Attention Mechanism

The core operation in transformer models that lets each token attend to all other tokens in the context. Attention enables an AI coding tool to relate a variable declaration on line 1 to its usage on line 200.