A vector is an array of numbers that represent data points. In attention mechanisms, queries, keys and values are represented as vectors to compute attention scores.
Query(Q): Represents a request or a question in the computation process. It is the question we ask of the model. In attention mechanism, the model decides to which input it gives the most focus on.
Key(K): Different pieces of information available to the system. Each “value” has a “key” vector associated with it.
Value(V): Corresponds to the actual data linked to each key. It holds information that the model uses to answer the query.