Dot Product

Dot Product

---

Prerequisites

Basis vector
Independent/Dependent

---

What is Dot Product

1. What is Dot Product?

\[\begin{bmatrix} u_x & u_y \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = u_x x + u_y y\]

This is exactly the dot product:

\[u \cdot x\]

A matrix represents a linear transformation from $\mathbb{R}^2$ to $\mathbb{R}$.

2. Geometric Meaning of the Dot Product

Direction
Similarity

\[a \cdot b = |a||b|\cos\theta\]

Where

• $

  a

  $ : magnitude of vector $a$

• $

  b

  $ : magnitude of vector $b$

• $\theta$ : angle between the two vectors

Angle	$\cos\theta$	Meaning
$0^\circ$	$1$	Same direction (maximum similarity)
$90^\circ$	$0$	Orthogonal (no similarity)
$180^\circ$	$-1$	Opposite direction

Thus, the dot product naturally measures directional similarity.

\[\text{cosine similarity} = \frac{a \cdot b}{|a||b|}\]

This results in:

\[\cos\theta\]

which measures pure directional similarity.

Ex. Transformer attention

\[score = Q \cdot K\]

• $Q$ : query vector
• $K$ : key vector

If their directions are similar, the dot product becomes large, leading to higher attention weights.

Thus,

The dot product measures how aligned two feature vectors are, making it a natural similarity metric.