Token + position embeddings·2 min
Why position embeddings
0 / 2
Two checks on the embedding sum — which rows get added, and why add instead of concatenate.
Knowledge check
For input IDs [15496, 11, 995], what is the embedding vector at position 1?
Knowledge check
Why are the token and position embeddings added together rather than concatenated, and what is the output shape?