A single attention head·2 min
Why mask the future
0 / 2
Two things about the mask: what it accomplishes, and why we use a big negative number instead of literal -∞.
Knowledge check
In a causal head, how do you stop query position i from attending to a later position j > i?
Knowledge check
Why set masked scores to -1e9 rather than literally -Infinity?