File size: 2,872 Bytes
c34c7de c0a0e96 c34c7de c0a0e96 c34c7de |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
<!-- Minimal HTML fragment: terminal-style ASCII attention masks -->
<div style="max-width: 940px; margin: 16px 0; border:1px solid #2a2f3a; border-radius:8px; background:#0b0f19; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', 'Courier New', monospace; color:#e5e7eb;">
<div style="display:flex; align-items:center; gap:8px; padding:8px 10px; border-bottom:1px solid #1f2430; background:#111827; border-top-left-radius:8px; border-top-right-radius:8px;">
<span style="width:10px; height:10px; background:#ef4444; border-radius:50%; display:inline-block;"></span>
<span style="width:10px; height:10px; background:#f59e0b; border-radius:50%; display:inline-block;"></span>
<span style="width:10px; height:10px; background:#22c55e; border-radius:50%; display:inline-block;"></span>
<span style="margin-left:8px; font-size:12px; color:#9ca3af;">attention-mask-visualizer</span>
</div>
<div style="padding:12px 14px; overflow:auto; font-size:12.5px; line-height:1.4;">
<pre style="margin:0; white-space:pre; tab-size:2;">
ATTN MASK β GPT-2 (causal)
Tokens: [The, cat, sat, on, the, mat]
Legend: x = can attend, . = masked (future)
The cat sat on the mat
The x
cat x x
sat x x x
on x x x x
the x x x x x
mat x x x x x x
ATTN MASK β PaliGemma-style (bidirectional prefix + causal suffix)
Prefix: [<i0> <i1> <i2> <i3> <i4> What is this]
Suffix: [A great duck]
Legend: β = can attend, β = cannot
<i0><i1><i2><i3><i4> What is this | A great duck
<i0> β β β β β β β β β β β
<i1> β β β β β β β β β β β
<i2> β β β β β β β β β β β
<i3> β β β β β β β β β β β
<i4> β β β β β β β β β β β
What β β β β β β β β β β β
is β β β β β β β β β β β
this β β β β β β β β β β β
--------------------------------------------------------------------
A β β β β β β β β β β β
great β β β β β β β β β β β
duck β β β β β β β β β β β
</pre>
</div>
</div>
|