🧭 位置编码：给每个词发张号牌🧭 Positional Encoding: a Number for Every Spot

上一片里，多头注意力一口气看完了整句话，每个词都跟其他词对了眼神。可它天生不知道谁先谁后——「猫追狗」和「狗追猫」在它眼里一模一样。这一片来解决这件事。 Last leaf, multi-head attention took in the whole sentence at a glance — every word glanced at every other. But it has no sense of who came first: “cat chases dog” and “dog chases cat” look exactly the same to it. This leaf fixes that.

第 1 步：每个词，发个号牌Step 1: A number for every spot

注意力算的是「两两相关、加权求和」，加法可不分先后——想让词有先后，就得给每个位置配一个专属数字。Attention scores pairs and adds them up — and adding ignores order. To keep an order, every position needs a number of its own.

词卡排队领号牌：猫、追、狗 — 把「猫」「追」「狗」三张词卡排好队，挨个发号牌：1、2、3。号牌就是「位置」的名字，词卡揣着它一起进网络。为什么要发？因为在注意力眼里，词卡像摊在桌上的一把牌——加法不分先后，不标注位置，它就真的分不清谁在前、谁在后。 Line up three word cards — cat, chases, dog — and hand out tags: 1, 2, 3. The tag is the position’s name, and every card carries it into the network. Why bother? To attention, word cards are a hand of cards spread on a table: addition ignores order, and without tags it truly cannot tell first from last.

号码直接写 1、2、3 不行吗？数字一大就出乱子——得换一种更聪明的方式。Why not just write 1, 2, 3? Plain counting gets messy fast — there’s a smarter way.

第 2 步：号牌写成波纹Step 2: Write the tags as waves

一条波浪曲线，不同位置不同相位，虚线连到词卡 — 直接数 1、2、3 有两个坑：数字会越写越大，模型还容易误会「位置 8 和 9」的关系跟「位置 1 和 2」差不多。真实做法是把每个位置画成一串波浪：位置 1 一组相位，位置 2 换个相位，位置 3 再换……每个位置都有独一无二的「波纹指纹」，大小恒定，相邻位置还长得像。这套波纹号牌，就是位置编码（Positional Encoding）。 Plain counting has two problems: the numbers keep growing, and the model may misread “8 and 9” as being as close as “1 and 2”. So each position is drawn as a wave instead: position 1 gets one phase, position 2 shifts the phase, position 3 shifts again… Every spot earns a unique “wave fingerprint” — always the same size, and neighbours look alike. This wave-based tag is called positional encoding.

号牌就位，Transformer 现在多看到了什么？With the tags in place, what extra power does the Transformer gain?

第 3 步：顺序一变，意思全变Step 3: Same words, new order, new meaning

左右对比：猫追狗 vs 狗追猫，词卡相同、号牌波纹不同、箭头相反 — 「猫追狗」和「狗追猫」的三张词卡一模一样，可号牌和波纹不同——谁在追、谁被追，箭头方向完全相反。注意力负责看清全场的词，位置编码负责记住谁先谁后；两样合起来，Transformer 才真正会读句子。 “cat chases dog” and “dog chases cat” use exactly the same cards — but their tags and waves differ, so the arrow flips: who chases, who gets chased. Attention sees the whole court; positional encoding remembers who came first. Together they let a Transformer actually read a sentence.

🎮 你来当单词导演（60 秒）🎮 Your turn: be the word director (60 seconds)

3 句话的词卡都被洗乱了。看「句意提示」，按正确语序依次点词卡：点过的卡变灰、号牌就位；排对整句自动进入下一句，3 句全对撒花。Three sentences got shuffled into word cards. Read each meaning hint, then click the cards in the right order: chosen cards turn grey and take their number tag. Finish a sentence to advance — three wins and it’s confetti.

一句话记住它：位置编码 = 发给每个词的位置号牌；注意力看全场，它管先后——两样合起来，句子才有意思。 Remember it in one line: positional encoding = a position tag for every word; attention watches the whole court, the tags keep the order — together they make a sentence mean something.

自注意力的加法不分先后：不额外标注，它分不清「猫追狗」和「狗追猫」Adding ignores order: with no tags, attention can’t tell “cat chases dog” from “dog chases cat” 号牌是一串专属数字（波纹相位）：每个位置独一无二、大小恒定、相邻相似A tag is a unique string of numbers (wave phases): unique per spot, never growing, neighbours similar 注意力看关系、位置编码记先后——Transformer 两样都要Attention tracks relations, positional encoding tracks order — a Transformer needs both

← 浏览全部 302 个知识点← Browse all 302 topics

内容参考 OpenStax 等公开教材，多来源核对 · AI 生成、人工审核 · 发现错误欢迎指正，帮这片叶子长得更好。 Based on OpenStax and other open textbooks, cross-checked across sources · AI-generated, human-reviewed · Spotted a mistake? Tell us — help this leaf grow.