🎭 多头注意力：几双眼睛一起看🎭 Multi-Head Attention: Several Pairs of Eyes

上一片里，每个词用自注意力看了一遍全句。但一种看法只盯一种关系，很容易漏。这一片给它多配几双眼睛。 Last leaf, every word looked at the whole sentence with self-attention. But one view chases one kind of relation — the rest slip through. This leaf gives it several pairs of eyes.

第 1 步：一双眼睛，只看一路Step 1: One pair of eyes, one track

先看单头自注意力：它一次只学一种关注方式。Start with single-head self-attention: it learns just one way of paying attention.

三个机器人戴不同颜色眼镜，同时看同一句话 — 单头自注意力只算一组“打分-加权”，像只有一副眼睛：如果它学会了盯主语，别的关系就顾不上了。可句子里从来不只有一种关系——谁做动作、谁指谁、在哪里，同时都在。 A single-head self-attention computes just one set of scores-and-weights — like one pair of eyes. Learn to watch the subject, and everything else goes unseen. Yet a sentence never carries just one relation: who acts, who refers to whom, and where, all at once.

一副眼睛看不过来，怎么办？One pair of eyes can't cover it all — so what next?

第 2 步：多开几组，各盯各的Step 2: Run several heads, each on its own thread

三色连线分别标注谁做什么、谁指谁、在哪里 — 多头注意力就是同时开几组独立的注意力，每组叫一个“头”（head）。一个头可能专门盯“谁对谁做了什么”，一个盯“它指的是谁”，一个盯“在哪里”。各算各的，互不打扰。 Multi-head attention runs several independent attention sets at once; each set is called a head. One head may track who-does-what, another who-refers-to-whom, another where it happens. They compute side by side, none crowding out the others.

几个头各看各的，最后怎么合到一起？The heads work apart — how do they come back together?

第 3 步：拼起来，才看得全Step 3: Merge the views into one

多色细线汇合成宽光带，流入发光大脑 — 把每个头算出的向量拼接起来，再过一层小变换，就是这一层的输出。每个词的新表示里，主谓、指代、位置全都在——几双眼睛的收获，汇总给同一个大脑。 Concatenate the vectors from every head, pass them through one small transform, and that is the layer's output. Each word's new representation now carries action, reference and place together — every head's catch pours into one brain.

🎮 你来当分头小队长（30 秒）🎮 Your turn: run the four heads (30 seconds)

道理讲完了。一句“小猫坐在垫子上，它在打盹”，四个彩色头各问一个问题——点出正确的词，帮它们各连各的线。Theory done. Take “the kitten sits on the mat, and it is dozing”. Four colored heads each ask one question — tap the right word and help each head follow its own thread.

一句话记住它：多头注意力 = 同时开几组自注意力，各盯各的关系，再把结果拼起来——团队比一个人看得全。 Remember it in one line: multi-head attention = several self-attention sets running at once, each following its own relation, then merged — a team sees more than one pair of eyes.

单头只有一种看法，容易漏掉别的关系One head has a single view and easily misses other relations 多头 = 几组独立注意力并行，各学各的关注点Multi-head = several independent attention sets in parallel, each learning its own focus 最后拼接汇总，每个词的理解更全面The results are concatenated — every word ends up more fully understood

← 浏览全部 302 个知识点← Browse all 302 topics

内容参考 OpenStax 等公开教材，多来源核对 · AI 生成、人工审核 · 发现错误欢迎指正，帮这片叶子长得更好。 Based on OpenStax and other open textbooks, cross-checked across sources · AI-generated, human-reviewed · Spotted a mistake? Tell us — help this leaf grow.