4 HN users recommended this explainer4 months2020–2025score 4.5
Years with an HN recommendation, 2007–2026
HN recommends it as an accessible, clear walkthrough of the transformer paper for people with some AI background.
Yannic Kilcher walks through the Attention Is All You Need paper, which introduced the Transformer, a sequence model based solely on attention without recurrence or convolutions. He reviews the architecture and the paper's translation results.
“The Yannic kilcher review is quite good.” (HN, 2023)
YouTube channel that explains the paper in detail: [this video] And subsequent follow-ups (ROME, editing transformer arch): [link] I find the channel amazing explaining super complex topics in simple enough terms for people who have some background in AI.
Detailed, simple explanation of the transformer paper
Everyone has a different way of approaching. I like video walk-throughs of papers. If you are the same, for the theory bit you may find Yannic Kilcher's videos [0,1] helpful. On implementation side, the best project out there is [2] with good documentation. Also spacy has transformer package [3] (I personally haven't tried it), so maybe it will be easier for you to jump in if you have prior experience with spacy. [0] [this video] [1] [link] [2] [link] [3] [link]
For accessibility I recommend Yannic Kilcher video review of "Attention Is All You Need" [this video] Yannic has been making about 62 other transformer paper reviews since. You can find the usual suspects. [link]
Accessible review of the Attention Is All You Need paper
The Yannic kilcher review is quite good. [this video] I can't ELI5 but I can ELI-junior-dev. Tl;dw: Transformers work by basically being a differentiable lookup/hash table. First your input is tokenized and (N) tokens (this constitutes the attention frame) are encoded both based on token identity and position in the attention frame. Then there is an NxN matrix that is applied to your attention frame "performing the lookup query" over all other tokens in the attention frame, so every token gets a "contextual semantic understanding" that takes in both all the other stuff in the attention frame and it's relative position. Gpt is impressive because the N is really huge and it has many layers. A big N means you can potentially access information farther away. Each layer gives more opportunities to summarize and integrate long range information in a fractal process. Two key takeaways: - differentiable hash tables - encoding relative position using periodic functions NB: the attention frame tokens are actually K-vectors (so the frame is a KxN matrix) and the query matrix is an NxNxK tensor IIRC but it's easier to describe it this way
> I love these word-salad exaltations of LLMs - people don't even know what they're writing half the time but it sure is provocative! Well FWIW I have done an implementation of Llama LLM (3 8-70b to be specific) in non-python so I do sort of know what I'm talking about. I'm not the originator of the hash table analogy. I got it from here: [this video] Hell, the vectors that are generated are called K/V, so. Yeah it's a hash table. And the idea that first order logic and other facets of intelligence can just be cleverly arranged lookup tables comes from here: [link]
Video list and Hacker News comment labels from HackerTube (prompted by @karpathy), released under ODC-By 1.0, built on the open-index/hacker-news archive. Hacker News data is provided by Y Combinator; comments remain the work of their authors. Video titles, channel names and thumbnails belong to their creators and are shown via YouTube. HackerTube Watch is an unofficial edition. Data export of 2026-10-31.
Plays via YouTube's privacy-enhanced embed
5 Hacker News comments
From the HN archive; up to 20 kept per video, favoring different commenters. Recommendations first.
Read more
Citations, criticism and jokes