How Might LLMs Store Facts? [Multilayer Perceptrons, MLP]

Time stamps:

0:00 - Where facts in LLMs live
2:15 - Quick refresher on transformers
4:39 - Assumptions for our toy example
6:07 - Inside a multilayer perceptron
15:38 - Counting parameters
17:04 - Superposition
21:37 - Up next

Preceding videos in this series:

• But what is a GPT? Visual intro to Transformers
• Visualizing Attention: A Transformer's Heart