LLM Foundations (early book draft)

by Dan Wilhelm

In this book, we develop a low-level understanding of Large Language Models (LLMs). To understand large circuits, we apply learnings from mathematics and the analysis of small circuits.

This book has an accompanying YouTube channel and project GitHub repo (coming soon).

To get the most out of this book, we recommend the reader be fluent in Python and know the basics of NumPy, linear algebra, and machine learning. Therefore, our target audience includes researchers, CS students, and software engineers with an interest in LLMs.

Currently, our focus is on analysis rather than training.

This book is hosted on GitHub, and suggestions/revisions are welcome. On every page in the upper right, there are icons for visiting the repo and editing the current page.