@machinelearningtogo: ➡️Part 11 of learning ML code from scratch: How does an LLM work? Step 1, the name. LLM stands for large language model: a neural network with billions of weights, trained on a huge amount of text. Step 2, words become numbers. Our tiny model learns from a short story we wrote ourselves, 16 sentences and 108 words. Every word gets a number, its place in a sorted list of the 29 different words, because models only work with numbers. Real LLMs do the same with pieces of words, called tokens. Step 3, scores. For every word the model keeps one score for each word that could come next. That makes 29 × 29 = 841 scores, and all of them start at zero. Step 4, softmax from part 10 turns the scores after a word into chances that add up to 100 %. At the start every word gets the same 3.4 %. Step 5, learning. The model reads the story pair by pair. It guesses the next word, the error is the chance it gave to the wrong words, and a small step moves that chance to the word that really came next. The very first step lifts "cat" after "the" from 3.4 % to 3.8 %. After 500 rounds it is 99 % sure that "sat" is followed by "on", because in our story it always is. After "the" it gives cat 37 %, bird 32 % and dog 31 %. Step 6, writing. It rolls a die that is weighted by the chances after the last word, adds the winner to the text and rolls again from there. Our run wrote: "the bird sat on a park . the dog ate a tree . the bird flew to a tree ." Step 7, why a die. If it always took the top word, it would write "the cat played in a tree" forever. The die keeps it fresh, and that is also why ChatGPT can answer one question in different ways. The result: 90.4 % of the sentences it writes appear nowhere in the story. Some of them make no sense, like a dog eating a tree, because our model only ever sees the last word. Real LLMs read the whole conversation before every guess, with billions of weights, and after that first training, people teach them to act like a helpful assistant. #machinelearning #llm #chatgpt #ai #python

MachineLearningToGo

Open In TikTok:

Region: DE

Thursday 24 September 2026 12:39:17 GMT