@luizasilva4567899:

marialuizaalvessilva
marialuizaalvessilva
Open In TikTok:
Region: BR
Thursday 16 July 2026 00:50:25 GMT
427
45
1
0

Music

Download

Comments

cristiano.lucio6
Cristiano Lucio :
😍
2026-07-16 01:01:09
0
To see more videos from user @luizasilva4567899, please go to the Tikwm homepage.

Other Videos

This is actually a systems design question. LLM is just the wrapper, bait, dress… whatever you call it… Three things get tested here. Why a normal cache breaks when two people phrase the same thing differently. Why a wrong cached answer is worse than no answer at all. Whether you can prove the cache works. Before designing anything, ask a few questions. Do answers change per user? How old can a cached answer be? What’s the cost of serving a wrong one? That last answer shapes the whole design. Hashing the prompt as a cache key seems simple. It breaks fast. So NO! “How do I reset my password” and “password reset help” mean the same thing. A hash treats them as different questions. So the cache needs layers. Exact match first. Embeddings next, to catch the same question worded differently. A verification step before anything gets served. A false positive is worse than a cache miss. A miss just costs another call to the LLM. A false positive confidently gives the wrong answer. That’s much harder to undo. This is where the interview gets interesting. People often say cosine similarity at 0.92. Few explain why that number. Push it too high and you start missing real matches. I’d measure false positives and negatives on real traffic instead. Different topics can run different similarity settings. Some questions can tolerate fuzzier matches. Others can’t. I’d roll this out slowly either way. Track hit rate, cost saved, latency, false positives, and answer quality. Building the cache is the easy part. Making it trustworthy takes longer. Glossary for my juniors or non technicals 🫶 : Embeddings — numbers that capture what a sentence means, so similar meanings sit close together. Semantic similarity — checking if two questions mean the same thing, even worded differently. Cosine similarity — a way of scoring how close two embeddings are, from 0 (unrelated) to 1 (identical meaning). Cache hit — reusing an answer that’s already stored. Cache miss — nothing stored, so the LLM gets asked again. False positive — the cache matches two different questions and serves the wrong answer. #Sys#SystemDesignM#LLMEngineeringc#TechInterviewf#SoftwareEngineeringEngineering
This is actually a systems design question. LLM is just the wrapper, bait, dress… whatever you call it… Three things get tested here. Why a normal cache breaks when two people phrase the same thing differently. Why a wrong cached answer is worse than no answer at all. Whether you can prove the cache works. Before designing anything, ask a few questions. Do answers change per user? How old can a cached answer be? What’s the cost of serving a wrong one? That last answer shapes the whole design. Hashing the prompt as a cache key seems simple. It breaks fast. So NO! “How do I reset my password” and “password reset help” mean the same thing. A hash treats them as different questions. So the cache needs layers. Exact match first. Embeddings next, to catch the same question worded differently. A verification step before anything gets served. A false positive is worse than a cache miss. A miss just costs another call to the LLM. A false positive confidently gives the wrong answer. That’s much harder to undo. This is where the interview gets interesting. People often say cosine similarity at 0.92. Few explain why that number. Push it too high and you start missing real matches. I’d measure false positives and negatives on real traffic instead. Different topics can run different similarity settings. Some questions can tolerate fuzzier matches. Others can’t. I’d roll this out slowly either way. Track hit rate, cost saved, latency, false positives, and answer quality. Building the cache is the easy part. Making it trustworthy takes longer. Glossary for my juniors or non technicals 🫶 : Embeddings — numbers that capture what a sentence means, so similar meanings sit close together. Semantic similarity — checking if two questions mean the same thing, even worded differently. Cosine similarity — a way of scoring how close two embeddings are, from 0 (unrelated) to 1 (identical meaning). Cache hit — reusing an answer that’s already stored. Cache miss — nothing stored, so the LLM gets asked again. False positive — the cache matches two different questions and serves the wrong answer. #Sys#SystemDesignM#LLMEngineeringc#TechInterviewf#SoftwareEngineeringEngineering

About