@remoteree: Training is a research job a handful of labs do. Serving is an engineering job every company shipping AI now has, and it repeats every day. The moves that actually shift the number, in the order they pay off: 1. Benchmark first. Every claim after this is measured against the baseline, or it is a vibe. 2. Quantize. Fewer bits per weight. INT8 is usually free. INT4 is where quality quietly dies, which is why the accuracy floor is part of the work and not an afterthought. 3. Continuous batching. Prefill and decode have different shapes, and exploiting that asymmetry keeps the GPU saturated without making one request wait for a whole batch. 4. Page the KV cache, so memory stops being the ceiling on batch size. 5. Speculative decoding, CUDA graphs, then export to ONNX for a portable graph. The artifact that gets you hired is the benchmark table attributing each gain to one specific change. Almost no portfolio has one. The build is on KhwajaLabs: Optimize LLM Inference, Quantization, Batching and ONNX. Eight steps, PyTorch and ONNX Runtime, and the discipline of measure, change one thing, measure again. #MachineLearning #LLM #MLOps #AIEngineering #AIJobs

Reshad

Open In TikTok:

Region: US

Thursday 20 August 2026 01:24:46 GMT