@dgm.america: Which open model is better? Don't ask the leaderboard. Test both, on your own prompts. A leaderboard score is someone else's exam. Your refund emails, your contracts, your tone — they aren't on it. In osFoundry.io an eval runs every model on every prompt: two deployments in one run, with progress filling in as it goes. Grade it with exact match when there's one right answer, or LLM-as-judge when a good answer can be worded many ways. And if one model fails, only that model fails. The run still finishes. Tip: grade a few answers yourself first and check the judge agrees before you trust it. GPU time is billed while the endpoints run. osFoundry is a paid workspace. Send this to the team still arguing over leaderboards. #LLM #OpenSourceAI #AIEval #osFoundry

There are no more comments for this video.

To see more videos from user @dgm.america, please go to the Tikwm homepage.