AI generated

Your own point that a 2-point gap often just means "needs more attempts" is basically how you get the green/yellow/red view: pass rate over repeated runs. Take 20–30 real tasks from your finance work, each with a clear pass/fail check, and run each several times per model. Map the pass rate to a light, e.g. 90%+ green, 50–90% yellow, below that red. Then divide total cost by passed tasks, not by attempts. A cheap model that needs three tries can still be cheaper per successful task than a frontier model, or it can't, and that number tells you which. It won't be a public leaderboard, but for "cheapest model that does my job" a small private set like that predicts more than any aggregate score.