Why I stopped chasing AI models

Simon Willison ran Kimi K3 through its paces this week. The model costs $15 per million output tokens, burned 13,241 reasoning tokens to generate a pelican SVG, and beats GPT-5.6 Sol on some frontend benchmarks. Qwen 3.8 Max dropped the same day. Claude Fable 5 shipped last month. GPT-5.6 Luna, Terra, and Sol arrived earlier this month.

I stopped reading model benchmarks six months ago.

Not because they are useless. Simon's pelican test is a great "hello world" for any new model. I stopped because chasing model releases was making me slower than using what I already had. The best AI tools are boring applies to models too.

On this page

• What is the AI model treadmill?
• Why model FOMO hurts solo builders
• How to choose a model and stop second-guessing
• When to actually switch

What is the AI model treadmill?

The AI model treadmill is the cycle of re-evaluating your stack every time a new model drops. Qwen 3.8 Max, Kimi K3, GPT-5.6 Sol, Claude Fable 5 — each one promises better benchmarks, longer context, or cheaper tokens. The treadmill keeps accelerating. In the first half of 2026, the number of models scoring above 1400 on the Arena leaderboard doubled from 12 to 25. The pace is not slowing down, and neither is the pressure to switch.

Why model FOMO hurts solo builders

Every model switch has hidden costs that compound over time. You retest prompts. You revalidate outputs. You update integrations. You lose the implicit knowledge of how your current model behaves — the quirks, the failure modes, the prompt patterns you tuned over weeks of use. For a solo builder, that knowledge is a real asset. Losing it every few weeks means you spend more time adapting than shipping. I saw this pattern in myself and wrote about it in The Founder Floor: the real bottleneck is not access to better tools. It is decision throughput. Every model switch adds a decision you did not need to make.

How to choose a model and stop second-guessing

Pick a model based on your actual workload, not benchmark scores. Run your own eval with your real prompts. Measure task completion rate, not token statistics. A model that nails your specific use case 95% of the time beats one with higher Arena Elo but unpredictable failure modes in your domain. Define three criteria: reliability on your core task, consistent output format, and stable pricing. Any model that meets all three is good enough. The marginal gains from switching models are almost never worth the hidden friction of adapting your entire workflow around a new one. The Codex Resets tracker is a good reminder that even platform-level decisions are unpredictable — building on top of constant change is exhausting by design.

When to actually switch

There are valid reasons to leave a model. Your current one deprecates an API you depend on. It gets noticeably slower for your workload. Pricing changes in a way that breaks your economics. But these are decisions driven by your stack failing, not by someone else's announcement. Set a quarterly review — the same cadence I use for tool audits in The design engineering toolstack is multiplying. Before you switch, ask one question: does my current model still do the job I hired it for? If the answer is yes, stay.

The model release cycle is not slowing down. Kimi K3 will have a successor before most teams finish evaluating it. The solo builder who wins is the one who builds with the model they have, not the one they keep almost switching to.