When I started adding integrations to Chief, the obvious way to give it a new capability was to give it a new tool. If you want it to read issues as well as pull requests, you add a tool that reads issues. The catalogue grows one entry at a time, and each entry is justified on its own.
I capped mine instead: three tools for a given integration, inside a six-hundred-token budget for the whole catalogue. Every new capability since has had to fit inside that, which in practice means expressing it as a parameter on a tool that already exists rather than as a new tool.
Reading other people's pull requests became a parameter on the existing pull request tool, saying whose. Reading issues became a query parameter on an existing tool rather than a tool of its own, so it's the same capability without adding to the catalogue.
Why a budget and not a guideline
The tool catalogue is text that goes into the context on every single request, and it's one of the things the model reads before deciding what to do. That gives it two costs.
The first is the obvious one: tokens, on every request, forever. That's real, but it isn't what changed my mind.
The second is selection accuracy. Every extra tool is another option the model has to choose between, and the ones that hurt are the near-neighbours. Two tools that sound similar but differ in a detail are exactly where a model picks wrong, so adding a tool doesn't just cost its own tokens, it makes the decision about every other tool a bit worse.
A guideline to be thoughtful about adding tools wouldn't have stopped me adding them, but a hard budget did. When there's no room, the question stops being "should this be a tool" and becomes "which existing tool is this a variation of", and that second question produces a smaller, tidier catalogue almost every time.
I've written before about keeping orchestration in code rather than in the prompt. This is the same instinct applied one level down: the fewer decisions you hand the model, the fewer it gets wrong, and a small catalogue is a way of not asking in the first place.
Measuring before removing
The constraint made me want to take things away as well, and that's where I very nearly got it wrong.
I'd assumed the smallest model in the lineup would be bad at tool calling and should have it switched off. It sounds right (small models are worse at structured output, and a wrong tool call is a visibly broken interaction), so it went on the list.
When I actually measured it, that turned out to be wrong. That model calls tools correctly when a tool applies. Where it went wrong was reaching for a tool when none was needed, so it was picking tools too eagerly, which is a different problem from not being able to call them.
Those two diagnoses lead to opposite fixes. Switching off tool calling would have removed a working capability to fix a problem it didn't have. The real fix was a router in front of the model that decides, without a model, whether a request is even a candidate for a tool. It's cheaper, it's deterministic, and it goes after the thing that was really failing.
The guard nothing exercises
I've also had to make a related decision twice now. Both times I designed a fallback for models that can't call tools natively, with a capability flag and a constrained-output path for when the flag is false. Both times I rejected it, and the second time is the one that stuck, because the first wasn't principled enough to.
Every model I ship calls tools natively, so the flag is always true and no real configuration can reach the fallback path. The only way to exercise it is a stub written specifically to exercise it, which means the test proves the test works.
A fallback nothing can reach doesn't prove anything, but it's still code that has to be read, maintained and reasoned about, and it gives you confidence in something that's never been tested. If I later add a model that can't call tools, that's when to build the fallback, against a real case and with a real way to know it works.
That reasoning is uncomfortable, because it sounds like an argument against defensive programming, and I don't think it is. The distinction I've settled on is whether the case you're defending against can occur in any configuration that ships. If it can, defend it and test it, otherwise you've written a branch that will be wrong by the time it first runs, and nobody will find out.
Where this leaves me
The budget has been the most useful constraint in the project, and not for the reason I set it. I set it to control context cost, but what it really does is push every capability question through a design step I'd otherwise have skipped.
What I don't know is where the ceiling is. Three tools per integration is a number I picked, and the fact that it's produced good decisions so far doesn't tell me whether it's right, or whether I just haven't wanted something that deserves a fourth. Ask me again when I do.