The spec said 1.5 seconds
I wrote a 1.5 second latency target into a spec and reasoned from it for weeks. Measured, the real figure was 60 to 110 seconds.
10 notes filed here
I wrote a 1.5 second latency target into a spec and reasoned from it for weeks. Measured, the real figure was 60 to 110 seconds.
An auth token held in memory and a SQLite file on disk both worked locally and disappeared in production, because my laptop provides durability the serverless platform doesn't.
Chief's daily brief runs on a small local model. Getting it to behave meant moving a bullet-count limit out of the prompt and into code, then measuring what a repeat penalty setting actually cost in accuracy.
Validating user input is the most familiar guardrail and the smallest part of the problem. The model's own output, what it hands to tools, and what gets rendered afterwards all need the same fail-closed treatment.
For an AI feature with guardrails, releasing it carefully requires two disciplines: coordinating multi-service promotions with evidence at each step, and keeping the validation environment faithful enough that human feedback can be trusted.
When you add an AI layer to a retrieval feature, data decisions that developers used to make by instinct become explicit contract requirements. Live demos are often the first place you find out which ones are missing.