> ## Content Index > Fetch the complete content index at: https://scottmallinson.com/llms.txt > Use this file to discover other available public pages before exploring further. # State you assumed would still be there - URL: https://scottmallinson.com/state-you-assumed-would-still-be-there/ - Published: 2026-10-26T07:55:00.000Z - Updated: 2026-10-26T07:55:00.000Z - Description: An auth token held in memory and a SQLite file on disk both worked locally and disappeared in production, because my laptop provides durability the serverless platform doesn't. - Author: Scott Mallinson - Tags: Engineering, DevOps, Architecture, Debugging Two failures, on two different projects, that took me a while to recognise as the same failure. The first: an authenticated integration would work, then stop working, then work again after I re-entered the credentials. Never during a session. Always across one. The second: a feature that reads stored history returned nothing in production, while working perfectly on my machine and in every test. Both are the same mistake. I had assumed something would still be there, and on the platform I actually deploy to, it wasn't. ## The credential that rotates The API in the first case rotates its refresh token on every use. You exchange the token, you get a new one, and the old one is dead. That's a reasonable design and it's documented. My code held the current token in memory. Which works until the process restarts. Then the in-memory copy is gone and the code falls back to the token in the environment variables, which was correct once, was consumed on first use, and has been dead ever since. So every deploy silently broke authentication. Every restart too. The failure surfaced as an auth error from the upstream API, which reads exactly like an expired credential or a revoked application, and sends you off to check the provider's dashboard. It looks like an authentication bug, but it's a storage bug. The fix is dull once you see it: the token store has to be durable, and the environment variables are for bootstrapping the very first run and nothing else. The diagnostic lesson is the part to keep. Here the failures lined up with deploys rather than with time, which pointed at where the token was stored, not at the token. ## The database that never existed The second one has a sharper edge to it. The feature reads history from SQLite. In production, on a serverless platform, that database is never created at all. The initialisation throws before any collector has a chance to run, and the filesystem it would have been created on doesn't persist between invocations anyway. There's no database, and there never was going to be one. Locally, of course, everything works. The file gets created on first run and persists because my laptop is a computer with a disk. The test suite passes for the same reason. Nothing in the development loop resembles the environment where the code actually runs. I didn't fix this by making SQLite work on a platform that doesn't want it to. I changed what the feature asks for. "Current location" is now a live call to the upstream API with a short cache, and the stored history is a fallback rather than the primary source. That's a smaller feature in one sense: it no longer depends on accumulated local state. It's also the version that runs where it's deployed, which the previous one didn't. ## The pattern, and why it's hard to see Both bugs are about durability assumptions that hold in development and dissolve in production. Memory persists between requests, until the process is recycled. The filesystem persists between runs, until it's ephemeral. Neither assumption is written down anywhere, because neither felt like an assumption at the time. They felt like how computers work. What made them hard to see is that my laptop has every property the serverless platform has, plus persistent memory and a disk, and my code had come to depend on both without my noticing. A worse local environment, one that recycled memory aggressively and threw the filesystem away between runs, would have caught both of these in minutes. That's roughly the opposite of how I had been thinking about development environments, where the instinct is to make local as convenient as possible and deal with the differences later. ## What I actually changed Not a framework and not a rule. Two questions I now ask about anything that stores something. Where does this live, and what event destroys it? If the answer to the second is "a deploy" or "a restart" or "the platform deciding to recycle the instance", and the value is expected to survive that, it's in the wrong place. Both of these bugs would have been caught by asking that once. And: what's different about the target platform that my machine is hiding from me? For serverless in particular the list is short and knowable in advance. No durable local filesystem, no shared memory between invocations, no guarantee that the instance handling this request is the one that handled the last. Every one of those is a thing my laptop provides for free and production doesn't. I'd like to say I now test against a production-like environment as a matter of course. I don't. Both of these were found in production, by noticing something was wrong and working backwards, and I don't have a good story for how I'd have caught them earlier without building infrastructure that would cost more than the bugs did. If there's a cheap way to get a hostile local environment, I haven't found it.