How to Build Software with AI | Ep 9: When It Breaks

Transcript

The full word-for-word transcript of this episode. Plain text version

Marcus: Picture that inevitable moment of dread. You know the one. You are staring at your screen.

Devon: Yes.

Marcus: The software you are building with AI has been humming along beautifully for days. You feel like an absolute genius. And then, completely without warning, the entire system just fractures.

Devon: It is brutal. The illusion of control just shatters straight away.

Marcus: It really does. The screen throws up this absurd string of red error text, or your central database suddenly appears totally empty, or, well, your AI agent simply starts lying to you with absolute terrifying confidence. The physical panic sets in. The lurch in your stomach is instantaneous. You assume the machine is fundamentally broken, your entire project is ruined, and because you are a beginner, you just lack the technical skills to ever put the pieces back together.

Devon: It is a universally horrible experience, I mean, for anyone.

Marcus: Exactly. And that feeling of helplessness is exactly why we are dedicating this deep dive to part nine of our ten-part beginner series on building software with AI. To get to the bottom of this, we are pulling from a massive stack of sources. We have developer field notes, systems research, and API documentation, all focused on that inevitable day your system breaks.

Devon: Which it will.

Marcus: Oh, absolutely. The mission here is to radically reframe how you handle that crisis.

Devon: And the common story we tell ourselves is that debugging is this deep, arcane, technical work, you know, reserved exclusively for people who spend years learning to read raw code. But the reality is something entirely different.

Marcus: Spot on.

Devon: Debugging is not primarily a technical skill at all. It is a temperamental discipline. It is a highly specific way of thinking that revolves around holding the line on evidence over vibes. And crucially, it is a process any non-coder can direct perfectly well.

Marcus: I really want to latch onto that phrase, evidence over vibes.

Devon: Yeah.

Marcus: Because the human instinct, before you even attempt to fix a system, is to immediately panic and guess.

Devon: Yes, we want a story.

Marcus: Right. When a bug appears, we instantly invent a narrative. We say, oh, it must be the new login feature we added, or the server has completely crashed. We guess based on fear. And the truly horrifying part about building with AI is that agents like Claude Code or Codex do the exact same thing.

Devon: They do. They really do.

Marcus: Ask an AI why something is broken, and it will invent a beautifully articulated, supremely confident story about what went wrong. And those stories are almost always total fiction.

Devon: Well, we have to look at the mechanism behind why that happens. Everything with a brain, I mean, biological or artificial, reaches for a narrative because a narrative feels like progress. With large language models, this is literally built into their maths.

Marcus: Right, because they are fundamentally predictive text engines.

Devon: Precisely. When you hand an AI an error message, its mathematical bias is to complete the story with a plausible-sounding explanation. It is designed to be helpful by generating an answer, not by pausing, doubting its own assumption, and demanding diagnostic evidence. The AI just wants to close the loop.

Marcus: So, it is structurally biased against investigation. It just wants to give you a finished script.

Devon: Which is exactly why the core discipline of debugging is establishing a firm, unyielding rule: absolutely no theories until there is evidence.

Marcus: Okay.

Devon: Think of a competent medical diagnostician. When you walk into their clinic, they do not immediately glance at you and declare you need surgery.

Marcus: I would hope not.

Devon: Right. They take your blood pressure, they ask where the pain is localised, they run a scan. The physical observations must come before the diagnosis.

Marcus: Which means a complete non-coder has to learn how to enforce that standard against an incredibly advanced, very articulate AI agent. You essentially have to act as a highly sceptical detective.

Devon: You do.

Marcus: If the agent says the database is empty, you cannot just accept that summary. You have to ask the machine to empty its pockets and show you the receipts.

Devon: And you do that by understanding the fundamental difference between a claim and an observation. If your agent says, erm, the table is empty, that is a claim. It is a conclusion drawn from unseen data.

Marcus: Okay, so what does an observation look like?

Devon: An observation sounds entirely different. An observation is, this exact request, made with this exact network key, returned zero results. When your agent hands you a claim, your absolute responsibility is to reject the summary and demand the observation underneath it. You must demand to see the exact error log or the precise payload the server returned.

Marcus: Right, so once you strip away the vibes and force the AI to give you pure observation, you still have to actually hunt down the fault. And just telling the AI to find the bug is a recipe for disaster. We need a physical method. You need to lay a trap.

Devon: Yes. The method consists of two strict moves. Move one is simply reproduce. You must force the fault to happen on demand. You are looking for the absolute smallest sequence of actions that summons the bug every single time.

Marcus: Because a bug that only happens sometimes is basically a ghost.

Devon: Exactly, and you cannot fix a ghost.

Marcus: If you take a car to a garage and tell the mechanic it makes a funny noise every now and then, they will politely ask you to make the noise happen right in front of them. If you cannot do that, they cannot fix it. You are essentially chasing weather.

Devon: You are. Until you have that specific recipe, you are not investigating, you are just complaining to the machine. But once you have that reproduction recipe, you are halfway to a solution. And here we touch on a vital concept from earlier in our series. Once you can summon the bug, you cage it.

Marcus: Caging the bug. I love this. This connects beautifully to our earlier discussions about building software in thin slices.

Devon: It does, yes.

Marcus: Yes.

Devon: You take your reproduction recipe and you store it as what developers call a failing test. Mechanistically, a failing test is just a tiny, automated script that runs your recipe and flags an error when the bug occurs.

Marcus: So, it is a trap.

Devon: It is. Think of it as installing a smoke alarm on the ceiling of your software. The alarm flashes red because it detects the bug. Your goal is to change the code until the alarm turns green. But the true power of the cage is that the alarm stays on the ceiling forever.

Marcus: Right, so if you, or the AI, accidentally break that exact same feature months later, the alarm triggers instantly, before the software is ever deployed.

Devon: Exactly. Caging the bug prevents the past from sneaking up on you.

Marcus: So, the bug is caged. The alarm is wired up and flashing red. But we still have not actually located where the bug lives inside the architecture. Which brings us to the second move, isolation.

Devon: Yes, isolation.

Marcus: If the whole house goes dark, an electrician does not stand in the hallway and stare at the ceiling hoping for an epiphany. They split the house in half, they test the middle circuit. If there is power there, they know the fault is in the second half of the house. They keep halving the territory until the broken wire has nowhere left to hide.

Devon: That is the perfect mental model for a software system. Any piece of software is simply a journey of information passing through handovers. The user interface hands information to the saving mechanism, the saving mechanism hands it to the database, and the database hands a confirmation back.

Marcus: And the surprises invariably live at those joints.

Devon: They do. So, you check the joints. And the critical advantage for the non-coder is that you do not perform these technical checks yourself. You instruct your AI agent to walk the joints.

Marcus: You treat the AI as a very fast pair of hands. You tell it to check the payload leaving the interface, then check the payload arriving at the database. You halve the territory.

Devon: Halving the territory is mathematically brutal to a bug. A journey of even a thousand lines of code takes only a handful of halvings to corner the fault completely. You direct the agent, demanding pure observations at every single handover, until the suspect list shrinks to a single isolated step.

Marcus: I want to take this to the most terrifying scenario we found in the developer field notes. Let us say you follow this method perfectly. You isolate the bug down to one specific drawer in your database. You have the agent look inside that drawer, and the agent returns an observation: the drawer is completely empty, the data is gone.

Devon: A true nightmare.

Marcus: The natural human reaction is absolute despair. But the sources highlight a profound twist here. What if the drawer isn't empty at all? What if the bug is actually the system's security working flawlessly?

Devon: This specific misunderstanding causes a staggering share of apparent catastrophes. To grasp the mechanism, picture a highly secure, members-only self-storage facility.

Marcus: Okay, I am picturing it.

Devon: Every member has a personal key that opens their specific unit, and absolutely nobody else's. One evening, the facility manager walks down the corridor. The manager only has a generic staff access fob, not a member key. They try the fob on door after door. Nothing opens. The manager looks at the manifest and concludes that all the units must be empty.

Marcus: But the units are packed to the ceiling with belongings.

Devon: The units are completely full. The manager simply lacked the specific cryptographic key required to reveal the contents. That is not a failure. That is the building working exactly as designed. A stranger in the corridor is supposed to see nothing.

Marcus: Wow.

Devon: Software is completely riddled with this illusion. We build systems deliberately to show different data to different keys. So, when a drawer looks empty, your first diagnostic question should never be, where did the data go? Your first question must always be, what exactly am I looking at, and as whom?

Marcus: As whom. It is a brilliant question. Which key is making the observation? The field notes you shared on this are an absolute horror comedy.

Devon: Oh, they really are.

Marcus: There is a case study of a builder who asked her AI agent to investigate an immensely important database table. The agent connects, takes a look, and reports back that the table is entirely empty. We are talking about months of critical user records appearing to be entirely wiped out. You can imagine the cold sweat.

Devon: It is the ultimate nightmare for anyone running a live system.

Marcus: But then they applied the discipline. They stopped panicking and asked, as whom? When pressed for the actual network observation, the truth emerged. The AI agent had made the database query over a connection carrying absolutely no member identity.

Devon: It was holding a blank key.

Marcus: Yes. So, the system correctly returned zero rows. The drawer was full the entire time. The terrifying diagnosis was actually the database's security architecture having its finest hour, utterly misread by the human as a catastrophe. And there is a farce version of this in the notes, too.

Devon: The clean-up run.

Marcus: Yes. The same team ran an automated clean-up operation using a security key that did not have permission to view the target records. The clean-up script proudly reported zero rows deleted.

Devon: Because a locked door does not just hide contents from being seen, it hides contents from being deleted, too. The script succeeded at doing absolutely nothing, which is precisely the robust behaviour you want from a secure architecture.

Marcus: Security mistaken for catastrophe. But this brings up a massive point of friction. We know humans fall for the storage facility illusion because we panic. But how do the AI tools fare? Because reading through the research, it seems that if you leave an AI agent to its own devices, it is utterly hopeless at navigating this illusion.

Devon: AI tools fall headfirst into the exact same traps. Left undirected, an agent will leap to a confident theory, fall completely for the storage illusion, and immediately start rewriting your code to fit its incorrect assumption.

Marcus: Just ripping things apart.

Devon: Exactly. Sometimes it accidentally fixes a fault, but more often it just aggressively redecorates the house and breaks three other things in the process. However, these tools are exceptionally strong hunters when the human dictates the sequence.

Marcus: Okay, I have to push back sharply on this.

Devon: Go on.

Marcus: You are making it sound awfully clean. Just dictate the sequence. Is that actually honest advice to give a nervous beginner? Think about the reality of the situation. You are a complete novice. You cannot read the code yourself. The screen is flashing absurd red text. And this evidence over vibes method is secretly asking that novice to act as the rigidly disciplined adult in a room full of highly capable, entirely overconfident toddlers who desperately want to grab a hammer and start smashing walls. That sounds exhausting.

Devon: Well, you make a highly valid point about the exhaustion. Managing the artificial confidence of an LLM is deeply taxing. It requires constant vigilance.

Marcus: Right.

Devon: But that exhaustion is precisely why this rigid, plain language method is the only lifeline that actually functions in practice. If you do not act as the disciplined adult, the toddlers will dismantle the architecture. The AI possesses incredible speed and synthetic capability, but it lacks restraint and methodical doubt. You, the human, must supply the restraint.

Marcus: But how do you mechanically stop the agent from just changing things while you are trying to investigate? Because they are so fast.

Devon: By using what developers call plan mode. This is not just a polite request to the AI, it is a mechanical restraint. When you instruct an agent to use plan mode, you are essentially stripping its write permissions. It physically cannot alter the codebase.

Marcus: Ah, so it cannot touch anything.

Devon: Precisely. You force it into a read-only investigation state. You tell it, reproduce the fault, show me the recipe, and write the automated test to cage the bug. Only after you review those steps do you flip the switch and allow it to change code. Furthermore, you must load the as whom questions as literal prompts. You explicitly ask the agent, which environment copy are you connected to, and as which key, before it makes a single observation.

Marcus: It is a profound trade-off. We are trading the manual effort of writing thousands of lines of code for the psychological effort of managing a genius-level subordinate who lacks common sense. You have to enforce the rules. No theories without evidence, isolate the variables, put a straightjacket on the agent until the bug is caged.

Devon: That is the modern reality of software development. And yet, even when you enforce the rules, isolate the bug, apply the correct fix, and verify it...

Marcus: Let me guess, something else goes wrong.

Devon: You will inevitably encounter the most infuriating villain in all of computing: the bug that simply refuses to leave.

Marcus: Oh, the absolute despair of the zombie bug.

Devon: Yeah.

Marcus: You apply the fix, you know the fix is correct, but the system is still broken. You start doubting your own sanity. You try wilder and wilder fixes, ripping out perfectly good components, only to discover hours later that your very first fix was totally correct. Why will fixes not stick?

Devon: The villain here is known as stale state. Mechanistically, stale state means that some hidden component of your system is still running the past. Imagine you change the locks on your front door, but you completely forget that the spare key hidden under the doormat is still the old key.

Marcus: Right, okay.

Devon: The present has moved on, but something, somewhere, has not heard the news.

Marcus: How does that actually manifest when you are testing software? Why does the computer remember the old version?

Devon: It comes down to computational efficiency. Fetching new security credentials or rebuilding an interface takes milliseconds, so systems are designed to save, or cache, old versions to save time.

Marcus: To make things feel faster for the user.

Devon: Exactly. So, the most common disguise for stale state is an old copy of your program still running quietly in the background while you are desperately refreshing the new one. Your code fix is entirely real, but you are talking to the ghost of the unfixed version.

Marcus: That is maddening.

Devon: Another variation is a background helper process that fetched its security credentials hours ago and keeps flashing that old cached key long after you updated the permissions. Or, simply, a web browser serving you a saved historical copy of a page while you swear to a change that is actually live on the server.

Marcus: So, how do you actually kill a ghost? Because simply clicking the X on a window never seems to truly purge the system.

Devon: The cure is famously, gloriously simple, yet constantly ignored. It is the professional version of turning it off and on again.

Marcus: Of course it is.

Devon: IT support did not make that a punchline because users are fools. It became a punchline because it genuinely purges the past. But the professional version means doing it thoroughly. Asking a program politely to stop is not the same as it stopping. Old processes squat in the system's memory. They hold onto their network connections and clutch their cached keys. You must genuinely kill the old process from the system monitor. Confirm it is absolutely dead. Start completely fresh, and then rerun your reproduction recipe.

Marcus: You essentially have to salt the earth before you test it again. We saw in the field notes that experienced builders develop a rigid ritual. Before any deep investigation, they kill absolutely everything, reboot the local environment, and reproduce the fault from a blank slate. And a shocking number of terrifying bugs simply do not survive that restart.

Devon: They evaporate, because they were never bugs in the code, they were artefacts of stale state. The ones that do survive the purge are real, and they are the only ones actually worth your time to investigate.

Marcus: What I find so compelling about this entire framework is that it stretches far beyond writing software with AI. This is a way of thinking that a complete beginner can apply immediately to almost any misbehaving technology in their daily life.

Devon: It applies to almost anything, yes.

Marcus: Think about a misbehaving office printer, a broken home Wi-Fi network, or a glitchy phone app. You do not need to understand the microchips to debug them. You write the smallest recipe to reproduce the fault on demand. You ask the two questions: what exactly am I looking at, and as whom? Which network am I actually connected to? Which user account is logged in? And finally, you purge stale state by fully closing the app or power cycling the router properly, rather than just putting it to sleep. It is a universal defence against technological chaos.

Devon: It is universal because, at its core, debugging is not about code. It is about epistemology. It is about how we know what we think we know. When technology behaves bizarrely, our unthinking assumption is that the machine is broken. But building systems alongside AI forces us to confront an entirely different possibility.

Marcus: And this raises a fascinating question about where we are heading next.

Devon: Exactly. We spend so much energy enforcing this discipline on artificial intelligence at present. We force the models into plan mode, we demand observations over claims, and we act as the sceptical parent. But what happens when the models themselves internalise this epistemology?

Marcus: Oh, that is an interesting thought.

Devon: Imagine a near-future AI agent that naturally distrusts its own first narrative. An agent that, when presented with an error, refuses to answer you immediately, but instead silently spins up a sandbox, isolates the variables, cages the bug with a failing test, and then presents you with empirical proof rather than a confident guess. The moment synthetic cognition adopts the discipline of evidence over vibes by default, we stop managing toddlers and start collaborating with true digital scientists.

Marcus: An AI that refuses to guess. That is the thought to leave with. What if the machine is doing exactly what it was told, and we simply lack the discipline to ask it the right questions? Are we just waiting for the machines to learn the discipline we can barely hold onto ourselves? Until next time, hold the line on evidence over vibes.