How to Build Software with AI | Ep 6: Memory and Meaning

Transcript

The full word-for-word transcript of this episode. Plain text version

Marcus: You are staring at a search bar on your screen, right? And you are absolutely certain that the document exists.

Devon: Oh, we all know this feeling.

Marcus: Yeah, you type in the word invoice, you hit enter, and the system confidently tells you there are zero results. Zero.

Devon: Right. And that is because, well, maybe six months ago, someone in the accounting department filed the document as a bill instead of an invoice.

Marcus: Exactly. Or you search for holiday, and the system completely misses the critical human resources form because it was filed under annual leave. I mean, it is an everyday, entirely mundane failure.

Devon: It is.

Marcus: But it happens constantly because human beings naturally remember meaning and intent, but traditional search engines, they only know how to match exact literal labels.

Devon: Yeah, it's a profound mismatch between human memory and machine storage. I mean, when we store information in our own minds, we categorise it by what it is actually about. We remember the shape of the idea.

Marcus: The vibe of the thing, basically.

Devon: Exactly, the vibe. We remember the frustration of the late payment, or the relief of the upcoming time off. But for decades, when we asked software to find something, we were forced to play this miserable guessing game.

Marcus: Trying to read someone else's mind.

Devon: Right. We had to try and recall the exact sequence of letters that someone else typed into a file name on a random Tuesday afternoon.

Marcus: And that structural failure is exactly why we are dedicating this sixth instalment of our beginner series for non-coding tool builders to making a massive leap.

Devon: It is a massive leap, yes.

Marcus: Because in our previous instalment, we established the filing cabinet approach, right? The supreme power of matching exact labels for exact tasks.

Devon: Which is brilliant, to be fair, if you're retrieving a specific client ID or a serial number.

Marcus: Oh, absolutely. But building truly useful systems requires leaping from matching labels to matching meaning, and I think that's the core of what we're looking at in this deep dive.

Devon: It is. And to really understand that leap, we need to compare a card index to a human librarian.

Marcus: Okay, let's unpack that.

Devon: Well, the filing cabinet from our previous discussion operates exactly like a library's card index. If you walk into an old archive and you know precisely what you want, the author is Melville, the initial is M, the card index is flawless.

Marcus: Right, you get the reference card, you walk to the exact shelf, and there is the book.

Devon: Exactly. But if you walk in and ask that same index for, you know, the book about the sea captain who ruins himself chasing a whale,

Marcus: The index gives you absolutely nothing.

Devon: Nothing at all.

Marcus: Because the index requires a precise string of text. You brought a theme, a human meaning, and the system just looks at the letters, shrugs, and fails.

Devon: Precisely. Now, give that exact same query to a librarian. She hears sea captain chasing a whale, and she is already walking you towards the shelf for Moby Dick.

Marcus: Right.

Devon: But then she pauses. She understands the underlying theme you are circling. So she says, well, if it is obsession that interests you, you might also want this book about a mountaineer, and perhaps this one about a chess prodigy.

Marcus: So she returns three books that share barely a single word between their titles or their back cover descriptions: a sea captain, a mountaineer, and a chess prodigy.

Devon: Exactly. They sit next to each other in her mind because they share a meaning. The card index matches what you typed; the librarian matches what you meant.

Marcus: That's a brilliant way to frame it.

Devon: And semantic search is the technology that gives a modern software system that librarian.

Marcus: Huh. But, and this is the mission of this deep dive, we have to establish a crucial warning for anyone building these systems.

Devon: Right, because there's always a catch.

Marcus: Always. The map of meaning that your librarian uses and the sources she has told to trust are not silent defaults you should just inherit. They are deliberate, often complex decisions you, the builder, must make.

Devon: Well, let us pull the mechanism apart first. Because if we are saying the machine does not actually understand us in a human, conscious sense, what is it actually doing when it searches for a plumber's invoice but correctly finds a document about a leaky pipe?

Devon: Right, it seems like magic.

Marcus: It does. If there's no human cognition happening, how does a machine replicate the librarian?

Devon: Well, we can explain the entire mechanism without requiring any maths at all.

Marcus: Thank goodness for that.

Devon: I know, right? Think about how a digital screen displays colour. Any colour you can possibly imagine on your monitor is just made up of three numbers.

Marcus: How much red, how much green, and how much blue.

Devon: Exactly, the RGB value.

Marcus: Every web designer's bread and butter.

Devon: Right. And because every colour is just three numbers, similar colours have similar numbers. Sky blue and powder blue sit right next to each other in the numbering system.

Marcus: While mustard yellow sits a long way away.

Devon: Exactly. By turning a colour into numbers, we have successfully turned the concept of similarity into a measurable physical distance.

Marcus: Oh, I see. So the AI is doing the exact same thing to language. It is assigning a coordinate to a word.

Devon: We do exactly that. When text goes into a modern AI system, a model reads it and assigns it a position on a map of meaning. But instead of just three numbers like a colour, it uses hundreds, sometimes thousands of numbers.

Marcus: Thousands?

Devon: Yeah. Imagine trying to plot a word on a map that has 1,500 different axes.

Marcus: Yeah.

Devon: That position, that massive string of coordinates on the map, is called an embedding.

Marcus: Okay, so if I am picturing this map, the word dog sits right next to puppy and labrador, they are clustered tightly together.

Devon: Yes, exactly.

Marcus: And a word like invoice sits on entirely the other side of the map, perhaps clustered near tax and receipt.

Devon: Yes. About the same thing simply becomes near each other on this map.

Marcus: Uh-huh.

Devon: And because nearness can be measured mathematically, the system can search by it.

Marcus: Right.

Devon: When you ask a question, the system turns your question into a coordinate on that exact same map, and then simply draws a circle and fetches whatever documents are stored nearby.

Marcus: That is semantic search.

Devon: That is semantic search. It is the literal engine behind every single chat with your documents feature that exists.

Marcus: I love how that strips the illusion away completely, which is your favourite recurring move, by the way.

Devon: I mean, it is vital to state clearly. The common story in the industry is that AI search is a magic understanding machine. The reality is that it is measurable geometry built from language patterns.

Marcus: Measurable geometry.

Devon: Yes. It is not human cognition. It is proximity on a mathematical map. And it only works well when someone consciously picks the map and ranks the sources.

Marcus: But okay, if this is purely geometry, someone has to draw the map, right? And the assumption, especially in the tech industry, is always that bigger is inherently better.

Devon: Oh, always.

Marcus: If I am building an internal tool for my company, my immediate instinct is to look at the brochure, find the most complex, most detailed map with the highest number of dimensions, and buy that one. More detail must mean a smarter librarian.

Devon: And that is precisely the instinct you must resist. The map is a choice, not a given. You have to choose which AI model draws the map and how detailed it is. And bigger is absolutely not automatically better.

Marcus: Why not? I mean, if an embedding with 3,000 numbers captures more nuance than an embedding with 300 numbers, why on earth would I want a simpler, less nuanced map?

Devon: Because every extra number in that coordinate costs more to store and, critically, takes longer to search.

Marcus: Ah.

Devon: Think about the physical mechanics of the filing index required to sift through those coordinates instantly. When a user hits enter, the system has to measure the distance between their question and every single document in your database.

Marcus: Right, so if you have a million documents and each coordinate is 3,000 numbers long,

Devon: The computational overhead is staggering. Past a certain size, fast filing indexes, which are often called vector databases, simply break under the weight.

Marcus: They literally choke on the math.

Devon: They do. The latency spikes, the system crawls to a halt, or you have to pay exorbitant cloud computing fees just to keep it running.

Marcus: Just to search your own files.

Devon: Exactly. Experienced builders do not pick the biggest number on the brochure. They pick the most detail their system can handle comfortably.

Marcus: The human stakes of this get very real for an organisation if they ignore that, though. Let us say I am a beginner, I am building a tool for my team, and I just let the system pick the map by default.

Devon: Which is very common.

Marcus: Right? I use an agent like Claude Code or Codex, and I just type add document search to my tool. What actually happens if I accept that silent default, and then six months down the line, I realise the system is painfully slow, costing a fortune, and I need to change the map?

Devon: Well, you trigger what is essentially a re-shelving crisis.

Marcus: A re-shelving crisis? Just the phrase sounds like a massive invoice waiting to happen.

Devon: It is incredibly expensive in both time and computing power. Remember, every single item you have stored, every document, every folder, every corporate note, has been positioned in the specific mathematical terms of that original map.

Marcus: Because it assigned a coordinate to everything.

Devon: Exactly. So if you decide to change the map later, you cannot just flip a switch in a settings menu.

Marcus: You have to redraw the map.

Devon: You have to redraw every single coordinate for every piece of information in your entire library. You have to pay an AI model to read your entire company history again and plot it on the new map.

Marcus: Oh, wow.

Devon: Yeah, you are completely re-shelving a library of millions of pages.

Marcus: So a casual silent default made on a Tuesday morning while testing some code results in a massive engineering and financial overhaul six months later.

Devon: Exactly. This is why the choice of map and its dimensions belongs in a formal decision record, complete with the reasoning for why it was chosen.

Marcus: That makes total sense.

Devon: When you use AI agents to do the wiring, because they will happily fetch the embedding model and set up the store for those positions without you ever computing a single coordinate yourself,

Marcus: Yeah.

Devon: you must hold the gates.

Marcus: This is where you have to use the plan mode we discussed in previous instalments.

Devon: Precisely. Before you let the agent unleash its default settings and quietly shelve 10,000 company folders, you ask it to output its plan.

Marcus: You check which map it has chosen.

Devon: You check the map, you verify its size, and you verify why. You make the choice deliberate rather than accidental.

Marcus: Okay, let us assume you do that perfectly. You pick your map, you shelve your library economically, and your system can now find things by meaning. But if we are relying entirely on geometric proximity to find answers, we hit a genuinely horrifying edge case regarding trust. I am thinking of the late payers policy scenario.

Devon: Ah, yes. This is the perfect example of the distinction between relevance and authority.

Marcus: Right.

Devon: Let us say your company has a formally agreed, signed-off policy on how to handle clients who pay late. It has been reviewed by legal, it is the official stance, and the document exists in the system.

Marcus: Standard corporate procedure.

Devon: But also in the system is a late-night, caffeine-fuelled email from an executive, musing out loud to a colleague about completely changing the late payers policy to something radically different.

Marcus: Perhaps something wildly aggressive.

Devon: Exactly. And both of those documents are about late payers. So, geographically, they sit right next to each other on the map.

Marcus: They do. To the embedding model, they share the exact same theme. A naive semantic search system will look at those two documents and treat them as absolute equals.

Marcus: And here is where it becomes horrifying and amusing in equal measure.

Devon: I thought you'd enjoy this bit.

Marcus: Well, think about it. A new employee asks the internal chatbot, what is our policy on late payers? Because the system treats them as equals, it might decide to confidently quote the 2:00 AM frantic brainstorm email, presenting it as settled company law.

Devon: Yes.

Marcus: And the employee then forwards that aggressive nonsense directly to a client. I mean, who actually pays the price when a confident digital librarian hands out a stray thought as official policy?

Devon: Well, the organisation pays the price, instantly, through reputational damage and internal chaos. But there is a technical mechanism to prevent this, and it is called authority weighting.

Marcus: Putting a thumb on the scales.

Devon: Exactly. We know that relevance is not authority. So we build rules that weight sources differently before the system even reads them.

Marcus: How does that work in practice?

Devon: Well, a signed-off policy document automatically outranks a stray email. A formal decision record outranks a brainstorm transcript. We could also use the honesty tags we covered in instalment three.

Marcus: Oh, right.

Devon: If a document is explicitly tagged as a wish or a draft rather than a fact, the system can automatically rank it lower when fetching answers.

Marcus: Okay, I am going to push back on this quite hard, actually.

Devon: Go on.

Marcus: You are calling it authority weighting, which makes it sound like a tidy, objective, technical fix. But deciding that a formal policy outranks an executive's email, or, more realistically, deciding that the legal department's strict compliance memo outranks the marketing department's optimistic sales guide, that is a massive, messy, human value judgment. You are hiding a vicious office political fight inside an algorithm. It is not just a technical setting. You are coding company politics into the search ranking.

Devon: Well, I will not deny it is a value judgment. But it is one that must be formalised into a technical rule. As a practitioner, my view is that the maths itself is tidy, even if the human decision behind it is fraught.

Marcus: But framing it as a technical fix allows management to pretend they are not making a subjective choice. They say, oh, the algorithm prioritised legal over marketing.

Devon: But the builder knows the truth.

Marcus: Exactly. A human being ranked the legal folder a 10 and the marketing folder a 5. If you do not formally own that decision and document it in your project paperwork, you are letting the machine decide what is true based purely on geometric proximity.

Devon: Which is far more dangerous than having the political fight openly. That is my point. It is the builder telling the librarian who to listen to.

Marcus: But if you avoid making that messy human decision openly, your tool is just a liability waiting to happen. You just inherit the chaos.

Devon: Look, we may have to disagree on how tidy the process is, but we entirely agree on the outcome. An unguided missile is worse than a negotiated compromise.

Marcus: Fair enough.

Devon: And if you fail to weight your authority, you guarantee failure in the ultimate currency of these tools, which is adoption.

Marcus: Adoption.

Devon: Yes. We have extensive field notes on this from engineers who build massive memory systems for global organisations. And the findings are stark. Accuracy is not actually the most important factor in whether a team trusts and uses the tool.

Marcus: I have absolutely seen this fail in practice. We once looked at a team that built a highly accurate, beautifully tuned tool, and nobody touched it. It turned out accuracy was not the problem at all. People just refused to trust it.

Devon: Because receipts matter more than accuracy.

Marcus: Receipts?

Devon: Yes. Systems that show their sources, systems that tell the user exactly which folders and documents they drew their answer from, they get trusted and used.

Marcus: While the other?

Devon: Systems that answer confidently but act like a black box and refuse to show their working get quietly abandoned by teams, even when the confident black box is usually giving the correct answer.

Marcus: Ah, the fall of the confident black box. It is such a pure human behavioural reality.

Devon: It really is.

Marcus: People do not actually need the digital librarian to be infallible. If she hands you a piece of paper and says, here's the answer, and here is the exact page of the employee handbook I found this on, you trust her.

Devon: Because you could verify it yourself in three seconds.

Marcus: Exactly. But if she just stares at you, recites the answer from memory, and refuses to say where she read it, your human instinct is to assume she is hallucinating.

Devon: Transparency earns trust. Unchecked confidence destroys it. The receipts matter more than the mathematical accuracy of the map. If the user cannot click a link and see the original document for themselves, the semantic search has failed its primary objective, which is augmenting human work.

Marcus: So, we have covered the leap from matching exact labels to matching meaning. We have covered the map dimensions, the sheer expense of the re-shelving crisis, the horrors of unweighted authority where drafts become policy, and the absolute necessity of bringing the receipts to earn human trust.

Devon: A lot of ground covered.

Marcus: And to lock this entire architectural shift in, you have a concrete takeaway for everyone listening, don't you?

Devon: I do. It is a very short paper exercise. It takes five minutes, and it requires no software tools whatsoever.

Marcus: Okay, let's hear it.

Devon: I want you to write down five short lines on a piece of paper. They can be anything from your daily life: a sentence from an email you received this morning, a task on your to-do list, a random fact you learned, a rule for your household. Just five separate, short sentences.

Marcus: Okay, five random lines.

Devon: Now, write down one question. The catch is, none of your five lines can answer this question word for word.

Marcus: Right.

Devon: But two of those lines must genuinely relate to the theme of the question. When you look at your paper, you will know instantly which two lines relate to the question, even though the words do not overlap at all.

Marcus: That instant realisation.

Devon: Yes, that immediate flash of connection in your mind. That is semantic search. An embedding map is simply a machine attempting to geometrically replicate the human instinct you just used to connect those ideas.

Marcus: That's a brilliant way to visualise it. And to finish the exercise, I want you to look closely at those five lines. Imagine that they actually contradicted each other. Imagine they violently disagreed on the facts.

Devon: Which happens in business every day.

Marcus: Constantly. Look at them and rank them. Which one would you trust first? Which one would you trust last? Why? Because the moment you do that, you have just written an authority weighting.

Devon: Exactly.

Marcus: You have just performed the exact human value judgment that belongs in your project paperwork, forcing the machine to respect your hierarchy of truth.

Devon: There is no magic here. The system's intelligence is entirely dependent on the builder making those deliberate choices. The geometry only works when a human draws the map and ranks the voices.

Marcus: The machine can build the shelves, and it can measure the distance between the books with terrifying speed, but it takes a human to know the difference between a masterpiece and a rough draft.

Devon: Perfectly said.

Marcus: And that leaves me with one final thought for you to chew on. If we spend the next decade successfully mapping all our corporate and personal knowledge into these vast geometric spaces, and we train ourselves to rely entirely on the digital librarian to find the connections between our ideas, what happens to our own ability to draw those connections?

Devon: A daunting thought.

Marcus: Are we building a perfect map of meaning, or are we just offshoring our own human intuition to a server farm? Until next time.