Garbage in, garbage out.

It seems much of the discussion on Hacker News circles around AI — usage of it, the latest model releases, thought pieces about it (this one included)…

Amongst all that I came upon an interesting quote:

Getting interesting information into it is still going to be just as slow as it is in the regular world. You have to build, tinker, experiment, get data, get knowledge, etc.
— https://news.ycombinator.com/item?id=50166775

This reminds me of yet another truism in many aspects of life (though it certainly does apply strongly in programming). This time from the Magic School Bus:

[The class is gathered around a computer terminal. Carlos’ cousin Mikey had scripted all of their school-opening chores into a program, but it went sideways and ran in circles. They printed out the code for review]
Carlos: Here’s your program, Mikey.
Mikey: Looks good to me. “Unlock and open the doors”, “Raise the Flag”, “Turn on the sprinklers”, “turn off the sprinklers”, recorded message, printer, coffee machine, bell.
Wanda: But what’s this? “Repeat everything every minute, except on Saturday and Sunday”.
Mikey: “Every minute”? It’s suppose to say “every day”. I… I made a mistake!
Ms. Frizzle: It’s the only way to take chances and get messy.
Carlos: No wonder everything is turning on and off every minute. The computer is doing exactly what you told it to do in the program. But if we put garbage in…
All: We get garbage out!

It’s easy to get sucked into the illusion that an LLM has catalogued all of the world’s knowledge. Certainly that seemed to be the case when search engines promised “the world wide web at your fingertips”. AI researchers have certainly gotten better at both the training (reinforcement learning, post-training, etc.), as well as improving their inputs, and so as of writing we’re trending steadily in an upward direction.

The concerning question is whether we’re still at the stage where we’re picking low hanging fruit (low effort for comparatively high gain), or whether we’re rapidly reaching that last 10% of optimization that takes 90% of the effort. I’m not asserting that there’s a “100% complete” end point for LLM research, just that as with most efforts, there’s a point at which no additional value can be extracted, and so you must continue in a different direction.

All that is to say that the fundamental problem exists within all large language models, that to put out a competent model, you need a lot of training data, and the training data currently available will never be perfect. Garbage in, garbage out.

This is why I still believe that there will be a place for software engineering and large language models to co-exist. It’s not an existential question of whether we will be supplanted by these intelligent and sometimes surprising computer programs, but whether we as an industry are able to utilise our existing skills in order to work *effectively with these tools in order to push past the limits of human cognition.

This — for the same reasons — is why I also believe that there are limits to vibe coding. LLMs abstract away the deterministic and highly opinionated and syntax-driven programming languages in favour of natural language. It’s easy to think that this is a natural extension of the human-computer interaction (a keyboard, how quaint!)

Crucially, though, it is not a stretch to also realize that human language is lossy, and that lossiness means that you yourself are fully-capable of providing that garbage in.

← Back to all posts