Six months of working with AI
Figuring out how to use AI has been my personal obsession for the past six months or so. The website just got rebuilt, so it feels like a good moment to look back and jot down some thoughts.
My rough impression: AI is fast. Absurdly fast. But it doesn’t make you capable of things you couldn’t do before (at least not yet). To put it another way, the level you used to reach only after endless hours of work, you can now reach remarkably quickly.
Try to go beyond that, though, and you tend to run into problems — flaws in the software design, or cases where you don’t understand the subject well enough to give clear instructions — and end up stuck in territory you can’t really manage. That’s been my experience, anyway.
So before you get into that mess, you do things like:
- abstract the design,
- break the problem down,
- have AI write tests for the obvious parts so correctness is isolated,
and basically use techniques that free humans from having to think about the lower-level details. Otherwise you end up buried under an enormous pile of code you don’t really understand.
In that sense, what humans should do is shifting quickly. Up until now (at least in my world), people who could write programs and logic were the valuable ones. Going forward, I think what matters is:
- people who can make design calls about which features matter and which to cut,
- people who can intuitively sense what’s wrong when debugging,
- people who can break AI out of the tunnel vision it sometimes gets stuck in,
— at least in software engineering. That’s my current hunch.
That said, humans can only hold so much in their head at once, so I’d rather let go of the low-level details as much as possible. Things like a vague sense of “I think we were talking about this at the start,” or “wait, does this match the system diagram we sketched early on?” — playing more of an intuition role is what feels best right now.
There’s a saying that comes up a lot in software engineering: “Occam’s Razor”. In the current age of AI-assisted development, I think this is a pretty important concept for the task of “keeping systems simple enough that humans can still intervene, instead of letting them spiral into unmaintainability.” The “KISS principle” is in the same neighborhood.
Things I’ve picked up over the past six months:
- Keep things simple. Express complexity as a composition of simple things.
- Keep the information density low enough that you can always understand roughly what’s going on.
- You can’t build something you can’t picture. Start by being able to imagine it.
These feel like things that mattered before and will keep mattering going forward.