The problem with AI coding

If AI has a better understanding of programming than most of us, why is the code AI writes so bad?

I think the biggest problem with completely vibe-coding some software is that the code is just unreadable. And the fundamental reason behind this is both AI’s biggest strength and weakness: it is very easy for AI to execute.

Back in 2010, well before AI coding was a thing, a group of people in JetBrains invented Kotlin because Java was a really bad language. Writing all the boilerplate was a waste of time, but that could be solved by an IDE with auto-code generation. More importantly, reading it was a waste of time. If you and I and even the IDE can tell you, without looking, what the boilerplate will tell you, it is a complete waste of everyone’s time to actually have the boilerplate written down in the source code. It can be compiled! This (and other things, such as null pointer exception safety) was the premise behind Kotlin. Now, writing apps that target the JVM is no longer a massive pain in the ass.

Do you really think Kotlin would have been created today? Now you can just tell Claude “edit the boilerplate in X way.” But I think we would have lost something if Kotlin had not been created. Interacting with Java would still be bad, and AI would merely have papered it over in the short term.

The reason Kotlin was created was because every line of code you manually write is costly. At the very least, it takes a good amount of time to physically type out each line of code. To put it in ML terms, there is a large penalty attached to the number of lines of code you write. Even if unintentionally, you will naturally gravitate towards the most simple and concise way to write code.

AI does not have nearly the same cost of writing code. It is very easy for AI to drop a thousand lines of code where a hundred would have done the job. Do you believe the actual codebase will align with your mental model of what it should be if it is verbose? I personally doubt it.

Why does AI prefer verbosity? Can’t you just RLHF it away? The problem is people don’t even value brevity as much as they should. You and I value brevity in the things we code or write in large part because it is easier to be concise. We take these lessons when we read what other people wrote and recognize that if something is concise, the author is more likely to know what they are talking about.

But the average person does not write, so they do not punish verbosity nearly as much as they ought to. You’ve probably heard that SAT Essay scores or AP exam essay scores are incredibly positively correlated with the length of the response.¹ For the same reasons, the longer responses actually get favored, and this likely translated over to code generation as well. Now AI labs are trying to fix this, but it might be too little too late.

What can you do about this? The most effective countermeasure is to not let AI touch your code and only perform research for you. But if you are in a situation where you want or need to write code fast (likely any software-related job you will work now), this may not be feasible. In that case, the next best thing you can do is just keep an eye out for verbosity and have your instructions punish the model for being verbose. (Ironically, this might make your prompts more verbose, but prompts are ephemeral, so this is less worrying than having your code be verbose.)