How LLMs change software engineering

In the build vs buy decision, the cost of "build" has gone down significantly, and this will have huge ramifications on all parts of the software supply chain

Ok so there are two triggers to this post.

First, yesterday, I came across this blogpost by Vaughn Tan about “boring tiny tools” - that with LLMs it is now possible to write bespoke software for a single user, since the amount of time taken for it is not much.

To make the module work properly would either require expensive customisation by sales engineers, or time-consuming pre-processing by Yuko’s team to convert vendor documents to fit into the standard module. It would be so troublesome or expensive that they’d decided to work around it by sticking to the spreadsheets they’d been using for years. She asked: “Why can’t we just have something small that does exactly what we need?” Then she sighed.

Then, later in the evening, I started my own vibe coding project. This is a system that I’ve wanted to build for a very long time now. Basically after every meeting I’m buzzing with ideas and want to record my thoughts. Sometimes when I’m walking or driving or commuting as well i want to record my thoughts. And then be able to parse and access it.

So I’ve built a simple system called “VoiceNotes” (I’ve put the code on Github; my apologies for having some hardcoded path links there; feel free to leave a PR fixing that) where I leave voice notes to myself on my iPhone. Then, when it comes into the same wifi network as my mac, the note gets synced to my Mac, where it gets automatically transcribed (using Whisper) and summarised into bullet points (using a local LLM, built on Ollama), and then gets put into a folder in my Apple Notes (I imagine that not too long into the future - or even if I get a newer phone (I’m on iPhone 13), I might be able to achieve the entire workflow locally on my phone; or maybe I can if I use APIs).

It took me maybe an hour of vibe coding to get this up. And then another hour of vibe coding to debug the app and actually get it working. The app is built using Swift and Python, with some strange gymnastics on xCode (I can read python but not really write it; I have absolutely no clue about Swift. So this is something I could have never done without Claude Code (or equivalent) ).

That’s it - I had a need for a product, and in some 2-3 4-5 hours of coding time (including debugging) the system was up, and usable. I don’t need to, any more, rely on some third party building an application that fits exactly my current needs, or take something that is close enough and modify it. I can write an application for this precise use case.

Enterprise software

So these two things (the blogpost on boring tiny tools and my app) got me thinking on how enterprise software will evolve, if it is possible to cheaply make pieces of software that satisfy one individual user.

At the core of it, it turns the build vs buy decision on its head, by making the “build” part significantly cheaper (even if you take into account the cost of maintenance, which we sometimes ignore while calculating the build cost). So at the margin, companies will start seeing value in building more things in-house, or getting more things custom-built, rather than buying stuff off the shelf and integrating.

Typically (correct me if I’m wrong) large traditional enterprises don’t have technology teams - they have IT teams. The job of the IT team is to identify pieces of technology that can serve their business well (could be products or services built outside), and then procure them and make things work together. The part of making things work together is typically handled by external “system integrators” (typically large IT companies like Infy or TCS), with the internal IT team program managing.

This outsourcing of the tech (both products and services) means that large traditional companies are slow-moving when it comes to tech. They have periodic “tech transformation projects” which sometimes take years to complete. This implies that large enterprises have traditionally been far behind state-of-the-art when it comes to tech.

The good news about “build” suddenly becoming cheaper is that enterprises need not be tech-backward as they have been so far. (In theory) They don’t need massive projects for firmwide transformation - they can instead bring in lots of “boring little tools” for each particular use case - as long as their architecture allows for it, that is.

What does this mean for everyone?

Ok here I’m pretty sure I’m hallucinating, or at least speculating, on how different parts of the chain of enterprise software will change based on this.

Software Product Companies

There will be fewer products bought, in aggregate, since build is now so much cheaper. And the products that are bought can be more easily modified using “boring tiny tools” - so this means product companies themselves don’t need to offer infinite customisation - that can be left to whoever implements the products.

Overall, bearish for software products. People claiming that (traditional; non-AI) SaaS will have a downturn have a legit claim, though everyone I know who is running a SaaS company has pivoted to some kind of AI or the other.

System Integrators

This will be interesting, actually. The large transformation projects might go away (since there will be fewer such transformation that are required), but these companies will remain as trusted partners to IT teams. And now that build is cheaper, they might be best placed to build / customise on behalf of their customers.

It needs a shift in mindset, and strategy, and capabilities, though. Apart from simply integrating existing systems, they’ll need to modify and deploy, which means smarter software engineering and use of AI. Knowing how they’ve operated through the decades, I expect them to make this transition, though they may not be particularly fast about the actual implementation of it.

Enterprise IT

Their job will get more complex. They will need a change in mindset since the build-buy equation is changing rapidly. “Build” here doesn’t necessarily mean build in house; IT teams are likely to rely on external consultants to do their builds, and as a consequence, will have more reliance on consultants than they do on product vendors.

What to build and what to buy gets more complex. Who is best placed to build a “boring tiny tool” is a question they constantly need to answer, and answering that needs new capabilities. Some enterprise IT teams will move faster than the others, and the skills required might be slightly different from what has been required so far.

Other technology consultants

It is possible that system integrators may not step up. In that case, IT teams will need someone to do the build for them. And for this, we might see newer (and possibly smaller) AI-enabled tech shops come up, that help customers quickly build (and maintain) boring tiny tools.

Their economics will be very different to that of the large established players (and they are unlikely to bill on person hour), but I expect them as a class to do rather well given how things are shaping up.

Business Intelligence

Well, this post is about tech, but I couldn’t help talk about my domain - at least here (unlike immediately above), I’m not talking through my ass.

Okay, I started writing this and realised I wrote enough in this one section for it to be a post by itself. So wait until tomorrow for this one!

Do we still need engineering?

I made a small edit in the beginning of this post. When I first drafted this, my app had taken 2-3 hours to build. Getting things more right took 2 more hours. And I had to constantly prompt it based on my requirements etc.

While I don’t understand Swift, it was useful to have my broad software engineering and product skills to prompt the LLM appropriately to make it make the correct changes. More importantly, this is still true, in conjunction with the maxim that “verification is always cheaper than creation”.

So even if vibe coding makes things more efficient, the efficiency gains are realised most by people who already have sufficient domain knowledge.