ChatGPT allows us to automate faster, but better?

I am looking at what has been shared about the capabilities of Large Language Models.  I’m just as impressed by what they’re capable of delivering as everyone else.

But.

What this brings to the fore for me is that the opportunities that LLM offers are very much a double-edged sword. While the capability to very rapidly build apps / code / automation is present, there’s an underlying assumption (and risk!) that what is produced by a LLM is “good”. 

How do we know if it’s “good”? 

Good doesn’t equate to functionality alone. “Good” has many facets. Being functional is one of them, but so is security, fragility, explainability, performance, maintainability… you get the idea.  So what does this mean for the automation world? (I have to work that angle in here, because it’s where my brain goes).

What it means is that the energy once spent on developing automation can now be redirected. Where?  To the activities that make automation maintainable, perform better, explainable, secure. The thing is, LLM don’t (directly) help with any of these problems, but what it does do is free up the time and attention we need to apply to solving these problems.  They also offer a way to ask the LLM questions that help with each of these disciplines.

What are we going to spend our time on?

In my opinion, explainability – the ability to explain not just what the code is doing, but how it is doing it – becomes the most critical skill to build and refine.   Being able to explain what is happening means that you are able to:

• develop tests that  ensure that the generated code does what it’s meant to.
• make assessments of the security or the behavior of the code and the surrounding ecosystem.

In short: if you are looking at LLM as being the way to do less work, you might be in for a rude shock. Yes, you’ll be able to outsource the generation of code, but the sustainment of the generated content is going to be where you spend your brain power.