SAASM (Software As A Slot Machine)

Pulling the AI lever and hoping to win big

I’m in pretty deep on a project. My laptop is hyperventilating, and the people next to me seem concerned. Claude has been firing on all cylinders for twenty minutes. The project I fully understood twenty-one minutes ago looks completely different now. I did my best to steer Claude in the right direction, following all the classic advice, going deep on the plan before letting it run wild. It starts to slow down; I stare at “flibbergibitting” until Claude spits out a “Here’s what I changed:”. Now it’s time for me to fulfil my side of the contract. I pick up my phone to test the changes. Something is not quite right. I pull the lever again and send Claude to chase down the issue. As it finishes up, I notice the fix is, again, not quite right. Time to pull the lever again. So much has changed; is it really worth it for me to sift through all the changes by hand when one more prompt could solve my problem with almost zero effort?

At every point of our work with an LLM, we are making a decision on how much of a partnership the work is. We can always dive into the nuance and apply our own reasoning and knowledge to the work to see if we can fix it ourselves. We can use language models to help us explore and understand the work. We can also ask those models to just do the work. The correct level of partnership varies across tasks, and it’s completely up to us to decide.

The more autonomy we give LLMs, the harder it is to take back. It’s high risk, high reward. A task that might have taken us a full day to complete could be finished in a few seconds when we hand the reins over to AI. This feels like magic. If the model makes a mistake at this level of autonomy, the reasonable next move is to prompt it to fix that mistake. This can loop indefinitely. At any moment you might be a single prompt away from a solution, but you could also be staring down the barrel of a full day of arguing with a machine.

Anyone who has spent time working with LLMs knows how dumb they can be. All it takes for our trust to be restored is one impressive, quick task completion. Gell-Mann amnesia is the effect of easily recognising something inaccurate in your own area of expertise1, and at the same time fully trusting something from the same source regarding a subject matter with which you are unfamiliar. In other words, we are quick to forget the shortcomings of AI the moment it solves a problem for us.

If a language model can solve our problem with a couple of extra prompts, it can be difficult to take the time to dive into the details. We have to remember that our value in knowledge work begins where the model’s capabilities end. Critical thinking is our superpower.

In an ideal world, we could hand the reins over to a model until it hits a roadblock. Then, we could drop in quickly to offer some advice, unblock it, and move on to something else while the model finishes its task. The issue comes with our ability to understand the huge amount of output a model usually has. To give thoughtful advice, we need to have a good understanding of the work being done. To have that understanding, we need to take the time to understand all of the model’s output. In the time it takes us to do that, we could probably have solved the problem on our own.

We need to resist the path of least resistance. Understand the limits of models and our own strengths. Use the models as subordinates to which to delegate mindless work. Learning when to hand off and when to stay in control is the most important skill to develop in the current AI era. One more pull on the LLM slot machine might spit out the answer you want this time, but it’s not a reliable long-term strategy.

Think for yourself. Learn for yourself. Use tools as tools.

Hop on the subscriber train to hear next time a post is leaving the station

1. Thanks, Jack, for telling me about this