I had an "Aha" moment with AI - and I had some thoughts

I had a couple of "wow" moments recently with just how good AI is becoming as a tool for some parts of knowledge work.

The first was relatively prosaic. I was preparing for a meeting and I wanted to pull some slides together: a bit of analysis, some basic charts. I found a PDF document with the data I needed, selected the data I wanted, Ctrl-C, Ctrl-V into Claude and said: "Can you turn this into a bar chart and a pie chart in ppt?"

Maybe two minutes later, three very professional-looking slides appeared, showing exactly the data I needed.

I stopped to check: is it just hallucinating? Is it making this stuff up? No — it had pulled exactly what I needed from a completely unstructured copy of a PDF. Into great, professionally formatted charts. And then I wanted to make some changes, so I just dictated into Claude what I wanted: put this in italics, change the headline to say this, reformat the chart so the axis goes from here to here, add data labels here, and so on. Again, after maybe a minute, the changes were made.

I'm pretty good at PowerPoint — I started my career in consulting at the turn of the millenium. Creating those charts would have taken me at least 20 minutes per slide, maybe more. It did it in a couple of minutes. More importantly, it meant I could spend that time thinking about what the insights were.

Now, the slides were in a format of Claude's choosing — and interestingly, when I asked it to reformat the slides, it also completely reformatted them - changed the colour scheme, turned a vertical bar chart into a horizontal one, I think maybe changed the font - which was a bit odd and probably would have been quite frustrating if I hadn't just been doing this as a bit of exploratory analysis to share internally with one colleague. But I'm sure you could pass it a well-structured template and a set of rules: axes are always in this font, this size, this colour; charts always start here and go there; and so on. And it would create them in exactly the form you wanted.

Very impressive.

The second felt like a much bigger "wow".

I recently read the EnergyNet paper, and listened to an epidode of the Volts podcast with the writer, Jonas Birgersson. It's an interesting concept (and I might write about it another time). One thing that I found really striking, though, was a claim that Birgersson makes that having one kilowatt-hour of battery storage could reduce the size of the electrical connection required by an average Swedish house from 16 amps to one amp.

I was genuinely surprised by that; so, in bed, just before I went to sleep, I asked Claude a few questions

First, I asked whether it could find me peer-reviewed or other references to that claim. Then I asked it to explain the mechanism to me as though I had the physics knowledge of an A-level student (I do...or did...) — because I wanted to make sure I actually understood it. And then I asked whether it could build a model to simulate the same principle for the UK:

If I wanted to model that type of reduction in supply capacity for a real UK electricity network, could you help me do that? I’m thinking I’d need to use PyPSA, and collect data on a real or representative distribution network (eg a substation).

It asked me a few questions about my preferences for the modelling, and then I went to sleep. The whole exchange took about two minutes.

When I woke up in the morning, it had built the model (with what looks like well-written and well-documented code), found appropriate data to use in the model, run it, produced neat graphical outputs in matplotlib, and provided a write up of the method (including the key parameters and how the code worked), the conclusions, and suggsted next steps - while I was asleep. Like a genie; or the Tailor of Gloucester's mice. It was, genuinely, amazing.

The graphical output is below:

I might put the code up on github in case anyone's interested, and also include Claude's write-up. If I do, I'll add a link here.

I might also do some more modelling to explore these questions further - again, I'll update / write something else if I do.

Some Observations and Thoughts on the Implications and Observations

As a result, as people say, I "have thoughts". And I'm sure many of these observations already out there in the discourse, with plenty of people making them, and I'm not claiming they are particularly original or full of deep insight - but it was the first time they'd struck me in that real "aha" way. I'll use the example of models, but I think the implications may well generalise to a wide range of related capabilities.

1. We're going to get more and more "analysis slop"; and more and more "vibe analysing".

When I say "analysis slop" I mean it in the same way "slop" is being used about AI-generated output in general: output that looks superficially like good quality analysis, based on well-thought through and rigorous modelling, but which hasn't had any quality control at all except what the first-shot at asking the AI for output produces. I expect a flood of "white papers" that look like this.

When I say "vibe analysing" I mean something more like what people are talking about when they talk about "vibe coding": tools or outputs that are genuinely useful to the person creating it, that enable them to build something bespoke to meet a particular need of theirs, that they wouldn't have had the capability or capacity to achieve before.

2. I think this has a couple of implications.

Firstly, the scope for people who know a little about an area (say, energy system modelling, or financial modelling, or...) to start building analytical models and drawing conclusions from them has just expanded hugely. I also think that, if you use them well, these tools are really good didactic partners - so it's possible to build capability surprisingly quickly: there's very little cost to asking a stupid question to an LLM, and they explain things well and their patience is only limited by their context window (though of course one has to guard against the risks that (i) you don't know when they're hallucinating if you're outside your area of competence; and (ii) they can be very persuasive and that can lead you to feel like you understand something more than you actually do). Overall I think this is a genuinely productive phenomenon.

Secondly, the scope of interesting questions that can be answered has expanded. Whether being done by an expert or an auto-didact, the effort required to just build the thing - as well as to create professional-looking output that humans can understand - has dropped massively. That means there's a lot of capacity now available to explore a whole range of questions that we previously just didn't have time/resource to look at - things that need simulation and complex modelling that are currently in the "too hard to understand and model, so we just won't bother" — that space is probably going to shrink dramatically, because the human cost of specifying, setting up, and checking those models has gone down a lot. And as a result there are going to be a huge number of questions that have been waiting for modelling / analytical horsepower that are suddenly going to be opened up. Which I think is quite exciting.

3. As has been observed by pretty much everyone in relation to computer programming, the value of knowing the syntax is dropping to zero. Being able to build — and maybe also being able to specify — a model or a piece of code is probably on a path to dropping towards zero in value, because Claude, ChatGPT, and whatever Microsoft has can just do it for you.

Now, I've often been struck by the insight of Clayton Christensen's 'law of conservation of attractive profit pools'. So with that in mind, if the value of knowing the syntax is going down - what are the profit pools that are going to increase in resopnse?

4. Possibly paradoxically, the value of expertise and the value of a brand that stands in as a promise for that, is probably going to go up. If anyone can now do this kind of modelling, how as a customer do you know which output is valuable and which is just slop / "vibe-based"? In an era when building the thing was complex and difficult, to an extent being able to do it at all stood as your credentials. That's no longer the case. So actually knowing what you're talking about - really knowing what matters in these kinds of models or analysis; and having a brand that people trust, may become relatively more valuable, not less.

5. Some IP has probably dropped in value — particularly anything well-specified, well-documented, and in the public domain. Note, though, my conclusion above: in many cases the value was always more in the expert knowledge and brand, than the IP. However the pricing model may well change (pricing based on time rather than the value of the output/outcome for the client is going to come under a lot of competitive pressure for this kind of work; though maybe value-based-pricing will too).

If you're a company that owns proprietary modelling IP, the risk of being out-competed by open-source alternatives has probably just gone up a lot. And if you were charging for people to do the modelling — rather than for the software itself — your margins are going to be eroded by competition. So you either need truly proprietary IP that the AIs can't replicate (likely in scope?); or you need to adopt these same techniques to make your modelling and analytical offer cost-competitive, time-competitive, and scope-competitive.

6. What does the ability to do more analysis actually mean for decision-making? I'm not sure the bottleneck on good decisions is, in most cases, the quantity or rigour of analysis. Sometimes it is. But sometimes it's just the ability of the decision-makers to take on board the data and analysis, agree on what it means, and decide on a way forward. That part doesn't get sped up. In some ways, if you can suddenly do loads more good analysis, it might actually slow decisions down: in organisational cultures that suffer from 'analysis paralysis', the ability to do more anlaysis, more quickly, may well be a hindrance rather than a benefit.

7. Finally, what does all this mean for people? Again, lots of people are writing about this. One thing I'm experiencing at the moment is a touch of the Claude-induced mania: I've got a growing list of projects and questions I'd like to look into, that I can now get AI to help me with in really pretty effective ways. And that sparks further ideas, further requests of Claude, further outputs...

I'd also like to be optimistic about the implications - along the lines of e.g Benn Stancil's substack essay from today (Side note: I'm a Stan and heartily recommend Benn's essays) - that AI, particularly in the context of knowledge work, could make the work more fun.