The first 40 months of the AI era

The first 40 months of the AI era

Here are my accumulated thoughts and ideas about AI since the launch of ChatGPT in November 2022.

40 months of ChatGPT

OpenAI first launched ChatGPT at the end of November 2022, nearly 40 months ago. I remember trying it at the time and being really impressed, just like everyone else. At first I just talked to it, and I remember being absolutely amazed. I remember what it was like to talk to older chat bots, more primitive programs like Cleverbot. ChatGPT was better, much better. So much so that it was immediately obvious that this was not going to be just another toy for internet nerds, the rest of the world was going to notice.

I experimented with asking it to write, to create content. I first asked it to create poems, then Dungeons and Dragons backgrounds and even a whole fantasy world including important characters, kingdoms, and lore. It was very impressive, just for the fact that the output was coherent but its ‘style’ was very boring and overtly inoffensive, which was (and still is) a clear limitation of the technology.

I was listening to the Linus Tech Tips WAN Show immediately following the launch and hearing Luke mention that ChatGPT was being prompted to produce fully functional programs. This made me very curious and I wanted to try prompting ChatGPT to test its capabilities. I first asked for simple hello world programs, which it produced perfectly. I was very impressed and continued to play with it, and it didn’t take long for me to realise that this bot could produce genuinely useful code snippets for common and well understood use cases. For simple things it was able to replace my typical research loop, I no longer needed to search on StackOveflow or other discussion forums for these kinds of solutions.

I remember the first time I vibe-coded a small project. It was an app that generated placeholder cards for my MTG collection. I prompted the bot (now Claude, not ChatGPT) to create an app that would fetch for the card metadata from an API, generate a qrcode, and correctly layout this information into a printable page of cards. The first output was very impressive, and mostly worked. I attempted to make adjustments with additional prompting but was unable to make meaningful progress. I gave up on using the bot, and finished the project on my own. Through each iteration of the project I replaced more of what the bot wrote with my own code. The final result hardly used any AI generated code at all. I debated with myself whether I had actually saved any time or effort, compared with having just done everything myself from the very beginning. Coding AI has come a long way since then, but even now I’m constantly asking myself just how much is this actually useful for coding?

Claude Code, my Review

Two months ago I purchased a Claude Pro subscription for the first time. Claude had been my free chat bot of choice for over a year and I had become increasingly curious about Claude Code. My initial impression was intensely positive. I immediately installed Claude Code on my workstation and began experimenting. I remember the initial excitement from being able to speak naturally to my computer. So long as I was careful to clarify my intent, I was now able to tell my computer what I wanted it to do and it would consistently do what I asked. It felt then (and now) like a brand new form of input and control of my computer along side my keyboard, mouse, and even command line terminal. I have a lot of doubts about using AI, but not for this use case. This is unambiguously good, useful, and just amazing. I would be very pleased to see this technology become fully commoditized. I would love to have a local LLM loaded on my GPU, or on a separate desktop appliance, that can do just this, and do so this well.

And of course I also tried vibe coding using Claude Code. The results were again very impressive. For the small projects that I attempted, I was able to get a good one-shot result (just like I did previously). This time the iterative prompting felt much more productive. The Claude Code interface eliminates the friction of copy/pasting from a chat interface, the bot can just make the edit itself. I was impressed with model’s ability to maintain coherency and context. I was amazed when it created solutions or found bugs that I failed to see. But even on seemingly simple projects it felt like I was struggling to keep it from eventually losing the plot.

I also tried to use Claude Code to help me start a business. Since I lost my job as an IT Technician last year I had considered building my own small IT services company. I tried to use Claude as a combination of executive assistant and mentor. I asked it to create a detailed pre-launch plan that I could follow, and then to track my progress. In hindsight it all seems painfully obvious and simple, but I must admit that the mere process of creating the plan was very inspiring and created a lot of confidence in me. I did manage to launch the business, and despite not yet attracting customers (except for friends and family) I’m continuing to work on it, but mostly without any ai assistance.

For my use of AI in this instance, I believe the term du jour would be ‘glazing’. I agree with the consensus view that this is a major concern with AI use, but at the same time I must acknowledge that I have a bad tendency to procrastinate, and creating that pre-launch plan did genuinely drive me towards action. If I end up making any money at all through this company then I will have to attribute that in part to having been ‘glazed’ by an AI.

How useful is AI really?

While I’m certain that this technology is producing some productivity improvements, I’m still genuinely (and frustratingly) unsure just how much of an improvement it is actually creating. I still replace a lot of the work that the bot produces, but not all. I also question the way I think about a project when I’m building it with an AI. To what degree did I expand scope because I knew I could do more using the AI? If I did expand scope, to what degree was that necessary or merely piling on more ‘nice-to-have’ features? I don’t believe the litmus test for AI should be time savings, that there can be value in adding more features or polish to a project within a fixed deadline, but by god does it feel difficult to judge. For now at least I’m keeping my Claude Pro subscription, but given the persistant rumors of undisclosed rate limiting, and ever improving performance of local LLMs, I can easily imagine that I’ll cancel my subscription before the end of the year.

Generating AI Content

As of now there is zero written content on this site that was generated verbatim from an LLM. I’ve experimented with prompting an AI to produce content for the site, but I just can’t bring myself to use even a single sentence of it. AI generated prose is at best boring, and at worst genuinely unappealing. I’m continually tempted, because in theory it should work well. The AI has perfect spelling and grammar, has more than enough context to produce article-length content, and can do in seconds what takes me hours. It always ends up being a waste because fundamentally this is not my writing, and the whole point of this blog is to a place for me to write.

Consuming AI Content

As a reader, viewer, or listener I find AI generated content really unappealing. The concept of the uncanny valley is directly applicable. It’s so very close to being convincingly human-made, but just different enough to create an uncomfortable sense of dissonance. When I see it, read it, or hear it I become instantly disengaged.

I keep telling myself that there is some way to make it work, a workflow or set of ground rules that can make this into a great and useful tool for creative output. A part of me wants to imagine, for example, that a small team of game developers could build the next AAA mega-hit game on a shoe