I had to take a road trip to Fredericton for work in the early part of this week, and with all of the OpenAI activity going on, it ended up being an inopportune time to spend ten hours in a car.
I'm usually pretty good about not having many tabs open in my various browsers on my various devices, and I even stay on top of my RSS feeds and news podcasts, but the last few days have broken everything. I now have dozens of tabs covering all sorts of takes and news updates and such on multiple devices.
I'm finally getting through them, and hopefully then I'll have read enough to be able to come up with some opinions of my own about the whole situation, which can maybe then take up a tab in someone else's browser.
In the meantime, I want to share two recent articles from the New Yorker which have interesting takes on AI and LLMs. The first is "Is My Toddler a Stochastic Parrot?" a charmingly illustrated and tastefully animated piece that is more profound than it may seem at first. For me the money quote is:
A toddler has a life and learns language to describe it. An L.L.M. learns language but has no life of its own to describe.
A lot of the infighting at OpenAI have been about different visions of how to approach Artificial General Intelligence, and it's concepts like this that make me think we're still quite a long way from that.
Airplanes and birds both fly, but they do so in very different ways. An airplane can fly faster and higher than any bird, but it's less nimble, and can't do it fuelled by worms and seeds. Computers can "think" in much more powerful ways than humans do, but they do it in a fundamentally different manner.
The second article is "A Coder Considers the Waning Days of the Craft" which many people have recommended to me, as I'm a coder as well, and have been wrestling with how to work with tools like ChatGPT and Copilot. I disagree with him on some key points though:
When I got into programming, it was because computers felt like a form of magic. The machine gave you powers but required you to study its arcane secrets—to learn a spell language. This took a particular cast of mind. I felt selected. I devoted myself to tedium, to careful thinking, and to the accumulation of obscure knowledge. Then, one day, it became possible to achieve many of the same ends without the thinking and without the knowledge. Looked at in a certain light, this can make quite a lot of one’s working life seem like a waste of time.
Maybe it's because I came to software development from a humanities background, but I've never liked the "mastering arcane secrets" aspect of programming. I want to do things and make things, ideally in as clean and simple a way as possible. There has been a lot of talk about how AI can write a lot of "boilerplate" code for you - but in my opinion if you have lots lots of boilerplate code to write you're programming wrong!
The biggest jump in programmer productivity I experienced in my career was when I switched from working with Java/JSP and PHP to Ruby on Rails back in 2005 or so. In the older languages and frameworks, I had to hand-craft every database query, manually set up every form and screen, and write lots of code mapping front and and back end data together. The big insight with Ruby on Rails is that the code could analyze the database schema for you and handle all the data mapping for you. You could define an entire data model with just a few lines of code, and the framework itself could automatically generate basic forms and listings for you.
The original demo from 2005 is still striking:
(DHH has been problematic over the years and Rails has had its ups and downs too, but the demo is still revolutionary, especially for 2005.)
In my experience, the challenge of building software isn't so much figuring out how to do things, it's properly understanding what needs to be done. People have been predicting "the death of the programmer" ever since COBOL came out in the 1950s. The job has changed quite a bit since then, but there's still a lot of us, and I think we're going to keep being busy for a while longer.
Anyhow, I need to go and close some more tabs now.