About ChatGPT Plugins

ChatGPT plugins are wild. Basically, you can extend ChatGPT with sensors and actuators; plugins that allow ChatGPT to search and scrape the web, to call APIs to fetch and even update traditional IT systems like flight reservation systems, and to even execute code generated by the LLM.

This is impressive on many fronts. It lets one augment the LLM with many capabilities implemented within traditional IT systems without the user leaving the chat system to manually follow instructions that ChatGPT produces, and to execute (inline) code that ChatGPT has generated. As an example, this allows ChatGPT to retrieve knowledge from the internet that is outside of its training set - a weakness of the LLM otherwise, and something that is believed by many to be the next big step in LLM capabilities. Moreover, it should allow a more direct form of memory than just augmenting the text content of the chat, by invoking a key-value storage API.

But what might be most impressive is how it is implemented. You give ChatGPT a description of the API (OpenAPI format) and a manifest that each describe what the API is and does, how to use the API including what the interface is and how to authenticate, and… that’s basically it. The LLM learns how to integrate, all by itself, in what is surely another portent of the nature of IT work changing in the future.

This is impressive for a few reasons. First off - this scales. Writing lots of glue code, and somehow having developers manage the intersection of an LLM and a structured, normalised API, would just not work well. Secondly - OpenAI could well have developed something more proprietary. A lot of people mock OpenAI for not being open in any sense - but they didn’t make a choice here to strengthen their competitive head-start by making exclusive and closed integrations with API providers. Finally, that this works at all is just very surprising to me. Given that OpenAPI document descriptions, and the associated code, is very common on the internet (and therefore in training data avaialble to OpenAI), perhaps it shouldn’t be - but this just blows my mind that ChatGPT can not only interface with another system in a schema-valid way, but also understand (to some likely currently-limited degree) what the side effects of doing so are - what the response is and means, how it fits into the broader goal of the chat, and so on.

LLMs continue to be one of the most popular technology developments of our time. Many have likened the importance of AI’s recent development to mobile, to Internet, to the development of early UIs - Bill Gates declares the Age of AI Has Begun - and coming off of crypto and web3 fizzling out, it would be easy to be skeptical, but it’s very easy to see real value to users here. As a user, if an AI model can automate a big chunk of my day, I will willingly pay a big chunk of my pay to do so. There are, ofcourse, many risks here, and many jobs will change and many will be lost, and the potential for inquality is great, let alone misuse. However - the potential benefits still shine through all of that. This is a very exciting time to be alive.

Liked this article? You can reach me on email or Twitter.