Image Credit: HITC
You can anticipate the integration of ChatGPT, the enormously well-liked AI chatbot, into an increasing number of your favorite tools and apps. The chatbot’s developer community now has access to ChatGPT’s features thanks to a new API from OpenAI, the company that created it.
In order to keep investors happy, OpenAI, a business with capped profits, had to find a way to make money off ChatGPT. When ChatGPT Plus, a premium service, was introduced in February, it took a step in that direction. And it took a bigger step by announcing an API that will let any company incorporate ChatGPT technology into their applications, websites, goods, and services.
Greg Brockman, the president and chairman of OpenAI (as well as one of the co-founders), claims that the idea was always to use an API. Prior to the official release of the ChatGPT API, he had a video chat with TechCrunch. He said, “It takes us a while to get these APIs to a certain quality level, I believe it’s more or less just a matter of being able to handle the scale and the demand.”
Image Credit: goo
According to Brockman, the gpt-3.5-turbo AI model, which powers OpenAI’s enormously successful ChatGPT, also powers the ChatGPT API. The “turbo” moniker refers to a tuned, more responsive variant of GPT-3.5 that OpenAI has been covertly testing for ChatGPT. GPT-3.5 is the most potent text-generating model that OpenAI currently offers through its API suite.
Brockman asserts that the API can power a variety of experiences, including “non-chat” applications, for a cost of $0.002 per 1,000 tokens, or roughly 750 words. Snap, Quizlet, Instacart, and Shopify are the early adopters of ChatGPT.
It’s possible that lowering ChatGPT’s enormous compute costs was the primary driving force behind the creation of gpt-3.5-turbo. Sam Altman, the CEO of OpenAI, once described ChatGPT’s costs as “eye-watering,” putting the computing costs per chat at a few cents. Brockman asserts, however, that gpt-3.5-turbo is enhanced in other ways.
“You never want an AI-powered tutor to simply provide the student with an answer, that’s an example of the kind of system you should be able to build [with the API]; you want it to always explain it and aid in their learning, Brockman said. “We believe that this will simply increase the API’s usability and accessibility.”
A reflection of the skewed data on which ChatGPT was initially trained, users were able to get ChatGPT to respond to inquiries in racist and sexist ways early on. ChatGPT’s training data is comprised of a wide range of internet content, including articles from Wikipedia, Reddit posts, and e-books. Additionally, ChatGPT engages in what is known as a hallucination in AI, which is the practice of creating facts without revealing that it is doing so.
Systems like ChatGPT are also vulnerable to malicious adversarial prompts or prompt-based attacks that trick them into performing tasks that weren’t part of their original goals. Finding ways to “jailbreak” ChatGPT and get around any security measures that OpenAI implemented has given rise to entire communities on Reddit. One of the less offensive instances involved a Scale AI employee convincing ChatGPT to divulge details regarding its internal technical operations.
Image Credit: Search Engine Journal
Without a doubt, brands wouldn’t want to be the target of an attack. Brockman is certain that they won’t be. He claims that one factor for this is ongoing back-end improvements, some of which have come at the expense of Kenyan contract workers. Brockman, however, focused on a fresh—and unquestionably less contentious—approach known as Chat Markup Language, or ChatML, from OpenAI. The ChatGPT API receives a text from ChatML as a series of messages with accompanying metadata. In contrast, the standard ChatGPT consumes unprocessed text that is represented as a collection of tokens. (The tokens “fan,” “tas,” and “tic” would be used to denote the words “fantastic,” for example.)
We are upgrading to a higher-level API. I should anticipate that, as a developer, you actually can be more robust [using ChatML] against these kinds of prompt attacks if you have a more structured way of representing the input to the system, where you say, “This is from the developer” or “This is from the user,” Brockman said.
Additional changes include more frequent model updates, which should stop unintended ChatGPT behavior. Beginning with the today-released gpt-3.5-turbo-0301 version, developers will be upgraded to OpenAI’s most recent stable model by default, according to Brockman. The advantage might be somewhat offset by the fact that developers can stick with an outdated model if they choose so.
Brockman points out that the introduction of dedicated capacity plans will give some customers — primarily big businesses with correspondingly big budgets — greater control over system performance, whether or not they decide to update to the most recent model. OpenAI’s dedicated capacity plans, which were just released, allow customers to pay for an allocation of compute infrastructure to run an OpenAI model, such as gpt-3.5-turbo. These plans were first described in the documentation that was leaked earlier this month.
In addition to giving customers “full control” over the instance’s load—calls to the OpenAI API typically occur on shared compute resources—dedicated capacity enables features like longer context limits. Longer context limits essentially allow the model to “remember” more text because they refer to the text that the model takes into account before producing new text. Higher context limits may not completely address the bias and toxicity problems, but they may reduce hallucinations in models like gpt-3.5-turbo.
According to Brockman, dedicated capacity customers can anticipate gpt-3.5-turbo models with up to a 16k context window, which translates to four times as many tokens as the ChatGPT model’s standard capacity. That might allow someone to paste in pages and pages of tax code and get reasonable answers from the model, for example—a feat that is not currently possible. In the distant future, but not soon, Brockman hinted at a general release.
That wouldn’t be overly surprising given the mounting pressure on OpenAI to make a profit following a multibillion-dollar investment from Microsoft.
A largely automated system will take the place of the company’s pre-launch developer review process. The change was justified by OpenAI by noting that “the vast majority of apps were approved during the vetting process,” and that its monitoring has “significantly improved. Brockman told TechCrunch that one of his team’s main goals has been to figure out how to be as welcoming as possible to developers. Our goal is to create a solid foundation upon which other people can build their businesses.”