OpenAI dev day

Day 278 / 366

OpenAI recently had a dev day, an event where they announced new features for developers who use OpenAI models and APIs to build products. These were some of the main announcements

Realtime API

This is a great way to write conversational applications using LLMs. For instance, if you wanted to create a bot that talks to your user, you would have had to first use a model to convert the user’s audio to text, then feed the text to the LLM to get a response. And finally, you would convert that text into audio and send it to the user.

With the Realtime API, the model natively supports speech-to-speech conversations, with no text intermediary. This greatly increases the speed at which you can respond to the user.

Prompt Caching

A lot of the time we make multiple API calls with mostly the same prompt with a few things changed. Prompt Caching reuses these commonly seen input tokens and thus saves up to 50% of costs for the developers.

Fine Tuning for Images

We use fine-tuning in order to improve the output of an LLM for a certain use case. Until now, OpenAI only allowed us to fine-tune text outputs. They have now introduced fine-tuning for images as well. You do it the same way as text, by providing OpenAI with a set of prompt and image output pairs.