Trying out more Open Source AI models
Day 19 / 366
I came across this post on Twitter today -
While the topic of the tweet itself is interesting, what caught my attention was that this person was able to fine-tune and host an open-source model, and use it as an API for his application. This was something that I thought was impossible last week, it’s just so tough to get information about AI stuff.
The person mentioned 2 platforms — Together AI and Helicone, so I decided to check those out.
Helicone AI
This appears to be a way for you to log the responses you get from your LLMs. I had been using just a Firebase db to do the same, and so far it had been free for me. I don’t understand the need for something like this, to be honest, but it is fully open-source, which is a good thing. Setting this git repo locally will surely be a great learning experience. If you want to use their hosted version, you get 50,000 requests per day for free, so should be good enough for my small apps for now.
Together AI
This site is exactly what I was looking for.
- It gives you the option to access 100+ open-source models through an API
- It gives you an OpenAI-like playground to test these APIs
- It allows you to create fine-tuned models as well
The pricing is very lucrative as well
So for a 70B model, i.e. a model with 70 billion parameters, you will pay 0.1 dollars for 1 million tokens.
Compare this to GPT-3.5 is 1.5 dollars for a million tokens, and GPT-4 is 40$!
Now of course, you won’t get as good a response with these models as you would with GPT. But that is where fine-tuning comes in. You can quite cheaply fine-tune a smaller model on your custom data and make it an expert in that particular domain.
So far I had been spending around 30 dollars a month in OpenAI credits for the few apps that I had made. I will work on migrating them to fine-tuned open-source models and see how much I can save.