I thought of an API which will mainly be an AI wrapper, with some other stuff going on- as a workflow automation.
But has anyone figured out how to price stuff like this? Because the chatGPT API as we all know is credit based, and I'd like to just do a subscription.
Is it better to set up a serverless deployment of an open source LLM, and then just pay hosting fees instead of having a potential bottleneck of hitting a credit limit with a 3rd party AI?
Thanks!
One option is to use langchain or another abstraction to finish your MVP/sales demo with OpenAI then set up your own model and do fine tuning after you have your first client. I saw a blog post about someone fine tuning llama with output from ChatGPT but I’m having trouble finding it now.
It seems to me that OpenAI agents are so expensive that you can’t really have markup on it or determine a fixed price. I created a chat demo for my client and we went over $40/mo with 4 people testing it out over a week.
Admittedly not an LLM expert but happy to be a sounding board if you’d like — kd at StackStudio.dev