My GPT project has become a local hit but I can't afford the bill

There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere.

Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a day. $500/month".

You'll know in like an hour if there's any advertising potential.

You could try to apply for grants. 20K+ chats sounds like a large number -- you must live in a 1M+ person metro area, right? It would probably be fairly easy to get a four figure grant from one agency or another. You could frame it as "localization of generative ai to the regional culture of X", or as a tourism thing, etc.

You could also sell advertising the old-fashioned way -- ask local businesses or perhaps a local university if they'd like the model to mention them in certain prompts.

But your best bet is probably to treat this as a loss leader and put up a small banner offering "In-Person Generative AI Consulting" in your metro area.

If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times.

You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.

You've fallen into a trap categorising yourself as a "chat app". If you're not connecting 2 different end users to talk to each other then you shouldn't call yourself a chat app.

Put a meter on the site, when the meter runs out, put up a timer that a user can bypass for everyone by chipping in N more $ to hit the goal

Traffic is most likely a spike. In a month it will probably be down to a few percent of that. I wouldn’t shutdown or optimize too much unless you see it persist for a few more weeks at this level.

If your lucky and it’s still going strong after that time you’ve got something worth more than your hosting bills.

Seems like a very low effort approach would be to create a few unobtrusive but visible ad spaces on your site, and make the ad banners say:

"## eyes see this space every day; rent this ad space. contact ..."

As for what to charge or how to structure the ad sale, do a little research or hire someone who knows that business (and give them a cut of ad revenue or some other attractive compensation).

Odds are high that this will go through a hype cycle and usage will taper off naturally in a week or two.

But the donations route could also be a fun way to lean into the local element of it if the cost remains high. Could have a simple line of text when the page loads that thanks one of the donors for that day.

Hey,

What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result.

Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt.

This might decrease quality, but maybe in "enough" cases, it does just fine.

I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB

Premium subscribers. Offer a token benefit to people who pay, can even be something worthless like flair on a username or listing their name in the “credits” or “hall of fame”. Then advertise it on your site and make it easy for people to pay.

Congrats on having a side project that is popular enough to break the bank - it's stressful to be sure, but you're a cut above the rest.

> I've already optimised the prompt and reduced the number of tokens I'm sending

Lets assume you've found, or will find, all possible technical optimizations for your AI spend on ChatGPT.

You can:

* Ask for donations

* Accept a $50/day cost for X days as a market research budget to find a monetization plan

* Have some "first 5 questions are free, next are paid (or whatever)" to rate limit cost while maybe raising revenue

* Look for a common pattern of questions to see if there's some value / insight into your users that can be used as a source of revenue

* Find a cheaper LLM that supports your needs while being serviceable

* Pull the plug because its a joke, claim credit for it, and take the learnings onto the next thing

edit: grammar & formatting

Assuming this is GlasgowGPT (based on the timing of that bot taking off on /r/glasgow and this post, and your username), a challenge you'd likely have with advertising is that the content from the bot is, whilst funny, not very SFW.

One option to get the costs down would be to use a local LLM. It isn't likely to be as fast/good as GPT-3.5 but at least would give you a fixed cost per month (hosting).

If you had local hardware, given that traffic is going to be pretty low, if you have decent bandwidth at home and some hardware, you could even host there.

That’s enough traffic to attract local sponsorships.

I bet you could find local sponsors who can pay you on a monthly basis

People respond well to financial transparency. If you have a donation button, show real-time stats on the amount it costs, and the amount that’s been donated.

Also because internet, expect 90% of cost to be from abuse, and likewise 90% of your dev time devoted to fighting it.

Is it GlasgowGPT by any chance?

Either way, there's definitely an opportunity for funding. In the case of Glasgow, there's a huge populous out there that are descendants of folk from the city/country taking pride in it (arguably more than a bunch of us that live here!) and like to get in on the banter.

That's certainly an avenue worth exploring if there's a similar relationship with folks or families of folks that emigrated away, but feel a strong connection.

Edit: You'll have to excuse my unfamiliarity with LLMs, but if you're storing messages and responses, and if there's a way to check that a message has already been asked, would it be possible to replay that (at least on the first message) instead of running a new query against the API? If they then engage further, then it would send the job out?

Midjourney has a model that worked for me ... you can queue into the "free lane" which is slower and limited by total requests for the day, and I assume is adjusted for how successful they get and are willing to spend. Or, you can subscribe for a low fee and get instant access.

> It's been a surprise hit and now I've had 20,000 chats coming through in one day

> The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day

If your traffic continues to increase at a rapid clip, you may want to look into investors for your project. You could still keep it mostly nonprofit if you think it is adding a lot of value for people.

You could charge for heavy use. At that many users and only $50/day you might be able to cover costs with premium users.

Alternatively you could try swapping out an open source model like llama on the back end (possible licensing issues here).

Another option would be to seek out ads for local companies directly since it is an explicitly local product. Cold calling would probably be the best place to start. You could also pay attention to who is paying for billboards in your area and try contacting them.

Thank you all for the suggestions and the incredibly generous advice as always. Yes, it's GlasgowGPT, and yes it's pretty offensive (https://glasgowgpt.davidhewitson.co.uk)

I've decided to go with a big old banner ad for now as I think there are some local businesses who might go for it. Whatever happens, it's been a fun 2 week ride! Awrabest

Make ibizabear (or your real name the name) the name and likeness of the AI. Then just pay for it out of pocket for 90 days and become a local celebrity?

Adsense is content based ads. You can ask other ad networks. Their revenues is even higher than Adsense. https://www.reddit.com/r/adops/comments/uvkyoo/lets_summariz...

Find the most popular queries, built content pages around it and add ads there

And you could try some alternatives to AdSense like:

Ezoic, MediaVine, or Adthrive

Contact local businesses and sell ads injected into your prompt

Allow your software to display ads, and other people purchase them directly.

It might be hard to implement it in a week but a hastily put AWS & stripe solution might work.

If you do ever implement it, do mention it on HN, I would definitely buy couple of slots.

Sell local ads yourself. Have five rotating slots each for $2k/month. Should cover costs and give you some margin.

I've been wondering how to create a ChatGPT-powered app and not have to spend a fortune on operating it. Use a cheaper model? Limit the input/output length? Train a custom model?

I helped my friend build Quordle last year, and before he sold it, he was making a surprising amount off of just donations. Try sticking a kofi link up, it's a pretty quick change.

Go ping some Angel and VC investors.

Tell them you’ve proven the hypothesis and achieved product market fit by showing the data from this one city and say you’re raising capital to take it to the next step.

Good luck!!

Inform people to use ChatGPT through OpenAI's web interface and publish your prompt?

You should look at selling local advertising (and get creative at incorporating those businesses into the prompts for in-chat placements). $1500/mo to a local business to get 5,000 local eyeballs per day should be a no-brainer - and that's for a takeover-style ad that gives 100% of impressions to a single advertiser.

The smarter way would be to open up sponsor opportunities for certain categories, for example: restaurants, retail (tire shops, hardware stores, clothing stores), common service industries (plumbers, electricians, landscaping, website design, general contractors, computer/cellphone repair)- then sell those placements for $200-500/mo to whoever wants to have the placement for that.

Replicate this for other cities and you have a healthy business - you can hire people to sell these ad spots in their respective regions and you can actually scale it up. Billboards work in the same way. Those billboard companies might even be interested in selling their placements to your viewers.

Two ideas.

Switch out the backend, there are open source alternatives, and the results could be "good enough".

Get a list of local sponsors, offer them 1k ad placements for X dollars/euros/etc... , and inject them into your prompts.

As it is a joke site, I don't believe people will mind if the chat has ads, especially if you mention somewhere that it's financially hard to keep the site up.

A simple way to monetize tools like this is to create a queue. You specify the amount of requests or concurrent sessions that you feel comfortable with, and then you make it possible to pay to «skip» the queue. This way people that really want to try it out will be able to be patient and wait out the queue, and the impatient rich people can pay for the rest.

I asked gpt for and it say partner with local business to promote them in exchange for some support. Have them give you coupons, so that they can track roi. Bill them ten cents or smth for every coupon uttered by your model. You can also make it fun, like a treasure hunt, but there are also other gamification opportunities.

I think the only sane model for openai wrappers is user's bring their own API key.

Ask it to be more concise in its answers if possible. This saves a ton of tokens. You can also include instructions to flatly refuse tasks like writing short stories, but this might also make it less fun for your users.

Tell users that in order to use your service they have to give you their API key.

Let me help you to reach out local businesses with cold emails to check if they are willing to support you. I have a cold email SaaS called Nureply and your situation can be a great case study for us.

Reach out to a monetization partner. They'll have dozens of ways to help find ad demand quickly. It will be more work than AdSense, but it will work- unlike AdSense. Here's a list of options: https://www.google.com/ads/publisher/partners/find-a-partner

Look for resellers or programmatic direct services to get started.

Lot of new LLMs that run locally are available now

My idea for something like this is just to have a donation pool. Once the donation pool dries up, the app stops sending OpenAI calls. There's still some cost to hosting the API, but it won't be $50/month.

If someone wants to do a SaaS for this kind of thing, that would be great too.

What about introducing monetized scarcity?

Make users to be able to use X tokens per month for free and offer them to buy tokens that would keep your business going.

You'll divide your audience and current traction. You'll loose many but also have a way to make your business sustainable.

Get 1-3 local businesses to buy a premium ad spot on your app. You can craft the ad with them and place it (side of the chat, top of the chat, etc). Charge them anywhere from $300-$1500 per-month. Get a few of those and that should cover you and then some.

What alternative ad platforms have you looked at? E.g. does AdForm allow using with llm apps?

I'd wait to see if it tapers back down before doing anything. Spiked usage is a pretty common thing for LLM apps.

If it persists or you think it will, reach out to me at parham@bearly.ai and we'd be interested to sponsoring it or helping how we can :)

Love this project. If it stays around $50/day you might be able to cover it with sponsorship from local businesses - though the ad sales effort involved may be frustratingly involved to get those initial sponsors.

Are you sure your app is used just by people in your city rather than abused by projects like gpt4free (where they use your API for purposes other than chatting)?

Is it this? https://glasgowgpt.davidhewitson.co.uk/

Charge for it. Simply charge each user $1 and cover your costs. Sounds like it's a fun tool but if you want to sustain it then get people to pay.

Wrap it as iOS or Android app with ads and send to Appstores. Even with integrated ads like Admob you can easily cover your expenses

Taxi companies, restaurants, gyms - see if you can get some people to pay you to put some native ads on your site for a while...

What about a credits type setup?

You get a few free uses each week and then you need to buy credits/sign up and unlock more?

Are the inputs and the responses in a non English language? LLM APIs can get costly for non English, sometimes as much as 10x due to more tokens being consumed. Not sure what's the solution here.

Also, maybe you can use sometime kind of caching combined with some mbeddongd search to serve the previous response, if the input is similar above a certain threshold.

Hello. Email me. I would love to know more. Best wishes, Klaus sekretariat@me.com

Reach out to local businesses and charge them to be featured on your homepage.

Not sure if "chat apps" are banned, but try checking out nitropay?

Patreon. Give everyone free n messages per day. Beyond that, patrons only.

Find a local business who want to show ad in your website.

How did so many people find out about your app?

If you want to try some open source LLMs, try our free serverless tier on PostgresML.org! You can run any LLM from HuggingFace and fine tune your own.

Would you consider a micro-acquisition? If you have traffic and interest why not sell for $1-$3M to a sucker VC?

Affiliate pricing perhaps?

jhdgsfgh

Not sure what you run this on and why its $50/day but sounds like a lot.

Convert your server cost to a pay as you go model. You are currently paying even for time when your server is idle. Containerise your app and run on something like AWS Lambda which has a pay for what you use model.

- s3 is cheap for storing your model data and as long as you are just reading that should be fine.

- AWS Lambda has 1 million free invocations per month in free tier.

- Post that you pay $0.60 per 1 million requests

- AWS stack is just an example, you can easily use GCP Cloud run or any other equivalent service.