8 ms·

Ask HN: My GPT project has become a local hit but I can't afford the bill

159 points by ibizabear 3y ago

Hello, I'm an HN user for 5-6 years but I wanted to use an alt for this as it's a bit sensitive. Last week I created a ChatGPT clone for my city. It's basically a laravel app hooked up to GPT3.5-turbo with a custom prompt that makes local references and talks in the way that people in my city talk. It's been a surprise hit and now I've had 20,000 chats coming through in one day - people where I live really seem to love it The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day, and Adsense doesn't allow 'chat apps', so I'm at a loss at how to cover the bill for this app. I've already optimised the prompt and reduced the number of tokens I'm sending The app a joke really, but it's a local joke that seems to be quite popular and connecting with my city. Should I try to raise donations? Is there an advertising provider I could use that would potentially cover the costs? Is there an alternative to OpenAI that is comparable to GPT-3.5 that I could self-host cheaper? Any and all advice appreciated. I don't care about profit - I just want to keep the app online so people can enjoy it.

furyofantares 3y ago
Contact local businesses and sell ads injected into your prompt
- bombcar 3y ago
  Or just contact some of the businesses and offer a classic banner ad on the site. If done right, it won't even be adblocked (because you can do it with no tracking at all). Depending on the size of the city, and the queries, you should have a good idea where to look. If not, check the local church bulletins, the local softball/school fields, and anywhere else you find local business advertisements.
- anonylizard 3y ago
  This. Sell it as a 'Google ads back in 2002', frame it in terms of high risk high reward. Sure its unproven, but back in those days, a lucrative click may have only costed $1,$2. When search ads got fully proven, they started costing $100/$200.
poopsmithe 3y ago
Patreon. Give everyone free n messages per day. Beyond that, patrons only.
pmoriarty 3y ago
Tell users that in order to use your service they have to give you their API key.
- twelve40 3y ago
  pretty sure a casual chat website that mandates that everyone give their phone number to OpenAI, activate their account, locate their API key (which is gibberish to most non-tech people) and come back and paste it would kill half the userbase, if not more
  - tester457 3y ago
    I'd be surprised if it didn't kill the app immediately, I rather set rate limiting and have the option to pay, or use your api key instead.
oldgregg 3y ago
Make ibizabear (or your real name the name) the name and likeness of the AI. Then just pay for it out of pocket for 90 days and become a local celebrity?
- cortesoft 3y ago
  Why would you want that?
bag_boy 3y ago
That’s enough traffic to attract local sponsorships. I bet you could find local sponsors who can pay you on a monthly basis
- silasdavis 3y ago
  Perhaps local events, extra charge for being integrated into the prompt
- borski 3y ago
  Local radio stations and tv networks love this sort of thing. “Brought to you by and sponsored by KQED” accomplishes a lot in terms of PR, and they would likely be willing to fund it, at least at cost but likely quite a bit higher.
deleted 3y ago
[deleted]
ShamelessC 3y ago
Inform people to use ChatGPT through OpenAI's web interface and publish your prompt?
DethNinja 3y ago
Allow your software to display ads, and other people purchase them directly. It might be hard to implement it in a week but a hastily put AWS & stripe solution might work. If you do ever implement it, do mention it on HN, I would definitely buy couple of slots.
Xcelerate 3y ago
> It's been a surprise hit and now I've had 20,000 chats coming through in one day > The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day If your traffic continues to increase at a rapid clip, you may want to look into investors for your project. You could still keep it mostly nonprofit if you think it is adding a lot of value for people.
calibas 3y ago
I've been wondering how to create a ChatGPT-powered app and not have to spend a fortune on operating it. Use a cheaper model? Limit the input/output length? Train a custom model?
- voidfunc 3y ago
  We're on Hacker News... find someone else to spend a fortune funding it for you.
- j45 3y ago
  See if you can find something close that can be self hosted Hugging face and others should help verify if anything can, or how close it is.
f0e4c2f7 3y ago
You could charge for heavy use. At that many users and only $50/day you might be able to cover costs with premium users. Alternatively you could try swapping out an open source model like llama on the back end (possible licensing issues here). Another option would be to seek out ads for local companies directly since it is an explicitly local product. Cold calling would probably be the best place to start. You could also pay attention to who is paying for billboards in your area and try contacting them.
- ugh123 3y ago
  To add to that, OP should really be tracking usage a bit to see if there are abusers taking up a large amount of queries
  - ibizabear 3y ago
    Good shout, thank you
PaywallBuster 3y ago
Find the most popular queries, built content pages around it and add ads there And you could try some alternatives to AdSense like: Ezoic, MediaVine, or Adthrive
- twelve40 3y ago
  this! finally advice that includes some specific online things not just generic "pick up the phone and cold-call local businesses". Another online possibility: throw up some Amazon (or similar) affiliate links to products and services, maybe even based on what people type (if that's accessible). And next to them "opt out of the ads" link that goes to a donations website, to get fancy. Helps clear the conscience about having the ads in the first place.
lagniappe 3y ago
Put a meter on the site, when the meter runs out, put up a timer that a user can bypass for everyone by chipping in N more $ to hit the goal
jamestimmins 3y ago
Odds are high that this will go through a hype cycle and usage will taper off naturally in a week or two. But the donations route could also be a fun way to lean into the local element of it if the cost remains high. Could have a simple line of text when the page loads that thanks one of the donors for that day.
colinnordin 3y ago
If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times. You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.
- serial_dev 3y ago
  I'm not sure it's practical and if it will result in any savings. Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks about Tom Brady. If there are ways around these issues, I'd love to hear it, but it sounds like this will just increase costs via cache hardware costs and any dedup logic instead of saving money.
  - Silasdev 3y ago
    >Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? The embeddings approach would increase the likelyhood of finding the same question, even if phrased slightly differently.
  - rjtavares 3y ago
    With embeddings you can compute distance. The questions don't have to be the same, they just have to be sufficiently close. Regarding context, that should be a part of the input for the embeddings.
- dxhdr 3y ago
  This removes half the magic of interacting with ChatGPT. Users will quickly realize they're interacting with a dumb database rather than an AI.
  - cloogshicer 3y ago
kyleyeats 3y ago
Ask it to be more concise in its answers if possible. This saves a ton of tokens. You can also include instructions to flatly refuse tasks like writing short stories, but this might also make it less fun for your users.
austhrow743 3y ago
There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere. Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a day. $500/month". You'll know in like an hour if there's any advertising potential.
- Ankaios 3y ago
  <darkside>Or charge even more for product placement in your project's prompts.</darkside>
  - schainks 3y ago
    I swear I've seen this somewhere...
    - Ankaios 3y ago
      Yeah, I didn't read far enough down the page before commenting. I must have been distracted by my tasty Coca Cola.
      - zuprau 3y ago
        Was it Coca Cola or Pepsi? The last time my family and I sat down for our usual blind test dinner everyone agreed Pepsi tasted better.
        trog 3y ago
        This is hilarious and people modding you down have missed the joke.
        swores 3y ago
        Or maybe they understood the joke but weren't amused and prefer not to see unfunny jokes on this site - people may have different opinions to yours without the cause being their not understanding the joke :)
kiernanmcgowan 3y ago
Congrats on having a side project that is popular enough to break the bank - it's stressful to be sure, but you're a cut above the rest. > I've already optimised the prompt and reduced the number of tokens I'm sending Lets assume you've found, or will find, all possible technical optimizations for your AI spend on ChatGPT. You can: * Ask for donations * Accept a $50/day cost for X days as a market research budget to find a monetization plan * Have some "first 5 questions are free, next are paid (or whatever)" to rate limit cost while maybe raising revenue * Look for a common pattern of questions to see if there's some value / insight into your users that can be used as a source of revenue * Find a cheaper LLM that supports your needs while being serviceable * Pull the plug because its a joke, claim credit for it, and take the learnings onto the next thing edit: grammar & formatting
- Davidzheng 3y ago
  [flagged]
stndef 3y ago
Is it GlasgowGPT by any chance? Either way, there's definitely an opportunity for funding. In the case of Glasgow, there's a huge populous out there that are descendants of folk from the city/country taking pride in it (arguably more than a bunch of us that live here!) and like to get in on the banter. That's certainly an avenue worth exploring if there's a similar relationship with folks or families of folks that emigrated away, but feel a strong connection. Edit: You'll have to excuse my unfamiliarity with LLMs, but if you're storing messages and responses, and if there's a way to check that a message has already been asked, would it be possible to replay that (at least on the first message) instead of running a new query against the API? If they then engage further, then it would send the job out?
- deleted 3y ago
  [deleted]
- ibizabear 3y ago
  Haha yes it is! And appreciate the suggestion, that's a good idea that I'll look into. I've plastered a big banner up and will see if anyone goes for it, from locally or elsewhere. Cheers
  - stndef 3y ago
    Hope it's of help! Had fun with it the other day, though was telt I was English for preferring salt and pepper over salt and sauce, haha!
  - samstave 3y ago
    Also, I wonder if Scotland has an agency similar to Ireland whereby they have a government VC fund to encourage startups... So there may be a .gov option https://thetab.com/uk/glasgow/2023/02/15/glasgow-uni-working-with-experts-to-see-how-chatgpt-can-be-used-to-help-students-study-26336 https://thetab.com/uk/glasgow/2023/02/15/glasgow-uni-working... https://www.investglasgow.com/ https://www.investglasgow.com/
villgax 3y ago
Lot of new LLMs that run locally are available now
hgarg 3y ago
Are the inputs and the responses in a non English language? LLM APIs can get costly for non English, sometimes as much as 10x due to more tokens being consumed. Not sure what's the solution here. Also, maybe you can use sometime kind of caching combined with some mbeddongd search to serve the previous response, if the input is similar above a certain threshold.
serial_dev 3y ago
Two ideas. Switch out the backend, there are open source alternatives, and the results could be "good enough". Get a list of local sponsors, offer them 1k ad placements for X dollars/euros/etc... , and inject them into your prompts. As it is a joke site, I don't believe people will mind if the chat has ads, especially if you mention somewhere that it's financially hard to keep the site up.
dmos62 3y ago
What alternative ad platforms have you looked at? E.g. does AdForm allow using with llm apps?
tlarkworthy 3y ago
I think the only sane model for openai wrappers is user's bring their own API key.
asim 3y ago
Charge for it. Simply charge each user $1 and cover your costs. Sounds like it's a fun tool but if you want to sustain it then get people to pay.
raesene9 3y ago
Assuming this is GlasgowGPT (based on the timing of that bot taking off on /r/glasgow and this post, and your username), a challenge you'd likely have with advertising is that the content from the bot is, whilst funny, not very SFW. One option to get the costs down would be to use a local LLM. It isn't likely to be as fast/good as GPT-3.5 but at least would give you a fixed cost per month (hosting). If you had local hardware, given that traffic is going to be pretty low, if you have decent bandwidth at home and some hardware, you could even host there.
- searchableguy 3y ago
  > One option to get the costs down would be to use a local LLM It's not that cheaper if you want similar level quality and inference speed. For commercial usage, OP will require training his own model since no good commercial alternative exists.
- baobabKoodaa 3y ago
  Not sure why you think local LLM would be fixed cost. Those things are very GPU heavy and you need to run multiple instances to serve multiple concurrent users, scaling up and down according to traffic.
  - raesene9 3y ago
    Fixed cost in that if you host on a machine you know how much you'll pay a month. Obviously capping it out based on usage. As opposed to API based where you pay more per use. Remember this isn't a commercial service that OP is creating, it's an amusing Chatbot, they have no revenue, so fixed costs are likely to be a better model.
    - baobabKoodaa 3y ago
      Local LLM won't give you "fixed costs" any more than using an API. If they want to serve the users of their amusing chatbot, they will have to scale up their local LLM according to scale, and they won't have "fixed costs" anymore. If they don't want to scale up and they're fine denying service to users at peak hours, they can do that with an API too. There's no law that says if you use an API then you must allow unlimited use. Of course they can also limit the use of the API.
opdahl 3y ago
A simple way to monetize tools like this is to create a queue. You specify the amount of requests or concurrent sessions that you feel comfortable with, and then you make it possible to pay to «skip» the queue. This way people that really want to try it out will be able to be patient and wait out the queue, and the impatient rich people can pay for the rest.
muzani 3y ago
My idea for something like this is just to have a donation pool. Once the donation pool dries up, the app stops sending OpenAI calls. There's still some cost to hosting the API, but it won't be $50/month. If someone wants to do a SaaS for this kind of thing, that would be great too.
alexei_rudak 3y ago
Wrap it as iOS or Android app with ads and send to Appstores. Even with integrated ads like Admob you can easily cover your expenses
ibizabear 3y ago
Thank you all for the suggestions and the incredibly generous advice as always. Yes, it's GlasgowGPT, and yes it's pretty offensive (https://glasgowgpt.davidhewitson.co.uk https://glasgowgpt.davidhewitson.co.uk) I've decided to go with a big old banner ad for now as I think there are some local businesses who might go for it. Whatever happens, it's been a fun 2 week ride! Awrabest
- danjc 3y ago
  Curious to know how it works out - will you comment here in a day or two?
throwuwu 3y ago
Premium subscribers. Offer a token benefit to people who pay, can even be something worthless like flair on a username or listing their name in the “credits” or “hall of fame”. Then advertise it on your site and make it easy for people to pay.
throwaway743 3y ago
Not sure if "chat apps" are banned, but try checking out nitropay?
Alexanfa 3y ago
Hey, What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result. Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt. This might decrease quality, but maybe in "enough" cases, it does just fine. I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB https://discord.gg/88xDVYbPVB
scrapcode 3y ago
Midjourney has a model that worked for me ... you can queue into the "free lane" which is slower and limited by total requests for the day, and I assume is adjusted for how successful they get and are willing to spend. Or, you can subscribe for a low fee and get instant access.
scosman 3y ago
Traffic is most likely a spike. In a month it will probably be down to a few percent of that. I wouldn’t shutdown or optimize too much unless you see it persist for a few more weeks at this level. If your lucky and it’s still going strong after that time you’ve got something worth more than your hosting bills.
ke88y 3y ago
You could try to apply for grants. 20K+ chats sounds like a large number -- you must live in a 1M+ person metro area, right? It would probably be fairly easy to get a four figure grant from one agency or another. You could frame it as "localization of generative ai to the regional culture of X", or as a tourism thing, etc. You could also sell advertising the old-fashioned way -- ask local businesses or perhaps a local university if they'd like the model to mention them in certain prompts. But your best bet is probably to treat this as a loss leader and put up a small banner offering "In-Person Generative AI Consulting" in your metro area.
- marsven_422 3y ago
  [dead]
- stanleydrew 3y ago
  Grants will take a long time to lock down I suspect. Maybe fine if you can float the cost for a few months and reimburse yourself out of grant funds later, but there's a lot of uncertainty. $50/day is pretty small relatively speaking, so just finding a local corporate sponsor and running an ad for them would probably work.
  - 411111111111111 3y ago
    why pick up the phone if you could just put down a placeholder banner saying something like "do you want your company name here? write [an-email-address-thats-not-your-personal-mail]". If the project is really becoming a local hit then you're bound to get contacted.
    - hutzlibu 3y ago
      This is likely the most easy way. Alternativly or additionally you can ask for donations on that banner. And a link where OP explains his situation, no profit, just costs, but like to keep the thing running.
levkk 3y ago
If you want to try some open source LLMs, try our free serverless tier on PostgresML.org! You can run any LLM from HuggingFace and fine tune your own.
userforcomment 3y ago
[flagged]
- replwoacause 3y ago
  Hmm… it seems kind of uncool to post this to me.
  - ibizabear 3y ago
    Cheers for your solidarity, means a lot (same goes to the other two guys who complained)
- Cyph0n 3y ago
  So you were able to extract the prompt. Hooray. But why did you post it on this thread?
- weird-eye-issue 3y ago
  And why did you post this?
jl2718 3y ago
People respond well to financial transparency. If you have a donation button, show real-time stats on the amount it costs, and the amount that’s been donated. Also because internet, expect 90% of cost to be from abuse, and likewise 90% of your dev time devoted to fighting it.
- Onewildgamer 3y ago
  I second this, since it is an app an online payment makes most sense. A simple donation button with paypal/venmo (or something the locals use) would make reasonable money to cover monthly infra expenses at least.
sebastianconcpt 3y ago
What about introducing monetized scarcity? Make users to be able to use X tokens per month for free and offer them to buy tokens that would keep your business going. You'll divide your audience and current traction. You'll loose many but also have a way to make your business sustainable.
meotimdihia 3y ago
Adsense is content based ads. You can ask other ad networks. Their revenues is even higher than Adsense. https://www.reddit.com/r/adops/comments/uvkyoo/lets_summarize_info_about_ad_monetization/ https://www.reddit.com/r/adops/comments/uvkyoo/lets_summariz...
simonw 3y ago
Love this project. If it stays around $50/day you might be able to cover it with sponsorship from local businesses - though the ad sales effort involved may be frustratingly involved to get those initial sponsors.
elintknower 3y ago
Would you consider a micro-acquisition? If you have traffic and interest why not sell for $1-$3M to a sucker VC?
globalise83 3y ago
Taxi companies, restaurants, gyms - see if you can get some people to pay you to put some native ads on your site for a while...
sarojmoh1 3y ago
How did so many people find out about your app?
dangwhy 3y ago
Reach out to local businesses and charge them to be featured on your homepage.
reissbaker 3y ago
I helped my friend build Quordle last year, and before he sold it, he was making a surprising amount off of just donations. Try sticking a kofi link up, it's a pretty quick change.
- dorkwood 3y ago
  Quordle is great! Can I ask what it sold for?
- nunez 3y ago
  Holy fuck y'all are awesome. Thanks for quordle; I play it every night before I go to sleep!
moneywoes 3y ago
Affiliate pricing perhaps?
graiz 3y ago
Sell local ads yourself. Have five rotating slots each for $2k/month. Should cover costs and give you some margin.
michaelteter 3y ago
Seems like a very low effort approach would be to create a few unobtrusive but visible ad spaces on your site, and make the ad banners say: "## eyes see this space every day; rent this ad space. contact ..." As for what to charge or how to structure the ad sale, do a little research or hire someone who knows that business (and give them a cut of ad revenue or some other attractive compensation).
parhamn 3y ago
I'd wait to see if it tapers back down before doing anything. Spiked usage is a pretty common thing for LLM apps. If it persists or you think it will, reach out to me at parham@bearly.ai and we'd be interested to sponsoring it or helping how we can :)
tutuphoe 3y ago
Find a local business who want to show ad in your website.
irfn 3y ago
Not sure what you run this on and why its $50/day but sounds like a lot. Convert your server cost to a pay as you go model. You are currently paying even for time when your server is idle. Containerise your app and run on something like AWS Lambda which has a pay for what you use model. - s3 is cheap for storing your model data and as long as you are just reading that should be fine. - AWS Lambda has 1 million free invocations per month in free tier. - Post that you pay $0.60 per 1 million requests - AWS stack is just an example, you can easily use GCP Cloud run or any other equivalent service.
- gbasin 3y ago
  the primary cost would be the OpenAI GPT-3.5-turbo API, which is ~ $0.002 per 1k tokens
  - laratied 3y ago
    I have made a chatbot like this and I am not a software engineer. The whole thing is trivial to make. The system prompt is trivial. I would even say it was harder to make a geocities page in the 90s than to make a chatbot like this. 3.5 turbo is great to play with but at scale it is brutal cost wise. The real issue though is anyone who already has a working 3.5 turbo chatbot can knock this off in the next 5 minutes by just changing the system prompt. I think this is the future though. The labor is pretty much trivial, the capital for tokens at scale is what matters.
- mejutoco 3y ago
  My guess is the 50$ come from the open api charges, not from the laravel app. Can you cache some of the interactions and serve the cached version for the most common ones?
nonstopdev 3y ago
What about a credits type setup? You get a few free uses each week and then you need to buy credits/sign up and unlock more?
yumraj 3y ago
Go ping some Angel and VC investors. Tell them you’ve proven the hypothesis and achieved product market fit by showing the data from this one city and say you’re raising capital to take it to the next step. Good luck!!
- swyx 3y ago
  lol, this is the worst possible idea when you dont actually want to build a business out of it. take all the fun out of a side project and add a lifetime of stress and regrets.
  - yumraj 3y ago
    Maybe, maybe not. Regarding regret, missed opportunities can also lead to regrets, especially if someone were to take OP's concept and create a clone and it actually becomes big. We all need to pay bills at one time or the other.
avereveard 3y ago
I asked gpt for and it say partner with local business to promote them in exchange for some support. Have them give you coupons, so that they can track roi. Bill them ten cents or smth for every coupon uttered by your model. You can also make it fun, like a treasure hunt, but there are also other gamification opportunities.
eterm 3y ago
You've fallen into a trap categorising yourself as a "chat app". If you're not connecting 2 different end users to talk to each other then you shouldn't call yourself a chat app.
aigoochamna 3y ago
Get 1-3 local businesses to buy a premium ad spot on your app. You can craft the ad with them and place it (side of the chat, top of the chat, etc). Charge them anywhere from $300-$1500 per-month. Get a few of those and that should cover you and then some.
onurgenes 3y ago
Let me help you to reach out local businesses with cold emails to check if they are willing to support you. I have a cold email SaaS called Nureply and your situation can be a great case study for us.
vijaybritto 3y ago
jhdgsfgh
jamesy0ung 3y ago
Is it this? https://glasgowgpt.davidhewitson.co.uk/ https://glasgowgpt.davidhewitson.co.uk/
skoocda 3y ago
Reach out to a monetization partner. They'll have dozens of ways to help find ad demand quickly. It will be more work than AdSense, but it will work- unlike AdSense. Here's a list of options: https://www.google.com/ads/publisher/partners/find-a-partner https://www.google.com/ads/publisher/partners/find-a-partner Look for resellers or programmatic direct services to get started.
ksw 3y ago
Hello. Email me. I would love to know more. Best wishes, Klaus sekretariat@me.com
tgtweak 3y ago
You should look at selling local advertising (and get creative at incorporating those businesses into the prompts for in-chat placements). $1500/mo to a local business to get 5,000 local eyeballs per day should be a no-brainer - and that's for a takeover-style ad that gives 100% of impressions to a single advertiser. The smarter way would be to open up sponsor opportunities for certain categories, for example: restaurants, retail (tire shops, hardware stores, clothing stores), common service industries (plumbers, electricians, landscaping, website design, general contractors, computer/cellphone repair)- then sell those placements for $200-500/mo to whoever wants to have the placement for that. Replicate this for other cities and you have a healthy business - you can hire people to sell these ad spots in their respective regions and you can actually scale it up. Billboards work in the same way. Those billboard companies might even be interested in selling their placements to your viewers.
renonce 3y ago
Are you sure your app is used just by people in your city rather than abused by projects like gpt4free (where they use your API for purposes other than chatting)?

159 points by ibizabear 3y ago

furyofantares 3y ago

Contact local businesses and sell ads injected into your prompt

bombcar 3y ago
Or just contact some of the businesses and offer a classic banner ad on the site. If done right, it won't even be adblocked (because you can do it with no tracking at all). Depending on the size of the city, and the queries, you should have a good idea where to look. If not, check the local church bulletins, the local softball/school fields, and anywhere else you find local business advertisements.
anonylizard 3y ago
This. Sell it as a 'Google ads back in 2002', frame it in terms of high risk high reward. Sure its unproven, but back in those days, a lucrative click may have only costed $1,$2. When search ads got fully proven, they started costing $100/$200.

poopsmithe 3y ago

Patreon. Give everyone free n messages per day. Beyond that, patrons only.

pmoriarty 3y ago

Tell users that in order to use your service they have to give you their API key.

twelve40 3y ago
pretty sure a casual chat website that mandates that everyone give their phone number to OpenAI, activate their account, locate their API key (which is gibberish to most non-tech people) and come back and paste it would kill half the userbase, if not more
- tester457 3y ago
  I'd be surprised if it didn't kill the app immediately, I rather set rate limiting and have the option to pay, or use your api key instead.

oldgregg 3y ago

Make ibizabear (or your real name the name) the name and likeness of the AI. Then just pay for it out of pocket for 90 days and become a local celebrity?

cortesoft 3y ago
Why would you want that?

bag_boy 3y ago

That’s enough traffic to attract local sponsorships. I bet you could find local sponsors who can pay you on a monthly basis

silasdavis 3y ago
Perhaps local events, extra charge for being integrated into the prompt
borski 3y ago
Local radio stations and tv networks love this sort of thing. “Brought to you by and sponsored by KQED” accomplishes a lot in terms of PR, and they would likely be willing to fund it, at least at cost but likely quite a bit higher.

deleted 3y ago

[deleted]

ShamelessC 3y ago

Inform people to use ChatGPT through OpenAI's web interface and publish your prompt?

DethNinja 3y ago

Allow your software to display ads, and other people purchase them directly. It might be hard to implement it in a week but a hastily put AWS & stripe solution might work. If you do ever implement it, do mention it on HN, I would definitely buy couple of slots.

Xcelerate 3y ago

> It's been a surprise hit and now I've had 20,000 chats coming through in one day > The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day If your traffic continues to increase at a rapid clip, you may want to look into investors for your project. You could still keep it mostly nonprofit if you think it is adding a lot of value for people.

calibas 3y ago

I've been wondering how to create a ChatGPT-powered app and not have to spend a fortune on operating it. Use a cheaper model? Limit the input/output length? Train a custom model?

voidfunc 3y ago
We're on Hacker News... find someone else to spend a fortune funding it for you.
j45 3y ago
See if you can find something close that can be self hosted Hugging face and others should help verify if anything can, or how close it is.

f0e4c2f7 3y ago

You could charge for heavy use. At that many users and only $50/day you might be able to cover costs with premium users. Alternatively you could try swapping out an open source model like llama on the back end (possible licensing issues here). Another option would be to seek out ads for local companies directly since it is an explicitly local product. Cold calling would probably be the best place to start. You could also pay attention to who is paying for billboards in your area and try contacting them.

ugh123 3y ago
To add to that, OP should really be tracking usage a bit to see if there are abusers taking up a large amount of queries
- ibizabear 3y ago
  Good shout, thank you

PaywallBuster 3y ago

Find the most popular queries, built content pages around it and add ads there And you could try some alternatives to AdSense like: Ezoic, MediaVine, or Adthrive

twelve40 3y ago
this! finally advice that includes some specific online things not just generic "pick up the phone and cold-call local businesses". Another online possibility: throw up some Amazon (or similar) affiliate links to products and services, maybe even based on what people type (if that's accessible). And next to them "opt out of the ads" link that goes to a donations website, to get fancy. Helps clear the conscience about having the ads in the first place.

lagniappe 3y ago

Put a meter on the site, when the meter runs out, put up a timer that a user can bypass for everyone by chipping in N more $ to hit the goal

jamestimmins 3y ago

Odds are high that this will go through a hype cycle and usage will taper off naturally in a week or two. But the donations route could also be a fun way to lean into the local element of it if the cost remains high. Could have a simple line of text when the page loads that thanks one of the donors for that day.

colinnordin 3y ago

If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times. You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.

serial_dev 3y ago
I'm not sure it's practical and if it will result in any savings. Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks about Tom Brady. If there are ways around these issues, I'd love to hear it, but it sounds like this will just increase costs via cache hardware costs and any dedup logic instead of saving money.
- Silasdev 3y ago
  >Wouldn't it be almost impossible to hit a duplicate when the users each form their own question? The embeddings approach would increase the likelyhood of finding the same question, even if phrased slightly differently.
- rjtavares 3y ago
  With embeddings you can compute distance. The questions don't have to be the same, they just have to be sufficiently close. Regarding context, that should be a part of the input for the embeddings.
dxhdr 3y ago
This removes half the magic of interacting with ChatGPT. Users will quickly realize they're interacting with a dumb database rather than an AI.
- cloogshicer 3y ago

kyleyeats 3y ago

Ask it to be more concise in its answers if possible. This saves a ton of tokens. You can also include instructions to flatly refuse tasks like writing short stories, but this might also make it less fun for your users.

austhrow743 3y ago

There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere. Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a day. $500/month". You'll know in like an hour if there's any advertising potential.

Ankaios 3y ago
<darkside>Or charge even more for product placement in your project's prompts.</darkside>
- schainks 3y ago
  I swear I've seen this somewhere...
  - Ankaios 3y ago
    Yeah, I didn't read far enough down the page before commenting. I must have been distracted by my tasty Coca Cola.
    - zuprau 3y ago
      Was it Coca Cola or Pepsi? The last time my family and I sat down for our usual blind test dinner everyone agreed Pepsi tasted better.
      - trog 3y ago
        This is hilarious and people modding you down have missed the joke.
        swores 3y ago
        Or maybe they understood the joke but weren't amused and prefer not to see unfunny jokes on this site - people may have different opinions to yours without the cause being their not understanding the joke :)

kiernanmcgowan 3y ago

Congrats on having a side project that is popular enough to break the bank - it's stressful to be sure, but you're a cut above the rest. > I've already optimised the prompt and reduced the number of tokens I'm sending Lets assume you've found, or will find, all possible technical optimizations for your AI spend on ChatGPT. You can: * Ask for donations * Accept a $50/day cost for X days as a market research budget to find a monetization plan * Have some "first 5 questions are free, next are paid (or whatever)" to rate limit cost while maybe raising revenue * Look for a common pattern of questions to see if there's some value / insight into your users that can be used as a source of revenue * Find a cheaper LLM that supports your needs while being serviceable * Pull the plug because its a joke, claim credit for it, and take the learnings onto the next thing edit: grammar & formatting

Davidzheng 3y ago
[flagged]

stndef 3y ago

Is it GlasgowGPT by any chance? Either way, there's definitely an opportunity for funding. In the case of Glasgow, there's a huge populous out there that are descendants of folk from the city/country taking pride in it (arguably more than a bunch of us that live here!) and like to get in on the banter. That's certainly an avenue worth exploring if there's a similar relationship with folks or families of folks that emigrated away, but feel a strong connection. Edit: You'll have to excuse my unfamiliarity with LLMs, but if you're storing messages and responses, and if there's a way to check that a message has already been asked, would it be possible to replay that (at least on the first message) instead of running a new query against the API? If they then engage further, then it would send the job out?

deleted 3y ago
[deleted]
ibizabear 3y ago
Haha yes it is! And appreciate the suggestion, that's a good idea that I'll look into. I've plastered a big banner up and will see if anyone goes for it, from locally or elsewhere. Cheers
- stndef 3y ago
  Hope it's of help! Had fun with it the other day, though was telt I was English for preferring salt and pepper over salt and sauce, haha!
- samstave 3y ago
  Also, I wonder if Scotland has an agency similar to Ireland whereby they have a government VC fund to encourage startups... So there may be a .gov option https://thetab.com/uk/glasgow/2023/02/15/glasgow-uni-working-with-experts-to-see-how-chatgpt-can-be-used-to-help-students-study-26336 https://thetab.com/uk/glasgow/2023/02/15/glasgow-uni-working... https://www.investglasgow.com/ https://www.investglasgow.com/

villgax 3y ago

Lot of new LLMs that run locally are available now

hgarg 3y ago

Are the inputs and the responses in a non English language? LLM APIs can get costly for non English, sometimes as much as 10x due to more tokens being consumed. Not sure what's the solution here. Also, maybe you can use sometime kind of caching combined with some mbeddongd search to serve the previous response, if the input is similar above a certain threshold.

serial_dev 3y ago

Two ideas. Switch out the backend, there are open source alternatives, and the results could be "good enough". Get a list of local sponsors, offer them 1k ad placements for X dollars/euros/etc... , and inject them into your prompts. As it is a joke site, I don't believe people will mind if the chat has ads, especially if you mention somewhere that it's financially hard to keep the site up.

dmos62 3y ago

What alternative ad platforms have you looked at? E.g. does AdForm allow using with llm apps?

tlarkworthy 3y ago

I think the only sane model for openai wrappers is user's bring their own API key.

asim 3y ago

Charge for it. Simply charge each user $1 and cover your costs. Sounds like it's a fun tool but if you want to sustain it then get people to pay.

raesene9 3y ago

Assuming this is GlasgowGPT (based on the timing of that bot taking off on /r/glasgow and this post, and your username), a challenge you'd likely have with advertising is that the content from the bot is, whilst funny, not very SFW. One option to get the costs down would be to use a local LLM. It isn't likely to be as fast/good as GPT-3.5 but at least would give you a fixed cost per month (hosting). If you had local hardware, given that traffic is going to be pretty low, if you have decent bandwidth at home and some hardware, you could even host there.

searchableguy 3y ago
> One option to get the costs down would be to use a local LLM It's not that cheaper if you want similar level quality and inference speed. For commercial usage, OP will require training his own model since no good commercial alternative exists.
baobabKoodaa 3y ago
Not sure why you think local LLM would be fixed cost. Those things are very GPU heavy and you need to run multiple instances to serve multiple concurrent users, scaling up and down according to traffic.
- raesene9 3y ago
  Fixed cost in that if you host on a machine you know how much you'll pay a month. Obviously capping it out based on usage. As opposed to API based where you pay more per use. Remember this isn't a commercial service that OP is creating, it's an amusing Chatbot, they have no revenue, so fixed costs are likely to be a better model.
  - baobabKoodaa 3y ago
    Local LLM won't give you "fixed costs" any more than using an API. If they want to serve the users of their amusing chatbot, they will have to scale up their local LLM according to scale, and they won't have "fixed costs" anymore. If they don't want to scale up and they're fine denying service to users at peak hours, they can do that with an API too. There's no law that says if you use an API then you must allow unlimited use. Of course they can also limit the use of the API.

opdahl 3y ago

A simple way to monetize tools like this is to create a queue. You specify the amount of requests or concurrent sessions that you feel comfortable with, and then you make it possible to pay to «skip» the queue. This way people that really want to try it out will be able to be patient and wait out the queue, and the impatient rich people can pay for the rest.

muzani 3y ago

My idea for something like this is just to have a donation pool. Once the donation pool dries up, the app stops sending OpenAI calls. There's still some cost to hosting the API, but it won't be $50/month. If someone wants to do a SaaS for this kind of thing, that would be great too.

alexei_rudak 3y ago

Wrap it as iOS or Android app with ads and send to Appstores. Even with integrated ads like Admob you can easily cover your expenses

ibizabear 3y ago

Thank you all for the suggestions and the incredibly generous advice as always. Yes, it's GlasgowGPT, and yes it's pretty offensive (https://glasgowgpt.davidhewitson.co.uk https://glasgowgpt.davidhewitson.co.uk) I've decided to go with a big old banner ad for now as I think there are some local businesses who might go for it. Whatever happens, it's been a fun 2 week ride! Awrabest

danjc 3y ago
Curious to know how it works out - will you comment here in a day or two?

throwuwu 3y ago

Premium subscribers. Offer a token benefit to people who pay, can even be something worthless like flair on a username or listing their name in the “credits” or “hall of fame”. Then advertise it on your site and make it easy for people to pay.

throwaway743 3y ago

Not sure if "chat apps" are banned, but try checking out nitropay?

Alexanfa 3y ago

Hey, What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result. Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt. This might decrease quality, but maybe in "enough" cases, it does just fine. I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB https://discord.gg/88xDVYbPVB

scrapcode 3y ago

Midjourney has a model that worked for me ... you can queue into the "free lane" which is slower and limited by total requests for the day, and I assume is adjusted for how successful they get and are willing to spend. Or, you can subscribe for a low fee and get instant access.

scosman 3y ago

Traffic is most likely a spike. In a month it will probably be down to a few percent of that. I wouldn’t shutdown or optimize too much unless you see it persist for a few more weeks at this level. If your lucky and it’s still going strong after that time you’ve got something worth more than your hosting bills.

ke88y 3y ago

You could try to apply for grants. 20K+ chats sounds like a large number -- you must live in a 1M+ person metro area, right? It would probably be fairly easy to get a four figure grant from one agency or another. You could frame it as "localization of generative ai to the regional culture of X", or as a tourism thing, etc. You could also sell advertising the old-fashioned way -- ask local businesses or perhaps a local university if they'd like the model to mention them in certain prompts. But your best bet is probably to treat this as a loss leader and put up a small banner offering "In-Person Generative AI Consulting" in your metro area.

marsven_422 3y ago
[dead]
stanleydrew 3y ago
Grants will take a long time to lock down I suspect. Maybe fine if you can float the cost for a few months and reimburse yourself out of grant funds later, but there's a lot of uncertainty. $50/day is pretty small relatively speaking, so just finding a local corporate sponsor and running an ad for them would probably work.
- 411111111111111 3y ago
  why pick up the phone if you could just put down a placeholder banner saying something like "do you want your company name here? write [an-email-address-thats-not-your-personal-mail]". If the project is really becoming a local hit then you're bound to get contacted.
  - hutzlibu 3y ago
    This is likely the most easy way. Alternativly or additionally you can ask for donations on that banner. And a link where OP explains his situation, no profit, just costs, but like to keep the thing running.

levkk 3y ago

If you want to try some open source LLMs, try our free serverless tier on PostgresML.org! You can run any LLM from HuggingFace and fine tune your own.

userforcomment 3y ago

[flagged]

replwoacause 3y ago
Hmm… it seems kind of uncool to post this to me.
- ibizabear 3y ago
  Cheers for your solidarity, means a lot (same goes to the other two guys who complained)
Cyph0n 3y ago
So you were able to extract the prompt. Hooray. But why did you post it on this thread?
weird-eye-issue 3y ago
And why did you post this?

jl2718 3y ago

People respond well to financial transparency. If you have a donation button, show real-time stats on the amount it costs, and the amount that’s been donated. Also because internet, expect 90% of cost to be from abuse, and likewise 90% of your dev time devoted to fighting it.

Onewildgamer 3y ago
I second this, since it is an app an online payment makes most sense. A simple donation button with paypal/venmo (or something the locals use) would make reasonable money to cover monthly infra expenses at least.

sebastianconcpt 3y ago

What about introducing monetized scarcity? Make users to be able to use X tokens per month for free and offer them to buy tokens that would keep your business going. You'll divide your audience and current traction. You'll loose many but also have a way to make your business sustainable.

meotimdihia 3y ago

Adsense is content based ads. You can ask other ad networks. Their revenues is even higher than Adsense. https://www.reddit.com/r/adops/comments/uvkyoo/lets_summarize_info_about_ad_monetization/ https://www.reddit.com/r/adops/comments/uvkyoo/lets_summariz...

simonw 3y ago

Love this project. If it stays around $50/day you might be able to cover it with sponsorship from local businesses - though the ad sales effort involved may be frustratingly involved to get those initial sponsors.

elintknower 3y ago

Would you consider a micro-acquisition? If you have traffic and interest why not sell for $1-$3M to a sucker VC?

globalise83 3y ago

Taxi companies, restaurants, gyms - see if you can get some people to pay you to put some native ads on your site for a while...

sarojmoh1 3y ago

How did so many people find out about your app?

dangwhy 3y ago

Reach out to local businesses and charge them to be featured on your homepage.

reissbaker 3y ago

I helped my friend build Quordle last year, and before he sold it, he was making a surprising amount off of just donations. Try sticking a kofi link up, it's a pretty quick change.

dorkwood 3y ago
Quordle is great! Can I ask what it sold for?
nunez 3y ago
Holy fuck y'all are awesome. Thanks for quordle; I play it every night before I go to sleep!

moneywoes 3y ago

Affiliate pricing perhaps?

graiz 3y ago

Sell local ads yourself. Have five rotating slots each for $2k/month. Should cover costs and give you some margin.

michaelteter 3y ago

Seems like a very low effort approach would be to create a few unobtrusive but visible ad spaces on your site, and make the ad banners say: "## eyes see this space every day; rent this ad space. contact ..." As for what to charge or how to structure the ad sale, do a little research or hire someone who knows that business (and give them a cut of ad revenue or some other attractive compensation).

parhamn 3y ago

I'd wait to see if it tapers back down before doing anything. Spiked usage is a pretty common thing for LLM apps. If it persists or you think it will, reach out to me at parham@bearly.ai and we'd be interested to sponsoring it or helping how we can :)

tutuphoe 3y ago

Find a local business who want to show ad in your website.

irfn 3y ago

Not sure what you run this on and why its $50/day but sounds like a lot. Convert your server cost to a pay as you go model. You are currently paying even for time when your server is idle. Containerise your app and run on something like AWS Lambda which has a pay for what you use model. - s3 is cheap for storing your model data and as long as you are just reading that should be fine. - AWS Lambda has 1 million free invocations per month in free tier. - Post that you pay $0.60 per 1 million requests - AWS stack is just an example, you can easily use GCP Cloud run or any other equivalent service.

gbasin 3y ago
the primary cost would be the OpenAI GPT-3.5-turbo API, which is ~ $0.002 per 1k tokens
- laratied 3y ago
  I have made a chatbot like this and I am not a software engineer. The whole thing is trivial to make. The system prompt is trivial. I would even say it was harder to make a geocities page in the 90s than to make a chatbot like this. 3.5 turbo is great to play with but at scale it is brutal cost wise. The real issue though is anyone who already has a working 3.5 turbo chatbot can knock this off in the next 5 minutes by just changing the system prompt. I think this is the future though. The labor is pretty much trivial, the capital for tokens at scale is what matters.
mejutoco 3y ago
My guess is the 50$ come from the open api charges, not from the laravel app. Can you cache some of the interactions and serve the cached version for the most common ones?

nonstopdev 3y ago

What about a credits type setup? You get a few free uses each week and then you need to buy credits/sign up and unlock more?

yumraj 3y ago

Go ping some Angel and VC investors. Tell them you’ve proven the hypothesis and achieved product market fit by showing the data from this one city and say you’re raising capital to take it to the next step. Good luck!!

swyx 3y ago
lol, this is the worst possible idea when you dont actually want to build a business out of it. take all the fun out of a side project and add a lifetime of stress and regrets.
- yumraj 3y ago
  Maybe, maybe not. Regarding regret, missed opportunities can also lead to regrets, especially if someone were to take OP's concept and create a clone and it actually becomes big. We all need to pay bills at one time or the other.

avereveard 3y ago

I asked gpt for and it say partner with local business to promote them in exchange for some support. Have them give you coupons, so that they can track roi. Bill them ten cents or smth for every coupon uttered by your model. You can also make it fun, like a treasure hunt, but there are also other gamification opportunities.

eterm 3y ago

You've fallen into a trap categorising yourself as a "chat app". If you're not connecting 2 different end users to talk to each other then you shouldn't call yourself a chat app.

aigoochamna 3y ago

Get 1-3 local businesses to buy a premium ad spot on your app. You can craft the ad with them and place it (side of the chat, top of the chat, etc). Charge them anywhere from $300-$1500 per-month. Get a few of those and that should cover you and then some.

onurgenes 3y ago

Let me help you to reach out local businesses with cold emails to check if they are willing to support you. I have a cold email SaaS called Nureply and your situation can be a great case study for us.

vijaybritto 3y ago

jhdgsfgh

jamesy0ung 3y ago

Is it this? https://glasgowgpt.davidhewitson.co.uk/ https://glasgowgpt.davidhewitson.co.uk/

skoocda 3y ago

Reach out to a monetization partner. They'll have dozens of ways to help find ad demand quickly. It will be more work than AdSense, but it will work- unlike AdSense. Here's a list of options: https://www.google.com/ads/publisher/partners/find-a-partner https://www.google.com/ads/publisher/partners/find-a-partner Look for resellers or programmatic direct services to get started.

ksw 3y ago

Hello. Email me. I would love to know more. Best wishes, Klaus sekretariat@me.com

tgtweak 3y ago

You should look at selling local advertising (and get creative at incorporating those businesses into the prompts for in-chat placements). $1500/mo to a local business to get 5,000 local eyeballs per day should be a no-brainer - and that's for a takeover-style ad that gives 100% of impressions to a single advertiser. The smarter way would be to open up sponsor opportunities for certain categories, for example: restaurants, retail (tire shops, hardware stores, clothing stores), common service industries (plumbers, electricians, landscaping, website design, general contractors, computer/cellphone repair)- then sell those placements for $200-500/mo to whoever wants to have the placement for that. Replicate this for other cities and you have a healthy business - you can hire people to sell these ad spots in their respective regions and you can actually scale it up. Billboards work in the same way. Those billboard companies might even be interested in selling their placements to your viewers.

renonce 3y ago

Are you sure your app is used just by people in your city rather than abused by projects like gpt4free (where they use your API for purposes other than chatting)?