5 ms·

Hey! I definitely understand the reservation. This is definitely me as well. My reasons for using the UI at this point: * GPT-4 is decently faster when talking

Hey! I definitely understand the reservation. This is definitely me as well. My reasons for using the UI at this point: * GPT-4 is decently faster when talking straight to the API * The API is so stupidly cheap that it's basically a rounding error for me. Half an hour of chatting to GPT3.5 costs me $0.02 Would be curious what you mean by integrating the backend-api?

joenot443 3y ago
Wow! Is it really that cheap? GPT4 is much more expensive, I imagine?
- kami8845 3y ago
  GPT-4 is decently more expensive -- I personally really like & use the therapist character a lot. In this scenario the session would cost me less than $1 which is still much cheaper than any therapist I've used previously :)
  - coolspot 3y ago
    What is your setup?
robopsychology 3y ago
How is it that cheap?! I ran three queries on langchain yesterday with two ConstitionalPrompts and it cost $0.22 - made me realize deploying my project for cheap could be expensive quick.
- kami8845 3y ago
  GPT3.5 Turbo pricing is 10k tokens or ~7500 words for $0.02. Though note that every API request includes the entire chat context and charges for input & output tokens. https://openai.com/pricing https://openai.com/pricing
- monkmartinez 3y ago
  You need to check which model you are using, also... LangChain runs through the model several times with increased token count on each successive call.
  - robopsychology 3y ago
    Yeah I assumed it would be doing several times but still more expensive than OP mentioned. I think the issue is I'm using davinci-003
    - drusepth 3y ago
      Yeah, davinci-003 is gonna be gpt3, which is more expensive than 3.5. One more anecdote: I've been running a half dozen gpt3.5 IRC bots for a few weeks and their total cost was less than a dollar. A few hours of playing around with LangChain on gpt3 cost me almost $4 before I realized I needed to switch to 3.5, though even then it still uses a ton of tokens every chain.
      - robopsychology 3y ago
        Thanks, I'll do that later
agotterer 3y ago
I’d love to see a comparison of the average cost to use this with the OpenAI API versus subscribing to chat-gpt plus. Maybe I’ll have to try this for a month and see if it end up costing more than $20. Thanks for creating it!
qwertox 3y ago
GPT-3.5 is really cheap (prompt and completion = $0.002 / 1K tokens), but GPT-4 is around 20 times more expensive (prompt = $0.03 / 1K tokens + completion = $0.06 / 1K tokens). But the benefit from using the API is that you can change the model on the fly, so you chat with 3.5 until you notice that it's not responding properly and, with all the history you have (probably stored in your database), you can send a bigger request with a probably better response once with GPT-4 as the selected model. I really wish the interface on chat.openai.org would allow me to switch between models in the same conversation in order to 1) not use up your quota of GPT-4 interactions per 3 hours as quickly and 2) not strain the backend unnecessarily when you know that starting a conversation with GPT-3.5 is efficient enough until you notice that you better switch models. OpenAI already has this implemented: When you use up your quota of GPT-4 chats, it offers you to drop down into GPT-3.5 in that same conversation.
- sebzim4500 3y ago
  Sure, but GPT-4 through the UI costs $20 per month, which is a lot of api calls.
  - moneywoes 3y ago
    Isn’t it 10 per hour?
    - arthurcolle 3y ago
      25 / 3 hrs

joenot443 3y ago

Wow! Is it really that cheap? GPT4 is much more expensive, I imagine?

kami8845 3y ago
GPT-4 is decently more expensive -- I personally really like & use the therapist character a lot. In this scenario the session would cost me less than $1 which is still much cheaper than any therapist I've used previously :)
- coolspot 3y ago
  What is your setup?

robopsychology 3y ago

How is it that cheap?! I ran three queries on langchain yesterday with two ConstitionalPrompts and it cost $0.22 - made me realize deploying my project for cheap could be expensive quick.

kami8845 3y ago
GPT3.5 Turbo pricing is 10k tokens or ~7500 words for $0.02. Though note that every API request includes the entire chat context and charges for input & output tokens. https://openai.com/pricing https://openai.com/pricing
monkmartinez 3y ago
You need to check which model you are using, also... LangChain runs through the model several times with increased token count on each successive call.
- robopsychology 3y ago
  Yeah I assumed it would be doing several times but still more expensive than OP mentioned. I think the issue is I'm using davinci-003
  - drusepth 3y ago
    Yeah, davinci-003 is gonna be gpt3, which is more expensive than 3.5. One more anecdote: I've been running a half dozen gpt3.5 IRC bots for a few weeks and their total cost was less than a dollar. A few hours of playing around with LangChain on gpt3 cost me almost $4 before I realized I needed to switch to 3.5, though even then it still uses a ton of tokens every chain.
    - robopsychology 3y ago
      Thanks, I'll do that later

agotterer 3y ago

I’d love to see a comparison of the average cost to use this with the OpenAI API versus subscribing to chat-gpt plus. Maybe I’ll have to try this for a month and see if it end up costing more than $20. Thanks for creating it!

qwertox 3y ago

GPT-3.5 is really cheap (prompt and completion = $0.002 / 1K tokens), but GPT-4 is around 20 times more expensive (prompt = $0.03 / 1K tokens + completion = $0.06 / 1K tokens). But the benefit from using the API is that you can change the model on the fly, so you chat with 3.5 until you notice that it's not responding properly and, with all the history you have (probably stored in your database), you can send a bigger request with a probably better response once with GPT-4 as the selected model. I really wish the interface on chat.openai.org would allow me to switch between models in the same conversation in order to 1) not use up your quota of GPT-4 interactions per 3 hours as quickly and 2) not strain the backend unnecessarily when you know that starting a conversation with GPT-3.5 is efficient enough until you notice that you better switch models. OpenAI already has this implemented: When you use up your quota of GPT-4 chats, it offers you to drop down into GPT-3.5 in that same conversation.

sebzim4500 3y ago
Sure, but GPT-4 through the UI costs $20 per month, which is a lot of api calls.
- moneywoes 3y ago
  Isn’t it 10 per hour?
  - arthurcolle 3y ago
    25 / 3 hrs