5 ms·

I’ve been using v4 flash for an app I’m building [1] and it’s amazing how cost effective and good it is coming from having always used gpt, opus and sonnet mode

by jmathai 2mo ago

I’ve been using v4 flash for an app I’m building [1] and it’s amazing how cost effective and good it is coming from having always used gpt, opus and sonnet models. It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it. [1] https://trysojourn.app https://trysojourn.app

weiliddat 2mo ago
I get where you're coming from, and the intent to make it easier for people to find examples and verses, but there's a fine line with LLMs giving you answers, is that it's interpreting it in some form. Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights?
- jmathai 2mo ago
  You’re correct. The hard line for me is ensuring that any scripture presented to the user is verified correct. LLMs can’t be trusted in this regard. One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source. I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
  - weiliddat 2mo ago
    I don't necessarily mean reguritating it, but choosing which part of the scripture to surface to the user is already some interpretation/choice. Even the devil can quote scripture (I'm playing the devil's advocate here).
    - jmathai 2mo ago
      Very true. The verses selected come from a tool call. The LLM queries the app for relevant verses. It picks the keywords - so perhaps there’s bias there but the verses are handed to the LLM. But there are ways to control and constrain the LLMs and what the user is presented with. These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
      - raynhardt 2mo ago
        How polished is the experience, I see there is an invite link on the website.
        jmathai 2mo ago
        It’s pretty polished but you’re asking the wrong person. Feedback has been positive so far :).
- hirako2000 2mo ago
  It doesn't prevent you from going to the source and struggle with the text, nor seek expert commentary. What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc. This app does that very well, LLMs are good at entity recognition.
  - weiliddat 2mo ago
    Yes it can point you to the right place, but choosing which part of scripture to point you to is already a choice/interpretation
- foltik 2mo ago
  Just let them have their AI pacifier.
- swat535 2mo ago
  > Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights? I think it depends, Catholics wouldn't be able to use this because the Magisterium is the ultimate authority on interpreting Scripture, so the personal interpretation isn't really needed. This is not to say that Catholics don't read the Bible, they are encouraged to do so since it deepens their faith On the other hand, for Protestant it varies, the High Church denominations are closer to Catholics (though none of them accept the Magisterium) in terms of scripture interpretation, but the Low Church ones (like Baptists or Non-Denominational ) are more open to personal interpenetration. Disclaimer: I'm a Catholic, so if I made a mistake here fellow Protestants, please correct me.
ComputerPerson 2mo ago
Is the difference between this and a frontier model that the scripture is guaranteed to be real? I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails. I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost. If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet. [1] https://ai.stepbible.org https://ai.stepbible.org
- kfse 2mo ago
  All models hallucinate. Everything in production with a model will too. It is not a "deal breaker"
- vmt-man 2mo ago
  > Is the difference between this and a frontier model that the scripture is guaranteed to be real? flash is suitable only for a toy apps, not for production environments :)
- willchis 2mo ago
  I hope the irony of being concerned about hallucinations in a bible study app is not lost on you or others reading this!
  - swingboy 2mo ago
    Sir, you dropped your fedora.
- jmathai 2mo ago
  Philosophically, I don’t believe we should outsource the accuracy of scripture to any single entity (let alone a for profit secular one). So it’s less about model choice and more about governance of scripture. I will check out the link you sent for sure!
tmaly 2mo ago
have you used it for anything outside of coding?
- jmathai 2mo ago
  I tried using Deepseek with Claude Code and it under performed. The use I was referring to was within a product and I feel it's a lot of value for the cost. https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code/ https://api-docs.deepseek.com/quick_start/agent_integrations...
achalxyz 2mo ago
[dead]

by jmathai 2mo ago

weiliddat 2mo ago

I get where you're coming from, and the intent to make it easier for people to find examples and verses, but there's a fine line with LLMs giving you answers, is that it's interpreting it in some form. Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights?

jmathai 2mo ago
You’re correct. The hard line for me is ensuring that any scripture presented to the user is verified correct. LLMs can’t be trusted in this regard. One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source. I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
- weiliddat 2mo ago
  I don't necessarily mean reguritating it, but choosing which part of the scripture to surface to the user is already some interpretation/choice. Even the devil can quote scripture (I'm playing the devil's advocate here).
  - jmathai 2mo ago
    Very true. The verses selected come from a tool call. The LLM queries the app for relevant verses. It picks the keywords - so perhaps there’s bias there but the verses are handed to the LLM. But there are ways to control and constrain the LLMs and what the user is presented with. These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
    - raynhardt 2mo ago
      How polished is the experience, I see there is an invite link on the website.
      - jmathai 2mo ago
        It’s pretty polished but you’re asking the wrong person. Feedback has been positive so far :).
hirako2000 2mo ago
It doesn't prevent you from going to the source and struggle with the text, nor seek expert commentary. What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc. This app does that very well, LLMs are good at entity recognition.
- weiliddat 2mo ago
  Yes it can point you to the right place, but choosing which part of scripture to point you to is already a choice/interpretation
foltik 2mo ago
Just let them have their AI pacifier.
swat535 2mo ago
> Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights? I think it depends, Catholics wouldn't be able to use this because the Magisterium is the ultimate authority on interpreting Scripture, so the personal interpretation isn't really needed. This is not to say that Catholics don't read the Bible, they are encouraged to do so since it deepens their faith On the other hand, for Protestant it varies, the High Church denominations are closer to Catholics (though none of them accept the Magisterium) in terms of scripture interpretation, but the Low Church ones (like Baptists or Non-Denominational ) are more open to personal interpenetration. Disclaimer: I'm a Catholic, so if I made a mistake here fellow Protestants, please correct me.

ComputerPerson 2mo ago

Is the difference between this and a frontier model that the scripture is guaranteed to be real? I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails. I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost. If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet. [1] https://ai.stepbible.org https://ai.stepbible.org

kfse 2mo ago
All models hallucinate. Everything in production with a model will too. It is not a "deal breaker"
vmt-man 2mo ago
> Is the difference between this and a frontier model that the scripture is guaranteed to be real? flash is suitable only for a toy apps, not for production environments :)
willchis 2mo ago
I hope the irony of being concerned about hallucinations in a bible study app is not lost on you or others reading this!
- swingboy 2mo ago
  Sir, you dropped your fedora.
jmathai 2mo ago
Philosophically, I don’t believe we should outsource the accuracy of scripture to any single entity (let alone a for profit secular one). So it’s less about model choice and more about governance of scripture. I will check out the link you sent for sure!

tmaly 2mo ago

have you used it for anything outside of coding?

jmathai 2mo ago
I tried using Deepseek with Claude Code and it under performed. The use I was referring to was within a product and I feel it's a lot of value for the cost. https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code/ https://api-docs.deepseek.com/quick_start/agent_integrations...

achalxyz 2mo ago

[dead]