5 ms·

>I'm also interested in their applications for journalism, specifically for dealing with extremely sensitive data like leaked information from confidential sour

>I'm also interested in their applications for journalism, specifically for dealing with extremely sensitive data like leaked information from confidential sources. Think it is NOT just you. Most company with decent management also would not want their data going to anything outside the physical server they have in control of. But yeah for most people just use an app and hosted server. But this is HN,there are ppl here hosting their own email servers, so shouldn't be too hard to run llm locally.

simonw 1y ago
"Most company with decent management also would not want their data going to anything outside the physical server they have in control of." I don't think that's been true for over a decade: AWS wouldn't be trillion dollar business if most companies still wanted to stay on-premise.
- terhechte 1y ago
  Or GitHub. I’m always amused when people don’t want to send fractions of their code to a LLM but happily host it on GitHub. All big llm providers offer no-training-on-your-data business plans.
  - tarruda 1y ago
    > I’m always amused when people don’t want to send fractions of their code to a LLM but happily host it on GitHub What amuses me even more is people thinking their code is too unique and precious, and that GitHub/Microsoft wants to steal it.
    - AlexCoventry 1y ago
      Concern about platform risk in regard to Microsoft is historically justified.
    - Terretta 1y ago
      Unlikely they think Microsoft or GitHub wants to steal it. With LLMs, they're thinking of examples that regurgitated proprietary code, and contrary to everyday general observation, valuable proprietary code does exist. But with GitHub, the thinking is generally the opposite: the worry is that the code is terrible, and seeing it would be like giant blinkenlights* indicating the way in. * https://en.wikipedia.org/wiki/Blinkenlights https://en.wikipedia.org/wiki/Blinkenlights
    - vikarti 1y ago
      Regulations sometimes matter. Stupid "security" rules sometimes matter too.
  - iamnotagenius 1y ago
    [dead]
- __float 1y ago
  While none of that is false, I think there's a big difference from shipping your data to an external LLM API and using AWS. Using AWS is basically a "physical server they have control of".
  - simonw 1y ago
    That's why AWS Bedrock and Google Vertex AI and Azure AI model inference exist - they're all hosted LLM services that offer the same compliance guarantees that you get from regular AWS-style hosting agreements.
  - IanCal 1y ago
    As in aws is a much bigger security concern?
- ipdashc 1y ago
  Yeah, this has been confusing me a bit. I'm not complaining by ANY means, but why does it suddenly feel like everyone cares about data privacy in LLM contexts, way more than previous attitudes to allowing data to sit on a bunch of random SaaS products? I assume because of the assumption that the AI companies will train off of your data, causing it to leak? But I thought all these services had enterprise tiers where they'll promise not to do that? Again, I'm not complaining, it's good to see people caring about where their data goes. Just interesting that they care now, but not before. (In some ways LLMs should be one of the safer services, since they don't even really need to store any data, they can delete it after the query or conversation is over.)
  - pornel 1y ago
    It is due to the risk of a leak. Laundering of data through training makes it a more complicated case than a simple data theft or copyright infringement. Leaks could be accidental, e.g. due to an employee logging in to their free-as-in-labor personal account instead of a no-training Enterprise account. It's safer to have a complete ban on providers that may collect data for training.
    - 6510 1y ago
      Their entire business model based on taking other peoples stuff. I cant imagine someone would willingly drown with the sinking ship if the entire cargo is filled with lifeboats - just because they promised they would.
    - vbezhenar 1y ago
      How can you be sure that AWS will not use your data to train their models? They got enormous data, probably most data in the world.
      - simonw 1y ago
        Being caught doing they would be wildly harmful to their business - billions of dollars harmful, especially given the contracts they sign with their customers. The brand damage would be unimaginably expensive too. There is no world in which training on customer data without permission would be worth it for AWS. Your data really isn't that useful anyway.
- mjlee 1y ago
  AWS has a strong track record, a clear business model that isn’t predicated on gathering as much data as possible, and an awful lot to lose if they break their promises. Lots of AI companies have some of these, but not to the same extent.
- Tepix 1y ago
  on-premises https://twominenglish.com/premise-vs-premises/ https://twominenglish.com/premise-vs-premises/
belter 1y ago
> "Most company with decent management also would not want their data going to anything outside the physical server they have in control of." Most companies physical and digital security controls are so much worst than anything from AWS or Google. Note I dont include Azure...but a physical server they have control of is a phrase that screams vulnerability.

simonw 1y ago

"Most company with decent management also would not want their data going to anything outside the physical server they have in control of." I don't think that's been true for over a decade: AWS wouldn't be trillion dollar business if most companies still wanted to stay on-premise.

terhechte 1y ago
Or GitHub. I’m always amused when people don’t want to send fractions of their code to a LLM but happily host it on GitHub. All big llm providers offer no-training-on-your-data business plans.
- tarruda 1y ago
  > I’m always amused when people don’t want to send fractions of their code to a LLM but happily host it on GitHub What amuses me even more is people thinking their code is too unique and precious, and that GitHub/Microsoft wants to steal it.
  - AlexCoventry 1y ago
    Concern about platform risk in regard to Microsoft is historically justified.
  - Terretta 1y ago
    Unlikely they think Microsoft or GitHub wants to steal it. With LLMs, they're thinking of examples that regurgitated proprietary code, and contrary to everyday general observation, valuable proprietary code does exist. But with GitHub, the thinking is generally the opposite: the worry is that the code is terrible, and seeing it would be like giant blinkenlights* indicating the way in. * https://en.wikipedia.org/wiki/Blinkenlights https://en.wikipedia.org/wiki/Blinkenlights
  - vikarti 1y ago
    Regulations sometimes matter. Stupid "security" rules sometimes matter too.
- iamnotagenius 1y ago
  [dead]
__float 1y ago
While none of that is false, I think there's a big difference from shipping your data to an external LLM API and using AWS. Using AWS is basically a "physical server they have control of".
- simonw 1y ago
  That's why AWS Bedrock and Google Vertex AI and Azure AI model inference exist - they're all hosted LLM services that offer the same compliance guarantees that you get from regular AWS-style hosting agreements.
- IanCal 1y ago
  As in aws is a much bigger security concern?
ipdashc 1y ago
Yeah, this has been confusing me a bit. I'm not complaining by ANY means, but why does it suddenly feel like everyone cares about data privacy in LLM contexts, way more than previous attitudes to allowing data to sit on a bunch of random SaaS products? I assume because of the assumption that the AI companies will train off of your data, causing it to leak? But I thought all these services had enterprise tiers where they'll promise not to do that? Again, I'm not complaining, it's good to see people caring about where their data goes. Just interesting that they care now, but not before. (In some ways LLMs should be one of the safer services, since they don't even really need to store any data, they can delete it after the query or conversation is over.)
- pornel 1y ago
  It is due to the risk of a leak. Laundering of data through training makes it a more complicated case than a simple data theft or copyright infringement. Leaks could be accidental, e.g. due to an employee logging in to their free-as-in-labor personal account instead of a no-training Enterprise account. It's safer to have a complete ban on providers that may collect data for training.
  - 6510 1y ago
    Their entire business model based on taking other peoples stuff. I cant imagine someone would willingly drown with the sinking ship if the entire cargo is filled with lifeboats - just because they promised they would.
  - vbezhenar 1y ago
    How can you be sure that AWS will not use your data to train their models? They got enormous data, probably most data in the world.
    - simonw 1y ago
      Being caught doing they would be wildly harmful to their business - billions of dollars harmful, especially given the contracts they sign with their customers. The brand damage would be unimaginably expensive too. There is no world in which training on customer data without permission would be worth it for AWS. Your data really isn't that useful anyway.
mjlee 1y ago
AWS has a strong track record, a clear business model that isn’t predicated on gathering as much data as possible, and an awful lot to lose if they break their promises. Lots of AI companies have some of these, but not to the same extent.
Tepix 1y ago
on-premises https://twominenglish.com/premise-vs-premises/ https://twominenglish.com/premise-vs-premises/

belter 1y ago

> "Most company with decent management also would not want their data going to anything outside the physical server they have in control of." Most companies physical and digital security controls are so much worst than anything from AWS or Google. Note I dont include Azure...but a physical server they have control of is a phrase that screams vulnerability.