5 ms·

Do you want anyone in the world to be able to synthesize dangerous viruses?

by johnsimer 3mo ago

invalidusernam3 3mo ago
What about allowing people to synthesize dangerous virus protection?
root-parent 3mo ago
We do. Its the only way we will get our jobs back.
orphea 3mo ago
So, security (safety) through obscurity?
- deleted 3mo ago
  [deleted]
- deleted 3mo ago
  [deleted]
- usef- 3mo ago
  The phrase "security through obscurity" isn't an argument against all information restriction. It doesn't imply we should, for example, publish step-by-step instructions for making widespread death easier.
  - qrios 3mo ago
    Another „great filter“: How to handle dagerous information?
- inglor_cz 3mo ago
  The argument against security through obscurity isn't that it doesn't work at all. It does to a degree, only it is not as strong as people think. An example from the meat world: not publishing your vacation dates well in advance for the world to see somewhat reduces your chance of being burglarized. That is security by obscurity; not reliable, but not completely inefficient either. But if you live in a fortress (security by key material), you can well declare your vacation dates without running the risk.
sterlind 3mo ago
you need a lot more than the nucleotide sequence to make a virus. you need the DNA or RNA to be synthesized, assembled, packaged properly. and long sequences are pretty hard to do. you need a lot of equipment, or you need to order from services. the oligo synth services can harden their KYC and/or screen for suspicious sequences. sure, a malevolent state actor could swing it, but they could make a bioweapon without Mythos's help already. also, vaccine production and disease surveillance have ramped up very quickly. they will ramp up further, despite political setbacks. it's a cat and mouse game that favors the defenders IMO. but the bioterrorism narrative is useful FUD to spin open-weight models as existentially dangerous. I am far more worried about Anthropic's own goals than the goals of some crackpot in a shed.
- theLiminator 3mo ago
  > it's a cat and mouse game that favors the defenders IMO How so? I'm actually against most of the "safety-tuning" that anthropic does, but this seems fundamentally untrue, a close analogue being video game cheat development. I think in general the cheat developer has an advantage and the cheats generally proliferate for quite a while before being patched.
  - ceigey 3mo ago
    Video games are an interesting analogy since they often trade security for performance, trusting clients about world state quite a bit. Finance and biology do come across as two similar high level systems. But while we can employ KYC, fraud detection, and various auditing techniques to finance, I don’t know what you do for biology. You can easily run an algorithm over every transaction a person makes in their account but there’s no equivalent for every cell, every bacteria strain, every virus in the human body.
    - sterlind 3mo ago
      (disclaimer: layperson remembering how the immune system works.) the adaptive immune system effectively does KYC by checking the antigens presented on the surfaces of cells. the thymus selects for B-cells (iirc?) which don't react to a corpus of the body's own antigens, but cover a wide library of everything else. when it sees something it doesn't recognize, it reproduces, warns the rest of the immune system and marks targets. that's why our immune systems can eventually conquer almost every pathogen we encounter, if we can survive long enough for it to do its work. but the KYC I was referring to was KYC that vendors of oligonucleotides (should) be doing, to keep people from ordering nefarious sequences.
iAMkenough 3mo ago
It's inevitable. Also, it's not like I get to vet who does or doesn't have access. Blind trust in the current selection made by an unregulated corporation just makes me anxious. Security in the form of "pay to play" is just kicking the bigger issue down the road.
sneak 3mo ago
I want everyone in the world to be able to perform unlimited cutting edge research on any topic at the maximum thinking level, instantly. The reason we are not being attacked is not lack of technology access.
- dyauspitr 3mo ago
  It is an access issue. If you could get step by step instructions on how to modify a virus so it kills all people over 6ft you bet your ass there would be people attempting it.
  - debesyla 3mo ago
    I guess in this theoretical "AI makes weapon" scenario one could use the same AI to make defences too? // Claude, make antiviral nanobots that defend me from 6ft virus. Make no mistakes.
    - dyauspitr 3mo ago
      I don’t know if you’re being silly but it is orders of magnitudes easier to modify an existing virus to selectively target certain snps than make “antiviral nanobots”
      - jex_the_ape 3mo ago
        Claude, modify the existing 6ft killer virus so that it only makes my balls itch slightly for a day and gives me lifetime immunity to all further stamms of the 6ft killer virus. Make no mistakes, double check so the virus causes no unforseen complications.
  - JumpCrisscross 3mo ago
    > It is an access issue Column A, Column B. Building a small explosive device isn't hard. Building a million is very difficult, doing it covertly virtually impossible without the resources of a nation-state. The problem with biologics is the self-assembly and replication machinery comes for "free." So the numpties who might otherwise blow up a trash can [1] now have a real chance of taking out a million people. [1] https://en.wikipedia.org/wiki/2016_New_York_and_New_Jersey_bombings https://en.wikipedia.org/wiki/2016_New_York_and_New_Jersey_b...
digitaltrees 3mo ago
It the tool was made available to anyone to build a virus, anyone would be able to build counter measures, if only a select few people have access they get to build the virus and everyone else is at a disadvantage. So, yes, I am leaning towards making these tools open rather than gated behind some priesthood and government that gets to wield exclusive power.
- usef- 3mo ago
  Compare the cost/ease of attacker vs defender if one person is given a virus to unleash anywhere in the world and another person is given a vaccine to distribute to the whole world. Or compare building a large bridge to someone disabling that bridge, etc. Prevention and repair is almost always more expensive than vandalism. I don't think there's an ideal solution here, but giving trusted people access to fix security issues before giving it to the wider public seems like a reasonable compromise. They're letting you use the model for all other uses.
jesterson 3mo ago
Do you believe people currently possessing best models act/will act in your best interest?

Do you want anyone in the world to be able to synthesize dangerous viruses?

invalidusernam3 3mo ago

What about allowing people to synthesize dangerous virus protection?

root-parent 3mo ago

We do. Its the only way we will get our jobs back.

orphea 3mo ago

So, security (safety) through obscurity?

deleted 3mo ago
[deleted]
deleted 3mo ago
[deleted]
usef- 3mo ago
The phrase "security through obscurity" isn't an argument against all information restriction. It doesn't imply we should, for example, publish step-by-step instructions for making widespread death easier.
- qrios 3mo ago
  Another „great filter“: How to handle dagerous information?
inglor_cz 3mo ago
The argument against security through obscurity isn't that it doesn't work at all. It does to a degree, only it is not as strong as people think. An example from the meat world: not publishing your vacation dates well in advance for the world to see somewhat reduces your chance of being burglarized. That is security by obscurity; not reliable, but not completely inefficient either. But if you live in a fortress (security by key material), you can well declare your vacation dates without running the risk.

sterlind 3mo ago

you need a lot more than the nucleotide sequence to make a virus. you need the DNA or RNA to be synthesized, assembled, packaged properly. and long sequences are pretty hard to do. you need a lot of equipment, or you need to order from services. the oligo synth services can harden their KYC and/or screen for suspicious sequences. sure, a malevolent state actor could swing it, but they could make a bioweapon without Mythos's help already. also, vaccine production and disease surveillance have ramped up very quickly. they will ramp up further, despite political setbacks. it's a cat and mouse game that favors the defenders IMO. but the bioterrorism narrative is useful FUD to spin open-weight models as existentially dangerous. I am far more worried about Anthropic's own goals than the goals of some crackpot in a shed.

theLiminator 3mo ago
> it's a cat and mouse game that favors the defenders IMO How so? I'm actually against most of the "safety-tuning" that anthropic does, but this seems fundamentally untrue, a close analogue being video game cheat development. I think in general the cheat developer has an advantage and the cheats generally proliferate for quite a while before being patched.
- ceigey 3mo ago
  Video games are an interesting analogy since they often trade security for performance, trusting clients about world state quite a bit. Finance and biology do come across as two similar high level systems. But while we can employ KYC, fraud detection, and various auditing techniques to finance, I don’t know what you do for biology. You can easily run an algorithm over every transaction a person makes in their account but there’s no equivalent for every cell, every bacteria strain, every virus in the human body.
  - sterlind 3mo ago
    (disclaimer: layperson remembering how the immune system works.) the adaptive immune system effectively does KYC by checking the antigens presented on the surfaces of cells. the thymus selects for B-cells (iirc?) which don't react to a corpus of the body's own antigens, but cover a wide library of everything else. when it sees something it doesn't recognize, it reproduces, warns the rest of the immune system and marks targets. that's why our immune systems can eventually conquer almost every pathogen we encounter, if we can survive long enough for it to do its work. but the KYC I was referring to was KYC that vendors of oligonucleotides (should) be doing, to keep people from ordering nefarious sequences.

iAMkenough 3mo ago

It's inevitable. Also, it's not like I get to vet who does or doesn't have access. Blind trust in the current selection made by an unregulated corporation just makes me anxious. Security in the form of "pay to play" is just kicking the bigger issue down the road.

sneak 3mo ago

I want everyone in the world to be able to perform unlimited cutting edge research on any topic at the maximum thinking level, instantly. The reason we are not being attacked is not lack of technology access.

dyauspitr 3mo ago
It is an access issue. If you could get step by step instructions on how to modify a virus so it kills all people over 6ft you bet your ass there would be people attempting it.
- debesyla 3mo ago
  I guess in this theoretical "AI makes weapon" scenario one could use the same AI to make defences too? // Claude, make antiviral nanobots that defend me from 6ft virus. Make no mistakes.
  - dyauspitr 3mo ago
    I don’t know if you’re being silly but it is orders of magnitudes easier to modify an existing virus to selectively target certain snps than make “antiviral nanobots”
    - jex_the_ape 3mo ago
      Claude, modify the existing 6ft killer virus so that it only makes my balls itch slightly for a day and gives me lifetime immunity to all further stamms of the 6ft killer virus. Make no mistakes, double check so the virus causes no unforseen complications.
- JumpCrisscross 3mo ago
  > It is an access issue Column A, Column B. Building a small explosive device isn't hard. Building a million is very difficult, doing it covertly virtually impossible without the resources of a nation-state. The problem with biologics is the self-assembly and replication machinery comes for "free." So the numpties who might otherwise blow up a trash can [1] now have a real chance of taking out a million people. [1] https://en.wikipedia.org/wiki/2016_New_York_and_New_Jersey_bombings https://en.wikipedia.org/wiki/2016_New_York_and_New_Jersey_b...

digitaltrees 3mo ago

It the tool was made available to anyone to build a virus, anyone would be able to build counter measures, if only a select few people have access they get to build the virus and everyone else is at a disadvantage. So, yes, I am leaning towards making these tools open rather than gated behind some priesthood and government that gets to wield exclusive power.

usef- 3mo ago
Compare the cost/ease of attacker vs defender if one person is given a virus to unleash anywhere in the world and another person is given a vaccine to distribute to the whole world. Or compare building a large bridge to someone disabling that bridge, etc. Prevention and repair is almost always more expensive than vandalism. I don't think there's an ideal solution here, but giving trusted people access to fix security issues before giving it to the wider public seems like a reasonable compromise. They're letting you use the model for all other uses.

jesterson 3mo ago

Do you believe people currently possessing best models act/will act in your best interest?