5 ms·

Gasoline and lighters are abundant. It takes only slight steering mistake to drive into the crowd. Yet we are still here. Yes, terrible stuff happens, but they

Gasoline and lighters are abundant. It takes only slight steering mistake to drive into the crowd. Yet we are still here. Yes, terrible stuff happens, but they are extreme outliers. Not sure how this works, but we can trust strangers to quite a high degree. I have no experience in the matter, but surely designing viruses is not a simple affair. (In any profession, having a blueprint is not the same as having knowledge, equipment and practical experience; if somebody would try, my bet would be that they would die during early stages of the process, due to inexperience of handling hazardous materials; kind of like Mr Darwin looks after us)

baq 3d ago
Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an aligned LLM to splice some smallpox into a generic research virus? BSL4 labs do things like this by hand all the time, so it can’t be too difficult and it only takes one lucky AI cultist for the rest of us to find out. I hope I’m wrong, but hope it’s not a process.
- tancop 3d ago
  All you need to pull that off is public info, student level knowledge and access to a lab with standard equipment. As you said labs do things like that all the time. Its already easy even without powerful AI. The reason we dont see any serious attempts to create a deadly pandemic is not that we keep insane people out of labs, because there is no way to find out if someone is a cultist or murderous psycho when they have every reason to actively hide it. Its because a) most people are not evil and b) a virus that can kill everyone it touches would probably kill its creator first. As someone with basically zero bio experience I think I could get a working smallpox or ebola level virus in a year or two if I had time to learn, lab equipment and Annas Library. No LLMs needed. But I dont want to do that, and I think its the same for all the people who could do it right now.
- falserum 3d ago
  Math and computers have benefit of not needing materials. Add a button here, add lema and assumption there - all are almost zero cost&risk. So iteration/experimentation with llm in hand is possible to some degree even for laymen. I assume virus creation is more like building a house. It involves physical work and chemical reactions that need time to finish in correct order. Each attempt costs a LOT more of time&money and involves actual risks.

baq 3d ago

Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an aligned LLM to splice some smallpox into a generic research virus? BSL4 labs do things like this by hand all the time, so it can’t be too difficult and it only takes one lucky AI cultist for the rest of us to find out. I hope I’m wrong, but hope it’s not a process.

tancop 3d ago
All you need to pull that off is public info, student level knowledge and access to a lab with standard equipment. As you said labs do things like that all the time. Its already easy even without powerful AI. The reason we dont see any serious attempts to create a deadly pandemic is not that we keep insane people out of labs, because there is no way to find out if someone is a cultist or murderous psycho when they have every reason to actively hide it. Its because a) most people are not evil and b) a virus that can kill everyone it touches would probably kill its creator first. As someone with basically zero bio experience I think I could get a working smallpox or ebola level virus in a year or two if I had time to learn, lab equipment and Annas Library. No LLMs needed. But I dont want to do that, and I think its the same for all the people who could do it right now.
falserum 3d ago
Math and computers have benefit of not needing materials. Add a button here, add lema and assumption there - all are almost zero cost&risk. So iteration/experimentation with llm in hand is possible to some degree even for laymen. I assume virus creation is more like building a house. It involves physical work and chemical reactions that need time to finish in correct order. Each attempt costs a LOT more of time&money and involves actual risks.