9 ms·

I'd just like to echo (verbally "vote") my concern. Every great step forward comes with the potential for an order of magnitude steps backward, i.e. destruction

by eezurr 4y ago

I'd just like to echo (verbally "vote") my concern. Every great step forward comes with the potential for an order of magnitude steps backward, i.e. destruction. The amount of man hours to e.g. blow up a building is far fewer than the amount needed to build it. Will the output of your product add a suicide timer to a cell? Will the output of your product prevent the cells from procreating/multiplying? Will the output of your product prevent pathogen creation? Will the output of your product require a specific, unnatural energy source that can only be man made? Professionals take great care in thinking about those problems, and sometimes still fail. (IIRC, a synthesized breed of mosquitos that were released in Brazil failed to die off and are now a part of the biosystem). [0] https://www.dw.com/en/genetically-modified-mosquitoes-breed-in-brazil/a-50414340 https://www.dw.com/en/genetically-modified-mosquitoes-breed-...

FerociousTimes 4y ago
> Will the output of your product prevent the cells from procreating/multiplying? How would unicellular organisms procreate exactly?! I am not being pedantic here but procreation usually entails sexual reproduction and I don't see how this is possible for these organisms.
- zxexz 4y ago
  I'm pretty sure procreation just refers to the process of reproduction, sexual and asexual inclusive.
  - FerociousTimes 4y ago
    My understanding of the pertinent terms in this specific context is as follows: 1) Multiplying: Asexual reproduction only. 2) Reproduction: Sexual and asexual reproduction. 3) Procreation: Sexual reproduction only.
    - zxexz 4y ago
      I appreciate the way you understand the terminology. However, I'm not sure everyone else understands it the same way. Although, I do admit the terms can be confusing. BTW: I don't see how 'multiplying' could refer to just asexual reproduction. People often use it in sentences like 'the deer population near here multiplied the last couple years'.
      - FerociousTimes 4y ago
        Well, I see how referring to reproduction in a herd of deer, as a collective, in everyday speech, by using the word "multiplying", but not on an individual level, which adds to my point. As you can see, getting terminology right is challenging and not always straightforward.
junofan 4y ago
Cart is way before the horse. Also grad students doing this type of work are absolutely not taking great care in thinking about these problems.
- evanlivingston 4y ago
  I fail to see how considering the impacts of a tool before building a tool is putting the cart before the horse. How is that not a necessary step in building a thing?
  - zmmmmm 4y ago
    I saw someone posted a reactive database just now on HN. I can't believe they didn't think about what child pornographers might do with that. There is literally no discussion of it on their web site. How could it not be a necessary step in building such a thing. Perhaps part of the disconnect here is not realising how vast the applications are for synthesizing DNA oligos are. It's a very basic thing, and anybody can already basically order them online for a very affordable amount. It's like being worried about someone open sourcing a way to make printer ink.
    - evanlivingston 4y ago
      I mean, at what point do stop and ask yourself if the thing you're making is going to cause bad things to happen? Do we only consider that when the thing is really obviously a weapon and ignore all the other creations?
  - pgayed 4y ago
    > I fail to see how considering [all] the impacts of a tool before building a tool is putting the cart before the horse. It’s interesting how people revise their arguments, or omit words from it, to make their interlocutor’s response appear more absurd.
  - pgayed 4y ago
    I hereby vote for the establishment of a Regulatory and Executive Committee to Understand and Reconsider Special Impacts On Nature. Err, wait, ummm, it looks like we first need a R.E.C.U.R.S.I.O.N. to establish the R.E.C.U.R.S.I.O.N.
TJSomething 4y ago
I'm not an expert, but if it's a chip that just synthesizes DNA from a sequence of base pairs, isn't what you're asking similar to making a computer that can't be used to perform evil? I suspect that computing if a given sequence is usable in a pathogen is equivalent to the halting problem. And practically, it seems that a lot of computational resources are required to figure out what a protein does, even for common cases.
- koeng 4y ago
  If we had really smart software engineers, well paid red-teams, and robust government policy collaboration with industry, I think we can make it at least 95% harder to create something dangerous. We have none of those, though. It's super hard, but until novel bioweapons are discovered, it is at least a tractable problem.
koeng 4y ago
> Will the output of your product add a suicide timer to a cell? Absolutely not. > Will the output of your product prevent the cells from procreating/multiplying? Absolutely not. > Will the output of your product prevent pathogen creation? Strictly defined as the output being a chip, absolutely does not. > Will the output of your product require a specific, unnatural energy source that can only be man made? Absolutely not. > Professionals take great care in thinking about those problems I am a professional in this field, and have been thinking about these problems quite deeply (if you check on my website, my first time writing about my concern for these problems was back in 2014). I have developed opinions on this over the years, but roughly they come down to the fact that many folks have a gross misunderstanding of the field in general, but quite like to think that they understand what is going on. For example, I mentioned I wanted to do oligo pool synthesis - how the hell would the output of an oligo pool synth run add a suicide timer? Or prevent replication? Or require a certain kind of energy source? In the context of the stated goal, these objections really don't make any sense. It is roughly equivalent to someone wanting to run a mining company and getting countered by "will the output of your product stop school shootings?". Perhaps better questions are along the lines of - how are oligos matching biohazard sequences prevented from being synthesized? Well, this is a question of both governmental policy (what IS a biohazardous sequence?) and of the integrated device (does it phone home for each synthesized sequence? What about hardware hacking?).
jmcgough 4y ago
This is like asking people if they've properly guarded against a malicious AI before making hobbyist computers. > Will the output of your product prevent the cells from procreating/multiplying? Why would this matter? We have immortal cell lines like hela cells that have been alive for decades. > Will the output of your product prevent pathogen creation? No, but no one is going to "accidentally" create a new pathogen (you'd need this as well as some expensive labware and a lot of expertise), and the people with the incentive can already do so in labs. > Will the output of your product require a specific, unnatural energy source that can only be man made? What lol

by eezurr 4y ago

FerociousTimes 4y ago

> Will the output of your product prevent the cells from procreating/multiplying? How would unicellular organisms procreate exactly?! I am not being pedantic here but procreation usually entails sexual reproduction and I don't see how this is possible for these organisms.

zxexz 4y ago
I'm pretty sure procreation just refers to the process of reproduction, sexual and asexual inclusive.
- FerociousTimes 4y ago
  My understanding of the pertinent terms in this specific context is as follows: 1) Multiplying: Asexual reproduction only. 2) Reproduction: Sexual and asexual reproduction. 3) Procreation: Sexual reproduction only.
  - zxexz 4y ago
    I appreciate the way you understand the terminology. However, I'm not sure everyone else understands it the same way. Although, I do admit the terms can be confusing. BTW: I don't see how 'multiplying' could refer to just asexual reproduction. People often use it in sentences like 'the deer population near here multiplied the last couple years'.
    - FerociousTimes 4y ago
      Well, I see how referring to reproduction in a herd of deer, as a collective, in everyday speech, by using the word "multiplying", but not on an individual level, which adds to my point. As you can see, getting terminology right is challenging and not always straightforward.

junofan 4y ago

Cart is way before the horse. Also grad students doing this type of work are absolutely not taking great care in thinking about these problems.

evanlivingston 4y ago
I fail to see how considering the impacts of a tool before building a tool is putting the cart before the horse. How is that not a necessary step in building a thing?
- zmmmmm 4y ago
  I saw someone posted a reactive database just now on HN. I can't believe they didn't think about what child pornographers might do with that. There is literally no discussion of it on their web site. How could it not be a necessary step in building such a thing. Perhaps part of the disconnect here is not realising how vast the applications are for synthesizing DNA oligos are. It's a very basic thing, and anybody can already basically order them online for a very affordable amount. It's like being worried about someone open sourcing a way to make printer ink.
  - evanlivingston 4y ago
    I mean, at what point do stop and ask yourself if the thing you're making is going to cause bad things to happen? Do we only consider that when the thing is really obviously a weapon and ignore all the other creations?
- pgayed 4y ago
  > I fail to see how considering [all] the impacts of a tool before building a tool is putting the cart before the horse. It’s interesting how people revise their arguments, or omit words from it, to make their interlocutor’s response appear more absurd.
- pgayed 4y ago
  I hereby vote for the establishment of a Regulatory and Executive Committee to Understand and Reconsider Special Impacts On Nature. Err, wait, ummm, it looks like we first need a R.E.C.U.R.S.I.O.N. to establish the R.E.C.U.R.S.I.O.N.

TJSomething 4y ago

I'm not an expert, but if it's a chip that just synthesizes DNA from a sequence of base pairs, isn't what you're asking similar to making a computer that can't be used to perform evil? I suspect that computing if a given sequence is usable in a pathogen is equivalent to the halting problem. And practically, it seems that a lot of computational resources are required to figure out what a protein does, even for common cases.

koeng 4y ago
If we had really smart software engineers, well paid red-teams, and robust government policy collaboration with industry, I think we can make it at least 95% harder to create something dangerous. We have none of those, though. It's super hard, but until novel bioweapons are discovered, it is at least a tractable problem.

koeng 4y ago

> Will the output of your product add a suicide timer to a cell? Absolutely not. > Will the output of your product prevent the cells from procreating/multiplying? Absolutely not. > Will the output of your product prevent pathogen creation? Strictly defined as the output being a chip, absolutely does not. > Will the output of your product require a specific, unnatural energy source that can only be man made? Absolutely not. > Professionals take great care in thinking about those problems I am a professional in this field, and have been thinking about these problems quite deeply (if you check on my website, my first time writing about my concern for these problems was back in 2014). I have developed opinions on this over the years, but roughly they come down to the fact that many folks have a gross misunderstanding of the field in general, but quite like to think that they understand what is going on. For example, I mentioned I wanted to do oligo pool synthesis - how the hell would the output of an oligo pool synth run add a suicide timer? Or prevent replication? Or require a certain kind of energy source? In the context of the stated goal, these objections really don't make any sense. It is roughly equivalent to someone wanting to run a mining company and getting countered by "will the output of your product stop school shootings?". Perhaps better questions are along the lines of - how are oligos matching biohazard sequences prevented from being synthesized? Well, this is a question of both governmental policy (what IS a biohazardous sequence?) and of the integrated device (does it phone home for each synthesized sequence? What about hardware hacking?).

jmcgough 4y ago

This is like asking people if they've properly guarded against a malicious AI before making hobbyist computers. > Will the output of your product prevent the cells from procreating/multiplying? Why would this matter? We have immortal cell lines like hela cells that have been alive for decades. > Will the output of your product prevent pathogen creation? No, but no one is going to "accidentally" create a new pathogen (you'd need this as well as some expensive labware and a lot of expertise), and the people with the incentive can already do so in labs. > Will the output of your product require a specific, unnatural energy source that can only be man made? What lol