6 ms·

Professional auditing and security testing should be necessary for any piece of software from which it's possible to drain large sums of money, regardless of wh

by nmrm 12y ago

Professional auditing and security testing should be necessary for any piece of software from which it's possible to drain large sums of money, regardless of who's running the software or holding the money. In fact, I'd argue anything less constitutes an ethical breach on the part of the lead engineer(s). I'm not really sure why that particular regulation is so onerous in any of these situations, since any responsible team would be thinking about security throughout, including and especially post-development. At least in principle (maybe you'd prefer different auditing standards or practices -- that's what I mean by in principle). edit: of course each of these is a different sort of service, and different levels of risk management are appropriate. In particular, the case of Blockchain seems like a real quagmire. I guess it would probably depend heavily on the revenue model. Really my only point is that I would have a really hard time sleeping at night if I had to sign off on not applying state-of-the-art auditing and testing techniques any of these, even Blockchain. Maybe I'm too crotchety and old-school for bitcoin.

Ryan_Singer 12y ago
For the Reddit Tip Bot?
- nmrm 12y ago
  The honest answer is that I don't know, because I don't know how much money the reddit tipbot handles or how long the money stays vulnerable to a potential bug in the software. This is criteria I suggest: > for any piece of software from which it's possible to drain large sums of money To which I might add "directly" or "quickly" or "covertly", but I think you get the intent. My edit to the parent comment also applies, since none of these is a binary value and risk management should match the assumed risk. edit: I would say that the fact that it's only a few bucks per user is not relevant, especially if that limit isn't hard-coded.
- fred_durst 12y ago
  How about this https://github.com/vindimy/altcointip/blob/master/src/cointipbot.py#L457-L466 https://github.com/vindimy/altcointip/blob/master/src/cointi... with this https://github.com/vindimy/altcointip/blob/master/src/ctb/ctb_action.py#L154-L174 https://github.com/vindimy/altcointip/blob/master/src/ctb/ct... You don't think that might allow someone to do a bit of skimming by playing a little loose with the exchange rates?
starkness 12y ago
We're in the process right now of writing up a more in-depth policy proposal, but you're spot on in terms of different levels of risk management. And we're not at all against security testing, which is also one of the great benefits of FOSS. ("Given enough eyeballs, all bugs are shallow.") And the audits comprise financial audits as well, which surely make sense for bitcoin exchanges and companies holding funds, but not so much for open source projects or technologies that are built around bitcoin but where no funds are held. That said, the actual regulatory proposal has many more requirements than even mentioned in the article (including quarterly reports to the NY State Superintendent, collecting of user data, and the possibility of being denied a license without a system for due process in place), and things that the creator of a Reddit tip bot surely couldn't comply with.
- nmrm 12y ago
  This is great to hear. The best way for software to keep itself relatively unburdened by (poorly implemented) regulation is for industry to hold itself to high standards. And why wouldn't we? We're proud of what we build. I'll be very curious to see how companies built up around client software but not directly handling money are treated in your proposal. I think safety-critical industries cover these in various different ways, normally under the assumption that the companies are producing either a) "components" for use in safety-critical systems; or b) tools which will be used for QA processes. I'm not sure either applies well, especially in the case of OSS. And I don't know of anything similar in finance.
- fred_durst 12y ago
  > Given enough eyeballs, all bugs are shallow. Tell that to OpenSSL.
  - starkness 12y ago
    Or tell that to Linus Torvalds, as it's his law.
throwaway2274 12y ago
> Professional auditing and security testing should be necessary for any piece of software from which it's possible to drain large sums of money, regardless of who's running the software or holding the money. In fact, I'd argue anything less constitutes an ethical breach on the part of the lead engineer(s). Would you include web browsers, OSs, system libraries and such in that definition? All those can steal users money if compromised. If so, who do you suggest be responsible for that in an open source project?
- nmrm 12y ago
  > All those can steal users money if compromised. Not in a vacuum; they have to be deployed in a setting where that's possible. > Would you include web browsers, OSs, system libraries and such in that definition? It's sort-of a moot point, because the major products in all of these areas are routinely analyzed from a security perspective. Apple and Microsoft both spend a lot of money on security, and security researchers spend lots of time and effort auditing linux. > If so, who do you suggest be responsible for that in an open source project? The organization deploying the software in a security-critical setting should follow best practices when selecting and maintaining components. There's a significant difference between engineering failures that happen even when you've followed best practices, and very preventable engineering failures that happen only because you've not followed best practices. Just because perfect security isn't possible doesn't mean we should give up entirely and not even both sanitizing input, for instance. Additionally, OS vendors should not encourage users to use their software in security-critical settings unless the vendor is following best practices w.r.t. security. This is where I could see some bitcoin projects getting into trouble.
pyvpx 12y ago
yes, it should. should that shouldness be codified into law and help entrenched market participants stay entrenched? because while I do see millions spent on bank software security, wellsfargo.com is still an enormous joke.
- nmrm 12y ago
  I'm referring only to the specific case of regulations which cover engineering practice (as opposed to more industry-specific regulations, for instance, which I don't know enough to comment on). In these cases, absolutely yes! The shouldness should be codified into law. The best mechanism (regulation vs. after-the-fact culpability; specific legislation vs. using existing frameworks, etc.) is debatable. But companies who cause public harm by not following best practices (either intentionally or due to poor trained engineers) should be held legally responsible for preventable disasters. Just like it's done in many more mature (as in older) engineering fields.

by nmrm 12y ago

Ryan_Singer 12y ago

For the Reddit Tip Bot?

nmrm 12y ago
The honest answer is that I don't know, because I don't know how much money the reddit tipbot handles or how long the money stays vulnerable to a potential bug in the software. This is criteria I suggest: > for any piece of software from which it's possible to drain large sums of money To which I might add "directly" or "quickly" or "covertly", but I think you get the intent. My edit to the parent comment also applies, since none of these is a binary value and risk management should match the assumed risk. edit: I would say that the fact that it's only a few bucks per user is not relevant, especially if that limit isn't hard-coded.
fred_durst 12y ago
How about this https://github.com/vindimy/altcointip/blob/master/src/cointipbot.py#L457-L466 https://github.com/vindimy/altcointip/blob/master/src/cointi... with this https://github.com/vindimy/altcointip/blob/master/src/ctb/ctb_action.py#L154-L174 https://github.com/vindimy/altcointip/blob/master/src/ctb/ct... You don't think that might allow someone to do a bit of skimming by playing a little loose with the exchange rates?

starkness 12y ago

We're in the process right now of writing up a more in-depth policy proposal, but you're spot on in terms of different levels of risk management. And we're not at all against security testing, which is also one of the great benefits of FOSS. ("Given enough eyeballs, all bugs are shallow.") And the audits comprise financial audits as well, which surely make sense for bitcoin exchanges and companies holding funds, but not so much for open source projects or technologies that are built around bitcoin but where no funds are held. That said, the actual regulatory proposal has many more requirements than even mentioned in the article (including quarterly reports to the NY State Superintendent, collecting of user data, and the possibility of being denied a license without a system for due process in place), and things that the creator of a Reddit tip bot surely couldn't comply with.

nmrm 12y ago
This is great to hear. The best way for software to keep itself relatively unburdened by (poorly implemented) regulation is for industry to hold itself to high standards. And why wouldn't we? We're proud of what we build. I'll be very curious to see how companies built up around client software but not directly handling money are treated in your proposal. I think safety-critical industries cover these in various different ways, normally under the assumption that the companies are producing either a) "components" for use in safety-critical systems; or b) tools which will be used for QA processes. I'm not sure either applies well, especially in the case of OSS. And I don't know of anything similar in finance.
fred_durst 12y ago
> Given enough eyeballs, all bugs are shallow. Tell that to OpenSSL.
- starkness 12y ago
  Or tell that to Linus Torvalds, as it's his law.

throwaway2274 12y ago

> Professional auditing and security testing should be necessary for any piece of software from which it's possible to drain large sums of money, regardless of who's running the software or holding the money. In fact, I'd argue anything less constitutes an ethical breach on the part of the lead engineer(s). Would you include web browsers, OSs, system libraries and such in that definition? All those can steal users money if compromised. If so, who do you suggest be responsible for that in an open source project?

nmrm 12y ago
> All those can steal users money if compromised. Not in a vacuum; they have to be deployed in a setting where that's possible. > Would you include web browsers, OSs, system libraries and such in that definition? It's sort-of a moot point, because the major products in all of these areas are routinely analyzed from a security perspective. Apple and Microsoft both spend a lot of money on security, and security researchers spend lots of time and effort auditing linux. > If so, who do you suggest be responsible for that in an open source project? The organization deploying the software in a security-critical setting should follow best practices when selecting and maintaining components. There's a significant difference between engineering failures that happen even when you've followed best practices, and very preventable engineering failures that happen only because you've not followed best practices. Just because perfect security isn't possible doesn't mean we should give up entirely and not even both sanitizing input, for instance. Additionally, OS vendors should not encourage users to use their software in security-critical settings unless the vendor is following best practices w.r.t. security. This is where I could see some bitcoin projects getting into trouble.

pyvpx 12y ago

yes, it should. should that shouldness be codified into law and help entrenched market participants stay entrenched? because while I do see millions spent on bank software security, wellsfargo.com is still an enormous joke.

nmrm 12y ago
I'm referring only to the specific case of regulations which cover engineering practice (as opposed to more industry-specific regulations, for instance, which I don't know enough to comment on). In these cases, absolutely yes! The shouldness should be codified into law. The best mechanism (regulation vs. after-the-fact culpability; specific legislation vs. using existing frameworks, etc.) is debatable. But companies who cause public harm by not following best practices (either intentionally or due to poor trained engineers) should be held legally responsible for preventable disasters. Just like it's done in many more mature (as in older) engineering fields.