8 ms·

OVH Security Incident

123 points by bjonathan 13y ago

computer 13y ago
The level of transparency in this report is great. Especially compared to things like the Linode incidents.
- p4bl0 13y ago
  OVH is always very good with transparency with its client. That's part of why I use them for my server, and my ADSL2+ (soon to be VDSL2) at home: when there is a problem, I know what it is and they report their progress solving it in live. Also their customer support is great and doesn't take you for a foul, if they feel you can take technical details, they will gave them to you and really discuss with you about the solution.
  - aroch 13y ago
    Your interactions are pretty much opposite of mine. OVH is constantly doing stupid things to their network (like updating all their BGP reflectors at the same time last week and then falling out of BGP because of it). What they have in transparency and they lose in being a bunch of asshats to deal with. They've taken a week to replace a blown PSU (caused by their own inability to wire a rack); they've dumped my servers because their own internal billing checks weren't aligned with my actual billing details; ffs they essentially root all their servers with their default install image. I won't even go into their blatant breaches of contract...They toss a ToS at you, but you can only read it when signed into your manager and you can only sign into your manager if you accept the ToS. They won't email you the new ToS either, you have to accept it without being able to read it.
    - hahainternet 13y ago
      > a blown PSU (caused by their own inability to wire a rack); Please explain. I spotted the BGP failure but this happens from time to time with basically everyone.
      - aroch 13y ago
        A set of dedi's a bought as soon as BHS opened were on a rack that wasn't wired properly, an electrical short blew a PSU.
        adamseabrook 13y ago
        I believe we also had a bunch of servers in that same rack. Downtime was around 5 minutes from memory.
- Nikkau 13y ago
  Yeah, as great as Github... Oh wait!
Robin_Message 13y ago
If I was a customers, I'd be asking if "based on SHA-512" means some kind of iterated algorithm, or if have they lost my password?
- hbbio 13y ago
  Maybe SHA-512, salted?
  - epsylon 13y ago
    Yep, in the french version of the message they specify that it's salted. http://travaux.ovh.net/?do=details&id=8998 http://travaux.ovh.net/?do=details&id=8998
  - tptacek 13y ago
    Salted SHAx passwords are basically the entire reason GPU John The Ripper and oclHashcat exist, although SHA2-512 is significantly slower than SHA2-256, so if you're going to use a terrible SHA-based password hash, SHA2-512 is your best bet.
    - JshWright 13y ago
      SHA512 is slower on most (all?) current GPUs, but there is plenty of hardware on which it is faster than SHA256.
      - tptacek 13y ago
        Hm. Example?
        JshWright 13y ago
        It'll (theoretically) be faster on any hardware that supports 64 bit operations, as SHA512 ends up doing fewer block operations. This assumes you're hashing something at least 8 bytes long, and that your hash implementation is smart enough to use the 64 bit capabilities of your platform.
peterkelly 13y ago
This is how you do a security incident disclosure. I hope Apple is taking notes.
- at-fates-hands 13y ago
  Are you referring to the recent disclosure, or just in general how they do it?
  - peterkelly 13y ago
    I'm just referring to this specific post from OVH (who I hadn't heard of until today). In reading it, I get the impression that they've handled the situation extremely professionally, and are setting a great example.
nnwa 13y ago
"After internal investigations, it appeared that a hacker was able to obtain access to an email account of one of our system administrators." That translates to password reuse, or an insecure password.
- jessaustin 13y ago
  ...or a client attack, or XSS, or poorly secured tokens, or whatever. If we always blame the user first, we're bound to miss something. Even if the fault were an insecure password, the admin site would still be to blame for not throttling and locking down the account in response to repeated attack.
  - nnwa 13y ago
    Fair point Jess.
jacquesm 13y ago
OVH has come a long way. They used to be cheap and bad at service and totally incommunicado about any issues. Then a few years back something changed and they started to work on their image. Their still cheap, but their service is good and getting better and they seem to have nailed the communications angle. Good for them. Between OVH, Hetzner and Leaseweb the EU hosting space is doing fine.
- seivan 13y ago
  Wasn't there a recent thing with their CEO hating Github because of "Githubs agenda of poaching developers" or some shit like that?
  - johnchristopher 13y ago
    Oh yeah: http://www.ovh.com/fr/a1136.interview-github-octave-klaba-ovh http://www.ovh.com/fr/a1136.interview-github-octave-klaba-ov... Google translate: http://translate.google.com/translate?hl=fr&sl=fr&tl=en&u=http%3A%2F%2Fwww.ovh.com%2Ffr%2Fa1136.interview-github-octave-klaba-ovh http://translate.google.com/translate?hl=fr&sl=fr&tl=en&u=ht...
    - gouggoug 13y ago
      I've always been a huge fan of OVH in my previous company. I had never seen this article. I was planning on moving my company's infrastructure from Rackspace to OVH. Reading this article makes me reconsider my decision. Note to future CEOs: your opinions matter when it comes to getting/keeping customers. Customers not sharing your opinions are likely to leave, if they can, or not subscribe to your services if they're not already subscribed. Bashing github is certainly the best way to alienate the developer community from OVH. Moreover, stating that the reason was because one of their developer got hired by another company makes me wonder what must be the working conditions at OVH. I had thought one day applying there, I won't think about it anymore.
  - temac 13y ago
    Their CEO is not the brightest guy in the universe. This security notice is well written and signed by him... but obviously not written by him.
ihsw 13y ago
Hacked again? https://bitcointalk.org/index.php?topic=186902.msg1936161#msg1936161 https://bitcointalk.org/index.php?topic=186902.msg1936161#ms...
- tallanvor 13y ago
  Quite a different incident, I think.
donohoe 13y ago
An email will be sent today with the new password Password in plain-text? I understand the convenience factor but doesn't sound very secure...
- mike-cardwell 13y ago
  If someone can read your email, they don't need your password. They can just initiate a password reset. https://www.ovh.co.uk/cgi-bin/nic/nicPassword.cgi But yes, in general, email isn't a very secure method of sending passwords.
mrb 13y ago
"The encryption password is "Salted" and based on SHA-512, to avoid brute-force attacks. It takes a lot of technical means to find the word password clearly" "clearly?" OVH is wrong. Based on this information alone, it is not sufficient to say how costly it is to recover the password. SHA-512 needs to be iterated to make it costly to brute force. For example, a raw SHA-512 hash, even salted, is not iterated and is easy to brute force. But multiple passes, as in crypt-SHA-512, are iterated and very costly to brute force.
- jlgaddis 13y ago
  If one's password is "password", the choice of hash algorithms and salts is irrelevant anyway.
  - yeukhon 13y ago
    Yes and no. If you salted a plaintext using any hash algorithm, the biggest advantage is on the population, not an individual password; so yes, it is irrelevant for any single password. If you have comprised a database, and you are interested in a single user, then you can easily recompute months. But if you are interested in compromising a large user population, because you want to find the weakest (like find all the users using 'password' or '123456'), then you have a hard time because with salt you cannot find occurrence. People think salt saves everything, that is not true. It only adds hardness to secure a population, not an individual password (because the salt is appended and stored!). Now when it comes to iteration, sure, adaptive hashes like bcrypt or scrypt can make computation longer. That's all. Using sha512 is still wrong: it's a hash designed for fast hashing, not for securing password. bcrypt is slowly broken. Slowly because workfactor is exponential.
- lmkg 13y ago
  I think you may be reacting to a mis-parse. The sentence is awkwardly phrased either way, but "clearly" could mean "it is clear that it takes a lot of technical means," or it could mean "to find a clear version (i.e. plaintext) of the password."
- huhtenberg 13y ago

123 points by bjonathan 13y ago

computer 13y ago

The level of transparency in this report is great. Especially compared to things like the Linode incidents.

p4bl0 13y ago
OVH is always very good with transparency with its client. That's part of why I use them for my server, and my ADSL2+ (soon to be VDSL2) at home: when there is a problem, I know what it is and they report their progress solving it in live. Also their customer support is great and doesn't take you for a foul, if they feel you can take technical details, they will gave them to you and really discuss with you about the solution.
- aroch 13y ago
  Your interactions are pretty much opposite of mine. OVH is constantly doing stupid things to their network (like updating all their BGP reflectors at the same time last week and then falling out of BGP because of it). What they have in transparency and they lose in being a bunch of asshats to deal with. They've taken a week to replace a blown PSU (caused by their own inability to wire a rack); they've dumped my servers because their own internal billing checks weren't aligned with my actual billing details; ffs they essentially root all their servers with their default install image. I won't even go into their blatant breaches of contract...They toss a ToS at you, but you can only read it when signed into your manager and you can only sign into your manager if you accept the ToS. They won't email you the new ToS either, you have to accept it without being able to read it.
  - hahainternet 13y ago
    > a blown PSU (caused by their own inability to wire a rack); Please explain. I spotted the BGP failure but this happens from time to time with basically everyone.
    - aroch 13y ago
      A set of dedi's a bought as soon as BHS opened were on a rack that wasn't wired properly, an electrical short blew a PSU.
      - adamseabrook 13y ago
        I believe we also had a bunch of servers in that same rack. Downtime was around 5 minutes from memory.
Nikkau 13y ago
Yeah, as great as Github... Oh wait!

Robin_Message 13y ago

If I was a customers, I'd be asking if "based on SHA-512" means some kind of iterated algorithm, or if have they lost my password?

hbbio 13y ago
Maybe SHA-512, salted?
- epsylon 13y ago
  Yep, in the french version of the message they specify that it's salted. http://travaux.ovh.net/?do=details&id=8998 http://travaux.ovh.net/?do=details&id=8998
- tptacek 13y ago
  Salted SHAx passwords are basically the entire reason GPU John The Ripper and oclHashcat exist, although SHA2-512 is significantly slower than SHA2-256, so if you're going to use a terrible SHA-based password hash, SHA2-512 is your best bet.
  - JshWright 13y ago
    SHA512 is slower on most (all?) current GPUs, but there is plenty of hardware on which it is faster than SHA256.
    - tptacek 13y ago
      Hm. Example?
      - JshWright 13y ago
        It'll (theoretically) be faster on any hardware that supports 64 bit operations, as SHA512 ends up doing fewer block operations. This assumes you're hashing something at least 8 bytes long, and that your hash implementation is smart enough to use the 64 bit capabilities of your platform.

peterkelly 13y ago

This is how you do a security incident disclosure. I hope Apple is taking notes.

at-fates-hands 13y ago
Are you referring to the recent disclosure, or just in general how they do it?
- peterkelly 13y ago
  I'm just referring to this specific post from OVH (who I hadn't heard of until today). In reading it, I get the impression that they've handled the situation extremely professionally, and are setting a great example.

nnwa 13y ago

"After internal investigations, it appeared that a hacker was able to obtain access to an email account of one of our system administrators." That translates to password reuse, or an insecure password.

jessaustin 13y ago
...or a client attack, or XSS, or poorly secured tokens, or whatever. If we always blame the user first, we're bound to miss something. Even if the fault were an insecure password, the admin site would still be to blame for not throttling and locking down the account in response to repeated attack.
- nnwa 13y ago
  Fair point Jess.

jacquesm 13y ago

OVH has come a long way. They used to be cheap and bad at service and totally incommunicado about any issues. Then a few years back something changed and they started to work on their image. Their still cheap, but their service is good and getting better and they seem to have nailed the communications angle. Good for them. Between OVH, Hetzner and Leaseweb the EU hosting space is doing fine.

seivan 13y ago
Wasn't there a recent thing with their CEO hating Github because of "Githubs agenda of poaching developers" or some shit like that?
- johnchristopher 13y ago
  Oh yeah: http://www.ovh.com/fr/a1136.interview-github-octave-klaba-ovh http://www.ovh.com/fr/a1136.interview-github-octave-klaba-ov... Google translate: http://translate.google.com/translate?hl=fr&sl=fr&tl=en&u=http%3A%2F%2Fwww.ovh.com%2Ffr%2Fa1136.interview-github-octave-klaba-ovh http://translate.google.com/translate?hl=fr&sl=fr&tl=en&u=ht...
  - gouggoug 13y ago
    I've always been a huge fan of OVH in my previous company. I had never seen this article. I was planning on moving my company's infrastructure from Rackspace to OVH. Reading this article makes me reconsider my decision. Note to future CEOs: your opinions matter when it comes to getting/keeping customers. Customers not sharing your opinions are likely to leave, if they can, or not subscribe to your services if they're not already subscribed. Bashing github is certainly the best way to alienate the developer community from OVH. Moreover, stating that the reason was because one of their developer got hired by another company makes me wonder what must be the working conditions at OVH. I had thought one day applying there, I won't think about it anymore.
- temac 13y ago
  Their CEO is not the brightest guy in the universe. This security notice is well written and signed by him... but obviously not written by him.

ihsw 13y ago

Hacked again? https://bitcointalk.org/index.php?topic=186902.msg1936161#msg1936161 https://bitcointalk.org/index.php?topic=186902.msg1936161#ms...

tallanvor 13y ago
Quite a different incident, I think.

donohoe 13y ago

An email will be sent today with the new password Password in plain-text? I understand the convenience factor but doesn't sound very secure...

mike-cardwell 13y ago
If someone can read your email, they don't need your password. They can just initiate a password reset. https://www.ovh.co.uk/cgi-bin/nic/nicPassword.cgi But yes, in general, email isn't a very secure method of sending passwords.

mrb 13y ago

"The encryption password is "Salted" and based on SHA-512, to avoid brute-force attacks. It takes a lot of technical means to find the word password clearly" "clearly?" OVH is wrong. Based on this information alone, it is not sufficient to say how costly it is to recover the password. SHA-512 needs to be iterated to make it costly to brute force. For example, a raw SHA-512 hash, even salted, is not iterated and is easy to brute force. But multiple passes, as in crypt-SHA-512, are iterated and very costly to brute force.

jlgaddis 13y ago
If one's password is "password", the choice of hash algorithms and salts is irrelevant anyway.
- yeukhon 13y ago
  Yes and no. If you salted a plaintext using any hash algorithm, the biggest advantage is on the population, not an individual password; so yes, it is irrelevant for any single password. If you have comprised a database, and you are interested in a single user, then you can easily recompute months. But if you are interested in compromising a large user population, because you want to find the weakest (like find all the users using 'password' or '123456'), then you have a hard time because with salt you cannot find occurrence. People think salt saves everything, that is not true. It only adds hardness to secure a population, not an individual password (because the salt is appended and stored!). Now when it comes to iteration, sure, adaptive hashes like bcrypt or scrypt can make computation longer. That's all. Using sha512 is still wrong: it's a hash designed for fast hashing, not for securing password. bcrypt is slowly broken. Slowly because workfactor is exponential.
lmkg 13y ago
I think you may be reacting to a mis-parse. The sentence is awkwardly phrased either way, but "clearly" could mean "it is clear that it takes a lot of technical means," or it could mean "to find a clear version (i.e. plaintext) of the password."
huhtenberg 13y ago