10 ms·

When MFA isn't MFA, or how we got phished

393 points by dvdhsu 3y ago

kerblang 3y ago
I don't understand: Why on earth does google want to sync MFA tokens? They're one-time use, aren't they? Or... feh, I can't even fathom
- kerblang 3y ago
  Answering myself, this helps a bit: https://www.zdnet.com/article/google-authenticator-will-now-sync-your-2fa-codes-to-use-on-different-devices/ https://www.zdnet.com/article/google-authenticator-will-now-... I guess we need a better way to handle "Old phone went swimming, had to buy another, now what?"
  - mbesto 3y ago
    Which is funny because the 2nd factor is "something I have", which means if you don't "have it" then you can ever complete the 2nd factor. This ultimately means the 2nd factor, when you're phone goes swimming, is ultimately your printed codes.
- jeremyjh 3y ago
  They mean they are syncing the private key used to generate the tokens on demand.
  - magospietato 3y ago
    Well that's even worse isn't it?
  - kerblang 3y ago
    Do all these 2FA apps - like say Microsoft Authenticator - have these hidden/not-so-hidden private keys? From other posts it sounds like you can view the token and write it down... MA doesn't have that, I don't think.
    - kerblang 3y ago
      Answering myself again, yeah, they all seem to have this private key hidden away somewhere. Didn't know that. https://frontegg.com/blog/authentication-apps#How-Do-Authenticator-Apps-Work https://frontegg.com/blog/authentication-apps#How-Do-Authent...?
    - e12e 3y ago
      TOTP (Time-based one-time password) need a shared secret (and two synchronized clocks) to work, so yes. FIDO2/WebAuthn relies on public key technology - so does also have a secret key - but is designed to be kept secret from the service/server one authenticates against. For use - FIDO2 is more like a multi-use id. Like a driver's license many services accept as id. If you lose it - you don't restore a backup copy from a safe - you use your passport until you get a new one issued. This makes more sense than with TOTP as the services only need your public key(id) on file. https://en.wikipedia.org/wiki/Time-based_One-time_Password https://en.wikipedia.org/wiki/Time-based_One-time_Password https://en.m.wikipedia.org/wiki/WebAuthn https://en.m.wikipedia.org/wiki/WebAuthn
      - rawgabbit 3y ago
        Which FIDO2 service do you recommend? I get tired reading all these security articles. The more I read, the more I feel they are hiding something.
        e12e 3y ago
        > Which FIDO2 service do you recommend? Generally what comes with your phone and one or two hw tokens for backup? Looks like token2.com is a reasonable choice if you just want NFC/USBc and FIDO2 (and not storage for ssh/gpg keys). But I have little experience with hw keys.
        drhuseynov 3y ago
        ssh and pgp keys are not based on the similar functionality. the keys from Token2 support *-sk key storage https://www.token2.com/site/page/using-token2-fido2-security-keys-for-openssh https://www.token2.com/site/page/using-token2-fido2-security... But not PGP
- unethical_ban 3y ago
  Syncing of "MFA codes" is really syncing of the secret component of TOTP (time based one time password). And it's a good thing, and damn any 2fa solution that blocks it. I don't want to go through onerous, incompetent, poorly designed account recovery procedures if a toddler smashes my phone. So I use authy personally, while a friend backs his up locally.
  - skybrian 3y ago
    A better way to fix this is to have multiple ways to log in. Printed backup codes in your safe with your personal papers and/or a Yubikey on your keychain. This works for Google and Github, at least. Passkey syncing is more convenient, though, and probably an improvement on what most people do.
  - itake 3y ago
    > I don't want to go through onerous, incompetent, poorly designed account recovery procedures if a toddler smashes my phone Why don't you use the printed recovery tokens?
    - monocasa 3y ago
      Who has a printer these days?
      - CameronNemo 3y ago
        Local libraries, print shops... but yeah that may be an attack vector.
    - unethical_ban 3y ago
      Not all websites offer them. Hell, no bank I use (several large and several regional) support generic totp. Some have sms, one has Symantec VIP, proprietary and not redundant. Edit: since I'm posting too fast according to HN, even though I haven't posted in an hour, I'll say it here. Symantec is totp but You cannot back up your secrets and you cannot have backup codes.
- AshamedCaptain 3y ago
  For me the question is "who the fsck uses Google Authenticator to store all their tokens, both company and personal?"
  - burkaman 3y ago
    Google Authenticator was I believe the first available TOTP app, and is by far the most popular. It used to be open source and have no connection to your Google account. Many people installed it years ago when they first set up MFA, and have just been adding stuff to it ever since because it's easy and it works. Even for technical users who understand how TOTP works, there is no obvious reason it appears unsafe to put all your tokens in the app (until you read this article). Look at the MFA help page for any website you use. One of the first sentences is probably something like "First you'll need to install a TOTP app on your phone, such as Google Authenticator or Authy..." It really did used to be the best option. For example, see this comment from 10 years ago when Authy first launched: > The Google Authenticator app is great. I recently got (TOTP) 2-factor auth for an IRC bot going with Google Authenticator; took about 5 minutes to code it up and set it up. It doesn't use any sort of 3rd party service, just the application running locally on my phone. TOTP/HOTP is dead simple and, with the open source Google Authenticator app, great for the end user. - https://news.ycombinator.com/item?id=6137051 https://news.ycombinator.com/item?id=6137051
    - fireflash38 3y ago
      I think technically Blizzard Authenticator (even the app) was available before Google Authenticator, but obviously for extremely limited use.
    - glandium 3y ago
      Also, since it doesn't allow to extract the private keys, you're kind of stuck with it once you've started using it.
bawolff 3y ago
While the google cloud thing is a weird design, that seems like the wrong place to blame. TOTP and SMS based 2FA are NOT designed to prevent phishing. If you care about phishing use yubikeys.
consoomer 3y ago
1. install pass otp 2. pass otp add whatever/otp/me 3. paste in "otpauth://totp/whatever?secret=whateveritis 4. pass git init; push to remote Now you you have MFA on any device that has git and your gpg key.
rahidz 3y ago
>The caller claimed to be one of the members of the IT team, and deepfaked our employee’s actual voice. The voice was familiar with the floor plan of the office, coworkers, and internal processes of the company. Wow that is quite sophisticated.
- tough 3y ago
  inside job?
  - mistrial9 3y ago
    how's that Zero Trust architecture working out for everyone ?
    - wepple 3y ago
      What’s this got to do with zero trust?
      - mistrial9 3y ago
        it is a cynical comment that is meant to hilite the relationship between humans where oppressive and untrusting employment leads to increase in antipathy, ill-will, feelings of being abused and all of that leading to insider theft and serious pre-meditated betrayal ?
        wyldberry 3y ago
        Zero Trust is such a bad branding for how the architecture works. It's just "always prove" architecture.
        tough 3y ago
        It does seem to sound pretty well on the mind of the executives signing the deals that hear the marketing talk
        3y ago
pyrolistical 3y ago
Naming/training issue imo. We need a better name than MFA. Something like “personal password like token that should only be entered into secure computer on specific website/app/field and never needed to be shared”
- mr_mitm 3y ago
  It's well known that OTP is not immune to phishing. Force your users on webauthn or some other public key based second factor if you're aiming at decreasing the incident rate.
  - unethical_ban 3y ago
    Well, push based 2fa with "select this number on your 2fa device" helps prevent some vectors. Simple totp doesn't do that. "Never give your totp or one time code over the phone" is good advice. "Never give info to someone who called you, call them back on the official number" is another. This is user error at this point.
    - AshamedCaptain 3y ago
      I disagree. Specially again that companies are centralizing on a couple 2FA companies (like Okta from TFA), this is just ripe for phishing. Okta itself is terrible at this; they don't consistently use the okta.com domain, so users are at a loss and have basically no protection against impersonators.
      - unethical_ban 3y ago
        For okta, if it is set up properly, the user should get push notifications. And in that push notification is a number they need to select to validate the push. This eliminates credential phishing and "notification exhaustion" where a user just clicks "ok" on an auth request by a bad actor. As much as I advocate for non cloud services, what okta provides is very secure.
  - jpc0 3y ago
wepple 3y ago
Why did they need to call? They could’ve phished the password and MFA by simply MITMing? Perhaps we need a distinction from phishable MFA and unphishable U2F/WebAuthn style
- rsstack 3y ago
  > The caller claimed to be one of the members of the IT team, and deepfaked our employee’s actual voice. The voice was familiar with the floor plan of the office, coworkers, and internal processes of the company. Throughout the conversation, the employee grew more and more suspicious, but unfortunately did provide the attacker one additional multi-factor authentication (MFA) code. > The additional OTP token shared over the call was critical, because it allowed the attacker to add their own personal device to the employee’s Okta account, which allowed them to produce their own Okta MFA from that point forward. They needed to have a couple of minutes to set things up from their end, and then ask for the second OTP code. A phone call works well for that.
  - wepple 3y ago
    Ahh, thanks and apologies for not re-reading before asking. That is indeed interesting; keep the con going a bit longer to get a proper foothold.
rolobio 3y ago
Very sophisticated attack, I would bet most people would fall for this. I'm surprised Google encourages syncing the codes to the cloud... kind of defeats the purpose. I sync my TOTP between devices using an encrypted backup, even if someone got that file they could not use the codes. FIDO2 would go a long way to help with this issue. There is no code to share over the phone. FIDO2 can also detect the domain making the request, and will not provide the correct code even it the page looks correct to a human.
- victor106 3y ago
  They could’ve just had employees use Okta Verify as opposed to Google Authenticator
- softfalcon 3y ago
  >FIDO2 can also detect the domain making the request, and will not provide the correct code even it the page looks correct to a human. I could not agree more with this sentiment! We need more of this kind of automated checking going on for users. I'm tired of seeing "just check for typo's in the URL" or "make sure it's the real site!" advice given to the average user. People are not able to do this even when they know how to protect themselves. Humans tire easily and are often fallible. We need more tooling like FIDO2 to automate away this problem for us. I hope the adoption of it will go smoothly in years to come.
  - miki123211 3y ago
    The problem with Fido (and other such solutions, including smartphone-based passkeys) is that they make things extremely hard if you're poor / homeless / in an unsafe / violent family situation and therefore change devices often. It's mostly a non-issue for Silicon Valley tech employees working solely on their corporate laptops, and U2F is perfect for that use-case, but these concerns make MFA a non-starter for the wider population. We could neatly sidestep all of these issues with cloud-based fingerprint readers, but the privacy advocates won't ever let that happen.
    - 3y ago
AYBABTME 3y ago
To deepfake the voice of an actual employee, they would need enough recorded content of that employee's voice... and I would think someone doing admin things on their platform isn't also in DevRel with a lot of their voice uploaded online for anyone to use. So it smells like someone with close physical proximity to the company would be involved.
- gabereiser 3y ago
  There’s a lot of ways to get clips of recordings of someone’s voice. You can get that if they ever spoke at a conference or on a video. Numerous other ways I won’t list here.
- V__ 3y ago
  One possibility would be to just call the employee and record their voice. One could pretend to be a headhunter.
  - skeaker 3y ago
    This would almost certainly be it. Calling someone to record them and using their voice later to impersonate them was done even before deep-fake voices were a concept. With the tools available now, even a short call + the grainy connection of a phone voice line would be more than enough to make a simulated voice work.
  - rlt 3y ago
    I'm already cautious about answering calls from unknown numbers. This could be a good reason to be even more cautious.
- themagician 3y ago
  Probably wasn't a "deepfake" just someone decent with impressions and a $99 mixer. After compression this will be more than good enough to fool just about anyone. No deepfake is needed. Just call the person once and record a 30 second phone call. Tell them you are delivering some package and need them to confirm their address.
brunojppb 3y ago
Fantastic write-up. Major props for disclosing the details of the attack in a very accessible way. It is great that this kind of security incident post-mortem is being shared. This will help the community to level-up in many ways, specially given that its content is super accessible and not heavily leaning on tech jargon.
- hn_throwaway_99 3y ago
  I disagree. I appreciate the level of detail, but I don't appreciate Retool trying to shift the blame to Google, and only putting a blurb in the end about using FIDO2. They should have been using hardware keys years ago.
  - duderific 3y ago
    It was also a bit weird how they kept emphasizing how their on-prem installations were not affected, as if that lessens the severity somehow. It's like duh, that's the whole point of on-prem deployments.
  - dvdhsu 3y ago
    Hi, I'm sorry you felt that way. "Shifting blame to Google" is absolutely not our intention, and if you have any recommendations on how to make the blog post more clear, please do let me know. (We're happy to change it so it reads less like that.) I do agree that we should start using hardware keys (which we started last week). The goal of this blog post was to make clear to others that Google Authenticator (through the default onboarding flow) syncs MFA codes to the cloud. This is unexpected (hence the title, "When MFA isn't MFA"), and something we think more people should be aware of.
    - hn_throwaway_99 3y ago
      I felt like you were trying to shift blame to Google due to the title "When MFA isn't MFA" and your emphasis on "dark patterns" which, to be honest, I don't think they are that "dark". To me it was because this felt like a mix of a post mortem/apology, but with some "But if it weren't for Google's dang dark patterns..." excuse thrown in. FWIW, nearly every TOTP authenticator app I'm aware of supports some type of seed backup (e.g. Authy has a separate "backup password"). I actually like Google's solution here as long as the Workspace accounts are protected with a hardware key. The only real lesson here is that you should have been using hardware keys.
batmansmk 3y ago
Are the claims of deepfake and intimate knowledge of procedures based of the sole testimony of the employee who oopsed terribly? This is a novelisation of an events Retool needs to revise the basic security posture. There is no point in complicated technology if the warden just gives the key away.
- hn_throwaway_99 3y ago
  > Retool needs to revise the basic security posture. Couldn't agree more. TBH I thought this post was an exercise in blame shifting, trying to blame Google. > We use OTPs extensively at Retool: it’s how we authenticate into Google and Okta, how we authenticate into our internal VPN, and how we authenticate into our own internal instances of Retool. The fact that access to a Google account immediately gave access to all MFA tokens held within that account is the major reason why the attacker was able to get into our internal systems. Google Workspace makes it very easy to set up "Advanced Protection" on accounts, in which case it requires using a hardware key as a second factor, instead of a phishable security code. Given Retool's business of hosting admin apps for lots of other companies, they should have known they'd be a prime target for something like this, and not requiring hardware keys is pretty inexcusable here.
  - dotty- 3y ago
    > Google Workspace makes it very easy to set up "Advanced Protection" on accounts, in which case it requires using a hardware key as a second factor, instead of a phishable security code. This isn't immediately actionable for every company. I agree Retool should have hardware keys given their business, but at my company with 170 users we just haven't gotten around to figuring out the distribution and adoption of hardware keys internationally. We're also a Google Workspace customer. I think it's stupid for a company like Google, the company designing these widely used security apps for millions of users, to allow for cloud syncing without allowing administrators the ability to simply turn off the feature on a managed account. Google Workspace actually lacks a lot of granular security features, something I wish they did better. What is a company like mine meant to do here to counter this problem? edit: changed "viable" for "immediately actionable". It's easy for Google to change their apps. Not for every company to change their practices.
RcouF1uZ4gsC 3y ago
One thing that is left out it to use unphishable MFA like hardware security keys (Yubikey, etc).
xorcist 3y ago
Stopped reading at "deepfake". It's the new advanced persistent threat, a perfect phrase to divert any resposibility. (Yes, there are deepfakes. Yes, there are APTs. This is likely neither.)
- skeaker 3y ago
  I am (genuinely, really asking) curious why you think so. I've got no hand in this, but the skepticism around phishing attacks from this site of all places really surprises me. People like Kevin Mitnick have done more sophisticated phishing with fewer tools. Why wouldn't someone intent on running a social engineering scam use one of the widely available voice faking technologies that are available now? Keep in mind that they're simple enough to use that people are making memes with voices generated from ~5 seconds of voice recordings.
  - xorcist 3y ago
    Making a meme is nothing like an interactive telephone conversation. It's not that it's impossible, but it's not trivial either. But mainly, it's just unnecessary. If the user is not fooled by a well crafted phishing, by doing the most trivial countermeasures such as calling back, they are not going to be fooled by a deepfake. In practice work on phishing is mostly better spent elsewhere. So while we shouldn't dismiss it completely, it's clearly not the case with a smallish company with limited economic value, so very unlikely the case here. There has been a handful of highly profile media cases involving deepfake. None of which has held up on further investigation. It is understandable, nobody wants to be known as the one who didn't recognize his own kid on the phone, but the truth is more simple and actually helps us when designing countermeasures.
    - skeaker 3y ago
      I suppose the disconnect then would be that we fundamentally disagree on what the simpler answer is. It's my understanding that a deepfake voice being used as part of a phishing scam is something that can be done trivially (or at least by a determined actor using free tools, so at least trivial enough for this case), so to me that would be the simplest, most obvious answer when compared to a company-wide conspiracy, but I can see your point if it is assumed that that isn't the case and that deepfake voices are actually hard to do.
account-5 3y ago
I wonder how long it'll be before a similar attack happens before someone's/a companies passkeys are synced to the cloud.
hn_throwaway_99 3y ago
Question for security folks out there: So often I see these kinds of phishing attacks that have hugely negative consequences (see the MGM Resorts post earlier today), and the main problem is that just one relatively junior employee who falls for a targeted phishing attack can bring down the whole system. Is anyone aware of systems that essentially require multiple logins from different users when accessing sensitive systems like internal admin tools? I'm thinking like the "turn the two keys simultaneously to launch the missile" systems. I'm thinking it would work like the following: 1. If a system detects a user is logging into a particularly sensitive area (e.g. a secrets store), and the user is from a new device, the user first needs to log in using their creds (including any appropriate MFA). 2. In addition, another user like an admin would need to log in simultaneously and approve this access from a new device. Otherwise, the access would be denied. I've never seen a system like this in production, and I'm curious why it isn't more prevalent when I think it should be the default for accessing highly sensitive apps in a corporate environment.
- joshxyz 3y ago
  i wonder on this too if people really use shamir secret sharing as part of some security compliance
- deleted 3y ago
  [deleted]
- fireflash38 3y ago
  You're looking for quorums, or key splits. They aren't super common. You see them with some HSMs (need M of N persons to perform X action).
  - joshxyz 3y ago
    not good with acronyms, what is hsm here?
    - coderintherye 3y ago
      Hardware security module https://en.wikipedia.org/wiki/Hardware_security_module https://en.wikipedia.org/wiki/Hardware_security_module
miki123211 3y ago
does iOS have a "is there a call in progress" API? If so, it would be a good idea for OTP apps to use it and display a prominent warning banner when opened during a call.
alsodumb 3y ago
Maybe it’s just me, but I am really skeptical about the DeepFake part - it’s a theoretically possible attack vector, but the only evidence they possibly could have to support this statement would be the employees testimony. Targeting a particular employee with the voice of a specific person this employee knows requires a lot of information and insider info. Also, I think the article spends a lot of effort trying to blame Google Authenticator and make it seems like they had the best possible defense and yet attackers managed to get through because of Googles error. Nope, not even close. They would have had hardware 2FA if they were really concerned about security. Come on guys, it’s 2023 and hardware tokens are cheap. It’s not even a consumer product where one can say that hardware tokens hinder usability. It’s a finite set of employees, who need to do MFA certain times for certain services mostly using one device. Just start using hardware keys.
- dvdhsu 3y ago
  Hi, David, founder @ Retool here. We are currently working with law enforcement, and we believe they have corroborating evidence through audio that suggests a deepfake is likely. (Put another way, law enforcement has more evidence than just the employee's testimony.) (I wish we could blog about this one day... maybe in a few decades, hah. Learning more about the government's surveillance capabilities has been interesting.) I agree with you on hardware 2FA tokens. We've since ordered them and will start mandating them. The purpose of this blog post is to communicate that what is traditionally considered 2FA isn't actually 2FA if you follow the default Google flow. We're certainly not making any claims that "we are the world's most secure company"; we are just making the claim that "what appears to be MFA isn't always MFA". (I may have to delete this comment in a bit...)
  - alsodumb 3y ago
    Thanks for the reply! What's expecting one. Since you might have you delete the reply anyway, can I get a candid answer on why hardware 2FA tokens weren't a part of the default workflow before the incident? Was it concerns about the cost, the recovery modes, or was it just the trust in the existing approach?
  - 3y ago
yieldcrv 3y ago
I just call them one-time passcodes (otp) Most of the time I am not using multifactor or 2factor the way it was designed But it is accurately a one time passcode
andrewstuart 3y ago
Some startup, please make a product that uses AI to identify these obviously fake emails. Hello A, This is B. I was trying to reach out in regards to your [payroll system] being out of sync, which we need synced for Open Enrollment, but i wasn’t able to get ahold of you. Please let me know if you have a minute. Thanks You can also just visit https://retool.okta.com.[oauthv2.app]/authorize-client/xxx https://retool.okta.com.[oauthv2.app]/authorize-client/xxx and I can double check on my end if it went through. Thanks in advance and have a good night A.
- deleted 3y ago
  [deleted]
macNchz 3y ago
Beyond having hardware keys, this scenario is why I really try to drive home, in all of my security trainings, the idea that you should instantly short circuit any situation where you receive a phone call (or other message) and someone starts asking for information. It's always okay to say, "actually, let me get back to you in a minute" and hang up, calling back on a known phone number from the employee directory, or communicate on different channel altogether. Organizationally, everyone should be prepared for and encourage that kind of response as well, such that employees are never scared to say it because they're worried about a snarky/angry/aggressive response. This also applies to non-work related calls: someone from your credit card company is calling and asking for something? Call back on the number on the back of your card.
- hinkley 3y ago
  Advice I haven't even followed myself: It's probably a good idea to program your bank's fraud number into your phone. The odds that someone hacks your bank's Contact Us page are small but not zero. The bedrock of both PGP and .ssh/known_hosts could be restated as, "get information before anyone knows you need it". Fraud departments contacting me about potentially fraudulent charges is always going to make me upset. Jury is still out on whether it will always trigger a rant, but the prognosis is not good.
  - GauntletWizard 3y ago
    At least once I have gotten a terribly phrased and link-strewn "Fraud Alert" from a bank, reported it to said bank's anti-phisihing e-mail address, gotten a personalized mail that responded that it was in fact fraud and that they had policies against using third party subdomains like... And then found out the day later that yes, that was their real new anti-fraud tool and template. There will need to be jail time for the idiots writing the government standards on these fraud departments before we get jail time for the idiots running these fraud departments before it gets better.
    - hinkley 3y ago
      Last time I talked to someone about this they pointed out that fraud depts are often outsourced. Which is a lovely plan because now your customers hate you for something an entirely different company did to them. And also they are directing you away from the official website every single time you interact with them. I'm not sure what grounds you issue arrest warrants on, but I appreciate the sentiment.
j-bos 3y ago
Where can one find a breakdown of how to build implement a TOTP generator? For curiosity's sake
- deathanatos 3y ago
  The basic premise is in https://datatracker.ietf.org/doc/html/rfc6238 https://datatracker.ietf.org/doc/html/rfc6238, although today I'd use SHA-256, not SHA-1, if possible. But I'd disfavor TOTP over hardware tokens that can sign explicit requests.
tamimio 3y ago
You know how I never get phished? I never answer any call or sms asking anything, and a link in a text message is ALWAYS a major red flag. I know everyone is talking about the MFA, but the entry point was the employees phone numbers, how they got that in the first place? Especially from the article the attacker knew the internals of this company.. As for the MFA, google should have the on demand peer-peer sync rather than cloud save, for example, a new device is added, then your Google account is used to link between these new device and existing device, click sync and you will be asked on your old device that a new device is requesting bla bla would you allow it? And obviously nothing saved in the cloud, just a peer-peer sync and google is a connection broker.
boblob-law 3y ago
Am I the only one questioning the deep fake of the voice?
- Izkata 3y ago
  If they have an audio recording of the person, there's a bunch of sites where you can create them on the fly for free. Don't know about the quality, though I imagine distortion can be dismissed as being from the phone rather than the fake.
  - SoftTalker 3y ago
    Yeah, just add some twangy dropouts like a poor cell connection has.
ocdtrekkie 3y ago
The only takeaways you need from this: - Your on-premise customers are the smart ones. Networks containing sensitive information should be isolated, not all pooled together. - Google still has actually no understanding of practical security. Literally ban their products from your networks.
u801e 3y ago
Unfortunately, MFA has become synonymous with SMS, email, and OTP. All of these methods require sharing a secret between two parties without any way to verify the authenticity of either party. Key based authentication where both parties have private keys that are not shared is a much better alternative. Unfortunately, client side TLS certificates, which are application level protocol agnostic, never really caught on.
- est31 3y ago
  There is U2F/FIDO keys / passkeys which are what you describe, latter just very recently becoming widely available. When/if they become successful is another question. U2F/FIDO etc keys are only supported by a subset of websites.
  - mooreds 3y ago
    > U2F/FIDO etc keys are only supported by a subset of websites. But a growing number. https://passkeys.directory/ https://passkeys.directory/ is a good place to check. Ask for it. MFA via SMS/email/etc was not very common 10 years ago, but it is now. That's due in part to people asking for it.
  - u801e 3y ago
    But they're not application level protocol agnostic. Based on my understanding, they require use of HTTP. If I want to get MFA using an email client communicating via SMTP and IMAP, then the email client needs to be able to interact with the HTTP API.
    - acdha 3y ago
      You can use FIDO tokens for other protocols: I use it for SSH, for example since OpenSSH 8.2 or so.
      - u801e 3y ago
        That requires the client to implement FIDO support. This was added to openssh 8.2p1. For example, mutt doesn't have FIDO support and you have to use an external script for oauth2 support. Both require implementing support for interacting with a HTTP API (which is not application level protocol agnostic). On the other hand, you can configure mutt to use a client side TLS certificate and SMTP servers (e.g., postfix) and IMAP servers (e.g. dovecot) both support client side TLS certificates without having to support sending HTTP requests or parsing HTTP responses.
fn-mote 3y ago
After reading all of the hype in the comments, I was disappointed by the actual article. There's about one paragraph of actual material about the ("spear") phishing attack. There are not any details about the progress of the attackers or the speed of the attack, which would have been interesting to me. There are no details about any losses from the attack (or profits to the attacker). Once the employee provided a TOTP code to the attacker, the only surprise is that they get control of the other codes by cloud sync (as extensively commented on here). Regardless of the hate, this could happen to anyone. But... big L for reading out your TOTP code to somebody. (If more details about the deepfake come out, then it might be more exciting.)
crabbone 3y ago
MFA is a scam resulting from Google first, and then others wanting to get users' phone numbers associated with more data they collect on them. It provides no tangible security benefits, creates a lot of headache for IT department, creates big gaps in developer's productivity (if used in a programming company) and, actually, creates a new attack vector (phones are lost or stolen a lot more often than any other means of authentication). Since Github now requires MFA, I'm throwing away my account: I'll never give them any physical evidence to connect me to other data they have on me. In the company I work today (20-something thousands employees) the latest security breach was through MFA. Data was stolen. Perpetrators made jokes in company's Slack etc. Last time I had to upgrade my phone (while working for the same company), it took IT about two weeks to give me all the necessary access again, which required a lot of phone calls, video conferences, including my boss and my boss' boss. It's mind-boggling that this practice became the norm and is recommended by IT departments even of companies who have nothing to gain from collecting such data.
- Terretta 3y ago
  I don't understand: > Google first, and then others wanting to get users' phone numbers associated with more data they collect on them Perhaps you mean SMS 2FA, instead of a non phone number related MFA such as T-OTP?
  - crabbone 3y ago
    Google were sued because they were selling to advertisers information about Android users that other advertising platforms couldn't possibly had. Advertisement data s.a. user preferences, their history of clicking on ads, browsing history etc would all be organized by the id derived from Android device. Once the court decided they cannot do that, and users should opt in to be tracked, they promptly created MFA that relied on collecting data about physical devices. Which they then again used to sell advertisement data. The whole point of this exercise is not to enhance security, but to have an edge as an advertisement platform. If today you can trick the system into not using a phone, it's a temporary thing. The more users join, the tighter will be the system's grip on each individual user, and the "privilege" of not divulging your phone number will be taken away. Google did this before with e-mail access for example, multiple times, actually. Remember how GoogleTalk used Jabber? -- Not having to use a proprietary chat protocol was a feature that made more users join. As soon as there were enough users, they replaced GoogleTalk with Hangouts or w/e it's called. GMail used to provide standard SMTP / IMAP access, but they continuously undermined all clients other than Google's. Started with removing POP access. Then requiring mandatory TLS. Then requiring a bunch of nonsense "trusted application registration". Finally, this feature is now behind MFA, which makes it useless anywhere outside Google's Web client / Android app etc. All of this was delivered as a "security improvements", while giving no tangible security benefits. It was a move to undermine competition.
tptacek 3y ago
We use OTPs extensively at Retool: it’s how we authenticate into Google and Okta, how we authenticate into our internal VPN, and how we authenticate into our own internal instances of Retool They should stop using OTPs. OTPs are obsolete. For the past decade, the industry has been migrating from OTPs to phishing-proof authenticators: U2F, then WebAuthn, and now Passkeys†. The entire motivation for these new 2FA schemes is that OTPs are susceptible to phishing, and it is practically impossible to prevent phishing attacks with real user populations, even (as Google discovered with internal studies) with ultra-technical user bases. TOTP is dead. SMS is whatever "past dead" is. Whatever your system of record is for authentication (Okta, Google, what have you), it needs to require phishing-resistant authentication. I'm not high-horsing this; until recently, it would have been complicated to do something other than TOTP with our service as well (though not internally). My only concern is the present tense in this post about OTPs, and the diagnosis of the problem this post reached. The problem here isn't software custody of secrets. It's authenticators that only authenticate one way, from the user to the service. That's the problem hardware keys fixed, and you can fix that same problem in software. † (All three are closely related, and an investment you made in U2F in 2014 would still be paying off today.)
- snagg 3y ago
  Worth noting that implementing FIDO2/Passkeys is more challenging than it looks both from a UX standpoint and from a threat modeling standpoint. We tried to cover some of this in a blog post, in case anybody is interested: https://www.slashid.dev/blog/passkeys-security-implementation/ https://www.slashid.dev/blog/passkeys-security-implementatio...
- drx 3y ago
  What would be your recommendation for replacing TOTP today?
  - akerl_ 3y ago
    FIDO2
- corford 3y ago
  For others new to WebAuthn and Passkeys (like me), worth noting that Passkeys come with important privacy/ease-of-use trade-offs (nice summary here: https://blog.passwordless.id/webauthn-vs-passkeys https://blog.passwordless.id/webauthn-vs-passkeys) Less of an issue though once more non-platform vendors start supporting them (e.g. Bitwarden https://bitwarden.com/passwordless-passkeys/ https://bitwarden.com/passwordless-passkeys/)
out-of-ideas 3y ago
> The caller claimed to be one of the members of the IT team, and deepfaked our employee’s actual voice. The voice was familiar with the floor plan of the office, coworkers, and internal processes of the company. huh.. this raises way more questions than it answers; my first two are: - how did the voice of some random employee (in IT for that matter) get learned by outside the company (enough to be deepfaked (and i presume on the fly) for that matter)? Maybe we should record less conversations (looks at Teams, Discord, Zoom) - where there already leaks of 'internal processes'?
drx 3y ago
Excellent write-up, thank you.
roamerz 3y ago
I was thinking about this the other night. Is there really a solution to this? Best case scenario lets say you have a hardware key and everything is sealed up really well. You get your phishing call but instead of asking for a MFA code they have a real time IA enhanced video call from your daughter or mom with a gun to her head and they just walk you through a set of steps that will expose your IT systems. Do you do as they demand with a loved one’s life at stake? Or maybe it’s a scam? What do you do? You have 5 seconds to decide. Me? I go John Wick on them but I’ve had more than 5 seconds so that doesn’t count.
lionkor 3y ago
MFA still means single point of failure - the person who has all the MFA is the one who can be hacked, like in this social engineering scenario.
Thorrez 3y ago
I don't think it's really accurate to describe it as not MFA. The attacker phished a password and 2 TOTP codes. So the attacker phished 3FA. So yes, Google Authenticator sync made the security worse, but it didn't downgrade the security from MFA to non-MFA. And even if the sync was off, the TOTP codes in Google Authenticator could have been phished as well, so Google Authenticator can't be blamed so heavily, because the attack could have been done without it. Disclosure: I work at Google but not on Google Authenticator.
j0057 3y ago
TOTP: better than nothing, but not a lot better. It depends on the human being infallible. U2F doesn't, and would have worked here to prevent the takeover and the need for blaming the employee for not being infallible, but hey, at least less than $50 of hardware costs was saved by the employer!
davinci123 3y ago
This is the biggest reason why MFA codes shouldn't be in the cloud. Use SMS-based MFA which is much more fool-proof though a pain in the ass. I have stopped using software based MFAs for this particular reason.

393 points by dvdhsu 3y ago

I don't understand: Why on earth does google want to sync MFA tokens? They're one-time use, aren't they? Or... feh, I can't even fathom

kerblang 3y ago
Answering myself, this helps a bit: https://www.zdnet.com/article/google-authenticator-will-now-sync-your-2fa-codes-to-use-on-different-devices/ https://www.zdnet.com/article/google-authenticator-will-now-... I guess we need a better way to handle "Old phone went swimming, had to buy another, now what?"
- mbesto 3y ago
  Which is funny because the 2nd factor is "something I have", which means if you don't "have it" then you can ever complete the 2nd factor. This ultimately means the 2nd factor, when you're phone goes swimming, is ultimately your printed codes.
jeremyjh 3y ago
They mean they are syncing the private key used to generate the tokens on demand.
- magospietato 3y ago
  Well that's even worse isn't it?
- kerblang 3y ago
  Do all these 2FA apps - like say Microsoft Authenticator - have these hidden/not-so-hidden private keys? From other posts it sounds like you can view the token and write it down... MA doesn't have that, I don't think.
  - kerblang 3y ago
    Answering myself again, yeah, they all seem to have this private key hidden away somewhere. Didn't know that. https://frontegg.com/blog/authentication-apps#How-Do-Authenticator-Apps-Work https://frontegg.com/blog/authentication-apps#How-Do-Authent...?
  - e12e 3y ago
    TOTP (Time-based one-time password) need a shared secret (and two synchronized clocks) to work, so yes. FIDO2/WebAuthn relies on public key technology - so does also have a secret key - but is designed to be kept secret from the service/server one authenticates against. For use - FIDO2 is more like a multi-use id. Like a driver's license many services accept as id. If you lose it - you don't restore a backup copy from a safe - you use your passport until you get a new one issued. This makes more sense than with TOTP as the services only need your public key(id) on file. https://en.wikipedia.org/wiki/Time-based_One-time_Password https://en.wikipedia.org/wiki/Time-based_One-time_Password https://en.m.wikipedia.org/wiki/WebAuthn https://en.m.wikipedia.org/wiki/WebAuthn
    - rawgabbit 3y ago
      Which FIDO2 service do you recommend? I get tired reading all these security articles. The more I read, the more I feel they are hiding something.
      - e12e 3y ago
        > Which FIDO2 service do you recommend? Generally what comes with your phone and one or two hw tokens for backup? Looks like token2.com is a reasonable choice if you just want NFC/USBc and FIDO2 (and not storage for ssh/gpg keys). But I have little experience with hw keys.
        drhuseynov 3y ago
        ssh and pgp keys are not based on the similar functionality. the keys from Token2 support *-sk key storage https://www.token2.com/site/page/using-token2-fido2-security-keys-for-openssh https://www.token2.com/site/page/using-token2-fido2-security... But not PGP
unethical_ban 3y ago
Syncing of "MFA codes" is really syncing of the secret component of TOTP (time based one time password). And it's a good thing, and damn any 2fa solution that blocks it. I don't want to go through onerous, incompetent, poorly designed account recovery procedures if a toddler smashes my phone. So I use authy personally, while a friend backs his up locally.
- skybrian 3y ago
  A better way to fix this is to have multiple ways to log in. Printed backup codes in your safe with your personal papers and/or a Yubikey on your keychain. This works for Google and Github, at least. Passkey syncing is more convenient, though, and probably an improvement on what most people do.
- itake 3y ago
  > I don't want to go through onerous, incompetent, poorly designed account recovery procedures if a toddler smashes my phone Why don't you use the printed recovery tokens?
  - monocasa 3y ago
    Who has a printer these days?
    - CameronNemo 3y ago
      Local libraries, print shops... but yeah that may be an attack vector.
  - unethical_ban 3y ago
    Not all websites offer them. Hell, no bank I use (several large and several regional) support generic totp. Some have sms, one has Symantec VIP, proprietary and not redundant. Edit: since I'm posting too fast according to HN, even though I haven't posted in an hour, I'll say it here. Symantec is totp but You cannot back up your secrets and you cannot have backup codes.
AshamedCaptain 3y ago
For me the question is "who the fsck uses Google Authenticator to store all their tokens, both company and personal?"
- burkaman 3y ago
  Google Authenticator was I believe the first available TOTP app, and is by far the most popular. It used to be open source and have no connection to your Google account. Many people installed it years ago when they first set up MFA, and have just been adding stuff to it ever since because it's easy and it works. Even for technical users who understand how TOTP works, there is no obvious reason it appears unsafe to put all your tokens in the app (until you read this article). Look at the MFA help page for any website you use. One of the first sentences is probably something like "First you'll need to install a TOTP app on your phone, such as Google Authenticator or Authy..." It really did used to be the best option. For example, see this comment from 10 years ago when Authy first launched: > The Google Authenticator app is great. I recently got (TOTP) 2-factor auth for an IRC bot going with Google Authenticator; took about 5 minutes to code it up and set it up. It doesn't use any sort of 3rd party service, just the application running locally on my phone. TOTP/HOTP is dead simple and, with the open source Google Authenticator app, great for the end user. - https://news.ycombinator.com/item?id=6137051 https://news.ycombinator.com/item?id=6137051
  - fireflash38 3y ago
    I think technically Blizzard Authenticator (even the app) was available before Google Authenticator, but obviously for extremely limited use.
  - glandium 3y ago
    Also, since it doesn't allow to extract the private keys, you're kind of stuck with it once you've started using it.

bawolff 3y ago

While the google cloud thing is a weird design, that seems like the wrong place to blame. TOTP and SMS based 2FA are NOT designed to prevent phishing. If you care about phishing use yubikeys.

consoomer 3y ago

1. install pass otp 2. pass otp add whatever/otp/me 3. paste in "otpauth://totp/whatever?secret=whateveritis 4. pass git init; push to remote Now you you have MFA on any device that has git and your gpg key.

rahidz 3y ago

tough 3y ago
inside job?
- mistrial9 3y ago
  how's that Zero Trust architecture working out for everyone ?
  - wepple 3y ago
    What’s this got to do with zero trust?
    - mistrial9 3y ago
      it is a cynical comment that is meant to hilite the relationship between humans where oppressive and untrusting employment leads to increase in antipathy, ill-will, feelings of being abused and all of that leading to insider theft and serious pre-meditated betrayal ?
      - wyldberry 3y ago
        Zero Trust is such a bad branding for how the architecture works. It's just "always prove" architecture.
        tough 3y ago
        It does seem to sound pretty well on the mind of the executives signing the deals that hear the marketing talk
        3y ago

pyrolistical 3y ago

Naming/training issue imo. We need a better name than MFA. Something like “personal password like token that should only be entered into secure computer on specific website/app/field and never needed to be shared”

mr_mitm 3y ago
It's well known that OTP is not immune to phishing. Force your users on webauthn or some other public key based second factor if you're aiming at decreasing the incident rate.
- unethical_ban 3y ago
  Well, push based 2fa with "select this number on your 2fa device" helps prevent some vectors. Simple totp doesn't do that. "Never give your totp or one time code over the phone" is good advice. "Never give info to someone who called you, call them back on the official number" is another. This is user error at this point.
  - AshamedCaptain 3y ago
    I disagree. Specially again that companies are centralizing on a couple 2FA companies (like Okta from TFA), this is just ripe for phishing. Okta itself is terrible at this; they don't consistently use the okta.com domain, so users are at a loss and have basically no protection against impersonators.
    - unethical_ban 3y ago
      For okta, if it is set up properly, the user should get push notifications. And in that push notification is a number they need to select to validate the push. This eliminates credential phishing and "notification exhaustion" where a user just clicks "ok" on an auth request by a bad actor. As much as I advocate for non cloud services, what okta provides is very secure.
- jpc0 3y ago

wepple 3y ago

Why did they need to call? They could’ve phished the password and MFA by simply MITMing? Perhaps we need a distinction from phishable MFA and unphishable U2F/WebAuthn style

rsstack 3y ago
> The caller claimed to be one of the members of the IT team, and deepfaked our employee’s actual voice. The voice was familiar with the floor plan of the office, coworkers, and internal processes of the company. Throughout the conversation, the employee grew more and more suspicious, but unfortunately did provide the attacker one additional multi-factor authentication (MFA) code. > The additional OTP token shared over the call was critical, because it allowed the attacker to add their own personal device to the employee’s Okta account, which allowed them to produce their own Okta MFA from that point forward. They needed to have a couple of minutes to set things up from their end, and then ask for the second OTP code. A phone call works well for that.
- wepple 3y ago
  Ahh, thanks and apologies for not re-reading before asking. That is indeed interesting; keep the con going a bit longer to get a proper foothold.

rolobio 3y ago

Very sophisticated attack, I would bet most people would fall for this. I'm surprised Google encourages syncing the codes to the cloud... kind of defeats the purpose. I sync my TOTP between devices using an encrypted backup, even if someone got that file they could not use the codes. FIDO2 would go a long way to help with this issue. There is no code to share over the phone. FIDO2 can also detect the domain making the request, and will not provide the correct code even it the page looks correct to a human.

victor106 3y ago
They could’ve just had employees use Okta Verify as opposed to Google Authenticator
softfalcon 3y ago
>FIDO2 can also detect the domain making the request, and will not provide the correct code even it the page looks correct to a human. I could not agree more with this sentiment! We need more of this kind of automated checking going on for users. I'm tired of seeing "just check for typo's in the URL" or "make sure it's the real site!" advice given to the average user. People are not able to do this even when they know how to protect themselves. Humans tire easily and are often fallible. We need more tooling like FIDO2 to automate away this problem for us. I hope the adoption of it will go smoothly in years to come.
- miki123211 3y ago
  The problem with Fido (and other such solutions, including smartphone-based passkeys) is that they make things extremely hard if you're poor / homeless / in an unsafe / violent family situation and therefore change devices often. It's mostly a non-issue for Silicon Valley tech employees working solely on their corporate laptops, and U2F is perfect for that use-case, but these concerns make MFA a non-starter for the wider population. We could neatly sidestep all of these issues with cloud-based fingerprint readers, but the privacy advocates won't ever let that happen.
  - 3y ago

AYBABTME 3y ago

To deepfake the voice of an actual employee, they would need enough recorded content of that employee's voice... and I would think someone doing admin things on their platform isn't also in DevRel with a lot of their voice uploaded online for anyone to use. So it smells like someone with close physical proximity to the company would be involved.

gabereiser 3y ago
There’s a lot of ways to get clips of recordings of someone’s voice. You can get that if they ever spoke at a conference or on a video. Numerous other ways I won’t list here.
V__ 3y ago
One possibility would be to just call the employee and record their voice. One could pretend to be a headhunter.
- skeaker 3y ago
  This would almost certainly be it. Calling someone to record them and using their voice later to impersonate them was done even before deep-fake voices were a concept. With the tools available now, even a short call + the grainy connection of a phone voice line would be more than enough to make a simulated voice work.
- rlt 3y ago
  I'm already cautious about answering calls from unknown numbers. This could be a good reason to be even more cautious.
themagician 3y ago
Probably wasn't a "deepfake" just someone decent with impressions and a $99 mixer. After compression this will be more than good enough to fool just about anyone. No deepfake is needed. Just call the person once and record a 30 second phone call. Tell them you are delivering some package and need them to confirm their address.

brunojppb 3y ago

Fantastic write-up. Major props for disclosing the details of the attack in a very accessible way. It is great that this kind of security incident post-mortem is being shared. This will help the community to level-up in many ways, specially given that its content is super accessible and not heavily leaning on tech jargon.

hn_throwaway_99 3y ago
I disagree. I appreciate the level of detail, but I don't appreciate Retool trying to shift the blame to Google, and only putting a blurb in the end about using FIDO2. They should have been using hardware keys years ago.
- duderific 3y ago
  It was also a bit weird how they kept emphasizing how their on-prem installations were not affected, as if that lessens the severity somehow. It's like duh, that's the whole point of on-prem deployments.
- dvdhsu 3y ago
  Hi, I'm sorry you felt that way. "Shifting blame to Google" is absolutely not our intention, and if you have any recommendations on how to make the blog post more clear, please do let me know. (We're happy to change it so it reads less like that.) I do agree that we should start using hardware keys (which we started last week). The goal of this blog post was to make clear to others that Google Authenticator (through the default onboarding flow) syncs MFA codes to the cloud. This is unexpected (hence the title, "When MFA isn't MFA"), and something we think more people should be aware of.
  - hn_throwaway_99 3y ago
    I felt like you were trying to shift blame to Google due to the title "When MFA isn't MFA" and your emphasis on "dark patterns" which, to be honest, I don't think they are that "dark". To me it was because this felt like a mix of a post mortem/apology, but with some "But if it weren't for Google's dang dark patterns..." excuse thrown in. FWIW, nearly every TOTP authenticator app I'm aware of supports some type of seed backup (e.g. Authy has a separate "backup password"). I actually like Google's solution here as long as the Workspace accounts are protected with a hardware key. The only real lesson here is that you should have been using hardware keys.

batmansmk 3y ago

Are the claims of deepfake and intimate knowledge of procedures based of the sole testimony of the employee who oopsed terribly? This is a novelisation of an events Retool needs to revise the basic security posture. There is no point in complicated technology if the warden just gives the key away.

hn_throwaway_99 3y ago
> Retool needs to revise the basic security posture. Couldn't agree more. TBH I thought this post was an exercise in blame shifting, trying to blame Google. > We use OTPs extensively at Retool: it’s how we authenticate into Google and Okta, how we authenticate into our internal VPN, and how we authenticate into our own internal instances of Retool. The fact that access to a Google account immediately gave access to all MFA tokens held within that account is the major reason why the attacker was able to get into our internal systems. Google Workspace makes it very easy to set up "Advanced Protection" on accounts, in which case it requires using a hardware key as a second factor, instead of a phishable security code. Given Retool's business of hosting admin apps for lots of other companies, they should have known they'd be a prime target for something like this, and not requiring hardware keys is pretty inexcusable here.
- dotty- 3y ago
  > Google Workspace makes it very easy to set up "Advanced Protection" on accounts, in which case it requires using a hardware key as a second factor, instead of a phishable security code. This isn't immediately actionable for every company. I agree Retool should have hardware keys given their business, but at my company with 170 users we just haven't gotten around to figuring out the distribution and adoption of hardware keys internationally. We're also a Google Workspace customer. I think it's stupid for a company like Google, the company designing these widely used security apps for millions of users, to allow for cloud syncing without allowing administrators the ability to simply turn off the feature on a managed account. Google Workspace actually lacks a lot of granular security features, something I wish they did better. What is a company like mine meant to do here to counter this problem? edit: changed "viable" for "immediately actionable". It's easy for Google to change their apps. Not for every company to change their practices.

RcouF1uZ4gsC 3y ago

One thing that is left out it to use unphishable MFA like hardware security keys (Yubikey, etc).

xorcist 3y ago

Stopped reading at "deepfake". It's the new advanced persistent threat, a perfect phrase to divert any resposibility. (Yes, there are deepfakes. Yes, there are APTs. This is likely neither.)

skeaker 3y ago
I am (genuinely, really asking) curious why you think so. I've got no hand in this, but the skepticism around phishing attacks from this site of all places really surprises me. People like Kevin Mitnick have done more sophisticated phishing with fewer tools. Why wouldn't someone intent on running a social engineering scam use one of the widely available voice faking technologies that are available now? Keep in mind that they're simple enough to use that people are making memes with voices generated from ~5 seconds of voice recordings.
- xorcist 3y ago
  Making a meme is nothing like an interactive telephone conversation. It's not that it's impossible, but it's not trivial either. But mainly, it's just unnecessary. If the user is not fooled by a well crafted phishing, by doing the most trivial countermeasures such as calling back, they are not going to be fooled by a deepfake. In practice work on phishing is mostly better spent elsewhere. So while we shouldn't dismiss it completely, it's clearly not the case with a smallish company with limited economic value, so very unlikely the case here. There has been a handful of highly profile media cases involving deepfake. None of which has held up on further investigation. It is understandable, nobody wants to be known as the one who didn't recognize his own kid on the phone, but the truth is more simple and actually helps us when designing countermeasures.
  - skeaker 3y ago
    I suppose the disconnect then would be that we fundamentally disagree on what the simpler answer is. It's my understanding that a deepfake voice being used as part of a phishing scam is something that can be done trivially (or at least by a determined actor using free tools, so at least trivial enough for this case), so to me that would be the simplest, most obvious answer when compared to a company-wide conspiracy, but I can see your point if it is assumed that that isn't the case and that deepfake voices are actually hard to do.

account-5 3y ago

I wonder how long it'll be before a similar attack happens before someone's/a companies passkeys are synced to the cloud.

hn_throwaway_99 3y ago

Question for security folks out there: So often I see these kinds of phishing attacks that have hugely negative consequences (see the MGM Resorts post earlier today), and the main problem is that just one relatively junior employee who falls for a targeted phishing attack can bring down the whole system. Is anyone aware of systems that essentially require multiple logins from different users when accessing sensitive systems like internal admin tools? I'm thinking like the "turn the two keys simultaneously to launch the missile" systems. I'm thinking it would work like the following: 1. If a system detects a user is logging into a particularly sensitive area (e.g. a secrets store), and the user is from a new device, the user first needs to log in using their creds (including any appropriate MFA). 2. In addition, another user like an admin would need to log in simultaneously and approve this access from a new device. Otherwise, the access would be denied. I've never seen a system like this in production, and I'm curious why it isn't more prevalent when I think it should be the default for accessing highly sensitive apps in a corporate environment.

joshxyz 3y ago
i wonder on this too if people really use shamir secret sharing as part of some security compliance
deleted 3y ago
[deleted]
fireflash38 3y ago
You're looking for quorums, or key splits. They aren't super common. You see them with some HSMs (need M of N persons to perform X action).
- joshxyz 3y ago
  not good with acronyms, what is hsm here?
  - coderintherye 3y ago
    Hardware security module https://en.wikipedia.org/wiki/Hardware_security_module https://en.wikipedia.org/wiki/Hardware_security_module

miki123211 3y ago

does iOS have a "is there a call in progress" API? If so, it would be a good idea for OTP apps to use it and display a prominent warning banner when opened during a call.

alsodumb 3y ago

Maybe it’s just me, but I am really skeptical about the DeepFake part - it’s a theoretically possible attack vector, but the only evidence they possibly could have to support this statement would be the employees testimony. Targeting a particular employee with the voice of a specific person this employee knows requires a lot of information and insider info. Also, I think the article spends a lot of effort trying to blame Google Authenticator and make it seems like they had the best possible defense and yet attackers managed to get through because of Googles error. Nope, not even close. They would have had hardware 2FA if they were really concerned about security. Come on guys, it’s 2023 and hardware tokens are cheap. It’s not even a consumer product where one can say that hardware tokens hinder usability. It’s a finite set of employees, who need to do MFA certain times for certain services mostly using one device. Just start using hardware keys.

dvdhsu 3y ago
Hi, David, founder @ Retool here. We are currently working with law enforcement, and we believe they have corroborating evidence through audio that suggests a deepfake is likely. (Put another way, law enforcement has more evidence than just the employee's testimony.) (I wish we could blog about this one day... maybe in a few decades, hah. Learning more about the government's surveillance capabilities has been interesting.) I agree with you on hardware 2FA tokens. We've since ordered them and will start mandating them. The purpose of this blog post is to communicate that what is traditionally considered 2FA isn't actually 2FA if you follow the default Google flow. We're certainly not making any claims that "we are the world's most secure company"; we are just making the claim that "what appears to be MFA isn't always MFA". (I may have to delete this comment in a bit...)
- alsodumb 3y ago
  Thanks for the reply! What's expecting one. Since you might have you delete the reply anyway, can I get a candid answer on why hardware 2FA tokens weren't a part of the default workflow before the incident? Was it concerns about the cost, the recovery modes, or was it just the trust in the existing approach?
- 3y ago

yieldcrv 3y ago

I just call them one-time passcodes (otp) Most of the time I am not using multifactor or 2factor the way it was designed But it is accurately a one time passcode

andrewstuart 3y ago

Some startup, please make a product that uses AI to identify these obviously fake emails. Hello A, This is B. I was trying to reach out in regards to your [payroll system] being out of sync, which we need synced for Open Enrollment, but i wasn’t able to get ahold of you. Please let me know if you have a minute. Thanks You can also just visit https://retool.okta.com.[oauthv2.app]/authorize-client/xxx https://retool.okta.com.[oauthv2.app]/authorize-client/xxx and I can double check on my end if it went through. Thanks in advance and have a good night A.

deleted 3y ago
[deleted]

macNchz 3y ago

Beyond having hardware keys, this scenario is why I really try to drive home, in all of my security trainings, the idea that you should instantly short circuit any situation where you receive a phone call (or other message) and someone starts asking for information. It's always okay to say, "actually, let me get back to you in a minute" and hang up, calling back on a known phone number from the employee directory, or communicate on different channel altogether. Organizationally, everyone should be prepared for and encourage that kind of response as well, such that employees are never scared to say it because they're worried about a snarky/angry/aggressive response. This also applies to non-work related calls: someone from your credit card company is calling and asking for something? Call back on the number on the back of your card.

hinkley 3y ago
Advice I haven't even followed myself: It's probably a good idea to program your bank's fraud number into your phone. The odds that someone hacks your bank's Contact Us page are small but not zero. The bedrock of both PGP and .ssh/known_hosts could be restated as, "get information before anyone knows you need it". Fraud departments contacting me about potentially fraudulent charges is always going to make me upset. Jury is still out on whether it will always trigger a rant, but the prognosis is not good.
- GauntletWizard 3y ago
  At least once I have gotten a terribly phrased and link-strewn "Fraud Alert" from a bank, reported it to said bank's anti-phisihing e-mail address, gotten a personalized mail that responded that it was in fact fraud and that they had policies against using third party subdomains like... And then found out the day later that yes, that was their real new anti-fraud tool and template. There will need to be jail time for the idiots writing the government standards on these fraud departments before we get jail time for the idiots running these fraud departments before it gets better.
  - hinkley 3y ago
    Last time I talked to someone about this they pointed out that fraud depts are often outsourced. Which is a lovely plan because now your customers hate you for something an entirely different company did to them. And also they are directing you away from the official website every single time you interact with them. I'm not sure what grounds you issue arrest warrants on, but I appreciate the sentiment.

j-bos 3y ago

Where can one find a breakdown of how to build implement a TOTP generator? For curiosity's sake

deathanatos 3y ago
The basic premise is in https://datatracker.ietf.org/doc/html/rfc6238 https://datatracker.ietf.org/doc/html/rfc6238, although today I'd use SHA-256, not SHA-1, if possible. But I'd disfavor TOTP over hardware tokens that can sign explicit requests.

tamimio 3y ago

You know how I never get phished? I never answer any call or sms asking anything, and a link in a text message is ALWAYS a major red flag. I know everyone is talking about the MFA, but the entry point was the employees phone numbers, how they got that in the first place? Especially from the article the attacker knew the internals of this company.. As for the MFA, google should have the on demand peer-peer sync rather than cloud save, for example, a new device is added, then your Google account is used to link between these new device and existing device, click sync and you will be asked on your old device that a new device is requesting bla bla would you allow it? And obviously nothing saved in the cloud, just a peer-peer sync and google is a connection broker.

boblob-law 3y ago

Am I the only one questioning the deep fake of the voice?

Izkata 3y ago
If they have an audio recording of the person, there's a bunch of sites where you can create them on the fly for free. Don't know about the quality, though I imagine distortion can be dismissed as being from the phone rather than the fake.
- SoftTalker 3y ago
  Yeah, just add some twangy dropouts like a poor cell connection has.

ocdtrekkie 3y ago

The only takeaways you need from this: - Your on-premise customers are the smart ones. Networks containing sensitive information should be isolated, not all pooled together. - Google still has actually no understanding of practical security. Literally ban their products from your networks.

u801e 3y ago

Unfortunately, MFA has become synonymous with SMS, email, and OTP. All of these methods require sharing a secret between two parties without any way to verify the authenticity of either party. Key based authentication where both parties have private keys that are not shared is a much better alternative. Unfortunately, client side TLS certificates, which are application level protocol agnostic, never really caught on.

est31 3y ago
There is U2F/FIDO keys / passkeys which are what you describe, latter just very recently becoming widely available. When/if they become successful is another question. U2F/FIDO etc keys are only supported by a subset of websites.
- mooreds 3y ago
  > U2F/FIDO etc keys are only supported by a subset of websites. But a growing number. https://passkeys.directory/ https://passkeys.directory/ is a good place to check. Ask for it. MFA via SMS/email/etc was not very common 10 years ago, but it is now. That's due in part to people asking for it.
- u801e 3y ago
  But they're not application level protocol agnostic. Based on my understanding, they require use of HTTP. If I want to get MFA using an email client communicating via SMTP and IMAP, then the email client needs to be able to interact with the HTTP API.
  - acdha 3y ago
    You can use FIDO tokens for other protocols: I use it for SSH, for example since OpenSSH 8.2 or so.
    - u801e 3y ago
      That requires the client to implement FIDO support. This was added to openssh 8.2p1. For example, mutt doesn't have FIDO support and you have to use an external script for oauth2 support. Both require implementing support for interacting with a HTTP API (which is not application level protocol agnostic). On the other hand, you can configure mutt to use a client side TLS certificate and SMTP servers (e.g., postfix) and IMAP servers (e.g. dovecot) both support client side TLS certificates without having to support sending HTTP requests or parsing HTTP responses.

fn-mote 3y ago

After reading all of the hype in the comments, I was disappointed by the actual article. There's about one paragraph of actual material about the ("spear") phishing attack. There are not any details about the progress of the attackers or the speed of the attack, which would have been interesting to me. There are no details about any losses from the attack (or profits to the attacker). Once the employee provided a TOTP code to the attacker, the only surprise is that they get control of the other codes by cloud sync (as extensively commented on here). Regardless of the hate, this could happen to anyone. But... big L for reading out your TOTP code to somebody. (If more details about the deepfake come out, then it might be more exciting.)

crabbone 3y ago

MFA is a scam resulting from Google first, and then others wanting to get users' phone numbers associated with more data they collect on them. It provides no tangible security benefits, creates a lot of headache for IT department, creates big gaps in developer's productivity (if used in a programming company) and, actually, creates a new attack vector (phones are lost or stolen a lot more often than any other means of authentication). Since Github now requires MFA, I'm throwing away my account: I'll never give them any physical evidence to connect me to other data they have on me. In the company I work today (20-something thousands employees) the latest security breach was through MFA. Data was stolen. Perpetrators made jokes in company's Slack etc. Last time I had to upgrade my phone (while working for the same company), it took IT about two weeks to give me all the necessary access again, which required a lot of phone calls, video conferences, including my boss and my boss' boss. It's mind-boggling that this practice became the norm and is recommended by IT departments even of companies who have nothing to gain from collecting such data.

Terretta 3y ago
I don't understand: > Google first, and then others wanting to get users' phone numbers associated with more data they collect on them Perhaps you mean SMS 2FA, instead of a non phone number related MFA such as T-OTP?
- crabbone 3y ago
  Google were sued because they were selling to advertisers information about Android users that other advertising platforms couldn't possibly had. Advertisement data s.a. user preferences, their history of clicking on ads, browsing history etc would all be organized by the id derived from Android device. Once the court decided they cannot do that, and users should opt in to be tracked, they promptly created MFA that relied on collecting data about physical devices. Which they then again used to sell advertisement data. The whole point of this exercise is not to enhance security, but to have an edge as an advertisement platform. If today you can trick the system into not using a phone, it's a temporary thing. The more users join, the tighter will be the system's grip on each individual user, and the "privilege" of not divulging your phone number will be taken away. Google did this before with e-mail access for example, multiple times, actually. Remember how GoogleTalk used Jabber? -- Not having to use a proprietary chat protocol was a feature that made more users join. As soon as there were enough users, they replaced GoogleTalk with Hangouts or w/e it's called. GMail used to provide standard SMTP / IMAP access, but they continuously undermined all clients other than Google's. Started with removing POP access. Then requiring mandatory TLS. Then requiring a bunch of nonsense "trusted application registration". Finally, this feature is now behind MFA, which makes it useless anywhere outside Google's Web client / Android app etc. All of this was delivered as a "security improvements", while giving no tangible security benefits. It was a move to undermine competition.

tptacek 3y ago

We use OTPs extensively at Retool: it’s how we authenticate into Google and Okta, how we authenticate into our internal VPN, and how we authenticate into our own internal instances of Retool They should stop using OTPs. OTPs are obsolete. For the past decade, the industry has been migrating from OTPs to phishing-proof authenticators: U2F, then WebAuthn, and now Passkeys†. The entire motivation for these new 2FA schemes is that OTPs are susceptible to phishing, and it is practically impossible to prevent phishing attacks with real user populations, even (as Google discovered with internal studies) with ultra-technical user bases. TOTP is dead. SMS is whatever "past dead" is. Whatever your system of record is for authentication (Okta, Google, what have you), it needs to require phishing-resistant authentication. I'm not high-horsing this; until recently, it would have been complicated to do something other than TOTP with our service as well (though not internally). My only concern is the present tense in this post about OTPs, and the diagnosis of the problem this post reached. The problem here isn't software custody of secrets. It's authenticators that only authenticate one way, from the user to the service. That's the problem hardware keys fixed, and you can fix that same problem in software. † (All three are closely related, and an investment you made in U2F in 2014 would still be paying off today.)

snagg 3y ago
Worth noting that implementing FIDO2/Passkeys is more challenging than it looks both from a UX standpoint and from a threat modeling standpoint. We tried to cover some of this in a blog post, in case anybody is interested: https://www.slashid.dev/blog/passkeys-security-implementation/ https://www.slashid.dev/blog/passkeys-security-implementatio...
drx 3y ago
What would be your recommendation for replacing TOTP today?
- akerl_ 3y ago
  FIDO2
corford 3y ago
For others new to WebAuthn and Passkeys (like me), worth noting that Passkeys come with important privacy/ease-of-use trade-offs (nice summary here: https://blog.passwordless.id/webauthn-vs-passkeys https://blog.passwordless.id/webauthn-vs-passkeys) Less of an issue though once more non-platform vendors start supporting them (e.g. Bitwarden https://bitwarden.com/passwordless-passkeys/ https://bitwarden.com/passwordless-passkeys/)

out-of-ideas 3y ago

> The caller claimed to be one of the members of the IT team, and deepfaked our employee’s actual voice. The voice was familiar with the floor plan of the office, coworkers, and internal processes of the company. huh.. this raises way more questions than it answers; my first two are: - how did the voice of some random employee (in IT for that matter) get learned by outside the company (enough to be deepfaked (and i presume on the fly) for that matter)? Maybe we should record less conversations (looks at Teams, Discord, Zoom) - where there already leaks of 'internal processes'?

drx 3y ago

Excellent write-up, thank you.

roamerz 3y ago

I was thinking about this the other night. Is there really a solution to this? Best case scenario lets say you have a hardware key and everything is sealed up really well. You get your phishing call but instead of asking for a MFA code they have a real time IA enhanced video call from your daughter or mom with a gun to her head and they just walk you through a set of steps that will expose your IT systems. Do you do as they demand with a loved one’s life at stake? Or maybe it’s a scam? What do you do? You have 5 seconds to decide. Me? I go John Wick on them but I’ve had more than 5 seconds so that doesn’t count.

lionkor 3y ago

MFA still means single point of failure - the person who has all the MFA is the one who can be hacked, like in this social engineering scenario.

Thorrez 3y ago

I don't think it's really accurate to describe it as not MFA. The attacker phished a password and 2 TOTP codes. So the attacker phished 3FA. So yes, Google Authenticator sync made the security worse, but it didn't downgrade the security from MFA to non-MFA. And even if the sync was off, the TOTP codes in Google Authenticator could have been phished as well, so Google Authenticator can't be blamed so heavily, because the attack could have been done without it. Disclosure: I work at Google but not on Google Authenticator.

j0057 3y ago

TOTP: better than nothing, but not a lot better. It depends on the human being infallible. U2F doesn't, and would have worked here to prevent the takeover and the need for blaming the employee for not being infallible, but hey, at least less than $50 of hardware costs was saved by the employer!

davinci123 3y ago

This is the biggest reason why MFA codes shouldn't be in the cloud. Use SMS-based MFA which is much more fool-proof though a pain in the ass. I have stopped using software based MFAs for this particular reason.