6 ms·

Tip for Meta engineers: when your service is failing, don't just log people out and prevent logins. Display a cute image that shows that the service is drastica

Tip for Meta engineers: when your service is failing, don't just log people out and prevent logins. Display a cute image that shows that the service is drastically failing (like a whale or something), and then people will know to stop trying to repeatedly log in. The public might even come up with a catchy name for the whale.

bandwidth 3y ago
Surely the repeated logins can't be helping the situation. I suppose it is entirely auth related across all Meta products. The repeated strain could pose a cold start problem for example.
brk 3y ago
This is a great idea! I know that I was flailing about trying to figure out why I couldn't login, so I'd suggest calling it a "Flail Whale".
zabzonk 3y ago
> catchy name Petunia?
boring_twenties 3y ago
Beyond unbelievable that going on an hour later, they're still showing "incorrect password" errors. How many hundreds of millions of people have wasted time frantically trying (in vain) to reset their passwords and pointlessly freaking out that their account might be compromised? What a bunch of careless, incompetent excuses for engineers.
- alwyn 3y ago
  Blame the managers and product owners that don't think this is an issue, not the developers that likely raised it a million times already.
- ConorSheehan1 3y ago
  What a poor bunch of overworked human beings, with almost no control over the product they work on. Frantically following the whims of managers, reduced to labour units in this late stage capitalist hellscape.
  - colund 3y ago
    That's such a beautiful comment I would almost consider printing it and putting it on my wall
  - jeltz 3y ago
    But well paid at least. Working at Meta seems pretty shitty except for the pay from the stories I have heard.
    - loeg 3y ago
      It probably depends what team you're on, but I would not describe it as "pretty shitty." Being oncall for a 24/7 service sucks, yeah, but for my team it is one week a quarter and I haven't had any outside-of-biz-hour alarms the last few shifts. Other than that -- my work is challenging and interesting, my colleagues are friendly and smart, and my manager is decent. Not a lot to complain about.
- nuz 3y ago
  Imagine how many hundreds of millions of users waste their time using instagram and facebook on a daily basis. Safe to say they don't mind wasting their customers time
  - cratermoon 3y ago
    This is your regular reminder that the users are not their customers. Advertisers and media outlets are the paying customers.
- matsur 3y ago
  Major outages are periods of intense stress and extremely difficult to operate in. The folks troubleshooting may be many things, but careless and incompetent are unlikely to be among them.
- Solvency 3y ago
  I can almost guarantee you're getting mercilessly downvoted because half of the people here are sympathetic Meta worshippers who desperately (1) wish they worked there and (2) know they'd probably contribute similarly to this same horribly engineered system.
newman8r 3y ago
How many other people here just assumed they had been banned for some arbitrary, uncontestable reason? I just use instagram to post hiking photos...
- TonyTrapp 3y ago
  Having had a look at desperate Twitter posts during a major outage of a big German email provider with similar failure mode (login failed silently), it seemed like many people assumed that their email account was hacked. Close enough.
sumitgt 3y ago
Could be an outage in the auth service itself
- Pxtl 3y ago
  Unless it's an outage in their ability to log into their own servers, they should be able to swap out the login page with a static HTML page explaining the outage. Maybe a 503 status code.
- remram 3y ago
  Yes of course it was. The point is, an hour later, they could have hit a circuit-breaker to get people to stop trying and going crazy over an error that is completely inaccurate.
Pxtl 3y ago
Right? I know I was like "oh, I haven't typed in my FB password in eons... maybe I changed it at some point and forgot? But if I change it what happens to all related services, is it going to log out my kids' Messenger Kids devices? Those are such a pain to log in. Should I change my password or not? What do I do?" Then I saw the news that it was an outage.
zerr 3y ago
For anything outside like-ing and post-ing, facebooks UI/UX is horrendous. Even Internet search does not help to find out how something trivial is done... The only way is to watch youtube videos.
- dspillett 3y ago
  > For anything outside like-ing and post-ing, facebooks UI/UX is horrendous. It isn't perfect for even that IMO. > Even Internet search does not help to find out how something trivial is done... The only way is to watch youtube videos. That says more about where the web is heading than about facebook. Video is easier to monetise ATM⁰, and these days people don't put helpful stuff out there just to be helpful as much as they once did¹, so content creators are making them instead of simple web pages. -- [0] Everyone wants to be the next big influenza who doesn't need a day job to get by. [1] That sort of people are still out there, though they are somewhat drowned out as the signal-to-noise ratio heads inexorably towards “WHAT? WHAT?! I can't hear a single thing above the manscaping adverts!”.
bluetomcat 3y ago
> Display a cute image that shows that the service is drastically failing (like a whale or something), and then people will know to stop trying to repeatedly log in. Probably not so easy to implement in behemoth apps, consisting of 20'000 source files...
mkl95 3y ago
What do you mean my login flow sucks? Its time complexity is really good!

bandwidth 3y ago

Surely the repeated logins can't be helping the situation. I suppose it is entirely auth related across all Meta products. The repeated strain could pose a cold start problem for example.

brk 3y ago

This is a great idea! I know that I was flailing about trying to figure out why I couldn't login, so I'd suggest calling it a "Flail Whale".

zabzonk 3y ago

> catchy name Petunia?

boring_twenties 3y ago

Beyond unbelievable that going on an hour later, they're still showing "incorrect password" errors. How many hundreds of millions of people have wasted time frantically trying (in vain) to reset their passwords and pointlessly freaking out that their account might be compromised? What a bunch of careless, incompetent excuses for engineers.

alwyn 3y ago
Blame the managers and product owners that don't think this is an issue, not the developers that likely raised it a million times already.
ConorSheehan1 3y ago
What a poor bunch of overworked human beings, with almost no control over the product they work on. Frantically following the whims of managers, reduced to labour units in this late stage capitalist hellscape.
- colund 3y ago
  That's such a beautiful comment I would almost consider printing it and putting it on my wall
- jeltz 3y ago
  But well paid at least. Working at Meta seems pretty shitty except for the pay from the stories I have heard.
  - loeg 3y ago
    It probably depends what team you're on, but I would not describe it as "pretty shitty." Being oncall for a 24/7 service sucks, yeah, but for my team it is one week a quarter and I haven't had any outside-of-biz-hour alarms the last few shifts. Other than that -- my work is challenging and interesting, my colleagues are friendly and smart, and my manager is decent. Not a lot to complain about.
nuz 3y ago
Imagine how many hundreds of millions of users waste their time using instagram and facebook on a daily basis. Safe to say they don't mind wasting their customers time
- cratermoon 3y ago
  This is your regular reminder that the users are not their customers. Advertisers and media outlets are the paying customers.
matsur 3y ago
Major outages are periods of intense stress and extremely difficult to operate in. The folks troubleshooting may be many things, but careless and incompetent are unlikely to be among them.
Solvency 3y ago
I can almost guarantee you're getting mercilessly downvoted because half of the people here are sympathetic Meta worshippers who desperately (1) wish they worked there and (2) know they'd probably contribute similarly to this same horribly engineered system.

newman8r 3y ago

How many other people here just assumed they had been banned for some arbitrary, uncontestable reason? I just use instagram to post hiking photos...

TonyTrapp 3y ago
Having had a look at desperate Twitter posts during a major outage of a big German email provider with similar failure mode (login failed silently), it seemed like many people assumed that their email account was hacked. Close enough.

sumitgt 3y ago

Could be an outage in the auth service itself

Pxtl 3y ago
Unless it's an outage in their ability to log into their own servers, they should be able to swap out the login page with a static HTML page explaining the outage. Maybe a 503 status code.
remram 3y ago
Yes of course it was. The point is, an hour later, they could have hit a circuit-breaker to get people to stop trying and going crazy over an error that is completely inaccurate.

Pxtl 3y ago

Right? I know I was like "oh, I haven't typed in my FB password in eons... maybe I changed it at some point and forgot? But if I change it what happens to all related services, is it going to log out my kids' Messenger Kids devices? Those are such a pain to log in. Should I change my password or not? What do I do?" Then I saw the news that it was an outage.

zerr 3y ago

For anything outside like-ing and post-ing, facebooks UI/UX is horrendous. Even Internet search does not help to find out how something trivial is done... The only way is to watch youtube videos.

dspillett 3y ago
> For anything outside like-ing and post-ing, facebooks UI/UX is horrendous. It isn't perfect for even that IMO. > Even Internet search does not help to find out how something trivial is done... The only way is to watch youtube videos. That says more about where the web is heading than about facebook. Video is easier to monetise ATM⁰, and these days people don't put helpful stuff out there just to be helpful as much as they once did¹, so content creators are making them instead of simple web pages. -- [0] Everyone wants to be the next big influenza who doesn't need a day job to get by. [1] That sort of people are still out there, though they are somewhat drowned out as the signal-to-noise ratio heads inexorably towards “WHAT? WHAT?! I can't hear a single thing above the manscaping adverts!”.

bluetomcat 3y ago

> Display a cute image that shows that the service is drastically failing (like a whale or something), and then people will know to stop trying to repeatedly log in. Probably not so easy to implement in behemoth apps, consisting of 20'000 source files...

mkl95 3y ago

What do you mean my login flow sucks? Its time complexity is really good!