9 ms·

Warning – Google suspended GCP services for 'verification', lost my business

476 points by alirsgp 4y ago

I operate multiple apps which were providing my entire livelihood, and I wake up one day to realize it's all gone. Long story short, Google froze my Google Pay account which is used to pay for anything related to Google services, including GCP. This was done as they need to temporarily suspend me to 'verify' me. The GCP team couldn't help me since it was an issue with Google Pay. Not to mention their chat support is useless and doesn't have any power to look into cases. Now I login to find out my Google Pay account is 'temporarily suspended' since I need to verify my account. I go ahead and add my drivers license and fill out their form, but while they verify details they suspend your ability to pay for any Google services. So now I can't pay for GCP usage (firebase) and all my authentication, database, cloud functions, are all not usable since the service is suspended. Unlike Amazon who offers phone lines for resolving billing issues, Google does no such thing. I wish I chose AWS. The end result is that I have apps I spent years of my life building and which currently feed me to only be rendered unusable because some AI at google thought my Google Pay needed additional verification. My whole life has done a 180 in the past week. I feel like I'm a man with literally nothing to lose and I can't live with myself anymore. This hurts much more than losing a job for me. Please stay far, far away from Google services of any kind. They are by far the worst of the big tech companies when it comes to developer support.

joshxyz 4y ago
another satisfied customer! geez guys keep your business away from google, it's tough not being able to reach a damn human for support especially when you're a paying one.
Aeolun 4y ago
Yeah, that sounds about par for the course for Google. I don’t understand why anyone would ever choose them for anything critical. Free support for AWS is better than anything Google has (at least based on my reading, for obvious reasons I have no first-hand experience).
- vbezhenar 4y ago
  Everyone praises their managed kubernetes offering. I have no experience with it, but if you ask people around, everyone prefers GKE to alternatives. That might be one reason.
  - x86x87 4y ago
    yes. it's a no brainer really. you get GKE and in return you may lose your business if you hit any snags. we're past the point where K8s is this exotic thing where people are not sure exactly where it is.
nonrandomstring 4y ago
This is really sad. I feel bad for you and hope you manage to rebuild your base somewhere else safe - hopefully with technology that you control. You won't be the last person to get screwed over by the callous machinery of dehumanised big-tech (AWS or Microsoft would be no better, sadly I think you are rationalising loss). We must all wake up to just how horribly precarious the lives of ordinary people are against the arbitrary whim of opaque, unassailable power centres that masquerade as "Free services".
- tracerbulletx 4y ago
  I don't have the statistics to back up any statement about the odds that one provider or another is safer, but AWS has had a person I could talk to and they really resolved my issues and seemed empowered to do so. That's a big and real difference.
  - kkielhofner 4y ago
    Anecdotal but I listed a few graphics cards on Amazon with the same account I had AWS on and they booted me without any recourse or contact information. Very strange.
    - tracerbulletx 4y ago
      There really needs to be some more legal precedent around this stuff. In the commercial real estate leasing industry there are legal commitments and a framework of legal understanding to protect business owners or at least provide predictability. That needs to be applied to leasing computing resources given how much business flows through the internet now.
      - cavisne 4y ago
        The frameworks largely exist, the above situation is like if you rented an office building and then started running an unlicensed side business out of it. It’s just a lot rarer (cloud computing has a much lower cost of entry then a commercial lease).
superchroma 4y ago
Unsurprising. I keep seeing this story over and over. I use google for nothing and have for years. For me, the key factor is that there's nobody to call. Want support? You had better be famous on twitter.
ilyt 4y ago
It's sad none (I think?) of the big cloud providers doesn't allow for prepaid account, it would solve both "our idiots thought that blocking long-term paying customers with no human intervention is a good idea" and "I woke up this morning with $20k AWS bill" problems
- fnordpiglet 4y ago
  Aws has no provision for shutting stuff down for exceeding a budget (or at least when I ran services there). We would rather reverse your $20k bill and keep you alive. I ate huge write downs from small and big customers accidentally blowing themselves up. I remember on guy almost in tears because he knew he would be fired due to his screw up that blew up the bill. It made me proud to help him. Amazon has a ton of serious issues but they are genuinely incentivized to care about customer problems (I.e., you are paid and promoted on how far you go for customers).
  - 0cf8612b2e1e 4y ago
    > We would rather reverse your $20k bill and keep you alive And as a customer, I never want to be in a position where I ever see $megabill. Maybe I might be able to get reversed if the gods smile upon me that day, but it would be incredibly stressful until I had resolutions. Seems an obvious blindside for “customer obsessed”. Anything I might host is practically a joke for which I would gladly have the provider pull the plug if billing suddenly spiked above pre-established threshold. I’ll take the downtime if it helps me sleep at night.
    - wingleader 4y ago
      I've had the exact polar opposite experience on any company running systems with paying customers. Usually it's "don't care what it costs, just fix it" which lines well with AWS' behaviour on this. When downtime costs tens of thousands of dollars _a minute_, people tend to stress less about some inefficiencies like this
      - 4y ago
twawaaay 4y ago
> I wish I chose AWS. Well... AWS has better support IMO (like actual people you can talk to). But I still chose to not tie myself to any of these platforms for anything critical. It requires additional upfront effort, but I can migrate all my applications off of AWS in a moment. There is no easy way around Google and Apple when it comes to your app staying available, but I am trying to minimise any chances I could be flagged for anything (basically not linking those accounts to any other services).
- joshxyz 4y ago
  im actually surprised aws gives you live chat support even at their free tier, pretty much why i also love namecheap.
- x86x87 4y ago
  being able to migrate != support GCP is not going to be a thing in a few years. Why would anyone pick GCP at this point with all the horror stories coming out of there? Yes, support is hard, yes it's expensive, yes it takes time. But at the end of the day this is how you make or break the trust of your customers.
- sebazzz 4y ago
  > But I still chose to not tie myself to any of these platforms for anything critical. It requires additional upfront effort, but I can migrate all my applications off of AWS in a moment. Then you are probably not doing "cloud native" development using their proprietary SDKs, so probably just running some EC2 (virtual machine) instance and RDS (database) instance. That is about the most expensive way you can use the big-tech clouds, at that point you might as well move to the smaller hosting providers like Linode.
  - npteljes 4y ago
    How well do smaller / more traditional providers support auto scaling and various other "elastic" features of AWS? I only have experience with renting a "traditional" VPS, and some of the more modern AWS offerings.
  - 4y ago
dmitrygr 4y ago
Same thing happened to me: “suspended, need extra verification” after YEARS of the account existing just fine. Nothing helped. Not doing what they asked (multiple photos of photo id), not emailing, nor calling support. I ended up calling an ex-coworker who still worked at google. He sent an internal email. It was resolved the same day. I never found out what the issue was. Don’t use google for anything you care about, unless you maintain close friendships with googlers.
- pteraspidomorph 4y ago
  That doesn't always work, (following anecdote is not about google cloud): someone I know lives in a street that's egregiously and very obviously broken on google maps (whole network of streets mapped with the same name, private pathways mapped as public streets, whole bunch of repeated door numbers, etc) and since everyone trusts google maps for everything this has caused a bunch of serious issues for the person as they interact with businesses and even government. Any attempts to get this fixed through the normal process get ignored (I have personally tried, several years ago, twice). Over the years I asked two different friends at Google for help, one of whom was management. They were baffled when the internal message thing yielded no results either. There doesn't seem to be any process for getting certain types of issues resolved at google. My point is that google can ruin people's lives even if the victims aren't the ones using it. Gmail blocking inbound mail is another example - if gmail decides to block your service, everyone will blame you, and there's nothing you can do about it other than wait.
Trencin 4y ago
It seems like a large overhead, but making your app multi-cloud compatible to counter for scenarios such as this might be not be as crazy as it sounds. 99.99% availability doesn't cover for scenarios like this.
- dahdum 4y ago
  I'd just make a backup AWS account with separate billing before I worked on multi-cloud.
  - Trencin 4y ago
    Could work, but if you're being targeted for some reason it's likely they would find your other account.
  - sparrish 4y ago
    That's short-sighted and poor engineering. You need your service to be able to keep going no matter what any single vendor does. Never intentionally create a single point of failure like that.
throwaway892238 4y ago
Google locked our company out of using our own domain on Google Workspaces because a former employee had signed up for a Workspace account using our domain, and we had no way to recover it. We literally manage our domain in GCP but they wouldn't accept that we own our own domain and thus won't let us use our domain with Workspaces, which screws you out of a lot of Google functionality. Never seen a company that was too incompetent to allow users to use its own product and make zero attempt to solve the customer's problem. Google Cloud CEO Thomas Kurian said "We need scale to be profitable." No, you need to stop treating your customers like crap to be profitable. We don't trust you, you are a pain in our asses, and we have alternatives, so there is really no need for us to spend our money on you. It's now a matter of when we all move to AWS and Azure, not if.
- userbinator 4y ago
  It's now a matter of when we all move to AWS and Azure, not if. If you really think that the "grass is greener on the other side"... No, the real solution is to stay away from Big Tech and cloud in general.
  - jiggawatts 4y ago
    I’ve never heard of anyone getting locked out of their Azure account like what seems a regular occurrence with GCP. They have human support and have proper Enterprise Agreements that don’t turn off your business because some heuristic tripped.
    - bozhark 4y ago
      This is by Google’s design. You have to get enough traction on social media to get on someone important at the company’s feed
  - seanhunter 4y ago
    That’s completely contrary to my experience. Running a small startup we encountered all 3. For us, MS/Azure were the most eager to please but in many ways the worst from an execution standpoint. GCP were very good from an execution standpoint but absolutely zero support and stories like this one scared us off. Lots of big GCP corporate clients actually use them through consultancies which essentially provide the support and are large enough on aggregate to get listened to by Google. AWS were good at actually doing things and also offered support the couple of times we needed it.
autotune 4y ago
Care to divulge a general idea of what these apps were before expecting sympathy?
- fnordpiglet 4y ago
  Why does what the apps do matter? The dude has software that puts the sushi on the table and google peed all over the table.
  - autotune 4y ago
    I want to know what the ban was for. What if the ban was due to a crypto mining operation? All I am asking for is a little more context before pointing any fingers.
    - justinwp 4y ago
      One of the apps by this developer: https://apps.apple.com/us/app/cuff-video-chat-make-friends/id1514536428 https://apps.apple.com/us/app/cuff-video-chat-make-friends/i...
      - autotune 4y ago
        This does not seem worthy of a ban on the face of it. My sympathy goes out to OP.
        shagie 4y ago
        The 18+ video chat is one of them... and (apparently) on the same domain is https://apps.apple.com/us/app/bitalert-bitcoin-ether-alert/id1577343755 https://apps.apple.com/us/app/bitalert-bitcoin-ether-alert/i... (note the privacy policy and the terms and conditions and look at the top level service). Having a service that is getting network traffic for both crypto related things and 18+ dating may have raised some algorithmic alarm bells.
      - mathverse 4y ago
f0e4c2f7 4y ago
Wow I'm really sorry this happened to you. I do believe your conclusions are correct about AWS. Azure generally seems fine too. Google's aversion to customer service makes them extremely dangerous as a cloud provider in my mind. This also goes for other critical business services, like Gsuite (I know! It's convenient). If you have GCP or Gsuite now and you're trying to evaluate how big of a deal it is my suggestion would be to pick up a phone and attempt to talk with someone at Google about your account. This experience can act as a preview of what that process might look like when services are turned off. If you try to call Amazon on the other hand it feels like Jeff Bezos might hop on the call if things aren't going well.
- roxgib 4y ago
  I've had some bumpy points with Azure, but if I really can't fix something I can pick up the phone and they'll try to help. The idea of having critical business services with a company that you can't actually talk to is terrifying.
- orbz 4y ago
  I know it’s just anecdotal, but once you have some spend (around 1k a month I’m guessing), you get an account executive who I’ve been able to reach on the phone whenever I’ve needed. In fact they tried talking to me on the phone more than I wanted.
  - more_corn 4y ago
    Google engineers are not known for being super available though.
    - orbz 4y ago
      My rep was able to get me connected to a few tech leads on various cloud products and one time when we were hitting weird quota limits the team protectively reached out to me to ask what the heck we were doing. Maybe we were just lucky, but I’ve had similar experiences in AWS and Azure too.
- 4y ago
dmak 4y ago
I have had GSuite with my domain for 10 years, so theres a lot of emails and apps that I used with Google auth. I always worry about what might happen if something out of my control on Google's end locked me out.
- sitkack 4y ago
  You should backup all of your data and ensure that your domain name is paid well in advance and that you can login to your domain name services w/o 2fa hitting the domain in question. You can always point your domain to a different provider and reset if you can recv email.
  - dmak 4y ago
    Thank you, and that part is covered, but I am not sure how I could replace Google Login if I point to a different provider.
    - konha 4y ago
      > I am not sure how I could replace Google Login if I point to a different provider. You can’t. You’ll have to rely on the account recovery mechanisms of the services you used Google login with. Some might be able to switch over to email login after verification, some might not even have a manual way to recover your account / switch it to another login method.
stevage 4y ago
That sucks. I don't like Amazon as a company, but AWS support is great. A couple of times in the past I've made some complaint about them on Twitter, and a rep has actually reached out and helped resolve the issue. And I'm a tiny fish, with monthly bills under $10.
- x86x87 4y ago
  this is not about how tiny you are. it's about having a system in place to fix issues and help your customers.
de6u99er 4y ago
I had this once with Google Drive, when I forgot to enter my new credit card because the existing one had expired. Took weeks to get my files back. It's like people at Google just wait for shit to happen instead of trying to think of the obvious things that could go wrong beforehand. That being said, if Google is looking for people to lay off there's some obvious candidates. I would include their mangaers too.
someRandoJunk 4y ago
I have seen similar stories like these in the past here. It's a shame Google hasn't done anything at all for so long to combat it.
rybosome 4y ago
I am very sad to hear that this happened to you. It must’ve been absolutely maddening to be unable to talk to a human being as your business was destroyed.
- alirsgp 4y ago
  I haven’t had anxiety like this ever before. I’ve worked all day to contact as many ppl as I could. I wish Google would separate their billing from Google Pay and not have this dependency.
MattyMc 4y ago
Upvoting in the hopes that someone on HN has a connection internally at Google, AND to raise awareness for this issue. My business uses Firebase, and I'm having serious second thoughts about that right now.
ryandrake 4y ago
Sigh. It's now been [0] days since the last "My [business | e-mail | critical thing] depended entirely on a single service with no backup, I got suspended, and now I can't do my [business | e-mail | critical thing]!" story. I do sympathize, and the right fix is for these services to have better support, but come on... These stories happen so often and get ample press. They should not be surprising. Please stop doing this, people! If something you rely on for your livelihood depends on a single company as a single point of failure, that is a flaming, burning emergency that needs to be solved right now!
- sparrish 4y ago
  I'll go one step further and say that if your website/app/service can be taken offline by any issue with a single vendor, it's more than an emergency, it was engineered incorrectly from the start. Are new devs taught that having something in two AWS zones equals redundancy? You drank the Kool-aid, people.
- x86x87 4y ago
  nope. nope. nope. are you supposed to write your own firebase? develop your own android? yes, OP learned a hard lesson about Google, but that does not mean that you cannot build stuff using just one provider. you need to do your due diligence of course. You need to have a plan, of course. But the cost of doing this on 2 platforms skyrocket. And especially for a small player the multi-cloud, massive investment is just not an option.
justinwp 4y ago
Based upon some of the submitter's previous submissions. It appears that they have some 18+ "dating" apps... Disclaimer: Googler, not in this area.
- nextaccountic 4y ago
  Like Tinder? Why does this matter?
- fbdab103 4y ago
  And we can only be outraged at the situation if the author built a business in a field you endorse?
- the-angry-dome 4y ago
  This does not matter. If they do not violate Google's terms of service, then Google is in the wrong. Disclaimer: Xoogler, who worked on Dasher and other paying-customer-facing tools.
- trenchgun 4y ago
  So what? Why does it matter?
fnordpiglet 4y ago
I’m sorry. Are you able to work with your customers to keep their business while you sort this? I would port your stuff to aws. Most services have close equivalents and it is probably worth while to try. As another poster noted it might be useful to have cross provider redundancy. I’ll be honest I have refused to work at gcp and have steered the mega corps I’ve advised on cloud choices explicitly away from gcp. Azure is better, but less technically sophisticated. It’s still a one shop cloud. And any company whose core business is surveillance and advertising won’t care about their customers - you’re just meat for their machine. Amazon was built around a customer centric business (retail) and aws inherited the ethos of that retail business. It’s technology is marginally less sophisticated than googles, but googles technology comes with an embarrassingly large amount of smugness and clever solutions that fail in clever ways. Amazon basically just wraps up common technologies in a managed control plane (aside from a few primitives like kinesis, SQS, sns, and DynamoDB) and doesn’t infect everything with their smartness. Good luck.
diebeforei485 4y ago
Paying customers should absolutely have access to customer service.
andrewstuart 4y ago
No one from Google ever turns up here and says they care and it matters. It's now broadly known that developers and businesses don't trust Google, but strangely as far as I can tell, the google CEO has never been held to account for this.
- trenchgun 4y ago
  https://news.ycombinator.com/item?id=33738014 https://news.ycombinator.com/item?id=33738014
ChrisMarshallNY 4y ago
I hear about this kind of thing, all the time. I guess they manage to not do it that often, so I guess I hear only the bad reports, but they are universally awful, when I do hear them, and they all have the same theme: A.I. Broke it, and the [rare] humans in the loop are powerless to mitigate the A.I.'s breakage. I was considering using Firebase, when I started the project that I'm working on, now, but got cold feet, and wrote my own server. It took a while, but I'm glad that I did.
andrewstuart 4y ago
Even if its hard, your business should have a strategy for coming back up when a critical vendor cancels your account. Not being able to survive a cloud vendor cancelling you is the 21st century equivalent of not having any backups.
- sparrish 4y ago
  You still need to have backups, even in the 21st century. Being brought down by a single vendor is poor engineering.
  - sebazzz 4y ago
    By convenience, backups are often stored on the cloud provider. Yes, this violates the rule of 3, but cloud providers often don't make it easy to do otherwise.
themihai 4y ago
I think this is to be expected(sooner or later) for small businesses dealing with "big tech". You are just a small number in an ocean of "users" so suspending you is really no different than suspending your twitter account. Start taking some responsability folks! Let's stop feeding these walled gardens!
toyg 4y ago
> I feel like I'm a man with literally nothing to lose and I can't live with myself anymore. Dude, I know the pain, but it's only money. Life goes on. If anything, now it should be your mission to stick it to the faceless corporation, not to give in to dark thoughts.
rygar 4y ago
Hi, I am sorry that this is happening to you. Please send me your info to my corp email - rygar@google.com - I'll do my best to help.
- alirsgp 4y ago
  I just sent you an email, thank you, please let me know anything I can provide you in email to help.
- jiggawatts 4y ago
  To quote a HN user talking about another similar instance of a cloud service failing to support their customers: > “You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.” From: https://news.ycombinator.com/item?id=20066239 https://news.ycombinator.com/item?id=20066239
- x86x87 4y ago
  not all heroes wear capes, but look at this person helping out on thanksgiving. this is both amazing and sad (that is necessary to do this to get help)
  - MrMan 4y ago
    its nice on a personal level but a sign of how broken things are at GCP that an angel has to emerge from the clouds just at the last moment to save your livelihood? that is super creepy and not the way the information economy is supposed to work, in my opinion.
    - x86x87 4y ago
      it is. this does not diminish the nice gesture though.
musk_micropenis 4y ago
Conversation in this user's comment history. Sounds like he knew the risk he was taking and knew what he'd need to do if it came to that. ----- alirsgp on Oct 24, 2021 | parent | context | prev | next [–] | on: 5 side projects in 6 years, earning $0 I've published ~20 iOS apps. Only 4 make money, but they allowed me to reach financial independence and never have to work for someone gain at the age of 24. krater23 on Oct 25, 2021 | parent | prev | next [–] What's when your apps are removed from Apple for any reason? Do you have enough money to live without this income? Or do you need then just start over again? alirsgp on Oct 26, 2021 | root | parent | prev | next [–] I'd have ~2 years and not hurting my lifestyle. Hopefully it doesn't come to that, but I can get a job if needed
grog454 4y ago
My first experience with AWS was around 2013 when I accidentally left a (cheap) ec2 instance running for months after I was done experimenting with it. Support refunded me no questions asked. Fast forward 2 years and that experience was an important factor in my decision to run servers costing 6 figures annually with AWS over their competition.
- ycombiredd 4y ago
  I came to write a very similar story involving a lazy error on my part (no error checking on a shutdown ec2 instance script, resulting in a 64 Xeon, 8 Nvidia Tesla server running 24/7 for about a month) and an astronomical bill resulting. I had a rudimentary batch job system running. Since I was charged for each power event, I wanted to minimize the power state changes and just queued up the work that needed GPU acceleration until there was enough to justify the cost of at least an entire hour and two power state events. When the cheap, small, always on instance received all the work output from the giant beast server, it would then send a power down command using the aws cli. Yeah, like I said, I was lazy so I just sent off the request and small server just went on about it’s business. Oops. Like I told the aws billing rep, it would have bankrupted me, ending that venture. I asked that, since it is all virtualized, that while yes I had the virtual hardware provisioned for all that time, if they could get someone to look at the actual utilization they would see it was essentially none. They got back to me and in a day or so to let me know they had forgiven the entire amount. AWS support actually listen, do real work, and very often it seems, do what is in the best interest of the customer even as it works directly against the company’s bottom line.
- thirdusername 4y ago
  When I was early in my career freelancing my AWS account was shut down for no payment for over a year, more than half a decade later I re-opened it through support and they wouldn’t even let me pay my old bill. I’ve been able to reduce reserved instances when the market has turned unpredictable. AWS isn’t a pushover, not even a little bit. Dealing with them externally has just been a good experience with professional understanding and a flexible business sense.
mrslave 4y ago
What are the alternatives? Is Bare Metal with OpenStack or K8s any more resilient to administrative DoS? Do we simply not care about this problem because the businesses being suspended are small fish?
db48x 4y ago
You have to sue them. If nobody sues them, then they will simply keep doing it.
nextaccountic 4y ago
Now you have posted here on HN, I'm curious to know if Googler saw this post and escalated the issue.
- alirsgp 4y ago
  Yes, they did and I am awaiting a response. Being thanksgiving holiday might be slow
rufius 4y ago
Google - “What kind of cloud would a clown make?” I just can’t fathom Google’s inability to sort this out. They’ve been the laughing stock of serious cloud business for years.
- x86x87 4y ago
  At this point I am having trouble understanding who would consider GCP. I mean maybe something along the lines of: something something, multi-cloud strategy, no lock-in options, let's also use GCP so we have multi-cloud. But I never see it as the option to pick. It's sad really.
  - Terretta 4y ago
    > trouble understanding who would consider GCP 1. SaaSy kids in SV or who wish they were in SV who also think Google Sheets beats Excel. 2. Big enterprise executives who heard Google was what the cool SaaSy kids do from someone in I.T. then got lured by too-good-to-be-true discounts from Google Cloud sales team as long as the executive is happy to be featured for their forward-thinking tech chops in pressers about picking GCP. To their credit, they will undo the Office to Workspace migration mandate almost immediately, after making their 140,000 users attempt it from IE 10 on Win 7, but there won't be press about that.
    - x86x87 4y ago
      > Google Sheets beats Excel I loled out loud
- joshxyz 4y ago
  Hahaha Google Clown Platform
gmbsuspended 4y ago
Sadly we hear so much of this on a daily basis as we are often sought out to support suspended Google Business Profile (GBP) accounts. It's not uncommon to have these same small to medium sized business owners ask for help with other suspended accounts in other Google apps like GoogleAds or Google Merchant Center. The support across all platforms is a growing eyesore for Google, and at this point I hope becomes the catalyst for real change in this space. It was bad enough when Yelp would act like the Mafia in regards to how they approached and treated businesses, but this level of negligence (from such a tech giant) is something to really see. We've been advocating on behalf of all these small businesses for years and truely hope that the outcry grows to a fever pitch where no more band aid solutions are used anymore.
iJohnDoe 4y ago
It’s not AI. It’s a jealous contract worker in India or a jealous Google employee because you were making money with your app. We really need to stop blaming a “Google AI”. Google has 118,000 employees. Someone at Google could help you if they wanted to, but they don’t want to.
deleted 4y ago
[deleted]
onphonenow 4y ago
I’m missing how a google pay account issue affects GCP? Use a different credit / debit card?
- alirsgp 4y ago
  GCP like all Google services force you to use Google pay. I have 4 different payment options there, but they are suspending me until they verify my ID. This is a big problem in cloud services since being down for days or weeks or longer can destroy apps and their standings in the app stores.
Zuiii 4y ago
Thank you for reminding me why I avoid Google services like the plague. Even relying on unmanaged apps on a single pet server is less risky than relying on Google. It's like playing russian roulette with your infra. Google Roulette.
speedgoose 4y ago
In the beginning of the year, I was recommended by a friend working at Google to test GCP using the tutorials on https://www.cloudskillsboost.google/ https://www.cloudskillsboost.google/ Google banned my account while I was doing one tutorial. One step failed, the provisioning of something, and I had to redo a few things again. It succeeded but I had used more egress bandwidth than allowed to run the tutorial correctly so they banned me automatically a few minutes later. The support unbanned me after some time but I thought it was a sign to not use GCP yet.
more_corn 4y ago
I too am currently in Google Pay support hell. Never in my life have I experienced such a Byzantine hell of weaponized incompetence. I’d post the thread but it’s just 48 exchanges of me calmly and patiently explaining the situation. And them failing to read the case notes, saying platitudes and then pasting the same non-useful canned response. Here’s a fun fact or two: Did you know there’s no way to get your money out from the web interface? Did you know there’s no way to select your profile id on iOS? These two features haven’t been implemented. If you’re clever you can reconstruct the rest. Their documentation indicates that they also don’t have an account closure reconciliation process. I’m hoping that’s just inaccurate documentation because that’d be illegal. Good times. Some states attorney generals should band together and sue those jerks.
albertopv 4y ago
I see many blame OP for relying on one single cloud provider without having backup and exit strategy. What if she/he is a single persone doing everything and having just barely the time to run the business? It's the same mentality I have seen from people complaining about Phoronix quality, ignoring It's a one man show website.
- alirsgp 4y ago
  Yes, I have a team of contractors I pay but essentially a one man show.
snird 4y ago
Google and GCP isn't capable or want to provide support to customers who pay less than ~$20k a month. Don't ever use GCP if you're not a big enough enterprise. Just as you should never buy a Dell laptop as an individual consumer. I'm sorry this happened to you, I hope someone will learn from your experience
- DarrenDev 4y ago
  Why should you never buy a Dell laptop? What can go wrong with that?
- t00l00 4y ago
  I’m £7.5k/mo in UK: have a named account manager, and an engineer who spends 2h/mo discussing whatever I want on a Google Meet, and bugs I can email directly. Got invited to their Leeds office to say hi and spend a day with them.
mathverse 4y ago
The difference between AWS/Azure and Google is that the former run a business. Google is a cult with a subscription fee.
thiht 4y ago
There are horror stories like this with all of Big Tech, but Google is hands down the worst offender. I do my best to avoid Google in every corner of my life: the only services I still use are Gmail (slowly migrating to my own domain with another provider though) and Search. But I’ll never use a new Google product anymore because of their unreliability on the long term.
splix 4y ago
Same happened to me too. They suspended the payment account. Two accounts actually, because different cards. Sent the driver license picture a dozen times, for one account someone eventually took a look and unlocked it (which took about a week). But another one is in review for several months already. And it's impossible to do anything with that.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
bfergsthesnitch 4y ago
Hey ali, you should know daniel is ridiculing you on another server.
csisnett 4y ago
I'm so sorry this happened to you this is why is important to not have a single point of failure. Have domain name in X company, hosting in another, etc. That way if one turns crazy there's More hope
t00l00 4y ago
Does this support depend on a region? I’m here in UK, my bill is sub £10k/mo and literally today few hours ago I had an hour with Google engineer discussing use of their MultiClusterIngress to mix GKE standard and Autopilot to handle spikes extremely fast. Afterwards dude sent me docs and his project he used as an example. It’s the second time this month: we don’t even chat about any specific bugs, more about my architecture goals. I also have an account manager and she’s currently working on making discount rates from our contract to work with our new hierarchy of multiple billing accounts, some VAT paying and some not, that my finance department requested. You also can have backup payment methods? I think we have DDs as a default and then CCs as backups.
heresjohnny 4y ago
I was tempted to use Firebase for a new project because it looked so easy. I’m fairly fluent in Angular and it integrates extremely smoothly, so at a surface level it would be the perfect choice for the backend. Stories like yours – thank you for sharing – plus horror reports about sudden 20k bills have prevented me from taking the “cloud” leap. I’m very glad I didn’t, especially because I also learned lots by setting up a web server and the application layer myself.
alirsgp 4y ago
I just want to note that I was able to solve this issue. Thank you to the googler who reached out to me to help, 'rygar', and the rest of the google cloud/pay support team for helping fix this issue.

476 points by alirsgp 4y ago

joshxyz 4y ago

another satisfied customer! geez guys keep your business away from google, it's tough not being able to reach a damn human for support especially when you're a paying one.

Aeolun 4y ago

Yeah, that sounds about par for the course for Google. I don’t understand why anyone would ever choose them for anything critical. Free support for AWS is better than anything Google has (at least based on my reading, for obvious reasons I have no first-hand experience).

vbezhenar 4y ago
Everyone praises their managed kubernetes offering. I have no experience with it, but if you ask people around, everyone prefers GKE to alternatives. That might be one reason.
- x86x87 4y ago
  yes. it's a no brainer really. you get GKE and in return you may lose your business if you hit any snags. we're past the point where K8s is this exotic thing where people are not sure exactly where it is.

nonrandomstring 4y ago

This is really sad. I feel bad for you and hope you manage to rebuild your base somewhere else safe - hopefully with technology that you control. You won't be the last person to get screwed over by the callous machinery of dehumanised big-tech (AWS or Microsoft would be no better, sadly I think you are rationalising loss). We must all wake up to just how horribly precarious the lives of ordinary people are against the arbitrary whim of opaque, unassailable power centres that masquerade as "Free services".

tracerbulletx 4y ago
I don't have the statistics to back up any statement about the odds that one provider or another is safer, but AWS has had a person I could talk to and they really resolved my issues and seemed empowered to do so. That's a big and real difference.
- kkielhofner 4y ago
  Anecdotal but I listed a few graphics cards on Amazon with the same account I had AWS on and they booted me without any recourse or contact information. Very strange.
  - tracerbulletx 4y ago
    There really needs to be some more legal precedent around this stuff. In the commercial real estate leasing industry there are legal commitments and a framework of legal understanding to protect business owners or at least provide predictability. That needs to be applied to leasing computing resources given how much business flows through the internet now.
    - cavisne 4y ago
      The frameworks largely exist, the above situation is like if you rented an office building and then started running an unlicensed side business out of it. It’s just a lot rarer (cloud computing has a much lower cost of entry then a commercial lease).

superchroma 4y ago

Unsurprising. I keep seeing this story over and over. I use google for nothing and have for years. For me, the key factor is that there's nobody to call. Want support? You had better be famous on twitter.

ilyt 4y ago

It's sad none (I think?) of the big cloud providers doesn't allow for prepaid account, it would solve both "our idiots thought that blocking long-term paying customers with no human intervention is a good idea" and "I woke up this morning with $20k AWS bill" problems

fnordpiglet 4y ago
Aws has no provision for shutting stuff down for exceeding a budget (or at least when I ran services there). We would rather reverse your $20k bill and keep you alive. I ate huge write downs from small and big customers accidentally blowing themselves up. I remember on guy almost in tears because he knew he would be fired due to his screw up that blew up the bill. It made me proud to help him. Amazon has a ton of serious issues but they are genuinely incentivized to care about customer problems (I.e., you are paid and promoted on how far you go for customers).
- 0cf8612b2e1e 4y ago
  > We would rather reverse your $20k bill and keep you alive And as a customer, I never want to be in a position where I ever see $megabill. Maybe I might be able to get reversed if the gods smile upon me that day, but it would be incredibly stressful until I had resolutions. Seems an obvious blindside for “customer obsessed”. Anything I might host is practically a joke for which I would gladly have the provider pull the plug if billing suddenly spiked above pre-established threshold. I’ll take the downtime if it helps me sleep at night.
  - wingleader 4y ago
    I've had the exact polar opposite experience on any company running systems with paying customers. Usually it's "don't care what it costs, just fix it" which lines well with AWS' behaviour on this. When downtime costs tens of thousands of dollars _a minute_, people tend to stress less about some inefficiencies like this
    - 4y ago

twawaaay 4y ago

> I wish I chose AWS. Well... AWS has better support IMO (like actual people you can talk to). But I still chose to not tie myself to any of these platforms for anything critical. It requires additional upfront effort, but I can migrate all my applications off of AWS in a moment. There is no easy way around Google and Apple when it comes to your app staying available, but I am trying to minimise any chances I could be flagged for anything (basically not linking those accounts to any other services).

joshxyz 4y ago
im actually surprised aws gives you live chat support even at their free tier, pretty much why i also love namecheap.
x86x87 4y ago
being able to migrate != support GCP is not going to be a thing in a few years. Why would anyone pick GCP at this point with all the horror stories coming out of there? Yes, support is hard, yes it's expensive, yes it takes time. But at the end of the day this is how you make or break the trust of your customers.
sebazzz 4y ago
> But I still chose to not tie myself to any of these platforms for anything critical. It requires additional upfront effort, but I can migrate all my applications off of AWS in a moment. Then you are probably not doing "cloud native" development using their proprietary SDKs, so probably just running some EC2 (virtual machine) instance and RDS (database) instance. That is about the most expensive way you can use the big-tech clouds, at that point you might as well move to the smaller hosting providers like Linode.
- npteljes 4y ago
  How well do smaller / more traditional providers support auto scaling and various other "elastic" features of AWS? I only have experience with renting a "traditional" VPS, and some of the more modern AWS offerings.
- 4y ago

dmitrygr 4y ago

Same thing happened to me: “suspended, need extra verification” after YEARS of the account existing just fine. Nothing helped. Not doing what they asked (multiple photos of photo id), not emailing, nor calling support. I ended up calling an ex-coworker who still worked at google. He sent an internal email. It was resolved the same day. I never found out what the issue was. Don’t use google for anything you care about, unless you maintain close friendships with googlers.

pteraspidomorph 4y ago
That doesn't always work, (following anecdote is not about google cloud): someone I know lives in a street that's egregiously and very obviously broken on google maps (whole network of streets mapped with the same name, private pathways mapped as public streets, whole bunch of repeated door numbers, etc) and since everyone trusts google maps for everything this has caused a bunch of serious issues for the person as they interact with businesses and even government. Any attempts to get this fixed through the normal process get ignored (I have personally tried, several years ago, twice). Over the years I asked two different friends at Google for help, one of whom was management. They were baffled when the internal message thing yielded no results either. There doesn't seem to be any process for getting certain types of issues resolved at google. My point is that google can ruin people's lives even if the victims aren't the ones using it. Gmail blocking inbound mail is another example - if gmail decides to block your service, everyone will blame you, and there's nothing you can do about it other than wait.

Trencin 4y ago

It seems like a large overhead, but making your app multi-cloud compatible to counter for scenarios such as this might be not be as crazy as it sounds. 99.99% availability doesn't cover for scenarios like this.

dahdum 4y ago
I'd just make a backup AWS account with separate billing before I worked on multi-cloud.
- Trencin 4y ago
  Could work, but if you're being targeted for some reason it's likely they would find your other account.
- sparrish 4y ago
  That's short-sighted and poor engineering. You need your service to be able to keep going no matter what any single vendor does. Never intentionally create a single point of failure like that.

throwaway892238 4y ago

Google locked our company out of using our own domain on Google Workspaces because a former employee had signed up for a Workspace account using our domain, and we had no way to recover it. We literally manage our domain in GCP but they wouldn't accept that we own our own domain and thus won't let us use our domain with Workspaces, which screws you out of a lot of Google functionality. Never seen a company that was too incompetent to allow users to use its own product and make zero attempt to solve the customer's problem. Google Cloud CEO Thomas Kurian said "We need scale to be profitable." No, you need to stop treating your customers like crap to be profitable. We don't trust you, you are a pain in our asses, and we have alternatives, so there is really no need for us to spend our money on you. It's now a matter of when we all move to AWS and Azure, not if.

userbinator 4y ago
It's now a matter of when we all move to AWS and Azure, not if. If you really think that the "grass is greener on the other side"... No, the real solution is to stay away from Big Tech and cloud in general.
- jiggawatts 4y ago
  I’ve never heard of anyone getting locked out of their Azure account like what seems a regular occurrence with GCP. They have human support and have proper Enterprise Agreements that don’t turn off your business because some heuristic tripped.
  - bozhark 4y ago
    This is by Google’s design. You have to get enough traction on social media to get on someone important at the company’s feed
- seanhunter 4y ago
  That’s completely contrary to my experience. Running a small startup we encountered all 3. For us, MS/Azure were the most eager to please but in many ways the worst from an execution standpoint. GCP were very good from an execution standpoint but absolutely zero support and stories like this one scared us off. Lots of big GCP corporate clients actually use them through consultancies which essentially provide the support and are large enough on aggregate to get listened to by Google. AWS were good at actually doing things and also offered support the couple of times we needed it.

autotune 4y ago

Care to divulge a general idea of what these apps were before expecting sympathy?

fnordpiglet 4y ago
Why does what the apps do matter? The dude has software that puts the sushi on the table and google peed all over the table.
- autotune 4y ago
  I want to know what the ban was for. What if the ban was due to a crypto mining operation? All I am asking for is a little more context before pointing any fingers.
  - justinwp 4y ago
    One of the apps by this developer: https://apps.apple.com/us/app/cuff-video-chat-make-friends/id1514536428 https://apps.apple.com/us/app/cuff-video-chat-make-friends/i...
    - autotune 4y ago
      This does not seem worthy of a ban on the face of it. My sympathy goes out to OP.
      - shagie 4y ago
        The 18+ video chat is one of them... and (apparently) on the same domain is https://apps.apple.com/us/app/bitalert-bitcoin-ether-alert/id1577343755 https://apps.apple.com/us/app/bitalert-bitcoin-ether-alert/i... (note the privacy policy and the terms and conditions and look at the top level service). Having a service that is getting network traffic for both crypto related things and 18+ dating may have raised some algorithmic alarm bells.
    - mathverse 4y ago

f0e4c2f7 4y ago

Wow I'm really sorry this happened to you. I do believe your conclusions are correct about AWS. Azure generally seems fine too. Google's aversion to customer service makes them extremely dangerous as a cloud provider in my mind. This also goes for other critical business services, like Gsuite (I know! It's convenient). If you have GCP or Gsuite now and you're trying to evaluate how big of a deal it is my suggestion would be to pick up a phone and attempt to talk with someone at Google about your account. This experience can act as a preview of what that process might look like when services are turned off. If you try to call Amazon on the other hand it feels like Jeff Bezos might hop on the call if things aren't going well.

roxgib 4y ago
I've had some bumpy points with Azure, but if I really can't fix something I can pick up the phone and they'll try to help. The idea of having critical business services with a company that you can't actually talk to is terrifying.
orbz 4y ago
I know it’s just anecdotal, but once you have some spend (around 1k a month I’m guessing), you get an account executive who I’ve been able to reach on the phone whenever I’ve needed. In fact they tried talking to me on the phone more than I wanted.
- more_corn 4y ago
  Google engineers are not known for being super available though.
  - orbz 4y ago
    My rep was able to get me connected to a few tech leads on various cloud products and one time when we were hitting weird quota limits the team protectively reached out to me to ask what the heck we were doing. Maybe we were just lucky, but I’ve had similar experiences in AWS and Azure too.
4y ago

dmak 4y ago

I have had GSuite with my domain for 10 years, so theres a lot of emails and apps that I used with Google auth. I always worry about what might happen if something out of my control on Google's end locked me out.

sitkack 4y ago
You should backup all of your data and ensure that your domain name is paid well in advance and that you can login to your domain name services w/o 2fa hitting the domain in question. You can always point your domain to a different provider and reset if you can recv email.
- dmak 4y ago
  Thank you, and that part is covered, but I am not sure how I could replace Google Login if I point to a different provider.
  - konha 4y ago
    > I am not sure how I could replace Google Login if I point to a different provider. You can’t. You’ll have to rely on the account recovery mechanisms of the services you used Google login with. Some might be able to switch over to email login after verification, some might not even have a manual way to recover your account / switch it to another login method.

stevage 4y ago

That sucks. I don't like Amazon as a company, but AWS support is great. A couple of times in the past I've made some complaint about them on Twitter, and a rep has actually reached out and helped resolve the issue. And I'm a tiny fish, with monthly bills under $10.

x86x87 4y ago
this is not about how tiny you are. it's about having a system in place to fix issues and help your customers.

de6u99er 4y ago

I had this once with Google Drive, when I forgot to enter my new credit card because the existing one had expired. Took weeks to get my files back. It's like people at Google just wait for shit to happen instead of trying to think of the obvious things that could go wrong beforehand. That being said, if Google is looking for people to lay off there's some obvious candidates. I would include their mangaers too.

someRandoJunk 4y ago

I have seen similar stories like these in the past here. It's a shame Google hasn't done anything at all for so long to combat it.

rybosome 4y ago

I am very sad to hear that this happened to you. It must’ve been absolutely maddening to be unable to talk to a human being as your business was destroyed.

alirsgp 4y ago
I haven’t had anxiety like this ever before. I’ve worked all day to contact as many ppl as I could. I wish Google would separate their billing from Google Pay and not have this dependency.

MattyMc 4y ago

Upvoting in the hopes that someone on HN has a connection internally at Google, AND to raise awareness for this issue. My business uses Firebase, and I'm having serious second thoughts about that right now.

ryandrake 4y ago

Sigh. It's now been [0] days since the last "My [business | e-mail | critical thing] depended entirely on a single service with no backup, I got suspended, and now I can't do my [business | e-mail | critical thing]!" story. I do sympathize, and the right fix is for these services to have better support, but come on... These stories happen so often and get ample press. They should not be surprising. Please stop doing this, people! If something you rely on for your livelihood depends on a single company as a single point of failure, that is a flaming, burning emergency that needs to be solved right now!

sparrish 4y ago
I'll go one step further and say that if your website/app/service can be taken offline by any issue with a single vendor, it's more than an emergency, it was engineered incorrectly from the start. Are new devs taught that having something in two AWS zones equals redundancy? You drank the Kool-aid, people.
x86x87 4y ago
nope. nope. nope. are you supposed to write your own firebase? develop your own android? yes, OP learned a hard lesson about Google, but that does not mean that you cannot build stuff using just one provider. you need to do your due diligence of course. You need to have a plan, of course. But the cost of doing this on 2 platforms skyrocket. And especially for a small player the multi-cloud, massive investment is just not an option.

justinwp 4y ago

Based upon some of the submitter's previous submissions. It appears that they have some 18+ "dating" apps... Disclaimer: Googler, not in this area.

nextaccountic 4y ago
Like Tinder? Why does this matter?
fbdab103 4y ago
And we can only be outraged at the situation if the author built a business in a field you endorse?
the-angry-dome 4y ago
This does not matter. If they do not violate Google's terms of service, then Google is in the wrong. Disclaimer: Xoogler, who worked on Dasher and other paying-customer-facing tools.
trenchgun 4y ago
So what? Why does it matter?

fnordpiglet 4y ago

I’m sorry. Are you able to work with your customers to keep their business while you sort this? I would port your stuff to aws. Most services have close equivalents and it is probably worth while to try. As another poster noted it might be useful to have cross provider redundancy. I’ll be honest I have refused to work at gcp and have steered the mega corps I’ve advised on cloud choices explicitly away from gcp. Azure is better, but less technically sophisticated. It’s still a one shop cloud. And any company whose core business is surveillance and advertising won’t care about their customers - you’re just meat for their machine. Amazon was built around a customer centric business (retail) and aws inherited the ethos of that retail business. It’s technology is marginally less sophisticated than googles, but googles technology comes with an embarrassingly large amount of smugness and clever solutions that fail in clever ways. Amazon basically just wraps up common technologies in a managed control plane (aside from a few primitives like kinesis, SQS, sns, and DynamoDB) and doesn’t infect everything with their smartness. Good luck.

diebeforei485 4y ago

Paying customers should absolutely have access to customer service.

andrewstuart 4y ago

No one from Google ever turns up here and says they care and it matters. It's now broadly known that developers and businesses don't trust Google, but strangely as far as I can tell, the google CEO has never been held to account for this.

trenchgun 4y ago
https://news.ycombinator.com/item?id=33738014 https://news.ycombinator.com/item?id=33738014

ChrisMarshallNY 4y ago

I hear about this kind of thing, all the time. I guess they manage to not do it that often, so I guess I hear only the bad reports, but they are universally awful, when I do hear them, and they all have the same theme: A.I. Broke it, and the [rare] humans in the loop are powerless to mitigate the A.I.'s breakage. I was considering using Firebase, when I started the project that I'm working on, now, but got cold feet, and wrote my own server. It took a while, but I'm glad that I did.

andrewstuart 4y ago

Even if its hard, your business should have a strategy for coming back up when a critical vendor cancels your account. Not being able to survive a cloud vendor cancelling you is the 21st century equivalent of not having any backups.

sparrish 4y ago
You still need to have backups, even in the 21st century. Being brought down by a single vendor is poor engineering.
- sebazzz 4y ago
  By convenience, backups are often stored on the cloud provider. Yes, this violates the rule of 3, but cloud providers often don't make it easy to do otherwise.

themihai 4y ago

I think this is to be expected(sooner or later) for small businesses dealing with "big tech". You are just a small number in an ocean of "users" so suspending you is really no different than suspending your twitter account. Start taking some responsability folks! Let's stop feeding these walled gardens!

toyg 4y ago

> I feel like I'm a man with literally nothing to lose and I can't live with myself anymore. Dude, I know the pain, but it's only money. Life goes on. If anything, now it should be your mission to stick it to the faceless corporation, not to give in to dark thoughts.

rygar 4y ago

Hi, I am sorry that this is happening to you. Please send me your info to my corp email - rygar@google.com - I'll do my best to help.

alirsgp 4y ago
I just sent you an email, thank you, please let me know anything I can provide you in email to help.
jiggawatts 4y ago
To quote a HN user talking about another similar instance of a cloud service failing to support their customers: > “You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.” From: https://news.ycombinator.com/item?id=20066239 https://news.ycombinator.com/item?id=20066239
x86x87 4y ago
not all heroes wear capes, but look at this person helping out on thanksgiving. this is both amazing and sad (that is necessary to do this to get help)
- MrMan 4y ago
  its nice on a personal level but a sign of how broken things are at GCP that an angel has to emerge from the clouds just at the last moment to save your livelihood? that is super creepy and not the way the information economy is supposed to work, in my opinion.
  - x86x87 4y ago
    it is. this does not diminish the nice gesture though.

musk_micropenis 4y ago

Conversation in this user's comment history. Sounds like he knew the risk he was taking and knew what he'd need to do if it came to that. ----- alirsgp on Oct 24, 2021 | parent | context | prev | next [–] | on: 5 side projects in 6 years, earning $0 I've published ~20 iOS apps. Only 4 make money, but they allowed me to reach financial independence and never have to work for someone gain at the age of 24. krater23 on Oct 25, 2021 | parent | prev | next [–] What's when your apps are removed from Apple for any reason? Do you have enough money to live without this income? Or do you need then just start over again? alirsgp on Oct 26, 2021 | root | parent | prev | next [–] I'd have ~2 years and not hurting my lifestyle. Hopefully it doesn't come to that, but I can get a job if needed

grog454 4y ago

My first experience with AWS was around 2013 when I accidentally left a (cheap) ec2 instance running for months after I was done experimenting with it. Support refunded me no questions asked. Fast forward 2 years and that experience was an important factor in my decision to run servers costing 6 figures annually with AWS over their competition.

ycombiredd 4y ago
I came to write a very similar story involving a lazy error on my part (no error checking on a shutdown ec2 instance script, resulting in a 64 Xeon, 8 Nvidia Tesla server running 24/7 for about a month) and an astronomical bill resulting. I had a rudimentary batch job system running. Since I was charged for each power event, I wanted to minimize the power state changes and just queued up the work that needed GPU acceleration until there was enough to justify the cost of at least an entire hour and two power state events. When the cheap, small, always on instance received all the work output from the giant beast server, it would then send a power down command using the aws cli. Yeah, like I said, I was lazy so I just sent off the request and small server just went on about it’s business. Oops. Like I told the aws billing rep, it would have bankrupted me, ending that venture. I asked that, since it is all virtualized, that while yes I had the virtual hardware provisioned for all that time, if they could get someone to look at the actual utilization they would see it was essentially none. They got back to me and in a day or so to let me know they had forgiven the entire amount. AWS support actually listen, do real work, and very often it seems, do what is in the best interest of the customer even as it works directly against the company’s bottom line.
thirdusername 4y ago
When I was early in my career freelancing my AWS account was shut down for no payment for over a year, more than half a decade later I re-opened it through support and they wouldn’t even let me pay my old bill. I’ve been able to reduce reserved instances when the market has turned unpredictable. AWS isn’t a pushover, not even a little bit. Dealing with them externally has just been a good experience with professional understanding and a flexible business sense.

mrslave 4y ago

What are the alternatives? Is Bare Metal with OpenStack or K8s any more resilient to administrative DoS? Do we simply not care about this problem because the businesses being suspended are small fish?

db48x 4y ago

You have to sue them. If nobody sues them, then they will simply keep doing it.

nextaccountic 4y ago

Now you have posted here on HN, I'm curious to know if Googler saw this post and escalated the issue.

alirsgp 4y ago
Yes, they did and I am awaiting a response. Being thanksgiving holiday might be slow

rufius 4y ago

Google - “What kind of cloud would a clown make?” I just can’t fathom Google’s inability to sort this out. They’ve been the laughing stock of serious cloud business for years.

x86x87 4y ago
At this point I am having trouble understanding who would consider GCP. I mean maybe something along the lines of: something something, multi-cloud strategy, no lock-in options, let's also use GCP so we have multi-cloud. But I never see it as the option to pick. It's sad really.
- Terretta 4y ago
  > trouble understanding who would consider GCP 1. SaaSy kids in SV or who wish they were in SV who also think Google Sheets beats Excel. 2. Big enterprise executives who heard Google was what the cool SaaSy kids do from someone in I.T. then got lured by too-good-to-be-true discounts from Google Cloud sales team as long as the executive is happy to be featured for their forward-thinking tech chops in pressers about picking GCP. To their credit, they will undo the Office to Workspace migration mandate almost immediately, after making their 140,000 users attempt it from IE 10 on Win 7, but there won't be press about that.
  - x86x87 4y ago
    > Google Sheets beats Excel I loled out loud
joshxyz 4y ago
Hahaha Google Clown Platform

gmbsuspended 4y ago

Sadly we hear so much of this on a daily basis as we are often sought out to support suspended Google Business Profile (GBP) accounts. It's not uncommon to have these same small to medium sized business owners ask for help with other suspended accounts in other Google apps like GoogleAds or Google Merchant Center. The support across all platforms is a growing eyesore for Google, and at this point I hope becomes the catalyst for real change in this space. It was bad enough when Yelp would act like the Mafia in regards to how they approached and treated businesses, but this level of negligence (from such a tech giant) is something to really see. We've been advocating on behalf of all these small businesses for years and truely hope that the outcry grows to a fever pitch where no more band aid solutions are used anymore.

iJohnDoe 4y ago

It’s not AI. It’s a jealous contract worker in India or a jealous Google employee because you were making money with your app. We really need to stop blaming a “Google AI”. Google has 118,000 employees. Someone at Google could help you if they wanted to, but they don’t want to.

deleted 4y ago

[deleted]

onphonenow 4y ago

I’m missing how a google pay account issue affects GCP? Use a different credit / debit card?

alirsgp 4y ago
GCP like all Google services force you to use Google pay. I have 4 different payment options there, but they are suspending me until they verify my ID. This is a big problem in cloud services since being down for days or weeks or longer can destroy apps and their standings in the app stores.

Zuiii 4y ago

Thank you for reminding me why I avoid Google services like the plague. Even relying on unmanaged apps on a single pet server is less risky than relying on Google. It's like playing russian roulette with your infra. Google Roulette.

speedgoose 4y ago

In the beginning of the year, I was recommended by a friend working at Google to test GCP using the tutorials on https://www.cloudskillsboost.google/ https://www.cloudskillsboost.google/ Google banned my account while I was doing one tutorial. One step failed, the provisioning of something, and I had to redo a few things again. It succeeded but I had used more egress bandwidth than allowed to run the tutorial correctly so they banned me automatically a few minutes later. The support unbanned me after some time but I thought it was a sign to not use GCP yet.

more_corn 4y ago

I too am currently in Google Pay support hell. Never in my life have I experienced such a Byzantine hell of weaponized incompetence. I’d post the thread but it’s just 48 exchanges of me calmly and patiently explaining the situation. And them failing to read the case notes, saying platitudes and then pasting the same non-useful canned response. Here’s a fun fact or two: Did you know there’s no way to get your money out from the web interface? Did you know there’s no way to select your profile id on iOS? These two features haven’t been implemented. If you’re clever you can reconstruct the rest. Their documentation indicates that they also don’t have an account closure reconciliation process. I’m hoping that’s just inaccurate documentation because that’d be illegal. Good times. Some states attorney generals should band together and sue those jerks.

albertopv 4y ago

I see many blame OP for relying on one single cloud provider without having backup and exit strategy. What if she/he is a single persone doing everything and having just barely the time to run the business? It's the same mentality I have seen from people complaining about Phoronix quality, ignoring It's a one man show website.

alirsgp 4y ago
Yes, I have a team of contractors I pay but essentially a one man show.

snird 4y ago

Google and GCP isn't capable or want to provide support to customers who pay less than ~$20k a month. Don't ever use GCP if you're not a big enough enterprise. Just as you should never buy a Dell laptop as an individual consumer. I'm sorry this happened to you, I hope someone will learn from your experience

DarrenDev 4y ago
Why should you never buy a Dell laptop? What can go wrong with that?
t00l00 4y ago
I’m £7.5k/mo in UK: have a named account manager, and an engineer who spends 2h/mo discussing whatever I want on a Google Meet, and bugs I can email directly. Got invited to their Leeds office to say hi and spend a day with them.

mathverse 4y ago

The difference between AWS/Azure and Google is that the former run a business. Google is a cult with a subscription fee.

thiht 4y ago

There are horror stories like this with all of Big Tech, but Google is hands down the worst offender. I do my best to avoid Google in every corner of my life: the only services I still use are Gmail (slowly migrating to my own domain with another provider though) and Search. But I’ll never use a new Google product anymore because of their unreliability on the long term.

splix 4y ago

Same happened to me too. They suspended the payment account. Two accounts actually, because different cards. Sent the driver license picture a dozen times, for one account someone eventually took a look and unlocked it (which took about a week). But another one is in review for several months already. And it's impossible to do anything with that.

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

bfergsthesnitch 4y ago

Hey ali, you should know daniel is ridiculing you on another server.

csisnett 4y ago

I'm so sorry this happened to you this is why is important to not have a single point of failure. Have domain name in X company, hosting in another, etc. That way if one turns crazy there's More hope

t00l00 4y ago

Does this support depend on a region? I’m here in UK, my bill is sub £10k/mo and literally today few hours ago I had an hour with Google engineer discussing use of their MultiClusterIngress to mix GKE standard and Autopilot to handle spikes extremely fast. Afterwards dude sent me docs and his project he used as an example. It’s the second time this month: we don’t even chat about any specific bugs, more about my architecture goals. I also have an account manager and she’s currently working on making discount rates from our contract to work with our new hierarchy of multiple billing accounts, some VAT paying and some not, that my finance department requested. You also can have backup payment methods? I think we have DDs as a default and then CCs as backups.

heresjohnny 4y ago

I was tempted to use Firebase for a new project because it looked so easy. I’m fairly fluent in Angular and it integrates extremely smoothly, so at a surface level it would be the perfect choice for the backend. Stories like yours – thank you for sharing – plus horror reports about sudden 20k bills have prevented me from taking the “cloud” leap. I’m very glad I didn’t, especially because I also learned lots by setting up a web server and the application layer myself.

alirsgp 4y ago

I just want to note that I was able to solve this issue. Thank you to the googler who reached out to me to help, 'rygar', and the rest of the google cloud/pay support team for helping fix this issue.