11 ms·
Rogers network outage across Canada hits banks, businesses and consumers
- AzzieElbab 4y agoI am in Toronto and I don’t use Rogers. Nevertheless Rogers slows down 4g traffic for most providers. It is fun watching how tones of services and sites simply stop functioning when internet slows down
- bob1029 4y agoI used to work in a debit processing command center where we would monitor all of the acquirer and issuer links in real-time. The amount of paperwork and phone calling required just for a <60 second MPLS VPN drop-out was pretty incredible. I can't imagine a day like this in that room. Would have been an absolute circus.
- cluoma 4y agoNot a good week for internet infrastructure in Canada. Yukon Territory had essentially no internet for most of this Wednesday after the only fibre was broken.
- walrus01 4y agothat, at least, was a very canadian outage caused by a beaver chew it's hard to build resilient rings of fiber between towns/cities where there is only one linear path (road) for right-of-way that you could economically build the fiber along...
- ilrwbwrkhv 4y ago
- seibelj 4y agoIt's a great weigh-station for immigrants who want to eventually move to the US. Canada is easier to immigrate to as a high-skilled person.
- ezekiel11 4y ago> Canada is easier to immigrate to as a high-skilled person. False. > It's a great weigh-station for immigrants who want to eventually move to the US. Many do not, they move back home.
- acchow 4y agoMany people who don't win the H-b lottery, or who do win the H1-b lottery but get turned down for an H1-b for other reasons, end up heading to Canada. Canada has a simple points-based residency system and it is very easy to get enough points if you have a bachelor degree and speak english fluently (already gets you a TON of points).
- ezekiel11 4y agoyou realize there are many many affluent immigrants who do not participate in the labor market but pay taxes, invest, and otherwise provide liquidity for the locals holding real estate portfolios?
- ezekiel11 4y agoI know many countries have problems but Canada is in a okay spot. Specific cities have looming crisis on their hands but this is largely the fault of provincial governance. Little of that I attribute to Canada as a hole, I just find BC in particular to be quite mediocre in terms of value compared to what you pay.
- brailsafe 4y agoWhy do you say that? I'm quite happy paying what I do renting in Vancouver. I usually find that people who actively want out of BC either desperately want to own property and can't, don't have any hobbies that suit the west coast, or can't make friends
- LegitShady 4y agowith the massive property price increases that have taken place Canada has become even more mediocre than ever before. With the increases we've seen some movement out of Toronto and Vancouver areas into other areas of the country, driving up prices even more. Young people are basically out of luck in terms of hopes of ever owning property. I know multiple immigrants who are talking about returning to their previous country, or trying to figure out getting legal in the US. There isn't enough housing nor enough being built. Discussing immigration is not allowed, socially speaking.
- 4y ago
- randlet 4y agoThe lack of communication from Rogers for an event of this magnitude is quite something. Three tweets[0] with zero useful information so far today. The first one not coming until ~4 hours after the interruption started. [0] https://twitter.com/rogers/ https://twitter.com/rogers/
- deleted 4y ago[deleted]
- prvc 4y agoYour mistake is assuming that their twitter account is for providing useful information to customers, rather than signaling their allegiance to state-sanctioned social and political causes.
- theIV 4y agoThey probably can't get online to tweet. /s
- jonny_eh 4y agoSeriously though, someone working there had to find a friend with a Telus hotspot to borrow, I imagine.
- alphaomegacode 4y agoThe timing of that update tweet is what gives pause for concern. If it were the same or similar thing as to what took down their network nationwide in April, it seems reasonable that they'd just say something like "Hey guys, sorry but the network is down due to <insert similar cause here>." A lot of us are in IT and for a network of national importance in a "developed nation" to be taken offline so easily is worrisome.
- zeagle 4y agoHopefully this will contribute to a regulatory denial of Rogers' upcoming purchase of Shaw, if only to keep a competitor with its own separate infrastructure to minimize outages through consolidation.
- brailsafe 4y agoomg I didn't even know about that. How could that be allowed to go through
- MR4D 4y agoFTA - "Rogers has not identified a cause." Seems like a rather large impact. Maybe Russia really wants its gas turbine back? [0] [0] - https://www.reuters.com/world/exclusive-ukraine-urges-canada-not-hand-over-gas-turbine-russia-2022-07-07/ https://www.reuters.com/world/exclusive-ukraine-urges-canada...
- ilrwbwrkhv 4y agohttps://blog.cloudflare.com/cloudflares-view-of-the-rogers-communications-outage-in-canada/ https://blog.cloudflare.com/cloudflares-view-of-the-rogers-c... Trust an American company to come up with a clearer picture of what's going on, highlighting the mediocrity of Canadian businesses.
- lupire 4y ago
- Waterluvian 4y agoTrust an American company that did the same boneheaded thing not long ago to be able to explain it well.
- 542458 4y agoIt feels like there’s been a rash of companies messing up BGP stuff lately - Facebook, cloudflare, and now this.
- goatcode 4y agoHaving formerly lived in Canada for 30 years, I can concur.
- jagger27 4y agoIf this doesn't show Canadians that telecommunications are a public utility, I don't know what will. The CRTC's (Canadian FCC) own phones are down[1] today because they're a Rogers customer. If that doesn't scream regulatory capture... 1: https://twitter.com/CRTCeng/status/1545421218534359041 https://twitter.com/CRTCeng/status/1545421218534359041
- nevereveragain 4y agoI mean... it isn't as if the CRTC is going to set up their own telco just to operate their own office.
- exadrid 4y agoYeah, but ultimately they are pretty much owned by the big Telco so we are fucked
- 99_00 4y agoCanadians like to complain about the lack of competition. For wireless Freedom Mobile was independent from 2008 to 2016. Mobilicity was independent from 2010 to 2015. They were always cheaper than the incumbents. However, they struggled to get customers because their coverage and speed lagged and they were eventually bought by incumbents for their wireless spectrum license. The government tried to foster competition by letting them bid on spectrum set aside for new players. And removing strict foreign ownership rules. Canada is a small market. Geographically larger than the US with the same population of California. It's just not an attractive telecom investment.
- betaby 4y agoSounds like those are good reasons to nationalize telecom infra.
- brutus1213 4y agoI don't understand how this is not on CNN/BBC news. This isn't the top news at the moment but significant .. I don't know if people in other countries know how this feels. People are saying this is pre-1990 .. but it is not really. People don't have landlines any more .. all our media is streamed via Internet. Stores can't take payments or orders. People can't call Ubers/taxis Millions of people don't have cellular. 911 issues have been reported. It has been over 12 hours and issue is not fixed. This video shows a senior person at Rogers giving an update on the situation. It feels like I am in the twilight zone: https://www.youtube.com/watch?v=MYfRZZaPrC0 https://www.youtube.com/watch?v=MYfRZZaPrC0
- cmrdporcupine 4y agoRight? I saw that and I was like... who gave this guy the right to talk to the media? If I were corporate communications at Rogers I'd be freaking that this person made it onto camera with this crap (Luckily my phone is Freedom and my Internet is an independent small Hamilton local ISP.)
- voisin 4y agoAren’t a lot of smaller ISPs going out of business right now because of the CRTC’s decision to allow the big 3 to charge more for wholesale rates? I recall TelSavvy being going outspoken about this.
- cmrdporcupine 4y agoI'm not sure of their business state, but yeah, it's not good. For me, I'm rural and I use a 900mhz point to point wireless connection that caps out around 15-20mbps on a good day. The upside is it's provided by a local old school indie ISP ("NetAccess"). It's $130 CAD a month, which is... expensive... but unlimited which is far cry from the theft that Rogers was performing on me prior with the 3g (then LTE) connection I had that at first capped me at 25G per month @ $100 a month and eventually went up to a "generous" 100G and then charged insane overage fees. My teen daughter ran us up a $400 bill one month. Bell and Rogers are now offering more competitive rural options (at the behest of the gov't) but I refuse to give them any more money.
- jollybean 4y agoI believe this is a Russian hack. Some group took credit for it. It's an act of war. I'm wary we may not find out for a long time. Debit cards were out, huge swaths of wireless and internet, this was a huge failure.
- dade_ 4y agoI always assume incompetence before malice, for Rogers, that goes triple. It is a shit network build on layers of shit. It is owned by a crazy family in a perpetual feud that could have inspired Succession. https://www.ctvnews.ca/entertainment/succession-actor-brian-cox-congratulates-edward-rogers-for-ousting-former-ceo-1.5736919 https://www.ctvnews.ca/entertainment/succession-actor-brian-... https://www.wsj.com/articles/rogers-family-feud-creates-boardroom-chaos-for-canadian-communications-firm-11635343453 https://www.wsj.com/articles/rogers-family-feud-creates-boar... Rogers and Bell executives are more interested in their ego than their networks and use their oligopoly money to buy sports teams: https://www.theglobeandmail.com/sports/no-love-for-blue-jays-in-rogerss-sports-spending-blame-an-accounting-rule/article28709405/ https://www.theglobeandmail.com/sports/no-love-for-blue-jays... Last time this happened, they blamed Ericsson, but let’s face facts, they aren’t competent: https://mobilesyrup.com/2021/04/20/nationwide-rogers-outage-was-caused-by-ericsson-software-update/ https://mobilesyrup.com/2021/04/20/nationwide-rogers-outage-... Their cellular business used to be called Cantel 30 years ago and we called it Can’t Talk. Their CIO is from Vodaphone. https://about.rogers.com/team_bios/jorge-fernandes/ https://about.rogers.com/team_bios/jorge-fernandes/ They have no oversight because the CRTC executive are best friends with the big 3 telcos: https://www.iphoneincanada.ca/carriers/bell/crtc-chair-on-drinking-in-pub-with-bell-exec-just-beer-with-a-friend/ https://www.iphoneincanada.ca/carriers/bell/crtc-chair-on-dr... Interac (Debit) apparently has their primary and backup network services with Rogers. They need a new executive team ASAP.
- throwaway742 4y agoI too think this is more likely to be a mistake, but incompetence also makes you easier to hack.
- EForEndeavour 4y ago
- mensetmanusman 4y agoWould Starlink work in Canada during an outage like this if the base stations don’t have internet?
- ttul 4y agoI don’t know but this makes me want to get out the Starlink and try. It is my disaster backup.
- kejaed 4y agoStarlink worked throughout the Rogers network issues for those I know using it in Canada. From what I understand of their current network setup, if their base stations in a region had no connectivity then the starlink customers in that region would lose their connectivity too. This is likely going to be able to be mitigated when Starlink inter-satellite communications is up and running and they can route packets in space.
- planarhobbit 4y agoI’m surprised there’s no chiming in from Bell customers. I’m on their Fibe or whatever plan, no issues all day whatsoever. My phone lines, however, are all down, as is merchant processing etc etc. Really strange day but not unexpected. We “know” the telcos are screwing us but we don’t have a way to gang up against them, as people easily buy into their feelgood propaganda.
- ncann 4y agoFrom what I've heard, some Bell lines are really just Rogers lines with a Bell label on top
- planarhobbit 4y agoI would imagine it’s the other way around, but maybe the infra has changed. I always thought it was Bell at the very top.
- swader999 4y agoWell here's one reason not to retire cash just yet.
- Hammershaft 4y agoJust wait until Rogers and Shaw finish their merger so that the entire country can be plunged into the stone age every time a technician trips over a wire in Toronto.
- cpncrunch 4y agohttps://www.cbc.ca/news/business/rogers-outage-cell-mobile-wifi-1.6514373 https://www.cbc.ca/news/business/rogers-outage-cell-mobile-w... "we're getting very close to understanding the root cause of the of the failure. And we're taking actions along with our network vendors to recover the situation." "We don't understand how the different levels of redundancy that we build across the network coast to coast have not worked," he said.
- Element_ 4y agoLate this afternoon a Rogers Executive in charge of the response gave an interview to the CBC. He said they hadn't even identified the root cause of the issue at that point (~12 hours into outage) https://twitter.com/PnPCBC/status/1545512971878662145 https://twitter.com/PnPCBC/status/1545512971878662145
- bonestamp2 4y agoThat seems reasonable if service is still not back to 100% -- they need to focus on fixing the problem. They can investigate what caused the problem later.
- mgbmtl 4y agoI think my weirdest experience was a server that was jammed on "apt update". I looked up the IP using whois.. and it belonged to Rogers.
- bonestamp2 4y agoWhere was the server?
- mgbmtl 4y agoThe server was setup to use ftp.ca.debian.org, which currently resolves to 207.210.46.249, and the whois for that mentions Rogers.
- ChoGGi 4y agoI see it seems to be a day for shitting on Canadian telecoms. May I say a hearty "Up yours CRTC". Seems as good as any time to share a Rogers story with HN: My mum passed away in 2009, she owed Rogers $200 and change. We took over her contacts/mail forwarding/etc to tie everything up. A month or two afterwards; we got a call from Rogers looking for their money, explained she passed away, they said how sorry they were and figured that was the end of it. 3/4 months after that we got a call from a collections agency, mentioned her passing, got apologies and hang up. This then continued every few months bouncing between Rogers and a collections agency. For some reason they kept passing the debt back and forth? The frequency was low enough, that we never bothered going any further with it, after a couple years it was less annoying and more a nice reminder of mum :) It took about 6-7 years before they finally clued in, and that's why I'll never have anything to do with Rogers.
- akg_67 4y agoIt seems like some sort of hardware/software rollout going wrong. We had similar outage last weekend in Japan with #3 service provider KDDI/AU, a 86 hour outage of both cell and data service. Lot of services like weather stations went offline, people were unable to reach/use emergency, banking, barcode/app based payment services. Some ATMs and Vending Machines went offline. Mobile and data service has become quite embedded in daily life.
- deleted 4y ago[deleted]
- stevenjgarner 4y agoClearly a correlation between 2 hard drives failing at HN [1] and the Canadian Rogers network outage [2]. [1] https://twitter.com/HNStatus https://twitter.com/HNStatus [2] https://www.reuters.com/business/media-telecom/rogers-communications-services-down-thousands-users-downdetector-2022-07-08/ https://www.reuters.com/business/media-telecom/rogers-commun...
- busymom0 4y agoAfter around 20 hours, it just started working for me. I am with Fido for mobile and use Tecksavvy for home internet but both of them rely on Rogers at the end, so both were down. Walmart was only accepting cash or credit card. Debit cards were not working as Interac was down.
- motohagiography 4y agoJust got back online a day later. Progress on side projects aside, the day was a write off. In terms of causes, I think it was one of a few possibilities in order of percieved plausibility and likelihood: 1. newb engineer updated router configs using a CI pipeline that caused them to stop talking to each other. 2. standard mistake of redistributing iBGP with a bunch of static routes for load balancing into eBGP causing their internet facing ASN to announce the internet, and it took down their internal MPLS routes for backhauling their cellular traffic over IP. 3. geopolitical situation where someone was making a deniable example of their capability against a target who can't fight back. That was just a friday in Canada, it's just as easy to do D.C., Chicago, or New York. 4. a political stunt and pretext for "a safer internet" with "more oversight," orchestrated by party hacks, or an exercise to see how effectively they could do this. Least likely, but that's the level of trust there is in this govt right now. If you don't want conspiracy theories, try being credible enough people don't have to invent explanations because nobody believes you are in control. I'd bet on 2 or 3. Items 1 and 4 are weak ideas, but 4 will likely exacerbate confirmation bias. What will likely not come out of this is liberalization of the telecoms market to add competition. However, regardless of teleological fallacies, I don't see this as a crisis that will be left to go to waste. It has been a good fire drill for business continuity and preparedness anyway.
- joshspankit 4y agoInterac. This outage was eye-opening. You’d think that Interac (a prolific payment method for using your bank account to pay for products that hit the scene early enough to get entrenched in Canadian habits) would be infrastructure-agnostic, but it’s clear that they are extremely dependent on Rogers’ network. All Interac was down. Places still processed credits cards as normal, the machines still worked, and it didn’t matter what ISP was used by any particular store. I do not understand how they were comfortable floating billions in transactions over a network with a single choke point.
- rafiki6 4y agoCanada's main issue is that we've allowed duopoly on ALL telecom (i.e. we essentially have 2 major national communications networks encompassing all forms of telecom, including phone, TV, cell phone, internet etc.) Rogers is one of those networks, and it basically covers 30-40% of the country's communications. There are a few smaller regional players, such as Shaw, Cogeco, Videotron etc. I think we might be the only developed country that's allowed this. I know in the US, there are 5 major mobile networks, some of who offer internet etc., but there are major internet providers separate from the mobile network providers. The reason this has happened, is because building networks is really expensive making it so that being a regional provider is a tough business, and the 3 major companies have played the laws to their advantage. They've done acquisitions over time, and used the rules around foreign investing to block a lot of major development in 3rd and 4th networks from happening. This is why the Rogers/Shaw deal should be blocked and the government needs to allow foreign investment to be able to come in and build out networks. This problem can only be solved in 1 of 2 ways, and only through regulation unfortunately: - A complete overhaul of our rules which will allow foreign investment in our communications networks in Canada - A break up of the 2 networks The second is not likely to happen. Rogers/Bell/Telus are some of the wealthiest companies in the world, and are at the top in terms of deep pockets in Canada. They will essentially fight the government on any sort of attempt to break them up and handily win. The government has a much better shot at changing the rules to allow foreign investment, but they don't want to do it. The reason that's the case is because the leadership of the telecom companies has many shared members with prominent politicians, and also outright owns all forms of media in the country except for CBC, Post Media, and Corus. Post Media is a known right-leaning org and Corus is effectively owned by Shaw, so their sway in the country's politics is not as big as Bell Media and Rogers Media who both own much bigger networks. That means that they can run a very effective smear campaign against any politician or part who dares to allow foreign competition to enter Canada.
- jleyank 4y agoGiven that private business won’t magically reduce their profits to have (more) backups and hire more competent it staff, I guess you all want the internet to return to being a regulated utility? The only really serious aspects were letting debit transactions be run by a single provider along with some aspects of border control. These should have mandated fall over to other vendors to keep running. The banks and credit cards did fine as long as the store was on line.
- drpgq 4y agoI was hoping there would be a discussion of this here with some inside baseball, but then HN was down.
- dredmorbius 4y agoThere's a Wikipedia article, though it's still sparse. It should have post mortem information once that's official. https://en.wikipedia.org/wiki/July_2022_Rogers_Communications_Outage https://en.wikipedia.org/wiki/July_2022_Rogers_Communication...
- deleted 4y ago[deleted]
- sophacles 4y agoThe Cloudflare blog suggests it's BGP problems: https://blog.cloudflare.com/cloudflares-view-of-the-rogers-communications-outage-in-canada/ https://blog.cloudflare.com/cloudflares-view-of-the-rogers-c...
- dekimir 4y agoHow does that relate to the Rogers cell service outage, though? Surely SMS doesn't require BGP? It feels like there's a big, fundamental problem affecting a variety of Rogers infra, where BGP and cell problems are just symptoms we can observe. CloudFlare says it looks more like a failure than an attack; I'm not entirely convinced.
- yardstick 4y agoI don’t think it’s an attack. I’m in Canada at the moment with my UK SIM and using Rogers internet just fine. Completely unaware of any problems until my friends warned me about needing cash and I then double checked and still, phone internet via Rogers on a roaming SIM worked fine. My guess is their voice, sms etc all are IP these days which was impacted by their BGP issues. Whereas my UK carrier roaming gives me a UK IP still, unrelated to Rogers advertised blocks.
- Victerius 4y agoInterac is down because Interac uses Rogers' network. Interac's backup network also happens to be with ... Rogers. Canadians across the country can't use debit cards, regardless of their financial institution. Credit and cash only. E-transfers are unavailable. Fun for people trying to pay their bills online. And that's just the financial side of the situation. Millions of people have lost Internet and phone service. Someone on Reddit wrote: "Maybe the worst part is that they have literally no open lines of communication. No twitter posts. All chat options (twitter, IG, FB, web based) are all unresponsive. Can’t call tech support, as the call will fail. Can’t even call billing or general inquires as it’s the same thing. Shit happens, I get it. But to leave your customers completely in the dark? Brutal. Just brutal."
- bawolff 4y agoIts crazy that interac does not have backup independent connectivity.
- shirononon_ 4y agothis is quite a significant outage and has been ongoing for over six hours now. would love to see a tech write up or blog post or even a Twitter hint but there seems to be absolutely zero transparency.
- randlet 4y ago> ongoing for over six hours now Closer to 12 hours now I think. Certainly it was out at 7am EST this morning when I woke up.
- timbit42 4y agoProblems started to appear around 4am EST and services became completely unavailable by 4:40am EST.
- jml7c5 4y agoIt's quite surprising. I wonder if the technical side and the PR side of the company just aren't sufficiently connected. It would be beneficial to have unvarnished tweets from someone in a technical role, written for technical peers rather than the general public. The press and tech journalists can filter the jargon into something useful for customers. "Dashboards are showing ABC, we're trying XYZ, so far it's looking like LMN", even if the public don't understand what is being said, is more reassuring than a generic statement. It suggests competence and hints that yes, it really is a difficult problem.
- RL_Quine 4y agoThe first withdraw was at around 7:45 UTC, a little over 13 hours at the time of writing this comment.
- nabaraz 4y agoLooks very similar to Facebook's outage. A combination of BGP and DNS problems that started with a configuration change. https://blog.cloudflare.com/october-2021-facebook-outage/ https://blog.cloudflare.com/october-2021-facebook-outage/
- comprambler 4y agohttps://radar.cloudflare.com/asn/812?date_filter=last_24_hours https://radar.cloudflare.com/asn/812?date_filter=last_24_hou... look at the IPV4 vs IPV6 traffic, when there is IPV4, there is no IPV6 and vice versa. Issues announcing IPV6 BGP prefixes and IPV4 at the same time?
- acoard 4y agoWhat evidence are you seeing that makes you think this is happening in the Rogers' case? I haven't seen any diagnostics or analysis by anyone.
- RL_Quine 4y agoRogers did just abruptly withdraw all their routes, then reannounced some, then withdrew them again. It's not dissimilar at least.
- deleted 4y ago[deleted]
- Ardon 4y agoYou probably saw it up thread but since rogers isn't talking this is all we really have: https://blog.cloudflare.com/cloudflares-view-of-the-rogers-communications-outage-in-canada/ https://blog.cloudflare.com/cloudflares-view-of-the-rogers-c...
- pigtailgirl 4y ago-- Started to fall to bits around 4:00am EST - got woken up by my phone non-stop glowing as news alerts about abe flooded in - got out of bed to check the news on my laptop - wasn't working - tethered to my phone & checked BGP - AS812 was announcing bogons & dropping peers like it was going out of style - thought maybe they were doing a config update - went back to bed - was incredibly surprised to see it dropped all the peers & was sitting dead when I woke up --
- tomComb 4y agoPeople in other countries probably don't understand just how powerful - and depended upon - the big telecoms are here in Canada. They have no presence outside the country - they are so bloated and inefficient they couldn't compete - but they are omnipresent here. There are lots of people who depend on Rogers for home Internet, home phone, mobile phone, TV, and home security, not to mention business services that consumers also depend on, like the payment networks that are down. Plus their media properties, which probably come in useful when the government starts to think about allowing more competition or decreasing the taxpayer money being funneled to them.
- Johnny555 4y agoI think people in the USA understand bloated telecoms with monopoly power that means they don't need to compete. In my neighborhood, I have one effective ISP, that is the cable company (who offers phone+internet+cable), the other is an old-school phone company who offers DSL (with bonded DSL, up to 12mbit). I thought that I could get a slow DSL connection to back up my cable, but no, there's some neighborhood line concentrator that means I can't get DSL at any speed. So I really have only one ISP to "choose" from. (well, in theory I could get Starlink, but my area is still waitlisted, as is Verizon 5G home internet)
- edgefield 4y agoHaving lived in Canada and the US, with telecom plans with providers in both countries, I think Canada’s telecoms take it to a new level. To give you a sense of how distorted the market is, it was cheaper for me to keep my US Verizon wireless plan than to use Rogers, Fido, etc. when I lived in Canada!
- jamal-kumar 4y agoI do this. Unlimited (international!) roaming and everything isn't that expensive (Only about 100$) compared to whatever insane amount I'd be paying if I had some ripoff, under 5 gigabyte or whatever pittance data capped Canadian phone plan for no reason. I also maintain a Central American phone plan that doesn't cost me very much either (2$ every 3 months just to keep the number outside of the country and receive SMS/use chat apps basically, but I don't do roaming when out of that region - When in the region, I spend maybe 20$ a month). Since the Rogers website is down (Great sales strategy!) I can refer anyone curious to a portion of their menu from some image I found; [1] 95$CAD (500MB until you hit overage) + 60$ for 'roam like home' (no mention of data, canadians still use SMS way too widely), and then probably more to up that data cap with no guarantee that applies to roaming? Yeah, no thanks! Oh yeah, the cops just install malware on your shit with reckless abandon, too. [2] I see no reason to consider Canadian nationality as anything but a nice passport at this point. [1] https://cdn.mobilesyrup.com/wp-content/uploads/2018/05/rogers-travel-packs.jpg https://cdn.mobilesyrup.com/wp-content/uploads/2018/05/roger... [2] https://www.theguardian.com/world/2022/jul/07/canada-police-spyware-admission-surveillance-experts https://www.theguardian.com/world/2022/jul/07/canada-police-...
- tra3 4y agoStaying with the inlaws that use Rogers' reseller for wired internet. Using Fido for mobile internet access (also by Rogers). Zero internet. It's wild to discover how reliant I am on the internet: - Can't get around this city (no waze, gmaps). - Can't figure out what's going on (AM radio is non stop commercials) - Can't figure out where we might get some internet (maybe WeWork somewhere? How do you find one without internet) This is just wild. Almost as critical as no water, or no power in the winter.
- jonny_eh 4y agoI'm genuinely curious now how you access HN.
- tra3 4y agoDrove around for 2 hours to see if another cell might be up, nope. Stopped a test stop that had internet. Looks like Fido is back up, but the isp is still down.
- antoniuschan99 4y ago680 news is AM radio version of CP24 at least. 680 news is owned by rogers and cp24 is owned by bell lol!
- EngCanMan 4y agoStill no internet at all? I have a suspicion that you found some…
- dogecoinbase 4y agoThe HN downtime was to add a post-by-postal feature
- doubled112 4y agoMy reseller connection (Start.ca) is down, and Telus’ cell phone service becomes near unusable because of the extra load. I expect the device I send this from will be offline again in the next few minutes, but I’m willing to be pleasantly surprised.
- routerl 4y agoIn Toronto, here's a small and non-exhaustive list of affected services: - Point of sale machines. Supermarkets are accepting only cash. - ATMs. Several banks are incapable of dispensing cash. - Public transit ticketing systems. The TTC is effectively running for free, today. - Public bicycle rental stations. - Public parking locations. How long before the public realizes that we are not Rogers' customers, but its hostages?
- dennis_jeeves1 4y ago>How long before the public realizes that we are not Rogers' customers, but its hostages? Won't happen. The average Canadians are one of the most sheep like docile people out there.
- KMnO4 4y agoRemember when we all banded together to prevent Verizon from introducing competition into our market because it would “take Canadian jobs” and “give our hard earned dollars to the states”? That happened because of a hostile attack ad campaign by the big 3.
- AlexandrB 4y agoI'm not sure an aggressive astroturf campaign is the same as "us banding together". There was even an anti robellus site[1] set up. [1] https://realfairforcanada.ca/ https://realfairforcanada.ca/
- voisin 4y agoWhat actionable steps can Canadians take?
- dennis_jeeves1 4y agoWell, elect politicians who encourage free markets, and have the least govt in our personal life as much as possible. Easier said than done. Canadians (like most other people) are too stupid to gauge what policies/politicians are good for them.
- tr1ll10nb1ll 4y agoI'm using a Roger's number in India currently. Unfortunately, I've been using that number for most of my 2FA so essentially, I've been unable to access a lot of services because of this (ones where I didn't use Authy/Google Authenticator). Lesson learnt: use a separate VoIP business phone number instead.
- bonestamp2 4y agoIsn't that susceptible to the same problem though? I mean, couldn't any service provider be down? I avoid SMS 2FA whenever possible, but sadly that's all some services offer.
- abrichr 4y agoSame here in the US. Which service will you port to?
- tr1ll10nb1ll 4y agoI'm just using OpenPhone now for services tied to my startup. At least, I'd be getting verification codes without having to worry about the service being down.
- bibstha 4y agoAre you using Wifi calling to receive 2FA? What would you change to?
- tvhahn 4y agoThe Canadian telecom space is dominated by 3 primary companies -- Telus, Bell, and Rogers. They form an effective oligopoly that is quite detrimental to the Canadian consumer. In the ISP space, there is a bit more competition. Namely, Shaw provides additional coverage in some regions of the country. However, Rogers wants to buy Shaw. You can imagine how bad that will be for Canadians. I do wonder what the Rogers outage is about. Ransomeware? State attack? Something stupid? If anything, it shows how we should not have critical infrastructure centralized. Competition between ISPs is important.
- ju45 4y agoThere's also Vidéotron and Cogeco but these are not Canada wide so .. yea.
- jonny_eh 4y ago> Something stupid? Whatever it is… it's definitely something stupid.
- walrus01 4y agoIn the ISP world, what will happen with Rogers buying Shaw... In the western provinces, AB and BC, Rogers runs a very widespread and strong LTE network. What they do not have is a DOCSIS3/coax and GPON cable TV plant. Nor do they have much terrestrial right of way for aerial plant or underground in conduits, which is where Telus (the historical copper POTS ILEC) and Shaw (the historical cable TV operator going back to the mid 1970s) are by far the strongest. Rogers is a facilities based last mile cable operator/terrestrial operator in Ontario. Shaw runs the landline cable tv networks in most of the metro Vancouver area. And many other small to mid sized cities in the west. Letting Rogers control both one of the largest/strongest mobile phone networks and the only viable land line terrestrial broadband competitor to Telus by acquiring Shaw's cable tv plant is an absolute outrage. To use a USA analogy, it's like if T-Mobile already owned RCN (Astound) in some big part of the country and then proceeded to buy Comcast. Or if Verizon bought Spectrum (Charter/historical TWTC cable).
- kache_ 4y agoAt what point does the anti-competitive nature and monopolistic qualities of Canadian telecoms become a national security threat? Maybe when it destroys the whole Canadian economy for a day
- TheCoelacanth 4y agoNah, that's not enough, but maybe when it does that twice in 15 months.
- honkler 4y agonever. It's a very deeply corrupt country.
- dennis_jeeves1 4y ago>Maybe when it destroys the whole Canadian economy for a day Won't happen. The average Canadians are one of the most sheep like, docile people out there.
- Jonovono 4y agoHarsh, but so true lol
- Thaxll 4y agoSounds like a BGP problem where people needs to get onsite to fix the issue but since they never did the drill they can't connect etc ...
- empressplay 4y agoGoodbye Rogers, Hello Telus! Glad that Best Buy has a large stock of Telus SIMs. Never again, seriously.
- brandon272 4y agoI see a lot of people noting the lack of communication and updates from Rogers. To me it would be shocking if they were providing updates. Canada’s telecommunication monopolies are not known for their customer service and, even worse, they seem to have internal corporate cultures of entitlement and arrogance that drive an “F U” attitude in general when it comes to external communications and accountability. So you won’t see a post mortem. You won’t see timely updates. You certainly will never see some kind of status page! I’m sure Rogers will be forced to do a post-mortem for large commercial clients where contractually required, but I highly doubt we will hear a word from them about the cause of this or what was/is being done to rectify it. They probably have an ETA but feel no obligation to share it. Rogers isn’t alone. I would expect the same behaviour from Bell or Telus. Canada has serious issues when it comes to its big telco carriers and it represents major risk to the Canadian economy.
- donmcronald 4y ago> I’m sure Rogers will be forced to do a post-mortem for large commercial clients where contractually required, but I highly doubt we will hear a word from them about the cause of this or what was/is being done to rectify it. They probably have an ETA but feel no obligation to share it. I'm sure there'll be industry wide participation in a post-mortem. Translation for non-Canadians... The head of the CRTC will book a tee time for him plus the CEOs of Rogers, Bell, and Telus.
- acchow 4y ago> it represents major risk to the Canadian economy. I actually think this outage will show that there's no major risk to the economy and every Canadian can just take every Friday off with no ill effects.
- deleted 4y ago[deleted]
- lriel 4y agoAgreed, although, Rogers has had notably more vaguely explained outages. With this, if the CRTC does not revise their decision to let Rogers acquire Shaw (CRTC 2022-76), then, as Canadians, we must expect to continue to overpay for shotty telecoms. But hey, there's maple sirup and affordable health care.
- walrus01 4y agoThis is funnier if you look at it from the English slang point of view when something gets thoroughly Rogered. This network has definitely rogered itself.
- markus_zhang 4y agoLooks like it is still going on. I only have 2G network. I also realized that I cannot use Google Map for driving without network.
- UkrainianJew 4y agoThe utterly inefficient and overpriced Canadian telecoms exist only because we allow one specific behavior: 1. A new local player comes to town, builds their own infrastructure (e.g. connects one building to fiber) and starts offering competitive service. 2. Rogers/Shaw/Bell/Telus immediately offer better terms for the residents of that building. 3. The competitor runs out of money and leaves. 4. The telecoms revert to their usual pricing. This hurts competition, this hurts customers, this hurts long-term infrastructure resilience, but not a single politician ever comes close to even admitting the problem. I understand you cannot do it on the federal level where both major parties are on the telecoms' payroll, but it could be a very low-hanging fruit for someone running for a municipal position to address. But nope, identity politics, words, feelings and fighting climate change with paper straws seem to be what the electorate wants instead. Sad.
- amelius 4y agoSounds like how the free market was designed to work. Not that I find it a particularly good design, though.
- culi 4y ago> free market > was designed
- Teever 4y agoUtilities like telecoms can't be free market due to the natural monopoly effect. We can't have a hundred different companies burying infrastructure or stringing up lines on poles.
- ornornor 4y agoMany places have solved this long ago, both for electricity and for telecom: the network is a separate, neutral entity, co-owned by all the players. One cable/fiber, rented by all providers. The network only has one goal: distribution. The providers are the ones competing.
- newsclues 4y agoI don’t understand have 911 service is impacted, doesn’t essential services have redundancy? Society is fragile
- KMnO4 4y agoThe actual dispatch appears to be fine. It’s just the network isn’t allowing mobile connections. Everyone has “No Service”.
- newsclues 4y agohttps://www.cbc.ca/amp/1.6514453 https://www.cbc.ca/amp/1.6514453 And some hospitals appear to have similar issues.
- kashunstva 4y agoMy spouse is a division chief in a surgical subspecialty - hospital is mandating anyone on-call to not leave the premises. People are sleeping in their offices.
- NKosmatos 4y ago“In April 2021, Rogers customers reported interruptions to wireless voice and data services for several hours. Rogers blamed that outage on a glitch tied to an Ericsson software upgrade”. Rogers has rolled out 5G with Ericsson equipment and I hope this outage is not related to Ericsson activities. Network upgrades are usually performed late at night during low traffic, so it shouldn’t be the case.
- pdmccormick 4y agoThanks for mentioning the last major outage. That one was memorable for me as my son had just been born a few days prior. If that had happened the day of, I honestly don’t know how we would have managed it logistically.
- 29083011397778 4y agoFor the record, I noticed at 1:50 PST or so - meaning a network upgrade would actually line up exactly.
- bandwidth 4y agoThe VP of Rogers admits they have yet to identify the root cause: https://twitter.com/PnPCBC/status/1545512971878662145 https://twitter.com/PnPCBC/status/1545512971878662145
- ttul 4y agoThis is absolutely astonishing.
- kogepathic 4y ago> This is absolutely astonishing. It's quite astonishing by any reasonable standard, but this is Rogers we're talking about, so pretty much par for the course.
- ju45 4y agoThat's why I use Bell or Vidéotron
- betaby 4y agohttps://twitter.com/Videotron/status/1360324916176896007 https://twitter.com/Videotron/status/1360324916176896007 last year