16 ms·

Inside the Transport for London cyberattack

101 points by alexbilbie 2y ago

com 2y ago
Lots of people who should have been establishing effective security practices and monitoring and improving it were doing … something … but not that. Total failure of management and governance at TfL and the British Library (which even had a “private sector security leader” on its board of governors for a decade or more before their total shitshow of a breach last year) But as usual, there will be no consequences.
- chrisjj 2y ago
  > no consequences. I recommend "repercussions" ;)
- jen20 2y ago
  TFL are better than most public bodies but are likely hamstrung on being able to pay anything like market rates for competent security people.
  - com 2y ago
    Totally get it that budgets are tight. But making sure that stuff happens isn’t highly correlated to tech staff or manager salaries. Unless they’re hiring inexperienced high-schoolers, it’s a failure of will and competence in management. And even that would actually be a failure of managrnent. I’m guessing - based on historic contacts with TfL - that this failure of management is probably manifest in too many meetings and intermediate products valorised over and above culture, knowledge and tech improvements. Avoidance of outcome-based monitoring and governance, and instead a focus on “process execution” like reorgs, agonisingly-slow checkbox actions and deckchair relocations is pretty common in low-ambition, low-performance orgs. Again, you don’t get this because you’re being cheap on security people.
    - avianlyric 2y ago
      I’m not really sure how you’ve ended up classifying TfL as a “low-ambition, low-performance” org. As transit agencies go TfL is one of the best on the planet, they’re constantly innovating and pushing the envelope of what’s possible. Every time they build a new line, or order new rolling stock, they use it as an opportunity to invent something new (which doesn’t always succeed). Everything from automated trains, to more complex and capable signaling systems, improved cooling and better customer signage. A “low-ambition, low-performance” transit organisation doesn’t run train services with a train every 90 seconds at peak, transporting 4 million people per day without a major incident or loss of life. There are nine Underground stations with annual passenger counts larger than the entire BART system in the larger Bay Area. The Underground system alone (only part of TfL responsibility) is the world fifth largest metro system outside of china by ridership. TfL built its own ticketing system, and invented the entire idea of using contactless bank cards for ticketing, including negotiating with Visa and Mastercard to create brand new rules for transit agencies. A system that it now sells to other mayor transit systems, such as the New York Subway. TfL isn’t without faults and problems, like any large public organisation. But to dismiss it as “low-ambition, low-performance” is to ignore many decades of safety operating the worlds oldest metro system, and developing and exporting new ways of improving transit for the travelling public.
lol768 2y ago
> Earlier this month Andy Lord, the boss of Transport for London, sat down at a scheduled board meeting and praised his organisation’s response to a “highly sophisticated” cyberattack, which began with reports of “suspicious activity” on Sunday 1st September. > “The vast majority of Londoners would not know this attack has happened,” the TfL commissioner told board members including mayor Sadiq Khan. Lord later added: “Because it’s been so well-managed people didn’t understand the scale and impact.” Are these people completely delusional? They've taken away passenger's visibility to see what they were being charged for; they killed all of the open data feeds (though a few of these have just now been restored in the last couple of days). Back in September, they disrupted all of their staff's productivity by locking everybody out and forcing them to try and do their jobs without any access to technology. And.. there's still no end in sight for a restore of the contactless portal. The way they've managed the incident and the collateral damage suggests there were not nearly enough security controls present in the first place (in terms of containing the breach). How many weeks on are we now without service restoration? For a cyberattack perpetrated by one seventeen year old? If it was an SME who didn't do anything technical and had been caught completely unprepared, I might be more understanding.
- avianlyric 2y ago
  I can believe, I live in London and depend on TfL all the time. It’s last 10 or so years, I’ve probably only bothered looking up my travel data a dozen or so times. 99% of the time I’m charged the right amount, and I don’t have to think about it.
bastard_op 2y ago
As soon as you read "outsourced their IT", one can always assume the aftermath would be a shitshow, as it is always done in response to the previous team not being able to run it, which means it is a goddamn mess. Having worked enough state and city government IT contracts in the past 25 years, you just assume the worst about everything and are often not disappointed. It's not a matter of if but when they'll be owned really, and most really wouldn't know what to do if they were still today. This is your relative tax dollars hard at work.
- avianlyric 2y ago
  Where does it say they outsourced their IT? The article mentions city hall outsourcing their IT to TfL, but city hall is sister organisation to TfL, they’re both organs of London regional government. The London Mayor is the chairman of TfL and the head of London regional government. It’s not like they’re outsourcing to some private organisation, every single organisation is either a state organisation, or a state owned company.
  - deleted 2y ago
    [deleted]
  - Groxx 2y ago
    I think they just read it backwards. Near the beginning in the bulleted list is: >Sadiq Khan’s office and the Greater London Authority outsourced their IT services to TfL this summer, meaning they were also badly impacted, paralysing services at the top of the capital’s devolved government. Which means TfL is the one doing other people's IT in addition to its own, not the reverse.
- mellosouls 2y ago
  always done in response to the previous team not being able to run it, which means it is a goddamn mess. : This is your relative tax dollars hard at work. I think you are underestimating the gross lack of realistic investment and corresponding demoralization and qualitative decline in some public services; which latter is then used by the decision-makers who've created the situation as justification for swashbuckling "transformation" projects - advised by and given to overpriced consultants - they can put on their CVs before hopping to the next gig. That's your tax dollars at work.
manojlds 2y ago
Why is the Mayor not even talking about this?
- avianlyric 2y ago
  What is there for the Mayor to say? The trains still run, the busses still turn up, the traffic lights still go red, yellow, green. For the vast majority of people, there little to no impact day-to-day. Sure the loss of live data is annoying, but trains still turn up every 2 mins, and busses every 5-10mins during the day. Even at night, busses still turn up every 15-20mins, so checking live data doesn’t give you that much of an edge.
- surfingdino 2y ago
  Because there is no political gain for him in this story.
bdndndndbve 2y ago
It's amazing how much bureaucracy they're willing to spend money on to means-test a fundamental service. If you just made transit free at the point of service you wouldn't have free cards for all under 16, and some over 16, and all over 60, and discount fares for people in poverty. Cities spend so much money outsourcing the IT for fare collection, and the administration of budget programs, and ultimately the experience is worse for the end users. It's a real case of the politically connected hoovering up tens of millions of dollars because suburban voters can't stomach a poor person getting to ride the bus for free.
- cdot2 2y ago
  All of this IT infrastructure exists to ensure the exact opposite of what you said. It ensures that rich people don't get to ride the bus for free.
  - truckerbill 2y ago
    The point is it’s easier to just tax wealth a bit more and let everyone get on the bloody train… It’s mostly people just going to work/school, makes sense to be free
    - KaiserPro 2y ago
      > easier to just tax wealth a bit more and let everyone get on the bloody train I mean its not. If it was, they would have done it. Wealth taxes are really fucking hard to do equitably, at least at first. For example OAPs tend to live in very expensive hosues. take rotherhithe for example one could have bought a house in the 90s for shit all, and now its worth the best part of 1.4 million. so now you're levying a 5% tax on a pensioner, or worse still a young couple mortgaged to the fucker. Now, but what about the super asset rich I hear you say? Well, they'll transfer all they own into a corporation. They can't tax assets like that on business because it'll crash the economy super quick.
      - bdndndndbve 2y ago
Doctor_Fegg 2y ago
Slightly OT, but really good to see London Centric on the front page of HN. Britain's local media has basically collapsed in recent years - it's now owned by three conglomerates (Reach, Gannett/Newsquest, National World) who are completely uninterested in any form of journalism, only in the sort of clickbait that would have embarrassed even Buzzfeed in its 2012 pomp. The London Evening Standard was one of the last remnants of even slightly decent local writing, and that too has now been shut down in favour of a weekly lifestyle paper called "The Standard". But there's a small number of indie publishers who are trying to fill the gap: the Manchester Mill and Liverpool Post, Bristol Cable, Oxford Clarion, and so on. London Centric is an attempt by an ex-Guardian writer to do the same for London and I hope it succeeds.
- jll29 2y ago
  Are people really no longer interested in what happens locally? I would pay for local news, even about places that I no longer live, but used to live and still have a passing interest in. But perhaps I would want to receive it as XML feed. Even events (concerts, readings, exhibitions, screenings etc.) are typically scattered across many smaller lists or mailing lists, depending on location. For example, there is way more going on in London than what is listed by Time Out: https://www.timeout.com/london/things-to-do/london-events-in-october https://www.timeout.com/london/things-to-do/london-events-in... Local communities may have their own sites or still rely on paper flyers. Some facts of communal interest are published in government outlets that are still mandatated in some jurisdictions. It would be wonderful if more of "what's going on" could be made available in curated digital form, for us to use and enjoy, and also to preserve it for future generations so that they can see what was happening in our age.
  - fragmede 2y ago
    > I would pay for local news Most of my local papers have moved to a subscription model, often with paywalls so I currently pay for local news. Have your local papers not?
  - refulgentis 2y ago
    No
walrus01 2y ago
It sure sounds like this "highly sophisticated" attack was a run of the mill cryptolocker.
lbriner 2y ago
I think lots of people who lack the experience have no idea quite how large and difficult cybersecurity is for a massive organisation whose systems span 20-30+ years or possibly even longer. There is no standardised tooling and very little that can be retrofitted to older systems. Firewalls are fine if the attack is against a port you do not need to use but otherwise you are left with a myriad of commercial offerings and a lot of "risk analysis". The one basic tool that does seem lacking, however, is just basic network segmentation. I could understand a single system being hacked, especially an old system that is massively complex to replace but having to shutdown multiple systems including WiFi and office networks just smells like lazy "just connect all the wires together to make my IT life slightly easier". Having air gaps with separate computers, separate networks (even vlans) etc. is probably the most cost effective way to reduce your attack surface.
- ChoHag 2y ago
  [dead]
paganel 2y ago
> Hundreds of thousands of Londoners are being overcharged for travel, while London Centric spoke to one teenager who is having to skip meals because of cashflow issues brought on by the cyberattack. This is just crazy, why not make public transport as cheap as peanuts to begin with? Why does everything have to be so damn expensive? Why the heck does a monthly transport pass have to cost, let me check, around 200 pounds?, what the fricking fuck?!?! Why don't the common people in the West rise up against this perverted shit? 2400 pounds per year just to have the privilege to take the bus/metro?
- ChoHag 2y ago
  [dead]
- HL33tibCe7 2y ago
  I don't live in London, but most people I've talked to who do don't have any monthly transport pass or anything like that. They just tap in with contactless. The transport is cheap enough that if you don't travel many times per day, there is really no need. As one example - a bus journey is 1.75 GBP regardless of the distance and number of individual buses taken, as long as all initial tap-ins are within one hour. Looking at the TfL website, people on benefits get 50% rate discounts; students get 30% off; pensioners and children get completely free travel. It's really quite a good system actually.
  - cutler 2y ago
    Not so fast. Contactless accounts for a huge volume of compensation claims due to faulty or badly-designed interchanges. I've lost count of the number of times my partner has been overcharged when travelling from Wimbledon to Waterloo.
    - shamiln 2y ago
      Is your partner not following the signs saying where to tap in?
      - 2y ago
HL33tibCe7 2y ago
> Cybersecurity experts claim TfL’s software may have not been up to scratch, with some public-facing systems coded to be compatible with long-defunct browsers such as Internet Explorer 6. This is rubbish, public-facing websites being compatible with defunct browsers is not indicative of any security issue
- JSDevOps 2y ago
  No but also … https://www.newscientist.com/article/2197453-exclusive-thousands-of-security-flaws-found-on-uk-government-websites/ https://www.newscientist.com/article/2197453-exclusive-thous...
cutler 2y ago
What exactly is it about supporting IE6 that makes it a security risk?

101 points by alexbilbie 2y ago

com 2y ago

Lots of people who should have been establishing effective security practices and monitoring and improving it were doing … something … but not that. Total failure of management and governance at TfL and the British Library (which even had a “private sector security leader” on its board of governors for a decade or more before their total shitshow of a breach last year) But as usual, there will be no consequences.

chrisjj 2y ago
> no consequences. I recommend "repercussions" ;)
jen20 2y ago
TFL are better than most public bodies but are likely hamstrung on being able to pay anything like market rates for competent security people.
- com 2y ago
  Totally get it that budgets are tight. But making sure that stuff happens isn’t highly correlated to tech staff or manager salaries. Unless they’re hiring inexperienced high-schoolers, it’s a failure of will and competence in management. And even that would actually be a failure of managrnent. I’m guessing - based on historic contacts with TfL - that this failure of management is probably manifest in too many meetings and intermediate products valorised over and above culture, knowledge and tech improvements. Avoidance of outcome-based monitoring and governance, and instead a focus on “process execution” like reorgs, agonisingly-slow checkbox actions and deckchair relocations is pretty common in low-ambition, low-performance orgs. Again, you don’t get this because you’re being cheap on security people.
  - avianlyric 2y ago
    I’m not really sure how you’ve ended up classifying TfL as a “low-ambition, low-performance” org. As transit agencies go TfL is one of the best on the planet, they’re constantly innovating and pushing the envelope of what’s possible. Every time they build a new line, or order new rolling stock, they use it as an opportunity to invent something new (which doesn’t always succeed). Everything from automated trains, to more complex and capable signaling systems, improved cooling and better customer signage. A “low-ambition, low-performance” transit organisation doesn’t run train services with a train every 90 seconds at peak, transporting 4 million people per day without a major incident or loss of life. There are nine Underground stations with annual passenger counts larger than the entire BART system in the larger Bay Area. The Underground system alone (only part of TfL responsibility) is the world fifth largest metro system outside of china by ridership. TfL built its own ticketing system, and invented the entire idea of using contactless bank cards for ticketing, including negotiating with Visa and Mastercard to create brand new rules for transit agencies. A system that it now sells to other mayor transit systems, such as the New York Subway. TfL isn’t without faults and problems, like any large public organisation. But to dismiss it as “low-ambition, low-performance” is to ignore many decades of safety operating the worlds oldest metro system, and developing and exporting new ways of improving transit for the travelling public.

lol768 2y ago

> Earlier this month Andy Lord, the boss of Transport for London, sat down at a scheduled board meeting and praised his organisation’s response to a “highly sophisticated” cyberattack, which began with reports of “suspicious activity” on Sunday 1st September. > “The vast majority of Londoners would not know this attack has happened,” the TfL commissioner told board members including mayor Sadiq Khan. Lord later added: “Because it’s been so well-managed people didn’t understand the scale and impact.” Are these people completely delusional? They've taken away passenger's visibility to see what they were being charged for; they killed all of the open data feeds (though a few of these have just now been restored in the last couple of days). Back in September, they disrupted all of their staff's productivity by locking everybody out and forcing them to try and do their jobs without any access to technology. And.. there's still no end in sight for a restore of the contactless portal. The way they've managed the incident and the collateral damage suggests there were not nearly enough security controls present in the first place (in terms of containing the breach). How many weeks on are we now without service restoration? For a cyberattack perpetrated by one seventeen year old? If it was an SME who didn't do anything technical and had been caught completely unprepared, I might be more understanding.

avianlyric 2y ago
I can believe, I live in London and depend on TfL all the time. It’s last 10 or so years, I’ve probably only bothered looking up my travel data a dozen or so times. 99% of the time I’m charged the right amount, and I don’t have to think about it.

bastard_op 2y ago

As soon as you read "outsourced their IT", one can always assume the aftermath would be a shitshow, as it is always done in response to the previous team not being able to run it, which means it is a goddamn mess. Having worked enough state and city government IT contracts in the past 25 years, you just assume the worst about everything and are often not disappointed. It's not a matter of if but when they'll be owned really, and most really wouldn't know what to do if they were still today. This is your relative tax dollars hard at work.

avianlyric 2y ago
Where does it say they outsourced their IT? The article mentions city hall outsourcing their IT to TfL, but city hall is sister organisation to TfL, they’re both organs of London regional government. The London Mayor is the chairman of TfL and the head of London regional government. It’s not like they’re outsourcing to some private organisation, every single organisation is either a state organisation, or a state owned company.
- deleted 2y ago
  [deleted]
- Groxx 2y ago
  I think they just read it backwards. Near the beginning in the bulleted list is: >Sadiq Khan’s office and the Greater London Authority outsourced their IT services to TfL this summer, meaning they were also badly impacted, paralysing services at the top of the capital’s devolved government. Which means TfL is the one doing other people's IT in addition to its own, not the reverse.
mellosouls 2y ago
always done in response to the previous team not being able to run it, which means it is a goddamn mess. : This is your relative tax dollars hard at work. I think you are underestimating the gross lack of realistic investment and corresponding demoralization and qualitative decline in some public services; which latter is then used by the decision-makers who've created the situation as justification for swashbuckling "transformation" projects - advised by and given to overpriced consultants - they can put on their CVs before hopping to the next gig. That's your tax dollars at work.

manojlds 2y ago

Why is the Mayor not even talking about this?

avianlyric 2y ago
What is there for the Mayor to say? The trains still run, the busses still turn up, the traffic lights still go red, yellow, green. For the vast majority of people, there little to no impact day-to-day. Sure the loss of live data is annoying, but trains still turn up every 2 mins, and busses every 5-10mins during the day. Even at night, busses still turn up every 15-20mins, so checking live data doesn’t give you that much of an edge.
surfingdino 2y ago
Because there is no political gain for him in this story.

bdndndndbve 2y ago

It's amazing how much bureaucracy they're willing to spend money on to means-test a fundamental service. If you just made transit free at the point of service you wouldn't have free cards for all under 16, and some over 16, and all over 60, and discount fares for people in poverty. Cities spend so much money outsourcing the IT for fare collection, and the administration of budget programs, and ultimately the experience is worse for the end users. It's a real case of the politically connected hoovering up tens of millions of dollars because suburban voters can't stomach a poor person getting to ride the bus for free.

cdot2 2y ago
All of this IT infrastructure exists to ensure the exact opposite of what you said. It ensures that rich people don't get to ride the bus for free.
- truckerbill 2y ago
  The point is it’s easier to just tax wealth a bit more and let everyone get on the bloody train… It’s mostly people just going to work/school, makes sense to be free
  - KaiserPro 2y ago
    > easier to just tax wealth a bit more and let everyone get on the bloody train I mean its not. If it was, they would have done it. Wealth taxes are really fucking hard to do equitably, at least at first. For example OAPs tend to live in very expensive hosues. take rotherhithe for example one could have bought a house in the 90s for shit all, and now its worth the best part of 1.4 million. so now you're levying a 5% tax on a pensioner, or worse still a young couple mortgaged to the fucker. Now, but what about the super asset rich I hear you say? Well, they'll transfer all they own into a corporation. They can't tax assets like that on business because it'll crash the economy super quick.
    - bdndndndbve 2y ago

Doctor_Fegg 2y ago

Slightly OT, but really good to see London Centric on the front page of HN. Britain's local media has basically collapsed in recent years - it's now owned by three conglomerates (Reach, Gannett/Newsquest, National World) who are completely uninterested in any form of journalism, only in the sort of clickbait that would have embarrassed even Buzzfeed in its 2012 pomp. The London Evening Standard was one of the last remnants of even slightly decent local writing, and that too has now been shut down in favour of a weekly lifestyle paper called "The Standard". But there's a small number of indie publishers who are trying to fill the gap: the Manchester Mill and Liverpool Post, Bristol Cable, Oxford Clarion, and so on. London Centric is an attempt by an ex-Guardian writer to do the same for London and I hope it succeeds.

jll29 2y ago
Are people really no longer interested in what happens locally? I would pay for local news, even about places that I no longer live, but used to live and still have a passing interest in. But perhaps I would want to receive it as XML feed. Even events (concerts, readings, exhibitions, screenings etc.) are typically scattered across many smaller lists or mailing lists, depending on location. For example, there is way more going on in London than what is listed by Time Out: https://www.timeout.com/london/things-to-do/london-events-in-october https://www.timeout.com/london/things-to-do/london-events-in... Local communities may have their own sites or still rely on paper flyers. Some facts of communal interest are published in government outlets that are still mandatated in some jurisdictions. It would be wonderful if more of "what's going on" could be made available in curated digital form, for us to use and enjoy, and also to preserve it for future generations so that they can see what was happening in our age.
- fragmede 2y ago
  > I would pay for local news Most of my local papers have moved to a subscription model, often with paywalls so I currently pay for local news. Have your local papers not?
- refulgentis 2y ago
  No

walrus01 2y ago

It sure sounds like this "highly sophisticated" attack was a run of the mill cryptolocker.

lbriner 2y ago

I think lots of people who lack the experience have no idea quite how large and difficult cybersecurity is for a massive organisation whose systems span 20-30+ years or possibly even longer. There is no standardised tooling and very little that can be retrofitted to older systems. Firewalls are fine if the attack is against a port you do not need to use but otherwise you are left with a myriad of commercial offerings and a lot of "risk analysis". The one basic tool that does seem lacking, however, is just basic network segmentation. I could understand a single system being hacked, especially an old system that is massively complex to replace but having to shutdown multiple systems including WiFi and office networks just smells like lazy "just connect all the wires together to make my IT life slightly easier". Having air gaps with separate computers, separate networks (even vlans) etc. is probably the most cost effective way to reduce your attack surface.

ChoHag 2y ago
[dead]

paganel 2y ago

> Hundreds of thousands of Londoners are being overcharged for travel, while London Centric spoke to one teenager who is having to skip meals because of cashflow issues brought on by the cyberattack. This is just crazy, why not make public transport as cheap as peanuts to begin with? Why does everything have to be so damn expensive? Why the heck does a monthly transport pass have to cost, let me check, around 200 pounds?, what the fricking fuck?!?! Why don't the common people in the West rise up against this perverted shit? 2400 pounds per year just to have the privilege to take the bus/metro?

ChoHag 2y ago
[dead]
HL33tibCe7 2y ago
I don't live in London, but most people I've talked to who do don't have any monthly transport pass or anything like that. They just tap in with contactless. The transport is cheap enough that if you don't travel many times per day, there is really no need. As one example - a bus journey is 1.75 GBP regardless of the distance and number of individual buses taken, as long as all initial tap-ins are within one hour. Looking at the TfL website, people on benefits get 50% rate discounts; students get 30% off; pensioners and children get completely free travel. It's really quite a good system actually.
- cutler 2y ago
  Not so fast. Contactless accounts for a huge volume of compensation claims due to faulty or badly-designed interchanges. I've lost count of the number of times my partner has been overcharged when travelling from Wimbledon to Waterloo.
  - shamiln 2y ago
    Is your partner not following the signs saying where to tap in?
    - 2y ago

HL33tibCe7 2y ago

> Cybersecurity experts claim TfL’s software may have not been up to scratch, with some public-facing systems coded to be compatible with long-defunct browsers such as Internet Explorer 6. This is rubbish, public-facing websites being compatible with defunct browsers is not indicative of any security issue

JSDevOps 2y ago
No but also … https://www.newscientist.com/article/2197453-exclusive-thousands-of-security-flaws-found-on-uk-government-websites/ https://www.newscientist.com/article/2197453-exclusive-thous...

cutler 2y ago

What exactly is it about supporting IE6 that makes it a security risk?