6 ms·

are you aware that all vehicles from Telsa, are permanently "tweeting" huge telemetry data to an amazon cloud infrastructure managing those petabyte of data to

are you aware that all vehicles from Telsa, are permanently "tweeting" huge telemetry data to an amazon cloud infrastructure managing those petabyte of data to for analysis, training AI, and whatever a fucked brain could imagine. So basically, there is not difference between "hi im having a pizza with friends at this new restaurant at the corner of my street" and "hi im item #12314 and i just turn left at 50mph speed on a busy road" Telsa and Twitter are both very mainstream business at acquiring huge amount of data, So intuitively they would share the same similar issue and challenge...

tayo42 4y ago
You missed the whole storing and showing and appending to time lines part along with inserting a relevant ad into it.
- netmonk 4y ago
  This groups under the banner “fucked brain can imagine”
rsynnott 4y ago
I'm not sure how much telemetry Teslas actually send (do buyers really tolerate their absurdly expensive car reporting their every move back?), but in any case I would assume it all gets stored for bulk processing. Twitter's pretty different; when [insert celeb with 100m followers here] tweets, that tweet should show up for anyone who's looking at their profile, immediately. And the like and retweet counts should update for everyone looking at the tweet that just went out to 100m people (actually, it would be possible for Twitter to fudge this bit, but they don't seem to). These might seem on the face of it similar problems, but really they're not at all. Ingesting lots of data is ~trivial; dealing with lots of data in realtime somewhat less so.
- netmonk 4y ago
  similar workload is live update trafic monitoring. collect data from all agents, and redistribute aggregated/processed data to all agent, and update route to optimise time or length or both of travel. collecting data, ingesting data, redistributing ingested/processed data. I remember back on 2015, there was an AMA on reddit from a former engineer from Telsa. I remember clearly how he explained how Telsa struggled to update thousand of car's firmware with legacy RH5 vm running on amazon and openssl 0.98 incompatibility with upper version. I found it very amusing, cause at the same time, i was working at one of the leader of payment industry and POS maker. And we where exactly struggling with the same issue updating firmware of hundred thousand POS terminal in the wild (except that our servers where on premise and not at aws)and dealing with this openssl issue. And i didnt claim workload are identicals, but i just claims that workload are very similar, leading to the fact that 90% of the solution might be common.
padolsey 4y ago
Twitter at scale is far more complex. With Tesla's telemetry it's just writing to a datastore, with less fuss around latencies, data duplication or consistency. There's no need for millions of clients to then get updated, as there is on twitter. Musk tweets and 110M+ users (+thousands of API clients) need to receive that almost immediately. Twitter has figured out how to do this in a very scalable fashion. Once you know how it's done it's not all that impressive on its own but when you take into account the littany of other features twitter has to maintain at scale you start to realize how many unique problem domains there are within the org.

tayo42 4y ago

You missed the whole storing and showing and appending to time lines part along with inserting a relevant ad into it.

netmonk 4y ago
This groups under the banner “fucked brain can imagine”

rsynnott 4y ago

I'm not sure how much telemetry Teslas actually send (do buyers really tolerate their absurdly expensive car reporting their every move back?), but in any case I would assume it all gets stored for bulk processing. Twitter's pretty different; when [insert celeb with 100m followers here] tweets, that tweet should show up for anyone who's looking at their profile, immediately. And the like and retweet counts should update for everyone looking at the tweet that just went out to 100m people (actually, it would be possible for Twitter to fudge this bit, but they don't seem to). These might seem on the face of it similar problems, but really they're not at all. Ingesting lots of data is ~trivial; dealing with lots of data in realtime somewhat less so.

netmonk 4y ago
similar workload is live update trafic monitoring. collect data from all agents, and redistribute aggregated/processed data to all agent, and update route to optimise time or length or both of travel. collecting data, ingesting data, redistributing ingested/processed data. I remember back on 2015, there was an AMA on reddit from a former engineer from Telsa. I remember clearly how he explained how Telsa struggled to update thousand of car's firmware with legacy RH5 vm running on amazon and openssl 0.98 incompatibility with upper version. I found it very amusing, cause at the same time, i was working at one of the leader of payment industry and POS maker. And we where exactly struggling with the same issue updating firmware of hundred thousand POS terminal in the wild (except that our servers where on premise and not at aws)and dealing with this openssl issue. And i didnt claim workload are identicals, but i just claims that workload are very similar, leading to the fact that 90% of the solution might be common.

padolsey 4y ago

Twitter at scale is far more complex. With Tesla's telemetry it's just writing to a datastore, with less fuss around latencies, data duplication or consistency. There's no need for millions of clients to then get updated, as there is on twitter. Musk tweets and 110M+ users (+thousands of API clients) need to receive that almost immediately. Twitter has figured out how to do this in a very scalable fashion. Once you know how it's done it's not all that impressive on its own but when you take into account the littany of other features twitter has to maintain at scale you start to realize how many unique problem domains there are within the org.