6 ms·
> 2 MB of logs per minute of user streaming time. 2MB/minute is 33KB/second. How is that impressive?
by asien 4y ago
> 2 MB of logs per minute of user streaming time.
2MB/minute is 33KB/second.
How is that impressive?
- zwily 4y agoI think the impressive thing is how much data that is for each user-minute. What could they possibly be storing in 33KB for each second of Netflix you stream?
- schmeckleberg 4y agoi think you and the above poster are in vehement agreement. ingesting 2 MBs of logs per minute is impressive in its pluperfect unimpressiveness. maybe the presentation was called "Timmy's first named pipe" or "Sally explores /etc/logrotate.d"
- FredPret 4y agoThat’s a LOT of text to describe me sitting on my couch. 2MB per minute is far more than the most detailed biography in existence.
- eightysixfour 4y ago2 MB/per minute/per stream at Netflix scale is crazy.
- deleted 4y ago[deleted]
- exdsq 4y ago220m users. Let’s imagine 50m are streaming concurrently. That’s 100TB an hour in logs lol. They could be storing an entire petabyte of logs a day. My friend did some data center stuff for the large hadron collider and wasn’t hitting these data ingestion states, and these are just to record me binging the office.
- ummonk 4y agoThe comment said "log processing system". Sounds more like it's a stream and not stored logs.
- jaywalk 4y agoI think it's impressive that they somehow found 33KB/second worth of data to log for each stream. I can't even imagine the amount of useless shit that must be logged to get to that number.
- westpfelia 4y agoThis is where I'm at. Like thats honestly not much log data. But what are they actually logging? I imagine there is a LOT of repetitive data.
- seti0Cha 4y agoDetailed logging can function as an on-demand APM. Not a bad idea if you have the bandwidth and storage for it.
- bpicolo 4y agoThat's per user. So a million (or ten, 50...) active users means a lot more per minute.