6 ms·

I work for facebook and I worked on the deployment of these large hiphop binaries of our code base. We considered multiple options - binary diffs, multicast, et

I work for facebook and I worked on the deployment of these large hiphop binaries of our code base. We considered multiple options - binary diffs, multicast, etc. The problem with multicast is that it's hard to configure and maintain in our complex cross region datacenter setup that has to travel through other peering networks. We ended up with a torrent deployment system that scales beautifully.

Ramonster 15y ago
We ended up doing exactly the same, I wrote a tracker for the bittorrent deploy system that is aware of our network topology and able to transport these 500Mb binaries to our frontend pool in < 3mins (300+ servers) The tracker is opensource: https://github.com/hyves-org/p2ptracker https://github.com/hyves-org/p2ptracker We're working on open sourcing more moving parts of our deploypipeline. I'd be interested to compare notes on your system ?