5 ms·
Probably not. In your ordinary browsing, you're not likely to download anywhere near the entirety of wikipedia. Plus, spreading out your requests over time rath
by _nb 13y ago
Probably not. In your ordinary browsing, you're not likely to download anywhere near the entirety of wikipedia. Plus, spreading out your requests over time rather than downloading it all at once is more gentle.
- gnosygnu 13y agoGood observation, but I just want to point out that the full dumps are downloaded from different servers. They are even mirrored by other institutions. See https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_XML_dumps https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_...
- cdash 13y agoThey could set up a torrent as well and almost assuredly people will seed it full time on servers.
- rakoo 13y agoThere is a torrent distribution channel: see https://meta.wikimedia.org/wiki/Data_dump_torrents https://meta.wikimedia.org/wiki/Data_dump_torrents However it suffers the number one torrent issue: they do not tolerate change. This means that - When an article changes, you need to generate a new torrent - When a new torrent describes the archive, it needs to be downloaded from scratch by all peers, so that the maximum number of peers are available for a newcomer. I hope you'll understand that this is not the official way to distribute archives...
- gnosygnu 13y agoYeah, but they're working on incremental updates, so this should make the "from scratch" part much easier. You can look at https://www.mediawiki.org/wiki/Incremental_dumps https://www.mediawiki.org/wiki/Incremental_dumps and http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-August/000915.html http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-Aug... if you're interested.