19-84

5 ms·

1.
by 19-84 8mo ago
there is nearly 10TB of youtube metadata available on archive.org https://archive.org/details/youtube-metadata
2.
by 19-84 8mo ago
redd-archiver uses postgres full text search. for static search you could use lunr.js
3.
by 19-84 8mo ago
the torrent has data for the top 40,000 subs on reddit. thanks to watchful1 splitting the data by subreddit, you can download only the subreddit you want from the torrent
4.
by 19-84 8mo ago
the data from 2025-12 has been released already, it is usually released every month, it just needs to be split and reprocessed for 2025 by watchful1. i will probably eventually add support for importing data from the monthly arctic shift du
5.
by 19-84 8mo ago
I included a metadata dump of every subreddit found in the torrent. it includes a status field which will show of a subreddit is private along with a much more details data catalog readme: https://github.com/19-84/redd-
6.
by 19-84 8mo ago
ive created tooling for an instance registry and team based leaderboard. the API has function to support this as well, so that we can collectively host archives in a decentralized and distributed manner registry readme: https://g
7.
by 19-84 8mo ago
I've updated the workflow and added a placeholder page that will serve before archives are created. thanks again! https://github.com/19-84/redd-archiver/commit/0dfd505ca81cb2...
8.
by 19-84 8mo ago
thank you for your comment, some example dot files were not copied in my original repo, they have now been added. https://github.com/19-84/redd-archiver/commit/0bb103952195ae... the docs have been updated wit
9.
by 19-84 8mo ago
thank you for your comment, I will support any platform that has complete dataset available. I will take submissions for any complete datasets through github issues. https://github.com/19-84/redd-archiver/blob&#x2F
10.
by 19-84 8mo ago
the API and MCP server is very powerful ;)
11.
by 19-84 8mo ago
I have also published sub statistics and profiling for each platform. these can be used to help identify which subs to prioritize for archiving. reddit: https://github.com/19-84/redd-archiver/blob/main/to
12.
by 19-84 8mo ago
the API should allow for a lot of different integrations
13.
Show HN: Self-host Reddit – 2.38B posts, works offline, yours forever(github.com)
286 points by 19-84 8mo ago|64 comments