11 ms·

Time to crawl! You can use something like Webrecorder, which IPFS also uses to pin tweets: https://blog.ipfs.tech/announcing-pin-tweet-to-ipfs/ https://blog.ipf

Time to crawl! You can use something like Webrecorder, which IPFS also uses to pin tweets: https://blog.ipfs.tech/announcing-pin-tweet-to-ipfs/ https://blog.ipfs.tech/announcing-pin-tweet-to-ipfs/ See https://webrecorder.net https://webrecorder.net Example https://replayweb.page/?source=https%3A%2F%2Freplayweb.page%2Fdocs%2Fassets%2Ftweet-example.wacz#view=pages&url=https%3A%2F%2Ftwitter.com%2Fwebrecorder_io%2Fstatus%2F1565881026215219200%3Fref_src%3Dtwsrc%255Etfw%257Ctwcamp%255Etweetembed%257Ctwterm%255E1565881026215219200%257Ctwgr%255Ee8c57cea50b2bf6a340bb4387a88a33f6e5fb36e%257Ctwcon%255Es1_%26ref_url%3Dhttps%253A%252F%252Foembed.link%252Fhttps%253A%252F%252Ftwitter.com%252Fwebrecorder_io%252Fstatus%252F1565881026215219200&ts=20221002034606 https://replayweb.page/?source=https%3A%2F%2Freplayweb.page%...

naillo 4y ago
I've been using this https://github.com/JustAnotherArchivist/snscrape https://github.com/JustAnotherArchivist/snscrape and with 5 threads and no vpn, just my laptop, have been getting about 1 million tweets a day (vs 1000 before being rate limited on the api). Lots of fun
kibwen 4y ago
Headlines one week from now: "Twitter announces they will be discontinuing the web interface"
- anigbrowl 4y ago
  I'm holding out for 'Wanna tweet? That will be $1' Incidentally they started sending out emails yesterday offering gold organization checkmarks for $1000/month + $50 per public-facing seat. I'm not sure how many businesses will want this, but I bet a lot of political nonprofits and media outlets will leap at the opportunity to boost their visibility (more boosting available for more $, kinda like promoted tweets).
unboxingelf 4y ago
Aside: can someone explain to me how IPFS, in this specific case, is any better than a typical 3rd party service storing the data? The linked announcement says they use a pinning service “web3.storage”. Web3.storage says it stores data on FileCoin. Neither website tells me how and where the actual data is stored, except “IPFS”. From reading the IPFS docs, a pinning service is akin to a node that has a copy of the files and is always online. If your decentralized network relies on a central node(s), how is this decentralized?

naillo 4y ago

I've been using this https://github.com/JustAnotherArchivist/snscrape https://github.com/JustAnotherArchivist/snscrape and with 5 threads and no vpn, just my laptop, have been getting about 1 million tweets a day (vs 1000 before being rate limited on the api). Lots of fun

kibwen 4y ago

Headlines one week from now: "Twitter announces they will be discontinuing the web interface"

anigbrowl 4y ago
I'm holding out for 'Wanna tweet? That will be $1' Incidentally they started sending out emails yesterday offering gold organization checkmarks for $1000/month + $50 per public-facing seat. I'm not sure how many businesses will want this, but I bet a lot of political nonprofits and media outlets will leap at the opportunity to boost their visibility (more boosting available for more $, kinda like promoted tweets).

unboxingelf 4y ago

Aside: can someone explain to me how IPFS, in this specific case, is any better than a typical 3rd party service storing the data? The linked announcement says they use a pinning service “web3.storage”. Web3.storage says it stores data on FileCoin. Neither website tells me how and where the actual data is stored, except “IPFS”. From reading the IPFS docs, a pinning service is akin to a node that has a copy of the files and is always online. If your decentralized network relies on a central node(s), how is this decentralized?