5 ms·

Someone should release a tool to scrape craigslist and post to padlister. Wouldn't craigslist have to counter-scrape and file a DMCA takedown notice for each li

Someone should release a tool to scrape craigslist and post to padlister. Wouldn't craigslist have to counter-scrape and file a DMCA takedown notice for each listing?

natrius 14y ago
I don't think Craigslist can legally issue a DMCA takedown for posts since they don't own the copyright to them. Padmapper stopped scraping Craigslist because it's against Craigslist's terms of service and Craigslist asked them to stop. A third party scraper as you've described would be harder to stop.
juiceandjuice 14y ago
I think you could probably make a chrome web app/plugin that does the scraping locally to get the data, then feeds it back in to padmapper.
briandear 14y ago
Sure. Let's build tools to steal other people's data and profit from it.
- huckleberries21 14y ago
  I guess I'm more sympathetic in this case because of the nature of the data: they're classified listings. The dynamic there is that landlords actually want lots of people to see their ad, and apartment hunters want to see as many good options as possible. So in this case, CL is a portal for that, but has set its policy to be such that it prevents the access both the landlords and the apt hunters want. Sure, CL is in its right as a for-profit company to not share info. But it's not in the best interest of the people who post (i.e. the generators of that data). It's frustrating that CL gets to tout itself as some uniquely altruistic for-profit company that's not all about maximizing profit and more focused on the good of its users, then refuses to give the posters more exposure for the sake of keeping their monopoly. Then they automatically assume that every other competitor out there is some greedy capitalist trying to profit off the community, which it doesn't seem like PadMapper has done. It's got a lot of users, but not much of a profit model, and it's been this way for years.

Someone should release a tool to scrape craigslist and post to padlister. Wouldn't craigslist have to counter-scrape and file a DMCA takedown notice for each listing?

natrius 14y ago

I don't think Craigslist can legally issue a DMCA takedown for posts since they don't own the copyright to them. Padmapper stopped scraping Craigslist because it's against Craigslist's terms of service and Craigslist asked them to stop. A third party scraper as you've described would be harder to stop.

juiceandjuice 14y ago

I think you could probably make a chrome web app/plugin that does the scraping locally to get the data, then feeds it back in to padmapper.

briandear 14y ago

Sure. Let's build tools to steal other people's data and profit from it.

huckleberries21 14y ago
I guess I'm more sympathetic in this case because of the nature of the data: they're classified listings. The dynamic there is that landlords actually want lots of people to see their ad, and apartment hunters want to see as many good options as possible. So in this case, CL is a portal for that, but has set its policy to be such that it prevents the access both the landlords and the apt hunters want. Sure, CL is in its right as a for-profit company to not share info. But it's not in the best interest of the people who post (i.e. the generators of that data). It's frustrating that CL gets to tout itself as some uniquely altruistic for-profit company that's not all about maximizing profit and more focused on the good of its users, then refuses to give the posters more exposure for the sake of keeping their monopoly. Then they automatically assume that every other competitor out there is some greedy capitalist trying to profit off the community, which it doesn't seem like PadMapper has done. It's got a lot of users, but not much of a profit model, and it's been this way for years.