4 ms·
Thanks. It crawls HN topics every one hour. So news in our application would delay up to one hour.
by federkasten 13y ago
Thanks.
It crawls HN topics every one hour. So news in our application would delay up to one hour.
- jkarneges 13y agoIf you're hitting HN's access thresholds, you might consider incrementally parsing data from http://hnstream.com http://hnstream.com That may help with analyzing comment content at least. Not so much with news rankings though.
- federkasten 13y agoThanks. hnstream is really cool. You're right. I have limited frequency of crawling due to HN's access thresholds. hnstream will help me. I will try it.