5 ms·
Show HN: Open-source script to get your site indexed on Google
- mortallywounded 3y agoIs this any different from logging into the Google search console and submitting your sitemap/index request?
- jklinger410 3y agoA little reading would get you the answer: https://seogets.com/blog/google-indexing-script https://seogets.com/blog/google-indexing-script
- paradite 3y agoThe info on 24 to 48 hours waiting time is wrong. I just submitted a few new pages (not yet indexed). They got indexed in less than 4 hours.
- tomschwiha 3y agoSubmitting manually takes sooooo long.
- goenning 3y agosame result from what I've seen, but not scalable for larger amount of pages
- rkuykendall-com 3y agoI submitted 1,900 pages in September and it has yet to look at 600. It did 4 this month. I wish I had been more picky with my sitemap but I thought including all URLs was the goal. I at least properly weighted them but that doesn't seem to do much.
- 15457345234 3y agoIf the organic results are too good someone might click on one of them instead of a sponsored link or advert. De-indexing/non-indexing helps to avoid that outcome.
- qingcharles 3y agoSame here.
- navigate8310 3y agoSo, Google stopped automating indexation because spam, humanity finds new way to resume automation to again propagate spam. It seems Google is trapped in its toxic game of search engine optimization.
- goenning 3y agoThe only possible outcome if for them to shutdown this API or make it work as documented. There's already at least 10+ SaaS offering this as service.
- RobotToaster 3y agoEh, you can get google to index your site by just submitting a site map, it just takes a little longer.
- mvkel 3y agoIf "a little longer" is six months+, you're absolutely right.
- beeboobaa 3y agoGoogle no longer finds new sites automatically? That might explain why it's been so trash the past few years. I remember running a few websites back in the day, and with zero interaction with google all of the pages showed up in the search index a day or two after publishing at most.
- harryp_peng 3y agolol
- deleted 3y ago[deleted]
- nextaccountic 3y agohttps://news.ycombinator.com/item?id=36195600 https://news.ycombinator.com/item?id=36195600 apparently
- xnx 3y agoThis script abuses the Indexing API which is intended for job posting and other specific purposes. https://developers.google.com/search/apis/indexing-api/v3/quickstart https://developers.google.com/search/apis/indexing-api/v3/qu... Use at your own risk.
- ldoughty 3y agoMight simply not work for your website: > Currently, the Indexing API can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject. I wanted to highlight (in addition to your statement) that JobPosting is a specific type of structured data. If the target site doesn't have these elements, it may or may not work... or it may work for now, but not once they realized it's being used incorrectly JobPosting structured data: https://developers.google.com/search/docs/appearance/structured-data/job-posting https://developers.google.com/search/docs/appearance/structu...
- datashaman 3y ago> If you dont have content that falls into those categories, the API won't help you there. https://www.youtube.com/watch?v=kvYb2bdtT7A&t=422s https://www.youtube.com/watch?v=kvYb2bdtT7A&t=422s IME it will silently drop and ignore anything that is not relevant.
- ahmedfromtunis 3y agoFrom Indexing API documentation: > Currently, the Indexing API can only be used to crawl pages with either `JobPosting` or `BroadcastEvent` embedded in a `VideoObject`. So this might come with the risk of seeing the site you want to boost rather penalized by Google.
- goenning 3y agoThis is not a boost, Index != Ranking
- ahmedfromtunis 3y agoEither way, this is clearly abusing the API by using it for things that it wasn't intended for. The only outcome I can see from this is a) contributing to the rise of spam and b) harming people managing sites and apps for which this API is vital.
- niemal_dev 3y ago[dead]
- federalauth 3y agoThank you for sharing. Can you quickly explain why keeping an MD5 avoids the abuse issue?
- niemal_dev 3y agoYou don't post the same URL twice with the same content therefore you maintain the logic of the API itself, so there's no spam or abuse of any kind.
- deleted 3y ago[deleted]
- gasparto 3y agoWhat happend to the good'ol sitemap.xml? You'll probably find an npm package with lots of dependencies that'll generate that sitemap for you if that's what you need...
- sjwhevvvvvsj 3y agoI’m failing to see how this isn’t just “hey look at my sitemap”!
- wut42 3y agoPer Google's docs: > For websites with many short-lived pages like job postings or livestream videos, we recommend using the Indexing API instead of sitemaps because the Indexing API prompts Googlebot to crawl your pages sooner than updating the sitemap
- adaboese 3y agoThe annoying thing about this is that it will ruin this feature for everyone else. I, and many others, use this for requesting to index time sensitive content.
- chiefalchemist 3y agoYes and no. I mean, just because something gets indexed doesn't mean Google values it and is willing to expose its customers to it. The consistent problem with SEO is that most SEOs don't understand Google's business model. They don't understand Google is going to best serve its customers (i.e., those doing the search). SEOs (and their clients) need to understand that getting Google to index a turd isn't going to change the fact that the content and the experience i'ts wrapped in is still a turd. Google is not interested in pointing its customers to turds.
- dindobre 3y agoI'm not sure I agree that the people doing the search are the customers here
- Sai_ 3y agoI’m really curious to know who exactly you think are Google Search’s customers in the context of this thread about SEO.
- nurtbo 3y agoAdvertisers?
- mcny 3y agobut think logically for just a second, why would advertisers advertise their turd if they could just have their turd show up on the search results for free? For a turd sandwich to work, you have to wrap your turd (ads) with high quality results so people actually search on Google and then you can show them the turd along with the good stuff.
- dewey 3y agoI've seen some people even wrapping and re-selling this as SaaS.
- ChrisArchitect 3y ago? "what I've noticed"...Google only indexing if a site has backlinks or is submitted by owner. Uh..yeah, how else would google know about a new URL? C'mon. This just seems like the usual SEO obsession/grift with some 'secret' way to get things done. It's straightfwd these days. Are you saying none of the pages you're queuing up are linked to each other? Most cases they would be in some way right? So the spider will start indexing them all based on a top url submission or a few key urls. Do event/job board sites really need all of their pages to be indexed immediately?
- mvkel 3y agoI recently launched a mini project and was shocked at how difficult and long it took to get any of its pages properly indexed on Google. It's almost as if Google is actively trying -not- to index anything as a way to reduce spam, by forcing the people who really care to jump through 100 hoops. A great way for the dark web remains dark.
- aarreedd 3y agoIt just takes time
- callalex 3y agoHow long is long in your case?
- lobsterthief 3y agoIt just takes time. Getting people to link to it by sharing it in other channels will help to shorten the timeframe.
- dakiol 3y agoIs there something for the opposite? I don't want google (or any other scrapper) to index my website. Afaik, robots.txt is not authoritative.
- speedgoose 3y agosudo iptables -A INPUT -p tcp --dport 80 -j DROP sudo iptables -A INPUT -p tcp --dport 443 -j DROP sudo ip6tables -A INPUT -p tcp --dport 80 -j DROP sudo ip6tables -A INPUT -p tcp --dport 443 -j DROP That should do.
- AznHisoka 3y agoAnother easy way is to just tweet it, which works for me - they usually get indexed < 1 hour later. Google has access to tweets and the URLs in those tweets.
- nhggfu 3y ago"to get your site indexed" => a nonsense claim + this technique might make engines aware of your content, but doesn't guarantee indexation whatsoever.
- leros 3y agoI just submit a sitemap URL to Google Search Console Tools. Is this any different?
- ninefoxgambit 3y agoI’ve seen a lot of indie startups lately that are basically selling faster google indexing then you can get for free using google search console. I guess they are probably using this feature under the hood.
- cranberryturkey 3y agoor just submit a sitemap.xml via google console.