5 ms·

You know there's something weird. You could prove your little hypothesis here wrong in less time than it took you to write it. So why state something that's wro

by rjf72 7y ago

You know there's something weird. You could prove your little hypothesis here wrong in less time than it took you to write it. So why state something that's wrong? I tested this by choosing to search for something completely random that would return unusual results and give a decent indicator of whether or not DDG was just copy-pasting results, so to speak. I searched for "extent space taco secure", no quotations. Bing seems to have exploded when I searched for this. The results are completely nonsensical and seem to have nothing to do with what I searched for. The first two hits are to Google, the first about webhp (apparently some virus?) and the other to their chrome page. Other links include things like links to fidelity.com, americanexpress.com, and the huffington post front page. DDG's results are [mostly] reasonable and populated mostly by things like some recent event where the head of NASA mentioned space security. Though it does have a link to the SF bay area craigslist alongside a couple of other really weird ones. In any case, you can see for yourself. Suffice to say they're definitely not just replicating results, at least not from Bing.

WD-42 7y ago
You could also read their documentation: https://duckduckgo.com/api https://duckduckgo.com/api > Because of the way we generate our search results, we unfortunately do not have the rights to fully syndicate our results, free or paid. That page links to: https://help.duckduckgo.com/results/sources/ https://help.duckduckgo.com/results/sources/ Where, after you get past a bunch of stuff about their "instant answers" gets to the root of it: > We also of course have more traditional links in the search results, which we also source from a variety of partners, including Verizon Media (formerly Yahoo) and Bing. So yes, maybe I was wrong saying results come only from Bing. But they definitely source their search results. I'm not knocking DDG here, I use it as my daily search driver. If you were to try and build a search engine today with limited resources would you really try to start from scratch? The way DDG has approached the problem (by sourcing results from other search engines) seems like the only reasonable way to be even remotely competitive.
- rjf72 7y ago
  Yes, they source their results. This is not the extent of your claims. You were stating that they directly rip from other engines meaning that, in your words, "In order for search results to get better on DDG, they first need to get better on their partner search engines". That is simply completely wrong, as my example showed extremely clearly. The one and only weakness of sourcing from third parties is that if they are not indexing some site, you also will not be indexing it. You were implying they are directly dependent on the ordering and quality of the other engines, which is obviously and provably false.
  - WD-42 7y ago
    First of all, I never claimed that DDG results were exactly the same as Bing's. I said the top results were the same. Admittedly, I used less random terms than you did. However, I did try your experiment, same terms. The top results from DDG were indeed found in the top results of both Yahoo and Bing. You are right though, not in the same exact order. Then I tried the same terms on Google. There is only one shared result on the front page (the Nasa chief one). I went through 4 more pages of Google, none of the results that both DDG, Yahoo, and Bing all seem to share were on any of them. This aligns more with the point I was trying to make: DDG is going to have a hard time improving their results if the information they're getting is from inferior sources to begin with. It's like working on incomplete information. No matter how good their algorithm is that ranks and filters results from other search engines, if those other engines suck, there is no way their results can be that much better. If, for a given term, Yahoo returns site A, B and C, and Bing returns Site D, E and F, is there a way for DDG to determine that actually site G is the better result? The results can't appear out of thin air. Also, you claim with emphasis, that the only weakness of sourcing from third parties is the indexing problem. That's absurd. Obviously, a huge weakness on sourcing from other search engines is that DDG are bound to the terms of those partnerships. Or a complete severance of them. No partners, no search results. One result of the terms in these partnerships is that DDG can't provide a search API. There was a time I thought perhaps I could write a developers search engine with DDG as the backend. Turns out you can't, and as a small search engine popular with HN types, I feel like that is a huge weakness.
    - 7y ago

by rjf72 7y ago

WD-42 7y ago

You could also read their documentation: https://duckduckgo.com/api https://duckduckgo.com/api > Because of the way we generate our search results, we unfortunately do not have the rights to fully syndicate our results, free or paid. That page links to: https://help.duckduckgo.com/results/sources/ https://help.duckduckgo.com/results/sources/ Where, after you get past a bunch of stuff about their "instant answers" gets to the root of it: > We also of course have more traditional links in the search results, which we also source from a variety of partners, including Verizon Media (formerly Yahoo) and Bing. So yes, maybe I was wrong saying results come only from Bing. But they definitely source their search results. I'm not knocking DDG here, I use it as my daily search driver. If you were to try and build a search engine today with limited resources would you really try to start from scratch? The way DDG has approached the problem (by sourcing results from other search engines) seems like the only reasonable way to be even remotely competitive.

rjf72 7y ago
Yes, they source their results. This is not the extent of your claims. You were stating that they directly rip from other engines meaning that, in your words, "In order for search results to get better on DDG, they first need to get better on their partner search engines". That is simply completely wrong, as my example showed extremely clearly. The one and only weakness of sourcing from third parties is that if they are not indexing some site, you also will not be indexing it. You were implying they are directly dependent on the ordering and quality of the other engines, which is obviously and provably false.
- WD-42 7y ago
  First of all, I never claimed that DDG results were exactly the same as Bing's. I said the top results were the same. Admittedly, I used less random terms than you did. However, I did try your experiment, same terms. The top results from DDG were indeed found in the top results of both Yahoo and Bing. You are right though, not in the same exact order. Then I tried the same terms on Google. There is only one shared result on the front page (the Nasa chief one). I went through 4 more pages of Google, none of the results that both DDG, Yahoo, and Bing all seem to share were on any of them. This aligns more with the point I was trying to make: DDG is going to have a hard time improving their results if the information they're getting is from inferior sources to begin with. It's like working on incomplete information. No matter how good their algorithm is that ranks and filters results from other search engines, if those other engines suck, there is no way their results can be that much better. If, for a given term, Yahoo returns site A, B and C, and Bing returns Site D, E and F, is there a way for DDG to determine that actually site G is the better result? The results can't appear out of thin air. Also, you claim with emphasis, that the only weakness of sourcing from third parties is the indexing problem. That's absurd. Obviously, a huge weakness on sourcing from other search engines is that DDG are bound to the terms of those partnerships. Or a complete severance of them. No partners, no search results. One result of the terms in these partnerships is that DDG can't provide a search API. There was a time I thought perhaps I could write a developers search engine with DDG as the backend. Turns out you can't, and as a small search engine popular with HN types, I feel like that is a huge weakness.
  - 7y ago