9 ms·

Netflix had an algorithm contest, when was it, like 20 years ago? A team made a highly effective content recommendation system and Netflix never used it. Why? I

by mvkel 1mo ago

Netflix had an algorithm contest, when was it, like 20 years ago? A team made a highly effective content recommendation system and Netflix never used it. Why? It killed revenue. Separately, there simply isn't enough content to recommend. It's not like you need help finding the perfect Wikipedia article. At any one time, there are ~50 things a user would probably watch. Feels more like a soft pivot to cash in on AI valuations since Hollywood is cratering. Probably worth the investment.

jubilanti 1mo ago
> A team made a highly effective content recommendation system and Netflix never used it. Why? It killed revenue. Nope. This is an internet urban legend. The Netflix prize was only ever for marketing and recruitment. They never intended to deploy whatever winning weights that win because they overfitted on the training and validation data. The dataset they released was not their actual full dataset. It contained zero personal information, just (userid, movieid, rating, date) which is only ever going to get you so far compared to what they do internally. And it wasn't "a team" that beat the baseline of a ranking algorithm Netflix never deployed, dozens of teams were within fractions of a point from each other. Hence the over fitting strategy.
- ekidd 1mo ago
  I think the biggest change from the Netflix prize days is that their catalog went away. Back in the DVD era, they had everything. And in the very early streaming era, they still had a huge number of things to watch. But their current catalog is badly impoverished, and they're just going to recommend the same 30 Netflix originals they always recommend to me, plus a few films or series that are rotating through on a temporary license. If I actually try to search for something specific I'd really like to rewatch, it's almost never there. They haven't quite regressed to the level of a small-town, early 90s Blockbuster, but it sure feels that way sometimes. So honestly, how much good can the Netflix recommendation algorithm do these days, given the much smaller catalog of movies and films it apparently has to work with? (The one that I don't get is the Kindle recommendation algorithm. If I read one really good book with a certain theme, the Kindle immediately replaces my recommendations with 40 bad knockoffs, 25% of them clearly AI written. There's apparently no signal for "actually good.")

by mvkel 1mo ago

jubilanti 1mo ago

> A team made a highly effective content recommendation system and Netflix never used it. Why? It killed revenue. Nope. This is an internet urban legend. The Netflix prize was only ever for marketing and recruitment. They never intended to deploy whatever winning weights that win because they overfitted on the training and validation data. The dataset they released was not their actual full dataset. It contained zero personal information, just (userid, movieid, rating, date) which is only ever going to get you so far compared to what they do internally. And it wasn't "a team" that beat the baseline of a ranking algorithm Netflix never deployed, dozens of teams were within fractions of a point from each other. Hence the over fitting strategy.

ekidd 1mo ago
I think the biggest change from the Netflix prize days is that their catalog went away. Back in the DVD era, they had everything. And in the very early streaming era, they still had a huge number of things to watch. But their current catalog is badly impoverished, and they're just going to recommend the same 30 Netflix originals they always recommend to me, plus a few films or series that are rotating through on a temporary license. If I actually try to search for something specific I'd really like to rewatch, it's almost never there. They haven't quite regressed to the level of a small-town, early 90s Blockbuster, but it sure feels that way sometimes. So honestly, how much good can the Netflix recommendation algorithm do these days, given the much smaller catalog of movies and films it apparently has to work with? (The one that I don't get is the Kindle recommendation algorithm. If I read one really good book with a certain theme, the Kindle immediately replaces my recommendations with 40 bad knockoffs, 25% of them clearly AI written. There's apparently no signal for "actually good.")