6 ms·
ByteDance's Recommendation System
- davidjfelix 2y agoContext: ByteDance is the parent company of TikTok. Frequently people talk about "the tiktok algorithm". This is that.
- pavlov 2y agoThere's also a heavy element of manual curation in TikTok. They have people putting their fingers on the scales to decide what content gets promoted. Where are those people, and what's their agenda? Who knows. Releasing the recommender on Github is a way to try to diffuse that criticism. But it's just one part of the puzzle that is Tiktok's content distribution.
- yunohn 2y agoThis is true for all social media algorithms. None of them are purely automated and for good reason. You need humans going in and tweaking the outcomes to ensure users have a good experience. Of course, when the conversation is about TikTok, this often becomes accusations of propaganda. But YouTube, Facebook, and Twitter all exert significant control over their algorithms and things like their Homepage, Trending Topics, etc. The conservative right often labels such curation as liberal propaganda.
- pavlov 2y agoSure. HN is very actively moderated, and most people here probably agree that it’s worth it. (Those who don’t like it presumably don’t stay here.) But at the massive scale of Meta or ByteDance, there is a difference between removing problematic content and actively promoting content. They’re two sides of the same coin, but the first is applied based on reactive guidelines (“we’ve previously decided this kind of content shouldn’t be here”) while the second is ultimately an in-the-moment opinion on whether more people should be seeing the content. The line is blurry, but these are not the same thing, and vibes-based content promotion is easier to manipulate. Are there CCP agents working at ByteDance? Of course there are because it’s practically mandatory — just like American telecom companies have NSA wiretap rooms. Do those CCP agents get consulted on which foreign political candidate should get the viral boost? Perhaps not. But it appears they’ve built a system where this kind of thing is possible and leaves little paper trail because the curated boosting is so integral to the platform.
- yunohn 2y ago> HN is very actively moderated I explicitly did not mention HackerNews, as the homepage feed is primarily based on user voting - neither algorithms nor chronology. Dang’s moderation is not comparable to other social media platform’s feed curation. > there is a difference between removing problematic content and actively promoting content Again, there is sufficient evidence that all major social media platforms do exactly this, not just TikTok. Hence why I said: >> The conservative right often labels such curation as liberal propaganda. > where this kind of thing is possible and leaves little paper trail Could you point to the paper trail that Meta, Google, Twitter provide on their curation actions? Otherwise, this just proves my point that people blindly want to accuse Chinese platforms of shady activities, and Western ones as paragons of virtue.
- dylan604 2y agoIf you don't have a way to manually push the algo, then you'd never be able to sell features like promoted posts and the like. And why would you not want a feature to sell?
- yunohn 2y agoAds can be inserted into any kind of feed, there is no difference between chronological and algorithmic. It’s usually a simple calculation of target ads displayed per posts viewed, with per user/advertiser caps.
- PittleyDunkin 2y ago> This is true for all social media algorithms. None of them are purely automated and for good reason. Purely chronological sorting of posts works the best and is fully automated. It's just really bad at keeping users eyeballs on ads.
- yunohn 2y agoI really don’t agree with this - chronological sorting favors larger/power users who spam their content through all hours of the day, versus smaller users that you probably care more about.
- davidjfelix 2y ago> They have people putting their fingers on the scales to decide what content gets promoted. Is this just your belief or is there evidence you can point to. How would you differentiate manual intervention from algorithmic intervention?
- rolandog 2y agoIt was previously discussed in HN [0]. [0]: https://news.ycombinator.com/item?id=34480003 https://news.ycombinator.com/item?id=34480003
- zenogantner 2y agoWhile this is from ByteDance, who also are behind TikTok, this algorithm is likely not the one behind TikTok. Instead, it is likely a component that powers ByteDance's commercial recommender system solution, which they market to e-commerce companies: https://www.byteplus.com/en/product/recommend https://www.byteplus.com/en/product/recommend This was mentioned in past discussions of the paper on HN. And even if aspects of this are used for TikTok: (a) it would be just one of many components of their recommendation system, and (b) the TikTok recommendation system has changed a lot during the 2+ years since this has been published. So take what you see here with a grain of salt. After reading the paper and the code, you will NOT know how TikTok's recommendations work.
- ryanar 2y agoPast discussion https://news.ycombinator.com/item?id=35573624 https://news.ycombinator.com/item?id=35573624
- pixelatedindex 2y agoThe article linked in that post is dead (403). Here is a cached version: https://archive.is/DDUjQ https://archive.is/DDUjQ
- xnx 2y agoLike Facebook before, everyone talks about the TikTok algorithm being some super secret and valuable mystery. In reality, both Facebook and TikTok succeeded because they were at the right place at the right time and didn't screw things up. The TikTok recommendation system is smart and very well implemented, but nothing novel that couldn't be implemented by a dozen other teams.
- mrfox321 2y agoNot true. Why do IG reels and YouTube shorts suck, then? They clearly built something superior. And it can't seem to be matched by the biggest tech companies.
- xnx 2y agoMore data beats better algorithms. TikTok has vastly more interaction data by nature of its design. IG and YouTube shorts don't have nearly the volume of engaged users and are reluctant to disrupt the cash cow of their traditional interfaces.
- newsclues 2y agoI question the assertion that TikTok has more interaction data than Google.
- xnx 2y agoGoogle has a ton of interaction data to be sure, but the app design decisions of TikTok (auto play, auto loop, easy swipe, easy like, etc.) extract so much more usable/actionable interaction data. The size of the like button on YouTube is a tiny percent of the screen. On TikTok the like button is the whole video.
- leobg 2y agoNot just that. The whole UI is designed for behavioral data aggregation. It’s not just “did you click the like button”. It’s “did you swipe it away? How long did you watch until you swiped it away? Did you come back afterwards? Did you let it loop multiple times before moving on?”. They’ll capture likes and dislikes you yourself probably didn’t even knew you had, just from tens and hundreds of these micro actions. And they’ll do it in the very first hour of you using the app, whereas YouTube won’t know too much about you even after months of you using it.
- tw1984 2y agofor those interested, Chinese laws forbid the export of recommendation systems, unless ByteDance is challenging the Chinese laws here, which is highly unlikely, this simple can not be the recommendation system used in their production. will be far more interesting to know say what is the difference, what got changed/removed to make them feel comfortable that such an open source variant won't get them into troubles with some 3-letters-acronym agencies back home.
- tasn 2y agoI very much doubt this is the TikTok algorithm. 1. It doesn't even claim to be that. 2. It's over a year old - so even if it was, this is no longer it. 3. There's zero incentive for them to release it, but every incentive to release a fake one. 4. TikTok is too much of a national asset, I doubt the Chinese government would not use it to their advantage (and transparency would be counter to that). Edit: there's also https://news.ycombinator.com/item?id=42471278 https://news.ycombinator.com/item?id=42471278
- paulluuk 2y agoFrom what I can tell, they didn't release the actual model, but rather a kind of framework for running and training models in a distributed way. The only actual "model" in there appears to be three Dense layers in a "demo" file.
- mrfox321 2y agoI mean, it's their serving infra, but not their features. See my reply in another parent comment.
- PaulHoule 2y agoThere are a lot of interesting projects in that GitHub account, I didn't know ByteDance was such an open source hero!
- deleted 2y ago[deleted]