9 ms·
Twitter doesn't say that 5% of all accounts are bots. While I don't have inside knowledge of how Twitter does things, the industry standard is to: 1. Build a c
by RandomBK 4y ago
Twitter doesn't say that 5% of all accounts are bots. While I don't have inside knowledge of how Twitter does things, the industry standard is to:
1. Build a classification model to determine whether an account is a bot or not.
2. Measure the precision/recall curve of that model, and pick a threshold where the precision would be >95% (i.e. >95% of what the model labels as "humans" are really humans).
(Higher recall leads to lower precision and vice versa. Higher recall means a higher mDAU, so it's a balancing act between a high number versus a high confidence in that number.)
3. Let the model loose on all accounts.
4. The number the model tags as humans is the "mDAU", though <5% of these might be false positives.
While this model takes a lot of training data to build, verifying its continued precision/recall characteristics can be completed with very little ongoing sampling - easily handled by the "100 a day" figure Musk has claimed.
- lostdog 4y agoTwitter describes how they get the 5% number (from Matt Levine [0]). They randomly sample 100 accounts per day and have human raters decide if they're bots or not. This is the right way to validate that your fancy ML algorithms are working well. [0] https://archive.ph/GtYzO https://archive.ph/GtYzO