6 ms·

I tested Meilisearch last year indexing 10mil documents it took 2 days! same dataset with the same server specs it took less than 2 hours to index in elasticsea

I tested Meilisearch last year indexing 10mil documents it took 2 days! same dataset with the same server specs it took less than 2 hours to index in elasticsearch.

Kerollmops 3y ago
Thank you for the feedback. You should probably look at our blog post explaining how to index in smarter way [1]. You should probably come and talk to us on Discord. We work with MrBeast and they don't have any issue indexing millions of YouTube videos everyday. Ho! And just to tease you a little bit. We highly improved the indexing speed of Meilisearch v1.6 (january 24). Here is a teasing tweet [2]. [1]: https://blog.meilisearch.com/best-practices-for-faster-indexing/amp/ https://blog.meilisearch.com/best-practices-for-faster-index... [2]: https://x.com/kerollmops/status/1734576622303404317 https://x.com/kerollmops/status/1734576622303404317
- simonw 3y ago
  Why does MrBeast index millions of YouTube videos?
  - Kerollmops 3y ago
    They recently released a new YouTube stats website and they use Meilisearch in the frontend. https://news.ycombinator.com/item?id=38681328 https://news.ycombinator.com/item?id=38681328
- milkshakes 3y ago
  a lot of people will never read your blog, or visit your discord but would be quite interested in using a tool like meilisearch. you're correct that the information is out there, but unless you put it right in front of them, such people will have a bad time (and sometimes complain about it, publicly!).
supz_k 3y ago
We recently re-indexed comments to Meilisearch (a PHP process that synced data from MYSQL to Meilisearch) after a Meilisearch version upgrade. It only took about 1 hour for about 12 million documents on a 16GB/4vCPU. In your case, maybe it was a config issue? Or, an old version?
- Kerollmops 3y ago
  And when you'll use the v1.6 you see a much better indexation speed. https://x.com/kerollmops/status/1734576622303404317 https://x.com/kerollmops/status/1734576622303404317

I tested Meilisearch last year indexing 10mil documents it took 2 days! same dataset with the same server specs it took less than 2 hours to index in elasticsearch.

Kerollmops 3y ago

Thank you for the feedback. You should probably look at our blog post explaining how to index in smarter way [1]. You should probably come and talk to us on Discord. We work with MrBeast and they don't have any issue indexing millions of YouTube videos everyday. Ho! And just to tease you a little bit. We highly improved the indexing speed of Meilisearch v1.6 (january 24). Here is a teasing tweet [2]. [1]: https://blog.meilisearch.com/best-practices-for-faster-indexing/amp/ https://blog.meilisearch.com/best-practices-for-faster-index... [2]: https://x.com/kerollmops/status/1734576622303404317 https://x.com/kerollmops/status/1734576622303404317

simonw 3y ago
Why does MrBeast index millions of YouTube videos?
- Kerollmops 3y ago
  They recently released a new YouTube stats website and they use Meilisearch in the frontend. https://news.ycombinator.com/item?id=38681328 https://news.ycombinator.com/item?id=38681328
milkshakes 3y ago
a lot of people will never read your blog, or visit your discord but would be quite interested in using a tool like meilisearch. you're correct that the information is out there, but unless you put it right in front of them, such people will have a bad time (and sometimes complain about it, publicly!).

supz_k 3y ago

We recently re-indexed comments to Meilisearch (a PHP process that synced data from MYSQL to Meilisearch) after a Meilisearch version upgrade. It only took about 1 hour for about 12 million documents on a 16GB/4vCPU. In your case, maybe it was a config issue? Or, an old version?

Kerollmops 3y ago
And when you'll use the v1.6 you see a much better indexation speed. https://x.com/kerollmops/status/1734576622303404317 https://x.com/kerollmops/status/1734576622303404317