8 ms·
Seems to be some normalization problem with the data, right in the 1st page of the default query there's a duplicate entry.
by paol 2y ago
Seems to be some normalization problem with the data, right in the 1st page of the default query there's a duplicate entry.
- afiodorov 2y agoGood spot, will deduplicate in the next iteration. However titles are repeated often due to the region/language variations.
- paol 2y agoSince you're denormalizing to a single table, I think the correct way to handle this would be to aggregate all the titles into the title column. Although "Untitled Pixar Animation Project" is basically garbage data, but that's a harder problem to solve...
- afiodorov 2y agodeduped all rows with a simple .uniq() call in polars before saving