14 ms·

> Another critical decision is that they only investigated pairs of model-dataset with good initial performance. groan

• bootsmann 3y ago
  Overfit in training and then complain that it doesn't generalize well in production, a true classic.
  • noobermin 3y ago
    This seems rather dismissive for something actually published in a real journal and not just on arxiv for once, right?
    • bootsmann 3y ago
      If you check the abstract, then this got published more for describing these drift patterns and showing ways to visualize and detect them rather than dropping the weird statistic the article makes it about.
  • epups 3y ago
    If this is what happened, what explains the fact that some of these models performed great for about a year before decaying?