6 ms·
Whether or not they've "seen" 3x more events is a little bit of a tricky question, because while they may have captured 3x the data exposure (see sibling commen
by evanb 14d ago
Whether or not they've "seen" 3x more events is a little bit of a tricky question, because while they may have captured 3x the data exposure (see sibling comments) experiments often operate blinded to the data. They can develop their analysis scripts, play out various different scenarios via Monte Carlo simulation, and get their whole pipeline working without the bias of actually seeing how each change in algorithm alters the outcome for the real data.
Then, at some point, they freeze their pipeline, "open the box", run the analysis on the real data, and report what they find. But they can only "open the box" once per exposure, after that you can worry that human bias can creep in.