8 ms·

If you can’t evaluate decisions or experiments on outcome, what can you measure them on? And why would you bother to? How would you learn from those decisions

If you can’t evaluate decisions or experiments on outcome, what can you measure them on? And why would you bother to? How would you learn from those decisions? The worst case scenario is bound to be horrifying, by definition. That’s not an excuse for waffling out of making a tough call.

ryanklee 4y ago
The evaluation of the quality of the decision should be made by taking into account the constraints and quality of priors that were inputs into the decision. Outcomes are probabilistic. Using results as the evaluation metric is mistake. We did not know how the pandemic was interacting with children and the school system. But we did know that it had deathly, chaotic, fast moving properties. The worst case outcome was not a fantasy. We had reasons to think that the worst case outcome was not improbable. Because we were dealing with an existential threat to possibly our most precious resource, we acted defensively, even though many of us were aware of the deep, problematic tradeoffs. It was the right decision.
- mattcwilson 4y ago
  Nonsense. Yes, at the very earliest part of the pandemic, we had no data. We had to prepare for the worst. Very quickly, we started to see the relative impact of different measures taken in different jurisdictions, we learned about the relative impact of the disease on different age groups and populations, and we started having tools and strategies for detecting, tracing, and inhibiting the disease. These became the priors that informed (or should have informed) the evolving decision-making process for disease response. But instead, in many cases, we saw a ton of inertia on many fronts, due to fear, political face-saving / ass-covering, or willful ignorance. That’s the part I take issue with. It wasn’t “the” right decision, it was “an” early and uninformed decision, followed by many opportunities for additional, more informed, better decisions.
  - ryanklee 4y ago
    Not going to reply other than to say characterizing my comment as nonsense is unfairly dismissive. Be a kind interlocutor if you want dispassionate, reasoned engagement.
    - mattcwilson 4y ago
      Sure, I respect that. If it seems like I don’t think you’re a good person, my apologies. I am enjoying having this debate with you and wish you the best. But I stand by the assertion that your previous comment does not make sense.

ryanklee 4y ago

The evaluation of the quality of the decision should be made by taking into account the constraints and quality of priors that were inputs into the decision. Outcomes are probabilistic. Using results as the evaluation metric is mistake. We did not know how the pandemic was interacting with children and the school system. But we did know that it had deathly, chaotic, fast moving properties. The worst case outcome was not a fantasy. We had reasons to think that the worst case outcome was not improbable. Because we were dealing with an existential threat to possibly our most precious resource, we acted defensively, even though many of us were aware of the deep, problematic tradeoffs. It was the right decision.

mattcwilson 4y ago
Nonsense. Yes, at the very earliest part of the pandemic, we had no data. We had to prepare for the worst. Very quickly, we started to see the relative impact of different measures taken in different jurisdictions, we learned about the relative impact of the disease on different age groups and populations, and we started having tools and strategies for detecting, tracing, and inhibiting the disease. These became the priors that informed (or should have informed) the evolving decision-making process for disease response. But instead, in many cases, we saw a ton of inertia on many fronts, due to fear, political face-saving / ass-covering, or willful ignorance. That’s the part I take issue with. It wasn’t “the” right decision, it was “an” early and uninformed decision, followed by many opportunities for additional, more informed, better decisions.
- ryanklee 4y ago
  Not going to reply other than to say characterizing my comment as nonsense is unfairly dismissive. Be a kind interlocutor if you want dispassionate, reasoned engagement.
  - mattcwilson 4y ago
    Sure, I respect that. If it seems like I don’t think you’re a good person, my apologies. I am enjoying having this debate with you and wish you the best. But I stand by the assertion that your previous comment does not make sense.