Yes, this is a counterfactual line of reasoning. It's certainly legit, and I believe it's an active field of research in ML, but at the end of the day, you're s

Yes, this is a counterfactual line of reasoning. It's certainly legit, and I believe it's an active field of research in ML, but at the end of the day, you're still poking around the inputs of a black-box model as opposed to examining a white-box model.