Core Problem
AI models often appear accurate while still being wrong for hidden reasons—due to opacity, confirmation bias, and flawed assumptions.
Key Concepts
Training vs. Test Data:
Even “successful” test performance may mask underlying conceptual errors.
Confirmation Bias in AI Models:
If a model learns the wrong correlations but appears accurate on test data, users may be misled.
Feedback Loops:
Especially relevant in predictive systems (e.g., predictive policing), where model outputs actively shape the world being measured.