Artificial intelligence models that pick out patterns in images can often do so better than human eyes—but not always. If a radiologist is using an AI model to help her determine whether a patient’s X-rays show signs of pneumonia, when should she trust the model’s advice and when should she ignore it?