Artificial intelligence (AI) is increasingly used in healthcare to support the diagnostic process, but concerns remain about the potential for error and bias. In this study, clinicians working at hospitals across 13 states were randomized to view a series of six AI diagnostic predictions with or without explanations and asked to determine the likelihood of whether the patient’s acute respiratory failure was attributable to pneumonia, heart failure, or chronic obstructive pulmonary disease (COPD). Each series of clinical vignettes also included three systematically biased vignettes. Clinicians’ baseline diagnostic accuracy for the three diagnoses was 73%; AI models with and without explanations improved diagnostic accuracy, but systematically biased models had a larger, adverse impact on diagnostic accuracy.