The AHRQ Patient Safety Indicators (PSIs) were originally developed as a means of screening administrative data to identify potential patient safety problems. However, they are increasingly being used for quality measurement and hospital comparison purposes. This study sought to evaluate the accuracy of surgical PSIs for identification of true safety issues, by comparing PSI-detected events to clinical data. The PSIs tested had only moderate sensitivity and specificity for detecting clinical adverse events, lending support to prior research, which concluded that PSIs should be used only for screening purposes. An AHRQ WebM&M commentary discusses the limitations of using PSIs for public reporting and hospital comparison purposes.