Department of Psychiatry

Penn Behavioral Health

Patient Feedback: A Quality Improvement Study in Outpatient Settings

NIDA-Funded Study of Quality Improvement in Substance Abuse Clinics

Principal Investigator: Paul Crits-Christoph, PhD


Quality improvement (QI) methods are a cornerstone of business and healthcare management throughout the United States, yet there have been few studies of QI interventions in addiction treatment settings. Investigators at the University of Pennsylvania and New York University School of Medicine tested the effectiveness of one QI system, Patient Feedback (PF), in increasing outpatient group therapy attendance and self-reported reductions in drug and alcohol use. The feasibility and acceptability of PF was initially established in a six-site study conducted within the National Drug Abuse Treatment Clinical Trials Network. The feasibility study was then followed by the study described below.

Study Design and Rationale

In the current study, 20 community-based outpatient treatment programs in the New York and Philadelphia areas, involving approximately 118 clinicians, were randomly assigned to PF or usual clinic practices. In the PF condition, patients were invited to complete a 12-item, self-administered survey in which they rated therapeutic alliance and treatment satisfaction, and reported substance use in the past week across 12 weeks of treatment. Ten clinics received 12 weeks of the Patient Feedback performance improvement intervention and ten clinics received no intervention during the 12 weeks. Over 1500 patients provided anonymous ratings of therapeutic alliance, treatment satisfaction, and drug/alcohol use. These anonymous surveys were faxed by clinic staff to a University of Pennsylvania data center where a custom software application converted the surveys into feedback reports and posted them to a password protected website. Clinicians then accessed their caseload feedback reports and aggregated reports for the whole clinic; supervisors were only able to access the aggregated clinic reports. Clinic staff met monthly as a team to review the feedback reports and develop QI plans intended to yield improvements on select QI indicators. Organizations may share their feedback reports with funding sources, regulatory agencies, policy makers, and other stakeholders. This centralized, semi-automated feedback system eases fulfillment of accreditation requirements and as such, reduces the cost of clinic operations.

Outcomes and Directions

The study did not demonstrate effectiveness of the PF system; however this may have been due to a variety of reasons and provides directions for future research. The low level of alcohol and drug use at baseline suggest that performance improvement studies for enhancing the effectiveness of substance abuse treatment need to either select only patient and/or agencies with higher baseline levels of drug/alcohol use or rely on other types of outcomes rather than recent substance use. It may also be that clinicians are less motivated by caseload-level feedback. This latter issue led to us designed a preliminary study that provided feedback at the patient level rather than the clinician level.

This study is CLOSED to recruitment.


Forman, R., Crits-Christoph, P., Kaynak, Ö, Worley, M., Hantula, D.A., Kulaga, A., Rotrosen, J., Chu, M., Gallop, R., Potter, J., Muchowski, P., Brower, K., Strobbe, S., Magruder, K., Chellis, A.H., Clodfelter, T., Cawley, M. (2007). A feasibility study of a web based performance improvement system for substance abuse treatment providers.Journal of Substance Abuse Treatment, 33, 363– 371.

Crits-Christoph, P., Ring-Kurtz, S., McClure, B., Temes, C., Kulaga, A., Gallop, R., Forman, R., & Rotrosen, J. (2010). A randomized controlled study of a web based performance improvement system for substance abuse treatment providers. Journal of Substance Abuse Treatment, 38, 251-262.

Crits-Christoph, P., Hamilton, J., Ring-Kurtz, S., Gallop, R., McClure, B.,  Kulaga, A., & Rotrosen, R. (2011). Program, counselor, and patient variability in the alliance: a multilevel study of the alliance in relation to substance use outcomes. Journal of Substance Abuse Treatment,  40, 405–413.

Back to Top