Perelman School of Medicine at the University of Pennsylvania

AEREO: The Consortium to Advance Effective Research Ethics Oversight


Do IRBs “work”? What does that mean – and how would we know?

Launched in May 2018, AEREO is a consortium of leaders in institutional human subjects research oversight, research ethics, and empirical methods. We have come together to advance the shared mission of evaluating and improving the effectiveness of Institutional Review Boards (IRBs) and Human Research Protection Programs (HRPPs). Consortium members work collaboratively to advance an empirical research agenda to support evidence-based approaches to research ethics review, oversight, and policy.

What does effectiveness mean in this context?

Effectiveness refers to whether and how well an intervention achieves relevant outcomes. The outcomes of interest for IRBs and HRPPs are:

  • Protection of research participants
  • Promotion of justice in research
  • A culture of ethical concern amongst research stakeholders
  • Public trust in the research enterprise
  • Socially valuable, scientifically valid, ethical research

These outcomes may be advanced by various procedural approaches to research ethics review and oversight. However, examining IRB and HRPP processes and structures alone cannot tell us whether these outcomes are being successfully achieved. Unfortunately, existing measures of IRB and HRPP quality focus on procedural and structural factors, such as the expertise represented on boards, how they deliberate, the types of standard operating procedures they have and follow, and whether they comply with relevant regulations and guidance. Our aim is to shift attention to IRB and HRPP outcomes. How can they be defined, how can they be measured, how can they be improved? This is what AEREO means by effectiveness, which we view as a central - but not exclusive - component of IRB and HRPP quality.

What's the difference between effectiveness and efficiency?

Much of the empirical data currently collected about IRBs and HRPPs focus on their turn-around time and how quickly they issue a decision on proposed research. These are important questions, but they relate to efficiency. The speed of protocol review does not tell us anything about how effective that review is in successfully achieving the outcomes of interest. AEREO seeks to move the discussion around IRB and HRPP quality beyond measures of efficiency to focus on the central question of effectiveness.