Perelman School of Medicine at the University of Pennsylvania

AEREO: The Consortium to Advance Effective Research Ethics Oversight


Do IRBs “work”? What does that mean – and how would we know?

Launched in May 2018, AEREO is a consortium of leaders in human subjects research oversight, research ethics, and empirical methods. We have come together to advance the shared mission of evaluating and improving the effectiveness of Institutional Review Boards (IRBs) and Human Research Protection Programs (HRPPs). Consortium members work collaboratively to advance an empirical research agenda to support evidence-based approaches to research ethics review, oversight, and policy.

What does effectiveness mean in this context?

Effectiveness refers to whether and how well an intervention achieves relevant outcomes. The outcomes of interest for IRBs and HRPPs are:

  • Protection of research participants
  • Promotion of justice in research
  • Creating a culture of ethical concern amongst research stakeholders
  • Establishing public trust in the research enterprise
  • Ensuring socially valuable, scientifically valid, ethical research

These outcomes can be difficult to evaluate directly, in part because they are not well-defined or easily operationalized. They also reflect some of the very same determinations that IRBs and HRPPs are expected to make themselves and about which there may be reasonable disagreement. In light of these challenges, existing efforts to evaluate IRBs and HRPPs have largely focused on more easily measured procedural and structural factors, such as the expertise represented on boards, how they deliberate, the types of standard operating procedures they have and follow, and whether they comply with relevant regulations and guidance.

Our aims are to identify the procedural and structural factors most likely to support IRBs and HRPPs in achieving their outcomes of interest, to specify the scope of reasonable IRB and HRPP determinations, and to find ways to measure how IRBs and HRPPs influence the experiences of research participants - a critical area that has been largely overlooked in prior evaluation efforts. Rather than focusing exclusively on what IRBs and HRPPs do in reviewing proposed research, we are especially interested in studying their impact once research is allowed to proceed.

What's the difference between effectiveness and efficiency?

Much of the empirical data currently collected about IRBs and HRPPs focus on their turn-around time and how quickly they issue a decision on proposed research. These are important questions, but they relate to efficiency. The speed of protocol review does not tell us anything about how effective that review is in successfully achieving the outcomes of interest. AEREO seeks to move the discussion around IRB and HRPP quality beyond measures of efficiency to focus on the central question of effectiveness.  

Learn more about the problems AEREO is trying to solve and some of our ideas so far from AEREO's co-chair, Holly Fernandez Lynch: