Effectiveness vs Efficiency

What does effectiveness mean in the context of IRBs and HRPPs?

Effectiveness refers to whether and how well an intervention achieves relevant outcomes. The outcomes of interest for IRBs and HRPPs are:

  • Protection of research participants
  • Promotion of justice in research
  • Creating a culture of ethical concern amongst research stakeholders
  • Establishing public trust in the research enterprise
  • Ensuring socially valuable, scientifically valid, ethical research

These outcomes can be difficult to evaluate directly, in part because they are not well-defined or easily operationalized. They also reflect some of the very same determinations that IRBs and HRPPs are expected to make themselves and about which there may be reasonable disagreement. In light of these challenges, existing efforts to evaluate IRBs and HRPPs have largely focused on more easily measured procedural and structural factors, such as the expertise represented on boards, the types of standard operating procedures they have and follow, and whether they comply with relevant regulations and guidance.

Our aims are to identify the procedural and structural factors most likely to support IRBs and HRPPs in achieving their outcomes of interest, to specify the scope of reasonable IRB and HRPP determinations, and to find ways to measure how IRBs and HRPPs influence the experiences of research participants - a critical area that has been largely overlooked in prior evaluation efforts. Rather than focusing exclusively on what IRBs and HRPPs do in reviewing proposed research, we are especially interested in studying their impact once research is allowed to proceed.

What's the difference between effectiveness and efficiency?

Much of the empirical data currently collected about IRBs and HRPPs focus on their turn-around time and how quickly they issue a decision on proposed research. These are important questions, but they relate to efficiency. The speed of protocol review does not tell us anything about how effective that review is in successfully achieving the outcomes of interest. AEREO seeks to move the discussion around IRB and HRPP quality beyond measures of efficiency to focus on the central question of effectiveness.