tamiko- AI HSR Risk Reference Tool v2.0

AI HSR Risk Reference Toolv2.0

Quick Reference Risk Identification and Mitigation Guide for IRBs Reviewing AI in Human Subjects Research

Tamiko Eto, MA CIP
Founder: TechInHSR

Welcome to the AI HSR Risk Reference Tool

This tool assists Institutional Review Boards (IRBs) and Ethics Committees in identifying and addressing AI-specific risks in human subjects research. It complements standard IRB review processes by providing structured guidance on risks unique to artificial intelligence and machine learning systems.

Important: This tool focuses on AI-specific risks and does not replace comprehensive IRB review. General research risks (privacy, physical/psychological harm, deception) should be addressed through standard IRB processes.

Key Features

  • Structured Framework: Organized around the 3-Phase AI HSR IRB Review Framework
  • Evidence-Based: Built on MIT AI Risk Repository, ISO 14971, and U.S. regulatory frameworks
  • Practical Guidance: Includes reviewer prompts and mitigation strategies
  • Validated: Tested at 23+ institutions with 21% improvement in reviewer confidence

Four Core AI-Specific Risks

⚖️

Misclassification

Incorrect categorization of participants, diagnoses, or outcomes that can lead to inappropriate interventions.

🔍

Explainability

Opacity of AI models where neither researchers nor participants fully understand how predictions are made.

👥

Participant Vulnerability & Equity

Uneven AI performance across demographic groups that may exacerbate health disparities.

🔒

Data Sensitivity & Privacy

Concerns about confidentiality, secondary use, reidentifiability, and HIPAA compliance with large datasets.

3-Phase AI HSR IRB Review Framework

The framework aligns AI research oversight with project maturity to avoid over- and under-regulation:

1

Discover/Ideation

Focus: Early exploratory work

Activities: Data collection, preliminary analysis, proof of concept

Risk Level: Lower - limited participant interaction

Key Considerations:
  • Data quality and representativeness
  • Initial bias assessment
  • Privacy protections for training data
2

Pilot/Validation

Focus: Model performance testing

Activities: Validation studies, algorithm testing, performance metrics

Risk Level: Medium - controlled testing environment

Key Considerations:
  • Model explainability requirements
  • Performance across subgroups
  • Error handling and safety mechanisms
3

Clinical Investigation / Real-World Deployment

Focus: Real-world use and impact

Activities: Clinical trials, deployment studies, post-market surveillance

Risk Level: Higher - direct impact on care decisions

Key Considerations:
  • Clinical decision-making integration
  • Monitoring and adverse event reporting
  • Long-term equity impacts

MIT AI Risk Domains

This tool focuses on four of MIT's seven major AI risk domains most relevant to human subjects research:

1. Discrimination and Toxicity

Concerns about biased or harmful outputs where AI systems may perpetuate unfair treatment or expose participants to inappropriate content.

  • Algorithmic bias across demographic groups
  • Discriminatory predictions or recommendations
  • Toxic or offensive outputs in generative systems
  • Perpetuation of stereotypes

2. Privacy and Security

Protecting sensitive research data and ensuring systems are resilient to breaches, leaks, and unauthorized use.

  • Data confidentiality and de-identification
  • Risk of re-identification
  • Unauthorized access or data breaches
  • HIPAA and Privacy Rule compliance
  • Model inversion attacks

3. Misinformation

Risk of false outputs or hallucinations that can mislead researchers and participants if left unchecked.

  • AI hallucinations (fabricated information)
  • Incorrect clinical recommendations
  • Misleading data summaries
  • Confidence in incorrect predictions

4. Human-Computer Interaction

Preserving human judgment in research and clinical application, ensuring that humans remain the ultimate decision-makers.

  • Over-reliance on AI recommendations
  • Automation bias in clinical decisions
  • Informed consent challenges
  • User interface design and clarity
  • Appropriate human oversight mechanisms

Interactive Risk Assessment

Select filters below to view relevant risks, mitigation strategies, and reviewer prompts:

📋 Reviewer Prompts for IRBs

Purpose: Use these prompts to communicate with research teams about specific risks identified in their AI protocols. Select the development phase and risk domain to see relevant prompts.

How to Use: Filter by phase and domain, then copy the relevant prompt(s) to include in your IRB correspondence, stipulation letters, or review notes.

Key Definitions

AI Human Subjects Research (AI HSR)

AI human subjects research is "Research" involving "human subjects", conducted to develop AI tools.

Common AI Model Types

  • Predictive Models: Systems that forecast outcomes based on historical data (e.g., risk calculators, diagnostic algorithms)
  • Large Language Models (LLMs): AI systems trained on vast text data to understand and generate human language
  • Foundation Models: Large-scale models trained on broad data that can be adapted for multiple tasks
  • Generative AI: Systems that create new content (text, images, code) based on learned patterns
  • Classification Models: Algorithms that categorize data into predefined groups
  • Computer Vision: AI systems that interpret and analyze visual information

Key Regulatory Frameworks

  • 45 CFR 46 (Common Rule): Federal policy for protection of human research subjects
  • 21 CFR 56: FDA regulations for IRB oversight
  • 21 CFR 812: Investigational Device Exemptions (IDE)
  • HIPAA Privacy Rule: Standards for protecting health information
  • ISO 14971: International standard for risk management in medical devices

Belmont Principles Applied to AI

  • Respect for Persons: Informed consent about AI use, explainability requirements
  • Beneficence: Maximizing benefits and minimizing harms from AI systems
  • Justice: Fair distribution of AI benefits and burdens across populations

About This Tool

Development & Validation

The AI HSR Risk Reference Tool was developed through a structured, iterative design process as part of a safety engineering project at the Center for AI Safety (CAIS). The tool has been:

  • Validated at 23+ institutions nationally
  • Shown to improve reviewer confidence by 21%

Methodology

The tool maps risks and safeguards from the MIT AI Risk Library and MIT AI Risk Mitigation Library against:

  • ISO 14971 (risk management for medical devices)
  • 45 CFR 46 (Common Rule)
  • 21 CFR Parts 312, 812, and 820 (FDA regulations)
  • HIPAA Privacy Rule
  • Belmont Principles and Good Clinical Practice (GCP)

Scope & Limitations

Current Version Includes:

  • AI-specific risks under U.S. human subjects regulations (45 CFR 46)
  • Focus on complex AI systems (predictive models, LLMs, foundation models)
  • Four core risk domains relevant to HSR

Future Versions Will Include:

  • International regulations (EU AI Act, GDPR)
  • ISO standards (42001, 23894, 42005, 24368)
  • Patient and community perspectives
  • Integration with IRB electronic platforms

How to Use This Tool

  1. Navigate to the Interactive Tool section
  2. Select the development phase of the AI system under review
  3. Choose relevant risk domains
  4. Review identified risks, mitigation strategies, and reviewer prompts
  5. Use prompts to guide IRB deliberations
  6. Document findings in your IRB review materials

Citation

If you use this tool in your work, please cite:

Eto, T. (2025). AI HSR Risk Reference Tool v2.0: Quick Reference Risk Identification and Mitigation Guide for IRBs Reviewing AI in Human Subjects Research. TechInHSR.

Resources

Acknowledgements

Special thanks to Professor Josep Curto, PhD, at the Center for AI Safety (CAIS) for invaluable guidance, and to colleagues Mark Lifson, Heather Miller, and the broader IRB community for their feedback and support.