On January 9, the Office of the National Coordinator for Health Information Technology (ONC), part of the Department of Health and Human Services (HHS), published in the Federal Register its final rule titled Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (the Final Rule).1 The Final Rule, a proposed version of which was published for comment in April 2023,2 is designed to enhance the access, exchange, and use of electronic health information, while also advancing equity, innovation, and interoperability in health information technology (HIT).3 The Final Rule imposes significant new transparency and risk management requirements for the use of artificial intelligence (AI) and algorithms used in certified health information technology.
Background
ONC is responsible for administering the ONC Health IT
Certification Program (the Program), which provides certification
criteria for health IT developers and their health IT modules4.
In 2010, the Program established certification criteria for
clinical decision support (CDS) within health IT modules5.
CDS encompasses a variety of tools to enhance decision-making in
the clinical workflow, including computerized alerts, relevant
clinical guidelines, and drug-disease interaction checks. In 2012,
ONC began requiring health IT modules to (1) support evidence-based
CDS grounded on a defined set of data elements, (2) support CDS
configuration for both inpatient and ambulatory settings, and (3)
display source attribute or bibliographic citation of CDS.6
In the past decade, health IT modules have played an increasingly
significant part in healthcare across various clinical settings,
but the use of AI in healthcare has gone largely unregulated. In
late 2022 and early 2023, the Biden administration published three
guidance documents outlining principles to prevent discriminatory
algorithmic decision-making and advance accountable AI systems.7 In
October 2023, the Biden administration directed HHS to prepare a
strategy for the responsible deployment and use of AI in
healthcare, which led to the present rulemaking.8
Decision Support Interventions and Predictive Models
Decision Support Interventions
The Final Rule replaces the Program's CDS criterion with a new certification criterion, "decision support interventions" (DSIs), which is designed to ensure that health IT modules "reflect an array of contemporary functionalities, support data elements important to health equity, and enable the transparent use of predictive models and algorithms to aid decision-making in healthcare."9 As discussed below, the Final Rule differentiates between "evidence-based DSIs" and "predictive DSIs" and imposes different transparency requirements on each.
Evidence-Based DSI
Evidence-based DSIs are non-predictive interventions actively presented to users in clinical workflow to enhance, inform, or influence decision-making related to the care a patient receives.10 Evidence-based DSIs rely on pre-defined rules based on expert consensus rather than empirical data to support decision-making, such as the SOFA Index and NYHA Heart Failure classification.11 The new requirements for evidence-based DSIs generally track evidentiary requirements that were part of the CDS criterion.12
Predictive DSI
Predictive DSI is defined as "technology that supports
decision-making based on algorithms or models that derive
relationships from training data and then produce an output that
results in prediction, classification, recommendation, evaluation,
or analysis."13 Predictive DSIs "learn[]
or deriv[e] relationships to produce an output."14
Predictive DSIs may include techniques such as algebraic equations,
machine learning, and natural language processing.15
Some of these tools might be used to predict, for example, whether
a given image contains a malignant tumor or whether a given patient
is at risk for sepsis.16 Large language models and
other forms of generative AI also will likely be classified as
Predictive DSIs, to the extent they are supplied by developers of
certified health IT and are used to support decision-making.17
Source Attributes
The Final Rule requires health IT developers to produce an expanded set of information, or "source attributes," related to both evidence-based DSIs and Predictive DSIs. "Source attributes" are categories of technical performance and underlying quality information used to create both evidence-based and Predictive DSIs.18 The new requirements aim to reduce uncertainty, enhance market transparency, and establish consistency in information availability.19
Requirements for DSIs
Under the Final Rule, evidence-based DSIs must now support 13
source attributes, including the developer and funding source of
the intervention, as well as the intervention's use of patient
demographics data and social determinants of health data.20
The Final Rule imposes more expansive transparency requirements on
Predictive DSIs, which must support 31 source attributes. Among
other requirements, developers of Predictive DSIs must produce
information about the intervention's training data set,
external validation process, and quantitative measures of
performance, as well as the process used to ensure fairness and
eliminate bias in the development of the intervention.21
The Final Rule also establishes capabilities that health IT modules
must support related to source attributes.22 First, the
module must provide plain language descriptions of all required
source attributes.23 Second, for Predictive DSIs,
the module must indicate when information is not available for
review for certain source attributes. If and when information
related to these source attributes is generated, the developer of
certified health IT must make this information available to
users.24 Finally, the module must
enable a limited set of identified users to record, change, and
access the required source attributes.25
Starting on January 1, 2025, and on an ongoing basis thereafter,
developers of health IT modules certified to § 170.315(b)(11)
must review and update, as necessary, required source attribute
information, as well as risk management practices described in
§ 170.315(b)(11)(vi) and summary information provided through
§ 170.523(f)(1)(xxi).26
Coordination With the Food and Drug Administration
Whether DSIs enabled by or interfaced with certified health IT
are subject to FDA regulation is separate and distinct from the
question of whether a developer or a particular technology is
subject to regulatory oversight by ONC's Program.27
In finalizing the rule, ONC declined to exclude from the definition
for Predictive DSI software that are FDA-regulated medical devices
or to exclude third-party software that qualify as non-device
software functions per the statutory exemption for certain CDS
software functions.28 Thus, technologies that meet
the definition for Predictive DSI within the Program may be
considered non-device CDS, be considered CDS with device software
functions, or lie outside of FDA's purview, depending on the
specifics of the technology.29 As explained by ONC, FDA and
ONC have separate and distinct authorities and regulate for
separate and distinct purposes with separate and distinct policy
objectives.
Although FDA-regulated CDS are not exempt from the Final Rule, ONC
worked with the FDA to minimize duplication of effort and maximize
alignment across the distinct and different authorities. For
example, ONC coordinated with FDA to ensure source attributes are
complementary and not conflicting with the information FDA
describes in its September 2022 CDS software guidance.30
For CDS software that are medical devices and the focus of FDA
oversight, the requirements of the Final Rule are consistent with
best practices and recommendations provided by the FDA.31
This consistency across agencies could reduce burdens on developers
who may be responsible for meeting both FDA and ONC requirements
for three reasons.32 First, an entity that develops
device software that also meets the definition of Predictive DSI
would be able to fulfill informational requirements for both FDA
and ONC purposes using the same or similar information. Second,
such software may be eligible to be considered non-device CDS
according to FDA guidance if the software developer fulfills
informational requirements pursuant to the Program. Finally,
burdens will be reduced across entities regulated by FDA and ONC
because an entity that develops device software that also meets the
definition of a Predictive DSI could leverage Program requirements
to enable users to select Predictive DSIs and access source
attribute information. These capabilities could serve as the
technical means to deliver information to users about the
credibility of the device software function that is necessary for
"independent review," without having to build a parallel
technological means to deliver such information.33
However, a determination regarding the information necessary for
independent review will continue to lie with the FDA.34
Intervention Risk Management
The Final Rule mandates that health IT developers apply intervention risk management (IRM) for each Predictive DSI included in their health IT module.35 Health IT developers will need to analyze potential risks and adverse impacts by considering the DSI's validity, reliability, robustness, fairness, intelligibility, safety, security, and privacy,36 and implement practices to mitigate those risks. Developers must also submit summary information of IRM practices through a publicly accessible hyperlink that allows any person to access the summary information directly.37
Implications
In addition to imposing detailed new requirements on HIT
developers, the Final Rule makes significant changes to access and
support for source attributes, transparency of predictive DSIs, and
intervention risk management practices. The Final Rule focuses
significantly on enhancing the trustworthiness, transparency,
racial equity, and innovation of DSIs to ensure high-quality
decisions that improve and support patient care. ONC believes these
requirements will help to address disparities and bias that may be
propagated through DSIs, as well as to establish consistency in
information availability, improve overall data stewardship, and
guide the appropriate use of data derived from health information
about individuals.38 ONC also believes the
increased transparency the Final Rule requires will allow users to
make better informed decisions about whether and how to use
emerging software.39
By taking advantage of the new transparency requirements, users of
Predictive DSIs can become smart shoppers in a rapidly evolving
health IT landscape. Going forward, health IT developers and those
interested in developing or collaborating on DSIs will be required
to make significant investments and updates in current and future
systems and technology to meet the Final Rule's DSI
requirements. Life sciences companies, labs, pharmacies, and others
with financial interests in DSIs should also be aware of ONC's
acknowledgement that financial arrangements with DSI developers
could implicate the Anti-Kickback Statue and that ONC is focused on
increased transparency around such arrangements to mitigate the
risk of bias or potential patient harm.
Footnotes
1 Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing, 89 Fed. Reg. 1192 (Jan. 9, 2024).
2 Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing, 88 Fed. Reg. 23917 (Apr. 18, 2023) (proposed rule).
3 89 Fed. Reg. at 1193.
4 Id. A Health IT Module is any service, component, or combination thereof that can meet the requirements of at least one ONC certification criterion, such as Electronic Health Record (EHR) software. See 45 C.F.R. § 170.102.
5 Id. at 1202.
6 Id. at 1231. ONC finalized its updated CDS criterion in 2015. See 45 C.F.R. § 170.315(a)(9).
7 See White House, Principles for Enhancing Competition and Tech Platform Accountability, Sept. 8, 2022; White House, Blueprint for an AI Bill of Rights (Oct. 4, 2022); E.O. 14091, 88 FR 10825-10833.
8 E.O. 14110, 88 FR 75191.
9 89 Fed. Reg. at 1196.
10 Id. at 1240.
11 Id. at 1246.
12 Id. at 1239.
13 Id. at 1244; see also 88 Fed. Reg. 23917, 23788 (proposing a broad interpretation of "intended to support decision-making").
14 89 Fed. Reg. at 1243.
15 Id.
16 Id. at 1245-46.
17 Id. at 1246.
18 Id. at 1196-97.
19 Id. at 1233-34.
20 Id. at 1431.
21 Id.
22 Id. at 1256.
23 Id.
24 Id. at 1256-57.
25 Id. at 1257.
26 Id. at 1254.
27 Id. at 1245.
28 Id.
29 Id. at 1262.
30 Id.
31 Id. at 1263
32 Id.
33 Id.
34 Id.
35 Id. at 1272.
36 Id. at 1274
37 Id.
38 Id. at 1234.
39 Id. at 1233.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
