Artificial intelligence (“AI”) isn't just a buzzword for eDiscovery practitioners. It's been a part of their standard toolset for a decade now, starting with early “predictive coding” tools. eDiscovery now supports a robust, mature selection of technology-assisted review (“TAR”) AI technologies, with Continuous Active Learning being the most widely adopted. eDiscovery Practitioners thus have direct experience with the benefits (and limitations) of AI. Much of this experience can be leveraged to help businesses in IBOR transition projects.

As eDiscovery practitioners have come to learn, results aren't all that matters. AI users need to be able to explain their results, whether they are flags on potentially privileged documents, recommendations for mortgage rates, or identifying IBOR provisions in contracts. A black box AI that takes inputs and produces conclusions without an explanation that makes sense to humans can be hard to defend. If its conclusion is wrong, there is no easy way to explain why or how to fix the problem. And if it's right, it can be hard to explain why. AI that is “explainable” is more likely to gain user trust and acceptance.

What explainability features should we look for in a TAR tool? The National Institute of Standards and Technology ("NIST") white paper " Four Principles of Explainable Artificial Intelligence," published for comment on August 18, 2020, defines four fundamental qualities to look for in a TAR tool:

  • Explanation. It supplies evidence, support or reasoning for its results. That means an audit trail for the TAR tools' estimates, not just for those of human reviewers.
  • Meaningful. Its explanation must make sense to its intended audience. An explanation that works for data scientists with PhDs in statistics is not likely to be understandable to a job applicant or a judge or jury. Defending the tool means convincing the final decision-maker, not a scientific body.
  • Explanation Accuracy. Its explanation must be right. That doesn't mean that the tool's results must be right (though, of course, that's preferable). But if its conclusion is wrong, the tool should be able to explain why.
  • Knowledge Limits. It must flag cases where it is less confident in its conclusions. Put another way, the tool should make it easy for its users to know how sure it is. Users can then make an informed decision about how much to trust the tool and how much to rely on human review.

The NIST white paper thus provides a framework to assess—and ultimately defend—TAR tools. It stresses that a well-designed and well-managed TAR tool, for all its complexity, may be easier to explain and defend than a process relying on unassisted human judgment.

As we employ AI beyond the eDiscovery realm and into other results-oriented projects like IBOR remediations, the NIST white paper gives us a roadmap on how to defend the results.

Visit us at mayerbrown.com

Mayer Brown is a global legal services provider comprising legal practices that are separate entities (the "Mayer Brown Practices"). The Mayer Brown Practices are: Mayer Brown LLP and Mayer Brown Europe – Brussels LLP, both limited liability partnerships established in Illinois USA; Mayer Brown International LLP, a limited liability partnership incorporated in England and Wales (authorized and regulated by the Solicitors Regulation Authority and registered in England and Wales number OC 303359); Mayer Brown, a SELAS established in France; Mayer Brown JSM, a Hong Kong partnership and its associated entities in Asia; and Tauil & Chequer Advogados, a Brazilian law partnership with which Mayer Brown is associated. "Mayer Brown" and the Mayer Brown logo are the trademarks of the Mayer Brown Practices in their respective jurisdictions.

© Copyright 2020. The Mayer Brown Practices. All rights reserved.

This Mayer Brown article provides information and comments on legal issues and developments of interest. The foregoing is not a comprehensive treatment of the subject matter covered and is not intended to provide legal advice. Readers should seek specific legal advice before taking any action with respect to the matters discussed herein.