Technology-Assisted Review (TAR) Takes Center Stage
Because technology-assisted review (TAR) leverages technology to capture and promulgate human judgments over large collections of electronically stored information, there are claims that it can substantially reduce costs while achieving results that are as good as or even better* than a more traditional manual review. Indeed, a key finding of the Rand Corporation’s Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery is that TAR can identify as many responsive documents as a linear manual review with the same level of consistency, “but with the potential of reducing the hours attorneys must spend [reviewing documents] by about three-quarters.”
These and other TAR-related concepts and claims engendered lively discussion at an April 19 conference hosted by Duke Law Center for Judicial Studies. The conference, which brought together more than 60 federal judges, industry experts, scholars, and senior-level attorneys well-versed in e-discovery, provided a forum to critically assess the claims about technology-assisted review and address its utility and efficacy in “a collaborative environment aimed at informing and developing consensus positions that can guide policy-makers and decision-makers, including the Advisory Committee on Civil Rules.”
H5’s executive managing director & general counsel Julia Brickell and principal consultant Dan Brassil attended the by-invitation-only conference. Mr. Brassil was a participant in the first panel of the conference – “What Is Technology-Assisted Review?” – with other panel members Constantin Aliferis (NYU Center for Health Informatics and Bioinformatics), Amir Milo (Equivio), Ian Wilson (Servient), and Gordon Cormack (University of Waterloo, Ontario), who served as the panel’s moderator. This panel set the stage for the day-long conference by explaining the methodologies underlying TAR as well as addressing confusion arising from various tools on the market and the vendors claiming to provide it.
Other panels examined TAR’s advantages and limitations and addressed protocols and best practices for conducting technology-assisted review. A judicial panel, with the honorable Judges Andrew Peck, Lee Rosenthal, Jeremy Fogel, moderated by Duke Professor Francis McGovern, tackled the management challenges that judges confront when reviewing TAR’s application. The jurists agreed that judicial training in this area is critical, noting that the lawyers must take the lead in bringing the important issues to the fore. The final panel addressed ethical considerations, obligations and concerns that attorneys face arising from the use of TAR.
Some key takeaways from the day-long event:
- When properly-executed, technology-assisted review can reduce e-discovery costs while maintaining quality at least as good as manual review; costs savings will continue to play an increasingly important role in addressing the discovery appropriate in a case.
- TAR is not “plug and play.” Expertise matters, and it’s an absolute necessity when it comes to making competent choices with regard to various aspects of TAR. Lawyers are not expected to become experts themselves, but they have an obligation to retain or confer with individual(s) who have the requisite expertise.
- ABA model rule 1.1 requires lawyers to maintain a level of knowledge that makes them aware of the benefits and risks associated with relevant new technologies. The technologies underlying TAR are likely to become more ubiquitous and counsel should take a proactive stance in gathering and evaluating information.
- It’s a “buyer beware” marketplace. There are a growing number of vendors claiming to provide TAR software and services, but there are also unsubstantiated/uninformative claims about the accuracy that some TAR tools provide. It behooves the buyer to undertake the due diligence required to assess the tools in light of the vendor claims.
- Only two approaches to technology-assisted review have been tested and shown to be effective: linguistic rules-based (an iterative keyword-based approach used by H5 and others) and supervised/active machine learning (aka “predictive coding”). The approach taken may depend on particulars of the matter and the goals of the client, but appropriate attention should be directed to process and measurement.
- TAR is a process and requires development of a workflow. Advance preparation is important and pre-testing is recommended. Lawyers should investigate tools/providers/approaches to ensure that the appropriate workflow process has been established before a matter arises in order to be able to hit the ground running when it does.
- Defensibility depends on the validation of (i) tools (ii) processes and (iii) results.
- Although there have been some judicial decisions related to TAR, judges are empowered to address controversies (Article 3); it is the role of attorneys to identify issues and bring them to the attention of the court rather than expect the court to opine on various aspects of TAR unless there is a particular issue raised in the case.
There is undoubtedly much more conversation to come about this topic and efforts such as this conference to raise the level of the awareness and provide a forum for intelligent discussion are much needed – and appreciated.
*(See Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, XVII RICH. J.L. & TECH. 11 (2011), http://jolt.richmond.edu/v17i3/article11.pdf.)