In what seems like a sudden development long in coming, “technology-assisted review” or “computer-assisted review” has finally made its way from the back room to the courtroom as Magistrate Judges Peck and Nolan opine in matters that that have raised some of the most challenging e-discovery issues since the amendments to the Federal Rules. (See Monique Da Silva Moore, et al., v. Publicis Groupe & MSL Group, and Kleen Products LLC v. Packaging Corp. of America, respectively.)
These days, “technology-assisted” or “computer-assisted” review are terms broad enough to be almost uninformative. However, some tools hovering under this umbrella have particularly topical import, based on the two matters cited above. Such tools include those with marketing monikers such as “predictive coding” or “computer based advanced analytics,” whose methods fall under the general rubric of “machine learning.” Despite the recent hype that would make it seem as though they’re new to the scene, machine learning tools have actually been around for a long time. Additionally, academic studies over the past 30 years have shown that approaches using machine learning achieve wildly varying levels of accuracy, mostly low.
How is a practitioner supposed to know something like that? Indeed, the challenge in understanding and evaluating technology-assisted review tools is that the details of their operation can be hair-raisingly complex. And trying to understand how accuracy is measured is not for the faint of heart or mathematically challenged (recall, precision and F1 scores, anyone?). Unless statistics, linguistics and computational analytics are up your alley, you may have no choice but to locate the alley where experts dwell.