Document Review: Human vs. Machine or Human AND Machine?
The by now age-old rivalry between person and machine continues ever onward with concerns around technology-assisted review. The idea of giving control of document review decisions to a machine can understandably give one pause. What’s going on in the machine and how is a human to know?
But there is a highly reliable way to use machines for review where you can actually know exactly what the machine is doing and how it’s making decisions.
Called deterministic or “rules-based” technology-assisted review, this approach leverages the power of the machine to replicate decisions that humans make (only better) to determine relevance as humans do. It represents a very different use of technology—one that is similar to linear review, but has much greater accuracy, consistency and almost unlimited scalability. This approach has been around for decades and in the right hands achieves extremely high accuracy (both high recall and high precision).
So how does this deterministic technology-assisted review work? It’s probably easiest to understand by contrasting it to traditional linear manual review. In manual review, an expert attorney(s) performs legal analysis to determine “what” should be considered relevant, and then provides a coding manual to other attorneys so they can emulate the lead attorney’s decision-making process. The coding manual is essentially a set of instructions, or “relevance rules,” that govern what should be considered relevant. In practice, the reviewers are identifying content or attributes that adhere to the rules. This can be as simple as the mention of certain key phrases, or complex combinations taking into account the sender, date and mention of certain subject matter. Reviewers go through documents one at a time and when they identify the presence of these characteristics, they code the document as relevant, effectively applying the “rules” the expert(s) set forth.
In the deterministic machine approach, that same expert attorney(s) essentially imparts the same relevance rules to the machine. The process typically involves help from individuals with linguistic and computer analytic expertise, as they translate the attorney’s relevance rules into instructions the computer can understand. The explicit relevance rules provided by knowledgeable humans enable the machine to emulate the attorney decision-making process.
The machine is thus executing a “rules-based” approach. In essence, it goes through the document population evaluating every document to determine relevance—that is, to see if it adheres to the rules—just as humans would, except without fatigue or distraction. Sampling protocols and quality control processes can be used to check the results and may require further consultation with the expert(s) to adjust the rules. If so, changes are quickly made to the rules, the review is rerun, and the entire document population reflects the change—something that doesn’t often happen with human review.
It should be easy to see that once the machine knows the rules, the execution of the process is fast and efficient, and therefore less costly. The rules, in the form of (usually) thousands of search queries, can be printed and provided to the judge or the other side, so the process is 100% transparent and defensible.
While deterministic (rules-based) approaches have been in use for many years, they are gaining increasing acceptance from the legal community because they make it possible to realize the scale and cost advantages of technology-assisted review with transparency, consistency and accuracy that is easy to defend and substantiate.
So if you’ve been stuck in the human vs. machine debate when it comes to modern document review, welcome to a new world where your review approach can be human and machine.