What we talk about when we talk about TAR, part 3

09
Jul
2013

Cynthia Courtney

Part 1 of this three-part article discussed the terminology of TAR:  Key word search, both simple and advanced, conceptual search and predictive coding. Part 2 discussed two specific statistical methodologies commonly used in TAR: Support vector and conceptual search, along with how they work, what they do and how they can be used. Our third and final part will focus on the practical: What do the cases say? How can parties use TAR in a defensible way? How can TAR help to achieve proportionality?

The Cases

Courts do not seem inclined to either require parties to use TAR or prohibit them from using it.  

• A court allowed the use of TAR over the receiving party's objection in Global Aerospace, Inc. v. Landow Aviation, L.P., et al, No. CL 61040 (Cir. Ct. Loudon Cnty., April 23, 2012).  The plaintiffs argued that the court's approval of TAR constituted its agreement that defendants need only produce a certain percentage of responsive documents. While allowing the use of TAR, the court explicitly preserved the receiving party's ability to later challenge the production's sufficiency.  
• Plaintiffs sought to compel the defendants to use TAR over objection in Kleen Products LLC, et al. v. Packaging Corp. of America, Civ. No. 1:10-cv-05711 (Aug. 21, 2012 N.D. IL).  After a protracted struggle over the issue involving two separate hearings and multiple written submissions, the plaintiffs withdrew their demand as to the existing document request, to which documents had already been produced. For future production requests, the parties agreed to meet and confer regarding production methods, raising the possibility that plaintiffs might renew their demands about TAR.
• A court issued a sua sponte order directing both parties to show cause why they should not use a single TAR provider in EORHB, Inc. v. HOA Holdings, LLC, No. 7409-VCL (Del. Ch. Oct. 15, 2012).  However, after a show cause hearing, the court lifted the order, allowing defendants to proceed with the vendor of their choice and plaintiffs to proceed without using predictive coding at all, given the low volume of documents they were expected to produce.
• In In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., No. 3:12-MD-2391 (N.D. Ind. Apr. 18, 2013), the defendant in a product liability case first used search terms to cull nearly 20 million documents over the objection of the Plaintiffs' Steering Committee, which urged the defendant to await a ruling on centralization from the Judicial Panel on Multidistrict Litigation before proceeding with document production.  When defendant thereafter used predictive coding, plaintiffs argued that using search terms had tainted the document pool and asked the judge to require the defendant to start over, using only predictive coding.  The judge refused to require a do-over, on the ground that defendant had fulfilled its discovery obligations.

By contrast, courts seem more likely to get involved in the rules of engagement around the use of TAR.  The best-known case addressing TAR is DaSilva Moore v. Publicis Groupe and MSL Group, 11 Civ. 1279 (SDNY), where the parties argued over the manner in which TAR would be employed. US Magistrate Judge Andrew Peck emphasized that parties intending to use TAR must choose a reliable vendor and program; and perhaps most importantly, must design an “appropriate process” that includes “appropriate quality control testing.”  Later, U.S. District Judge Andrew Carter, in reviewing Judge Peck's TAR decision, approved Judge Peck's protocol on the basis that it was reliable, standards-based, and allowed participation by the plaintiffs, who were given the ability to object once production was made if they believed it to be insufficient. 

Judge Peck's opinion specified a number of steps that parties should take to assure the reliability of a party's use of TAR:  

1. Bring both vendor experts to a court hearing to respond to the magistrate judge's questions.
2. Allow the requesting party to view the documents that were used to train the TAR system, both those that were marked responsive and those that were marked non-responsive.
3. Allow the requesting party to view the additional documents that were used to “stabilize” the TAR system, whether or not marked responsive.
4. Do not adhere to an arbitrary number of documents that will be produced, without reference to the statistical results.
5. Do not limit up front the number of iterative reviews used to “train” the system, but rather, assessing whether the system had “stabilized” before stopping the iterative reviews.

The DaSilva Moore principles of cooperation, transparency and attention to a strong workflow have a concrete and defined role in the case of In re Actos Products Liability Litigation, MDL No. 6:11-md-2299 (July 27, 2012).  In Actos, the parties entered into a detailed protocol, wherein three experts from each side met, collaboratively reviewed the training set of defendants' documents together, and agreed upon relevance determinations for all non-privileged documents in the set. Robust protections to guard documents subject to privilege and confidentiality were included. This process may not be right for every case in which TAR is used, but the opinion is a good exemplar of a transparent process.

TAR and Defensibility

It is early days yet in the world of TAR court decisions.  Parties can choose, but cannot be forced, to use TAR applications to help them identify production-worthy documents and make decisions about review.  Producing parties who use TAR can expect the receiving party to press for transparency:  Sharing of the initial sample or seed set, disclosure and perhaps agreement on key statistical metrics, and possibly joint review of the initial sample set to agree upon relevance, among other criteria. However, it is not yet known whether courts will require the sharing of this information.  On the one hand, courts traditionally have stayed out of micro-managing the details of a party's decisions about a production unless it is challenged as insufficient.  On the other hand, a producing party who uses new and complex technology as an aid or device to more efficiently identify relevant documents should expect the receiving party to ask questions about the methodology.  

It will not be a particular statistical metric or number of sample documents that defines a defensible protocol, but rather a documented workflow that details the steps that will be followed during the TAR project:  Initial sampling and review, stabilization of the system, sampling of the results.  Some of these details will need to be shared with the receiving party; it is not yet clear which details or the extent of sharing that will be required.

TAR and Proportionality

In addition to its obvious potential as a culling and organizational tool, TAR can be an instrument of proportionality.  With a sound workflow and reliable sampling, parties can estimate early in the discovery process how much review must be conducted to achieve a production of most of the relevant documents.  With this knowledge, costs of review can be calculated and informed decisions made about what to review, and perhaps whether to request relief from the court, such as limiting discovery or cost shifting.

Conclusion

Technology Assisted Review, or TAR, is a provocative technology that litigants are using to cull irrelevant documents and identify documents that are important to a case.  Both producing parties and receiving parties are feeling their way with TAR, with relatively little guidance from courts.  Strong workflows and careful attention to protocol will improve the results and defensibility of TAR.