The Search Duel: What is the Best Method?
.jpg)
At Legal Tech’s recent conference in New York, most of the discussion was around search: is it defensible, how good is it relative to other methods, what did Magistrate Judge Facciola really mean in his O’Keefe opinion, how do you know if the search methodology picked up all the relevant documents, what is the best search engine, etc. To borrow a famous quote from Winston Churchill, it felt like the consensus regarding key word searching could be summarized as follows:
It is the worst form of search except all the others that have been tried.
The problem with debates about how best to judge the effectiveness of a given search methodology is that it often ignores two important considerations: what is the cost tradeoff and what is the “gold” standard to which to compare it. The “gold” standard is usually viewed as the ability of experienced litigators to find the relevant documents by reviewing (reading) all the documents in discovery.
A friend recently shared the following real experiment that he did at his firm. He challenged a group of four seasoned litigators in his firm to see if using key word searching of approximately five boxes of documents (2,000 documents or 10,000 pages) he would find more relevant documents in the case than they would find reviewing every document the old fashioned way. It took our friend under one hour to search the documents and he found approximately 500 responsive documents. The four lawyers took a total of 32 hours to review all the documents. They found about 300 responsive documents. When they compared responsive sets, it was clear that the lawyers had missed almost 40% of the responsive documents in the data set. Not only had the review team missed a lot of responsive documents, the cost differential was significant: $200 versus over $6,000 (assuming $200 per hour).
Although key word searching has many limitations, and like any method, needs to be carefully thought out when used, it is a very effective way to quickly and cost effectively focus a review.
