Document review is the most expensive part of the EDRM. It’s not even close. Looking at the EDRM, it’s easy to see why; the costs of everything else in it are largely task oriented. The amount of time it takes for someone to push a button to process ten custodians is roughly the same as it is for twenty. There are hosting charges and processing charges based on volumes, but those costs pale in comparison to the hourly rates charged by attorneys to look at documents. Every other portion of the EDRM leads into, or results from, Review, which is the heart of a legal project. Review is on an individual document level, meaning everything needs to be considered and analyzed.
George Socha (currently with Reveal), one of the creators of the EDRM, created a chart which adjusted the size of the individual areas by cost, which demonstrates these cost disparities:
Cutting out expensive review is the easiest and simplest way to lower litigation costs. Plain and simple. Other aspects of the EDRM are simplified by technology and automation. We have now reached the point where review can also be automated through AI, lowering your project costs considerably. How much? Let’s compare it to human managed review.
The Cost Comparisons
When it comes to human managed review, the costs are directly tied to volume and the review rate. If you have 1 million documents and the quoted review speed is 50 docs/hour, the amount of time it will take reviewers to go through the entire set is 20,000 total hours, and then just multiply that by the attorney hourly rate to get the reviewer cost.
Even then, some variables can throw off those results. Some reviewers are generally slower, meaning faster reviewers must make up the difference. In addition, the more categories you need to tag, the slower the review as there are more factors for which the reviewers are required to tag. In recent years, the trend has been to lower the quoted review speeds compared to where they were five or ten years ago when they were 60 docs/hr.. If the rate goes down to 40 docs/hr., that 20,000 total hours now balloons to 25,000 total hours. That number is just for first-level review too; add multiple layers to it, such as a QC review, privilege logging or redaction review, and the hours increase further. QC review usually adds another 10% since that’s usually what is requested to QC. Privilege logging costs vary on the volume to log and usually the level of detail required for the log summaries. Redaction reviews can be simple or flat out ugly, again depending on volume and level of detail.
Another way human managed review increases costs is by pyramiding the internal review structure of a project. Sure, the 25,000 hrs. mentioned above relates to the reviewer hours required to complete first level review, but there is always a review manager, organizing and overseeing the review team. That also increases the cost. In recent years we’ve seen another layer, “review leads,” increase costs, inserted between review management and the review team; these are usually better reviewers but help coordinate the other reviewers for the review manager. Those better reviewers cost more, even though they really serve no additional purpose, and it’s a way to pass on cost from the review provider to the customer to keep better contract reviewers longer term. So, for a project length of 25,000 hrs., if you have 25 reviewers going through documents, you will have one Review Manager, and likely five leads, with five reviewers reporting to each lead.
The costs of the review at that point become formulaic and easily calculable once you know the hourly reviewer/ lead/ manager rates:
- 1,000,000 docs/40 docs/hr. = 25,000 First Level Review hours
- 25,000+ (25,000 x 10%) = First Level + QC Review Hours (27,500 hrs.)
- 27,500/25 reviewers = 1100 hrs., 40 hrs./week = 27.5 weeks (about 6 and a half months) to complete.
- 27,500 x Hourly Reviewer Rate = Cost of First Level Review + QC
- (5 x 1100) x Hourly Team Lead Rate = Cost of the Team Leads
- (1 x 1100) x Hourly Review Manager Rate = Cost of Review Manager
- “d” + “e” + “f” = Total Cost of First Level Review and QC.
Assuming a reviewer, lead and manager rate all at $50/hr. (which is never the case; leads and managers cost notably more), that’s an overall cost of first level review and QC of $1,705,000. And that’s not even including Privilege Logging or Redactions.
Which brings us to the cost of an AI review. Much like every other aspect of the EDRM, the cost of an AI review is fixed to the volume of documents you want to run it across. So, for example, if the cost is $0.45 per document, the cost of an AI review of 1,000,000 documents is $450,000. You get a relevancy determination, a summary and an explanation as to why the AI coded the document that way. The summary is saved to a field that one could use for Privilege Logging as well. Running the documents through AI only needs to be done once and you get the results in 24 hours, not 6 ½ months later. What you choose to QC is entirely up to you, though some small additional review work for QC sampling and to validate results should be conducted as a part of any AI review protocol for defensibility.
Assuming an Hourly Review Rate, a Team Lead Rate, and a Review Manager Rate all the same of only $16.50 per hour already puts you over that cost. Think about that; a rate close to the minimum wage in many states is more expensive for document review than an AI review.
The Quality Comparison
Document reviewer quality varies wildly. To be sure, there are some good reviewers out there making a legitimate career of contract review work. However, there are also numerous reviewers out there that aren’t concentrating on the work, who barely want to understand the project, and are clicking on category boxes just to pull down a paycheck for a gig while they look for something else. With remote review becoming the norm after COVID, it’s difficult to be sure reviewers are even paying attention to your matter for the time they bill, and not just wandering Netflix for hours before breezing through 250 documents without a care to how they are coded. The larger the project, the more likely there are several reviewers like this on your project.
In addition, consistency from reviewer to reviewer varies, even among smart attorneys paying attention. What one reviewer regularly captures may be something one consistently misses. It’s been known for some time that reviewers pale in both recall and precision compared to technological features such as search terms and predictive coding. In fact, one way reviewer errors are regularly captured now is in comparing their coding decisions to the predictive coding applications.
AI has progressed to the point where it can capture the correct information faster, better and more consistently, and is orders of magnitude better at capturing complex conditions, than manual human reviewers are. The results are not even close. To many, there is an apprehension that the “human element” has been removed from the process, but AIs have proven better in every way than the human reviewers. If you conceptualized 25 different computers running the same AI program and gave them human names, you would have a better sense of the consistency, quality, speed and results of the output than you would with 25 human reviewers.
Summary
The cost and quality analysis of an AI review beats human review hands down. There really is no reason not to use AI for your review projects. If you are still apprehensive, contact us about a POC where we can run it against a sample of a recent project you completed and compare the results for yourself.
