Applying Moneyball Based Judicial Analytics to Construction Contract Disputes

When construction contract disputes go through project level negotiation, a dispute resolution board, or mediation and are not resolved successfully, the parties must entrust the outcome to third-party decision makers in arbitration and litigation.  However, a party’s anticipation of the received response of third-party decision makers is often reduced to informed speculation, gut feels, and guesses. Today, however, the current state and future of legal analytics offer construction attorneys the promise of greater predictability on how those arbitrators and judges will perceive and decide cases. 

Since the early 19^th century, attorneys have tried to gain an advantage over each other by attempting to predict dispute outcomes with varying levels of success. However, in the past decade, the digitization of court records and arbitration decisions has led to a new tool in this quest: analytical evaluation of court, arbitrator, and attorney decisions and/or filings. Not only can analytics provide insight into how an arbitrator, judge or attorney may act, but it can even predict, with high levels of accuracy, what an arbitrator or judge may grant, deny, or award.

In other words, statistics and complex computer programs can provide a scientifically proven way to not only keep your clients out of legal disputes, but when disputes do occur, analytics can provide an increased chance of winning based upon proven biases found in the arbitrator or judge who may oversee the case.

This article introduces construction attorneys to the field of litigation-judicial-arbitration (lit-jud-arb) analytics, how attorneys can use analytics for arbitration or court, and the likely future of legal analytics. When negotiation, Dispute Resolution Board’s (DRB’s), and mediation fail in resolving construction contract disputes, the next steps of arbitration and litigation generally result in long and costly procedures that can take years to complete.  Legal analytics is an invaluable tool that allows construction attorneys and their clients to more accurately and scientifically predict outcomes and identify weaker or futile arguments, claims, or defenses before plunging into the effort and expense of arbitration or litigation. 

What is Litigation, Judicial, and Arbitral Analytics?

Litigation, judicial, and arbitral (“lit-jud-arb”) analytics is a broad term which quantifies the writing, decisions, awards, or any other defined metric that can evaluate a mediator, arbitrator or judge for their potential to win or rule in a case.  In addition to measuring attorney and arbitrator metrics, analytics—derived from collected metrics on judges, attorneys, or arbitrators—can be utilized for a plethora of other purposes such as: due diligence, win/loss ratios, judicial history, cases before a particular judge, legal writing styles, precedent preferred by a particular judge, discovery grant rates, and many more undiscovered analytics.  In addition to measuring judicial and arbitrator metrics, analytics allows parties and their attorneys to pick and choose the specific attorney team for a particular case, as if they were picking players of a baseball team, in order to increase the odds a motion or claim is granted or denied based upon their percentages before a particular arbitrator or judge.

Given the broad number of analytics possible, it is best to categorize these into three areas: “(1) descriptive analytics – gathering, organizing, tabulating, and depicting data; (2) predictive analytics – using data to predict future courses of action; and (3) prescriptive analytics – recommendations on future courses of action.”

For example, in Arditi and Tokdemir’s paper, they set out a list of thirty-four input and output factors that affect litigation outcomes when training Artificial Intelligence (AI) to predict construction dispute outcomes.  Some of these features are as follows: contract value, type of designer, misrepresentation of site conditions, contractor coordination, misrepresentation of supervision, defective or deficient contract documents, surety bond issues, and many more.  From this information, Arditi and Tokemir were able to predict construction disputes in courts with a 83% accuracy rate.  In contrast, arbitrations are confidential and are rarely published, so most metrics from arbitrations likely follow publicly available metrics derived from judicial cases.  Since arbitration metrics are derived from judicial sources, their usefulness has limits since awards could be based upon unknown factors not found in public sources.

Essentially, analytics provides a methodical and novel method for measuring attorneys, arbitrators, or judges who rule or act on behalf of parties to construction contract disputes.  Thus, all construction attorneys should review any analytics they may find on their legal research services: Westlaw, Lexis Nexis, Premonition, or another service, but these metrics should be heavily weighted but not dispositive since there is always a margin-of-error in any prediction.

How to Use Lit-Jud-Arb Analytics

Lit-jud-arb analytics is an emerging field without many defined borders, but construction attorneys should focus on certain “bedrock” analytics for insight on their case and developing the best possible strategy.  For example, analytics can be used to determine which cases and precedent arbitrators or judges are likely to accept, the ratio of grants/denials/partials an arbitrator or judge has historically done, which attorneys are successful with those arbitrators or judges, and many more detailed analytics.  Analytics can also provide—in limited circumstances—predictions on arbitrator or judicial rulings prior to any case being filed in a specific arbitration tribunal or at a specific courthouse.  

Since analytics is still a burgeoning field, many metrics have yet to be adopted or investigated in arbitration tribunals and courts throughout the United States; however, pioneering work in analytics across the globe and in limited circumstances the U.S., provides some guidance on where the U.S. is likely heading.

One prime example of the insight analytics provides when metrics are available is a 2016 study of Dutch court decisions.  With the available metrics, the author found a higher win rate for female attorneys representing female clients compared to male attorneys in the same scenario.  In order to find this bias, author S.T.L.A. Mulders web scrapped this data from Dutch court websites for 13,212 cases decided between 2013–2016.  In this data, the gender of the attorney was found by cross-matching the licensed attorney database to the name scrapped from the court cases.

Another fascinating arbitral analytics example is provided in a case involving the Russian Federation as a party.  In this case, the Russians were successful in partially overturning an arbitration award against them based upon several arguments including a discrepancy in the billable hours by the arbitration panel.  Essentially, the tripartite arbitration panel was required to write the final arbitration award—under the rules provided in UNCITRAL and accepted by the parties—equally, but upon closer inspection, the Russians discovered the arbitration secretariat was billing the majority of the written award hours in violation of the UNCITRAL rules.  Thus, a Dutch court in 2016 quashed the original arbitration award based upon other grounds while neglecting to address the billable hours argument.

However, after further inspection of this case, stylometric analysis, the analysis of words and the writing style of any person, would have also proven the arbitrator panel’s secretariat was writing the award instead of the full panel since the award’s wording and writing style closely matched the secretariat’s writing style.  Thus, the award was not written by the panel as the UNCITRAL rules required and would likely be reversable if proven on appeal.

These specific tools used by the Russian Federation could also be useful for construction arbitrations when a client wishes to challenge an arbitration award.

Closer to Home: Use of Lit-Jud-Arb Analytics in US Securities Arbitrations

Legal analytics is a growing field that will continue to evolve and expand over the coming decade.  Given where the field stood in the preceding decade, it can be hard to predict what new fields analytics will lead to in the future.  In certain respects, the future is now.  Arbitrations involving financial firms, their brokers, and investors administered by the Federal Industry Regulatory Authority (FINRA) are published and available on-line and offer a substantial source of data for use in analytics.  Co-author Davies applied lit-jud-arb analytics to this data source to extract fascinating, valuable, and specific insights to prospectively predict outcomes and identify arguments and language that are more likely to be successful in FINRA arbitrations.  In his forthcoming law review article Arbitral Analytics: Moneyball Based Litigation/Judicial Analytics Used to Predict Arbitration Claims and Outcomes, co-author Davies demonstrates the effective use of AI to not only increase the accuracy of arbitral analytics, but the ability of analytics to predict outcomes based upon stylometric analysis of words used in arbitration awards.

As an example, co-author Davies found the word “remaining,” when written in an arbitration award, was seventeen times more likely to result in a granted arbitration award as compared to a denial of an award.  Additionally, the top twenty words associated with a high grant rate for FINRA arbitration awards has been included in Figure 1 below.  Thus, an attorney should consider utilizing certain words over others in FINRA arbitration claims to increase their win rates.  Moving forward in the construction industry, this type of analytics could be developed and applied to construction contract disputes before an arbitrator or a judge.

Figure 1:

Conclusion

In conclusion, the application of lit-jud-arb analytics, combined with AI, is likely the future of the legal industry; and with continued development, could be the future in the legal arena that serves the heavy civil infrastructure construction industry for the resolution of construction contract disputes.  

Stay tuned for more to come on this subject.