chevron-down Created with Sketch Beta.


Automating the Jobsite: Leveraging Artificial Intelligence for Risk Mitigation and Disputes

Marshall Harris


  • AI is helping construction firms reduce project risk, improve performance, mitigate impacts, and improve jobsite documentation.
  • Enabling deeper insights into labor and equipment.
  • Generating cost analytics with machine learning and natural language processing.
Automating the Jobsite: Leveraging Artificial Intelligence for Risk Mitigation and Disputes
pcess609 via Getty Images

On a recent project assisting an established international builder in quantifying the delays and associated increased costs on a large condominium project, we ran into a problem that I would say was caused by the human element. We found that the builder’s own project-control protocols had not been rigorously followed during the project: The daily reports were incomplete, unreliable, or missing for large chunks of time; daily site photos had not been taken consistently, and the photos that had been taken were not filed or labeled; daily manpower tracking had been done manually but was incomplete and often inconsistent with the site entry turnstile data; and the cost-estimating team failed to consider “lessons learned” information from the firm’s prior projects in the region when the bid was being developed, which led to underestimating certain elements of the work.

Due to these issues with the client’s documentation and processes, supporting the client’s delay claim was a significantly more arduous task than anticipated, and while we were still able to prepare a limited claim package, our client’s submission could have been significantly stronger with better data and documentation. It was also clear to our client that, if better data had been available during the project, the company could have essentially had an “early detection” system that identified potential delays and impacts before they became crises.

All over the world, construction managers, general contractors, and subcontractors face a similar issue: They lack documentation and actionable information on their projects, and often this issue is the result of lapses by their own project teams. Frustratingly, this can prevent the same parties from taking timely action to avoid project impacts. And in some instances, claims for excusable delays and increased costs that would otherwise be valid are instead denied due to these internal failures.

Fortunately, with the emergence of new software and hardware technology solutions to improve jobsite documentation and make use of existing, readily available data—on an automated basis using artificial intelligence (AI)—many of these issues can be avoided. AI is a general term for tools that mimic human cognitive functions such as problem solving, pattern recognition, and learning; and AI itself is a broad category that encompasses a range of subset technologies, such as machine learning, natural language processing, and computer vision, among others.

Despite the usefulness of these tools, a recent survey of 130 of the largest architectural, engineering, and construction firms in the United States found that less than 8 percent of respondents were using AI technologies on more than a handful of pilot projects, and at least 46 percent of respondents were not using AI technologies at all. BD+C Giants 300 Technology and Innovation Study 2019 (Building Design + Construction, Apr. 27, 2020) (survey conducted by SGC Horizon LLC).

With these technologies finally starting to mature, contractors and subcontractors should consider investing in solutions that automate traditionally manual processes and leverage AI to make actionable, real-time information available in a digestible format to identify potential issues and impacts. The ability to do this quickly will, ideally, minimize the need to submit a claim package at the end of the project. Currently available technologies leverage AI to do many things automatically, including recording progress on a jobsite over time, keeping track of the quantity and locations of personnel and equipment employed on the project, and making sense of the company’s historical cost and performance data. Construction companies and developers can rely on these technologies to gain insights that they can use to identify and reduce project risk, improve performance, and assist with forensic project analysis.

The Impact of Automated, Rich Visual Documentation

Today, the effortless, automated generation of comprehensive visual documentation enabled by new technology applications is a game changer that has countless applications for enhancing real-time risk mitigation and improving dispute outcomes. These technologies often employ AI not only to capture the visual data but also to make the visual information easier to access, interpret, and act on.

Frictionless, structured visual data capture. One software solution is enabling construction personnel to effortlessly create comprehensive, 360-degree images of the jobsite and view the imagery in an interactive interface. The imagery is captured passively via hardhat-mounted, 360-degree cameras worn by the people who are already on the site, such as the safety manager or site superintendent, and the AI-based software automatically tags the images to locations on the project’s floorplans.

In the most basic use case, project stakeholders can leverage the viewing interface to click through the rich images and “walk” the entire jobsite. Users can also see how progress changed at a single location over time. Issues identified during image capture can also be “tagged” to their location in the field.

The benefits of accurate and comprehensive jobsite photos are significant with respect to identifying and addressing risks. During a jobsite meeting, it is simple to pull up imagery of any specific location on the project site to discuss and resolve a design issue that may be having an impact on the work. Photos can also be easily employed to identify issues that need resolution via requests for information or field observation reports. Comparing the images against the project’s 3D design model in real time to identify discrepancies between the design and the installed work is yet another real-world risk mitigation use case.

Forensically, the 360-degree images can be reviewed to identify problem areas or compare progress against the planned schedule when the necessary information or level of detail is not available in project schedules (or when schedule updates are not available). The availability of 360-degree imagery also provides the potential for greater specificity when quantifying the delays to a project’s critical path forensically, as monthly schedule updates provide only a static snapshot and may lack insight into what caused impacts between the schedule updates. More recently, an even more powerful application of 360-degree photo platforms has been to use AI to identify the specific type and quantity of work installed at any given time, which can further enhance the capabilities discussed above.

Increased value of visual data from disparate sources. AI technologies such as computer vision and machine learning are also permeating jobsite photo and video management, which can assist project teams to take informed action based on images that typically sit on project hard drives, disorganized and untouched. One company is using its AI engine to identify the indicators of project risk on a jobsite in the areas of safety, productivity, and quality. By aggregating photo and video content from disparate sources into one system, the system also enables project teams to access a dashboard that ranks safety risks and employs predictive analytics to reduce safety incident rates.

Forensically, these visual search capabilities support an expedited and more efficient review of the relevant scopes by claims experts. The AI-powered tagging system enables users to quickly search through images for specific scopes—for example, a search on the word “rebar” across all images and video on a project in which the rebar installation was alleged to cause delays—to help narrow down the pool of images that could be of use in analyzing an issue. While it is possible to sift through months or years of general project photos or video files manually, relying on powerful software driven by machine learning to expedite such work reduces time, improves efficiency, and increases effectiveness.

Enabling Deeper Insights into Labor and Equipment

Construction innovators are envisioning a time soon when every person on a construction site is given a smart watch, smart hardhat, and augmented reality safety goggles that will work together to feed information in real time to a project management command center. By employing today’s internet of things (IoT) hardware technologies such as wearables designed specifically for the jobsite, project teams already have tools providing access to real-time data that can be used to improve safety, make timely decisions, or establish reliable support needed to substantiate or refute a claim.

Real-time, actionable IoT data. One of the major companies in construction jobsite wearables provides clients with its personnel clips and equipment tags to enhance site visibility and security, improve worker safety, and enable productivity gains. The personnel clips and equipment tags work within a site-specific mesh network to contemporaneously identify the general location and activities of personnel and equipment. From a safety perspective, the personnel clips contain hardware that automatically identifies in real time when an accident such as a fall has occurred.

Additional risk mitigation can be achieved by using wearables and IoT tags to confirm—even remotely, if needed—that the expected personnel or equipment are working in the areas where the project schedule anticipated they would be working on a given day. This can help identify areas on a jobsite that may be suffering from inefficiencies, such as locations where there are more personnel than necessary (and in which those personnel are not accomplishing any more progress than anticipated). If equipment tags are employed, the dashboard can highlight which pieces of equipment have not moved recently, which indicates underutilization. Project teams can take action to address these issues by reassigning personnel or equipment, reducing staffing, or removing underutilized equipment.

Improving claims and dispute outcomes with IoT data. Jobsite wearables can also have profound impacts for disputes and forensic analysis. Jobsite wearables data can be used to automate timesheet creation, which can improve the accuracy of daily reports. This improves upon the typical approach of developing daily reports manually by project personnel, which is often completed days after the work was done and can be prone to inconsistencies or lapses.

Automating these processes also ensures daily reports are regularly prepared and include accurate manpower counts and work locations, which can be key information for analyzing project impacts and supporting or refuting a claim. The increased level of detail enables the forensic analyst to analyze the precise areas that are the subject of a claim or dispute, which is currently very difficult to piece together based on typically available project documentation.

Automating Cost Processes and Insights

The lack of integration between various sources of information on a jobsite is an area where there is still vast room for improvement—especially with cost estimation and accounting. However, technologies are now leveraging machine learning and natural language processing to provide automated insights into project costs. These solutions alert estimating teams and management to information that may affect the cost of the work, despite the fragmentation of data sources.

Generating cost analytics with machine learning and natural language processing. One solution establishing itself in this space offers a construction financial forecasting and intelligence platform that uses machine learning and data analytics to provide contractors with insights on their project costs that help them to make decisions both on site and at the corporate level. By leveraging an abundance of readily available structured and unstructured data from sources such as company accounting records and public documents, the machine learning–based engine can identify trends that humans would typically either miss completely or identify only based on “gut feel.”

While projects are being built, contractors can rely on such software to identify correlations in their historical project cost data—or even in public records data—that no humans can do themselves without tremendous time and effort. The software delivers analytics and predictions into the data, which helps contractors to either identify cost anomalies or validate the reasonableness of their own cost forecasting, among other applications. By providing real-time, early detection of cost anomalies, these insights can help a contractor identify an at-risk project early enough to take action that addresses the potential cost impact. The result is a powerful and much-needed bridge between a construction firm’s accounting and operations teams.

In a dispute in which a contractor submits a claim, it would certainly put the contractor at ease to know it had used software before the project started to confirm that its cost forecasts were reasonable based on its historical data and that publicly available records had been leveraged proactively to confirm that any external variables were considered. In addition, it could be instrumental to use AI software to help support the reasonableness—or proactively identify potential deficiencies—of a contractor’s budgeted costs.

Looking Ahead

As the potential impact of AI-based technologies is increasingly recognized, market adoption is accelerating and it is estimated that by 2026 the global market for AI in construction will reach $4.5 billion. “Artificial Intelligence (AI) in Construction Market to Reach USD 4.51 Billion By 2026 : Reports And Data,” GlobalNewswire, July 23, 2019. As technologies that leverage AI mature and become more widely accepted, the underlying algorithms will become more accurate and will therefore produce even better results. Construction technology start-ups that leverage AI are being formed at a rapid pace: In the past five years, the AI and advanced analytics segment had the highest proportion of new companies out of all the construction technology segments analyzed in a recent report—and that trend is expected to continue. Katy Bartlett et al., “Rise of the Platform Era: The Next Chapter in Construction Technology,” McKinsey & Company, Oct. 30, 2020.

As exciting as future developments may be, the automation of traditionally manual processes using AI-based tools is already here, which makes this the right time for contractors to consider investments in these solutions. Applying these tools will help construction firms gain greater insights into their work, reduce project risk, improve performance, and ensure that needed documentation is accurate and readily available in the event a dispute becomes unavoidable.

Embracing AI-powered technologies and investing in integrating this technology will enable construction companies to improve their bottom line and expand their market share; but perhaps more importantly, those that fail to start adopting AI-based technologies today may find themselves falling irreparably behind those that do.