Defining a System of Trust (SoT) as a Keystone Tool for Supply Chain Security

In the community of public policy considerations associated with public health and epidemiology, we will continue to amass lessons learned to fill a university curriculum for decades to come. And in other zones of the “glass half full” community, there are and will continue to be myriad beneficial lessons learned—both constructive and critical—about the management of medicines, devices, and ancillary technologies.

For one such ancillary area, information and communications technology (ICT), there has been a decades-long effort to wrestle to the ground the villain of “supply chain risk.” Behavior among ICT network, device, and component producers in the development and manufacture of microelectronics (semiconductors) and ubiquitous software of every description has been riddled with instances of both malicious and negligent disruptive behavior for decades. Research and development into tools and techniques to address ICT supply chain security (SCS) and apply to the trillions of dollars’ worth of computer and network commerce globally has achieved some success in recent years. Target markets for these techniques have always been thought to include complex government and national security systems, finance and banking, aviation and vehicle manufacture, and healthcare and pharmaceuticals.

Early in 2020, one such R&D investigator, the not-for-profit FFRDC MITRE Corporation, achieved pilot stage maturity with a set of tools to address SCS—in particular, the core characteristic of “trustworthiness” of suppliers and their devices. It should come as no surprise that while exploring the applicability of these tools to the manufacture of drugs and devices supporting the war on COVID-19 the team of investigators not only accelerated their efforts dramatically but received important funding to support a broad deployment of their tools into a number of potential user market segments.

What follows is a description of the concept of a system of trust and the theory and construct of MITRE’s associated tools applied to SCS risks and agnostically adaptable across virtually any supply chain in any line of business. In the not-too-distant future, the legal community may well come to understand the concept of “trust” and “trustworthiness” as a core element of the due diligence obligation associated with any commercial contract for the sale of complex goods and services. When that day comes, the SCS system of trust is likely to inform that embedded due diligence process.

The topic of trust and trustworthiness in product supply chains is one confronting communities around the world and of heightened interest today; this includes all of our company’s U.S. government agency sponsors and the thousands of commercial industries they rely upon. As the discipline of supply chain security (SCS) has been brought into sharpened focus by COVID-19, the unique expertise and legacy of deep SCS engagement across many organizations has placed a high demand on our SCS subject matter experts and their shared expertise and rich body of knowledge. Prior to COVID-19, our company, The MITRE Corporation (an operator of federally funded research and development centers (FFRDCs) focused on systems engineering in areas of information and communications technology (ICT) supporting federal government homeland security, national security, and critical infrastructure missions), had over forty active SCS projects underway with various government sponsors. Today the number is much larger.

This article was in part prompted by the unexpected explosion in attention to the discipline of SCS generally, in areas of economic activity far afield from the ICT issues we have addressed for more than two decades and by an associated appetite for tools and solutions to address the extraordinary risks to those supply chains uncovered by the pandemic. A remarkable recognition of SCS as a critical consideration in our globalized economy is one of the very significant and unexpected by-products of the COVID-19 pandemic and its impact on the production and availability of medical devices, pharmaceuticals (both therapeutics and vaccines), personal protective equipment, and other artifacts essential to the management of this health crisis.

Many of our colleagues supporting agencies different than our historic “beat” in the Defense, Intelligence, and Homeland Security agencies have come in search of supply chain advice and expertise for health agencies and their components, including the Food and Drug Administration, the Centers for Disease Control and Prevention, Health and Human Services, and VAHealth. They seek to understand operational issues we are confronting in dealing with the embedded, often long-standing challenges involved with establishing approaches to assuring the trustworthiness, safety, and security of computer and network hardware, software, network systems, and their components. These other disciplines, like we in ICT, have found that much of what we have learned relevant to ICT SCS over many years is very applicable to the supply chains of many other areas of commerce, including lingering challenges.

What are these other disciplines, and who are those asking these questions about SCS? They are as diverse as the immediately COVID-19-impacted segments of the biomedical and pharmaceutical complex suggested above, as well as large-scale manufacturing, including heavy equipment, vehicles (air, land, and sea), and tools of every class, agribusiness, and the many source sectors that provide logistics services and devices to the rest of the industrial economy—including extractive industries, chemical manufacturing, and other producers of fabricated material.

The questions and pleas for advice, templates, and analytical tools come from their business managers, engineers, and lawyers—in-house corporate, agency staff, and retained counsel. They are acquisition officials in and out of government, from communities far beyond the ICT technologists who form the rank and file of our daily practice. They are policy practitioners seeking to adapt rules recently written by Congress and the agencies to address supply chain threats—both active, adversarial threats and embedded risks from production processes and negligent artifacts of manufacturing and global commerce.

In this article, we outline emerging answers MITRE and some partner researchers and client agencies have evolved over decades, which have truly ripened in the past three to five years and which MITRE has serendipitously moved into broad piloting and testing in 2020 coincidentally with the arrival of the pandemic. We are driving off the root of our prior work, which sought to define the elements of the concept of trustworthiness, and methods and tools for both assessing entity SCS posture and applying that conceptual trust framework to the acquisition environment.

To do this, we begin by addressing the generic SCS considerations of the complete ecosystem involved in the procurement of products and services, many aspects of which have renewed importance within the pandemic context. What does it mean to have unqualified confidence that whatever you buy and the organizations that sell to you meet all the conditions of your demand and specifications—and thus will meet your needs and therefore merit your trust?

We particularly describe the elements of an analytical system of trust (SoT) for SCS that is currently under refinement and tuning. While the ICT-focused SoT detailed in this article is based on the decades of SCS experience we have accumulated, it also draws on the insights and practices of the broader communities of procurement departments and standards organizations from many other economic disciplines beyond ICT that confront these same SCS concerns in the community writ large, as well as important, sometimes unexpected new insights about resilience exposed by the pandemic. And we remain very solicitous of observations about the scope and scale of potential applicability of these concepts (and adaptations of the specific SoT tool set) both to the immediate life-saving challenges of health-related supply chain challenges exposed by the COVID-19 pandemic and virtually every other integrated line of commerce—like pharma, food, and critical items of supply for aerospace, transportation, and other critical infrastructure sectors.

To date, the SoT has matured through design into implementation. Our team has validated a simple model and placed into the hands of technology acquirers a tool for consistently representing the diverse range of SCS risk concerns. Each user can identify and assess those concerns applicable to their use case/user environment. The tool then enables responses for each acquisition instance to over 500 specific risk questions from over forty different public and MITRE-internal supply chain risk resources. These responses are then mapped to the SoT model, relying on an initial set of seventy-two validated data sources. This SoT analytical system is positioned to enable a clear, well-informed decisional determination by an acquirer about whether to purchase from a particular entity and whether to purchase a specific item/part number from that entity.

To support this analytical structure, the SoT effort has collected, correlated, and mapped 295 SCS-related policy, statutory, and regulatory issuances from the past seventy-plus years. These authorities shape and control the legal and governance framework within which acquisitions are conducted and define the transactional elements for what many in government need to do (or are precluded from doing) with respect to suppliers, supplies, and services.

The System of Trust

The supply chain SoT has four objectives. The first objective is to gather and organize into a single structured corpus all of the concerns surrounding trust with respect to organizations, products/components, supply chains, and service offerings—so that those trying to make choices share access to a broadly available, commonly understood, stable basis to consider their options, risks, and decisions.

Second is to capture the techniques and mechanisms available to gather substantive evidence of which potential concerns with trust are or are not substantiated. For example, a verified, vendor-originated bill of materials, whether for software or hardware, can provide a streamlined, repeatable, and scalable method for collecting evidence about salient trustworthiness aspects for supplies used to build and operate devices and systems.

The third objective is to provide a mechanism for winnowing and tailoring the overall SoT to a profile-linked set of weighted concerns and investigative questions that assesses the resources of an organization, the significance of the system or service to its operations, and the consequences that could result from failing to fully vet observed concerns.

Finally, the SoT aims to put in place mechanisms for objective scoring of these accumulated findings that an organization can individually adapt and tailor to its line of business and unique priorities, operational sensitivities, and experience with its type of business and partners.

Along with introducing these elements of SoT, this article describes how various community efforts—like the Industrial Internet Consortium (IIC), 2020 and its focus on trustworthy Industrial Internet of Things (IIoT) systems in operation; Plattform Industrie 4.0, 2020 [2], with its ideas on trustworthy organizations and our collective reassessment of supply chain practices and norms with our new understanding of impacts from sudden change; and the Open Group’s Trusted Technology Provider standard, 2018, which explores several aspects of supplier behavior that contribute to trustworthy products with a certification regime—and other endeavors to define and convey how trustworthy systems can be acquired and integrated into operations.

By meeting the above objectives and meshing with the related activities in the broader community, the SoT will be well positioned to become the generally accepted principle for SCS, similar to the generally accepted accounting principles (GAAP) used in all businesses in the U.S. or the globally equivalent international financial reporting standards (IFRS).

Context for the SoT

The supply chain security SoT is a MITRE initiative aimed at defining, aligning, and addressing the specific concerns and risks that stand in the way of organizations trusting suppliers, supplies, and services. The SoT effort includes an examination of the risks that can undermine trust in a supplier and/or in the supplies and services it provides. This examination encompasses the multiple contributors to the development and provisioning of systems and services, the supply chain of the supplier, and the suppliers’ offerings of services to maintain systems and proffer upgrades/updates.

In ICT, the risk basis (threats and vulnerabilities) of demand for SCS tools has been understood for decades; the principal challenge has been to develop analytical models and tools that have broad sectoral relevance, and unique subsectoral applicability. Thus, we have spent much time addressing the means of addressing entity (both vendor and customer) posture with regard both to systemic-level supply chain concerns, and key vulnerabilities in core subtechnology areas, which for ICT are microelectronics manufacturing (semiconductors and other integrated circuit devices) and software. In contrast, for healthcare, for example, the details of a risk-focused analytical SCS assessment and a SoT transactional methodology will necessarily be adaptable to areas such as pharmaceutical manufacturing and connected medical device manufacturing for monitors, ventilators, fluid pumps, and other therapeutic administration tools. The same need for adaptation will exist for any other line of commerce and its integrated production supply chains.

The topic of trust and trustworthiness is one confronting many communities around the world. At the same time, these communities often reflect distinct perspectives, with differing purposes and objectives. The IIC, for example, has been exploring how to define a trustworthy IIoT system and how to demonstrate or establish that it is trustworthy. Simultaneously, Germany’s Plattform Industrie 4.0’s use of the term “trustworthy” suggests what it means to work with trustworthy partners. These two groups have been collaborating to align their work on the trustworthiness of organizations and systems, and the Open Group’s effort is focused on helping identify those suppliers that can provide trusted technologies. The concept of trustworthy supply chains, the appetite for trustworthy partners, and the need for trusted systems are global—and an emerging and keen focus for many. Having a reliable path to an actionable understanding of the risks that can impact that trustworthiness is essential, and must be understood, shared, and usable at scale.

When we explore what many define as supply chain issues, most organizations tend to focus on whether delivered goods and associated components and raw materials will be available where and when they are needed—the classic “logistics” approach to supply chain. When industry and government use the terms “trust” and “trustworthiness,” however, they focus on risk management, most often in terms of financial and regulatory perspectives. These considerations are essential but are each portions of a much larger aggregate of concerns, which are the “piece parts” of a comprehensive and effective supply chain SoT capable of being applied to ICT, pharmaceuticals, or aircraft.

Indeed, parties to commercial transactions frequently overlook another fundamental source of transactional risk: how to decide to trust those individual, disparate, and different parts. Moreover, the acquiring entity may not have a complete understanding of the different areas of risks that could disrupt the supply chain, or, importantly, the variety of risks that could undermine trust in a system or a supplier. A recurring, familiar result is that when two parties to a transaction “talk to each other,” they are instead talking past each other because of the differences in each party’s definition of the word “trust” and the accompanying concept of transactional trustworthiness.

As illustrated in Figure 1, the various entities and concern areas involved in transactions exhibit different intentions, along with individual unique understandings of the meaning of trustworthiness. Real-world examples include parties concerned with creating or using trustworthy commercial-off-the-shelf (COTS) systems that incorporate ICT, or those interested in having trustworthy high-value items like medical devices with embedded ICT capabilities, or those concerned with having trustworthy surveillance cameras across a large manufacturing complex—never mind those concerned with data, counterfeits, or the suppliers themselves. These various parties undoubtedly often have different intentions and focuses, along with individual, unique understandings of the meaning of trustworthiness.

While each of the above roles, risk areas, and types of concerns use the terms “trust” and “trustworthiness,” generally the various entities presume different concepts, scope, and level of veracity underlying those terms. This causes immense confusion and misunderstandings that negatively impact organizational transactions and trust and trust relationships (primarily external and potentially internal). The SoT harmonizes the vocabulary of trust and trustworthiness while supporting the subtexts and nuances that delineate varied areas of concern and various roles among suppliers and those using their products and services. Without this harmonization by the SoT, each of these unique participants—with their unique experiences and perspectives—will continue to talk past each other due to their use of differing concepts or scope of issues when they talk, consider, and act on trust and trustworthiness topics.

Supply Chain Security’s Challenging Evolution

From a SCS perspective, the least understood risk to most organizations comes from our toleration of evolved technical complexity and the seeming need for each user to have the skills of a programmer to use even the most common devices—everything from a vehicle, to an elevator, or to today’s medical devices, never mind trying to manage a “smart” power grid. Complex software- and network-enabled ICT has become a key enabler for a large portion of people’s homes and businesses, with new and enduring supply chain challenges. In other lines of commerce, the SCS challenges may be similar, especially where there are electronic elements of and software present in devices; or they may be rooted in other unique concerns, such as the availability of chemical precursors for pharmaceutical compounds. For some, the supply chain consideration may be largely logistic, such as the availability of accessible domestic manufacturing capacity—as was true of ventilators in the early months of the COVID-19 pandemic.

While organizations have always relied on external suppliers, the emergence of a complex, multilayered, and multiparty supply chain as a unique source of economic harm and national security risk came in World War I, when “anti-tamper” became a practice espoused for protecting both the goods and services of the munitions industries and combat material; as a result, a focused law enforcement effort aimed at defeating sabotage and counterfeiting arose. This was followed by World War II–era efforts to protect nuclear and other defense industries’ production and critical infrastructure products throughout their full life cycles. Subsequently, we saw the introduction of just-in-time (JIT) manufacturing, where flexibility, predictability, and multisourcing considerations came to bear to take advantage of global manufacturing models and to achieve pricing efficiencies. These structural changes in the commerce of technology and defense products created a much-expanded universe of supply chain partners and all manner of supplier qualification issues, including the transitive financial stability risks of those upstream of an organization and the need to trust a supplier to deliver the right quantities on time. If a supplier’s supplier is disrupted, the finely tuned JIT supply line might not just falter, but crumble. And it ultimately spawned the development of the modern supply chain risk management and cyber supply chain risk management subdisciplines—SCRM/C-SCRM—which are among the dominant elements of the present practice of SCS.

In the 2000s, many US federal government practices of supply chain logistics management from the Cold War era were extended into the broader commercial IT marketplace, as those technologies and the efficiencies they brought to business and government started to become key enablers of the information economy. As if this were not enough of a challenge, the computerization of everything gave rise to pervasive cyber threats. For many suppliers serving the US Department of Defense (DoD) with commercial goods, the concept of a “cleared industry partner” became part of their way of life. In parallel, the globalization of ICT manufacturing and markets, coupled with security concerns such as the growth in export licensing of sensitive US technologies, extended the definition of, and areas of concern with, the supply chain. Consequently, visibility and control of highly complex microelectronic component supply chains and their global network of specialized software developers has become a difficult, perhaps impossible, technical management problem.

Naturally, the resulting potential for disruption and harm from the supply chain has become a key concern at the personal, organizational, and societal levels. Such supply chain–rooted risks require specific attention and different perspectives about an organization’s practices across the many business functions of both commercial enterprises and government.

The SoT Creation Approach

The SoT is an amalgamation of decades of experience accumulated by MITRE and others supporting the national and homeland security communities in contributing to their government customers’ efforts to address supply chain risk issues. This included engagement in various national and international standards efforts on the topic, associated published work from MITRE, the standards developing organizations, and others working in various aspects of SCS.

Our construct of a supply chain SoT permits us to transform that knowledge base into a coherent whole that serves as a taxonomy and implementation model. Acknowledging the vast collection of past and current SCS work from across the world and the variety of questions each effort has established, the MITRE effort is building both from the ground up (using individual questions) and from the top down (using supply chain risk taxonomies).

Moreover, the SoT effort is arranging unilateral nondisclosure agreements (NDAs) with a variety of industry organizations. These unilateral NDAs are almost identical to the ones MITRE previously used to create its Common Weakness Enumeration (CWE) Version 4.0, 2020 body of knowledge. Like other community efforts MITRE has pursued, feedback and engagement with experts and practitioners will be critical in creating something that promotes integration, eases the burden of working with others, and streamlines communications and understanding when multiple parties discuss a topic or share information. These efforts will enable broad access by many other lines of commerce and areas of economic activity to the SoT we describe here and its larger SCS context.

In addition to the community-published materials, MITRE’s past internal work, and the knowledge shared through the unilateral NDAs, the SoT will also integrate legal-focused content to allow for the SoT to identify areas of concern that an organization might be obligated to investigate (or be precluded from investigating).

By collecting the various sets of measures and questions currently in use across the various aspects of the SCS community, MITRE is establishing a wide-ranging set of risk-determination questions for input into the SoT. However, rather than just have these as open-ended questions or questions requiring subject matter experts (SMEs) to interpret when used, the SoT will refine them into a series of yes/no questions that embed SME experience into the questions themselves so that less-experienced practitioners can use them to identify where and at what level one might trust a supplier, its supplies, or its service offerings.

To illustrate this, consider the different modes of taking a measurement of human blood pressure. If we ask only for the systolic and diastolic measurements, there is an implicit requirement for someone with appropriate training and understanding to interpret the measurements (the evidence) into a finding of “normal,” “elevated,” “hypertension stage 1,” “hypertension stage 2,” or “hypertensive crisis.” If, however, we ask about these as separate range-based questions that are yes/no–based, the medical expertise is embedded into the questions themselves, for example:

• Is the systolic number less than 120 and the diastolic less than 80?
• Is the systolic number higher than 180 and/or the diastolic higher than 120?

This approach can be seen in the area of financial health, where many use the “quick ratio” to get insight into an organization’s liquidity and cash flow risks. Based on the total assets, liabilities, and any surplus, it calculates a ratio; if it is less than 1, it is usually consider high risk, equal to 1 is moderate risk, and above 1 is low risk. The experts have baked into the ratio calculation the knowledge of what creates risk. Anyone with access to the data can follow that methodology to determine where a specific organization is with respect to these aspects of financial risk.

To the extent possible, the SCS SoT will follow this paradigm of embedding informed risk expertise into the questions so that the SoT produces objective metrics that can easily be used by those who are not supply chain risk SMEs.

Rather than collect these items into a spreadsheet, word processing document, or some other unstructured form, the SoT uses a knowledge base along with an application tool to support viewing, organizing, and tailoring the content or subset of the content within the SoT. Further, this application tool will support evaluating a supplier, supplies, and services against the tailored SoT subset, and will support adjusting the contributing weights of specific SoT questions and areas of concern. The SoT tools focus on assessing the SCS concerns captured in the SoT taxonomy against specific supply items, service offerings, and companies. The SoT tooling should not be confused with general risk management tools in common use at many organizations.

The SoT Structure

The MITRE SoT is organized along categories common to device and software development; these include those encountered by suppliers and evidenced in the production of devices/supplies and services and address twelve top-level decisional risks associated with trusting them.

These are the risks that any agency or enterprise must evaluate and make choices about during the full life cycle of their acquisition activities. Leveraging the full breadth and depth of MITRE expertise, industry efforts, and government research, the SoT includes twelve top-level risk areas, seventy-six risk subareas, and 410 detailed questions to date. These incorporate seventy-two objective data sources in order to advance a probabilistic risk assessment of the trustworthiness of a product, service, or supplier. The methodology is currently prototyped in a web app.

Depending on the role of the acquired items in the mission or operations, and the consequences of disruption or loss of control, variable levels of energy and attention are necessary to examine and assess these concerns. Each risk factor, when considered by an analyst, will either advance or diminish a “trust” case for a given supplier, supply, or service.

Risk factors belong to one of three trust aspects, each with top-level risk areas identified. Each of the dozen top-level risk areas—seven about suppliers, three about the supplies, and two about service offerings—extend into a myriad of subrisks that detail lower-level concerns and identify where to gather evidence of the existence of risks writ large. During the evaluation process, subject-specific questions are posed to establish the presence or absence of the individual aspects of the concern and are aligned with best practices from MITRE, the US government, and industry.

Scoring of the risks is determined through a set of contextually driven, tailorable, weighted measurements that are used as inputs into a scoring algorithm. The scoring results are then used to identify supplier strengths and weaknesses against risk categories that can be used by an acquirer to evaluate and analyze the supplier’s “trustworthiness” for supplying components or services.

The interests of a specific assessment may focus on the supplier, a specific item, the legal authorities that the assessing organization is under, or a combination of elements. The SoT will offer profiled subsets of an overarching risk map for investigation of each assessment. The results of this analysis can be the basis for a “trustworthiness” discussion with a supplier or a basis for mitigation requirements levied on the supplier to address supply chain risk.

Applying the SoT: Going Live

Starting with an untailored full SoT set of the risk areas and subrisks, a SoT assessment is envisioned to start by asking the user to answer a few questions that will narrow the SoT content to something appropriate to the context of assessing a product, service, or supplier, for example. This subset will be aligned to the assessment focus, resources, available time, and legal authorities of the assessing organization and its actual present acquisition challenge. Additionally, indicating whether purchased data sources or only open-source information can be used would change the risk subset offered for assessment. Similarly, if samples of the components are available for analysis and the requisite testing capabilities (i.e., chip or software analysis) are available, the portions of the SoT related to those risks would be offered.

The process for combining the component subrisk assessments into a probabilistic risk score is being developed following industry and MITRE best practices for analytics and scoring. Figure 2 shows the summary-level risk scores and risk profiles from the first SoT use-case pilot. As a key aspect of the development of the SoT, we developed several use cases focused on exploring the use and utility of the SoT for situations aligned with sponsors. The initial set of pilot efforts included (1) assessing a set of companies for general concerns, (2) assessing a specific company as a supplier and service provider of critical infrastructure systems, (3) assessing a product for use by a specific community within the federal government, and (4) assessing the risk posture of a critical supplier type for a sector of technology. Each of these four SoT pilots utilize different subsets of the tool outputs.

The preliminary results for one of the four pilots are illustrated in an unweighted bar chart and radar plots of five data-driven scores from the supplier risk areas, leveraging fifty-two questions in those areas for eleven companies of interest. These examples all utilize data sources available within MITRE’s analytics capabilities, with results that clearly show a larger risk profile for the last company when compared to the others. This sampling provides a proof of concept that offers early evidence of this tool’s utility, with deeper and broader analysis to follow as the SoT is completed. In the next phase of the SoT effort, weighting and score contributions will be tailored to allow for focus and emphasis of specific subrisk areas to be used in an assessment.

While only a subset of data sources was used for the pilots, the numerous sources of public, private, and restricted data sources available to use when evaluating the individual subrisk questions are being catalogued and captured within the SoT. Similar types of analysis were done for three other pilots, with one exploring the services area and another focused on the software supply portion of the SoT.

Moving Ahead

In addition to continuing to build and validate the SoT, add weighting and score contribution capabilities, socialize the concepts the SoT entails, and gather ideas for additional questions that explore how to make concrete the concerns organizations have with their suppliers, supplies, and services, the SoT effort will continue creating its assessment tool and explore its utility in SCS of pharma, medical device, and other non-ICT domains. The SoT Risk Model Manager tool will allow an organization to tailor the SoT to just those select areas of concern an organization feels are the most useful for its decision making. In this next phase, it will support tuning the weights and combinatorial mechanisms used to combine the individual answers into an overall trustworthiness finding. Finally, in this coming year we will evolve the mechanisms used for answering the questions, from predominantly manually to integration with data analysis tools, and explore the use of both quantitative and repeatable qualitative methods.

While we have identified many questions that can be answered with one or more of the seventy-two public and private information sources we’ve documented, we will continue to evolve and refine the catalog of sources of evidence about the different aspects of the SoT. Many communities exist for the topics being amalgamated under the SoT. Their respective past, current, and future work will need to be explored and integrated, where appropriate, so that the SoT and those efforts can coexist, proffer synergistically, and ensure their respective work, foci, and best practices are clearly understood and mutually leveraged.

Finally, there will be an immediate and evolving need to explain and train individuals and organizations on the use and utility of the SoT. Additionally, many of those currently working in the areas touched by the SoT work will, we hope, evolve their efforts to align with the integrated SoT vocabulary and concepts as we collectively solve the trust and trustworthiness challenge, with a shared taxonomy and methodology and the SoT tooling.

A Piloting Plan for the SoT

In view of both the novelty of the SoT concept as an element of SCS abilities and the anticipated broad scope of interest from organizations across industries and government (together with the broad applicability of the SoT’s concepts, taxonomy, evaluation methodology, and tool set), we have decided to embed the practices of frequent testing, expert evaluations, and interim exercises (both mini/tabletop exercises and full formal simulations) to test assumptions about the structure, functioning, and applicability of the SoT’s elements to the full technology life cycle.

The SoT must be, and so has been designed as a dynamic, living system that can evolve functionally with changing needs as well as maintaining the broad beneficial adaptability to the breadth of commerce that will become its hallmark. In addition to these continuing practices of testing and evaluation throughout the creation and maturation process of the SoT, there will also be two or three pilot implementations of multiple weeks to provide further evidence of functionality and to support further editing and restructuring, gap analysis, and overall refinement.

Finally, a regular cycle of exercises and user evaluations is intended to be a signature element of the SoT’s “maturity model” during its future life. Ongoing exploration of applying the SoT to other acquisitions, suppliers, and products/services will occur in other transactional environments, including various intergovernmental and critical infrastructure transactions, as already described. The flexibility and adaptability of the SoT are expected to become key elements favoring SoT incorporation and adoption in these diverse transactional settings.

The material contained in this article comes from the System of Trust team’s research and integration efforts as well as from work with individuals and groups from MITRE, our sponsors, and industry.