AI systems are far from perfect, as shown by some spectacular (and very public) examples of racist chatbots, financial programs that routinely deny certain minority groups credit/mortgages based on their ethnicity, discriminatory hiring practices, and generative AI programs that hallucinate fictional legal cases, to name a few.
Canadian acquirors of AI products/services should filter and consider their purchases against the requirements of pending Bill C-27, Canada’s proposed Artificial Intelligence and Data Act (the “AIDA”), whose purpose is to expressly regulate certain types of AI systems and ensure that developers and operators of such systems adopt measures to mitigate various risks of harm and avoid biased output. While AIDA will only apply to AI systems that are “high impact” systems (terms are as yet undefined), prospective acquirors should still ask the “hard questions” around the vendor’s bias mitigation practices. Does the AI vendor have an internal AI ethics review board? What kinds of data sets have been used in training the AI product/service? Has the company established measures to identify, assess, and mitigate risks of harm or biased output that could result from a client’s use of the product/service? What steps has the AI vendor taken to ensure the quality and accuracy of its data, to ensure that it is class-balanced and unbiased? Was the source of the AI vendor’s data sufficiently diverse, or was the AI system narrowly focused on a small sample of data that could lead to unforeseen and harmful consequences? Has the AI vendor explicitly tested for bias and discriminatory outcomes? If so, how? Does the company have a plain language description of the AI system that states how it is intended to be used, the types of content that it will generate, and the recommendations, decisions, or predictions that it will make, as well as the strategies to mitigate against bias?
Use Rights/Intellectual Property Considerations/Licensing Concerns
You should review the draft AI contract to ensure that your client has the necessary rights to use the AI service/product as contemplated, including its affiliates and customers, as applicable. It’s critical to drill down in the prospective AI contract to determine what the vendor says about (i) the ownership of its own intellectual property (AI models, tools), including any licensed third-party content; and (ii) who owns the content/output generated by the AI product/service, as applicable (i.e., the vendor or the client). Since laws are still evolving in this area, all desired client rights must be expressly defined in the AI contract. Many AI systems are built on data sets that have been scraped from other publicly available third-party content, which opens these vendors up to prospective litigation, so a positive affirmation in the vendor contract regarding ownership is essential. Look for language in the AI vendor’s contract to ensure that all rights that make up the AI system have been listed and protected, and that the AI vendor has the right to license the AI technology for its intended uses (any restrictions should be carefully noted).
AI systems are rife with privacy concerns. They are myriad, and include (i) ensuring that vendors have the legal authority to process personal information used by the AI product/service, particularly that of minors, in relation to the data sets used to train, validate, and test generative AI models; (ii) individuals’ interactions with generative AI tools; and (iii) the content generated by generative AI tools. Similarly, the AI system should contain mitigation and monitoring measures to ensure personal information generated by generative AI tools is accurate, complete, up-to-date, and free from discriminatory, unlawful, or otherwise unjustifiable effects. Detailed questions should be asked as to whether the AI vendor has put in place sufficient technical and organizational measures to ensure individuals affected by or interacting with these systems have the ability to access their personal information, rectify inaccurate personal information, erase personal information, and refuse to be subject to solely automated decisions with significant effects.
It is therefore critical to understand what the AI vendor says about its own privacy/cybersecurity practices, and whether it has incorporated “privacy/security by design” principles in the development of its AI systems. While AIDA has not yet passed in Canada, existing Canadian privacy laws still require vendors to limit the collection of personal information to only that which is necessary to fulfill the specified task and to ensure that the AI system is not indiscriminately grabbing content solely for the vendor’s benefit. AI vendors should incorporate adequate, reasonable security safeguards to protect against threats and attacks against stored data that seek to reverse engineer the generative AI model or extract personal information originally processed in the datasets used to train the models. The standard AI contact should include detailed language relating to comprehensive privacy protection and mandatory breach notification. Ideally, the vendor will also state in its contract that it adheres to meaningful cybersecurity standards, such as NIST (National Institute of Standards and Technology), which just published its AI Risk Management Framework in January 2023. These requirements and accountability measures must also flow down the vendor’s entire AI supply chain, especially when AI models are built upon one another.
It is worth noting that starting September 22, 2023, Québec’s Law 25 will grant individuals new transparency and rectification rights related to the use of automated processes to render decisions about individuals (“Automated Decision-Making Systems”) that use the personal information of such individuals. An individual will have the right to: (i) be informed when an enterprise uses their personal information; (ii) request additional information on how the individual’s personal information was used to render a decision, as well as the reasons and principal factors and parameters that led the Automated Decision-Making System to render such decision; (iii) request to have the personal information used to render the decision be corrected, and (iv) submit observations with respect to a decision to a member of the enterprise to review the decision made by an Automated Decision-Making System.
Lastly, it is important to be aware of any “reverse” privacy/security requirements that the AI vendor may incorporate in its standard agreement that create onerous burdens on clients. These may include obligations for clients to notify the vendor of any vulnerabilities or breaches related to the client’s AI service/product and provide details of the breach, provide legally adequate privacy notices, and obtain necessary consents for the processing of client data by the AI vendor, complete with actual representations from the client that they are processing such data in accordance with applicable law. Some AI vendors even require clients to sign separate Data Processing Addenda. It is important to be aware of these additional vendor data requirements and neutralize any that are unacceptable to your client.
Additional Sources of Liability
Besides the risks above, additional sources of liability include noncompliance with both AI-specific legislation and regulations (which are not limited to Canada, given pending AI regulations in Europe and the United States), but also existing federal and provincial laws (privacy, consumer protection legislation, consumer disclosure requirements). Old laws still continue to apply to AI vendors, and AI systems that are defectively designed would still be subject to product liability laws.
Unfortunately, AI products/services are usually offered by vendors on an “as is, as available” basis, with minimal to no legal representations and warranties. Standard contract terms typically contain disclaimers that limit any damages to direct damages with very low dollar liability. You should therefore seek to include express legal representations/warranties regarding the following: (i) the vendor having all necessary rights, including ownership and licenses to make the AI service/product available to the client and for the client to use the AI product/system as contemplated/described; (ii) non-infringement, including no infringement when used by the client as intended; (iii) vendor’s (and the service’s/product’s) compliance with all applicable laws, including privacy laws and jurisdictions outside of Canada (customize as required); (iv) the AI service/product not containing any viruses, malware, etc. that would otherwise damage the client’s systems; and (v) no pending third-party claims or investigations existing that would impact the vendor’s ability to provide the product/service.
Similarly, many AI vendors do not provide indemnities in their standard legal agreements but rather include reverse indemnities from the client. For example, clients are asked to indemnify the vendor, its affiliates, and personnel from and against claims, losses, and expenses (including legal fees) arising from or relating to: the client’s use of the AI services/product, client’s content, any products or services that the client develops or offers in connection with the AI services or product, or client’s breach of vendor’s terms or applicable law. You should endeavor to minimize the client’s indemnities and balance the agreement through the addition of such critical vendor indemnities as indemnification for vendor’s failure to comply with applicable laws, fraud, negligence/gross negligence, willful misconduct, intellectual property infringement (especially patent and copyright), breaches of confidentiality/privacy and cybersecurity breaches, customer data loss, and lastly, personal injury/death (depending on the product/service). While I do not recommend trying to seek unlimited indemnities as they are generally no longer considered “market,” I recommend instead seeking “super-caps” (i.e., higher caps) for the most critical of these, such as IP infringement; confidentiality breaches and privacy and cybersecurity breaches; customer data loss; fraud; gross negligence/negligence; and willful misconduct. These super-caps may be based on the greater of a specific dollar value or a multiplier based on contract fees paid or payable, or some other formula. Lastly, the scope of the indemnity should include affiliates, contractors, and third-party representatives of the AI vendor as applicable/appropriate.
You should review what the standard legal agreement says regarding dispute resolution, as many AI vendors seek to restrict a customer’s rights at law (and equity) to deny their day in court. Instead, vendors will insist on mandatory arbitration, naming a US arbitration regime that will prove expensive for the client should it wish to assert its contractual rights. Some agreements also include compelled informal dispute resolution that results in a hold period (i.e., sixty days) before a client can assert a claim. These restrictions may not be in the best interest of the client and should be removed. It is, therefore, important to look at the governing law/jurisdiction clauses carefully and note any special restrictions/differing rights depending on the client’s jurisdiction.
Lastly, don’t forget to look at the termination provisions, as AI contracts often contain robust termination rights in favor of the vendor—i.e., the vendor can terminate the agreement immediately upon notice to client if the client (allegedly) breaches its confidentiality/security requirements, for “changes in relations with third-party technology providers outside of our control,” or to comply with government requests. Also, the vendor may have broad suspension rights that allows suspending the client’s use of the AI system if client is allegedly not in compliance with the AI product/service terms, the client’s use poses a security risk to the AI vendor or any third party, if fraud is suspected, or if the client’s use subjects the AI vendor to liability. Often these broad rights require additional negotiation and tightening to balance the client’s interests. It is also important for the contract to expressly address, in plain language, what happens following contract termination. For example, must the client immediately stop using the service/product and promptly return or destroy the AI vendor’s confidential information? If so, does this include the client’s outputs? Does the client have ongoing usage rights regarding outputs? Will the AI vendor continue to use any ingested client content or personal information, or will this be erased? If yes, consider the protections/restrictions necessary for your client to comply with applicable privacy laws and any particular industry requirements.
While AI technology may be new, seeking to create balanced legal agreements that correctly apportion risk and liability is not. Notwithstanding the daunting list of risks associated with the use of AI systems, there are a number of risk mitigation measures that prospective buyers (and their counsel) can deploy to manage these concerns. It is critical to negotiate AI contracts with teeth in order to ensure that clients will feel comfortable acquiring and using these products and services on a going-forward basis.