Quick answer
Choosing the right AI services partner takes four pass/fail gates first, then a weighted score across six example criteria, as of November 2026. The gates are a SOC 2 Type 2 report, a no-training clause, a pilot on your own data and data export at exit. In our made-up test, the top scorer at 4.15 out of 5 failed a gate. Run the gates before any scoring meeting.
You've sat through the demos, three bids are on your desk, and security and legal each want their own list before anyone signs. Choosing the right AI services partner means agreeing on what knocks a bid out before anyone compares scores. An AI services partner is a firm you pay to build, run or plug AI into your systems, from a consultancy to a model-as-a-service vendor. In our made-up test below, the bid with the highest score had no no-training clause.
Each section below feeds the example scoring sheet. The paid pilot on one problem carries its heaviest weight, 25 of 100, and it's also a gate. The frameworks a partner maps to and its SOC 2 report back the risk and security scores. Data rights in the contract and export at exit are gates too, and testing each sales claim tells you what the pilot should measure on your own data. The last section shows why the highest score still lost.
Key takeaways
- Our made-up top scorer failed the no-training gate and dropped out.
- A SOC 2 Type 2 report tests controls over time; Type 1 doesn't.
- The FTC says firms that break no-training promises may be liable.
- The Copyright Office protects AI output only with human-determined expressive elements.
- OMB's federal AI memo lists lock-in terms any buyer can copy.
Start with one problem and a paid pilot
A pilot on your own data shows whether a bid works before you've signed anything. The GSA's Buy SI procurement best practices page, last updated November 4, 2026, puts the problem first. Then it says to test in a sandbox or a pilot with a small user group before a large purchase. Write the problem as one measurable sentence, such as cutting claim-intake time from 12 minutes to 8 (example figures). The pilot gets scored on that sentence.
Federal buyers get the same advice. OMB memo M-25-22, issued April 3, 2025, says agencies should seek demonstrations and tests of proposed AI where practicable, and weigh a testing environment on agency networks. It binds only federal agencies, but a private buyer can copy what fits, such as its ownership and lock-in terms.
Skipping the test has a record. In December 2023 the FTC said Rite Aid would be prohibited from using facial recognition for surveillance for five years, to settle FTC charges. The Commission vote was 3-0. The complaint says Rite Aid failed to seek any information from either vendor about the extent to which the technology had been tested for accuracy. It also failed to implement or enforce any procedure for tracking the rate of false positive matches. Ask each bidder for written test results. Then run your own on a sample of your data, scored against the sentence you wrote before the demos. Keep both sets in the bid file for the scoring sheet below.
Diagram by DigitalTreed: the buying order this guide follows, from the GSA and OMB pages cited in it.Frameworks a partner should map to
Once the pilot is set, ask each bidder which framework it follows. In the US, the common reference is the NIST AI Risk Management Framework, released January 26, 2023 for voluntary use. NIST says AI RMF 1.0 is under revision as one piece of the White House AI Action Plan, so ask which version of it each bidder maps to. The framework names seven traits of trustworthy AI, from valid and reliable to fair with harmful bias managed, and a good bid shows how each one's tested.
The framework's core has four functions, GOVERN, MAP, MEASURE and MANAGE, per the AI RMF 1.0 document (NIST AI 100-1). NIST warns that its actions "do not constitute a checklist", so a partner who hands you a ticked list hasn't done the work. The part buyers should read is GOVERN 6, which calls for policies on AI risks from third-party software, data and supply chains. GOVERN 6.2 adds contingency plans for failures in high-risk third-party data or AI systems. Ask how the partner meets both for any model it resells, since that model's maker is a third party to you as well.
If the bid uses generative AI, ask for the mapping to NIST's Generative AI Profile, NIST AI 600-1, a companion published July 26, 2024. A partner with its own AI program may also hold ISO/IEC 42001 certification. The IEC webstore listing for ISO/IEC 42001:2023 says it sets requirements for an AI management system. It's meant for any organization that provides or uses AI products or services, and it came out December 18, 2023.
NIST keeps crosswalks from the AI RMF to other standards. They're useful when one bidder cites NIST and another cites ISO.
A certificate covers the partner's own management system, not the build it'll do for you. Ask which sites and services its scope names and which body issued it.
Read the SOC 2 report, not the badge
A certificate proves a management system exists, while a SOC 2 report shows a CPA firm's test of controls. The AICPA describes its SOC suite of services as CPA reports that help users assess the risks of outsourcing services. A SOC 2 examination covers security, availability, processing integrity, confidentiality or privacy. It's measured against the AICPA's Trust Services Criteria.
Check the report type before the logo on a sales deck. A Journal of Accountancy explainer on SOC reports says a Type 1 report gives an opinion on whether the description is fair and whether the controls are suitably designed. Type 2 goes further. It adds an opinion on whether the controls operated effectively, and it lists the auditor's tests and results. We'd require Type 2. In it, read three things: the period covered, the services in scope and every exception the auditor found. If the services you're buying aren't in scope, the report doesn't cover them at all, however clean its opinion reads.
SOC 2 reports aren't public. They're restricted to specified parties, per a February 2026 Journal of Accountancy article on fast SOC offers, so ask each bidder for its report early. That article tells buyers to check the CPA firm's qualifications and peer review results, and to ask whether an offer is "unrealistically fast, easy, or cheap". As of November 2026, the AICPA's SOC page carries a notice that it's looking into allegations about a compliance vendor's business practices. It also lists an exposure draft of proposed 2027 SOC 2 criteria, dated October 26, 2026.
Diagram by DigitalTreed: SOC 2 Type 1 vs Type 2, from the Journal of Accountancy explainer and AICPA pages.A vendor's report says nothing about your own access controls, and our cybersecurity tips for businesses cover that side.
Data rights to write into the contract
A SOC 2 report won't tell you whether the partner may train on your prompts. The FTC's January 2024 post to model-as-a-service companies says firms that break promises not to train on customer data may be liable. Those commitments count "regardless of how or where the commitment was made", marketing and terms of service included. In past cases, the FTC has also made companies delete any models or algorithms they built with data they had obtained unlawfully.
Terms can also move after you sign. A February 2024 FTC post on quietly changed terms of service warns about switching to AI training through a "surreptitious, retroactive amendment" to terms of service or a privacy policy. It says that may be unfair or deceptive. We'd put a no-training promise in the signed contract, with a clause that those terms can't change without your written consent.
Ownership of outputs needs its own clause. The Copyright Office's Part 2 report on AI output, January 29, 2025, says AI output gets copyright only where a human author has determined sufficient expressive elements. That excludes "the mere provision of prompts". If a partner's deliverables are mostly machine-made, you'll own them through the contract more than through copyright. OMB M-25-22 asks agencies to set ownership and IP rights clearly when agency data trains or fine-tunes a system. The seven lines below adapt those points for a private buyer, one clause each for legal.
- Your data, prompts and outputs aren't used to train or improve any model without your written consent.
- The partner keeps your data only as long as the contract needs it, then deletes it and confirms in writing.
- You own the deliverables, including prompts, configurations, fine-tuned weights and code written for you.
- The partner lists every subprocessor and model provider that touches your data, and tells you before adding one.
- The partner reports a security incident affecting your data within an agreed number of hours.
- Terms that cover your data can't change without your written consent.
- At exit, your data and outputs come back in a format you name, with help moving them.
They're a starting list, not legal advice. Have your attorney fit them to your state and sector.
Privacy law can add more terms depending on whose data the partner handles, so read our data protection rules for businesses before legal finalizes the list.
How do you test an AI partner's claims?
FTC Chair Lina M. Khan said in the September 2024 Operation AI Comply announcement that "there is no AI exemption from the laws on the books". One case in that sweep was DoNotPay, which the FTC said claimed to offer "the world's first robot lawyer". The complaint alleges it didn't test whether its AI chatbot's output was equal to the level of a human lawyer. DoNotPay agreed to a proposed order that would require it to pay $193,000.
Treat every number in a pitch that way, whether it's an accuracy rate, hours saved or a cost cut. Ask what data it came from, who ran the test and whether you can repeat it in the pilot. The NIST AI RMF document itself notes that a developer may not be open about the risk metrics or methods it used.
Security claims need proof too. The CISA, NSA and FBI guide on AI data security, May 22, 2025 covers data risks across the AI lifecycle, from development and testing to operation. For AI agents that act on their own, CISA's May 1, 2026 guide to adopting agentic AI advises against broad or unrestricted access to sensitive data. It also says to begin with low-risk, non-sensitive use cases. Send each bidder the six questions below in writing, and file the answers with its pilot results and SOC 2 report.
- Which data did your accuracy or savings figure come from, and can we rerun it on ours?
- Which NIST AI RMF version and which ISO/IEC 42001 scope do you map to?
- Which services and dates does your SOC 2 Type 2 report cover, and what exceptions did it find?
- Which model providers and subprocessors will see our data?
- What systems will any AI agent reach, and with which permissions?
- How do you track errors after go-live, and how fast do you tell us?
If a bidder can't name the data behind its accuracy or savings figure, score its pilot and leave the figure out.
Our red flags for hiring a developer apply to AI builders too. The Grok vs ChatGPT privacy comparison shows how far two vendors' data terms can differ.
Diagram by DigitalTreed: each sales claim and the document that backs it, from the FTC, AICPA and NIST pages cited here.Plan the exit before you sign
OMB M-25-22 lists ways to cut vendor lock-in risk: knowledge transfer, data and model portability, and rights to code and models made under the contract. It adds clear licensing and pricing transparency. The memo asks agencies to set sunset criteria where practicable, since a change in cost, needs or model performance can mean it's time to stop. Cost needs a trigger of its own. GSA's Buy SI page says these services are typically billed like software as a service and usage costs can grow quickly, and it advises usage limits and regular consumption reports. Write a monthly cap and a report into the contract, and don't accept a usage price you can't check against your own consumption numbers each month.
We'd also name an exit review when costs pass, say, 120% of the pilot's per-user rate (example threshold; pick your own).
If the partner hosts models on its own hardware, our high-density colocation guide covers that side's terms. For a phased build, our agile vs waterfall budget guide ties payments to working releases.
Example scoring sheet with pass/fail gates
The sheet has two parts: four gates that end a bid on a single "no", then six criteria scored 1 to 5 and weighted. The weights below are our example, not a standard, so change them to fit your risk.
Weighted score = sum of (weight x score) / 100, which keeps the result on the 1-to-5 scale.
| Criterion (example) | Weight | What earns a 5 |
|---|---|---|
| Pilot results against your success measure | 25 | Beat the target on your own data |
| Security evidence | 20 | SOC 2 Type 2 for these services, no open exceptions |
| Data terms | 20 | All seven contract lines accepted |
| Risk practice mapped to NIST AI RMF | 15 | Testing, monitoring and incident steps shown |
| Exit plan | 10 | Knowledge transfer and rights to code and models |
| Cost clarity | 10 | Usage pricing, caps and monthly reports |
| Total | 100 |
The gates: a current SOC 2 Type 2 report for the services you're buying, a signed no-training clause, a pilot on your data and export at exit. Don't score a bid that misses one.
What we tested
We built the sheet as a script, test/score_partners.py, and fed it three made-up bids on November 6, 2026: Firm A, Firm B and Firm C. Firm A scored 4.15 but failed the no-training gate. Firm B scored 3.9 and passed all four, so it's the only bid left to negotiate with once the gates ran. Firm C scored 3.25 and failed two gates.
| Bid (made up) | Weighted score | Gates passed | Result |
|---|---|---|---|
| Firm A | 4.15 | 3/4 | Out: no no-training clause |
| Firm B | 3.9 | 4/4 | Shortlist |
| Firm C | 3.25 | 2/4 | Out: Type 1 report only, no export format |
Why the highest score still lost
Ranked by score alone, Firm A would have won, since its math was 25x5 + 20x4 + 20x3 + 15x4 + 10x4 + 10x5, divided by 100, which gives 4.15. A 5 on the pilot and a 5 on cost clarity carried it past Firm B's 3.9, even with a 3 on data terms. The gate caught what the average hid: its default terms let it use customer prompts to improve its service unless you opt out.
Firm C scored a 4 on its pilot and still went out, since it's the bid that offered only a SOC 2 Type 1 report and named no export format. The bids and scores are made up to test the sheet and say nothing about real firms.
Drawn by us: weighted scores and gate results for three made-up bids, from test/results.json.Our take: run all four gates in week one, before anyone's booked a scoring meeting. In our run, they removed two of the three made-up bids before any weight mattered.
For the uses that come after signing, such as AI in email, chat and phone, our guide to AI for business communication covers the US rules. If the partner will connect to finance or operations data, our cloud ERP guide lists what those systems hold.
Frequently asked
Is the NIST AI RMF mandatory for private companies?
No. NIST's crosswalks page calls it voluntary, rights-preserving, non-sector specific and use-case agnostic.
Does ISO/IEC 42001 certification replace a SOC 2 report?
No. The IEC listing shows ISO/IEC 42001:2023 is a 51-page management system standard, while SOC 2 is a CPA's controls opinion.
Can an AI vendor start training on our data later?
It can't do that through a quiet terms change without legal risk. The FTC post says retroactive privacy changes for AI training may be unfair or deceptive.
What does the OMB AI acquisition memo replace?
M-25-22 rescinds and replaces M-24-18 and covers solicitations issued 180 days or more after April 3, 2025.
How we made this guide: The scoring script's output sits in the What we tested table, run on November 6, 2026. We read the OMB memo and NIST AI 100-1 as PDFs, saved the other 18 pages in Sources, and matched each quote to those saved copies. Facts checked by the DigitalTreed team on November 6, 2026. This guide isn't legal advice, and an attorney should review the contract before you sign.
First published August 2026 as a general essay on how enterprises pick AI partners, with one unsourced line on security incidents. Rewritten November 6, 2026: buying steps, NIST and ISO frameworks, SOC 2 types, contract terms, FTC cases, exit terms and a tested scoring sheet. The unsourced line was cut and the year left the title.
Sources
- NIST: AI Risk Management Framework
- NIST AI 100-1: Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023
- NIST AI 600-1: Generative Artificial Intelligence Profile (July 26, 2024)
- NIST: Crosswalks to the AI RMF 1.0
- IEC Webstore: ISO/IEC 42001:2023, AI management system
- AICPA & CIMA: System and Organization Controls, SOC suite of services
- AICPA & CIMA: SOC 2 reporting guide
- AICPA & CIMA: 2017 Trust Services Criteria (revised points of focus, 2022)
- Journal of Accountancy: Expanding Service Organization Controls Reporting (July 2011)
- Journal of Accountancy: Promises of fast and easy threaten SOC credibility (February 2026)
- OMB M-25-22: Driving Efficient Acquisition of Artificial Intelligence in Government (April 3, 2025)
- GSA: Buy SI, procurement best practices (updated November 4, 2026)
- FTC: AI companies, uphold your privacy and confidentiality commitments (January 9, 2024)
- FTC: Quietly changing your terms of service could be unfair or deceptive (February 13, 2024)
- FTC: Crackdown on deceptive AI claims and schemes (September 25, 2024)
- FTC: Rite Aid banned from using AI facial recognition (December 19, 2023)
- U.S. Copyright Office: Part 2 of the AI report released (January 29, 2025)
- U.S. Copyright Office: Copyright and Artificial Intelligence
- CISA: AI Data Security best practices (May 22, 2025)
- CISA: Guide to secure adoption of agentic AI (May 1, 2026)



Comments
No comments yet. Start the conversation below.