Even the legal industry's own AI research tools gave wrong answers more than 17% of the time in a 2024 Stanford RegLab and HAI study. Westlaw's AI-Assisted Research was wrong more than 34% of the time. If tools built for lawyers stumble that often, a general chatbot reading your lease deserves a short leash.
That doesn't make AI useless for contracts. It makes it a decent reader and a bad authority on the law. Used as a fast way to find, quote and summarize the text in front of you, ChatGPT, Claude and Google's tools can turn twenty pages of fine print into a list of questions in a few minutes. Used to tell you what the law says where you live, or whether you'll win, they invent things.
This is general information, not legal advice, and no AI answer substitutes for a licensed lawyer who knows your facts and your jurisdiction. With that said once, here is the workflow we'd follow, in order, and the places it breaks.

Check the data settings before you upload anything
A lease or an employment agreement carries your full name, address, income, sometimes a bank account. Whether that text can end up in a training set depends on a toggle most people never open. Defaults are policy, and on the individual plans of the big three, the default is not on your side. Here is what each vendor's own pages say.
| Service | What the vendor says for individual plans | Your control |
|---|---|---|
| ChatGPT (Free, Plus, Pro) | OpenAI says it may use your content to train its models | Turn off "Improve the model for everyone" in Settings > Data controls; temporary chats are not used for training |
| Claude (Free, Pro, Max) | Anthropic's 2025 consumer terms update lets you choose whether chats improve Claude; retention is five years if you allow it, 30 days if you don't | Privacy settings; business and API accounts are under separate terms |
| Gemini apps | Google says a subset of chats is read by human reviewers; reviewed chats are kept up to three years, even if you delete your activity | Turn off Keep Activity to stop future chats from being reviewed to improve Google services, or use a temporary chat |
Google's own advice is blunt: with Keep Activity on, "don’t enter data that’s confidential or that you wouldn’t want a reviewer to see or Google to use to improve its services." A contract fits that description. Our privacy framework for AI tools goes through how to read these policies in general.
Google's notebook products fall on the better side. Google's Gemini Notebook help page, where NotebookLM now lives, says your data "is not used to train Gemini Notebook unless you provide feedback", and that feedback can expose the full interaction to review. Notebooks inside Gemini work differently. Google's privacy hub says files you add as sources are not used directly for training its generative models, but your chats with the notebook follow the Keep Activity setting and can be reviewed by humans. That split is why we'd check which product's page applies before uploading. For a closer look at how these tools hold documents, see our comparison of NotebookLM, ChatGPT Projects and Claude Projects, and the NotebookLM listing.
Our pick for anyone pasting a real contract: the training toggle off, or a temporary chat, before the first paste. If the deal is a business one with trade secrets or a client's data in it, a paid business plan where training is off by default is the better home. OpenAI states that ChatGPT Business, Enterprise, Edu and the API aren't used for training by default. One catch on ChatGPT: even after you opt out, OpenAI says that if you rate a response with a thumbs up or down, the whole conversation attached to that feedback may be used for training. Skip the thumbs on a contract chat.
Strip the personal data first
Redaction takes five minutes and it works on every tool, whatever its settings say. Find and replace is enough.
- Names of people and companies become [TENANT], [LANDLORD], [CLIENT] or [VENDOR].
- Street address, unit number and parcel or registration numbers become [PROPERTY].
- Bank, card, tax and ID numbers, phone numbers and emails get deleted outright.
- Signatures, signature images and anything scanned from an ID are cropped out before you upload a PDF.
Keep the dollar amounts, dates and clause numbers. The AI needs those to be useful, and they identify nobody by themselves. If a scanned PDF has no selectable text, run it through OCR first and check the output, because a misread digit in a rent figure will flow straight into the summary.
One more habit. Tell the AI where you live. Leases, non-competes and consumer terms vary by state, province and country, and the model will otherwise pick a legal system for you.
Ask for quotes, not conclusions
The single biggest improvement to any contract prompt is to forbid paraphrase. You want the exact sentence, where it sits, and a plain-English gloss kept visibly separate from the source text. That separation is what lets you check the answer.
I'm a non-lawyer reading a residential lease. Jurisdiction: [state or province, country].
Use only the document pasted below. Do not rely on outside knowledge of the law.
For each topic, give me:
1. The exact sentence(s), copied word for word, in quotation marks.
2. The section number or heading.
3. One sentence on what it means for me, labeled "Plain English".
If the document does not address a topic, write NOT ADDRESSED. Do not guess.
Topics: rent and late fees, deposit and how it's returned, term and renewal,
ending the agreement (both sides), repairs, landlord entry, subletting,
penalties, who pays for what, how disputes are handled.
[DOCUMENT, personal details replaced with placeholders]For a terms of service or privacy policy, swap the topic list: what the company can do with your content, how and when it can change the terms, how to cancel and get a refund, arbitration or class-action waivers, and who is liable when something breaks. For a freelance or vendor contract: payment timing, ownership of the work, termination notice, liability caps, auto-renewal and exclusivity.
The second prompt goes after what isn't there. This is where AI earns its keep, because a human skimming a lease sees what's printed and rarely notices an absence.
Based only on the document, list provisions that a [residential lease / freelance
services agreement] in [jurisdiction] commonly contains that this one lacks.
For each, say in one line why a person might want it and label it "common"
or "varies". Do not say any provision is legally required. End with five
questions I should ask the other party or a lawyer.Treat the output as a question list. An AI saying a standard clause is absent has told you where to look, nothing more; it may be wrong about both the standard and the absence.
Checking quotes takes ten minutes
Open the original document next to the chat. For every quote the AI gave you, use the search function (Ctrl+F or Cmd+F) and paste a distinctive phrase from the quote. Three outcomes are possible: it matches, it matches with words changed, or it isn't there. The second and third both happen, and you'll want to know which clause you were about to rely on.
Two quick tests catch most trouble. Does the section number the AI cited exist, and does that section say what the quote says? Is the clause the AI quoted modified by another clause later in the document, such as a definitions section or an addendum that overrides it? Cross-references like these are easy for a model to skip, and easy for you to confirm.
Re-read your answers. For each quote, say whether it appears word for word in the
document. List every quote you shortened, reordered or combined from different
sections, and every statement that goes beyond the text.That third prompt is useful but it isn't a check. You are asking the same system that made the error to grade itself, so we'd treat it as a prompt for your own search, not a verdict. Our guide on how to fact-check AI output covers the general routine; for contracts, the document itself is the only source that counts.
When the document is long, split it. Paste one section at a time, or ask for a table of contents with section numbers first, then ask about specific sections. We'd expect shorter inputs to be easier to check, since each answer points to a short stretch of text.
Where the answers go wrong
Stanford's researchers have documented the failure patterns well, and they map directly onto contract questions. In the 2024 benchmark write-up, they point to an earlier study of general-purpose chatbots that found hallucination on legal queries between 58% and 82% of the time. (A separate 2024 Stanford piece reports 69% to 88% for specific legal queries, which is a different measure, so don't mix the two.) That earlier work also found models tend to accept whatever premise the question assumes, which it calls contra-factual bias. Ask "why is my landlord required to give 90 days' notice?" and you may get reasons, when no such rule exists where you live.
Those figures are from 2024, on models that have since been replaced. We don't know today's rates for the newest assistants, and we doubt any vendor can give a number that covers contracts in every jurisdiction. The failure types are the useful part, because they haven't gone away.
Jurisdiction. Tenancy, employment and consumer rules differ between states, provinces and countries, and sometimes between cities. A model may default to U.S. norms if you don't say otherwise. State your location in the prompt, then confirm any claim about the law on an official site.
Invented or misattributed clauses. Stanford describes a second kind of error it calls misgrounded: the tool states the law correctly but cites a source that doesn't support it. In contract work, the equivalent is a real-sounding clause attributed to section 9.2 when section 9.2 says something else. This is why the quote check above is not optional.
Outdated law. In the 2024 study, one legal tool recited a standard that the Supreme Court had overturned in 2022. Rules about deposits, notice periods, fees and non-competes change often. A model's training cutoff is always behind the statute book.
Agreement with your assumptions. If your prompt says "this clause is illegal, right?", you will often hear yes. Phrase questions neutrally: "what does this clause allow the landlord to do?" is a better prompt than "can they really do this to me?".
Do you need a contract-specific tool?
Probably not for a single lease. Purpose-built products such as Spellbook are aimed at transactional lawyers, and the case study of Spellbook running on Claude describes volume work by legal teams. Spellbook sits 546th among AI apps on ToolRank. Our contract review and drafting board and the AI legal assistants board list the options, and lighter browser tools like Legalese to English exist for simple translation of dense wording.
The Stanford result is the reason to be cautious about the pitch. Tools that retrieve from legal databases cut errors compared with general chatbots, and Stanford says so, but its own summary was that "even these bespoke legal AI tools still hallucinate an alarming amount of the time." A business with a stack of vendor contracts a month may justify the subscription. A tenant with one lease does not. Check any such tool's data policy as carefully as the chatbots', since you'd be uploading the same documents. For the broader field, see the legal services board.
When to pay a lawyer
AI can lower the cost of a lawyer's time by making you arrive prepared: a marked-up list of the clauses, the missing items, and your questions. It can't replace the hour. Our own rough sorting of tasks, from low risk to high, looks like this.
The further right on that line, the more it costs to be wrong. Paying for a licensed lawyer makes sense when real money, a deadline, your home or your business sits behind the signature: a commercial lease, a contract with a personal guarantee, a non-compete, a partnership or investment agreement, an eviction or demand letter you've received. Ask up front whether a short consultation or a flat-fee document review is available. That is often cheaper than open-ended hours.

Cost is the usual objection, and there are free routes. The American Bar Association's Find Legal Help site points to lawyer referral, free legal help for people who qualify as low-income and its ABA Law for Consumers pages with straightforward explanations of the law. USA.gov lists legal aid programs, including LawHelp.org and the ABA's free legal answers service for people with low incomes. Outside the U.S., your national or provincial bar association and consumer agency fill the same role. Those sources are where to confirm what the law says in your state, not a chatbot.
One last use of AI that we like: before the lawyer meeting, ask it to turn your checked quote table into a one-page summary of questions. You pay for less of the lawyer's reading time, and the lawyer sees the actual text rather than the AI's version of it.
Frequently asked questions
Is it safe to upload a lease to ChatGPT, Claude or Gemini?
It depends on your settings. On individual plans, ChatGPT may train on your chats unless you opt out, Claude asks you to choose, and Gemini's human-review sampling depends on Keep Activity. Turn the relevant setting off or use a temporary chat, and remove names, addresses and account numbers first.
Can AI tell me whether a clause is legal or enforceable?
Not reliably. Enforceability depends on your jurisdiction, current statutes and the facts, and the Stanford work shows models getting both the law and its recent changes wrong. Use AI to find and explain the clause, then confirm the rule on an official source or with a lawyer.
How do I stop the AI from making up clauses?
Make it quote. Require word-for-word text and a section number, allow the answer NOT ADDRESSED, and check every quote with the search function in your own copy of the document. Nothing in a prompt removes the error, but the quote requirement makes it visible.
Does this work for terms of service and privacy policies?
Yes, and these are the easier case because the documents are public and usually need no redaction. Use the same quote-first prompt with topics like content ownership, changes to terms, cancellation, arbitration and data sharing.
Which AI tool is best for contracts?
For a one-off lease or a policy, any of the three major assistants can do the reading job if the data setting is right. Contract-specific platforms make sense for teams reviewing many agreements. Neither choice replaces the quote check.







