For a client I recently reviewed a redlined contract where the counterparty used an "AI-powered contract platform." It had inserted into the contract a provision entirely contrary to their own interests. So I left it in there. Please, go ahead, use AI lawyers. It's better for my clients.
Not to mention hallucinated citations
I really did think that AI might have some value churning out some kind of boilerplate that I could adapt or doing very rote adaptation of existing work where there are minimal changes but it's just complete garbage for everything I've tried it on.
It’s complete garbage. I was so confused when I started receiving bad first drafts that were so horrible. And so long. Repetitive. Internally inconsistent.
Its like asking an improv troop to write for you. It just yes ands, until the end
to the degree this is useful, standard templates exist
Do you not just use templates for such things?
The more tailored/tuned systems do have benefits but the generic products are not there yet
Kinda sorry I'm retiring from practice next year, I see huge opportunities for new clients who previously relied on AI-powered contracts.
Marc Benioff is on CNBC rn, saying how AI has cut referrals to humans from 10k per 86k (daily?) inquiries down to 5k per 86k inquiries. So it might be good for Trust and Wills dot com.
lol amazing
“savvy” The word has come to mean its opposite.
Oh man, I wish I could see this. That sounds savory.
An expert in one of our cases cited multiple fictional journal articles in his report. It came to light when opp counsel asked for copies of articles cited in the report. Our expert used AI to write his report. We had to withdraw the expert, and get an extension to name a new one. Ugly.
I don’t know if OC was being kind or if he hadn’t thought it through. I would have sat quietly and destroyed the expert in deposition. But it def raises the bar on expert retention and review. Be careful out there, kids.
How many hours did he bill to prepare the report? FFS
Another way that "AI" is forcing people to do *more* work because people have to check for others silently using it. I've certainly plucked a handful of things out of an expert's report to look at, but it was never my practice to check if they're real. Now you have to put "no AI" in the retainer.
You know the number of supposedly intelligent and professional people doing this is shocking
Right??? Ffs.
I call it the "eyeball problem", AI doesn't increase the number of them, just increases the demand for increasingly strained eyeballs.
The ay eye ball problem.
Friend of mine works in a very specialized IT field. They thought they'd found their new employees, but something didn't feel right to one of the interviewers. Checked ChatGPT on a hunch and the applicant's answers were identical.
What was the clause?
Ugh, so now if I hire lawyer, I’m going to also have to figure out a way to ensure that they themselves read (& write) all the stuff I would be hiring them to read (& write), not farm *any* of it out to AI. I can screw up with AI on my own, if I’m hiring a human, I want the human to do it.
To be fair, you can't sue GPT-4o for malpractice, but you can sue a lawyer who misuses GPT4o.
I guess you'd have to have a pretty good idea that they were using it. Is that the kind of thing that could come out in discovery?
I mean, if the citations are completely hallucinated, that's usually a good sign.
You can have that put into the terms of your engagement letter with an attorney. However, getting the attorney to actually comply with the terms of that engagement letter can be... Tough. In my extensive experience in the area, immediately forgetting what they've promised in the EL is common
It seems to me all AI stuff should be fact checked. What happened to good ole legal research.
The problem is, whoever is doing the fact checking has to know enough to spot any errors. If I'm hiring a lawyer, it's because I don't know enough to do it myself, so I'm less likely to be able to successfully fact check, or error-check, the AI output.
I mean the lawyers should fact check any AI document. I wouldn't expect the client to. I'm being totally misunderstood today. 🙄
There is a duty of supervision and a duty of competence which, in theory, are accountable via professional discipline / licensure and malpractice and court sanctions, although no court is around for the contract stuff like Max’s example
Hiring a lawyer is a lot like hiring a plumber or electrician. There’s no way for most clients to supervise the details. Do some lawyers, electricians, and plumbers cut corners? Unfortunately yes. So we have to do our best to vet for experience and integrity before picking our professional
Sorta feel like with plumbers and electricians the one that advertise on TV might want to be avoided.
Sounds like a lot more lawyers
Lawyers are supposed to disclose when they use AI. You can include a prihibition in your contract. But be aware, it wil be cheaper for them to use AI as a start and verify/edit from there.
Where does the requirement that lawyers disclose their use of AI come from?
I said supposed to, not required. I would argue it's part of Rules 1.4 (duty to communicate) and 1.6 (confidentiality) and certainly 8.4.
Honestly, this problem precedes AI. The lawyers who are using AI now just didn’t do the work at all before, or found other ways to cut corners. The AI cheating shows up more blatantly and hilariously, but it’s the same problem it was before, and the same problem that riddles any profession.
Oh god I love it
I’ve claude to start drafting motions/replies. And it always cites fake cases. I even tell it “those are fake cases, check your sources” and it replies “yes sorry those are fake, here are some cases” and then those are fake. It also replied “yes, those are fake you should do your own research” 😂
Making stuff up that sounds real is exactly what a LLM is supposed to do! That’s its purpose! So funny and stupid the things that people will try to use this technology for.
It also likes to attribute hallucinated quotations to correctly-formatted real cases.
My company got us a trial of an AI tool to do corporate research. It consistently just made up nonsense, with nonsense citations, seemingly at random. When we complained about this during the trial, they told us to check it. Which means having to do all the work the tool was meant to automate.
And then, after pointing out that this saves us no work and increases the chances of errors in our important research, they turned around and bought licenses anyway. We don't use it, but that money definitely left the budget for raises this year.
All computers work on the ‘garbage in, garbage out’ principle. If you don’t give it the information, all AI models can do is basically a fancy version of predictive text. Absolute waste of time.
This!
Even if you do give it the information, all it can do is predictive text. It doesn't know what any of the words mean, all it "knows" is that some strings of characters are statistically more likely to be in conjunction than others. It is literally only a fancy predictive text engine.
I have noticed, that GPT models especially 4o is much easier to get to stay on real material. Provided that you have the materials. So in that use case a case database that the model can query is a must.
They should invent a job for someone to look up actual cases and cite them correctly
Call them a law person or something.
Heh. The point is, LLM models can be useful and provide rather reliable output. But getting that right requires a lot of domain knowledge and skill at building agents. Want it or not, AI models will streamline a lot of work away. The big problem is where we get next generation of experts?
LLMs will never stop bullshitting because all they can do is confidently predict the next word in a sentence. Stop using it as search engine, and for God's sake, stop using it as a lawyer
It’s not that simple, anybody who thinks it is will basically become obsolete. LLMs are natural language processing tools, if you want to avoid bullshit, you need to provide correct reference inputs. LLMs will be a force multiplier for a professional.
No, they're language mimicking tools. They have no understanding of what the inputs mean, only the statistical probability of the next word. They are never going to be reliable no matter how much correct information you put into them.
They are already good enough for many use cases. Absolutes don’t really work.
Those use cases being ones where the output's veracity doesn't actually matter.
Enjoy generating plausible sounding lorem ipsum. The rest of us will be creating something of value.
It literally is that simple
It is that simple. That fact might make some obsolete not because they're wrong, but because the ppl making hiring and purchasing decisions don't know or care that it's true. LLMs are not "language processing" in any meaningful sense. Your imagination is doing all the work of bridging the gap
Invoice multiplier for litigation lawyers handling the mess of contract and litigation nonsense generated by a model that generates words
Kuule... ei näin
The problem there is that LLMs are precisely not natural language processing tools. Producing the statistically likely next paragraph is not how humans naturally process language.
They are currently the best available for the task, obviously there will be new innovations in the future. And world won’t wait. In general I don’t think computers ever do things same way as our brains do. But close enough to be useful.
How "obviously"? Like, I can look at airplanes and say, everybody will have flying cars, it's obvious progress from a solved technology. Didn't. There is no such thing as obvious progress. There is progress you can look back at and say "that made sense". That's not the same as "this will happen".
Why doesn’t openAI provide the “correct reference inputs”, instead of using chatGPT to unleash BS on society? Sam Altman sells selfish lies and fantasies like Elon Musk
But just like professional grade power tools will not make you a master craftsman, no bot will replace human expert, but it will make said expert a lot more efficient.
So, yes it is true, that I a software engineer can’t use LLM as a lawyer. But a skilled lawyer can reduce the hours spent on a client case by skilled use of LLM of properly tuned agent.
How is that
A question I have for you is why, even if we granted all your predictions were 100% correct, which we don't, you think we should be encouraged at the idea of you proposing to put a bunch of law clerks and paralegals out of work. What are all the people whose job is done by AI do after?
The idea of somebody automating portions of their work, allowing them to do less work for the same pay, giving more time for leisure is good. The idea of the people at the top obsoleting people below them, putting them out of work entirely, is bad. There've been movements about this. It's not new
Our intelligent agents are called junior lawyers and paralegals.
And as a legal secretary friend's boss told the associate who was bullying her, new junior associates are much easier to find than good legal secretaries.
Lawyer here at a big firm that has invested a LOT of money in potential LLM use case experiments to little avail. We may yet see narrowly purpose-built tools that prove useful. But I’ll bet they will be focused on data processing and search—not analysis or reasoning.
LLMs are good at producing things that LOOK like legal analysis, but they get it wrong often enough that a decent lawyer needs to check everything, so no efficiency gain. To tweak the pipe metaphor, imagine they produce an actual physical pipe but 25% of the time the pipe is poisoned.
Yes, but it's pretty marginal gains, at best, so far. And only in select areas. And in ways that become fairly meaningless if you have access to a decent bank of past work.
That latter piece is so important and never gets brought up.
It's will always be amusing to me (developer) and wife (litigator) when outsiders assume that our productivity's main limiter is typing speed.
and in any case NOT WORTH BOILING THE OCEAN OVER IT
But will they? Or will they just be lazy? Because its my experience people just get lazy.
Not anywhere near as much as you might think.
They are high on their own supply.
For real. bsky.app/profile/phas...
I have been rather happy with the results on my own field so far. But you are free to disagree, I kinda remember people complaining about early spell checkers.
The thing is, the hours a lawyer would spend are researching the cases, not drafting some text.
This is the cool thing about the LLMs though, is that they don't read articles, they develop a model of what a plausible article reads like. They don't, like, "understand" the article, even if you give it "good" inputs it, by design, will just synthesize fictional ones.
It is exactly that simple. You repeatedly asserting something doesn't make that thing true. Any more than an LLM producing a citation for a nonexistent case makes it exist. Since you think that surface appearance is equivalent to truth, I'll point out that you appear to be an insistent troll.
That hasn’t been my experience. In my experience, I get back plausible-sounding nonsense. Granted, I work in a very specialized area, but it’s the sort of thing that you won’t notice unless you’re already skilled in the area. As a QA professional, it’s both deeply reassuring and deeply scary.
What did you use as input? Just a prompt without knowledge base will get less than satisfactory results. And ChatGPT and other general purpose chatbots are rather useless for actual work.
A lot of detail of what I wanted to do. I will say they’re getting better, in that RAG systems can point me to where to look.
Yeah currently I am using Langchain as integration library.
The output is reliable in that it will always generate output, but it's not "reliable" in terms of being accurate or useful. Fact-checking the statistically generated text of a software program is not a good use of my time.
It's not streamlining if you've got to do just as much work to correct the error-riddled output of the GIGO machine.
Heck, it's worse than GIGO. You can put perfectly good information in and it'll still generate bullshit because it just slams words together in a statistically probable order.
You aren't wrong. LLM AI *will* get better, especially once people start applying serious expertise to it. This has been the way of *all* technological advancement in the last 25000 years. The problem is all the corpses left behind, as they twiddle and tweak it into something useful.
No, it won't, because if you're trying to use a language model to solve a factual problem, the best it can give you is stuff that sounds like facts, or like something somebody in the past said that sounds fact-like but is wrong. Improving screwdrivers makes them better pry bars, not better hammers.
No. LLM models do NOT provide reliable output, because they don't model reality, they only model things their input feed said. Any accuracy they have happens because they've mostly been fed correct statements, and output stuff that sounds truthy, but it is in no sense reliable, nor does it try to be
yes but not that kind of work
Lawyers and paralegals are useful and provide much more reliable output. It takes more time and effort to go through a motion or brief with a fine-toothed comb than it does to create one from a template, and you run WAY less risk of having errors slip through.
Back in May, researchers tested Lexis and Westlaw AI research tools. The hallucination rate was 1 in 6, so even given the best legal databases,these models can't do research LLMs are not search engines, we have to stop using them like one. hai.stanford.edu/news/ai-tria...
That only tells that group couldn’t do it. Next on will hopefully read their results, and try a different method. Also LLMs don’t research, you use it to process language to build an agent that does.
LLM is not a magic box, it is a programming environment and language for processing things.
But a caveat, it has come apparent that I lack vocabulary to explain things to legal experts. I do my best, please have patience.
Like LLMs, you may have vocabulary, but you don't have models for the facts about what lawyers do, so throwing vocabulary in potentially-grammatical sentences isn't going to let you say anything factual about facts except at random. Improving screwdrivers doesn't make them better hammers.
I checked the article, it states that RAG agents do produce much better output than a plain base model, but not yet good enough. So I trust the teams keep on improving and finding use cases where it is good enough to assist rather than hinder.
And other thing to note, the team doing testing were intentionally trying to get LLM agents to answer wrong. Which is a perfectly valid test, such tool needs to be robust. However a user would learn ways to use the tool for best most reliable output. Thus skill in using tools is important.
cc: @thescottperson.bsky.social
I "haaaaaate" that people are just accepting "hallucinate" instead of "providing garbage output". It can't hallucinate; it doesn't think. It's just bad at performing the task that has been set for it.
Hate all you want. But it’s kinda accepted as a common term. Same as LLM applications are called agents for some reason. They have no agency, but that is the “official” common name.
"Agent" doesn't specifically refer to a person. "Hallucinate" is marketing bullshit to push a narrative that LLMs have greater capabilities than they do.
However they are the “official” industry terms that are in use. Feel free to use different words, but don’t complain to people who use common accepted terms. In AI, agent doesn’t imply personality, use of term comes from agent based computing from 1990s.
I'm noting that you pointing out that applications being called agents although "they don't have agency" is a meaningless distinction as that's not what the word means. It means "the one doing" at its Latin root, which is why "Agent" is used for computer programs that perform tasks.
lololol. "AI will streamline your business and give you valuable answers if you hire a large staff of AI training experts to tinker with the LLMs until they mostly stop hallucinating....and then train all the users up yo expert level is AI query syntax. It'll be smashing!" Lololol.
the same way we got the current generation of experts? And I take issue with the idea that AI replacing a lot of work is 'inevitable.' Reminds me of crypto chuds saying I'd 'inevitably' take my paycheck/pay rent/pay bills in some crypto over fiat.
Holding a tool to a "rather reliable" standard means it's useless at best.
Oooh. A resistor of bad information via human intervention! Who works at a bar! Maybe we could be clever and call this new job a "barrister!"
All GPTs are plausible-BS generators. You can't tell it what to do because it isn't applying logic, only statistics.
My standard test is to just ask about well-known SCOTUS cases (Brown v Board, Roe, Korematsu, Dred Scott, etc). Doesn’t take long to get a completely bogus description. One model insisted Korematsu (the Japanese internment case) was a “landmark civil rights decision” that paved the way for Brown.
I suppose that it was a landmark civil rights decision.
I want to start a legal tech startup that would solve this problem and would love to chat with you.
Here's a brain teaser for the LLM folks out there. If you're generating a model to be a research assistant, why are you even coding the capability to generate data? When is that ever going to be useful in a use case where accuracy is of highest importance?
My old boss used to run entire draft contracts I gave him to review in chatgpt and then paste the response to me. It was so fucking annoying.
I didn't realize he was doing it for months, I just figured he was pasting basic contract principles he found on the Internet. Once I found out, I freaked out that he was apparently sharing internal IP, including privileged work product, with OpenAI. (He wasn't a lawyer.)
So, you are telling me my laziest 7th graders are messing with the algorithm by inputting their incorrect, badly written essays into the system?
Yes.
Ah, the joys of corporate in-house legal practice :-)
Bad lawyers are expensive, good lawyers have a high billing rate etc etc
I guess that’s ethical? They should proofread their work
Max's ethical duty is to his client. If there is a long term relationship to consider, then it might be in the best interest of the client that Max reveal the error to OC; otherwise Max is basically ethically obligated to shut up.
Lawyers have ethical obligations that extend beyond just the client.
But not to opposing counsel.
I use such a service and routinely find the same thing when reviewing its suggested changes. The idea of relying on it alone - LOL
Read an integration clause that was so integrated that it removed the price term from the contract by excluding all appendices. Alas, I didn't think that I could push that one through.
Sometimes rules based systems are better than stochastic random word printers - you know like understanding that price is a necessary term in a contract.
OK, I guess sometimes! :)
That's kind of the whole point if something is being bought or sold. Right? Did no one check it for logical sense? Or would they say they'd rather not pay for that work like they effectively did with the contract? Of course, that would likely mean they're not bound, and so it's not a contract?
LOL!
I'm still laughing at the lawyer that submitted a motion written by an AI that included citations to cases that do not exist. So not only did the lawyer not write the motion, he didn't even read it.
Multiple lawyers have done this! It’s so baffling esp after the first few were publicized and dinged for doing it.
It was baffling for me the first time I saw it reported because I never submitted anything or went to court to make a legal argument without reviewing every citation to make sure I wasn't going to be humiliated for citing bad law or quoting out of context. That's just how I was taught to do it.
This all has implications for scholarship, too. It's been established for a couple of years now that undergraduates use ChatGPT as a plagiarism machine, but there was one chair's meeting where the Dean raised the specter of faculty unethically using AI to generate research papers
In theory, careful peer review should be able to sniff out fake citations, but given the amount on our plates these days, just how careful are we in checking sources and whatnot? It's at least good that AI generally does not generate prose that rises to the level of peer-reviewed scholarship
Right?
I have very little experience hiring lawyers -- we hired one to write our wills and when we bought our house. Other than responding to questions promptly, I don't necessarily know how I'd tell a good one from a bad one, but this would DEFINITELY tell me I hired one who was just phoning it in.
If a lawyer you've hired submits AI generated paperwork that includes made-up citations to non-existent cases, I strongly suggest contacting the relevant state bar and filing a complaint.
I'd also expect that multiple lawyers would look over any court filings and would at least be a little curious if they saw a citation they weren't familiar with and look it up and it REALLY surprises me that something AI hallucinated would make it all the way through.
That said agree on the surprise that it gets to filing
The times it's happened, it seems like it's been inexperienced solo practitioners (which is an explanation, but no excuse at all).
Multiple lawyers may not look over things before they're filed, but multiple lawyers (some of them with fancy titles like "Judge") are going to look at them after they've been filed, which is why any semi-competent sole practitioner is going to review citations before filing.
And any semi-competent lawyer is going to turn down work they can't perform, which is part of it being inexcusable that a lawyer resorted to having software compose a motion or brief.
And, one more point: a lawyer is generally responsible for what goes out over their name, and absolutely so where court filings are concerned; even if an attorney delegates drafting to a partner, associate, student, etc., they've got a responsibility to review it before signing off on it.
Agree with this
That's true -- I'm not a lawyer, but generally when I write something that goes out to a client, I'll ask someone else to take a look at it, partially for fact-checking and partially for proofreading and making sure I didn't present something in an overly complex way.
I mean, I respect that, done properly, there's a lot of research, work and editing that goes into what is, at the end, usually a relatively short output and I'd be surprised that someone (another lawyer at their firm, corporate counsel, etc.) wouldn't put a second set of eyes on it.
Would you as a client want to bill for two separate lawyers’ time doing the exact same thing - cite checking cases? When I sent briefs to more senior colleagues it was expected that I’d done the key work of sourcing and checking cases.
Although, in this case, I am sure that the client was still billed at a full rate, for a "standard time", even though no lawyers actually did any work. Just the AI.
This is going to vary depending on who's doing the work--is it a solo practitioner, someone in a law office where the attorneys are really just pooling staff/office space, a large multi-attorney firm, etc., and if it's a firm, how is workload organized?
I was a public defender; on the one hand, the office I was in consisted of a dozen attorneys mostly independently doing their own thing; on the other hand, it wasn't unusual for someone to pop their head into a neighbor's office to say, "You got a minute to look over this for me?"
That said, getting one of your colleagues to look over it wasn't an expectation that they'd check your work--everyone was busy and had their own caseload; it was mostly a "Does this look mostly right and did I miss anything?" deal, not a rigorous review.
So, so, so many people are dead set that "the bad thing won't happen to MEEE" and convincing them otherwise is like trying to teach a fish to fly
Meeting of the artificial minds.
My Contracts professor had us redline AI drawn contracts as part of our final & the sales agreement I had interchangeably used ‘principal’ and ‘principle’ (among other worse issues) which made for hilarious reading.
An actual good use of AI!
Many such cases (in theory; few in practice because we're only interested in using it to replace people)
Those AI-powered contract drafting/negotiation tools are nowhere near ready for real world use, despite what the sales people claim. In 10+ years from now, it may be a different story.
are you able to explain what provision says without compromising the case?
Soon enough, the problem of contracts nobody read is going to get supplemented with the problem of contracts nobody wrote.
In a glorious convergence of billable hours. "Use AI for legal" seems penny wise, entire-net-worth foolish.
I am running out of ways to say that you shouldn't believe the things a liar tells you.
Oh! I was wondering why contracts had started getting weird like this. Thinking back on the timing this makes sense. Like I was worried that there was some kind of serious designer drug going around or something so this is actually a relief.
@ Melissa, I went through the same journey. Why am I being sent so many incoherent and internally inconsistent contracts and contract provisions? And why can’t the attorney for the counterparty explain to me what they actually mean? UGHHHH
Also, don't forget about the COVID brain damage.
True but forgetting which side is a deep kind of error I would not expect to see while still being basically coherent.
It’s a bad combo
You weren’t exactly wrong
Fair. Hilariously bad isn't new, but the hilariously bad for your side is ... a new kind of bad and like it feels like cheating even though the ethical thing to do is not say anything.
This I mostly expect see as the extra, weird crankiness that happens when people feel details slip, not an entirely unusual situation in a profession where people work in to their 80s
As the saying goes, if you think hiring a lawyer is expensive, wait until you see how much it costs when you don't.
As a as a joke, I let Clyde write my column for new world notes. It was hilarious. Pompous, verbose and embarrassing. It cracked me up
Ask Jeeves, Esq.
We used to see this even before AI where opposing counsel cut-and-pasted without re-reading. Sometimes they caught it before signing. Sometimes they didn't. Not our responsibility to represent their client.
📌 As my work is contemplating "AI" for contracts
"Smart contacts"
that seems disbarment worthy negligence on the part of the counterparty's legal team
Surely malpractice. I can’t think Judges are going to love lawyers who are *trying* to lead them into error. Michael Cohen has already demonstrated the error of these ways…
If there is a legal team outside of the chatbot…
Can't disbar a chatbot, can't stop a billion dollar corp giving dud legal advice either
I was under the impression that giving legal advice without a license was illegal for basically this reason
As AI cannot apply for a license, does this then make every user who uses AI as part of generating legal advice a law breaker? And every corporate entity that provides the AI in violation of the law?
I mean, I’m not a lawyer, but I would imagine the first client who loses due to AI BS will come back against the firm
I just spent a few days and will Be spending a few more fixing some AI aided decisions by Quickbooks. Companies will fall.
Likely already are failing. And when it comes to math, a lot of people aren't good at it so they won't even necessarily notice mistakes being made by Al.
The stuff it’s getting wrong isn’t even maths related. It’s classification of expenses that are essentially random. Anybody trying to track costs or spending can’t.
This is why I talk a lot about keeping people in the process with my clients
I have to take a walk around the block after I receive an AI-drafted contract. The provisions are incredibly long and convoluted, often the meaning of a provision is unclear, and the contract is internally inconsistent. Ugh.
Need to have your AI assistant summarize the points for you...
Haha. If I were to have an AI assistant summarize for me, I’d go from needing only a walk around the block to needing a walk around the entire park plus a shot of whiskey. And then a nap.
I am not a lawyer, but I did read a LOT of contracts in my day. I remember one NDA which did not allow the Company to disclose anything but did not prevent me from doing so... The clause was written backwards! I laughed and signed it.
A company I worked for had a non-compete clause that was impossible to breach because there weren't any companies that fit the description they gave in the area they specified. I assume there had been at one point and they never updated their contract template.
Haha. Yep and if you point it out to the HR person doing intake they have no clue, having never read it.
Why point it out? Let them keep losing the employees smart enough to notice it
Exactly. It’s a natural fact this is an adversarial relationship, do not give up any advantage.
I absolutely *LOVE* this for them 😂😂😂
😱😂 <--- 2 things can be true at the same time. And AI can get dangerous. Thing is, nobody is doing anything about it.
📌
We let cars drive for us and we let AI think for us. Before we all lose our last operational grey cells, let’s direct our brains to have us re-watch WALL•E, paying particular attention to the Buy ‘n Large segments.
AI also ain't paying for no malpractice insurance, so good luck, suckers.
I have to wonder if they won't be able to invalidate that contract because there was no meeting of the minds.
On the other hand, the inevitable end result of AI Galaxy brain is going to be something along the lines of: "sure, you can contract directly with an AI since they have an 'artficial' mind."
It would be funny if somebody claimed an undisclosed conflict of interest based on Claude's training materials
It’s like I always tell clients. You can pay me now, or pay me much more later. Up to you. Clients relying on AI legal “drafting” is just a new version of an old story.
Oops! 🤣 AI sucks
As a health coach, I don’t like Claude taking away potential clients.
A good contract provision when hiring any kind of written work product could be that no AI will be used, or must be disclosed and requires written approval to include in product. Not sure what teeth would work to enforce.
We've started including AI provisions* in our supplier agreements. *meaning, prohibitions or limitations on the use of AI as well as disclosure of any use. also generally prohibiting their use of our company data that they have access to to feed AI engines.
right? Also shows that on top of using Claude, they didn't even have it proofed. Not even to ensure it all works in their favour. Talk about lazy and irresponsible.
I once had to point out to a lawyer that his lease edits had eliminated all notice-and-cure rights for his client. And he got mad at me.
How is whoever is operating Claude not guilty of the UAPL?
This is the kind of non-stop hagiographic coverage of AI that caused me to stop listening to his podcast. bsky.app/profile/moxf...
There are a few NYT podcasts that have become unlistenable. I had to stop with The Daily after realizing I often wanted to throw my phone at a wall because the “analysis” was so bad/biased.
📌
Hey, not your job to help the other party from shooting themselves in the foot legally, they have a guy for that.
@anchorbabynotes.bsky.social
well... I mean... for a while it generates more work for those who have to go in and fix it?
I wrestled with the ethics for a hot minute when I had this happen I don’t think there is a _professional_ ethics obligation to tell a counterparty they did something stupid but in the past, I have been on the giving and receiving end of ‘I think you made a mistake’ and it felt . . . Good?
Generally speaking, there are lawyers in the other side of deals you feel good discussing errors with (going either direction) and then there is the other kind. The reason for the error is irrelevant.
I tell my clients that its my job to take the harsh position b/c the other party doesn’t need to like me. Once the deal is done, unless something goes wrong, they should never talk to me again. Having said that, I also tell my clients that a harsh position doesn’t mean being harsh. (Files/honey)
For sure!! After a very tough negotiated deal, I had the attorney on the other side refer a client to me a few months later. Negotiations don’t have to be nasty and can build respect between the attorneys.
I’ve picked up clients the same way. It always makes me think of one of those war movies where the opposing soldiers provide a grudging salute.
I imagine you do get more files if you treat the other party very generously, typo aside it still works
You’re the first person to mention it, but the typo has been bothering me since I posted it.
also isn't "Claude" the one that kept telling people they for sure had cancer
They're about to see that therapist a lot more...
I had a client use an AI platform to "check my work". It added a provision that was completely wrong and would have gotten the client sued. Thankfully the client vetted the new contract with me.
Error checking does seem like a good use of AI, functionally we’ve been doing that forever in Word. But the default should be to only highlight possible errors, not add substantive text.
I just had that lightbulb moment about an incredibly stupid contract that included a hilarious self-indemnification clause that I reviewed for a non-profit some time ago. Incompetent AI makes so much more sense than "the lawyer got drunk half-way through the clause, and reversed terms."
I may have seen that same self-indemnification clause! Wild times.
I’m convinced that “Kevin Roose” is an AI-generated persona and you can’t prove otherwise. Who’s even named “Kevin” anymore? Do better, AI.
😂
Wonder if they billed the regular amount for drafting.
I wondered too. The platform's model seems to be "AI does the contract, a lawyer reviews and approves it" and, well, the only way this is profitable is if the lawyer is paid a trivial amount and really does not give a shit what's in it, they just click "approve."
Well that's one way to lose one's license
Lawyers using AI is an “old man shakes fist at clouds” moment for me, tbh. IANAL, but work in the orbit of contract lawyers & civil litigators. Haven’t seen AI used like that in the wild, that I’m aware of, but the eager and fast integration into fucking everything is not promising, imo.
It might eventually get better? 🤷♂️ The AI tech lords appear to be quite content to train their products by releasing weak beta versions. There's no guarantee that it will get better. But there are other areas of AI that were weak for decades but eventually became excellent. 🤷♂️
I've seen that without the intervention of AI. @fascinatorfun.bsky.social
The other side might have been willing to take the risk: They get a quick low-cost turnaround, and the bad-for-them insertion never arises IRL for them and -your- client.
It would be their attorneys ethical responsibility to advise them if just how ridiculously stupid that plan is, and why they'd have to be an idiot to even consider such an idea.
Sure, but part of the calculus from has to be that there's some degree of cost risk associated w/ their own ignorance re: the extent & impact of the potential exposure that they're opening themselves up to. If they're proactively making self-harming edits based on AI, it strongly suggests ignorance.
“You’ve got lots of claude” is my new go-to phrase for “You do not brain good.”
I may have to steal this.
I feel like the whole point of redlining is the thinking part. The redlines are the product of the thought, which the AI can’t do. I do think there might be some value for screening low-risk procurement contracts.
As a couple therapist, anybody who thinks they can see me "75% less" because they are individually asking an AI how to handle their relationship problems is welcome to their ongoing, intractable relationship problems.
Going to say soon to be worsening to the point of failure relationship problems
Potato, potato!
I saw a sentence in a contract not so long ago that ended with “including, for the avoidance of doubt.” I wondered if it was just carelessness or if AI was also involved.
Could have been carelessness and using a template, ie, they were supposed to take that part out if there was nothing to list after "including"
LLMs haven't invented bad documents from lawyers. ;-) I worked at a startup, and the law firm sent them an Employee Manual to use which was their boilerplate. I refused to sign it because in the IP section it said that my spouse's work belonged to my employer. 🤣 A 100 other people signed it. 😜
If you feel comfortable disclosing, was the platform Ironclad?
This, and the problem is that they’ll never know, or at least until it’s too late, meaning neither they nor their “AI platform” can learn from the error.
📌
The “Ask Jeeves” of AI?
Ask Jeeves was a Hell of a lot smarter and more useful than any AI.
Only ask jeeves was useful and at the time better than the alternatives
Look if someone is going to send an important document without human review it's their own damn fault. However, AI can often extract very relevant data very quickly and efficiently. It just needs human review to make sure it's not garbage. Human + AI = very powerful indeed.
LLM (it's not AI, c'mon) = (very powerful) autocomplete. Once you have that paradigm firmly lodged in your brain, everything starts making sense. The whole point of it is to best guess what comes next and suggest it to you, regardless of its usefulness or truth value. The rest is up to you.
LLM's. Whatever. They are a very powerful tool that only 2 years ago became available. Their "best guesses" have become invaluable for me, especially as I like to learn, they are great teachers.
They're a tool, nothing more, nothing less. No magic under the hood, no matter what the marketing people say. You still have to understand and sometimes refine their suggestions. That said, I find it's always easier to improve a generated answer than to start from scratch.
Absolutely. Especially when it comes to writing code. Gemini AI is remarkable. I can't code...but with Gemini I can. Night vs day
“often” LLMs are inherently unreliable. And unreliable output provided quickly and efficiently is still an impediment, not an advantage.
I have to respectfully disagree. This is a hunan- ai synergy. With human intelligence and LLM speed, things that were impossible (or more accurately tedious and difficult) are now possible. It is a very useful tool.
You have an admirable level of trust in Monte Carlo simulations.
When those Monte Carlo simulations can write useful code for me I say: Bring it!
Uh huh.
There are those who know...and those who will be left behind. Choose wisely.
lol OK
Can I ask which platform?
The "outside counsel," who I checked is indeed a real licensed attorney, had their email address at Ontra, which was not the counterparty to the contract. Seems to be their "AI-powered, human-in-the-loop contract negotiation platform." www.ontra.ai/products/con...
I wonder if they have malpractice insurance. I think "generative AI" may eventually be a useful tool for lawyers, but if its the main selling point to the client, you're using it wrong. I'd go so far as to say fillable forms a la "legalzoom" are a better approach.
A friend of mine is busy trying to start-up his own contracting AI company... But it isn't that one.
Pretty funny that the article says they’re “falling for it.” They sure are. 😬
Save money now by having Claude write your trust. Your kids can pay a lot more for lawyers to fix it after you're gone!
Aren’t documents presented to the court that were partially or wholly generated by AI rejected by Courts? They weren’t prepared by a court officer or something?
📌
Brian Eno's warnings about the danger of relying on the surface appeal of mediocre AI applies to the law every bit as much as it applies to art. bsky.app/profile/mada...
I agree with everything Eno says in that quote, but the fact that he even feels the need to say it is kind of freaking me out. Shouldn’t it be obvious to everyone that if you’re truly interested in art & not just commerce, the process of making it is just as important as the finished product?
Anyone who knows anything about art knows that the process of making it is just as important as the finished piece. It's why you can't take an anonymous piece of art that you don't know the provenance or history of and sell it for a million bucks.
AI use is a byproduct of generations of bored schoolkids being rewarded for their writing based solely on word count and vocabulary usage, not actual content, meaning or analysis.
I prefer to use my Magic 8 Ball for all my lawyering
At least "reply hazy, try again" doesn't commit you to anything dangerous.
AI for therapy? That's a giant NFT¹ from me. — ¹ No Fuckin' Thank You
"make-shift therapy" JFC.
People used the original Eliza as a “makeshift therapist”. This does not mean Claude is smart. It means people are inclined to project personality and inner life on things. I suspect this helps us build empathy, but like many evolved traits, it doesn’t work in the modern world.
Toy Story 5 is gonna be wild.
📌
I think it would be good to get financial advice from a chat, but if lots of people are doing it because then you know which way the herd is moving and you can use that to make your own decisions
I love this. I shouldn’t, but it’s really golden.
We're going to have to update that saying about "fool of a client"
*for
Spelled "Claude" pronounced "clod"
Wealthy people using AI for legal matters is like "life finds a way" except for redistributing their wealth
I’m forwarding this to our head contracts lawyer 😂
How dare you. I'll see you in court. Bleep 😋
I mean it cites cases that don’t exist and gets black and white legal questions wrong. Why not let it write contracts too?!
I love the blind faith in tech such that their new magic box can master literally anything LOL. Good for them, I hope it all crumbles for them.
It's an extension or logical follow-on to the idea that it only takes watching a few YouTube videos to make anybody on the internet an expert in whatever topic is currently in the news.
Every tech product is just a regulatory breach, making money until it gets shut down or costs imposed. It ends up being way more expensive. See, e.g., Uber. However, in this case, it also completely fucks the users. The work product is abysmal, a first year could do a better job without review.
I think outside of coding tools, the most i've seen targeting a sector is legal writing, which is weird because if you can ground in case law, my impression of legal writing is preciseness of language. maybe it can be helpful, but still requires a lot of editing.
I think that might be non-lawyers looking at legal writing for contracts and similar, noting that it is quite formulaic and frequently templated. And thinking, well AI can do that as easily as an expensive lawyer. But, the detail matters, so they are very wrong.
Yep. In a similar way to air traffic control, a lot of it is systematized, but the trick is in knowing how and when to use the various parts of that system. Just because you can make a template of something doesn’t mean that a totally untrained AI will know how and when to use it.
Also, an LLM type of AI's isn't even using the template. It is just generating something that is (possibly) stochastically similar to the template (and polluted by the rest of its training data). An intern, on their first day, set loose with some templates would be better / less risky.
Agreed. I always hesitate to commend “common sense”, but in this case it’s perfectly apt. An LLM simply can’t notice the kind of basic errors that even a totally untrained person will recognize immediately.
I’m in law and good at explaining tech, so I’m the one who has to explain to the SVP of Legal that an LLM can only make a lawsuit-shaped object. It’ll have a caption and citations and a prayer for relief and an sig block, and every word of it is no better than “lorem ipsum sit dolor amet.”
Oh gods, now I have a vision in my head of Elon convincing Trump that he should fire all the ATCs and replace them with an LLM.
When I was a trainee controller in 2009, my boss was absolutely convinced that we were all in danger of being automated out of existence in relatively short order. My fear was, and remains, that just what you’re suggesting is more likely. The automation doesn’t work, but it gets used anyway.
Thank you! Back when you could listen to ATC traffic on United, I figured out quickly I wouldn’t last 15 minutes in that job. The controller at SFO doing takeoffs and landings, *and* a GA flight. Being on the first plane to go around when the wind reversed at ORD. And they just calmly handle it all.
Just like plumbing: 25c for having the boilerplate paragraphs on your computer; $250 for knowing which boilerplate paragraph to insert to meet this client's needs.
My school teacher wife uses ChatGPT to make incorrect answers for multiple choice quizzes because it is better that being confidently wrong than anything else.
That is clever
Oh my God an actual use case for ChatGPT. Puts it one ahead of B*tco*n at least.
Now that's how you teach media literacy and critical thinking skills in a modern environment.
That’s a good use of it! I’ll try that. Wrong answers take too much time to come up with!
Need to make one or two by hand still IMO to test for specific errors or misconceptions, but it’s great for answers built to catch lack of effort
Hmmm. Given the training data and how LLMs work through approximation, it might be pretty good at coming up with misconception errors! But of coarse you should check, and/or create a lot of possible wrong answers to chose from.
She asks for a list of incorrect answers and chooses the best ones
I have mostly written STEM multiple choice tests, which should have at least one or two options embodying common math/physics errors; LLMs aren’t good enough at math to make the right sort of error, or even to have a reasonable shot at plagiarizing an error appropriate to the context.
Wow, a good use case!
For anyone interested she shared her prompt process with me: "Just something like "create a 30 question multiple choice test for "Text" by Author. Include 4 options for each question and include and answer key" Then you can also write, "create a 50 question review assignment based on the above test"
"Then you just put in your own answers for the right answers and keep the wrong ones. Edit as necessary. You can also input your own questions and ask for multiple choice answers"
Anyone else think Claude’s face resembles an…
If a lawyer messes up, you can hold them responsible. But you can't sue AI for incompetence. I think there is likely a place for AI, especially when it comes to e-discovery and sorting and summarizing info, but the final product needs to be reviewed, and you would need to deal with privilege issues
Why can't you sue it for incompetence? If someone advertises their AI can do a certain thing, charges you $$ for that thing, and fails to provide that thing competently, why can't that be the basis for a suit?
A decent tech startup *should* be consulting with a lawyer that will write into their TOS that you can't sue if there is a defect in what the AI generates. But you're right, not all tech startups will consult with a lawyer for their TOS
ToS isn't the be-all-end-all of contract or tort law. You can't necessarily disclaimer your way out of something if elsewhere on your website or marketing you make contradictory claims, for example.
Also if you're high on your own supply and have your legal AI write your own ToS... god help you.
I am convinced AI will end up generating even more business for lawyers.
Honestly I think the same thing. There are going to be so many incomprehensible or contradictory contract provisions. Makes me wonder if some clever lawyer will say the parol evidence rule shouldn't apply to AI contracts because you need to know what they were thinking when the K was generated.
Is "K" here an abbreviation for "contract"? Perhaps to avoid confusion with "C" for "client"?
Yes. For reasons I don't even know, lawyers use K to mean contract.
See also federal procurement. bsky.app/profile/pete...
Kan konfirm that.
I think the doubts expressed in that explanation are worth holding in mind. Things do not always make a clean, continuous progression of borrowings from medieval practice to modern practice.
Oh boy, let’s let the AIs battle it out. He’s what chatGPT says. At least Claude acknowledges that it doesn’t really know.
ROFL at Black's Law Dictionary and the UCC both using "K" for shorthand.
ChatGPT is the Confidently Mediocre text diarrhea spigot, yes
I don’t know. The same lawyers who carelessly use AI are likely careless when not using AI.
I wonder whether the use will generate business in narrow segments of the legal world. Large firm malpractice carriers and risk folks send a million reminders about being careful about AI (and I’d bet have some pretty good exclusion language). Could hurt those who can least afford it.
AI is a much more efficient way to be careless, which you’d assume would have an impact.
I *can* see putting a doc, or a paragraph, into an AI, and asking if there is a better way to say this, or better support. But neither of those are automatic cut-and-pastes.
It’s going to change the practice. Work will still exist in abundance but lawyers have to be able to know how to use these tools effectively (and ethically!).
What’s privileged in this situation? The prompt that the company fed into the AI? Did they have an attorney look at it first? Does that matter if the company ratified it? The AI “guts” that spit out the contract? The process in choosing and eliminating words? But AI doesn’t think, like you said
Is the prompt parol evidence?
Now that is an interesting question!
Late and inexpert but I think that's a functional question. Does the prompt function as parol evidence or... Hm. If the prompt was not the product of discussion between the parties, but the product of only one party, then maybe? Maybe especially if it introduces ambiguities? Like contra proferentem?
Probably only if both sides were involved in writing the prompt. If only one side knows the prompt, it would not be evidence of the mutual intent of the parties.
What happens if there is a system update in the middle of a case?
Deposition question #322: Please provide all prompts used in the development of this contract…
This is like that employment contract someone generated for their company a little while ago, that inserted a “no termination without just cause” provision. Cool, I’m for it, that would help my person.
Gotta love it for future litigators!
In the same way the Internet did. The cheap easy stuff (will templates, rental contracts) became widely accessible, so supply opened up for other kinds of work. The history of automation is easy stuff gets cheaper, specialist stuff gets more expensive, and market expands in the middle.
You didn't need an Internet to have cheap easy standard contract things; you bought a Nolo Press book that had them. Once PCs were around, sure, it was easier to have a Word or Word Perfect template version of it to fill in the blanks and print, but that fit on a floppy disk, no internet needed.
Similar situation in the tax preparation arena. The proliferation of "AI prep" is going to be like TurboTax on overdrive... more business for those who do representation work.
I talked with a lay person recently who was planning to use AI to generate an agreement and I urged him to ask a lawyer for a template instead. Even if it’s wrong for him it won’t be total trash.
Even pulling someone else’s agreement from EDGAR and changing the party names would be better than using a chatbot. And a lawyer doing that would be committing malpractice.
a public law library in their state/county/city will have attorney-drafted sample clauses for a variety of agreements available, usually for free or for the cost of copies. the larger libraries may even have them available in a database, for easy emailing or downloading.
Who needs a lawyer when you have Claude. 😂
I am very excited for the arguments about whether AI can have scienter!
Looking at a case where the use of AI may (or may not) have jeopardized attorney client privilege….
One of my students in Legal Profession (PR) shared that their parent, a lawyer, had done quite well untangling messes made by Legal Zoom contracts and attempted corporate docs
This. I write you a contract? approx. $500-5000, depending on complexity. I litigate a situation you created with a badly-written AI contract? take the above numbers and add multiple zeroes.
(the fun fact is I'd *also* prefer the one that costs less, since it's also a *lot* less work and less stress for everyone involved)
So much work of good lawyers is convincing clients to do things that result in less work for us.
It’s always better to do 10 easy things for a hundred bucks than incredibly annoying thing for 1k.
Basically, NEVER use AI for something you don’t know. I use it often to give me first passes at stuff that needs generic, formulaic language, have first look at legislative language, and proofread stuff. But I ALWAYS check the output . It does save time , but it is not reliable at all.
Yeah, this. It's pretty good at code, which isn't surprising considering the types of people that make it, but you still have to be able to read it to see where it goes wrong
This is the best use IMO. Helps to get things rolling and move off from a blank page.
Nice.
📌
Is AI replacing beer as the cause of, and solution to, all of life's problems? (H/t Homer Simpson.)
No because you will need a lot of beer to deal with all of the solutions AI creates
I find it very troubling that the tech term for this is AI hallucinating. This is AI lying. I think we should be very worried about a computer program that can just make things up and lie.
Calling it lying would imply that the LLM knows what it's saying isn't true ¯\_(ツ)_/¯
‘Lying’ is at least as accurate as ‘hallucinating’ if not a more valuable metaphor. In these cases, the LLM is performing as designed, and trained, which I don’t think is deliberate or of its own agency in the same way as “knowing what it’s doing” but is working according to its internal structure.
Hallucinating =/= lying. When someone hallucinates, their brain is "working according to its internal structure" also. That's why people show signs of fear when they hallucinate something scary. Are they "lying" about what they're seeing? No. Is it real? No.
No hallucinating isn’t lying. But GenAI isn’t hallucinating either. It’s operating normally producing misleading or wrong out put. To the extent that an LLM does anything on purpose, it’s closer to a lie than a hallucination.
Lying is making a statement you know to be untrue, in order to deceive. LLMs have no conception of what "true" even is, let alone the ability to deceive. Hallucinate is more accurate, i.e. creating plausible outputs based on internal mechanisms instead of factual information/ external stimuli.
I agree GenAI doesn’t have agency, nor perception. I prefer lying as a metaphor, it’s operating as intended. There is no difference between a hallucination and any other output. Unlike hallucinations, when some part of the signal/ interpretation chain isn’t doing as expected.
If we have to choose a human verb, what these guessing boxes do is bluff.
Yeah that’s a pretty decent metaphor.
We just got pitched an AI contract thing with a redline option I’d never ever use but at least it had a massive disclaimer banner at the bottom reminding all users that these are just suggestions and ultimately it’s each attorney’s professional responsibility to blah blah blah. No, thanks.
I have been grateful over the last few days that this thread is still going places. Thanks!
