Bigger Picture: The Impact of AI for Attorneys
Free. Watch without an account. Sign in to save progress and take the exam.
Up next
Sign in to take the examPass it for a verifiable Certificate of CompletionThen
TranscriptThe player shows YouTube’s automatic captions, which are machine captions we do not review. The full transcript is below.
00:42:02quality. I think AI will be eventually, I think there'll be some hiccups along the way, but I
00:42:08really think that it'll really be a powerful tool to make access to justice a lot more accessible
00:42:15than what we currently see today, right? I mean, imagine being able to use an AI agent to write
00:42:21briefs and motions. And I know this because it's a chance neutral. I hear pro se cases on occasion
00:42:26and I can tell you that I can't wait for the day that pro se litigants can use AI to write their
00:42:32briefs and, you know, tell them write a brief following these rules, using these facts and this
00:42:38information, right? I mean, I think there's just huge value to that. And I think it increases the
00:42:44overall quality of the litigation and the process. And I, I'll, I mean, my question for you guys is,
00:42:50how do you train the lawyer? Right? So let's assume by the time you get there,
00:42:56I'll go back to my earlier statement. We won't be in court because if we can have the tools,
00:43:01pull the digital facts forward that you need that are objectively reliable because they're
00:43:07computer captured, machine device captured. If I could put the evidence in front of you,
00:43:14you wouldn't want to litigate either because it's a losing battle. People litigate when
00:43:18there's an absence of evidence, not where there is an abundance of it. Well,
00:43:24what I think is usually, usually, usually stay with that. I'm sorry, guys.
00:43:31One of the things I think is going to be, you guys make great points, but I think one of
00:43:33the things that's going to be really interesting is the metadata factor surrounding AI, right?
00:43:40Talk about chat GPT or the dictation in the phones or write the predictive language when
00:43:49you're writing in Microsoft in a Word document. You know, do people have choices between what they
00:43:54wanted to, you know, use for words, like just being able to have common ground for litigation
00:44:02purposes as to how all of this works and have a common understanding as to what it is to request
00:44:09and how to request it. I think that this is just going to be the next wave, like I mentioned before.
00:44:14I think there needs to be, I think, real institutional knowledge within the legal
00:44:20profession about how this stuff works and how you ask for it and what are we going
00:44:25to be fighting over because things were, you know, I mean, when you look at the past battles,
00:44:30right? You look at the jurisprudence over the past battles and say like,
00:44:33can you believe we were arguing over that kind of stuff, you know, 15 years ago?
00:44:38Yeah, but I would argue that most lawyers have no idea how computer-assisted learning works
00:44:43and relativity deploys in its true mathematical form, but almost every single lawyer uses
00:44:50relativity's computer-assisted learning at some functional level. I think that it's more about
00:44:55lawyers are going to be required to come up with checks and balances and controls
00:45:00to ensure and validate the results, right? I mean, that you're getting. I think that,
00:45:06you know, technically and mathematically, it is a very complicated thing to understand
00:45:12how the actual large, you know, chat GPT is actually five or six different LLMs,
00:45:17all compliant in a neural network. But it's quite complicated, right? Or can you look at
00:45:22Lambda from Facebook that released their own large language model, right? I mean,
00:45:28I think that it's quite complicated for lawyers to understand it, but I think lawyers can create
00:45:32controls and processes to manage that. And I think that trust will have to be developed,
00:45:38but I don't know, council regard, council gal, what your thoughts are there?
00:45:42I think it's a Volkswagen Beetlebop problem, you know, the new Volkswagen Beetlebop, the bug,
00:45:48the modern version of it is such a pleasure to drive. I mean, power steering, air conditioning,
00:45:54satellite radio, no one knows how to fix it. The old one, everyone knew how it worked.
00:45:59I don't think users want to know how it works. They want to know what their user experience is.
00:46:04And if the user experience is succulent, they're going to want it. Okay. Even if they can't
00:46:10explain it. Yeah, I mean, I don't know, council gal, what do you think?
00:46:17I just think that there needs to, you know, how much litigation was, if you read these like
00:46:25the like tar case primers and stuff on the amount of jurisprudence that came down just
00:46:30arguing over how to apply tar and discovery review, you know, across the United States.
00:46:37I mean, it's like unbelievable how many cases over the years that have been on it.
00:46:41I'm just wondering if the industry, you know, if the profession could get ahead of the curve,
00:46:46the judiciary can get ahead of the curve on on AI to be able to, you know, avoid a lot of disputes
00:46:52surrounding the technology and just come to a more common technical understanding.
00:46:57The courts have accepted, they accept the use of computer assisted learning. Like there's no longer
00:47:03in controversy, when people produce things, you don't have to disclose whether or not you use
00:47:08computer assisted learning to do your production. I mean, if you're asked or the court orders you,
00:47:13okay. But I mean, for run of the mill litigation, right, you don't have to disclose, I did this,
00:47:20you know, there's an acceptance among the courts of using the technology. I think the courts have
00:47:27a lot more resistance to computer assisted brief writing, but I don't think in the e-discovery
00:47:34realm of finding responsive information, the courts, there's going to be a fight about it.
00:47:39Because I don't think the, I think people, I mean, maybe I'm wrong. I mean, I don't know. I just,
00:47:48because I mean, have you heard of a computer assisted learning dispute recently where
00:47:53it wasn't some egregious set of behaviors? Yeah, a lot of them actually. Yeah, it's kind of
00:47:59great. I mean, I don't know. But those are tied to egregious behaviors, right? They're tied to like,
00:48:05no, look at all the litigation and diisocyanates in front of special master James Francis. Okay.
00:48:14A lot of litigation over a protocol around how to apply tar and which side was entitled to use it
00:48:21and which side was obligated to have the other side, give them the constraints on how to use it,
00:48:27when to use it, and where to use it. So I think there's more respect. So that was a protocol that
00:48:34should have been, been okay. So yes. Okay, fine. When the protocol, when it is in controversy,
00:48:40it is certainly heavily litigated. But if it is done, you know, like in 3M or in like a lot of
00:48:48the cases, right, they, they kick the tires, they have conversations, they go before the special
00:48:52master. And, but you know, there's, you know, I agree, poor protocol writing pro, like, but that's
00:49:00a function of a lack of knowledge and a lack of understanding and buyer be, I mean, or assumptions
00:49:06are being read into a protocol or process. And there's a lack of understanding when people
00:49:11agreed to it, right? I mean, I'm hearing an arbitration now where they just blindly agree
00:49:15to use a base tools and take whatever's produced. And I just got 2.2 million, 2.2 million documents
00:49:22produced in arbitration. And they agreed to the protocol. And I'm like, you know what it's going
00:49:27to cost to host that, let alone review it. And I mean, so yes, it does come into controversy,
00:49:33but I don't think they're debating the actual underlying technology veracity. They're debating
00:49:40the parameters upon which it operates. Nobody's saying it's not working. They're saying maybe
00:49:46the operators don't know how to use it. They're saying, well, let's go back to tar versus AI.
00:49:52Okay. When you say AI, I think people think of something that is largely self, self customizing,
00:50:02self adjusting. But you look at early tar models, there was a lot of litigation because
00:50:08people said, we think tar is great. We just don't think you know how to pull the levers
00:50:12and push the buttons. That's what I was meaning to get at before.
00:50:19But do you think they're going to litigate the AI models in the same way?
00:50:23Well, it's even worse because not only do I not think you know how to push the buttons,
00:50:28I don't even think you know which buttons are getting pushed because it's all getting done.
00:50:31They don't know. Okay. So then I retract my prior statement. And I'm convinced that
00:50:37you are 100% right. They're going to go to the bananas. They're going to go to town because
00:50:42they have no idea. I can't tell you that people are like, it's the new Volkswagen. Let's take it.
00:50:48It looks great. Time to use. Let's go for a joyride.
00:50:52Here's the good answer. Okay. We're not ready to let it run on autopilot,
00:50:57but it has huge value. You can use it to navigate the top cream of your documents.
00:51:04You can go to the high density. You can learn a lot and just realize that may be the starting
00:51:09point, not the ending point, and use it to branch out into your themes, your custodians,
00:51:14and your timeframes. Yeah. Okay. I buy that. And they're going to go to war over all of that.
00:51:19I can see that. Okay. I mean, I have a couple, but I just have such a short threshold for it.
00:51:25I'm like, are you really going to fight with me about the math? I like, I'm like,
00:51:30you really want to, like, you know, there's sometimes communication.
00:51:34I'm not going to fight with you over the formula. I'm going to fight with you over what
00:51:38values you put into your formula. I agree. And to me, my response is well,
00:51:42then just show me the values. And if I think they're reasonable, we can have a conversation
00:51:47and they say, Oh, I don't want to tell the other side of the product. Sorry.
00:51:50Yeah. That's my favorite client work product. I'm like, Oh, really? I go, well, let me,
00:51:56let me know how the work product works out. If that's how you want to fall on your laurels,
00:52:00I go, you got one of two choices. You can review it all or you can share it, right? Because I mean,
00:52:04there's no, there has to be a sense of confidence in the way the system performs. And you have to
00:52:09be able to demonstrate that to a level that the performance of a system works properly.
00:52:16Well, yes, but it's sort of like the throw the documents down the stairs method, right?
00:52:22I'm going to throw the documents down the stairs and what lands on the floor,
00:52:25I'm going to produce. And you say, that's, that's a horrible methodology.
00:52:29You don't know why it works, Dan. And I'd say, well, I don't need to know how it works
00:52:32because I sampled the documents that were left on the stairs. And from a statistical perspective,
00:52:38you got what you were supposed to get. Right.
00:52:41Except for the 5,000 other documents at the top of the stairs, it didn't get thrown down.
00:52:46Well, that's a different issue, but I, we can test the system from the
00:52:50inputs without knowing what happens in the middle.
00:52:53I agree. But you have to know that what, what, what the problem is with AI,
00:52:58particularly, you have to know what the data set was that, that left the station.
00:53:01And the problem is with AI and the data sets, right?
00:53:06When you're set up now, you'd have to have an AI that's designed to know the inputs
00:53:10and the outputs. Then you're okay.
00:53:13Well, they're the results where the results in and of themselves are self-validating because
00:53:18of the technology. And the early days of the automobile, people had problems with
00:53:25flat tires, no roads, bad gasoline. And, and people who were using horses would laugh at
00:53:32the automobile broken on the side of the road. And we're just kind of doing the same thing right
00:53:35now. We're laughing at the people who have a advanced technology failure, sort of a schadenfreude,
00:53:41but we're going to solve these problems. This is clearly the future. We just need to
00:53:46know. I agree a hundred percent. I think, I think AI, as it said, you know, on the slide here,
00:53:52it says it will move. I think the review phrase, I think the review phase, I think,
00:53:58I think it'll take a couple of years, at least two to three years for this to actually come
00:54:02fully. But I think eventually it will remove a large part of what the cost structures are
00:54:08today. I think the hosting will still prove to be problematic, but I do think that hopefully
00:54:14there'll be smaller data sets and that it'll be more efficiently done. And the cool thing about AI
00:54:20is that it's going to be so wide scale, right? Who here knows, you know, in your family,
00:54:25who actually knows what Cal is, right? I mean, who knows what Taurus, but most people in your
00:54:29family know what chat GPT is. Well, they tell you what, most people in my family do not know
00:54:34what chat GPT is, but they certainly know what Netflix is. Okay. Netflix, there you go.
00:54:38Okay. Netflix recommends movies. They know that. And they know when Spotify recommends
00:54:43songs. And so they understand that a machine can make a recommendation based on prior preferences.
00:54:49And that's really what matters. Right. And I think that I read an article yesterday in the
00:54:54Wall Street Journal that said, CIOs aren't really driving the generative AI discussions. It's the
00:54:59CEOs, which makes this technology trend a lot different than in prior transformations,
00:55:05where the CIO was really driving the technology transformation. And now it's largely the
00:55:10business people that are starting to drive it, which I think is good,
00:55:13because there'll be a lot more widespread and wider adopted in the broader community,
00:55:18which will make it more cost efficient. Right. And I think that will be the positive of it all.
00:55:24I mean, the real question I have, you know, is how do you train lawyers for people to properly
00:55:31find to use, to do this work? Right. I mean, think of the, what you learned in law school
00:55:37or what gets taught in law school today and ask yourself, how does that actually map into
00:55:44being able to functionally do these important tasks or even oversee the tasks?
00:55:51Look, I think there's going to be a big role, especially in the next 15 years,
00:55:5420 years for special masters and expert witnesses and consultants, because our ethics rules say,
00:56:01if you don't know it, you affiliate with someone who does. And that's perfectly appropriate.
00:56:05Well, I mean, council regarding a great business. So I mean, but I agree. And I think it's only
00:56:14going to get more complicated because I think there's ethical duties under the ABA rules at
00:56:181.1 common eight. And one point, there's a litany of them that require that. And I don't think,
00:56:25I mean, all those, you know, and I agree. And I think, but I think the future will be interesting
00:56:30and dynamic. And I think that AI has a huge potential to make discovery much different.
00:56:38I think in a decade, if we have this seminar in a decade, we'll be like, oh my God,
00:56:42is that really how we used to do it? How draconian? Well, I mean, we're limited by the tools we have,
00:56:48but I'll say this. There's a spectrum of the way you can use AI. And there are some
00:56:53actions on that spectrum that are very edgy today, but there are other actions that are
00:56:58absolutely not edgy and AI can bring great efficiency with high reliability today to
00:57:04large data sets and large problems. And that's where we see success. Just it's, how do you go past the
00:57:13automated version to the highly customized version that requires specialty knowledge can get you
00:57:19very cutting edge results with the right guidance. But again, there's this whole other bucket of
00:57:26reliable, dependable move forward. You can use it.
00:57:29Yeah. So here's to the old Volkswagen being reliable and dependable. This is
00:57:35you can fix it with a nickel, man. You could turn every screw.
00:57:40You're off to the races. I'm going to totally I'm sold. Thank you guys so much for your time.
00:57:45It was a pleasure and a privilege as always. And if people do have any questions again,
00:57:51please feel free to email us questions or anything. Use the supplemental materials
00:57:57and the presentation as you see fit. Just please don't solicit us. And with all that said,
00:58:03thank you. I'll say we're giving away copies of fact crashing. Everybody wants a copy of the book.
00:58:08Send me an email. I'm going to email you. I want one. Done.
Automatically generated; it may contain errors.
Lesson notes
Broader changes AI technology may bring to the legal profession.
Study materials
Notes, flashcards and practice questions for this lesson, reviewed before publication.
Study notes
AI and access to justice
The panel saw AI as a potential tool for broadening access to justice. Pro se litigants, for example, might use AI to draft briefs that follow the applicable rules using their facts. Better tools could raise the overall quality of litigation.
A related point: if technology can bring forward objective, machine-captured facts, many disputes may never reach court. People litigate when evidence is absent, not when it is abundant.
Lawyers do not need to understand the math, but they need controls
Most lawyers use technology-assisted review features in review platforms without understanding the underlying mathematics. Large language models are even more complex. The panel's view was that lawyers do not need to understand how the engine works; they need checks, balances and validation processes that show the results are reliable.
One panelist compared this to the modern Volkswagen Beetle: pleasant to drive, but no one knows how to fix it, unlike the old model everyone understood. Users care about the experience, not the mechanism. Similarly, most people understand that Netflix or Spotify recommends content based on past preferences, which is the core idea behind many review tools.
What matters is that a party can demonstrate the system performed properly. Panelists described testing a process through its inputs and outputs, for example by sampling what was left behind, without knowing what happens in the middle, provided the data set fed into the system is known.
Why AI review protocols will be litigated
Courts now accept computer-assisted review; parties generally need not disclose that they used it unless asked or ordered. But disputes have arisen over TAR protocols: which side may use it, under what constraints, and how. The panel's view was that those fights concern the parameters, not the validity of the technology:
- "I'm not going to fight with you over the formula. I'm going to fight with you over what values you put into your formula."
- With early TAR, opponents accepted the tool but doubted the operator knew how to pull the levers.
- With AI, disputes may be worse, because parties may not know which levers are being pulled at all.
Poorly drafted protocols cause problems. One example was an arbitration where parties agreed to a protocol and accepted whatever was produced, resulting in 2.2 million documents, with significant hosting and review costs. Claiming the model's settings are work product forces a choice: share enough to establish confidence, or review everything.
AI metadata is another emerging issue: records of how predictive text, dictation or generative tools shaped a document. The profession needs a common understanding of what can be requested and how, to avoid repeating the extended disputes that surrounded TAR.
Using AI as a starting point rather than autopilot
The practical consensus:
- AI is not ready to run on autopilot, but it has significant value now.
- Use it to navigate to the most relevant, high-density documents first, and treat that as a starting point from which to explore themes, custodians and time frames.
- Some uses are cutting edge and require specialized knowledge; others are reliable and efficient today on large data sets.
- AI is expected to remove a large part of today's review costs over the next few years, though hosting large volumes will remain costly.
Adoption is broad: business leaders, not only technology officers, are driving generative AI initiatives, which should spread the technology and lower costs.
Competence duties and the role of experts
The ABA Model Rules, including the competence rule and its technology commentary, require lawyers to understand the tools they use or to associate with someone who does. Law school does not yet prepare lawyers to perform or oversee this work. The panel expected a large role for special masters, expert witnesses and consultants over the coming decades as courts and parties work through AI-assisted discovery, and predicted that a decade from now today's methods will look primitive.
Flashcards
12 cards. Test your recall, then flip to check.
Select the card, or press Space, to flip it. Left and right arrow keys move between cards.
1 of 12
Practice quiz
Practice — not graded, not the final exam