The governing principle: applying old discovery rules to new AI data
Free. Watch without an account. Sign in to save progress and take the exam.
Up next
What AI evidence actually exists: inference layer and model layer artifactsTranscriptThe player shows YouTube’s automatic captions, which are machine captions we do not review. The full transcript is below.
00:00:00Yeah, I see. I mean, it's an arbitrator, I see those discovery requests all the time, they find it out.
00:00:05So it's not a thing, right?
00:00:09No, I mean, literally, they say all documents related to scoring, weighting, and relating to copyrighted material. Okay, let me tell you something.
00:00:18The corpus upon which they train may result in a substantial part of what we know as the internet. So everything there has a weighting score.
00:00:28So just so you're all clear, they may not even have all the logs. Like, they take snapshots of logs and like, don't even keep all of them.
00:00:37And then they're like, oh, spoliation. And then you're like, oh, my Lord. So like, I mean, it can get complicated because people don't understand.
00:00:44So both of your points, it's really important to have clarity as to what you're talking about, just to save yourself and your client money and time.
00:00:52And getting an outcome that at least is based on what you're seeking.
00:00:59So, governing principle, old rules, new data. So news flash, they haven't changed the federal rules.
00:01:07I know this is sometimes shocking to people, but they haven't modified the federal rules, so they still govern.
00:01:15I don't know, Mara, you used to do this a whole lot. You want to point out here?
00:01:19I mean, I feel like a broken record half the time when I tell people about like 37, I'm like, it's not how it works.
00:01:27Yeah, nothing much more beyond what you said, Daniel. The federal rules apply. Discovery is broad.
00:01:34Try to make your request proportional and try to convince a judge of the things Daniel said. Try to explain to them why something is just far, far, far too burdensome if you don't want to produce everything under the sun.
00:01:47And you'll see in all of the litigation that's happening across the country that really judges are struggling with this question because they don't understand what exists and what doesn't exist.
00:01:59And they need to be taught what burden looks like in this context.
00:02:04And it sort of reminds me of like when cell phone data first started to be collected and sort of the fights we had about that and about messaging data and sort of everyone needing to understand
00:02:14how that data was kept, how it was used, you know, the ability that parties would have to go and collect it.
00:02:20So we're all sort of going through this sort of education process right now. But yes, the federal rules still apply and we're figuring out how to interpret them or apply them here.
00:02:29And I would just add, so that's a very real problem. One hundred percent it's a real problem.
00:02:37Another very real problem is that we're dealing with a lot of new companies with immature document handling retention.
00:02:47No. No, I mean, not immature. They don't exist.
00:02:53They don't exist. And so which is also not OK. Right. But and a different problem from the problem I think we're discovering, you know, we're going to talk about just the inherent challenges of discovery in an A.I. or a generative A.I. tool.
00:03:09Right. That doesn't mean that you can't have reasonable document retention plans for ordinary stuff. Right.
00:03:18And so so the combination, I think, is where a lot of the agita is coming from right now. Right.
00:03:26Just because they can't produce this stuff doesn't mean they can't produce anything or shouldn't be to your point.
00:03:32Well, this is my whole point. Right. And so I hear both sides. I hear the plaintiff in the defense. Right. I want everything. You can't.
00:03:38It costs too much. And I hear back and forth, even between large corporations, you know, very large litigants on both sides of the V.
00:03:47And it's an interesting problem if once you get through and educate people, the real question is what is the benefit they realized by training the corpus, even if they didn't preserve the data, they still realize a substantial benefit.
00:04:02So how do you rate because they're like, oh, it's going to cost a bajillion dollars. But you're like, well, your company is now worth a bajillion dollars literally.
00:04:11And you realize that value in part because of how you know what you did and, you know, you know, so and to Karen's point all and to both Karen and Mari's point, these problems could be addressed if they did it up front.
00:04:25The problem is they go from like, and don't get me wrong. I don't fault them. Right. I mean, Tyra went from zero to I think it's worth fourteen billion dollars in five years and two thousand or three thousand employees.
00:04:38And I mean, there's just a lot of growth and a lot of complication. And that's just the company.
00:04:44But if the company puts in place the right pieces and the right tool sets, then it's in a position where it can actually respond to things in a constructive fashion rather than doing it retroactively.
00:04:56I'm sure Mari's brought in often to help clean up, like just privileged disasters because they use AI to like have full on conversations and like the engineers do it, too.
00:05:09And then they have the conversation with the AI, then they send it to counsel saying privileged on the email, but they don't realize that all the work they did to generate that isn't privileged.
00:05:20And, you know, it's it can be a headache. So, but I mean, so what actually exists to be discovered, I think this is really important just so you understand.
00:05:31Right. Because it reminds me of predictive coding back in the day or TAR or more. They had like all these fights about the words, but there's these concepts.
Automatically generated; it may contain errors.
Study materials
Notes, flashcards and practice questions for this lesson, reviewed before publication.
Study notes
Why the federal discovery rules still govern AI data
Daniel B. Garrie opens by stating the governing principle of the lesson: the federal rules have not been changed or modified to account for artificial intelligence. Even though the data involved (training corpora, scoring, weighting information) is new, the old rules still apply. Mari Grace confirms this directly, stating "the federal rules apply" and that discovery remains broad, but requests must be made proportional and arguments about burden need to be explained clearly to a judge.
The challenge of locating and defining AI training data
An early exchange in the discussion involves a discovery request asking for "all documents related to scoring, weighting, and relating to copyrighted material." Daniel B. Garrie explains that the corpus used to train an AI system may represent a substantial part of what is known as the internet, and that everything in that corpus has a weighting score attached to it. He points out that companies may not even have all the logs related to this process because they only take snapshots of logs and do not keep all of them. This creates situations where a party raises spoliation when, in reality, the data may never have existed in the first place because of how snapshotting works.
Proportionality and burden arguments for AI discovery requests
Mari Grace stresses that because discovery is broad under the federal rules, a party resisting production needs to make a proportionality argument and explain to the court why a request is "far, far, far too burdensome" if it does not want to produce everything. She notes that judges across the country are struggling with this question because they do not understand what data exists and what does not exist, and they need to be taught what burden looks like in this specific context.
Comparison to earlier technology discovery disputes
Mari Grace draws an analogy to when cell phone data first started being collected, describing the fights that arose over messaging data and the need for everyone to understand how that data was kept, how it was used, and what ability parties had to collect it. She frames the current AI discovery landscape as going through a similar "education process." Later in the discussion, Daniel B. Garrie makes a similar comparison to predictive coding (also referred to as TAR), noting that in that earlier era there were many fights over specific words rather than the underlying concepts, and he suggests a similar dynamic is happening now with AI discovery.
Document retention problems at AI companies
A point is raised that many new AI companies have immature document handling and retention practices. This is corrected in the discussion to clarify that the issue is not merely immaturity — in many cases, such retention processes do not exist at all. It is noted that this absence of retention practices is "also not OK," but it is treated as a distinct problem from the inherent challenges of discovery in an AI or generative AI tool itself. The point is made that a lack of reasonable document retention for ordinary business records is a separate issue from the technical difficulty of preserving AI training data, and that the combination of both problems is a major source of friction ("agita") in current disputes.
Weighing benefit realized against cost of production
Daniel B. Garrie frames a recurring tension: parties on one side argue they want everything, while the other side argues production costs too much, and this back-and-forth happens even between large corporations on both sides of litigation. He suggests that once people are educated on the issue, the real question becomes what benefit a company realized by training its corpus, even if it did not preserve the underlying data. He argues that if a company's value increased substantially because of what it did with that data, that benefit should be weighed against claims that production costs are excessive. He references a company that grew from zero to a valuation of fourteen billion dollars within five years and two to three thousand employees, framing this rapid growth as the source of a lot of complication, while stating he does not fault companies for this growth.
Building retention systems before disputes arise
Daniel B. Garrie argues that many of these problems could be addressed if companies built proper retention processes up front rather than reacting after litigation starts. He states that if a company puts the right pieces and tool sets in place early, it is positioned to respond to discovery requests constructively instead of retroactively.
Privilege risks when using AI tools internally
Daniel B. Garrie raises the issue of privilege disasters arising from AI use inside companies. He describes a pattern where engineers have full conversations with an AI tool, then send the output to counsel labeled "privileged" in the email, without realizing that the underlying work used to generate that output is not actually privileged. Labeling the email to counsel as privileged does not protect the underlying AI conversations that produced it, which is how these privilege problems arise.
Flashcards
13 cards. Test your recall, then flip to check.
Select the card, or press Space, to flip it. Left and right arrow keys move between cards.
1 of 13
Practice quiz
Practice — not graded, not the final exam