Developments in eDiscovery
Free. Watch without an account. Sign in to save progress and take the exam.
Up next
Bigger Picture: The Impact of AI for AttorneysTranscriptThe player shows YouTube’s automatic captions, which are machine captions we do not review. The full transcript is below.
00:13:04it's important to distinguish, as data, I think, Counsel Regard, as you noted in the beginning,
00:13:09has gone everywhere, right? I do discovery now on vehicles. I do discovery on medical devices.
00:13:15And that is expensive discovery, because at the end of the day, there are a few forensic tools that
00:13:19can soundly collect data off of a CAT scan in a way that is defensible or collect data to scale off
00:13:26vehicles, right? I mean, so that, but then there's the germane, like, not germane, but the more
00:13:33mainstream, I guess you would say, you know, the email, the Slack channels, the messaging, but
00:13:39even mobile devices, right? I think, I think that maybe there are some mechanisms where this has
00:13:45all been, you know, developed or the tech maturity is calm, but even mobile devices present challenges,
00:13:51right? Not all iPhones are made the same, and not collecting from all iPhones is easy, right? Or
00:13:57androids or anything else. So I think that the diversity of data sources that are coming into
00:14:02the market make it more complicated. And there's no, and to your point, right, there's no one
00:14:08size fits all, right? If it's a bet the house antitrust litig, you know, or bet the house like
00:14:13mass torts case, right? There may be a need to search X, Y, and Z, or A, B, and C, right? There
00:14:19may be, but if it's like someone slipped and fell, but everybody knew about it because they circulated
00:14:23a meme, right? That may not be the reason to go looking, right? So, but I think that lawyers,
00:14:30it's incumbent upon them to be more strategic about what they're looking for to litigate their
00:14:34case and not litigate the discovery. I do think AI will help in that, but I do think at the end
00:14:40of the day, eventually, you know, because there'll be AI systems where you can query video, right?
00:14:46I'm already using them, right? Where you can ingest 50,000 hours of video and I can query
00:14:51all of the video and find exactly what I'm looking for, right? I have to get the data
00:14:55ingested, but then it, you know, real time sort of ingestion, lowering some of the costs.
00:15:00But I'd love to hear your thoughts about where is it going, right? I mean, it's been around,
00:15:04and I think council regarding council go, we both make excellent points. Like AI has a lot of
00:15:09potential, but AI is really broad, right? It is a lot. We see AI able to apply models that it's
00:15:19been trained on or programmed with to analyze data, where I think we're still going to have
00:15:24challenges though, is where our data systems are evolving and the AI models haven't been
00:15:30trained on what is emerging. So when Slack first came out, if you had already had an AI system
00:15:36and it didn't know Slack, it's not going to be able to help you process Slack the way it might
00:15:40for iMessage. And so we constantly have this whack-a-mole issue with emerging technologies.
00:15:48It's one of the issues that we've found challenges and experts and we've overcome it
00:15:52is how do you become an expert on a technology that's so new that no one has ever testified on
00:15:58it before and do so to a Daubert standard? And now we need to migrate that to an AI world.
00:16:08Well, I think another big challenge though is maybe not just sort of like culturally,
00:16:13but institutionally, the litigation process and the rules when they were initially
00:16:20promulgated, the rules of civil procedure contemplated that people wait for more or
00:16:26less. The spirit of them was as far as pre-trial discovery is concerned, practitioners would have
00:16:32or should have essentially what would be a moral level of certainty as to what was reviewed and
00:16:38what's being produced. Because as long as you had as many eyes on a set of documents,
00:16:45even if you were dealing with hundreds of thousands of pages over a period of time,
00:16:50you get several associates to do that. Then it developed into manual review,
00:16:55but just on document review tools, but sort of page by page and then keyword searches.
00:17:00And as things develop, now we're at a point in time where the rules have just recently
00:17:06in the grand scheme of federal litigation and even state litigation, depending on what
00:17:11jurisdiction you're in, just recently been amended to contemplate this idea that
00:17:17you are not going to be able to take a scorched earth approach on review because
00:17:21it's humanly impossible to do so. Even though AI has been around for a long time,
00:17:28I'm thinking that the idea of just relying upon the technology to do its job and knowing that
00:17:37you are not going to be able to physically review everything because it's humanly impossible
00:17:44is really interesting proposition. Well, a couple of quick comments to that.
00:17:50First of all, with the plethora of systems we have, which is one of the fundamental foundational
00:17:55definitions of big data, volume, velocity, and variety. We're talking about variety right now.
00:18:02With the variety of systems we have out there and the need for proportionality,
00:18:06we need to have a method to identify, qualify, and prioritize sources.
00:18:12This is such a big topic. We've actually written a book on that called Fact Crashing,
00:18:16which addresses exactly that, but it's not a problem that's going to go away,
00:18:20and one book's not going to solve the problem. It certainly helps.
00:18:24Related to that also is, Dan, you heard Council Gary make a comment earlier at the beginning
00:18:33that we're dealing with data that's human generated. I would say, actually, we're not.
00:18:38We're dealing with data that is human triggered, but a lot of the data we deal with is computer
00:18:44entered, but it's entered because a human caused the event to occur. Whether that's GPS
00:18:50stated where your vehicle traveled, well, you drove the vehicle. The food you consumed,
00:18:55your refrigerator data, you opened the refrigerator, you put the food in there.
00:19:00We have human actions being captured by automated systems, which is phenomenal
00:19:05from a discovery perspective, because it's much more objective, much more reliable,
00:19:10much more voluminous, and we can get huge insights where before
00:19:14we had blanks in the factual timeline. Now we have this great information source.
00:19:20Finally, I would say AI, most of the people that I speak with who are hesitant about it,
00:19:27it's because they don't want AI to replace the human decision making process,
00:19:32but the people who are being successful are the ones who are using it as
00:19:36AI to augment humans, not replace humans. AI is not going to replace humans,
00:19:41but this has been going on since the 60s, but getting past the hippie dippy 60s argument of
00:19:49replacing human beings, humans will evolve and new things will come about and it will be used
00:19:54as a tool to augment and make people more effective and efficient. You introduced a really
00:20:00interesting set of points, Council Regard, because I deal in cases where I know how much
00:20:05weight you put on the Mercedes when you sat in on the left side and how long you were sitting in it,
00:20:09so I know who's telling the truth, and I can tell you in seconds, and I play in forensics,
00:20:15where I know if your vacuum was running around your house and where it went.
00:20:19So there's vast amounts of data that are coming into play today, and I think the only way to
00:20:24manage that is through using some sort of algorithm to search and query and analyze document sets,
00:20:31and then there'll be various quality control mechanisms that will be set up to establish
00:20:36the quality of the product you're getting, right? Just because a surgeon says, I'm really good at
00:20:40operating on you, you're not going to go in and be like, okay, cut me open, I'm good to go,
00:20:45and he went to school at Daniels Medical School, right? You're good as dead.
00:20:49So like, I mean, there's a level of quality and controls that will certainly come into play,
00:20:54and the courts are going to compel that. The parties are going to compel it. One really
00:20:58cool thing about our litigation, while it may be expensive and exasperating at times,
00:21:03is it's adversarial. And by making it adversarial, there's always the constant pushing and probing
00:21:09of making people demonstrate and prove things. And I do think, as technology, I think collection
00:21:16with AI autonomous agents in the near future, maybe not for everything, but we'll solve a lot
00:21:21of the collection issues, not forensic issues, but collecting, saying, find me this information
00:21:26in my enterprise, and it will query your enterprise and find you the information, right?
00:21:31And then you're going to have to have controls in place that will be effective for ensuring
00:21:35that it operated properly and it didn't miss the server that the IT guy told it to skip, right?
00:21:39But, you know, that's just putting in the right controls to make sure that just like
00:21:44anything else, right? Like a surgeon, right? That's operating on you. He may have residence,
00:21:49but you sure want the guy making that one master cut to be the surgeon,
00:21:53not the resident who started in July 2nd, and you're in the hospital July 5th,
00:21:57right? So I think that will be addressed. And I think same thing for review. And it's been
00:22:01around forever for review, like tar, car, I don't know what they call it now,
00:22:05cow or whatever. I don't know what they call it. But the idea of training a system to find
00:22:10documents that's more efficient than human beings is statistically and mathematically
00:22:14proven to be better, right? So I don't, I think review, I think the cool part will
00:22:18be is you're going to connect it to data sources as a lawyer and say, find me all
00:22:22the relevant information and it will organize it and prioritize it. And then you can, you know,
00:22:27I think there's a lot of real, not now, I think three to five years from now. I don't think it,
00:22:31you know, I think Google is connected bar to Google drive and you can start to query it
00:22:35if you're using it properly to find your records and information in certain repositories.
00:22:40But I do think that it will be, that AI will make litigating a lot more efficient
00:22:46and accessible for a lot of people. I think it may take a little bit of hiccuping and costs,
00:22:51but I think eventually people will realize a substantial benefit. I think the cost of discovery
00:22:56will drop. I think, I think hopefully we get to the point where I can have an AI tool that can
00:23:03bridge the federated search problem. Our problem is not searching our systems. It's integrating
00:23:09them because they're all isolated. But if you could bridge the, the isolation of our systems,
00:23:13the silo of our systems and have an AI tool that you can communicate with the way that
00:23:17you're describing, I would also load the case law into my AI tool and say,
00:23:22look for the problem before I get sued. Just tell me in advance so I don't get sued in the first
00:23:27place. No, I agree. I mean, we're built, I'm working with some professors that like,
00:23:33that are building federated AI querying agents to automatically create connectors.
00:23:38And the AI is creating connectors and then querying the system to actually search
00:23:42these huge disparate ridiculously old school and some very new esoteric systems to look for things.
00:23:49Because to a council regards point, right? It's not about searching it. It's about searching it
00:23:54in a way that doesn't cost $5 million to load the tapes, extract the data by, I mean,
00:24:01people don't realize how complicated these things are, but like a tape is really complicated. You
00:24:05got to get the library that, you know, and everything's got to be loaded. And I think
00:24:09as these solutions get more and more sophisticated, I think it'll take a couple of years,
00:24:14but I think you'll see that because it just, it makes sense, right? I mean, there's just huge,
00:24:19I mean, forget the legal implications. Find me all the legal, like the business risks,
00:24:24there's so many things that AI will be able to help facilitate and make things better and more
00:24:29efficient. I just, so I mean, but I don't know, council governor sitting there staring like,
00:24:33what is Daniel smoking? No, no, no, no, not at all. Not at all. No, I mean,
00:24:39what can I circle back what I was saying earlier is I think that getting your head around
00:24:43psychologically, the discovery process, which has been for some time now, particularly since
00:24:52the adoption of TAR as being a best practice, is the idea that you can have, you know, one rule
00:25:02of procedure that says, for example, answering and discovery interrogatories under pains and
00:25:07penalties and perjury, but the next rule, a document production under rule 34, there's a
00:25:13best practice, but you're going to be relying upon the technology to assist in producing an
00:25:19accurate set of documents. It's humanly impossible to review everything yourself.
00:25:25It's just the one thing you want, right? You wouldn't want to, but I'm just saying,
00:25:29I think institutionally it's taken some time for us to get where we got eventually on TAR.
00:25:37The question, what I'm curious to see is it's going to happen the same way now with sort of like
00:25:41AI and the vernacular and the federal litigation system, if that makes any sense.
00:25:46Even though I, I agree with you. I just don't think they've hung out with millennials to
00:25:51watch how much data they generate in four hours on their phones is going through garbage.
00:25:55Like the amount of the idea of having someone look through four, you know, four hours of
00:26:01some teenager's video stream, just this, I don't, I don't see it, but my counsel,
00:26:06I don't think we're going to have to though, Dan. I think what we're already seeing is
00:26:11going back to the human generated versus computer generated data, computer generated,
00:26:16but human triggered. We're starting to do litigations now with our clients where we have
00:26:20no email collections at all. And I don't mean because we're collecting text messages.
00:26:25We're not collecting any unstructured content. It's all unstructured contextual metadata.
00:26:31We can solve problems because my clients have come to the question. If AI can read the
00:26:38documents and tell me what people wrote, I would rather know what they did than rather,
00:26:44rather than what they wrote that they did. Yeah. So we, we call that the crypto companies now.
00:26:50Okay. But in all seriousness, like, um, like, you know, I, you know, there's,
00:26:56we see this on the bleeding edge as well. I think that like litigation, you know,
00:27:01because all it is, is, is an algorithm that's constantly improving to find things more
00:27:05efficiently. Right. And so everybody's wed to email or messaging or WhatsApp or
00:27:11Signal or I don't know, you can pick, but all that is, is one slice of what you get.
00:27:16If you think about it at a practical way, when you sent the message, who else was called on
00:27:20the phone, what other instances all happened simultaneously is probably a better record to
00:27:25corroborate it. And then you can look at the content, right? But I think that, you know,
00:27:30I can't tell you a number of times it was, that was not my iPhone. It, my iPhone,
00:27:34I left it at work and it was connected to my workstation station. And it wasn't me looking
00:27:40at those pictures. And then we're like, yeah, it wasn't me. And we're like, okay. Well then
00:27:45funny enough, when we look at your iCloud and we analyze the other data devices that were triggered
00:27:50in the passing of information, they must've had access to a whole lot of things. It must've been
00:27:54a really sophisticated hacker. But I mean, yeah, I agree. I mean, that's the likely future. I think,
00:28:01yeah. Do you think that like, Mike, like Office 365, the next generation co-pilot is pretty
00:28:10fantastic. And I think that will streamline most email, pretty much all of it, pretty frankly,
00:28:18you'll be able to query systems within 16 months that are across the enterprise of Microsoft's
00:28:25je ne sais quoi to more effectively find things, I think.
00:28:29Yeah, but the challenge there for our corporate clients is even if they elect to have Microsoft
00:28:34as their dominant platform, their clients that they deal with every day have chosen to use
00:28:40telegraph or signal or fill in the blank and it doesn't stay within that paradigm.
00:28:46It does not. So, but I mean, at least you take that part of the paradigm off the table.
00:28:51So like I tell my clients, at least your stuff will be less of a headache. I go,
00:28:55all of the messages that you decide to change with them over telegram, I go,
00:28:59I don't know if you should be using telegram in the first place. But hey,
00:29:03if you are and you think it's an effect, you know, you know, but, you know, to your point,
00:29:08I mean, I know, Daniel, you you have cutting edge criminals. I'm sure that use all sorts
00:29:12of sophisticated, creative tools and his clients are criminals. I'm sorry, clients that are
00:29:17defendants that are wrongly accused, accused, my fault accused. They, I mean, I work now
00:29:26both as an expert in civil and criminal cases. And I see really creative uses by the accused
00:29:31of technology. I don't think it's going to change. I think if anything, it becomes more balanced.
00:29:37I think that in the criminal side of the house, the defense really has a hard time because
00:29:42everything sort of gets dropped on them at once. And I think having tools that will allow them to
00:29:47automatically query it and find what they need for their defense will be a really powerful
00:29:52application. I don't know, Counsel Gail, what your thoughts are there?
00:29:56Well, I mean, I, I think naturally, because there's such a proliferation of
00:30:04mobile technology with smartphones and, you know, that you want to get in tablets and exactly the
00:30:12wearables and all that stuff that a lot of it is you're not dealing with, you know,
00:30:19tons of different types of software applications across a huge, you know, enterprise organization.
00:30:28Most of, I think in some respects, particularly in criminal cases, you can run into complex,
00:30:35you know, search and seizure related issues and fourth amendment issues. But for the most part,
00:30:42when you're dealing with things like self-right extractions or encased, that kind of stuff,
00:30:46it's, it's, it's pretty contained. But my question is like, what about the new work technology,
00:30:53self-destructing messages, right? I can't tell you the number of times I tell people
00:30:58self-destruction means that either they took a snapshot of the message they got and they
00:31:02have it in their photo library because they're paranoid or it's bye-bye. Like I don't, I don't
00:31:09like, especially on a phone that's using a solid state disc on it and the trimming and garbage
00:31:15collection. I mean, the odds of finding that message are infinitesimally small.
00:31:21Yes, agreed. If ephemeral messaging is its own panel discussion, we're not going to get into
00:31:29that here. I think the bottom line is you raised some great points, counselor Gary, that AI can
00:31:36really help a litigate, especially, you're right, in the speed of the criminal trial
00:31:42when the prosecution drops all other evidence at one time and the defendant is challenged with both
00:31:48time and resources, AI tools can help them process that and really try to understand that Brady
00:31:54disclosure a lot better. And we do the same thing though in protracted civil discovery where we can
00:32:01use these tools to get a heads up and a handle of what we're dealing with. So many times I see
00:32:07parties spend hundreds of thousands of millions of dollars to produce discovery, the burden of
00:32:13discovery to the other side. They've met their burden, but they haven't themselves benefited
00:32:18from what they've collected at all. The biggest challenge, I think, in terms of
00:32:25from an individual client standpoint on a criminal case, it's relatively contained
00:32:32in terms of the discovery of involving the individual defending clients. Now, obviously,
00:32:39it's more complex if it's a corporate defendant, but most criminal cases are individuals for the
00:32:45most part. But the challenge- Did you read all of the files I give you?
00:32:53But that's what I'm turning to, is that the data dumps, that's really the biggest challenge.
00:33:00Yeah, that's what we're saying. You'll connect your data dump into an AI thing and say,
00:33:04here's our defense, find me the documents. Right, so the big challenge is
00:33:09the Janks Act, right? So Janks material. When Janks was passed, they weren't thinking about, well,
00:33:19you're going to have to wait until this government witness testifies, until we're then going to produce
00:33:26to you a terabyte worth of data, and you can go through and then figure out what's in
00:33:32material. Like, it's just, it's impractical, right? So that's really the big, the issue is dealing with
00:33:39the data dump. Now, I think AI is going to be extremely powerful in being able to deal with
00:33:44that because there are inputs and there's functions that would be able to analyze context,
00:33:51which is so difficult, especially when you're under the gun on time.
00:33:55So, Counsel Gelb, tell the audience, not everybody knows what AI does. How does AI,
00:34:03what is it about AI that helps it go through a terabyte of data in under a day in your scenario?
00:34:10For example, if you were to have a program to be able to analyze, what is, for example, most
00:34:26detrimental, something that's in context, right? Like, what is most, for example,
00:34:35consistent between these individual statements that were made to these agents between this time
00:34:42period, you know, something along those lines. It builds a language construct.
00:34:50It takes pieces of language that are in two separate documents, it connects them together.
00:34:55It creates an entity extraction analysis, understands people, places time and events.
00:35:00And it can overlay other data, right? That's why it's so interesting, right? You give it
00:35:04the terabyte of data, it'll connect your entire set of facts for your arguments and cases,
00:35:10put together the context, and then you can query the system saying,
00:35:13please identify conversations between this agent, these agent interviews and these parties at these
00:35:19times. And then they'll query it and they'll find things that you're not even thinking about,
00:35:23like, and they look at the phone call logs of X, Y, and Z and say, well, we also identified this
00:35:29anomaly in X, this is impossible or whatever it is, right? But it'll be trained using, but,
00:35:35you know, you'll have to train it to counsel regards point, you'll have to have some training
00:35:40of it, right? So that it knows, you know, not right from wrong, but left, you'll need to know
00:35:46left from right. So it has to know your case specific rules of physics, you know, this can
00:35:51happen, that can't happen. Right, exactly. How does witnesses A statement conflict with
00:35:58witnesses B statement? Yeah, but I'll read all of it and tell you why it conflicts, how it conflicts,
00:36:04and give you their history and everything else. And then it may also look at other things that
00:36:09you haven't thought of, because depending on how you've trained the algorithm, right, it may actually
00:36:14analyze the data looking for discrepancies across the macro level, right? So, but the cool part
00:36:20is that you'll be able to take that terabyte, call it 400 gigs, just because OneDrive is one terabyte,
00:36:26and you'll be able to bring that into OneDrive and use, I don't want to say a rough shot,
00:36:32but like you'll be able to use a widely commercially available AI agent co-pilot on OneDrive,
00:36:39I assume, at some point in the near future to say, query it to find, assuming it can read all
00:36:44the documents, so again, counsel regards point, that does need to be able to read the data
00:36:48formats. But assuming that they're producing it to you as a load file, right, they're going to give
00:36:52you the text and the data files, hopefully, then it would just query the systems and you would then,
00:36:59you know, hopefully have trained it on your other case materials and it would find it.
00:37:02I mean, that will be the jerry-rigged version of what will, I envision probably 95% of the
00:37:10legal marketplace will use rather than buying more sophisticated tools, but that will be the
00:37:14reality, I think, in a lot of ways, right? And then you can say, write the motion, right? That's
00:37:20the crazy part. You can then tell it to write the motion and it'll write the first draft of the
00:37:24motion with the evidence that you just got and it'll give it to you in, you know, 10 hours. I
00:37:28don't know, if you're putting on your home computer, maybe it'll take a couple days, but,
00:37:32you know, if you're using some cloud-based Microsoft solution and you have a fast pipe,
00:37:37probably getting two, three hours. I mean, you need a gigabyte pipe, but anyways.
00:37:43Your bottleneck is how fast can you load the data into the processing ingestion engine?
00:37:48Well, that's the thing. OneDrive would have it built in, right? It would have its co-pilot
00:37:51built into the engine. It's about getting it into OneDrive and then OneDrive querying the total,
00:37:57the total amount of information. It won't be like fantastic, but it'll be
00:38:02much more efficient than you reading one 400 gigs of data and trying to turn that around. So.
00:38:09Even if it would tell me, even if the system wouldn't write the final answer, if it would
00:38:16merely say these are the documents you should focus on and here's an outline that I would
00:38:21recommend, but you should read the documents and verify it, that's already a huge leg up.
00:38:27Oh, we're using that now in my organization. Exactly.
00:38:30I mean, we're already there. We just, we run our own, you know, we took our own large
00:38:35language model and built it out. But yeah, I mean, yeah, that, that, yeah, that, that's
00:38:42there, right? I mean, cause you don't even have to train it at that point, right? You,
00:38:46you just need to give it context because. The point is not using AI, any algorithm,
00:38:54to make a final decision. Everyone would agree we're not there yet and we may never be there
00:39:00for certain issues. So I like to use the phrase, don't use AI for adjudication. Use AI for
00:39:06navigation. Yeah, but that's what you use it for. It's like navigation. Yeah. I don't know.
00:39:13Google maps, man. Like, like, like you use it to get from A to B, but you're driving
00:39:19the car and know what's north and south. Well, I'll give you this thought. Maybe we
00:39:23could have started with this, but 10 years ago when I was driving to my in-laws house
00:39:28and my little girls in the back seat, we have the iPhone mounted on the dashboard and she's
00:39:34watching the map of the car driving on the map. You know, what's that blue line? That's where
00:39:39we're driving. What's the arrow? That's the car. What's that lock at the bottom? That's
00:39:44when we're going to get to grandmother's house. There was a pause. And then from the mouths
00:39:49of babes, right? She says that thing knows the future. So yes, my navigation tool was a
00:39:57fortune teller, right? And that was AI then. Yeah, I mean, it's way more advanced 100% and
00:40:05people don't realize that, right? They don't understand, you know, that, you know, that,
00:40:11you know, and it's, you know, and when you look at audio files, I've had cases recently where we're
00:40:16trying to ingest lots and lots of audio files and you can now the audio files can be actually
00:40:24automatically transcribed. It's about 90, 80, whatever, but you can then, but you can actually
00:40:30train to look for intonation now and all these other inflection and tone and voice. And we don't
00:40:34use any of that, but like AI also will make process because handling audio files and discovery
00:40:41and video files is actually one of the most expensive parts of the process because there's
00:40:47no efficient way to search and query massive audio and video libraries today. And it's also very
00:40:53impactful in front of the judge and jury. Yeah, when they bring it up, your deposition testimony
00:40:58and they're like real time, here's what Mr. Gary testified to. That just happened to me.
00:41:03It's impactful. I mean, you know, it means a lot, right? I mean, a video of you testifying brought
00:41:10up during cross can carry, can go as a strong message, right? As long as you're dressed well
00:41:15and sound credible, but otherwise, you know, but it's credible and juries pay attention.
00:41:20I agree. Look, I will say this. We're almost at the end of the hour. It just dealing with these
00:41:29technologies, watching their impact on large volumes of data has been impressive. Like you,
00:41:35I grew up as a computer scientist. I wrote AI systems and watching them now at this scale
00:41:42still impresses me. The faster we get our heads around the ways to apply this to practically
00:41:50solve our problems, the better off we all will be in terms of resolving disputes faster and cheaper.
00:41:56I think, and I, you know, just to build on that, right, is that I also think you'll increase the
Automatically generated; it may contain errors.
Lesson notes
New work technologies that have added complexity to the eDiscovery process.
Study materials
Notes, flashcards and practice questions for this lesson, reviewed before publication.
Study notes
The growing variety of discoverable data sources
Discovery now reaches far beyond email. Panelists described discovery from vehicles and medical devices, where few forensic tools can collect data defensibly, as well as Slack channels, messaging apps and mobile devices. Even phones vary: not every iPhone or Android device collects the same way.
Volume, velocity and variety are the classic features of big data, and variety is the pressing problem. With so many systems, counsel needs a method to identify, qualify and prioritize sources. There is no one-size-fits-all approach:
- A bet-the-company antitrust or mass tort case may justify searching many sources.
- A simple slip-and-fall case may not, even if relevant content exists somewhere.
Lawyers should be strategic about what they need to prove, and litigate the case rather than the discovery.
Human-triggered, machine-recorded data
One panelist reframed much of today's ESI as data that is computer-entered but human-triggered: GPS records exist because someone drove the car, and appliance logs exist because someone opened the refrigerator. That data is often more objective, more reliable and more voluminous than what people write, and it fills gaps in factual timelines. Examples included vehicle sensors showing how much weight sat in a seat and for how long, and data showing where a robot vacuum traveled.
Some matters are now litigated with no email collection at all, relying on contextual metadata, because clients would rather know what people did than what they wrote about what they did. Device and account data can also test claims such as "that wasn't me on my phone."
How the rules adapted to technology-assisted review
The rules of civil procedure were drafted when lawyers could have near certainty about what was reviewed and produced, because teams of associates could read every page. Practice moved from page-by-page review to review platforms and keyword searches. More recently, rules and practice have accepted that exhaustive manual review is humanly impossible.
Technology-assisted review (TAR), in which a system is trained to find responsive documents, became a best practice because it has been shown statistically to be more efficient than human review. Panelists noted the tension this creates: a party answers interrogatories under penalty of perjury, yet document production under Rule 34 relies on technology to produce an accurate set. Adopting TAR took time institutionally, and the same adjustment is now happening with broader AI.
Limits of AI and the need for quality control
- New data types. AI models trained on existing systems may not handle emerging ones; a model that did not know Slack could not process it like iMessage. This creates a constant whack-a-mole problem, and experts must establish reliability for new technologies under the Daubert standard.
- Siloed systems. The hard problem is often not searching each system but integrating isolated systems, a federated search problem. Researchers are building AI agents that create connectors to query disparate legacy and modern systems.
- Controls. AI collection agents may soon find information across an enterprise, but controls must confirm they did not skip a server. Like a surgeon supervising residents, a qualified person must remain accountable, and the adversarial process will push parties to prove their methods.
- Augment, don't replace. Practitioners succeeding with AI use it to augment human judgment, not replace it. As one panelist put it, use AI for navigation, not adjudication.
- Ephemeral messaging. Self-destructing messages are rarely recoverable from modern phones unless someone captured a screenshot.
Using AI to manage data dumps in criminal and civil cases
Criminal defendants often receive enormous volumes of discovery at once, and the timing of Brady and Jencks Act disclosures was not designed for terabytes of data produced when a government witness testifies. AI can help by building language models of the material, extracting entities (people, places, times and events), connecting related passages across documents, overlaying data such as call logs, and flagging inconsistencies between witness statements. It must be given case context, the case-specific rules of what can and cannot happen.
Similar tools help in civil matters, where parties often spend heavily to produce discovery without learning from what they collected. Panelists expected widely available AI assistants built into enterprise platforms to let lawyers query produced load files, identify the documents to focus on, outline arguments and even draft a first version of a motion, with the lawyer verifying the results. Audio and video, among the most expensive data to review, are becoming searchable through automated transcription and indexing, and video evidence can be powerful before a judge or jury.
Flashcards
13 cards. Test your recall, then flip to check.
Select the card, or press Space, to flip it. Left and right arrow keys move between cards.
1 of 13
Practice quiz
Practice — not graded, not the final exam