Legal Cyber Academy

Developments in eDiscovery

Free. Watch without an account. Sign in to save progress and take the exam.

Up next

Bigger Picture: The Impact of AI for Attorneys
TranscriptThe player shows YouTube’s automatic captions, which are machine captions we do not review. The full transcript is below.

00:13:04it's important to distinguish, as data, I think, Counsel Regard, as you noted in the beginning,

00:13:09has gone everywhere, right? I do discovery now on vehicles. I do discovery on medical devices.

00:13:15And that is expensive discovery, because at the end of the day, there are a few forensic tools that

00:13:19can soundly collect data off of a CAT scan in a way that is defensible or collect data to scale off

00:13:26vehicles, right? I mean, so that, but then there's the germane, like, not germane, but the more

00:13:33mainstream, I guess you would say, you know, the email, the Slack channels, the messaging, but

00:13:39even mobile devices, right? I think, I think that maybe there are some mechanisms where this has

00:13:45all been, you know, developed or the tech maturity is calm, but even mobile devices present challenges,

00:13:51right? Not all iPhones are made the same, and not collecting from all iPhones is easy, right? Or

00:13:57androids or anything else. So I think that the diversity of data sources that are coming into

00:14:02the market make it more complicated. And there's no, and to your point, right, there's no one

00:14:08size fits all, right? If it's a bet the house antitrust litig, you know, or bet the house like

00:14:13mass torts case, right? There may be a need to search X, Y, and Z, or A, B, and C, right? There

00:14:19may be, but if it's like someone slipped and fell, but everybody knew about it because they circulated

00:14:23a meme, right? That may not be the reason to go looking, right? So, but I think that lawyers,

00:14:30it's incumbent upon them to be more strategic about what they're looking for to litigate their

00:14:34case and not litigate the discovery. I do think AI will help in that, but I do think at the end

00:14:40of the day, eventually, you know, because there'll be AI systems where you can query video, right?

00:14:46I'm already using them, right? Where you can ingest 50,000 hours of video and I can query

00:14:51all of the video and find exactly what I'm looking for, right? I have to get the data

00:14:55ingested, but then it, you know, real time sort of ingestion, lowering some of the costs.

00:15:00But I'd love to hear your thoughts about where is it going, right? I mean, it's been around,

00:15:04and I think council regarding council go, we both make excellent points. Like AI has a lot of

00:15:09potential, but AI is really broad, right? It is a lot. We see AI able to apply models that it's

00:15:19been trained on or programmed with to analyze data, where I think we're still going to have

00:15:24challenges though, is where our data systems are evolving and the AI models haven't been

00:15:30trained on what is emerging. So when Slack first came out, if you had already had an AI system

00:15:36and it didn't know Slack, it's not going to be able to help you process Slack the way it might

00:15:40for iMessage. And so we constantly have this whack-a-mole issue with emerging technologies.

00:15:48It's one of the issues that we've found challenges and experts and we've overcome it

00:15:52is how do you become an expert on a technology that's so new that no one has ever testified on

00:15:58it before and do so to a Daubert standard? And now we need to migrate that to an AI world.

00:16:08Well, I think another big challenge though is maybe not just sort of like culturally,

00:16:13but institutionally, the litigation process and the rules when they were initially

00:16:20promulgated, the rules of civil procedure contemplated that people wait for more or

00:16:26less. The spirit of them was as far as pre-trial discovery is concerned, practitioners would have

00:16:32or should have essentially what would be a moral level of certainty as to what was reviewed and

00:16:38what's being produced. Because as long as you had as many eyes on a set of documents,

00:16:45even if you were dealing with hundreds of thousands of pages over a period of time,

00:16:50you get several associates to do that. Then it developed into manual review,

00:16:55but just on document review tools, but sort of page by page and then keyword searches.

00:17:00And as things develop, now we're at a point in time where the rules have just recently

00:17:06in the grand scheme of federal litigation and even state litigation, depending on what

00:17:11jurisdiction you're in, just recently been amended to contemplate this idea that

00:17:17you are not going to be able to take a scorched earth approach on review because

00:17:21it's humanly impossible to do so. Even though AI has been around for a long time,

00:17:28I'm thinking that the idea of just relying upon the technology to do its job and knowing that

00:17:37you are not going to be able to physically review everything because it's humanly impossible

00:17:44is really interesting proposition. Well, a couple of quick comments to that.

00:17:50First of all, with the plethora of systems we have, which is one of the fundamental foundational

00:17:55definitions of big data, volume, velocity, and variety. We're talking about variety right now.

00:18:02With the variety of systems we have out there and the need for proportionality,

00:18:06we need to have a method to identify, qualify, and prioritize sources.

00:18:12This is such a big topic. We've actually written a book on that called Fact Crashing,

00:18:16which addresses exactly that, but it's not a problem that's going to go away,

00:18:20and one book's not going to solve the problem. It certainly helps.

00:18:24Related to that also is, Dan, you heard Council Gary make a comment earlier at the beginning

00:18:33that we're dealing with data that's human generated. I would say, actually, we're not.

00:18:38We're dealing with data that is human triggered, but a lot of the data we deal with is computer

00:18:44entered, but it's entered because a human caused the event to occur. Whether that's GPS

00:18:50stated where your vehicle traveled, well, you drove the vehicle. The food you consumed,

00:18:55your refrigerator data, you opened the refrigerator, you put the food in there.

00:19:00We have human actions being captured by automated systems, which is phenomenal

00:19:05from a discovery perspective, because it's much more objective, much more reliable,

00:19:10much more voluminous, and we can get huge insights where before

00:19:14we had blanks in the factual timeline. Now we have this great information source.

00:19:20Finally, I would say AI, most of the people that I speak with who are hesitant about it,

00:19:27it's because they don't want AI to replace the human decision making process,

00:19:32but the people who are being successful are the ones who are using it as

00:19:36AI to augment humans, not replace humans. AI is not going to replace humans,

00:19:41but this has been going on since the 60s, but getting past the hippie dippy 60s argument of

00:19:49replacing human beings, humans will evolve and new things will come about and it will be used

00:19:54as a tool to augment and make people more effective and efficient. You introduced a really

00:20:00interesting set of points, Council Regard, because I deal in cases where I know how much

00:20:05weight you put on the Mercedes when you sat in on the left side and how long you were sitting in it,

00:20:09so I know who's telling the truth, and I can tell you in seconds, and I play in forensics,

00:20:15where I know if your vacuum was running around your house and where it went.

00:20:19So there's vast amounts of data that are coming into play today, and I think the only way to

00:20:24manage that is through using some sort of algorithm to search and query and analyze document sets,

00:20:31and then there'll be various quality control mechanisms that will be set up to establish

00:20:36the quality of the product you're getting, right? Just because a surgeon says, I'm really good at

00:20:40operating on you, you're not going to go in and be like, okay, cut me open, I'm good to go,

00:20:45and he went to school at Daniels Medical School, right? You're good as dead.

00:20:49So like, I mean, there's a level of quality and controls that will certainly come into play,

00:20:54and the courts are going to compel that. The parties are going to compel it. One really

00:20:58cool thing about our litigation, while it may be expensive and exasperating at times,

00:21:03is it's adversarial. And by making it adversarial, there's always the constant pushing and probing

00:21:09of making people demonstrate and prove things. And I do think, as technology, I think collection

00:21:16with AI autonomous agents in the near future, maybe not for everything, but we'll solve a lot

00:21:21of the collection issues, not forensic issues, but collecting, saying, find me this information

00:21:26in my enterprise, and it will query your enterprise and find you the information, right?

00:21:31And then you're going to have to have controls in place that will be effective for ensuring

00:21:35that it operated properly and it didn't miss the server that the IT guy told it to skip, right?

00:21:39But, you know, that's just putting in the right controls to make sure that just like

00:21:44anything else, right? Like a surgeon, right? That's operating on you. He may have residence,

00:21:49but you sure want the guy making that one master cut to be the surgeon,

00:21:53not the resident who started in July 2nd, and you're in the hospital July 5th,

00:21:57right? So I think that will be addressed. And I think same thing for review. And it's been

00:22:01around forever for review, like tar, car, I don't know what they call it now,

00:22:05cow or whatever. I don't know what they call it. But the idea of training a system to find

00:22:10documents that's more efficient than human beings is statistically and mathematically

00:22:14proven to be better, right? So I don't, I think review, I think the cool part will

00:22:18be is you're going to connect it to data sources as a lawyer and say, find me all

00:22:22the relevant information and it will organize it and prioritize it. And then you can, you know,

00:22:27I think there's a lot of real, not now, I think three to five years from now. I don't think it,

00:22:31you know, I think Google is connected bar to Google drive and you can start to query it

00:22:35if you're using it properly to find your records and information in certain repositories.

00:22:40But I do think that it will be, that AI will make litigating a lot more efficient

00:22:46and accessible for a lot of people. I think it may take a little bit of hiccuping and costs,

00:22:51but I think eventually people will realize a substantial benefit. I think the cost of discovery

00:22:56will drop. I think, I think hopefully we get to the point where I can have an AI tool that can

00:23:03bridge the federated search problem. Our problem is not searching our systems. It's integrating

00:23:09them because they're all isolated. But if you could bridge the, the isolation of our systems,

00:23:13the silo of our systems and have an AI tool that you can communicate with the way that

00:23:17you're describing, I would also load the case law into my AI tool and say,

00:23:22look for the problem before I get sued. Just tell me in advance so I don't get sued in the first

00:23:27place. No, I agree. I mean, we're built, I'm working with some professors that like,

00:23:33that are building federated AI querying agents to automatically create connectors.

00:23:38And the AI is creating connectors and then querying the system to actually search

00:23:42these huge disparate ridiculously old school and some very new esoteric systems to look for things.

00:23:49Because to a council regards point, right? It's not about searching it. It's about searching it

00:23:54in a way that doesn't cost $5 million to load the tapes, extract the data by, I mean,

00:24:01people don't realize how complicated these things are, but like a tape is really complicated. You

00:24:05got to get the library that, you know, and everything's got to be loaded. And I think

00:24:09as these solutions get more and more sophisticated, I think it'll take a couple of years,

00:24:14but I think you'll see that because it just, it makes sense, right? I mean, there's just huge,

00:24:19I mean, forget the legal implications. Find me all the legal, like the business risks,

00:24:24there's so many things that AI will be able to help facilitate and make things better and more

00:24:29efficient. I just, so I mean, but I don't know, council governor sitting there staring like,

00:24:33what is Daniel smoking? No, no, no, no, not at all. Not at all. No, I mean,

00:24:39what can I circle back what I was saying earlier is I think that getting your head around

00:24:43psychologically, the discovery process, which has been for some time now, particularly since

00:24:52the adoption of TAR as being a best practice, is the idea that you can have, you know, one rule

00:25:02of procedure that says, for example, answering and discovery interrogatories under pains and

00:25:07penalties and perjury, but the next rule, a document production under rule 34, there's a

00:25:13best practice, but you're going to be relying upon the technology to assist in producing an

00:25:19accurate set of documents. It's humanly impossible to review everything yourself.

00:25:25It's just the one thing you want, right? You wouldn't want to, but I'm just saying,

00:25:29I think institutionally it's taken some time for us to get where we got eventually on TAR.

00:25:37The question, what I'm curious to see is it's going to happen the same way now with sort of like

00:25:41AI and the vernacular and the federal litigation system, if that makes any sense.

00:25:46Even though I, I agree with you. I just don't think they've hung out with millennials to

00:25:51watch how much data they generate in four hours on their phones is going through garbage.

00:25:55Like the amount of the idea of having someone look through four, you know, four hours of

00:26:01some teenager's video stream, just this, I don't, I don't see it, but my counsel,

00:26:06I don't think we're going to have to though, Dan. I think what we're already seeing is

00:26:11going back to the human generated versus computer generated data, computer generated,

00:26:16but human triggered. We're starting to do litigations now with our clients where we have

00:26:20no email collections at all. And I don't mean because we're collecting text messages.

00:26:25We're not collecting any unstructured content. It's all unstructured contextual metadata.

00:26:31We can solve problems because my clients have come to the question. If AI can read the

00:26:38documents and tell me what people wrote, I would rather know what they did than rather,

00:26:44rather than what they wrote that they did. Yeah. So we, we call that the crypto companies now.

00:26:50Okay. But in all seriousness, like, um, like, you know, I, you know, there's,

00:26:56we see this on the bleeding edge as well. I think that like litigation, you know,

00:27:01because all it is, is, is an algorithm that's constantly improving to find things more

00:27:05efficiently. Right. And so everybody's wed to email or messaging or WhatsApp or

00:27:11Signal or I don't know, you can pick, but all that is, is one slice of what you get.

00:27:16If you think about it at a practical way, when you sent the message, who else was called on

00:27:20the phone, what other instances all happened simultaneously is probably a better record to

00:27:25corroborate it. And then you can look at the content, right? But I think that, you know,

00:27:30I can't tell you a number of times it was, that was not my iPhone. It, my iPhone,

00:27:34I left it at work and it was connected to my workstation station. And it wasn't me looking

00:27:40at those pictures. And then we're like, yeah, it wasn't me. And we're like, okay. Well then

00:27:45funny enough, when we look at your iCloud and we analyze the other data devices that were triggered

00:27:50in the passing of information, they must've had access to a whole lot of things. It must've been

00:27:54a really sophisticated hacker. But I mean, yeah, I agree. I mean, that's the likely future. I think,

00:28:01yeah. Do you think that like, Mike, like Office 365, the next generation co-pilot is pretty

00:28:10fantastic. And I think that will streamline most email, pretty much all of it, pretty frankly,

00:28:18you'll be able to query systems within 16 months that are across the enterprise of Microsoft's

00:28:25je ne sais quoi to more effectively find things, I think.

00:28:29Yeah, but the challenge there for our corporate clients is even if they elect to have Microsoft

00:28:34as their dominant platform, their clients that they deal with every day have chosen to use

00:28:40telegraph or signal or fill in the blank and it doesn't stay within that paradigm.

00:28:46It does not. So, but I mean, at least you take that part of the paradigm off the table.

00:28:51So like I tell my clients, at least your stuff will be less of a headache. I go,

00:28:55all of the messages that you decide to change with them over telegram, I go,

00:28:59I don't know if you should be using telegram in the first place. But hey,

00:29:03if you are and you think it's an effect, you know, you know, but, you know, to your point,

00:29:08I mean, I know, Daniel, you you have cutting edge criminals. I'm sure that use all sorts

00:29:12of sophisticated, creative tools and his clients are criminals. I'm sorry, clients that are

00:29:17defendants that are wrongly accused, accused, my fault accused. They, I mean, I work now

00:29:26both as an expert in civil and criminal cases. And I see really creative uses by the accused

00:29:31of technology. I don't think it's going to change. I think if anything, it becomes more balanced.

00:29:37I think that in the criminal side of the house, the defense really has a hard time because

00:29:42everything sort of gets dropped on them at once. And I think having tools that will allow them to

00:29:47automatically query it and find what they need for their defense will be a really powerful

00:29:52application. I don't know, Counsel Gail, what your thoughts are there?

00:29:56Well, I mean, I, I think naturally, because there's such a proliferation of

00:30:04mobile technology with smartphones and, you know, that you want to get in tablets and exactly the

00:30:12wearables and all that stuff that a lot of it is you're not dealing with, you know,

00:30:19tons of different types of software applications across a huge, you know, enterprise organization.

00:30:28Most of, I think in some respects, particularly in criminal cases, you can run into complex,

00:30:35you know, search and seizure related issues and fourth amendment issues. But for the most part,

00:30:42when you're dealing with things like self-right extractions or encased, that kind of stuff,

00:30:46it's, it's, it's pretty contained. But my question is like, what about the new work technology,

00:30:53self-destructing messages, right? I can't tell you the number of times I tell people

00:30:58self-destruction means that either they took a snapshot of the message they got and they

00:31:02have it in their photo library because they're paranoid or it's bye-bye. Like I don't, I don't

00:31:09like, especially on a phone that's using a solid state disc on it and the trimming and garbage

00:31:15collection. I mean, the odds of finding that message are infinitesimally small.

00:31:21Yes, agreed. If ephemeral messaging is its own panel discussion, we're not going to get into

00:31:29that here. I think the bottom line is you raised some great points, counselor Gary, that AI can

00:31:36really help a litigate, especially, you're right, in the speed of the criminal trial

00:31:42when the prosecution drops all other evidence at one time and the defendant is challenged with both

00:31:48time and resources, AI tools can help them process that and really try to understand that Brady

00:31:54disclosure a lot better. And we do the same thing though in protracted civil discovery where we can

00:32:01use these tools to get a heads up and a handle of what we're dealing with. So many times I see

00:32:07parties spend hundreds of thousands of millions of dollars to produce discovery, the burden of

00:32:13discovery to the other side. They've met their burden, but they haven't themselves benefited

00:32:18from what they've collected at all. The biggest challenge, I think, in terms of

00:32:25from an individual client standpoint on a criminal case, it's relatively contained

00:32:32in terms of the discovery of involving the individual defending clients. Now, obviously,

00:32:39it's more complex if it's a corporate defendant, but most criminal cases are individuals for the

00:32:45most part. But the challenge- Did you read all of the files I give you?

00:32:53But that's what I'm turning to, is that the data dumps, that's really the biggest challenge.

00:33:00Yeah, that's what we're saying. You'll connect your data dump into an AI thing and say,

00:33:04here's our defense, find me the documents. Right, so the big challenge is

00:33:09the Janks Act, right? So Janks material. When Janks was passed, they weren't thinking about, well,

00:33:19you're going to have to wait until this government witness testifies, until we're then going to produce

00:33:26to you a terabyte worth of data, and you can go through and then figure out what's in

00:33:32material. Like, it's just, it's impractical, right? So that's really the big, the issue is dealing with

00:33:39the data dump. Now, I think AI is going to be extremely powerful in being able to deal with

00:33:44that because there are inputs and there's functions that would be able to analyze context,

00:33:51which is so difficult, especially when you're under the gun on time.

00:33:55So, Counsel Gelb, tell the audience, not everybody knows what AI does. How does AI,

00:34:03what is it about AI that helps it go through a terabyte of data in under a day in your scenario?

00:34:10For example, if you were to have a program to be able to analyze, what is, for example, most

00:34:26detrimental, something that's in context, right? Like, what is most, for example,

00:34:35consistent between these individual statements that were made to these agents between this time

00:34:42period, you know, something along those lines. It builds a language construct.

00:34:50It takes pieces of language that are in two separate documents, it connects them together.

00:34:55It creates an entity extraction analysis, understands people, places time and events.

00:35:00And it can overlay other data, right? That's why it's so interesting, right? You give it

00:35:04the terabyte of data, it'll connect your entire set of facts for your arguments and cases,

00:35:10put together the context, and then you can query the system saying,

00:35:13please identify conversations between this agent, these agent interviews and these parties at these

00:35:19times. And then they'll query it and they'll find things that you're not even thinking about,

00:35:23like, and they look at the phone call logs of X, Y, and Z and say, well, we also identified this

00:35:29anomaly in X, this is impossible or whatever it is, right? But it'll be trained using, but,

00:35:35you know, you'll have to train it to counsel regards point, you'll have to have some training

00:35:40of it, right? So that it knows, you know, not right from wrong, but left, you'll need to know

00:35:46left from right. So it has to know your case specific rules of physics, you know, this can

00:35:51happen, that can't happen. Right, exactly. How does witnesses A statement conflict with

00:35:58witnesses B statement? Yeah, but I'll read all of it and tell you why it conflicts, how it conflicts,

00:36:04and give you their history and everything else. And then it may also look at other things that

00:36:09you haven't thought of, because depending on how you've trained the algorithm, right, it may actually

00:36:14analyze the data looking for discrepancies across the macro level, right? So, but the cool part

00:36:20is that you'll be able to take that terabyte, call it 400 gigs, just because OneDrive is one terabyte,

00:36:26and you'll be able to bring that into OneDrive and use, I don't want to say a rough shot,

00:36:32but like you'll be able to use a widely commercially available AI agent co-pilot on OneDrive,

00:36:39I assume, at some point in the near future to say, query it to find, assuming it can read all

00:36:44the documents, so again, counsel regards point, that does need to be able to read the data

00:36:48formats. But assuming that they're producing it to you as a load file, right, they're going to give

00:36:52you the text and the data files, hopefully, then it would just query the systems and you would then,

00:36:59you know, hopefully have trained it on your other case materials and it would find it.

00:37:02I mean, that will be the jerry-rigged version of what will, I envision probably 95% of the

00:37:10legal marketplace will use rather than buying more sophisticated tools, but that will be the

00:37:14reality, I think, in a lot of ways, right? And then you can say, write the motion, right? That's

00:37:20the crazy part. You can then tell it to write the motion and it'll write the first draft of the

00:37:24motion with the evidence that you just got and it'll give it to you in, you know, 10 hours. I

00:37:28don't know, if you're putting on your home computer, maybe it'll take a couple days, but,

00:37:32you know, if you're using some cloud-based Microsoft solution and you have a fast pipe,

00:37:37probably getting two, three hours. I mean, you need a gigabyte pipe, but anyways.

00:37:43Your bottleneck is how fast can you load the data into the processing ingestion engine?

00:37:48Well, that's the thing. OneDrive would have it built in, right? It would have its co-pilot

00:37:51built into the engine. It's about getting it into OneDrive and then OneDrive querying the total,

00:37:57the total amount of information. It won't be like fantastic, but it'll be

00:38:02much more efficient than you reading one 400 gigs of data and trying to turn that around. So.

00:38:09Even if it would tell me, even if the system wouldn't write the final answer, if it would

00:38:16merely say these are the documents you should focus on and here's an outline that I would

00:38:21recommend, but you should read the documents and verify it, that's already a huge leg up.

00:38:27Oh, we're using that now in my organization. Exactly.

00:38:30I mean, we're already there. We just, we run our own, you know, we took our own large

00:38:35language model and built it out. But yeah, I mean, yeah, that, that, yeah, that, that's

00:38:42there, right? I mean, cause you don't even have to train it at that point, right? You,

00:38:46you just need to give it context because. The point is not using AI, any algorithm,

00:38:54to make a final decision. Everyone would agree we're not there yet and we may never be there

00:39:00for certain issues. So I like to use the phrase, don't use AI for adjudication. Use AI for

00:39:06navigation. Yeah, but that's what you use it for. It's like navigation. Yeah. I don't know.

00:39:13Google maps, man. Like, like, like you use it to get from A to B, but you're driving

00:39:19the car and know what's north and south. Well, I'll give you this thought. Maybe we

00:39:23could have started with this, but 10 years ago when I was driving to my in-laws house

00:39:28and my little girls in the back seat, we have the iPhone mounted on the dashboard and she's

00:39:34watching the map of the car driving on the map. You know, what's that blue line? That's where

00:39:39we're driving. What's the arrow? That's the car. What's that lock at the bottom? That's

00:39:44when we're going to get to grandmother's house. There was a pause. And then from the mouths

00:39:49of babes, right? She says that thing knows the future. So yes, my navigation tool was a

00:39:57fortune teller, right? And that was AI then. Yeah, I mean, it's way more advanced 100% and

00:40:05people don't realize that, right? They don't understand, you know, that, you know, that,

00:40:11you know, and it's, you know, and when you look at audio files, I've had cases recently where we're

00:40:16trying to ingest lots and lots of audio files and you can now the audio files can be actually

00:40:24automatically transcribed. It's about 90, 80, whatever, but you can then, but you can actually

00:40:30train to look for intonation now and all these other inflection and tone and voice. And we don't

00:40:34use any of that, but like AI also will make process because handling audio files and discovery

00:40:41and video files is actually one of the most expensive parts of the process because there's

00:40:47no efficient way to search and query massive audio and video libraries today. And it's also very

00:40:53impactful in front of the judge and jury. Yeah, when they bring it up, your deposition testimony

00:40:58and they're like real time, here's what Mr. Gary testified to. That just happened to me.

00:41:03It's impactful. I mean, you know, it means a lot, right? I mean, a video of you testifying brought

00:41:10up during cross can carry, can go as a strong message, right? As long as you're dressed well

00:41:15and sound credible, but otherwise, you know, but it's credible and juries pay attention.

00:41:20I agree. Look, I will say this. We're almost at the end of the hour. It just dealing with these

00:41:29technologies, watching their impact on large volumes of data has been impressive. Like you,

00:41:35I grew up as a computer scientist. I wrote AI systems and watching them now at this scale

00:41:42still impresses me. The faster we get our heads around the ways to apply this to practically

00:41:50solve our problems, the better off we all will be in terms of resolving disputes faster and cheaper.

00:41:56I think, and I, you know, just to build on that, right, is that I also think you'll increase the

Automatically generated; it may contain errors.

Lesson notes

New work technologies that have added complexity to the eDiscovery process.

Study materials

Notes, flashcards and practice questions for this lesson, reviewed before publication.

Study notes

The growing variety of discoverable data sources

Discovery now reaches far beyond email. Panelists described discovery from vehicles and medical devices, where few forensic tools can collect data defensibly, as well as Slack channels, messaging apps and mobile devices. Even phones vary: not every iPhone or Android device collects the same way.

Volume, velocity and variety are the classic features of big data, and variety is the pressing problem. With so many systems, counsel needs a method to identify, qualify and prioritize sources. There is no one-size-fits-all approach:

  • A bet-the-company antitrust or mass tort case may justify searching many sources.
  • A simple slip-and-fall case may not, even if relevant content exists somewhere.

Lawyers should be strategic about what they need to prove, and litigate the case rather than the discovery.

Human-triggered, machine-recorded data

One panelist reframed much of today's ESI as data that is computer-entered but human-triggered: GPS records exist because someone drove the car, and appliance logs exist because someone opened the refrigerator. That data is often more objective, more reliable and more voluminous than what people write, and it fills gaps in factual timelines. Examples included vehicle sensors showing how much weight sat in a seat and for how long, and data showing where a robot vacuum traveled.

Some matters are now litigated with no email collection at all, relying on contextual metadata, because clients would rather know what people did than what they wrote about what they did. Device and account data can also test claims such as "that wasn't me on my phone."

How the rules adapted to technology-assisted review

The rules of civil procedure were drafted when lawyers could have near certainty about what was reviewed and produced, because teams of associates could read every page. Practice moved from page-by-page review to review platforms and keyword searches. More recently, rules and practice have accepted that exhaustive manual review is humanly impossible.

Technology-assisted review (TAR), in which a system is trained to find responsive documents, became a best practice because it has been shown statistically to be more efficient than human review. Panelists noted the tension this creates: a party answers interrogatories under penalty of perjury, yet document production under Rule 34 relies on technology to produce an accurate set. Adopting TAR took time institutionally, and the same adjustment is now happening with broader AI.

Limits of AI and the need for quality control

  • New data types. AI models trained on existing systems may not handle emerging ones; a model that did not know Slack could not process it like iMessage. This creates a constant whack-a-mole problem, and experts must establish reliability for new technologies under the Daubert standard.
  • Siloed systems. The hard problem is often not searching each system but integrating isolated systems, a federated search problem. Researchers are building AI agents that create connectors to query disparate legacy and modern systems.
  • Controls. AI collection agents may soon find information across an enterprise, but controls must confirm they did not skip a server. Like a surgeon supervising residents, a qualified person must remain accountable, and the adversarial process will push parties to prove their methods.
  • Augment, don't replace. Practitioners succeeding with AI use it to augment human judgment, not replace it. As one panelist put it, use AI for navigation, not adjudication.
  • Ephemeral messaging. Self-destructing messages are rarely recoverable from modern phones unless someone captured a screenshot.

Using AI to manage data dumps in criminal and civil cases

Criminal defendants often receive enormous volumes of discovery at once, and the timing of Brady and Jencks Act disclosures was not designed for terabytes of data produced when a government witness testifies. AI can help by building language models of the material, extracting entities (people, places, times and events), connecting related passages across documents, overlaying data such as call logs, and flagging inconsistencies between witness statements. It must be given case context, the case-specific rules of what can and cannot happen.

Similar tools help in civil matters, where parties often spend heavily to produce discovery without learning from what they collected. Panelists expected widely available AI assistants built into enterprise platforms to let lawyers query produced load files, identify the documents to focus on, outline arguments and even draft a first version of a motion, with the lawyer verifying the results. Audio and video, among the most expensive data to review, are becoming searchable through automated transcription and indexing, and video evidence can be powerful before a judge or jury.

Flashcards

13 cards. Test your recall, then flip to check.

Select the card, or press Space, to flip it. Left and right arrow keys move between cards.

1 of 13

Practice quiz

Practice — not graded, not the final exam

Question 1 of 7In a straightforward slip-and-fall case, opposing counsel demands collection from every device and messaging app the defendant uses. What principle did the panel emphasize?
Choose an answer to check it.