The AI Revolution in eDiscovery

Thank you very much for that grace, And, happy to be here, happy to talk with you guys. So let's have some fun and talk legal tech.

I know for a fact, actually, we got a lot of things right. One of the things you were alluding to that you said here very well is that we were looking at how do we handle and help you deal with your entire legal portfolio, not just a project. Right? And, you know, that was beneficial to, you know, both us as a company as well as the end client in a ton of ways.

Right? It definitely built some predictability into it, the spend, the resources, and the like. You know, nobody, like, back then, and I mean, still now to a degree, legal spend is looked at as a cost, a pure cost, not an investment into the business, a necessary cost nonetheless. But it's like, how do we control that?

How do we put our arms around that? And we gave businesses that ability. Right? And so that definitely resonated with client base.

I know for a fact also the people that company kinda saw where the world was going with cloud computing.

You know, a lot of our customer base were were law firms, and they had a way of doing things.

You know, sometimes they were very beneficial where if they've already moved to a cloud, whether it be public or private, they understood the benefit of doing that. Some were very resistant to let go of the data because they had only historically held on to things. So, you know, we were able to make very compelling cases in both instances to say that, okay, here's a better way of doing this. You know, especially with, some of law firm clients.

Right? I mean, because you had finite or they had, rather, say, resources. Right? They had a SQL guy for the firm or two SQL people for the firm as opposed to us at that company having multiple SQL developers and admins and the like for all of our Relativity instances.

We eliminated the conflict at certain places between, say, a human and or technical resource working on does outlook go down or does relative to go down? Which one do you use first? Where do you fix things? Right? So we filled a lot of gaps for a lot of people, and we see that a lot of the service delivery models that exist today.

So I would say very early on, the beginning of proselytizing and going to market with managed services, we learned to listen to the customer. Right? Some very smart people all got in the room, came up with, here's how we're gonna deliver eDiscovery managed services, and here's what everybody needs. And we went to the market, and there were a lot of people who did need that, But there were a ton of people who needed different things, whether it was different technologies, whether they had contractual obligations to certain things, whether they just ran their business differently than, you know, our core base.

So, you know, obviously, with just about anything, you can't be so bespoke that you're reinventing the wheel every time. But we learned how to get the wheel to a point where it was the customizations provide a minimal disruption or, like, cost spikes and the like, unless there were some just extreme outlier. So, you know, understanding the problem that each individual entity was trying to solve for, right, as opposed to just thinking, hey. I have got the solution.

Now what's your problem? Right? Or tell me about your problem that I've already solved. Sometimes it worked that way and sometimes it didn't.

So that was a huge part, especially the very early days. As the offering matured, at times got easy to kinda look internally. Right? I mean, we're we're successful.

We're doing this. We're doing everything we said we did or what have you. But continuing to evaluate, you know, different things in the market, whether it be tools or trends or, you know, thought leaders. If you're paying attention to that stuff and really getting to the signal of it all was very instrumental and very relevant to how we continue to deliver those services.

A lot of that is the culture of the organization. Right? And, you know, maintenance mode, while not where I wanna reside, you know, for any length of time, it's a marker of success. Right?

You built the thing that you sought to build, and it's working for that entity, that individual, that group, whatever, you know, the basis. But you get to make this road. You love that. You're happy for that.

But what's the next generation? Right? And there's always a next generation. So whether it's just, you know, the regulatory environment, the world at large, you know, that particular industry that you're servicing or the ancillary industries that go along with that.

What's happening in those places that you need to account for, right, that you need to look forward and say that, hey, this thing's coming up or these three things could come up. How do I adapt to a, for just the people who are in maintenance mode, and what's my next message to the market as the market shifts? Because, you know, what's gonna be prescient in three years, you know, may not be prescient today for most people. But, you know, how do I prepare myself for that?

How am I not being dragged by the tail on that? So it's, you know, it's really important to, you know, understand your DNA, who you are, and who you wanna be as a company, as a provider, as a entity, or what have you, and make sure that, you know, you're always looking at that, okay. I got where I got largely because I solved the problem. How do I solve the next problem?

Healthy combination of the two. Right? So there's definitely the life cycle. There's the get the deliverable to the client.

Right? They asked for this, they came to us, they asked for it, let's get it to them in the time that we said we would give it to them. Then, as I was saying about builder mode, you know, we'll look at, okay, what could we have done better? What problems do we foresee or other issues we foresee where we can adapt the product now or in the next version of it to what we're doing?

And then you have the market. Right? You know, more people are gonna want it or need it. They're gonna need it better.

They're need it just a little bit different or what have you. So you start tweaking that baseline product, that MVP into, you know, however you can scale it.

Sure thing. So it means a number of things, but at its core, it's again, like I said, how the world changes and the world adapts. And here we go. Here's this thing that the world's doing in a certain number of ways and how the world's working.

How do we use this tool that's been given to us? How do we use the fire to heat our homes, cook our food, and do all the other things that we need fire for? Right? So that's really the thirty thousand foot view of how we look at that.

Then it gets a little more bespoke. So you mentioned some of the products that we've developed. Right? Those products solve for different problems that arise, whether it's internal to the firm, whether it's the firm's client, you know, whoever else in the market might need some of these things.

So there is a very constant need to look at, okay, how do we tweak this thing? How do we harness this fire? How do we, you know, just just make sure, you know, we're not burning ourselves or anybody else? So that life cycle, it's always, you know, kinda build, evaluate, deliver, repeat, tweak, update.

We've done that in a great number of ways and sometimes it's, very simple where, you know, we put a wrapper on some of those large language models that you talked about and, you know, we beefed up the security and the likes, so we're making sure that certain data is not making its way out into the world. Some other things become, like, a lot more bespoke. Right? You talked about the pilot, for example.

Yeah. That started out with a client in a certain industry. Couple other clients in that same industry felt the desire to use it and do some things a little bit differently than the initial use case. We were able to kind of tweak on the fly and get it to work for them.

And then we were able to pitch it to some other industries all altogether who share some of that regulatory burden and share some of those sort of use very problems. But, again, different industries. So where, how, what data is being procured or processed or what have you changed a little bit. We've had the framework for that already and we're able to tweak that.

Then we were able to take it from shield to spear. Right? So we started investigating some other industries where, hey, we have an inkling at least, a little more than an inkling, but we've got an inkling that this could be beneficial to you guys and here's how.

So, you know, that gets you the entrance a lot of times, but then that's when the listening really starts and how we solve for that and the products have progressed.

The fundamental component of being a human being and being an active listener is never to be overlooked. You're often described as kind of a legal engineer of sorts who designs these complicated technical workflows in the context of legal work. But let's talk a little bit more about when you think about architecting a workflow in the context of using AI for an investigation. Daniel's gonna love this question, but how do you design the human in the loop characteristic and checkpoints so that you can move quickly, but you can also have that kind of look back and audit trail in order to defend each step to make sure that the AI is not just kind of running free on its own.

So we try to keep the human at the center of the loop. You know, it's our core belief that, you know, AI is not gonna go full Terminator or Skynet. It's people with AI. Those are the people that are anything.

So, I mean, obviously, there'll be less humans in some cases that need things, but a human needs to be at the center of that. There are some things that are just very cut and dry. Right? Like, did country x update their regulations with these five documents?

Yes, no. Right? Very quick, very easy decision tree. But now what do you do with those documents?

How do you integrate those documents into whatever you've been doing for that particular industry or that particular client, what have you? Are we sure that it's right? I mean, you mentioned all the popular large language models are out there. Who you know, you've gotta click a box first.

Say, hey, this thing hallucinates or this thing may not be perfect. It may not have access, quite frankly, to everything you needed to have access to. So, you know, while the AI tool is very powerful, very fast, you know, very, very quickly, somebody has to check that. And unfortunately, we've seen the news stories where, you know, with matters cases being cited that were cases in movies and not real cases or what have you.

Some somebody's gotta look over that stuff. Somebody's gotta realize that when something fails for whatever reason, right, whether it's somebody using combative AI in some ways or obfuscating things from, you know, how you've been looking for them. It is paramount for a human to really be at the center of that to ensure that the deliverable is correct. These models are gonna get more powerful, obviously, and, you know, there's certainly an argument to be made that there will be even further reduction in the need for immunity.

But I don't see elimination by any stretch of the imagination. And that goes for things that we're doing, you know, inside of eDiscovery, outside of eDiscovery, and the like. You know, inside discovery, one of the arguments against has always been, oh, well, I know my case better than any machine could know my case. And it's like, well, you know your case.

Do you know your data? How quickly can you parse through the data and the like? So, again, you need a subject matter expert really at the center of that to ensure that the machines are doing what you told them to do.

So we've already seen with some of the technology you mentioned, the TAR or or just even databases. Right? I mean, you wanna go back to, like, concordance databases.

That replaced case rooms that had thousands of boxes of paper in them. And those boxes oftentimes were looking at copies of things, and you had team a working on this, team b working on that, or what have you. So now you've got this database, and not only do you everybody not have to be in that room in particular, they don't have to be in a city to get to those same documents. And often have they can get through them more quickly, even if they're just going page by page.

Right? They're able to share work product better. So we're seeing that in so many different aspects and areas of life that you said much faster. You know, somebody we're able to get to the core of things.

We're able to answer the questions a lot faster. We're able to parse through more data. Right? We're creating more data than we used to.

I think you guys are both well, I'm probably the elder statesman here. I think you guys can appreciate, like, what a actual work computer itself. Right? That used to be someone who looked at a spreadsheet and computed the pluses and minuses.

We have computers now that do that as fast, if not way faster, than any computer in that era did. Right? So that's always been how technology has worked in our civilization. You know, we don't have fifty people try writing and copying books.

You know, the printing press came along and now, you know, large scale printers have even blown that away. So all that's for me to say that the next evolution, I do believe, is going to be an elimination of the billable hour. And it's not just a technology challenge that's driving that. How that's gone, you know, a lot of corporations look at, you know, how their balance sheets ebbs and flows and how they value projects.

And it's not based on an hour of time. It's based on getting the work done, and it's based on outcomes and the like. So the outcomes and the like. So the legal industry, which had been a bit of an outlier, think the technology is bringing that kind of closer to center on how, like, the corporate world and how the kind of the world at large works.

So that's where, you know, we talked about at the top of this, listening to the client. Right? Understanding what problem they're trying to solve. Obviously, it takes a little bit of prognostication to say that, okay.

Today, this tool can solve that problem, but here's how your problem's probably gonna morph, and here's how this tool may or may not morph. That's a big part of it. It's really understanding or getting that the true comprehension of, yeah, what they need to do or want to do. I would say next is a very strong governance model.

Right? You have to have some guardrails around how and why we're gonna do this, how and why we're gonna add new tools. Because you're right. I mean, the tools themselves change every day.

There are people who come to market every day, whether it is repackaging things or whether it's just getting faster to create some new things. So what's your model? How do you vet tools? How do you vet the security, the safety, accuracy of those things?

You absolutely need to know that. And then you have to have a risk tolerance that's greater than a lot of people in the legal space has had. Maybe it's not quite as drastic as, say, the software or banking industries go, but there are gonna be some stumbles. There are be some shortfalls.

There are gonna be some drawbacks. And, you know, in the legal space, you know, we've been taught that those are all wrong. Those are all bad things, but business doesn't move like that. So you have to have some patience that, you know, these things are going to happen.

Obviously, you mitigate them as much as you possibly can, but to think that, you know, you're gonna have a hundred percent accuracy, you're never gonna get to a product. Right? Nobody develops anything that's gonna be a hundred percent on day one.

So what is it that you can tolerate is something that you need to know. But how do you want this to work? Is it a suite of things? Are you looking for one thing that's gonna do everything, everything well, everything kinda well?

Do you want a million different bespoke products and the like? Do you wanna have a hundred tools in your drawer at all times? Right? So I try to understand those things and talk people through what I feel are some best practices for those, what I think, you know, based on my experience with them, whether it's longer or shorter, how the things in the market kinda work and fit what they are currently doing and what they wanna do going forward.

Yeah. I think it definitely starts with governance and risk tolerance. Right?

Do you touch on some of the kind of competing interest? You know, efficiency isn't always a positive word, you know, in certain businesses or certain business models and the like. So you definitely have those competing interest at play.

It's really navigating, you know, how much do you wanna solve this problem, how much do you wanna solve it quickly, and what are you willing to do and tolerate to get to that point. You know, I used see that all the time, right, where you had conflicting stakeholders that, you know, the privacy guy, the security guy, the infrastructure guy, they were all on the same team, but they were all trying to accomplish different things. They they all had different incentives and all. So, you know, we still see a great amount of that, you know.

So it's really you know, it can be quite tricky, quite frankly. Right? Getting those very interest on the same page, getting them to speak where we read for the same thing, hymnal, so to speak. So, you know, that in of itself is another reason, you know, going back to the question about human in the loop.

Absolutely. So that's really it's the same sort of mindset, the same sort of acting path.

Hey. Thank you guys for having me. I enjoy what you're doing, this podcast, how it's serving the community. Gold, you know, you've always been one of my favorite people, so I was happy and honored to get the invite.