Future-Proofing Legal Workflows with Artificial Intelligence

Oh, thank you so much, Daniel. Really appreciate it.

Yeah. It's a it's a great question. Certainly, it can feel sometimes that you're on a hockey stick or sort of like an exponential takeoff. And certainly, the last couple of years have certainly been very transformational.

But as you said, I've been working in this field for a very long time, and I started as actually a computer engineer. So my first four years in undergrad, I built simulations. I worked on this you'll appreciate this. Configurational bias Monte Carlo simulations for lipid bilayers.

Now, it turns out that simulations of how our bodies interact require tremendous computational power. And some of the very first early uses of GPUs for things besides gaming were actually in simulating molecules and molecule interactions in our bodies and other places. So I worked on some of that stuff very early. So there's been a lot of interactions in this space for a very long time.

So after I did my undergraduate degree, I did a bit of a pivot. I went into applied mathematics and computational science.

And the two sort of major projects I worked on in grad school were in simulating telescope mirrors that are deflecting to sort of make light's way through the Earth's atmosphere, so large ground based telescopes. We're working on a thirty meter telescope in Hawaii, actually, on the island that I was born. And I worked on the control systems that are being built for that telescope. After my telescope endeavors, I actually went into semiconductor lithography, which is chip manufacturing.

I worked on technologies that improve the design of semiconductors, including the phones that some of you might have in your pockets. And that sort of, in turn, really got me into the center of what's we were using a gradient descent algorithm. These are some of the fundamental building blocks that people use in modern AI systems. And I was using supercomputers.

And a lot of the lessons learned, a lot of the technologies, a lot of the algorithms, all of those things have been carrying forward with us for a decade.

And I think here's a fun fact I think a lot of people don't sort of kind of always keep in their heads. We think about the evolution of algorithms, and we think about the evolutions of computers. What most people don't realize is that they coevolve.

So when we design mathematical algorithms as mathematicians and computational scientists, we design them to take advantage of computer hardware.

Simultaneously, we complain to the computer developers that we don't have enough hardware or the system's not parallel enough or we don't have the right level of concurrency on a particular system. And over the last fifty years, many of the design and development you've seen in algorithms and in software and hardware have actually sort of happened together. And the design of GPUs, which power all modern LLMs today outside of tensor processing units and other things from specialized hardware, those co evolved in a sense. NVIDIA looked at what was needed to make large language models work very, very fast.

And at the same time, the transformer architecture is a way to make large language models work on GPUs in massively parallel architecture. And so there's this sort of beautiful sort of coevolution of technology and algorithms that I think most people sort of take for granted. I sort of got to ride that wave for some time. So I was in grad school.

I worked at IBM. I did a postdoc. I lived abroad for a few years. I continued to work in this sort of like supercomputing, large scale analysis place.

Came back to the US after a year in the UK, and I started working on, let's say, more mathematical army defense use cases.

So coastal defense, coastal modeling, storm surge, things along those lines. We're going all over the place with my history. But eventually, I found my way into work with DARPA and co investigations with groups like the FBI, with the Department of Defense, with the SEC, looking into criminal behavior, wrongdoing. Silk Road was one of the things that we sort of kind of helped take apart and looked at. And I think that's where I really sort of caught the bug investigation and sort of bringing large scale data and finding what was in it and taking it and using it for good, hopefully.

Yeah. I think we're adopting this technology at an incredible pace. So I take a look at how many humans are reviewing documents in Relativity and how many documents are being reviewed by machines in Relativity. And the machines are drastically speeding up.

And so I mean, taking your question sort of on the face of it, like, you know, what could make adoption go faster? Are we gonna hit some limits? I think first and foremost, the real challenge I don't know if you've read, there's a really great paper called AI is Normal Technology by a professor and his graduate student at Princeton University. And they talk about how technology has both sort of these phases of development and implementation, but then there's this sort of long phase of sort of socialization and that's when you hit all sorts of things. You hit legal rules, you hit precedent, you hit organizational transformation.

And so we're in that phase, right?

And now it's a question of adoption and pace of adoption at the organizational level, not, let me say, at the, say, proof of value, but more about how you get to value.

I think that being able to add information and context to information that we have doesn't have like a good or bad assigned to it. It's how we use it. Generally, I think that we're gonna find positive uses for the massive amount of information that we're collecting and creating. Better classifying it, better organizing it, better being able to understand.

For example, if I have a algorithm that can identify what the pictures of my son laughing are, I wanna know every single one of those. And so from a personal level, I take advantage of advances in AI technology. Coming back to some of the framing though of, hey, we have these pockets of information. Discovery is a particular place and time in which we're trying to answer questions, are there other times in which we want to try and answer questions about our data?

I believe that's a hundred percent resounding yes. What I will sort of put as a sort of cautionary lens or at least sort of a mitigating lens to that framing is that data classification is always done within a context of a question you're trying to answer or a problem you're trying to solve. In discovery, usually the problem is pretty well defined.

From a corporate governance perspective, sometimes there are questions that you sort of know you're always asking and you want to sort of refresh those questions and make sure you understand them on some cadence. But what will happen frequently is the number of questions that you try to answer overwhelms sort of your ability to effectively kind of answer those questions over the volume of data you're creating. There will be limits to how much you can, let's say, forward govern and there will always be sort of question answering at the time of discovery because sometimes information doesn't arrive that you need to know to answer a question until you're in discovery.

ChatGPT, when it was released, created a massive adoption wave of technology outside of legal technology. It was sort of this new thing. People sort of started interacting with it in all sorts of ways that were both expected and unexpected. It's a very useful technology.

It is, on the face of it, something that maps very well to the way that we like to communicate and the way that we like to think, and that's a very powerful thing. It's a very powerful thing in motivating adoption. If you look at how TAR in comparison works, you have to explain to people that, well, it's a statistical model over the word counts that are happening and words that are more frequent are going to show up in these documents. Then hold on.

We got a histogram that's going to show you all of this. And so there's two things at play. One is that TAR required a lot of technical explanation. The second thing is that anybody who understood how TAR was validated looked at how generative AI worked at a document by a document level and said, I can use the same techniques that I studied hard.

And it took me six months to learn and explain to all of these customers, I can use those exact same techniques in generative AI. And that is a massive boon to adoption.

And it's in the same way that people learned that TAR didn't always get every single document right. It's the exact same thing with generative AI. And actually, some of the same things that can cause you to mislabel documents in TAR can actually cause you to mislabel documents in generative AI. If you don't know that there is an issue that looks just like yours in the case and that it can distract from the truly responsive documents, In tar, you sort of find that out as you're starting to label documents.

In the same way with generative AI, you can write an initial prompt and after you've run it through your first iteration of documents, you might realize that actually, oh, no, There's two things here, and I need to make sure I adjust my prompt.

Yeah. I mean, I certainly think that one facet of it is that it's easier to, in some sense, explain to users, even though large language models are extremely complicated things and how they were built is very complicated, their architectures, etcetera, the idea of how they're used is easier. The idea of how they're validated is known.

And I think as you sort of said, so you can explain how they work and it's a little simpler than machine learning. You have the advantage of validation already being a known thing. And then I think this third facet is that they're just less effort.

So if I want to use TAR, I have some amount of time I have to spend either training the model or operating and reviewing documents in a way where it's not sort of providing a lot of value to me yet. Whereas with generative AI, I'm sort of almost immediately capturing the value of the model. I can review that to make sure it's looking for what I want, and then I'm off to the races with my documents. So that it just that sort of simplicity and that feeling of control and understanding For some of our users, it's sort of, I don't want them to think it's magic, but it feels magical.

And that's important. Right? So you want them to feel magical, but then also rigorously validate. When I can find users who have both of those things, those are my absolute favorite users.

So I recently interviewed a candidate for a role on not on my team, but basically appeared to me. And he's a director of engineering based in Poland.

And so there's really kind of two things in here. One of these is you have to use the AI technologies on a daily basis. You have to use them on a daily basis. And I encourage my teammates.

I encourage my peers. I encourage my colleagues. I encourage our customers to just constantly be playing with it. Find what you like about it.

Find what's interesting to you. Do you like generating art? Do you like writing with it? Do you like learning?

All of these things are ways that you can engage with it. Do you like programming? And so those are all things that are part of it, but there's another part too and that's the filter.

The single most important thing you have is you have to be able to filter out all of the noise and all of the hype so that you can get to great results. And so for me, I found that following maybe fifty to one hundred people who are sort of fairly prolific, who amplify great results that are important and have impact is to me the way that I find out most of my information. And of course, I also have an applied science team and they love telling me about all the things I don't know, and so I spend a lot of time listening to them as well. And so between them, I tend to feel like, again, not everybody knows everything that's going on in the industry or all the applications and what's going well and what's not, but having a diverse professional network that I communicate with.

My social medium is LinkedIn. That's where I actually keep track of a lot of the research developments, actually, I find out through LinkedIn.

I have to admit that I've got a four year old and a seven year old. So most of the way that we talk about the responsible use of AI is trying to stump it and having it, you know, think of an animal and then have the kid try to guess the animal. We haven't really gotten so much further than that. Going in into sort of middle school, high school, college age kids, it's the same level of force the engagement and use it as much as you can and it's just like any other tool that you need to be great at.

So in high school, I had to learn how to use a graphing calculator because that was an important way to learn trigonometry and eventually get ready for calculus. Well, if you are studying almost anything, a generative AI tool and again, for example, like OpenAI comes with a study mode and there are other tools that are specifically designed to help you learn didactically as opposed to or pedagogically instead of sort of just answering questions for you. But there's a book by a man named Neal Stephenson called The Diamond Age. Don't know if you've read this book.

But Diamond Age is this sort of book that sort of imagines a world where we had a technology like large language models and it sort of becomes this primer that gets handed to the wealthiest and most privileged of children. And then of course the story is it gets to somebody who isn't wealthy and privileged and what comes out of it. And I've always just been delighted by this idea. I've been sort of inspired by, well, what if we could give these resources to everybody who wants to learn a topic?

Or so coming back to, like, what is the lesson? The first lesson is engage with it. And then the second lesson is, you know, of all of the things that people can interact with, social media, like, these other sort of things, like, you know, that they're encountering in their daily lives, this is quite possibly a great force for good. Now, I don't recommend interacting with certain large companies chat systems because they're not responsibly developed, but there are chat systems overall and there are certain companies.

I'll I'll name Anthropic as a positive example of a company that really focuses on trying to build AI responsibly and deploy it responsibly. And so these are systems that I think you can, again, not fully trust. Right? There's a trust but verify piece of it.

But trust your kids to engage with.

That's right. Four wonderful years of Relativity.

So I hope this is as not a disappointing answer, but I am not interested in the present. I just want to understand where we're going in the future. So I want to understand what are the future pains? What are we not solving yet?

Where are we going? What's coming? What are the things that six months from now, two years from now, we're all going to be working on? And so I'm I'm really just have a huge eye towards what are the big problems of twenty twenty six and twenty twenty seven that we can solve next.

Oh, it's so great to be here. Thank you so much.