From CIA Data Science to AI Law

Thank you so much, Daniel and Eric. Really, it's a delight to be here.

Yeah. I will be the first to admit that I have a very nontraditional path to where I have come to and really kind of been pushing the envelope my entire career, which is a fun place to be, but kind of a, you know, dangerous place to be at times as well.

I became a data scientist quite by accident. When I was at university, I had put together my own degree where I was studying turning points in history. So I was always been driven by this keen curiosity about why do societies set themselves up the way they do?

How do they respond to ecological or economic pressures? So studied history, philosophy, economics, anthropology, theology, all the ologies, right, that make you no money at all in the world.

But really fascinating, especially focusing on technological revolutions and how those affected the last seven thousand years of history. And I came out of class one day in my senior year and got approached by the agency to come work in a new data analytics shop that they were setting up in the director of intelligence.

And I was like, I'm not sure you have the right guy. Like, I don't really like math. I really hate statistics. I failed calculus. They're like, yeah. We know all that. That's not why we want you.

And so they said, look. So if you remember that this is nineteen ninety two. So remember the early nineties, what was happening technologically is you had the World Wide Web and email was just coming out of academia. Cell phones were just coming on the market. And so we were, as a society, creating this new digital contrail that was very personalized.

And the agency rightfully saw that that contrail would just become more rich and more voluminous as time went on. And so what my job was, what can we learn about people or groups of people based on that digital control? And so over the next eight years, working with some of the best data scientists in the world, what can you learn from those about people? And so who's a good guy, who's a bad guy, whatever that meant that particular, you know, day or week.

But can you predict their behavior? Can you influence their behavior? Every social media company, every app, every cookie on your like, everybody does that today. But at the time, it was pretty cool science.

And so got into data science really accidentally by studying all these things that had nothing to do with data.

I'd always wanted to go to law school. That was my intent. And so in two thousand, I went to Georgetown Law, and then I went on to get a graduate degree in data analytics at New York University.

And so I spent my entire career in BigLaw in one of the top ten firms in the world and have really focused on so the how do you get insight out of massive quantities of data?

The interesting so I graduated from Georgetown right when kind of the eDiscovery revolution started. Remember back to two thousand five, this is when the federal rules of civil procedure, they redefine documents to include electronically stored information, and every company in the world had a heart attack over that. Right? How in the world do we find all documents responsive to anything? Systems were not searchable. You could not do you know? So I was put in charge of, like, literally the second week.

One of these firms trying to figure out this electronic discovery problem. And so brought a lot of data classification algorithms into that process. Right? Because that's what I had worked on a lot at the agency is, hey. I have this thing. Go find more things like this.

And so worked with a lot of the beginning eDiscovery companies and the Sedona conference, things like that. Right? Like, to bring order and process to really horrible process.

So I spent about ten years doing that. Right? And so in twenty fifteen, the federal rules were amended again that to basically chill out. So a lot of the you know, all the big sanctions, all the big things have been taken off the table. And really in twenty fifteen, the eDiscovery law kinda settled, and it's remained settled. And now it's about to blow up again because of AI. But Yeah.

So at that time, I had constantly been pushed upstream in the creation of data.

Right? Because you learn that when you're trying to search for document, right, that if a company has their data organized well, it's easier to do that. Right? So that kinda pushed me into information governance, and that just kept pushing me upstream.

So where, you know, companies that their data is one of the most valuable assets that they have. Right? And their acumen, their decisions, their work product, it's all locked in their data, and that data basically lies around and does nothing. Many studies have shown after ninety days, ninety percent of the data is never touched again.

And after a year, it's closer to, like, you know, ninety nine percent of data is never touched again. So I moved my practice almost entirely into automated decision making systems. So the algorithms that run our lives. So everybody is like, the algorithm, it runs our life.

Right? Well, it does. They do.

And so think when you apply for credit card or a mortgage or a loan, there's an algorithm that basically decides how risky you are and how much I'm gonna charge you for that credit or for insurance. You see it in labor and employment. So now somebody posts a job, ten thousand resumes come in, some kind of AI agent now that's still highly algorithmically based is sorting through those resumes to decide who gets an interview. And so my job was helping companies design those algorithms so that they were accurate, but mainly that they were legal.

You know, like, in labor employment, if you only hire all white men, then you've got a problem. Right? And so during my career, I've been able to represent pretty much all the big tech companies, the vast majority of the Fortune one hundred, because this problem with data, with the discovery, with algorithmic decision making is across all industries. And so when GenAI came out, I was a partner chief data scientist and global AI chair for one of the top AmLaw firms.

And we represented the GenAI companies that were coming out as well as the established ones.

So it was really involved. You remember when first GenAI first kinda hit the market, there were all these meetings at the White House and congressional briefings, and I was on the scientific advisory board for NIST that drafted the AI risk management framework, which is I really like it. But also, like, a negotiator in the EU for the EU AI act and working with many foreign countries, ministries of AI or their equivalents. And so became really passionately aware of the fact that we are at this tipping point in society that this technology is literally the most transformative and disruptive technology since electricity.

More than the computer, more than the Internet, more than like, this is a fundamental shift in society. And about this time, the head of the firm that I was working for asked me, hey. What do you think, you know, the effect of GenAI is on the practice of law?

And so we undertook this really cool analysis. So lawyers, as you know, we write down what we do every tenth of an hour. Right? So every six minutes, we keep our time.

And the billable hour is the holy grail of the law firm. Right? Everybody from the lowest paralegal to the highest partner, it's how many hours do you bill. That's what everything is based on.

So we took about ten years worth of fourteen thousand attorneys' time entries to try to figure out what it is lawyers actually do. What steps do they take to create their particular legal product or service? And so it was surprising to find out really that no matter what your practice area is, corporate litigation all over the place, right, that lawyers basically do six or seven things over and over and over again. And that conservatively, eighty percent of what is lawyers currently do today is better done by AI.

And so now the twenty percent we'll talk about this. The twenty percent that's left is absolutely essentially human. Right? It's just critically a human capability.

But when eighty percent of what you do is better done by AI, that's a problem. Right? And law has never been particularly open to efficiency gains because when you build by the hour and you do something more efficiently, there's no benefit to you. You're just giving all the efficiency gains to the client. So basic Adam Smith, it just doesn't the incentives aren't aligned.

So I figured out two things at once. One, the way that law is practiced today is dead. That generative AI is an existential threat to, especially, the bigger law firms. But on the flip side, a law firm that completely takes advantage of all the power of what AI can do can be a completely different generational kind of law firm. So I decided to take some of my team, and we went and formed Clarion AI Partners, the firm that I'm at now.

And everything that we do is AI based, augmented, enhanced.

And so we, like, built our own Jarvis, like, you know, for a Ironman world.

Like, our favorite analogy is Oh, I love I love that analogy.

And this is what most of our clients don't understand, and a lot of people are very afraid of AI taking over the jobs. Well, it's not like you're gonna unplug a person and plug in a bot. So we say think Ironman, not the Terminator. Right?

If you take a person that you've spent years training and they've gone through school and they've got all this experience, and that is your greatest asset along with your data. Right? The data that where your artifacts of intelligence are housed. When you bring those things together and encase them with enabling technology, all of a sudden, you've got Ironman in his flying suit in Jarvis.

Right? And so decided so my arc I will tell you that when I was at this firm, it was the pinnacle of my career. Like, I had this very unusual path. I had been at kind of the cutting edge of data and pushing the envelope on eDiscovery and then information governance and now algorithmic fairness and now AI and walked away and set up this AI based law firm.

And all my friends are like, you're an idiot. What are you doing?

But there's not, to our knowledge, another law firm in the world that will help you with AI legal questions.

Your AI strategy, like, how do you transform your company safely and effectively with this technology, and then can help you actually build the tools that you need to do so.

And I think the answer to that is probably not.

Mostly because, like you said, the eighty percent of the stuff that goes away will now allow us to do that twenty percent of stuff at greater dimensions of detail.

I don't think we are exposed, like, the intellectually curious people are not quite grasping, like, oh my gosh.

We're gonna be able to focus on just the stuff that we love. I don't have to worry about time and billing. I don't have to worry about taking the garbage out. That'll all happen euphemistically. But the question I have for you is when your firm is at this kind of precipice of a new brand new day, how are you controlling, like, the excitedness, the eagerness, the demand to just jump into something and start using it in light of what we know can be really problematic. It can yield a ton of information. Some of that information can be completely wrong.

You basically can't, as a human, take the easy button and just take with what AI is providing at face value. So how are you kind of chilling people out here and saying, oh, slow the roll here. This is an amazing moment in time, brand new day ish stuff, cotton gin, maybe printing press, electricity. How are you telling clients to, like, hold on? Because there's a lot more work that we need to learn about.

The first thing we say to them is here's the two problems we run into the most. One is that if you are using AI to do what you do today, better, faster, stronger, that's not enough.

That's literally table stakes. That is the minimum that you can do to just survive this era.

The other is that the biggest mistakes that leaders make is that they spend too much time defending how they have historically been successful.

And so think about law firms. Law firms are one of the bigger problems because, one, they're partnerships, so everybody's equal. Even though you've got a management committee and you've got a chair, you know, there's only so much control you have. And they're usually older. They're usually very established in their ways. They are not the most tech savvy people in the world, and so you're you're pushing against all of this inertia against efficiency and then changing what you do.

And if you don't change what you do, you're like the dinosaurs the day before the meteor hit.

You're getting run over right now.

Exactly. And, you know, the problem is more acute with generalizable tools. Right? So any of the chat kind of tools, even tools that are specifically made for lawyers.

Right? So, you know, there's some really wonderful products out there and that can do some great things because the LLMs that they are created on are legally based. Every court decision, every treatise, every law review article. Right?

And so they are unbelievably powerful and helpful. Right?

But they've been controlled in the background by this idea of constitutionality. Right? Like, every one of these tools has controls. If you go and say, hey.

Help me build a bioterrorism weapon with the chemicals in my bathroom sink or under my bathroom sink. None of them is gonna answer that question. Right? They've been controlled at the back end to not do that.

Beyond that, the moral question becomes very important.

Inherently flawed. A hundred percent, Eric.

There's two answers to that. One is on kind of the general purpose tools. Even those tools, you can inject your own moral reality into how they act.

So this idea the way that I try to help both technical and nontechnical people understand these systems is that when you're building an agent as simple as a chatbot, you're not coding it. You're not programming it. You're forming its psychology.

And so giving it a persona statement. Right? This is what you are. You are this.

And then giving it a series of constitutional rules, you are defining its moral compass. Right? And that definition is coming from whoever is creating that personification statement in constitutional and even how you measure those things. Even with these generalizable tools, you can reflect your moral view on the world into how the tool works.

I do. Mainly because we're getting out of the world where any human is labeling anything. Right? Because our whole problem was that we had humans labeling stuff and humans have different opinions about things.

And and nine times out of ten, they always just threw it in other anyway. Right? So it wasn't really helpful. The way that these frontier models work, though, is that they find patterns however those patterns are distributed.

And so they're very good at finding more like this. Right? Like, because of how they vectorize information, when you start to vectorize information, it's just mathematical similarity.

And so labeling and all those things, finding GenAI should be the best advancement in eDiscovery ever.

What I think is gonna happen at the enterprise level is that data isn't going to live as flat ASCII data anymore. Data is gonna be vectorized from the very beginning, and that makes it much more accessible and much more amenable to frontier model understanding, manipulation, gathering, pattern finding. Right? And so I think the actual way that we store and use data is going to change, and that that crosses over all kinds of data formats and modalities, whether it's pictures, videos, words. Right? So access to data and access to correct, quote, unquote, data is gonna become easier and easier over the next few years here, which should make eDiscovery much simpler.

And frankly, as you know, once you get through eDiscovery, cases almost always end because it's pretty clear Yeah.

Who's at fault. Right?

So maybe. It depends on all of the main discovery tools. When you look at them, they're all really, really diving into trying to get JNI into the workflow, right, and to more or less success.

But, inevitably, they will get it in there and they will get it right. But eDiscovery tools have always been an anomaly.

The only reason that they have existed is because the platforms that we created data on inside of a company didn't have search capabilities built in.

Right? And so the whole concept of having to take data out of a company and do something with it and then give it back to the company to turn over to the court is ridiculous. Like, it's been ridiculous since day one.

Yeah. Sorry to all my friends with the eDiscovery space, but it is. It's the only reason that all of that industry exists is because that capability was not built into the systems on which we created the data because they didn't foresee that need to do so. Right?

So how is that gonna play out? Initially, it's gonna play out with the same eDiscovery tools that all of us use today. They're working really, really [inaudible].

It's gonna be a tough sell.

Remember back in the early days of eDiscovery when you had to prove there was a question whether a search term worked? If I put in the term rose that it's actually going to return, remember all the track legal track work with Jason Barron and Mara Grossman and all those early pioneers. Right? And then we got to TAR.

And TAR, go my gosh. Like, convincing a court that, you know, you could predictably classify this stuff into tiers and you could do sampling and there then TAR kind of settled. Well, now GenAI is like it's just like this whole huge magical new thing that there's gonna be we're gonna go through the same process of everybody testing it and some people using it and finally settling into how it's gonna work. Now the question in my mind is, is that going to happen internally or externally to a company?

And that is frankly gonna decide the fate of the discovery tools that we see today.

An absolute pleasure. Thank you for having me.