Special Reunion Episode: AI, Human Intelligence, and the Future of Legal Operations

You know, Eric, if I could detour for just a moment, something that Daniel said caught me when he was talking about empathy.

And I just wanted to say that I think one of the reasons that this podcast has been so successful is that Daniel Gold is one of the nicest guys on the planet. And I'm not just saying this because we're on your big anniversary podcast. Your great BDO cat and I have talked about this many times. And if there is anyone who shows empathy in our profession, it is you, counselor.

And so I'll bet I speak for everyone here when I say thank you for all you do, and thank you for being the quintessential nice guy as well as just the knowledgeable guy you are. And I still need to have some more of your coffee. But, Eric, back to your question. Yeah.

You know, we see all our friends from the profession in the industry here, my Relativity colleague, Chris Haley and Dave Cohen, one of the, education all star lawyers. And we all get to the same thing. It is not the AI. And, you know, I've got an article coming out soon on not blaming the AI and not blaming others.

But you think back on it, Would you ever submit a case to the court without shepardizing it first? Or, you know, if you're on West, key citing or whatever you do, and you check the citations.

And it's not the AI. And so when we talk about TAR reinvented years later. It's the same thing on we're back in law school or our paralegal training programs or our technology training. It is the more things change, the more they stay the same. And, hopefully, we can all not keep doing the same things.

Yeah. It is exactly the same. It has not changed. And to talk about David's point and not repeating the same mistakes that we did with various technologies, including TAR, My biggest question is, what is When we are selling this, quote, unquote, internally, externally, clients, deploying within organizations, how are you framing?

To me, here's the really critical point. It's a tool, not a solution. A tool, not a solution. Its true value really lies not in itself, but in thoughtful application.

So when you talk about application, that's where the human insight comes into play. That will be critical to achieving success. And from my perspective, what I'm telling folks is that insight, that experience, that expertise does not come from just merely observation.

So if you're not doing it and you're not getting involved, you will be quickly left behind.

Well, I think I mean, I look back about a year and a quarter when I was on the podcast, and I made certain predictions, and I had certain thoughts of where Gen AI was at that time, and I talked about guardrails a lot.

And I think that there's been so much improvement and so much progress in the Gen AI, in the models, in the prompts, in the training, etcetera. I'm not saying that there is not any bias at all, but I think that I did not expect the GenAI products to move so quickly in terms of becoming adoptable without taking on, like, huge risk. I think now that the companies that have provided the main Gen AI tools have really done I mean, they were aware of the concerns, right, of privacy, bias, etcetera, security. And I think that I'm not saying that, you know, you shouldn't be cautious and don't go into using it with your eyes closed.

I mean, as Z said, these are tools. Right? And so we are the ones that have to master how to use the tools. And we have to be aware, you know, of the limitations, of the hallucinations, etcetera.

I think that the technology, like, really within the last year, has really been perfected more than I thought it was gonna be in a year.

And so I'm sure there's some bias in some tools. There's so many different AI offerings now that I couldn't, across the board, say, of course, there's no risk. But I do think that, you know, the key tools that have been brought forth by, like, the huge players are being perfected more so. But we have to go into this, you know, still with our eyes open and with caution, but it's also, as we said, as lawyers, it's our ethical duty, right, to check our output and our usage of it. But I'm seeing, like, tremendous acceleration in adaption and adaption that I didn't think was gonna happen so quickly in a year.

Yeah. It's a great question. I think that, you know, for example, you mentioned images or videos. Right?

What happens when we go ahead and we take the images and videos? We use GenAI to summarize the video or to give a description of the image. Right? It's a image of someone's front yard with a cracked sidewalk.

And, you know, it's the question is around damages or something like that. When you start using the Gen AI to then use Gen AI from a review standpoint or from an analysis standpoint to then analyze that summary.

So now you're two steps into AI to do it without necessarily having a human who went in between to look at that and say, yes. That is a picture of a house with a cracked sidewalk versus the, you know, no. It's actually an airplane, and the Gen AI got it totally totally wrong. How much are we reliant on that?

And how much are we gonna be looking at that? And where do the humans come into play there? And that's part of why I think saying that we're gonna be replacing humans is wrong. I still feel that.

It's all about how do we supplement them and how do we make them super efficient. How do we get them to the point where, you know, they can look at that and identify right away? Yeah. It is that house with the cracked sidewalk versus something else.

And instead of spending, you know, a minute and a half on a document or whatever they would have done, if they can do that in ten seconds, read the summary, move on, those types of things are gonna be really important going forward, but it definitely introduced risk the more we utilize this technology ongoing. Meaning, the more we remove the humans, there is a element of risk that is there. But the technology is getting better and better, and the providers across the board are getting better and better at providing guardrails to make sure it's reined in.

Yeah. So I'm fresh off the heels of being at the Masters, you know, New York conference where I saw I saw Phil there. I think a couple others may be on here. But I had some of the best discussions with my opposing counsel on GenAI and realize we're basically in the same boat internally.

Like, people in, you know, my role in LitSupport trying to convince not necessarily clients. It's convincing the own attorneys we work with. And I'm about to start a whole bunch of testing in GenAI on real case data, finally. It's taken a year.

But the biggest selling point of all this is the transparency fees. I can go in and click a little hyperlink and it says, here's the exact text that is making this document responsive.

And my hope is the more people familiarize themselves with that, I'll have less pushback than, you know, TAR. I still get pushback on TAR and active learning. I just do it in the background, and everybody knows. So yeah. But the transparency is gonna be key, and I know we're gonna have probably a ton of plaintiff defense disputes over what needs to be turned over with, you know, prompts, if anything, and, you know, the coordination of all of that and hopefully cooperation. So, yeah, this is, as a nerd, a real exciting time for me.

Sure. I think it would go back to the basics. If we talk about the tools that we're talking about as black boxes, they're only gonna be as good as what goes in. So garbage in, garbage out.

So that taxonomy that you're talking about, that classification, that's not done without somebody on the front end or human on the front end helping to educate the black box. So it's gonna be important. But it to your point, I think, maybe Daniel said that it will supercharge. It has the ability to do the amount of work that would take hundreds and hundreds of people to do so quickly that as long as we keep the human in the loop, I think we'll be okay on the back end.

Kudos to all the nerds out there who are working on making sure that there's some discipline and construct around what goes into feeding the black box that will then feed us. I also loop back to maybe something we've talked about, and that is the human in the loop. In the end, you just have to have accountability. There has to be a person that looks at this, and based on their experience and their judgment and their wisdom.

And I think one of the things I really liked hearing was the thought that humans have empathy. And I think one of the interesting things to me is that all of this for lawyers, for knowledge workers who are trying to implement AI, really, there's a lot of change management. There's a lot of anxiety about being replaced, about falling behind, about how do I succeed in a paradigm that's different. And I think it's incumbent upon all of us to make sure that we recognize that, that we have empathy not just in the specific instance, but empathy for the people who are using AI To tell people over and over again, use AI.

Use AI. Use AI without having them really understand what the AI is can be really anxiety producing, especially with the specter of losing their job or being replaced in the background.

And that is, you know, the fear that I think a lot of people have is, you know, especially some folks who are not lawyers that are other legal professionals. Oh, well, you know, I'm gonna be replaced. And that is obviously a concern that a lot of people have, but I think it goes back to this idea of you can't have technology without the humans.

Right? You have to be able to have that. Z, I think you might have some perspective on this as well. Correct?

Just a very quick comment and something that I've been talking to others in the industry that we need to be cautious of.

We need to bifurcate the discussion instead of saying what is GenAI tool.

There are problems that GenAI tool can create, and there are solutions that GenAI tool can provide.

Because lawyers are not by design by embracing technology for unsophisticated audience about talking to deepfakes and other bias, etcetera, issues, which JEDI can create those problems.

They get scared of the use cases where JNI can provide a really good answer. And so something that we need to think about and the folks who are at this intersection of law and technology clarify the messaging. Otherwise, we're gonna have adoption issue.

Yeah. So there's so many new tools being developed. And, you know, we go to these legal technology conferences, and we see them. And there's a huge wow factor. But I think it was, Stephanie who noted that the big problem is going back and selling them to your firm and your lawyers who are not as technology proficient.

Although AI has been ninety five percent great and only five percent problematic, it's the five percent that everybody hears about. It's the hallucinations, the fake sites. Just today, Bloomberg came out with a report where AI mistakes crept into a judge's opinion, where there were fake quotes and a wrong citation and so on. And that's what everybody hears about, and so there's a lot of caution.

So I think what where we focus is, hey. What can we really get people comfortable with? You know, it's gonna be a journey, and the AI is gonna keep getting better. So for example, at Reed Smith, one of the things we focused on early was deposition transcript summarization.

Many products now do that and do that very well. And if you show the attorneys who are used to having paralegals or associates do those summaries, if you show them those summaries side by side, in some ways, AI ones are better because of their ability to characterize the testimony and categorize it and the ability to search across multiple transcripts, summaries at the same time, and so on, that we view that as one that would be an easier adoption and one because of the hyperlinks that you can check and make sure that the summary is accurate, you know, if you're getting a product with hyperlinks. So, yeah, we gotta be careful of just running after that shiny new toy and make sure that whatever we're running after is something that's really gonna get adoption and can convince people to use it. And, yeah, this AI is good, not just risky and dangerous.

Well, obviously, Dave Cohen is an institution of knowledge.

And Agreed.

I think about the various technologies. I remember in CLEs fifteen, twenty years ago when Dave used to bring the dollar bills out to make a point about the old days of tar. And it's funny. I mean, my continuing effort to emulate and be Daniel Gold, Esquire, I thought it was interesting when you mentioned that you have justice Kennedy and judge Peck on your wall.

Oh, I was gonna kinda add to the prior before David's discussion. How right now, I keep getting inundated with this new GenAI tool's gonna fix figure problem. And I talked about this a little bit earlier this week, but we're kind of, in my world, trying to go back to just fixing the foundation. Like, before we can use some of these GenAI products, I need to fix some of our workflows now that technology has changed since, you know I'm almost at a twelve year anniversary at my firm.

And looking at what I did in twenty fourteen, I just cringe. It's like looking at your high school term papers. If you can call, like, a lot of this info before it gets into review, the Jet AI is gonna just work so much better. So it's try to go back to the beginning and set up the workflows to reflect using Gen AI eventually.

Yeah. So, Daniel, thanks for having all of us on. I know we've been talking a lot about eDiscovery and discovery in general.

One thing I'd like to talk about, if I could, is just like the normal legal analysis and putting thoughts together.

I'm getting a lot of things from clients, even new clients, putative clients that come in pretty hot and already have a really good take on the dispute, on the issues on the contract, whatever, because they've used a, you know, frontier model, and it's really darn good.

And I just wanted to, you know, maybe talk about that for a second because it's going to increasingly ratchet up the pressure to do really good work and be very, very responsive to client needs and demands for lawyers regardless of your practice area, regardless of how long you've been doing it. Because more and more and more companies of all sizes, individuals, you name it, are able to get kind of almost ubiquitous pretty good knowledge and ubiquitous pretty good analysis. And it's not always an a plus, but it's really, really good what we're seeing. And it also is kinda nice if you are conversant with using AI to have that conversation with your clients.

Say, this is awesome. Let me affirm that. That's great. There's gonna be areas where my team will have, you know, better answers, and we're going to, you know, provide that to you.

But, wow, we really short circuited a lot of that kind of beginning process because now we're almost you know, we're in the same chapter, maybe not the same page. I'm very excited about that.

I mean, I just think about, like and this may be slightly different than what you're looking for, but what's resonating to me throughout this conversation is we've done this before. This is for most of us on this call. But by the way, Daniel, great to see you. I'm so proud, and look at this group.

I mean, wow. I'm just I'm honored to be here and be a part of this. But we've all done this before, and looking around, like, we've all done this many, many times. This is not our first technology revolution or evolution.

My entire career, we all have to improve and get better and learn to delegate and those kinds of things. This is just an the next evolution of that and deciding what low risk use cases or where I need help that that I can delegate that to and that I can trust in the results that I'm verifying as I go. I'm doing these things. This is a process, a mechanical process that we've and habits that we've developed over our careers to leverage technology innovation since the beginning.

And I don't know that I view this as any different than that.

And I've spent my entire career wishing I could do more and more and more. I've never billed by the hour my entire career. I've never spent in any day where I got done with the day and said, I did everything I wanted to do that day. And so, like, if this this gets me one step closer or hopefully five steps closer to providing more value, getting more done in the same or less amount of time. That's the goal here. This is an opportunity just like every other technology revolution, and you gotta choose to get on the train or not. Right?

And so this is that opportunity, and everyone on the screen here, I suspect, has embraced these technological changes and those opportunities before, and that's why we're all here today and better for it.

Right.

Yep. It wasn't, but that's okay. I'll comment on that and then add some. There will always be a need for lawyers, and there will always be a need for a human component. We're seeing If anybody's using generative AI on a daily basis, we all know that.

This is a very different technology in terms of for a number of different reasons. One, it's faster than anything we've ever encounter.

We are seeing changes within weeks, not years. And the other thing is we're starting to get pressure from the business. So we've been talking to a lot of the corporations and the businesses. The CEOs are saying Use generative AI.

It's also an opportunity for the legal department because there's a spotlight being shined on us now because there's concern around governance.

Oh my gosh. Now we need a policy. Now we need to figure out what we're gonna do. And it is a time where legal can truly create a competitive advantage and a legal operations role, which is kind of the definition, creating a competitive advantage for the business.

I was talking to a GC yesterday, and here's one of the things that makes generative AI a little bit different. They are able to take some of their existing Google technologies and write within an hour a script so that it can go through his email and pull out documents and do an analysis on those documents so that when he's now scanning his email, he can get a summary. It's those kinds of things that are going to move the needle. And we're all talking about, oh, it's not gonna eliminate jobs, but it already has.

So that is just incorrect. Microsoft had a ten percent reduction in force not too long ago. And as we become more able to scale more quickly, we are going to start losing roles, and we should start to lose some of those roles. So if we are not getting ahead of this, we will become irrelevant to the business who is our internal client.

And then one quick comment on billing is it is because we can collect data so much more easily and quickly, we are going to know how much something should cost.

And so this may be the first technology that actually moves us away from the billable hour, and we start using fixed fees.

Well, thanks. And, Rocky, I see your hands up too. So if you wanna jump in on this, I know you and I are kind of kindred spirits. So I don't wanna lose sight of, you know, whatever you wanted to add as well.

But I took a note as I was listening, and, you know, I wanna be respectful of time. But I'm grateful to chip in because ever since we spoke and you interviewed me, I've put my head down and have gotten to learn. I don't know if everybody's familiar with this, but if you're not, you should be. You know, they're talking about different levels of capabilities, t shaped, I shaped. And I was talking to someone, and when I was describing kinda where I saw things going, she's like, oh, Maureen, you're exactly right. She's like, you right now, I have t shaped capabilities.

So what I've done is I put my head down and really got, you know, in with global privacy, global enterprise IT, cybersecurity, governance. And it's been the best six months of my career because I see the conversations, and I hear the conversations that they're having. And I think it was Connie who just said that the business is driving so much more than I think the legal eDiscovery industry even understands.

So when we say we've been here before, no. Not even close. Right? Like, this isn't even about Tar because those aren't the conversations that are being had right now.

So I can't divulge what I'm working on, but, eventually, within the next couple months, I'm gonna roll out some new frameworks that I've been working on because the conversations around everything is changing so much. It's being driven by AI. It's being driven by data, and it's being driven by collaboration and prioritization across these different teams. And it's a pretty incredible experience to be a part of.

My passion's coming through, isn't it?

Yeah. I think it's Between that change of the rapid pace of change together with because this technology allows us to do more earlier. And I think that's gonna actually be the biggest shift that our industry sees because, again, it's not the same old same old. It's not like, okay. We've been here, done this before, and we know how to do it.

Everything is shifting to take place earlier. Like Nicholas said, the fact that he can get something from a client that's already prepped, he might have to go through it again.

All of this is shifting things towards the client. And that's what's really important here because the client's the one who pays the bill in the end of the day.

So I think the real impact of Gen AI is gonna be seen by really taking what we've traditionally thought of as the EDRM and flipping on it on its head and allowing us to do so much more, so much earlier that that is gonna have the biggest impact throughout the entire process.

And to add on to that, the only way it's gonna be solved is through enterprise technology, not through legal technology. It'll be a part of it, and it'll still play a role, But the whole evolution around technology is changing as well, which is pretty exciting.

Yeah. No. I feel like for anyone that is good at what they do or passionate about what they do, they will always find ways to be useful. And part of that is changing and learning and adapting to what you're doing. But for somebody that is stuck, is unhappy, is not passionate.

I mean, their jobs are gonna be automated. Like, that is gonna happen. But I think, you know, for the majority of us, like, the being on the edge or adapting to the change is what shows your value. And as much as I love Gen AI, it is doing, you know, incredible, incredible things.

But it is also doing, like, really dumb stuff too, you know, that it's, like, so frustrating. Like, you ask it to come up with, like, a random number and it gives you, like, seventeen every time or something. Other stuff that is not ready for deployment, but then there are, you know, other ways where it's just like, it is absolutely magic, and you're seeing stuff that, like, for a human person to do it mechanically, you wouldn't wanna do it because it's the work is so boring. You know?

Like, I come back to this tool all the time. I don't know if anyone remembers this, the bait stamp from way back in the day. And it's like, I was doing that at the start of my career, and now we have software that does it. And nobody would ever do that manually now.

You know. And I feel like we're at this point with Gen AI where we're blessed that it can do some really extraordinary things that we no longer have to do because the work is just so boring, and we can do the more interesting stuff. You know? And that's a great quote for me.

I'm pleased that it's on the number one podcast for BDO.

Yeah. You know, I gotta tell you, Maureen, the tea, the pot in the comb philosophy, Heinemann writes about it, and a guy named Bjorn Talman used to run Coca Cola GC. He writes about it. And I think that really kinda goes to, like, the old medical philosophy of about cross training and really getting doctors to share information about how they practice and diagnose. It's an amazing concept. But anyway, Nick, I know you had your hand up. I just wanted to call back on you to see get your thoughts.