BDO's Legal Tech TalkTech Agility in Law: Embracing AI and Innovation
About this episode
Daniel Gold and Jared Crafton talk with Todd Itami, Director of AI and eDiscovery at Covington & Burling, about how technologists and lawyers think differently. Todd explains why generative AI will reach the practice of law faster than TAR did, how he builds a tech-agile culture through liaisons, office hours and small wins, and why lawyers should apply the same validation habits to AI that they apply to Westlaw.
Key takeaways
- Technologists expect tools to change constantly. Lawyers want to learn a tool once, which slows adoption.
- Generative AI has lowered the barrier to entry because anyone can use it, which is why Todd expects it to spread faster than TAR.
- Building a tech-agile culture matters more than any single tool. Todd uses pilot groups, liaisons and office hours.
- Small wins create believers: one macro turned a 300-hour exhibit stamping job into a three-hour task.
- Existing information governance and confidentiality rules still apply to generative AI, and its output needs validation like any other research tool.
Full transcript
Transcript from BDO's published episode, edited for transcription errors. Speaker labels added by Gold Standard Leadership.
Welcome to BDO's Legal Tech Talk podcast. We're joined by judges and legal professionals to discuss emerging trends, regulatory updates, and the latest headlines.
We'll provide tips to help your law firms and legal departments make the most out of legal technology.
Hi, everyone. I'm Jared Crafton, BDO's forensic technology practice leader.
And I'm Daniel Gold, forensics principal and leader of the eDiscovery managed services practice.
Let's get started with this episode's exciting topic.
Welcome back to another episode of BDO's Legal Tech Talk. We are thrilled to have in the virtual recording studio with us, Todd Itami from the law firm Covington & Burling. Todd is the director of AI and eDiscovery. Todd, welcome to the program.
Hey. Thanks for having me, guys. I'm a big fan. So, excited to talk to you today.
What's your favorite piece of technology at home right now?
That's a good question. So I've always loved hardware. You know, I'm a hardware enthusiast. So anytime, new products from NVIDIA come out and, of course, I'm talking all their consumer lines. So they're all their RTX series.
AMD, my dog, I named my dog Ryzen.
Of course you do. Of course you do. Ryzen. So everybody's like, well, that's a weird name.
I'm like, yeah. Don't worry about it. Ryzen just came out or AMD just came out with their, new nine thousand series. Right?
So I just put the 9950X in my machine building machines around that, but waiting for the new chipset to come out in the motherboards. And that's how big a nerd I am. And those so new stuff will come out. I'll overclock it.
I'll see how it does. I'll benchmark it. And then, you know, just put it up on my wall. And then I'm waiting for the next wait for the next stuff.
But it does come in handy sometimes, you know, crunching through big datasets. And it's, like, really cool now that you can download, like, you know, the small llama model. You can run that on a forty ninety and, like, actually have an LLM at your house that meta trained. It's kind of nuts when you think about it, but I love it.
And I think that we're at a place right now where there's so much competition going on, and there's just money, VC money flying around everywhere. And I just can't get enough. I'm like, wow, this is great. I hope the competition continues as it has to so we can do cool stuff, like download an eight billion parameter model from Meta.
It's brand new. They just trained it, and I can run it here. You know? So I think kind of the combination of in the last five to ten years, we've really seen hardware, like, kind of wake up again, you know, in the way GPUs are advancing with the thirty r t x thirty series, you know, especially for commodity GPUs.
Like, it got really better a lot better all of a sudden. Everybody's like, this is making progress about the right time too with all the AI stuff. But I don't know. My dream is that I get my hands on some of the big NVIDIA GPUs, but I think that's, like, a little bit impractical.
I can just go, like, rent cloud compute like everybody else.
I have a feeling it's going to happen then.
There is a sense of curiosity in you, Todd, though, that when you're talking about building one of those towers to those computers, this idea of just, let me just load it up. Let me overclock it. Let me see what it can do and then iterate. And I think that that goes beyond it has to fundamentally for you go beyond just, I wanna build a cool computer and see what I can do. I have to think there's a crossover into the world of the law.
I think that's right. And I think, you know, the idea of tinkering or experimentation or, you know, thinking about legal problems or eDiscovery problems like a data scientist or like an enthusiast.
You know, it's a little different. There's a lot of these panels now on GenAI and how's it going to work with document review or these different things. And, you know, as lawyers, we have this very linear thinking.
You know, will it do x? Is it ready for prime time to I'll give it some request for production and we'll hit play and then it'll spit out the answer like tar and like, is it ready? Yes or no? You know, and that's kind of like, wow, that's like a really one dimensional way of thinking about a software product.
And it would be crazy if we kind of thought of like deterministic coding or database technology in that same way, because there's like this whole spectrum of things it can do. And I, you know, I think about that like way, like, man, you know what would be great if you had any level of LLM, kind of in house behind the firewall, whether that's at your vendor or at your law firm or whatever, to do a million different kind of in between that solve the whole document review problem with GenAI or nothing. You know, it's kind of QC. So you've done your TAR or you're reviewing documents for deposition and you say to it, have I missed anything here?
Could we create a RAG workflow to look at this set, look at everything else, and would you suggest I miss anything and then read between the lines? Am I missing something here? Basically, it's kind of finding, you know, how would a data scientist how would somebody who's an enthusiast think about using these tools? And it's really the creativity.
Talk to me about the difference between because you live both worlds. You're a computer scientist, right, and also a lawyer. How does a computer scientist think that is different from how a lawyer thinks?
In some ways, the practice of law has not changed since eighteen seventy.
Yes, we have the Internet and Westlaw and the sticky note. And those three things, like, really improve the Braxton Law, but on some level, you know, it's this noble profession where we're going to go sit in an office on our own and, like, draft up something brilliant. And so I think lawyers are mostly looking at their tools in a more one dimensional sense. Maybe they're thinking about discovery or the stops on discovery as, like, some framework set out by the rules of procedure, like, and then we will review the documents.
And really, I'm kind of constantly surprised in the industry as a whole, the amount of time that lawyers think I need to dive into this software, and this is gonna help my life. Even if you take, like, the most common lawyer program, Microsoft Word, how many people out there are using this with, full understanding of how styles work and templates and ability to create macros to check the eight things they've always checked or, you know, understanding Copilot or the variety of tools out there they can use as plugins to make sure all their definitions line up, you know, very few. And so it's kind of like one is first a difference in mindset in terms of expecting new technology, expecting to learn new things, expecting it to change.
And so the first difference in mindset is, you know, we're flexible. We like new stuff. We think the software, there's always another version coming out. There's an update to the PEP to the style guide.
There's an update to the code itself. There's a new keyword. It's exciting to us and we understand this process of learning new things. I think lawyers aren't expecting that.
You know, they want to learn the tools and then learn their practice and get involved in the law and be better at the law and strategy and writing.
And so I think that's your first difference. You know, we're looking for new tools. We embrace them. We think it's cool.
That's one piece. We want to learn new things. Lawyers generally don't. I think two, you know, there's a way of thinking about workflows for the data itself that we see a document in a database as just a row in a spreadsheet.
We see a piece of data sitting in a data warehouse as just that. You know, one of many signals collected by Hadoop and, you know, there's these great tools, but, you know, lawyers are kind of like, well, this is like a document.
We're gonna do something to the document and kind of are still trying to put into this data framework the physical world of lawyering, of facts. And so, you know, we'll look at a database and we'll go, oh, there's a, b, and c we can do to it. Let's move the data like this. Here's a nice Visio with a workflow.
And I think lawyers are like, how do we get this document? You know, just show me what to read. Like, I want to do my job. And so it's interesting when lawyers come to me in various contexts, whether this is casual or at the firm in my role as kind of helping with eDiscovery and AI solutions.
I'll find that maybe half the time when I'm being approached to say, hey. Is there an AI thing that can do this? I'll end up saying, oh, yeah. Well, here is kind of, good principles of workflow, classic data management stuff, kind of let's make the Visio. Let's sit down together and map out the data flow and just make it visual and think about these things in units and in terms of validation and in terms of thorough put, you know, kind of like a data scientist would think, how are we gonna get through this data? And that will be the answer and they'll go away happy and they'll be like, this is great.
And, you know, and we never really engaged any sort of fancy, you know, foundational model or anything, but just, you know, kind of applying the basics, they'll be so happy about it.
Todd, it sounds like the second point that you're making about the differences between how computer scientists like myself and attorneys like Daniel Fink or, you know, people who have combined skill sets like yourself is really, you know, thinking about data strategically and really being able to, you know, understand the full picture of data and the impact of decisions you may make with that data. And also it sounded like you're saying a lot of attorneys that that you've seen are really missing the forest for the trees when they just get, you know, so focused on some of the tactical aspects of working with data. Now do you think that's something that we can help bridge the gap with with education or, you know, what strategies can we employ to help the attorneys that we work with to get that more global and strategic view of the data that they're working with?
There is some profound point that this is probably where I differ from a lot of other lawyers in this space that there is a level of technical education and technical understanding that I think the baseline, we have to move it up.
You know? In some ways, I think we can retool as an industry and really try to understand better these tools. You know, in some ways, it's about ready to change.
I think it's really hard, like, there is so much of a hype cloud out there with AI.
And for people, I think your average lawyer is off put by it. I think in a lot of ways, it's off putting to me too and I consume this stuff constantly. I'm drinking, you know, the updates because it's kind of like, you know, there's so many voices out there. There's like your AI influencers love to talk about prompt engineer. You know, like, just like, wow. Okay. Like, interesting.
There's the popular media portraying these things. And, you know, we see like character AI, second most visited AI tool in the world. Like, wow. You know, Wall Street where the valuations we see don't match the technology in on both sides, but type one and type two errors. It's going up and down venture capital. Again, it's great energy. We got developers and then we get salespeople.
And it's kind of like all those things, you get this sense, I think, your average person, and I do, like, something here is off putting, you know. It's almost like there's an in crowd with this new AI stuff or it's like there's some exclusivity and I'm just like, wow. That is kind of the opposite of my world of what I've seen over the last, whatever, thirty years of being so into this stuff and building computers. And it's kind of like, no. Inclusivity, education, getting people up to speed, making them feel comfortable with these tools because they're the customer. You know, they're the people who know the client. We need to create this technologically agile culture that's welcoming, that doesn't feel like there's some people in the know with, like, AI stuff and others that don't, but really stepping up the baseline.
What do we expect these attorneys to know? I'm kind of like, wow, it needs to be so much higher than it is right now and it's coming.
There's this Churchill quote that I love, that I heard last year. I can't remember which speaker it was at, Web Summit in Lisbon. And there's this Churchill quote, and it says, you were given the choice between war and dishonor.
You chose dishonor, and you'll have war, depending how big a history nerd you are. There's the Munich Agreement, which was, you know, historians have some disagreement about what level of awful it was, but, you know, Hitler was over here and they'd oh, let's let's do this Munich agreement. It's gonna stop this oncoming tide. And, you know, Churchill's like, no.
You agreed not to fight him and you that was dishonorable, but you're still gonna have this crazy and, of course, he was right, you know, in the coming months. And it's kind of that way at the tech revolution and AI, and it's coming. And all the hype makes us kind of think like, oh, maybe not, you know. But right now, let's say it doesn't advance at all.
The ability of Gen AI today, you know, September twenty twenty four Gen AI technology to change the legal world is here. It's over. We can use these technologies. We can think of new creative ways to integrate it. Even if it only marginally increases in value, it's going to change the practice of law. You can ignore it and you can pretend it's not going to be better at drafting a brief than you soon, but it's coming.
And I think we have on some level, despite the hype, grossly underestimated both how fast it will progress and its impact on the law.
There is so much, Todd, that I have to unpack with you. Wow. I think that was just so apropos, and I'm thinking of this epic Greek mythological war that's at play today. And you spoke with such eloquence, and yet some might say it's hyperbole that we are at such an inflection point.
But when you speak of this point that we're at right now, I have to ask you a couple of questions. The first one is, why is this any different than fourteen years ago when we said, oh, technology system reviews gonna be the best thing? And there was this hype. It was in The Wall Street Journal.
Right? It was. Right? Tom Gricks was there. He was the cover photo of The Wall Street Journal, had the great remember.
That great case in Virginia. Right? Right after judge Peck, who was on this podcast as well, said that, you know, you could use technology system review and everybody in the sales said, sign an NDA. We've got the best algorithm for technology system review and then comes along a couple other technologies that said, oh, you don't even need a seed set.
You just code one document and wham, I'm going.
Right? You don't need to do anything. Right? So one, why is this any different? And two, when you talk about this culture of inclusivity and you talk about this culture of bringing people in, I wanna know it's beautiful, but how do you do it? Yeah. So counsel, I asked you a compound question.
Yeah. These are good challenges. So number one, yeah, you're preaching to the choir. I mean, maybe two or three years ago, you know, I'm still having trouble convincing some people to use it.
For me, I'm kinda like, yeah, that's a piece of the education. If they really understood what they are doing, they wouldn't. But why will the uptake be different? Why is this different than, you know, the TAR or all the web three stuff?
I think that the difference is, one, we have bridged some gap between the overall usefulness of the technology to the populace.
You know, Gen AI, everybody can use it. Everybody's kind of seeing like, oh, wow. It, like, does things. We're seeing, you know, younger people.
ChatGPT is more popular than Google for searches with certain demographics. And we don't even have search GPT yet, which they're promising here soon. So one, there's kind of this thrust of popularity or energy in the public at large, and now we've reached a point where I think that's gonna get harder to deny. You know, we're seeing RFPs from clients and potential clients, and I don't know, eighty percent of them are asking, how are you using GenAI?
You know, some of those are asking because they wanna make sure it's safe. Others are asking because, like, hey, we've heard of this thing. You know, we're seeing it mentioned in all these companies. Will that hype die out and it, you know, fizzles?
Well, maybe.
Guess that's a possibility, but I just think, one, the technology is groundbreaking enough and helpful enough that we're gonna be able to ride the wave. Clients are gonna demand it.
And the technology itself lowers barriers to entry for the technology.
The technology itself, it's so helpful for coding. It's so helpful for making new apps. And, you know, we had this kind of emerging data landscape a couple years ago. We're like, wow.
There are a lot of collaboration problem, you know, platforms. A lot of cloud stuff popping up. I mean, it's kinda crazy. But now, get ready.
You know? Because everybody can develop. And it's so nice for me when I'm sitting there and I'm like, I remember how to do this in Java, but and I can ask, and it'll tell me how to do it in three different languages, x thing. You know?
It'll translate it for me. I can give ChatGPT or, you know, anything, any of these models, my code and say, what would you do to optimize this? And sometimes it has bad ideas, and other times I'm like, wow, I literally thought I was the smartest in the way I built this piece of code and it has better ideas, you know. And so I think the technology itself is lowering barriers of entry for companies.
Smaller people, smaller shops can get in and build things. I think this momentum will kind of carry it forward and make it more undeniable and progress the technology.
What does the tipping point look like, though? I mean, are we gonna reach you know, is it the first time somebody submits a brief and they get all the way through a successful result at a trial and they say, guess what? That was all in high.
There's not case one where somebody reviewed their documents with GenAI and then said, told the court and the court went, okay. You know, and we see popular platforms changing their suggested workflows to say like, oh, well, actually, could you review ten thousand documents with people first before you trust our GenAI plug in? Yeah. I don't know when it tips or when it becomes acceptable.
And a lot of that is this idea we talked about before where it's kinda like, you have to have the prepackaged thing, yes or no, rather than the steps in between that are gonna be really helpful. So when will the legal community actually adopt it? When will it really change the world? That's that's a tougher question.
Certainly companies are demanding it. We see lots of very traditional law firms all of a sudden thinking it's a good idea to create AI based positions.
I don't know. I'm, I think, if I had to guess right now, in a year, if you're not using it in your big law firm and can explain really well how you use these tools responsibly, that's gonna be a major downside for your RFP. That's gonna hurt you. Companies are gonna realize, okay, so I'm not getting as much value out of you.
That's what I think. I think it's, it's coming soon. When lawyers will have to learn this stuff better. I don't know.
That's interesting because there's a couple of things that come to mind. Number one is, you know, we've all heard the phrase, the ostrich effect And a lot of the CLEs I give, I talk about how the ostrich effect was actually a cognitive bias theory that was developed by two, economists, had nothing at all to do with psychology, but it has delved into other areas such as, you know, when we talk about technology.
So cognitive bias is one and also cognitive dissonance as well is, something else as well that I think is very interesting. Leon Festinger is the one who created that theory, and I think that there is a lot of cognitive bias and dissonance inside of the legal world because to your point before, we are practicing law, like, with still the nineteen hundreds.
Yeah.
But yet here we are facing all this technology. But there is this weird dichotomy that you have brought up, Todd, and that is, on the one hand, we're still practicing law against the nineteen hundreds. Right? And we still just wanna look at a document even though nothing is really a document anymore.
There really isn't anything called a document. Text messages are not documents. I hate to break the news to everybody. Secondarily, right, on the other A lot of law firms are creating roles with AI and here you are, you are the director of AI at the law firm.
Yeah. So is it hype?
Is it I have to meet my clients where they are? And where they are is the fact that they wanna see their law firms say they do something with AI. So whether or not we do or we don't, we're gonna give somebody a title. Where do you see the entire landscape right now?
Yeah. I think the honest view is across the industry. The rubber has not met the road yet. You pull the associate out of any big law firm, and they're not understanding how to use these tools or are given the tools even in house, you know, from, from infosec or, you know, acceptable use perspective. Like, they don't have them at their fingertips and integrations into these platforms are not there.
And, you know, it's one thing for people like myself, but that we really need it at scale. So what's the state of the industry? Yeah. It's a like you just said, the latter. You know, where people are trying you know, we have our in house tools that we're rolling out to every associate. But still, it's in its nascent stage, even at the firms you know, my friends at firms who are more progressive externally.
You know, the kind of the reality on the street to me, seeing this is, of course, just my view, is that like, yeah, we're just getting started. It's not there's nobody that has a culture of using this stuff to its maximum ability right now.
Can we talk a little bit about your role specifically as director of AI? I mean, I assume some of the things you just mentioned are on the roadmap, of course, but you know, what's kind of your charge here in the first couple of years? What are you trying to accomplish?
So I think that my focus is noticing the hype elements of AI.
And as much as those are helpful and people ask, well, what is Covington doing that we're doing the best stuff. You know, of course. But it's really like, okay, how are we gonna build a culture around this? How do we kind of ride this wave, this excitement of Jenny I to build a tech agile culture?
Because that's more important at the end of the day than integrating whatever is the shiniest new tool. Building a culture where, hey, you know, a pilot, that's something that happens around here. There's pilot groups. We're used to it.
We're used to a subset of lawyers trying things and we've identified people at the firm who like it. And, you know, there's okay. So half the patent group actually knows c. Great.
I bet you they'll try our tools and give us some interesting feedback. You know? So it's things like that is creating a culture that is expecting change, that is expecting, hey, there's this new thing. It's it's learning about tech, and you're gonna need to dedicate some time to that.
But really, let me tell you, it's gonna make your life way better. And these tools are going to eliminate repetitive parts of your job or parts of your job that aren't as fun and let you focus your big legal brain on strategy and creative thinking. And they're gonna give you ideation that might seem wild, but if you sit around and think about it, like, it's very helpful. And so my kind of large scale focus is, okay, how are we going to deliver the most value to clients?
It's creating this culture that is flexible, creating a culture that uses and deeply understands the tools, as opposed to kind of going like, alright. Well, here are the three use cases for the current models on the market, which, by the way, will be radically different in six months. And this is what we're putting in place. Of course.
Of course. You know, we're developing our own in house wrappers that can use the models in our own environment and those sorts of things. Yeah, we're doing all that. We're evaluating.
We'll look at new tools all the time and try to get them in people's hands. But the real strategy, the real focus is creating that culture.
I'm very interested in the cultural aspect of this. Right? Because I mean, you're describing a ton of change management. Right? But it's that cultural point, which is, you know, a very, very soft skill.
Right? Yeah.
That's probably most important. Right? That's really gonna drive adoption. So I'm curious if you have, you know, tips, techniques, you know, how do you get through to attorneys, you know, to really, you know, get them thinking along a cultural line, like this is something that is just part of my day to day now.
I think it's a combination of, you know, promising benefits.
And so I think, one, it's repeating your words over and over with everybody you talk to. And I think one on one or one in a small group, you know, these are smart people. Lawyers, they're smart. They get it.
You know, there's a reason why they did these things. And so one on one, it's easier and you can kind of convince them, yeah, do you kind of see how this is gonna change things? And do you notice how our industry is slow to move? And so that kind of, you know, proselytizing person to person, that's important.
Those sort of things. Okay. Now, can you focus on liaisons?
Can you identify the people, your other missionaries, the other people that are going to engage in that kind of arm's length discussion? So sure, going to each individual practice group or unit within your business or your legal department doing the presentation.
But also in the presentation, hey, we have Sarah over here and Jess and they are the liaisons.
Go talk to them with your ideas. They want to chat. They have some office hours. It's open to talk about.
We're lowering like, you don't have to know about this stuff. You don't have to pretend like you know about it. If you want to talk to them about the absolute basics, great. Or if you have some pie in the sky, can computers do this idea, great.
You know, so creating more points of contact, more places for that kind of person to person discussion to happen. And of course, creating policies that are flexible. Creating policies that fast track infosec approval, that fast track these things or set clear standards out. And this is the ISO certification we're looking.
This is what the scope of that ISO certification has to look like, you know. So kind of deciding these things beforehand and creating processes that you can move it through. So there's this boring policy side that you gotta get your, you know, your policy index out and highlight everything and go through and sit down and make sure the stakeholders are on board with you. Make sure you have the staff to do that, to move faster.
You're gonna have to hire a couple people, probably big institutions.
But that piece and then this proselytizing idea of getting the ideas out there, providing the trainings, on demand trainings, in person trainings, office hours, and the liaisons.
Legal teams are struggling to manage resources, maintain compliance, and adopt new technologies.
BDO's legal operations team can help. We enhance efficiency, reduce costs, and support compliance so you can run your legal department like a business.
Schedule a meeting with us today at bdo.com.
What What are the small wins that I think everybody listening to this podcast is going to appreciate to learn from you?
How have you been able to find new followers from small wins?
I think that the magic happens with technology, and this is AI or creating a simple macro to help a paralegal stamp exhibits.
The magic happens when there is ownership in the process, you know, people feel like they had an idea and helped. And so I can kind of trace a lineage of people at our firm who are very tuned in to this idea that like software can help me or, you know, I bet you if I go and talk to Itami or some of the other folks at the firm, they'll have an idea to do this. And so small wins are seeing things like that happen. You know, the paralegal example is one where on a patent case, stumbled across the, you know, paralegals.
I love to talk to the people who are, you know, actually doing the repetitive work because they have the best ideas of how to innovate. Hey. What are you guys doing? Oh, you're stamping these exhibits.
Okay. So there's a giant folder full of twenty thousand documents. You're fishing out an exhibit for trial. You're renaming it.
You're putting a PDF stamp, and you're naming the stamp. And I'm like, yeah. You know, I think with a little DOS and, like, ten lines of JavaScript within Adobe. So using all of our own tools, we can create an automated workflow where we'll just you click, it'll add the stamp, it'll rename the stamp, it'll rename the file, it'll scrub it and it'll put it in a folder for you.
You know, we changed literally kind of a three hundred hour process into three hours. So that was a win for the client to the tune of, you know, two hundred ninety seven hours.
So they liked it. I liked it. I like the money savings, but the paralegals are like, wow. You know, this was great that we don't like stamping exhibits. And then those people took that. They put it in our trial manual and they're thinking about it in more expansive and creative ways and have subsequently asked me about other kind of some intractable problems that are really fun to solve them. Like just seeing those wins, seeing people get happy and the light bulb clicking on and then all of a sudden they're thinking about it creatively.
They're thinking about it in a totally different light, kind of endless possibilities as opposed to I know this software does A, B and C.
I love those examples of small wins and, you know, getting that excitement in people and getting them to that point where they're just running with the ideas is fantastic. But there is another side to it as well. Right? I mean, AI, you know, has some risks. And earlier you mentioned, you know, a lot of policy building that you're doing. So I'm kind of curious how you're thinking of and it's probably still early, I'm guessing, but how are you thinking of, you know, embedding, you know, some of the ethical thinking? You know, how do you get them to think through, like, oh, there might be some bias here actually or, you know, maybe I actually shouldn't be doing this.
Yeah. A hundred percent. Or like, hey, have we created some crazy antitrust risk with this company that's coming and presenting their product like it's a moat, but it's actually not. I think that it's seeing this AI technology and tomorrow's AI technology as producing some sort of new novel risk that's outside of existing information governance frameworks.
That's a mistake. No. No. No. Remember all the bad ways you could use your computer to violate the rules of ethics last year.
They're still there. Okay. Whatever you could do with that Google and that email to make bad stuff happen or disclose client information or, you know, get wrong ideas from the Google or from the Westlaw search. Those are still there.
It just that Gen AI tool is no different. And kind of, you know, saying like, yeah. Yeah. It's going exactly back to the culture.
We have a framework for evaluating information security and thinking about confidentiality.
And I've had clients who copy and pasting words into an online translator and being like, wow. So okay. Don't do that anymore.
That's, you know, it's not that's not a good practice. And, you know, this thing kind of getting people thinking, oh, woah. Hold on. Like, here's our acceptable use policy.
This is the proper way to handle data. These tools are no different. Is this approved? Can you put confidential information in here?
Yes or no? It's the same everywhere. If it's in Google, if it's in some website, and it's again getting people used to operating within this framework and understanding the risks associated with it. And so what are the limitations of the tool?
You know, would you go into Westlaw or Lexis, type in some natural language search two years ago and think it showed you all the cases?
Would you pull up one secondary source and think it's showing you all the cases on x? There's a name for that. It's malpractice. You know what I mean?
Like, that's not how to practice law. That's not how to operate in our common law system. You gotta dig. You gotta look for cite who's citing this?
Who's citing and it's always been this way. But at bottom, it's that, hey, there's a way to use these technologies. There's a responsible way. This has a certain level of confidence associated with it.
Some technologies, we need to validate that output. How do we do that? How do we validate Westlaw's output? Again, thinking like a data scientist.
When it tells us all the cases, do we trust that? Absolutely not. These are the steps you build. It's eighty twenty rule.
You know, it's gonna take you a whole bunch of time to build confidence in the results you got initially. And so it's just teaching that culture, evaluating these tools just like you would anything else. But showing them these parallels.
Teaching the methodology as opposed to this is some new thing and it's gonna hallucinate like, yeah, no kidding. But we need to build that, show them that with an information governance framework, with an infosec framework in a way that we're teaching methods rather than shiny new tool.
It's so interesting. I didn't even have to ask you the question, Todd, because you really honed in on something very valuable for all of our listeners, which is the how. How do you have lawyers understand the very delicate balance between becoming a data scientist, a computer scientist, and practicing law. And there is a very distinct bright line of understanding, and I and I think you really talk through a lot of that.
And it's you know, Jared zeroed in on it before as well. This language that you use about being able to build out this culture, I think it is really remarkable. And I think it makes you stand out, Todd, from a lot of other folks in this industry who are trying to do the same thing as you, that culture of inclusivity and how you are preaching and how you are building, you know, followers, I think is pretty remarkable, which makes me wanna ask you what I hope to be a very thoughtful question for you, which is when you look back on your career so far and you think about what's to come for you, Todd, what is the legacy that you hope to leave?
Yeah. That's interesting.
You know, what is the legacy? I guess it is a legacy of adopting these attitudes toward the technology, sharing the excitement is really, I guess, what gets me off the most. Having people see how cool this stuff is. There's something kind of so special about it and so personal about, you know, these new technologies that it's there for everybody.
It's equal opportunity in so many ways.
And the culture of tech as a whole, you know, we have seen these technologies released in a largely open source manner.
And, you know, CUDA is sitting right there. Anybody can use it and it's actually amazing. You know, it's anybody can use PyTorch. You could learn it in two hours and be building your own models and that sort of accessibility and that sort of, you know, you don't need a special card to play.
I loved it. I love tech. And so the more I can kind of turn people on to this or the more people that really kind of sit back and see, wow, the big picture, like, this is exciting. This is fun stuff.
I wanna be a nerd too, you know, to the extent that it I could turn people on to this kind of nerdery.
And this just it's, you know, silly and fun, and there's always something new. Like, the more people who kind of get in my boat for at least a second or to some degree, you know, I love that. Yeah. The more, the better.
Well, Todd, your enthusiasm is infectious. Your passion is contagious, and, unfortunately, we're out of time. But thank you so much for coming on today. This conversation has been an absolute blast.
Oh, thank you. I had a great time. I really appreciate it. Thank you so much for having me.
Thanks for joining us on BDO's Legal Tech Talk podcast.
If you're enjoying these podcasts, don't forget to rate, review, and subscribe for more episodes.
Head over to www.bdo.com for a list of all of our episodes, transcripts, resources, and how to get in touch with us to continue the conversation.
Until next time, this has been another episode of BDO's Legal Tech Talk.
Daniel Gold is an attorney and a Principal at BDO USA, where he leads eDiscovery managed services. He hosts BDO's Legal Tech Talk, co-hosts the NeLI Pod and founded Gold Standard Leadership.
More about DanielGet the next one in your inbox
Daniel Gold writes weekly on leadership, legal technology and eDiscovery.
Subscribe on Substack