Measuring Success: The Metrics Behind Legal Operations

Thank you so much for having me and great to be here.

Been at DocuSign for about four years now. Been in legal ops for most of my career, other than what I like to call the scenic detour over to product marketing. So I did that for about two and a half years. But, otherwise, I've been in the legal ops community for well over a decade now. Started out my journey at NetApp where I met the wonderful Connie Brenton, which is why I'm here today. It's once you're in her ecosystem, you just never leave. But I'm very grateful to her for introducing me to legal operations and this amazing field that I so enjoy today.

At DocuSign, my team does kind of all core legal operations type work. So whether that's managing our vendors, our billing, managing our metrics program, knowledge management, training our our team members on everything from how to use the billing software to all the AI technology that's coming out today. We do kind of all that core work. And then the other part of our work, because it's really unique to DocuSign, is we spend a lot of time doing DocuSign at DocuSign.

So that's really about using our own products in house, providing feedback to the product team about what we like about the product, what we don't like about the product, what could be better, how we could really help the legal team adopt more of our technology. And that's one of my favorite parts of the job is getting to work with the product team and really influence the direction that our DocuSign I, product is going in. So that's the unique part about my legal ops role that is definitely atypical, but probably the most fun part of my job.

Yeah. Absolutely. So, you know, change management, I think, is the hardest part of the job. You can build the most beautiful workflow.

You can have the best technology out there, but if people don't use it, then you haven't done the job. Right? So I have a couple of strategies. One is really understanding what the pain points are that the legal team members are experiencing.

So if you can really understand what is their primary pain point, like, you might think that they need better redlining software.

You just, in your head, have that in mind, and then you go talk to them. And they're like, actually, the hardest part of the job is just getting the sales team to email me the request instead of Slack me and call me. So we get feedback like that, but really digging into what is the pain point and then solving for that pain point. So that's one of my best tips. And then the other one is finding some champions.

So it is far more effective if I can get one or two kind of champions within different parts of the legal teams. Let's say the commercial legal team. I have my best buddy over there, Peter. And if I can get him on board with whatever it is that I wanna roll out, he can then be a champion for the rest of the commercial legal team.

And that's just been so much more effective than me going in there and being like, I built this new thing. You're gonna love it. New technology. I've just paid a bunch of money for it.

It's gonna be best thing since sliced bread. That's not how it works. Right? It's like getting a champion, a couple of champions to be kind of those early beta testers, help build the product or tool or functionality or whatever with you.

And then they go and they be the evangelist for you.

And you're also doing what is required, what's necessary within almost every single legal department. And that is, and also in law firms. And that is, you need to be able to have someone who's going to champion your efforts.

Yeah. We call ourselves customer zero. So we get a lot of the kind of tools and new features before it goes out even for a beta test. So we test things.

We're part of that really kind of early product development life cycle before it goes out to beta. So the product team might make some changes before they even send it out to beta with customers and then make some more changes before it goes out into kind of GA. But the other really cool thing that our legal team has just started doing is what we call the legal education series. And we actually sit down with the product team on a weekly basis and spend an hour just talking about some aspect of legal.

The other day, we spent an hour talking about hyperlinks in master services agreements, and this concept was, like just totally blew the product team's mind. They'd never thought about what happens when you have an embedded hyperlink in a contract. Right? So we get down into those things that you think, how could you possibly talk about that for an hour?

But, wow, it's, like, way more complex when you actually dive into it. And I think that's been really cool because that's even kind of before product has even rolled out kind of an alpha or a pre beta kind of thing for us to play with. We're really providing the feedback of this is how we actually do things. These are our real pain points.

Now build something to fix that, solve it. It will be broadly useful for the rest of the market out there. So that part's been really, really fun.

Yeah. No. I think measuring is super difficult, but we absolutely do it because how else do we know if we've been successful?

So a couple of different ways we go about that. One is I'm thinking of a use case most recently, which is AI assisted review, our functionality around that.

We've recently rolled that out as part of our product suite, and our team say that they save kind of, like, at least fifteen minutes per NDA review, thirty minutes to an hour per master service agreement using that kind of tool. And that's a great story for us to go out and tell the market that our teams have now saved time. That time that they used to spend reviewing redlining contracts because AI is now helping them do that. Another example that I love to give is that we used to have a person on my team who spent a huge chunk of his time reviewing and validating metadata on contracts.

So we would, you know, we'd get a final executed contract. He would go through and make sure that that contract got tagged with who the party was, what the expiration date was, execution date, kind of all that data, and make sure that was filed away in our contract repository.

I now don't have a person on my team doing that anymore because we have navigator, which is our AI enabled repository that extracts all that metadata automatically. So I no longer have someone on my team doing that because I have a really high level of confidence that the data that's extracted is accurate, you know, above ninety percent accuracy. So I freed up good chunk of this resources time to not do that anymore, to go work on other things. So those are a couple of the metrics. Another example is our commercial legal team used to be almost twice as large as it is today, And we're not doing less volume, so there's not the volume has not decreased.

We just have fewer people that are able to support similar volume of contracts because of some of the tools like AI assisted review, bring up a lot of their time.

Yeah. It's a mixed bag. I wouldn't say every single person on the legal team is fully sold that AI is, you know, gonna solve all their problems. But I think it does help that we work at a company that truly has embraced AI incorporated into our product.

Perhaps the trust is more from, you know, that we have built over time a level of confidence in using the tools and that they've seen the progression of you know, the first time they tested the AI wasn't that great, to be honest. This is a while ago before we launched it to customers, but they were really involved in that testing process.

So, you know, we had our our lawyers in there reviewing the AI output. It was pretty bad at first. Then they reviewed the next iteration. It was a lot better. Then it got a lot better. So I think it's really building trust over time.

But because they've been along on that journey of watching the product grow, I think that's really helped with building the trust. I'd say outside of the contract space, it's, you know, probably a little bit more mixed in terms of our adoption. I think people are finding great kind of day to day productivity gains. They pop something in to have it summarize. They're dropping bunch of documents into notebook l m and seeing what gets spit out, but it's fairly ad hoc adoption across the team at this point. Some folks are just more resistance to adopting new technology in the first place, and some functions just aren't as right for AI use yet.

So it you know, like, our employment legal team, they're not quite ready to dump everything that they've got into AI. That data is sensitive. They're, you know, they're definitely concerned about that. Our commercial legal team, little more open to it.

Our product legal team is like, get me more AI. I cannot get enough. But they just tend to be the folks who are, like, wired that way. Right?

They're, like, very forward looking. They're always thinking about what's next in the product space. So those folks are totally on board, and they're they've been the most fun to work with.

I'll go back to some of the concepts maybe that I've mentioned and extrapolate them out, which is, you know, if you could think about really identifying the pain points that someone on the legal team is experiencing, and then you come with that solution. Like, doing a lot of listening, I think, is super important. Right? So going around, talking to people, really understanding what is it about their job that's so hard.

You know, legal operations folks are I think of us as, like, problem solvers, or at least I certainly think of myself that way. Like, I identify the problem, but I don't just identify it and point it out over there. I actually go solve it. But that requires a lot of listening.

So going around, talking to a lot of people, really understanding what those pain points are, and then, you know, starting with just something.

And and so that's really stuck with me over the decade plus. So that's certainly a concept that I would extrapolate out. And then the other one that I also mentioned earlier, but I think is so key is really finding some champions outside of just legal operations. So you have to have a champion over in the commercial legal team and a champion in the privacy team and a champion in the employment legal team who are all gonna partner with you.

And I've tried to do that formally where I've created, like, a legal ops working group. And I'm like, okay. I'm pulling all these people, and they're gonna be so helpful. We're gonna have this group, and everyone's gonna meet regularly.

And I personally didn't find that very effective. And so I've taken more of the approach of building those relationships kind of with these individuals, pulling them in to partner with me on products, and then kind of sending them back to their organization. Like, go be the champion for this change. Right?

Here's the new process. Here's the new way we're gonna do things. Okay. Now you go tell your team about it.

And I found those two kind of techniques to be really, really effective.

To me, they've done such a great job of building a community, and that is what makes all the difference at this conference. So I come here. I see lots of people I know. I'm, of course, meeting new people, like the two of you, which is so wonderful.

But a lot of it is building this community that extends far beyond just this event. So sometimes you go to a conference, you learn a bunch of things, you see a bunch of people, you go off about your life, and life goes on. You never think about it again. What is really unique about this conference is because of the emphasis on community and relationships, you walk away with a number of new contacts that are gonna be there for you in two weeks when you have another question about, you know, a challenge that you've run into that you didn't know about two weeks ago.

And And so to me, that has been the huge value is it's not even about just what I've learned what I learned today or tomorrow or this week. It's about the relationships and the people that I now know that I'm gonna learn from in two weeks, in a year, then be able to pick up the phone and call and say, hey. My GC just came up with this wild idea to do x y z thing. Have you done this before?

Like, I could text someone and say, you know, I'm planning a leadership off-site next week. I need a great icebreaker. Like, it could be a silly thing like that. To me, that's really the biggest takeaway from these events, and it's not even so much what did I learn sitting in a session.

Although they do such an amazing job putting together content.

Definitely kudos to particularly Jeff who spends a lot of time really curating some amazing content.

Great to be here. Thank you.