Data-Driven Decisions in Legal Tech: The Power of KPIs

Oh, thank you. I'm glad to be here.

Thanks, Daniel. Oh, well, my background is in software engineering.

And then when I got into the software back in late nineties, the beginning of the dot com, of course, we were seeing that a lot of data being collected as software tools and all that. And I got involved in a project on doing the research on fuzzy logic. It's kinda like a subject back in the old days, and that was my introduction to data. And when I joined the legal industry early two thousand, building matter management softwares, that was the early time that the data sheet and the basically, the time sheets that I supposed to be the sheet paper was converting into electronic billing, collecting a lot of data, and then happened to be building softwares for the matter management systems.

And the first thing that come up, okay, we collect all this data. Now what? Of course, we do the analysis on the, you know, billing rules and check the codes and rates and all that stuff. But there's a ton of information in those UTPMS codes and the data that you collect. And that was the reason I got really fascinated with what you can do. At the same time, I joined the KPI Institute back in two thousand three, and I saw that metrics and KPI is being used across all industries.

And in the legal sector of the KPI Institute, there was, like, a section for KPIs with, like, five four metrics.

And so that was a time that I start getting involved with building dashboards and all that for the corporate legal departments and office of general counsels and bump bump into those cases where you create something and you see blank faces and not excitement, but you're excited and see the disconnect.

And that was the reason I got more and more, curious as to how this disconnect exists in the legal industry where we don't have it in the other industries.

That's a good question. I think that one of the disconnect I noticed that was we started normally with the data because that's easy. We collect it, and then we figure out what to do with it. But if you start from the end question and see what is it that we're trying to accomplish or what is it like the objective for the analyzing this data.

Let's say if you wanna do cost saving, are you planning to do shift work from some, let's say, partner job to the associate and things like that, then you see that then the value of those codes become meaningful. And then you start saying, okay. So these codes can mean something for me, phase of the, let's say, litigation case. Right?

But if you don't ask question and do the analytics, then the people that they enter the code, if they are not being asked, they are not entering it correctly. And this is like the human behavior.

So by putting those questions on upfront and creating the analytics, then you see anomalies in the code. And, actually, we did a paper on this. Then using those anomaly, you can figure out how to have the better coding. Now you ask about narratives.

If you look at an invoice and you see, let's say, three pages of narratives and you see the code, the first thing you don't look at the code, you look at narratives. So let's say twenty years ago, we didn't have machine learning and AI with the power of now. Right? Now what have the benefit is using the AI to read the narratives and match the code, make sure the codes are correct.

And if the codes are correct, then we can do the analytics on the code. So it definitely there's a lot of tools available out there that they do this, but absolutely, narratives plays a, a key role here. And also the other way around this exists. Right?

If you have a descriptive code, but narrative is very, like, one word or, like, very simple with a number of hours, let's say, four hours unlike research, you can do the reverse. We need more information about these four hours. What did you do and all that stuff?

Well, definitely, data quality plays a really key role in doing the analytics and also the KPI and decision making process. If you don't have a good data, decisions are not definitely, valuable as much. But at the same time, I wanna mention that a lot of people, they're getting to data quality. They wanna make sure the data is hundred percent correct. Right? And something in KPIs we do it is just the benefit that you get from a metric compared to the cost of producing the metric.

Right? So let's say if you're collecting data and there is a percentage of, like, mistakes in the data, and in order to fix that and bring it from eighty percent to ninety five percent, you have to be spent, let's say for the sake of argument, dollars ten thousand. Right? And then as a result, you create a metrics that is gonna give you maybe a thousand dollars benefit on decision making. It's not worth it to spend that much time on data quality. Now I don't wanna dismiss your question about data quality. I just wanna mention this because sometimes we get into the data cleanup so much that we lose focus on the actual what matters.

And I've seen this so many times that I'm gonna get back to I'm gonna open a tangent here, but I have seen so many times that we get so focused on make it like a perfect to the penny. Yeah. And we create a report and show it to the upper management. And they say, well, that's not important that much to us. Even the CFO say it's okay to have, like, one or two percent stakes. Right?

But we have other issues that we need to focus. Now data quality, I think one of the key things that I think is very important, again, going back to start from the end and then backtrack and see what are the fields and what are the informations important to you and make sure there are best practices for those. Let's say, do not give them a text field for data entry. Make sure it's a drop down.

Use code as much as possible and things like that. But data quality is one of the area that we're getting more promises on using AI to clean the data, which is something in the past it was very difficult to do. There are more and more data mining tools and data warehouse tools available that can actually automatically, let's say, if you were to enter Los Angeles and it's mistaken a spelling, automatically AI can understand the location even of the data fields.

Absolutely. I think that was one of the things that I would say is one of the hardest thing to do, but it's been tested in other industries. Let's say one of the method I'm gonna mention is like balance and scorecard from Kaplan. So starting from region and mission and objective, and then go to the initiatives, and then go to the, basically, mission basically, creating the critical success factors, and then to the KPI.

So you go and start and say, what is it important to our organization?

What are the initiatives that we are taking this year? Let's say, if you wanted to expand to Latin America. Right? The merger acquisition is the goal.

Right? That drives some of the initiatives for all of the practice areas within the legal department to align the metrics that they wanted to track toward that goal. Right? One of the things that a lot of time happens is a cycle of these implementations.

Sometimes we're sure some some companies, especially nowadays, right, the strategy and the alignment changes maybe in a few months.

So we can't have a silo of the business intelligence and KPI created from two years ago and deliver it now, and then the executives are gonna say, this is not our priority right now. So that's definitely one of the area alignments of the metrics and KPI.

I have a concept of, like, KPI graveyard. It's okay to retire some of these KPIs and just put it in a graveyard. Maybe later you can use them, but you need to shift it and always look at what it is important right now.

That is correct. One of the things that make a slightly different between the legal industry versus, let's say, transportation and aviation is the urgency and the real timeliness of the activities. Right? Let's say if you're in aviation, you have a set of KPIs working for some of them for, like, decades.

Right? That delay in the flights and all that. So immediately, you have to adjust things to make sure that customer satisfactions and all that being taken care of. In the legal, the KPIs may if it's a lag of one month or two months, it's okay.

It's not like real time. But still, we have to make sure that we stay aligned with the c level and actually with the general counsel. But general counsel makes sure that when they go to the board, ask these questions. What is it like the important of our business?

And, of course, CFO is always gonna ask for cost saving. Right? But maybe, you know, as I mentioned about measure acquisition, the CEO may wanna have a different agenda.

That's a very good question. And, honestly, I haven't seen many of them done in the corporate legal departments to that degree. It's pieces of this, but one of the thing I've seen culture wise, yes. First of all, one of the thing is very important is get the leadership buy in and get them involved in this process.

So it's not just they come in and just say that, hey. This is important. We just need to get them involved and even the to the process of defining the metrics and KPIs and all that. That's very important because if the leadership is involved, and I'm talking about, let's say, the CLO or general counsel, then the managing counsel in different department, they are involved.

And it's getting something it's part of their basically goal that, hey. In our department, this is what we're gonna do. Let's say, if we wanna increase the customer satisfactions in an insurance company, for instance. Right?

That is not just, like, one department that they are managing one. It's a holistic area. But defining the metrics for each of the department, that's the kind of, like, a part of the KPI program team that they sit down and look at the success factor and translate the success factor based on the smart rules, specifically what it is that the activities you're doing is gonna contribute to the success factor. And as a group, all these success factors, operational success factors are gonna contribute to the mission and vision.

So I think it's just a it's a group activity of all of those. But tactically, in some areas can be done. Let's say, if the goal is to do the saving or maybe reduce the that's something very common. Right?

For instance, the OKR is another term that is here is very useful. So OKR is defining that we're gonna reduce the cost by twenty percent. Right. But that OKR by itself has multiple KPI in it or multiple metrics in it.

So that's another thing that there's a lot we can talk about it, but some people, they get to the focus of, oh, okay, that means that we need to cut the cost by twenty percent no matter what. That means if I have a panel firm, critical firm, and they are doing a great job and they are very critical for us, they have to go twenty percent down. Right? Which is not the case because in some cases, their reputation and the risk is another area that we have to protect the company, not just by cutting costs.

The KPIs, sometimes I use this metaphor of driving. Let's say you drive a car and you have a dashboard. Your OKR is the map that you have a destination you need to go there. And let's say you're going to either a field trip or you're transporting a large cargo to another location.

Right? The OKR is telling you this is the location, this is what you need to get to. The KPI is your speed, and we have another concept, KRI, key result indicator, which is basically it's like KPI buzz looking at the past. That can be your MPG, mile per gallon.

Right? And the reason between the difference between the speed and mile per gallon is speed is executive function that is under your control. You control it to get to your destination based on your OKR. Because OKR is telling you that you need to get to the destination within the, let's say, two, three hours.

So if you go too slow, you're not gonna get there. You're gonna be safe. And if you go too fast, you're not gonna be safe. So the speed is the KPI.

And, of course, in the organization, you have so many KPIs. Right? But that's the KPI. Some people, they mix it with o the KRI, which is basically an MPG.

You don't check your MPG every minute. Right? At the end of your trip, if you're transporting a cargo and your driving behavior is not good, your MPG is gonna be high, and then your boss is gonna say, how do you drive and all that stuff. Right?

Still important. So the way that I explain is KPI is for the executive c level, and the KRI is for the board. They look at your performance in the past. Right?

And OKR is the goal, as you mentioned, yearly, sometimes, like, quarterly, is the objective that this is what we're gonna accomplish. Now the art is to define certain KPI, and we don't have more than five normally, that they contribute to your OKR effectively.

And that those are the one that is a kinda like the team needs to sit down and figure out what are the activity that has maximum impact towards your OKR. Like, for instance, one KPI can be AC of your car. Right? And the AC can go up and down.

But how much it contributes to your destination? Of course, if it's too hot, that's gonna be not comfortable. But still, we can't drive. Right?

I use this metaphor because there are so many things that in the legal department or the department can be mapped to this basic metaphor.

That's a good question. I think I hate to say this, but this depends to the organization and whether what is their mission at the moment. Right? If they are focusing on, let's say, growth and, like, at this point, their competitiveness is more important.

But I think that at the same time, it has to be a balance between them. But one thing I can tell you about the doing the decision between this, a lot of time I see that organization, they don't do measurement. It's just because they trust too much on their intuitions and gut feeling. And they think that if they are in the industry long enough and they have seen it enough, they don't need numbers that much.

And it can be done based on evidence and things that you see on the table quickly. But one of the thing I always say, we are using measure and metrics every day in our life, right? If somebody's getting these, like, glucose meters are getting very popular, right? And that's something that we it's so important and they just wanted to understand their body, right?

And then they measure it, they they control it and they study about it, right? And the same thing with the they go to gym and they measure everything. Right? But then when they go to the office and they deal with a big organization, millions of dollars at stake and risk, they just go with the gut feeling.

And that's something that it requires change the culture and perspective, but also it requires thinking from the top. And then one of the thing I wanna just add to this is the transformation of, let's say, general counsel to CLO, getting legal department transformed to the business unit. That means that the CLO become part of the club of other c levels as far as the business, and the language of the business is numbers and KPIs.

So if they wanted to speak in that language, storytelling using number, then they come back to their department and say, we need to have this done, like other department they are doing. If you look at hundred years ago, right, finance, CFO didn't exist. It was accounting.

And then the finance become a key role partner of the c level because now finance come in and is strategically planned for the future, not just looking at the book and say, you're over or under budget. Right? And the same thing with IT. IT used to be support of technology, but the CIO now and CTO, they are a strategic planner of the future of the company using AI and things like that. And the same thing is with the CLO, and I think that the CLO, they need numbers and measurement and KPIs like any everybody else.

Actually, I did a presentation at CLOC, twelve mistakes of implementing KPIs and myth, and I would be happy to share that with you. But top ones, I would say, not aligning metrics with the mission vision and all that. And the other one is measuring everything and everything that you have, like creating noise for the management and missing the important.

I would say the other one is just treating the KPI program as a one time project. A lot of time, we define a project and say, hey. We just need to create a dashboards and metrics and all that, and they just IT or whoever, they did a jambal k away. Not knowing that two months later, the requirement changed.

You need a new set of KPIs. You need a set of measurements. And then people are stuck with a dashboard that they look at like a gallery. They don't use it, and I call it, and sometimes it's like that little yogurt in the fridge, nobody touch it and it's there and I normally say, hey stop delivering that report and see who scream.

And sometimes two months later nobody complain because nobody uses those reports.

Thank you so much. I learned a lot. Thank you. Thanks for joining us on BDO's Legal Tech Talk podcast.