On this episode of The Geek in Review, hosts Marlene Gebauer and Greg Lambert interview Thomas Suh, Founder and CEO, and Ken Block, Senior Account Executive of ⁠LegalMation⁠. The company provides AI-powered tools to help litigators automate repetitive tasks and work more efficiently. The conversation focuses on LegalMation’s products, overcoming resistance to adopting new legal tech, and predictions for the future evolution of legal service delivery.

Suh provides background on founding LegalMation about seven years ago to help streamline the “scut work” litigation associates spend time on. The flagship product automates drafting responses to lawsuits, discovery requests, demand letters, and more by leveraging a firm’s historical data. LegalMation initially built an automation tool internally at a law firm before deciding to spin it off into a standalone legal tech company. The product found an early champion in the form of a corporate legal department interested in licensing it. Today, LegalMation serves large corporate legal departments, law firms, and insurance companies.

Suh and Block discuss common roadblocks to adopting new legal technology like lack of trust and skepticism. Suh notes the importance of identifying the right use cases where efficiency gains matter most. For high-stakes litigation, efficiency may be less of a concern than for high-volume routine matters. Corporate legal departments are often early adopters because they are focused on efficiency and supplementing personnel. Law firms still incentivized by billable hours may be warier of efficiency gains.

For the YouTube Viewers, Block demonstrates LegalMation’s Response Creator tool for automating drafting of responses to complaints and discovery requests. The AI leverages a firm’s historical data to maintain proper tone and style while speeding up document preparation significantly. Lawyers can still review and edit the AI-generated drafts before finalizing. Suh explains that because the AI relies solely on a firm’s data, it maintains consistency rather than attempting to generate random creative language.

Looking ahead, Suh predicts that the litigation process will become more modular, with different firms or providers specializing in discrete phases rather than handling a case end-to-end. Block emphasizes that younger lawyers expect to leverage more technology and are unwilling to slog through repetitive manual tasks, which will force law firms to adapt. Technology stacks and automation will become selling points for recruiting top young talent.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Contact Us: 

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠

Threads: @glambertpod or @gebauerm66

Voicemail: 713-487-7821

Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠Transcript

⁠

Marlene Gebauer 0:00
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert 0:07
And I’m Greg Lambert. So Marlene, a couple of weeks ago, I mentioned another case where someone submitted the brief to the court. Did they use the generative AI tools in? We also referenced, you know, again, once again, this is why we can’t have good things in the in the legal industry,

Marlene Gebauer 0:28
because I remember Well,

Greg Lambert 0:31
turns out that although the facts behind this somebody actually did submit a case, a brief that that had some hallucinating citations that were that were made up. I said it was the 10th circuit court, the federal court, and I was I was wrong is actually the 10th Court of Appeals. Here in Texas and Brazos County,

Marlene Gebauer 0:55
you’re partially correct. I was perfectly I got

Greg Lambert 0:59
I got the 10th part, right. 30%? Yeah. Well, apparently, I sent some people up in Denver scrambling, when they heard that at the at the actual 10th Circuit Court. And they reached out to me, and finally we figured it out to wasn’t them. It was my mistake. So sorry about that. I’ll try to be a little more careful the next time and not rely upon ChatGPT too. Write all my introduction there.

Marlene Gebauer 1:35
Well, thank you. Thank you very much for catching it for us. So we do, we do apologize for the mistake. But you know, enough of Greg’s apologies for past episodes. Let’s jump into this week’s episode with a couple of great guests from LegalMation. We’d like to welcome Thomas Suh, Founder, CEO and board member of LegalMation, a legal technology company focused on assisting litigators with AI tools. And Ken Block Senior Account Executive at LegalMation. Thomas and Ken, welcome to The Geek in Review are having us. Thanks for having me.

Greg Lambert 2:08
So I know we go back and Ken and I go back a good ways. But Thomas, do you mind just kind of starting us off by giving an overview of what LegalMation does, and a bit of the history behind why you started the company?

Thomas Suh 2:27
Sure, to keep it as short as possible, being ex lawyer litigator that’s asking for a lot. But so we are focused exclusively on automating a lot of the no other way to put it a lot of the scut work that litigators and legal professionals in litigation, do the cutting and pasting some of the digging up fire samples and information to respond to lawsuits or discovery request or demand letters, all those things that, you know, no one no one go to law school dreams about doing this not working. And the reality is that, that we all have to do that as the associates. And it was par for the course. And something that of course it can bill for obviously, you know, your your I don’t know what the rates are now for associates in New York, but I presume $500 per hours by not out of the out of the realm. Whether you’re in trial, or whether you’re cutting and pasting, you’re still charging for the same. The same rate. And so there was really very little incentive to do anything more efficiently. Well, we decided to back it was managing a law firm before this. And we decided to experiment with AI. This is about the the initially that the deal was drawn up about seven years ago. And the one of the one of the moments that really spurred us into looking into this was one evening, I’m walking down the hall and I see one of our associates working late at night. Now this was probably 11 o’clock at night or so. Some some attorneys here may say Well, that’s nothing.

Marlene Gebauer 4:25
It’s sorta late, maybe

Thomas Suh 4:30
Anyway, let’s just see what’s coming late. And I asked what are you what are you doing? And of course, he said, Well, I’m working on Discovery responses, and we have a case and this discovery we’re working on. So I said, Well, when is this due? And he said well, it’s you know, we’ll respond by tomorrow. And I said tomorrow, you still working on this and the client has to review most likely and you have to maybe go back and forth a couple times. And his response was very telling. He’s responsible Well, I hate doing this part of my job so much that I procrastinate until the very, very end. So I can’t say that was the moment, obviously, my co founder, James, really, ultimately came up with the idea for the company and some of the tasks that we’re going to automate. But that was one of those moments where I remember going back now, and asking ourselves, Well, that’s the kind of stuff that machines should be doing. Computers should be doing so that we can focus on higher value work. Everyone talks about it, but no puts into practice. So when you start to experiment ourselves, and we created a tool within the firm, to automate some of those early stage litigation tasks, so that we could elevate a paralegal and legal state attorneys and associates to do other type of work. And we had no intention of spinning off a new company, we’re lawyers, and we’re just doing fine. And there was no need to jump into a completely new sphere, where we’re not going through any sort of midlife crisis or anything like that. But we present this to our, one of our large corporate clients. At that time, they’d love what we’re doing. And we asked them, if we were to create this for you, would you be interested in being one of our licensees? If we were to spin it off as a separate company, of course, they said, We will be your partners right off the bat. And that was about six years ago. Since then, of course, now we’re in multiple large legal house departments, large law firms and insurance companies as well. And the rest is history, as they say, award, you give you the long winded answer. Very simple question. Well, here it is.

Marlene Gebauer 6:43
I was definitely a point six. Yes, before we jump into the LegalMation, the product, let’s get a little philosophical and discuss the bigger topic of adopting new technology in the law firm environment. And you know, this is, this is a big topic, a lot of people are talking about it, particularly with some of the Gen AI discussions. Now, I know you’ve experienced some common roadblocks when it comes to lawyers adopting new technologies. So you know, lack of trust, skepticism, those, those are awful, often big hurdles. So can you can you expand on why those mindsets are so prevalent in the legal industry? And what’s worked well for you in building the trust that’s really needed to overcome that skepticism?

Ken Block 7:39
Yeah, it is an interesting question. Because in my experience, and I’ve been selling software in the legal space for a very long time, all the things you mentioned are part of it. There’s skepticism, there’s we’ve done it a certain way for a long time, why should we change that? There’s the I know better than you, because you’re not a lawyer. And I am. You know, there are other things that go on in the firm that related to the partners that come into the back office, whereas there’s some fears and skepticism in there that we can talk about actually, at length, it’s a very significant part of it. And I think what you find is, it’s very personality based, because there are camps, that will never change, they’re not going to change. Some of it is age based, they’ve just been doing this for so long. They’re just not in any mode, the change they’re doing well, they’ve done well, you know, nothing’s gonna change, it would be a risk to do that. For them. It could be a risk to their income, it could be risk, their lifestyle, I mean, all those things are real, these are all human things come into it a lot. But then there’s another camp. And this is sort of the challenge. And where we’ve had to go find those that say, Well, yeah, life is comfortable. But yeah, we need to advance, you know, we need to do new things, we need to be thinking the, you know, if it’s not broke, break it philosophy, that those are kind of the people that we’re looking for, I’ve looked at my entire career, you know, how do you? How do you say, Yeah, this is all really good. But you know, somebody’s probably going to do it better than us if we don’t look at other ways of doing things. So I think that the challenge for us is finding those folks that are open minded enough to be willing to take a risk to some extent, but but, you know, a really controlled risk, which is I think, a lot about what we’re doing. It’s a very controlled risk.

Marlene Gebauer 9:20
Do you have a sense? I mean, just from your experience, do you have a sense of, you know, are there certain personas? Are there certain sort of types of folks that sort of fit that? You know, you know, if it ain’t broke, we need to check. You know, we need to try and break it. Are you are you finding sort of groups like any similar characteristics that you could share?

Thomas Suh 9:47
Yeah, I mean, if I were to cluster our client base, between the large corporate or insurance in house staff versus law firm I’m still very different. Because there’s different incentives on both sides. I think I’ll tackle the law firm sort of Persona first and I will give this okay, if I give shout outs to two folks, because there is certainly one person that I like to use as example it does Ron Chapman over at Ogletree Deakins, you guys may or may not know, Ron, he’s actually in Dallas. He is a very forward thinking, business minded attorney. And, you know, I hate to say that’s a rarity in our industry, because he thinks about the big picture. He thinks about the the, sort of the benefit to the client, which ultimately transfers to a better relationship with the client and longer term relationship, doesn’t worry about what the implications of efficiency tools can have on the rich in the short term, where he may lower some billable time where he may actually create more efficiencies, which go against the point six, point 1.2, as money noted, so that is a type of person that I think, has been quite successful. And it shows because, you know, I do know, for a fact, he’s a very successful attorney there. And he’s got great clients. And that’s the type of lawyer that if I were to, if our client and I had a case, a complicated case, in employment, obviously, I would go too wrong, because not only is he is he thinking about my business, or, or efficiencies, but that also tells me that he’ll think about the best way to solve my problem. And that’s what I really care about as a client. So that’s the person on the law firm site. In the opposite of that, in fact, Canada, we recently had a had a conversation, of course, I will not mention, the person didn’t say

Greg Lambert 11:59
it’s probably best, you don’t mention this one.

Thomas Suh 12:03
Yeah, on that one, I will not. But it’s not this is probably more the rule than the exception in this particular case. And this is a person who’s been very successful. The managing partner of this office, that was part of a larger office, and when we initially showed our tool, as a benefit to his practice, and of course, his clients. And we did a full demo of his he saw the outputs, and he was very impressed. He they have any questions about the substance. But immediately, he asked questions about well, but a bill for my time, my associates bill for the time and and how is this gonna work? So immediately, he went to where most lawyers go, which is, hey, I’m paid for my time. Bottom line. Right, exactly. And of course, you can sell you want and say, Well, you know, you could do more, or you could do other things with your time and so forth and so on. Don’t you have enough to do with your time as opposed to doing this kind of work? But I think folks are built a certain way, if you don’t see it right away. If you don’t see the benefits of a tool like this right away, I think it’s hard to change that behavior, that mindset. But like I said, I failed. He’s probably the rule, not the exception, whereas Ron, it’s certainly the exception. And but I think it’s going to change, it’s going to start changing as people realize that AI is not so we talk about AI now, still, in many ways as a, as a novel idea, or something that will wash you look into that. It’s a new tool. No one talks about, do we research online or internet? We’ll talk about oh, my mean, are you on the internet? No one says that today but remember

remember guys back in, you know, the mid 90s When Netscape and just started coming out, you know, people thought, Wow, you can like look at stuff and you can do some, some look at cool stuff. And you can buy stuff you can actually like do transaction online. People had that was such a novel thing. I would say, you know, give it a few years, no one’s got Hawk about AI, or automation or efficiency, they just kind of talk about it as Oh, that that’s just something that’s just part of our workflow. So you will change. I don’t know if it’ll take a generation. But you know, there is a saying, right, in the law, firm business, change will come one retirement at a time. And I think that’s true. I think we need sort of a fresh view of things. And, you know, we’ll eventually get there. That’s the law firm side, the the corporate side, I think it’s a bit different because now corporate, you know, you saw some of the retail announcements of a of a some insurance companies having to restructure and start laying off some professionals and that sort of thing. Well, what better way to To, to increase your efficiencies, then to use tools and technology to be able to supplement now the lack of personnel, but more so provide better value to your customers and to your to your buyers. So it all sort of meshed together and it’ll come together. There’s always some casualties of automation. But that also tells us that there’s new jobs that have been created and so forth. But not that may be a separate topic for another day.

Greg Lambert 15:31
I want to dig just a little bit deeper in because this kind of relates back to, you know, you’ve got this the spectrum of the way that people approach the problems. And then kind of looking at, you know, the typical story that we hear of a lot of tech companies, as they, you know, identify a discrete problem, but I want to twist it just a little bit. And that is, you know, sometimes the change, especially within a legal department or law firm starts with identifying what the problem actually is. And a lot of times, primarily, it’s a partner, or general counsel, that, you know, they are facing that discrete problem, and you finally find a way to identify that to classify it, to explain it to them. And I’m sure that’s something that you work with, you know, one client at a time. So I want to ask if you can give us kind of a specific example of a problem that you’ve sat down with, with one of your clients and talked about using the technology to solve that. And then how do you kind of build the, you know, the understanding of here’s the problem first, and then look at the technology to say, okay, ultimately, we can apply the technology to solve this part of the problem. You have you have some examples that you can use, maybe maybe can you want to?

Ken Block 17:12
Actually, I want to take your question and just sort of add a little twist to it also, which is, I think, in our case, what I’ve learned, you know, I’ve known LegalMation, for a long time been with the organization for several months. And what I’ve learned in a very quick period of time is that a vast majority of the people we’re talking to do not even realize that this is a problem, specific area that we’re tackling of automating complaint and discovery responses. For the some of the reasons we said earlier is that people just this is the way we do it, we’ve always done it this way, that’s it’s those people in the back office that start the process, like it’s just not even part of their mindset, to even think that this is a problem. And I think one of the challenges that falls on us is to really explain to them that, you know, it is the old way of doing things it is causing you to be slow the example Thomas gave earlier of his late night associate there, they’re just not thinking that there’s anything you could even do about it. And then you know, so we have to get their attention that there’s this thing that you’re doing that is it is it is backwards. There are better ways that you can do things, we have a very simple solution that is extremely powerful, that you can use, and you can implement in a very rapid period of time. That can be beneficial to the organization and to the clients to you whomever. You know, that’s sort of the first challenge that we have is to even get them thinking that way. And I think a lot of folks, and I saw it at the booth, you know, last week, I mean, there were some people that came by and kind of looked at it for a second. And then I think walked away and blew us off. I mean, they just you know, yeah, okay, right. But then there were some others. And there was there was one gentleman in particular that I’m thinking about, who I don’t think he Thomas, he really realized it was a problem was actually an associate from a large law firm came by our booth. This is just a really interesting story. He looked at the back panel of our display, which was very simple. I think it said three things on it. And there’s nothing to it, but the word response caught his eye. And he literally did the whole card tune thing with the head. You know, flipping everything. Walked out, I kid you not I mean, that’s what he did. He walked up to us, he asked us what we did we explain what we did. We gave him a demo. And as we’re going to do a demo in a bit. It’s it’s all of five or 10 minutes to really get a sense for what we do. He looked at us and said, Why isn’t there a line at this booth is the greatest thing I’ve ever seen. But he wasn’t even thinking that this even existed till he walked past our booth. And then once he got it oh my goodness at any explain. I proceeded to explain the entire here’s the client I’m working with and here’s the work we’re doing. He gave us you know, he spit out the entire thing to us in two minutes flat and Thomas I looked at each other and said, Oh my goodness, if this guy doesn’t get started with this next week, there’s there’s something cracked in him. But the light bulb just all went on. I mean, it was just it was just fascinating. thing to watch happen, but that just, you know, that doesn’t happen every day of the week, we had to fly down, you know, to Florida, in the back of an exhibit hall, you know, to see that happen. So that was kind of a long answer to get there. But, you know, once we get them in there, maybe Thomas, you can take it from here, you know, once we get them, you know, rocking then, then then good things happen.

Greg Lambert 20:21
And I want to kind of see if I can direct us in a certain way. And you get this associate that’s, that’s doing the work that’s in that’s in the trenches doing this, that again, that’s how he’s always done it, how she’s always done it. And then all of a sudden, the light bulb me like you said, the light bulb comes on. And so that may may work really well to for that light bulb to go on on the associate. How do you then catapult that into talking to the powers that be? To explain to them this is this is a problem. This is you know, and here, here’s one way, you know striving your clients crazy, whether you whether they’re saying it or not. And here’s a solution to this issue. How do you get it to that next level?

Thomas Suh 21:15
That’s that’s a great question. The way I would ask her is the, the problem has to be recognized by the powers to be otherwise you’re you’re never going to get through. For them to even listen to what you have to say in this particular case, just piggybacking on what Ken said, this firm was handling volume, they started getting it is litigation arbitration cases. on consumer arbitrations, though there was high volume I, we have no idea that this many cases were being filed against this one particular client of theirs. And it turns out that because they wanted to earn the business, they were handling these on a flat fee basis. Now, when I hear flat fees, obviously, then the opportunities open up, because the law firms then should be thinking about how to make things highly efficient. Because if you were to cap your fees, or you you agreed on the flat fee with your client, it’s up to you to lower your operating expenses. So that you can you can increase your profit margins, that that should be a no brainer. In this case, and that’s what he was this associate knew that this was a flat fee. So the moment he takes this, and then in fact, we already have a follow up call set up already for next week, because he’s already spoken to the higher ups. And of course, think of it, you know, they already have a flat fee, the other client, now if they could lower, let’s say, their actual costs of the associates and servicing costs by even just 20% 30%. That’s directly to the bottom line. And of course, Equity partners know this pretty good person who may be on the on the bonus structure, I don’t know what the structure is, but the partner who is benefiting from this. That’s the one thing that lawyers understand is money and profit, and PPP and all the other good stuff. So I think the right use case has to be there, the incentives have to have to be there. But if you are a firm that handles very bespoke, you know, high end price fixing antitrust cases, efficiency is not the first thing on your mind, because your clients are coming to you for their bet the company cases, I don’t think they’ll care about you saving another 10% here 15% When they’re when their company is at risk of dying. So that’s sort of, it’s important to recognize that AI is not for everything, and it shouldn’t be for everything it is for the right use cases and the right practice areas and the right people as well.

Marlene Gebauer 23:46
Yes, so this is a really interesting conversation, I have a couple points to hit on based on it. So can you are saying that, you know, this, this associate that came by, didn’t even know that a solution existed? Until, you know, serendipity happened, and you happen to walk by and had the aha moment. And, you know, we and firms we before everybody, you know, we don’t do a great job of informing our, our people about solutions. And you know, before our listeners start throwing tomatoes at me, I know we try. It’s like I know, we try we give them food, we go to practice, we go to practice group meetings, we do it, you know, on onboarding, we try and get in front of them. But it’s that that never ending struggle that you know, this is not their job. And this is to support their job and to try and get that attention. So have you know, have you seen sort of good ways to do that or ways that you have seen that work?

Greg Lambert 24:57
Very easy question here can Well,

Ken Block 25:04
no, because I keep thinking about right answer i No, no, no. But I think that the analysis isn’t complete. Because there’s two parts of the story, right? I think you have to talk about both sides of the equation. Before you answer that question. There’s the you know, Attorney associate and the partner side of things that we just were talking about. And then there’s the part of the firm that does a lot of this work, because they’re also part of this equation. Significantly, in many cases, we’ve found they can make or break our opportunity in a heartbeat. And they can do it for very, I’m going to use the word selfish, but I don’t mean that in a very nasty way. But in realistic way, you know, there’s things that there’s most folks do either by perception or reality that they think that their jobs could be at risk, they think that the, they’re doing a lot of overtime work in this particular area. And they don’t want to lose that overtime, because that’s a significant part of their incomes. So, you know, and we’ve seen, you know, I know, one of our we did a trial recently for a law firm, where at the beginning of the process, the paralegals are extremely excited. And yet, somehow over the course of the next two or three weeks, they barely used it. And the only conclusion we come to at the end of it is that somehow they perceived all of a sudden, hey, wait a minute, what am I going to do with my time, if all of a sudden these discovery responses take, you know, an hour instead of four? What am I gonna do? You know, whatever I’m getting over time, you know, am I going to lose that income and can’t blame them, by the way, I mean, perfectly understandable. But it’s also a very big part of that equation. So when we, you know, go to onboard new clients, or we go to, you know, close an opportunity they have, there has to be comfort on both sides that, you know, from the management’s perspective, I’ll define management as partner associate, you know, that they’re going to get something good at this, but they’re also not going to create an enemy of the folks that support us every day of the week. You know, so they’re, it’s a balancing act. I’m not trying to be coy about how he answered. But you know, each situation we walk into, we have to figure out who the players are, and who has at the end of the day real power. And the example I just gave you, the real power was actually the folks that were the paralegals and legal assistants had all the power. And they put the kibosh on it ultimately. Right.

Marlene Gebauer 27:24
Alright, so that’s, that’s interesting, because that sort of deals with my second point, and it’s about inclusivity. And perhaps fear, you know, who are the decision makers? Like, it was interesting, you bring up the paralegals, because I was going to point out that, you know, generally, the decision makers are the folks at the top the folks who have control of the money, it’s like, okay, are we going to spend on this or not? You know, oftentimes, perhaps not all of the right, team members are kind of empowered to have a say, or to sort of think of, you know, think outside the box in terms of, of different ways of doing things. So what tactics have Have you found that are useful to get the right people in the room or in the even in the process, the overall process. And also get them kind of aware that there might be other ways of doing things.

Ken Block 28:29
You need leadership to be strong, and very communicative, and able to really talk about why it is they’re bringing in someone like us to do this function that they’ve all been doing for a very long time, they have to be able to communicate that, you know, the reason we’re doing this is, you know, we want to streamline this process and save some time, because we’d like to bring in more business might be one reason, right, we’d like to do more, or we have more, you guys are already working your tails off, and you can’t get it done. And then we’re behind. And then you know, we’re taking hits, because we’re behind this is a way that we can catch up on the work. And I just heard that from an insurance company last week. It just underwater and this will help them get like, that’s the biggest thing I think we’ve learned in the stories that I’ve heard is that when there’s someone who has, you know, a real clear vision of what they want to do, why they want to do it, and they communicate that vision to everyone that can be involved. project goes smoothly project goes quickly, everybody’s you know, ready to go. There’s no problem when that communication is not there. Then, you know, the people that are doing the work that again, the legal associates, the paralegals they can stop this thing they have all the power because no one’s helped them alleviate their fears, which are absolutely real 100% if they if someone’s threatening my income essentially, you know, these two guys from this company, we’re invest are coming in to take my money away. It’s not happening. It’s not happening. I don’t care what you know what anybody

Marlene Gebauer 30:00
says, But if someone gotta be, there’s gotta be an incentive,

Ken Block 30:04
100% incentive or just, you know, reality, here’s, you know, we’re all in this together, here’s what we’re doing it, it’s going to do things, this is going to be good for you in the long run, I’m gonna explain to you why, you know, this is going to happen, when that happens, these projects go very, very well, when that doesn’t happen, they don’t go at all. All right, that’s,

Greg Lambert 30:21
that was just insane. Just thinking about this conversation kind of pulling back just a little bit. A lot of times we talk about getting something in the door, especially sales cycle on for law firms, especially, is extremely long. But in I have a phrase that I actually haven’t used the last few episodes, but I’m going to use it here is that, you know, all problems are communication problems. Because, yes, you know, there’s decision makers who bring could have had the power to bring things in into, into the law firm into the legal team. So there’s that power, but you know, it’s the the users that can take a you know, it could be a multi million dollar investment, and turn it into shelfware. If that communication isn’t there, to explain, you know, here, here’s the vision that we have of why we’re doing this, here’s the output that we’re looking at, here’s where you play a piece, in accomplishing that goal, here’s the reward, at the end of the day, for doing that, if you don’t have that conversation, and you introduce not just a new product, but you introduce uncertainty into it, that’s when you have you know, the people that are actually in the in the trenches doing doing the work, cans, you know, sink, that that whole project, if you’re not careful, if you don’t have that clear vision of where we’re going with it.

Ken Block 32:06
100%, and I’m a guy who’s sold some of that shelfware over the years, and it was just, you know, disappointing to see that it just didn’t work out because all the intentions were there. But no one had set the expectations properly. And at the end of the day life is a lot about expectations. So that’s we need to do a better job of

Greg Lambert 32:24
want to talk a little bit about the response creator, which is one of the newer tools there that you have it at LegalMation. And I want one tie this into this conversation that we’re having about how this impacts it, but also kind of broaden it up a little bit and talk about, you know, the discussion of generative AI and in the concerns and the roadblocks that that you encounter when you bring in software or new way of doing things that create that downstream impact. So contain this in how how do you address this, as far as helping them kind of create that story to tell the people that are using this is why we’re doing it, this is the effect? How do you how do you approach that.

Ken Block 33:19
So with our applications, we actually have two approaches. There’s a lot of things that we do that are really smart productivity tools, I call them, you know, where we’ve taken some existing technology that works really well that people are familiar with OCR, and, and some other smart things that have absolutely nothing to do with AI in many ways, so there’s no, there’s something they can do without that we hope that, you know, makes them a little bit more comfortable. But secondly, there are, you know, some AI options, as Tom has mentioned, they’ve been around for a number of years now we’ve been, you know, sort of battle tested AI. But in our case, you know, we’re not looking at the the entire universe, you know, to create our AI tools, we’re not you know, bringing in a website since 2003. And all that stuff. You know, there’s nothing scary about it, only thing that we do with those AI technologies is work straight pointed at the client’s own data set and nowhere else. So you know, the approach we take is, you can kind of take this as you want to at your pace, if you want to start off just being very smart about how we, you know, turn, you know, complaints into fully answered responses. Having nothing to do with AI, we can do that. That’s fine. You don’t have to bring in that component to it if you want to ease your way into it. And then we can explain exactly what the AI component is. And again, it only works on the firm’s data set. So it’s not a scary thing, because everybody’s you know, having worked in the in the first wave of this stuff, you know, many years ago, and I think that’s when I met both of you, which didn’t deliver His promises in large part. This is not that this is a much smarter approach to it. And that’s the discussion we have with folks who have that concern Right, Thomas center if you want to add anything to that,

Thomas Suh 35:01
just just just a bit more. So one of the challenges, of course, with January I, as you’re familiar with GPT, and even the not just the ChatGPT, four and five coming up soon, they do rely on Large Language Models and information and data that is publicly available. And it’s sort of a massive amount of information, and it can distill for you a bunch of stuff. Our approach is certainly using the same general techniques, but focusing on just clients on historical data. So there are no religious nations. In fact, the system is never wrong, you cannot make up things, because it’s not drawing from any other source other than clients on historical prior responses and, and answers. So and I do want to give one specific example where this works really, really well. One of our large sort of a large part of our business is working with auto manufacturers, automakers and law firms that serve automakers with lemon law cases. And, you know, we’re in California and California is the biggest market for lemon law lawsuits for a number of reasons. And our system is able to be trained on a client’s own historical responses to discovery requests, RFAs, RFPs, and near auditories, as an example, and what used to take about three to four hours per set. Now, even with the sort of the process of using our tool and editing and reviewing is down to less than an hour. Now, that’s a huge difference, because there are 1000s of cases one, in fact, we’re on track for one auto manufacturer to get hit with. Boy, I want to say, maybe 3000 cases just this year alone. So you can imagine the savings that they’ll they’ll benefit from using our tool. And the law firms actually surprised that you may or may not be surprised, but one other automaker is a client because of their me law firm that approached us. And they have proactively actually reached out to us and to figure out how they’re going to serve the client better, because they understand again, the long term, you know, the long game here is to maintain the relationship to earn more business potentially from the same client with other matters. And so they actually came to us. There, they signed up, they worked with us to train the system, of course, the client is fully aware. But now that relationship has been really solidified. And that’s an example of the right type of cases where you have a lot of competition among law firms where you have to be competitive, you have to offer something different other than, hey, we went to the best schools and we have 20 years of experience, give us the work? Well, if you’re if you’re answering requests for proposal for a portfolio of cases, and all you have is your usual resume? Well, I think those days are over at the you got to bring some different angle. And technology is the great equalizer. And this is why now you have not just not just a you competing with another large firm that but now you’re going to be competing with smaller, more nimble law firms that can do probably as good or better work than you because technology allows them to elevate themselves and is served as a great equalizer. Anyway, another story, I kept dropping stories of clients without mentioning names, which I hope helps, but these are real life examples where again, the right type of cases, the right matters. And by the way, these high volume cases make up 80 plus percent of litigation in the in the country anyway. So why are we messing around with the point ones point twos when you know that a bulk of these cases eventually is going to come to you in the form of clients asking you the firm about what are you doing with tools? Or are you gonna be or the worst thing that can happen is they’re going to move to a work away to another firm that’s already using these tools. So your choice.

Marlene Gebauer 39:13
I see I love that story of you know, firms sort of taking an innovative approach to the client relationship and looking you know, not only at the the legal services, but but also looking at the business services and the the the process and sort of like how do we work together since we’re, you know, sharing these these documents, how do we, you know, work in the best way to make that as easy as possible. I think that’s terrific. Cool. Another exciting thing, and I think Greg, this is a first, we’re going to I think it’s the first we’re going to do a live demo for the YouTube viewers a little later. So I’m excited about that. But before we get to that, let’s hear about what the product does and the problem that you’ve set out to solve with response creator.

Ken Block 40:06
So the problem is that answering these documents responding to them, is, by and large, with very few exceptions, a 100% manual process involves things like copy and pasting from PDFs that come from the courts into a brand new empty document, or sifting through a maze of templates to try to find the one that matches the one you’re working on now, or looking in particular templates, where there’s a list of answers. It’s as long as a roll of toilet paper to try to find the one that matches up with the question that you’re looking at now. And some of these processes, you know, they may be quicker for complaints, which don’t tend to be as long but but you get into some of the discovery documents, some of those responses required number in the hundreds. And it can take days, it could take a day just to set the format of the answer up, let alone even answer the question. Right. So that’s going on every day of the week, all the days that ended why because because we know the working overtime on Saturday and Sunday in corporates in law firms, and you know, even service providers are doing this sort of stuff all over, you know, we just know that that’s just a fact. So what this product targets, is taking the process that they’re doing every single day of the week, often repetitively answering the same kinds of questions in the same style, and allowing technology to be smarter about answering those in a more automated way. It’s not an automatic answer that’s not answering and automatically filing we don’t have we’re not scaring anybody that way, you know, you’ll you’ll build the answer. And someone will look at it before it goes out the door, of course. But the idea is to, you know, set up the answer format, in some cases, maybe automatically drop the answer in to the right spot, because we know what the answer is going to be all the time that repetitive cases Thomas was talking about before, you know, we know what the answers are going to be, but we still have to answer them. And then in some cases, giving us the option to plug in language where we need to with just point and click kinds of things. You know, where you’re you are using some judgment because you need to, but at least you’re automating that process. And the language that’s being used is more best practices, right? It’s coming from the way the firm wants to answer it, or the way the client wants the question answered. Or maybe it’s tailored to a particular attorney style of answering I laughed out loud a few weeks ago, when our one of our trial customers, the paralegal said, Well, I work for two attorneys, they both answer the questions exactly the same way, except the attorney be like citations at the end of there’s, and that person current process has to go find the template of the one with the citations and the one without them and drop them in? Well, in our world, we can, you know, tag it and make it easy. And they can just point and click and, and do that sort of thing. But that’s real life that happens all the time. So so that’s the problem that we’re the problem that exists. And that’s the problem that we’re addressing and trying to solve.

Greg Lambert 43:06
Now, you had mentioned and we talked over over the past couple of weeks about this, in that the way that response creators is set up is one that works with your with your history of documents that you that you’ve used. So it kind of picks up that tone of how the firm responds to these types of matters. And you’re using the AI to do it. Now one of the things I think a lot of people are very concerned about, especially with with some of the creativity that they’re seeing, and some of the generative AI tools, for better for worse is that you know, on some of these things, you you don’t want to necessarily be all that creative. You want it to fit your format, you want it to give a standard answer. But at the same time, you don’t want it to look like you’ve just cut and paste something, you know, you want to be able to have that ability to kind of use your your own judgment, your own creativity. So how do you set this up so that it can learn and adapt over time to fit those needs and keep that speed of reaction of responding, you know, at an acceptable level?

Thomas Suh 44:38
Probably the one thing that’s good about our system is that creativity that you talk about, you know, cuz I don’t have I’m sure Greg Marlene, you guys have both experimented with ChatGPT at least a free version or perhaps you have a your firm less than some of the other paid versions. But you run the same query for the same documents they ask it to summarize, let’s say you know some document You get a different answer every time. And that’s a problem for lawyers. Because it’s unpredictable, right? So we don’t have that problem, because the system will simply pick the best answer from the existing database of prior already present all the pre approved and served responses from the past. So you’re not going to have that issue number one. Number two, will the system continuously learn? Absolutely. And that’s the, that’s the difference here, where the wished we we can onboard a client in a month or two or less, depending on the workflow with just a seat Sebold call a seat set, which is a smaller sample of historical responses. And then when users start actively using the system, remember, that becomes essentially training data, because we capture everything that typing every keystroke is capture and that we store, which we then use as training data to propagate back back propagate to the Data Warehouse. And that becomes new response options from which the system can choose from so it’s never making up stuff. It is just it is a it is picking the best options from an existing set. And for legal, that is what lawyers and paralegals need, they need that sort of predictability, that consistency, right lawyers freak out when they can’t predict what’s gonna come next. So that that’s been that’s why, again, it’s not for every case, right? It is for the more higher volume matters, which comprises again, most of the litigation volume in the country. So that’s why it’s been quite successful for us. And we’re very excited about that it had it was quite painful actually to develop the system, it’s very difficult. In I think it’s going to require, it’s always been fine tuned and improved on. And one thing, though, what this allows, then, at the sort of higher level, it allows some corporate clients to say, hey, you know, here’s what we’ve noticed, once you sort of put this up, and we’re testing, you’d note, we’ve noticed that our old outside counsel has such variations in their responses for the same request, or for the same allegation in the same jurisdiction. Completely inconsequential, but because we’re all artists, Greg, absolutely.

Speaker 5 47:30
Yeah. And

Thomas Suh 47:33
we like to sort of, you know, create our own sort of responses, because it’s, it’s, it’s artistry, well, at the corporate level, that’s a problem. Because oftentimes, you may land ledges and trouble. And so when you want this system like this allows you to create sort of that consistency across the board to pick perhaps the best answer or the the pre approved answers that shouldn’t be used for certain allegations, without all the extra tenuous things or, or also remember, human, you know, human, let’s, let’s face it, a huge part of our business, because we’re service business, there’s a human element where, look, I don’t function as well, at four o’clock in the afternoon, as I do at eight o’clock in the morning. So I may rush through some of my answers as an associate, I know that, you know, associate me mean, you tried to rush quickly through doing their work, because they have happy hour coming up in 15 minutes? Well, I mean, let’s face it, and machines, like I care about that. So there’s a lot of benefits to, to that option, as opposed to relying on the current state of affairs.

Marlene Gebauer 48:42
So if lawyers are concerned about losing the human touch in the process, you know, either based on quality concerns, or maybe even based on sort of education and learning concerns, you know, you kind of have to do the scut work in order to learn this stuff. You know, how do you reassure them that, you know, AI generated documents, you know, will be of the same quality, and that, you know, people doing the work are still going to learn, as part of the process, you know, where, you know, where does human Review and Editing, you know, come into play in that regard?

Ken Block 49:24
Yeah, no, that’s an excellent question. So, you know, in terms of the learning process, I think we can look at it a couple of different ways. Some of these cases, some of the, you know, the points that are made, you know, they are virtually identical every single time particularly talking these high volume cases, when you see the same stuff. The person is going to learn in about three answers exactly what he’s supposed to he or she is supposed to answer every single time. You know, whether I point and click and drop the answer in, or I manually type it in. The learning curve is pretty quick with those sorts of things. So anytime it comes to something that’s not of the cookie Have a variety like that, you know, then you’re going to have the opportunity to either put in a nuanced answer, or you are going to type the answer in automatically. Because remember, one of the great benefits our system is that we create the answer format in the first place, which in itself is a drudgery, right? So we’re, by speeding up that process, you know, we’re opening up a space for you to answer it in several different ways, as you’ll see when we get to the demonstration. So, you know, the learning comes in different ways and learning advances over time. And it’s generational. As Thomas mentioned earlier, I don’t think that you’re gonna lose anything, all you’re gonna do is speed up the curve with things that don’t merit taking a whole lot of time on, and give you the opportunity now with more time to learn and spend time to things that you do need to spend more time on that. That’s, that’s been my take on it.

Thomas Suh 50:46
Yeah, I would agree. I mean, 90, what is it mid 90s, we were not given access, first year law school, we were not given access to Westlaw or LexisNexis. Because the school’s philosophy was, hey, you need to learn how to go to the library and pull up your the books. And of course, you know, you’re happy to do that until you realize that the shepardize insert was missing because your classmate took it. And they were reading it somewhere down the hall. And they forgot to put it back.

Marlene Gebauer 51:20
So forgot. Yeah.

Thomas Suh 51:23
Yeah, forget that. Exactly. So I mean, that literally, I remember professor saying, no, no, no, this this this side, this is like a supplement. Yeah, I can help you this, this, this West Law stuff can help you later on. But right now you have to learn to do book research and

Marlene Gebauer 51:37
learn the structure, you gotta learn the structure of how organized Yeah, so the answer

Thomas Suh 51:41
is no, we know that do we? Actually do we’ve committed malpractice? If you wasted clients time to go to the library and do your research in the old fashioned way, without using Westlaw or using Westlaw or LexisNexis? exclusively? That’s just one example. It may not be a perfect example. But you get the idea is how much are you learning from cutting and pasting or going back to old samples and trying to do your sort of the, the matching process that you have to do let system do it for you. And then when you get a draft and you read through it, you understand, you’re gonna understand why that response was there. And that’s, I mean, that’s how you learn. I

Greg Lambert 52:22
know, when I was at WWL, I heard a really good quote, and again, I wish I can figure out who it was it said it so I could credit him with him. But the quote was, you know, law firms are rushing to be second, especially when it comes to technology, no one wants to be first, but they don’t want to be the last either. So now that you guys are out there, you have a product like response creator, that is in the hands of, you know, a number of competitors to to, you know, the people that want to be second, what’s been the response so far that you tell them? And what you know, can you give an example of a story where you can share in terms of the metrics, that in the efficiency gains that people that have implemented it, or are bragging about?

Ken Block 53:14
Yeah, there’s a lot of stories, I mean, I’ll start and Thomas will fill in, I mean, the benchmark I use when I’m talking to folks who want to learn more about us, is to assume an average of 30 minutes per document prepared saved. By itself, just call it 30. Some complaints may be shorter, some discovery documents may be longer, but let’s do some math and call it 30 minutes faster. So depending on your perspective, you know, if you’re in a large you know, insurance company, and you’re doing hundreds and hundreds of these things, I mean, we just did a calculation for a prospect a while back that it would save them six bodies of time in a year at 30 minutes and they looked at me and said You’re nuts. 30 minutes is too light. But you know, I don’t want to I don’t want to make it too crazy cuz six was crazy as it was. You know, that that’s where I start. But we’ve done some other studies, you know, that that make it more specific, that actually makes us confident that 30 is light, but it’s it doesn’t make people think we’re nuts, that kind of thing. And maybe Thomas, you want to tell a specific story.

Thomas Suh 54:17
Yeah, I typically use more percentage savings, time and efficiency as a starting point. And that’s 50 50% or more 30 minutes is of course, Ken’s referring to some of the very high volumes, sort of auto part crash cases, you know, very routine type of high volume cases. But I will give you one example one of the large pharma company that was starting to get some lawsuits in one of their products. These complaints tend to be rather long, they’re not the typical, you know, four page complaint for slip and fall case or or accidents. Some of these complaints were 190 to 300. Allegations paragraphs. I don’t know what that translates to pages. But for this far a couple of these share that one of the EMLA 100 firms that they engaged was taking was billing about 10 hours to draft an answer for each complaint. And they want to see what we can do well, we got their prior samples. And just as a test, we ran it through, we sort of created the Astra tool for them. And even though the system wasn’t able to answer all the paragraphs, he was able to ask for, let’s say, 80%. Because the training data would that would that would give me too much into into the weeds. But ultimately, what now they were able to do is from, from the 10 to even 20 hours of billable hours that they were paying, they’re unloading the firm just for the answer. Their in house counsel could do this in an hour. Because they saw it says the system would spit out a draft like the fully drafted answer with each allegation answered, and then a few that were missing, of course, they were very unique, so that they have to go in and but even though review time, if you could just really quickly review. It wasn’t it out, it’s a complete game changer. That’s, that’s less common than sort of the bulk, other type of cases that we handle. But that’s an example of the power of a system like this. It doesn’t matter how long it is, it might take a little bit longer to process by another minute or so by our system. But that’s the type of efficiency that you can gain on one extreme. And then the other extreme, of course, even if you save 30% of your time, on your high high volume matters. Well, you multiply that by the number of cases, the big picture is actually very significant.

Marlene Gebauer 57:03
So we are we are at the point in time where we come to the crystal ball question. So we ask both of you to sort of peer into your crystal balls and see what challenges are changing in the legal industry is going to happen over the next two to five years. So Thomas, I will start with you.

Thomas Suh 57:26
A boy of acid is made different ways and I shift quite a bit. But I think the core position that I have remains, which is it? I always thought because I did. I was at a law firm. I always thought to myself, you can take the lifecycle of a litigation matter. There’s no reason why one team, one person or one firm should handle everything from A to Z. Now 30 That’s contrary to what everybody would say, which is Wait a minute, you need to know the fast from the beginning all the way to trial. But if you really think about what you can do, and can you modularize the practice of law, can you modularize the cycle of litigation as an example where one firm or one team only does depositions. One team does only written discovery one Kim does he does initial pleading stage and one team does the expert depositions as a separate sort of, and then one team of course, that’s the trials. I think that is possible. And I think that not only creates some efficiencies and obviously savings that come along with that, but I think it gets its gives you the best results possible. An example is you know, I had a partner who was fantastic, have written and work right there. m&s J’s, and they’re, they’re depleted pleadings and everything else that they did was just like poetry. Fantastic. But you put it with followed up for trial and it crumbles. Why is it the person who’s gonna take the case all the way through and that this is why you do have examples where you have the parachuting trial attorneys come in sort of 2030 days before the trial, the biggie up to speed and Nick is still driving case. That’s just one example of what you could then apply to the rest of the litigation lifecycle. So on top of that, that’s part number one. There are big chunks of that then that can be handled by a non wall firm entity, right? We already have the LSPs trying to legal service providers and you have, you know, they gotta be new law firms that would be that are gonna be owned by non lawyers, as you know, the a lot of that is already coming to reality in Arizona. So you’re gonna have such options. If you’re a client, you have so many different options to handle your cases, especially high volume cases. That I think if you’re a law firm, I would be more think came about specialization, as opposed to a full service and law firm, which has worked until today. But I don’t think it’s going to be as lucrative going forward or really a smart going forward.

Marlene Gebauer 1:00:15
I like that very much that is super disruptive. So I hope I hope we get a lot of feedback on that. Ken, what does your crystal ball say?

Ken Block 1:00:26
Yeah, mine is probably a big, bigger picture kind of thing. I think a lot about what Tom said before about generational and I think about the younger people that are coming in now and what their lives had been like for the last, let’s call it 25 years, something like that. And so my perspective is having three of those children in that age range. And watching what’s going on with things like the the real resistance to return to the office, and you know, the desires of these kids to live a more quote, normal life and still want to be lawyers and still want to do those things. And also demanding technology to be part of what they do, they don’t want to do the grunt work, they think they can figure it out. Like I said, earlier, I see it three times, I’m good to go. All my kids are like that, I don’t need to, you know, practice this, I know what it is, you got you told me I got it. There’s an element of BS to that, of course. But I think in large part, this is the way they are they know they don’t, they’re not going to go through all this stuff that a lot of us have a slightly older age went through in the past, where yeah, you’re gonna, you know, you’re gonna do that for a year and get your 10,000 hours and all so they don’t have the patience for any of that stuff. Because they think that the technologies that help them get where they want to go much faster. And I, I’ve even said years ago, I was in another place and I challenged our marketing person to go find me all the law firms, they could were the managing partner was in their early 50s, at latest. Because they’re just going to be more technology. You know, it’s not, I’m not being nasty, better ageist about it. The fact is, they’ve been exposed to more technology in their lifetimes, and can relate to what I do, versus someone who just hasn’t. And I see that trend happening, you’re going to see these younger people in those positions, and they’re going to demand a LegalMation Come in, or all these other technologies. It’s not an if and they’re going to look at the firm more than ever and say, Okay, what is your technology stack are using this are you using that you’re not using that, that’s fine, I’ll take the offer from the folks across the street, that’s a better gig, even if it’s for less money. So that’s what I see over the next year is gonna be interesting to watch with these, dare I say kids are going to do and they’re going to want to happen, you know, in this profession, and others too, but this one in particular,

Marlene Gebauer 1:02:40
one retirement at a time. That is true, much faster

Ken Block 1:02:43
now, I think than before.

Greg Lambert 1:02:47
So for those who are listening on the audio platform, if you want to see a demonstration at LegalMation, you can head over to the YouTube channel to see more. And for those of you that are watching on YouTube, now you’re in for a treat, we’re going to do a little bonus content here and do a do a demo.

Ken Block 1:03:09
Okay. So we’re going to demonstrate two aspects of the system. Here, we’ll give you an example of processing a complaint response and also a discovery response. And just for completeness, we also do some pricing data analytics as part of what we do that kind of flows from it. But let’s let’s walk through a complaint response first. So if I’m working in the system, odds are I have responsibility for certain state or I’m only barred in a certain state, we happen to support about 33 states on the complaint side now, which represents most of litigation in the United States. So we tend to cover just about everybody, but you will only see the list that is relative to what you do. So for this example, what I’m going to do is I’m going to process a complaint from the state of Florida. And what we’re going to do is we’re going to take a document, it’s going to go into my database. There we go. I’m going to open that document, and I’m going to process it and we can have the system be prepared for different kinds of practices, depending on what the firm does. In this case, it’s an insurance claim. So I’m going to process and what the system is doing is two things right now, number one, it’s going to take that PDF, and it’s going to OCR that, and it’s going to begin the process of completing it and turning it into an answer format. The second thing it’s going to do is it’s going to prepare a confirmation page of the main details of this case, literally for confirmation. And part of the reason for that is it is as good as OCR has gotten over the years, as long as I’ve been doing it, it is not perfect. And we want to make sure that there’s certain pieces of information that are correct all along. We also use some of this information to extract out because it can be used for other analytical purposes. So here if the firm is using a matter management system, they can put in a matter number, whatever their format is, and then we’re gonna roll down the page, check the jurisdiction, check the captioning, you know, all the different things that are important if we’re delivering service that makes sure all of that information is correct here on the lower side, we can assign an attorney and all those interesting things takes a minute or two worth doing. pages aren’t exactly perfect again. So OCR just needs to be checked up on normal part of the process. Now, when we click go, we’re now going to create the answer format. So the way the screen is laid out, as Thomas and James and our colleagues have done as attorneys is they’ve tried to lay this out as attorneys tend to work. So what you see first off in the left hand side, this is the complaint itself, and its original format, you always want to be able to refer to that, and it’s side by side with the right hand side, which is the answer format. And here, we’re going to show a couple of examples of the how the system can be used and to the firm’s preferences. So in this case, what we’ve done is we’ve actually automatically replied to the allegations, one through four automatically, these are fairly generic in the way they’re presented, you can kind of glance here at 123, and four. And so you know, we would again, in a high volume situation, we often will answer these in exactly the same way. In this case, this client that were put together, sees the questions, they’re answered in a certain way. And the answer is fill in themselves automatically. Again, nothing is set in stone, if I want to make an adjustment to allegation number three here, the response, I can come in here, and I can change it any way I want to. In some cases, perhaps maybe we don’t have an answer for that ready made on the situation. And so I’ve got a blank spot number five, that I now can fill in. And so here’s where the options come in to for our clients. Number one. With the answer now formatted in all of about 30 seconds, I can simply start typing, I can certainly manually enter information, I can copy and paste from other documents, I can do things you know quite easily, anytime I want to that’s always available, format it up, make it look nice. The other thing that we can do here to make it even faster is we can use our virtual assistant on the left hand side where the answers have already been canned, they’ve already been written. For this particular situation, you can just drop them in. And again, I’ll just pick one here for just for example sake, but point click Add drops the answer and and this list can be customized to the user’s preferences with this gear box here, we can go down and we can update add to we can create additional answers, you know, that are specific to the way that they’re working with their clients or with the attorneys they’re working with. They can make this look exactly the way they want to in accordance with whoever the best practices are. And they can add in anything they like. So once those answers have filled out, you may have noticed that the system does keep track of the responses that were expected and the ones that we have completed. The other thing we may want to look for to see if we want to add any definitive defenses into this response. Well, we’ve got a list of affirmative defenses available also in the virtual assistant. And these will look familiar to the audience for sure. Let me just pick a couple of these here just to illustrate how the system can work. And you see, what I’ve done is I’ve I as I’ve entered the man, it has dropped them into the proper place for affirmative defenses, it has numbered them consecutively as you would expect them to be and place the text. And again, you can come in here and you can edit, we may need to fill in a name or a date or something like that. But the the form and the basis of that defense are already in place with the proper language. But then we can also do other little things that I think may sound trivial, but I think are actually quite interesting and just get rid of we’ll call them annoyances, which are what happens if we decide in review that we don’t want the second affirmative offense. So we want to be able to remove it quite easily. And what you’ll see the system has done is it’s removed it, it has moved up number three into number two’s position and change the heading to second affirmative defense. It’s a little thing, but it’s a little thing that slows things down as we’re preparing them in the manual world. So if we’re satisfied with where we’re at with this particular complaint, was simply click go, the system will process the answer. And it will be available here in this download screen. So we can open up as a Word document, we can edit it again, we can forward it to another user if they’d like to look at it and have a final confirmation sign off. And of course, this is a list of all of the different complaints that we’ve created so far. So that’s the process of creating a complaint. The second part of this is to create a discovery response. And the process will be virtually identical. So one of the things we refer to a lot more talking to folks that are interested is the idea that if there was turnover in a particular office, and we need to train someone new, by the time I’m finished with this, I’ve shown it once I’m going to show it again. You’re pretty much trained. Greg and Marlene, you’re already ready to go. So here for discovery, this will be an RFP. I’m going to select something from the state of Texas. I’m going to drop this file in to the spot. It’s going to be an asbestos case. Personal Injury. I’m going to click Go the processes I Danika this one will take a little bit longer because a little bit more to it. But again, in the scheme of things, the speed with which we’re going to create this answer format is a fraction of what is needed in the manual world, we’re going to do the same two things, we’re going to open up a confirmation page. So we’ll make sure that all the bits that are scraped off of the basics of the case are correctly cataloged, we may need to enter a piece of information as well. So we’ll do that in this example, just to show how that works. And then we’re gonna go proceed to the answer format. So here again, I want to put in a matter number 123, dot 456, whatever the matter format is, we can integrate this with the matter management system as well. We’ll go here, we’ll confirm all the bits and pieces, maybe something’s missing, I don’t have the responding party will be able to plug that in. Maybe I want to add in the particular attorney, so we’ll add an attorney there bar number will come up, their contact information will come up because we’re going to need those on the response format, we’ll put that out, we’re going to click go. And it’s now going to create a similar but slightly different format. So again, here on the lower left, we have the original, that was been processed, we always have that to refer to.

Again, on the right hand side, we now have the answer format. In this case, nothing has filled in by itself, we’re going to answer these one at a time. So let’s again walk through the answers and the options of how to do this. The first is to simply manually enter information once again, or copy and paste whatever you need to from a different document. And we have multiple clients that have licensed our software just to do this just to get to this point only. As Thomas mentioned earlier, as we were talking, you know, the idea that just creating the answer format saves a dramatic amount of time, let alone the answers. You know, they simply avail themselves of this option, and it’s a winner for them. Second option over here on the left, again, we have some objections that are already written, you’ll notice some of them are in green, which means we’ve seen these kinds of questions before. And what we’ve seen in the past is that these are the objections that we tend to put in for those things. It’s not required, it’s not automatic. But they’re there and available as a suggestion if desired. And again, it just fills in the answers automatically right here, we can go to another one. And maybe there’s some other responses we would like for this particular one. And again, we can drop in whatever we want to there, we can again here customize as needed for the particular person who’s using the system, maybe they’re working for different attorneys. And the answer is slightly different. Or maybe they’re working for a client who has a very specific way that they want those questions answered. So we can group them by an attorney, name, a client name, what have you. And then when they come back in the system, they all know what the Filter icon looks like. And then you can just simply pick the answers that are applying to one of them this case, that’s one of the ones I picked. Alright, so they always have the ability to really customize this to the way that they work. And then with discovery, we have another option that’s available here. Let me make this a little bit bigger. This is our response library. So what we’re able to do here is this teach the system using a seed set of documents from the firm only, our software is only looking at the documents that are within this firm’s repository itself, as they’ve provided to us to see if we’ve answered these questions before in these kinds of situations. So when I click into this particular one, it’s looking out in the database to see if we’ve answered it before you can see here real quick producing and all that good, too late. It’s all playing us. It’s the same language over here. And in the past, we’ve answered this question in this way. We can also tag those questions and answer pairs, you know, this one’s related to relative to furnaces, or turbines or products or what have you, you can always refine that a little deeper. But once you’re satisfied, this is an answer you want. Click that answer, drop it in, again, customize it if you want to delete but you want to nothing final about this, it doesn’t automatically create a final version, it sets it up and is ready to go. We again keep track of answers that we’ve completed versus how many are left to be done. It’s great if you need to stop work for something else, and then come back in. Once we’re finished, we click go. And it processes this document right here. So the process is very similar for complaints and discovery again, for training purposes. You know, it’s not a lot to do to learn to use the system, the legal part of it and what has to be done. Well, that’s got to be that’s got to be communicated either from the client or from the partnership or whomever that’s going to happen regardless. So picking up this tool and using it right away, is possible. And then compared to the many of the other legal technology applications have been involved with for many years, that have taken months of planning and sometimes years to roll out. This is not that this solves a very, very specific problem creates a very, very big opportunity and allows folks to jump on it and start using it in a very, very short period of time. That’s my demonstration of those two functions.

Marlene Gebauer 1:14:51
That was very, very interesting. And oh my gosh, I can see how it would reduce the time immensely. I’m just by using Right,

Ken Block 1:15:01
right. Yes. Is that not the fastest software demo you’ve seen?

Marlene Gebauer 1:15:07
That’s like, that’s like perfect. Just like, you know, it’s like, I think we’re like, what? Under 10 minutes on that.

Ken Block 1:15:12
And that’s what I tell people. I think I’m out of my mind. That’s right. Soup to nuts. Great.

Greg Lambert 1:15:19
Ken and Thomas, thank you both for taking the time to come in and talk with us. Give give the demo. So that was great. Thank you.

Marlene Gebauer 1:15:32
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter, LinkedIn, and MBA Bauer 66 on threads.

Greg Lambert 1:15:49
And I can be reached at on LinkedIn or at glambertpod on Twitter or glambertpod on threads. Thomas can if someone wanted to continue that conversation, learn more, where’s a good place for them to find you online?

Ken Block 1:16:04
They can go to our website. There’s a communication tool there they can see us on LinkedIn. Both Thomas and I are there. You can email us directly I’m ken.block@legalmation.com

Thomas Suh 1:16:14
or Thomas thoughts legalmation.com Kham Kham remain the main contact.

Greg Lambert 1:16:20
reached out to Ken

Marlene Gebauer 1:16:23
at least initially. So yes, listeners, you can also leave us a voicemail on our Gigan review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 1:16:38
Thanks, Jerry. All right, Marlene, can Thomas talk to you guys later? Yeah, yeah.

https://open.spotify.com/episode/3B4A7EFJBE1WzqUteJXXRr?si=mOC-OyQ4Qhe1glFffLysdg

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer,

Greg Lambert 0:14
And I’m Greg Lambert. Marlene. Last week we get to do the LINKS conference. So thanks to Colleen Cable and to Kris Martin at Harbor in in Joanne there, had a great time we took we took copious amounts of notes to be prepared to speak at the end. And then it turned out we didn’t we didn’t use hardly any of any of those notes at all.

Marlene Gebauer 0:42
That was very funny. It’s like and it’s fairly common, like you basically do all this prep, and then it kind of takes a different course. And you know, that’s okay. So well, but yeah, prepared. Really good conference. Yeah, was a really good conference and really enjoyed the speakers and the panels. Got a lot of good information. I should note, I should note for our listeners, so I am working really remotely. I’m working at my parents house today. So, so apologies for any background noise or my like how it sounds. I’m kind of doing this off of my iPhone, so

Greg Lambert 1:19
Well, you’ll just test my editing skills. And they Oh, like, how well I can get the breakfast dishes sound out of the way.

Marlene Gebauer 1:29
That’s no joke.

Greg Lambert 1:31
Yeah. Oh, I could I could hear them. So Marlene, we are lately just chock full of special episodes in this week is no exception.

Marlene Gebauer 1:43
Yes, we are very excited to have Christina Wojcik. Joining us today on the podcast. As many of our listeners read recently, Christina is the new managing director of corporate for LexFusion and brings over 20 years of experience driving innovation in the Legal Services and Technology. She has a very unique background, having worked on both the legal service provider side with companies like Pangea3, IBM unseal software, as well as leading innovation strategy internally at Citi. Christina Wojcik. Welcome to The Geek in Review.

Christina Wojcik 2:16
Thank you so much for having me. It’s an absolute pleasure being with you today. And Marlene, just like you’d said, I’m, I’m kind of in transit right now. So I’m in a hotel room in New York City city after a whirlwind week here, and I’m just really excited to be here and to be with you today.

Greg Lambert 2:30
Well, Christina, you bring an amazing amount of experience in legal tech and innovation from, you know, a lot of different perspectives. So what initially drew you into this field, and what’s what’s kept you engaged in this field throughout your career.

Christina Wojcik 2:51
You know, it’s, it’s one of those things where it was kind of a combination of good luck meets bad luck meets timing needs, you know, at eclectic meats, not being afraid to do something a little outside the box. You know, I went to law school, not necessarily knowing if I wanted to be a lawyer or not, but it just seemed like the right thing to do at the time based on my undergraduate degrees. And when I graduated, I moved to Washington, DC to focus on international human rights, especially with women and children being trafficked. But unfortunately, what I realized very quickly is that doesn’t actually pay back your very expensive Law School student loans. So a little bit of bad luck there a little bit naivete there. And so I quickly pivoted over to kind of the business side of law. So I was involved in the very early days of, you know, moving document reviews from the the law firms to, you know, the the temporary attorneys with the contract attorneys helping set up and run some of those shops at Kelly Law registry. You know, I got really interested in more of the academic side of how one runs an in house corporate legal department. So just my kind of curiosity took me to the Corporate Executive Board or that I moved to see B. And now it’s been acquired by Gartner to help you know, in house counsel recognize best in class practices for their legal departments through research and literature. And from there, it just kind of following my interest in following people that were doing really cool, disruptive, innovative things. It’s just kind of captured my attention, and I just keep following along that path.

Greg Lambert 4:26
Sounds exciting.

Marlene Gebauer 4:29
Christina, speaking of exciting you were at Pangea three during its high growth and acquisition period when it was acquired by Thomson Reuters. So what key lessons did you take away from being part of the pioneering legal service community?

Christina Wojcik 4:45
So working with Pangea three, I actually discovered them very, very early in their stages, just when outsourcing legal work to India was becoming a thing. Um, I was at the Corporate Executive Board at the time and I I learned a little bit more about what they were doing just through the research of the CV organization. And I’m like, Oh, that’s cool, right? Like I really believed in the vision I really believed in the value that they were trying to drive. And I really believed in the leadership that was growing that organization and growing that that category that hadn’t really existed before. So Pangea, three’s leadership with David and Sanjay and Greg Paulin and Jonathan Goldstein, Kevin Cole Angelo, were just incredible. And I was working very closely with all of them in the US when I had joined. And, you know, watching the team that David and Sanjay built around that which includes Joe Borstein, and people like me, or Marilyn peruviana, who are just brilliant and passionate people about the things that they believe in. And so when I think about key lessons that I took away from pioneering a legal services company, or probably any company is like really believing, believing in, you know, what you’re doing, what your team is doing, and trusting in the leadership of the organization that that you’re joining. If all those three things come together, you can have incredible amounts of success. And fortunately for Pangea, three, we that’s exactly what we had experienced.

Greg Lambert 6:12
Yeah, that’s quite an all star group of people that you worked with.

Marlene Gebauer 6:15
Absolutely.

Greg Lambert 6:20
So making the jump from legal services over to legal tech, you know, starting with IBM, then you went to sealed software, what stood out as some of the biggest differences that you saw between the two sides of the industry?

Christina Wojcik 6:38
Oh, my thinks that there, there are more similarities probably than than differences, right. So the services side and the tech side, we’re all trying to solve critical problems that law firms and in house counsel we’re experiencing, right. And I think that when we’re looking at the my experience at IBM, we were really trying to tackle some of the incredibly large challenges. So it was around the time that Watson had, you know, won Jeopardy, and really focusing at that time on how we could potentially apply cognitive analytics to the legal space. And there was a competition essentially within IBM as to where is where should the product position of Watson’s capabilities really be focused on and at the time the decision was made? To look at oncology? Right, what can we do in the medical field with with Watson what type of predictions can be made about cancer treatments, and, you know, legal was coming in kind of that, that next round of of investigation and evaluation. But then when we think about seal software, exactly what I had said about life experience at a Pangea three, which is, I had a group of incredibly talented leaders, with all sedbergh, Kevin, with Steve talker, Dan Taylor, and many others. And I apologize for not listing everyone there, but really incredible leadership, who were really trying to shake the way things have been done traditionally, in the past, to provide better professional and personal experiences for those people within law firms and corporations who were really struggling to ensure that they were adding value to high, you know, large scale, repeatable type programs and projects. So I think in terms of leadership, and in terms of goals and visions, very a lot of similarities between those two, two startups. And I think the common thread between all the startups is just the fearlessness of the people who are engaging with it to go and tackle something that, you know, truly hasn’t been done before. Has it been proven out that knowing that it’s the right thing to do, and really just sticking to to, to that cause of driving value?

Greg Lambert 8:57
I’m just curious while you were at IBM, and since this was during the the Watson IPE, how exciting. Was it being part of that at the time, and

Christina Wojcik 9:13
yeah, I read this is like where I couldn’t have had time that again, this is where the luck comes back. And I couldn’t have timed that any better because the amount of geek out sessions that we could have about the potential of cognitive analytics as it could be applicable to the legal space before anyone else was really doing or thinking about it was such a fun ride, right? And it’s so much fun to just sit down with some of the brightest brains in the the in the world, really thinking about what impact could we have, if we could have, you know, actual thought based processes to different areas within legal. So it was an incredibly exciting time, you know, frustrating as well because I wanted to see things happen as quickly as possible. possible because I appreciate the impact that it could make. But recognizing and I think this is a little kind of alluding to what we’re experiencing today with generative AI, also doing it cautiously, right, because this is before a lot of the podcasts and things going well, this was actually before podcasting. But around like, will the robots win? Right? Are the robots going to take over everything? Very, very, I think, a lot more fear, uncertainty and doubt than used to be at that time, given, like the limitations of the technology. But still, that’s really where we started the gears turning on, what could the future look like if the robots are potentially in the opportunity to have the opportunity to, to take over, we weren’t quite there yet at IBM. But, you know, it’s been an incredible journey, watching how it’s progressed to where we are today.

Marlene Gebauer 10:46
Yeah, I do remember the very sort of us versus them kind of feeling when everybody was kind of watching to see if, in fact, Watson would win. Heading up innovation and tech strategy internally at top financial institution like city, you know, that must have given great insights into legal department needs. So what would you say are some of the biggest pain points that you see in legal teams, and what they’re facing today?

Christina Wojcik 11:16
I think what a lot of the legal departments are facing today is a common is a common story, especially around you know, some of the the tier one financial institution, and it’s about, you know, spend analytics, big data cost reductions. It’s also about selecting the right type of solutions to fit your needs, and the fear of potentially making a mistake in the selections of those solutions. You know, especially at organizations like Citi and the other tier one financial institutions and other highly regulated organizations, they’re under constant pressure from regulatory oversight from regulatory involvement. You know, if if, if the correct actions are taken, there might be opportunities for significant fines or immediate exposure. And legal is kind of at the forefront of a lot of that in terms of mitigating risk and ensuring that the organization is as protected as we could possibly be from a lot of different different a lot of different areas. And so, you know, when I think about things that legal departments, really in house legal departments want to ensure that they’re doing is, is getting access and visibility into information in different ways that maybe historically was was needed. And how do we do that, especially when you have as much data as much information that flows through an organization, the size of city? And how do we ensure that data is visible and being applied appropriately, as well as ensuring that all the other things that go into running a major legal organization, outside counsel management, you know, law, firm performance, legal technology, billing, matter management, and then some of the the challenges that we face when we’re looking at the intersection of where business and legal meet, and ensuring that that intersection, and the technology being used offers a seamless experience, so that way, information flowing upstream and downstream, is, is is as untouched by manual processes as possible in order to try and reduce any type of human related errors. So when you’re looking at a legal organization, the size of a tier one financial institution, there are a lot of pain points, but there’s also a lot of incredibly smart people who are working on, you know, addressing, solving and mitigating those pain points every single day.

Marlene Gebauer 13:39
You mentioned that fear factor. And I’m wondering, in relation to legal technology, are there discussions that you’re aware of, in terms of, you know, generative AI, and sort of what decisions should be made around that, particularly since the technology seems to be changing very rapidly? And regulations aren’t really keeping up?

Christina Wojcik 14:05
I think there is significant. You know, I don’t know if if I would use the word fear, but caution, right, very much kind of a proceed with caution. And it depends on the organization. Different types of organizations have different types of risk exposure, right. And there’s certain types of organizations where you do not want them to potentially get this wrong, because could it bring down the global financial market? That’s a very concern, you know, very big concern. But there’s other organizations where the impact if they use generative AI, or if any of their information is exposed in a way that will have less global impact, and they have the opportunities to be maybe a little bit more fearless with their pursuit and use of technology technology like generative AI, you know, within the tier one financial institutions. It’s very much a closed door. or situation right now, where testing is going on in kind of a lockdown environment using large language language models in areas where it can’t necessarily be exposed to, to outside influences, right to ensure we understand exactly what it does, how it works, and what are the risk and control mechanisms we need to put around it to ensure that the monster doesn’t come out of the box, right. I alluded to like, I was just at a conference this week incredible conference on Gen AI ai for legal. And, you know, I alluded to the fact that it was kind of like the monster behind the door. And I don’t know if Stranger Things is a good reference to make, right we have all these scientists who are kind of behind the door poking and prodding at the Gen AI ai, understanding exactly how it’s going to react, where it’s gonna get angry, where it’s okay and, and how we can use it to potentially benefit but not necessarily ready to open up the gates to let that monster out yet, because once it’s out, we if you saw Stranger Things, you know, it can have negative impacts and implications. So I think we’re still kind of at least at the tier one financial, still monster behind the door, let’s start looking at it for things that are within the realms of understanding, we know what it will do we know how to control it. But also, you know, you can take a step back from Gen AI, even when we’re looking at more of the predictive AI. When it comes to things like machine learning natural language, processing delays, semantic indexing, those things where you don’t have to worry about the machine, whether it’s hallucinating or lying to you, you just need to worry about whether or not a model is learning away from what you taught it right. One of the examples I use is, you know, I teach it to find a fork, like a dinner fork that you eat with over time. Is it also pulling in like a pitchfork, right? Like, you can see why the model might get confused as a fork. Its prongs like it does things kind of similar, but like if I want to eat dinner, although sometimes I think my my family would say it looks like a meeting with a pitchfork. I prefer a dinner fork to eat dinner. So with with that title. You know, with that type of predictive analytics, it’s a much for the uses I had within my legal department. He was much less risk if it found out we have verbs leering towards pitchfork than a Gen AI ai, you know, Large Language Models, which is completely hallucinating everything that it’s just told me.

Greg Lambert 17:20
Yeah, well, I can, I can see why the guys over at LexFusion wanted you on board. I can just feel the, the the energy and the experience coming off of you. So but I have to say I did read your title. And it looks like somehow or another, you didn’t just land a job at LexFusion, you landed three jobs. So from what I can tell your man, you’re managing director for corporate Legal Operations, commercial transactions, and innovative accelerations. So I want to congratulate you on the three jobs that you’ve accepted there. And then ask you, you know, how did Joe Casey and Paul, the co founders there at LexFusion? How, how did they twist your arm to get you to join? Join them there.

Christina Wojcik 18:17
So those are my three titles for today. After this past week in New York, we probably have 17 more to add. I will I won’t be encouraged Joe, Casey or Paul to to do that at this time. So we’ll just keep it to those three. You know, it’s I’ve, like I said, I’ve known Joe for many, many years. Since my Pangea three days, we were together, he enjoyed a little bit after me. But we were together through the acquisition by Thomson Reuters. And, you know, he’s incredible. He’s just incredible. He is a wonderful person, he’s brilliant, he’s just a joy to work with. So from a personality perspective, like immediate fit, Casey is a genius. And anyone who’s had the pleasure of seeing him speak, you know, recognize that he is probably one of the foremost experts in kind of the next thing, or the things they’ve done in the past within the legal space. And he’s just incredible. And Paul also had some time at Thomson Reuters, and the kind of the Pangea three crowd and, you know, he is one of the most incredible, and I hate to use the term salesperson, because I feel that that doesn’t do justice to what he brings, which is really having the desire to fix problems, having the empathy to recognize that people are struggling, and he knows how to help solve those problems. So he’s delivering, you know, solutions to things that, you know, keep people up at night. And so it’s just the combination of the three of them is you know, I talked about leadership and following the leadership, a combination of the three of them it’s a no it was a no brainer for me from a leadership perspective, that you know, they were they they’re the right crowd to to join Right. The other component I talked about earlier was vision, right, the vision of what they’re trying to do. And Joe told me a bit about the organization a couple of years ago when, when they got together and started out, and I could see an inkling of a good idea, but I hadn’t had enough experience at Citi at the time to really understand the value or the impact that they can make, both on the law firm side, and then from, from my personal bias, the in house, corporate side. So, you know, I reconnected with with, I’ve been connected with all of them for always, and, you know, I just started, started kind of processing it a little bit more or myself thinking like, you know, the amount of impact I’ve been able to have at City by creating an innovation lab creating a program called lawyer of the future, creating an AI for legal programs. So I could get my lawyers and legal department on AI tools so they can understand what it does what it doesn’t do. I spent most of my career on the buy, or on the selling side, excuse me. And so I know, I know, when the vendor or the solution provider and technology is giving me demo magic, or you know, just kind of leading me down a path that I know that I’ll be able to deliver on, don’t ask me how I know that I led pre sales for a couple, you know, a couple of years. So I kind of see I know demo magic when I see it. And I really hope I was allowed city to make better buying decisions when it came to professional services when it came to legal technology. And so what I thought about my experience in that eclectic experiences I have throughout my career on the on the sell side, and then my incredible journey at Citi on the buy side, I really felt that I could add value to LexFusion to help their membership, as well as to help their law firm and corporate clients, you know, make better and more informed product development and decisions, help them develop products in a way that makes it easier for organizations as complexes, tier one financial institutions to buy and, you know, apply against your business and use cases. And then really help the in house counsel kind of cut through the madness that is legal technology these days, to really hone in on those organizations that will solve their problems within their lifetime. Onboarding is quite a process. So if we can help accelerate some of the onboarding, because you know, we’re introducing the right people at the right time, then I’m just hoping I can really add value to both sides of the house. And so LexFusion turned out to be a perfect match for that. And every time I open my mouth, they’re they’re like, great, Christina, we’re gonna add that to the things for you to do. So I’ll probably stop here before we give them any more ideas.

Greg Lambert 22:44
Yeah, yeah, I was gonna say no, no good deed goes unpunished. So be careful. factly.

Marlene Gebauer 22:51
So so obviously, there is a lot going on. But can you hone in on a few things in terms of what really excites you about the new role and what you hope to accomplish?

Christina Wojcik 23:05
Well, a couple of things that are really exciting to me, one I was, I’ve spent the past week in New York, getting to know some of our members, I’ve known a lot of them for, you know, a number of years, but you know, really getting to know their leadership. And like I said, it’s really about the leadership and the vision they have for their product and how they’re delivering. And I think that the members that we are working with their eyes, right, they are, they are incredible, they have the right people. And I think that’s one of the other things that I really like about LexFusion, they can be very selective about who they work with just because they have an extensive network. And they can really kind of hone down on those solutions and leadership that are driving value. So incredibly excited about our membership, I’m really excited about expanding our our membership network, to other areas within kind of the commercial transaction, corporate space, that will help solve additional challenges that in house counsel face. And then I’m also incredibly excited about, again, working with my former peers, I still consider some consider myself a peer, but my former peers within organization to help them set up, you know, potential strategies to allow them to build innovation and technology programs if they don’t have one already, along the lines of of how I supported at at at City. So again, taking that knowledge and trying to help others build a program that will see their organization successful. But at the end of the day, too, I really want to see people have kind of that personal and professional individual satisfaction because maybe they’re making an impact they always hope they’re making or maybe because we’re helping drive solutions that allow them to go have dinner with their family at night. Right. I know that’s kind of an old adage, but still people like to Well, some people like to have dinner with their family and I some people it’s a great excuse not to go home and have dinner with their family but if I he’d help drive whatever their personal professional goals are. Through my experience, like that’s, you know, that’s, it’s, it’s so exciting for me.

Greg Lambert 25:09
What does? What’s the customer that you’re attracting look like? What? What’s the reason that someone would reach out to you? LexFusion to help them see, what’s a problem that you would help them solve?

Christina Wojcik 25:26
Oh, well, you know, I have a very wide range of, of knowledge, some of it’s useless. But hopefully some of it’s also useful. So I think I’m the member side, what we can offer is not only our incredibly extensive network, you know, our reach into law firms or reach into in house corporations, but also our extensive understanding of what it takes to build a successful organization from the ground up. Right, some of our members are very well established, some of our members have recently been acquired. Some of our members have already gone public, right, but some of our members are looking to build bro scale, and then, you know, potentially, in the future future have an exit strategy. So if I can take my experiences on the services side from Pangea, three, through the technology side from permaseal Software, and help them kind of craft that vision into whatever the leadership’s ultimate goals are, you know, I really hope to be able to help support that on the membership side. But again, like I said, I also want to bring in members that have a bigger kind of allows us to expand our portfolio to have more offerings that will serve as more of the the corporate clients for the the corporate side, you know, what I really hope is, I can help them accelerate their time to bring innovation onboard. And, you know, I think that, you know, I’m getting very excited about the opportunity. So I want to be sure your question didn’t get it didn’t get away from me a little bit. But I think that the the value that LexFusion brings, is both on that membership side helping build companies and and get their products in front of the right people. But then also on our you know, our our other customer side, making sure that we are bringing them the right products that solve for the the challenges that they’re facing. And knowing that it’s coming from a trusted source that we’ve been through with you, we’re not here to do anything other than to support your your goals inside.

Greg Lambert 27:32
Now, there was a an interview you did last week, week four with Caroline Hill, who’s who’s great with the Legal IT Insider in in that you said something that kind of caught my eye with your experience at Citi. And we kind of talked about it here. And that, you know, you don’t necessarily want to do something that will cause the whole, you know, global financial infrastructure to crumble. But that, you know, that, that also kind of can can can prevent a, you know, a good conducive environment to thoroughly test things to really kind of reach out and try new things. And you talked about, like failing fast was actually a good thing. I would love to scratch it that a little bit more. What would you mean by failing fast is actually a really good thing.

Christina Wojcik 28:30
Yeah, it sounds a little counterintuitive, right? Where no one ever wants to have the word word failure anywhere near their their name. But, you know, I, I think that failures, especially

Greg Lambert 28:42
in legal,

Christina Wojcik 28:42
it’s actually yeah. And I think that, you know, what, when I, when I had joined city, they had many different types of technology support available, depending on whether you are, you know, in on the business side, the legal side, more of kind of a overall strategy for city as a global organization. And historically, within our global legal support on the technology side, they’re an incredible team at taking a solution that has been selected, implementing, deploying, you know, building it out, and then moving that into a business as usual Bau status, which is essentially just ensuring that the technology continues to behave as expected, making sure we’re putting the right patches on it, making sure we’re getting the right updates to it. And this is more probably appropriate for when we had everything on premise. You know, as we move to the cloud, it became a little less important because there’s, the lift is different when things are on cloud versus on prem. Right. But what I what I found is that as I’m exploring it as I’m looking at cool technology, especially in the emerging category, I want to be able to tinker with it in a safe environment and understand whether or not it truly works against our use cases. Because the onboarding process says it could take up to two years, right? So if I pick something, and then I go down the road, and two years later I go to implement and deploy and find out Yikes, that didn’t quite meet the brief because maybe, maybe a salesperson got one past me. And I didn’t quite see the demo magic, or maybe we didn’t test it in the right way during like a POC or pilot. So I wanted to, you know, put together an area within the organization that would allow me to really rigorously test different products against my my organization’s legal organizations, and also partially business organizations, business requirements and use cases. And I also wanted the ability to, to look at building technology stack solutions, especially when it came to regulatory responses, right? How can we reduce the amount of manual touches that we have, during a large scale regulatory response? How can we add automation to it? And I love to use analogies of this may or may not hit my former team at City. Vicki, James, Janine, Brian, Diane, if you’re listening to this, you’re probably laughing at me wrong. So I’m gonna try an analogy. So when I when I think about it, my innovation lab that we built, it was based in London and Tel Aviv and components coming out of Dublin and New Jersey was really kind of the the place where we made the sausage now I love vegan sausage, I love meat sausage, this is generic, just sausage, right? So the lab allowed me to kind of build the recipe, right? So we could source different types of of products, we could look at different types of combinations, different types of flavor categories, look at different texture options in a way that we could quickly say, you know, we thought that by adding blueberries that was going to make the sausage delicious, it did not let’s remove the blueberries and try maybe fennel instead. Right? So it allowed us to kind of like play a little bit before we had that final solution architected to move over to deployment and to Bau. So at the end of the day, what comes out of the lab can be handed over to that technology function that is incredible at those things that I mentioned, implementation, deployment, and moving to view. So we go to the sausage factory created incredible sausage, and then we can move that into handing over to the grill master handing it over to the hosts to distribute. And then we have the recipe that we can use again next time, right. And then when we want to do something new, we can go back into the lab and start tinkering again. So it gave us the freedom to to pull things in to move things out in a way that did not commit us to a two year onboarding cycle prematurely. And so we had much better architected solutions coming out of the lab, and accelerated some of that deployment and implementation challenges that we may have struggled with previously. And by the way, can I say follow now

Marlene Gebauer 32:56
I’m hungry. I’m starving now. So it’s like, I gotta go get breakfast. So Christina, we’re we’re just about at the end of our time, and it’s been fantastic. And usually how we close is that we, we ask all of our guests our crystal ball question. And that is, what challenges or changes do you see in the legal industry over the next two to five years?

Christina Wojcik 33:26
So I think some of the changes are absolutely going to be in the the legal tech space, right? I think that there it is ripe for some consolidation, I would anticipate, especially in the CLM space, if you go to any conferences, you see 800, new CLM vendors every single year, that will start to start, we should start seeing a consolidation of that market, we need to we absolutely need to because it’s muddled. And it’s no longer good enough to say that I’m different because I have AI, right? Yes, you’re different. Now, if you don’t have AI, and I’m all for.

Marlene Gebauer 33:57
you and everybody else get getting lost, get lost. Yeah, you know,

Christina Wojcik 34:02
and I think we’re also going to see those control factors coming in over the generative AI ai. So those organizations that are bit more risk adverse, will be using it in production instances, because they understand kind of the nature of the beast, what it can do, and what the opportunities to to bring value to the organization in the appropriate control framework. So I think that those are some of the things that we are are absolutely going to see. And, you know, the final thing that we we might see, and we talk about this all the time, so I’m even hesitant to say it but, you know, kind of war pressure on the law firms from new and innovative ways to provide legal advice to organizations with technology as a foundational layer. So instead of building a firm, looking at potentially law firms as our lawyers as like the first key component, potentially looking at technology As the first key component and then building the lawyers on top of the technology, don’t help but think that’s very, very intriguing.

Greg Lambert 35:07
Christina Wojcik, Managing Director of Corporate for LexFusion along with now I think we’ve we’ve added up 20 jobs that you have, I want to thank you very much for taking the time this week, especially since you’re you’re in New York, doing other things to come on the show and talked about us. With your new role there with LexFusion.

Christina Wojcik 35:29
Thank you so much.

Marlene Gebauer 35:31
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague, we’d love to hear from you. So reach out to us on social media, I can be found at @gebauerm on Twitter, LinkedIn, and @mgebauer66 on Threads.

Greg Lambert 35:48
And I can be reached on LinkedIn as well at glambertpod on x and glambertpod on threads and blue sky and all the all the other things that are out there right now. So Christina, if someone wanted to learn more was in you know, they wanted to reach out, where’s the best place you could be found online?

Christina Wojcik 36:09
You can absolutely reach out to me, LinkedIn, but you can also email me directly at Christina.Wojcik@LexFusion.com. And I look forward to hearing from everyone.

Marlene Gebauer 36:19
listeners can also leave us a voicemail. Greg, we still have voicemail. Sure,

Greg Lambert 36:25
we’ll say okay.

Marlene Gebauer 36:27
Think you could still leave us a voicemail on our geek and review Hotline at 713-487-7821. And as always, the nice up here is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 36:38
Thanks, Jerry. All right, Marlene, I’ll talk to you later.

Marlene Gebauer 36:42
Bye bye

This episode of The Geek in Review podcast provides an in-depth look at how the AI assistant Paxton, created by Tanguy Chau and Mike Ulin, is transforming legal work. The hosts speak with the founders of Paxton to explore the pain points their technology aims to solve and how generative AI can enhance lawyers’ capabilities.

Tanguy and Mike discuss their backgrounds in AI, regulatory compliance, venture capital, and management consulting. This diverse experience informed their vision for Paxton as an AI assistant specifically built for legal and compliance professionals. They explain that Paxton is trained on millions of legal documents and regulations, allowing it to search this vast knowledge and retrieve highly relevant information rapidly. A key feature they highlight is Paxton’s accuracy in citing sources, with every sentence linked back to the original text.

One of the key features of Paxton is that it can automate repetitive, low-value legal work to make lawyers more efficient. Tanguy notes that tasks like reviewing thousands of sales contracts clause-by-clause or compiling 50-state surveys that once took weeks can now be done by Paxton in minutes. Mike discusses Paxton’s advanced document comparison capabilities that go beyond keyword matching to understand meaning and intent. This allows quick, substantive analysis of contracts, marketing materials, and more.

Exploring the future, Mike predicts that like software developers, lawyers who embrace AI will become much more productive. But higher-level strategic thinking will remain uniquely human. Tanguy shares an analogy of a human on a bicycle outpacing a condor, the most efficient animal. He believes combining human creativity with AI tools like Paxton will enable radically new levels of efficiency and capability.

Paxton.AI’s Tanguy and Mike make a compelling case that AI-powered tools such as Paxton will fundamentally transform legal work. By automating repetitive tasks, AI will free lawyers to focus on high-value, client-facing work. Overall, this episode provides great insights into how generative AI may soon become indispensable for legal professionals seeking to improve their productivity and capabilities.

As a special treat, we wrap up the interview with a demonstration of Paxton.AI’s capabilities. (YouTube only)

Links:

• ⁠Paxton AI (try the Beta for free)⁠

• ⁠Forbes Article: Unlocking The 10x Lawyer: How Generative AI Can Transform The Legal Landscape⁠

• ⁠Using Generative AI to analyze the 45 page Trump Indictment using Paxton AI⁠

• ⁠Unveiling Paxton AI’s Newest Features: Boolean Composer and Document Compare⁠

• ⁠Instantly Analyzing the Congressional UFO Hearing with Generative AI powered by Paxton AI⁠

Transcript:

Greg Lambert 0:02
Welcome to The Geek in Review the podcast focused on innovative and creative ideas in the legal profession. I’m Greg Lambert and I am flying solo for a little bit. I’m hoping Marlene can jump on while we’re doing the interview, if not, everyone is stuck with me today. So I want to start this off by warning the audience that we’re probably going to get pretty geeky on this week’s show, as we dive into some examples of what generative AI can do with specific legal task with the guests that we have this week. So we’re really really excited to have the cofounders of Paxton.ai Tanguy Chau, CEO and CTO Michael Ulin. Glad you guys can both join us. And hopefully we can all geek out here. So Tanguy and Mike, welcome to The Geek in Review.

Mike Ulin 0:55
Thanks for having us, Greg. Yeah, excited to be here and a big fan of the show.

Greg Lambert 1:00
Thanks. Thanks. Thanks,

Tanguy Chau 1:01
Greg. It’s such an honor. Thank you.

Greg Lambert 1:04
So it’s great. Great having you here. So Tanguy, let’s start this off. Because not everyone may have heard of Paxton. Yet. Do you mind giving the audience a little bit of your background? And the history of Paxton and why you and Mike decided to launch this new venture?

Tanguy Chau 1:23
Yeah, absolutely. Well, Greg, like every good company, the story of Paxton starts with the relationship between Mike and I as friends and as founders. Mike and I are AI engineers. We are enterprise experts. I did my master my PhD, my MBA from MIT. Mike and I met a decade ago, we work together while at McKinsey and Company. This is the largest management consulting firm. And we were strategic advisors to a lot of fortune 500 companies. But really what sets us apart as legal tech founders is that both Mike and I have a deep understanding of Legal and Regulatory Affairs. Now on my end, I spent almost a decade as an investment advisor in the venture capital industry, eventually serving as the Chief Compliance Officer for an SEC and FINRA regulated, firm. So it’s really through this personal experience and shared desire to help regulatory and legal professionals make sense of millions of laws, rules and regulations that Paxton was born.

Greg Lambert 2:35
And Mike a little bit about you.

Mike Ulin 2:38
Yeah. Thanks for having us on. As Tanguy alluded to, we got a, we met and became friends while doing a stint at at McKinsey and Company. And then after that, I made my way into the world of legal tech, working in data science for a company called RPI x, which was involved in serving the General Counsel’s Office of large tech firms on intellectual property matters. So work they’re developing natural language processing algorithms, working with with patent data. And then I was part of the founding team, along with a few other ex McKinsey colleagues of a company called zesty AI, where I was the founding AI engineer and, and head of AI for XSD, where we focused on building enterprise AI models for the US property insurance industry. So we were actually the first AI model approved by the California Department of Insurance for use in underwriting and pricing of property insurance in that state. So worked very closely with regulators there, and then worked closely with regulators in 10 other states to get our AI models have proved. So bring, bring a deep experience in AI to the space and we’re really excited about the possibilities of applying generative AI in the legal context.

Greg Lambert 4:09
Let’s say we’re riding up the elevator together. So I’ll let one of you pick who wants to give the speech here. But if you were going to introduce me to Paxton and what it does, what what’s the what’s the elevator speech that you give?

Tanguy Chau 4:26
Yeah, I’ll volunteer for that. You know, Paxton is really a generative AI assistant that is built for legal and regulatory compliance professionals. That’s really what it does. And unlike other assistant it is really trained and build from the ground up. We have painfully collected millions and millions of regulation rules. Statue use and use this material to train a very accurate legal AI assistant.

Greg Lambert 5:08
And Mike, I want to cover some. Because usually, when people create something like this, there’s some problem that you’re set out to solve. And so can you. And I know that a lot of what you’re what you’re doing and you’ve elaborated on in the media is about the biggest pain points and inefficiencies that you see in the legal sector. And so can you elaborate on on some of those pain points that you’re trying to solve? They’re Tanguy.

Mike Ulin 5:41
Absolutely great. So a lot of this is kind of born out of our experience in the legal and regulatory space. So, Paxton is really designed to take a lot of the pain points away with regards to legal and regulatory research. So having done a lot of regulatory work in the insurance sector, you know, insurances is one of those areas that is not regulated at the federal level, it’s actually regulated at each of the individual 50 state levels. So if you’re a a national player, like some of the companies I worked with, at my prior company, we we worked with companies like Berkshire Hathaway, farmers, AAA you have to figure out the different regulatory regimes of each one of those those 50 states, and then talking to legal professionals, this can be a pretty daunting, tedious and, and very labor intensive task. So you know, what we’ve done with with Paxton is, we have all of the laws, rules and regulations from not only the federal level, but also each of the 50 states. So one, one particular use case that’s been gaining a lot of traction with our clients in, in financial services, or the insurance industry is conducting 50 State Surveys in a matter of minutes. So you can simply ask Paxton for what the regulatory regime is on a particular topic in each of the states and get a response back that you can then use as a as a starting point for your research, rather than, you know, spending a lot of money doing this, or, you know, we’ve heard that these things can take months to complete,

Greg Lambert 7:39
I can verify that it takes a long, long time to pull those together. And, and so now I think a lot of partners are going to have to figure out new projects to give summer associates because that’s a big task that a lot of summer associates get is take this issue and give me a 50 state survey. Now a smart associate would then do it in the 10 minutes, that it would take and then wait around for a couple of days and then hand it in to them. So it didn’t seem like it only took 10 minutes.

Mike Ulin 8:18
You know, and we want to we want to empower smart associates here at Paxton. Here we go.

Greg Lambert 8:24
So, Tanguy, you recently wrote an article that was in Forbes, and it was called unlocking the 10x lawyer how generative AI can transform the legal landscape. And in the article, you discuss how generative AI can help lawyers become more efficient by automating low value tasks. So I think most of us probably have some examples. But I want to I want to see if you can provide, you know, some examples of low value legal work that you specifically think can be automated with with AI. And, you know, how much time do you estimate that this could actually save a lawyer say, even on a weekly basis?

Tanguy Chau 9:11
Yeah, absolutely, Greg. So let me start by defining what we mean by the 10x lawyers. And what we mean here is that with the proper tool, with the proper support, a lawyer can be 10 times as efficient as a lawyer that doesn’t have access to tools like Paxton. And so, you know, let me let me share in this piece, right, we analyze a survey that Wilson Sonsini, one of the largest law firm in the world did, and that survey explains that 67% of in House lawyers feel buried in low value, highly repetitive work. And let’s give some samples here, right? You look at an m&a process, and a lawyer might, a law firm might need to look at the sales contracts of 1000s and 1000s of vendors, and go through the through the language of all of these sales contracts, and it’s very repetitive. They have to look clause by clause over all of these documents, and, and identify variations in the language as it’s being used. It takes hundreds of hours to review each of these contracts. This is a perfect use case, for an AI agent such as Paxton. A second example, for example, would be, you know, when when we talked about 50 State Surveys and, and looking and reading vast knowledge databases or fast amount of information, it requires reading through 1000s of pages, just to identify and find critically important information that is relevant to the to the case, Paxton, for example, is able to read through 30 million pages of legislature has perfect memory, perfect recollection and accurate retrieval of the most mundane, but critically important information. And so what we think is, you know, this is this is what Tanguy is outstanding for handling very mundane, very boring, repetitive work, and to free attorneys to focus on the high value work, the highly engaging the client facing work that all attorneys and all lawyers love to do.

Greg Lambert 11:47
So, I want to ask you, because we had a guest on a couple of weeks ago in Marlene has joined us as well. Hi, Marlene.

Tanguy Chau 11:57
Hi, Marlene, Marlene.

Greg Lambert 12:01
Tanguy, I did want to follow up on that we had a guest on believers last week that talked about how you can structure the generative AI, even even when it’s trained on, you know, legal specific things that because of the because of its ability to be creative, you know, it could take it could take liberties with the with quotes on it, change things around. And so he talked about some ways that they did the multi token, I forget how multi token authorization or multi token something but basically where material that is like quoted stays the same so that you don’t get the any type of loosen hallucination. What are you guys said

Marlene Gebauer 13:00
they’d like they’d like Link words. So it wouldn’t be just like separate words. They’d link it so that things wouldn’t get separated? And it was there less of a chance for hallucination

Greg Lambert 13:11
and how it worked. Yeah, and especially in legal where you want to be very specific and precise. When you’re quoting, legal materials. So how are you looking at? If I’m going to cite something, that I am assured that the answer that I’m given matches what’s in the documents? Is that do you link to the documents? Or do you have something else going on in the back end to kind of make sure that you’re, you’re getting very good answers?

Tanguy Chau 13:49
Yeah, absolutely. I think this is a this is a very important problem that we spend a lot of time thinking about how do you actually do this accurately? There are many different ways that one can approach this problem, both from a technical approach different techniques, and machine learning techniques that one can can use. What we have focused on and what has resonated with our users is very accurate sourcing. So for any, you know, the use of AI, really, it’s really important that the user trusts the answer. And there’s a saying that says trust but verify. And this is what, Paxton is really good at every sentence, every word, every paragraph is has a footnote that you can verify the where the information is coming from directly to source text to source regulation and it takes you to the source of the information. So this is or approach or approach is you The the ability to very quickly verify that the answer is given. Because eventually, you know, it is still on the user’s responsibility to check and make sure that the information that the tool provides, is, is accurate when they when they file a motion, for example.

Marlene Gebauer 15:23
And is it a footnote? Or is it like a reference in in the text?

Mike Ulin 15:27
It’s a, it’s an actual citation in the text. And, you know, folks are free to, you know, check it out for themselves at Paxton.ai. Ai. You know, we’re offering free free beta access right now. But you can actually, when you get a response, every response comes with a footnote, as Tony was saying, and that is a link direct back to the primary source material. So you can link to the regulation or the statute or the rule, or, or materials that you upload yourself to Paxton. So you can verify what what the system is saying is accurate? Yeah.

Marlene Gebauer 16:07
And it’ll go back to that specific area. For reference, yeah,

Mike Ulin 16:11
it won’t. It won’t just, you know, link you to the entire

Marlene Gebauer 16:15
California section of the tax code.

Mike Ulin 16:19
Just an entire chapter. Yep, there you go.

Marlene Gebauer 16:25
There’s my weekend.

Greg Lambert 16:26
Yeah, I’ve actually played around with it. It’s, it’s really cool. And thank you guys for letting it during, during this phase of you guys launching, letting everyone take a look at it for free. because not everybody’s doing that.

Mike Ulin 16:45
Ya know, we’re very feedback driven culture here at Paxton. And, you know, we’d love for and thank you for, for testing out the the tool, we’d love for your audience to take a look and let us know what you think we’re always on the hunt for ways to improve it.

Marlene Gebauer 17:03
How specifically, are you designing Paxton.ai use of artificial intelligence to augment the capabilities of lawyers? What is unique in the way that you’re leveraging AI now that you may not have that may not have been possible, just a year or two ago?

Mike Ulin 17:20
Sure. Happy to have to take this one. So I’ve been in the field of AI for for, you know, my whole my, almost my whole career over over a decade now. And we really designed Paxton with with the needs and workflows of an attorney in mind. And really taking advantage of the new generative AI, the new capabilities that the generative AI ai offers, offers today. And, you know, if folks want to get a sense of, of how that’s how that’s different. You know, you can really see it for yourself, you know, just just take a look at what’s available, you know, when you do autocomplete on, on your phone, or autocomplete on, on Gmail, you know, you would just get a phrase or a couple words, as a suggested next thing to say, and that was kind of the state of the art for a while. But but, you know, today, and you can experience this with yourself for yourself on on, on Paxton, we can now generate, you know, entire documents, based on on really accurate source materials. You know, we have models at Paxton that can take into the, you know, can take into account like the context of about a book worth of material at a time, versus, you know, what we were what we’re dealing with just a few years ago, where it was like, what’s, what’s the next word that I want to take someone? So I think that that is kind of the the change in how AI has, you know, what AI systems are available today that’s really enabled these types of use cases.

Greg Lambert 19:18
So, you guys have a really good presence on medium. And I think you’re doing a really good job of kind of reaching out beyond the, you know, the Twitter, LinkedIn, although you guys do a great job on on those as well. But, you know, on medium, you write some, some very specific and interesting ways about how to use Paxton AI. And so we want to go over just a few examples of some of those articles that that we’ve read through and Tanguy I’m going to start with you. There was an article on Paxton’s drafting capabilities. And in the article, it mentioned generating documents from templates. So can you expand more on how lawyers can customize those templates? And do they have some flexibility to develop templates attune to their own specific needs?

Tanguy Chau 20:24
Yes, absolutely, I think, let’s remember, right. Paxton really is an intelligence layer that sits atop different knowledge databases. And so you can really ask Paxton, to perform different tasks to adjust and to adapt, you can use the curated database that we collect, and we maintain, and we monitor, such as, you know, the, the federal and state statutes, rules, regulations, and more to come, you know, obviously, but you can also ask Paxton.ai intelligence, and apply it to your own knowledge databases. And you do that by simply connecting it either to your document management system, or uploading your own set of documents or cases, or your own templates and the language that you want Paxton to use. And so you can then ask Paxton to read, and Paxson will adopt your template, and respond in the style and interest in the style and then draft following, you know, draft motion drafts legal memos, following your custom firm templates. And so this is really how we’ve made Paxton easy to, to find for our users to fine tune to their specific knowledge, as well as their specific templates.

Greg Lambert 21:59
And, and just a follow up on that one. Because I can hear my, my IT security guy screaming right now. So how do you work with, with with firms, with lawyers, with your users on making sure that that information is secure, and that someone else won’t be able to get access to that?

Mike Ulin 22:26
Yeah, so we have a very deep background and serving large enterprises. So so both Tanya and I served very security, conscious, fortune 500 companies in our, in our prior life at McKinsey, and, you know, when we were developing AI models for the insurance industry, you can imagine those, those companies also had very strict security requirements as well. So we really took that that experience and you know, that knowledge of what, what organizations care about, and that’s, that’s built into the design of Paxton. So we meet all the requirements for for sock two. And, you know, we work with security teams at law firms all the time to understand their individual requirements, and really tailor Paxton.ai security settings and data retention policies to meet the needs of the firms that that we’re working with.

Marlene Gebauer 23:31
So as law librarians, the article on Paxton Boolean search composer caught our eye and, you know, I think there there are still librarians out there that say, you know, boolean is the best is the best way to search and that, like natural language is kind of the, the death of the ability to, to get really down and granular in your search. And while while, you know, experts, searchers are very good at this sort of thing, you know, sometimes more casual searchers are not. So, you know, what are some of the examples of complex legal research questions that the search composer can simplify that, you know, lawyers and maybe more casual users struggle with? You know, how does the explanation help lawyers learn in the process?

Tanguy Chau 24:23
Yeah. Yes, this is, this one is a very, very popular feature within the Tanguy platform. And it’s a feature that was requested by our community. I think, I think it speaks to like the how Mike and I are thinking about being very building a very user and consumer centric for a company first so. So for the audience. We built a, a Boolean composer and Uh, what we heard is a lot of our users didn’t enjoy or didn’t like writing Boolean queries. You know, they’re complicated. They’re unintuitive. Shocking, shocking, right? Yeah.

Mike Ulin 25:17
This net natural language thing is just a fad. Yeah. Right.

Tanguy Chau 25:22
But Marlene, I mean, it’s so true, right? We were forcing users to adopt the language that machines use, right? It’s not like you and I, like speak using Boolean, right to read speak using natural language. And so instead, we thought that instead of asking users to adopt the language of machines, we reverse this process. And we, you know, we had Paxton learn the language of users instead. And so, you know, instead of coming up with complex Boolean expressions, or users have the ability to very simply explain in plain English, what do they want from Paxton, and it might be retrieving all of the cases for over the last 10 years for a particular issue. And, you know, and if and when needed, Paxton can transform these requests into clear Boolean expressions that the users can then use in tools like LexisNexis, and Westlaw. And then Paxton will also explain how this expression was built stage by stage what each of the Boolean queries is used for. And so makes it very easy for a user to draft a Boolean query, but also adjust it adapted and, and takes a lot of the mystery out of Boolean searches.

Greg Lambert 27:00
Well, that’s interesting, because I feel like definitely welcome. Yeah. Well, and one of the things that and I know we do it here at my firm, and I’m sure other firms do the exact same thing is, like, like, if somebody is searching on Westlaw, and they’re having problems, we suggest that they call one of the research attorneys at at Westlaw, that helps them basically create a search structure for that. So you know, I don’t I don’t want to run the people out of out of a job up in Eagan, Minnesota. But, you know, sometimes you don’t necessarily want to talk to somebody, you want to get something that you just want to figure it out on your own. And I like the fact that you’re not just giving giving an answer, but you’re actually explaining how you’re getting to that result, which maybe that will teach some of the attorneys how Boolean works. So

Mike Ulin 28:01
absolutely. Yeah, you know, we’re just a very, I mean, this this, as Don, you mentioned, this was just a suggestion from from one of our users that we then took and adapted to the platform, just just to make the lives of attorneys just a little bit easier. So we’re glad folks that our folks are getting use out of it.

Marlene Gebauer 28:21
And the transparency builds trust, so yep, that’s good.

Greg Lambert 28:25
Absolutely. So So Mike, the there was another piece that featured document compare. It talks about what what’s the logic behind the the document difference? In highlighting? So does it just look at the wording of the document itself? Or is there some kind of more semantic analysis going on? It’s identifying the, you know, the substantive differences in the documents when you’re comparing them?

Mike Ulin 28:59
Yeah, great, great, great question. And, you know, this this feature of Paxton is really meant to go beyond just like the the redlining or track changes that that folks might be used to, it’s not limited to just keywords or, you know, sequences of words in the document, it actually looks at the meaning and intent of the text in the document. So you can use it to say, compare it to NDAs. You know, maybe you have a template NDA that’s been approved and one from a vendor that you want to understand you know, what their their similarities and differences are. And Paxton can really quickly in just a matter of seconds, highlight what what the substantive differences between those two documents are. And it doesn’t matter if those claws Is are in a different order. Or if you use a slightly different language to describe something, Paxton is able to just tell you, you know, this particular clause covering this particular issue is present in one document, but not in another. And that really can save folks a lot of time in what used to be a pretty, pretty tedious exercise. You know, other other clients are using it for, say, marketing collateral review. So if you have a set of requirements or, or a regulation that that covers your marketing material, you can just plug that into the document compare tool and see if the document you’re evaluating like a webpage or a piece of print collateral complies with the requirements that you want to evaluate. And it just saves a lot of time versus versus what folks were used to doing before.

Marlene Gebauer 31:02
So you seem to have a little fun with the topics you use for your articles by looking at real world examples of how Paxton can be used on things like the congressional UFO hearings, or the ability to quickly go through most of the recent indictment of former President Trump. How was that marketing working? I mean, are you getting a lot of response from that?

Mike Ulin 31:26
Yeah, we, you know, we like to have a bit of fun, and, you know, I think demonstrating a lot of these, these capabilities on on topics that are all topical, and you know, in the public consciousness helps really bring home the value, you know, even you know, even though folks will probably not use this to analyze a UFO hearing, they’ll probably use it.

Marlene Gebauer 31:54
Never know.

Mike Ulin 31:58
But a lot of UFO attorneys in your audience, Greg and Merlin, yeah,

Marlene Gebauer 32:03
we’re gonna find out.

Greg Lambert 32:04
It’s definitely a growth industry. So we’ll see how that will speak. Speaking of the congressional UFO hearings, in your article on that, you the analysis mentions being able to instantly process a lengthy transcript. So what’s the key to Pakistan’s natural language processing that enables such a quick turnaround time on that? And and how do you train the AI model to handle that large amount of text?

Mike Ulin 32:43
Yeah, and, you know, we wanted to pick this more fun example. But you know, what, what we’ve heard from from users is, you know, there’s a real pain point about working with video content, whether it’s an agency hearing a congressional hearing at the federal or state level, or, you know, depositions or other types of long form content that, you know, you may be pressed for time, and you have to really get the key takeaways from it rather quickly. And what, what really enables this is Paxton.ai ability, like, like we discussed earlier, Paxson is ability to understand a large amount of information at one time, you know, in the generative AI space, this is what’s known as the context window. So the the amount of information that Paxton is going to consider before providing you an answer. And, you know, as opposed to some of our competitors, Paxton is able to take in a lot more information at once. And, and we really see the returns on that, in terms of the quality of our answers. And in terms of the quality of the responses. Paxton generates, so Paxton.ai Paxton.ai trained on on legal data, and, you know, it really becomes a very flexible tool. And so it doesn’t matter if we’re applying it to UFO hearings or something a bit more serious. We can we can use this tool in a wide variety of areas.

Marlene Gebauer 34:30
When Paxton does analysis on these large documents or transcripts, what specifically does the summarization capacity focus on extracting? I mean, are you using you know, semantic are you using Are you it’s the amount of information submitted generally is the amount of references of a particular thing. So, you know, does it identify or does it specifically, you know, identify people charges, key evidence quotes. How does it work?

Tanguy Chau 34:59
Yeah, I mean I think this is this is what’s really special about about this generation of of AI tools, right? The flexibility that it has. So for example, if talking about the documents of the Trump indictment, you can use Paxton to summarize indictment. Generally, you can ask Paxton to identify which people were charged, what are what were they charged with? What are the key evidence? What are the specific quotes. And I think this is something that wouldn’t have been possible a few years ago was prior generations of AI systems. But really generative AI has an understanding of the intent behind the queries and an intent of the source documents. And and that’s what makes this so powerful, especially in the legal field is that it’s no longer searching for words, it’s no longer searching for phone numbers or pattern it really is searching for what is the intent of the user. And this is, this is something that, you know, users games have tremendous flexibility and how it’s being used.

Greg Lambert 36:27
So we kind of have danced around this a little bit. But they’re kind of a common phrase, it’s saying that it’s being said, and that is that generative AI tools like this aren’t going to replace lawyers specifically, but it’s going to give a significant advantage to lawyers who leverage the AI tools against those who don’t. And so as we, as we probably see this happening more and more as as AI TAS take on more of some of the base work that that lawyers do, how do you how do you see the day to day work of a lawyer changing? And do you have any suggestions or thoughts on some of the skill sets that the AI enhanced the lawyer is going going to need as far as a skill set, or even a specialization?

Mike Ulin 37:31
Sure, you know, and I think this is, this is just another another tool. And I think that’s what, what really makes us makes us human is the ability to leverage and use tools that can actually make us more efficient and enhance our work. You know, just think about the switch from looking up materials and in a book or a library versus now you have Westlaw and LexisNexis, you can think about this as kind of a new iteration or wave of that. So I think really partnering with these, these tools, and learning how to utilize them will be will be critically important for attorneys in the future. But you know, those things that are really, you know, kind of uniquely human, higher level strategic thinking, thinking about how different pieces of of a problem fit together, and really, interacting and guiding clients. Will will be really, I mean, they’re important skills today. And I think those are going to be even more important in the future. But hopefully, you don’t have to, you know, banging your head against the wall reading, reading a lot. A lot of Boolean queries going forward.

Marlene Gebauer 39:03
Damn. Well, speaking of attorneys in the future, yeah, Mike, and Tanguay, we ask all of our guests are crystal ball question. So please take a peek in the future for us and let us know what you see as a change or challenge in the legal industry over the next two to five years.

Mike Ulin 39:26
Yeah, happy to go first. You know, I think you know, it’s notoriously hard to predict the future but I’ll give it a shot. You know, one of the professions that has really readily adopted the use of of generative AI I think a lot more quickly than a lot of the others has been software engineering or software development. And I think if you want to understand a bit of what the future is going to look like we can we can see how this person Session has adapted to generative AI. And, you know, I think there was a study out of Microsoft around, you know, software engineer, software developers that utilize generative AI tools are up to 60% more productive than their peers that do not. And, you know, just thinking about the way I code when I, when I still have time to code. And, you know, our team at Paxton, when they’re coding, utilizing these tools, it’s really dramatically changed their workflow from from a year ago. And they’re a lot more efficient and a lot more productive. And I think we’ll, we’ll see a lot of this in the legal profession, you’ll, you’ll have an assistant like Paxton, that that will make you more effective, more productive, and allow you to get back to, you know, give you more time in your day back and allow you to focus that, on higher value, add sorts of things. And I think you’ll see these, these AI assistants proliferate, not just in legal but but in other professions, like medical as well. Tanguy,

Tanguy Chau 41:16
and yeah, you know, I think, to look 1020 years into the future I, I start by drawing inspiration and looking into what happened in the past. And so I have a, there’s a, there’s a fun analogy, that I think we can learn a lot from. And so in the 70s, I don’t know if you’ve seen that study, but Scientific American did did a study on locomotion across species. And so they calculated the amount of energy that was required to travel a given distance. And so, you know, the study concluded that the Condor was the most efficient animal, you know, an expense, the least amount of energy to go from point A to point B. And, you know, on the other hand, humans spend a lot of energy to perform the same task. And we’re kind of in the bottom half of, of efficiency. But what was really interesting about this study is that it also measured the efficiency of a human on a bicycle. And surprisingly, when, when a human is aided by a machine, they expended the least amount of energy to perform the same task and was even more efficient than than the condors. And so I think this is a really good analogy of what’s going to happen in the next 1020 years. There are a lot of tasks that we perform today, where we’re really not very efficient. And AI assistants such as Paxton.ai, I see them as the bicycles of tomorrow, you know, you combine a human with a machine, and we invented the most efficient way to perform a task, while minimizing the energy spent. And that’s the true power of innovation. And whoever adopts AI tools, like Paxton first, will do the same work faster, better and more efficiently.

Greg Lambert 43:19
Mike, I want to make a quick comment on on your analysis with the software engineering field in law. And this is just a personal anecdote. And that is, so I’ve been using the generative AI tools out there to help me kind of relearn some programming. And, and so it’s been, you know, I would say, measure it in that it makes me much more efficient, because it can give me a great place to start, say, I’m doing a Python script. So it can give me some something to start with. And, you know, and I get to a good result much faster. But I will also say that on the flip side is the net motivates me to now take on additional projects that I would have never have done before. So you know, I just wonder, do you see that in? In the future for lawyers is that yeah, it may may only take me now 10 minutes to get to, you know, something that usually took me two hours to do before. In Do you see the same analogy with say, Well, now I can take on, you know, these other things that I didn’t, you know, wouldn’t have taken on before, as well. Is that how you see it?

Mike Ulin 44:46
Yeah, no, I think that’s a really great example. And there’s there’s a lot of great stories of folks like yourself, being enabled by this, this new set of tools And, you know, you can, you can look at, you know, what happened with with the spreadsheet a few decades ago, you know, in the time before Microsoft Excel, you know, the bookkeeping profession used to be a lot more prevalent. But after, after we made the transition from from physical spreadsheets to software based spreadsheets, we saw a dramatic increase in the number of accountants, then you can think of accounting as a, you know, it’s a more higher level form of that group of work than bookkeeping. And I think I think something similar will happen. With the legal profession with with other professions, when it comes to generative AI, you’re, you’re able to dedicate a lot more time, whether it’s personal time to activities you enjoy, or activities that are more higher value, add more strategic in nature. And, you know, hopefully, that that that’s inspiring for folks, and you know, you, you get to spend a lot more time on things that you actually like doing versus versus just trying to make do with with inadequate tools.

Greg Lambert 46:23
So Well, Mike and Tanguy. From Paxton, Ken, thank you very much for taking the time to talk with us. And tell us more about your work there at Paxton AI. Thank you.

Mike Ulin 46:36
Yeah, thanks for having us.

Tanguy Chau 46:38
Thank you, Greg. Thank you, Marlene.

Marlene Gebauer 46:41
And, of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague, we’d love to hear from you. So reach out to us on social media, I can be found on LinkedIn, and I gave our M on x. And also look for The Geek in Review on threads.

Greg Lambert 46:59
So and I can be reached at LinkedIn or glambertpod on X or glambertpod on threads or everyplace else. Plus, we are launching our Geek in Review channel on YouTube. So that one as well. So Tanguy and Mike, if someone wants to learn more about Paxton, or reach out to you, where can you guys be found online?

Mike Ulin 47:27
Yeah, if folks want to learn more, just check out Paxton.ai. And right now, as we mentioned before, we have free access to Paxton so people can really experience it for themselves. Just sign up with an email at Paxton AI. And if you want to reach Tony and I you can just send us an email at Hello@Paxton.ai AI. And as we said, we’re really hungry for feedback on on Paxton or if you guys want to chat and chat about AI or anything else we’re always happy to.

Marlene Gebauer 48:04
And listeners, you can also leave us a voicemail on our Deacon review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry. Thanks, Jerry.

Greg Lambert 48:15
All right. Marlene, I will talk to you later.

Marlene Gebauer 48:19
Okay, so yeah,

Greg Lambert 48:21
for those of you watching on YouTube, we get a little extra for you that we’re not going to have on the audio podcast. And so I asked the guys coming out of left field here to give us an example of something that they think is cool. So I think Tanya, you’re going to share your screen. Yeah.

Mike Ulin 48:48
Greg’s way of making sure the software works.

Tanguy Chau 48:53
Quality control. So yeah. So there goes so. So this is the plaxen platform. And so you can see you can query curated databases here laws, rules and regulations. You can add videos or audio or deposition transcripts or connected to your databases, you can use a document comparison features. This is the the famous Boolean composer that that we talked about. And so you see we connect to these sources, starting with the administrative code of Alabama, all the way to this Wyoming State statutes. And so if for example, I am a I’m an investment manager and I want to understand what are the rules to sell under Rule 144 This would be my eye select my source. Here, I’m going to query the Code of Federal Regulations, we’re working on an updates where Tanguy searches through the entire body of knowledge. And then I simply hit submit. And what happens when we click Submit is passing reads very quickly and very rapidly, but very accurately through the 30 million documents and growing that are in the database. So you can, it does a semantic match around the language that is here. So the rules to sell under 144, and then retrieve the relevant information. And so you can see Paxton writing, and I’m going to increase my screen here, you can see the response that passage gives, and how every sentence that we have is properly footnoted, when you click on the reference material, you see where it pulled the information from. And so here, we’re connected directly to the Securities and Exchange Commission. We pulled this information from title 17. And as you click, you see the actual language of that Tanguy used to provide the answer. So this is an example of what we do for body of, of knowledge that we curate. In this version, you can, you can create a new folder, a new matter, and then drag and drop 1000s of documents here, and then adapt and, and adapt the knowledge base that Paxson has to your needs. You can also upload videos. And see for example, this is a video that that of the joint hearing about President Biden’s border policies in Arizona and you can query those videos, we have like a quick summary that we provide. And then you have a document comparison, and a Boolean search tool. Such as, for example, if I wanted to look at know, what are all of the rulings, please find me all of the court rulings needed in the last 10 years for someone that illegally operated a motorboat in the state of Hawaii for for example. And I wanted in, in Boolean query to use in in a platform like like Westlaw. This is the search query that we will provide. And this is how the search query is, is built. And so the user can simply just copy the search string and then use them on their platforms. So that’s that’s a quick overview of what Paxton the plaque detection platform.

Greg Lambert 53:01
Awesome. Wow. Very cool. Yeah. enjoy that very much. Thank thank you for sharing for sharing that with us.

Tanguy Chau 53:11
Thank you for giving me the opportunity. All right.

Transcribed by https://otter.ai

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. So we have a very special episode this week with our friend Kris Martin. He’s the Executive Vice President with Harbor. Kris, thanks for joining us and being available. We’re very excited to talk about a number of things that are going on at Harbor, formerly known as HBR. We’ll, we’ll get to that later. So Kris, welcome to The Geek in Review.

Kris Martin 0:14
Marlene, and Greg, thank you.

Greg Lambert 0:19
So it is that time of year again, where conference time, we’re getting ready for the Lynx conference. And so Harbor has the Lynx conference set up this week. And it kicks off with a keynote on unleashing human potential in the age of AI and technology. And of course, we have to talk about AI and technology this year.

Marlene Gebauer 1:04
I like that we’re throwing in human potential in addition to AI and technology.

Greg Lambert 1:09
Absolutely. So Kris, tell us a little bit about links. And then with this type of keynote, what what’s the theme that we’re we’re working with this year?

Kris Martin 1:18
Yeah, well, I mean, starting off with that keynote, wow, it’s those of you who know, Nita come ll her story, and sort of kind of the journey that she’s been through. We’re really excited about that start to Lynx this year, every year, we’ve had the benefit of some some really good keynotes to get us started. This is the time of enhanced focus, yes, on AI, but also on psychological safety on a newer Diversity. And really, if you start to think about where those themes fuse, the less, we have to be focused and thinking about how we’re going to produce a letter to send to our boss or to our client, or so forth, versus what the meaning is, or what the depth of that is, you can start to see where that human power becomes even more important, and even more read through when the machine is writing all of our emails for us. The only way to really differentiate what’s good or bad, what’s true and genuine versus not, is going to really just come down to who we are as humans, and the sort of intersection between that human power and that and that technology. Power is a really fantastic and fascinating topic to dive into.

Greg Lambert 2:25
I imagine there’s a lot of people that want to just have the AI write everything for them right now. So having that empathy, having the humanity behind is probably going to be more important than ever.

Marlene Gebauer 2:39
Kris, I know that you have worked with Gino Grady on the strategic start stop survey. And you’ve guys have collaborated for the past few years. And that the survey covers vendor behavior and AI adoption this year. So what are some of the key AI tools and vendors law firms should be considering right now? How can law librarians help evaluate these tools and guide procurement decisions?

Kris Martin 3:05
Well, let me also first kind of roll back and talk about the Stop stop survey for a second give it give us a little history. Yeah, Jean. And I as many of you that we’ve all known each other for many, many years, Jean launched the start soft serve a good decade ago and such a fantastic topic to really just get a pulse on the entire legal community. What products are we starting to use? What products are we stopping to use? What behaviors are we starting? What what behaviors are we stopping and just just as a as a quick like point in time polls and when you look at that year over year, in fact, when we go back in time and look at old ones, it is a pretty fascinating study, just to see how that’s changed. Is all of it purely scientific? No, that’s almost the opposite of the point we keep the questions predominantly for those of you who have participated in it are predominantly like open ended questions, right? So, so really allowing people to just kind of express what they’re doing. And then we do all the hard work of actually compressing that into useful data. I’ve loved the survey. So Jean did it on her own for years. And a couple years ago, through COVID, she had a bit of a gap. I wasn’t sure if she had sunsetted it or what so we talked and we collaborated and kind of decided like, we can help her with the analysis and whatnot. But it really is still jeans work of art here and a really cool output. So now to answer the question, though, I don’t know that there is really a clear winner. In fact, that’s almost maybe the point there are 93% of firms are moving forward and doing some kind of activity around AI that can be as little as demoing things and exploring it that the 7% by the way are firms that have said we’re not doing absolutely anything with AI right now. We were too afraid to take that next step or to jump into that universe. So but from a product side, I think what’s interesting certainly CaseText had a head start on the universe and in the space and their name come A lot, we do a lot of this just kind of crunching it down to see what names, what vendors what themes repeated themselves throughout the survey. So definitely what that team is working on and activity coming into Thomson Reuters, which happened just as the survey was launching was probably the most prominent name that you’re going to see repeated throughout the survey. While I’m looking for

Marlene Gebauer 5:19
I’m looking forward to seeing the results,

Greg Lambert 5:21
we may look back on it and think that the 7% that are sitting on the sidelines right now may actually be the smart ones. I mean, I mean, think about it links. The seconds conference. Last year, the only AI we were talking about was was extractive AI, which was kind of, you know, kind of a bummer.

Marlene Gebauer 5:41
Yeah, I’ve been around.

Greg Lambert 5:44
Now 10 months has changed the whole thing. Speaking of of links a second year, this is actually the third year coming up. And Marlene and I have had the pleasure of doing the wrap up session for for each of the first two. And we’re gonna do that again this year. So we actually have I think the the easiest job, we just got to listen and take notes, and then color commentary. And then, but, you know, in the three years now that you’ve been doing this, you know, what’s been the value that Lynx has brought to the community?

Kris Martin 6:19
You know, the lakes conference itself really stemmed out of May 1 To just give credit where credit’s due, this was absolutely calling cables, brainchild. But before there was links, there was a survey. And the survey is the bliss survey is really a labor of love that we began in 2016. It’s an absolutely free to participate survey, we put an extraordinary amount of time and effort into into creating the output with the with the survey results, because we know that those who participated put an extraordinary amount of time and effort in in providing us their answers. It’s not a quick and easy, you know, six question multiple choice it, it’ll take most, you know, well well over an hour to go through and complete and pull together all the facts. So we want to give them a solid and quality output. The findings is as that survey started to see change happening year over year, we ask about 80 to 90% of the questions each year are consistent questions that we ask every year. And then we layer in some more current trends type questions. And so of course, you know, no doubt this year is around AI last year was around analytics. So each year we’re gonna we’re gonna make a little bit of modification to some of the some of the changing behaviors, we wanted a different outlet, we and I think we’re blessed really started from was we had a webinar where we shared some of the results. And it we did that through double A double L. And we’re just they were shocked. We were shocked by the amount of interest in in joining that webinar. So we took that and we evolved that into this conference with the idea that weaved throughout each session is facts and figures that come out of the bliss survey, but also really trying to bring in some some fantastic thought leaders, you know, we’ll we’ll put some links in the show notes for those of you who catch this episode with with time to spare. The conference itself is probably the cheapest out there $59 For a participation, we find that a lot of people don’t even bother asking for permission to attend, they just use their personal guard to run that up in part because the entire $69 is actually a contribution to the George Strait minority fund from with double A double Well, so far in the last two years, we’ve given up fold of about $13,000. This year, we’re on track to definitely keep pace with the with the prior years if not exceed it. And even more exciting some of the actual participants or sorry, some of the actual recipients of the scholarship fund will be joining us during the conference. So it’ll be super cool just to like see that full circle of giving and and incorporation of of those individuals. Maybe even one day in the future, we’ll have them as speakers that will even be more cool to to see that all the way come through.

Greg Lambert 9:05
So Kris, I’m gonna shift gears just a little bit. Back in June, speaking of calling cable you Coleen wrote an article and it was entitled leveraging the law firm Library’s untapped value. The library can be a secret weapon for differentiating your firm and bringing previously untapped resources to bear. We had a we had kind of a prior discussion on this last week where Marlene was going, why does the library need to be a weapon? So I like it, I like it. So but in the article itself, you discuss the importance of strategic purchasing practices to mitigate the cost that law libraries assume for that for their firms. You know, so What specific steps in the article did you recommend the law librarians take? When and when something that we actually need to do is make that audit of our library resources. And in using that to negotiate with the vendors.

Kris Martin 10:11
Yeah. So and really every year we call in and I tried to launch a platform paper this year, I think with all the goings on with Harbor, it got delayed a little bit later into the summer into early summer. But it’s typically something we tried to kick the year off with as essentially a platform paper, we had gone a little bit quiet on cost savings type focus, for the last couple of years, I think, not to say that cost and budget isn’t a priority for libraries every single year. But it’s it’s certainly come back into into focus. And so we did bring that one back forward. Again, it is it is one of the areas that that we feel that libraries are going to need to think about differently maybe than they have in the past, I’ve been working with libraries and cost cutting cost savings for geez, actually nearly nearly 20 years at this point, it was the law library that kind of brought me into this space. The big challenge that we’re facing now is cutting is no longer the the way to prosperity, right, we had a decade plus for most firms to be able to continually look at print collections and sort of the more obscure titles and types of things as well, even into the electronic side, to say, you know, we can reduce that in order to reduce cost. Now, we’ve seen pretty substantial consolidation of products to large vendors who are also themselves investing pretty significantly in the new technologies that are really expected of them as vendors. And so trying to really find that balance then becomes a bit of a different art. It’s not simply about how much do I use this product, a lot of the products, you don’t actually necessarily use the way that you may be now paying for them to use. So now it’s about really trying to evaluate what is the right product that aligns with my firm for the future. If I’m already stealing my crystal ball moment, I’ll come up with another one later. But the image I think some of us are starting to form of the future when it comes to these products is the vendors are going to find their way as a content set into the tools that we operate and utilize. You’re seeing already, of course, some conversation around how products like Westlaw and Lexus are integrating with tools like Microsoft, and surfacing themselves inside of the tools that the lawyers are already interacting with. We’re gonna see more of that. And so understanding what these products are, becomes less about the place where your lawyers are gonna go, the fact that your lawyer likes or doesn’t like the interface of product X, that’s definitely still relevant today. And it’s not that that’s going away in next year or the year after. But the interface of these tools becomes much less relevant than the content within them, and the technology behind them. Because as those things progress in advance, we’re going to see ourselves into a different kind of future where the lawyer is really just talking to a completely differently.

Marlene Gebauer 13:16
So Kris, you know, you’ve mentioned, look, we’ve trimmed the fat on the library budget, right, we’ve done that, that job. And now we’re at a point where we still have to continue to look at cost savings, you’ve also mentioned that a lot of our beloved products are becoming more integrated with other beloved products. So these are basically quite sticky. And they’re, you know, the firm’s have them. However, it may prevent exploration into other tools, you know, perhaps even newer tools that don’t have that type of integration that aren’t as mature. So how do you continue to to balance cost savings and then still provide access to essential research tool, legal research tools, and that meaning, you know, looking at some of these new products that could really revolutionize and change the way that we work.

Kris Martin 14:15
And I think that part of the answer to this is about the library and those who are responsible for acquisitions of both new things and existing renewals is understanding and working closely with the permanent strategy. The cost justification around this is probably no longer going to be well, your budget was x last year, and so will allow for maybe a 3% increase in budget, but we also want you to go and purchase this seven figure plus new technology. It doesn’t work. The math itself doesn’t work. Now, does that mean that we can’t get the best possible deal on that seven figure product that you go and purchase? Well, yeah, that’s where our job comes back in and That’s how we need to think about how we’re going to negotiate that the best price on that. But the idea that we’re going to somehow fit these tools, because we’re going to categorize them as research tools inside of a library budget, that’s probably a bit of a fallacy.

Marlene Gebauer 15:14
Yeah, I was I was gonna say, I mean, again, with more of the integration, this could be a cost spread across different departments. I mean, the more integration that there is, some of these tools like when you’re talking about Microsoft, I mean, that’s traditionally an IT tool. But if you’re sort of using aspects of of an information tool that are integrated, you know, perhaps this is something that could be spread a little more broadly,

Kris Martin 15:35
the analogy that I’ve tried to draw this to this is not the same as a product releasing its newest generation. This is more akin to as we, as we talked about the AI tools, specifically, this is more akin to when law firms had to make that decision to put a desktop computer on every single lawyer’s desk at $10,000, a pop was a pretty substantial investment, thinking about doing that. And you know, there were times in the very beginning where only not even just partners only select individuals were granted the computer. And usually those were the ones that had a technology client, so that they could demonstrate that they were using their product, and they were really just a prop to grow business. And I’m sure we’ve all heard those stories. But with time, the firm began to understand the overall value proposition of the tool and cost justify as utilization that’s where we’re at with making kind of financial decisions around these products right now.

Greg Lambert 16:32
And Kris I’m gonna, I’m gonna blow up this next question and go in a different direction a little bit. And I’ll let Marlene pictures pick through the bones of that question, if there’s something that she wants. One of the things I just got back from Vanderbilt University where I talked with a bunch of law students in in a few of the professor’s there, we talked about doing an audit of the resources that the law firms have. And I had mentioned some stories that not just me, but other colleagues of mine have had, which is with the excitement, the hype, the expectations around what generative AI is supposed to do to the law firm environment, just the practice of law itself, it has enabled us to start having conversations that we probably should have been having for the past 510 years, which is you’ll have someone come up and say, hey, I want to change how I practice in this area, or how I do this piece of work. Because I know that ChatGPT can can help me with this, and allows us to then have that conversation of, well, generative AI may do a little bit of that work. But really, this tool that we bought you five years ago, that you asked for and are using, that’s the tool that will help you will change streamline your the way that you practice or or the output that you have, or or speed up or make it more effective. And so we’re having those conversations, are you seeing kind of the same result that when you do the do the audit, that you’re exposing tools that you’ve already had that have essentially been shelfware, for years are now coming out in showing their value? Because we’re actually having that conversation? Are you seeing that? I think

Kris Martin 18:32
there’s kind of two interesting things in this. I will not share the vendor, the product or the vendor name here, but suggests that, you know, there’s there’s an analytics tool in particular, the vast majority of large law firms have purchased, they have accessible on an enterprise scale. So every single one of their lawyers can can utilize the tool, but the actual utilization of this product is minimal. And why now, if you showed the lawyer the output from this tool, and you said, Is this useful? Well, they all say yes, they all love it, they love it to death. But when you actually say, alright, now sit down to start to use it. That leads to the other theme, which is citizen coder. And, you know, the theme of citizen coder is really interesting and legal, which is to start to really get the lawyers more conversant or more capable of having sort of a dialogue in the language of code. Citizen coders in in sort of the conventional sense are using low code, no code tools to create and develop product. But more importantly than that, it allows them to then think about what their process is in terms of a tool or a product or software to then translate it and be able to speak to people who know these things bet. All right, so I’m bringing these two themes together to say the more the lawyers can talk about what their needs are, and the process of what they’re doing in terms of actual technologies, whether they exist today, and they are things that are inside the firm just not being used, or things that the firm or the firm’s vendors should be creating, we now have a little bit more of a circular conversation here where the lawyer can communicate to an internal stakeholder, the internal stakeholder can talk to a vendor and a vendor can come back and sell a product to the firm that actually has utility, maybe a bit of a roundabout answer to your question. But I think this concept of getting the the lawyers a little bit more conditioned to talking about what they’re doing in a framework of what is actually possible with the tools that do actually exist presently, today, again, whether the firm has it, which they may or not,

Marlene Gebauer 20:47
alright, so I like this this concept of citizen coder that you’ve brought up, because it sort of touches on the various shaped lawyers that we’ve we’ve talked about on the podcast about sort of attorneys being more comfortable with, with technology and speaking the language of technology. But you know, yet this remains a real challenge, because, again, that’s not what well, it is not perceived that that is what their day job is that that that sort of tech, that sort of information is not part of what they’re doing. They are lawyering, they are dealing with law, and you know, that sort of the technology is is left for someone else, and the low code stuff does help. But again, have you seen kind of best practices to kind of get that type of engagement from attorneys, you know, what sort of incentives are they given? To do that, that that or what sort of processes are you seeing in firms that are changing to embrace that type of idea?

Kris Martin 21:58
Yeah, it really starts to become more about and it’s it’s certainly few and far between, at this stage, I worry about and I think about really even going back to Casey’s Kia study a decade ago now, Is that about right? I

Marlene Gebauer 22:11
think the will is there, I think the will is there for many. And that’s

Kris Martin 22:15
what I was gonna say I feel like between the fact that generations have progressed in terms of like, new entrants into into the partnership, and just the overall demand and energy around this particular theme, more so than do I know how to write a formula in Excel. It’s do I know how to write up a useful prompt in ChatGPT? That will give me an answer that will actually help me with my practice. I also find that generative AI isn’t all that dissimilar from the Socratic method. And so the actual way the thinking happens is pretty well aligned to maybe the way a lawyer behaves, or thinks, which will make it feel a little bit more natural. And so then this, this starts to then allow us to evolve the conversations, we at Harbor, sit down with groups of lawyers and help them think through that. So have we seen it happen? Absolutely. I mean, we’re, we’re helping to facilitate that happening. But the actual kind of product or outcome from that really is so variable based on, again, the fluency or the ability to be conversant around the tools when we’re talking in hypothetical about things that we don’t understand. It’s like trying to order a Big Mac and in Paris, you do way better if you knew what it was actually called there.

Marlene Gebauer 23:35
Isn’t it lit Royale with cheese?

Greg Lambert 23:37
Now that’s a Quarter Pounder with Cheese.

Marlene Gebauer 23:41
All right, so we’re shifting from the hypothetical, the theoretical to the more practical Kris. The recent rebranding of HBR Consulting to Harbor brings together six merged companies into one unified brand. As the legal industry undergoes rapid change what motivated the rebranding to Harbor now,

Kris Martin 24:02
so from our view, and and I think each kind of grouping or pocket of clients is sort of viewing what’s happened here a little different. From our view within Harbor. This isn’t a rebranding, this is a brand new launch, we are a altogether net new company, bringing together six very impressive and prestigious brands under one roof, we have a singular goal of really being able to provide an end to end solution to our clients. And we look at this in sort of three steps from the assessment of what a problem is to build, identify the problem, understand it, build a general solution or direction around it, to help you implement the answer to the solution that was discussed, whether we help get you there or not. And then finally, as needed to help you manage that and in the ongoing those three pieces really define when we talk about how we can support any legal organization, whether that’s a law firm or a law department and to end It’s kind of across those three areas with sort of a fourth appendix. And this is really for everybody. And this is sort of this is really how we interact with the community is through our insights, just like we were talking about earlier with bliss, just like we were talking about earlier with LINKS Harbor as a whole has an intention to really be in the market to provide the front end the cutting edge information about what’s going on in the legal space.

Greg Lambert 25:22
Now, how big is Harbor now, how many employees roughly,

Kris Martin 25:27
so about 650 employees within the group that I lead, which is research and intelligence, we have about 230. And so predominantly, librarians and other kinds of information professionals.

Greg Lambert 25:40
And I know a lot of law firms go through this when they when they merge, or they acquire, you know, either boutiques or or other firms. So I know you guys have have had a number of, you know, individual companies that have come under the Harbor umbrella. And now that it’s all one, one company, what have you guys done to kind of allow you to realize that, that cross selling, I guess, to the different expertise within Harbor, how are you learning to kind of collaborate within so that you know, when to reach out and get help on an issue that a law firm may have?

Kris Martin 26:21
Yeah, no doubt. Fantastic question. I mean, I think I’ll start very humbly here and say, this is a this is an ever evolving process, that really any naysayer against the way that that an organization like this outside of legal within legal doesn’t matter, whatever be a, when you bring together all these different units, do they function in individual silos. And really, it’s just a composite of parts? Or is it actually truly one company and so the way that you do that is a few things. One is true and actual alignment across one vision and one set of values which we’ve done, this whole process of, of bringing organizations together, started last year, and really just was announced formally to the market a few weeks ago, the majority of the process today hasn’t been about the mechanics of bringing our organizations together, it’s been about the actually bringing of the people together under under one roof with one set of culture, one set of vision. The power here comes with taking thought leaders and individuals from different disciplines and spaces, subject matter experts, and integrating them kind of in a cross function sense into different practices. So we’re set up with four different practices, we’re not expansive out into every little niche does its own little thing. Everything sits within one to four, strategy and transformation, research and intelligence, vendor governance, and strategic sourcing, and legal technology. So everything that we do fits inside of those buckets, those buckets are just as focused on the stakeholders that they serve as the the solutions that they’re intended to deliver. And so by being able to partner then at that stakeholder level, we’re sharing and interacting on needs versus trying to just push through a single solution. This allows us to bring over and cross in other parts of our organization as needed based on based on what we’re hearing from the client.

Marlene Gebauer 28:22
So Kris, I really love the name Harbor. So whoever thought that went up, you know, kudos, kudos to you. I just, I think back about, like, when I used to practice and health care, you had safe harbors, you know, these areas where, you know, hospitals could, you know, you know, have have a safety net. And so I think it’s just a very apt name, I just kind of have this vision of a bunch of people, or a bunch of boats, like in a port, you know, and just, you know, safe from the storm. So, how does this notion of Harbor as sort of a safe port, as a safe point of support and knowledge exchange? How does that resonate with the company’s mission to help clients achieve breakthrough outcomes during industry transformation? Yeah,

Kris Martin 29:10
the overall direction of where we are expecting to go. And I think this really started with the initial vision of why now, why even do this point in time was a true shift in what we saw to be the direction of the law firm. And so as we see, law firms increasingly focus on their back end operation, the hated and dreaded terminology of the non lawyer. You know, what, what’s in a name, right. And there is a fair amount of lawyering these days that’s happening by non lawyers by those so as much as that might be seen as a as a negative term within the law firm. The reality is, a great amount of the actual delivery of law happens by the two of you, by your peer group are the ones who are really making a lot of magic happen. And so building an organization that they’re and designed to support that really became the the wine now for us, we really were at that precipice at the at the point of the development of, of the legal community where, where this was actually demanded or needed.

Greg Lambert 30:20
And I also noticed it, and it took me a couple of days before I realized it shouldn’t have it was that HBR and Harbor essentially had the same letters in in them as well as got it. Kris we ask all of our guests our crystal ball questions. So we’re going to ask you to pull out your crystal ball and peer into the future for us. What do you see on the horizon, that would be a challenge or a change in the legal industry over the next, say, two to five years?

Kris Martin 30:53
Yeah, you know, briefly a bit in passing earlier. But the idea that legal research is really going to integrate itself into lawyer process, the need for and let’s start with, with with the role of the library, or, or where many librarians will start to fit in. Gino Grady had done a post about pumped engineers ability to really think through and evolve, the way that these products are integrating into that legal process will be a pretty key element of the way that lawyers act and behave. And so just thinking how that’s going to work, how we’re going to interact with vendors differently through that process, the pieces that we were talking about as well in terms of lawyers surfacing, new process that will leverage and evolve from the Large Language Models that are out there, and the way that they’re creating the hierarchies inside their organizations. I think all of this is a pretty substantial shift in behavior and not so far on the future that we shouldn’t be thinking about that from how we organize our teams, how we hire new people, the types of resources that we’re looking to acquire, all of those topics should should fall into this lens of we’re probably know there will be a future and date of where people are going to places to get information versus just having it elevated and surface towards

Greg Lambert 32:12
Well, Kris Martin, executive vice president at Harbor, want to thank you very much for joining us on The Geek in Review. And hope to see you later this week at the links conference as well. Again, we’ll put a we’ll put the link on for links on the show notes. Yeah.

Marlene Gebauer 32:32
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague, we’d love to hear from you. So reach out to us on social media, I can be found at @gebauerm on Twitter, LinkedIn, and at m gay Bauer 66 on threads.

Greg Lambert 32:51
And I can also be reached on LinkedIn or glamoured on Twitter, and glamour pod on threads and blue sky. So, again, so many places to post these days of Kris, what about us, if someone wants to continue the conversation or learn more? Where can you be found online.

Kris Martin 33:10
So you can definitely come in through our website is not just Harbor but Harborglobal. So Arbor global.com is the website that that we just launched a few weeks ago and and can connect through there. I personally can be reached at Christopher.Martin@HarborGlobal.com. And if you’re still interested in links, you can either reach out to me directly, or use the links email address is just that word LINKS at Arbor global.com.

Marlene Gebauer 33:42
And so listeners, Greg and I have been talking about retiring the voicemail. So it’s kind of like Twinkies, you know, we’re gonna be getting rid of it, unless anybody uses it. So you can still leave us a voicemail on The Geek in Review Hotline at 713-487-7821 but perhaps not for long. And as always, you hear is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 34:11
Yeah, thanks, Jerry. Alright, Marlene and Kris. See you guys later.

Marlene Gebauer 34:15
Bye. Bye.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. Well, Marlene, you’ve been after me for months years, I’m not sure how,

Marlene Gebauer 0:22
at least a year

Greg Lambert 0:23
To do some type of video interaction and have a have a YouTube channel, whatnot, so so that we could join the cool kids that do podcasting, but then also have a YouTube channel. So I think we finally figured out that we’re going to try something.

Marlene Gebauer 0:42
We’ve stumbled onto a solution? Yes,

Greg Lambert 0:44
we have we have. So I think it’s gonna be a combination of we’re going to make some of our archive available online, and then going forward, adding in video, a video component to the podcast as well. So just wanted to give the audience a heads up. And we’ll be announcing things as we know more, which right now we know very little.

Marlene Gebauer 1:08
So this, this is very exciting, because our our listeners may or may turn into viewers and so they’ll get to see and hear all of the great content that our guests provide.

Greg Lambert 1:23
Yeah. And they’ll they’ll figure out why it is I have a face for radio.

Marlene Gebauer 1:29
I just open that up right for you.

Greg Lambert 1:32
Thank you. You taught me the softballs. Oh, knock them out of the park. So this week, we are joined by Brandon Wiebe, who’s the general counsel, head of privacy at transcend Brandon, welcome to The Geek in Review.

Brandon Wiebe 1:51
Thank you, Greg. Thank you Marlene, really excited to be here.

Marlene Gebauer 1:55
So Brandon, would you mind giving us a little history of transcend and what you do to support the mission there?

Brandon Wiebe 2:01
Yeah, absolutely. So Transcend is a data privacy infrastructure company, by background. And the full mission of Transcend is to develop technology solutions to some of the really complex privacy challenges that companies have faced over the last 10 years. And, by way of example, historically, a lot of privacy organizations were tackling privacy challenges through more manual and organizational measures, rather than true technical solutions that integrated into like the data layer or the tech stack, that where data was actually being processed. And we saw a lot of data processing throughout organizations get increasingly more technical love, on the marketing side, and the sales side, and everywhere, but the legal and privacy side. And so our founders, Ben and Mike, several years ago, decided that a solution that was much more technical in nature, is where privacy and privacy governance was going. And so they founded Transcend. I came on board about a year ago, to lead the legal and privacy organization, which is unique at a company that is a privacy tech company, because it’s all sort of the traditional stuff that you would do, as the first in house lawyer at, you know, a fast growing startup. So there’s corporate and IP and employment and all of that, and our own internal privacy compliance as well. But I also have the unique opportunity to work really closely with our product team and our engineering team, and our sales organization, to help guide where the products are going. And to work really closely with our customers as they are adopting and implementing our products and tools. And, and one of the things we’re going to talk a lot about today, I think is sort of the next chapter and transcends history. And I and I think in a lot of privacy organization’s history, is the adoption of AI and how the mandate around privacy governance is becoming much broader, given a lot of unique, the unique, a lot of the unique risks that AI presents, and

Greg Lambert 4:31
I like how you phrased it as you had the unique opportunity, when really I read that as I’m doing all the work.

Marlene Gebauer 4:42
And I do I do have a follow up. Before we get into the AI. It’s like let’s, let’s take a little trip down memory lane. So you know, you had mentioned, you know, historically that things were done very manually. I mean, do you have a specific example that you could share in terms of Have something that was done very manually that sure that technology eventually helped out with?

Brandon Wiebe 5:06
Absolutely, yeah. And I’ll, I’ll use an example from my own history. So before I was at Transcend, I was leading the product and privacy counseling function at Segments, which was eventually acquired by Twilio. And I continued on in that role at Twilio. But when we were at Segments, we looked around at the technology available to do a data map, to get a sense of where all of the data in our systems resided. And at that point, the solutions that were out there were very manual and processed in manual in scope. And what that meant was, you had to go around and interview every data system owner in your company, and write down what data they said that they had in that system, why they were processing it. And from that, you would be able to build a data map, it would be out of date very quickly, because systems change very rapidly. And you had to trust that the person who was telling you this knew what they were talking about, and that they put in the time to actually evaluate that system and understood what data was in there. More modern privacy technology, like Transcend, attacks these problems from a technical level, so is actually going through systems and programmatically finding what systems may have data in them. And then digging even deeper and analyzing the contents in those systems, and programmatically building a real time data map for you. So that’s an example of where technology helps solve one of these issues. It was much more manual, you know, even five years ago,

Greg Lambert 6:48
yeah, I bet it was eye opening, when you would run this process on a company’s data to determine and, you know, be able to show them, well, you’ve got all of these other data, data repositories, databases, places that store information, and I bet many of those had been built for like one off projects, or they didn’t even know existed. So that’s definitely a, you know, the automation is definitely something that helps uncover, you know, the, the not just the good data that’s out there. But just the the mess of data that can be out there as well.

Brandon Wiebe 7:30
That’s exactly right. That’s exactly right. And privacy and legal are not the only teams interested in finding that to, you know, the security team, very interested, but also the engineering team and the hosting costs of holding on to data that you really don’t need. So it definitely has impact cross functionally as well.

Greg Lambert 7:49
So, you know, we wanted to jump into the AI conversation as well. What types of unique data privacy and AI ethics challenges? Have you encountered as the General Counsel there at Transcend? And, and how do you navigate that to reduce the risk?

Brandon Wiebe 8:09
It’s a great question. So I’ll talk a little bit about some of the internal challenges that we’ve encountered at Transcend, and some of the solutions we put into place. But we’ll also spend a little bit of time talking about some of the solutions that we’re building right now that we hope other folks can implement, as well, as they build out their own AI governance policies internally. So I think like a lot of really fast moving startups, Transcend is really eager to adopt AI, as quickly as possible, and as responsibly as possible. And one of the challenges that I’ve seen is that there are a lot of tools that are out there, that are subtly adding in new AI functionality to a tool that we’re already using, that we already have a contract with. And it can be very hard to identify and vet those, but they do present some unique risks that did not previously exist in the way that we were using the tool in the past, I think so. No, sorry. Go ahead.

Marlene Gebauer 9:21
No, you go ahead. Because say it was it’s not even subtly, like, I mean, everybody’s announcing it, like, Oh, we’re gonna be having generative AI as part of our, you know, existing projects or products. And so yeah, it’s it’s it’s big news.

Brandon Wiebe 9:34
That’s right. And, and I think the volume of tools and the pervasiveness of how these features get added, is very hard for legal teams sometimes to get their hands around or to keep track of. So you know, this, this came up in the coding area, where our engineering team started using GitHub co pilot and thinking through the implications of using something like Copilot to generate code where that code resides. That’s a complex IP problem to think through. And if the legal team is not aware that that’s even happening to begin with, that can be a big challenge. There are also, you know, similar risks presented by net new vendors that have an AI functionality to them. So we use Jasper, for example, on our sales and marketing side to generate some content, and analyzing what we can do with that, how we can use that, the IP rights in that. Making sure we’re vetting it for marketing claims, and being thoughtful about that entire process is another new and unique challenge, presented by AI. So when we began to see some of these new features and new tools emerge, I wanted to take a step back and think about, well, practically, what is the impact of adding these new AI, this new AI functionality to our technology stack? And is there a way that we can more thoughtfully or more programmatically analyze these issues, without slowing down innovation without freaking out the teams that are trying to adopt these. And what we started with was, I think what a lot of organizations are doing, which is a code of conduct around generative AI, that laid out at a high level, what some of the big risks to the organization and to our customers, or to data subjects, could be from using generative AI, and some guidelines, some roadmap, markers that folks can use to understand when it’s okay to use these tools, what use cases are always acceptable, what use cases might present some risk, and you should come to our team to talk to us about that. And we can help guide you. And which use cases are always a no go. And this sort of lightweight code of conduct. adoption, I think does a couple of things. One, it makes folks feel more comfortable about how they’re engaging these tools and using artificial intelligence in their work. And they don’t feel like they have to sneak around and use, you know, ChatGPT on their personal login, they feel like the company has given them pretty clear guidance on what is okay. But I also think it sends a really strong signal that we want to adopt this stuff, and use it to improve efficiency to be at the vanguard of technology development. We just wanted to make sure we’re doing it thoughtfully and responsibly.

Marlene Gebauer 12:55
So hold that thought. You know, we sort of went through this this, like, initially, like everybody loves generative AI, and then we were like, everybody’s afraid of generative AI. And now we’re kind of at the point where it’s like, well, we kind of have some healthy skepticism about Gen AI. So, you know, in your view, well, you know, where do you think corporate government is lacking? And I mean, you know, you may have addressed that just now, when it comes to oversight. But given the, you know, the constant change of, of the tools, the constant change of how we interpret, you know, what our relationship should be with these tools, you know, how should those governance models evolve in a timely way?

Brandon Wiebe 13:50
It’s such a good question. So I would say, I think corporate governance, right now has a lot of policies in place already for for, you know, robust organizations, mature organizations, that touch on some of the risks presented by AI systems, for example, a company that has a really robust privacy governance, or is going to be able to vet AI tools for a lot of the privacy risks that are presented by those tools. And so I do think, you know, we think about AI as this totally new technology, but it shares a lot of similarities with other technology that we’ve encountered previously. Right. So I do think that there is some something that we can borrow from current corporate governance policies that will allow us to get part of the way there. But we know that, for example, thinking about the privacy governance example, we know that AI systems can present risks and never touch personal information at all. They can present risks around trustworthiness or, or bias without ever processing personal data in their training set or in their input or their output. And so that there are going to be areas where current corporate governance is going to miss some risks. And so my perspective on this is, you need AI specific governance models that include both organizational measures. So Code of Conduct policies and general guidance and principles around how to responsibly adopt and implement and build AI systems. But you are also going to need some sort of technical enforcement mechanisms, as well. And that probably starts with technical solutions that give you insight into what AI systems you actually are using. So you can catalog those, I analogize, this very similar to you know, the data mapping principle where you can’t govern the data that you’re processing unless, you know, you know where it resides, or what data it is why you’re using it. You cannot govern AI systems where apply a code of conduct, if you don’t know what systems you’re using, what systems your teams are building. And so a technical solution that at least begins to categorize those and gives you some observability, I think it’s gonna be really important. It also has the benefit of being incredibly flexible, because regardless of how your code of conduct may evolve, or regulations may end up evolving, knowing what systems you have, is always going to be step one, in order to do a gap analysis and, and begin to actually enforce a governance model on those things.

Marlene Gebauer 16:51
And, and how important is sort of communication as to the why the governance is the way it is, like, you know, not all organizations are created equal, I think, you know, some have have clients or the type of work they do that requires, you know, stricter privacy methods and others, you know, perhaps they have, you know, foreign regulation that applies to them. So, how important is the communication, given the fact that, you know, everybody, I shouldn’t say everybody, but you know, a lot of people want to kind of get on this bandwagon and use it.

Brandon Wiebe 17:31
I think communication is incredibly important. And if legal teams are leading the charge in this area, I think getting buy in from stakeholders at all levels of the organization is going to be really important. And that that comes with communicating and articulating really clearly, you know, what the risks are? And I don’t think the risks are exclusively, you know, regulatory, I will say, you know, I see a lot of folks say that AI is unregulated right now. And that folks are drafting regulations. And eventually, we’re going to get to a state where, you know, particularly in the EU, but we’re also seeing movement on the federal level in the US that we’re going to have specific discrete requirements around AI systems. I sort of scratched my head at that, because I think AI is regulated in many respects already. And from the privacy perspective, for example, we already have very clear guidance in the EU, and in the US around processing of personal information and what is acceptable and what is not. There are a whole bunch of other risks that AI systems present that will be regulated as well. But I don’t think we as you know, attorneys, who are advising our internal clients on this, have to wait for those regulations to be able to point to some regulatory risks. I also think it’s really important for legal leaders to be able to articulate risks beyond just regulatory risks. So there are brand and trust related aspects to adopting AI. And we saw at the time of recording this just last week, a very popular video communications tool, have an issue that arose around how they were processing data are how they stated they were processing data for their AI systems. And this was a an example and I don’t know internally, you know how that decision was reached at this organization. And I don’t want to speculate, but this is an example of where changes to how you are approaching AI and how you’re communicating that externally can vary significantly and very quickly impact your brand and So legal leader has been able to articulate that side of the risk equation as well, I think is going to be incredibly important in bringing, you know, executives along with you.

Greg Lambert 20:11
Now, and just hearing you talk and reading some of the articles, the blog posts that you that you’ve put online, I know that you have have stressed that you believe that the G, the governance in ESG, is poised for a drastic change with your legal teams helping lead the charge on this change. So what kind of specific governance updates are you seeing coming into play now?

Brandon Wiebe 20:44
Yeah, so I think we’ve seen a pretty similar trajectory with privacy where we started with organization organizations engaged in something like voluntary compliance, where they were simply trying to comply with privacy principles, as a baseline. Certainly in the US, we saw this, you know, before CCPA, came out, where organizations were, were really just trying to adopt privacy policies. Because there, there was sort of a an FTC, sort of Section Five II type of requirement for you to be truthful with what you were doing with, with personal information, but no other really regulations around that. And then, just in the last five years, we’ve seen us go from that to very sort of principles based privacy regulation, to privacy regulation. Most recently, I think CPRA. And all of the other state laws have come out as a good example, privacy regulation that is much more specific and technical in nature. And my guess, and again, I don’t want to prognosticate too much, because when I, whenever I do that, I usually get it wrong. But my guess is that we’re going to see something very similar for AI regulation and AI governance, where we go from voluntary compliance to principles based regulation to very technical and specific requirements. Except the only difference for AI is it’s going to happen much faster than it ever did for privacy regulation. I think we’re going to see that evolution happen in a matter of a couple of years, rather than, you know, the 20 plus years from, you know, the time that we had the privacy directive in the EU, to what what we see right now and in the in the landscape in the privacy landscape. So I do expect that we will see, governance, internal governance requirements have to become much more technical in nature. And as I was talking about earlier, I think technical measures and controls are going to be adopted. And I think the industry and regulators are going to coalesce around those much faster than we saw happen with privacy.

Marlene Gebauer 23:17
efficacy, it almost has to right. You talk to a lot of other general counsel in the course of your work, how are you advising them to take a leadership role on issues like ethical AI practices and data privacy?

Brandon Wiebe 23:35
Yeah, another great question. So I think GCS and legal executives are only going to be able to take this sort of leadership role. If they are going to do what is often a challenge for, for in house legal organizations wishes to be very proactive, and deeply engaged in the AI implementation and development process at their organizations. You know, oftentimes, we, as legal leaders, can be put into a position where we have to be very reactive to, to changes. The way I see that I think AI is going to require really deep technical understanding from legal teams. And it’s going to cover both legal and non legal functional areas. So as we were talking about, you know, some of the risks that are presented by AI right now, beyond things that are already regulated, sort of fall into a non legal area. But I think organizations are going to have to rely on their legal teams to tackle those. And then the legal team has been engaged really early on in the process is going to be key. And so when I think about those things like this deep technical understanding, coverage of legal and non legal issues, and really early engagement. What this reminds me of is a product counseling function in an organization. And for listeners who are not familiar with the product counseling concept, this is something a lot of technology companies have adopted in the last several years, I think it was started at Google, where you have a counsel in house that doesn’t have a necessarily one specific subject matter area that they are really deep on. So they may not be, you know, a patent counsel, they may not be a privacy counsel, but they have a really good general understanding of a lot of these subject matter areas. So they have breadth, and they’re a generalist, and they are embedded with a particular product team, so that they can learn that product and that technology really deeply and be able to issue spot and identify early on what what issues may arise during that product development process. I think organizations that are going to effectively govern AI, and legal leaders that are going to develop leadership in this area are going to have to follow a very similar model in how they work with their product teams, or, you know, whatever cross functional team it is that is implementing or adopting, or building AI technology. So my view is, you know, that early engagement, even if it’s very lightweight, even if it’s just a listening exercise in the beginning, that is where structuring this sort of legal, cross functional support is going to start.

Greg Lambert 26:47
So I saw where you had a recent announcement where you’ve created the AI risk assessment template, there transcend. So how, if I’m a GC at another company, how would you advise me to use this template to help me get kind of a head of any of the issues before they become a regulatory crisis?

Brandon Wiebe 27:14
Yeah, and I think that AI risk assessment template is a good example of a way that a legal team can engage early on in the process before a tool is built before a new tool gets implemented. So this is really just a lightweight questionnaire that, you know, we looked at a number of the different risk frameworks that are out there right now. So we looked at the NIST risk management framework. We looked at guidance from the OECD, we looked at the EU AI act drafts. And we tried to develop a set of questions that legal and product teams could answer together about a new AI system to begin to probe what potential risks existed. We also tried to draft this in a way where the steps included a fact finding portion, as well as a risk identification and mitigation portion, with the outcome being driving towards a released product or a released implementation that has properly balanced the risks that were identified, but gets to something that is launched. So the goal of these templates is not to slow down the innovative process is to document what’s happening, and to guide teams early on, so that they can avoid having to pull a product down the road or having to deal with, you know, a major regulatory crisis. When regulation gets passed eventually. The thing that I typically advise product teams, or other attorneys that are trying to engage with product teams, on this, on these types of issues, is early engagement is like pointing a rocket, a couple of degrees when it’s on the ground, rather than launch it into space, and trying to you know, now navigate it 100,000 miles in the opposite direction, it is much easier early on to make very small nudges one direction or the other, before you’ve built something. The inertia of an implemented system or a launch tool is so great, that it becomes very difficult to pull that back in or change it once it’s out in the wild. So I think again, early engagement is super important. And these sorts of risk assessment templates are a great way to guide teams on analyzing that risk and helping to guide early on in that development process.

Marlene Gebauer 30:01
So file this question under, you know, generative AI is replacing all our jobs. You know, how are AI tools influencing and reshaping the role and responsibilities of GCS?

Brandon Wiebe 30:16
Yeah, it’s it’s a great question. Again, I think, for attorneys themselves for in house counsel, you know, there are certainly plenty of AI tools out there that we are going to start adopting, to speed up our lives to make us more efficient. And teams that are open to doing that, in a responsible way, are going to accelerate and provide a level of impact and service that that is much higher than their peers who have chosen not to do that. Certainly, for any listeners of this podcast, this is top of mind for them, you know, I listened to several episodes as well. And there are so many great solutions out there for in house counsel to speed up a lot of internal process and time suck. That, historically has been what we have had to focus on, and allow us to become much more strategic and thoughtful. I think there’s also a lot of opportunity for lawyers that are in house and working with product and engineering teams to develop AI tech in house that can help them as well, I’ll just give you a quick example. At transcend. You know, we built a something we call privacy GPT internally, which is a tool that we have trained on all of the most recent privacy regulations and developments. Since, you know, the the GPT-4 cutoff date of September 2021. Most of the regulatory issues that I encounter on in terms of privacy, or AI, have developed in the last couple of years, they’re not something that you could go to, you know, your own version of ChatGPT. And, and get any sort of guidance on. So we began to train our own model on this, and have began to begin to use it internally. And again, when paired with a code of conduct and guidance around how to understand and actually use AI and know what the technology is, and know that it’s not going to give me a 100% accurate answer all the time. But it’ll give me a place to start my research that has allowed me to, you know, speed up answers and increase the level of service that I can provide internally, I think a lot of in house legal organizations are going to have to adopt that sort of mentality, or they’re going to get left in the dust.

Marlene Gebauer 33:00
And my question was kind of tongue in cheek, because I know most, you know, most general counsel and their departments are very overworked. And so this is actually a welcome a welcome solution.

Greg Lambert 33:14
When I present, though, find some some way to fill those gaps, and with more work. So So Brandon, I was reading just this morning about a new tool that transcend is developing called Pathfinder, with the idea of, again, helping businesses secure AI governance. Can you elaborate a little bit on how pathfinders? And from what I’m reading it has this customizable art architecture that allows for better compliance and risk management? How, how can you kind of give us a little bit of a intro to what Pathfinder does?

Brandon Wiebe 34:00
Yeah, happy to. So we’re really excited about this new tool that we’ve been developing. And we are right at the outset right now have a signing up early access partners to test this out. So Pathfinder is really meant to solve that technical side of the governance problem that I was talking about earlier, which is to to stitch observability and auditability, and governance into the technical layer of, of using AI systems. So the way that Pathfinder works, in essence is it acts as an intermediary between an organization’s tools that they are building internally, and the foundation model LLM that they are building those tools off of so for organizations that are using open AI to build an internal system or A tool or to build a chatbot, or to analyze data, whatever it happens to be. Right now, we see organizations begin to develop a lot of those simultaneously on many different teams, and not have great insight or oversight on how those are connected to all these different foundation models, what data they’re processing, where they reside. And you can do an impact assessment and document it manually that way. And I think that’s a good first step. But longer term, again, I think a technical solution is going to be important. And so Pathfinder acts as that interstitial layer that can observe and audit what is happening with these tools and how that information is flowing back and forth to these foundation models. And then longer term will be able to provide governance rules and layers over that, so that organizations can unblock unwanted contents, filter, you know, strip out personal information, that sort of thing.

Marlene Gebauer 36:09
How was transcend preparing for potential changes in the AI regulations? And how will how will this Pathfinder product adapt to those changes?

Brandon Wiebe 36:20
So we are keeping our ear very close to the ground. And, yes, and reviewing, you know, every new draft of of every proposed regulation that comes out right now, I think the the approach that we’re taking is similar to the approach we’ve taken with privacy governance, which is technical, technical solutions ought to be regulation agnostic in the way that they stitch into the technology itself. And what I mean by that is, legal teams, and oversight teams are going to need some sort of technical access, and insight and control over what is happening at sort of the ground level truth of the technology. So the actual data packets that are flowing. Once you have that established, once you have that infrastructure in place, it becomes much easier to quickly change the logic layers over the top of that, that govern what is flowing. So when regulations update, and we see this on the privacy side. So again, by analogy, I’ll talk to that. But when we when we see a regulation change from an opt in regime to an opt out regime for a particular type of data, right? If you have the ability to govern that data at the technical level, it’s a very easy switch to flip to say, well, now we’re going to require that for all of this data, we get an opt in rather than an opt out, or vice versa. Right. So so that that is a similar approach that we’re taking for for AI systems is once you are technically stitched in at that layer, it doesn’t so much matter where the regulations end up. Because repurposing the logic over that sits over the top of that is much easier than having to rebuild your whole infrastructure every single time.

Greg Lambert 38:26
Now, be honest, are you having the AI review all the regulation changes, proposed changes that are out there, give you a first draft?

Brandon Wiebe 38:37
Know, a first draft is always helpful, but I’ve seen this thrown out several times on LinkedIn and elsewhere in the industry, but you know, we view generative AI ai as the overeager intern. And so it’s a good first draft, and it really tries hard. But you always need to double click a little bit on on, you know, whatever you get from that.

Greg Lambert 39:05
I like that the over in turn. I like that a lot. Well, Brandon, we ask all of our guests at the end of the interview to our crystal ball question which is, you know, let’s let’s peer into your crystal ball and look into the future and let us know what changes or challenges do you see in the next two to five years that you as a GC and transcend as as a company may end up facing?

Brandon Wiebe 39:41
Again, I hate to prognosticate too much. But I again, I think the how swiftly AI technology has been adopted and the overwhelming inertia from, you know C suite to be at the leading edge of, of either developing their own AI systems or implementing them or even just re skinning what they have as an AI tool is going to require that GCS become experts in this technology. And if they can’t do it themselves, they’re going to need to hire develop teams internally that can become experts in the tech, I think becoming experts in that technology. And getting comfortable with that technology is almost more important than being an expert on all the discrete subject matter areas or legal risks. Because comfort with how the technology is actually operating, will allow legal counsel or allow legal counsel to provide much more specific and discreet and practical advice to clients. I see a lot of teams that look at AI technology with trepidation. Not because they think that there is are they not because they think they can point to a specific type of risk, but because they don’t truly understand how the technology is actually operating. And so I do think that, you know, legal teams that are going to be successful over the next two to five years are the ones who understand the technology at a really deep level.

Greg Lambert 41:38
All right, well, Brandon Wiebe, General Counsel and head of privacy at transcend. Thanks for taking the time to come on The Geek in Review and talk to us.

Brandon Wiebe 41:47
Thank you so much, Greg, and Marlene, really appreciate it. This was great.

Marlene Gebauer 41:51
And of course, thanks to all of you, our listeners and subscribers, for taking the time to listen to The Geek in Review podcast. We really appreciate your support. If you enjoy the show, share it with a colleague, we’d love to hear from you. So reach out to us on social media. I can be found on LinkedIn at gay Bauer M on x and m gay Bauer 66. On threads. We also have The Geek in Review account on threads.

Greg Lambert 42:14
That’s, that’s a lot of places anymore to be a lot of places I

Marlene Gebauer 42:18
gotta narrow that down next time next time.

Greg Lambert 42:21
So and I can also be reached at glamoured on x and glamour pod on on threads. And actually, I just finally got my count for blue sky two, which is also glamoured pods. So who knows now now we’ve got 15 different places to post.

Marlene Gebauer 42:41
Wherever you are, we’re there

Greg Lambert 42:42
we are. We are so Brandon, if somebody wanted to continue the conversation or learn a little bit more about what you’re doing where, where can they find you online?

Brandon Wiebe 42:52
Absolutely. Well I can be found on LinkedIn at bbb that’s B W i E, B as in boy E and transcend can be found online@transcend.io on LinkedIn at transcend and on x at transcend underscore IO.

Marlene Gebauer 43:11
And listeners can also leave us a voicemail on our kijken review Hotline at 713487782 and as always, the music you hear is from Jerry David DeCicca Thank you Jerry. Thanks Jerry. All right, Marlene. I buy by

Unknown Speaker 43:32
the way Hey welcome

Unknown Speaker 43:47
back. Devils backbone. Bill’s back

This week on The Geek in Review, hosts Greg Lambert and Marlene Gebauer spoke with Katie DeBord and Kristin Zmrhal, two vice presidents from legal tech company DISCO. Greg kicked off the episode by discussing his recent work with a Houston nonprofit called Project Remix Ventures that helps at-risk youth. He took their leader on a visit to innovation hub The Ion to showcase reinventing old spaces for new purposes, like DISCO has done with legal tech. The hosts then welcomed Katie DeBord, who moved from being Chief Innovation Officer at law firm Bryan Cave Leighton Paisner to DISCO. In her current role, Katie focuses on leveraging technology like AI to improve the litigation process for lawyers. She drew experience from her past analyst role at the CIA, where she honed her skills in synthesizing complex data sources.

The hosts also introduced Kristin Zmrhal, who has over 20 years of experience in the legal tech space. At DISCO, she helped build their eDiscovery products and services. Kristin explained that DISCO’s vision is to create great legal technology that helps lawyers find evidence faster. Their product suite now covers the entire litigation lifecycle, from intake to discovery to case management. DISCO uses AI tools like their new Celia application to automatically surface insights from case documents, allowing lawyers to review documents more efficiently. They are also careful to cite sources to ensure transparency.

In terms of company culture, Katie and Kristin discussed how DISCO values rapid experimentation, quick decision-making, and collaborating as a team. They also emphasize empathy in how they treat each other and design products for users. Being a public company also gives employees a sense of ownership. On the innovation side, Katie sees billable hours changing due to advancing legal technology, which will impact law firm profitability models. Kristin predicts AI adoption will reach a tipping point in legal tech within 2-5 years, drastically improving processes like eDiscovery. However, regulating AI poses challenges for the legal industry.

For giving back, DISCO has community service and pro bono programs. DISCO Cares allows employees to volunteer locally. Through DISCO Pro Bono, they donate their technology to support pro bono legal matters. This aligns with their mission of making legal services more accessible. When asked for parting thoughts, Katie emphasized lawyers needing to leverage professionals from adjacent disciplines as part of their teams. Kristin reiterated that this is the most exciting time in her 20 year legal tech career, with AI poised to transform legal workflows.

This engaging discussion provided insights into DISCO’s innovative products and empathetic culture. With seasoned experts like Katie and Kristin leading the way, DISCO seems well-positioned to help shape the future of legal technology. Listeners can connect with Katie and Kristin on LinkedIn and find out more about DISCO’s offerings at csdisco.com. Be sure to stay tuned to The Geek in Review for more insights from leaders in legal tech.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript

Marlene Gebauer 0:07
Welcome to The Geek in Review. Podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer,

Greg Lambert 0:14
And I’m Greg Lambert. Marlene, this week, I got to do a couple of really interesting things related around the same, same thing. You know, we’ve talked about the Houston Ion before where they’ve converted the the old Sears building in Midtown Houston into an innovation hub. Really cool stuff that they’re doing. And new times. Yeah. And and so this week, it wasn’t related to The Ion but I got kind of a similar experience. One of our lawyers works with a nonprofit that pointed me to another nonprofit who was looking for some podcasting help in this project in Houston is called the project remix ventures, where they took an old Juvenile Detention Center and have converted it into a community center. And where they’re helping at risk youth are, I think they call it what is it is justice involved youth is the phrase that they’re using for kids that have been in the justice system, and giving them educational opportunities, both with working on their GEDs, and with learning a life skill, more of an entrepreneurial type focus. And so I’ve been working with a woman down there, Diana Rodriguez, who’s been outstanding. And actually this morning, we both went to The Ion because I wanted to show her kind of the parallels of taking something that was for one purpose, and and reinventing it for another. So Diana, thanks for hanging out with me this morning. And I’m sure we’ll be doing a lot more projects together.

Marlene Gebauer 1:57
Yeah, it sounds like a great project. And I love to hear the fact that they’re getting kids involved in this and sort of learning new skills and how to apply them. And you know, who knows, maybe we’ll get some interns.

Greg Lambert 2:08
Oh, I hadn’t even thought about that. But yeah, train them up to come in here and do the editing for me. That’s right. Well, speaking of innovation, we’d like to welcome in Katie DeBord and Kristin Zmrhal. Did I say that? Right?

Kristin Zmrhal 2:29
Well done, Greg.

Greg Lambert 2:34
They are both vice presidents in discos product strategy, organization, DISCO, being an AI enabled litigation Management Suite for law firms, corporate legal departments, and government agencies. And a lot of us know it from the eDiscovery group and the massive wonderful parties that you guys throw it, please. Translate legal week. So Katie and Kristen, thanks for joining us here on The Geek in Review.

Katie DeBord 3:02
Thank you so much for having us.

Kristin Zmrhal 3:04
Yeah, we’re thrilled to be here. Thanks so much.

Marlene Gebauer 3:07
So Katie, I’ll start with you. Your journey being the Chief Innovation Officer at BC LLP to DISCO is is really quite unique. What does your current role entail? And what are your main goals and responsibilities at DISCO?

Katie DeBord 3:20
Yeah, it is unique. And actually it goes back further because I was a complex litigation partner at Bryan cave, Leighton Paisner for many years before I moved into the Chief Innovation Officer role. And the move really does integrate my experience as a litigation partner and all those years of litigation with that tech and innovation role at the firm. And it really is driven by my desire to think through how lawyers and especially complex litigators are using technology in this increasingly data driven complex world. You know, most tech companies have some idea of how litigators work across some aspects of the lifecycle, but many don’t know, really the details of how they fully work. And so for me, the move to DISCO was driven by like, how can we insert products throughout that lifecycle to make the experience magical, you know, as magical as Spotify is for me, you know, and, you know, how can we take analogies, litigators use analogies all the time, right? So how can we take analogies of where AI is being used in other industries and apply that to the litigation lifecycle and that’s, that’s really where my energy comes alive. And, and that was what drove my move.

Marlene Gebauer 4:43
Yeah, so so definitely innovative, you know, using looking at certain tools and applying them in new ways.

Greg Lambert 4:49
And we’ll dive in deeper, but what a great time to be in the industry right now. Isn’t it? So, so cool. First and I want to throw the same question at you, you have over 20 years of experience in the legal industry. You’ve worked as a consultant and leader in various organizations, including Google. So how did you get interested in legal innovation? And what kind of led you to the path of joining DISCO?

Kristin Zmrhal 5:19
You know, I had, I’m not a lawyer, many people think I am. Whether that says something about my personal traits or my village, argue, I don’t know. But I had originally thought that I wanted to be a litigator. And so I started working after undergrad and was going to work for a couple of years and then go to law school. And the first job I had was at Navigant Consulting in Chicago, and we were working on these very complex, massive cases, whether it was regulatory investigations or litigation. And I kind of looked around the room where we had set up 200 contract attorneys and had first second year associates sort of working all night with us on these cases and said, You know, I don’t think I want to be a lawyer anymore. But what I loved was solving the really hard problems with technology. And so that’s, you know, spun off my career to just stick on the technology side of the law as a consultant. Then when I was a consultant, Google became my client and recognize that they there was a better way to do their internal sort of ediscovery program. So we started that back in about 2010. And I did that for a number of years. I was actually trying to get out of ediscovery, after about 15 years, but I think once you’re in you can never you can never get get out.

Greg Lambert 6:34
It’s like the mob. It really.

Kristin Zmrhal 6:38
Yeah. And everybody knows everybody, right? Yeah. But I had but QE, you know, our CEO and founder and the team came in pitch to Google actually the DISCO product. And while I was listening to their pitch, I was really inspired by what they were trying to build. And so rather than just buying the product I came to work with with them. So I’ve been at DISCO for about five years, building out, you know, our corporate litigation suite with a heavy focus in what we’ve done in ediscovery. And AI.

Greg Lambert 7:05
Well, I don’t think you missed out by not going to law school.

Kristin Zmrhal 7:10
I’m pretty I’m pretty happy with where I ended and I get to work with and for lawyers every day,

Greg Lambert 7:16
and you didn’t have to pay off the student loans. That’s right. Well, funny

Kristin Zmrhal 7:19
you say that, because that’s the joke I always make, because I did end up going to get my MBA at Berkeley.

Marlene Gebauer 7:25
I made. Nevermind, nevermind that. So Katie, you you mentioned that you were a practicing attorney. You didn’t mention that you have worked with the CIA. So we need to hear about that. Oh, yeah, yeah,

Katie DeBord 7:41
no, I mean, you know, just the none of it was I worked on the analyst side. And, you know, on that side, and there’s, there’s analogies and parallels here, which is why I’m talking about it. But you know, on that side, you’re you’re streaming through massive amounts of information and data coming from lots of different sources, whether there are top secret human intelligence sources, or signals intelligence or publicly available information. And you’re, you’re trying on a daily basis to condense all of that, so that you can see the wheat through the chaff and see what’s happening, and tell the policymakers and the president, right, hey, we think this is about to happen based on the you know, these facts. And so that’s, that’s what I did for five years at the CIA. And I think it actually really primed me for law. Yeah,

Marlene Gebauer 8:37
it’s like I was thinking, it’s like, oh, is it really like Jack Ryan? Is that what the analysts do like, special ops, and

Katie DeBord 8:46
you’re not imagine a bunch of people in short sleeved dress shirts, and pocket protectors, they literally have pocket protectors, like a think tank, you know, it’s they’re, they’re the most amazing people in the world. I love them,

Marlene Gebauer 8:59
you know, having this background, right. How did you, you know, what lessons did you sort of take along the way? And how have you kind of applied that to legal innovation and developed, you know, that expertise and the passion around that?

Katie DeBord 9:16
Yeah, I mean, I think I think the first and foremost is to always work with really smart people. You know, and that’s, I’ve done that my entire life. I’ve been very blessed in that in that regard, to find the smart people and work with them. And then the second element of it is sort of what I said before, you know, the world is three dimensional problems are three dimensional. There’s a lot of different factors influencing the problem, right? There’s a lot of different players at play. There’s human dynamics at play. And, you know, the better that you can see that three dimensional problem and tackle it in a three dimensional way, the more effective you’re going to be and in my view, You know, tech helps us see problems more three dimensionally it helps us see where gaps exist in information, you know, where we know we should have information, but we don’t have it. Or where we see human dynamics change. Chris and I are emailing each other every day. And then something happens and we stop emailing each other. And in tech can help surface that and say, Hey, what’s going on in this dynamic? And what happened here? And that’s to me, you know, the interesting thing, and the kind of the theme across my entire career is, is really thinking about things three dimensionally and dressing them three dimensionally.

Greg Lambert 10:39
Yeah. So So in other words, that’s, that’s when you guys jumped off the email went to WhatsApp, right. And you know, something’s going on. That’s right. Hopefully, they’re just planning a birthday party. I want those messages, though. And for the listeners that want to hear a little bit more about Katie’s experience with the CIA, Steve poor, has a great interview that you did a few months ago was that I highly recommend listening to that, because I think he dives a little deeper into that issues. And of course, Steve always does a great job. Yeah,

Katie DeBord 11:22
it’s great. Kristen, I’m

Greg Lambert 11:24
going to ask you, I’m sure that most of our listeners know, have heard of DISCO. But I want to, I want to kind of get a high level introduction for those who may not know what it What’s your kind of mission and vision is there? What are one one? What is the vision and mission of DISCO? And what are some of the key products that DISCO offers? To the clients?

Kristin Zmrhal 11:51
Yeah, I think thanks for asking. We are mostly known for our ediscovery offering that’s been around for about a decade now. But our goal is to build a great technology for lawyers to help them find evidence faster. And so our product and offer and services now span the entire litigation lifecycle, from the initial intake of a complaint or request some of our clients actually, we host these law enforcement portals for law enforcement to seek emergency information in the event of a you know, an accident or an issue to litigation, hold management, witness management across the whole case, of course, ediscovery that has had aI available and actually for 10 years as well. And then the end to end sort of case, in fact, management. So there’s really exciting ways that we’re now taking AI and embedding it directly into these products as well, where you can imagine a world in the future and not too distant future really, where AI can help you analyze an issue a subpoena a complaint a suit, and quickly identify the facts of the case, help you then go seek evidence to, you know, prove or disprove those facts, and even start reviewing those documents for you. So at the end of the day, our core mission is always about finding evidence, you know, supporting justice, and we’re now able to use all of these products and services to support that. I often like to think of you know, how, why I’m so passionate about what we do at DISCO. And as somebody that was a project manager and a consultant for a number of years, or in the, you know, early 2000s working in this space, the answer was always let’s go hire a bunch of people, like throw a bunch of people at it. And I love the cliche as it as much as it is of work smarter, not harder. And so anytime that we can look around the room at the way people are working, and find ways that technology can make it faster, make it higher quality and less costly. That’s what we aim to do. And so the world of where we were hiring 200 contract attorneys in temporary space, just to get a document review done, we’ve already changed the game there where we can do that with far fewer people. And generative AI is just going to ensure far fewer people and a higher quality output. That’s the only way that we can operate these days. I mean, there’s Katie alluded to, data’s growing, if the issues are more complex, the way in which people are communicating is is, you know, nuanced. And so you need this technology to sort of keep pace of it.

Marlene Gebauer 14:25
Kristin, I’m curious if you’ve had the opportunity, working with clients to use DISCO technology in ways other than just surfacing evidence. The technology I think can be used to surface a lot of different content. And so I’m wondering if, you know, this is, you know, if you’ve had the experience of using DISCO in a way that maybe had not been originally envisioned.

Kristin Zmrhal 14:51
We actually have a number of clients because the eDiscovery database was built to be highly scalable with Google fast you know, speed At search speeds, we have a lot of clients that end up using this as a repository and for compliance issues or employment issues, any sort of investigations. Absolutely. And then we also, you know, we also have the capability to upload basically depositions, the videos, the transcripts, and annotate and collaborate on both the video and the testimony that becomes really, really important. We’ve had clients who’ve come to us say, we never would have looked at this video before, because the technology that they had was sort of antiquated or hard to use or, you know, crashed, find evidence that they didn’t know existed, and they probably wouldn’t have had it not been for modern technologies. Those are sort of the best client calls that we get to go on where they tell us that our technology helped them find something that they never would have, they never would have known before. And I you know, I think about sort of I mentioned, we’re sort of the connectivity. So for a corporate legal department, we’re already doing interesting things like being able to connect directly into Slack and Google and Microsoft and the way that people are communicating more moderately, you know, you can imagine a world where your compliance department investigations department security department now has more heads up or monitoring capabilities into what’s going on and can reduce risk in the future with modern technology.

Marlene Gebauer 16:21
So you’re actually changing like the workflow and the work methodology in that manner? Absolutely. We had Curt Meltzer on a few weeks ago, talking about legal technology companies and their efforts to give back to the community and to the public. You have a few missions as part of your community service efforts, DISCO cares and DISCO pro bono, do you mind telling us a bit about those programs?

Kristin Zmrhal 16:46
Yeah, with DISCO cares. This has been around almost since the inception of the organization. And we our original work headquarters was in Houston. And I believe it was one during one of their hurricanes where a lot of our our colleagues and their families were impacted. And so the DISCO team just naturally and organically started helping people by whatever they could do if they had boats that they could help transport people or, you know, rescuing them. And so that became like, that was just core to who we are without having it under this official umbrella of DISCO cares. But recognize that the people that we bring into this organization, while they’re very intelligent Are you know, are also quite passionate about their own communities. And so we do events, in almost every location, wherever we have an employee, whether it’s in our headquarters in Austin, or New York, or field reps that are spread out all over the world, have the capability to tap into DISCO cares to get back to their local community through, you know, a number of different missions. One of our other things is we love to partner with our law firm clients as well to put on those types of events. Do you I can go into the pro bono to that. Yeah, please do, please. Yeah. And so because DISCO is is so passionate about our, you know, seeking justice and making justice available for everybody. We do support pro bono matters by providing the technology at no cost, mostly through our law firm partners. And we’ll offer services on top of that for for those small matters. That, you know, again, another opportunity to give back and giving our technology away for free for these types of this type of work is fulfilling,

Katie DeBord 18:33
you know, in terms of leveraging DISCO generally, whether it’s pro bono or for a high stakes litigation matter. I think the thing that people love about DISCO is that it’s actually fun and easy and intuitive to use, which is not necessarily the case with most technology. And and it’s easy to use for the lawyers, right, I really advocate for lawyer first technology, meaning technology that the lawyers will actually interact with and use as opposed to support teams, right? It’s great when support teams use the technology. But when the lawyers aren’t using it, they are extracting the full value of what the technology can bring to them from a strategic perspective from, you know, a client results perspective, in my view,

Greg Lambert 19:25
that was a very polite way of saying is so easy. A lawyer can use it.

Katie DeBord 19:33
It’s fun, right? I mean, that’s the other thing is like, we interact with technology that’s fun to interact with, like I love, you know, Spotify, because it’s fun, and it finds music that I like that I wouldn’t have found otherwise. Right? And that’s a really cliche example. But it’s true. And the reason why it’s so lawyer friendly is that we have lawyers on our development teams. We have lawyer leadership we under stand that are in use or has a lawyer brain and lawyer desires and you know, lawyer competencies. And you know, we we build for that. And we designed for that. And that’s how I think all legal tech should be.

Marlene Gebauer 20:18
I’m curious about how you leverage AI to enhance your products and services. Tell us a little bit about the new Cecilia AI tool that I actually got to see a demo of. It looks very impressive and cool. And before for you go into details, why the name Cecilia,

Kristin Zmrhal 20:35
this is a it’s a great story. So there was a woman named Cece, who was Kiwis very first, our CEOs very first assistant. And as you know, DISCO progressed through its journey out of the law firm and into its own organization. She became the CFO, you know, the finance team, the sales team, the customer success team, she was just a little bit of, you know, everything to the organization. And we really wouldn’t be here. Without her. She has since retired. But there’s a little bit of a nod to her in the naming of our initial generative AI ai application. So that’s kind of fun. I actually had the opportunity to have dinner with her last week, and it was really great. So So yeah, so Celia was I was launched at legal week 2023. So that was in March. And it was the our first sort of jump into generative AI, we’ve had aI through predictions in the product for almost 10 years. But this was on the backing of all of what we’re hearing about generative AI and ChatGPT and Large Language Models. And we were able to get it to market so quickly, because it’s something that we had been working on in our AI Lab for some time. So the thought behind the Celia, and how a lawyer will engage with her is she’s basically a legal assistant. And so something that a lawyer might ask an associate or a legal assistant to go research or look into, could have taken them, you know, day, many, many hours, days or weeks to go and find the answer to you just asked Cecilia. And what’s unique about it is it’s since the bot is only learning on the documents that you’ve put into your database. So the evidence that you’ve already collected, it’s not using publicly available data, that reduces, you know, the risk of hallucinations. And so it’s really powerful in that initial identification of issues and how you’re trying to prove or refute certain facts in your case. And the evidence that supports it, we also have will be this is a sneak peek for all of you to is, by the time that this airs, we will have launched what we call automatic timelines at the ilta conference that’s coming up. And I mentioned earlier that we have the capabilities to intake complaints, or subpoenas, we can now with the release of this product actually generate a statement of facts from that specific complaint. And later this year, we will have capabilities that will take issues that are defined by a lawyer and be able to go and predict whether or not any individual documents in a database of hundreds of 1000s, or millions of documents is related to those specific issues in the case. So really exciting stuff coming out of our AI Lab. We’re not stopping there. And it’s just incredible to me to think about the velocity that we’ve been able to accomplish just in the course of the past, you know, first eight months of this year, and where we’re gonna go next year, I really, I really think that this is the tipping point in litigation and ediscovery of the massive adoption of technology to improve, you know, case outcomes at a fraction of the cost or the time.

Marlene Gebauer 23:48
You know, we had some guests on actually just the other day, and they were talking about sort of different ways to prevent or reduce hallucinations. And I’m curious what you all are doing in that regard, because that that seems to be a big sticking point with a lot of firms in terms of using it for obvious reasons. So how are you approaching that?

Kristin Zmrhal 24:14
Yeah, the key part of it running within an ediscovery database is so that it’s scoped basically, to the documents that exist there, and will only provide answers of what exists in that database not going out to the world wide web. And then our team has been working very closely with a number of the Large Language Models, providers and tweaking and fine tuning those. I think we tested probably a dozen or so of them because that that is very important. The other key thing that we do to provide some surety to the lawyers that are using the application is cite our sources. So when a narrative response comes back from a question that I’ve asked, in addition to providing the answer, it also shows the evidence and allows you to go Have a look at the actual evidence directly in the database to validate the response. So while these are all, you know, new technologies, and there’s a lot of what I consider fear out there, I think when we’re talking to our client aren’t really innovative clients, they recognize that this is a tool in your toolbox, and you still hire a lawyer for their legal minds and analysis. And this is just something that’s going to support and assist them it’s not going to replace.

Greg Lambert 25:27
Yeah, I do have to tell one story when I was looking at the demo of Cecilia, that and I’m not an ediscovery. Guy. So just let me put that out there. So I came came in, from my perspective on using the tool and, and of course, the training data that they use in the demo is the Enron data, which a lot of companies use, because it’s just a massive amount of data that that’s out there to be used. Whenever I get these demos, I want to try and ask questions that are outside the scope of the demo. And so I use, I asked him, I said, well ask Cecilia, if she can explain what mark to market accounting practices are based on the on the documents. And they did say, well, we’ve never had anyone asked this before. So let’s see and plunked it in there and immediately came back with defining what it was also pointed out some documents directly that were that were using these accounting practices. And so it was, like I said, it’s not my profession. So maybe that’s a common thing. But I was super impressed with the fact that you can ask it a question like that in plain English, and get back something that even I could understand.

Kristin Zmrhal 26:44
Yeah, it’s been a lot of fun watching our clients interact with it. And we’ve seen instances again, where the question was asked, and we in in a discussion we were having on a live case with them. And the attorney immediately sent the documents that we found to the rest of the case team, because it was so impactful, and they weren’t sure if they would have ever found it. So just it’s Yeah, super exciting to watch.

Katie DeBord 27:10
Yeah, that’s what I was gonna say. It’s like when you’re on the phone with these amlaw 50 partners, who oftentimes say, tech won’t touch the stuff I do, because it’s too complex and too bespoke and gotta yatta yatta. You know, and you see their wheels turning as they get this technology. And they realize, Wow, this, this is actually cool for me. And I will use it right. I mean, I think that’s what that’s what gets really fun.

Marlene Gebauer 27:37
It’s the aha moment, you know, those are always good.

Greg Lambert 27:40
Yeah, those are, those are fun. So let me shift away from the product. And back to the business. DISCO is one of the few legal tech companies, it’s actually a public company, interested in learning more about the culture and values there at DISCO, and talk to us a little bit about the principles and indoor beliefs that drive the success there. And I forget Kristin, if it was you, or Katie, that said, you know, want to surround yourself with really smart people. So if you get a whole group of really smart people, how do you foster a culture of innovation and collaboration among all of those really smart people who may be used to kind of doing things on their own and not collaborating?

Katie DeBord 28:30
I mean, I can, I can talk about sort of my observations a year in with DISCO and, and it really is that culture of innovation, that, that made me want to go to a tech company, and in particular DISCO, you know, where innovation is, you know, kind of the core driver of revenue and value and all that kind of thing. And what what I’ve experienced is the speed, right, the speed of decision making, you know, once decisions are made, there, we stick to that decision, unless there’s a compelling reason to not to reverse the decision. I’ve been part of meetings where we’ve redesigned, you know, some services, you know, where, you know, it’s kind of, you know, tech enabled services. And the speed in which that went from kind of discussion to market release was stunning to me. And it’s not like it was a careless speed, right. It was a thoughtful, well developed plan, but it marched forward quickly. And that’s and I have dozens of examples of that, where just the speed has been mind boggling to me. I mean, it’s really cool because someone was gonna throw out an idea with our CEO, and he’s like, let’s do it, and it’s done. And that’s awesome to be in that kind of environment. And then the other thing I would say is that I See a lot of empathy in this company, you see it both internally, like amongst peers, I’ve had a couple of things, you know, happen recently where I’ve, I’ve needed some grace and empathy. And so you see it with that you see it with how people work together. If people aren’t familiar with a concept, there’s patience, and there’s sort of a group understanding of, of where people are coming from and hearing people out. And then you see it on the customer facing side, where we are truly empathic to what our end users are going through and dealing with the risks that they’re undertaking as lawyers, and really designing with empathy. And so I think, to me, those are sort of the two major things that I’ve seen in DISCO that have been really fun and awesome to experience. I love

Kristin Zmrhal 30:47
the that culture of sort of rapid experimentation, we fail fast. So it’s okay to try something and it doesn’t work out, well, you just stopped doing that. And then you pivot and that and that, that is really in our culture. It’s like, let’s, let’s do rapid experimentation. If we experiment and it is, you know, amazing, we’ll put additional resources out if it’s not, we just cut it. But let’s, let’s make decisions quickly, and figure out what works. That and it really is the the people, there’s, you can probably tell just how passionate Katie and I are. But it’s everyone at the organization. And almost everybody just has this believes in the vision that we’re driving towards and gets behind it, I have to share one of my favorite stories of DISCO. Because it’s just we invest a lot in the people that we bring on, especially early in their careers. And we we were at a big DISCO dinner at a restaurant in Austin. And we you know, there was a lot of us. And we were ordering a lot of wine and a lot of food. And this waiter that we had was just incredible, incredibly personable, really nice. So we started talking to him, Well, long story longer. We ended up I said, you know, if you’re ever looking for a job, send me your resume. We’re always hiring, you know, junior people. And not only did he send me his resume, the next day, I introduced him to one of our sales managers that has a number of sort of early career sales representatives, they we hired him and he just got promoted into a higher level account executive role. And, and it was just, this is just like one of many examples where you see people that are that are going to progress and make their career at DISCO, because we care and we treat it like a big family. And we really want to bring everybody up alongside us. It’s not competitive. It’s collaborative.

Katie DeBord 32:39
And I do think the fact that it’s a public company, there is also sort of a deeper sense of ownership of personal ownership. And I think that it creates a whole other layer of dynamic there.

Greg Lambert 32:52
Katie, you mind diving into that one a little bit more. But how do people that work at DISCO feel like there’s a personal ownership?

Katie DeBord 33:00
Well, everybody at DISCO has some sort of stock benefit as part of their employment agreement. And so certainly, it is personally beneficial, right to each individual employee, when the company does well. And I just feel like it is like we are all stockholders in the company, right? And we’re all working together for not just our, obviously our science, but also our personal benefit. And I just think kind of adds another dynamic of kind of ownership and seriousness around success.

Marlene Gebauer 33:40
But you, Katie and Christina, you do raise good points. I mean, as it seems everybody, you know, is incentivized kind of from the ground up to sort of work together like that. There’s definitely that that we all win. You know, if one of us wins, we all win type of mentality, which, you know, it sounds quite refreshing, actually. And that’s terrific.

Greg Lambert 34:02
Totally. So Katie, and Kristen, we ask all of our, all of our guests to answer our crystal ball questions. So Kristen, we’ll start with you, if you don’t mind pulling out your crystal ball and peering into the future. For us. What do you see as a change or a challenge in the legal industry over the next two to five years?

Kristin Zmrhal 34:27
Yeah, I mean, I think where the the challenge is going to be around this, the use and regulations on some of these generative AI capabilities, but I think that’s also the opportunity. I am very bullish that in the next two to five years, the whole concept of ediscovery is completely changed because of the the application of this technology. And it’s it feels different this time. You know, when I would go to the legal tech conference 15 years ago, people were talking about predictive code Doing and tagging and all this. And I still even with large technology companies, so they, they are skeptical to adopt AI. I think the this is this is the time. And so it’s the opportunity for us to look back and five years from now and think about how we used to handle, you know, complex, massive litigation or investigations in a much more sort of manual way than than what we’ll be able to do. But but we need the lawyers and the policymakers to keep up. And so that’s, that’s, I think, a risk here for us of getting slowed down through concerns of of these technologies. So that’s the challenge.

Marlene Gebauer 35:38
How do they do that, though? You know? Like, how do they keep up?

Greg Lambert 35:44
They need to leverage generative AI to help

Kristin Zmrhal 35:49
have it right, it’s its own policies and procedures, I

Greg Lambert 35:52
could go wrong.

Katie DeBord 35:54
I really believe and more and more lawyers are doing this. But it is so important for lawyers to leverage adjacent professionals, not non lawyers, adjacent professionals with expertise in these areas, as part and embrace them as part of their team, their client service team, right. In that’s going to have to happen. And to the extent you know, they’re not doing it already.

Greg Lambert 36:22
And so, Katie, same question, can you peer into your crystal ball in and what do you see in the next two to five years,

Katie DeBord 36:29
so mine’s not original, but it’s really, really important, which is the billable hour, technology isn’t coming, it’s here, it will change law firm profitability models, right. And my personal opinion is an old drive more and more flat fee, you know, engagements, or alternative fee arrangements. At the very least, I you know, if I was going to make a prediction, I mean, you know, we have lawful, really good, best of the best in class law firms like Bartlett, Beck, who went away from the billable hour model decades ago, and so we have precedent for this, we can do it. It’s just an industry thing that probably needs to happen in light of the technology’s increasing role in legal service delivery.

Marlene Gebauer 37:18
I wanted to go back to the, you know, everybody’s incentivized, you know, what takeaways or lessons can can law firms, you know, have from that? Because, you know, I don’t think most firms are sort of, you know, they’re private, and they’re not, they’re not set up that way. So what, you know, what lessons can they take away or apply?

Katie DeBord 37:39
You know, it’s a really good question. I mean, I think, I think one takeaway or lesson is the importance of investment back into the business, right, instead of just cashing out at the end of the year, and starting all over again. And when I say investment in the business, I mean, not only, you know, tech infrastructure investment, right, but also investment in training investment, again, and people with different types of skill sets that are adjacent to the lawyers, you know, and really investing in how is client service delivery changing today? How will it change in three years, and really thinking about how law firms need to position themselves to meet that goal? If people feel invested in, right, from every level in, in, across every discipline, including critical non lawyer disciplines. I hate that term, but I’ll use it here. You know, then people will operate together as as a team. And I’ve seen that I’ve seen law firms do it well, and it’s absolutely possible. But it does take top down leadership and really concerted decision making and tough and tough decision making. I mean, it’s hard to reserve dollars at a law firm.

Marlene Gebauer 39:08
True, true. Well, Katie DeBord and Kristin Zmrhal. Thank you both for taking the time with us here on The Geek in Review.

Katie DeBord 39:17
Thanks so much for having us.

Kristin Zmrhal 39:18
Thanks for having us. This was fun.

Marlene Gebauer 39:20
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found on LinkedIn, it gave our M on Twitter, and m gay Bauer 66 on threads.

Greg Lambert 39:37
And I can be reached @glambert on Twitter, or x or whatever it’s called next week. Or Lambert pod on threads. And of course, LinkedIn. I seem to be on LinkedIn a lot more lately. So Kristen, and Katie, if someone wants to learn more, or reach out, where’s a good place for people to find you online? Kristen, you US.

Kristin Zmrhal 40:01
Yep, you can find you can find me on LinkedIn. I’m pretty active on there. Or if you have more questions about DISCO, you can find us on our website at CES disco.com. Katie,

Katie DeBord 40:10
you can reach me at Katherine DeBord on LinkedIn. You have to do the Catherine k th ry N, or DeBord Katie on Twitter or Cisco’s website of course.

Marlene Gebauer 40:20
And listeners, you can always leave us a voicemail on our Giken review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 40:32
Thanks, Jerry. All right, Marlene, I’ll talk to you later.

Marlene Gebauer 40:35
Okay, bye bye

In this episode of The Geek in Review, hosts Greg Lambert and Marlene Gebauer interview three guests from UK law firm Travers Smith about their work on AI: Chief Technology Officer Oliver Bethel, Director of Legal Technology Sean Curran, and AI Manager Sam Lansley. They discuss Travers Smith’s approach to testing and applying AI tools like generative models.

A key focus is finding ways to safely leverage AI while mitigating risks like copyright issues and hallucination. Travers Smith built an internal chatbot called YCNbot to experiment with generative AI through secure enterprise APIs. They are being cautious on the generative side but see more revolutionary impact from reasoning applications like analyzing documents.

Travers Smith has open sourced tools like YCNbot to spur responsible AI adoption. Collaboration with 273 Ventures helped build in multi-model support. The team is working on reducing dependence on manual prompting and increasing document analysis capabilities. They aim to be model-agnostic to hedge against reliance on a single vendor.

On model safety, Travers Smith emphasizes training data legitimacy, multi-model flexibility, and probing hallucination risks. They co-authored a paper on subtle errors in legal AI. Dedicated roles like prompt engineers are emerging to interface between law and technology. Travers Smith is exploring AI for tasks like contract review but not yet for work product.

When asked about the crystal ball for legal AI, the guests predicted the need for equitable distribution of benefits, growth in reasoning applications vs. generative ones, and movement toward more autonomous agents over manual prompting. Info providers may gain power over intermediaries applying their data.

This wide-ranging discussion provides an inside look at how one forward-thinking firm is advancing legal AI in a prudent and ethical manner. With an open source mindset, Travers Smith is exploring boundaries and sharing solutions to propel the responsible use of emerging technologies in law.

Links:

• Hallucination is the Last Thing You Need
• Who Owns the Law

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠ | YouTube (NEW!)

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Voicemail: 713-487-7821 Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. So Marlene this week, once again, we find out why in legal,

Marlene Gebauer 0:21
we can’t have nice things

Greg Lambert 0:22
nice things. You may may have seen that there was filing an appeal filing in the 10th circuit. It was an ex parte case, but it was actually reviewed by an attorney, a 23 year prep long practicing attorney that had somewhere in the area of eight made up citations that you know, and I can tell you, I took that brief and I passed it through Lexuses brief analyzer and it immediately found those eight. So, you know, again, so, so word of advice, and I went around I was in Dallas this week, talking AI with the attorneys there. And the one thing I told them is this is not a legal research, pure legal research tool. Please do not use citations that are in there. And if you if you do you better check them. Yeah, and if something sounds too good to be true, it probably is. So

Marlene Gebauer 1:24
it’s just trying to please you. It’s just trying to make you happy. Absolutely. Absolutely. We’d like to welcome three guests today. Oliver Bethell Chief Technology Officer, Shawn Curren, Director of Legal technology. And Sam Lansley, artificial intelligence manager from Traverse Smith from their London office, Sean Olly and Sam, welcome to The Geek in Review. Thanks, fine.

Greg Lambert 1:48
So Olly, I want to throw this to you first, because we do have although we have a lot of audience in the UK and Australia may not be familiar with Travers Smith. So would you mind just giving us an overview of what Travers Smith does and exactly what your team does to support the mission there?

Oliver Bethell 2:09
Absolutely. And look, Greg, Marlene, thank you very much for the invitation to come and join us today. I’ve been a longtime listener, but it’s great to be able to join you first time as a contributor to say Travis Smith, we’re a UK law firm. But we’re a leading full service UK law firm, we regularly conduct cutting edge in industry, we first work for clients across lots of industry sectors. But we’ve particularly carved out a market leading reputation for our expertise in international asset management, cross border m&a, and also particularly global dispute resolution investigations work. So really a full service firm. And the team here were part of the the wider technology team here at Travis Smith, and very lucky to work with Sean and Sam, on on our strategy for AI, which hopefully we’re gonna get to touch on today.

Marlene Gebauer 2:53
I imagine in the past year or so has been absolutely insane. For the three of you, with the advancements in technology, the acceptance and the demands made of API’s. And of course, the awakening of the public to generative AI ai. Shawn, how have the expectations of attorneys and others there at traverse Smith, change the way you approach and launch technology at the firm?

Shawn Curren 3:18
I think for you know, I’ve been in the legal tech industry now for about 20 years. And I sort of talked about this a lot, which Yeah, I was around, you know, before the digitization of legal services, but everything was paper based. And when we used to archive fields physically, and we didn’t have document management systems. And you know, I’ve been through a lot of many revolutions over the last 20 years, say document management was one nice thing, you know, the BlackBerry was one, and I think there’s been a few others. But the improvements of the technology have always been quite marginal, I think. So you know, let’s say there was a process that was 90% people, people process and 10% Tick, then a new piece of technology would come out a new feature a new product that would maybe make that 12% Tick if the 18% Tick, taking a little bit of eating a little bit of the people in process. Certainly think with a lot of the hype on AI over the last eight months there is this expectation that the tech is going to eat everything a law firm does, that equation is going to go to 100% Tech and 0% people and process that has been quite challenging, I think for for people to operate. And so we took the strategy very early on that we needed to get our hands on this technology, get it in front of it or get hands on generative AI, get it in front of lawyers, get them testing it, getting them understanding what it’s good at and what it’s not so good to understand where there was opportunities for marginal gain efficiency was this tick and where there’s also opportunities for significant efficiency. And I think probably come on to talk to a couple that you’ve picked up on. And so we deployed an enterprise chat bot about a month ago. And I think one of the first firms to do that blocked ChatGPT very early. We didn’t want to be granting an irrevocable Well, worldwide royalty free license to data to big tech into open AI. And we’ve been testing and trialing these models across the business since then trying to understand exactly where we should put our focus. I don’t know, Sam, if there’s anything you’d add?

Sam Lansley 5:34
Yeah, I think I just all I’d add to that is that we’ve been very open and transparent with what the model can and can’t do. I know sometimes more technical people can choose to exaggerate the technology, which has sometimes produced a bit of skepticism from lawyers, we’ve been very honest about hallucination problems. And we’ve kind of explained to lawyers that we have to be really careful when using it to develop certain contracts or anything like that. So we’ve been very cautious with our approach. And we’ve also taken the approach of basically showing what is good and what showing what it’s bad at. And to try and do that we basically introduced community into our YCM bot, which essentially allows lawyers to promote something he’s good at, and also promoting as bad. And then they can have a bit of a discussion as to why it’s bad and why it’s good. So that the idea there is that we’re just trying to make lawyers really aware of how to use the technology and how not to use technology. I think that’s been really useful for helping them kind of get the expectations of what it can do and what it can’t in, and they are using that community. Yeah, they are using that community. It’s like a whole social network they’re liking they’re commenting on it, and we’re finding some really useful prompts from

Greg Lambert 6:38
Shawn, I want to elaborate on something that you said that you said you you had blocked ChatGPT, early on, do you mean for your users to use it or for open AI to come in and kind of scrape your materials or

Shawn Curren 6:55
Yeah, so we I think, Ollie and I actually were at Libya, in November, I think, or December. And it was all he actually talked to me about this and said, Have you seen this new technology called ChatGPT? And you’ve talked a bit of sophistication? I thought, I know. I’ve not heard of it. But clearly, if it’s got the ability to ask and answer questions, then it’s going to be phenomenal. But the concern is going to obviously be around making sure that you don’t improve the model by giving it data. And you’re asking the models providing a change of control clause, you’re just getting a version of that from Wikipedia, or law teacher dotnet. And that’s one thing, providing it a change control clause, which is proprietary, valuable intellectual property for that organization. And in asking it to provide some analysis on that. We recognize that that may be not saying that it is, but it may be a certain strategy for companies who are using data to train algorithms to improve the quality of the algorithms. So I think it was January, we were like, We need to block the looked at the terms of ChatGPT. And we knew that all of the lawyers who may be dabbling with it and testing and trialing it, were agreeing to the consumer teams, because we didn’t have an enterprise relationship with open AI. And those consumer teams were very explicit in saying that your content will be used to improve our services. And we thought, no, that’s not It’s not acceptable for us, we blocked ChatGPT, January, February, and Sam and his team, actually, it was in about two or three weeks, built a really thin chatbot layer that sat on top of the enterprise API’s of open AI. So we became a b2b customer of theirs. And it sat on top of the enterprise API’s, which meant that we were not subject to the same consumer terms are subject to enhanced enterprise teams, I couldn’t take our data to improve and develop services. And we then looked to roll that out, and then we’ll safe and secure way. But we did so was the kind of badge of experimentation and education as it wasn’t to be used and what product, we’re very concerned about hallucination. We’re very concerned about copyright. We’re concerned about an outlier risk, but the potential legality of the model having scraped away up and question marks around that. But we do want to allow us to understand the sophistication of this technology, and we’re sort of that hate that it was existing in the market, you couldn’t not do anything, right. You couldn’t just hold and wait, you had to give them something. But you had to do it in a safe and secure way. And that was effectively a strategy. So I think in March, we rolled out an enterprise version of a run chapelry, which we’ve developed in tannic, and we had to build logging and auditing capabilities on that, too. We can track him it’s been used.

Greg Lambert 9:34
So Sam, I noticed that and I love the fact that your title is AI manager. And I’m assuming that’s a long term title that you’ve had for multiple years, right?

Sam Lansley 9:47
Yeah, for 25 years. Just to be really clear on the AI manager title that does involve people it’s not me just managing an artificial intelligence but so thank you Obviously, the title is a reflection of the unique demand that was put on our firm and every firm in the last three to six months. I think prior to this, obviously, I was a software engineer inside the engineering team, we did focus heavily on AI as well, we actually won an award for a cog x with our platform s Turner, which was a review and labeling platform for reviews that we would do internally. But yeah, I think obviously, we decided in March that it was probably a good thing to create a whole separate team, which could really focus on this generative AI and how it can be leveraged inside the firm. And I think obviously, the decision there was made by the firm, that I’d be the right person to lead that team. And obviously, now we’re looking to expand the team more and more, I think we’ve looked into hiring, you know, ml ops people, machine learning engineers, software engineers, and it has produced already a tremendous amount of value for the firm.

Greg Lambert 10:48
If you get I know that there’s a big push for prompt engineers, are you guys looking to hire someone specifically? Or how are you training people to, to kind of manufacture those prompts in a good way?

Sam Lansley 11:05
At the moment, obviously, the community feature we added into YCM bot has been tremendously helpful, that’s allowed lawyers to kind of come up with prompts themselves and share useful ones, I think we probably will look into prompt engineering in the future, you can kind of see the quality difference between a really sophisticated prompt and a really poor prompt. So I think it’s something we will definitely explore, I think someone does, maybe need to be dedicated to that, to get the best out models, but it’s going to be really interesting field, it’s crazy how that prompt engineer, no one even would have thought that would exist, you know, six months ago, and all of a sudden now, I think nearly everybody’s gonna be hiring prompt engineers in the next couple of years.

Shawn Curren 11:39
We’ve been working with this board or now for the last eight months, and an experimental and educational basis. But part of that experimentation, the education is working with our clients and looking at whether there is ways for us to use this in certain instances. And so because we’ve got right now, you know, six months ago, we were b2b customers of open AI. Now we are b2b customers of Microsoft, we’ve got access to the GPT models in Europe, actually Amsterdam. And it’s just another Cognitive Services epi that we use regularly for what the work that we do. So we’ve been testing it a little bit more and engaging with clients on that. And we have been sampling certain types of use cases, particularly around the search for relevance in litigation, and discovery. And what was found already, is that the way that the lawyers may instruct a human to look for relevance is completely different to the way that you architect and structure prompts to search for relevance. And there’s a kind of computational law aspect as a computer science aspect and a legal aspect or kind of being sandwiched together. And we have, you know, I’ve been involved in a couple of these where I’ve really enjoyed understanding the facts of the case, the point of issue, the particular area that the legal team would like to hone in on on that set of data. And then the wheather asked the question, to get the right outcome and the right output from the models. I think that’s going to be Olly and I were talking about this yesterday, and you know, what that is going to be, but the next prompt engineer, whether it’s kind of like legal intelligence engineer or something, but there’s certainly an area between the law and the computational aspect of these models, that needs to be a gap that needs to be filled with some form of skill set that’s going to need to evolve.

Marlene Gebauer 13:30
Well, Shawn, that that raises a really good point. You know, Olly, I know you’ve been working with Michael Bommarito. Hi, Michael. And Dan Katz 273 Ventures on Gen AI ai resources like the YCN bot, which we’re going to dive into. But you know, first, what sparked your interest in exploring how Gen AI ai? You know, As Sean mentioned, there’s sort of this gap that needs to be filled between that and a law. What got you interested in sort of delving into that area?

Oliver Bethell 14:03
Sure. I was shouts to Mike and Dan, because they’ve been great to work with so far and this. As Sean said, we were having one of our particularly in depth one to one meetings in a brand new quarter one evening, and we had quite a long standing strategy around application of artificial intelligence that Sam was spearheading mainly around extractive AI so similar to technologies such as Kira, which was, which was going well and proceeding the way in whether we’d expected and then having an avid technologist. So reading about the capabilities of ChatGPT obviously set up many light bulbs in my head. I remember the conversation with Sean. Imagine instead of being able to extract a clause you could say, then say write me a clause that sounds like this, that is by a friendly that is competitive, that is off market, however you choose to describe it. And we had this theoretical conversation back and forth. Which then led to Shawn and Sam and, and several members of the team starting to investigate what was possible. And really since then it’s it’s been Shawn and Sam spearheading what, what we’ve done around the open source projects and what we’re doing. latterly, I think what’s been most interesting has been initially we look obviously looking at the generative use cases. But What’s impressed me is that is the reasoning potential. So there’s we’re going to talk quite a bit about hallucination today. And it’s obviously a concern for the legal industry around the risk of hallucination. But there is so much capability there today, around the ability to reason. And I think that’s where we’re starting to have some interesting experiments at this stage. And, and we’re just, we’re just uncovering and unearthing what the potential of this technology could be.

Marlene Gebauer 15:47
Would you say reason? Can you expound a little bit more on that?

Oliver Bethell 15:51
Yeah, absolutely. So when we think about to slightly incorrectly answer your question, we think about generative, we’re asking it to create new content. So write me a clause or prepare a citation. So it’s generating almost new content, which carries with it all of the risks of copyright and hallucination, which which we’ll dive into. But if you were to ask a question such as, does this batch of emails contain any concerning language, or any derogatory language, that that is a sort of more of a reasoning exercise. So you’re not asking it to generate anything new and but you’re asking it to carry out a task that were very difficult to programmatically describe using previous generation technologies, because you would have to train the model on what you meant by all of those different things. But straight out of the box, we’re seeing this huge capability. And I think, clearly, lots of legal tech vendors are seeing the same thing. So what are the announcements by the big platform providers that are saying they’re bringing these capabilities to market? And I think, well, your previous podcast episodes, talked about a lot of technology that’s being launched by press release at the moment. But clearly, the capability is there. It’s just about being able to harness it, I think,

Greg Lambert 17:08
as a very polite way to put it.

Oliver Bethell 17:13
Innovation by press release, I’ve called it.

Greg Lambert 17:17
Shawn, we kind of touched on this. But I want I want to pick your brain a little bit more on some of the due diligence that you’ve done with the generative AI. And it sounds just from the brief conversation that we’ve had right now is that you have both a passion for the technology, but also a passion for understanding how the the legal process works as well. So what have you seen as some of the biggest obstacles in preventing generative AI tools from accurately conducting legal work and processes? And how did that coming to under to understand those obstacles help you in setting up a use case for the YCN bot? There have it, Travers Smith?

Shawn Curren 18:07
Yeah. So I think, to follow on from what Wally has said, we are we have a slate knows we can’t show on this. But we have a slate, which is a bit of a spectrum between generative use cases, and extractive use cases. And on the generative side, we’ve got issues with hallucination and copyright. You know, the more creative the model, the more risk that is for hallucination. The more verbatim the output is, then the the higher the risk there is of copyright, because, you know, it’s verbatim exactly taken from a website that hadn’t rebutted us clause, it said, Don’t scrape me. So on that generative seed, we just leave. For us right now. It’s like high risk, low value. We’re not too interested. So the obstacle is figuring out whether we actually want to be providing legal advice, which is a derivative of publicly available data sources, like law teacher dotnet, and law dotnet, and all these websites. So that needs to be figured out. We’re not going to come out and see we’re using generative AI to draft contracts as obviously, but but on the extract has said, we we haven’t really come across any obstacles. And to Olly’s point, that is where we see the most value immediately and over the shortest term. And we think that if we’re to talk about legality of the models, sort of big problem that’s that’s occurring. Using the reasoning engine of the model is the furthest away from risk. Using generative, you’re creating content as copyright issues. You give that up draft a contract, we give that to a client we try and draft we try and grant them rate and title of the copyright of that output. The clients then got contaminated content, based from US based on open AI based on a website is scraped and as a kind of dz chain of copyright. Yeah, that’s a bit of an issue. But but on the extractive seat, we end the reasoning side. So the reasoning side, yes, the models wouldn’t be as sophisticated as you were today, if they didn’t ingest the internet, and doing so that has potentially scraped websites that they weren’t, you know, maybe weren’t supposed to call taking corporate they’re not supposed to. But we’re giving it to it asking a human equation is really smart. We’re giving it most of the data. And we’re just asking you to compute on a very specific point. And so we don’t really see any obstacles on that side. And that’s the area that we’re going to focus on.

Greg Lambert 20:33
Have you seen that the discussion around generative AI and just the hype, and it reminds me of the interview that we had a few weeks ago with Tony Thai from from HyperDraft. And that there are certain tools that do very well at certain tasks. And right now there’s this kind of idea that generative AI can do it all. Have you been able to leverage the the hype and excitement around the the generative AI to point people back to the tools that you’ve actually had maybe for years, that say, you know, okay, this has AI in it, too? Why don’t you use this because this is much better at you know, drafting documents or reviewing certain things has that helped in that?

Shawn Curren 21:26
Yeah, this is why, you know, eight months ago, when companies law firms are coming out and saying that overnight, they had decided to, as far as it seemed, pivot that entire document automation and document production strategy, to an LLM model that was streamed on the internet to seem to me to be a bit counterintuitive, because as you see, you’ve got those these businesses, our business, and they make many law firms have got closed banks, we’ve got document automation investment, we’ve got precedent banks, got content management systems with tags that are rich and useful to find documents or clauses that are relevant got by saving CLC, you’ve got tons of information that almost definitely surpasses what the model can produce. So those are the kind of use cases where we didn’t jump in. And actually, we think that we should continue to use the technologies that we have, because there isn’t a transformational difference there. To jump on appointment automated. We’ve been working on AI for the last four years, and it’s been around the entity extraction use case, Kedah luminance, is it a restrictive covenant as it is a control clause, this natural language ability of the model, where you can explain much more context, give it the background of a case, tailor, the contract that is reviewing stitching together amendments and variations across a large set, whatever it is, you can give the model that’s context. And it can then provide a natural language output, which not It’s not under percent, sometimes it’s wrong. Sometimes humans get it wrong. But the workflow that you get back from the more than one form of an Excel spreadsheet with documents down the left questions along the top, it wouldn’t be impossible to discern the difference between that and that that Excel spreadsheet being created by an AI bot, or a human. Some will be wrong, some will be raised, but it’s as it’s not a million miles away from, from Kenya. I think that’s revolutionary. I think that’s game changing. I think it’s new. And some people will say, Well, we’ve had semantic search for the wheel, true to the ability to cross reference based on meaning and context, not text. But we’ve never really had contextual natural language input, with contextual natural language output as an API. That is, in my view, new and it’s an exam degree, it’s revolutionary. I was on a roundtable with the Financial Times in March, and there’s lots of other firms and there was a question as to whether this was rich, any aid was revolutionary, or evolutionary? Was it marginal gain efficiency, significant inefficiency, I view as it’s certainly revolutionary, but it’s not going to solve for all problems. But actually, it might solve a really big pain point for legal, which is that we are in the industry of assessing risk. And that most of the time, we don’t have enough time to read all of the information to assess the risk appropriately. And that is now what’s going to change which I think it’s going to be a game changer for the legal services industry.

Marlene Gebauer 24:25
I wanted to I wanted to go back to your point about contaminated data, in the context of are you guys seeing any challenges in terms of clients saying, you know, I don’t want you know, I don’t want my data used in terms of, you know, drafting things like, you know, I don’t want that continue. I don’t want others, you know, competitors to sort of gain from that. Is that something that you’re seeing and how are you addressing that?

Shawn Curren 24:56
So we’re not seeing that because we are not, at this point off for them any generative use cases of the model. And we are very clear with their lawyers that they shouldn’t be offering generative use cases of the model was in terms of corporate risk, and hallucination risk. You know, we may come on to talk a little bit later about initiation is all you need the paper that we wrote. But just because just because a case exists, or doesn’t exist doesn’t necessarily mean that if it does exist, it’s been referenced or quoted correctly. And it hasn’t made up some fact that takes the face in a different direction or causes an issue. So we just on the generative side, we’re just, we’re not there yet in terms of having issues with our clients, because we’re not promoting that seat, when we get a big job. And it’s a kind of analyzed lots of stuff, and ask and answer lots of questions of stuff to get a view of risk. That’s where we are starting to see to the legal team slick. You know, your clients may be dabbling with ChatGPT, or an enterprise Chatbot. Patient? Why don’t we start the conversation now to see whether they would like us to do an paddlewheel review with EA, you do it on a human basis. And we’ll have any out of you alongside and we’ll just let them see what the output is. And we can cross reference that and we can see what we think and designate just take the clients on that journey. You know, and a lot of our clients aren’t we had breakfast with a lot of lgcs, they are very interested in AI. They want to understand how it can be applied to the businesses that they work in, but also their legal departments. And we’re trying to have that conversation early on. Because it’s not, it’s not going to go away. My view is that it’s not. You know, I’ve got certain views about blockchain and smart contracts. And I think this is perhaps a lot more interesting. And allowing this innovation is going to go away.

Marlene Gebauer 26:47
Sam, I’m going to jump to you. There was an article in May, which talked about how you were approaching YCN bots functionality. And you proposed multi model as one solution, can you walk us through how that approach would work and what impact it might have? You of

Sam Lansley 27:07
course, so one of the things we’re seeing is everybody becoming extremely dependent open AI, or a zero open AI. So they’re dependent on this one model that’s been created by this one company. And for us, we thought that first of all, was a massive risk, we would be completely dependent on one company for all the innovation in this space, given how important we think generative AI could be to the firm, we didn’t think that would be the right approach. The risks possibly being obviously this company could get quite restrictive in their usage, they might start, you know, adding terms that we can’t comply with, they may start limiting access to it. Which obviously would be a disaster if we built a large strategy on top of that was also in terms of just innovation, though, and I think, just do it, the doubling of nearly any technology has benefited from a wide array of companies and people working together to develop a product. I think with us, we looked at this multimodal approach, and we said okay, for YCM bot, this chatbot interface is going to be crucial for the way we deploy this technology internally. And we looked at it and went, Okay, we can’t just be dependent open API, we may want to integrate with other vendors like Google and use the BOD API, we may want to develop our own model and interact with it completely from scratch. So I think one of the things we looked at and said, Okay, we need to keep, obviously using this technology, because they are the front runners at the moment. But we need to be able to have an option. So what we did with our weissenbach was we made it extremely easy to plug in a brand new model. And you could just use it as if you were using, you know, open AI or pod or any other model.

Marlene Gebauer 28:34
So Shawn, in preparing for this interview, you brought up the idea of multi tokenization with AI and common law, and can you can you tell the listeners more about this so that they get the benefit of of our conversation from earlier.

Greg Lambert 28:49
So they can be excited when about it? And when we heard it the first taxi?

Shawn Curren 28:54
Absolutely, we would encourage anybody who hasn’t already to look at to review. Elimination is all you need paper. And to summarize, this paper basically says that there’s subtle, non obvious errors in generated common law and case law from these models provide a significant create a significant risk for an industry, which is that those subtle non obvious errors may be fed into motions to the court, the judge may accept certain arguments, they may miss the fact that previous common law has been slightly misquoted because it sort of had the same semantic meaning or there was a subtle word difference because the model just generates an actual slightly word that may get fed into a new judgment. And we potentially contaminate case law. They continue to do that. So regulation and legislation, lower risk because that’s a very centralized process. And the chance of that happening is low. But when you’ve got a decentralized process, like common law, where you’ve got a circuit court in the foothills of Wisconsin with one judge and one local solicitor. The probability and perhaps uninformed on AI, the probability of them, double checking the case it exists, teaching the facts of the case using it to support the argument. And then that being wrong, is much higher. And that’s a risk that we have violated. And it was it was really concerning. So in your paper, we talked to how you could perhaps solve for that. And so when the model is providing outputs, on common law, and it sort of says the facts of the case of this, this is my the judge says the facts of the case of this, this is my opinion. And I’m making this decision based on inverted commas, the the decision that has come before it 1000 times, or 2000 times or 3000 times, it is possible that the model swaps out what’s when it’s when it’s rating, and say that quote, it’s possible that it puts in a different quote, we had examples in the paper, where the cause of action definition was switched. With a more technical legal version, it was switched with a more easy to understand version from a different set of common law. But it was sort of same. In this case, this is the cause of action definition, but actually, it was a different cause of action that actually, so we started talking about how it with Kinsley so for that, and the multilink tokenization approach is that when we’re tokenizing models, which is breaking up words, so here’s an empty sequence. Here’s the next word, when you get to the next word, and you’re going, Oh, this is a judge talking. And this is a quote, that exists a million times and common law, you know, from COP, carbolic, smoke ball company, you know, Steve Donahue versus Stevenson, we’re not gonna, we’re not going to probabilistically guess the next word and say these quotes, we’re going to take verbatim, every single word. And so when you’re building your models, and you’re tokenizing, the idea is that when you tokenize, that quote, you take it, you take the whole thing, you don’t break it up based on what you don’t have. So it’s not guessing the next word, it’s guessing the next word up to the point in which it has to guess the next quote, because we cannot manipulate in any way, the spoken word of a judge. And so it’s just one idea to try and to try and resolve this the way that Lexis are dealing with this, which is really suitable as the take the court and the search for debt, and then make sure it exists and the rank it and the law, it’s important and that sense of one, that’s smart, but it’s this idea that we could get that rate at the model creation, and the model architecture fees, rather than after the fact always checking if the models Okay, to make the model better its source. So it’s a sort of theoretical is a theoretical approach to trying to resolve the issues of hallucination within the need an industry.

Oliver Bethell 32:53
Shawn, in your, in your research, did you come across some examples of where case law citations had been incorrectly summarized by judges, and therefore, if a model was to use a probabilistic outcome, it would just potentially regurgitate mistakes

Marlene Gebauer 33:10
would regurgitate human error?

Shawn Curren 33:12
Yeah, yeah. So so there was one instance where a court in Belfast and Northern Ireland quoted Denning, talking about the big red hand, the more onerous and unusual the term, the more notice has to be provided to it. And the judge, or the clerk of the court, whoever writes up the judgment, misspelled the word read. And instead, the word read was in the judgment. And we were doing a comparison between the, you know what the actual court says, and we remember looking at that one and going, Oh, interesting. The model has pervaded the word read, instead of it, we got it the other way around. But we thought it was the model who got it wrong, but it was the model, the model corrected it by putting in reads because probabilistically read was more relevant than that read was more relevant than read this is fascinating. But that’s the exception to the rule. Don’t want people to see that. You’re rubbish and esbs. But most of the time, most of the time, the model was, you should look at the paper, but it was it was what the model was suggesting. So basically, what we did, we took a judgment, and we took a bit of text, and we would ask the model to fill it in what the model was suggesting was semantically the same. But it was It wasn’t what for what so it was, which was amazing, like the model understood the key slot, understood the context understood what it was meant to do. And it explained, you know, the quote, so the bigger you are, the more unusual a terminal notice has to be faded to it may have said something like the court, you know, the term has to be obvious. So it can get the point understands the point, but it’s not quoting it perfectly. That might be close enough sending them as a quote that it doesn’t really matter. It doesn’t take their days off in a different direction or it may be so you know, it may take off and direction in the face may depend on that and then that potentially causes was a CD session. So that’s the kind of genesis of the paper. And so yeah, that’s why we will be articulate that to our lawyers. And we talk about we showed them the cause of action definition sweat out and see, would you notice that in practice, and go, Oh, my god, yeah, cool. We’re not going to touch this for legal reset.

Greg Lambert 35:17
You know, you probably made every legal geek leaned over and turn up the volume on this last part, because it’s super, super interesting. And I think it’s something that answers a lot of questions, and kind of settles a lot of doubt that people have on how, how are we going to stop with the hallucination issue? How are we going to get past this? Eventually? And we will, but I think this is the most clear answer I’ve heard on how we get past hallucinations of eventually. So thank you for that. And I did want to say we’re gonna put a link to the paper, which is called hallucination is the last thing you need, which is on the archive.org. Site. Great, great read, I usually don’t read these types of articles. But I went through this one, I love how, you know, essentially, you had what one in every 20 Or one and 20. Results were actual, actually correct in quoting the resources. So you know, if you’re a legal geek, like, like I am, this is definitely a must read. So thanks. Thanks to the three of you for writing as it was great. Olly, I want to I want to turn back to you and kind of talk more. We’ve, we’ve mentioned the YCN bought a number of times, and I do want to clarify that and tell me if I’m wrong, that YCN stands for your company name, right? It’s you. And so would you mind just kind of giving us a high level overview of what the YCN bought does. And more importantly, talk about why you’ve cited there at Travers Smith, that you dedicated the resources in building this and developing it, but then you’re turning around, and you’re making this an open source project. So that basically anyone can go out and use the sweat of your labor to to have access to this. So what was the reasoning behind that?

Oliver Bethell 37:36
Yeah, so to start off, start off at the top YCM. But your company name bot is the product that Sean described earlier on in the episode, which is a chatbot interface to allow people to use or benefit from the capabilities of generative AI, specifically ChatGPT models, but through a safe enterprise environment, and it was something that we thought was necessary, in order to give people the ability to be able to use an experiment and learn from this technology, but provide some safety around it, we provided that safety through the enterprise terms with initially open AI, and then Microsoft, but also with some additional terms of use that just guide people and and then we were able to add some new features such as person’s name, entity recognition, which is the ability to spot if we were inputting a prompt that contained an individual’s name, or perhaps the results might have somebody’s name. And we’ve subsequently added an override for that, but just certain, I guess, governance safety features that have allowed us to be able to use this technology in a safer way as possible. That gave us the ability to be able to have really fantastic conversations internally. So people could experiment, we can have these great discussions. But we also recognize it was real potential for our clients and the community to be able to use this as well. And we’ve got a long standing history that we’re really proud of, of open sourcing technologies. So we did that with other products such as matte mail, and also our original document labeling platform at a toner that Sam was involved or actually pioneered and wrote. So we open sourced that made that available. And it’s been a fantastic way for us to be able to demonstrate our capability, have great conversations with people in the community, but also it’s been really fantastic for attracting and retaining talent as well. So we see that as part of be part of our people position. So why YCM bot has been great. We obviously promoted that. We’ve had some fantastic discussions with clients and also other other law firms as well. And we believe that several businesses have taken it and implemented it. We’ve done it under the least restrictive license, which is MIT, which is great, because we believe in that. However, that doesn’t prevent us from being able to see actually who has deployed it. But But we understand, as I say several businesses have and if we can play some small part in businesses being able to safely use this technology then then we’re really proud of that.

Marlene Gebauer 39:55
So what was the collaboration like with 273 Ventures on the Y See and bot. So

Sam Lansley 40:01
collaboration with Dan and Mike as they were a big part of us deciding to make the model make the platform more friendly to multiple different model types. So I think before we actually built the interface, we decided quite early on that it would be worth investing in an interface that could be user friendly and allow us to consume enterprise API’s. But I think what was really useful was talking to Dan and Mike, is they explained how useful it would be to be able to open up the different GPT models. And I think, based on that we actually added those features to it. They also had a little bit, look over the code, give us some advice on that. But yeah, that’s probably the most the most of the collaboration between us and 273 Ventures.

Shawn Curren 40:41
I think the one thing I just add to that is, we’ve now be open sourcing for four years, we open source enterprise technology that, you know, we all believe we could potentially sell. But what we’re finding is that a lot of law firms who want to take that technology, don’t have the sophistication of an engineering team who could implement it. And so we are trying to create partners in the industry who have a bit of an engineering ability or capability to change to be able to support taking that open source code and implementing it within law firms. Obviously, we don’t get a benefit from that commercially, but it means that people can benefit from the product.

Greg Lambert 41:14
So I’ve been going around my firm giving these talks on generative AI uses. And one of the common things that I both address and I also feel questions on is the safety and security of the generative AI ai tools. And Ollie, you kind of mentioned earlier about being you know, the difference in the licensing between being a business to business license versus a commercial license that most of us use when we jump on, jump on and use these tools. So when it comes to the YCN bot itself, what are you doing to make it safe for users to go out and experiment with the generative AI tools that are out there,

Oliver Bethell 42:00
I think there’s a couple of bits to touch on here. What is around the multimodal strategy components. So YCM bot has been built so that it can connect by API to any model. And then obviously, when you contract on enterprise terms, then you’re specifying that you’re not prepared to allow your prompt data to be used to improve and train the model. And the additional benefit, obviously, that everything that is prompted and then return back is is encrypted under all of the expectations that we would have. That’s then gets extended through through to the Microsoft model as well. And those those enterprise terms will be what we would expect when we integrate with any model. There’s an interesting point, I think it’s worth raising, which is, it will be interesting to see how the fur throwers describe it. As these lawsuits get brought to open AI. Is there a possibility that in future we may see there being restrictions applied to some of these models? Because it is generating content that is contaminated? And if that is the case, products that are built on top of those models? May they need to be restricted in some sort of way. And therefore, we are we’re obviously in the very early stages at the moment of experimenting with this technology. And so is there a way of maybe getting on to a bit of a crystal ball question here. But is it possible that even though the interaction with these models is secure, and under the right terms, as see the training data that has been used, could be brought into question that might mean that the use of those models needs to be thought about more deeply. And so having the ability to switch between different models, I think is going to be necessary in order to mitigate for the risk of that. We could even see there being a point where the most conservative of clients may say, we only want you to use a very restricted model that you can confirm that all of the training data is legitimate. And we want that you to run that on your own infrastructure. That is a potential extrapolation that we could see in the future. And it’s one that we’re thinking about quite quite deeply. So we’ve obviously taken all of the steps that you would expect of anyone that’s working with enterprise software today. And that’s how we’ve we’ve engaged with Microsoft. But we can see in the future, there might be a need to think more deeply about the quality of the training data of these underlying models.

Marlene Gebauer 44:16
And I’m wondering, I mean, again, you know, when you’re talking restrictive, less restrictive type of type of models. I mean, is there also a pricing component to that, in terms of what firms will, will charge to use them? Yeah,

Oliver Bethell 44:30
I don’t know enough yet about the cost of running these different models. But I think you you had a guest on the HyperDraft podcast a couple of weeks ago. And they made an interesting point. I think you’re talking about smaller models. And one of the things that we are thinking about at the moment is, would we potentially have smaller, more restrictive models that are finely tuned that are able to perform very narrow tasks, and perhaps they’re the ones that get used for the most sensitive of use cases, but they could potentially be quite expensive to train. So there might be a cost element there. Anecdotally, when it comes to cost, what I would say is that just the experiments that we’ve had so far have been pretty, pretty low costs in terms of how much is your technology that we’re using for the build is pretty minimal. So we know that at the moment, that cost isn’t that great. But I can imagine that potentially increasing quite significantly as we start to use more GPU resource.

Shawn Curren 45:26
And yeah, I think I didn’t just add as a fee, allocate some monetary value to every word that exists on the internet, and are talking like 0.00000001 cents. And there is a mechanism for that to be returned to the copyright holder, the person who wrote the blog, the person who wrote the story that fascinated the academic research, whatever it is, then clearly the cost of these models on entrance, right now it’s fairly low, and it’s nothing, it’s less than a month for us, you know, maybe don’t put that in, it’s very low. I don’t know if that’s anticipated, but it’s very low. But you can see how if we just sort of crystal bogeys, if the authors who are seeing their work has been taken in as a corporate issue in this, and the big tech vendors want to retain a lot of that work, because it’s the model, there might have to be some form of negotiation and renumeration back to the data owner. And that might be in the form of pennies. But it may still be something that has to happen. Similar to the music industry, where now you’re getting royalties that are in this sort of pennies and pounds, that kind of capability could evolve yester models, take the data and aggregate it in the output, it may be difficult to link it exactly back to the training data. But it’s not impossible. And there might be a way to do that on a roster or off basis. And so if you think about the fact that we might have to renew MIDI, the information creator. And if we don’t do that, I don’t know what the future of these models is because I’m not going to be creating any more information on the internet for somebody else to benefit. So if we think of it that the cost of inference has got to go up, it’s got it’s got to trend up quickly, we’re getting a really cheap free version of it. I don’t want to use the word Napster. But, you know, it’s but it’s, it’s good to trade up. Or as Olly said, the models have just got to become more narrow and focused on if you go into holding face to deer hunting faces a publicly available data asset police with different licenses of data for training data, there’s 3000 Lexis warning, these 1200 Odd packages of data that are MIT licensed. So, you know, for us sensible, sensible thing to do for any business is take all of that training in one model, get your own data that you want to cooperate for, refine it, fine tune it, see if that model is any good. See, if it can do some tasks, then you’ve you know, you’re hedged, you’re protected. But there’s also lots of detail on eigene CaseText, under more restrictive licenses, your patchy etc. And it’s going to be more difficult to use those for commercial use, and not continue, you know, if you use an Apache license that says that you’ve got to continue to open source, anything that you do, which effectively with throws would mean all of your data needs to be open sourced. I don’t know how we deal with that. So it gets very complicated. So I do think the cost is going to go

Greg Lambert 48:10
up. Yeah, we’re almost in the equivalent of a zero interest rate model, where things work really well when money was was super cheap. That don’t work well, when the interest rate is now at 6%. So yeah, yeah.

Shawn Curren 48:27
I mean, in one of the things I think is fascinating, it’s just happened yesterday, not to pick on certain companies. But there is a leading audio physeal vendor, who have been in the news mainstream for injecting into the ATMs, a clause that said, Any conversation that happens on their platform, platform similar to this can be used to will be retained and can be used to improve and develop the services. That clause has been around bags, but boilerplate clause has been around for five years. Nobody’s paid any attention to it. Yes, II I think it’s not really Buhl. What do you mean data can improve models? We don’t really get it. And now it’s had to be interesting. Like yesterday, it’s at the mainstream. So there’s gonna be lots of people focusing on that and going, Oh, is my that is my data, valuable data, retain it? Do I need to get some return and some renumeration from it if I’m going to allow algorithms to be trained to provide some form of service that sort of automates what we used to pay a lot of money for? And I think that’s going to be really interesting is it kind of consumerization of data labeling and the importance of it for algorithms in AI?

Marlene Gebauer 49:31
So clearly, there’s a lot to think about when it comes to AI models. What advice would you give to lawyers or other legal professionals who are interested in experimenting with AI in their practice? You know, how can they leverage the technology, both responsibly and ethically?

Greg Lambert 49:53
At everybody, all the ones

Oliver Bethell 49:58
on the conversation here Often, I think what if I was a lawyer wanting to understand how I could leverage this capability, I would be investing my time in learning how to prompt as effectively as possible? How am I going to want to develop this technology myself? Probably not. Hopefully, I’ve got a great team. I want to Travis Smith that I can lean on for that. But my input into the development of the products would be around the prompting. And one of the things that Shawn, Sam and I were talking about just this week was, there is going to be significant IP generated in sophisticated prompts on their own. And so actually, perhaps some of the products that are going to be coming to market over the next six to 12 months, a lot of the investment is going into building sophisticated prompts that will be abstracted by user interface controls. So as a lawyer, spending time on investing your energy into understanding how you can leverage technology with more sophisticated prompts will probably be to the benefit of your organization.

Greg Lambert 51:00
You know, we’ve seen the big players in legal information, Westlaw, Alexis Bloomberg, others, lots of others jumping in on the journey of a game. But as, as developers, as users, what would you like to see them working on that would help you know, all of you and your duties to to help advance the technology there, TraversSmith? And, Sam, you want to take a stab at that one?

Sam Lansley 51:31
Yeah, I think, obviously, with these with these vendors, the key for them is if they’re going to invest time in legal research is to focus on dealing with a loose nation problem. I think that’s the big focus, I understand they’re already doing that to a degree as well. But I think for us, we can’t really consume those legal research tools properly, without having a better understanding of how they’re going to deal with hallucination problem, I understand that we’ve obviously come up with one exciting mechanism of how to deal with that, and people can read about it in our paper. But I think at the moment, it’s just that inaccuracy point. And it’s it can be really subtle. It’s not as simple as just doing a case law name search, it’s much, much more complicated than that. Because sometimes, the facts can just have such a small difference, if you just added a couple of words here or there, which aren’t accurate. I think this part is really key. Because I think, obviously, there’s the judicial process will have a role to play here as well. But this really could be scary if it goes badly if everybody starts relying on these open AI tools to summarize legal information. And then if people use those in a legal argument, and the judges actually accept that argument, that becomes part of common law. And that point would literally allowing AI to write our legal system for us, right the laws that we will have to comply with. So think what I’d love to see from Westlaw and organised organizations like that is some real serious thought taken to any feature they offer. I think they are doing that, which is great, but just whatever else they can do to really stop this loose nation problem. And probably encourage them in a way because of the risks associated with it. If they actually can’t deal with hallucination problem, I think they should probably maybe even think about putting the feature out there in the first place. Because it is we’re already seeing it. And I think we’ve seen examples so far where judges have caught it, which is great. What does frighten me a lot more is situations where judges don’t catch it. And then we have a situation where we’re trying to rewrite the last 20 years of legal precedents.

Shawn Curren 53:21
One of the things I’m struggling with, and you mentioned, Greg, that you were involved in a librarian conference. And actually, there’s a there’s a librarian at the William and Mary Law School, who wrote a very interesting article, The Wall floor library. And as you wrote in tonight, there’s 2019 law before LLMs became a thing. And the article and you could perhaps share it is who owns the law? And I think that’s a really interesting question. When it comes to to waistline Lexis, which is to the to the one the law does the legislator on the law. And if you think about what LLMs Genie AI are doing to many other industries, investors right now aren’t looking at the intermediaries who take data and then resell it so violently, you know, under the hood for like, you know, Travelocity and Expedia and booking and all these companies is a data provider who provides them a lot of information about flights and hotels, and all that sort of stuff. And very enter the data providers market cap is 2 billion. All these intermediate ease is like 50, for 60 billion, because you’ve been a customer if at the margin. And actually a data provider doesn’t want to engage in deal with that. And so if you think about that model, the investors are seeing the data providers know, the value creation, and the intermediaries, perhaps and troubling and struggling a little bit. And so if you think about how do you apply that to earn industry, if there’s an algorithm that can take a problem and can compute that against accurate, accurate up to date, legislation, regulation and common law to provide an answer that is as good, if not better, but certainly as good to what we could do sales, who gets the commercial benefit of that, who gets the game. And it’s effectively the provider, it’s the person who owns the law, this is maybe 1020 30 years in the future. But that’s something I think we have to figure out. And so we believe that the laws will open, does an open justice movement in the UK. And we believe that the law should be open. And it should be accessible to everybody. And it means that any business could take that feed into a model and use it to compute outcomes. And I think that has to be the future that I’d be very interested to see 20 or 30 years from now, if actually, large tech organizations could have effectively on the law, with these algorithms able to sort of ask and answer legal questions and have more automated pieces. I’m not sure how equitable that equitable that’s going to

Greg Lambert 55:50
be here, in that paper, and we will put a link on that is Lesley Street and David Hanson, Lesley, still at William and Mary, and I think David’s still at Duke. Great, great paper, and, as you said ahead of his time. And I can tell you that everyone loves the fact that you cited law librarians paper in your research.

Marlene Gebauer 56:15
So given all that we’ve been talking about, what’s the future vision for developing testing and applying the YCN bot to help transform legal work over time? What’s you know, what’s the focus?

Sam Lansley 56:29
So obviously, we’ve looked at TS bar. So YCM bar, we obviously got to spot because we’re Travis Smith, but we are looked at is producing a tremendous amount of value. But it is limited in its sense that it cannot consume documents. So we’ve actually looked at it. And we’re working on a bunch of other tools that can be used to consume documents instead of chat. So I think that’s the biggest feedback we’ve had internally. And it’s one of the things we’re prioritizing now is opening up features where you can just for example, upload a series of documents, ask a series of questions about those documents, and then get this back in an automated response. So for example, one of the things we’re doing as you can upload 50 NDAs, you can ask a series of questions about those NDAs. And then what will happen is these GPT models will go and iterate through every single document, ask those questions of every document. And then you can get the answers back in the user interface or you can get an Excel export. So I think that’s where we almost see the future of this product. Because I guess we do see as TS bots having a role. But I think this will be more of a focus, because I think TS bot will be still there for quick questions for things where you don’t require a full on Document Upload. But I think if we just look at some of the processes we do, it’s crucial that we actually have a process to upload massive large amount of documents, and then we can get questions answered of each of those documents.

Greg Lambert 57:45
And I love the fact that you that you use TS bot to replace these your company name because that just drives home the why it says your company name for so. So if I had a V the JW bot, right? So we would call it exactly

Marlene Gebauer 58:00
is there any thought to sort of looking kind of internally at at the firm’s own documents or administrative things, or as opposed to client related documentation.

Shawn Curren 58:14
So we just think it’s just looking internally, to the extent that we take your data to tokenize it and fine tune models. I think certainly we’ve got data in our document management system where we are 100% confident that we want to cooperate, because not everything in your document management system is owed. Sometimes it’s the other side, sometimes it’s the client. So you know, you might use it to improve and refine models. We had this question actually from one lawyer was it? Can we just take a piston bank now and just train a model? And then ask the model the question to give us the output for the precedent bank. And the point I made was that, you know, the benefit of the Large Language Models is that they understand the concept of market understand the question, understand friendly and buyer, buyer friendly and seller friendly, just by taking an untrained model and taking those precedents, it’s not going to know any of that. So it’s not going to be any more sophisticated than what you’ve got just now. And this is the point that we talked about earlier, as you talked about a previous contributor talk, you know, talking about existing tooling, and is natural language input to get a clause and a precedent bank better than 50 tags curated, maintained, kept up to date, by experienced knowledge lawyers, thinking about common law developments, legislation, developments, regulatory developments, and applying that to that precedent bank. I’m not 100% sure whether the natural language interface to get that output is significant gain efficiency, or they’re just going into the basement bank yourself and and looking for that. So but certainly on the training side, I think that might be to complement an existing large model. I think that’s an area that we might look at the future.

Greg Lambert 59:56
Well, guys, we all we ask all of our guests that The crystal ball question. And it is that time. Otherwise, if we continue this conversation, we’ll have to split this podcast into two episodes. So I want all of you to pull out your crystal balls and peer into the future for us. And you can either answer it individually or as a group. What are some of the challenges or changes that you see, I would say specifically with AI, and in the law in the next two to five years?

Oliver Bethell 1:00:30
I’ll go first, Greg, I think it’s what I touched on earlier, which is the adoption of these models, whilst they’re still potentially question marks over the training data. So I think we’re at a stage where there’s clearly a rush to deploy these technologies as quickly as possible, there’ll be a few bumps in the road. Clearly, there’s enduring innovation here. So it’s going to have a fundamental impact in the long run. But there’s some figuring out to be done about how to safely apply the models. And that’s where I think being model agnostic is going to be critical to the success of any business that’s going to leverage this technology.

Shawn Curren 1:01:05
I would say that, I feel like that there’s a bit of a Napster music industry moment happening with existing advanced alien names and ad information based industry. And I think that over the next five or 10 years, we’re going to have to figure it out, whether we want Spotify and an Apple Music, dominating all of those industries, or whether we want the fair and equitable distribution of AI to go, to beat him up to be spread out more evenly across businesses. And so be open source where we create a bit open source, that’s where we want the law to be public. And I think that’s going to be a really interesting development to watch over. I do think, you know, when my kids get to college each and every 20 years, I think there’s gonna be so much change from where we are today. But it’s exciting. And I think it’s positive.

Sam Lansley 1:02:04
Yeah, my opinion, I think it will kind of follow the, the progression of this the general AI industry. And I think it’s really interesting there, because obviously, when I was first starting my career chatbots were a thing back then, obviously, they were nowhere near as sophisticated as they are now. And I think in 2012, I think it must have been I was developing chatbots. And then I think we gradually you see how far it’s progressed. And that has been impressive. But one of the things I wonder now is if we’re going to move beyond that, I want to mean by that is, there’s a limitation at the moment, if you use these chatbots, you’re still heavily dependent on a human per human agent going in there developing the right prompts. And then they still have to say something, if you go to a lawyer, for example, and say, you know, you have this case, they’re still going to have to break down that case into a series of prompts, which can then be asked of these GPT models. And then that’s the same for most industries. And I kind of wonder if Auto GPT. So this is an open source project where you can give it high level goals, and it can actually go out there and achieve those goals. For example, if you say, you have a goal to grow your businesses like, you know, marketing process by blah, blah, I can actually go out there and take certain actions to do that. And the idea being, for example, if you want to grow your online presence, it may actually go out there and start, you know, writing tweets for you, in my next stop posting on LinkedIn. And I kind of wonder if that’s going to be the next level in terms of AI. And I kind of wonder if there’ll be more focus on actual agents where you can maybe this is not two or three years, maybe this is 510 years. But I do think that eventually, someone’s going to have to deal with the problem of the limitation rather, of people being so involved prompting the algorithm correctly. And I think we could see something with these algorithms are so sophisticated, where you can give it a high level goal, it can break down those goals into a series of sub tasks, ask the user to approve the subtasks then actually go out and do some of the actions that are required. I think it might happen in law, it might not. But I kind of see that’s where it’s gonna go, to be honest. Well, we

Marlene Gebauer 1:04:01
may have to have you guys back in a year or so just to see, see how things turn out. So Shawn Curren, Olly Bethell and Sam Lansley, thank you very much for taking the time to speak with us here at The Geek in Review. Thank you. And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found primarily on LinkedIn but also at gay Bauer am on Twitter and at m gay Bauer six six on threads.

Greg Lambert 1:04:36
And I can be reached on LinkedIn as well and Lambert on X slash Twitter, whatever it’s called the sweet X. And, but more and more on glamoured pod on threads. So gentlemen, if someone wants to learn more, reach out and find you online where Best place to do that.

Sam Lansley 1:05:02
Yeah, the best place to be on LinkedIn. So if you could please follow our Travis Smith AI page that would be great.

Marlene Gebauer 1:05:07
And listeners, you can also leave us a voicemail on our Gigan review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca Thank you, Jerry. Thanks,

Greg Lambert 1:05:19
Jerry. All right, Marlene, I’ll talk to you later Hey

Unknown Speaker 1:05:34
Take me away. Welcome

Speaker 3 1:05:45
back, devils back home. Back home

In this episode of The Geek in Review, hosts Marlene Gebauer and Greg Lambert interview Laura Leopard, founder and CEO of Leopard Solutions, about succession planning challenges facing law firms. Leopard explains that many firms have partners nearing retirement age but no concrete plans for transitioning clients and leadership. This lack of succession planning threatens law firms’ futures.

Laura mentions that to make matters worse, the path to equity partnership is getting longer, making it harder to retain promising senior associates and counsel. Firms have added non-equity partner roles, keeping equity partner numbers small to inflate profits per partner. Leadership lacks incentives to retire, with no retirement plans or continued compensation. All this will hamper recruiting efforts, as younger generations prioritize work-life balance.

She recommends that in order to retain mid-career attorneys, firms must rethink policies on remote work, billable hours, and flexibility. Virtual firms with better lifestyle offerings are growing competitors. But firms seem unwilling to change. Leopard argues everything should be on the table for analysis by outside consultants. Phased retirements and succession mentoring could also help transition clients and power.

Though Laura Leopard (and even Bruce MacEwan) cannot point to examples of firms that have executed succession planning well, it is possible with courageous leadership. She advises setting retirement age limits, crafting written plans, and easing older partners’ exits. A too-big-to-fail mentality persists despite serious business vulnerabilities if talent is not retained and recruited.

Looking ahead, Leopard predicts the rise of virtual firms will shake up the legal industry as they encroach on Big Law territory with alternative fee arrangements. The pandemic accelerated dissatisfaction with law firm partnership and policies. As generational divides grow, flexible virtual firms will keep gaining ground over more rigid large firms.

This engaging discussion unpacks the complex dynamics around law firm succession planning and existential threats posed by lack of preparation. As partners cling to power, can bold leaders emerge to implement creative solutions and secure these institutions’ longevity? Tune in for an insightful examination of forces reshaping the legal landscape.

Links:

• Leopard Solutions’ LinkedIn
• Leopard Solutions’ YouTube
• Laura’s LinkedIn

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠

The Geek in Review podcast hosts Marlene Gebauer and Greg Lambert interviewed ⁠Nicole Clark⁠, CEO of ⁠Trellis⁠, about their new Law Firm Intelligence tool (LFI). This tool allows law firms to analyze aggregated and normalized state trial court data to gain competitive intelligence across cases, practice areas, and performance. Collecting this unstructured data from county courts is very challenging, but provides valuable business insights.

The Law Firm Intelligence tool enables firms to identify growth opportunities, benchmark themselves, and drill down into the data to find strategic insights. Firms can slice and dice the data by region, practice area, time period, and other parameters to get to the most relevant information. LFI also gives litigators specific insights into judges, opposing counsel tactics, and case outcomes.

Trellis uses both technology and human QA processes to ensure the accuracy of the raw trial court data. The data comes directly from the courts, without any alterations by Trellis. This allows Trellis to spot trends like hotspots for certain case types, which can inform law firm strategy and policy implications.

As a newer legal tech company, Trellis initially had to overcome skepticism and get large firms to try their product. But steady growth has now built their credibility. Nicole Clark discussed the challenges of selling into the legal industry as a startup.

Trellis has exciting new AI capabilities in development that will leverage the trove of state court data they have aggregated. While widespread adoption of AI in legal is coming, though the timeline is uncertain. Clark predicts more law firm consolidation and AI startups, but cautions against overestimating what legal tasks AI can solve.

Links Mention:

• ⁠Trellis Releases ‘Law Firm Intelligence’ Providing Insights into Major US Law Firms for the First Time ⁠
• ⁠Law Firm Intelligence by Trellis⁠

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Threads: @glambertpod or @gebauerm66
Voicemail: 713-487-7821 Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript

⁠

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer,

And I’m Greg Lambert. Marlene I am teaming up with with the combination of some former guest and some fellow three geeks members for a legal value network webinar on Tuesday, August 15, at noon, Eastern, and we’re going to have a conversation about the now I guess it’s either famous or infamous Goldman Sachs report that talks about how 44% of legal work will be replaced by AI. We we had some issues with that report. So we will view it we thought we would dive into some of the details. And luckily, because we had some former guests, which includes Kris Satkunas, from CouncilLink and Nita Sanger from Cherry Bekaert, so they’re going to join Toby Brown and Ryan McClead. And I and I’m going to be moderating, because this is way above my abilities to talk about this. But we’re gonna we’re gonna have fun conversation on this. Ryan McClead also published a three part series that’s on three geeks in a law blog detailing some of the findings that we had. And I do have to put a warning out for everyone. There is math involved in this. So luckily, I’m just the moderator, I don’t have to worry about the math. I’m going to leave that to everybody else. So. So I’m going to put a link on the show notes for that. And as you know, which is near and dear to my heart with the webinars also free so

well, you know, maybe maybe you’ll get lucky and you’ll get a response from from Goldman Sachs.

Maybe God, I hope not.

That that’d be interesting. That would be interesting. Well, I’m fresh off a trip, finishing a trip to West Texas.

Was it 110 or 112?

It was actually cooler than it was in Houston. Believe it or not, it was actually much more pleasant. But you know, definitely different moving from coastal plains to, you know, high desert. So, anyway, good trip, but and I’ll be making it in another couple of weeks. Yeah.

Go Red Raiders.

Go Red Raiders. And speaking of geek and review alumni, we have one this week, who’s returning to talk to us about some really great features for state docket information. We’d like to welcome back Nicole Clark, CEO and co founder of trellis. Nicole, welcome back to The Geek in Review.

Nicole Clark 2:46
Thanks so much. Great to be here, guys.

Marlene Gebauer 2:48
So can you provide an overview of Trellis has recently released law firm intelligence feature and how that aims to benefit law firms.

Nicole Clark 2:57
So for those who know Trellis, you know that we deal in state trial court data, State Trial Court dockets and documents, we’ve worked really hard to be able to look across the trial court data from a law firm level. So that was really never possible before. And what we allow now in our law firm intelligence product, is the ability to look across cases across practice areas across performance, it’s really for law firms to understand themselves and their peer firms from a competitive intelligence standpoint, in a in a new way, with a lot of transparency,

Marlene Gebauer 3:34
I know that, and we talked about this before, that being able to obtain and aggregate trial court data at the state level is is not easy. In fact, you you may be one of one of the few that have been able to kind of figure this out. So do you mind kind of just telling us a little bit more about how you’re gathering the information and creating it for this new law firm intelligence tool?

Nicole Clark 4:06
Absolutely. For anyone who’s gone on to a been a, you know, had to look up information in a in a state trial court, you’ve likely gone to a county court website. Those are very difficult to to navigate at best. What we do is. Yes, we have to go into aggregate this data, it is county by county, just like it would be for a lawyer going on to look about particular docket of a case. We go in county by county, we aggregate all of the cases, both historically, as well as new cases being filed and current cases that are moving forward. And then we structure that data. So that data is going to be different county by county, the way they number cases, the way they describe case types, almost everything and we structure it and we classify it so all the way down to what every motion is who brought The motion what was the outcome, so really everything classified. And then all we do is we allow you to look across the state trial courts. So from a single interface, not have to go on to 1000s of different county court websites, but be able to pull up information, not only docket information, but document information as well be able to search really the body and text of those documents. And so I really like to think of us as a far more innovative version of Pacer, but with a state trial court system with some really, really fantastic analytics service on top of that data, and law firm intelligence being our newest analytics service on top of this data.

Marlene Gebauer 5:39
I remember when I was at the Oklahoma Supreme Court, and I may have mentioned this last time we talked, the most trouble ever got into was during a training session, explaining how one counties court information may be completely different than another counties court information, even though it’s on the same system. And I had one one county clerk that that called me into her office and yelled at me because she thought I was being insulting into it. I’m like, No, I’m just telling you that things are, you know, if you go from Payne County, to Oklahoma County to Tulsa County, it’s going to be input different. So it must be a challenge to kind of normalize that that information.

Nicole Clark 6:24
It absolutely is. In fact, we see it down almost to some judges with clerks and putting information differently, and you can see the same judge same county, and information will be input differently. So Challenge is a euphemism for collecting and aggregating all of this state trial court data. But we’ve made leaps and bounds probably even since last time we chatted, we will be 44 states by the end of this year, and 1000s of counties. So we’re really and I’m just so proud of the team and everything we’ve accomplished in terms of coverage and how we’re continuing to move forward.

Marlene Gebauer 6:56
So once you sort of bring a state on board attempt, do you find you know, you mentioned the judges and sort of unique, you know, three unique things that they’re doing? Do you find that it changes? Like, once you’ve, you know, once you’ve kind of mastered, mastered a particular state, do you find that then sometimes they change and then you have to adjust again?

Nicole Clark 7:17
They change daily? The monstrous act of structuring and normalizing this data. Yeah, they change their systems all the time. And so if you think about from our side,

Marlene Gebauer 7:30
How are they allowed to do that? It’s so unfair.

It is hard to tell elected officials to change the way they do things.

Nicole Clark 7:39
Their systems also break. That’s the if they’re not changing, then they’re breaking because they’re, they’re, you know, government proposals that weren’t probably the highest and best. And so the the websites are difficult,

Marlene Gebauer 7:54
at best, to put it mildly, right? Getting back to the the law firm intelligence feature, in what ways do you sort of see how law firms can leverage that that feature to gain a competitive advantage in the legal industry?

Nicole Clark 8:12
There’s two ways to the law firms side that we see this being utilized. One is the business and development. So business development marketing side, which is the ability for a couple of things. One is to review your own internal data, how are your cases tracking over time? And what practice areas are you growing most? And then sort of have an understanding of that be able to discuss it at a business level? Is that something you want to continue to spend into is that something where there’s another practice area, that could be an opportunity that you’re not currently thinking about marketing, and just as a tracking cases over time is also sort of a leading indicator to revenue? So we find that that is just from a business health standpoint, a good check in? But also from the, you know, competitive intelligence standpoint? Well, where do you stack up against some of your peers in terms of number of cases, types of cases, regions that you are growing in dramatically? And is there opportunity where some of your peer firms are growing the most for you also to make some of those business decisions that could help you grow there. And that’s just on the high level business sort of number of cases by by region by practice area over time. But you can drill down to each of these if you want to see the underlying cases in torts for Massachusetts that your firm is actually on right now. And then look at performance. So there’s so many different ways, but from a high level, and I can get into some of the more of the use cases, but from a really high level, it’s business intelligence, and then for the actual litigators. Its strategic insights on those actual cases throughout those actual documents in motion. ones that make up a lot of BS analytics.

Marlene Gebauer 10:02
I gotta wonder that even at that high level and an evidentiary approach, those conversations must be difficult to have just from the, you know, okay, how are we doing in sort of this sector of, you know, litigation versus other ones? Because, you know, obviously, that’s going to impact people’s livelihood. And I don’t know if it kind of goes down to the granule, granular level, in terms of individuals and sort of how they’re doing. You know, have you gotten any feedback like that from customers?

Nicole Clark 10:34
Well, the interesting thing about about this product, and one of the things that I’m so proud of is that we do go down to the granular level on everything that we’re showing. So a lot of AI new technology out there, and you just have to accept it. How are they coming up with that information? Are they coming up with those answers? We don’t really know. But with us, because we have the underlying data, we actually source everything on the bottom of the page that you’re looking at. So if you wanted to look in torts in Oklahoma, you could actually see because you’ll see the individual attorneys assigned to those cases. So even for origination and information like that, you can dive down into the underlying cases to see that information. But yes, there there is a lot of that, to consider in terms of how are we actually doing what does that mean, but I think if you look at it with an eye towards the opportunities, that that’s the right way to think about it for the law firm. In general, I think that having that information is going to allow them to make better decisions. We also see for marketing departments, there are some things that certain firms excel at, and each firm is a little different, some handle cases much faster. Some have higher summary, judgment, win rates, right, there’s a variety of different areas, some firms will excel at. And in with this data, the marketing department can actually see that, and now they can market those statistics, because they see where they’re excelling where that information might be hidden. Otherwise,

Marlene Gebauer 12:05
well, Nicole, I’m going to put on my data geek hat here, because in the article, it talked about slicing the trial court data. And I love the slicing and dicing of data and beating it to until it tells you what you want it to tell you. So but do you mind elaborating a little bit on you know, what you mean by slicing the data, and how that ends up helping law firms get access to the relevant data that they want.

Nicole Clark 12:37
So for everything that we’re we’re looking at whether it be cases, business growth over time, clients, judges, outcomes, all of these have filters where you can drill down, just over and over to a specific state to a specific time period to a specific practice area, to particular matter types. So every metric that you’re looking at can be drilled down in whatever way you’re looking at. So let’s say opposing counsel, for instance, you want to look at the opposing counsel that the law firm goes up against most often. Okay, that’s a great starting point. Well, what about in this particular practice area? What about in this region? Now dig into those cases? What are the outcomes of those cases? Is there information that the litigation team can use and looking at those cases, to maybe change saw their tactics or to understand their opposing counsel a little bit better. So the ability to look high level, but then to drill down to what’s actually important for whatever you’re trying to understand at that moment? That’s the slicing and dicing, that’s so valuable, even clients, right? So you can also look at your client list by the matter type, how you know, who’s the highest volume makes up the biggest sort of book of business? But then you can drill into those cases? Well, what are the what are the regions that we represent them in the most often? Are there opportunities for other regions? Can I look at my peer firm? Did they represent this client to interesting? What are the types of business that they have? Are there opportunities for us to cross market and maybe take some of that business? So everything that you look at, you can drill down to see where there’s opportunity, whether it be by a particular region, by a practice area, by a particular matter type? So that’s what we mean by the ability to slice and dice is any metric that you’re looking at. You have the ability to slice and dice for the persona that’s looking at it, what’s important to them, and what are they trying to achieve? And you can go all the way down to the individual cases, the individual documents and that’s what we mean by slicing and dicing the data.

Marlene Gebauer 14:45
Clearly there’s there’s a use case for business development and using using trellis and using the law firm intelligence tool. So if we’re going to ship to to say actual practice, like you know what types of insights Can firms and practitioners extract from, from the tool to enhance their litigation strategies and related decision making processes?

Nicole Clark 15:10
So this, I think the best way to do it would be in opposing counsel and motions tab. So you’re going up against a specific opposing counsel, you know who they are, take a look at that firm. And you can drill all the way down to their actual performance. What are their motion grant rates across a wide variety of motions, by practice area, by region by court. But not only that, if you’re in and you’re saying, Okay, we want to understand how they do on summary judgment motions in Connecticut, we can drill down and you can look at the motions by showing only the granted ones, I want to see what those look like. So that’s the ability to drill down all the way to high level, we ought to understand performance in general. And then we want to understand those individual motions, and how they position those issues. And now, you’re really in a position to be able to be proactive in the way that you respond or the way that you think about litigation strategy against particular opponents. And so for the actual litigation strategy, I think the motion and understanding opposing counsel is key. I also think that looking at your own firm, let’s say you have a case, and it is before a judge that you’ve never appeared. Now, of course, we have judicial analytics. But what if you want to talk to someone else in your firm? Well look at the cases that your firm has had before that judge, which is another area that you can drill down on, and then see who was the partner on those cases. Now you can reach out to that colleague and say, Hey, I saw you had a piece of went to trial. Before this judge, tell me a little bit about your insights. So you even know internally who to reach out to now that might have information that’s useful, in addition to the high level analytics information.

Marlene Gebauer 17:01
Just to kind of dovetail on to that I know that you mentioned in the article that there was a way that firms can analyze their own performance using a kind of a self analysis feature. Can you talk a little bit more about what that feature does?

Nicole Clark 17:17
Yeah, so that would be looking up their own firm as a as a starting place. And for every firm, whether you’re looking at it from an internal sort of benchmark standpoint, or you’re looking at an opposing counsel for, you can look high level at a glance, across case types across practice areas, you can dig into the underlying cases, you can look at the opposing counsel that you appear before most, you can dig into your own client list and understand who’s making up the bulk of that business. By volume. Certainly, you can go in by judges, judges that your firm has appeared before. And then finally, you can look at your own outcomes. So by practice area, by region, by particular court, how are maybe the case type that I tend to handle the most? How are these cases that the outcomes actually coming out? Where my firm has been handling this? And is there something we can learn from those outcomes that maybe we can lean into if it’s great, or we can do a little bit differently if there’s ability to improve there. So it’s being able to really assess yourself, whether it be high level firm, or individual attorney, I want to see sort of, you know, information that’s going to help me on my phone caseload right now, that helps me understand how to improve, and how my opposing counsel tends to handle their cases, their sort of litigation strategy, in their cases in this practice area. Do they settle? Do they take it all the way to trial, these are pieces of information you can take back and use certainly in setting expectations with a client, but also the way that you think about planning on the case and you’re on litigation strategy.

Marlene Gebauer 18:56
It’s almost like a wellness check for firms and in terms of their their litigation health. I learned you know, where you know, where where, where are we good, where do we need some help, and some work so

flatlining where our cholesterol really

hopefully no Blaze. Nichol we talked a little bit earlier about the challenges of how state court their their you know, things break down or things change on a regular basis. And that’s got to be challenging in terms of gathering the data normalizing the data, so you know, how do you ensure the accuracy and reliability of the trial court data that’s presented in in trellis and in the the law firm intelligence feature?

Nicole Clark 19:45
So you’re absolutely right, it is not easy. We have taken on a very, very large

Marlene Gebauer 19:52
I know we talked about this last time, too, because we were you know, we were we were and are seriously impressed that you kind of like took this this on? Because this is this has always been no small feat. No, it’s always been the challenge for years and years and years just because of the things that you’ve highlighted.

Nicole Clark 20:10
Yep. So we have both technology and human sort of QA processes. So think about just tons of tests that are running constantly that let us know immediately if something is broken, if we need to rewrite to make sure that we can adjust to the way that the law that the county is doing something a little bit differently, if data isn’t coming in as quickly if any anomalies are happening. So that’s all on the technical side, where we have all of these checks and balances that let us know from a technology standpoint, changes that we need to make. And those are every day. That’s just the cost of doing business. We’re trying to bring in this data, then we have the human side of that, well, where there are these anomalies, let’s look at it. It might, in fact, be accurate. But let’s have a human confirm. Yes, that is based on the data actually, what’s going on. It is an outlier compared to other judges are compared to other things. But we can look at the data and see that that is the data itself is the trial court data. So we there’s there’s no we don’t put our own data in there. We don’t put firm data in there that is directly from the trial courts. So we have what the courts have. But our goal is to stay as up to date as the courts are. And to always provide that data. Now, in general, the courts might be slow, and they might be behind. But they’re not wrong, usually. And if they are, we’re going back and refreshing that data. And it’ll be something like a spelling of a name or you know, minor things. So with the data being refreshed with the court, that becomes it’s going to be accurate, as the Court updates that information. But none of the data, we don’t make changes to the data, the data is what the court provides is the same thing that if that attorney you were to, you know, find the relevant court website and dig in, assuming they could find the case, then that’s what they would find on from the court directly, as well. And then for us, the accuracy also comes from well, we’re doing these analytics, but let’s look at the underlying data constantly, and assess. And that’s what we’re so proud of with law firm analytics is that, you know, we have all that data, and we can provide the analytics on it, but pretty able to provide the underlying data for every piece of analytics in real time. Law, firm intelligence also updates with every new case, it’s being filed with anything that’s changing. So it’s real time updates. And we’re really proud to be able to provide the underlying data for the law firms, attorneys, corporate counsel to be able to dig in, if they have any questions, or if there’s information that might be valuable for them in some of those more granular pieces, the cases or the documents as well,

Marlene Gebauer 22:52
no small feat, just like what Greg said,

Well, you know, one of the one of the nice things about having so much data at your fingertips, and having it normalized going through and making it useful and standardizing it is that you can identify some of the trends that are going on and look for patterns. Do you mind just kind of giving us some examples of some of the trends that you’re able to find and how law firms can can look at those and capitalize on them?

Nicole Clark 23:25
Sure. So trends, certainly within their own law firm, right? What are the the practice areas that they handle the most? But then also, what is the the Kennedy match that to something like length of a case? And then think about that in terms of business units? So business economics, so there’s so much there in sort of understanding their own data that are trends, but yet service, but you can also think about it from understanding even a specific Corporation, highlighting the areas that they’re getting sued in what regions are hotspots for them? Are there particular case types that end up being the the worst and most expensive or the highest risk for them based on outcomes based on verdict information? So there’s so many different things you can think of, and that’s from sort of individual law firms, individual corporations, but then there’s also the court system itself, where are we getting backed up? What are the new cases that are being filed? Is there a new statute or a new novel case type that is coming out that we’re seeing now in complaints and uptick in complaints, but that we know might be a bigger hotbed of litigation, and we have this leading indicator of new complaints being filed related to that. So there’s so many different ways to think about trend analysis, whether it be important to a particular corporation important to a particular law firm, or almost in the way we think of policy itself across our state trial court system, areas where we’re too backed up things we can make, do a make more efficiently and all of that can be sourced out of our data.

Marlene Gebauer 25:01
So I mean, trellis is a relatively new technology company relatively, as sort of a newer player in this this field, you know, what were some of the key challenges that charlas has had to overcome in developing the product, the law firm intelligence tool? And, you know, how were you able to address these challenges?

Nicole Clark 25:26
Yeah, and it’s such a good question, especially in legal tech, because it goes sort of forward the industry in general, it is a difficult industry to bring new products and new technology into. And I think for us early on, you know, we’re maturing more as a company now, and we’ve come a really long way. But early on, it’s getting that credibility with particularly large law firms, I think that they’re hesitant to make tend to be the first to try something, right, they don’t want to be the one to take that risk, they want to be in the group of people that are all doing, you know, the, using the same technology. And so early on overcoming that and proving that No, our data was valid, that we were providing something that wasn’t available otherwise, and getting over the hump of getting folks to to think of you as credible, and to try the product in the first place. And we’ve grown a ton our and almost 50% of A and 100, we have 1000s of law firms. So we’ve come a really, really long way. But as an early legal technology company, it’s it’s not an easy industry to sell into. And to get people to take a risk and try a new product for the first time. And that just takes time. It takes time if people seeing your company, understanding your product, and slowly coming to the the opinion that like, you know, 1000s of other firms. It’s a real product, and they’re going to believe that you’re not just going to disappear, you know, the next day and not be able to provide the service that you promised.

Marlene Gebauer 27:09
Yeah. When I was at WWL, a couple of weeks ago, there was a saying that I wish I couldn’t remember who said it. And so I could give them give them credit for it. But she said law firms are rushing to be second. And I thought that’s, that sounds right. No one wants to be first, but no one wants to be last. So that’s exactly that Nailed it. Nailed it.

That’s awesome. Well, speaking of firsts, as a legal technology company, you know, what other innovations or features does trellis have in its pipeline to further support and empower law firms in the future?

Yeah, share all your secrets with us. Yeah, please,

with all of our listeners see

Nicole Clark 27:51
my lighting up. Because I am really excited, we’ve got some huge things in development right now, I will say, law firm intelligence, we call it LSI. Because it’s a mouthful, when you say it, a lot of times. But that was a fun big project that took a lot of time, particularly because the way the courts present that data, and it just wasn’t possible to do previously. So we spent so much effort getting to where we could actually do that. And with so many customer requests, so we’re really happy to be able to provide that to folks now. But that’s not the only thing that we have coming. So we have a lot of exciting AI capabilities that are going that are in development right now, that are going to be incredibly useful. And one of the things that I’m most excited about from us on the AI standpoint is, again, it’s that data piece. So we’re not relying on a model that is the internet, which is a ton of great information, but it’s not necessarily relevant. And there’s a huge ability to hallucinate without underlying data there. Because we have this massive trove of data. And it’s data where the majority of lawyers are actually practicing. It’s where the majority of cases are actually moving forward. And we have all that underlying data. So we can do so much in being able to surface questions faster to help with motion drafting, to help with research, because of our actual data set. And I feel like that’s really, I’m excited because I genuinely believe that’s going to differentiate us from other folks that are out there. There’ll be a lot of a lot of legal tech companies that are going to have some sort of generative AI sort of plug in on their website. I think the technology is all Step No, almost becoming commoditized at this point, and I and I think everyone’s going to try and have something but I’m so excited that we have the underlying data to be able to one provide information that is not a hallucination, or that limits those as much as possible. And then to source it for our lawyers. You don’t have to trust us and a model in General, but here’s the underlying data cases documents that we use and coming up with that answer and feel free to dig in yourself. And that’s what I’m really excited about.

Marlene Gebauer 30:09
Well, Nicole, it’s been about 16 months or so since you were last on. And I took the time before the interview to go back and read your crystal ball answer from April of 2022. Okay, so here’s what you mentioned. Okay. You talked about the use of API’s. You talked about single sourcing for data collections. And you talked a lot about consolidation in the legal tech and Information Industry. So looking back, I think you did very well, on your projections. Well done. Well done. Yeah. So we do the crystal ball question, part two. And so what, as of August of 2023. Now, what do you see for things like AI, state court analytics changes that you see in the industry over the next two to five years?

Nicole Clark 31:08
You know, I think we probably all agree that AI is having a massive impact in a variety of ways, whether it’s actually being utilized in the day to day right now, everyone’s talking about it in legal. And I think there’s more of a a willingness to try new technology that I’ve ever seen before, which is really, really interesting. And I think it bodes well, for legal tech in general, that we’re coming off of COVID, where people were forced to embrace technology. And now there is this technology, where they’re being told 40% of their jobs could go away, well, we got to understand this hub, let’s, let’s look at what that actually means is that accurate, so they have to dig in and understand it a little bit. And then I think you’ve got the sort of hype circle, which is law, you know, legal tech companies are going to want some form of AI to be able to talk about and be exciting. And so there will be AI, in most legal tech companies, I believe that it’ll be utilized in some way or another, whether it’s relying directly just on you know, or GPT, or there are models that are actually being, you know, honed and molded by data. I think that’s one of the questions. But I do think that law firms in general are looking to, at the very least say that they are open to AI and are exploring those things. And I think for some firms, though, it will be adopted. And once that happens, it will likely create a wave where other firms will hop on. Now, when that massively transformed, legal or not, and how fast that happened. I think those are the two things that are up in the air. I think AI will certainly and has already started to impact legal, how fast it really does things like take away some people’s jobs, or make new jobs where you have to become better at, you know, your prompts, etc. That’s a timing thing. I think it will happen. But I think we don’t know how fast. So that’s something to be seen over the next few years. And I’m excited to watch it. I think it’s interesting. And I think there’s some things that we’re way over selling. And I think there are some things where it can be a huge benefit, and can help people do things faster and perform their jobs better. So it’s a mixture of both in AI, it’s not going to be a holy grail, wherever you lawyer’s job is done, and it’s not going to be nothing. It will help folks. It just depends on the particular skill set that you need and the particular task that you’re trying to do. And I think in legal in general in the industry, you know, I talked about so consolidation of companies. And I think that that didn’t happen as much as we thought. I mean, there’s been some big ones, right, that that we know about, but I think I’m in a will continue to move a little faster right now. So I think over the next few years, we’ll see m&a of the larger players actually buying some of the smaller companies to add to their portfolio to make sure they’re staying innovative, more than I would have thought before. And that’s one of the changes of AI. We’re also seen an influx of companies start in legal where because and it’s somewhat caused by investors who are recognizing that this is an industry that maybe they can make large returns on. And so companies are being started that wouldn’t have been funded previously. And so there will be a lot of companies that continue to come over the next year or two. Not everyone’s going to be able to make it and so that’s where I think we’ll continue to see a little bit more consolidation. But there’s an influx in companies starting that we never had before.

Marlene Gebauer 34:42
Yep, I had one follow up for you. And when it talked about law firms adopting AI, and I want to I wanted to ask you if you’re seeing something that I’m seeing, and that is because there is this interest in AI it’s causing people to actually come forward and ask questions and kind of look at different ways of performing the tasks that they’re doing. And it could result in that it’s has nothing to do with AI to improve the process at all. It could be looking at the data and putting it into an old fashioned spreadsheet. So I’m wondering, Nicole, are you seeing people coming to you asking questions off the basis of, you know, how do I get the AI to improve my process? And then you give them the adviser says, Look, that’s not an AI problem. That’s, that’s an X problem. Right? Are you seeing that as well?

Nicole Clark 35:41
It’s so funny that you mentioned that because yes, and we’re having internal conversations about it about customers saying we want x and then we we and you know, we want AI to do X. And then we huddle up. And we’re like, well, it’s not AI. Like it might be but it still might solve what they’re looking for. But it is it you know, that goes to the the excitement that maybe AI can solve everything. And then we’re starting to recognize that there’s so many things that AI is not the right solution for but then we can still come up with technology solutions that are going to make everyone’s lives better and easier. And so I like that. It’s that it’s causing the catalyst for people to question how things should be done. But not everything is solved by AI. Amen. Yeah.

Marlene Gebauer 36:27
I mean, they may not know that the appropriate tool, but like it’s got them excited enough to start thinking about how do we explore new ways of doing things?

Nicole Clark 36:36
That’s exactly right, which is just a boon for the industry and trying to figure out new better ways instead of it’s the way we’ve always done it.

Marlene Gebauer 36:44
Exactly. Well, I look forward to see if these predictions come true. Cole Clark, CEO and co founder of Trellis thank you for joining us today, and letting us know more about what you’re doing with trellis and the law firm intelligence feature.

Nicole Clark 36:58
Such a pleasure, guys,

Marlene Gebauer 36:59
Thanks for being here. And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter, and also on LinkedIn.

And I can be reached @glambert on I guess, do we call it X? No, I don’t I don’t know what we call it.

Oh, yeah, we gotta call it I don’t know. Did it change? Yeah. Change yet? Oh, yeah. Oh, yeah. Oh, dear. But

I’m actually at Yeah, sorry, actually spending a lot more time on Threads at @glambertpod.

Nicole Clark 37:36
I was going to say are you finding that the legal tech cohort is going over to Threads?

Marlene Gebauer 37:39
Finding some, some, I think until threads adds a few more features such as direct messaging and good bear groups is going to take a while so even saw one of my favorite podcasters Kai Ryssdal announced that he’s probably going to go back to Twitter until Thread gets better.

Gets its act together. So

well, but speaking of all the social medias Nicole if someone wants to reach out and learn more, or continue the conversation where where can you be found online?

Nicole Clark 38:13
Absolutely. So I can be found on LinkedIn at Nicole A. Clark. And then you can also reach out for any questions about Trellis to info@trellis.law, or just jump onto our site and start searching and experimenting yourself with searches across the state court system at trellis.law/search.

Marlene Gebauer 38:33
Listeners, you can also leave us a voicemail on The Geek in Review Hotline at 713-487-7821 and as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.

Thanks, Jerry. Alright, Marlene, I will talk to you later.

Okay, bye

Nicole Clark 38:48
Bye, guys.

by 3 Geeks (Ryan McClead, Greg Lambert, and Toby Brown)

This is part 2 in a 3 part series. The first part is here. Part 3 will come out tomorrow.

The Big Idea: We found a much better dataset, though still small, from which to extrapolate actual effects of Generative AI on the legal industry.

Key takeaways:

• We got anonymized and summarized data for 10 corporate legal departments from LexisNexis CounselLink
• The data showed that almost 40% of time entries, representing 47% of billings, could potentially use Generative AI.
• We estimate that a realistic initial upper limit for Generative AI would be to reduce that work by half, or 20% of time entries and 23.5% of revenue

In the previous post, Ryan got tired of hearing the Goldman Sachs “44% of Legal is going away” stat being quoted uncritically and decided to actually look into the underlying data used in their report. Ryan’s exploration of the data is an interesting story in and of itself, but the bottom line is that the data is fuzzy at best, the sample size is laughable, and the breathlessly unquestioning reporting on Goldman’s study has been remarkably sloppy.

After writing up his findings, Ryan shared that post with Greg and Toby, and the question quickly arose, “can we find some actual, useful data to better understand the effect that Generative AI might actually have on law firms?” Gregreached out to Kris Satkunas from LexisNexis CounselLink, a recent interviewee on the Geek in Review, to see if CounselLink could share some anonymized benchmark data for us to analyze.

LexisNexis CounselLink Data

As a reminder the Goldman data was using survey questions about how important certain “work tasks” were for their jobs. Those tasks included things like “Getting Information”, “Identifying Objects, Actions, and Events”, and “Scheduling Work and Activities”. These are quite vague and wide open to interpretation.

In an attempt to find more useful data for our purposes, we asked Kris for the percentages of all time entries that included the keywords “Draft” or “Review” in the description. Our assumption is that those two terms will capture a large percentage of actual time entries in which lawyers are likely to use Generative AI. We fully recognize that this simple heuristic will not produce a clean data set from which to extrapolate definitive results, but as a first pass at some real data, we believe this gives us a nice estimate of tasks that could potentially be ripe for automation with Generative AI.

Kris suggested that the best way to do this would be to look at law firms’ time entries for individual corporations who use CounselLink e-billing for all of their legal work. This gives us the percentage of time entries for a company’s entire legal portfolio that contain our keywords. She selected data from 10 CounselLink customers of differing sizes and representing a broad range of industries. We don’t know the companies or the law firms that they used, but we know that 7 of them had a material portion of their legal work performed by AmLaw 50 firms, and the remaining 3 had material work performed by AmLaw 100 firms.

This is still an extremely small data set, but we believe these numbers are significantly more representative of actual legal work than Goldman’s O*Net professions survey data.

As you can see above, “Draft” and “Review” time entries make up about 39% of all entries in our data set. Even if Generative AI could completely replace all of that work (which it can’t and won’t), less than 40% of legal “work tasks” (as Goldman considered them) could be automated using Generative AI. If you take an optimistic view, AI could potentially replace about half of the hours in those entries. In other words, a more realistic estimate would be that about 20% of legal “work tasks” could potentially be automated by Generative AI. That is not “nothing to see here”, in fact, it’s significant and impressive, but it’s not the horror headlines that so many publications ran back in March and April.

Follow the money

Of course, the number of time entries, isn’t really that interesting on its own, even though that is essentially the data that Goldman Sachs was actually reviewing. The real question, and the headline that should truly strike horror into the heart of every law firm partner is, “What is Generative AI going to do to my paycheck?”

Thankfully, Kris took the same data she provided above and represented it as a percentage of total fees billed.

While the number of “Draft” and “Review” entries made up 39% of all billed tasks, those same tasks make up 47% of all fees on average. Using our back of the napkin estimation from above, if Generative AI could potentially remove 50% of that work, it would result in a revenue reduction of about 23.5% to law firms in the very near future.

It was at this point that Toby pointed out that Generative AI is unlikely to hit law firm revenue evenly across all timekeepers. In fact, it’s much more likely to replace or reduce lower level legal work like that typically performed by associates.

I think we just heard many partners just breathed a huge sigh of relief, but they shouldn’t. Because if Generative AI is going to affect associate work more than their own, that means that firm profitability (i.e., the money the firm splits and hands to partners at the end of the year) is going to be substantially smaller.

But more about that in the next post.

In the next installment, we’ll look at how firm profitability is likely to be effected by Generative AI.

This will be discussed further in a free LVN Webinar on August 15th.

• Reporting about this report is sloppy
• Reporting within this report is sloppy
• The underlying data doesn’t tell us much meaningful
• 3 Geeks attempts to find meaningful data

Law.com  – March 29, 2023
Generative AI Could Automate Almost Half of All Legal Tasks, Goldman Sachs Estimates
“Goldman Sachs estimated that generative AI could automate 44% of legal tasks in the U.S. “

Observer – March 30, 2023
Two-Thirds of Jobs Are at Risk: Goldman Sachs A.I. Study
“The investment bank’s economists estimate that 46% of administrative positions, 44% of legal positions, and 37% of engineering jobs could be replaced by artificial intelligence.

NY Times – April 10, 2023
A.I. Is Coming for Lawyers, Again
“Another research report, by economists at Goldman Sachs, estimated that 44 percent of legal work could be automated.”

What Goldman Sachs actually said in the report

What the underlying data tells us

In particular, we use data from the O*NET database on the task content of over 900 occupations in the US … to estimate the share of total work exposed to labor-saving automation by AI by occupation and industry.

…we classify 13 work activities (out of 39 in the O*NET database) as exposed to AI automation, and in our base case assume that AI is capable of completing tasks up to a difficulty of 4 on the 7-point O*NET “level” scale (see Appendix for more details). We then take an importance- and complexity-weighted average of essential work tasks for each occupation and estimate the share of each occupation’s total workload that AI has the potential to replace.

Having dug into the underlying data, I can now begin to translate some of that:

• O*NET database – The Occupational Information Network, a Department of Labor sponsored database containing voluminous occupational information. This is where all of Goldman’s data comes from.
• 39 Work Activities – O*NET defines 39 work activities that apply to each job title they track (I count 41, but maybe Goldman dropped a few intentionally?)
• 13 Work Activities – Goldman identified 13 of the 39 work activities that are potentially vulnerable to AI automation, including:
• • Getting Information
  • Monitoring Processes, Materials, or Surroundings
  • Identifying Objects, Actions, and Events
  • Estimating the Quantifiable Characteristics of Products, Events, or Information
  • Processing Information
  • Evaluating Information to Determine Compliance with Standards
  • Analyzing Data or Information
  • Updating and Using Relevant Knowledge
  • Scheduling Work and Activities
  • Organizing, Planning, and Prioritizing Work
  • Documenting/Recording Information
  • Interpreting the Meaning of Information for Others
  • Performing Administrative Activities
• “Level” Scale – O*NET gives each work activity a Level from 1 to 7.  O*NET simply describes this as a “level”, but extrapolating from the Level Examples (called Anchors by O*NET), and published in the report Appendix on pg. 17, Goldman interprets the “Level” metric to mean “Difficulty”.
• Base Case – For the purposes of this study, Goldman assumes that Generative AI is Capable of performing work activities up to level 4. They suggest that findings may be significantly different, one way or another, depending on how much capability you attribute to AI.
• Weighted Average – In addition to a level, each work activity is given an “Importance” score.  Goldman combines the importance- and complexity- (level?) scores of essential tasks to come up with a percentage of the workload for each occupation, that AI could potentially replace.  They don’t provide that formula.

What Goldman Sachs does not say in the report

O*NET classifies Work Activities by individual Job Titles, not by industry or profession, so what should we consider “legal” in this report?

• Administrative Law Judges, Adjudicators, and Hearing Officers
• Arbitrators, Mediators, and Conciliators
• Judges, Magistrate Judges, and Magistrates
• Judicial Law Clerks
• Lawyers
• Legal Support Workers, All Other
• Paralegals and Legal Assistants
• Title Examiners, Abstractors, and Searchers

• Office and Administrative Support
• Legal
• Architecture and Engineering
• Life, Physical, and Social Science
• Business and Financial Operations
• Community and Social Service
• Management
• Sales and Related
• Computer and Mathematical
• Farming, Fishing, and Forestry
• Protective Service
• Healthcare Practitioners and Technical
• Educational Instruction and Library
• Healthcare Support
• Arts, Design, Entertainment, Sports, and Media
• Personal Care and Service
• Food Preparation and Serving Related
• Transportation and Material Moving
• Production
• Construction and Extraction
• Installation, Maintenance, and Repair
• Building and Grounds Cleaning and Maintenance

Level Classifications of Work Activities are not Integers

Sample Size

• Administrative Law Judges, Adjudicators, and Hearing Officers – 26
• Arbitrators, Mediators, and Conciliators – 23
• Judges, Magistrate Judges, and Magistrates – 19
• Judicial Law Clerks – 16
• Lawyers – 31
• Legal Support Workers, All Other – 0
• Paralegals and Legal Assistants – 34
• Title Examiners, Abstractors, and Searchers – 16

Goldman is not a disinterested party

Putting it all together

• Goldman Sachs – an investment bank that likely stands to benefit substantially from greater hype surrounding the delivery of Generative AI –
• Published a study about the effect of AI on ALL Jobs in the US – based on surveys completed by thousands of people, but only 31 lawyers and 134 people in legal related jobs –
• That indicates that Generative AI could potentially replace 44% of  “legal work activities” – which are at best vague and fuzzy “activities” and at worst kind of meaningless –
• As long as you factor in a proprietary importance/complexity weighted average – that Goldman does not share –
• At some point between now and the heat death of the universe – although, likely sooner.
• The press latched on to this and reported that Goldman Sachs says 44% of either legal tasks, jobs, or total legal work could be replaced by AI – depending on which account you read –  and
• Lawyers and Legal Journalists all freaked out.

Tony Thai and Ashley Carlisle of HyperDraft, return to The Geek in Review podcast to provide an update on the state of generative AI in the legal industry. It has been 6 months since their last appearance, when the AI Hype Cycle was on the rise. We wanted to get them back on the show to see where we are on that hype cycle at the moment.

While hype around tools like ChatGPT has started to level off, Tony and Ashley note there is still a lot of misinformation and unrealistic expectations about what this technology can currently achieve. Over the past few months, HyperDraft has received an influx of requests from law firms and legal departments for education and consulting on how to practically apply AI like large language models. Many organizations feel pressure from management to “do something” with AI, but lack a clear understanding of the concrete problems they aim to solve. This results in a solution in search of a problem situation.

Tony and Ashley provide several key lessons learned regarding limitations of generative AI. It is not a magic bullet or panacea – you still have to put in the work to standardize processes before automating them. The technology excels at research, data extraction and summarization, but struggles to create final, high-quality legal work product. If the issue being addressed is about standardizing processes or topics, then having the ability to create 50 different ways to answer the issue doesn’t create standards, it creates chaos.

Current useful applications center on legal research, brainstorming, administrative tasks – not mission-critical legal analysis. The hype around generative AI could dampen innovation in process automation using robotic process automation and expert systems. Casetext’s acquisition by Thomson Reuters illustrates the present-day limitations of large language models trained primarily on case law.

Looking to the near future, Tony and Ashley predict the AI hype cycle will continue to fizzle out as focus shifts to education and literacy around all forms of AI. More legal tech products will likely combine specialized AI tools with large language models. And law firms may finally move towards flat rate billing models in order to meet client expectations around efficiency gains from AI.

Listen on mobile platforms:  Apple Podcasts |  Spotify

Contact Us:

Twitter: ⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠, or ⁠⁠⁠⁠@glambert⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠

⁠⁠Transcript

Marlene Gebauer 0:07
Welcome to The Geek in Review, a podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer,

Greg Lambert 0:14
And I’m Greg Lambert and I’m fresh back off the plane 2:30 In the morning arrival time back from Boston. Yeah, it was, it was terrible. I know more about the because I had a layover in Baltimore. So I know a lot about the Baltimore airport now more than anyone should really know. Yeah. So but it was great conference. 1800 of my closest friends showed up.

Marlene Gebauer 0:39
They literally they are eighteen hundreds of his closest friends.

Greg Lambert 0:42
Yeah, my wife and my sister in law always tag along with me. And so they noticed that I can go about seven to 12 feet at a time before getting stopped by the next person to catch up. And so they they’ve just decided to stop even being around me, and they just go out and do their own thing. So which is which is probably best for everybody.

Marlene Gebauer 1:01
And I should let everybody know that we’re actually recording from The Ion today in downtown Houston. The Ion is a innovation hub area for a lot of startups and other innovators. So we’ve been here before. And in fact, we’ve actually interviewed Joey Sanchez, who basically runs the place. So we’re happy to be back here again.

Greg Lambert 1:26
Yeah, our original plan of drinking beer and podcasting fell through

Marlene Gebauer 1:32
Greg’s plan, never my plan.

Greg Lambert 1:35
It’s always my plan. If you hear beer, you know, it’s me.

Marlene Gebauer 1:38
So, we are also very happy to have Ashley Carlisle, CMO at HyperDraft. And Tony Thai founder and CEO and Chief Engineer of HyperDraft. Back on the show, Ashley and Tony, welcome back to The Geek in Review.

Ashley Carlisle 1:52
Thanks for having us. Happy to be here.

Tony Thai 1:54
And by the way, Greg, I’ll be joining you on the on the drinking side.

Greg Lambert 1:57
There you go! So back in January, we talked with both of you about the hype around generative AI, and we even titled The episode ChatGPT. If it sounds too good to be true. So we’ve had six months since then, and Marlene and I have recorded more than 25 episodes of The Geek in Review since then. And most of those on the hype of generative AI. So we kind of kind of made this arrangement when we talk to you initially that we wanted to bring you back on. And so let’s figure out where we are in the discussion.

Marlene Gebauer 2:35
And you know, before we do get into that discussion on generative AI, what’s been going on at HyperDraft? Since we last talked in January, I know you’ve been busy, and we’ve been trying to get you both on the pod for about two months now. So I’m glad we were able to arrange it.

Greg Lambert 2:49
I hear not all of that was HyperDraft. related.

Marlene Gebauer 2:52
That’s true. of it was some of it was personal stuff. So that’s good.

Greg Lambert 2:55
Ashley anything new in your life?

Ashley Carlisle 3:02
Well, thank you for your patience. And Tony can go into this in more detail. Because this is you know, HyperDraft is our whole team’s baby. But really, you know, his original baby since what 2018 2017. It has been a busy few months for us, we really are scaling because our customer base has grown, which we’re very appreciative of. But, you know, just trying to make sure because we do customization for every client that everyone’s getting kind of the level of service that will be like has been a lot. So thank you for your patience as we gotta go through our pipeline that way. But yes, I went on my honeymoon. So happy to be back.

Marlene Gebauer 3:35
Congratulations.

Ashley Carlisle 3:36
Yes. Thank you so much. And just as a brief intro as to who Tony and I are because I know Tony is probably not going to go into that. We are both former corporate attorneys. We met at Goodwin Proctor and HyperDraft. We are a trusted legal tech partner that works for law firms, legal departments providing AI powered document and workflow generation. So we get a lot of questions about generative AI. And, you know, we got a lot of questions and comments after being on your podcast in January, too, because I think we’re of the mindset that you should be honest in the hype cycle. When it comes to legal tech, you know, I think there’s a lot of people in legal tech that are just like we have to evangelize everything and only say the positive because we don’t want lawyers to, you know, not be interested or freak out. But I think Tony and I are of the belief that if we’re not honest, then people don’t actually get the tech literacy that they need to make informed choices and actually have digital transformation within their organizations. So yeah, to answer your question, I think everyone’s burnt out on the AI chat to some extent. We’re true. I think we’re at the end of the end of the hype cycle, but it has been an interesting one. And Tony I don’t know if you want to jump in there. I know you have a lot of thoughts with that.

Greg Lambert 4:49
Well Tony, Ashley has been very busy, but what about you?

Tony Thai 4:53
I don’t do anything?

Greg Lambert 4:54
That’s what I thought.

Tony Thai 4:55
Yeah, I as you know now from personal experience, everything grinds to a halt when Ashley’s not around. I would say that the last few months, especially after we had our initial chat in January, a lot of clients come to us and ask for more consultative services around learning about AI, how they can apply it in their business. And yeah, I would say that that’s taken up a good chunk of time that we didn’t anticipate kind of going into this year and coming out from last year. And I would say the other part of like, what’s kept us busy is, you know, as you know, the legal tech sales cycle takes quite a long time. And so all the seeds that we planted over the last few years have started to sprung. And so we’re starting to onboard just a great number of clients nowadays, as opposed to just planting seeds. My opinion on the hype cycle, hmm.

Marlene Gebauer 5:49
Yeah,

Tony Thai 5:50
We, I mean, you know, I don’t think it’s quite died out yet, I still see a lot of posts about it, I still see a lot of activity, especially as a nerdy engineer on GitHub, which is what we use to kind of figure out what are popular projects, still a lot of AI talk, but why I think has started to happen a little bit more of, and I’d be curious, in your opinion, what you’re seeing in the market as well, is that the general population has kind of been saturated with at least a baseline level of knowledge. And so now that they kind of know what it is now they’re thinking, okay, great, there’s a solution, where’s the problem that we need to fit this into? So I think that that’s why it’s been a little bit more quiet, in terms of the hype that’s been coming out is is just like, consumers, the buy side of the equation is figuring out, is this an actual solution for a problem that we have? What was the problem in the first place? And that’s what we spent been spending a lot of our time talking to folks about.

Ashley Carlisle 6:56
it to add on to that point, I think, though, another thing we didn’t realize was, which makes sense. Hype cycles also can lead to just misinformation bubbles. And so a lot of times people come to us with questions, and they think they know what they’re asking, or they think they have kind of, you know, the technology they want for a certain use case. And we’ve been surprised at how much like education people need, and they want and so I think that’s gonna be a big thing in the next couple of years is I don’t know if it’s companies popping up, or you know, us doing more consultative stuff, and other companies doing that, too. But you know, we’re hoping that that misinformation can be corrected, because it helps us. And then Tony gets a lot of weird questions that him and the engineers just like, are like, wait, what? Where did you get this? Is not Right.

Tony Thai 7:38
It’s equivalent to like showing up to your doctor and saying, Hey, I Googled what’s wrong with me? And I think that this is the answer. And that is useful in some respect, but not when you’re adamant that, you know, I need this or I want to do this, our GC who, yeah, you, Marlene and Greg have met, Sean has said, like, will show up selling a car. And they they’ll ask they he doesn’t fly yet. And her whole point is, your current mode of transportation is walking. We’re offering you a vehicle and you’re asking for flight. So it’s, there’s a, there’s quite a disconnect.

Marlene Gebauer 8:13
You know, between the misinformation and then just sort of? I mean, are you finding that people just aren’t really sure about what they want, just because it’s such a new technology? And then, you know, people just don’t fully understand it. And, you know, are you seeing that, that there’s some that kind of have some definite concrete use cases? And they really know what they want, and they can move forward? Or is it more people are just in exploratory mode and saying, you know, I want this this thing, and I want to be able to tell my clients that we have this thing, but they don’t really have they haven’t identified as you said, Tony, they haven’t identified the problem that they want to solve with it.

Tony Thai 8:53
Yeah, I’m gonna let Ashley take the first stab at this because I worry that I go into rant mode,

Ashley Carlisle 8:58
He will. Because there’s something it’s a great question, because we’ve gotten both, in fact that this week alone, Tony and I have been conversations with major law firms or, you know, fortune 10, companies will have different perspectives on this. I think the main theme we’re seeing is there, people in law firms or in legal departments are getting a lot of pressure from other people in their organization being like, you need to be on top of this, like, I need to know what’s happening in AI and how we’re going to use it. And people have different responses to that based on what they know. And so some people come to us and they’re like, we know this, blah, blah, and we’re like, well, your assumptions on this are wrong, but we can kind of clean this up and figure this out for you. Or people come to us and they’re like, so I know I need to use this. Please tell me like the business case and the use case and basically every soundbite that I should tell to my board. And both have been very interesting conversations. We’ve had like a litany of each but it is very interesting to see how legal is being pressured to use legal tech or AI which I think a lot of us have vendors are secretly happy as you know, like the adoption side A golfer legal Tech has been slow for, what, 30 years now. So it’s nice to have kind of a tipping point. But it does lead to a lot of awkward conversations. To start with.

Greg Lambert 10:10
Where do you think that pressure is coming from?

Ashley Carlisle 10:12
I think it’s coming from the fact and Tony is open about this, when you talk to these people, they’re just now realizing a lot of the management of firms or these big organizations that AI has been used, and maybe their r&d department, their engineering department, their development department, and that there are a lot more business use cases that they realized. And that legal being a cost center should be something that’s also focusing on this.

Tony Thai 10:34
Yeah, let me add, just touch. I think what’s really frustrating in the, in the hype cycle is when, and I’ve said this, like numerous times. Now I said, like, with prospective clients, I always tell them, hey, come up with a wish list of features and problems that you want to solve for, send them out to all the prospective vendors. And then any vendor that says they can solve all of them, you should remove them from your list, because that’s just not the way that it works, right? Like anyone that’s promises you everything under the sun isn’t going to deliver on it. But the most important thing that I think is hard for many organizations to hear is you must put in the work to get something out of it, right? Like there is no magic bullet AI does not solve, right AI does not solve for the fact that your team is disorganized, or slow to move, or slow to standardize, right. And then when we talk about generative AI, this is where I get really ranty is, you know, you hear about all this, like, Hey, let me generate these these provisions, these documents, all this custom generation, especially randomly seated generation, that’s, you know, what this a lot of this generative AI does, is the antithesis of standardization, right? If I generate the same legal concepts, and and trying to, you know, explain it via some contractual provision 50 different ways, I’ve not standardized crap. And so what I spent a lot of time talking to clients saying, like, hey, it’s good to like develop a system, do the gap analysis, and then figure out how AI can be applied to the process. But we got to be thinking in a process oriented fashion. And in a view of standardizing the process, as opposed to saying, like, Hey, I got a really cool toy, I want to use it everywhere. We see it all the time, too. When it comes to engineers, when I teach a junior engineer and new pattern of writing software, you see them write it everywhere, even if it doesn’t work that way. And the same thing applies with lawyers, like you teach a lawyer, a neat provision or a way to write something you see it plastered everywhere. That’s just the nature of you know, new technology or new learnings. But at the end of the day, what needs to happen is people need to be willing to do the work. And what I’m glad to see with AI is it’s opening people’s eyes and in their willingness to put in the work so that they can get the output that they desire.

Marlene Gebauer 13:00
I liked the point that you raised about, you know, if a vendor says, Okay, we’re going to satisfy all of your pain points that you should throw them out right away. Are you seeing that that firms and organizations are starting to recognize that perhaps it’s not going to be like a one size fits all type of solution, not gonna buy like one thing or apply one thing in terms of generative AI? That, you know, you may have a lot of different points of AI, whether that be through trusted vendors, whether that be, you know, a model that you train yourself, whether that be something that’s commercial that you purchase? Are you seeing any of that? And what are your thoughts about that?

Tony Thai 13:43
I think the bulk of firms are not even remotely they’re in that thinking, right? It’s so nascent in terms of the build versus buy versus acquire calculations that then they need to do they’re not even thinking that way. I got off the song last week with a, I would say top 10 law firm that told me that they’re going to wait for the perfect solution. That was they were like, exactly right, like wait for the perfect solution, which made me chuckle because I’m like, yeah, good luck, right.

Greg Lambert 14:12
I think that comes out in Q3. Right? Or maybe Q4?

Ashley Carlisle 14:19
Q3 of 2030 Maybe?

Tony Thai 14:20
Yeah, yeah. Oh, so I don’t think folks are quite there yet. I don’t I don’t mean to bash these organizations. It’s the nature of running a large organization, you have to get consensus amongst the team, and it’s slower moving. I think that’s largely why. Kind of segwaying over to Greg’s point on builder spy require, like Thomson Reuters decided to acquire, does that surprise us? Not really, in the sense that, you know, Thomson Reuters is a massive organization for them to decide on a dime, pivot, or not pivot but started to develop their own in house AI that’s competitive and as focused as the CaseText teams is, would be, you know, nearly impossible. So I thought it was a good move by them to pull on the expertise and the team that they need to really round out their, their offerings.

Greg Lambert 16:06
Yeah. You know, were you were you guys surprised by this? Or did you think this type of acquisition merger is what was going to happen? And do you think you’ll see more of it in the future?

Ashley Carlisle 16:17
So I had heard rumblings, but I had both reactions. At first, I looked at the price. And I was like, Oh, that’s good for our industry. At first, and then I thought about it more. And I was like, should the price be more? I’m not quite sure. I know, Tony, as an m&a attorney might have some more thoughts on that. But then the other thing that was interesting, you know, Bob Ambrogi, does a great job of covering our industry. And right after the acquisition, he covered an investor call that was done by Tr to its investors explaining the acquisition. And a lot of questions. Were basically on the premise of why CaseText? You know, a lot of companies have like affiliations with open AI, why them? And I was really happy with kind of the answers that the TR team gave kind of showed that they, I mean, we assumed but they know what they’re doing. And their answers are basically, with these Large Language Models. It’s all about data and kind of the right guardrails. And a lot of these other tools have been promised, but actually haven’t been built like Harvey AI and a lot of this other stuff that’s kind of been marketed widely. So I thought it was very smart of CaseText, to be the first one to actually build something I know, they’re currently in pilot with most firms before this news, but still to just have something because I think that’s another thing we talked to law firms and legal departments about is they just assume the stuff is already built. And a lot of times a lot of this marketing and press is about things that are in beta, or things that they can just merely sign up for. And so I thought it was very smart the way the CaseText team did it. And I know you guys have talked with, you know, some of their leaders. And I think we’ve always valued them as someone who’s not afraid to do things before other people do, and someone who really innovates in this space. So I think I’m curious to see how this acquisition impacts that part of their identity, because being a part of the TR family that might be a little bit frictional to them kind of being the first mover in legal research. And then the last thing, which is something we talk a lot about with our potential clients, if people have questions is, this kind of shows the limited use case of generative AI ai as it stands today, you know, CaseText is known for legal research. And with CoCounsel. From what I’ve seen, it is legal research, data extraction, some summarization, and people are often surprised when Tony or other members of our team have to explain to them that that’s really the best use cases for them at this point, because people think it can do everything. And Tony can go into this. But based on how the model is based, generative AI is not great for all legal work. So I really think there’s so many lessons from how case Texas navigated this.

Marlene Gebauer 18:48
Yeah. If you get a you know, transactional attorney who’s looking at that, and they’re like, Well, you know, that that’s not important to me. It’s like, I want something that drafts and it’s like, yes, but this has been trained on train to do legal research. It’s been trained on case law. So you know, that’s what this this does right now.

Greg Lambert 19:06
Yeah. Well, speaking of that, that call the TR did announcing the the acquisition, or there was one kind of theme that that they were saying throughout it, and I wanted to kind of get your opinion on this. It was that one of the reasons that they really wanted to, or they found CaseText more attractive than the other ones that were out there was CaseText had this long standing relationship with open AI, it was one of the first to get access to GPT. Four and start playing around with it. But do you think, you know that as we as we see other Large Language Models, whether it be Anthropic or Google bar or llama or any of these that come out, do you think that that that advantage will will quickly shrink?

Tony Thai 19:55
I think that the first mover advantage is useful for We’re learning about how to build guardrails and the training and the moderation piece of it. But yeah, I mean, depends on your definition of wait a great lawyer answer.

Greg Lambert 20:16
It depends on what your is,

Tony Thai 20:18
it depends on your definition of what, like, the competitive advantage is right? Like, what does it mean for an LLM to be better than another? Model? Right? Like, who’s in whose opinion? Is it? That one model is, quote, unquote, performing better than the next one? Right? Are we doing it exams are like, what is that standard? You know, from a statistical perspective? Yes, we can kind of observe that and compare the numbers. But I think that with the release of like the llama, two models and, and more open source projects, I think that that competitive moat definitely gets smaller and smaller. I think that the advantages from being a first mover and had the opportunity to spend more time with clients and customers earning their trust and learning their use cases that that’s, in my opinion, that the the actual competitive moat, not just the partnership, I don’t think that means much, anything, what I’d say on the competitive moat side, in the way that we are trying to gauge what is a good model or what what it means to be good AI, for usage, is evaluating this last mile concept that I talked to a lot of our customers about, which is AI is really good, except for the last mile, last mile still needs to be managed by a human being who has judgment and the ability to, to have intent and judgment that that’s, you know, made evident from the fact that, you know, you see those news articles about lawyers who are relying solely on ChatGPT to source cases. And then throwing them into memos and the in these are not, you know, Junior attorneys, they’re seasoned attorneys that should know better to do that kind of last mile.

Greg Lambert 22:02
Check whose name now is, is a verb.

Tony Thai 22:05
Yeah. So oh, is it? Yes.

Greg Lambert 22:08
Just shorts up a brief. Yeah, that’s a Schwartz.

Tony Thai 22:11
That’s still I feel bad for anyone that lasting swirl shorts. I haven’t realized that it was a thing. But that’s pretty funny. But yeah, I would say the last mile is probably the point of conversation that I bring up a lot with, with clients now is for them to kind of take a step back and think like, oh, yeah, what is that final mile? When I say that something’s approved, right? Whether or not I’m reviewing a draft of the memo, or a brief that needs to go out? Should I be the last person or who should be the last person that makes that decision? That’s kind of an interesting point, that we’ve realized that a lot of folks haven’t thought about when it comes to AI. And then that actually lets them work backwards into figuring out what the good use cases are for AI.

Marlene Gebauer 22:53
I had another question, sort of, pertaining to the hype cycle. So you know, initially, everyone’s like, yeah, yeah, gotta have it. Gotta have it. Gotta have it, like, clients wants you to have it. And you know, now we’re starting to see clients saying, Well, wait a minute, I don’t want you to use an NDA. I don’t want you using that on my stuff at all. And and so now we’ve kind of gone the opposite way. And, you know, I’m thinking, you know, do we have a situation where we’re going to have to have a bunch of mini models that train on very discrete amounts of data? And our clients are our security folks, are they ever going to be comfortable with, you know, whatever, guardrails are put up? And then if not, at least a lot of questions I’m giving you. And then if not, how are we? How are we going to take advantage of this? Technology?

Tony Thai 23:47
Great question, Marlene, I wish. I don’t know what your availability is like. But I wish you could just join us on most of these calls, because then you could ask them questions for clients, and then that would streamline the conversations a little bit. Yeah, no, those are those are really good questions. I’ll tack them one at a time. Ashley, obviously jump in, whenever. So let’s talk about like the ethics and confidentiality concerns. Many models, yes, like, you’re starting to see it with the gatekeeping that’s being done, or the dates that are coming up with a lot of the data sources, right Reddit, whether, you know, name, the aggregation source, they’re blocking open AI in a lot of these scrapers from being able to scrape that data to feed into the models. The next natural progression of that is each one of those companies are going to offer their own model, right, and build their own model so that they can they can make use of their their content. So yes, on the kind of boom of Mini models, I think that is definitely going to be thing. But by the very nature of machine learning, like you need massive data sets to train these models. And when you’re clear I want to tell you, Hey, you’re not allowed to mix my data with other people’s data, then it becomes a tunnel vision issue. Because if you ask a model, Hey, have you seen this type of provision? Or you seen this type of case or fact pattern before? And it tells you definitively, nope, never seen it? You don’t know if the answer is because they’ve been segmented from this other client data, or that’s truly market information. Right? My favorite specter working at major law firms is, I think, from a client perspective, everybody thinks that there’s some simplistic needed system that tells us transactional attorneys, what a market provision looks like. But the answer to that, and the dirty little secret is, and usually an email goes out to the entire firm wide saying, Hey, I’m looking for this provision. Can anybody tell me what they think market is, which is wild, right? This day and age, we haven’t even figured that out. So that’s the reason why I bring that up is I think the models are not going to be that much better. And figuring out whether or not something is market or not, with these kind of Mini models going through. And then just to segue into your other question of like, well, then how the heck do we make use of AI? If it’s not perfect? I, that’s the stuff that Ashley and I deal with on a day to day basis worse, I think, because we always had AI as a back office tool for us, we know how to interact with it. But we’ve never made it like a major premise of any service offering. Because at the end of the day, nobody cares how the sausage is made, they just want the result. And so I think that that shifts a lot of people’s thinking into, okay, maybe I can’t use machine learning here. But I know it can be automated, how do we do that? That starts a conversation with talking to engineers, realizing, Oh, I could do this, I just don’t get to do with the machine learning where I just have to spend the time and the effort. I keep saying this, right, like fundamental laws of physics applied, cannot get something for nothing. And if you put in the work, then you can get an output that you desire. So if there’s something that you feel should and could be automated, whether it’s through AI or not, now is the time to be able to ask that question and then talk to the engineers and say, Hey, can we do this with AI? And if not, AI? You know, this machine learning Large Language Models specifically? Can we do it some other way? Yeah, I’m just curious,

Greg Lambert 27:21
because I know, we’ve been using AI, on the back end on a lot of products, probably for the last seven years or so maybe a little bit longer. But really, you know, full force the last seven years or so, does this hype around generative AI so this slice of AI? Does that kind of cool down the market? For a number of advancements on the back end AI that may not may be more extractive than then generative. I’m just curious as as a vendor was, you’re developing products, you may have a solid source that you’re using to help kind of tweak the back end to automate it to use the AI in that way. And then all of a sudden, in November, the switch gets flipped, and everyone’s talking about Gen AI ai, which may or may not help what you’re doing at all. So sorry, I was kind of a round roundabout questionnaire wrote, hopefully, you get get the gist of it. No, I

Tony Thai 28:23
got it. I think it takes a lot of discipline. And it’s something that, like Ashley and I and the rest of the team talk about a lot is to stay disciplined and focused on the end product. It’s hard not to jump on the hype train. And have Ashley’s actually stated this more more concretely than I have like, with the vendors that have jumped on it so strongly out the gate, like it’ll not look so great next year, or the year after that. And so it’s really about staying focused on the end product. And as we mentioned, like the end product for us. And what we work on with a lot of our clients is that consultative approach of saying like, Hey, what is what is the problem, you think that you have that we’re going to solve, like, don’t worry about the tools, the tools, we as the vendor, and as the engineers, we’re gonna worry about that. But the fact that somebody shows up and says, like, Hey, here’s a wrench, paint my walls, it’s not going to help us, right. Like, if the problem is we need to paint the walls, we will deliver that solution for you. So a lot of it has to do with staying focused and then no know, educating the client, as Ashley has mentioned, in terms of like, how it’s and I think this is part of your question, tell me if it’s not, but in terms of how like generative AI has affected AI research in general, it’s only for the positive, which is fantastic. I would say the glimmer of, of hope and optimism that I pulled from this is is a sense that it’s gotten people to read more and learn more about what’s possible. what worried me and Ashley He, when we had this conversation in January is that people made it overblown and then over relied on it, which has already started to happen, right. But the education and the focus on the AI, like, that’s fantastic that leads younger generations and kids to want to learn about this stuff. If being able to generate a picture from just prompts inspire someone to become an artist, or to look into programming as a way of, of expressing themselves artistically. That’s fantastic, right? Like, I’m, I’m excited about that portion. Do I think it there are people in the academic research side that are not so happy with it? Absolutely. Do because there’s plenty of people that haven’t talked about generative AI that much because they’re like, dude, I’m busy. You know, and developing AI models that help solve actual problems, like, figure out different drug dosages for patients, which you know, some of my friends are working on, as opposed to hat fight generate a picture of a weenie dog holding a hot dog on a boat somewhere. Right? You know, when we compare those, those two problem cases, one seems ridiculous, and the other one seems worthwhile. So hopefully, we’ll steer towards the more worthwhile ventures as a hype cycle dies.

Greg Lambert 31:17
Now I feel bad for that country music song I had to create for me. So.

Ashley Carlisle 31:23
I mean, that’s for our entertainment. Right, Greg? That’s all good. I think one thing that I think I’ve joked with Tony about but every day, I just think it’s true. To echo with what he said, for our team, we kind of view Gen AI as the gateway drug to the AI stuff that we’ve tried to talk to lawyers about for years, and they’ve just gotten bored. So that’s good and bad, you know, open AI, and I think I mentioned this in January, but just my thoughts on has kind of been cemented, they picked the marketing tenants for AI and execute it really well, like the tenant of marketing of show not tell that is open AI and ChatGPT in a nutshell. And so that’s why it popped off. The good. And the bad of that is the good people want to learn more about Gen AI ai, they realize the limitations, they go over to the more AI side, like Tony said, hopefully we can find a practical solution, maybe not what they originally thought. But also it kind of makes this weird juxtaposition because as someone that’s watched, Tony, obviously, I’m not an engineer, but we’ve had conversations about how he’s built our product over the years and how he’s added on new things. From my understanding as non engineer, the whole process of building ml models and adding these AI features is very unsexy. So the juxtaposition of ChatGPT coming in and being like a marketing, darling and being very like flashy to the reality of everyday AI that people have been using in the background for years. For some people, it’s a very weird conundrum. So in some ways, it’s great. But in other ways, it kind of makes it awkward, because Tony will tell you, ML and AI, he did it before he even went to law school. It’s a very unsexy like thing that requires a lot of patience.

Tony Thai 32:55
Absolutely.

Marlene Gebauer 32:56
I had a question about your thoughts regarding how Gen AI is sort of impacting kind of other technologies that we’ve seen in the industry like expert systems or RPA, or blockchain?

Tony Thai 33:14
Yeah, I would say it’s, it has a negative approach. And the reason why I say this is, like with rpa, robotic process automation, if you know where you want to go, let’s say we’re driving a car, we’re trying to get from point A to point B. And the solution set is, okay, I need to go to the grocery store, what streets do I need to do? And you know, what do I What steps do I need to do to get to that grocery store, you have to learn it, you have to figure it out, you have to kind of reverse engineer and then you write a list of instructions for a person to go do it now with with, you know, GPT, and a lot of these, you know, Gen AI ai models are LLM that are focused on collecting all this data and then making a statistical probabilistic estimate as to what the steps should be. To get from point A to point B. It’s just war, wasted energy and more wasted time. Rather than just reading a damn map and saying, let me just go from point A to point B. And why risk doing the wrong thing. And when I bring this up it you know, a lot of people are caught on a you’re just a hater. No, not I’m just I would love to use it. The problem with it is, let’s say we have an API right? Let’s see, we have a banking API. And if I’m going to use ChatGPT, and its plugins to transfer money, which has a nonzero chance of sending it to the wrong bank account, why the hell would I do that? Right? Why not just read the documentation, know where I want to go and then write the instructions down. It takes less time, less effort, as society we’re just so incredibly lazy, that we want to be able to just say, hey, I’ll read my mind. We’re incredibly lazy. I mean, the other thing, it’s the same thing applies when I talk about like, oh, Tony, can you make auto suggestions for me to respond on a? On a contract provision? Absolutely. I can make recommendations, it will never match your style perfectly. It will never match your intent. It cannot know your intent. And if you’re going to write the damn intent down, then write the damn contract language down that did this. That’s the most frustrating part is saying like, yeah, I want this. Just write it down. Just write it down. Right? It’s it’s better than asking somebody else that’s me, like, like me saying, like, Hey, Ashley, can you write this stuff? No, just do it myself. Anyways, that’s Rant over. Just so frustrated with Tony,

Greg Lambert 35:37
I’m not sure I’m understanding your passion?

Ashley Carlisle 35:43
Well, I think Tony brings up a good point, which we see time and time again, that I don’t know if vendors really talk about they should. Obviously, Tony and I are former lawyers, we all know, as former lawyers are working with lawyers that a lot of times lawyers do things to do things, you know, they don’t think about working smarter, there’s just like a way to do it. And unfortunately, when that comes to try to figure out what to automate. That’s the mindset they bring into that conversation. And some vendors are smart enough to check them and be like, Okay, we need to like completely reset your mindset to make this productive. And then others just want to close the deal. And then people are unhappy with the result, and you have to fix it afterwards. And that’s something that we have all the time people come to us and they’re like, Well, why wasn’t this implemented correctly? And we’re like, well, hold up, that’s another vendor. And then also, like, did you have these conversations? Like, were you just thinking like, we have to do this? Because we’ve now committed and so we’re going to automate everything? Did you actually like talk amongst your team gathered data, like, it’s a very interesting thing, which I get where it comes from. But I think a lot of people need to be more candid in the process, and more honest about it,

Tony Thai 36:43
they’re not going to be absolutely because like, there’s it like a perverse incentive to oversell and under deliver. That’s why every time we show up to these meetings, we’re the third or fourth vendor that these folks have gone through, and they’ve been like, beaten down. It is usin, this last week, I said, I said to people, like, we’re never the first boyfriend, right? Like we’re always the third or fourth relationship. And there’s all this baggage and hurt and pain. And we have to sit there and talk and I, you know, we’re more therapists than we are like tech solutions providers, we have to be realistic and cognizant that a lot of folks will just lie, right? And it’s hard to say, you know, no, we don’t want your business because that’s not possible within the budget that you want. Right. And that’s a tough decision to make. And that’s one of the benefits of not being venture backed. And the fact that we’ve built this business from scratch and built it into a profitable business, is that we are not reliant on the numbers on how many clients we have, it’s how much money and profitable partnerships can we build, as opposed to just pure numbers play of trying to get every client and logo that they can. I like talking to you, Marlene and Greg about this, because like, you guys have a realistic view, being on the inside and seeing how bad it is when like these vendors come out. And all they want to do is, is sell you the moon. By the time you get there a year and a half later, and political capital spent aside, you’re left with nothing, and just complaints and turn the

Ashley Carlisle 38:16
tables, I’d be curious to know your thoughts on that lawyer laziness, or just dealing with vendors, I’m trying to figure out whose candidate not

Greg Lambert 38:23
I think, you know, the proof is in the pudding, so to speak, and that when you get a product, you actually get a product in front of you that you get to test. Wow, you really see see it warts and all on there. And you know, things like taking, you know, two to five minutes to answer one question, and then not being able to answer it to ask a follow up, that really affects the view, you know, kind of burst the bubble. So I you know, I think one of the things that we’re seeing is a lot of over promising and under delivering, and that it won’t cause another AI winter because I think, you know, this is this is one the first time that this is really real from the consumer side side of things. But, you know, if if you keep over promising and under delivering that has a long term effect for for the next next boyfriend that comes along.

Marlene Gebauer 39:24
And I’ll say that it’s kind of been a mix. In my experience, it’s like, you know, I’ve seen that there’s a lot of interest from from attorneys and asking a lot of good questions being very thoughtful. But as we talked about earlier, when it kind of comes down to okay, we need to decide on what do we want it to do? And you have again, you have a lot of different people with a lot of different opinions. And you know, that’s you know, that’s part of the decision making process but it but it is kind of messy, and in terms of vendors you know, again a mix I’ve I’ve seen some vendors that are, you know, very candid about things. And then I’ve seen some others where, you know, you do get the sense that, you know, you’re not getting a clear answer. And that is indeed frustrating.

Tony Thai 40:16
It must be hard for both of you, because you’re fielding these questions. And then there’s, you know, other information online, that, you know, this goes back to the whole, like, self diagnosing thing when you go to your doctor, like, they’re saying, like, oh, well, you know, you can do this, like, why don’t have access to it? And then you have to sit there and say, like, well, one, our clients will let us just feed it a bunch of their data. That’s one right to, it’s not perfect, right. And I know that in a perfect world, you’d have something that can automate the tedious parts of your job, but some of it is required. I don’t know if we’ve already talked about it on this podcast. But in conversations, I’ve had this discussion with other engineers before I said, like, I think our approach at adopting AI is very biased, because we’ve done the work before, right? Everybody here knows, the thought process that needs to go into determining whether or not a contract provision is valid, or whether or not a specific case citation makes sense. The younger generations don’t have those neural pathways. So where do they start? And how do they determine whether or not an output is good or bad? They still need to do the work to build those muscles. And those neural pathways, and that’s what worries me the most is like, yes, they’re, you know, don’t throw the baby out with the bathwater, right? Like, yes, there is a lot of tedious stuff that we’ve had to do. But some of it is part of the training, like some of it is like I read the case, right? I didn’t wait for the summary, I read the entire thing. And then I figured out how to interpret it. And that’s what worries me about folks that try to oversell the AI is I think that it’s a disservice to a good chunk of the younger generations, because they don’t know how to how to do the actual work before they automate it.

Marlene Gebauer 42:06
It’s like, it’s like I tell my kids, like, you know, it’s okay to be bored sometimes, because, you know, that basically leads to, to creativity and coming up with good solutions and learning stuff. So we’re talking about, like, all the things, you know, we’re being real critical on on Gen AI, but, you know, let’s, let’s turn it around and be like, you know, what does it actually useful? You know, I think we, Tony, you might have hinted sort of at an assistant type of role, you know, is that one, or is there some other ones that you or and Ashley can think of,

Tony Thai 42:38
for me, I think of it in the stuff that we build, right our team builds is like second brain activity, which is, you know, stuff that I suck at remembering, or that I hate having to do that feels tedious, for example, like, searching through my email, or having to set up reminders to send additional emails, I’m sure everyone here has used Gmail and went reminds me, Hey, you didn’t respond to this email three days ago, like, those are the types of AI features that I find super helpful. And then on the more on the contract drafting side, it never gets it right. But it gives me inspiration in terms of like how I should approach a provision, or it gives me a list of things to think about, sort of like a checklist, so that those are good use cases, for me in terms of the Gen AI ai applications in the legal industry.

Ashley Carlisle 43:29
To add on to that because I get to be privy to a lot of these conversations that you have with our clients or potential clients. You know, obviously, the CaseText example, I explained how like research, data extraction, it can, it can do those things, brainstorming admin tasks. I was on a panel on Monday, where we went through and people were saying, um, how they’re using it kind of in a law firm in business contexts, or in brainstorming, not really illegal work product. And I think that trend will continue. Tony, I don’t know if you would mind getting into this. But one thing that I’ve found helpful, Tony explaining to our team and potential clients is, it’s not the gin AI is bad or not useful. But you have to know how the model is set up just like on a second grade level so that you can understand where it is limited, especially in regards to making legal work product. It’s good and research because it’s a statistical model, which Tony can go into. But in regards to creating consistent documentation, which as you know, lawyers try to act like we invent a new cool thing every day. Really, it is like quality control and consistency is what keeps us at the top of our job. That is not what it’s built for. So I think that is something that is a common misconception we have to work through with people is if you want new ideas, or you know not really creative, but as creative as Genie AI can be ideas. Sure. But if you’re looking for quality control, this is not that because it is I don’t know Tony’s statistical model. I don’t know how you would sum it up. But I think people are very surprised by that.

Tony Thai 44:58
No, I think you’re perfectly absolutely Uh, you know what I would say is Marlena, you know, bring it back to your kids not to single them out. But I’m sure that there are situations where you ask them to do something multiple times, right? Like, hey, you know,

Marlene Gebauer 45:11
I live in my house, I’m sorry.

Tony Thai 45:15
Or you’re like, hey, take out the trash, and they took out the trash, or there’s a bag of trash sitting next to the trash can and that they didn’t take that out, either. You’re like, no, really,

Marlene Gebauer 45:23
really does live in my house.

Tony Thai 45:25
Exactly. Right. I’ve, I’ve, I’ve got siblings. And then you go out to the trash. And you’re like, they took out the trash and put it next to the trash can not inside the trash can. And at some point you’re like, would have been faster had I just done it myself. And so the goal of talking to your kids about it is like, the goal is eventually they will do it themselves. Right, like, and I think that’s ultimately a good goal for Gen VI is, if you give it enough data, then it’ll figure out how to do it.

Marlene Gebauer 45:53
Five years from now, it’ll get it right. Exactly, exactly.

Tony Thai 45:57
You know, at some point, you’re just like, I’ll just take out the trash myself, right? Like, it

Marlene Gebauer 46:01
doesn’t benefit me anymore. They’ll figure it out, then, you know,

Tony Thai 46:05
exactly. So there is that cost analysis that gets done a lot. And when we talk to clients, or many clients, we’re like, we don’t, you know, we’re glad that you don’t focus your sales pitch on AI because like, we won’t take a product that that sells us AI because we’ve gone through all the iterations. And we’re sick of it, we know exactly what we want, we want you to build it. And then we want you to tell us what it can or can’t do in the future. But we don’t care about the training of how to take out the trash like we needed to take out the trash today.

Greg Lambert 46:36
Ashley, before we get to our crystal ball question I want you to put back on your cmo hat because I’ve talked with a number of CMOs and law firms that really love some of the some of the summarization some of the idea generation, has there been any specific examples where you’ve used it with your cmo head on?

Ashley Carlisle 46:57
Yeah, I think it’s very interesting. And I think in legal marketing circles, people are trying to figure out new use cases all the time, obviously, for content for brainstorming, it’s good. I think Google has gone back and forth on how it SEO ratings automated versus on automated content and how that flags. So I think a lot of people are staying tuned to make sure they’re up to date on that. Because the last thing you want to do is, you know, get dependent on ChatGPT to create a lot of content, and then it not actually show up in the rankings. For now. It’s fine. But I think that hasn’t been a big discussion point. But I will say it’s not as good as if a lawyer was writing it or a legal marketing professional was writing it. So it really depends on kind of what your end goal is. But when you have a large volume of stuff to do, it is helpful, you know, so I think we do use it for brainstorming and for content. But like Tony said, I’ve had to really work with the prompts to make sure that I’m giving it enough info, because if you do not give it enough info, it is the most vanilla diluted, airhead content you could find on the internet. It is it is

Marlene Gebauer 47:57
general, it’s it’s very generic, you know, so it’s a good base. But I often wonder if sometimes having that base is a bit of a crutch. And then it’s like, okay, I don’t actually get as creative as I would have been if I had done it myself. All right, so crystal ball question now. So you guys know the drill, we ask our guests to look, you know, two to five years into the future. And, you know, tell us what you think you see in terms of generative AI.

Greg Lambert 48:28
But But since we may have you back on in six months, let’s just sell a short note, shorten that for you a little bit has to do you

Tony Thai 48:34
wanna take a first step? Sure,

Ashley Carlisle 48:37
I’ll just go kind of, obviously, its predictions. So it’s fun. We don’t know exactly, I guess we’ll find out. But I think for our team, kind of what we’re predicting, which I kind of alluded to was the hype cycle dies down. And people are just focusing on tech literacy and knowing what AI is. And so I guess our hope is that that just sustains. Also, I think, like you said, we’re going to be in another chapter of the buy build partner phenomenon with legal tech, which is kind of plagued our industry for a long time. But this time with many models or coupling kind of these large LLM with specialized legal products with AI. It’s going to be a whole new phase for our industry. And I’m just curious to see how that pans out. Because as you guys probably very much know, a lot of lawyers think like, oh, we can build that we can hire someone to build that. And then they get bored. And then they come to us. And then Tony in our team has to clean up their mess. So I think unfortunately, for better for worse. We might have that again, just with different AI problems.

Greg Lambert 49:33
Tony, which What’s your prediction?

Tony Thai 49:36
That I’m Yeah, I hate to. I hate to jump on that bandwagon, but I think that’s exactly it. Also, you

Ashley Carlisle 49:42
hate to say them, right? So

Greg Lambert 49:45
I don’t know. He said you’re right. A number of times.

Ashley Carlisle 49:48
What about it? I know what I have this on record.

Greg Lambert 49:51
We’ll just we’ll send you a link but yeah, all the compliments will just put it on a loop and send it to you.

Ashley Carlisle 49:58
Greg and Marlene, Do people ever ask you What’d you think? I mean, obviously, I know you have a lot of these conversations and they very different parts of, you know, the industry. But in regards to AI, what do you think we debate

Marlene Gebauer 50:08
stuff all the time, you know, just just internally and, you know, the attorneys come and they, they, you know, they want to know, and they want, they want feedback from, you know, the innovation group is to Okay, well, what should we be doing? What do you recommend? So, yeah, there’s a lot of, you know, there’s there’s a lot of that type of conversation going on, I think.

Greg Lambert 50:28
Yeah. And I would say, probably one of the things that I’m, I’m hoping will happen over the next six months to a year, is that even if this isn’t the answer to their question, that because of the hype going on around here, they’re going to ask questions, they, you know, they’re going to come to you and say, Hey, I need I need, you know, ChatGPT, because that’s what everybody calls it. I need ChatGPT to, you know, to do X, Y, and Z, because, you know, I’m having, I’m having trouble, I need to, I need to make that more efficient. And then as we get into discussing it, then we realize, well, really what you’re doing, you have a data issue, or you have a process issue, and it gives us a chance then to eat, you know, if we’re not the right people, at least direct them. But you know, one of the benefits that I’ve loved so far is that it’s gotten the conversation going, when it may not have gotten gotten going before.

Marlene Gebauer 51:27
I’m sorry, I had another question. Oh, go ahead. Tony, you were gonna answer? Sorry,

Tony Thai 51:31
I want to hear that question. But I have like a side prediction that will probably put egg on my face and on both sides. But I’ll say it anyways. All right.

Greg Lambert 51:41
We will, we will bookmark this in the transcript.

Tony Thai 51:44
I predict that with Gen AI and the kind of magnifying glass that goes on legal tech and just efficiency in general, that some major firms are going to move towards a flat rate model in the next 12 to 18 months, which I know we’ve been talking about for over three decades now. But I actually think that that is going to start to have to be a thing. Because the client, the client awareness is just there. Right? That’s, that’s the part of the conversation that that this firms have to stay up to today on? Yeah, that

Marlene Gebauer 52:22
wasn’t my question, although we have talked about that as well, internally that like this might be the thing that pushes the pushes us over the edge. My question was about prompt engineering. And, you know, do you see any opportunities, you know, for this type of role in firms or in other places? And, you know, what types of folks do you think would be, you know, would be sort of moving into those roles?

Tony Thai 52:53
I think it’s a it’s a skill set that gets added to existing legal professionals. I can’t imagine having the need to have a have like a full time person on it. Yeah, I can’t, I can’t do that pathway. But I am seeing a lot of that pop up with even our clients that they have, like open job postings for prompt engineers, to help them with it. But I think the major question is like, what’s the long term use case for them? You know, once they’ve engineered these prompts that way,

Greg Lambert 53:24
like being a webmaster in 1994?

Tony Thai 53:28
I love that reference. Heck, yeah.

Greg Lambert 53:53
Well, Tony Thai and Ashley Carlisle from HyperDraft, we want to thank you for coming back on. You want to do this again in January.

Tony Thai 54:02
Always do it. All right.

Marlene Gebauer 54:05
And thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 54:19
and I can still be reached on Twitter at glamour, but I’m finding myself more and more young threads on threads. So I’m glad you join Lambert pod so the same glam group but with pod on the end. Somebody beats you to glamour. Yeah. Well, it’s because it was Instagram. So someone on Instagram beat me to it. And so you gotta take your handle from Instagram. Yep. So well, Tony and Ashley if anyone wants to learn more about HyperDraft or continue the conversation with you, Tony, where can you be reached?

Ashley Carlisle 54:51
So they can join our waitlist on our website, www dot a word.

Greg Lambert 54:56
I like to I wish this is time I wish it was video. I wish I would have been so so okay, let me let me rephrase that. Ashley, where can where can people find out more?

Ashley Carlisle 55:07
So if people want to learn more, talk more about AI or document or workflow automation or really anything legal Tech, we get a lot of random questions happy to answer. They can either join our waitlist on our website www.hyperdraft.ai. They can follow us along on LinkedIn where HyperDraft in there and on all other social media platforms, or they can just email us I’m ashley@hyperdraft.ai and Tony, similarly is tony@hyperdraft.ai.

Marlene Gebauer 55:34
And listeners, you can also leave us a voicemail on our decon review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca. Thank you.

Greg Lambert 55:44
Thanks, Jerry. And Jerry has a new album out.

Marlene Gebauer 55:47
Oh, terrific. Well, we’ll have to talk about that next.

Greg Lambert 55:49
All right. All right. Talk to you guys later.

Greg Lambert 0:07
Welcome to The Geek in Review. Podcast focused on innovative and creative ideas in the legal industry of Greg Lambert. Marlene Gebauer is out this week. So once again, you are stuck with me as the solo host. But I think you’ll enjoy our guest that we have on the show this week. So today we are honored and delighted to have Kriti Sharma, Chief Product Officer of Legal Tech at Thomson Reuters, where she leads the development and delivery of innovative and impactful products that leverage data analytics and AI. She is also the founder of AI for Good, an organization that creates intelligent, ethical and scalable technologies for social good. Kriti is a leading global voice on AI ethics and its impact on society and has advised governments, businesses and international organizations on how to use AI responsibly and inclusively. And she has given a TED talk on how to keep human bias out of AI and has been named in the Forbes 30 under 30 for advancement of AI. So wow, that’s a that’s a lot. Kriti, welcome to The Geek in Review.

Kriti Sharma 1:18
Hey, Greg, thank you so much for having me. And as a self identified geek, I feel like I found my people.

Greg Lambert 1:23
All right. I think we have a guest host in the making here hopefully. So, Kriti, we’ve worked with a longtime friend of ours, and actually our very first person on The Geek in Review, Zena Applebaum. And we asked her to come back onto the show. And as much as she wanted to, she said, you know, really, we should go and talk with someone from what she referred to as the trifecta of TR’s, a legal team. And so that was that was you as the chief product officer, Mike Dahn, who a lot of people from the show should recognize who’s the head of Westlaw Product Management. Or we should talk to Emily Colbert, who is Head of Product Management for Practical Law. And so as we were talking, I mentioned to Zena, that I love the fact that two of these three points in this trifecta are women. And I’m curious as to your thoughts on that. And actually, as Zena said, I should I should prod you on this as well. How do you think this type of Diversity in the leadership there Thomson, Reuters legal products affects, you know, how the organization is run? And how products are designed and maintained and enhanced?

Kriti Sharma 2:39
Yeah, so Greg, first of all, we really have an awesome trifecta that Mike, Emily and myself, and then Zena, and many other colleagues very closely working with us. I think what’s really special about Thomson Reuters is the inclusivity. And not only our Diversity of backgrounds, but also our experiences. We come from many different backgrounds. I, myself, have a computer science background, I started building robots when I was very young, as a kid. We talk about those robots another time. Now, I’m just really passionate about solving real world challenges that bring access to justice for all bring a more of a technology driven approach and AI background to this field. Mike and Emily have such incredible depth in understanding that product spaces, the world of legal the practice of law. And it’s just absolute incredible to see those different experiences and fused together. And the reason why it’s so important, Greg, not just the Diversity at a leadership level, but across the ranks across the organization. Because as we go into the world of AI, machines learn like humans do, and they’re designed, and they bring in experiences and knowledge from the creators from the underlying data sets. And if we are to build ethical, sustainable AI technology that we can trust, we got to have a diverse group of people building it, it has never been more important. And we’re very well aware of the challenges of biases in AI, the issues of sexism, or racism and all these challenges that we hear about and observe and experience in our daily lives. Now, when it comes to the next wave of AI technologies, we have to do this right. And one of the key questions here is who creates this technology? And is this group diverse enough in their backgrounds and their knowledge and their experiences?

Greg Lambert 4:33
Yeah, like I said, I was super impressed with the makeup of what Zena called the trifecta. Thanks for talking about that. Now, I want to talk a little bit I know there is just so much going on with legal technology and AI. And you know, we’ve been talking since November of last year, you know, on how it can enhance content. And I know that at Thomson Reuters has been working a lot on this as well, not just since November, but for many years. Now, I know there’s a lot going on with, with the regulatory actions coming up with the potential acquisition of CaseText. That was announced a few weeks ago. So we won’t go into any specific questions about that. Because I know you’ve probably been told not to say certain things about that, so we won’t. But with those disclaimers out of the way, I did want to ask, What is your perspective on TR’s research and content advancing is all of these generative AI and whatever’s next is coming along? What’s the kind of the path that you think TR wants to take?

Kriti Sharma 5:42
See, Greg at TR, we are very much at the forefront of applying AI into the most important tasks and jobs that our customers are doing. And our approach is very much of an augmentation of human work, we take a co pilot approach rather than an autopilot. It’s the companion that expands human skills and does the work for them and humans are always in control. And the reason why we went down this path is we deeply studied our customer needs, our advantage where TR can bring real value to our customers. And we strongly believe that in our ability to ground or train AI in trusted content, we can produce trusted answers. And say the biggest investment that TR is making, you’ve heard about our 100 million dollars per year commitment in the advancement of artificial intelligence, the biggest part of that 100 million dollars, is going into building trusted and responsible AI. And we don’t think it’s ever been more important to do this and to build this technology in the right way than today. At this point in time, because AI is going to change in profound ways how we experience services, how we live, and work in in our communities. And we want to make sure we do it the right way. Just expanding more on what that means in action is we are turning and bringing our content and research knowledge which is trusted, it’s well recognized, viewed spent to your point decades, enhancing and perfecting our approach there. We’re using that trusted data to train the AI models so they can give more clear answers. We can provide citations and links back to those trusted sources like Westlaw and Practical Law. And most importantly, drive adoption and make it easy for people to use this technology by bringing it right where they work. So an example of this is our partnership that we announced with Microsoft Co-Pilot for Word where, right where you draft, we can help you get to a first draft within minutes by bringing in the most trusted answers and content using the power of Westlaw practical law and our document intelligence service. And that loops back to what makes AI really work at scale and production and not just in a POC and pilot land is building it in a trusted and responsible way. And that’s what they’re all in for.

Greg Lambert 8:26
Yeah, I know that, you know, the trust issue is huge. In fact, I was talking with a couple of friends of mine yesterday about… first we started talking about the Goldman Sachs report that suggested that, you know, up to 44% of lawyers work could be replaced by AI tools. And and you know, and we were going back and forth. And I mentioned I said look, you know, we that trust that you talked about, I think people are suddenly feeling the lack of trust with the tools that are out there on the on the commercial market. And you know, this has been driving most of our general counsel’s and our security teams to just talk about the risk side of using generative AI for legal practice. And not only that, but then the uncertainty of whether data that is put into the AI tools by attorneys is really secure and maintains the confidentiality of the client and of the law firm. As we were talking, I was feeling like one there’s still a huge gap of legal professionals that have even used generative AI tools so far. And then there’s also the kind of this push back so are you seeing any type of you know, kind of pushback let’s let’s slow this down? And maybe just be temporary but but are you seeing kind of a cooling in the market when it comes to using AI tools in in legal practice?

Kriti Sharma 10:00
So, Greg, what we are seeing is instead of cooling down around here IV are seeing more for speed up towards thoughtfully created AI. And that’s, that’s an important nuance if you allow me to expand a bit, yeah, we went through the space in Cuba and and everybody was excited and to spy, more commercially, consumer facing commercially available tools like ChatGPT, being one of those prime examples of a foundation model that made the interface so easy that anyone could start using it and experiencing the power of these models. But when he got past that and realize, okay, there are all these challenges with this technology, and you can’t use it for every every use case. And so that was the reality hitting moment radicalist. Chatting was so Q1. Q2, and Q3 is all about now let’s make it real and make it work for us. And that’s where we see much stronger drive and enthusiasm and interest from our customers. And our partners really want to benefit from this technology, and they want to adopt it, but they want to do it safely, responsibly and thoughtfully. And that’s the big reference. And it’s not new. I’ve worked in AI for a very long time and many different industries. I’m newer to legal, I came to Thomson Reuters in the legal world, just over 18 months ago. But I’ve spent a very long time in my career working across various applications of in finance and in small businesses and health tech and social justice. And it is always the crux comes down to can humans trust this technology? Is it designed in a way where it doesn’t black box information, it starts to white box it. It’s gotta give you good show It’s working, it’s gonna show you where it learns information from can you go back to a trusted source to explore more? What’s the impact on humans can get started easily and get going. And it’s a real benefit to be generated by applications of this technology as being true to its capabilities and doesn’t know its limitations. And that responsibility of clarifying these questions, it very much leans on the developers or the creators of that technology such as ourselves, and also our customers and partners who really joining us in driving that change process across the board.

Greg Lambert 12:23
Yeah, I remember going to Mike Dahn’s presentation, I think it was for WestlawNext. And he was talking about how they were implementing what would have been AI at the time. And that was that the system was supposed to learn as people were using it, you know, and how to react to certain things, what should drift up to the top? And I remember people being worried about that probably, you know, eight years ago, though, and I know that’s just exponentially shot up since then. But I think, you know, a lot of us in the industry are really looking toward the big players like Thomson Reuters to bring us something to us that we do feel confident in.

Kriti Sharma 13:09
Greg, I would just add that we take that responsibility extremely seriously. We know that given our scale and the quality of the content, there is a responsibility for us to bring a solution to a market that sets the gold standard sets the tone of building and deploying quality AI solutions. And I’ve been just so impressed and inspired by the the partnership from our customers by teams were really at the forefront of building this technology in a trusted responsible way.

Greg Lambert 13:45
So you were on a terrific panel in June, sponsored by tech.eu. On Charting a Way forward for ethical AI: balancing innovation and responsibility. And in one topic you mentioned was that there is a significant lack of regulation around AI and that you were really disappointed by that. So are there any specific regulations that you would like to see the EU, the US or others implement? That would help kind of set the Trust for the technologies going forward?

Kriti Sharma 14:22
Yes, a great question, Greg. And one that I’m very passionate about.

Greg Lambert 14:27
I could tell from your panel that you are passionate about

Kriti Sharma 14:32
I think at one point I did say I’m deeply, deeply, deeply disappointed.

Greg Lambert 14:37
Yeah, I left out a couple of the deeply’s on that.

Kriti Sharma 14:41
Yeah, I did have to turn down to another deep please. But I I just think Greg if the future of this how this technology is deployed, the scale at which should not be left to individuals and geeks like me to figure out. At Thomson Reuters for example, with be obviously establish our AI policy and our standards and our governance and our principles and how we are going to create trusted, responsible ethical AI. But the true adoption of this technology for benefit of humanity and society and business is going to come from specific rules and guardrails that have to be put in place across all producers and creators of this technology, not just the ones who opt in to do this, right. And this is where the regulatory frameworks are very, very important. If we have to give people confidence that they can feel safe in using this technology for more positive outcomes. What we are seeing in the regulatory space, and I’ve spent a lot of time going through the EU AI act, for example, is that the regulators are taking at a somewhat divergent opinions or approaches to this technology. For example, in the EU, the draft EU AI act is taking a risk based approach to there are some applications that are outright banned. Others that are considered high risks, some are medium, and Others are low risk. And that comes as a framework, there are applications of AI that we shouldn’t be deploying in society, because it is unacceptable, for example, large scale surveillance and so on. And then there are others. For example, in the legal and information world where we can use trusted data, and ground AI in facts, we can really create access to justice and much bigger scale. And so the thoughtful approach around creating a tiered and risk based model is a great step forward, the UK has taken a different approach to the EU in that they are going to have historically taken more of a lightweight light touch approach to regulation, relying on existing regulators to figure it out. And the US is in somewhat earlier stages. What we really need, at a very fast pace, is global coordination around regulation for AI, then we bill in an increasingly global and digital world, when we build tools and solutions for people around the world, we need that coordination. So we can drive adoption in the right way and in this in a safe way. And that is my big call and ask for help and call for action for policymakers, business leaders to really lean in and get our point of view and our needs across through coordination between industry and policymakers and civil society.

Greg Lambert 17:27
Yeah, I think you’re right, it has to be a global approach. I don’t know how practical it’s going to be, given the current political climate, I think this is going to be a almost an AI arms race. Do you see that for countries that that are going to look for ways to give themselves an advantage and just disadvantage other countries?

Kriti Sharma 17:48
I think, Greg, from what I have seen for a very long time in the AI space is that this is a very polarized tech community. There’s a group of people, thinkers of our times, who believe the existential risk is far greater the longer term risks and yeah, they would put put out, us to slow down or pause, you know, and then there’s the other community of people and builders who are more concerned about making sure that we use this technology for the right applications, and we build it the right way. It’s more near term needs around eliminating bias from systems, creating more accurate AI that can help you be more effective in your work. And that world has been in, or that space has been across this spectrum for a while. And what I’m hoping we’ll get to, and we are getting a lot closer to it is meeting in the middle of. Let’s make sure the technology we’re building today is trusted, responsible, so we can briefly get the benefits out of it. And let’s start to work through regulatory frameworks that help us in the long term, and identify applications where we shouldn’t use this technology and we should use it more in others. This is a technology like any other technology, it’s more about what humans do with it.

Greg Lambert 19:07
I think I’m gonna use that quote at some point. You know, we’ve been talking about the development of in the tools and the things that are available, and the trust issue. But you know, this is going to be a huge shift and how, I’ll just limit it to the legal profession, but you know, how legal professionals do their day in day out job, you know, what kind of skills development and talent do you think we’re going to need going forward as the nature of work changes?

Kriti Sharma 19:40
Yeah, Greg, this is such an exciting time for people to come and build products. And I do believe it firmly. I went down this traditional part of in my career of building technology since I was a kid and then go into engineering school and then computer science and that part to really get to grips around how to build AI systems at scale and build large scale technology platforms, that game has changed. Now we’re moving past that teaching everyone to code movement. Yeah, that that kind of obsession, we had a three year old must know how to write code. We’ve moved past that, to this new world where AI can do that to a lot of that work of processing information or writing code, even Yeah, which is the most expensive form of knowledge where that’d be created. We’re now moving to this world where human role is to ask the right questions to get the right response back from the machines. It’s about curiosity, its emotional intelligence and empathy. Those human skills have never been more relevant and important than it is before. So And yet, there’s a very famous quote going around in the tech circles right now that “the newest programming languages is the human language,” so it’s such I mean, only speak two human languages, Greg, I speak 17 computer languages. So I’m gonna go retrain. There, this is such an exciting time, that barrier of getting into the world of technology or building a career that or creating new solutions and tools and products has gone down significantly. And I can’t wait to see a much broader group of people with many different experiences coming into this field. A few years ago, as I was building an AI product at the time, which is similar, Large Language Models, conversational system, we hired the first AI personality trainer slash conversation designer. And at the time, the that role was about help the machines learn how to talk to humans. So it’s a tone of voice, how to interpret their needs, and how to be transparent. And for example, declare that you’re talking to a machine and not a human and what the limitations are. And that was the most important and most impactful role that we had in the team. And that person had no background in tech, but they were excellent at language and communication. So we would love to see that. Yeah, I think we’re about to see that come in spades. And I, the technologist of the future, is gonna look different. They’re gonna come from many different backgrounds. And I think going back to your Diversity point, and your first question, this is how we’re going to solve it, I really, I really hope that this is this is how we solve some of the Diversity gaps or challenges we see.

Greg Lambert 22:28
Yeah, I wouldn’t have thought about that about having to train the personality of the AI, especially in interactive AI that you have, because I’m a solid Gen X person. And I’d fit that stereotype of, if you have to do more than two emails, pick up the phone, because you’re obviously talking past each other, not to each other. So I imagine that, you know, it’s the same way with the machine. And we’ll talk a little bit about this later. But I was watching a video on one of the products that you helped develop rAInbow, which clearly states your you are talking to a machine. And but it still allows people to do that interaction. And I think having that knowledge upfront, helps set the tone and confidence of the of the conversation going forward. So like you said, emotional intelligence, human interaction are going to be more important now than probably ever.

Kriti Sharma 23:24
Yeah, absolutely. And, Greg, the example you mentioned, of rAInbow, we launched this tool in prior to my Thomson Reuters life, to help people in vulnerable conditions get access to help that they needs. And in this case, rAInbow, specifically targets people who are at risk of violence, domestic violence, or gender based violence. And it is often a very difficult circumstance, taking the first step feels hard. There’s judgement, shame, stigma, issues, or challenges around getting access to services that often only work Monday through Friday 9am to 5pm. And will be found is by building a tool that’s not there to solve the problem, but provide information to help make taking that first step a bit easier, was the purpose it was designed to be it is designed to be transparent and clear that they’re talking to a machine what its limitations are. And when we did that, and it was clear about the human and machine roles and hierarchy of that relationship. We were able to impact and positively and augment some of those, those needs in a very difficult situation.

Greg Lambert 24:37
Well, since we brought up rAInbow, let me just jump into the conversation about AI for Good so you, you’re not new at all to the AI discussion. So you were inspired by your experience with the Obama Foundation Summit all the way back in 2017. To start an organization called AI for Good what happened at that summit that drove your desire to start AI for Good. And you know, what was the problem that you wanted the organization to solve?

Kriti Sharma 25:10
Yeah, it’s a great, it started from this realization that we are, we have such a profound and powerful technology in our lives. And that is available to us. And I just did not want to look back in time two years now and think the best thing we did with this tech is make people click more ads, or get them hooked to their phones. And it was just this at the same time, we’re seeing huge needs from frontline organizations we’re solving. So the hardest problems that our society and communities face, we didn’t have all the resources to scale. And so it originated and grew into a movement. And now we’ve evolved as even to the next stage of AI for Ggood as a movement and not just an organization. So whether you’re a business, you are a nonprofit, you are an individual, you can start to build applications of AI in a safe, responsible way that addressed some of those bigger challenges, or at least augment them. And, yeah, I have to admit, candidly, it is the most rewarding and the most terrifying thing I’ve ever done in my life, to take it to work on challenges of mental health and loneliness and violence into in a topic that I don’t know very much about, but using it as an opportunity to work with people who are so experienced in that world, in that field who have so much to offer, and just could benefit from technology supporting those needs. And I would say it’s been very humbling.

Greg Lambert 26:55
Now, can you tell the audience a little bit more about what AI for Good does, and kind of what you see it doing over the next few years?

Kriti Sharma 27:03
Yeah, it’s an AI for Good builds, products, solutions and experiences for some of the biggest social challenges that face our world, our communities. So we we’ve done a lot of work on supporting frontline organizations who address and provide services to people at risk of violence. We are deeply passionate about extending that to enabling access to justice, which very much aligns with my work at Thomson Reuters in the legal world, we do a lot of work on enabling and democratizing access to trusted information in in the mental health space for young people. Our focus has been historically been primarily in developing countries in South Africa and India. It will be seen over the last few years in particular, Greg is this shift to a movement in that there AI technologists, researchers, civil society members, business people, it’s such a strong movement that our people are very curious to learn how they can contribute or how they can use AI in a meaningful way. And our hope and dream is we’ll stop the conversation about AI over time, and we will all be talking about positive impact that we can create and solid sustainable development goals related problems that we can solve. And then ideal success will be it’ll be more about good and less about AI.

Greg Lambert 28:32
So, you know, there’s a lot of discussion on the potential for the new AI technologies to help underserved populations gain access to their legal services or health services. And I know you’re doing great work with AI for Good, but how do you seek big bigger companies like Thomson Reuters and others, taking this potential of access to justice, access to health care, and turning that into actual services that deliver to the community rather than just solely focusing in on how do I get more people to go go use Westlaw?

Kriti Sharma 29:10
Yeah, I think he definitely will. We’ll see more people using Westlaw because it becomes a lot easier to use it to do your work. I think the potential of AI is is across multiple domains, right? It’s about in your professional work, getting your work done faster, and more effectively and more efficiently and with greater confidence. That’s the work you’re seeing Thomson Reuters produce in the legal professional space with faster access to research and answers to questions and expertise in drafting. Then there is the opportunity of changing how and improving how our court systems work, for example, by giving them access to the same trusted information by making them more effective, giving them digital tools and that we do a lot of that work to our core products like Case Center, and then increasingly A big responsibility incumbent upon all of us is to then make this information more easily available to a broader community of, of people. And, for example, in the US, Thomson Reuters is the service called Findlaw, where we have over 100 million visitors who come to our site to understand their legal information rights and connect to a trusted attorney. And we will see more and more of that support grow towards large communities of people around the world. What I think is a real opportunity with AI, is how we can solve the unsolved or the underserved problems. Legal is definitely one of those worlds where there is such a big opportunity to do do more. And they’re passionate people who want to solve this problem that there is an underserved community, I really, really hope that with these advancements in AI, with the productivity improvements with the augmentation of human skills, we can bring quality legal services to more people

Greg Lambert 31:06
Tying into that, I know the question was around the potential for access to justice, access to healthcare, but part of that requires kind of a better understanding of the technology, what it can do, and experimenting with AI’s, AI tools to help governments or companies better understand the technology so that they can turn around and hopefully, kind of fill these needs. So do you think that having things like, you know, regulatory sandbox or experimental zones or experimentation zones, can be an effective approach to how we can learn what the tools can do?

Kriti Sharma 31:46
For sure. We’ve seen this successfully deployed in the FinTech world before, right when there was more collaboration between industry to strike the right balance between innovation and regulation. We also have a gap and availability of talent and skills to work with and for regulators to start. And so some of these questions and problems, we know how critical this skill set is, and how scarce it can be at times and in our world. So absolutely, I definitely think more collaboration between industry and government and policymakers. And I’ll pitch my dream again, Greg have more international alignment.

Greg Lambert 32:29
Yes

Kriti Sharma 32:29
I’m deeply, deeply, deeply disappointed, we really hope to see more of that coordination. And we all need to get out of our comfort zones and do more of it. And it’s incumbent upon it upon all of us to get this right. At Thomson Reuters, we are taking a leading position in the top leadership around this space in leading the conversation with our many different stakeholders in this space to really champion the needs of our customers to the best of our abilities.

Greg Lambert 32:57
Kriti we’re at the point of the show, where we ask everyone our crystal ball question. So I’m gonna ask you, I know you’re, you’re in a hotel room. So hopefully, you’ve packed your crystal ball with you. But if not just, you know, think, think out and look into the future for us. And let us know, what do you think are going to be some of the biggest challenges or changes in the legal industry? And I always say, over the next two to five years, maybe that’s a little too far out. But, you know, over the near term, and longer term, where do you what do you see as challenges?

Kriti Sharma 33:29
Since you said challenges and changes, I’m going to talk about three things that I deeply care about, I think it’s the most important, it’s a helpful framework to think about AI. First thing is we got to be very intentional about who creates AI. And by that, I mean, people from many different backgrounds who have a good mix of skill set who are diverse in their opinions, their experiences, and then knowledge. Yeah, I think it would be a real sad state. If it’s only geeks like me, like me, who create this technology where it needs to be a broad group of people from expertise in particularly in the legal world to understand us who are who really know that. And that should reflect in the data and the content that’s fed into these AI systems that content, it needs to be trusted. Second, is how we build it. It’s one thing to just create a POC or a quick wrapper on top of GPT-4 and check it out or, and quite another to build it thoughtfully to understand customer needs that that really need to be solved the right problems that need to be addressed. And create the transparency and trusted link backs or citations back into the information so you can use it confidently. And then the third one, perhaps the most important one, Greg is what do we use it for? Go back to that point where we build this technology to make people click more ads. Or we could use this to meaningfully in a significant way. drive productivity enhancements. give people time back or create easier access to systems or information or make it easier to enter into this profession and create more impact in a much more positive way. So my crystal ball would be that we will know the answers to the questions of who creates it? How do they build it safely? And what did it build it for?

Greg Lambert 35:20
All right, well, Kriti Sharma, Chief Product officers of Legal Tech, they’re at Thomson Reuters, thank you very much for taking the time to talk with us this morning. Appreciate it.

Kriti Sharma 35:30
Thank you so much, Greg.

Greg Lambert 35:31
And of course, thanks to everyone listening for taking the time to listen to The Geek in Review podcast. If you enjoy the show, please share it with colleague we’d love to hear from you. So you can also reach out to us on social media. Marlene can be found at @gebauerm on Twitter, I can be reached at glamour on Twitter and I’m now on threads so @glambertpod so I had to add the pod there to the end of it and not only that, but now the podcast has a thread account which is at @TheGeekInReview. Kriti if someone wanted to find out more and or can continue the conversation where’s a good place that they could find you online?

Kriti Sharma 36:10
They can reach out to me on LinkedIn and Kriti Sharma are on Twitter at @Sharma_Kriti.

Greg Lambert 36:16
So listeners you can also leave us a an old fashioned voicemail on our Deacon review Hotline at 713-487-7821 and as always, the music you hear is from Jerry David DeCicca who actually has a new album out so check that out. All right, thanks everyone.

In this episode of The Geek in Review podcast, host Marlene Gebauer and co-host Greg Lambert discuss cybersecurity challenges with guests ⁠Jordan Ellington⁠, founder of ⁠SessionGuardian⁠, ⁠Oren Leib⁠, Vice President of Growth and Partnership at SessionGuardian, and ⁠Trisha Sircar⁠, partner and chief privacy officer at Katten Muchin Rosenman LLP.

Ellington explains that the impetus for creating SessionGuardian came from working with a law firm to secure their work with eDiscovery vendors and contract attorney staffing agencies. The goal was to standardize security practices across vendors. Ellington realized the technology could provide secure access to sensitive information from anywhere. SessionGuardian uses facial recognition to verify a user’s identity remotely.

Leib discusses some alarming cybersecurity statistics, including a 7% weekly increase in global cyber attacks and the fact that law firms and insurance companies face over 1,200 attacks per week on average. Leib notes SessionGuardian’s solution addresses risks beyond eDiscovery and source code review, including data breach response, M&A due diligence, and outsourced call centers. Recently, a major North American bank told Leib that 10 of their last breach incidents were caused by unauthorized photography of sensitive data.

Sircar says law firms’ top challenges are employee issues, data retention problems, physical security risks, and insider threats. Regulations address real-world issues but can be difficult for global firms to navigate. Certifications show a firm’s commitment to security but continuous monitoring and updating of practices is key. When negotiating with vendors, Sircar recommends considering cyber liability insurance, audit rights, data breach responsibility, and limitations of liability.

Looking ahead, Sircar sees employee education as an ongoing priority, along with the ethical use of AI. Ellington expects AI will be used for increasingly sophisticated phishing and impersonation attacks, requiring better verification of individuals’ identities. Leib says attorneys must take responsibility for cyber defenses, not just rely on engineers. He announces SessionGuardian will offer free CLE courses on cybersecurity awareness and compliance.

The episode highlights how employee errors and AI threats are intensifying even as remote and hybrid work become standard. Firms should look beyond check-the-box compliance to make privacy and security central in their culture. Technology like facial recognition and continuous monitoring helps address risks, but people of all roles must develop competence and vigilance. Overall, keeping client data secure requires an integrated and ever-evolving approach across departments and service providers. Strong terms in vendor agreements and verifying partners’ practices are also key.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript

⁠

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer,

Greg Lambert 0:14
And I’m Greg Lambert and I have been playing over the past couple of days with the new meta Twitter clone called Threads and so far it’s not too bad it’s it’s pretty simple but I think that’s probably the best way out of the box there. So Marlene, I don’t think you’ve joined in yet..

Marlene Gebauer 0:37
Not yet, but soon. Yep, yep, well do. We have an episode today that isn’t about Large Language Models and Gen AI. Believe it or not, it’s still about data. But today we’re focusing on how firms can do the best job possible to protect theirs and their clients. Law firms spend a great deal of time and money on security measures. They invest a lot of certification establishment a proper protocol and training. But as we see in the headlines, even all this may not be enough. And April both TechCrunch and Bloomberg Law reported that Proskauer Rose exposed client m&a data for six months, because a vendor they used to create an Information Portal on a third party cloud based storage platform failed to properly secure it.

Greg Lambert 1:21
Yeah, and I think there was actually another Bloomberg article that came out last night that expanded that to a to a third firm. But you know, if you think this is just a risk of doing business in our you know, this high tech world that we live in a you would be wrong. There are services and technologies that can even you know further reduce the risk by monitoring and preventing such errors. And our guests today are going to speak about it. So we would like to welcome Jordan Ellington, founder of SessionGuardian, Oren Leib, Vice President of growth and partnership there at SessionGuardian. And Trisha Sircar, partner and CO Privacy Officer at Katten. We’d like to welcome you all to the show. So Jordan, Oren, and Trisha, welcome to The Geek in Review.

Jordan Ellington 2:09
Thank you.

Oren Leib 2:10
Thank you. Nice to be here.

Marlene Gebauer 2:12
Jordan, can you tell our listeners a bit about yourself and how you came to develop SessionGuardian.

Jordan Ellington 2:17
So my career started in the early 90s at Weil, Gotshal in New York, and continued for the last for the next 30 years, focusing on what I call secure document collaboration systems. This basically sharing sensitive documents on extra debts with outside parties. And that resulted in me having an ongoing focus on cybersecurity and making sure that only the right parties had access to sensitive information. Many of my colleagues for awhile, landed jobs at other parties over the years, whenever they needed to build a special matter specific application, they would give me a call. Often this would result in a new client. And this is really the story that led to the creation of SessionGuardian. About five years ago, they started working with an analogue 50 firm to develop a security system that they could use with ediscovery vendors, specifically providing contract attorney staffing agencies. And they had found that there were a variety of security postures, some staffing agencies, so worked on premise, some staffing agencies had remote users, some staffing agencies rented space in a Regis center and their computers. So there was a significant cybersecurity disparity across the agencies. So the goal of SessionGuardian was simply to normalize these security posture. So regardless of the agency that the firm was working with, they would be able to have a high security standard. So we realize the technology could be leveraged to provide secure access to sensitive information anywhere. So we pursued a plan to continue that vision and marry facial authentication as a way of validating that the physical user logging in was the intended user. So ultimately, we want to create a system that will enable firms and corporations to engage staff securely remotely with an with an appropriate information security posture anywhere. To be honest, there was pushback. So when we suggest that SessionGuardian could be used on sensitive matters remotely, we were told no. sensitive matters could only be worked on premise in a supervised document review setting. And then COVID came the phone started ringing in the rest of the history. I’d like to say that COVID accelerated the future at some point we would have reached this work from anywhere environment, and SessionGuardian overnight became the only game in town with a solution specifically focused on addressing cyber threats of users working remotely on their home computers.

Marlene Gebauer 4:57
And Oren, how did you get involved?

Oren Leib 4:59
Yeah, well, first of all great to be with everyone. I would say, you know, as an attorney and legal innovation evangelist, I like to describe myself at times. What really appealed to me about SessionGuardian. Frankly, the reason I came on board is that this is truly a privacy first company. And as Jordan mentioned, we’ve created technology, I think that truly walks the walk in that regard. I think we can all agree that data privacy is a fundamental and sacred, right. We live in a country where a court of law recently ruled that the All Writs Act couldn’t compel Apple to unlock an iPhone belonging to an accused terrorists. I don’t think that demonstrates just how greatly we value our privacy rights. That said, when I first came across SessionGuardian, I was Alm and I was astonished to learn that basically, you know, as we’re moving into a remote work based environment, any attorney on a project could come into a live session access privilege regulated data, without anything more than an MFA to quote unquote, authenticate. Now, keep in mind that anyone could share credentials with an unauthorized user. And that’s a simple and common way to circumvent MFA as users could take a screenshot or photograph sensitive data residing on the screen during the course of a doc review or translation, basically, any scenario where they’re in a remote work environment, and they have access to privilege regulated data. So I mean, from our perspective, these are the kinds of non negotiable security gaps that SessionGuardian protects against. And ultimately, and I love this about what we do, our solution enables our partners, law firms, corporations, and third party service providers to basically go out there and hire the best talent anywhere on the planet without ever compromising security. And frankly, that’s a huge net positive, not just for our customers, but the overall job market writ large.

Greg Lambert 6:43
We talked a little bit about this in the introduction, law firms spend a great deal of time, and probably a lot more money on complying with security requirements and keeping the client’s data safe, as well as their own. So Trish, I want to turn this to you, you know, what’s the perception of security in the market? And do you think that, you know, we’re all doing a good job?

Trisha Sircar 7:12
Yeah, look, you know, I can’t speak to law firms generally. But I do know for a fact that data security and privacy is top of mind, for law firms and all industries, especially as cyber attacks have increased in the pandemic, the digital economy is here to stay and just growing. And the hybrid workplace, which is, it’s still persisting, you know, for law firms, keeping data security and privacy at the core of their practice is especially important, because they are handling client confidential information and intellect intellectual problem. So it is I know that at the core of our most authentic culture,

Greg Lambert 7:51
yeah. And we’ve joked for many years here that it is job used to be about maintaining the network. And now it’s almost, I would say, probably half the job is security. And you’re just seeing so much in the securities operations of the firm is, does it feel a little overwhelming to you, Trisha?

Trisha Sircar 8:13
Look, sometimes it does, you know, we we are an AmLaw 100 national law firm. And, you know, our clients are very varied. And across all sectors. So know, as a data privacy and data security, and IP lawyer, I have to access a lot of data of different clients across different industries. So, you know, sometimes Zscaler, or other ad networks pictured when we access certain websites, and had subtypes be a challenge. But again, I think it’s very important to take those extra steps and get those provisions from IT security, to enable access. I know we can get frustrated, because everything in light of the pandemic DEI has not stopped. It’s a seven day work week for most law firms from partners. And speed is important, but it’s good to take a minute. I’m sure you had the right provisions, the right access the right it’s, the data is in the right people’s hands.

Greg Lambert 9:12
I think we all saw a senior associates, PowerPoint that talked about how just how long those weeks are for everybody.

Marlene Gebauer 9:21
So Jordan, or Oren, or both of you, and I’m very interested in hearing the answer to this this question because, you know, just we’ve heard so much about these certifications in the past. But you know, why do you think ISO and sock two certifications are not sufficient anymore for law firms?

Jordan Ellington 9:42
Thanks Marlene. So an ISO and a sock certification means that a you have policies and procedures and be that you’re following them at a certain point in time so they represent a snapshot in time. They do not represent the reality of what’s happening. On a matter that’s outsourced to perhaps a service provider as to how that information is being treated at that point in time. And so our perspective is that ISO and sock is the trust, right? It’s to establish trust with the vendor with the partner, that they have certain policies and procedures in place. But then we need to verify we need to make sure that ongoing, that information is treated appropriately. And that’s, that’s why we suggest using technology to enforce these policies and procedures in real time.

Oren Leib 10:38
Yeah, thanks, Jordan. I just wanted to take a moment to talk about the lawyer’s ethical responsibilities around safeguarding sensitive client data. I mean, especially when that data is accessible remotely, I know it’s not talking about Rules of Professional Conduct really aren’t a very sexy topic and, you know, sort of get dwarfed by the regulatory regimes out there, and the usual suspects like GDPR, since the PAP been a host of others. But I recently came across an ABA formal opinion for 77 R, and I want to read it verbatim, because it talks about the lawyers responsibility when it comes to security information. And it says and I quote, a lawyer may be required to take special security precautions to protect against the inadvertent or unauthorized disclosure of client information, when required by an agreement with the client or by law, or when the nature of the information requires a higher degree of security. I think that last part where a higher degree of security is triggered is always a red flag for us lawyers to heed. But what does that mean, practically speaking, right doesn’t really give you a prescription on how to do that. It’s pretty vague. Well, I mean, I would pause if MFA says we’ve established aren’t sufficient in authenticating credential users, for example, we’re safeguarding sensitive data that lawyers need to take extra precautions with both continuous identity authentication technology, and implementing security measures in a practical way that prevent things like screenshots and screen shares of sensitive data, which again, I think are you know, we mentioned non negotiable security gaps. So I think that becomes part of the lawyers evolving set of obligations and technical competency requirements as it relates to cybersecurity, that maybe the certifications that we talked about, don’t comprehensively address enough.

Marlene Gebauer 12:23
Trisha, what are some of the challenges firms face in keeping data private? I imagine the various and changing regulatory requirements in different jurisdictions are an extremely large complication. Do regs address the real world problems in a practical manner?

Trisha Sircar 12:42
Yes, I’ll answer your first part of that question. Number one is and will consistently be employee related issues, employee security threats, human errors across the board, in all industries and sectors are the number one reason that fail to keep private information, private, continuous employee education and training are vital. Number two, I believe, is data retention, particularly for US law firms in the US, we have a culture of retaining data for longer than is necessary. And law firms maintain a lot of data. And as we’re seeing these new laws, the California Consumer Privacy Act as amended by the California Privacy Rights Act. I know there’s some proposed laws by the FTC. And the SEC as well regarding data retention requirements, as we see more of these laws develop, I think, is going to promote a culture in the United States of data minimization. So that’s important. Number three, physical security, particularly in the hybrid world. I don’t know, I do know, one law firm. I only know really a 1am law 50 wall firm that’s fully remote. Every amlaw 50 law firm that I know that’s quite a manual, every am law 50 law firm, or am law 100 wall for my nose, generally hybrid. And I think when you take documents home, you share a house or a condo with other people, when associates have roommates in New York City, physical security is imperative. And I think, really, really have to educate and continuously, you know, alert employees to this, you know, made sure there’s, you know, safe document destruction when data is done, you know, we all print out documents to read them as lawyers. So I definitely think data. Physical security is another big issue more so the fifth in the remote world and hybrid world. I mentioned employee errors, but also insider threats. Right. As we’re seeing, you know, the macroeconomic factors at play right now. We may be seeing some reductions in force, and other issues that we already saw in the pandemic, but again, we’re going through that cycle again, macro economically. We’ve all seen the headlines about law firms doing layoffs and roofs. You want to just make sure that you don’t have a disgruntled employee. and all access and any kind of devices that they are now wiped clean and denied immediately. So, again, tying that to employees and insider threats. And then answering the second part of your question, did the regs address the wheel real world problems in a practical manner? You know, they do. Generally however, it’s hard for global law firms right that sometimes have to compete with conflicting regimes. The General Data Protection Regulation prescribes a time limit to keep certain data us regs require that to be kept for for a different period of time, certain standards regarding customer data, employee data, so they can be quite conflicting. And particularly in the US, we have a patchwork of regulations, whether it’s the gramm leach Bliley act for financial institutions, Children’s Online Privacy for companies that are in the EdTech space, then there’s the consumer protection laws. Now we have, I believe, the 11 state that has a consumer protection law with Oregon, so it can be challenging that the regulations can be competing. And we don’t always invest resources in understanding them and adopting them. But I think the biggest challenge with the regulations that we are seeing is in AI, as law firms and clients using AI more and more, we’re still waiting for the regulations to really provide the proper guardrails to use AI ethically and responsibly. Yeah, we’re having a hard time keeping up writing off and complying with, you know, the current privacy laws. It’s it’s just moving too fast. And I think, you know, the EU has made some great developments in that area of regulation. But other jurisdictions are still catching up. So I think AI is really an area where the regulations are lagging.

Greg Lambert 16:54
I know we’re really good at advising our clients on how to comply with these regulations. In just in your experience, or, you know, as you’ve talked with peers, our law firms good at taking their own medicine on this. Are we good at it complying with these regulations?

Trisha Sircar 17:12
I think so. I will read things. So, you know, I can only speak to can, and we have a great team. So it is front and center a priority. I’m in regular contact with my general counsel and my seaso an IT security as a career Privacy Officer of the firm, also my HIPAA officers. So yeah, absolutely. You know, we’ve we’ve heard the headlines, but there are a lot of law firms. You know, you’ve mentioned a couple of names that are only a couple, one hand. So I do think, generally, the more sophisticated and more 200 law firms or sophisticated boutique offense, do a good job.

Greg Lambert 17:55
Yeah, well, we’ve kind of set the stage here on where we are in the industry and kind of what the floor should be on complying with regulations and security issues that we should all be looking for. But, you know, the reason we brought Jordan and Oren on here was to talk about how do we go above and beyond what just is the basics here. So I’m gonna turn it over to the two of us and describe how SessionGuardian specifically addresses things like cybersecurity gaps, what are some additional protections that you provide beyond that floor that we just said, so, kind of give us some real bass introduction to SessionGuardian? And then kind of tell us what it is that makes you guys unique?

Jordan Ellington 18:44
Absolutely. Thanks, Greg. I think the first thing to address also is before a cybersecurity gap is a cybersecurity posture, and who’s actually responsible for for cybersecurity, of your client. So the client will have their set of cyber security rules and postures, and certainly they will be coordinating and working with a trusted law firm that also certainly has a high level of cybersecurity postures. But what we’ve noticed is that as there is what I’d call a delegation of responsibility downstream to a service provider, for example, from a practical perspective, they do not have the the budget and the IT resources leave most of them do that the clients a law firm says, and then further downstream, they will often hire a contractor who will often be working on their own device, their BYOD device, and certainly they have zero budget from a cybersecurity posture. When we’re looking at a cybersecurity gap, we try to look at it from the bottom up, where is the risk and the risk is at the end of the line. It’s who’s actually accessing it. information. And, and we will look at some very, very basic things such as, can a user take screenshots of the information they see on their screen? Can they copy and paste? Can they accidentally share this information on an evite on a web meeting, and you’d be surprised to find out that most vendors will ask for direction from their clients as to whether to enforce screenshare and screenshot protection. But I would submit that most most clients that are engaging vendors probably think that screenshot and screenshot protection is table stakes. So that is what we try to provide, we try to provide a minimum basis of security so that if even someone didn’t think about it, that security is is there.

Oren Leib 20:50
Yeah, just to add a little bit to what Jordan was saying and set the stage kind of on some alarming stats with regard to cyber hacking. And some of the use cases that may or may not surprise you that go beyond just, you know, your run of the mill, document review project or source code review project. So Bloomberg released an article. You know, basically, it was a report. From checkpoint research, the rate of global weekly cyber attacks rose by 7%. In the first financial quarter of this year, as compared to the same quarter last year, organizations face an average of over 1200 attacks weekly. One out of every 40 of those attacks targeted a law firm or insurance carrier, and more than a quarter of law firms and a 2022. Aba survey said that they’d experienced a data breach up 2% from the previous year. So pretty alarming set of statistics there when it comes to cyber hacking. In terms of use cases, I mean, I know that we’ve been known as a company to secure doc reviews and source code reviews, but our offering our security solution extends to any scenario or sensitive or regulated that is exposed in a remote environment. So that would include for example, things like data breach matters. That’s an obvious one. m&a and IP due diligence matters, contract review matters. Third party call centers, outsource it firms. And even in the education space remote learning and testing. we’re even seeing a growing incidence of interview fraud where, you know, the people that show up to work weren’t actually the ones that interviewed for the position. But naturally, our solutions sparked a lot of interest across regulated industries like the financial sector, health care, life sciences, and so on. Recently, we met the head of Global cybersecurity for a major North American bank. And they said to the last 10 Breach incidents were actually triggered by shoulder surfing and photographs of sensitive data.

Marlene Gebauer 22:46
People again. Just what Trisha said.

Oren Leib 22:49
Yes, exactly. And recently, I think everyone’s probably seen the Coca Cola IP theft story where a Chinese operatives snap photos of a series of trade secrets and formulas on our computer screen that she was going to use for a copycat operation in China. She found her way around all of the military intelligence grade data tracking software, by simply pulling out this thing called a mobile device and snapping photos, it was just that easy. So for example, you know, our mobile detection software was designed to prevent these kinds of instances from occurring. So just so practical examples in terms of the use cases that go beyond legal in into an enterprise wide environment,

Marlene Gebauer 23:28
Jordan and Oren Can Can you explain what you see as the role of SessionGuardian in ensuring business continuity and minimizing the impact of cybersecurity incidents on law firms operations?

Jordan Ellington 23:41
Yeah, absolutely. So cyber hacking has become a, it’s not just a question of continuity, it’s a question of survival. So it is a threat to a business, when their information or their clients information get gets breached. And this is compounded now, with a work from anywhere environment, where you’re no longer sure if if the person you have authorized to look at information is in fact, that user, what SessionGuardian tries to do is establish a physical security posture regardless of where the user is located, and ensures that they’re coming in from a known device at the appropriate time, from the appropriate location. And by keeping these types of controls and providing a security log that allows you to verify what areas of a system a person has been in and what levels of access did they have? Where did they log in? Were they by themselves or were they having heard they have someone looking at the screen with them for example. These types of controls greatly limit the risk of a cyber breach, and when it occurs it provides you a roadmap as to what areas of the business may have been breached, you know, by where and by whom?

Marlene Gebauer 25:08
So, I mean, is this Does this mean I can’t like log in at like, you know, four in the morning and start start working? You know, are they going to be like, Why is she on so early?

Jordan Ellington 25:18
That’s exactly right. So so that really depends. So it could it could be an observed behavior. So perhaps if it’s just starts happening before a large round of layoffs, what’s happening? Is there data being copied? Or is something strange happening? So visibility into at the end of the day, human behavior that is not normal? is what helps prevent or limit the scope of a breach?

Greg Lambert 25:46
And do you do this in a way that is seamless on the end user side? Because I can see, you know, if we, if the security guy had his druthers, you know, we may not be able to get to actually log in and do our work, you know, unless we, you know, do 10 things before we log in. So how do you how do you kind of build that level of security and, but not hampered the actual work productivity of the user?

Jordan Ellington 26:19
Yeah, that’s a great question. So the important thing is to tailor or customize the security posture based on the information that’s being looked at. And so, you know, in this way, you can achieve a context sensitive security where through, you know, most of your transactions, maybe you don’t need to have the webcam only on or maybe you only need to have it once. And then as you graduate into more sensitive information, you can dynamically be prompted to go through additional checks. The good news is that a lot of these checks are automatic. So for example, where you’re logging in, from what time you’re logging in, at, you know, can you perform a screen share a screenshot? All these things happen automatically? Do you have an a malware running? Do you have a vile anti virus running, these are all automatic, passive checks that don’t take any time. So, you know, it’s really important to balance you know, the security requirements that the firm has your obligations to your clients with the practicality of doing work. So the appropriate configuration, we do this as part of our service, we will consult with our clients to provide a recommended set of security policies in the system based on what needs to be protected.

Greg Lambert 27:40
So we’ve kind of laid the groundwork here on how to prevent any kind of cybersecurity. But as everyone knows, it’s probably not a matter of if but when this happens to a law firm, so what specific features or functionalities and SessionGuardian assists law firms in an incident response and the remediation after a cybersecurity breach happens.

Jordan Ellington 28:07
By far the most important feature that SessionGuardian provides is an extremely granular audit trail of again, who was logged in, where, when, what type of device and what were they doing what was happening in their environment. So that that audit trail paints a picture or a roadmap as to what may have happened very, very quickly, I think the scariest thing that can happen in a cyber breach event is just not knowing what was hit. And so understanding the you know, where that attack occurred is helpful. Secondly, by using technology, such as SessionGuardian, you greatly limit the freedom of movement of what happens when you get inside. If you have security set up properly. Zero trust is kind of like the the new buzzword. Well, you know, if in order to get that credential and start doing something that’s extremely sensitive, you need to show your face and show that it’s actually you, you make it that much harder for an attacker to run wild in an organization. Lastly, you know, if, if it comes time to pay the piper, and there are damages, being able to demonstrate that you took all the reasonable steps that you could have to limit and there’s an audit trail that shows that you have these security features in place. And I’m not a lawyer, but I would think that that would help limit your liability in these types of cases, or

Oren Leib 29:44
I am a lawyer, but I would definitely defer to Trisha on this one. I mean, I would I would presume that you’re right, you know that those would be mitigating factors. Some things just you know, as as Greg said, you know, it’s not a matter of if but when these attacks occur and liability is going to be based on whether or not you took all the reasonable precautions in terms of technical competency and resources to prevent that as best you could, knowing the software environments not perfect, but I would definitely defer to Trisha on more specifics on that if she likes to delve into it.

Trisha Sircar 30:18
In terms of, you know, if you’re doing everything by the book and dotting your I’s and crossing your T’s, in terms of any liability that you might experience, when you do face a breach, you know, there’s, of course, as you know, notification costs, ID Protection costs, and all those kinds of costs that most companies will provide. Not all, I believe only one or two state laws in the US require it. But it’s really a commercial decision. But again, given the amount of law firms there are, and how important reputation is and goodwill is, you know, I do think that’s where that that comes into play. So really, I think the biggest issue is really those kinds of costs, any mitigation costs, you know, forensics always cost a lot of money to do a deep dive and root cause analysis. And then the reputational costs that, you know, that might impact the law firm. However, again, it’s not if it’s when you do it. So it really those reputational costs will be minimized. If you’re doing everything, right, obviously, you know, consumers can bring a private right of action under the California Consumer Privacy Act. However, if reasonable, it’s based on a reasonableness standard based on the information security practices, so yes, there’s a cost to defend that. But generally, hopefully, you know, the firm will come out on the right side. And of course, you know, there’s other penalties and fines that can be imposed. But again, when we’re seeing those kind of enforcement actions, we’re really, you know, the Attorney General’s of California, New York, the FTC, and other the GDPR enforcement tracks, they all provide examples of what those fines and penalties look like. So that can give you a good understanding of where it’s based on negligence or no assault or be competent.

Marlene Gebauer 32:16
So we we touched a little bit about WFE, earlier and Jordan, you know, that’s another area where the cybersecurity landscape has changed, you know, because we have this incredible need for mobile access, you know, to be able to work from from anywhere. So why is multi factor authentication no longer adequate in this evolving landscape?

Jordan Ellington 32:41
Thanks, Marlene. That’s a really interesting question. And it’s, it’s a subject that’s I spent quite a bit of time thinking about. So want to take a step back down down memory lane in terms of information, security standards, when we just had usernames and passwords. So we had usernames and password. That’s how you accessed sensitive information. And then someone realized that, well, we need to have complex passwords, they need to have a certain level of complexity. So okay, now we have usernames and slightly more complex passwords, and everyone couldn’t figure out how to create that complex password. And then we figured out that that wasn’t enough, either, we need to have an MFA because there are ways to even find out what it complex password is. So that we had usernames, complex passwords, and MFAs. And then COVID happened and everybody started working remotely. And what we found is that that MFA was really designed to protect my password from being used by someone else. But if I wanted to actually have someone else use my password, I can give them my MFA code. And so from a remote work scenario, the MFA by itself, no longer gives the assurance of whoever is responsible for protecting the information that that the individual using the MFA is actually the user that you expect them to be. You know, we think that the next step now is to provide assurance of correct usage of of the information, who is the user is? Pardon the cliche, the answer is staring us in the face. It’s the face, it’s the person the person needs to become the factor of authentication to access information. And so, you know, we built a business around it, we have adoption, we think that this is the appropriate way to secure information remotely, and ensure that you know, who’s about to log on to secure system.

Greg Lambert 34:51
And, Trisha, let’s get your opinion on that from your view. You know, are there regulations being updated? Did is there? Or is there not kind of this clear guidance from from the regulations on on what, what law firms and clients should be doing?

Trisha Sircar 35:10
Yeah, look, I think in terms of privacy and data security regulations, it is quite clear. However, you know, they’re not always, they sometimes conflict, right. And they’re also, they’re expensive to comply with, especially for smaller law firms are, you know, smaller, even a solid practice, law firms really need to take the time, like any industry, you know, it’s not just a checkbox compliance, it really, you know, it has to be a culture of privacy by design, especially, again, given the digital economy that we live in the hybrid world, and just how the cultural shift from the pandemic, to this kind of, especially in the US, with this mainstay hybrid world. Again, I’m going to reiterate it again, and again, employees are still our biggest assets, but also our biggest threats. So any kind of work from home, policies and procedures need to be regularly updated and enforced. And it’s those, you know, penalized penalties for non compliance should include termination, when developing these policies and procedures for work from home, where it’s complete remote, where it’s hybrid, where it’s in the office, you know, legal IT and HR need to work this together. California employees will have different rights to New York employees. We have law firm partners that have worked between London and New York, or Chicago in London, you know, different countries require different things. This is a dynamic process that has to be constantly reviewed and updated, not only for the laws, but also the changes in the threat landscape, and has to involve senior management leadership, legal IT and HR.

Marlene Gebauer 36:55
Trisha, we’ve we’ve talked about the need to follow regulations and need to be aware of ethical considerations. We’ve talked about using technology to prevent things from happening when they happen. But that’s only part of it. Right? General Counsel risk, IT departments, they still have to address things like terms and conditions with vendors, ownership of security keys, building strong remote file inclusion and master service agreements. So from a negotiation standpoint, what are some things to look out for when negotiating with a vendor?

Trisha Sircar 37:38
Yeah, no, I look, again, I think most law firms will tell you, especially the larger law firms, nationally and global law firms, typically, if a vendor is handling personal data, or confidential data goes through an IT security process typically won’t even get to legal to review the document until it goes through that process. If a firm doesn’t do that, they should absolutely do that. Okay. And then, in terms of when he gets to legal, when we negotiate the MSA and the provisions in that MSA, the law sets the minimum requirements, other provisions in the GDPR, or the California Consumer Privacy Act, or gramm leach Bliley Act, or HIPAA commercial provision. That includes you know, who bears a cost for data breaches. Audit who pays for an audit, when negotiating these agreements, these are commercial requirements, you should appropriately allocate risk based on which side of the agreement you’re in, you know, the law firm can be the client or it can be the service provider. So depending on that relationship, your risk posture will change. And, you know, you obviously, wouldn’t negotiate terms and conditions favorable to you. But that includes limitations on liability, indemnification audit rights, in addition to what the regulation requires additional commercial requirements. And then you also want to, I always ensure that any vendor that we use for my clients have locaton have minimum cybersecurity insurance requirements, in addition to other types of insurance, and get those certificates of insurance. Those are really the big ones, you know, a limitation of liability, indemnification insurance, and then, you know, appropriately tailoring, you know, communication when there is a breach, and repays, what’s the cost? How is that quarterbacked that should all be in the MSA.

Marlene Gebauer 39:37
So this is the time when we ask our crystal ball question to everybody. So looking out at the next two to five years. Tell us what changes or challenges lay in store for us in the area of cybersecurity. Trish, you want to start?

Trisha Sircar 39:54
Sure, like again, number one is not going to change. It hasn’t changed in my career. Employee Leave. So we just we just have to consistent with trading and the bring education and making it at a very rudimentary level that old generations plan process and understand that and also tailoring employee training based on the work from home, but the hybrid, and there are complete in the office model. So again, I cannot emphasize that role that will not go away. The second biggest issue, and we’ve mentioned it again before is the ethical use of AI, and how we use AI and cybersecurity in a compliant way and managing privacy as well. So AI is a hot topic. So I think it’s going to be in cyber privacy, in machine learning everything.

Greg Lambert 40:47
So, Jordan, same to you the crystal ball question, what do you see as a change or a challenge for the next two to five years,

Jordan Ellington 40:55
I would say that the change is going to be also within the next few months, let alone the next two to five years. And I would say that the unethical use of AI in hacking is by far the biggest danger and threat to cybersecurity today. And we’re going to see increasing incidences of AI impersonating individuals, even within an organization. You know, sending an email from the CEO that sounds like came from the CEO with topics and click on this and it’s game over. I think the importance of establishing that a person is real is going to become ever more important. And I think we’re also going to see a separation between personal emails and work emails and matter specific emails that perhaps do not include the ability to email in and out of a closed group of people. Just because it’s going to be too simple to have an email impersonating a topic that you think is relevant. You click on it, and then you have a breach. And Oren,

Greg Lambert 42:09
how about you?

Oren Leib 42:10
Yeah, that’s a great question. I definitely agree with Trisha and Jordans insights on the future cyber threats and the increasing role of AI powered hacks and deep fakes. You know, look, we know the pandemic hyper accelerated digital transformation without any time error. But I think we need to step back, think about how lawyers who handle some of the most sensitive data out there need to step up their cyber defense game. I mean, it’s no longer voluntary. It’s obligatory. You know, we’ve seen ABA formal opinions about this that suggests that lawyers need to take special security precautions when it comes to sensitive data that includes privileged, confidential and regulated information. And as lawyers, we just can’t simply pass the buck to cybersecurity engineers and experts in the event that there’s a data breach. I mean, I understand there’s no such thing as a perfect cyber defense. But we need to demonstrate his lawyers that we’ve undertaken all reasonable efforts to safeguard data. That includes technology and tools and methodologies that need to be employed in our clients, the regulatory enforcement agencies out there, and even the attorney ethics boards, they won’t accept an excusable neglect in this regard. And by the way, this isn’t just a call to action for big law players out there. All attorneys, regardless of their firm size, or raking are equally accountable. And that includes law companies as well. LSPs LSPs, all of them must comply with data privacy regs, and RPCs. Where applicable. I mean, just think about this, virtually every day, multiple times a day, privileged and sensitive information becomes exposed to subcontracted attorneys working in BYOD environments somewhere out there in the world. When that happens, the level of data protection exponentially recedes. And there’s a cyber hacker waiting to pounce, I don’t mean to sound overly dramatic. But we already covered some of the alarming stats earlier in the conversation on this. What we’ve noticed is there’s a lack of cybersecurity awareness amongst attorneys. To that end, SessionGuardian is not only committed to providing a robust cybersecurity defense, but also educational awareness. And I think that’s the best way to tackle some of these issues that are present day and in the future. To that end, we’ll be offering free on demand CLE courses that fulfill mandatory cybersecurity CLE credits. And for more information, feel free to ping us at cle@sessionguardian.com. Yeah,

Greg Lambert 44:35
well, if it does play out, I hope most of the firms learn from the mistakes of others and then from their own mistakes. Absolutely. All right. Well, I want to thank Jordan Ellington, Oren Leib, and Trisha Sircar, for coming on and taking the time to speak with us. Think thank you all very much. Thank you.

Oren Leib 44:54
Thanks a lot,

Trisha Sircar 44:55
Thank you.

Jordan Ellington 44:56
Thanks for having us.

Marlene Gebauer 44:58
And of course, our audience thanks to all of you for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 45:11
And I can be reached @glambert on Twitter, everyone, Trisha where if they want to reach out to you? Where can they find you?

Trisha Sircar 45:20
Know, you can find me on my Law Firm bio page or LinkedIn

Greg Lambert 45:23
And Jordan.

Jordan Ellington 45:25
You can find me on LinkedIn.

Greg Lambert 45:27
And Oren,

Oren Leib 45:28
you can also find me on LinkedIn.

Greg Lambert 45:30
I’ll make sure we get all this LinkedIn links.

Marlene Gebauer 45:33
LinkedIn it is listeners if you want to leave us some feedback. You can also leave us a voicemail on our geek review Hotline at 713-487-7821 and as always, the music you hear is from Jerry David DeCicca Thank you Jerry.

Greg Lambert 45:51
Thank you Jerry and talk to you later Marlene.

Marlene Gebauer 45:54
Bye bye

Isha Marathe, a tech reporter for American Lawyer Media, joined the podcast to discuss her recent article on how deep fake technology is coming to litigation and whether the legal system is prepared. Deep fakes are hyper-realistic images, videos or audio created using artificial intelligence to manipulate or generate fake content. They are easy and inexpensive to create but difficult to detect. Marathe believes deep fakes have the potential to severely impact the integrity of evidence and the trial process if the legal system is unprepared.

E-discovery professionals are on the front lines of detecting deep fakes used as evidence, according to Marathe. However, they currently only have limited tools and methods to authenticate digital evidence and determine if it is real or AI-generated. Marathe argues judges and lawyers also need to be heavily educated on the latest developments in deep fake technology in order to counter their use in court. Regulations, laws and advanced detection technology are still lacking but urgently needed.

Marathe predicts that in the next two to five years, deep fakes will significantly start to affect litigation and pose risks to the judicial process if key players are unprepared. States will likely pass a patchwork of laws to regulate AI-generated images. Sophisticated detection software will emerge but will not be equally available in all courts, raising issues of equity and access to justice.

The two recent cases where parties claimed evidence as deep fakes highlight the issues at stake but did not dramatically alter the trial outcomes. However, as deep fake technology continues to rapidly advance, it may soon be weaponized to generate highly compelling and persuasive fake evidence that could dupe both legal professionals and jurors. Once seen, such imagery can be hard to ignore, even if proven to be false or AI-generated.

Marathe argues that addressing and adapting to the rise of deep fakes will require a multi-pronged solution: education, technology tools, regulations and policy changes. But progress on all fronts is slow while threats escalate quickly. Deep fakes pose an alarm for legal professionals and the public, dragging the legal system as a whole into an era of “post-truth.” Trust in the integrity of evidence and trial outcomes could be at stake. Overall, it was an informative if sobering discussion on the state of the legal system’s preparedness for inevitable collisions with deep fake technology.

Links

• Turbocharged by Election Season, Deepfakes May Soon Face Legal Reckonings (free registration req.)
• Deepfakes Are Coming to Courts. Are Judges, Juries and Lawyers Ready?

Listen on mobile platforms:  Apple Podcasts |  Spotify

Contact Us:

Twitter: ⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠, or ⁠⁠⁠⁠@glambert⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠

⁠⁠Transcript

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. So Marlene, this is our five year podcast anniversary anniversary. Yay. So I just, I just didn’t want that to go pass us by without saying something. Because I think it’s been, you know, just a fantastic five years. That’s great. And, you know, it really feels like this year, we’ve almost started with all the generative AI and advancements in technology that’s been going on this year. It’s true. I

Marlene Gebauer 0:42
mean, you know, we’ve always had great interviews, and but this is sort of a whole new area that that is new to everybody. And so I think that makes it even more exciting.

Greg Lambert 0:52
It’s been fun because work. I sound like the expert.

Marlene Gebauer 0:57
It’s like everybody’s an expert now. Right. You know, and as the as you noted, Greg, we have had a lot of episodes devoted to generative AI tools, you know, what they can do, how to address risk and using them. Today, though, we’re delving into a darker area, one where AI is used to purposely fool the viewer and impact decision making in court. So welcome to the world of deep fakes at court, are you Are we ready? Say yes.

Greg Lambert 1:28
Yes, yes.

Marlene Gebauer 1:30
Our guest today has been speculating on that question. Isha Marathe a legal tech reporter for American lawyer media recently wrote an article. Deep fakes are coming to a court, our judges and lawyers ready, where she delves into the implications on litigation. Isha Welcome to The Geek in Review.

Isha Marathe 1:49
Thank you so much for having me. I’m excited to talk about this. Yeah, we are two very interesting. Yeah, both

Greg Lambert 1:57
are ghosts. This is our ghost story episode. So I’ll say yes. To underlay it, so well Isha. Before we kind of dive into the article, let’s back up a little bit and just talk about, you know, what exactly are is a deep fake. And I know that we, you know, I think we’ve seen a lot of these on, you know, social media, and we probably heard of some other darker recesses of the web that these have shown up, but can you kind of just give us an overview of it?

Isha Marathe 2:31
Yep. So I’m a deep fake. I mean, it’s like in the word, it’s just a blend of deep and fake. And the deep comes from deep learning, which is a form of machine learning. So that’s like the AI bit of it. And then the fake is more obvious. It’s comes from the fact that this technology is used to create an image, which usually incorporates the likeness of another person. And it does so like in a really convincing and a really good way. So I really see a deep fake as like a really, really good, super realistic Photoshop situation.

Marlene Gebauer 3:12
How are they created? I’m very curious to know that, because you mentioned this is a really, really good Photoshop, and you know, like, I’ll show my kids pictures, and they can tell me right away, it’s like, oh, that’s been shot. And I can’t tell I can’t tell. But they can tell. But apparently, these are so good that even maybe they can’t tell. So like, how are they created?

Isha Marathe 3:33
They are they’re really good. And they’re getting better. I almost want to say like every two weeks, oh, my god, like I know, only like two months ago, they couldn’t really create hands too well, or there were like too many teeth in people’s mouths. And like now they’re like, wait, the hands look great. And the teeth look great. So it’s really, it’s, yeah, they’re really good Photoshop to deal with. I shouldn’t even say Photoshop to you. I just say that, because that’s the only kind of barometer to equate it with. But they’re made very differently. It’s not a cut and paste situation. There’s many ways of actually making deep fakes. And back in the day, and by that, I mean, pre the democratization of generative AI, which has happened over the past few months, pre that era, there was this like kind of complicated and expensive and lengthy process where you needed high end computers, you know, which allowed you you have to use like an encoder decoder algorithm where two people’s faces were run through two encoder decoder pairs. The encoders kind of learned the latent facial features of these people during the training. And then at the end of the process, the decoders were swapped so that the image kind of was decoded to produce the facial features of the other person. And it this was like sounds complicated. It was complicated and It took so long because you had to repeat this process over and over, like hundreds of times for each frame of a video or to train the algorithm sufficiently, you know, to create a believable image, or deep fake. And that’s just one example. There’s like, there were tons of ways to make these kinds of time consuming and Complex, deep fakes. But that, of course changed. After we kind of open AI came on the scene with, you know, the first tool was ChatGPT, which doesn’t create a deep fake, but it kind of democratized generative AI, as I said, so now we have like stability AI and mid journey and mid journey is famous slash infamous for the Pope and the puffer jacket, deep fakes,

Marlene Gebauer 5:48
which we all love to, you know, it’s like, that’s the thing. Yeah.

Isha Marathe 5:52
Though, it’s so great. And, you know, there’s lensa, which we saw, I think it was like on everyone’s Instagram story at one point, and it kind of like looked cool, and, you know, look just like people’s faces. And there were all these cool backgrounds that looked realistic. And anyway, like this thing that’s kind of just like Mima fide and like made fun and viral has kind of changed this lengthy, expensive process that needed a lot of skill and good equipment, into something that a beginner can now do, just by clicking a button and create these super realistic images. You can feed a text prompt, which is what I think the pope in the puffer jacket situation was, or you can feed images into some of these tools. And the software can be found super easily. It’s on GitHub, which is an open source development community. So it’s really approaching and click of a button technology.

Greg Lambert 6:49
So you know, so much easier than it used to.

Marlene Gebauer 6:53
It really is so, you know, yeah, you know, we’re talking about the Pope and the puffer. And, you know, that was fun and amusing. Everyone had a laugh, but, you know, the Pope and a puffer aside, say that quickly? Why, you know, you reported that deep fakes, you know, are sort of creeping into litigation. And, you know, why are deep fakes, so dangerous in that context.

Isha Marathe 7:18
So there are really two ways for deep fakes to come into litigation as of right now. And one of them is a deep fake at the center of a case, say it’s a defamation case, say the Pope was like, I didn’t like that, you know, or like, whatever. But the other, which I think, right now, to me, scarier is a deep fake coming in as evidence for a case that has nothing to do with a deep fake. And that second possibility is the one that we talked a lot about with her really great sources for that piece that you’re talking about. And I think that it poses the most challenges right now. Because without the education, without the technology, and without the regulations in place to tackle this technology that is really evolving at breakneck speed, and is suddenly like in the hands of everyone, the legal system doesn’t have too many ways to stop it, to understand it from entering trials and courtrooms and even even coming in front of a jury, which is the most scary, and I’m sure, you know, we’ll get into that part more later. But it’s, I don’t know, it’s really a challenge to deal with evidence that just is not what it looks like, or sounds like upon first instance. And I think, to me, this really like this technology really dragged the legal industry into, like a similar post truth era that we found politics to be in 2015 2016. That’s scary to me.

Marlene Gebauer 8:52
I mean, I’m kind of wondering, like, you know, are people who deal with evidence? Are they ready for this? Are they aware of this? And how are they going to deal with it?

Isha Marathe 9:02
Um, okay, so we, so there’s many people who could technically deal with it, there’s, like, different

Marlene Gebauer 9:11
and maybe we should start with, like, who deals with it, maybe just sort of answered that call, right. And we can go into sort of what to do later. The very

Isha Marathe 9:19
first line of defense, according to our reporting, so far has been that it would be ediscovery professionals, because not only are they likely to be the first ones to catch something amiss, but they also probably have the most tech education. My colleague Cassandra acquire, was also on the tech desk actually wrote a really great piece about the three lines of defense that ediscovery professionals have right now. When it comes to tackling deep fakes and, you know, they go the first line is the metadata, which may give ediscovery folks enough information, even before review, just by kind of trying to track the lifecycle. of how the data has been flowing, that’s connected to an image that may be a deep fake or not. The second line of defense is the forensics, which is actually analyzing the image, running it through software, or doing a semantics analysis. And then the third line of defense is the one that’s like the most forward looking and I think extends beyond legal, which is waiting for good tools to come to market that are going to detect AI generated output. So yeah, that’s that’s the that’s the discovery portion of it. I think.

Greg Lambert 10:34
Are you seeing any examples that are already entering the court? And are there particular types of cases that are more, I guess, vulnerable to this?

Isha Marathe 10:45
So um, I think that, you know, we’ve seen, we’ve seen too, as of late May, in terms of where a party said that are claimed that the evidence that was presented was a deep fake, and one was in a Tesla lawsuit involving an image of Elon Musk. And another one was related to the Gen six riots, involving former President Barack Obama and Trump, oh, sorry, involving former President Donald Trump. And both judges determined that the evidence was, in fact, a deep fake. And it came up and it kind of went, and it wasn’t, it didn’t cause too much of an effect on the case itself. So we haven’t seen anything that so far has, like altered a case dramatically. But for people who talk to you, they’re like, even these two cases are enough to signal something in the pipeline.

Marlene Gebauer 11:47
So you know, you you quite rightly mentioned that, you know, eDiscovery, folks are sort of the first line of defense in order to identify deep fakes. Are there any others? You know, before things get to court? And then once it gets to court, you know, how do they make this determination? And you know, how successful you know, are these methods to try and shine the light on deep fakes.

Isha Marathe 12:16
So a lot of the questions around how successful

Marlene Gebauer 12:21
the methods do we even know?

Isha Marathe 12:24
All right, yeah, it’s just early days for that. And I wish I knew, but yeah, it is early days for that. But, you know, to me, it’s kind of a lot of it falls on counsel and, and then being educated enough to inform the bench about the technology that’s coming in. And Lee Tedric, who I actually interviewed for the article, she does workshops with judges, where she has, I believe, like monthly sessions to just educate them about the basics of this technology. It’s at the education part kind of falls on everyone. But right now, if we’re talking before even coming into court, I think ediscovery experts would be like, the key player here.

Greg Lambert 13:14
Yeah. Well, I mean, they’ve been the key word or, you know, fake documents, or Yeah, documents that have been altered. So it would make sense. But I know, Professor Ted rich had talked about the amount of education that the judicial judiciary needs for this. But, you know, I’m hoping that it doesn’t cost any money because I don’t have any money. So how, I mean, is education going to be enough? I guess,

Marlene Gebauer 13:46
now, it’s not going to be free.

Isha Marathe 13:50
And you know, it’s not going to be consistent and equitable. I remember when I started reporting on lion tech, and a source said to me, if you know, one cord, you know, one cord, and that’s just, like, super true. So I think that’s going to be very true here. You know, we saw over the pandemic, that there were some courts who were super prepared, you know, they already had cloud integrations and video conferencing technology, and their judges knew how to use Zoom. And then there were other folks who were like taping cameras to the ceiling trying to do just like some type of a video conferencing situation. And I think that’s how this is going to be, you know, the, the tech that’s coming up to detect AI generated images is super new. The education is super sparse right now. Because, you know, who even knows what’s going on, including the people who are making the technology? So, yeah, it’s kind of I wish I had a better answer for that other than, no, it’s just gonna be rough.

Marlene Gebauer 14:53
Yeah, I mean, because I know Professor Diedrich was mentioning that, you know, judges should have have, you know, trusted technical experts on ham, who can authenticate? And as you know, Greg very rightly pointed out, you know, the courts don’t have financial or administrative resources to hire these experts. And I just, I wonder how we’re going to deal with it? Well, yeah, I

Greg Lambert 15:19
think he should already said, we’re not, we’re not. Yeah, no, I’m

Marlene Gebauer 15:24
just hoping if there’s another answer. I don’t I don’t know

Isha Marathe 15:29
that like all hope is lost. I don’t mean to say that. I just think in terms of the education, I just think it’ll be slow. But there are tools in the in the toolkit, currently, even for judges and, you know, one of the one of the tools that comes up often is the Federal Rules of Evidence series, 901, and 902, which gives the court a rubric for dealing with computer generated evidence. So there are things in there you know, there are there are little pieces that judges can rely on. And some people, you know, who we spoke to believe that that’s enough. And they believe that a widget is a widget to quote, one of my sources, and it shouldn’t make a difference if the deep fake or if it’s a text message, or a Slack message or an emoji, you know, it’s all at the end of the day, the same. There are people who believe that, yeah, yeah. Yep. So there are there are these rules,

Marlene Gebauer 16:26
going back to sort of how this is going to impact the courts? You know, what about the existing backlog? Will having to deal with the deep fakes further delay litigation, and you know, how much more pressure will litigants have to settle or arbitrate or mitigate?

Isha Marathe 16:42
I personally do not know about the effect on settlements. And I, I wonder, again, if that is, there’s not been enough cases to know. Yet. However, it is super likely that deep fakes will complicate litigation. And, you know, it’ll just because of the situation where there’s a lack of education, lack of blanket comprehensive education around this technology, and also a lack of technical experts, for judges, you know, what is going to happen is that both parties just have their own experts if they can afford them, and they’re going to be expenses, which brings up a lot of access to justice issues. But it’s going to be expensive. And it’s going to lead to what folks called Battle of experts, where there’s just an image of Person A meeting person be, one party says the image is fake. And, you know, without universal watermarking standards, and without a court that’s equipped to handle the situation, each side is just going to bring its own expert, and they’re just going to duel it out in this confusing and expensive battle of experts. So I think,

Greg Lambert 17:55
well, and obviously, you know, the, if someone purposefully sends in fake evidence, to a case that there would be repercussions for that, if caught. And so it’s a very high risky move to make. Do you know, I know. Mr. Schwartz from from New York just got fined $5,000. Have you seen any, any kind of actions or sanctions so far? Because like the courts just came out with with new rules on you gotta tell us if you’re using generative AI, are there any has anyone jumped on this about you had to verify that these are not deep fakes?

Isha Marathe 18:42
I do not know of courts or judges, specifically who had said this yet, from the reporting that I’ve done until the end of May. But I believe there are regulations coming out of Texas, Washington, and California, the California law might actually have sunsetted, earlier this year, but there are state laws coming up, kind of like data privacy regulations, which we often call a patchwork, because they’re slightly conflicting. And and, you know, in the absence of Federal Regulations, of course, that’s what we have. So I think that’s what we have right now when it comes to any authentication, compliance for AI image generators. But nothing I know of for a court, in particular.

Marlene Gebauer 19:33
So if a deep fake does get past safeguards, you know, if it gets in front of a jury, you know, even if it’s discovered and stricken, you know, what sort of impact do you think that has, you know, do you think jurors can just walk away from what they’ve already seen? I mean, images can be very compelling.

Isha Marathe 19:51
Yeah, there’s really kind of a debate here among our sources for the story about deep fakes and chord at least and half of the folks who we talked to say, well, a jury is almost like sacrosanct in a way where you can’t even question they’re, they’re just a cog in the machine. And we just have to trust what they’re doing. And that’s just how it’s always been. And you can’t really get into their psychology too much hear around deep fakes, but then there’s others. And this is a part that I’m more interested in, who say that a jury, at the end of the day is just a set of people and shaking off that first impression of an image or a video, just because a judge says, oh, disregard that, you know, it’s not really realistic. And taking that a step further, if either side is then dissatisfied about the jury’s ability to unwind their knee jerk presumptions about a fake image or video, it could lead to calls for Miss trials, again, lengthy legal battles. So I do think, you know, the potential to influence a jury here is really alarming.

Greg Lambert 21:02
So Isha, the, you know, the, we’ve talked about, you know, the Pope and the puffer jacket, and really kind of images that are being used as are deep fakes are being submitted as evidence, but there’s other types of evidence that could also be manipulated through such as video recordings that are introduced or even witness testimony or transcripts. So, you know, what steps do you think can be done on these things as well?

Isha Marathe 21:32
I really think that starting education, or Hurley and I know, that’s always like an easy, just like an easy thing to say, because you can’t really touch it. It’s not so tangible, but I do think starting education early for judges, you know, having mandatory kind of education courses, or CLE credits around, this is the way to go. Not just for judges, but for counsel for ediscovery professionals as well, you know, just flagging that, hey, like this is coming. We might see things we might hear things that aren’t what they seem to be. And just keep that in mind. Aside from technology,

Greg Lambert 22:12
yeah, those are courts incorporate. So I was gonna ask is if do you think the technology for example, you know, we’ve talked about people using AI and submitting AI for class classwork, you know, for homework, I imagined being being used for trials as well. But, you know, there’s systems out there that supposedly that you can, you can pay something in and they can tell you, you know, this is probably 80% Chance written by AI. Is the technology to catch deep fakes, at least kind of keeping a close runner up to the advancements in creating the deep fakes.

Isha Marathe 22:56
Yeah, that’s really a good, that’s a good question that I don’t know the answer to. I’ve heard both things. I’ve heard things, you know, folks saying no, there’s a lot of interest it to create this technology. And then I’ve heard folks saying that it’s really there’s not enough money being put there.

Greg Lambert 23:15
That’s how people want to spend their money.

Isha Marathe 23:18
That’s right. That’s right. And I don’t know, like, honestly, which side is true? I, but I think there are solutions. Certainly, there’s like, there’s sensitivity multigo ZeroFOX, like, among among several solutions that claim to be able to identify deep fakes. But until there are enough of them, and until there’s enough kind of causes to do this analysis. I think it’s going to be hard to say, though, a lot of what I know is just experts kind of looking at an image and then saying like, there was most recently that video that Florida Governor Ron DeSantis, posted of, or his campaign, I apologize. On Twitter posted of it was like a mishmash of pictures of Donald Trump. And one of the pictures was Donald Trump hugging Dr. Fauci. And, you know, that one has since been kind of debunked as a as a deep fake. But then experts came out and they said, Oh, his collar doesn’t line up, and his buttons are a bit weird. And his like, neck doesn’t look real. But on first instance, it kind of looks real. Yeah. And that’s really the concern, right? Like, it’s, you know, by the point people see it, it’s already too late. And that’s, that’s really, that’s really the concern that technology.

Marlene Gebauer 24:40
Yeah. And I keep going back to this, this idea of like, how it’s going to impact the credibility of evidence, and, you know, is this going to force a change in in sort of the rules of evidence, you know, how, how does this proliferation of deep fakes you know, how is that going to impact act burden of proof for legal professionals and standards of admissibility for evidence in court. I mean, do you have any thoughts on that?

Isha Marathe 25:08
You know, so far, what we’ve heard is that it’s super unlikely for the rules themselves to change what is more likely is, so I’m looking at the fre 901 and 902 series. And you know, which basically says that the evidence produced must be, quote, sufficient to produce to support a finding that the item is what the proponent claims it is, and quote, you know, in the pre deepfake era, like circumstantial evidence was often enough to determine the authenticity of something because courts wanted to avoid the lengthy trials, which involved, you know, expensive forensic experts for every step, and, you know, like affidavits for every authentication. But I think now, the impact might just be that attorneys have to rely on forensic experts more than then they have. And you know, it’s just going to be these, again, expensive kind of practices that were shed by courts that are just going to come back. And I think that’s just going to impact people who really can’t afford can’t afford trials that go like that. And then, you know, that’s who I really worry about the most.

Marlene Gebauer 26:24
So this has been just a fascinating and scary conversation. And so, yeah, there is a great burden on on legal professionals here to educate themselves. So how can they, you know, most effectively educate themselves and sort of stay updated on the latest developments in deep fake technology, to you know, better navigate its impact on litigation and evidence? Um, no easy answer, I’m sure. Yeah,

Isha Marathe 26:55
there’s many ways. From my perspective, I’m always like, you know, read the right publications, or check out the scholarship that’s coming out. And there is some really interesting scholarship coming out about this. Off the top of my head, Rebecca Delfino wrote a great paper about deep fakes in trials and seek out workshops, seminars, talk

Marlene Gebauer 27:17
to your ediscovery people, you know,

Isha Marathe 27:19
yeah, talk to your ediscovery people talk to your IT department

Greg Lambert 27:22
Read Isha’s articles.

Marlene Gebauer 27:24
Yeah,exactly. All of them all the time.

Isha Marathe 27:29
And I would also say, recruit and hire the right people, you know, whether they’re tech experts for like judges back pockets, or for counsel, or just an ediscovery departments? i There’s no, I wish there was one answer to that. I would love a single answer. But there isn’t. I don’t think that,

Greg Lambert 27:46
well, you know, we work in an industry that loves looking back at things and what kind of precedent is out there? So I know, I know, deep fakes are kind of the new new rage. But you know, before that, there was Photoshop, before that there was, you know, photo manipulation and evidence manipulation? Do you know if there’s any legal precedent? Or if there’s landmark cases out there, where this type of either deep fake now or what would have been, you know, essentially a deep fake, you know, two years before that, kind of significantly changed the outcome of a case? And are there any lessons that this industry can can look back on and, and kind of apply to what we’re what we’re approaching

Isha Marathe 28:36
I definitely don’t know, of, of any cases like that about deep fakes as as the technology we’re talking about today. But when it comes to the past, the only thing that comes to mind, I think, and I don’t even know if this directly connects, but I think about, you know, like we’ve talked a lot about deep fakes in elections and politics, but the place where litigation most is going to come as defamation and my opinion, and not just in the US, but also internationally, which is what we’ve reported. And, sadly, the majority of deep fakes are seen in revenge porn and porn that is using, you know, faces of celebrities in videos. And there was a case that actually struck down I’m trying to try to look it up. But it was a federal child pornography law, I believe, that ruled that the First Amendment, you know, protects the graphic manipulation of images, even if it appears that you know, it’s it’s minors who are engaging in these activities because it’s not It’s generated, you know, synthetically, and it’s not real and nobody was harmed during the making of it. And that’s the only thing that comes to mind how it’s going to directly connect to deep fakes, I don’t know, except to say that, you know, there’s really strong First Amendment protections for defamation and why litigation might go up is really unlikely to, to kind of succeed. You know, because it’s hard to prove, and there’s a lot of protection for creativity and parody and for synthetically produced materials, and I think and political speech than political speech. That’s right. So I don’t know if that was a bit of a stretch. But that’s all that really came to mind.

Marlene Gebauer 30:48
So, you know, we we’ve talked about more having more education, we’ve we’ve talked about different types of technology that can perhaps spot deep fakes, what other measures? Or are there any other measures that should be taken to, you know, adapt to, you know, deep fake technology that’s impacting eDiscovery, and the overall judicial process? You know, are there any things that that need to change? I mean, you know, apparently the rules don’t need all the experts are saying the rules don’t need to change. But you know, what does

Isha Marathe 31:20
You know, there’s two things. One is really keeping an eye on the AI detection technology that’s coming out and incorporating it as quickly as you can. And by you, I mean, courts and ediscovery professionals. And the other thing is really, and this is the slower process, but if there was a federal kind of federal statute that would make AI image generation companies watermark their output. That could be another thing that would be super helpful in this fight against deep fakes. I. But yeah, that’s what I think other than education,

Marlene Gebauer 32:03
and triple damages.

Greg Lambert 32:05
Yeah. Yeah, I was gonna say come down hard on the first first one that gets caught. So Isha, we’re at the point now, where we ask all of our guests, our crystal ball questions. So we’re going to ask you to pull out your crystal ball and peer into the future for us. And so what do you see happening with a deep fakes and its effects on the legal system, say over the next two to five years,

Isha Marathe 32:30
Hmmm, let me think about this. So I think I really think that we haven’t seen deep fakes affect litigation to a significant extent yet. But the dam is going to break I think, and, you know, courts and counsel and juries and ediscovery professionals, if they’re unprepared, I think it has a real risk to to the judicial process. That’s one prediction. And at the same time, I think that like data privacy statutes, we are going to see a patchwork of conflicting, but a patchwork nonetheless of laws that that regulate AI generated images and audio and video. And I also think we will have sophisticated detection software. I really do. But I think that the system is inequitable, and so some courts will have it and some courts won’t.

Marlene Gebauer 33:28
Well, I’ll leave it on that. Isha Marathi tech reporter with ALM Thank you very much for taking the time to share insights on the effects of deep fakes with us.

Isha Marathe 33:37
I had such a good time talking to you guys. Thank you so much for having me. And of course, it’s

Greg Lambert 33:41
still scared though.

Isha Marathe 33:44
I know I hate like I always, you know, I always used to watch these Read, read news articles and be like, Oh, I hate the alarmist. But now, I fear I sound like

Marlene Gebauer 33:55
it seems like every day is just another alarm that’s going off. So, but thank you for sharing this with us and educating us in this area, of course. And of course, thanks to you, our audience for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 34:17
And I can be reached @glambert on Twitter Isha if someone wants to reach out to you where’s the best place?

Isha Marathe 34:24
It’s also on Twitter at @IMarathe

Marlene Gebauer 34:29
e and listeners. You can also leave us a voicemail on our Deacon review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 34:42
Thanks, Jerry. All right tomorrow and Happy anniversary.

Marlene Gebauer 34:44
Thank you. You too.

Our guest this week is ⁠Kristina Satkunas⁠, Director of Analytic Consulting at LexisNexis. Kristina discusses the recently released LexisNexis CounselLink Enterprise Legal Management Trends Report for 2023. This annual report provides insights and benchmarks on key metrics related to corporate legal spending and outside counsel relationships.

The 2023 report found that law firm hourly rates increased 4.5% over the past year, the highest year-over-year increase in the 10 years LexisNexis has published the report. While rate increases are not surprising, the magnitude is noteworthy. Kris attributes the largest drivers of the increase to economic factors like inflation as well as lower demand for certain types of legal work. However, average blended rates (the rates charged for entire matters rather than individual timekeepers) remained relatively flat. This suggests in-house counsel are mitigating rate hikes by changing the mix of firms, timekeepers, and types of timekeepers working their matters.

The report also found the ongoing trend of consolidation to fewer outside firms continues, with 61% of companies using 10 or fewer firms for 80% of their legal spending. Kristina expects this trend to remain relatively stable but notes there are benefits to using both a smaller number of firms (e.g. better rates, stronger relationships) and a larger number of firms (e.g. subject matter expertise, competitive rates). She recommends companies determine when to use large firms versus smaller or midsize firms based on factors like matter complexity, risk profile, and cost.

Alternative fee arrangements (AFAs) have not gained significant traction according to the report, remaining at about 12% of matters. Kristina is an advocate for wider AFA adoption and believes companies need to ask for and consider AFA proposals, especially for appropriate matters. AFAs can help buffer rising hourly rates. She acknowledges AFAs require effort to evaluate and implement but thinks legal operations teams and outside counsel should work together using data and analytics to develop reasonable AFA proposals.

The report provides new data on international lawyer rates in 22 countries. Rates differ significantly between countries based on factors like a country’s economy, political stability, and role in global trade and commerce. Many companies are leveraging international firms for regulatory, litigation, IP, and other legal needs outside the U.S. Benchmark data on rates in different countries provides helpful context, especially when engaging firms in new countries.

Kristina sees two significant changes on the horizon:

1. Determining how to properly and effectively employ AI and technology to increase efficiency and reduce costs; and
2. Continued access to data enabling both in-house and outside counsel to make smarter, data-driven decisions.

When asked what metric in-house and outside counsel should focus on, Kristina recommends using available data, whether from the survey or a company’s own systems. Data is a “two-way street” that should be shared collaboratively to improve decision making.

Links

• ⁠CounselLink® 2023 Trends Report: In-depth Perspective on Rising Outside Counsel Billing Rates⁠ 
• ⁠CounselLink⁠

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify⁠

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠ Voicemail: 713-487-7821 Email: geekinreviewpodcast@gmail.com Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript⁠

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. So Marlene, I was listening to a recent podcast on Stephen Poor’s Pioneers and Pathfinders where Steve was interviewing Natalie Anne Knowlton. And there was something that Natalie had said during that episode, I think is very relevant to the conversation that we’re about to have. And that is that the legal world is it uses a lot of anecdotal stories to describe how things are going on, especially in the operations of law and those relationships between corporate counsel and law firms. But based on what she was saying, we really need to do the research and collect the data, and use the data to back up the stories that were telling.

Marlene Gebauer 0:59
Well, our guest this week is also very much a proponent of analyzing the data to describe what’s happening in the legal market, and has recently produced a Trends report focused on what’s happening with outside counsel rates.

Greg Lambert 1:11
So we’d like to welcome Kristina Satkunas, Director of analytic Consulting at LexisNexis. Kris, welcome to The Geek in Review.

Kristina Satkunas 1:11
Yeah, thanks to you both for hosting me. I’m glad to be here.

Marlene Gebauer 1:23
Kris, the 2023 edition of counsel link enterprise legal management Trends report was released recently, and you titled it insights into seven key metrics in depth perspective on rising outside counsel rates, we wanted to bring you in and talk about this year’s report and dive into some of those numbers. But before we do, would you mind giving us a little bit of background on what the annual report measures and what the objective is for council link when it compiles the report each year?

Kristina Satkunas 1:53
Yeah, I’d be I’d be glad to do that. There are seven as the title would imply, there are seven key metrics that we track every year. And most of them are a cut of hourly rates. So hourly rates by practice area, or by law, firm size, by geography, etc. But there are two that are not rate related at all. So there are seven seven metrics that we have consistently tracked over those years so that we can track the trends of them. And then as to the purpose of the report, we’re sitting on a gold mine of useful data, or we have a benchmarking database that has well over $50 billion in legal spending, and that data spans a broad set of industries and practices. And within you know, my own 20 years, or 20 plus years at this point in the legal industry, one thing I’ve learned is that people want benchmarks. They want to know if what they are doing, or what they’re experiencing is like what everybody else is. So we mined the data to be able to share insightful benchmarks to others in the industry. And then I think there’s a side benefit that comes from being able to see analytics presented in a different way so that readers can have ideas of new ways for them to think about their own data. So for instance, we present both timekeeper rate statistics and then also blended matter rate statistics and show how that data can be presented. So I think that’s useful for people to think about with their own data.

Greg Lambert 3:38
And, Kris, how long have you been doing this specific report?

Kristina Satkunas 3:42
We’ve been doing this reports counseling has been doing this report for 10 years. This is the this is the 10th year of producing the the trends report. So it feels like a good time to to really be broadcasting then the value of and some of the changes that we’ve seen.

Greg Lambert 3:58
Yeah. So in the most recent report, you found that law firm hourly rates have increased significantly. I don’t think anyone’s too shocked by that. But what are some of the key drivers that you’re finding behind those rates and our clients seeing a value in return for the hike in the rates? Yeah, that’s

Kristina Satkunas 4:20
it’s a, it’s definitely two different questions. And I’ll try to I’ll try to hit hit both of them. So the the increase that we saw in the past year was the highest that we’ve seen in the 10 years that we’ve produced the report. And you’re right, I don’t think anybody surprised by it. We certainly heard anecdotally that rates were increasing at super high levels, that I heard from a number of our customers that they were seeing double digit rate increases. So So while the average that we report in the in the trends report is 4.5% was the average partner rate increase So that actually, I think 25% of partners actually had those double digit rate increases. So I think oftentimes, we get skewed by the by the average, but there were a lot of much larger increases out there. So as to why, why last year? Why why the largest increase that we’ve seen? You know, I think there are a number of drivers, their economic drivers the effects of inflation. Demand is certainly a part of it. Right? Lower m&a demand, other corporate types of legal work that we’re hearing about our I wouldn’t say drying up but but are certainly dropping in terms of demand. And as you both know, firms typically go to two levers to compensate for lower revenue and lower demand. They raise rates and or they reduce headcount. And I think we’re seeing or hearing about both of those things today. And as to clients seeing value. I think that it’s fair to speculate that some do, and some don’t. But the interesting thing that that is a separate metric in the report is that at least at a macro level, clients aren’t really feeling the pain of individual timekeeper rate increases. So another key finding in this year’s report is that blended matter rates were pretty flat relative to the prior year. So what’s happening is that customers are changing the mix in order to offset individual rate increases. So it’s either the mix of the firm’s they’re engaging, or the specific timekeepers, that are working on their matters, or the types of timekeepers, you know, partners, versus senior associates versus sivut, Junior associates, etc. So there are ways to mitigate the rate increases. And I think that’s an interesting finding to this year, as report

Marlene Gebauer 7:04
did investment or use of technology? Did that have any impact on the higher rates?

Kristina Satkunas 7:10
No, I don’t think so. So I think that it would be the opposite, right? Like we would expect investment of technology to help drive efficiencies, which, if anything, like drive down some rates? So I don’t I don’t think we’re seeing any effect at this point of investment in technology. But hopefully, hopefully, that’s a future thing specific to AI in particular.

Greg Lambert 7:32
I’ve heard people arguing the opposite. And that if the efficiencies go up, the rates go up. Because basically, they could be wrong. But I’ve been hearing arguments on both sides that efficiencies will drive down cost, whether that be maintaining rates, or if time drive up cost in Drive downtime. So it’s going to be interesting to watch how that how that plays out over the long run.

Marlene Gebauer 8:01
it’s gonna be very interesting to see how the generative AI impacts that.

Kristina Satkunas 8:06
Yeah, I agree. I agree with both of you. And it will be interesting to see what if any of the responses from corporate counsel, as if in fact, rates do go up as a result?

Marlene Gebauer 8:17
Kris, you found major differences in rate increases between practice areas? Why did rates increase so much more in areas like m&a versus others? What’s driving that demand?

Kristina Satkunas 8:31
I think the answer to that is that high value work tends to get the least pushback from corporate clients. So be it, m&a work, other types of corporate work, regulatory and compliance is a good example. Right matters in those categories are often deemed, as you know, either high risk, highly complex, critical, whatever the term is, that’s going to be used to categorize them. And nobody wants to haggle over hourly rates when the outcome is that crucial to the business. So even though demand is dropping in some of those categories, I think firms can continue to raise rates without fear of significant pushback. I mean, to to a limit, of course, but I think the the the perceived value of that work is is going to continue to fuel an increase in those rates.

Greg Lambert 9:26
And Kris, you had mentioned earlier about the blended rates, and those going up a little more modestly than the hourly rates. So I want you to kind of expand on on that. So how are the legal departments managing those costs in light of the higher rates? And are you finding any of the strategies that are there more effective than others?

Kristina Satkunas 9:50
Yeah, I think that one of the shifts that I’ve observed in the 10 or so years, I’ve worked with corporate counsel is that they’re getting savvier. and corporate legal departments have realized that while we often focused on individual timekeeper rates and approving those rate increases that are being asked for that total fees are the product of rates and hours and who is billing those hours. So legal departments can make choices to shift to different firms. And to make sure that high rate timekeepers aren’t doing work that somebody else perhaps could be performing. They can also ask for budgets, they can ask for staffing lengths if they really want to manage it closely. And the data suggests that this is happening, because the mix is shifting. And as to I guess, one other one other thing as to strategy. So the the mix is certainly a strategy paying attention to the mix. But I’d be remiss not to point out that if any sort of fixed fee, AFA arrangements are put in place, right, who cares at that point about hourly rates? Right, it doesn’t matter how much they’re going up if you’ve negotiated a a reasonable flat fee. So despite the fact that AFS are not being broadly adopted enough, in my opinion, they are still a tremendously effective strategy when dealing with soaring rates like we’re seeing.

Marlene Gebauer 11:22
So law firm consolidation is continuing, and with most companies using 10 firms or less for 80% of the spend, and you know, I imagine, you know, it’s it’s probably easier managing fewer panel firms, maybe you can, you know, do something with alternative fees across the board with a firm, what are some things that are fueling the ongoing consolidation? And do you expect this trend to continue?

Kristina Satkunas 11:53
Well, you’ve already so you’ve touched on a couple of reasons that, that that trend is, is there that it’s just easier to manage fewer firms, right, that they’re that they understand how, what the policies are of a given organization, under bills are hopefully going to be more in compliance with your guidelines, those things of that nature. But, but also, I think that one of the biggest benefits of consolidating to a relatively small number of firms is that those firms are going to have both a deeper and a broader understanding of a given clients work of their business, which should lead to better outcomes, as well as efficiency. And then I think, well, you touched on this already, that from a cost perspective, you should also be able to negotiate better fees be that better discounts on hourly rates, or AFS, because you’re more comfortable with the relationships that you that you have with those firms. So that increase volume should give you some negotiating power. And as to the trend itself, so in this most recent report, 61% of our customers use 10 firms or fewer for 80% of their of their of their work at their spend 10 years ago, when we started this report, the metric was 54%. So it’s bumped up to you know, somewhere around 60%. And it’s actually been pretty stable in the past several years, you know, it may shift up or down by a percentage point. And I think that most corporations have gone through this exercise of consolidation. And I don’t really expect to see it increased significantly at this point, because there are there are lots of industries where it’s just not possible. So take insurance carriers, for instance, they require firms in multiple jurisdictions, so they’re never going to consolidate their legal work to just a few firms. So I think we’re fairly stable, maybe it will go up just a little bit more, but but I don’t expect it to go up much beyond that. 60% It’s

Marlene Gebauer 13:56
interesting, because, you know, different firms are good at certain things as as opposed to others are focused on certain things as opposed to others. And, you know, you have boutique firms that, that are, you know, have a lot of expertise in, in certain types of areas. And yet, you know, you see this consolidation, you know, just it’s just kind of interesting that the ease and costs, and some of the factors that you mentioned, you know, seem to be seem to be important to firms as opposed to kind of picking and choosing, you know, who’s the absolute best?

Kristina Satkunas 14:34
Yeah, that’s the, I think there’s, there’s the question of who is the absolute best and also, could I potentially get a better rate by going to a smaller firm? So there are there are definite trade offs to having this strategy in place. You know, my my advice to our customers is to make sure on the pricing side that they really thought about the strategy, you know, when do you need when do you need to go To the largest firms that are out there, perhaps Perhaps there’s certain types of work where you could go down to the next tier of firms and still get just as just as high a quality of advice, but for a for a slightly lower rate. So I think there’s a balance. And that’s the other reason that that I don’t think we’ll see it go much above 60% Is that many, many feel like they need that they actually do better by having a bigger number of firms within their pool that to choose from. So it’s a great, it’s an interesting question. And I think that we’re maybe we we’ve hit that balance of having a small number of firms versus a slightly larger number of firms on their panels,

Marlene Gebauer 15:41
Do you see perhaps as a trend sort of going more to sort of the the smaller or the mid level firms and sort of saving the, you know, bet the ranch type of stuff for the larger ones.

Kristina Satkunas 15:51
So I haven’t seen it, you know, we one of the statistics that we track, as part of the research I do in the trends report is what percentage of work is going to the very largest firms versus the next year of firms. And overall, it really hasn’t shifted much. And I look at that by practice area as well. So we haven’t seen it to date in the data. Will that happen in the future? Again, I think people are getting savvier. And they’re thinking about their strategy. So it wouldn’t surprise me if we saw some strategy of of thinking about when it would be better to engage other other types of firms besides the the biggest in our country. Yep.

Greg Lambert 16:31
Kristina, you had highlighted earlier that alternative fee arrangements or AF A is a really kind of a good strategy long term to kind of buffer those rate increases. But the amount of AFS has stayed pretty steady for a number of years at around 12% of matters. So it from what you’re saying, Why aren’t these AFA is gaining more traction? And what do you think needs to change for it for it to do so?

Kristina Satkunas 17:03
I’m not gonna lie. And maybe picked up on this from my answer to the other question. This metric is a metric is a bummer, I think to the industry. I’ve been in legal for a long time.

Greg Lambert 17:15
It sounded like you were pro AFA, but I didn’t want to I didn’t want to come right out and say,

Kristina Satkunas 17:20
I’m definitely pro AFA. And I think, you know, both on the law firm side, I worked, you know, for a good 10 years or so with large law firms. And now with corporate counsel for the last 10 or so years, and both sides claimed to have this tremendous interest in adopting non traditional billing. But the numbers speak for themselves, right the needle. Maybe it’s moving just a little bit, but it’s it really, for all intents and purposes hasn’t moved in the last 10 years. So why I’m speculating a little bit here. But you know, some of this is based on what I learned from our own customers. So some of it I think, is staying with the devil you know, versus the devil you don’t write that’s that’s one reason. Practically speaking, I think the reason AFA is aren’t used more is that it’s one less thing to think about. And if you think about departments that are staffed leanly legal departments that are staffed very leanly thinking through whether a proposed AFA is appropriate and reasonable, that takes time and also takes a certain skill set. So while legal departments have moved in recent years toward having Legal Operations functions, and some of them include people with analytical strengths and those skill sets, but many don’t, right, and that I think that would certainly help in my opinion, to have people who have the, the know how and the instincts to to break down and analyze an AFA. And then another factor is that AF A’s depending on the type of AFA can be harder to demonstrate the value of specifically, I’ve talked to procurement professionals who are involved in the fee negotiation process. And many of them prefer to stick with negotiating hourly rates because it’s easy to say, you know, look, I negotiated that lawyer’s rate from $2,000 an hour to $1,500 an hour and by doing so I saved the business X amount, right? That’s a really easy way to show my value as somebody who’s a procurement individual, but it’s not as simple with non hourly billing and and I think in order to try to demonstrate the value than many that do engage in a ifas go down the unfortunate path of requiring shadow billing, which takes away from the simplicity and the efficiency, which is one of the key reasons to have an AFA. So that’s lots of reasons maybe more than more than you wanted to hear, but I will say on a positive note If we do see non hourly billing across the board, right in all practices, all sizes of law firms are engaging with them to some extent. And I think that clients just need to ask they need to work with their most trusted law firms to be creative, and then use their historic data to validate whether a proposed AFA is reasonable. So I do I’m still hopeful, I’m disappointed. But I’m still hopeful that we will start to see a face pickup.

Marlene Gebauer 20:33
Surprisingly, high numbers of lawyers build minimal hours on matters. How can legal departments better manage staffing to address that issue? You know, what should they watch out for? And outside counsel billing practices?

Kristina Satkunas 20:46
Yeah, I included that metric about how many lawyers bill and then how many hours they typically bill, this is the first time in this Trends report that that I analyze that data. And it’s actually supplemental analysis to the seven key metrics that, you know, depending on the feedback that we get to this year’s report, we may, we may choose to include that every year and move on to move to having eight key metrics that we start measuring beginning with this year. And I did that analysis for a few reasons. One is that it’s been 10 years felt like it would be nice to have something new to analyze and look at. And then also because I I’ve actually had several clients asked me about that metric, you know, what’s what’s typical? How many lawyers should I expect to bill on this type of a matter? So, so So as I said, first time that we’ve done that analysis, and I have to say the results were surprising to me to, on average, matters, on average matters have 37% of lawyers billing fewer than five hours over the lifetime of the matter. And I should state that that’s only looking that’s only using a population of matters that have material spent, I use 25,000, is that threshold, right? So I looked at matters that have gone through their full lifecycle and have accumulated at least $25,000 in spend. And of those 30 37% of those matters, excuse me, of those matters. On average, 30% of lawyers Bill fewer than five hours over the lifetime of the matter. And it’s worth stating to that that’s just lawyers, right, we know that there are many other types of timekeepers, that are billing on those matters, I’d focus just on lawyers. So these these are indeed a high portion of lawyers that don’t do a whole lot of billing. So as to what to as to what they should watch out for what what should legal departments be paying attention to. This data is readily available for any legal department requiring a billing, which is certainly the vast majority of companies of any size these days. And my suggestion would be I’d start out by just monitoring the largest matters, right? monitor those high profile, large matters, pay attention to how many people are billing and how much they’re billing and then it comes down to communication, as so many things do. Setting expectations with, with law firms, and asking questions when the numbers of billers start piling up. One other thing I want to mention, related to this is that automation can help. Right I can only speak for counseling, because that’s who I work for. But for example, counseling can flag a new biller on a matters invoice. And that can lead to a conversation with whoever is looking at that invoice to say, well, there’s a we already have 50 people billing on this matter, why did a new one suddenly pop up? Right. Also automation related? Our customers can require staffing lists, which will enforce that only pre approved timekeepers are billing on a given matter so so in addition to kind of auditing and having some sort of analytics and reporting to be able to track what’s going on automation can can make a big difference.

Greg Lambert 24:23
Or our friend Toby Brown would would also point out that there needs to be better scope on the on the matters so that you don’t see this or if it Yeah, and I guess a prime example which would be legit is let’s say you’re doing some kind of deal. And then there’s a tax issue. Well, there might be a tax lawyer that comes in that gives two or three hours of advice on that particular issue. Is that in scope outs out of scope that you know, those sorts of things? And of course, you know communication everybody’s terrible at talking to one another apparently.

Kristina Satkunas 24:56
Yeah, that’s it’s a really good point. Right. So I so I I started speaking about the numbers. But there are certainly legitimate reasons that people that lawyers need to be brought on to a matter for just a short period of time. But, you know, to your point, it comes down to a conversation. So make sure that make sure that you that you understand the reason for that, and, and are communicating expectations and what is in and out of scope. The related thing to scope, of course, is budgets, too. That’s the other tool that that both sides can use, let’s create a budget and be specific in that budget and talk about who needs to be involved in the matter in order to be able to hit those budget numbers. So yep, good, really good point.

Greg Lambert 25:38
Speaking of adding some new data to the list, you provided some new data on international lawyer rates this time around? So what were some of the key drivers behind the different rates between countries? And our, our and how are US companies leveraging international firms and their work?

Kristina Satkunas 26:00
Yeah, so this is the third year that we’ve actually included non US rates in the trends report, but each year, we’ve added additional countries and or practice areas. So so we have reported on on international rates for quite a few years. But I did expand that a little bit this year, as we have more and more volume to be able to speak to. So I can’t really speak to what drives specific individual country rates, you know, differences are definitely going to be affected by by economies of that country of those countries. political stability, you know, multiple factors are going to come into play as to as to why the rates in India are much lower, for instance, and the rates in Japan, right, but But as to how international firms are being leveraged. Many of our customers need outside counsel advice outside the US for multiple reasons, you know, maybe regulatory advice in a specific country, I litigation might pop up from a specific country, patents is a really big reason you want to file a patent in a particular country. So you need some advice within those countries. In fact, 10% of the spend flowing through counseling is from non US law firms. So it really, it’s definitely material, and it’s growing. So we knew that more and more of our customers are needing to seek outside, outside the US advice. The benchmarks, I think, are super valuable to a company who either has never needed advice from from a firm outside the US or just needs to engage a firm from a country they haven’t used before. So you know, say suddenly, I need a lawyer in Brazil. And now I can look at those benchmarks and know that the rate that is being proposed to me is at least within a reasonable range. So again, I think the value of benchmarks in general is, is having a comfort level that something is is normal or a similar to what to what others are paying. So I expect to see more and more of the international data coming through counsel Inc. And we will continue to expand on what we include in the in the trends report.

Marlene Gebauer 28:28
So if you compare today’s data to five to 10 years ago, what do you see are the biggest shifts in how companies are managing costs and working with outside counsel?

Kristina Satkunas 28:41
So as you both know, our industry moves slowly. So there aren’t huge changes that I’ve seen over those 10 years. But you know, there is glacial glacial glacial movement, and, and even 10 years of data, but there are some if I go back 10 years ago, then I do start to find some differences, like I mentioned, the 54% consolidation to 10 firms or fewer versus 60. Right, that doesn’t, that’s actually pretty significant over the course of 10 years. So that’s one, I think, slightly more consolidation that we’ve seen in the last 10 years. And then that increased attention to timekeeper MCs. You know, I think I really feel like in the years that I’ve worked with corporate counsel, it’s always been about timekeeper rates, individual rates, I’m not going to pay this much for that person, but anecdotally, I am I’m hearing and we’re seeing the data, especially this year, that people are paying attention to that mix. And anecdotally, the other thing I would say that I hear more now than I used to from our customers is a greater willingness to change firms, or even those that they’ve had long standing relationships with and part of that is And this is outside of the scope of that Trends report. But I think that many legal departments are taking more things into account when they consider whether those relationships should continue. So Diversity is definitely one within the last few years that while I would say every customer I talked to said it was important five years ago, now I’m hearing that they will actually move, leave work away from a firm that is not improving or showing efforts to improve and Diversity. So So yes, absolutely some some shifts just awfully slow within our within our industry.

Greg Lambert 30:39
So Kris, if you were to present this in person to a group of in house attorneys and outside counsel, and you were to hand each one a copy of this, what would you tell the in house counsel, what they should know? What’s the number one thing they should look at? And what what would you tell the law firm leader, the number one thing that they should look at?

Kristina Satkunas 31:02
So I think my answer to that question is a high level answer to that question. It’s really about using data. And this is where, where I geek out, when I talk to our customers, I tell them, my job is to help you use data to improve all of the decisions that you make within your departments. And so here in the trends report, we are providing external benchmarks. But in house counsel, again, assuming that they have some sort of enterprise legal management system in place, they have their own data to benchmark internally. And that’s really even more important than looking externally right as my own are the rates that I’m paying across the spectrum of firms that I work with consistent am I paying rates that are increasing a lot every year and then sure that external benchmarks are valuable, but that using your own data is the place to start. And then the same applies to law firms, right? So if we’re talking, we’re talking about a partnership here where corporate counsel rely on their outside counsel to help advise them. So they have this internal data that they can be using, but law firms have their own internal data. So as an example, you know, going back to AFA is I’ll just keep beating my AFA drum. So if a given client says we’ve got no experience in using a ifas, at all, right, and should we use it for employment work, for instance, and a law firm that they partner with can say, you know, 20% of our customers who are in your industry are using flat fees for immigration work, or whatever it might be. And this is something you should continue that you should consider because our data helps support this, right. And so as long as you’re within the bounds of privacy, sharing that information, data really is a two way street, I would love to see that data being used proactively and collaboratively between in house and outside counsel,

Marlene Gebauer 33:05
Kris, so now’s the time where we ask our crystal ball question. So you pull out your crystal ball and you look into the future. And tell us you know, what changes or challenges do you see over the next two to five years?

Kristina Satkunas 33:20
Well, you kind of teed me up earlier to do this, but I can’t walk away from this discussion as a LexisNexis employee without mentioning AI. Right, figuring out I think figuring out where AI is figuring out where AI can and should play a role in increasing efficiency, and ultimately reducing costs, in my opinion, is going to be a focus for for many, it’ll be an interesting path to see how we get there. And I think we’re at the very, very, very beginning of that path. But hopefully, within five years, many will have have have figured that out as to what role AI can and should be playing. And then I think the broader change, I think we’ll see is that data, and the and having access to data continues to make us all smarter.

Greg Lambert 34:16
All right, well, two very good predictions, I think. So. Chris, Kristina Satkunas, Director of analytic Consulting at LexisNexis. Thank you very much for coming in and discussing the survey with us today. This has been fun. Thank you. My pleasure.

Marlene Gebauer 34:31
Thank you enjoyed it. And of course, listeners, thanks to all of you for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 34:47
And I can be reached @glambert on Twitter, Kristina, how can people reach out to you if they want to learn more?

Kristina Satkunas 34:54
They can find me on LinkedIn that probably be them the easiest way.

Greg Lambert 34:58
All right. And we’ll also make sure that we put a link to the survey as well on the show notes as well.

Marlene Gebauer 35:06
Listeners if you have questions or comments, you can also leave us a voicemail and our geek and review Hotline at 713-487-7821 and as always, the beer is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 35:19
Thanks, Jerry.

Marlene Gebauer 35:20
Okay, bye bye

This week on The Geek in Review, Marlene Gebauer and Greg Lambert talk with ⁠Curt Meltzer⁠, principal of ⁠Meltzer Consulting, LLC⁠. Meltzer has over 40 years of experience in the legal and legal tech industry. He discusses his interest in pro bono and community outreach programs in law firms and legal tech companies. He notes that while 95% of AmLaw 200 law firms highlight pro bono work on their websites, many legal tech companies do not prioritize these efforts.

Meltzer emphasizes that pro bono and community work is good for business. It enhances company culture, helps with recruiting and retaining top talent, and strengthens customer relationships. He argues that legal tech companies should consider emulating their law firm clients’ community programs. This could include donating software or services, allowing employees paid time off for volunteer work, or collaborating directly with organizations that law firm clients support.

Meltzer highlights LexisNexis and Thomson Reuters as leaders in the legal tech industry for their work promoting access to justice and the rule of law around the world. However, he notes that companies of any size can contribute, whether through recognizing employees who volunteer or donating resources. He published a list of 41 legal tech companies that do highlight community outreach on their websites to raise awareness, though he found 39 companies with no mention of such efforts.

Meltzer sees both opportunities and challenges ahead. Private equity investment in legal tech companies may prioritize short-term profits over community programs. However, companies that do not respond to customer interest in their pro bono and corporate social responsibility initiatives risk losing business to competitors. Overall, Meltzer aims to foster conversations about strengthening the relationship between the legal tech community and the broader community. Corporations that embrace ESG programs and give back to the communities they serve will thrive.

Listen on mobile platforms:  ⁠Apple Podcasts⁠ |  ⁠Spotify

⁠Links:⁠

• ⁠Legal Tech Vendors Give Back List⁠
• ⁠Driving Social Responsibility⁠

⁠

Contact Us:

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert⁠⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠

⁠⁠⁠Transcript⁠

⁠

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer,

Greg Lambert 0:14
And I’m Greg Lambert. So Marlene, I can tell that it’s summertime because one we’re getting those afternoon thunder showers. So you may hear some thunder in the background here. But the biggest way I can tell is that, you know, I get up in the morning go to work and my family who works at public school system, or just sound asleep still as I as I walk out, and it will be that way until August until they go back. Usually it’s the other way around users. They’re, they’re out the door before I am.

Marlene Gebauer 0:44
Yeah, that sounds like my family too. It’s up. They don’t work in the public school system. They just attend the public school system.

Greg Lambert 0:53
Well, we have a longtime friend on the podcast this week, Curt Meltzer, who is principal of Meltzer Consulting, LLC, and formally with organizations like Litera, Orrick, Norton, Rose Fulbright, I’m sure there’s a few more that we don’t want to necessarily run down the entire list. But, Curt, welcome to The Geek in Review. It’s good to see you.

Curt Meltzer 1:17
Thanks, Greg. Marlene, I really appreciate the opportunity to speak with you today.

Marlene Gebauer 1:21
So Curt, you wrote a couple of LinkedIn articles recently, where you point out that 95% of the amla 200 firms have dedicated websites for their pro bono access to justice and community outreach efforts. In addition, many ediscovery vendors and legal tech companies are also involved in supporting their communities. But there are some companies and legal tech who do not prioritize giving back the same way that law firms and corporations do. So what was your motivation to start looking at this area of pro bono and community outreach efforts in the legal industry?

Curt Meltzer 1:56
Yeah, as you as you noted, I’ve been in the legal tech industry a long time, over 40 years, and I’ve been about half the time on the law firm side and about half the time on the vendor side. So I’ve got a perspective based on participating in both. And while I was at these law firms, I was worked at five different law firms, every single one of them had extensive pro bono programs. And they inveigh engaged in various community projects as well. I volunteered time working at local schools in Minneapolis through one of these firms, and participated and other community activities as well. But of the six legal tech vendors, I work for only one, the largest one of those LexisNexis ever did anything remotely along those lines. So it was, you know, presented a pretty significant contrast in my mind, and made me wonder, why is that so different. And a lot of this has come about by the recent growth in the ESG movement, particularly within larger corporations, who have noted the value to their companies about their engagement with the community, it’s just good business, it’s very simple. It’s good business, there are many studies that have shown the financial performance is improved. Once corporations embrace an ESG program, once their own company’s b2b, customer relationships are enhanced, because of mutual involvement in these activities, they also find a tremendous increase in attracting and retaining employees. And it increases employee productivity and their motivation as employees feel increasingly good about their employer. So there’s a lot of good reasons for every company to be doing these things. And obviously, some have more resources to devote to that than others. But there is benefit associated with everybody. And I see a big gap in the legal tech community. And this is perhaps my way of trying to get back by alerting people and making them aware of it, and try to get a little more activity on the other vendors suck.

Greg Lambert 4:04
This is probably an obvious question here. But do you think the law firms are doing it only because this is something that basically the Bar Association’s kind of require us to do that there, that there has to be so many hours of community, you know, working in community, or pro bono work? And there’s not necessarily that same requirement on the tech side? And let me let me spend it another way. If we didn’t have that requirement, would we still would we be more like the legal tech companies and and not not doing it because we don’t have some regulatory board looking over our shoulder?

Curt Meltzer 4:44
Well, I’m a glass half full kind of person. So I believe that law firms would do it regardless. I think it’s part of the tradition of the legal profession is to give back it’s not a requirement that not a regulated industry, but they want to do it And, I think provides a lot of fulfillment to lawyers, many people in law firms who are engaged in pro bono activities have more client engagement with their pro bono clients than they do with their corporate clients. And they see things through to fruition. And, and it, it’s, it’s a different motivation in that in most of large law firm work, you’re helping, rich people get richer. But in pro bono work, you’re helping people with really serious personal issues, relieve those issues and help them have better lives. And while it may be, there may be some economic incentives to do that. There’s a different level of satisfaction you get from doing that sort of work. And I think that there’s enough of this in the traditional legal profession, that it will carry on, no matter what Bar Association’s might require. And secondarily, and maybe even more important, in some cases, clients are asking law firms about their pro bono activities, they want to know they want to encourage them to do more, and they may choose to work with law firms that do more for the community than those that do not. It all depends on their requirements. But there are a lot of incentives to have law firms continue to get back to their community.

Marlene Gebauer 6:21
Do you see this? Maybe changing from a generational standpoint? You know, there’s studies that, you know, certain generations like that that is more important to them in terms of, you know, giving back to the community. Do you see that there’s any trend, or will be any trend the other way?

Curt Meltzer 6:38
I think it’s very clear that there’s a trend in that direction. And that’s why corporations have been growing their ESG programs in recent years. It’s I’m a member of the baby boomer generation, don’t hate me. And I hear I know that there’s very little other than what may be a minimum requirement or minimum obligation field for majority of people in that generation. But as as we reach younger generations, like millennials, in particular, they are doing a lot more for the community, because a lot of proof of that. So maybe it’s just a matter of time, but I’m not patient enough to wait.

Marlene Gebauer 7:23
Well, what are some specific examples of community outreach efforts that have been undertaken by large law firms.

Curt Meltzer 7:32
So there’s lots as we’ve talked about the 95% of the ABA, 200 least that I could find, emphasize their pro bono programs, and many of them emphasize it in their career pages. So they’re trying to recruit people who are interested in giving back to the community. Most large law firms offer paid days, or hours of service to their employees. Many, in fact, use Martin Luther King day as a day of service. And they’ll do things as as groups or, you know, an office of the firm will do sing something together on that day, that’s become increasingly popular in recent times. And many of the efforts beyond simple pro bono work, focus on causes that they believe in supporting and preserving civil rights, protecting the environment, or ensuring access to justice, the Access to Justice, is a huge issue for law firms, because we know that a large percentage of the population doesn’t really have access to the legal system because they can’t afford it. And finding new ways to enhance the ability to deliver legal services to those with less resources is is a very important goal for ESG. In many law firms.

Marlene Gebauer 8:51
Do you find that most of the pro bono is sort of, I would say, more traditional pro bono, which involves representation? Or are you seeing any trend towards use of technology or sort of working in technology to develop, you know, access to justice solutions, or things like that?

Curt Meltzer 9:11
There have been a number of really interesting developments on the tech side for access to justice and typically comes from one of individual legal aid organization, there’s so many, you know, 1000s of them around the country. And if one gets somebody who volunteers time to say, you know, what are the issues that are hitting your customers more often, they’ll develop something like chat bots would be great opportunity to help people answer questions they have about legal services. It is interesting that many people who are impoverished still have phones that they use to access the internet. You’ll see on the streets in New York City, the public access to the internet and charging, you’ll see homeless people charging their phones. And you know, using the service for free there because it’s almost a necessity just to live nowadays. So it’s easy to access people with relatively low income, because almost all of them have access to the internet through their phones. And, and so building applications and accessibility to these services, using these applications has really taken off in recent years. And I don’t see any end to that in the near future. I think it’s encouraging. And and with all the generative AI work being done and chatbots being developed, it’ll probably grow even more in the short time,

Greg Lambert 10:44
that would seem to be the path that we’re on. So, Curt, when you when you approach legal tech companies and you start discussing this idea, how do you approach them and say, here’s kind of the benefits that you would get as a legal tech company from by engaging into this corporate social responsibility initiatives that you’re talking about.

Curt Meltzer 11:11
I think it’s very similar to any other corporation, everybody’s motivated by increasing performance, be it revenue or profitability, or what have you. And there are many studies to show that this is the case, and also saves you a lot of money by enhancing your employee relationships, increasing retention, enhancing your recruiting capabilities, getting and retaining employees is very expensive. And anything you can do to help that will save the company a lot of money. And but in terms of the disparity between law firms and legal tech vendors, it’s really easy to have the conversation to say, look at what your customers are doing. Who are your 10 biggest customers look at their websites, you’re more likely to see what they’re engaged with. And think about how can you collaborate with them hand in hand to support the causes that they believe in? So those are ready made? You know, nobody’s gonna say no. If you’re a software company, you can donate software to whatever organizations they’re working with. Many of them will have use for it, some might not. But plenty will. You can talk about the kinds of services that you deliver if you’re not a software company, and how can you al help legal aid organizations? It has their ability to run their business or what have you. So collaborating with your customers and saying, What do you guys do? We want to help you further your causes. It’s a very easy discussion to have. It’s like when I was CIO of a law firm, and a partner asked me as a CIO, what would you do to increase my business? How should I be thinking about increasing my business? And my response was very simple that any marketing person would say, go talk to your clients, ask them, sit down with them, don’t charge them, and ask them what’s going on in your business? And what are the challenges you’re coming up with and listen to them, because one of the things that clients always say the most is, my, my outside counsel doesn’t understand my business. And that’s because they don’t ask, or they want to put it on the clock to ask. But if you want to enhance your business, and create strong relationships, listen to your customers, it’s a very simple thing. So legal tech vendors should be turning around to their customers, maybe the ones that they’re having trouble with maybe the ones that they’ve let down in some way, and turn to them and say, Hey, we want to help you, in your efforts to whatever is important for or have access to justice or environmental concerns, civil rights, whatever, how can we help you in that cause and develop a relationship based on mutual interests, and that can only enhance the relationship and then the profitability for the firm as well. So it’s, it’s not complicated stuff. I think people just get wrapped up in what they’re working on and forget how simple it is to have those conversations, who’s gonna say no to having that kind of conversation? Or that a engagement? Anybody be happy to have that? So it’s, it’s really not a mystery. It’s pretty straightforward. Mix. You know, it’s just common sense.

Greg Lambert 14:35
One of the things that we started discussing you coming on to talk on the podcast is because you had written a couple of LinkedIn articles and discussed from the law firm side of things and your perspective on that. And then on the legal tech company side of things in your in your second article, promising a list of players In the industry that were doing good, good works in the second article, but what are there any particular legal tech companies that kind of stand out for their commitment for for giving back? And and how can law firms effectively drive change in their legal tech providers regarding corporate social responsibility, in other words, kind of put the pressure on maybe even start the conversation with them that we could use your help doing some either ESG or pro bono work? Yep.

Curt Meltzer 15:36
I think the two that really stood out the most in the vendor community are LexisNexis and Thomson Reuters, they really put a lot of effort into ensuring the application of the rule of law around the world. Many people outside of the US and some of the Western countries are not as fortunate as we are to live under the rule of law, their safety, livelihoods are significantly challenged, they have a lot of variables that are out of their control. And while we may think things are out of control in this country, we’re pretty safe. By and by. So these two companies work all over the world with governments and corporations, to further the rule of law. Now, if every nation filed through a law, that probably would be good for their business, too. So it’s not entirely selfless. They have a selfish motivation. But that’s great, because it works for everybody. And so that’s really a win win. But they are having worked at Lexis and I know, talk to a lot of people at Thomson Reuters, that the employees are really proud of their company’s efforts along these lines, and they’ve been sustained for many years at a great cost. And yes, those are very large corporations. But

Greg Lambert 16:53
that was gonna be my question.

Speaker 3 16:54
Yeah, they are giving back. And it’s a it’s a significant commitment to them. And their top level people talk about it routinely. It is a really a I believe, of part of their fabric. And and it’s very much admired as, as much as we might criticize them for some things from time to time. I think they both do a really great job. And Bloomberg does, as well as do many others in the legal research area.

Greg Lambert 17:21
I was just wondering, since you mentioned, really the two big players in the legal tech and legal information market? Is this something that can be scaled down for smaller companies? Or is this an excuse for smaller companies to not do it? Because they said, Well, look, you know, you need to be a Thomson Reuters or a Lexis in order to really have that type of type of capability is, do you see it that way?

Curt Meltzer 17:46
No, I think any company of any size can do that. Snell i, if we look at the low end, if we look at startups, and there are so many startups in the industry, you know, those individuals are working 100 hour weeks, and they’re losing money, it’s hard to say that they could devote some of their resources to, to giving back to the community. But at some point, if they’re making any profits at all, these companies have to start thinking about giving back. So it’s, it’s pretty easy for us, for most software companies, they can offer their product for free to any type of charitable organization, most notably legal aid organizations to enhance the access to justice. But there’s a cost associated with that, because then they have to help them implement it and support it. So yes, it will take some of their time. But they don’t have to charge for the software, right? That should be within reach of many, many companies. And again, if they’re looking for organizations that work with just turn to their customers, they could probably give them a dozen right off the top of their heads. So it’s I don’t think it’s asking a lot to do that. And the other point is that a lot of people in every walk of life volunteer their time to help their community in one way or another. Their employers should reward them for that and recognize them that for that they don’t have to give them money, but just recognition and thanking them for doing that. And then talking about it to their customers where we have X number of people volunteer y number of hours per year, and making it important part of the culture of the organization. I think almost anybody can do that doesn’t have to cost money. You don’t have to spend millions of dollars to do this. It’s just everyday people do things to give back to others. And it’s important to recognize and appreciate that and and let your customers know that you’re doing it as well. I don’t think it’s asking a lot at all.

Marlene Gebauer 19:51
Current your second article you had mentioned you were going to provide the list and you did so so you have the the list of I think it’s 41 companies and links to their community focus pages. And you encourage readers to go and check those out. What motivated you to to actually put a list together of the vendors giving back to their communities? And have you gotten any feedback on it?

Curt Meltzer 20:15
That’s a great question. Marlene, I believe that transparency is critical to foster change within any organization. And so it occurred to me that I should highlight those organizations that I could readily find, that are doing something to give back to the community. So I did find 41. But I also found 39, that do not appear, at least from their website to give back to the community, they’re not talking about it. So I decided not to include those 39 At this time, because I don’t want to embarrass anybody, I want to encourage them, you attract more flies with honey than with vinegar, right? So in the course of this process, I spoke to one vendor and said, This is what I’m gonna do. And he said, Well, we do a lot of stuff for the community. But we don’t have anything on our website about and like, why not? Why wouldn’t you do that, that’s that that’s a positive for your company. So two days later, they had a community page on their website, which was great. So I’m really about starting a conversation, I don’t think we’ll have a sea change anytime in the new near future, I think it might take a long time to really make an impact. But I thought this would be someplace to start off. And the next step really, is to have customers ask their legal tech vendors about their efforts to give back to the community. And the more people that ask, the more these companies will recognize that it’s important, and they’ll do something, and we’ll see results. And that would really be great. But you’ve got to take small steps in order to reach the goal. And so I’m at a point in my career now where I can do this, I don’t imagine I’m gonna get any income from this effort. I’m doing it because I think it’s the right thing to do. And I’m happy to be able to get back in this small way. And hopefully, you know, it’ll make a difference in in a handful of vendors in the near term. And more and more over time.

Greg Lambert 22:17
Curt, I know, I know, when you were pulling this list together, you had some concept of how you were going to kind of pick the companies that you want to look for what what kind of criteria did you use to gather, who was going to be looked at, and then also what was going to be looked at,

Curt Meltzer 22:36
I like to think of myself as a person who believes in science. But in this case, I implemented a very unscientific methodology. The first thing I did was I went to the iltacon, sponsor list. And I started at the top, and I worked my way down the list. And I looked at most of those vendors, I know I left off a few smaller ones. And then the second place I looked to was legal week exhibitor list. And I at least pulled out companies I had heard of there, a lot of companies that hadn’t heard of him, not an ediscovery expert, there’s a lot of ediscovery vendors at legal week. But I pulled out a bunch more that were not on the iltacon list. And then the third source was thinking off the top my head, what are the companies that law firms spend their technology dollars on the most? So number one is always Microsoft. Right? So let me put Microsoft in there. And then I added Dell and Cisco and many other very large corporations to list because I thought it would be interesting and enlightening for people to see, oh, what are the big guys do? And you’ll see they have very extensive coverage of their ESG efforts on their websites. Because again, it makes good sense from their business perspective. So I do not claim to be comprehensive in this list. And in fact, of the 39 where I couldn’t find anything. Maybe they do have things on their websites. And and you know, I just missed it, that might say something about how they could enhance the ability to bind it. I didn’t I didn’t go into all the press releases that may have said something about that. There are so many press releases. I didn’t I didn’t focus on that at all. But I was really looking for ESG efforts, not I’m leaving out the whole Diversity area, the DEI spaces, I think, a different discussion. And so I purposefully left that out. You know, that is also very important topic, but not part of the conversation that I’m having at this point. I’d also like to encourage any company that I may not have listed that has something to share, to reach out to me as I say in these articles, and I’d be happy to add their names to the list. The next time I update it.

Greg Lambert 24:59
As you were going through and doing our observation and investigation into the the websites, for those that you found word doing some type of social outreach, or ESG? Did you observe any kind of pattern or trends among those those vendors on what they were committing to and how they were giving back? Was that was there any trend to that?

Curt Meltzer 25:24
As I said before, obviously, the larger the company, the more likely they’re going to do something, the more b2b engagement they have, the more likely they’re going to be doing something. I think it’s interesting that most of the companies that do something, apart from the very largest ones, don’t talk about their involvement in the access to justice. But when it’s all about community activities, it might be, you know, supporting various charities in their neighborhoods, raising funds for them volunteering their time, many of the companies have a headquarters office where the largest group of their employees will be located. And they’ll work with those communities locally. And that’s all great. It’s wonderful that they would do that, you know, they’ll give their employees time off, and they’ll have employee recognition programs for their volunteer efforts. But most of them are not involved in access to justice. And again, that’s what their their customers are doing. So I think, I think they should consider how they can assist in the process, as I said earlier, by collaborating with their customers,

Marlene Gebauer 26:37
you know, you have the data on what companies are doing, you’re aware of trends and patterns that we’ve talked about. So it’s a natural next question to, to say, you know, do you envision some sort of rating system or ranking system for the community efforts? You know, what factors would you consider in this process? Or would that just kind of destroy the whole purpose of of what you’re trying to accomplish?

Curt Meltzer 27:05
It might, I think it’d be controversial. That really is a great question that I don’t have a simple answer at all. But it is something that I’ve spent some time thinking about. There are various rating systems for DEI efforts amongst law firms, some of them are more controversial than others. Some require the law firms to pay fees to participate, so that it creates less than ideal incentives, I think, for participating. But, you know, if I were to suggest something, it would probably have to vary a lot, depending on the type of products or services that each company offers, I think there’d be some sort of combination of both objective and subjective data. Simple question for software companies, that is do they offer their software for free to, to legal aid organizations or any other type of charities, research companies provide a lot of free Research Services? You know, depending on their products or services, who are they supporting? But measuring the scope of their commitment, I think is is important as well, because if it’s a one time, you know, we donated a dozen old computers to local charity, like, okay, that’s nice, but that’s not really the commitment here, we’re gonna get rid of them anyway, we have to figure out a way to rank that sort of thing. We should look at, you know, what number what percentage of the employees are participating in these programs? Or what’s the total value of the products or services donated as a percentage of your, you know, corporate gross, some people aren’t going to tell us that private companies, but they can provide those percentages. And maybe that would be meaningful to some law firms. But really, I don’t have answers to this. And I would love to have some suggestions from any of your listeners, on the best way to create a rating system, maybe they have some examples of similar things that I could turn to. And I imagine, you know, if I ever get to that point, or enough people start asking for it, then I’d have to pull a group of people together and get a diverse perspective on how to do that in as far away as possible. And I bet it’d be a lot of effort and create a bunch of controversy, but it may be worthwhile to do so at some point.

Greg Lambert 29:30
We’ve mentioned ESG a few times on this and how corporations are using that to help them do their community outreach and some of their efforts in in supporting access to justice and environmental issues, and so on. And there’s kind of a I think, a few years ago, there was a big movement to do this, and there’s kind of been a little bit of a backlash. To that, one side of one side of the coin is saying that it’s greenwashing that you’re just saying that you’re doing this, but you’re really just kind of doing busy. You know, you’re highlighting a few things while you’re, you know, kind of underplaying some of the other stuff that you’re doing. And then there’s some other folks that may be saying, Look, you’re, you’re a business, you know, stay out of this other other stuff, the kind of wishy washy stuff and do what you do best. And by doing that, that’s the best way for you to serve. Are you seeing any of that, hitting some of the some of the legal tech or even even the law firm areas, getting some back, you know, a backlash for ESG?

Curt Meltzer 30:45
I don’t think so. I think this law firm business is different because of the history of pro bono service. They’ve been giving to the community for a long time, and they strongly believe in the value that it provides, not only to them, but to their community. So very few other industries have that sort of commitment. I’m sure the medical profession has similar perspective. And they do quite a lot for the for the general public well being. But most large corporations don’t have this kind of history. And so it’s easier to question their motivations. But in the legal profession, it’s been rock solid for decades, if not over a century. So

Greg Lambert 31:35
the legal tech industry will benefit by, you know, kind of correlating or, you know, aligning themselves with the with what the full legal, you know, with the law firms and others that are in the practice of law are doing, will that kind of buffer them from some of the blowback that they may have hit more of the corporate environment?

Curt Meltzer 31:56
I think because most of the companies in the legal tech profession are private companies, as opposed to public companies, they’re not subject to as much scrutiny. And the scrutiny that they care about most is their customers. If they’re collaborating with their customers on their ESG efforts, they’ll get nothing but positive feedback. So I think it’s very different. You know, there’s a handful of public companies, and maybe their pushback will be different. But most of the legal tech professional, private, so it shouldn’t have an impact.

Greg Lambert 32:28
Well, Curt, we’re at the point now where we ask our crystal ball questions. So I want you to pull out your crystal ball and peer into the future for us. And what kind of changes or challenges in this area? Do you see over the next say, two to five years, both on on the law firm side, and on the legal tech side?

Curt Meltzer 32:51
Well, let me address the the legal tech side first, there is a sea change in going about in the legal tech industry these days. And that’s due to the amount and volume of private Equity that’s being invested in the legal tech community. There’s a lot of big PE companies that have made investments, very well known ones such as Bain Capital, and Oakley capital and Warburg Pincus. And then there’s many, many others that have made investments in legal tech. And private Equity companies are not always the best long term stewards of their member companies. They make a lot of decisions based on short term bottom line decisions. So their motivations don’t always align long term with their companies, their employees, or the customers most importantly. So that’s makes it a lot harder to try to convince these companies that they need to give back to the community. So the only way to get their attention to have is to have increasing numbers of customers, ask the vendors about their ESG efforts. And then frankly, if they don’t like the answers they’re getting, choose a competitor, and make sure that’s known to these companies. And once they see their revenue impacted, then it becomes clear that they have to do something about it. So that’s really a big motivation for me to raise awareness. Because if nobody asks, nothing’s going to happen. And as these private Equity companies increasingly come into the industry in which it seems there’s there’s no end to that insight. They’ll have to realize that this is a cost of doing business in the legal community. And they’re gonna have to consider whether or not they’re, they want to do it. So that’s kind of a cloudy future. From my perspective, I’m really unsure about that and and maybe that’ll blow over. And, you know, maybe that won’t be a big issue. But I’m very, very concerned about it. And then, for law firms who are, who have a long history of working with relatively small vendors, in the legal tech community full of very brilliant entrepreneurs that we’ve all met over the years, it’s easier to affect change among smaller companies as the size of these vendors grow. It’s harder to get them to be responsive. So I think that there’s a couple of big challenges there. And we’ll see how it affects ESG efforts in the near term. But if you don’t start asking the questions, if you don’t raise awareness, nothing’s happening. And that’s why we’re talking today. I really appreciate it.

Greg Lambert 35:51
Curt Meltzer, principal of Meltzer consulting LLC. I want to thank you very much for coming on here and talking with us here at The Geek in Review, and keep keep doing the good work.

Marlene Gebauer 36:02
Thank you, Kurt.

Curt Meltzer 36:04
Thank you both. It was a pleasure.

Marlene Gebauer 36:06
And of course, thanks to all of you, our listeners for taking the time to listen to The Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at @gebauerm on Twitter,

Greg Lambert 36:20
And I can be reached @glambert on Twitter, Kurt, if people want to continue the conversation, where can they find you online?

Curt Meltzer 36:27
I am see Meltzer on Twitter. I’m also on LinkedIn as Curt Meltzer and you could also go to my website, Meltzer consulting.com. Me LTZ er consulting.com. Thanks,

Marlene Gebauer 36:42
listeners. If you want to leave a comment or a question, you can leave it on our voicemail at The Geek in Review Hotline at 71348778 to one and as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.

Greg Lambert 36:55
Thanks, Jerry. Alright, Marlene, I’ll talk to you later.

Marlene Gebauer 36:58
Okay, bye bye

hey, hey. Welcome back. To back, devils back home. Devils back

This week we bring in Christian Lang, the CEO and founder of LEGA, a company that provides a secure platform for law firms and legal departments to safely implement and govern the use of large language models (LLMs) like Open AI’s GPT-4, Google’s Bard, and Anthropic’s Claude. Christian talks with us about why he started LEGA, the value LEGA provides to law firms and legal departments, the challenges around security, confidentiality, and other issues as LLMs become more widely used, and how LEGA helps solve those problems.

Christian started LEGA after gaining experience working with law firms through his previous company, Reynen Court. He saw an opportunity to give law firms a way to quickly implement and test LLMs while maintaining control and governance over data and compliance. LEGA provides a sandbox environment for law firms to explore different LLMs and AI tools to find use cases. The platform handles user management, policy enforcement, and auditing to give firms visibility into how the technologies are being used.

Christian believes law firms want to use technologies like LLMs but struggle with how to do so securely and in a compliant way. LEGA allows them to get started right away without a huge investment in time or money. The platform is also flexible enough to work with any model a firm wants to use. As law firms get comfortable, LEGA will allow them to scale successful use cases across the organization.

On the challenges law firms face, Christian points to Shadow IT as people will find ways to use the technologies with or without the firm’s permission. Firms need to provide good options to users or risk losing control and oversight. He also discusses the difficulty in training new lawyers as LLMs make some tasks too easy, the coming market efficiencies in legal services, and the strategic curation of knowledge that will still require human judgment.

Some potential use cases for law firms include live chatbots, document summarization, contract review, legal research, and market intelligence gathering. As models allow for more tailored data inputs, the use cases will expand further. Overall, Christian is excited for how LLMs and AI can transform the legal industry but emphasizes that strong governance and oversight are key to implementing them successfully.

Listen on mobile platforms:  Apple Podcasts |  Spotify

Contact Us:

Twitter: ⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠, or ⁠⁠⁠⁠@glambert⁠⁠⁠⁠
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠

⁠⁠Transcript⁠

Marlene Gebauer 0:07
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert 0:14
And I’m Greg Lambert. So Marlene, there’s been just a ton of things going on in the legal world involving using generative AI. You know, the tools like ChatGPT, Bard, Bing Chat, all the others that are out there. And you know, of course, the big news that everyone’s heard about over the Memorial Day weekend was the the lawyer Mr. Schwartz in New York, who apparently just copied and pasted his brief that he filed with the court straight from ChatGPT. Now, I just found out that this guy was a 30 year lawyer. He wasn’t like a newbie on this. So to me, he broke rule number one of the internet, which is don’t do anything stupid.

Marlene Gebauer 0:59
Yeah, it’s like Gen AI Gone Wild. It’s just like, really not. You know, it’s just it’s like, it’s like, just common sense that you would of course, check your work before you submit to the court.

Greg Lambert 1:11
But don’t check your work with ChatGPT is if it will double down on the lie. So the second big news that came out this week is I saw that Northern District of Texas, Judge Brantley Starr created a new rule for his court for anyone appearing before him that they must certify that if they’ve used any generative AI tools, that they have to sign it and certify that a human has reviewed the work before it was filed with the court. Any human Yes, it just has to be a human. And it actually says human in the court rule. Yeah, I have to admit, at first, this kind of pissed me off when I saw it, because I’m like, Oh, great. And I think can’t remember who it was on Twitter, it was like, well, never underestimate the ability for a judge to make something even more complicated than it needs to be. But I did have someone at my firm kind of calm me down a little bit. And just basically pointed out that, you know, this is a rule 11 thing. This is basically just reminding attorneys, that they are the ones responsible for anything that they turn in to the court. And it basically keeps the lawyers from breaking rule number one of the internet.

Marlene Gebauer 2:40
Yeah, like Mr. Mr. Schwartz is our is why we can’t have nice things.

Greg Lambert 2:43
This is why we can’t have nice things in legal. Oh, there’s so many other reasons for that as well. I am sure so. Well, you know, and the thing is, is this is really kind of put the legal world kind of on the edge or on edge at the moment because of this news. And you know, in teams are kind of breaking into into camps of your team AI versus teams of no AI. So you know, I guess Are you are you Team, sparkly vampire? Are you Team werewolf? I think that’s how we need to, to put the in but the you know, the two big reasons are the, you know, hallucination issues. And and the privacy issues, which, you know, makes total sense on that.. And I think, you know, I think the legal industry can can walk and chew gum at the same time. So I don’t think this is a you know,

Marlene Gebauer 3:41
I don’t think it’s an either or. I think that would be a mistake to think about it that way. Yeah. And this is this is something that will be used, we just have to figure out how it will be used.

Greg Lambert 3:56
Just like any other tool there exactly. But it will continue on. But at least our guests this week is you know, we brought him in and we can talk about both of these issues and a lot more.

Marlene Gebauer 4:09
Yes. So we’d like to welcome back to the show Christian Lang. Many of us know Christian through his great work through Reynen Court through the New York legal tech meetup and as an advisor with LexFusion. But now he’s taking on a new opportunity by starting Lega, a platform that provides a safe space for Large Language Models exploration. So Christian, it is great to have you back on The Geek in Review, and you’re gonna I know, you’re gonna tell us what LEGA stands for.

Christian Lang 4:33
No, it’s great. It’s great to be back. It was originally an acronym. It was just a generic placeholder for a name a Large Language Models enterprise governance application. And then when I saw the acronym I was like, well, that’s a that’s a nice name. There’s no way there’s going to be IP is going to be clear on that. And I did a check and didn’t know much about IP law. I’m not sure I didn’t check right but I was like, oh, that’s actually could be a great name. And it’s close to legal and it’s close to lego. So we just ran with it.

Marlene Gebauer 4:57
That sounds great. That’s immediate.

What I thought when I saw was LEGO

Greg Lambert 5:00
If any industry needs a safe space, it’s legal. So thank you.

Marlene Gebauer 5:05
So a lot has changed since we last had you on the show back in 2019. Can you talk to us about your transition from the really cutting edge work that you were doing a Reynen Court? And how, after all that wound down your idea for starting LEGA?

Christian Lang 5:19
Sure. Well, it’s great to be back with you. Thanks for having me. Let’s see the Reynen Court transition. highly relevant. I mean, you know, Reynen Court fundamentally, was all about trying to help legal enterprises get access to the best, like the latest and greatest tools, while retaining control over their data, which is a challenge in a cloud computing context. You know, the SAS paradigm has you duplicate your data and send it out outside the security perimeter or somewhere else. But there’s new tools in the market that let you bring the app to the data in it. So spent years trying to focus on how we solve that challenge for legal and really was leaning into standardization, as a as a mechanism by which you can provide speed and nimbleness so like firms and legal enterprises can like navigate a shifting landscape. And I think we’re seeing that change, accelerate. And it was really kind of through that lens that this whole opportunity cropped up. Because this new wave of generative AI technologies, I mean, the pace of change is absolutely breathtaking, and almost paralyzing. But I think we have with the right mindset and right tool set, you can start to again, give people the tools, they need to retain some of the control so that they can lean in and try to find the commercial applications and use cases. So it dovetails nicely. I mean, it’s very different, very different business, but that the perspective that we cultivate it Reynen Court, the relationships and the sensitivity that I developed to the way that large law firms in particular, think about enterprise administration of applications and data governance is kind of where all this where all this came from. Yeah.

Greg Lambert 6:49
Does the LEGA model that you’re using does that differ quite a bit from how you were setting up Reynen Court’s? You know, the focus on enabling adoption of cloud based legal applications?

Christian Lang 7:02
Yeah, very much. So I mean, it’s it’s kind of born from a common perspective and kind of built on some of the same learnings. But like, an entirely different thing at you know, as I’m sure we may talk about a little bit, it’s LEGA is focused on a couple of things. But the the tip of the spear is around kind of governance. And it’s about helping to drive the visit visibility and transparency, and kind of operational hygiene, you need to be able to get started and start getting your hands on tools and capture learning from tools and start to scale that learning etc. You Reynen Court, we didn’t always do a great job of articulating it elegantly. So everyone really understood it. But there was a root, incredibly robust technology layer beneath that doing the actual deployment and container orchestration that facilitated that cloud native portable deployment paradigm. In this world in the generative AI world. I’m excited that this may not be the case for long. And we maybe will talk about that in the future. But right now, everyone’s kind of consuming these large hosted models that are hosted by big tech companies. And you can hit those to the consumer applications. But you can also hit them via API. And so a lot of what we’re building in the LEGA platform layer is about facilitating those API connections, but in a very particular way that helps keep the enterprise a little bit more in control of the data, the access and the policies.

Marlene Gebauer 8:21
Right. So following up on that response, I mean, you’ve talked a little bit about the rapid pace of change. You’ve talked about the need for governance, you’ve talked about how firms are looking more at sort of the API options, thinking about all of those points, you know, why do you see the need for a solution that’s aimed at helping law firms safely implement and govern the use of Large Language Models? I mean, what is sort of different about them, as opposed to like the rest of us?

Christian Lang 8:52
This is a you know, I don’t I’m sure no one who’s listening to this sort of podcast needs to be told this. But this is obviously an exceptional moment. It might be unique. I mean, it’s extraordinary, how not only how incredible and kind of mind bending that the technology is, it’s extraordinary how quickly it’s become a part of the zeitgeist, and it’s just there, and everyone understands it. So to answer your question, Marlene, I mean, we’ve got a situation where we, you know, every firm has hundreds if not 1000s of professionals playing with things like ChatGPT in their own personal accounts outside the visibility of the firm doing God knows what you know, maybe putting confidence apply data even if they’re not supposed to do these systems.

Greg Lambert 9:32
They are.

Christian Lang 9:32
You know, this they know. Yeah, and we know that that’s the context right? As cool and exciting as the open AI models are that’s just literally the tip of the iceberg. We’ve already seen now Google Bard come to market and Anthropic is Claude model, and there are dozens, if not scores more today, and we’re about to have hundreds more. So you’ve got everyone playing in the consumer tools. You’ve got a bunch of new tools coming to market and everything’s moving so fast. There’s like this analysis paralysis where It’s kind of hard to get started. And so you put those factors together, it just occurred to me looking through that lens that we were talking about earlier that we just we need, there was a real opportunity to provide a simple, elegant platform layer, we get channel users out of the consumer tools into enterprise accounts, where the firm of the enterprise is kind of controlling the account and the API key access to the underlying model. You give them the tools, they need to kind of configure solutions that they can provision access to those users. And then just through kind of a governance layer, you just do some very simple things, you give them the ability to manage user access to solutions using the AD and SSO the way you do for the entire rest of your tech stack. When you have policies you want to implement, you’re not sending those out, just by email, you know that people are going to not read in the first place or forget, two days later, you’re going to make sure that every time somebody hits a solution, there’s a configurable compliance checkpoint, so they know exactly what they can and can’t do in respect of this solution at this moment in time, because that’s going to change dramatically based on what model is driving the solution and the factor of time, like you might be comfortable tomorrow, something you’re not comfortable with today. And then finally, in most importantly, if you’re channeling the traffic to the API driven solutions of the enterprise that you control, and you control that traffic routing, you can take everything going into a model and everything coming out of a model, and put it into an audit log, so you can see what people are doing. And that’s important on day one for compliance purposes to make sure people are adhering to the policies in Greg, as you were saying, not doing anything stupid that they shouldn’t be doing or not paying attention, right. But I’m actually much more excited about the broader opportunity to be able to see what people need, because I think we’re in one of this phase. Now we just need to get our hands on, we just need to play with it, we need to identify use cases. And when we find good use cases, or we engineer great props to unlock value, we need easy and elegant ways to scale that across the enterprise. So the audit trail aspect of LEGA I think, is the tip of the spear, the foot in the door is the compliance angle, but actually think the commercial opportunity behind it is huge. And so that was kind of the notion of why I thought, you know, law firms kind of needed something like a LEGA. And that’s kind of that was the genesis of how that all this came to be.

Marlene Gebauer 12:07
Is part of the solution that you’re you’re talking about here, you know, in terms of like figuring out use cases and things. Is it also about sort of which of these LLMs are good at what because you know, some of them are better at certain things and others or you know, some firms that are testing it, they’re limiting them in different ways. And so it’s kind of hard to compare in terms of what’s the right thing to use for a particular purpose.

Christian Lang 12:34
I think that’s exactly right. I think there’s some great people out in the market doing really important and interesting things. In respect of your specific models. There’s some companies built around amping up GPT-4 for legal that do great work. But we may want it like, what’s an example, you know, if I’m if I’m trying to get a taxi brief on a client before I go to a meeting, a GPT=4 model that where the training data stopped in September to 2021, that’s not actually going to help me with current information, right, I might need something driven by Bard, I might need something driven by Claude or any number of other models, I think one of the coolest developments that’s coming that we may see is the ability to host significantly smaller models in your own estate. So they can interact with your proprietary datasets in a way that gives you confidence that it’s not be exfiltrated, or going somewhere, it’s not supposed to go right. And I think that’s exactly the point. We need those sandboxes where we can try and test things alongside of one another, see what’s good at what, and kind of play and tinker. And I think that’s in part, what has informed the way that we’re attacking LEGA, like when you want to tinker, you don’t want us to go hire a bunch of developers to build a UI for everything you want to try and test right you need a way to kind of get apps or solutions kind of in hand quickly. And then because that application layer is going to constantly be changing, you don’t want the governance tools in the application layer, because that’s going to be changing, you want that to be constant. So LEGA’s essentially distilling out of the application layer that user management, the policy management, the governance and the audit logging. So that that doesn’t change. So that’s consistent across solutions. And to your point, Marlene, across different models. So when you’re trying and testing different things, it you don’t have to have this balkanized kind of governance landscape and data setup. It’s consistent. And that’s really the gist of what of what we’re trying to do.

Greg Lambert 14:19
He gave a pretty good explanation about how you secure the platform. But I’m curious what kind of what kind of questions are you getting from the IT and security teams because I was I’m going to kind of bifurcate this discussion a little bit into the security piece, which we have, we have a whole group and you know, IT has jobs in the past decade has focused much, much more on security than than it has before. And then on the actual use case side of the argument as well. So sticking with the with the IT and security parts, what kind of questions are they coming to you with? And is there any type of certification that they’re doing or statement of work? Or Whatever, what it was kind of some basic things that they’re asking you at this point?

Christian Lang 15:17
Yeah, it’s a great, it’s a great question. It’s funny, those two constituencies can map onto the two parts of the LEGA infrastructure. And I’d be very curious to see how this plays out over the next 12 months. In terms of what kinds of questions they’re asking, you know, people are really just trying to get their brains around how this technology works, and how data goes and where it lives and how it’s secured. I think a lot of people, particularly in these large enterprises, that that we all know and love are putting a lot of stock in the fact that Microsoft has leaned in so aggressively to incorporating some of the technologies. But you know, the dirty little secret, and it’s not even so much a secret anymore, is when you start asking the hard questions to the Microsoft folks. They don’t even have answers yet there because it’s all happening so quickly, like I’m not, you know, and and it’s a there’s a lot of open questions. And so I, you know, I’ve been lucky to not have conversations, I don’t know, 50 6070 law firms talking about their strategy, how they’re thinking about things, I think the vast majority are still trying to get started kick tires with with information that’s not necessarily confidential, not the certainly not the kind of deeply confidential crown jewel type of information. And so then they’re just trying to figure out, okay, well, who’s hosting, what, where’s it going, et cetera. But I think there’s some really significant security issues that we’re all going to have to delve into. AI models generally are pretty susceptible to adversarial attack because of how they’re set up. And, obviously, when things are moving this fast, that’s essentially like just fundamentally antithetical, like, I think we’re gonna have a pretty tough context. That said, I think it’s really important. And you mentioned the advisory work I do for LexFusion, Casey Flaherty has done a lot of great work and writing about this recently, we can’t get bogged down in the security conversation, we got to be focused on how, where this goes and from a substantive perspective on the types of solutions we can drive. Particularly if we look beyond law firms, and we start thinking about corporates and enterprises, if the lawyers want to be sitting at that table, a part of that conversation driving where this goes, we can’t we can’t be focused on the limitations and the no’s. But it is interesting that everyone, Greg is starting from square one trying to figure this out, figure out what the security means. And a lot of people are putting a lot of faith in proxies, but kind of with their throwing up their hands a bit being like why I hope on Microsoft knows what it’s doing. So I’m not sure I what I’m doing.

Marlene Gebauer 17:38
So I like your focus about like, you know, how do we get to yes, as opposed to all of the No’s. What types of solutions? And I mean, this is a huge question. I know, what types of solutions can can law firms build and configure on the LEGA platform?

Christian Lang 17:53
Yeah, it’s a great question. So I think, you know, the short answer is the sky’s the limit. Let’s let’s talk more specifically about kind of what that is, in its earliest kind of MVP, state, what like it is today, just just the pure thing that we’ve launched in the market and is actually being used today. That’s consuming via API, the big found a hosted foundation models that are out there, you know, the GPT, 3.5, Turbo, GPT-4 , Bard, Claude, etc. Everyone clearly wants and is going to use some sort of live chat bot that’s like a ChatGPT alternative, right? There’s some low hanging fruit that are also very big, abstract things where you can just tighten up instead of, you know, you could always end you could always do this work through a chat window if you knew the right kind of prompt commands or the way to engineer the prompts. But you know, there’s ways to craft solutions. So some of that prompt engineering is done in advance utilities like summarizing documents, I mean, the use case, I’d love to talk about it, when I’m talking to having a conversation about this at a law firm. It’s just a it’s a silly little use case. But it’s powerful. I mean, when I was in practice, when you’re a mid level associate, and you’re trying to start get involved in business development, that the number one thing you do, almost the only thing you do is keep an eye on the news, and see what’s happening that might be relevant to your key clients, you know, with whom you want to deepen relationships and send it to them. I wanted to make sure you saw this or you know, I was thinking of you or just to say front of mind, right? And you can’t just send a busy client, a document or a link, right, you have to send them the key takeaways in the body of the email. And I think everyone understands, if you don’t understand, let me tell you, it’s amazing how that 20-30 minutes is a real challenge when you’re billing 18 hours a day, and you’re going flat out. So like just the ability to have something summarized Well, documents which these Large Language Models are phenomenal at is a great simple, easy use case. There’s some translation use cases, etc. However, there’s a whole different layer of much more specific things that are pretty exciting. And if you have the right tools to twist dials, you can create them very easily. Like here’s an anecdote so we put it out at a launch press release on Wednesday. One of our early adopters Womble Bond Dickinson the chief knowledge and innovation officer Bill Koch was quoted in that very happy to have them on board when we were doing our launch workshop at Womble and I was, with Bill talking him through how the configurations work. Without delving into private information, he had a brainstorm in the moment thinking like, Oh, I see this solution. Can we do this other thing that looks a lot like that. I’m like, I don’t know. Let’s try it. So we literally just like duplicate a solution, we go into the configuration parameters in our admin panel, and start playing with the system prompts. This was an open AI driven solution. So we could tell the model what hat to wear. And we changed the kind of hat wearing instructions of the model to do what Bill was just suggesting. And four minutes later, we had a working solution doing the thing he was he was trying to do. And I think that’s emblematic of where we can go with if we have the right kind of Workbench sandbox type of approach here. There are tons of marketing use cases, we have firms thinking about talking about, we talked earlier about taxi briefs that came out of a, I was having a breakfast with a CIO at a Magic Circle firm A couple of months ago, and we were chatting about the whole concept of Oh, you’re running into a meeting, you need a quick download of information, everything that’s happening, you might ask a paralegal, you might hit your associate, you might throw it down to the library, well, you can literally configure a solution that just will spit it out in the in the data and the information you want, you can just drop in a company name and bam, it’s it’s in your hands and five seconds. And there’s I mean, these are just silly little examples that don’t seem that powerful. And this is just the tip of the iceberg. I mean, we can we’ll really start to be able to lean into really substantive things, particularly as soon as we have access to models where we can control what data goes in and where it lives.

Greg Lambert 21:26
Yeah. And I think one of the things that a lot of people worry about, I think, actually, they should probably flip and worry the in the other direction is that you’re going to see things like this, and you’re going to hear people go, Oh, if they’re not coming to me for that any more than you know what, what use, am I? The problem is, they’re not coming to you in the first place, because no one has the time to do this. And what’s going to happen is, all of a sudden, they’re going to come to you with much more extensive request, because they’re coming with more information up front. And I think, you know, the problem with that is, the people that never use you are now all of a sudden going to use you. So you’re going to be overwhelmed with this. And so, you know, it’s one of the things that I don’t think a lot of people are thinking about is how do we react to the changes that are going to be coming to us? And so, you know, especially if you can create something in four minutes, and then all of a sudden you got, you know, going going out for probably months or years from there. Again, I think it’s one of the things that is not on people’s radar at the moment, but I think it should be.

Christian Lang 22:44
Yeah, I think it’s a great point, Greg, I’m at look, let’s let’s be real there. Of course there are practice areas, if you’re thinking about firms, or maybe there’s roles within firms where the party in question is purely kind of rent seeking, they’re not actually adding any value. They’re just kind of coasting like, of course those things exist. But I think people dramatically underestimate how much work and more valuable work there is to be done. That’s just not getting done today. And to your point. I mean, once once you start having these enabling tools, it helps guide you to where you need to be to do that stuff. And the market for legal services is dramatically underserved. You know, I pick your number out of a hat, but only 8% of the legal services market is actually served today for all kinds of reasons, there’s plenty of plenty of work to to fill up any any minutes that might be created by kind of expediting processes that shouldn’t be done in a kind of a manual way.

Marlene Gebauer 23:36
So Christian, you partnered with Betty Blocks for legacy technical infrastructure? Why did you partner with them? And how does that partnership benefit customers?

Christian Lang 23:47
Yes, this is another relationship and concepts, both conceptually but also very tangibly that came out of the Reynen Court experience, because they were close partner of ours at Reynen Court, we used to work closely with them huge fan of that he blocks I think they’re, you know, the best or one of the two bests, you know, application development platforms that plays in the legal space.

Greg Lambert 24:07
I’ve always loved their name.

Christian Lang 24:08
Yeah, they’re great. I mean, they were right in the Goldilocks zone. They’re very few of them that have user friendliness, but huge power behind it. So you can do almost anything, usually you kind of have to pick or choose between those two things. And then the lesson from reigning court that generated that relationship. I’m just deeply sensitive to how the great to your point about the fact that the IT teams are almost exclusively focused on security for the last 510 years. I mean, I just knew what was going to be kind of acceptable and not acceptable from an architecture at a data security perspective at these at these top firms. Not necessarily these two, maybe we’ll talk about this, these top firms are necessarily legis best customers. I just that’s where we’re getting started. And I just knew if you show up with a business that was started three months ago and say, Hey, trust me, I did these things. I don’t have any certain certifications yet. I don’t have I don’t have a big team of QA engineers doing these things yet, but just you know, trust me that that’s a nonstarter. So, Betty Blocks not only is a phenomenal platform, they are an incredibly mature operation that serves some of the most security conscious customers in the world, not only law firms, but beyond law firms in the intelligence sector and other things. So they just have, they have all the things, they check all the boxes that you need to give somebody confidence about where the data is going, how, where it’s living, who has access to it, etc. And so I knew that if we wanted to start quickly, and I knew that starting quickly was imperative here, given t