Ep.217 – AI Implementation Secrets: Avoiding Common Disasters with Mike Luttenberger of Netrix Global

Jason Kaufman (00:04.726)
Welcome to the podcast that is designed to fuel your success in selling technology solutions. I’m your host, Jason Kaufman, Principal Solutions Architect at Telarus, and this is Next Level Biz Tech.

Jason Kaufman (00:21.164)
All right, everybody. Today we have another exciting episode for you around AI. You may have heard of it. It is that term that’s getting thrown around a lot, sometimes in a joking manner and sometimes serious, but it’s probably the most requested topic that we get in here. So we wanted to give you some tidbits here. So we brought this episode titled Implementation Confessions, the AI Disasters at Disasters They Don’t Want You To Know and How We Solve Them.

I’m here with Michael Luttenberger with field CTO of modern work at Netrix Global. Hey Mike, how you doing, sir?

Mike Luttenberger (00:52.205)
good. Doing good. How are you today?

Jason Kaufman (00:54.072)
Doing well. Thank you for joining us. so everybody’s probably wondering, Mike, why are you the chosen one on this AI podcast here? So, Mike, I’d love to get a little background on you. Can you share a little bit of your professional journey? You know, how’d you venture in the field of cloud, AI? You know, give us that, give us those you know, insights. One thing that maybe, you know, that inside joke that you’ve always had, some embarrassing moments or something. What’s that background of Mike?

Mike Luttenberger (01:17.039)
Sure. So yeah, we’ll actually go quite a few years back to get that started. so I didn’t I’m not one of these folks that started off, you know, writing programs at five or anything like that. I was actually quite terrified of computers. I didn’t touch a computer until I was well into college, really. And college was a big deal in my house. My father didn’t graduate high school, he went right into the military, he ended up getting his GED later, but to him it was like college is everything, you have to do it. So it was very, very important. So that I’m like setting the story for that.

So, okay, cool. So I go to college and I wasn’t really sure what I wanted to do. I was I loved science. So something in science I thought maybe like plant biology or something, because that’s that’s what I was good at. and in that part of college, I had to take a computer class, just a you know, CIS 101 class. And I didn’t want to take it for obvious reasons because I was terrified of computers, but I obviously had to take it, so I took it. And he kind of gave us like the syllabus the first day of class and said, like, this is what we’re gonna do the whole semester. And I finished it all in two weeks. I was just like, I

Loved it. I loved everything about it. So I said, you know, do I have to show up anymore? And he’s like, Of course you do. You know, I’ll teach you some programming. So I’m like, okay, cool. I don’t know what programming is. So he taught me, I don’t remember what it was.

Visual basic or like that. and started building on it. So then I was just like, okay, this is what I want to do. And I just started taking all these computer classes. Fast forward a little bit. I one of my buddies is like, hey, there’s my buddies hiring like all these people for computer stuff. You’re doing computer stuff. I’m like, I’m in college, like I know literally nothing. He’s like, you know what, you should do it for interview experience. I’m like, that’s actually a great idea. So I put a half-hearted, you know, resume together, threw it out there to the to the place. They call me up a couple of weeks later and

I like, okay, cool. Like, you know, I was like, what do I have to lose here? showed up, they’re talking ASP and all this fun stuff. And I had the look on my face of I’ve no idea what you’re even talking about. And he paused the interview and he goes, You have no idea what active server pages are. Are you? I go, I don’t, you know. And he’s okay, well, that’s kind of our thing here. You gotta learn it. he’s like, you know, go go learn it. come back in a year and let us know, and we’ll interview again. Well, I was like, nope, this is what I want to do. So I left the interview. I went to Barnes and Noble or Borders when they were around and kind of

Mike Luttenberger (03:27.483)
Got three books on ASP. And I went home and I just started studying it. And then I had class that night, one of my CIS classes. So I went in and I went to my teacher and I said, Hey, I just had this interview and they’re interested in ASP. Can you tell me more about it? And he’s like, never heard of it. And I’m like, wait a minute. Like I’m supposed to go to college to learn this stuff. My professor doesn’t know any of this stuff.

So I made it a choice that night to never go back to class again. So I was like, obviously I knew my dad would lose it. but I’m like, I gotta do this. So I took every minute I had off from work, you know, studying, staying up till four or five in the morning. you know, sitting in my car during break, like reading the books and all this stuff. So I call the guy back seven days later and like I’m ready to interview. And he’s like, There’s no way you learned ASP in seven days. I go, I don’t think you know how bad I want this job. He goes, Okay, fair enough. He goes, then

Next interview won’t be with me, it will be with a Microsoft MVP in ASP. And he goes, Either you are hired on the spot or never call us again. He goes, Do you really want that? I go, I really want that. so I interviewed, I I got interviewed and got hired on the spot, and that’s kind of my intro into computers, and everything is pretty much self-taught. lots of stuff throughout the you know, obviously things have changed. Started off with SharePoint, heavy in development, detoured about 2014 into

machine learning and augmented reality and then obviously the cloud and all that fun stuff which led us to about you know three years ago with chat GPT I saw that and was like this is it this is the next thing this is it yeah so I’ve been exactly I thought it was plants yes yeah

Jason Kaufman (05:01.933)
That’s what I was made to do. I thought it was plants, but now now it’s ChatGPT and AI.

Yeah. Fun fun fact about me, I had the same same thing when I went to college. I started an actuarial science, which is like I was always good at math. And then all of a sudden I made a wrong turn, walked into a random class and ended up being computers one one. I was like, Yeah, I like what I heard. Stayed and never went to my other class. So totally understand that one. That’s an interesting background. That’s awesome.

Mike Luttenberger (05:22.105)
Yeah. Yeah.

Mike Luttenberger (05:27.683)
Yeah. Yeah, yeah. Dad was definitely not happy when I came home with that news, but I you I told him, I’m I’m gonna get this job. I’m telling you, I’m gonna get this job and he was like so, yeah.

Jason Kaufman (05:38.455)
That’s awesome. Giving give you got seven days and you’re going up against an MVP and you’re either going to get hired on the spot. It’s, you know, walking off the plank, life or death situation, and you just passed it. Just crushed it. That’s awesome.

Mike Luttenberger (05:41.924)
Yeah.

Mike Luttenberger (05:45.744)
Yeah. Yeah, and I always f I always find that that when you know, when the game’s on the line I always want the ball in my hand, you know, so always. So that that’s kinda my personality, if you will, right. Yes. I don’t have any shoes though, so

Jason Kaufman (05:58.52)
Well, in the tech game, you gotta be like Mike too. You got to do it on the basketball court, and now you gotta do it in the tech game. Yeah. So so all right. Well, thank you for the background on on yourself. So awesome background, you know, leading the tech, self, you know, self-starter, self-learner, you know, building into you know machine learning, AI, and all that stuff. You know, you’re definitely the right person here. And now, how about Netrix Global? So, you know, we’ve definitely done some intros. You guys are a big name on the engineering side.

so tell us a little bit about the background of Netrix and how you landed there and why, you know, Netrix starting to be like an AI leader in the space.

Mike Luttenberger (06:31.257)
Sure.

Yeah, so Netrix was founded back in nineteen eighty-nine. And we I kind of came over about 10-ish years ago. There was a bunch of us who kind of will say it migrated over, right, from a previous employer. that kind of got sold off and and pieces like that. So we kind of all migrated over here, if you will. And really on the professional services side, that’s where we were. But we Netrix, again, through acquisitions and mergers and things like that, we’re about a thousand-person firm. 80% of that is engine.

Again, we’re MSP and you know, like most folks, but we also are professional services where again, obviously, the AI piece comes in. We are also AWS and Microsoft partner. So we are looked at from Microsoft and both AWS as kind of the go-to partners for a lot of things because we we pretty much do it all, especially in Microsoft, except for dynamics. That’s about the only thing we don’t really get into. but we have folks that we partner with that we you know consider almost our our own.

Jason Kaufman (07:29.261)
But you guys get out of the hyperscalers too, right? I mean, you guys are very flexible on the different models. So if somebody wants Chat GPT Enter Enterprise, Google Gemini Anthropic, you guys got chops there too?

Mike Luttenberger (07:38.658)
Yep, yep. We are AI agnostic. We there is nothing that we focus on. We don’t just say we’re just doing copilot or we’re just doing chat GPT. We do it all. We have done with our customers, we have done ChatGPT stuff, projects, however you want to look at that. lots of Claude, obviously, as of late. Claude has been such an uptick for us, I would say, over the past like month and a half, two months. and then obviously copilot, we’ve doing that since it was released as a Microsoft partner. we were one of the few folks that Microsoft allowed.

and kind of their partner program around Copilot early on. So we’ve been doing that for what I think it’s like almost three years now or two and a half years whenever it came out. So yeah, yeah, we kind of do it all. There’s nothing we we don’t touch in the AI space.

Jason Kaufman (08:21.495)
Awesome. So yeah, I mean it goes to show why you guys are, you know, so dynamic and so so easy to work with because you’re flexible. You know, you don’t you know everybody’s got that one trick pony. Hey, we do AI, but this is what we do. Not, hey, we want somebody that comes in that can talk agnostically across the AI board, what different technology tools, what are different use cases. And that’s why I think we’ve landed and expanded pretty well with Netrix. And it goes to show exactly what you’re saying there. Top engineering staff, 80% heavy there in engineers.

And then, you know, across the board, you guys can talk pretty much any AI game. so can’t wait to get the t-shirt. so you know, we we’ve talked about AI, we’ve talked about Netrics, we talked about different OEM technologies from Chat GPT to Azure, AWS. So for our advisors, you know, that are listening around the world here, who aren’t deep in the weeds with AI, you know, let’s get a little precise primer on what what is AI and how’s it being used today. And are there any risks to it?

Mike Luttenberger (08:54.37)
Mm-hmm.

Jason Kaufman (09:17.449)
You know, let’s let’s be honest. You know, is it is it an easy flip of the switch button and everything works, you know, handy dandy, or are there some complexities that are involved with it?

Mike Luttenberger (09:25.921)
Yeah, a hundred percent there are complexities. so how I liken to think of AI or how I like to explain it to, you know, someone who’s non technical or is what what is this AI thing is Sure.

Jason Kaufman (09:35.639)
Yeah, just print pretend you’re talking to me. Non technical. You know, let’s break it down. You know, maybe a coloring book or something like that.

Mike Luttenberger (09:41.686)
Sure. So kind of what I like to say is if you think of before AI, you would be very much in what I would say like keyword searching when you’re doing things, right? Think your Googles, your five people who use Bing still. but what you do is you you put in your keywords and then you’re hoping to get just something back and then you kind of do the work from there. AI is very different, right, to that to where it’s what I say is context aware of what you’re asking. So kind of the example I always give folks is let’s pretend that you have a 401K document, right?

Sitting in Google or you’re it’s sitting in SharePoint. And if you just use the native search tools, those are most likely looking for keywords, right? So if I type in the word benefits, but the word benefit is nowhere in that document, it’s not tagged, it probably won’t find it because it’s not context-aware, right? It’s looking for keywords. Now let’s bring in AI to kind of the fray. Now that same exercise can happen once you’ve turned AI and pointed at your environment, and you could say, Tell me all my benefits, and it will 100% understand that 401k.

is in fact a benefit and pull that for you. So that’s how I like to kind of explain it. But obviously it’s it’s much bigger than that. There’s kind of different arms to it to where there’s agents and all this fun stuff and different models doing different things. And we obviously help our customers kind of go through that and guide through that. We have again kind of customers on all sides or both spectrums, right? Where we have some I don’t even know what AI is, but I am told from leadership that we have to get into it because we’re we’re going to be behind to no we’ve

Already deployed AI, we just kind of need you to come in and help, you know, augment our staff with this, you know, project or even create that pop, you know, project for us because we just don’t have the technical chops to do that and kind of everything in between with that. But to your point, it is not something you just turn on. and again, we’ll talk about definitely some of those of some issues, but clean data, security, those are all things that have to be kind of the foundation of AI. I have lots of customers who think that.

AI will just fix those things. They will not. It will make it actually quite worse if you are kind of run in an environment that is doesn’t have the right, you know, data structures and those things are just kind of all over the place in duplicates. it could become a nightmare for sure.

Jason Kaufman (11:56.674)
I know I know you talked about like agents, you know, a lot of us, you know, take that data and then we’re able to ask it human language stuff. You meant you mentioned the intent. So even if benefits was spelled wrong into a Word document, you know, we’d be able to figure that out. so but but what about some other use cases that we see all the time? Like when people think AI, they think you chat GPT, which we kinda went over, you know, get some knowledge and ask it a question, then we get an automation. You know, how do we how do we automate stuff that a human normally does that’s repetitive and mundane or any of that stuff?

And then also on the other side, the data side. Like how do I get those insights and get those metrics that are really business impactful rather than let me get a series of spreadsheets and then copy everything and have somebody do all these calculations and stuff like that? You know, can you paint us a quick picture on those other types of use cases besides the the large language model stuff?

Mike Luttenberger (12:42.595)
Sure. Yeah. And and definitely we’re once we kind of talk about some of those big picture things, we’re we’re now we’re kind of expanding outside of just AI in itself, right? And to your point, we’re bringing in automation and things like that, anomaly detection, things like that. So what’s great about those is or kind of where you’re moving into that next piece of it is to your point, yes, if there’s something that I do every day or or even I want done automatically, that’s again a great spot for AI. Again, you could think of it as an example of let’s pretend that I have an agent or I

Have AI monitoring my inbox, right? And anytime someone says something about pricing or cost or something like that, that I have an auto-generated, you know, kind of email, if you will, that will attach our pricing sheet and you know, but also have LLM behind that that will actually answer it, you know, not just copy and paste something in, but we’ll intelligently know that here’s how I should answer it. Here’s some supporting materials and can get that stuff out there without you having to do it, especially if it’s if you’re a marketing, that’s something that you do, you know, all the time. So that’s

a great example of that. And then again we could go even ad you know a little bit more advanced where we have actually helped customers save tons of money where they’re it will have anomaly detections in it where it will notice like, you know what, this customer’s like never ordered this type of stuff before. So we’re gonna kind of flag that and now a human needs to review this because this is this is not making sense, right? Based off of historical stuff, you know. Yeah.

Jason Kaufman (14:03.319)
Character. Yeah.

Okay. No, thank you for the for the rundown. Yeah. I there’s AI is just so blanket the way everybody says we need AI, we need it now. We don’t know what we need it for, but we do need it now. And we also need it yeah. So we we get the same request over and over again. we okay, well who’s who’s making this objective? Well it’s our CEO or it’s our board or it’s our owner, you know, ’cause we want to get a competitive advantage. So I AI. Well what do you what do you need it to do? well we we don’t know that yet. What you know, what what do you can you tell us? I’m like

Mike Luttenberger (14:10.903)
It is.

Yes, I love that meme. I see it. Yeah.

Jason Kaufman (14:33.121)
Well, you know, let’s let’s talk about priorities first. Let’s see what what tasks or what data or what you know, all these things are w we’ll get into shortly. I don’t want to land the thunder before we give the lightning. so we’ll we’ll get there when we start talking about some of these legitimate opportunities that you guys are doing for us as well. getting some really good stuff. so one of the things that, you know, we hear horror stories all the time of companies comp companies implementing AI on their own or using wrong partners.

Do you see that there’s like a general theme that that the issues arise that are repetitive, templated? Do you see a do you see an issue that’s reoccurring that comes up all the time?

Mike Luttenberger (15:08.663)
Yes. So there are four things that I see all the time. One from the partner perspective, and again, I hear this probably at least once a week, where a customer will say, Some company came to me and said they have this exact solution of what we’re trying to do here, right? And I always question them to say, you know, talk to them, find out when how long they’ve been in business, right, as a company. Usually it’s just a couple of months. I would ask them very specific things about what they’re doing around AI in here, right? Like one of the things I love to always tell them is

Go ask about tokens. Like what models are you gonna be using? And that one I always hear a lot. They’ll come back and say, they told me we had to use the latest and greatest. And I’m like, probably not. They’re probably not that’s not accurate, right? You don’t have to use the latest and greatest depending on what you’re trying to do, especially if it’s an agent or something. You can go back very, very old, very old is you know six plus months, in AI and use older models to do exactly what you want at a much cheaper, you know, token cost if that’s something you’re doing. That’s something I hear one.

Jason Kaufman (16:09.165)
So you’re seeing the same thing to where companies just go g open a chat GPT enterprise account and just open open AI and get a you know, cost effective model that’s, you know, two years old and then they just go buy a twenty dollar domain dot AI and now they’re an AI company and they’re on Google paying for Google marketing.

Mike Luttenberger (16:24.045)
You better believe it. Yes. And we have there’s been a lot of our customers that we have where they’re like, no, we’re gonna go with this other one because they’re cheap or something. And then they come back to us in about three months and they’re like, Yeah, we fired those guys because they they weren’t like doing anything. We’re like, We told you. You know, like th there are so many kind of fly by the night, I would say AI firms now because of this, it’s the hot thing, as you were saying, you know. Yeah.

Jason Kaufman (16:46.719)
Mm. One of those bitter bittersweet moments. we yeah, that that didn’t work out. Yeah. Okay. Well we got your notes right here. We we knew you’d be coming.

Mike Luttenberger (16:51.651)
Yeah.

Yes. Yeah, exactly. the other components I again I I say there’s four, but these two could be kind of one is data not clean. again, a lot of folks just think that it’s okay, we’re gonna throw it and AI will figure it out, it won’t figure it out. Kind of the example I like to give with bad data is let’s pretend that you had a policy document and there was twenty twenties and then they made a copy of it, called it twenty twenty-one, changed a couple things, copied that one, called it twenty twenty-two, changed a few things.

So you get the idea. So now we’re sitting at you know 2026. And now me as an end user just goes in and says, Hey, what’s our policy, right? On whatever, say travel. And let’s say that you know, back in up until 2023, travel was $500, and then we changed it to $200 at 2023. Every time I ask Copilot or Chat GPT or Claude or any of them, hey, what’s you know, again, this the reason I called all those out is because this is not the model problem, right? This is the data problem. If I ask them, hey, what

What is our policy on travel? It might pull me 2023s, it might pull me 2020s, it might pull 2026. If I have all of those kind of versions of in there of those different, you know, years and policies, it’s gonna just kind of pick one of them, right? At random, unless I’m being very thoughtful or or you know, saying in my prompt, like tell me the 2026 one. But again, most people don’t say that. They just say tell me the policy. so what will happen is that will come back and say, hey, guess what? You can just spend $500, and then I’ll go and I’ll do that. And then next month my claim will get

And I’ll hey, why did I get denied? It’s 500. And no, we changed it in you know 2023. And I’m like, but AI told me. Right. So now there’s a perception problem with that. So that’s the data side. The other side is the security side. And again, true story here. we had a customer, we really advised them, you really want to make sure the data is, you know, where it should be. You don’t want to make sure executives’ salaries could be found, all that fun stuff. Because again, if we go back and use kind of what I was talking about earlier about the you know, the 401k policy piece, it is context-aware of what you’re asking. So, this is a true.

Mike Luttenberger (18:54.033)
The IT staff was keeping a password document just a couple levels down and in an area that no one really went. Again, it was open to the whole company. they didn’t, you know, want to do anything really about that. We really advised them like, no, security data, we really have to clean this stuff up. They’re like, No, it is good. We’re confident it’s good. They turned it on. What had happened now is they had called the password document something like grandma’s recipes or something like that, but it was a password document. So

Jason Kaufman (19:24.055)
No nob guess that’s password document. You know, on a business on a business server, grandma’s recipes. Yes, I’m gonna get some cookies, yeah.

Mike Luttenberger (19:25.919)
Exactly. So all all they said was, Yep. So it’s context aware now. So they to they’re typing in password document for IT and all the the all the company, all the employees are finding all this stuff, right? So needless to say, they put the kibosh on it. We had to put you know everything in place and then they turned it back on. But that’s the reality, right? And that’s why those are really important. And the last one is no training. I can’t tell you, and again, one of the stories I’ll tell today is company, you know, that won’t will spend

all this money on licensing for AI and even in getting the data and the security but not doing the training, not getting folks AI literate. And then we come in and we find out, you know, not even a third of the the company is even using AI because they just weren’t properly trained on it.

Jason Kaufman (20:13.335)
Yeah, I I love that because like most of the most of the you know, I’m at an event and we do the speech, we break it down in those individual lanes that you were talking about, the sear, you know, life cycle of AI. And I’ve seen people like, hey, we we’re development heavy. We built this amazing tool set for this company, it was perfect. We tested it. It was it almost had a hundred percent Q, you know, QA, no, no hallucinations, no nothing. Like everybody loved it, but nobody used it. And it comes down to the training and to ensuring the adoption. That’s where you get the ROI. So you could build the most perfect thing in the world.

But if people don’t use it and the the, you know, company gets to experience people using the more efficiency and all that stuff, then effectively your ROI is nil because nobody if if you don’t get a if it’s not going to be adopted, then hey, it’s it’s just something you put a ton of R and D and capital expenditure in and nobody uses it to actually get the benefit out of it. So it’s amazing how much that’s missed and a lot of these conversations that we’re having. That’s a a massive gold nugget right there.

Mike Luttenberger (21:09.355)
It is. It is. And I would say that’s one of the things that we do well, right? we’ve traditionally historically have had kind of a change management training kind of area within Netrix for years, obviously through different things, like when you know the cloud was new and everybody was moving from on prem to understand what that meant. So that is absolutely one of the most crucial things I think that a lot of folks just think to your point, we’ve spent all this money, we’ve got all the stuff, we’ve we’ve tested it, it works. Now we’re gonna give it to the people and just expect them to

to use it and use it properly and it’s generally not the case. Right. Right. Yep. Yeah.

Jason Kaufman (21:42.712)
that that’s definitely some great stuff right there. A lot of a lot of great nuggets to watch out for. Like, you know, just know that AI isn’t something you turn on, you walk away from. There is a true life cycle in playbook in order to be successful. Netrix has it, you know, Telarus it, you know. So we we come in with a team approach to make sure that’s successful as it can be. But one thing that we really wanted to do here, and I know it’s in the title and everybody’s probably itching, you know, what

Let’s talk about some of the opportunities here and get some specifics around these issues and how you guys came in to save the day. So I know, you know, one thing we wanted to talk about here. Let’s talk about one of the more important ones that you see day in and day out. It’s probably one of the more repetitive issues that come up from a customer. Let’s talk about a real life use case here, and let’s unpack it a little bit. Do you have one in mind that you want to discuss? Many. Well, let’s start with one and we’ll go from there. We we

Mike Luttenberger (22:33.623)
Many. un unfortunate unfortunately many.

Jason Kaufman (22:39.287)
They told I couldn’t record a twenty six hour podcast. So we’ll have to we’ll have to get it down a little bit and focus on one or two and then we’ll say, Hey, contract contact Mike at Metrics if you can go all day.

Mike Luttenberger (22:49.593)
Yeah, so so let’s talk kind of the training one, right? So kind of the paint the picture, this company had about three thousand or thirty five hundred users within there. and they had it th this particular one was co-pilot, right? So they had had co-pilot for about two years, I think it was, at this point in time. And then they came to us and said, Hey, we don’t think we’re getting the value out of this. Can you kind of come in and just

Figure it out. Like figure out why this isn’t working for us because we’re we’re not real no one’s like using it. We have all sorts of other like AI in the environment. Can you just kind of come in, do some interviews, figure out, give us kind of a roadmap game plan what we should do? Absolutely. So we started meeting with folks, right, doing interviews and a couple of things we were hearing, right? So one we’d say, like, like you’re using Chat GPT. Why are you using Chat GPT? Some of the answers were I didn’t have AI. Or it was Copilot didn’t do

said thing, right? so we again not judging, just taking notes, just trying to understand that. And then some of the folks would say like, well, we didn’t have AI. And I’m like, what about copilot? And they’re like, I we don’t have a copilot. And I’m like, did you not did you get like a anything that lets you know you had AI like as a company? And they’re like, no. You know, and again we asked them how did you launch it? And they said, well we just sent an email out to everybody and said, you know, hey, you have AI now. And again, let’s not kid ourselves when it comes from corporate, maybe 50% of the people will read it, right? but

But also there was no training. Right. I mean

Jason Kaufman (24:14.253)
Well, since I’m on a recorded line I read every email that comes from corporate and I read it thoroughly and sometimes twice. Okay. Okay, perfect. Just

Mike Luttenberger (24:20.737)
I meant them, not me, not you. Not us. Them. That was their problem. but yeah, from there, we started doing these interviews and we’re finding out really quickly that either they didn’t even know they had copilot, so they went and bought something else, or because they weren’t trained on the capabilities of it, they didn’t even know that copilot did this stuff, right? So we listed all this stuff out. I think there was like forty five different AIs, and again, I’m in this space every day. There were some I never even heard of. So I

I was like, I don’t even that’s probably a security nightmare, you know, of what you have. I would get rid of this. So one of the things we did was kind of a matrix, right? Of here’s because they said we we really want to focus on one where possible. And and again, that’s definitely my stance on it. I think that if you have you should have your user should have a base, right? So if you’re in Microsoft, it makes sense it’s Copilot. If you’re in Google, it makes sense it’s Gemini. But then you have edge cases for when that won’t fit, right? And again, there is no LLM out there that does all the things, right? I think you have to have a mix, but I

Don’t think every user should have a mix. I think, okay, well, marketing came to us with this use case. That’s why they need ChatGPT, let’s say, on top of Copilot. Not that the whole organization gets it. I digress, sorry. but that’s one of the things that we kind of went in because he said, you know, we really want Copilot to kind of be our foundation and just if there’s something else, then we’ll do it. So we did this matrix of what Copilot does, what the use cases were that they said they needed it. And what we found out was that they could eliminate I think it was about 35 or 40.

40 of them and it was going to save them six hundred thousand dollars a year in licensing because everybody just had all the licenses for all these things. So all of that just to take a step back, right? And that’s all we did was essentially took a step back and did a proper kind of launch of AI for the organization, some email campaigns and some training. And it and the we watched the usage go through the roof, right? Of that after just doing that very simple thing. Again, there was not much to that kind of launch of it. It was a few.

training sessions, some prompting sessions, and then we had like an email campaign that kind of went along with it. And we went back at the end and looked at the usage and it was like a hockey stick from that. So

Mike Luttenberger (26:31.797)
It just goes to show you how powerful something and that is just as simple as training. Again, doesn’t always take a lot to do that or cost a ton or take weeks and weeks and weeks to get those things up and running. But that makes the difference of, you know, in their case, six hundred thousand dollars a year and licenses they were paying for to do the same stuff that, you know, essentially Copilot was doing.

Jason Kaufman (26:51.597)
It’s amazing how many like companies of that size, like if you’re saving $600,000 a year, you know, you obviously got some got some chops, you know, you assume that they already do some of those things. Like if something’s not working, they would go in and find stakeholders or users, get real feedback, you know, make sure they know how to use it, do a controlled rollout, you know, get proper feedback to QA everything. You know, you think they would do this natively, but as you as you can tell, they don’t.

So bringing in somebody that you would think, you know, an expert team, that’s the first thing you want to do. Let’s go back to basics and make sure that this was done correctly and done it done in a controlled environment. Get those user stories, get that feedback, you know, create a matrix that’s easily consumed by the people that are in charge of this stuff, the true stakeholders, and and then redo it and make sure that it, you know, you follow the use cases, you answer those use cases and build to it. And then you train the people on how to use it to solve those problems that they didn’t think they already had a solution for.

So it’s just amazing taking it back to its core functionality on, hey, you know, I you know, the company’s already doing all this stuff. You know, we have to think way outside the box to make this a success. You really don’t. You’re coming back to something that’s core and you think like it’s part of that one hundred things that you already know that you should, you know, do no you know, normally, they don’t do it. They it was just a misstep because they think p you know, you release AI, you send somebody an email, now everybody’s got that email blast, they’re gonna get right in and know how to use it. And that’s not the case.

Mike Luttenberger (28:11.876)
Yeah.

And and I think that’s that’s a big part of it. I think a lot of people think, well, AI is just easy. I just talk to it or just type it and it’s gonna do all these things. And to a degree, I would agree with that, but I think that you absolutely have to kind of get some literacy in there of just understanding how it’s gonna work and the things it will do, because one of the things we see quite often is that if they’re not trained or they haven’t gone through kind of that again, AI literacy as I like to call it, is they just use it as a search, right? They you’ll you will talk to them and like show me what you’re doing with AI and they’re like

Just doing like internet searches that they were once doing in Google, now they’re doing in their AI, right? They’re not having it build anything because they really didn’t know or understand that it could do those things. So again, if you’re paying for this stuff, you you want folks to maximize everything that they’re doing with it. But I think that’s a lot of it. A lot of people think, well, yeah, it’s so easy. We’ll just launch and they’ll get it. that is I can tell you that’s not the case.

Jason Kaufman (29:04.301)
Yeah, I mean if we can sum that up in like three or four bullet points on things to listen to for that specific type of use case, which obviously it was the first one you mentioned, so it comes up a lot. So it’s not a one-off, is you know, check, check out the, you know, the the rollout. You know, how was it done? Was it just communicated via email or were there multiple steps involved? Or do the people know it’s actually, you know, released? do do they know it solves the use case that they’re looking for? Do you get feedback on it to make sure it’s done?

And then are they trained and know how to do it? I mean, it seems like really easy concepts, but if I’m an advisor and I’m listening to somebody that’s telling me that they didn’t have a successful rollout of an AI implementation and they st and they don’t mention those things, that’s a meat light bulb, immediate light bulb on, hey, I know somebody that can bring in that’ll make that’ll get this right on track and make sure you get that ROI back from that AI implementation. I think it’s a home run, something easy to listen for that that you wouldn’t think you would, you know, you would have to because you would assume that somebody’s already doing it.

Mike Luttenberger (30:00.558)
Yeah, agreed. Yeah. It it’s I I think it’s it’s kind of low hanging fruit, easy question to ask, and but again, doesn’t get ans asked a lot because I think a lot of people just assume you spent all this money, of course you went through training. Right, but that’s not the case. They do.

Jason Kaufman (30:14.519)
Yeah, they put all the budget into the programming, but they don’t they don’t put a money into what is it gonna look like at a rollout? and it’s amazing how much that’s missed. 100% agree. so that’s a that’s a great one. And I know you said you have many, many you want to go through. but let’s let’s choose one more here. I want to be cognizant of everybody’s time listening. I’m sure they’re sitting in a parking lot at their destination, like, I want to get to the end of this because this is great stuff. Jason, you’re way better than the other guy that does that runs this podcast. You know, whatever you guys want to think of while you’re listening to this.

Mike Luttenberger (30:19.979)
Exactly. Yeah.

Jason Kaufman (30:43.141)
but let’s go over another opportunity. You know, we went through the training, you know, maximizing the ROI, the adoption. You know, is there another, you know, use case that you’re seeing a lot that’s repetitive?

Mike Luttenberger (30:52.867)
Yeah, and and like I said, I’ll kinda combine them. It’s the data since security side. A lot of people just think that we’re just gonna buy AI, we’re gonna get the coolest LLM, the most powerful one, and then we’re gonna point it to not very good data. And they think that that’s gonna sell you know, solve everything, which again it actually makes things worse. once they do that. And again, that kinda walks the fine line of security too, because I know you said a little earlier, could this could this be dangerous? again, I think there’s definitely different layers of what dangerous means, but I think for most companies

finding the you know CEO or executive team salary through AI is probably dangerous, right? And if you’re not doing that security piece of it, that’s that’s something that can and absolutely has happened at a customer of ours where they launched it, they were able to find all sorts of salaries that they shouldn’t because it was sitting in some site that HR didn’t realize and it was open and everybody had access to it. Needless to say, they shut it down the next day.

That person that was running the project, it was like the IT director, got let go. again, we advised, we said, absolutely we should be doing security and governance. And he was like, No, I know my environment, I know it’s perfect. obviously it was not, right? So these are some of the horror stories that can happen with it. Again, AI is just gonna make those things worse, right? The security and again, your your data governance around those things. If the data is all messed up, it’s just it’s not gonna work, right? It’s not, it’s gonna give you inconsistent answers when you ask.

and again, obviously we want this to r you know help us and save us time and do all these things. And if it’s I always have to chase after it because I don’t think that was right or that was an old policy, it’s not helping me anymore and I won’t use it right.

Jason Kaufman (32:36.621)
That’s a great, great point. Once you once you can poke holes through it, people lose confidence in it, especially if it’s something new and you know revolutionary and all that stuff. And it’s amazing how you know how it all comes back to and we’ve all heard the terms crap data in equals crap data out. That’s it that’s astronomically more important now with the machine learning algorithm moving in fractions of a millisecond on something and and making inference and you know, making its own decisions based on this data rather than something because it’s only programmed to do what it’s programmed to do.

So, you know, it it’s amazing how quickly not controlling the data derails the conversation. Now, if we’re getting a little tactical on this and we’re talking about the data, you know, what are a few key terms that we could talk about here? Is, you know, is it just mainly data classification, data lineage, data deduplication? You know, like if we could break down a few of those different things that if I’m a visor and I throw these different terms out and I talk about it like I know what it is, what are a couple of those things that we can that we can say?

Mike Luttenberger (33:33.124)
Yeah, you’re absolutely right.

Duplication is horrible, right? because if one gets updated, that’s a problem. classification, I think, is important, right? And especially from a security perspective of having those classified, whether they’re confidential or not, because you know, some customers have it where yes, you can use AI, but you can’t put like customer data in it. So classifying the customer data so that you could safely use, you know, AI is important, in my opinion around that. So yeah, those are some of the things that I definitely listen for. And also even

retention policies and again this could kind of fall under data and security. having that stuff purge itself or archive itself to get it out of the environment because

Maybe you you do have good data and maybe it is you know, we don’t have duplicates and we didn’t make, you know, 2021, 2020, we didn’t do that. But we have thirty years worth of stuff. Well, that’s also not relevant anymore, right? There are things that might be an environment that again, clean data, but if it’s from twenty-five years ago, it probably doesn’t relate to today. So having those retention policies, like I said, to either get rid of that data, try and have it have always kind of the most current things that are relevant to the business are very important too.

Jason Kaufman (34:44.961)
Great, great stuff and great, great nuggets to look for and great insights to provide. You know, so that kind of leads us into you know the final couple topics, which is, you know, the advice for partners. You know, we have the tech tech advisors out there that we know we team with, we partner with, you know, they’re doing a lot of our work with the clients, you know, opening up those conversations. You know, if we wanted to take everything that we talked about today and kind of, you know, TLDR it and sum it up into three or four points, where do you, what would you say you would like

Partners to take from this episode.

Mike Luttenberger (35:17.751)
don’t rush into it.

all the things kind of we talked about to date are more foundational. Again, I know people want to get going really quick on this stuff, but quick can be a disaster if those things aren’t in place. I say kind of take the time to get those kind of things in order that they need to be. And then once it’s turned on, it’s you know, you’re good to go, in my opinion, around those things. So that to me is kind of those taking a step back. I know no one wants to take a step back. Everybody wants to jump forward and do this thing. Cause again, I hear my executive team has told me we have

To do this, my competitor is doing it. I get it, but I promise you it’s gonna hurt so much more if you rush into it and the data is everywhere and no one’s trained, and you spent hundreds of thousands a year, millions of dollars a year on licensing, no one’s trained on it, the data’s a mess, people are finding salaries they shouldn’t. It’s HR nightmare. I I think those are kind of the just thinks take a step back on everything.

Jason Kaufman (36:13.421)
I think that’s that’s great stuff right there. And I got one more question before we before we end this, and it’s the the best one that we always look forward to. so let’s say you you had that crystal ball, you know, you gotta you gotta say what’s what’s going on in the future with AI. You know, you’re you’re the you’re the foremost AI knowledge pr repository in the world. You know, you know, not talking about Elon Musk, not talking about Sam Allman, not talking about all of them. I want to know what Mike, be like Mike says about AI. What are we gonna see here in the future?

Mike Luttenberger (36:41.82)
sure. So this is something I’ve been saying for over ten years now.

it is going to be wearables. So it’s gonna be some sort of pennant that sees the things and hears the things that you’re doing all day. It’s obviously AI is driving that. Or glasses. I think that is going to be the next piece of this. We’re already starting to see a little bit. Again, I think Google Glass was ahead of its time. I just don’t think people were ready for that. but I think that is coming back. And again, I think what’s happening now is we’re we’re being trained right now to type things into AI. And obviously it’s getting more conversational where I think people are going to be hitting that

Jason Kaufman (36:47.425)
Wearables? Okay.

Mike Luttenberger (37:15.437)
Some do, but most don’t. They’re gonna hit that microphone button and have that conversation and that’s gonna help kind of bridge that gap with obviously wearing glasses or something where I’m not gonna have a keyboard, but now I can talk to AI to find me directions, find me a recipe. If I’m sitting in front of my stove and I wanna help, you know, create you know, create a meal or something like that, all of those things I think are gonna get be in play. So glasses and or some sort of pendant or pins, some sort of wearables, that is the next step of this.

Jason Kaufman (37:43.349)
basically just described in a really long way, augmented reality is gonna be the future. If we’re gonna put it put a word to it, AR, augmented reality, wearables. Yeah, now now apples get in that game too. That’s gonna be exciting ’cause I’m an Apple person.

Mike Luttenberger (37:47.203)
You got it. Yep. Yeah. You got it. Yep. All the way.

Mike Luttenberger (37:56.887)
Yeah. Like I said, I I I kind of made my my shift in twenty fourteen when they showed AR kit at one of the WWDCs and I was like, this is this is it. And then again, you’re bringing in machine learning. So I’ve been on this train since twenty fourteen. and I’m not I’m not getting off it. This is definitely the next thing.

Jason Kaufman (38:15.073)
All right. Well thank you, Mike. Let me let me say it right here. Just a reminder for everybody, episodes drop every Wednesday on Apple, Spotify, wherever you listen to the best of the best podcasts, we’re always top ranked. If we’re not top ranked, you know, you can find us real easy under Telarus and next level biz tech.

Jason Kaufman (38:34.903)
So thank you for listening. I’m your host, Jason Kaufman, Principal Solutions Architect at Telarus. This was about implementation confession, the AI disasters they don’t want you to know about and how we solve them. With Michael Luttenberger field CTO of Modern Work at Netrix Global.