AI in ITSM: Balancing Innovation with Responsibility - Fireside chat with Forrester

artificial intelligence is rapidly reshaping the way IT teams deliver value in this webinar we will explore how to harness its potential without compromising trust security or human connection my name is Gary Blower i'm a service management practice lead here at Efficode and I am joined by my fellow panelists who I will allow to introduce themselves starting with Julie hi my name is Julie Moore i'm a principal analyst with Forester in the my coverage areas are in ITSM ESM and knowledge management thank you Julie and Magnus hello everyone thank you for having me my name is Magnus Sunset i'm located here in Stockholm Sweden i'm also a U service management practice lead here at Ethico trying to answer all these questions and on that note if anyone does have any questions as we go through the uh fireside chat here then you can enter them in the Q&A panel below in Zoom um but what we'd like you to also do if you ask a question do include where you're from because we're fascinated to find out where our audience is participating but let's kick things off with our first uh question really for the panel uh and we're going to start things fairly simple and we're going to ask what does AI in IT service management really mean today and Julie as someone I know who's done a lot of research on this I'm going to throw it open to you first so it's it's a it's a wonderful topic because AI probably three or four years ago meant something very different than it does today it it is a collection of capabilities everything from traditional AI to machine learning natural language processing but of course now we have the new entrance into the area which is the generative AI and of course agentic AI um but essentially what the addition of AI is doing in ITSM is it's allowing us to take a lot of the rope work or the the repetitive tasks that we have and put more reliable automation behind uh the scenes so it it can span everywhere from incident to problem to change to config um making what we do uh more reliable oftent times adding uh a higher level of consistency in the environment um you know there there are capabilities like being able to summarize note not note fields and and put that into a record you know often the human in the loop with the pressure of getting on to the next call or the next ticket might short change what they write but the AI right they're able to create that summary and add it to the ticket so we get really reliable consistent results when sometimes the human in the loop um doesn't have the time or the patients to do some of these activities so it is uh a very comprehensive set of capabilities that span across IT service management thank you J Magnus we got any sort of thoughts on this as well yeah I I I totally agree with that and also to emphasize a little about bit about this summarizing thing with with I I've seen companies having tons of comments for instance in in their tickets and it's quite cumbersome to read through all of that so so giving a practical example of that would be you know if you have a major incident people have been working maybe for a couple of days on this huge incident that there would be like hundreds of those um comments and just by summarizing the whole ticket you can get up to speed rather quick when you're coming in it's like you're coming into the sort of the situation room and something is going on there what's going on but on the subway there or on the car you can just summarize what's going on you're coming in there so people in the room they don't have to sort of brief you about everything that's happened during the last two uh 5 hours or whatever because there's so many things going on at the same time maybe so as long as people are are writing down things into those tickets and so we can get these summarizing so I think it's both a lot of automation yes but as long as people actually add information and write down what they've done then you can get good insights and summarize uh summarize what's going on and I've seen this multiple times when you have like so many tickets and so many comments and then um after 12 hours at least here in Scandinavia you're you're not supposed to work more you have to go home and then the next team comes in and they then they need to to have a brief and you're super tired in the head you need to go home and well I didn't just read the ticket then you'll understand getting that summarizes summarizing the whole ticket would be super good so I think that's a good value thanks Magnus and um funny enough we were talking about this before the webinar about an hour ago and I was explaining how uh one of our uh clients actually been working with in the past week presented us with a problem that previously would have taken us quite a bit of uh effort to actually resolve and they've got a the customer basically receives a lot of requests and most of those requests come by email and there all sorts of different sorts of requests and today it's a kind of fair well up until now it's a fairly laborious task to kind of triage each email and work out what the the customer is actually trying to ask but by using um an agentic AI we've been able to streamline that so that the aentic AI can actually read the emails interpret them determine what type of inquiry it is classify the ticket and then speed up that first step so it's really a kind of you know from my point of view I see it kind of as an enabler as Julie said um to removing some of that less important lowv value activity that we we previously had to do manually um so so far we've only kind of discussed the the benefit to us as IT professionals there is also a significant amount amount of value to customers so when you push the the AI through channels like chat or through your portal um you know just using a rag search you're able to better connect content to the end user you're better able to understand what they're looking for you can surface answers instead of a list of results you know typically when you're using something like a a Google search right you're just given a list of results to go through this will can actually generate an answer for the end user um kind of simplifying their access to information and the ability to interpret what they're asking for so these capabilities are not just giving it people superpowers but they're also really helping to make a a better employee experience for end users what would you say just to follow up on that Julie what would you say for kind of providing self-service to customers so that they can answer those questions themselves what's the kind of key things that need to be in place for that to work successfully uh you know there's a there's a variety of of things that you have to pay attention to and I I still believe wholeheartedly in journey mapping because it really does help you to understand exactly what's going on with your customers um you know I I think over time the portal will actually become not as important of an interface to customers because the more that we put through chat and the more conversationlike it is for end users you know I don't have to stop working in the application that I'm in I can pop up that chat window right there and be working you know to help resolve something or to access or get answers to something so it's keeping the end user in the flow of doing their work which is um so much more productive you know the disruptions and the queuing and things that we've traditionally done in IT service management we can create a much better employee experience for that um so I think it's really important for us to focus on that where can we reduce steps in the process where can we add value where can we automate so that the end to end experience of the end user is so much more enhanced than it is in the way that we're doing things today uh same similar question to yourself Magnus um so how can we yeah how can AI improve and accentuate the experience for the for the end customer yeah just speaking of the knowledge and all the information that is out there if you have really crappy knowledge database wherever it sits you need to sort of start reducing the amount and and go through it um and sort of unpublish things archive things and make it maybe if you want to make AI unaware of it for for several reasons i mean if you send an an attachment in an email it's I think it's there's some baselines I've read 10 years ago about it saved 10 nine times or so in different locations and then it sort of evolves over time so reducing the amount of documentation that you have you can also use AI to do that so your knowledge databases are becoming better and you can rewrite it easily with AI and and make it more up to-date and then when you're at it you can also use technology to to uh put like a clock on it to be revised uh within a year or so but then also we have those at least here in Europe the AI act coming up and that sort of brings other legal aspects to the table if it's AI produced or not which is uh something we all all need to be aware of I would say but using AI to to go through the documentation see where it sits update it that that is super helpful I've seen um kind of um following on from what you both said actually an example I wanted to to share was uh a chatbot I'd seen that um did what I consider to be a really good thing from a customer experience point of view in that it attempted to resolve the the issue for the customer through that kind of basic chat in exchange through whether it's like a pop-up window workflow or whether it's through like a customer support portal but if you said to it you know um actually I think I need to escalate this to to to to an actual you know help provider that it actually gave a really precise summary of the whole kind of conversation that had been had and also gave context uh when it actually handed over to the human who then picked it up so it made that a very seamless experience and I think anything that can be done to improve both the customer experience through the AI but also through that kind of um collaboration with the kind of human at the end of it I think is a is a is a is a goal to achieve as well i mean quality assurance of documentation in that sense is super important that you actually know it's not an hallucinated article somewhere it should be sort of qualified as real documentation so we don't uh you know spread too much of the like wrongful information you it should be a person behind reviewing it it makes sense yes still uh because the quality I've seen sometimes it it does hallucinate as you know Gary we've seen that when you ask an agent can you do this ask the same question couple of times it sort of keeps the conversation in a repository and then then it sort of derails right yes um well the I I I want to talk about that knowledge piece because you know we can't necessarily fix poor knowledge management ment practices by applying AI the the hallucinations can happen for a variety of different reasons if you don't put the appropriate guide rails then you know the the system is trying to do what you're asking it to do and it doesn't have the information in its training and so it essentially tries to make it up um that that's the the primary hallucination that you know gives us incredibly uh great stories in the media you know it it definitely gets a lot of PR but there are also hallucinations that happen because it's surfacing out ofdate no longer relevant information um and it's really important under to understand that yes within a a knowledge management practice you can use AI AI to improve the results of knowledge management that's that's for sure but these engines these um uh large language models they require really good solid if you have greater confidence in the training data you'll get better results um coming out of the the questions that you're asking and so yes we can look for duplicates of knowledge we can use it to help you know create templates we can use it or or apply templates to knowledge we can use it to even create new knowledge but the under underlying fundamental practice that we need as an organization to make sure that we're regularly going in and making sure that information is updated if you do find a hallucination what do you do about that um you know there are new roles within the organization that are going to pop up that are are around the nurturing and caring and feeding of these models to ensure that the right knowledge is getting in there for training um and that we're not feeding it misinformation following on from that then Julie so obviously to nurture and maintain knowledge like that that comes with a significant investment and cost so how how can teams actively measure the return on investment they get from making these improvements in terms of the success they get from their AI so what I've seen so far and it this is a is a really good question because what I'm seeing is that people are looking at our traditional metrics and looking at the impact of those so for example applying uh you know some some form or many forms of AI within incident management we might look at things like first contact resolution uh def deflection being solved by self-service without actually having to come in to to speak with a live agent um and MTTR right those are our traditional anchors but often these systems are are capable of and can work autonomously without any human in the loop so how do we measure the value of something that it is you know found an anomaly and it's corrected that it's determined that that anomaly needs a corrective action and it's corrected it you know that's that's where we have value but we don't necessarily have metrics today that are a good indicator and when you look at the traditional metrics like uh you know first contact resolution or MTTR when do we start measuring that do we start measuring it when the technology is trying to resolve it or do we start right to the end user that's one experience it doesn't just start because a human is now in the loop so we have to look at what we're measuring and understand what is the end to end value that's being uh provided to end users um you know not just you know are we uh solving things more often on first contact or you know the the total meanantime to repair so we we've got to really look at those metrics i believe that the return on investment lie in new measurements and new things that that we have yet to develop i I think it's a great use case for the AI itself that when you know we are in in a perpetual cycle of using and improving that we ask the AI how do you measure the value what have you gained for us how many minutes have we saved how does that equate over time because if it's fed with a lot of data from our service management system it knows how long it typically takes for that type of incident to happen and it can say wow we've end to end we've actually saved three minutes and that equates to x amount of hours uh per year so I think it's an opportunity for us to look at evaluating how we measure and come up with new ways to understand the value of these systems we're going to be doing things completely different than the way we've always done them you know we we have you know processoriented operations that's run like a a kind of a factory factory floor this is going to require new models new ways of thinking um new ways of of serving customers and I think along with that we need to evolve our metrics to understand the return on the investment and the efficiencies that are gained thank you for that and I it's one of my favorite subjects i don't want to derail the fireside chat actually is IT experience management and I can see value in measuring um you know customer experience and things like XLAS as well as as a way of measuring the effectiveness of the AI in terms of the overall the value return to the customer so yeah you're you're right there's lots of um there's going to be a lot of change I think is what you're basically saying over the next uh next few years but organizations are not thinking about this i I've spoken to a lot of companies that are early on the adoption cycle and unfortunately they are just not you know putting thought process into how am I going to measure the value of this because if it's going to expand in an organization you know for different use cases you need to be able to prove its value and that should be something you're thinking about before you roll it into the environment so that you can kind of baseline where you are today and then understand that impact uh on the environment so it's definitely something that absolutely needs attention yeah thank you for that um so I'll go to this one for to yourself Magnus first so if um you know if a organization or a team is looking to start their AI journey with IT service management today where would you advise that they they you know their starting point should be where you stand no start where you are and and really optimize your area where you can actually uh start with as an individual maybe use a couple of tools start learning the what it is about and and try it out yourself so you know understand what everybody else is talking about and when you're entering the team as a team uh how can you improve the ways of your working you should always sort of put AI sort of as an appendix to every process that you have maybe you have incident management how can we uh optimize and automate through AI you know like a continuous service improvement idea so that should always be and that's that's nothing new really but AI is a new as a tool so we should consider it as a new tool as well right now we're experimenting with the assistants and agents and and these things and when it comes to when you actually start using them you need to put them into the correct area where they belong sort of and understand what they're actually doing take ownership of the agents or whatever you have out there they're doing some they perform these tasks uh somebody need to update these agents maybe uh over time because the process are changes changing etc so taking ownership of of your agents the agent bots and and what we call them uh is super important and just don't open up oh yeah now let's everybody starts using it yeah but what do you want out of it because it could be counterproductive if you don't understand what what what is going on there for instance if you have you start developing really fast using CI/CD pipelines and you're doing co-pilot creating faster and faster software then the rest of the organization need to follow suit it's the same thing with AI the pastry I can produce my material well then it becomes the other guys could be a bottleneck if I create tons of blogs maybe and the the editorial needs to go through it before publishing and then that becomes a bottleneck so why should I then create so many articles if it's never been published so because I can use AI to do those those things and a little bit dark behind it and then it it it sort of we will create a lot of new bottlenecks that we haven't experienced before because some areas may not be fit for this purpose of using AI and we we're moving 10 times faster you know it's like you have these guys sprinting super fast in marathon yeah they'll sprint all the time and in a couple of minutes they run their mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar mar marathon but then you have the other guys that not using a be slower so there's like like um rubber bands some people are like yeah fast and some are not we need to that's the new reality I think okay and and Julie what does the I know again part of your research what does the what are the sort of um if you look at uh organizations that are starting their that have started their AI journey and the ones that have done that successfully what's the kind of common attrib