Episode 6: The business impact of GenAI on software organizations

hello everyone and welcome to our webinar about uh business impact of of uh generative AI in in software organization all in all as you might hear we are with with my colleague Marco Clement here in the the same room so we're using one one microphone i'm heading the um Debox and AI business at Defy Code and then Marco is our CTO who will run the the first part of the the the webinar on with with many quite interesting topics and then we will discuss a bit more about the the business impact of size but but with these words I will actually hand it over to Marco to to start the the the contents of today thank you and welcome everybody as said I'm the CTO at FOD also a Microsoft M MVP in Azour AI and I will be talking you through some of the trends of generative AI today for software development and organizations overall i will also show a few demonstrations few ideas on how AI is being used currently it will take roughly 30 minutes and then we'll have a discussion with Henka and then the last 15 minutes we will be going through your questions and as a reminder you can ask questions as we go in the questions part and as we get moving forward Hanka will either read them out loud as we move forward or then we'll handle them in the latter part of the presentation today as we already know uh generative AI is changing the way we do work not only software development but overall how we do work and how we consume anything uh anything within within the digital services and as you have seen the last two years of course the democratization of AI starting from chat GBD but also today lots of other models models available and some of the benchmarks are showing that that AI will relatively soon surpass human capabilities not only in the traditional AI parts but also in areas such as logical thinking and and uh uh reading comprehen mention and and areas such as that and we already know that generative AI has has done big improvements in generating code and what this all does is we have recognized that generative AI is one of the most transformative forces across all industries not only software industry and all of the analyst organizations see the annual growth rate is somewhere between 40 to 50% so at the same time as generative AI is changing the way that traditional organizations operate it will also open up lots of new opportunities for organizations to work on business and on the way that they produce salt i will trace back a few steps backwards on organizational uh organizational functionalities and when organizations start to introduce generative AI within their within their systems in the value stream we see that there are a few things that have been brewing already for the last 15 years and of course uh FOD as a DevOps organization has been in the very center of this looking at organizations trying to automate parts of the processes in their dig digital creation software developments and as part of this not only the automation but organizations have had to adopt adapt into a more fluent value stream reducing the bottlenecks and reducing uh like sending messages over and and and having kind of these uh uh silos within either the value stream or within the organization and the movement in development has been clear the movement has been from centralized functions where the responsibilities are shared uh across only the functional areas or components into more decentralized and this has kind of opened up opportunities for the value stream and the value to be passed through the organization faster and especially now with generative AI um it's very important that as the development is decentralizing the tool chains centralized because if we have a decentralized way of working and decentralized tooling it becomes a mayhem so one of the kind of uh underlying underlying movements with generative AI is also the way how tool chains and the ways organizations operate is centralization and if we look at the traditional way of of de DevOps way of working where developer teams are building owning and running their own uh applications and they have the infrastructure platforms at their disposal um the traditional way of looking at DevOps is that the developers will just be working on their on their code and then they will deploy it and they'll they'll be working together with ID or DevOps DevOps teams and then everything will be fine and dandy however in these organ in in this way of working the developer teams actually have to understand and consume multiple tools and multiple practices multiple processes before they can even see if the code that they have created works or not and now when we look at the decentralized development organization and the vast amount of tooling that the developers have to understand it it creates problems it creates bottlenecks and of course it creates cognitive load within these organizations and now as we're adding on top also the AI tooling it will be another layer on top of the development teams the development organization to consume and understand and instead of building efficiency we see it it creating problems and hindrances within the organization for this purpose I see one of the key movements with generative AI is platform engineering and the concept of platform teams within the organization and if we look at the developer teams working in a modern way in a modern fluent way the key concept is to provide all of the tooling and uh the environment where developers work as a service and essentially this works so that platform team or a similar team within the IT or DevOps team depending on how the organization is built provides all of the services as a centralized functionality whether it's the components they're working with whether it's DevOps tooling continuous integration security automation test automation or in the future also the AI agents and assistants it also includes the knowledge base and service management building a community around the the products that the organization is building and it's it's interesting if we now go back a bit and look at the traditional software development life cycle which starts from the portfolio management and product discovery all the way into releasing and are our users using the actual services and we look at how this traditional software development life cycle is changing in the era of generative AI um many of these parts of of the the value stream changes but one thing will remain clear if the organization hasn't uh paid attention to how they work with problems they might find along along the line the cost of fixing these problems will exponentially become more expensive the later in the value stream they find and I've been talking for the last 15 years about the shift left movement and Epicode of course has been advocating the shift left for at least 15 years now the shift left as a concept naturally decreases the cost of the defects um and if we look at finding the problems in production environments usually in traditional organizations it might be even 30 to 100 times more expensive and shift left as a movement has been even without generative AI the way how organizations need to adapt themselves in in kind of building better value for the customers but now with generative AI organizations as they start suboptimizing first on the value stream items like development or testing or security or compliance or service management and observability the same problem will apply throughout the value stream unless the organizations could focus on optimizing also the value stream at the same time i think this is a a pretty interesting like phenomenon all in all that like even though some of the sub optimizations like optimize that part but it at least in the beginning side it will still help on the shift left movement that actually have a business impact of of kind of a like finding the the the the bugs and errors faster right correct and I said this is a movement that has to happen whether there was generative AI or not exactly and it's an important basis for whatever I'm going to next talk about generative AI so as you start applying generative AI in your organizations it's important to understand that suboptimizing value stream items with generative AI will bring problems when you scale up certain parts but then you realize that for example you improve your developer capacity by 50% but you have your say manual testing or quality assurance in place which will be a bottleneck for the next phase of the speed and kind of the shift left movement or even start left movement that that helps organizations to start reacting to problems faster is actually a mental way of changing the organization to adapt into whatever generative AI will bring in the future i talked a few few words about platform engineering i I showed how the platform team should be sh serving the developer teams and here as an a picture of the so-called traditional value stream platform engineering is something that glues together kind of the business decision making and the customers and everything in between you know platform engineering we see is one of the methods how organizations can scale up their skills through the value stream and not just suboptimized in in certain areas of the of the value stream as an added bonus we've also seen of course that platform engineering will improve the organizational productivity on average by 25% and it will also reduce the tooling related costs by 40% in the longer term and now if you look at how many different licenses you have in use from different tooling and you add in once again the AI tools stamp on top it is definitely an area where organizations have to also pay attention to and maybe a a a short kind of a discussion item here is that uh like many organizations have taken with AI now the approach of like let all all flowers flow so so kind of a every team wants to try their own right so so so kind of a it's somewhat opposite to this but then this is the way to actually take the control back and then get the business benefits in the long run right and and that's quite interesting movement is that like we are letting many many organizations to kind of a try out and experiment a lot but then when we go to a larger scale adoption you have to take the platform engineering ways of working into use into to to get the benefits and even if you didn't call it platform engineering the concept of building uh a common platform throughout the value stream is something that organizations have to one way or another solve in in their future and as Henka said kind of the benefits that you see in your innovators and early adopters of the organization the ones that have automatically started to use all of the new generative AI tooling and have even applied it in their in their department or the software that they're developing it is we see still an area where organizations tend to think that now we've implemented these tools within the organization and we're getting we're getting all of the benefits out however organizations will actually get the biggest benefits when you scale these practices and tooling across the whole organization this is also the reason why I started with the concept of platform engineering so having the majority using the best practices using the common tools and having being introduced the new tools the early adopters and the innovators of your organization's organization has will be the key to unlocking the benefits and this is why in order to break the barrier between the innovators and early adopters in the organization you have to find a way to scale them up to the majority of the organization and one of the best ways of doing this is the concept of platform engineering or platform team or a common place where all of the developers consume these services i have to raise shortly about the like the talks where we are in when we discuss with CTOs and and and many organizations they are like we are like heavily on AI when they actually mean we have few innovators who have done so so the business benefits really start to come as with the the the the whole uh uh picture shows when the maturity actually gets on with with AI and when you integrate it to be part of AI and I think this is super important thing to understand from the business benefit point of view for the AI it's the same movement we'll we'll see the same movement as we originally saw with agile all of the organizations say that yes we are already doing agile where I could say that the organizations aren't doing agile even today yet and similarly organizations say that yes we are doing devops where is the organizations definitely aren't yet doing DevOps and this even applies today even while most of the the practices and public cloud have become a mainstream many organizations still strugg struggle with parts of the value stream and building kind of the fluent capability of releasing to production environments good this concludes my introductory part into what organizations need to put in place to actually unlock the generative AI benefits in the future and next we'll move on to looking at the AI adoption and maturity framework that we build for the purpose it all starts of course from assistance so today most of you are already using some sort of generative AI assistance uh chpd could be one something that helps you achieve a task or understand something many of the things that AI has brought into the into the regular people and within organizations where like that's the first step of the the AI adoption uh framework so AI comes in and enhances the productivity and modernization of applications in this five steps uh maturity framework the next step is then agents and we've divided the AI agents and autonomous AI agents on purpose because the AI agents are something that's initiated by an action that we do as working in the value stream in a various roles like a subsoftware developer for example this is where most of the organizations are now we see that we are not talking about percentage uh benefits but instead we are talking about orders of magnitude and the organizations who have been able to implement agents within their organization in a successful manner they will be able to uh get to 30 to 50% uh speed increases in the development and they can overall get to kind of this 2x efficiency within the organization and today if we look at how uh how agents for example would work you all know cursor AI you know uh GitHub copilot and co-pilot would work by saying for example create a react and bite project to the root when I click enter my agent will start working for me on this particular uh particular uh task and what it will do it will create the agentic mode now you're using right correct this is the agent oh yeah this is the agentic mode from GitHub copilot it's been out for a few months already so what it does you'll see from here on the left it created me a structure for the project i started from an empty project now it installs the dependencies for me which will take uh a few seconds you will see them installed there and now as they're installed it will make some more extra configurations for preparing for the generative AI development and then it will eventually uh either start the the project automatically or then I'll skip to start it but as you'll see this is the kind of a work if I started from scratch um yeah yeah if I started from scratch I would have to create and the bootstrap or the project I would go and have to see how this react and project would be started and now we've moved into the world where agents from my initiation will create all of these kind of basic repetitive things for us and it doesn't limit to just giving the answer it will also be actionable and now as I open this page you'll see I have V and React project here and if I wanted to start working on it I could just go here into the agent mode back and tell what I want this homepage or application to show me and this then you can already imagine if you com connected this to your design system and your template library starting these projects from your organization's template library with the kind of design you want to enforce this is super efficient for your organization and maybe just to add that that no one gets the wrong impression is that like this isn't only about for the coders it's the same for like pro product product owners or requirements management or testers we can actually kind of we've done already with multiple different organizations these agents that do these things automatically for them and and we've seen such a huge uh uh like efficiency gains from from these agents yeah and then the next the third step of the maturity framework is autonomous agents and these are the kind of agents that start autonomously and a few examples um I'll I'll mention that going from agents to autonomous agents is where we see currently the leading organizations are today so they're already moving from this kind of agentic way of working into building autonomy into how the agents work so instead of working in a browser I mean instead of working in an editor I would for example when I have um a requirement this is from Atlassian using Atlassian robo in in Atlassian environments the you can create agents relatively easily i I've said when issue is created and the issue type is story then run this agent called readiness checker and now every time I create a new tickets what will happen is when I create the ticket and I give it a description it will run me a readiness checker automatically so kind of autonomous agent which is initiated in the background runs a process for me and and then either does an action or informs me somehow and now for example this readiness checker what it does it checks the completeness of the task provides some details but lacks the key information like acceptance criteria well I don't have acceptance criteria here clarity is good uh my readiness checker deems this this clarity to be okay but then when I look at the auditability and estimation there is no way for a human nor the AI agent to understand what is the current auditability or or estimation of this particular task this is one example of autonomous agent the other which was released in in Microsoft build which is I think super interesting is the co-pilot coding agent it's a new concept of working i have this application called uh weather app which shows uh it's a weather monitor I use back at home you'll see I'm finish because there's sauna it's still a bit warm because of yesterday evening and uh now if I you'll see from here add battery indicator to the temp boxes so here are the three temp boxes or temperature boxes and they are using the ruby tag technology but I don't know what is their current battery status so as they're emitting the Bluetooth uh signal and they will of course consume battery and I don't know the battery status so my previously I would have started to work on it on an editor and like two months ago I would have worked on it in an editor but today um in in GitHub GitHub you have in the issues add battery indicator to the temp boxes I've created this requirement with the same information I had for an example in in the Atlastian environment and what I've done is I've assigned it to co-pilot and what co-pilot has done it's created me a pull request add battery indicator to tempbox component it's made a branch of it it told me as a description what it did did as the changes and because I'm following continuous deployment practices which I'll talk to you about more in a second I already have then the preview away available it's not in production environments but it's in a production like environment with the functionality here as the red dots next to upstairs bedroom which tells me that these are the battery indicators or the the tags that I should go and change the battery to and I have not written one singl