Episode 5: Building AI on your Terms—Insights from Eficode’s AI agent factory

welcome to everybody who's attending this webinar series top AI tools for DevOps and software development And in this episode we're talking about building AI on your own terms insights from Ethicode AI agent factory So we're going to talk a little bit about what we've been building what we've been building internally and what kind of uh demo environments we've been doing about the whole AI factory concept and talking about what's actually going on around that Hi my name is Heredo and with me I have Alex here as well from Eicode Hello And uh what's on our today's agenda we have a little bit of slideware as that's that's what consultants do I'm going to run you through a little bit of the background on what even does an AI factory mean what's what's actually happening there and then Alex is going to show a little bit of a demo on maybe not after after you get all the definitions right maybe not a 100% true AI factory but pretty close to what we've been building what we've been building on there to actually build up those software software factories using AI and then at the end we deser have some time for questions It's it would be good to put the first poll out to get some some nice uh feedback from the audience I just want to a little bit of scope on uh what you guys are thinking about what have you done AI in your company or in your context So I know a little bit about what what I want to talk to you Most people are seem to be well still planning Yeah And using tools is currently the leader candidate in here and then a little bit of a distribution on all the other other fronts that we have in here So mostly we have people and I think this is quite a typical typical distribution of people and when we're talking about AI that some are still planning on using it most people are using tools like JTP copilot a lot of the other branded tools to already engage with AI and I think that's now common place in everywhere and you should be using these tools Then we have some people who are already going into the custom assistance and custom agents part and that's a little bit of what we're going to discuss and uh I'm kind of well I don't know if I'm glad but agent frameworks or agents using agents framework is the kind of like thing that we are talking about today We are talking about the end level of uh how we want to build hybrid AI and human interactions in an organization How do we want to build that okay Uh so I want to talk about a little bit about the context so that we get everybody gets on the same page So mostly when we're talking about AI we're talking about in many contexts people are talking about that is bit is the biggest opportunity now it's going to change everything is going to be so great and uh then in the reality not many people are actually actually getting it into production and there are multiple reasons for that uh I see the opportunity pretty much and and that it will change stuff about 60% of software teams using AI see significant productivity gains I believe that 100% I I myself I'm also doing a PhD on software engineering and how do you do software software cord products and how your software is actually actually affects your business and because of the way that you can use AI to change that I see a lot of lot of productivity gains in there as well and exactly the truth has been that there is many many of data nowadays that shows that airport tools actually help people deploy deploy faster but then the problem is also that many companies are not fully prepared and integrating getting that AI into existing workflows is harder than expected and this is the kind of like part that we've encountered a lot in F code and it the sad thing that is it is it pretty much comes down to basic DevOps you have to have all of your metrics in place you have to have some automation frameworks in place you have to have access to data and all of that to actually do AI so you first have to do DevOps and then you can jump onto the AI train but there are some low hanging fruits that you can do as well and by the way I was reminded that please if you have any any questions you can use the chat you can ask me just uh just use a chat and ask at any point if you have anything that I'm kind of glazing over you want to hear a little bit more about or then put some questions on the Q&A to answer them at the end of the whole presentation So this is what I was talking about when I was talking about the five five phases They were named a little bit differently in the poll because uh because of easier understanding but basically what I'm talking about is many companies now are going through this five-step ladder and I think it's a ladder that you have to first just figure out the tool usage on the phase one just get AI tools into developer productivity and then you do software you take those co-pilots you take those lowh hanging fruits you take those tools off the shelf and start using them next I'm kind of like mashing together software agents and assistants Because assistants are what I call static in the sense that they cannot operate independently They only operate from a human trigger and that's what triggers them to do something And an agent can work on any kind of other trigger as well Be it another agent or be an environmental trigger or be a build order or something like that So so that's the kind of definition that I'm using between assistants and agents here And I'm mashing them all up now together to software agents that then do single agent workflows For example check this code Could do do code review for this bit of code Do this single thing for me And the next part is that we are going to talk a lot lot more about is AI orchestrated workflows How AI actually triggers other pieces of AI as part of your software life cycle and your software whole cycle Phase four takes all of your business requirements and start integrating that into your whole business And how do you run your software development team the integration goes much deeper than uh than just inside of your software team And this is probably going to be the most difficult part in your organizational culture to do because you have to go into marketing data You have to go into all these other places that then feed into your software priorization and all that to do it And phase five is pretty much I haven't seen any pretty much any companies working on phase four or phase five And I gladly hear if somebody already has a company that works like this that basically AI is integrated into all steps I can do autonomous decisions on phase five It can do business experience and can do that We are not there yet We are somewhere between when many companies you are in phase one phase two and our demo is mostly talking about phase four and phase three Three and a half is is what we're going to demo on the software factory front on how do we actually get to the whole AI native in software business and thinking which is the dangerous word here as well Uh and it's I said it's mostly about fear how do you get people do the AI will replace development no it won't It will empower them give them a lot of more power to do the same thing in software workflows Who owns it who controls it this is an important step that we have to have a human controlling all of that to actually scale because scaling is difficult because it's all about how do you manage complexity so that you actually can make the AI behave as an asset not a risk The second then is as as well can I trust AI in coding that's why you use a lot of these different normal day-to-day operation styles For example in coding why why do we have code review why do we have tests why do we have acceptance test because you cannot trust the developer you cannot trust anybody here So so a trust is a little bit of a strong word but you basically have all of these methods to already build that trusted So you just basically integrate your AI into the same workflow that you already have with the same tools as you integrate a new team member It's the same way you integrate a new feature into your development Just integrate in the same workflow and Alex is going to demo a little bit on that and how do we actually do that in in uh Jira we've actually done something horrible which is made Jira into a software development tool but you'll see more about that uh and and the fun thing is that the last lack of business case is something that we've heard a lot as well about why are we not taking AI into because our company does not see it being profitable because we do not see how it boosts us and that's that's something uh that this talks about us because I stole our marketing slides for But basically it says that you have to look at the road map and how integrated all of that because AI is a booster in pretty much all stages of software development that you can use in all stages where you're using knowledge that can be replicated in some way that should be replicated in some way you can use that for long-term success And this is pretty much outlining the same phases that we have I'm not going to dive deep into that but I just want to highlight the fact that we see in all of these steps huge coefficients for more efficient business that's going to happen And you start from all the tools and you progress through the whole whole all of the five steps that we have here and you're going to end up in some place that is going to be superbly more efficient in building it and also like how to do it And this is that then the question of what kind of tools and what kind of tools and what do you do to actually get to the phase five or phase four because even phase four or even phase three is going to be a huge leap in many kind of phases and this comes back to uh agile principles or other principle that there basically two ways to manage complexity and two ways to uh scale because that's what scaling is you build your company to be bigger and bigger and bigger so you accept more and more complexity into your company and into your process So how do you then manage this and basically make that scaling profitable you can scale in a way that makes you unprofitable and then go gloriously bankrupt But you can the ways to do this is create ways to manage complexity So adding control structures adding quality gates and adding all of that around your complexity or getting rid of that complexity by simplifying processes and streamlining other stuff You probably heard this from management call many steps But these two things are key in AI integration into your culture Not just uh building the actual tools but how do we integrate that AI into your company using AI to manage ways to manage complexity or using AI to get rid of complexity doing refactoring or doing systems that can go around that uh and here's an example this is what typically when we talk about software development we typically talk about that hey we we do coding we we do coding that's the thing that that's where the value is done but that doesn't really happen it's actually if you zoom in on that left there's two tickets that says check out from version control and developer does a commit it's basically a manual step nowadays and it's been a long time that's what happening Uh but if you look at the whole value stream there's a lot of other stuff that not just the code generation it's not just the testing or something like that It's many of the value stream steps happen outside of the developer doing tests and all of just a small part of that's been typically said as the value creation happens when the code is done But actually getting sure that what should be coded Are we coding the right thing are we doing Oh sorry Sorry about that this one Are we doing the right thing did we actually do the right thing can we verify that it is actually performing the way that we do and how did we actually get it to the front of the customer to actually use it these are all part of the whole value stream that can be seen And this is where the kind of like AI can be used in many of these steps to build your software factory It's not just about copilot writing code but it's about first step basically getting the first points of data When you get something from your customer for example a customer request you can get it from a call center and transcribing and integrating that into R&D An AI system automatically transcribes those phone calls automatically takes them up and then puts down to the R&D backlog For example this is then goes into requirement specification management where we look into what should we actually do choose some of them and then do a lot of like adding additional information to those tickets to make sure that we doing the right thing And this this is where pretty much we start our demo from and getting that done The third balloon which has three dots and it is basically the old style of just just automating the code generation just doing GitHub just doing all of those tools to help you to code The second step automated note generation for example automatically generating release notes from what has changed in code what have your tickets ids been and all this there's been let's say normal meth methods for this al generation as well just taking the headers of your dura tickets that were closed for example but typically if you know how tickets work that doesn't really map out to the actual features but enhancing that with an LLM who actually reads those creates those change locks for example would be much better and then when you're actually operating ing it have an AI check what's actually happening in the logs make up alerts and make up audit reports and acceptance test with Kerov for example to just generate a lot of these other steps in software development pipeline basically what we want to build is that we we want to build an AI system from simple basics this is this is probably the bas basic system and again stepping up the ladder to read you lead you to the AI factory is is how do you do proper prompts that you feed into LLM to get result the basic agent workflow or or assistant work of the hey I want you to do thing X please please please do it in this and this way and give me the result this is what you get and we can for example use this very basic strategy to just generate test for getting backups or getting specifications for for example for testing but by itself if it's not or integrated into your whole work stream on what you're doing and what's your context on this typically these are quite useless so what you want to do is you have some kind of memory and feedback loops in here So the most basic version of this is for people you write those tests a human checks that they are okay You save them into the repo and then you have those as part of your whole repository to feed him back into the LLM This is basically building memory into your test cases on what your software should be doing and we can augment that then with many other ways and this is where we start delving more and more into this whole AI factor and using multiple agents to do this and we start adding rag So best realizes generation we start adding more and more context and more and more layers for the LLM to work with We have some kind of knowledge base that runs a database that has the data in it This database in our case should be any kind of data about your product about your clients about the needs what's actually going on in there and how do you integrate that into the LLM to provide you answers from your context and not just the uh basic general level context that you get from a basic LLM And this this is why I think these custom agents and custom assistants are very important because you give them your context and give your context priority over general knowledge and one that then is very important in this this space then at least is when you do this you have to add guardrails into it and when we're talking about software development this is pretty much testing and validation You could even say that the first bubble should be unit test and the second one being acceptance tests on what we get and those form the core basis of memory and guard fails for your system so that you can build up these AI factors to automatically conform to these cards and these limits that you are actually building and this enables us to do very rapid development on new features or new development phases that we have and that's pretty much the core of what we want to talk to you today For example I'm just showing you an example of a step or agent that we could add into the workflow is that you could have a security test professional can with context like hey can you give me some kind of instructions to use non CVS photo attempt this kind of stuff and add this into your workflow Hey can can a testing a security test professional check my ticket comment on it what is actually dangerous in it and should we do something about it and then you get that to the ticket in the same as a human security professional could maybe comment on that And you build up this stack of knowledge that you can then use And this is the thing that in the last space you can start adding a lot of tools now into these AI systems with MCP servers and others to pairing up that context from outside of your space as well We we actually did our demos before MCP Uh but basically building around the same kind of idea that MCP does MCP still has problems like it doesn't really do security It doesn't really do aification and it can allow for a lot of nasty things like remote code execution But there are other frameworks that you have to build on top of it to do that security to do the access control to get it work But it will open up the world there There's multiple different everybody's not building an MCP server in their application And that's actually one of the problems that you're going to have hundreds of different MCP servers and running different versions running different stuff So it's going to be management nightmare So that's why I'm expecting that there's going to be a something else to replace MCP or some higher level MCP like uh Kubernetes came in to control Docker for example and that's what basically what we've been building around the whole system and how do we run our agents and this is I said PMCB but we are now also integrating SCP into it but basically the idea being that we can run AI agents in any any platform or it's just compute it's basically you anyway want to run your agents where your data is because moving data moving all of that your context is going to be difficult It's going to be painful I don't think you really want to introduce new places where you want to copy all of that data You want to run that wherever your data is build on top of that be Azure ABS cloud or even locally with NV and MVR and NV partner as well O been to playing around with that and then have a central AI broker and a management software I said like Kubernetes to manage all of those agents which you can then build on top of all of these different agents that we're going to soon demo assessment advisory creation and on boarding evaluation monitoring a lot of different phases that we can do which then work on your data in in the multiple different data sources that you have and this brings uh brings us pretty much into demo time so it's not just me showing showing you nice cogs from slides but we actually have Alex here who's going to actually demo something that we had built which which I think is quite cool on on on what AI can do for you in for example J Go ahead Alex Thank you Henry and thank you Henry so far for the presentation All right let's go to the demo But um before we actually start it I want to I want to show you some background information before so that you can follow better the demonstration here So let's go to this uh book speed solutions which is our frictional