DevOps news: DevSecOps and AI-driven automation

[Darren] Technically speaking, following the rules, the io domain should be retired within the next five years. [Marc] Welcome to the DevOps Sauna, the podcast where we dive deep into the world where software, security, and platform engineering converge with the principles of DevOps. [Darren] Enjoy some steam, and let's forge some new ideas together. [Marc] We are back in the sauna. Good afternoon, Darren. [Darren] Good afternoon, Marc. [Marc] Correct as usual. Yeah, it's always a good day when I get the time of day correct. Indeed. I would love to tell our listeners that at the time of recording, it's a beautiful autumn day in Helsinki. It is, however, an autumn day in Helsinki at the time of recording. I think it's time, Darren, to have a look. We try to do this about once a month for our listeners. Let's have a look at the news. [Darren] Yeah, it's been quite a while since we've been able to discuss this, so we're probably not just covering the months that's occurred, but I guess everything in interest that's happened since we went on our summer holidays. So got a few months of stuff to get through. [Marc] Yes, indeed. There's going to be, as usual, lots of AI topics in the news lately. AI will soon do this, and AI will soon do that. But one that popped up that's quite interesting, I think, is that AI will soon automate DevSecOps governance. So what's going on here, Darren? What is this automating the governance of DevSecOps? [Darren] Yeah, there was this panel at the DevSecOps Virtual Summit where the Chief Security Strategist for OpenText was talking about this new model for automating governance in DevSecOps. And it was interesting, but it's kind of like automating tools that have already existed for a while and processes that have already existed for a while. So, to put it simply, it's the idea that leveraging AI will allow you to cut down lead times towards governance of whatever model you're using, something like ISO or whatever. And it's like, it makes sense, that's exactly what we should be doing. But the key thing here is the idea is that AI will reduce buy-in, it won't remove it. And so a lot of the companies who are not buying into security already, I don't think will start buying into it just because of AI making it cheaper. [Marc] Okay, and break this down for me a little bit. What is buying into security, and why is this a choice? [Darren] Because the EU mandates for things like NIST 2.0 are not fully in place yet. They've actually been written, I think, into local laws now, and then they'll be put in place over the next year. But security has always been a choice. Just look at the IOT, look at embedded systems, security is a choice. It's a stupid one not to say yes to, but it's always been a choice. And now we're having a choice of whether we want to shorten the amount of work, the amount of man hours by including AI in the mix. And that's going to be a good thing for some people, but it's probably not going to tip the scales for people running minimum viable products. [Marc] Is this one of those where it's like, if you think security is too expensive, wait till you see how expensive the lack of security is? [Darren] Yeah, if you can't afford security, you can't afford a breach. It's exactly that. But it was an interesting panel, and I hope we see more on that in the future. [Marc] So, one of the things that I have been using LLMs and AI for the most is working with the information I already have. Whether we're talking about a RAG system or we're talking about your tools like Copilot or GitLab Duo, it's about working with information that I have oftentimes. So, is there something about the DevSecOps governance that actually is about feeding in our internal processes and compliance documents and things like that and then helping to speed the governance along or increase the level of security buy-in? Yeah, basically. [Darren] So it's likely going to involve using some kind of retrieval augmented system, the RAG system that we're talking about, to gather up security documentation and apply the methods in those pieces of documentation to pipelines. It's kind of like, it's going to be a double-edged sword of how it's actually implemented because then it just, it kind of offloads the responsibility to the AI if you're not careful about it. If you're just assuming the expertise on the AI side, it's like I think we've talked about AI before as either an augmentation tool or a replacement tool. And I think the focus very much needs to be on the first. So, it's not completely replacing anything. People still have the knowledge, people still have things like secure development life cycles in place and understand them and not just be like, okay, the AI is going to catch it because I think that way it's going to end problematically, but it's promising. And if it gets more people on the side of security, I'll be pleased by it. [Marc] All right. Another one is going multiplayer mode in your internal developer platform. And I went through this a couple of times and I think I'm starting to get a grasp of what we're after here. But what is multiplayer versus single-player mode in your IDP? [Darren] Yeah, this was an article on devops.com that I was kind of entertained by. Not just because it's kind of referencing the team topologies by Matthew Skelton and Manuel Pais, was it? But so we have this idea that no single individual should handle everything. And again, it feels like a topic that perhaps shouldn't be surprising in 2024. I think the thing that surprises me most is how often we run into that, how often we might run into a developer who's also maintaining infrastructure or an infrastructure person who's also coding. And then this article just goes into extreme detail, building on the work of Matthew Skelton and Team Topologies to basically create these silos. I don't know when these podcasts will be released, but we have the episode with Dan and Emma. [Marc] Dan and Emma on platform engineering. [Darren] That like covers a lot of these topics and this was just an interesting article to come across after that podcast. They're basically saying many of the same things. [Marc] I mean, this sounds like four eyes to me, that in order to make any significant change you need two humans, the person doing the thing and the person who is kind of oversight of the thing. Is that what we're talking about? [Darren] In a way, but also not quite like obviously oversight is needed, but also people should be focusing on what they want to do, on what they are capable of doing, and not having coders running infrastructure, for example, not having this overlap of lacking skill sets. So you have this, it's kind of like a collaboration model and it kind of encompasses the idea of platform engineering because platform engineering is like this platform as a service pushed in the direction of internal developers. And this multiplayer mode idea comes from that where it's the same thing, platform as a service, where this is your product, you are the platform team, and you're pushing it out to your internal devs. And it's like, that's what they say by multiplayer mode, by having these not quite siloed, but not quite fully transparent teams and the abstraction around those. It's just an interesting read from my perspective as a more technical person on how these kinds of people management things tend to go. [Marc] So what I'm kind of getting here maybe now is that, so pull requests are usually asynchronous, but pair programming is synchronous. We do it together or mob programming or something like that. So what I think I'm hearing here is a bit more real-time realization of perhaps the team topologies where when we need things, we work together in order to resolve those. And if I'm more of a platform or an enabler, then maybe when I'm getting the requests, I'm trying to automate them out of existence. So the next time I don't get this type of request, and by working together, we create something that's greater than just you throw me a ticket, I build you some infrastructure, or you do it yourself, and God knows what's running there. [Darren] Yeah, exactly. Looking at one plus one, it is somehow more than two. [Marc] Yeah. For especially for very large orders of one, large values of one. Hey, HashiCorp is back in the news, and we know that IBM announced back in the spring of 2024 that it would buy HashiCorp for nearly six and a half billion dollars. But then what was the event where HashiCorp has basically not... HashiConf. HashiConf. Yes. Thank you. So, at HashiConf, this didn't really come up. So, what do we know about this? [Darren] It was just kind of interesting. Maybe something of a footnote. HashiConf announced various things to move to Terraform stacks, which has been informally referred to as like Terraform 2.0. And that's going into public gateway now, which is a public beta, which is, to me, good news. It's just the idea that they'd have this big conference six months after the announcement of the purchase and very little mention of IBM. [Marc] It's kind of curious. It's a good catch. By the way, they also announced HCP vault secrets. There's this Terraform Migrate, migrating common DIY workflows into the Terraform Community Edition to HCP Terraform or Terraform Enterprise, Nomad GPU enhancements in 1.9, a bunch of other stuff. But, you know, one of the things that I kind of hope for, and I see this in some other areas where, you know, large venerable corporations buy younger companies, is they actually do try to keep them separate to some degree so that the culture and all of the pathologies of the parent don't just spill over onto the child. So it could be a good thing. It could be. [Darren] I think we've actually seen that locally, wasn't it? Nord Cloud, one of the local providers here in Finland, was purchased by IBM some time ago, but I'm pretty sure they basically tacked on an IBM company after the name, and that kind of carried them through, and they continued as expected. So maybe we'll be seeing that for HashiConf as well. [Marc] All right. Something more practical here. I read that .io domains may be retired in the next five years, reading on Reddit a couple of weeks ago. So what's going on here? [Darren] Basically the British are handing back domain, handing back territory. So there are the Chagos Islands in the Indian Ocean that have up until recently belonged to the United Kingdom. They are being returned to Mauritius who are nearby. The problem is the .io domain, which is quite frequently used by tech startups, by anyone because of the input-output connotation. We even use it at Eficode. This .io domain is actually tied to British Indian Ocean territory. So technically speaking, following the rules, the .io domain should be retired within the next five years. And if we were doing clickbait, this is exactly where we'd end the conversation and make people panic. But this has actually happened before. So the SU domain from the Soviet Union is still in use despite it being dissolved in 1991. And what happens is basically ICANN, who handle these domains, can probably make an exception for this domain given its frequent use and the amount of money that's being pumped into it. So it's probably nothing, but it's a good bit of information to have on the radar. [Marc] I think this one's just going to come back down to capitalism. [Darren] Yeah, 100%. [Marc] There's quite a lot of money in those domains. So, let's see how that goes. [Darren] Yeah. I don't think you're wrong. [Marc] All right. Are there some security advisories that you'd like to bring up? [Darren] I think the biggest one was the OpenSSH vulnerability. Yeah. Well, in short, I think, when was it? Back in July, there was a researcher who found a potential vulnerability in OpenSSH, which allowed for arbitrary code execution, which sounds extremely bad, and it is extremely bad in cases where it's applicable. This vulnerability is called regression with two S's, SSH in the middle for stylized reasons. It was kind of touted by most media outlets as a big problem, but when you dig into the actual testing, you find that it was tested against 32-bit systems and required a race condition which took like 10,000 tries to win. So it's one of those things where if this was a problem to you, your SSH was probably overexposed to begin with because you could restrict it by IP. You could have any kind of restriction software like failing to ban something block listing to remove these 10,000 attempts that would happen over the course of a few hours. And also, just the testing in general being performed on 32-bit systems while also acknowledging that earlier versions of operating systems weren't vulnerable by default. So Ubuntu 18 and 20 weren't vulnerable. The vulnerability started in 22 and 24, so it's like if you get a very new version of an operating system and run it in its 32-bit incarnation, then you have this system that could be exploited if you didn't have any kind of defense in place. And it kind of highlights this need for new metrics in cybersecurity that I've talked about before and we'll talk about again at some point, but we just don't have a good metric to show the impact of something against the internet as a whole. So, while this was touted as a high-severity vulnerability, it was very overrated. [Marc] I think we should probably have an episode specifically on this. By the way, everybody, don't you appreciate your security professionals? I think we should all go give our security professionals a big hug as soon as we see them next. Darren, frequently when we're discussing CVEs, you give a valid argument why the metrics are wrong due to potential blast radius or many, many other factors. Perhaps we should have an episode on that. Yeah, I don't disagree, but I feel like I've talked about it a lot. Let's focus on it. In other news, GitLab released their 2024 Global DevSecOps Report. Our good friends over at GitLab, there's a few things here that I like very much. We're going to be releasing our, we do a DevOps Trends document that we send to people on the FECode mailing list, send out to our customers once a year. It's usually a Christmas gift where we talk about how we see what's going on in our customer spaces in general, what's going on with technology. But GitLab has a really interesting way. They've got the top IT investment priorities in 2024. Number five, cloud computing. Number four, automation. Number three, DevSecOps platform. Number two, AI. And the number one top IT investment priority in 2024, Darren? That's security. [Darren] Absolutely. And it's actually good that it's there. I will notice that the year-on-year change for it is actually down 5%. So before it was 24%, whereas today, 19% are stating that it's the top investment priority. Whereas for example, AI, DevSecOps platform, automation, these have all increased by at least 6%. AI, perhaps unsurprisingly, increased by 8%. So I think we're going to see that rocket into the forefront shortly. [Marc] I think there's a name for these kinds of questions that you can ask somebody to understand how deeply they know their area of expertise or something. But I think that there's probably a question around how many attacks you see per day. That, I think, would be an interesting question. We talk about this with everybody from huge enterprises that talk of the sheer force and volume of attacks that they see on a daily basis, down to really small independent players that maybe wouldn't necessarily be on anybody's radar, but they're still getting multiple attacks per hour of one type or another. [Darren] Yeah. This is actually probably the bit of trouble I see at the start of projects the most, in that there's always some mentality, that we are not large enough to be considered for an attack. And we just have to sit them down and scroll through one of the malware sites or the ransomware sites and see the list of websites that have been hit. And most of them are not on anyone's radar because that's just how attack traffic works. You attack everything and see where you get in. We've left the age of subtlety and just moved to the age of throwing the kitchen sink at it. But there is something else in this DevSecOps report from GitLab that I think we should talk about. And it's their approach to discussing AI. It's the topic that everyone's talking about, including us, it seems. But they have this idea, they have the title, the time for AI is now, but the adoption should be intentional. And I think that kind of sums up how it should be and how it's currently not, with a lot of people just kind of gluing AI to everything they possibly can. And this doesn't seem like intentional pushing of AI, but trying to catch up with the hype. [Marc] There's a lot of different ways to look there. We do get customers that come in and say, we need AI. And then it's like, okay, I'll tell our guys to start coding and then let's start talking about your requirements. That's the joke. So there is this. However, the thing that I am enjoying at the most is that a lot of us are looking at how can we use AI at the moment to help access and process the information that is available to us, but difficult to find, parse, or understand. And one of my favorite use cases lately, and I think that we'll have an episode on this in the near future, is refactoring legacy codebases. So if you take your codebase and you load it into a database, and then you build a reg model around that, all of a sudden, you can start to create tests where none existed before. You can go into even things like contract testing on internal APIs, you can do all kinds of really cool stuff, validate it at the human level, and you can start to take a really proactive approach towards not only technical debt but the ability to work without regression. I think this is something that we see a lot that are really practical steps, but what do our listeners think that they need AI for? I think that that's something that we'd love to hear from you. [Darren] To me, I'm just reminded the refactoring is an extremely good point, not just for small subset parts like testing, but just being able to port your entire code easily to another language is so extremely useful. But I always come back to the announcement back in April. I think that Logitech were putting out a mouse with an AI button, and I feel like for every good use case, there's always going to be someone like Logitech trying to put an AI button on their mouse. [Marc] Oh, goodness. I don't know, when the Windows key first appeared on my keyboard, and I was a Linux user, I was kind of mad about it, but the command key on the Mac is okay for me today. [Darren] Yeah, it would be an interesting discussion, which arrived faster, the Windows key on the keyboard or the Netflix button on every single remote at the moment? I feel like the adoption of Netflix may have been quicker than the Windows key. I've waited and it was everywhere somewhere. I do think there's one final part of this report I think we should cover. It was just the quick idea that toolchains remain a barrier to Developer Experience. [Marc] Now, how can a toolchain be a barrier to Developer Experience? I mean, developers are spending, what, 40% of their time on toolchains at the moment. [Darren] Well, that seems like quite a barrier if they want to spend, let's say, more than 60% of their time developing. [Marc] I'll tell you my take first, and then I'd love to hear yours, Darren. So when I'm coding, and I'm getting into a flow state, it's kind of like whatever the problem is in front of me, I'm going to work on it. And the problem is if I start having an issue with a tool or an issue with an action or a build or something that I might go really deep down that rabbit hole and completely forget the work that I'm supposed to be doing. And that's kind of one of the things for me that if you have an all-in-one integrated tool chain, like GitLab, for example, then there's a lot less opportunity to have to go down those rabbit holes and instead just focus on the actual work that you're supposed to do to bring value to your business. [Darren] I feel like that's a common problem that people would be drawn into those rabbit holes. And I also feel like it is a problem that we may have created in a way. We're always talking about these fast feedback loops, but if we create like blocking feedback loops, which are causing failures in builds, causing failures in pipelines, then you are going to have people who are digging into these things. From my perspective, I kind of like the infrastructure, so I'm always happy when I get to do something like that. But I imagine the pure coders who are just wanting to get on with their work are going to have a better time with these GitLab styles, all included, everything doing what it's supposed to. And I guess that's the value of platform engineering and this extrinsic platform engineering team managing this tool set. [Marc] You used a couple of words there that I have to grab onto. The extrinsic platform engineering team is an interesting idea. I learned the words extrinsic and intrinsic from Management 3.0 by Jorgen Appello. And usually I'm talking about extrinsic versus intrinsic reward systems, and feedback and things like that. It kind of harkens back to the DevOps is not a team. It's not a guy on a team. One of the greatest organizations that I've had the pleasure to work with as a consultant, they called their 700 software people DevOps engineers. And when I saw that originally on their org chart, I'm like, oh my God, you have 700 DevOps engineers. How many developers do they serve? It's like, well, that's the developers. It's DevOps, right? So exactly. So the platform team extrinsic, I think one of the things that platform engineering is to us really is it's not just even taking a customer-facing approach to the internal developer platform, but it's kind of taking this pull approach like we talked about in the previous episode on platform engineering. And I think that this is a really interesting thing. So is it intrinsic or extrinsic, the platform engineering team in your organization? [Darren] Yeah, it's going to be a difficult question based on the discussion we had in the previous episode, I would define it working ideally as being extrinsic. I don't know how many people will agree or even have that setup. [Marc] All right. This is the news for October. We're in the third week of October 2024. At the time of recording, we'll get this out to you as quickly as we can. Thank you once again, Darren, for this gregarious conversation. Thanks, Marc. It's always fun. It sure is. And hey, we'll see you back in the sauna. Bye. We'll now tell you a little bit about who we are. Hi, I'm Marc Dillon. I am a Principal Consultant at Eficode, specializing in platform engineering and enterprise transformations. [Darren] Hey, I'm Darren Richardson, Security Architect at Eficode, and I work to ensure the security of our managed services offerings. [Marc]