Efficiency vs. Security: Striking the Right Balance with AI-Generated Code | Varun Manoj

Hey, everybody. It's been a lovely conference. The session that I'm doing today is called Breaking AI. There will be a demo, I'm going to try my best - to show you how AI works in software development - and what are some of the things we should keep in mind - when using AI, essentially. But yeah, a bit about me. So, I've been at Snyk for nearly five years. I'm a solutions engineer. So, essentially, if you come to talk to Snyk, I'll be giving you a demo. And at the end of the day, if you want to try it, - then I'll help you, kind of POC Snyk. Snyk's all about DevSecOps. So, adding security into development processes. As you see there, it's in brackets. Because to be honest, DevOps done right has security as part of the process. But we've had this shift of injecting security. And now we've also got this added world of AI. This is what we're going to cover today, right? How to AI in development? What are the common use cases? And then, we're actually going to try to use AI - to build an app with Copilot today, right? And we'll see how that goes. And then, finally, learnings and takeaways. There's been a lot said about AI in this conference, right? There's a lot of people talking about AI. I won't rehash old ground, but this is quite a good graphic, - I think, just to show how popular it is and how much is being adopted. So, Google Translate took 78 months to reach 100 million users. ChatGPT took two months. It's just absolutely insane. So, it's coming whether you like it or not, AI has been used. And I think there's a reason why it's quite popular in development, AI. Because your average developer, at the end of the day, - what do they get paid for? They get paid to ship a product. That's their main KPI. So, that's what they're paid to do. That's why they come to work. That's what they do. And AI makes that easier. So, AI is adopted by developers in various ways. And these are common activities that you do when you do software development. You maybe start with comments, - you can summarise code, refactor code. And the last two, pair programming and generating code, - they say AI or Copilot is the new Stack Overflow, right? So, you can ask questions, get answers - relevant to the code base that you're working on. And that's really how it's been adopted, right? So, I think most developers these days use AI, right? And they say that they take less and less time - to finish their tasks when they use AI. But again, being from Snyk, - we also see the other side effects of this - or some of the consequences of this. Because when you're generating code, you have to look at the validation of, - like, does this introduce vulnerabilities, et cetera. So, Copilot does generate code without maybe that security lens. And it's up to the developer or to the organisation to validate - and make sure that code is actually secure, right? A little example, artificial imagination, right? Maybe you've seen this one before, I think it's a slightly old example, - but if you ask ChatGPT that question, - I'll leave it to the audience on the quick maths on that, - but like five times nine plus four times three, - does anybody know? ChatGPT says 51. And even when you press ChatGPT to say, is that right? It's pretty confident. It says, "Yeah, no, that's right. That makes sense." But then, when you actually ask it a question, direct 45 plus 12, - oh, you sure about that, ChatGPT? And then, it apologises. So, quite confident. Then, you press it, give some other evidence, and say, "Oh actually, maybe." But that's the fallacy on my part, because at the end of the day, - AI doesn't have a comprehension of mathematical concepts. An AI model is trained on a data set. And then, it's finding statistical probabilities - of what should come next from your question, right? So, there's no inherent understanding from an AI model - about the nuances of mathematical concepts like this. So, you might need to validate the output you get from an AI model. And also, maybe, you know, understanding - that you have to also maybe choose the right model for the job. So, for example, DeepMind from Google came out - with an AI model called Alpha Geometry, - which is a model specifically trained to solve geometry problems. This is maybe a more specialised tool - to solve a specific problem at the end of the day, right? So, that's an important consideration as well. What about code? So, I think the rest of this - will be actually to try and build an app today. Okay, so that's what I'm going to try and do. We'll see how this goes. [chuckles] So, a little bit of housekeeping about what we're trying to do. I'll be working from this specific repo. And it's a repo about conferences, - which I think is quite good because we're at a conference, right? This is in GitHub, and along with this, I have IntelliJ IDE. Okay, and this is a job application. And what I'm going to do is I'm going to pretend I'm a developer - working on this application. And actually, inside my IntelliJ, I've got a paid version of Copilot. Okay, so I'm going to just go over here. Let's just see straight off the bat what we're working with, right? So, if I maybe run this app as it is now, - so you see here there's some source code, - I've got some Java folder, I've got some resources, - there's like an index page, there's some fragments, right? So, now my app is ready. Because obviously Java is so very fast, very, very fast. [chuckles] So, if I look at what this looks like. Okay, so there's not much there. All right. So, let's start working on this application then, right? So, I think the first thing that we're going to try to do - is we will try to create a homepage, okay? So, with a top banner and a conference schedule table, right? So, inside this Java app, I've got an index page, - and I can see I've already got some fragments. So, I can start to use Copilot straight away, right? So, inside this container, what I'm going to do is, - I'm going to interact with Copilot with comments, - and I'm going to say something like, insert nav fragment. Let's just say that. See what happens, right? And then, Copilot will generate a suggestion, - and you can see here that it's saying global header, - so it's using my global header fragment. That looks good. And then, maybe I want to also insert this table fragment. So, if I say another comment and say, - insert event table fragment, right? Okay, great. So, I've got another fragment. However, and just notice that - I asked for event table, it gave me event table. But my actual fragment is events table, right? So, this is an important point. Copilot will give you what you asked for, right? So, instead of event table, I actually want to say events table. Small point, but it is important, right? Because now I get the right thing that I wanted, right? So, now I've got these two fragments. Let's see if that has made a little difference. Oh, actually, before I do that, I'll actually make... Yeah, perfect. So, I've got something a bit more interesting. I've got my talk. I've got a list of talks. I've got some search potential. I can go and ask for something. And I've got some speakers. And hey, look, I'm there, too. With a description and start dates and end dates for my talks. So, if I try to maybe, I don't know, - try to search for something, nothing's found. Okay, so clearly, the search needs some work, right? What I can do is, I could look for my search. Okay, so if instead of closing all this, I look at my Java project. I can see there's a bunch of different folders. And inside this repository, we can see some interesting files here. And there's also, for this app, a home controller as well. So, you can see here that there is a mapping - where when I add some text into here, - that input will get passed to a search repository function, right? So, if I look at my search repository, and notice here, by the way, - that there is an old search repository which has an implementation, right? But then, there's also a new one which maybe I'm going to be using. This is an important point because, - obviously, when you use Copilot in a company, - there's going to be legacy code that you may be working with, right? Actually, a little survey to the audience, - how many people here work at companies where - there's zero vulnerabilities existing in the code base? Could anyone raise their hands? Yeah? Everybody's got some known issues inside the code base, right? Whether you know about them or not. So, that's the problem for using something like Copilot, - which I'll show you now, where you take the search talk, - so we'll try using Copilot here, - okay, so I'm going to ask Copilot to build a search function. So, I'm going to say, lowercase the input, right? Looks good. And then, I'm going to continue to say, - create a query - to match the input with the, - I'm going to make maybe another line, - with the description, - title or speaker username, - and format the code - so that it is 80 characters per line, - just to make it a bit more readable. Okay, so what's happened here is, we've taken the input, - lower-cased it, then I ask Copilot to create a query, - which it seems to have done, right? It made a query. And this is taking that lower input, putting it into this query. And then, what I'm going to ask Copilot next to do is execute the query. Execute the query. And return the result. Yeah? Successful. Can anyone see any problems with the code that I've just written? Very fast, by the way, right? That took me very, very little time. [chuckles] Okay, okay, yeah. Okay, that's definitely a possibility. Let's actually see if this works though, right? So, what I'm going to do is, I am going to see - if my little function here does do anything, right? So, here we go. Nice. So, in a sense, you're right. But search is working. But as that gentleman said, there is potential here for a SQL injection. And there's a way we can check this as well, right? So, I work at Snyk, - and one of the things Snyk makes is an IDE tool, right? So, a plugin. And you can download this plugin, - install it, and run a scan with Snyk, - which will set off a scan for whatever is opened up inside the IDE. And Snyk's now going to go find out, - oh, is there an issue inside my IDE, right? So, I can see here there is a SQL injection, right? So, this is what Snyk is doing. We're understanding where does data input enter the application. So, it starts in that home controller function, - goes into this search talk method, - and then it's passed into my search talk method to another line, - to another line, and finally, into my SQL injection. Clearly, there is an issue with this, - and it's not because Copilot is inherently insecure. It's just that I've asked Copilot very simply to do something, - and it's generated me some code. But it's also looking at the previous context of what's already - in the code base and is propagating any existing vulnerabilities - into the implementation of what I've asked for, right? Because it's, again, statistical probability. It's trying to find what is the next thing that it should probably be, right? So, if I wanted to actually understand this from a security perspective, - what I would do is, actually, I'll start off by getting rid of this, - to start, start again. And something that I'd also call attention to is that, - when you use this plugin, you'll see little links like this - to learn about a vulnerability as well. Because at the end of the day, security issues - are more of an education problem - than, you know, as I said at the start, - most developers need to ship products to build apps, right? Anything that gets in the way of that can be seen as a blocker, - can be seen as something that we complain about. However, if it's an opportunity to learn, - that becomes a much better, I guess, pitch, right? Because this will help you become a better engineer. So, SQL injection, what is it? How does it work in action? And then, finally, what are the ways to maybe mitigate an SQL injection? So, one of the things that we suggest here is parameterise queries, right? So, maybe what I'll do is, instead of just asking what I did before, - what I'm going to do is I'm going to ask a specific thing from Copilot, right? So, I'll start again with lowercase the input. It's a good place to start. Here we go. And then, I'm going to say, - create a query with named parameters, - that matches the input with the description. Again, same thing. Or speaker username. And then, I'll ask also, - format the code so that it is 80 characters per line. Okay? There we go. And that looks a little bit better. Named parameters that matches the input. And then, what I will do then is set the input parameter. So, set the input parameter. Parameter. There we go. And then, I will return the result. Okay, so now what I've done is I've asked Copilot, - instead of just giving me a query, actually, - with my new security education, - I've asked it a specific ask of using name parameters instead, right? So, here we go. We've got something else now. In fact, I'm a bit worried that that's the same. So, what I'm going to do is I am... Well, let's just see if it works, right? We'll do it live. So, I'm going to stop this now. Let's see if my app will run again, now with my new code. And in fact, before we do anything, - why don't we even just check it to see if that has made any difference? Actually, to be honest, - maybe my demo has given me not the right thing there. Okay, we still have our SQL injection. Uh-oh! [laughs] [audience laughs] Right. I think what I'm... Create a name, - create with named parameters. Do you know what? I am going to do something to just make this a bit easier. I'm going to just copy this, and add a new thing here. Okay. Sorry. I think what happened was, I didn't add the right prompt. But you see here, the new prompt is asking for very specific details. Again, using parameters, but also, setting a positional parameter, - and then executing the query and returning the results. So, now if I take this code, and I maybe try to scan with Snyk - to see if there still is an SQL query, - you can see that that SQL injection is gone, right? So, we can test this by running the app again, - now with my new shiny search function. Here we go. So, remember, what we want is an ability to search. So, let's see if I can search. Yeah. And if maybe I add like a payload to this. So, if I see if I can maybe add a payload like this one. So, oh, yeah. If I just take this, maybe see if I can get anything from that. Okay. So, I tried to add a payload that would maybe - engage that SQL injection, and nothing's turned up. So we've essentially input validated that little search box, right? So, that was just a demonstration to show how you can use Copilot - to generate vulnerable code, - maybe how you can use different development practices - to check your code, validate your code, - and then ask the right things from Copilot, - and it will give you the right kind of feedback. So, yeah, I think I will end with the takeaways, - So, the shit, shit, learnings and takeaways. So, there's probably a couple things to do before you start using - generative AI tools inside your company or inside your organisation. The main thing, I think, is understanding that you might need - to come up with standard operating procedures for using generative AI. If you're going to use Gen AI tools, - you should know the risks involved with using these tools. For example, validation, making sure that you're not adding company data - into a model which might be used and be outputted somewhere else, right? And then, making sure that it's embedded into processes - to actually validate the output of these models. So, the common saying is trust but verify. With AI, you should definitely not trust to start with, right? Because at the end of the day, it's not, - it's going to give you exactly what you asked for - and not really looking at the wider implications of it. And then, a little plug to say, - yeah, it's a good idea to maybe start testing while you're coding as well. So, yeah, as I mentioned, I'm from Snyk, - and we do have some white papers that can help you - understand how to use AI in an organisation. And one thing I will also say is, that Snyk Learn stuff I showed you, - we actually have a set of lessons - that talk about the OWASP Top 10 when it comes to LLMs and Gen AI, - which are really good lessons. And these are free, so you can actually access this content - right after this session. And they are a very practical type of sessions - that you can send to a developer or send to people using Gen AI. So, yeah, something to look into as well. Hope that was fun. Thank you. [applause] [outro music] [music ends]