Dev-environments-as-code on any infrastructure

Lian (00:05): My personal philosophy is if you can solve a problem with no code, that's better. If you can solve a problem just by design, or just with the process, I would prefer that. Marc (00:18): This season, Andy and Marc are back with a fantastic group of guests. Andy (00:22): I bet the depths remain classified, and Marc keeps his head in the clouds. With our combined experience in the industry, we can go from the bare metal to the boardroom. Enjoy your time in the DevOps Sauna. Marc (00:41): Okay, we are back in the sauna. As usual, I have my cohort here, Mr. Andy Allred. Andy (00:48): Hello-hello. Marc (00:50): Hello, Andy. And we are really excited this week to have Lian Li, a Developer Advocate at Loft Labs with us. Hello, Lian. Lian (00:58): Hey, excited to be here. Thanks for the invite. Marc (01:01): Super, super cool. We've been talking about open source quite a bit lately. And you have a project called DevPod. Would you like to tell us about it? Lian (01:09): Absolutely. So this is actually brand new. We released it about two weeks ago, I believe. And the idea here is that DevPod is supposed to help you spin up a dev environment wherever you want. So other open-source projects so far have been very focused on Kubernetes. But we've seen that sometimes, you don't want to set up an entire Kubernetes cluster on your local machine. Or maybe you don't need it. So we were thinking how can we use all the power of a container, but then make it a bit more free, I guess. So what we've done is we've looked at the market right now. And we've seen that GitHub has this thing called Code Spaces, which we think is super good. And there is this open standard, they're calling it dev container JSON, which we feel is also like a very good standard. And we think that it's here to stay. However, Code Spaces is closed and proprietary. So that was the thing that we were missing in our development flow as well. So we were thinking how about we port the standard into an open source project that can basically put your development container wherever you want to, whether it's a bare metal VM or vendor Kubernetes, whatever you want. So this is how the idea of DevPod came about. And, yeah, we can talk about it a bit more if you want to. But this is the short gist of it. Andy (02:29): Yeah, it was interesting that I've been working as a consultant and juggling between projects and customers for quite a while now. And the idea of how to get the dev environment set up everywhere you go is complex one, and I just love Code Spaces. But I can't always use GitHub, and I can't always be connected to the Internet, say, there would just be something. So then I saw the announcement of DevPod. And it's like this is awesome. This is just the thing. But then I told Marc about it. And he Googled he's like, you mean the thing out of Uber? Yeah. This is like two years old, right? No, not that one. So what's with the name? Lian (03:05): Yeah, you're asking the wrong person here. Yeah, definitely the industry as a whole, we need to get better at naming things. There's DevPod, there's DevSpace, there's GitPod, there's Code Spaces. There's DevPod by Uber. There's a bunch of stuff that sounds very similar. And I think it gets a bit confusing. Like in this industry, either we name things in a way that doesn't make any kind of sense, where you don't have any kind of relation to like, what is it that it actually does or it just reusing buzzwords we already have. I'm not the best person at naming things either. You should see my code. It's all something-something manager. Andy (03:41): But isn't that one of the hardest things in software engineering anyway, naming stuff? Lian (03:46): Yes. That and counting. Marc (03:48): Yeah, I had a colleague that would name his variables sequentially like 1, 2, 3, 4 or A1, A2 because it was so much easier for him than coming up with a new name. I like it, DevPod, it says what it is to me. If you had called a celery or carrot sticks or something, then maybe it would be easier to associate, but I think it's cool. So how did you get started? I looked a little bit at your background, Lian. How did you get started at this? I think you've got some interesting stories there. Lian (04:18): Well, I have some stories. We can argue about whether they're interesting after I told them. Actually, after I finished school, I went to law school for about three years. I was just really interested in the law and helping people. I still think that that desire to use my knowledge and my skills to help people who don't have the same access to resources that I do, I still have that. That's kind of the things that's been driving me since I was in high school. I was very politically active actually and started going to law school and really believed that I want to become a lawyer or politician and change the world. However, it turned out that that is actually, first of all, not that easy. And also studying law is really dry and kind of boring. So I decided to not do that. And I thought back, so what were things that I love to do when I was younger? And when I was about 12 years old, I started building websites with a friend from school. This was late 90s, early 2000s. So the Internet was still a bit of a Wild West a little bit. So yeah, it started a bit with HTML, a bit of JavaScript. And it was just super fun and like building things and seeing how you write something here and then see something be created over there. So I remembered that. And then yeah, just started with an apprenticeship in software development. So I'm from Germany, and Germany if you want to go into these trades, you can either go to university or go to trade school. So I went to trade school. And yeah, afterwards, I started as a web developer. So I was doing PHP, Java, and JavaScript and the front end. And yeah, I was pretty happy there as an engineer, but then after some time, I wanted to expand my horizon, I moved to DevOps a bit. And then I did a little bit of machine learning, just really exploring everything in the space. And somehow, I ended up moving to Amsterdam, and becoming a cloud native consultant. It just happened. I just got lucky, found someone on Twitter who gave me this opportunity. Yeah, I was a consultant for almost four years. And then I switched to developer advocacy because I felt like this is the next big step that makes sense with my passions and my skill set. Here we are. Marc (06:38): Excellent. And this connection with open source, did this come from your legal background, or because in the pregame, we talked a little bit and I could already see some passion for open source alternatives, and you're talking about as a difference with Code Space. How do you see the importance of open source to you? Lian (06:57): Eventually, the road would have always led me to open source because I really love being part of community. In fact, I think if it wasn't for community, I wouldn't still be in tech. But I'm just not that interested in going super deep into technology, or optimizing for like the last couple of bits here and there. But I really, really love the tech community that that was the first time I felt like I had a space in tech, once I started doing stuff for the community and have the community give back to me. When you start as an engineer, you always touch open source tools, like open source is the lifeblood of all technology, and learning about the -- those are my first years of apprenticeship, probably that I learned about there's these tools out there that are not owned by anyone, and everyone can technically contribute. And everyone's opinion matters. And everyone's perspective matters. And that I found very interesting. However, as it turns out, that is the goal. But a lot of open source projects are actually not that super welcoming, and inclusive, unfortunately. So it took me a bit to get warm with actually like contributing to open source like daring and not being afraid, and even showing maybe some weaknesses that you have, or some things that you don't know yet. But I did eventually when I was at the consultancy, and I really worked heavily with Kubernetes, and Helm, and I also got to know the people behind those projects. That was when I first felt comfortable to contribute actual pull requests, code documentation, it really took someone from the contribution or maintainer group to let me know that it's okay. And then I felt invited to really contribute. Marc (08:42): I think there's a bunch of really interesting things here. So trying to find the parts that complete you within a community and then not everyone can be as inclusive as the others. But I think that there's a journey here to finding the ones that might be the right match for you from a cultural and inclusiveness perspective. They give you the skills that you need to maybe do some of the deep-dive things on certain areas of tech and allow you to do the things that you do best. One of the things we talk about a lot in this podcast is when people join a community, all of a sudden, they realize that they have a lot more ability to make positive change in the world. Lian (09:23): Yeah, definitely this feeling of being needed and wanted, and there's something that you can contribute that is unique to you. You don't feel like, it doesn't matter whether it's needed submitting this pull request or not. But you have a very unique and specific perspective that adds value to the group. Marc (09:41): Cool. Let's go back to the tech side of things a little bit. We still have customers where it takes two weeks to get the laptop when something is getting up and running. So can you tell us a little bit about the value proposition for getting the development environment up and running faster? Lian (10:04): Of course, yeah. So there's two big advantages that I would say you get from DevPod, the one thing is that your developer environment is completely portable, it's inside a container. So all the dependencies are in a Dockerfile. And you can run it anywhere that you can run a Docker container in the exact same configuration. So I can use my development environment setup, and then you as my new teammate, I can just give you this configuration and be like, hey, here you go. Just use it. And you're done. So when I was consulting, I was working for this huge FinTech company, like 5000 people or something really, really big. And unfortunately, with a lot of these big corporations, the fact is that a lot of the engineers don't have as much agency and autonomy in which tools they want to use. And which websites they can go to or which files they can download. And it got to the point where literally, they were building in house. I think it was an eclipse extension, or it might have been IntelliJ, where if you open an issue in the ticket tracking system, then it will automatically spin up everything and open your IDE and just show you that part of the code that you had to fix. Well, obviously it didn't work out because a lot of times when you have a bug, it's not just in one place, right? It's because there's this and then there's something else. But on the plus side, it got people productive fairly quickly because they didn't have to care about, now I have to like put this over here. And I have to set up this configuration and all that. So onboarding, and getting people up and running quickly. That's a big plus. And then the other thing is that because it's portable, you could build something right now on your local environment. And you can play around with things, but now you maybe want to test integration, or maybe now you need a really beefy GPU because you're doing machine learning stuff. So now you want to run this on EKS cluster, or not even a cluster just on a virtual machine somewhere with a vendor. And then same thing applies, you can just use the Docker file and your DevPod configuration, and then just just run one command. And it's just setting up the entire thing, again, on a separate environment. So that's really the big value proposition. And then the other thing, obviously, is that it's open source. So if you need a specific setup on EKS, that the provider that we built doesn't give you, feel free to just copy paste it or create your own provider, customize it, and then also share with the community so other people can reuse it. Andy (12:34): I tried to set up something like this, where I can do most of my development inside a Docker container previously. And it's a brilliant way to get all your dependencies in there, get any special tools you need to do stuff kind of pre-built pre-baked in reproduce at all. But then one issue I always ran into is how do you do the port forwarding and actually connect to the service? So how does DevPod handle that? Lian (12:59): So DevPod, the way that I understand it is it basically it thinks of your development environment as a closed and isolated thing. So if you need a dependency, we will probably want to install that within the dev container. We have another open-source project. It's called DevSpace. And that is more on an application level where that space works exclusively for Kubernetes. And it's a similar idea, though, that you have a Yaml. And in that Yaml, you can specify that you want to do port forwarding, so you could use DevPod with DevSpace. And then you have like these two separate Yaml’s, where the DevPod Yaml says, this is my container. And in this container, I need dev space. And in the DevSpace Yamo, you can specify okay, I need port forwarding. And then you have to obviously forward that pod also on your container level. That's a bit more involved. But you can do just that DevPod itself does not care about your application dependencies. It only cares about your environment dependencies if that makes sense. Andy (13:59): Yeah. I was meaning more when I try to spin up, for example, a web server inside a DevPod. How do I connect to that web server and see it in my browser? I think that's pretty automatic. Lian (14:12): Yeah, so I there is a demo for that. I'm pretty sure where we have everything set up. So you can just look at the Yamo. And it will tell you this, but from what I understand DevPod itself doesn't start your application either. What it does, it starts your IDE. So it basically just starts your development flow without starting the actual application. But you can obviously write it there and you can like expose things, you can customize it. This pod forwarding, I think it's probably built into the providers specifically. So either it is already built into the let's say Kubernetes provider, or you can write it yourself. It's definitely possible. But as I said provider specific, maybe I should explain what providers are also so it makes sense. Andy (14:56): Please. Lian (14:58): Okay. Yeah, the idea is that you have this Yamo that defines how to build the dev container on the one hand, and on the other side, you have a provider that is specific to where you want your dev container to live. So if you want to run your dev container on your local desktop, Docker desktop, you need to the Docker provider, if you want to run it on EKS, you need an AWS EKS provider. And these providers are also open source. You can look at them, there's a bunch of them already from the community, like the day after, and going from Pulumi already built a scale provider. Thanks so much to the community. And yeah, as I said, you can do all kinds of fancy stuff with the providers like collecting your def container to other services would be part of that provider. But from what I know, we don't have that yet because it's so specific. If you want to use a specific AWS service as a Kafka, broker, whatever, then your provider needs to specifically know about that. And that's why it's more specific and less general. Andy (16:01): Okay. I was playing around with it. And I just was using the, I think the Docker provider and my localhost and SSH to another server I have I'm on. And when I said, hey, start up this Python file with a web server. As soon as I said, start service on port 9000, it said, okay? Port 9000 is now available on your local host. Yes, but it's the local host of the container. And I opened it up. It was also the local host of my laptop where I was running DevPod from, but I guess I just got lucky and the providers I had chosen when I happened to do that test. Lian (16:35): Possibly. Yeah, as I said, there's definitely a demo where we do that where basically, we open the browser and the IDE, so you can basically type stuff in the ID and also in the browser. I can look it up. I just don't have it in front of me right now. Andy (16:49): Okay, yeah, I was playing around with it. And at least for the simple use case, I was using, [inaudible] how the demo worked. I said, start the server. And then I said, hey, this is available. And I click the link and it opened my browser and localhost. And it was forwarded to the application I just started inside the container, but one of them was localhost Docker, and one of them was SSH tunnel. So maybe I just got lucky in my choice of providers. Lian (17:15): This is the thing also about building tools to make developers more productive, like DevPod is a world class ships with a CLI. But also, it has a graphical UI. And I think this is really nice for developers who don't need to understand Kubernetes. But then on the other hand, as is the case with me, for example, I'm just not super sure how exactly behind the scenes things actually work. It could theoretically lead to problems, right? So there's always the question about if you're using tools to make things easier, do you still have people on your team who understand how you did it originally before you had this tool? And would you be able to debug without the tool or really understand what is happening behind the scenes. I think that's still important. Andy (18:02): Absolutely. I like the idea of having abstractions and abstracting things away. We were doing something recently and talking about originally, I needed to set the pin, set the jumper pins on my motherboard for the CPU to work. And then that's abstracted away. And then I needed to build a kernel. And now you just get those and then need to build an OS around the kernel. And now that's for free. And kind of, we're abstracting higher and higher all the time. But as these are still kind of early, it's very important that somebody understands how the abstraction works and how to go deep. Lian (18:37): I think it's really nice to enable so many more people by abstracting these things away. But yeah, I agree, you still need someone who understands. Who can pull the plug if necessary. Marc (18:53): Hi, it's Marc again, as a Kubernetes Certified Service provider, Eficode offers Kubernetes consulting support and training for organizations embarking on their Kubernetes journey. We also constantly contribute to the open-source community. If you'd like to hear more, I'll leave you a link in the show notes. Now back to the show. Marc (19:18): There's a safety aspect here as well, where I've seen organizations that do the anti-pattern of exactly what you described, where all our developers must be experts in Kubernetes. And it's like, how many people are going to be happy with that coming down as a mandate from their management that doesn't necessarily even understand what Kubernetes is. What kind of safety aspects have you worked with or worked around or worked through that your work may help to contribute to. Lian (19:47): That's a big one. I liked how you were talking about antipatterns because I feel like DevOps itself, that is what a lot of people say like DevOps itself is an antipattern, right? We're trying to solve one problem. But now introduced like this entire industry of problems to solve, but to get to your actual problem around safety. So I'm really interested in people like that's my main thing, even in technical questions, I think, ultimately, what we're doing is we're always solving people problems, right? My personal philosophy is if you can solve a problem with no code, that's better. If you can solve a problem just by design, or just with the process, I would prefer that. But a lot of times we do need tech or code. And then the question is, what are you focusing on? If you are focusing on just using the specific technology, like a lot of companies just want to use Kubernetes, or blockchain or I don't know, just because it's hot or because they heard from somewhere that this is like how we do things now. Once again, without having people to really understand why you're doing it, how you're doing it. As a consultant, I've seen that a lot of times, right? We're doing Kubernetes now. We're doing GitOps now. But then the developers, they were never asked to be on board with this, like their opinion wasn't asked. Even their processes and how they work and how they like to work was never taken into consideration. So already, you have this huge gap between the people who make decisions and the people that actually then have to like, work with these decisions. So yeah, in my work as a consultant, I was really focused on how do you create an environment where people can speak up if they want to, can contribute to important conversations if they want to, but don't have to if they don't want to, but then you also need to understand sometimes it's important to get by in from everyone. Sometimes it's important to commit even if you disagree. There's all these different levels of consensus and agreement. And these are all things that I got from by the way at university when I was doing political work there, because in that group, and I was actually a member of the Pirate Party back then. In that group, there were a lot of people who studied social sciences, and really focused on how -- they would call it nonviolent communication, and nonviolent collaboration. So how do you communicate and collaborate without talking over each other without being mean, or insulting people, these kinds of stuff. Following that, I've been involved in a bunch of community events, and specifically also the Code of Conduct teams of these community groups. For example, there is a global one-day workshop. It's called Global Diversity CFP day. And what happens is that for one day, all around the world, experienced speakers will organize a workshop to help underrepresented people give their first presentation in tech, and I was part of the COC team there, which was so interesting because of the fact that we were targeting marginalized people specifically. We wanted to be extra careful and extra vigilant in how we interact with each other. So compared to regular events that does target everyone, we were probably on the overcautious side and thought very deeply about power dynamics, and how to act on interpersonal friction. And yeah, that was just super fascinating. And I think it's useful to everyone to really think about the fact that if you include everyone, you will always get a better result than if you didn't. Even if someone gives you their opinion, and you don't agree, that's fine, but at least you have it now, and you can make a better decision than before you had that opinion. Andy (23:23): I just love this, Lian. One of the taglines of our podcast is that everything we do with machines is to help the people or to help the humans or as I sometimes snarkily, say with Marc, everything in cyberspace is actually to help meet space. But you just take that idea to such another level. I love it. Just the idea of a conference basically done to help people give their first tech talk, who normally wouldn't feel comfortable, they get up and do it. How did I not think of that? How did nobody think of that? Well, somebody did. But that was just amazing. And this is awesome. Lian (23:59): Somebody did and it's all run by volunteers. Unfortunately, it's been on hiatus for a bit because people were busy. And also, a bit burned out, I guess. But just the fact that it was global on the same day from Asia all across to America, obviously we had a bit of time zone things. But yeah, it was just really great to see people from all over the world come together and help each other and also help each other globally because we had like shared, I think we're all on the same discord. In terms of the feeling of community in terms of belonging, that was amazing. And I just want to add that one of the one of my favorite catchphrases is that is code is debt. Every line of code you write is technical debt. So the less you write, the better. Absolutely. Marc (24:48): Why don't you repeat the name of the conference for our listeners that came in and went quickly? Lian (24:53): Yeah, it's called Global Diversity CFP day. And if you are interested in reviving that, you can also reached out to me, I still have close contact with the original organizer. I would love to get something like this back off the ground. But we will have to see, I know that things have really changed since Corona. And I think this aspect of open-source to me has become so much more important because now I work completely remotely like my team is in San Francisco, I'm an Amsterdam, in the Netherlands. And I almost work more with people not from Loft Labs than with people actually in my company, because I work in so many community things. And yeah, since then also I've noticed that being intentional in your communication when you work async is so important. I love meetings because you can just chat. But it's not good for getting work done. If you want to get work done, you have to really think about what do I have? What do I need? Really be concise and think about what all those things, and then send someone a message. And it's all about efficient communication. I think this is good, actually. I do like that aspect. Marc (26:05): I absolutely love everything to do with this conversation. And you just came back around Lian to the nonviolent communication aspect, which I just gave a speech on, which is this is what I observe. This is how it makes me feel, this is what I need to be able to continue. And like this whole thing with Western organizational culture, we're all the messenger, right? So there's lots of different ways to categorize an organization. But if we all learn nonviolent communication, and we all learn to communicate, and say what our needs are in order to be successful. And I think it helps everybody be able to be more generative and create more. Lian (26:41): 100%. There's a couple of things that I feel like we can all do better as an industry or just as human beings as a whole in our culture. One thing is to disagree. I feel like a lot of people are very afraid to disagree and to create conflict. Disagreement doesn't need to be bad. You can feel bad for a second because someone disagreed with you, but ultimately, it is a good thing. Right? Like, it gets us further once again. It's better to have a disagreement and think about that and be like, okay, I hear you, but I don't care for whatever reason is still better than never getting that feedback. I'm going to leave it at that because there's much more to this, that I don't want to open like this entire bucket, but maybe like listening to pupils also, or like being curious. I don't know if you watch TED last year. But there was one episode where he talks about the curious, not judgmental, and this is such also in tech and an open source. It's such an important thing because I've seen this email threads online, and are really, really old open-source projects. Where do you have these old school engineers who are very much of the culture of, I'm your senior. And that's why I'm right. And you're not allowed to raise this because you were just. I don't know, you're just not as famous as I am, something like that. But sometimes, again, someone has a completely different background than you, completely different perspective. And sometimes this is something that you would have never thought of. And it's really helpful to hear that out. In a lot of meetings, someone suggested something and someone else has said, no, that happens so many times. And I will always say, let's not say no, just ask, why do you think this way, right? You don't have to say, I agree with you. You don't have to, like compromise your integrity or whatever, but just give people some space to express themselves. And then you can still say, I disagree because blah, blah, but it's the understanding. Both sides know where you're coming from, and then you can move to the next step, which might be commit, even if you disagree. We don't have data. So this is my opinion, this is your opinion, fine. We still have to make a decision. All right, let's just make this decision. Maybe set up some ground rules, like okay, we're writing this as an experiment for two weeks, and then we'll circle back to it. Once you are open and hear people out, there are so many options that are not just yes or no. Andy (29:09): As a recovering grumpy old sysadmin, I can very much testify that my career took a turn for the better when I stopped saying no, that's stupid, and started saying, help me understand why you think that makes sense. And then moved on to okay, hey, help me see that. Instead of saying no because I know better. Say, help me understand your perspective changes everything. Lian (29:38): Yeah, exactly. And people are much more likely to want to work with you. Andy (29:42): That's very true as well. Marc (29:44): Recovery is a never-ending process. So one a few more things actually. But one of your titles I think it was on your Twitter profile is chief karaoke officer. What's up with that, Lian?