How to use AI in software development

It's a jungle out there. So many new technologies and methodologies are appearing on the scene, and so many paths to choose from. The good news is: we at Eficode have kept a close watch on the landscape all along. And we are rooting for you, so it's time to share some important insights.

Read on and learn the AI use cases that will most likely bring significant positive results to your software development organization. High-value use cases just sitting there, waiting to be exploited.

In this guide, you will learn:

• The basic tools you need to have in your software development

• Deploying and managing GenAI

• AI assistants and agents, and more. So let's dig in!

As with anything related to creating a competitive advantage, it's good to know which "rules" or conventional wisdom you can skip. If you are too dependent on old ways of working and methods, you miss the big-picture changes that are needed in the age of AI. So what's worth forgetting, or remembering?

Rethink the V-Model

In software engineering, the V-Model has always guided two aspects:

1. What should we implement

2. How should we validate it

In the olden days, this involved equally as much work in both sides of the model—perhaps even lopsided to the left side of the V. While the V-Model is not going away, this needs transforming.

In the future, because GenAI makes the left hand side more effortless. People in software development must shift their attention from coding efficiency to ensuring the proper outcomes through rigorous, iterative validation loops.

As a developer, this means you will spend more time validating your work and less time writing code.

Drop the classic cost estimations

Estimating the cost of software changes is difficult—especially for new work. Generative AI changes how we think about cost and effort. You are no longer limited by how fast you can write code. You can now much more easily reuse what you have built before—or what someone else wrote before. This means you can create new things more quickly and affordably.

IDEs are more than just code: They are context hubs

Your integrated developer environment (IDE) is becoming more than a place to write code. It's a central hub for all your work. It connects your code to user stories, business goals and customer feedback. This helps you see the impact of your work. The future of platform engineering is the whole value chain.

You need a strong safety net

In modern software development, because GenAI allows anyone to be incredibly destructive in a fraction of time, you need a strong DevOps safety net. To ensure that the final product is still what it should be, you need:

• Agile feedback loops so you can make changes quickly

• Traceability from CI/CD pipelines so you can track your work

• Automated security checks to make sure your code is secure

• Comprehensive test automation to make sure your code works

• Platform-centric development so your code can handle lots of users

• AI-driven insights so you can improve your code over time

• This safety net helps you work quickly and avoid mistakes. It also reduces risks for your team.

We are truly seeing the dawn of a new era of software development. And in this guide, I will walk you through the AI use cases every software development organization should implement to adapt and thrive.

Each use case gives you actionable insights and practices to help your organization enter this new era of software development.

GenAI tools are quickly becoming the standard for software development. Teams that use these tools will have a significant advantage. Those that don't risk falling behind. In this chapter we will show you the most essential GenAI-based solution that can boost your team's productivity and the quality of your software.

2.1 Code completion and coding assistants

TL;DR: GenAI tools like GitHub Copilot and GitLab Duo help you write code faster and with fewer errors. They also make it easier to improve the quality and consistency of your code.

These assistants are the very entry-level of GenAI for software development. They help you write code. They are available now, and you should start using them right away. They offer several benefits.

They can boost your productivity and code quality. They use vast code repositories to give you relevant suggestions. You can avoid errors, write cleaner code and reduce technical debt. It also makes it easier for your team to work together and onboard new members.

GitHub Copilot and GitLab Duo are two popular examples of tools that help you deliver software faster:

• GitHub Copilot suggests code, creates documentation and even writes tests.

• GitLab Duo integrates with your CI/CD pipeline. It gives you feedback to improve your code in real time.

A quick word about privacy

When using these tools, be mindful of your company's code. You don't want your proprietary code to be used for training public models. Check the terms of service and privacy policies to understand how your data is handled.

Both GitHub and GitLab offer ways to protect your code. You can turn off telemetry and use self-managed environments to keep your data private.

In short: Why these tools are helpful

• Code completion tools help you write code faster and with fewer errors.

• They use large code repositories to suggest secure and well-tested code. This helps you avoid errors and write consistent code. It also helps you keep your code clean and easy to change in the future.

• When you use these tools in your everyday work, your team can become more comfortable with AI, which can lead to using AI in more ways in your software development.

2.2 GenAI for requirements management

TL;DR: GenAI tools can help you write, analyze and share requirements. This can save you time and reduce errors. It can also help your team agree on what they are working on.   

You can use GenAI to manage your requirements—a big step toward making your software development better. GenAI helps you capture, analyze and communicate your requirements more effectively. 

Products like Atlassian Rovo and technologies like “Talk to your knowledge base”, implemented as RAG achitecture, use GenAI to automate requirements documentation. They can help with: 

• Real-time insights to help you refine your project goals

• Extracting requirements from discussions

• Keeping you aligned with organizational priorities 

• Generating user stories or acceptance criteria.  

These features save you time and reduce ambiguity in your requirements, which then helps you avoid costly mistakes later in development. 

GenAI can also analyze your requirements to find inconsistencies and overlaps, helping you improve the quality of your project deliverables. It also makes it easier for your team to collaborate and onboard new members. 

A quick word about privacy – again   

Just like with the assistants we discussed in the previous section, when you use GenAI for requirements management, you need to protect your data.

Review the platform’s security features to make sure they comply with your organization's policies. Implement access controls to protect sensitive information. Encrypt your data both in transit and at rest. Regularly audit the usage of tools like Rovo to mitigate risks. 

 In short: Why these tools are helpful 

• They automate tedious tasks like documentation, saving time and reducing human error

• They keep stakeholders aligned and requirements clear, complete in line with project objectives  

• They help you successfully execute through better communication, onboarding and team alignment   

2.3 Mine test cases and specifications from your legacy code and project documentation

TL;DR: Use GenAI to turn your old documentation into a resource you can easily query. This can help you find test cases and specifications that are missing or hard to understand.

Every company has legacy code. This code often lacks proper documentation, specifications and test cases. These are important for understanding and maintaining your code. Unfortunately, many companies don't prioritize creating these assets, which can cause problems later on.

In large projects, teams often rely on extensive documentation to communicate. Since this documentation is often written in natural language, it can be ambiguous and may not reflect the current state of the software. It can be difficult to know if the documentation is up to date.

GenAI can help you make sense of this documentation—even find missing information. Tools like AnythingLLM can extract information from your documentation. You can run this tool on your own laptop. This lets you create a RAG (Retrieval-Augmented Generation) system. You can then query your documentation using a chat interface.

You can also use advanced coding assistants and knowledge graphs to analyze your code.

These tools can help you understand how users interact with your software. You can then use this information to create more comprehensive tests. For example, you can identify potential user flows and edge cases that you might have missed otherwise.

How to set up your RAG

1. Choose a suitable LLM (Large Language Model), either from a SaaS vendor or locally run.

2. Embed your data into the RAG. Make sure the model only focuses on the relevant documents.

3. Start querying your documentation. For example, you can ask it to:

• List all functional scenarios described in the document

• Format the scenarios in Gherkin syntax

• Identify non-functional requirements like performance and security

This turns your static documentation into an interactive resource. You can use it to generate test cases that you didn't have before.

Why this is helpful

• You need test cases and specifications to understand and refactor your code. GenAI can help you generate these from your existing documentation.

• GenAI can help you bridge the gap between business needs and technical specifications, making it easier to understand and maintain your code.

2.4 Produce design assets during development

TL;DR: AI tools like Stable Diffusion and Flux help bridge the gap between software development and image creation. Fine-tune them with your brand assets to quickly generate high-quality, consistent visuals, saving time and keeping development on track.

Timing misalignment between software development and the creation of design assets is a common challenge.

While high-quality, visually appealing designs are critical for a great user experience, developers want to run their software before final assets are ready. Stopping mid-development to create placeholder visuals with tools like Gimp can disrupt the workflow, taking time away from building better software.

AI image generation tools like Stable Diffusion and Flux can quickly and cost-effectively produce high-quality, brand-consistent visuals. Trained on extensive datasets, these models can generate diverse and sophisticated images tailored to you.

You can maintain visual consistency across all assets while speeding up the creative process by fine-tuning these tools to align with your brand's identity—like your unique style, color palette and design philosophy.

Once you have fine-tuned these AI models, they can generate a variety of visuals for different needs, including UI elements, marketing materials and product mockups.

How to get started

1. Set up your AI image generation environment with the right model

2. Fine-tune the model using your brand assets and associated labels

3. Generate placeholder images that serve as stand-ins for final designs

Why these tools are helpful

With AI-driven tools, you can make your development process smother, generating on-brand, high-quality images almost instantly, so that you can focus on creating outstanding software.

Now you know how to use generative AI in your software development. But how do you actually put these tools into practice? This section will show you how to deploy and manage generative AI in your organization.

You'll learn about different deployment strategies and how to choose the right one for your needs. You'll also learn how to manage your AI models and keep them running smoothly, geting the most out of your AI investments.

A quick word about LLMs

Generative AI is changing how you create software. But how you use it depends on what you want to achieve. As we have covered previously, you can use it to improve your development process. But you can also use it to add new features to your software.

When you use LLMs in your development process, they help your team work better. For example, you can use tools like GitHub Copilot or ChatGPT to generate code, fix bugs, or learn new frameworks. You can also train your own model on your codebase. This can help you automate tasks and focus on more complex problems. The LLM helps you write better code, faster. But it's not part of the final product.

You can also embed LLMs into your software. This adds new features to your product. For example, you can use an LLM to provide customer support, translate languages or personalize user experiences. The LLM becomes a core part of your product.

The main difference is in how you use the LLM. You can use it to improve your development process or to add new features to your product. Both ways show how versatile LLMs are.

3.1 Deploy LLMs strategically

TL;DR: Deploying large language models (LLMs) requires careful consideration. You need to think about where you deploy them (cloud or on-premise), how you keep your data safe, and how you comply with regulations. You also need to think about performance and scalability.

You can deploy LLMs in the cloud or on your own infrastructure. Cloud deployments are good for scalability and flexibility. On-premise deployments are good for security and low latency.

A hybrid approach might be the best solution. You can use your own infrastructure for sensitive data. You can use the cloud for other tasks. This gives you the best of both worlds.

You might want to start with local processing. This can help you get started quickly. You can then move to a hybrid approach as your needs grow.

Key things to know

• Keep your data safe by deploying LLMs on your own infrastructure

• On-premise deployments can help you meet compliance requirements

• Reduce latency and improve performance with on-premise deployments

• Deploy in the cloud if you need flexibility to scale your solution

3.2 Optimize GenAI costs with a balanced hybrid infrastructure strategy

TL;DR: You can save money on GenAI by using a mix of third-party tools and your own infrastructure. Start with third-party tools and then move to your own infrastructure when you're ready.

Using GenAI can be expensive if you're not careful. You can end up spending a lot of money on third-party services. These services often charge you for each token you use and the amount of data processed when using the models. This can limit your flexibility and what you can do with the technology.

A better approach is to use a hybrid strategy. Start with third-party platforms to get started quickly. These platforms give you the speed and flexibility you need in the early stages. Once you have validated your models, you can move them to your own infrastructure.

While this move can sound complex, it gives you more control and can save you money in the long run. You can also use your existing hardware and software.

Why this is important

• Balance short-term flexibility with long-term cost savings

• Reduce your reliance on external vendors and their pricing models

• Scale your GenAI solutions cost-effectively using your own infrastructure

• Manage your AI operations in a sustainable way

3.3 Fine-tune and manage your LLMs

TL;DR: Fine-tuning LLMs ith your own data helps them understand your business and give you better results. You also need to manage your LLMs, which includes retraining them regularly and making sure they are secure and compliant with data privacy regulations.

You can fine-tune LLMs to make them work better for your specific needs. You do this by training them with your own data. It helps them understand your business and your goals. Just be careful not to include personal information.

If you don't fine-tune your LLMs, they might not be very useful. They might not understand your business or your data, so fine-tuning helps them gives you better results.

You also need to manage your LLMs. This means keeping them up-to-date and secure. You need to retrain them regularly with new data. You also need to make sure they are not being misused.

Key things to remember

• Fine-tuning helps your LLMs understand your business and provide relevant information

• Manage your LLMs throughout their lifecycle to keep them up-to-date and secure—and get the most out of your investment

3.4 Expand GenAI capabilities to vision and multimodal applications

TL;DR: Generative AI can work with more than just text. You can use it for audio and video too. Try using large vision models (LVMs) for tasks like monitoring. To use these models in your products, you'll need to integrate them with edge cloud and use powerful GPUs.

Generative AI is not limited to text. You can use it with audio and video as well. This allows you to create new and innovative products and services.

You can start by using large vision models (LVMs). These models can analyze images and videos. You can use them for tasks like monitoring and alerting. But keep in mind that LVMs don't have a good long-term memory. So, focus on use cases that need short-term analysis.

To use LVMs in your products, you should consider integrating them with edge cloud. This allows you to process data closer to the source. You also need to invest in powerful GPUs to handle the workload. Combining edge cloud and LVMs can lead to new and exciting applications.

Why this is important

You can use generative AI for more than just text. This opens up new possibilities for your products and services as you can process and analyze richer data sources in real time.

To wrap up this section, let us have a look at a great use case where developers can incorporate visuals:

3.5 Generate a GitHub Action (YAML) from a visual illustration of a CI/CD pipeline

TL;DR: Creating GitHub Actions workflows in YAML can be tedious. Tools like Eficode's Online Pipeline Game let teams visually design pipelines, which can then be converted into YAML using AI tools like ChatGPT. Just share a screenshot of your design with a prompt, and let the AI generate ready-to-use workflows for you.

GitHub Actions uses YAML to define workflow configurations consisting of one or more automated jobs. But creating the first version of a workflow can often feel daunting.

Visual tools for designing YAML-based pipelines have become increasingly popular to make this process easier. Even though you don't have access to commercial tools or 3rd party solutions for this purpose, you can use LLMs to turn a visual expression of your pipeline into a YAML format.

Since YAML is a widely recognized markup language, a combination of large visual models and large language models can interpret and then convert visual pipeline designs into GitHub Actions workflow files.

How to do it

To create a visual design, for example using Eficode Online Pipeline Game – a collaborative platform where software development teams can design and refine optimal workflows. Then, use GenAI for YAML generation. Take a screenshot of your pipeline design and hand it to your preferred AI chatbot (e.g., OpenAI ChatGPT 4). You can use the following prompt (or tailor it to your specific needs):

In short: Why this helps

Using visual tools to collaboratively design CI/CD workflows, in line with the agreed test strategy, and complementing them with GenAI, makes designing and implementing GitHub Actions workflows a simple, efficient and collaborative process.

“You are a competent software developer and technology architect. You know extremely well Infrastructure as Code concepts and YAML. I need to create GitHub Actions workflows in YAML based on the visual description of CI/ CD pipeline. Your role is to analyze this image, and produce GitHub Actions pipeline files in YAML for me. Make sure that the output is accurate, correct and usable right away

We've already seen how GenAI can help you write code with assistant-tools like GitHub Copilot. But GenAI can also create more general assistants and agents.

They help automate tasks and provide insights. But they don't replace humans. Humans are still in control and are the underlying source of the work.

You can use GenAI assistants for many tasks. They can manage portfolios, optimize code and automate testing. They can also access data in real time and make decisions.

You can create your own assistants for specific tasks. You can also use assistants pre-built by 3rd parties or your colleagues. You need to find a balance between customization and scalability.

You can use assistants to solve specific challenges in your development process. For example, use them to manage your portfolio, refactor your code or help you write code.

To get the most out of GenAI assistants, they need to be managed properly. This includes codifying them, embedding them into your systems and managing their lifecycle.

In this section I will show you how to use GenAI assistants effectively. You'll learn how to deploy them and scale them across your organization.

AI Assistants vs. AI Agents

Understanding the difference between assistants and agents is crucial for implementing AI effectively in your organization.

4.1 Standardize and scale GenAI Assistants with templates, RBAC and marketplaces

TL;DR: Use templates and access controls to manage your GenAI assistants effectively. Create a central library of assistants that everyone in your organization can use. It can help you avoid redundant work and ensure everyone is using the same standards.

Using GenAI assistants can be easy for individual tasks. But it can be harder to use them across your whole organization. You need a structured approach to manage them.

You should standardize your assistants. You can use templates to do this. It makes it easier to deploy and customize them. You should also use role-based access control (RBAC) so you can control who can create and modify assistants.

You can also create a central library of assistants so your team can share and reuse assistants. It also helps you avoid duplication and improve efficiency.

There are types of assistants:

1. Local assistants, for specific tasks

2. Common assistants, for general use across your organization

Use a centralized system to manage your assistants. It helps you track their lifecycle and keep them up to date.

Why all this is important

• Reusing and improving your assistants saves you time and effort

• When you can deploy your assistants across your organization you will use GenAI more effectively

• You can customize your assistants to meet your specific needs

• Your team can work together to build and improve assistants

4.2 Simplify portfolio and requirements management

TL;DR: AI assistants can help you manage your portfolio and requirements. They can automate tasks, find problems and help your team agree on goals. This saves time and improves collaboration.

AI assistants can change how you manage your portfolio and requirements. They can automate documentation, analyze requirements and improve communication. They can also create user stories, acceptance criteria and status reports.

They help you see what's happening across your projects, and also help you make better decisions and avoid errors.

Why these are important

• You improve collaboration and transparency within your team

• There will be fewer errors in your documentation

• Everyone agrees on the goals and how to achieve them

4.3 Modernize existing software and codebases with refactoring assistants

TL;DR: GenAI tools can help you modernize your old systems. It can automate tasks like creating documentation, updating databases and refactoring code. This can save you time and money while making your systems more reliable.

You need to keep your IT systems up to date. But this can be difficult if you have old software and poor documentation. GenAI can help you with this. It can automate tasks like creating documentation and analyzing your systems, saving you time and money.

GenAI can also help you improve your code. It can find problems and suggest improvements. It can even translate your old code into newer languages.

This helps you keep your code up-to-date and secure. For example, you can use GenAI to update old parts of your code with more modern and efficient approaches. Your code becomes easier to work with and less likely to have problems.

GenAI can help you overcome the challenges of modernizing your IT systems. It can help you create better documentation, improve your code and make your systems more reliable.

Why this is important 

• Reuse and improve your assistants to save time and effort, just like reusing code modules

• Deploy assistants across your organization to use GenAI effectively, similar to how you deploy software updates

• Customize assistants to meet your needs, like configuring your development environment

• Collaborate as a team to build better assistants, similar to how you work together on code

4.4 Make testing more efficient with QA & test-case GenAI assistant

TL;DR: Use GenAI to help you test your software. AI tools can create test cases, automate testing, and find bugs early. They improve the quality of your software and help you release it faster.

You can use AI to improve your software testing. AI-powered tools can analyze your code and documentation to generate test cases and automate your testing process, removing lots of manual work.

These tools can also find potential problems early on, which helps you improve the quality of your software before you release it. You can integrate them with your CI/CD pipeline to test your code continuously.

Why this is important 

• Improve the reliability of your software and reduce bugs

• Save time by automating test case generation

• Deploy your software faster and with more confidence

GenAI agents are like digital workers in your organization—extensions of your existing team. They can do specific tasks, therefore freeing up your human workers for more complex problems.

You need a good way to manage these agents. We can call this "AgentOps".

AgentOps helps you integrate AI agents into your development process. It also helps you automate tasks and improve your workflows, reducing time required for repetitive work. For example, you can use agents to automate testing, deployment and monitoring.

A good AgentOps strategy includes:

5.1 Develop evaluation framework (validators, oracles) to validate AI agents and processes

TL;DR: Validate the inputs and outputs of your AI agents, making sure these are working correctly and following the rules.

You need to make sure your AI agents are working correctly. You can do this by validating their inputs and outputs. In other words: checking the data they receive and the results they produce.

You can use frameworks like LLamaIndex and RAG to help you with this. They allow you to build the evaluation engine which is interacting with your systems to validate the responses and outputs.

Integrating validators (or oracles) into your development process can be challenging. You need to ensure your agents meet functional and non-functional requirements. Functional requirements are things like test cases. Non-functional requirements are things like performance and security.

How to get started developing validators

1. Connect your agents to your regression suite for functional specifications

2. Connect your CI/CD workflows to your product and service documentation for both non-functional and functional requirements

3. Integrate with your ITSM systems to validate user-related processes

4. Incorporate your CRM systems to verify customer interactions

Validating your AI agents helps you reduce risks and ensure they are working as expected.

Why use evaluation frameworks

• Make sure your AI agents produce accurate and reliable results 

• Make it easier to follow ethical and regulatory requirements 

• Expand your AI-driven workflows without introducing errors

• Reuse and improve your assistants to save time and effort, just like reusing code modules 

• Deploy assistants across your organization to use GenAI effectively, similar to how you deploy software updates 

• Customize assistants to meet your needs, like configuring your development environment 

• Collaborate as a team to build better assistants, similar to how you work together on code 

5.2 Build and manage AI agent factories

TL;DR: You need a good way to manage your AI agents. An agent factory can help you create, deploy and maintain your agents. It also helps you keep them aligned with your business goals.

More and more companies are using AI agents. These agents can automate tasks and improve decision making. But you need a good way to manage them. Otherwise, things can get messy.

An agent factory is a system for managing your AI agents. It helps you with the entire lifecycle of your agents, from creation to retirement. It uses standardized processes and tools to design, develop, deploy, update and retire agents.

Managing these lifecycles involves a range of tasks:

• Governance—or setting rules for how the agents are used

• Monitoring their performance in relation to your goals

• Version control—tracking the changes to your agents

• Making sure the agents are compliant

• Handling incidents

You need this systematic approach to stay on top of your agents, getting the most out of them and managing the risk of using them.

In short, this is why you want to use agent factories:

• Create and deploy new agents quickly and easily

• Make sure your agents meet your business goals and ethical standards

• Reduce development time and inefficiencies

• Reuse components to save time and money

5.3 Establish secure agent runtime environments

TL;DR: You need a secure environment to run your AI agents. You also need to control who has access to them. This helps you protect your data and your systems.

You need a special environment to run your AI agents. This environment gives your agents the resources they need, like computing power, memory and access to data sources.

It also includes tools to monitor your agents and make sure they are running smoothly. These tools might include performance monitoring, logging and alerting capabilities, helping you spot any bottlenecks that come up.

You also need to control who has access to your agents. You can do this with role-based access control (RBAC). This allows you to give different people different levels of access, based on their roles and responsibilities. For example, you might give developers access to modify agents, but only allow administrators to deploy them to production.

You can also use RBAC to restrict access to sensitive data and workflows, so that only authorized personnel can interact with them.

Why runtime environments are important

• Prevent unauthorized access to your agents and protect sensitive workflows

• Define clear roles and permissions for your team to enforce accountability

• Expand your use of AI agents securely and efficiently

• Make sure your agents are running effectively in a controlled and monitored environment

Now it's time to take action. Here are some things you can do to get started.

Learn more about AI agents and assistants

These tools are becoming essential for modern software development. You need to understand how they work and how you can use them in your workflows. Make sure you understand how they interact and how you can use them to automate tasks or make better decisions.

For example, you might need to run agents in different places, like your business systems or research and development processes. But you can often keep assistants centralized on your IT hardware.

Assess your needs

Take a look at your current development process. Find areas where GenAI can help you improve. This could include automating tasks like writing documentation or generating test cases. It could also include improving code quality or managing your requirements more effectively.

Choose the right tools and platforms for you

When you select tools for your team, you should consider a few things:

Make sure the tools work with your existing systems. Many AI solutions offer APIs and integrations to connect with different tools and platforms. You can connect tools like GitHub Copilot to different code repositories, or you can use GitLab Duo in multi-cloud environments.

Stay flexible as AI tools evolve. The best tools today might not be the best tomorrow, so make sure you can adopt new technologies without problems.

Prioritize tools that follow industry standards and support open-source frameworks. These tools often have better community support and work in different environments.

Help your developers be more efficient and innovative. Let them choose the tools they like to use. This helps them do their best work.

By applying these principles, your organization can take charge of the future of software engineering—equipped with the tools, methodologies and mindset to succeed in a rapidly transforming landscape.

Let this guide serve as your starting point for what's next.