Skip to main content Search

White paper

Agile for the Age of AI

An autopsy and reincarnation

Security and Compliance illustration

Agile has been declared dead many times. 

Yet the underlying ideas that made agile powerful have not disappeared — they have simply been misunderstood, misapplied, or left unscrutinized in a dramatically changed context.

This whitepaper revisits the foundations of agility, maps values to principles and practices, examines common reasons why agile transformations fail, and critically reassesses the original Agile Manifesto principles in light of modern product organizations.

Based on this analysis, we propose revised Agile principles that better reflect today’s complex product environments, and we examine how the rise of Artificial Intelligence further reshapes how these principles should be interpreted and applied.

The purpose of this work is not to discard agile, but to reinterpret it for the environment in which it is now used.

 

This whitepaper is for:

  • Leaders responsible for product and organizational transformation 

  • Agile coaches and transformation practitioners

  • Product managers and architects operating in complex environments

  • Organizations exploring how AI influences modern ways of working

Mikko Korkala

Mikko Korkala

Senior Advisor & Organization Coach, PhD. Advisory & Coaching.

With a career spanning several roles—a software engineer, consultant, and change maker, and a researcher—Mikko Korkala has established himself as a respected figure in product development. A pioneer in Finland's adoption of agile methodologies, he has spent over two decades implementing Agile change with a diverse array of clients, bringing his unique expertise to both development teams and senior management.

Henri Hämäläinen

Henri Hämäläinen

Co-CEO & Chief Product and Partner Office

A seasoned consultant, agility coach, and leader, has dedicated his career to enhancing organizations' success and efficiency. With roots in software development, he transitioned to consulting, leveraging his diverse experience to drive structural, procedural, and cultural transformations in many product organizations. For over a decade, Henri has worked with over 100 clients, serving as a trusted advisor on numerous transformation programs.

Why Agile is declared dead — and why it isn’t

Like many works, this also began with a thought: “Is agile really a failure, or is it actually dead?” These are questions worth exploring, since there has been quite a stir in the product development community for some time over these viewpoints. We decided to search for an answer to this and find out if this is really the case. In the process, we noticed that instead of revolving around a single question, we embarked on a journey that led us to places we did not expect us to go. It was quite an adventure, and we would like to invite you to hear about it!

This work reflects the storyline of how this effort proceeded. First, we decided to start by exploring what agile is about and concluded that it is as much about practical techniques and frameworks as it is about the underlying philosophy. This led us to exploring the philosophical underpinnings of agility, and we noticed that they have never (to our knowledge) been mapped to existing agile frameworks and practices! This was something we were compelled to demonstrate ourselves, and we present how principles are connected to agile values and to agile frameworks, using Scrum as an example.

Our conclusion is that one reason agile efforts fail is that the underlying principles are poorly understood or poorly defined within organizations. Change is seen more through the lenses of ceremonies and roles instead of their intended purpose - why are we doing this, and what drives this behavior we would like to see in the organization. In addition, we summarized key insights from our collective experience on why agile is seen as a failure in organizations. From our discussion, the following conclusion can be drawn: Agile is not a failure and is definitely not dead. It is nothing more than a set of frameworks and practices, driven by principles, that aim to instill behaviors that make organizations agile. We agree that it is very easy and tempting to pronounce agile dead, and this conclusion is a result of not properly paying attention to the forces that drive it.

This, finally, led to the biggest question. If the agile principles are the key forces driving agile behavior, are they, in fact, valid, especially in the context of modern agile development? They emerged as an experimental approach to product development in the context of small efforts over 20 years ago and have stood almost sacrosanct ever since. To our understanding, they have never been scrutinized before. In this section of this whitepaper, we examine them in the modern context and conclude that some of the principles still hold, while others are obsolete and require revisiting. For the next step in agile evolution, we propose our Modern Agile Principles. Ultimately, it was clear to us that we cannot ignore the impacts of Artificial Intelligence on our revised principles, and we discuss these in the final section of this work.

We hope that you enjoy reading this work as much as we were delighted to approach it! If you see something that raises a question, is outright wrong, or anything else, we warmly welcome you to reach out to us! This is our interpretation, and therefore, open to discussion and debate.

Contents

Part 1

What is Agile — And why is it questioned?

What Is Agile? Definitions, perspectives, and philosophical foundations

It is worthwhile to begin this discussion with the definition of agile. Given this, the first challenge agile implementations face is the lack of a universal definition of agile. Over the years, a plethora of definitions have emerged. Below are some of them that have surfaced over the years from different perspectives. For example, Erickson et al. (2005, p. 89) defined agility as follows:

“At its core, agility means to strip away as much of the heaviness, commonly associated with traditional software-development methods, as possible to promote quick response to changing environments, changes in user requirements, accelerated project deadlines, and the like”.

Some definitions in turn have approached agility from a more philosophical perspective. For example, Williams and Cockburn (2003) condensed agility as being “about feedback and change”, whereas Highsmith and Cockburn (2001) emphasized the “soft” aspects by stating that the central idea behind agility is recognizing people as the primary drivers of project success together with an intense focus on maneuverability and effectiveness. Other definitions emphasize that, in its essence, agility is a set of light but sufficient rules for the project, people, and communication, as Cockburn did in (2002). This very brief glance at the definitions of agile indicates what Laanti et al. (2013) state, that what is understood as “agile” has evolved over time, and that different people mean different things when discussing agile software development and agility in general. While this overview is unarguably historical, the multifaceted nature of agile and agility is still prevalent - people still see agile and agility differently from their individual positions.

In our experience, business-focused people often see agile ways of working as a means to increase productivity, improve resource utilization, and ultimately increase revenue and profits. Technical people, on their behalf, associate agile with processes, tools, techniques, and frameworks. These viewpoints are valid and, to an extent, correct despite their narrow perspectives. As we shall discuss later in this work, agile and agility are something that encompass the whole organization. This includes both the fundamental change in the ways that organizations themselves and the work done within them need to be seen. Agility naturally encompasses the effective implementation of techniques, processes, and frameworks as well. These need to be guided by the understanding of what agile and agility mean to the particular organization. This latter claim is particularly important for any organization utilizing agile and agility, and is by no means a new finding. Abrahamsson et al. (2009) concluded that each organization using agile methods needs to define what agility means to them in their particular context. This definition needs to start by understanding and defining agile in terms of its philosophical underpinnings.

 

Agile values and principles

At the core of agility lies a set of values and principles that serve, or should serve, as the foundational guidelines for implementing agility in the organization. The Agile Manifesto emerged during February 11-13, 2001, at the Snowbird ski resort in Utah, United States, by 17 prominent agile practitioners. Their collective experiences were distilled in the Agile Manifesto, which has remained the key underpinning of the Agile movement ever since. In the spirit of the manifesto, agility is distilled into the values and principles discussed below.

“We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

  • Individuals and interactions over processes and tools

  • Working software over comprehensive documentation

  • Customer collaboration over contract negotiation

  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

- Agile Manifesto (2001)

What is paramount to understand regarding these values is that they are about finding an appropriate balance between each item, since both of them are essential. As an example, devising a plan is essential. However, it is not reasonable to execute the plan as defined if circumstances suggest it needs to be revised. What must be stated about the agile values is that they are open to interpretation due to their very high level of abstraction. Furthermore, there is no single definition describing what these values mean in practice. Instead, they implicitly account for each organization's unique context.

It is absolutely necessary for the organization to define how these values are understood and how they are realized in action. Unfortunately, according to our experience, this is something the absolute majority of organizations neglect altogether. This importance is highlighted by Rigby, Elk, and Berez (2020), who also provide an approach and guidelines for creating an “Agile Manifesto for Leaders”. This approach is aimed at articulating the organization's values and accompanying agile principles, encouraging the implementation of those behaviors. Luckily, this is no higher science. Human beings are very good at finding meaning in words. The key point is to understand what each agile value and principle means for your organization and how they are being realized within it.

The 12 principles of Agile development are more concrete guidelines for incorporating agile ways of working within the organization and, as mentioned, need to be elaborated in the specific context of the organization. These principles are as follows:

  1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
  3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  4. Business people and developers must work together daily throughout the project.
  5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  7. Working software is the primary measure of progress.
  8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  9. Continuous attention to technical excellence and good design enhances agility.
  10. Simplicity — the art of maximizing the amount of work not done — is essential.
  11. The best architectures, requirements, and designs emerge from self-organizing teams.
  12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

 

These principles and values are important since they provide the foundations and rationale for agile processes. These principles are derived from Agile values, each promoting a different important aspect of an agile organization. The following table lists agile principles mapped to Individuals and Interactions over Processes and Tools as well as what these principles emphasize in this context.

Table 1: Individuals and interactions over processes and tools with corresponding agile principles, and what they emphasise in this context.

Value: Individuals and interactions over processes and tools

Implementing Agile principles Emphasis on
Business people and developers must work together daily throughout the project. People and their interactions
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. People and their empowerment
The most efficient and effective way to convey information within and to a development team is a face-to-face conversation. People and their interactions
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. People and their capability to manage work effectively
Simplicity — the art of maximizing the amount of work not done — is essential. People and their ability to independently decide how to implement something (empowerment)
The best architectures, requirements, and designs emerge from self-organizing teams. People and their ability to decide together what works best (empowerment)
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. People and their ability to learn what works and what does not, and act accordingly (empowerment)

As shown above, this value, in this context, is linked to people and their capabilities, as well as to the empowerment to take appropriate actions when required. This clearly puts people front and center in an agile organization and illustrates empowered decision-making, a core element of any successful agile organization.

The following table, in turn, explores agile principles mapped to Working Software versus Comprehensive Documentation, and illustrates their emphasis in this context.

Table 2: Working software over comprehensive documentation with corresponding agile principles and what they emphasise in this context.

Value: Working software over comprehensive documentation

Implementing Agile principles Emphasis on
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software Delivering real customer value in the form of usable outcomes
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale Fast feedback through fast value delivery (learning)
Working software is the primary measure of progress. Emphasis on outcomes instead of outputs, such as documents*
Continuous attention to technical excellence and good design enhances agility High quality leading to minimal rework and easy modifiability of software
Simplicity — the art of maximizing the amount of work not done — is essential. Simple solutions for easier maintenance and fast value creation, and avoidance of waste

*This does not undermine the importance of documentation, which remains essential for ensuring that outcomes are maintainable and understandable. However, progress should primarily be measured through the creation of working software, through which customer value ultimately realizes. effort should be primarily measured by the progress of creating the actual software through which the customer value realizes.


In this context, the agile principles emphasize fast value creation for the customer and shorter feedback cycles based on actual deliverables, which enable learning. In this particular case, the customer can be both external and internal. While real end users are obvious examples of external users, Product Owners and other internal stakeholders represent internal customers.

The table below shifts our focus on discussing Customer Collaboration over Contract Negotiation in a similar fashion.

Table 3: Customer collaboration over contract negotiation with corresponding agile principles and what they emphasise in this context.

Value: Customer collaboration over contract negotiation

Implementing Agile principles Emphasis on
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage Flexibility of contents based on what has been learned in order to provide the best value for the customer (learning)
Business people and developers must work together daily throughout the project Shared understanding of the state of the effort in order to create maximum transparency and best value for the customers
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation Communication that enables the fast exchange of information and the rapid resolution of matters
Simplicity–the art of maximizing the amount of work not done--is essential

Light-weight contracts enabling easy adjustment of plans if required

Here, the agile principles focus on flexibility, collaboration, and communication between both internal and external stakeholders. This assumes that changing circumstances are recognized and responded quickly to produce the best possible outcomes for customers, whether external or internal.

And finally, Responding to change over following a plan will be explored in the context of agile principles.

Table 4: Responding to change over following a plan with corresponding agile principles and what they emphasise in this context.

 

Value: Responding to change over following a plan

Implementing Agile principles Emphasis on
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage Change emerges through learning. It must be responded to appropriately.
Business people and developers must work together daily throughout the project Fast decision-making by all relevant parties involved in the effort
Simplicity--the art of maximizing the amount of work not done--is essential As light-weight, good enough plans as feasible with the premise that plans reflect only the best current understanding of the situation at any given time

These principles emphasize the importance of recognizing that any plans, no matter how well devised, are flawed and subject to change. This has long been recognized. As Prussian Field Marshal Helmuth von Moltke the Elder stated in 1871,

No plan of operations extends with any certainty beyond the first encounter with the main enemy forces.

Inevitable changes will impact any plan. 


This is due to the inherent uncertainty (lack of knowledge) caused by emerging situations. This, however, does not mean that plans and planning should be neglected in agile development. On the contrary, they are indispensable but should be seen as something that reflects only the best current understanding of the topic at any given time.

From this, it can be concluded that agile values and principles emphasize

  • People and their capabilities and empowerment

  • Fast value creation, feedback, and learning

  • Flexibility, collaboration, and communication

  • Recognizing and accepting the inherent uncertainty in the development efforts

These focal points inevitably cascade into actual methodologies. In our experience, not many organizations understand the relationship among values, principles, and methodologies, despite their importance. They are not just there to illustrate what is important in the methodology, but also to serve as the key guidelines for conducting the work throughout the organization. 


To illustrate how values and principles are realized in development methods, below is an example of the relationship between agile values, principles, and ceremonies of Scrum, as well as their intended purpose. 

A few notions concerning our approach to Scrum need to be addressed.  In official Scrum, the backlog should be kept in good order consistently throughout development. Our experience shows that having a dedicated Backlog Refinement session is extremely beneficial. It provides a dedicated, regular, and focused session to keep the backlog healthy. Therefore, we have included Backlog Refinement as a separate ceremony in our discussion.  In addition, we like to make a distinction between reviewing the output of a Sprint internally between the team and the Product Owner, and with a wider audience. We label Sprint Demos as sessions where the outcomes are presented to a wider audience of stakeholders for feedback and input. Reviews are conducted after each Sprint, and Demos when appropriate. Finally, while Sprint is not a ceremony per se, we include it here to illustrate the full Scrum approach.

Table 5: Scrum ceremonies mapped to Agile values and principles along with their focus.

Scrum ceremonies
Agile values Agile principles Emphasis on
Backlog refinement Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan 
#1, #2, #6, #10

People and their capabilities and empowerment

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Sprint planning Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan
#1, #2, #5, #6, #8, #9, #10

People and their capabilities and empowerment

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Daily scrum/ standup Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan
#2, #4, #6, #8, #10, #12

People and their capabilities and empowerment

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Sprint Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan

Working software over comprehensive documentation
#2, #3, #4, #6, #8, #9. #10

People and their capabilities and empowerment

Fast value creation, feedback, and learning

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Sprint review Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan

Working software over comprehensive documentation
#1, #2, #3, #4, #6, #7, #9 #12

People and their capabilities and empowerment

Fast value creation, feedback, and learning

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Sprint demo Individuals and interactions over processes and tools

Customer collaboration over contract negotiation

Responding to change over following a plan

Working software over comprehensive documentation
#1, #2, #3, #4, #6, #7, #9,#12

People and their capabilities and empowerment

Fast value creation, feedback, and learning

Flexibility, collaboration, and communication

Recognizing and accepting the inherent uncertainty in the development effort

Retrospective Individuals and interactions over processes and tools #12 People and their capabilities and empowerment

From these principles, principle 1 - Our highest priority is to satisfy the customer through early and continuous delivery of valuable software, can be seen as more of a general guiding principle for an agile organization, instead of something present only at the level of methodologies. The case is similar to principle 5, considering motivated and empowered people. 


As a conclusion, agility is as much about the mindset as it is about tools, processes, and frameworks. Mindset without the instruments to implement it does not lead anywhere, whereas instruments without understanding how to use them will likely lead one on an unfruitful journey. This journey very often leads the organization to declare agile dead, or at least, a failure. Now is the time to take a look at other reasons this happens.

Part 2

Diagnosing Agile: where transformations go wrong

The Autopsy: Why Agile is seen as a failure

This section describes common reasons why agile is seen as a failure. The top reason is that the expected business benefits from agile are not realized. This results from either inflated expectations or from a multitude of reasons leading to business failure. We discuss these next.

Oftentimes, organizations see agile as a competitive advantage against their competitors. While adopting agile was indeed beneficial from this perspective in the early years of agile, that benefit no longer exists. Since agile is a mature concept and, more or less, a widely adopted mainstream development approach with a significant body of knowledge, it is increasingly difficult to utilize agile as a competitive edge. This is especially true when the competition is doing agile equally well.

It should be understood that agile is a tool for operational effectiveness, not a strategy in itself. Porter (1996) also discusses the concept of the “productivity frontier,” which represents the sum of all best practices at any given time. New innovations and technologies push this frontier forward, and those who adopt them first may gain a temporary advantage over competitors. However, competitors will eventually reach the same frontier, eroding any momentary benefits gained by early adopters. When organizations do not recognize this nature of agility (or any other innovation) and the expected business benefits are not realized, agile is often pronounced a failure. 

This should not be a surprise. Leaders and their contributions to agile transformations are pivotal to success or failure. Leaders need to make their intent clear to support the effort and act accordingly. In practice, this means they set an example of acting in accordance with the new behaviors agile will introduce, provide people with adequate time and resources to adopt and adjust their new ways of working, and be very clear about the change vision that draws the desired future image of the organization. This requires that leaders understand what agile is essentially about, how it will impact the whole organization, and understand that a transformation is most often a very lengthy process requiring persistence and patience to see the desired effects manifest. In short, leadership needs to be very adamant with their intent to change the organization and support and enable it effectively. Unfortunately, we have witnessed poor understanding of and support for agile on the leadership side. It should be noted that if leadership does not put sufficient effort into the transformation, the rest of the organization can become cynical about the initiative and abandon it altogether. Yet again, agile is seen as something that does not work.

Sometimes organizations approach the required role changes (and agile in general) in name only. Agile literature proposes a flock of roles with their intended purposes,  which will often require a serious amount of work within the organization to implement them to serve the organization’s unique context in the best possible way. Unfortunately, the introduction of new agile roles may lead to people being assigned a Scrum Master, Product Owner, Epic Owner, Release Train Engineer, or any other title, without consideration of what should fall within the domain of each role. Nominations can be only nominal, whereas the ways of working do not really change along with the nominees’ responsibilities. This becomes an Agile Masquerade of sorts, eroding how people perceive agile and agility, turning them into empty words that lead to failure.

Agile implementations and transformations always require that the structures where the transformation takes place are understood and adjusted. A simple yet very common example is adopting an agile development method and applying it to a current team structure. More often than not, teams are organized based on efficiency - teams consisting of specialists in a certain field working on many different projects - instead of effectiveness. In these cases, agile methods create additional friction, tension, and frustration within teams because they fail to address the challenges they were supposed to address and ultimately deliver the desired business outcomes. By this we mean that the underlying policies, structures, roles, and accountabilities remain unchanged. In such cases, agile transformation is reduced to merely adopting a framework, without addressing the deeper organizational elements that must evolve alongside it. Instead, an organization starting a transformation journey must undertake the crucial exercise of reconstructing its people structures to achieve the most effective and frictionless value creation possible.

What needs to be understood is that it is a new way of understanding and conducting work, not something that complements the existing ways of working. It is crucial to consider, e.g., meetings, there will be overlapping meetings from the existing meeting structure, as well as from what has been introduced by agile. This is especially true for larger organizations that use scaling frameworks to manage a broader work context. Seeing agile as an additional management structure is a surprisingly common phenomenon among larger organizations, which inevitably need more control and coordination mechanisms than smaller ones. What this means is that organizations begin to see agile as a burden, with meetings and ceremonies taking an increasing amount of time away from the work they are supposed to be doing. This creates frustration towards agile and may lead to discarding it altogether.

This misconception is strongly related to the understanding of why agile and agility are being used. Senior management may see agile and agility as something aimed towards “getting as much horsepower out of the engine as possible”. Agile ways of working are seen as something that automatically increases productivity and speed just by being adopted. Efficiency is way too often seen as doing as many things as possible as fast as possible, combined with full utilization of people, and this is what agile is too often seen as the way to achieve. However, agile and agility do not miraculously add extra hours to a day. Further, this view of agile and agility as performance-improvement initiatives is stressful for people in the organization, because the intended outcomes are rarely realized. This outcome is natural since there has not been any real transformation - agile ways of working are just a new framework for doing the exact same work in exactly same manner. Further, the management will not see the supposed outcomes and consequently declare that agile does not work. What should be instead understood is that the speed and efficiency are byproducts, which come from focusing on most important initiatives and ensuring their delivery instead of trying to advance too many things at the same time.

This manifests itself in two separate forms. People implementing the selected approach easily become occupied with managing the process and its different steps and ceremonies. The focus is set on how to improve different ceremonies (such as those of Scrum or SAFe) and how to make the artifacts (such as backlog items) more refined. In a broader context, many of the organizations we have worked with focus their attention on these aspects, as well as roles, of agility. Improving roles, practices, processes, and artifacts is undoubtedly important, but it is often misleading and may place the process at the heart of agile, where it does not belong. This easily results in people beginning to serve the process, trying to conform to what is the “textbook approach”. Textbook approaches are idealizations and simplifications that effectively remove the organization's unique context from the equation. This approach also most often fails to address the organization's actual challenges. For example, continuous refinement of what should be documented for an Epic often just addresses the symptoms. The real challenge might be that the organization lacks a clear understanding or agreement of what it should be developing in the first place, and the resulting outcomes are vague as a result. If the workflow is not addressed holistically, it creates frustration and disillusionment with agile ways of working and fails to deliver the anticipated benefits. This further increases the chance of claiming agile as something that does not work.

The second aspect belonging to Process over People category is the selection of an inappropriate methodology for the given situation. For team-level processes, Scrum is appropriate for a team developing new products or services with relatively low amount of maintenance work. However, Scrum is unsuitable for teams focusing on maintenance work or work that emerges sporadically from multiple different sources. Attempting to use Scrum in such a context creates significant confusion and frustration within the team and will greatly influence team members to see agile in a very negative light. In addition, using scaling frameworks in small companies involving a small number of teams will create unnecessary complexity to development. This does not imply that organizations should avoid coordination mechanisms altogether. Smaller organizations with multiple teams still require structured planning and alignment practices. In our experience, lightweight adaptations of scaling approaches are sufficient in such contexts, while larger organizations often benefit from more comprehensive scaling framework implementations. In summary, if the transformation does not involve a broader context than development and if the selected frameworks are not fit for purpose, this will likely have a negative impact on how agile and agility are perceived in the organization, increasing the likelihood that it is seen as something that does not work.

Very often, agile is perceived as something that involves R&D only. The changes in ways of working occur only at the team level and perhaps scale up to practices for coordinating larger entities, such as multiple teams. This is a dangerous premise, since it excludes product and portfolio management, for example. Further, it dismisses the need to consider the organization as a whole, including its management and leadership practices, and strategic guidance. R&D can only produce as good outcomes as its inputs are. Full transformation covers the entire organization and involves taking significant steps to understand and reconstruct how it operates. If this is left undone, the outcomes of the transformation effort are very likely to be dissatisfying, and agile may be declared unfit and dysfunctional.

Expecting quick outcomes from an extremely complex effort, such as a transformation, is one of the cardinal misconceptions prevalent in organizations. Smaller quick wins are, of course, important in any change efforts, but anticipating a full transformation with desired business results realized within six months, or even within a year, is an illusion. Transformation initiatives are lengthy journeys undertaken in many small steps, with much learning and trial-and-error. Our experience is that agile transformations can take several years, depending on the size of an organization. Even after that, the transformation is not complete, and improvement should continue indefinitely. If this is not recognized and the initiative and its participants are not given enough time to learn and adjust, agile is, again, easily concluded as a failure.

One can say that the software development world experienced a major shock regarding planning activities when agile methods emerged over 20 years ago. Since agile emphasizes flexibility and responsiveness to change, this was interpreted that there is no need to identify requirements or discuss scope. Instead, the teams can just proceed as they see best and adjust their work as they go. This was likely a counter-reaction to planning intensive waterfall efforts where conformance to documents was seen more important than delivering the actual outcome. While this anarchistic approach may work even today for start-up’s trying to figure out it they meet customers expectations as fast as possible, it hardly works with established organizations delivering large projects involving multiple stakeholders, both external and internal, as well as several teams working together in a complex domain. Luckily, organizations have come to understand that today’s complex software projects will require appropriate requirements identification, agreement on scope and schedules, and effective planning even when agile methods are being used. Misconceptions regarding agile and planning are still prevalent, but reside on the management side.

Plans are still in many cases seen as commitments by the management. This is understandable since management and leadership is accountable for the organizations financial performance, which requires understanding when something will be provided for customers. Plans, as we have discussed, always represent the best current understanding on how something should be implemented and at what time-interval. They should not be treated as absolute commitments, and this is what organization’s management and leadership need to understand. This is important since things will inevitably change over time through learning. The same is true for the scope as well: the inherent uncertainty of product development may well cause needs to reconsider the scope, if feasible. 

What agility brings to the table here is that it enables effective, fast, and regular feedback on the project that enables learning. This high transparency provides a mechanism to make effective decisions regarding the plans, and the scope, when necessary. Instead, if organizations think agile as a silver bullet that miracuously ensures that plans hold, the effort finishes on a set date within the scope and budget, they are sure to discover the consequences and declare agile dead.

This is one of the main reasons agile and agility fail in the organization. Agile methodologies are seen as blunt instruments or an alternative way of implementing software without considering the overall change the organization requires. Transformation initiatives often begin by introducing different frameworks and corresponding roles to the organization, which is understandable. Roles, tools, and processes are the “tangible” part of agility and a very natural place to begin with. But behind every step and ceremony lies fundamental questions that need to be asked and understood. The key questions to ask are: why are we doing this? What are we trying to achieve with this? What are the principles we are following here, and why are we following them? And what is preventing us from doing this effectively? To answer these questions, it should be remembered that organization needs to define what the values and principles mean to them. Defining and following them will help organizations to get solid answers to these questions and implement sustaining agility.

Continuing from this, this would be a good time to remind ourselves once again about the origins of the Agile Manifesto. For over two decades, it has stood immutable and untouched by further scrutiny. While the values and principles as they are defined remain solid guidelines for product development, it should be remembered that, as with everything, the context in which they were constructed is a defining factor. Agile and agility emerged as a movement originating in the trenches, aiming to address common challenges of product development within a team. Since then, both the context in which agility is being implemented and the complexity of the outcomes these organizations create have changed. Product organizations can consist of thousands of R&D professionals creating complex, large-scale solutions using agile ways of working. Given the major changes in context and scope, where agile ways of working are now used, the question we ask and explore next is whether the agile principles have stood the test of time. 

Part 3

Rebuilding Agile for the age of AI

Agile manifesto dismantled

In this section, we take a critical look at agile principles.  It should be noted that there is no single “right” definition for any agile principle. We interpreted them earlier in the context of mapping them into Scrum in this chapter, and will discuss them from our perspective while exploring them as they were defined over 20 years ago, and reflect them in the current product development landscape where they are being used.

During this discussion, we will elaborate on certain principles iteratively. Therefore, their definition will change as we proceed with our discussion.

Principle #1:  Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This principle can be seen as a reminder from a world where Waterfall projects could take years to complete and only after the completion were the outcomes released to the customer. We argue that the phrase “satisfy the customer” is problematic because it leaves much to interpretation. It is not beneficial to an organization to always respond to every customer request, especially not continuously, except in cases where the effort is a one-off, tailored delivery. This is particularly true when the organization delivers products or platform services to a wide audience of customers. Responding to every need of the customer would be financially detrimental to the organization, especially if the requested outcomes would not be aligned with the strategic goals of the organization.

The other aspect that requires further consideration in modern software development is the concept of early and continuous delivery. This depends on the organization's domain and its intended customers. Some organizations are definitely able to deliver their outcomes as quickly as possible, even several times a day. For some organizations, this is not feasible. Consider a global organization delivering a complex solution involving mechanics, software, and hardware, implemented in a globally distributed fashion, with a need to comply with strict safety standards. These organizations have a different meaning for what is early and continuous due to the fact that their releasable outcomes are large and will take a lot of time to complete. Further, the product's business logic may dictate that frequent releases are not feasible. For these organizations, an annual release cycle, for example, can be the optimal release cadence. Therefore, early and continuous delivery should focus more on an appropriate, optimal delivery cadence.

The term 'customers' nowadays does not refer only to external customers but also to internal ones. In many cases, teams deliver outcomes that benefit their organization. Here again, the size of a deliverable and the value of the outcome are the factors defining what is considered early and continuous delivery.

Given this, we conclude that this principle does not apply as such in the current product development landscape. In the context of a modern software development environment, we would rephrase this principle.

Revised principle: Our highest priority is to deliver meaningful value for our external and/or internal customers at an appropriate delivery cadence to sustain our business, and grow it aligned to our strategic objectives

Principle #2: Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

This principle can be seen stemming from the rigid contractual structure and general approach of the Waterfall model. These efforts could spend considerable time identifying and elaborating on requirements, which were then frozen and contractually agreed upon. Conducting any changes in the latter phases of the effort was extremely difficult and costly. Therefore, the emphasis on embracing change, as Kent Beck put it in (2004) is understandable. The need to change is a result of learning, which is indeed the core of agile ways of working and of agility itself, but the concept of “change” needs closer examination.

The first determining factor whether a change can be made late in the development is the size and impact of the change. Small, inexpensive changes with minimal impact on the overall product can be implemented late in the project due to their low risk and low impact. Major changes with significant impacts, e.g., on schedules and fixed deadlines, should be carefully considered due to their risky and potentially very expensive nature. Therefore, whether the required change will be implemented depends on the nature of the change, the expected benefits to the customer, costs, as well as other factors, such as contractual aspects, brand, and public image-related issues, imposed on the organization.

Also, the concept “late in the development” requires scrutiny. For some organizations and application domains, even major changes are feasible very late in the effort. Consider a small start-up implementing a Minimum Viable Product for an emerging market. In this case, the organization does not necessarily have a customer base, it is not bound to deadlines, and may not have competition. This context gives it freedom and flexibility to operate as it pleases and harness it to incorporate learnings with very low risk. On the other hand, larger established organizations operating in regulated environments, facing strict deadlines and stiff competition, need to approach what “late in the development” means from a different perspective. With these organizations, changes can be costly, as mentioned above, but the time horizon for change relative to the project’s completion is considerably shorter due to the less flexible environment in which they operate. Furthermore, these companies often deliver complex solutions where decisions need to be made and locked in earlier in the project.

Therefore, it can be concluded that this principle is valid for some organizations under certain circumstances, but not all.

Lean Software Development (Poppendieck & Poppendieck, 2003) suggests the concept of Last Responsible Moment for decision-making. This concept is about setting a deadline for locking in a decision. All relevant information related to the topic is collected to reduce uncertainty as much as possible by the date of the decision at the latest. The decision is not postponed after the decided date. Larger organizations operating in this context should use this approach when dealing with change. Given our discussion on this principle and the concept of Last Responsible Moment, we revise this principle as follows:

Revised principle: We recognize that change emerges from learning, which will be beneficial for our customers and for ourselves. We welcome major changes in the effort until we agree on our Last Responsible Moment, which we communicate clearly to our stakeholders, customers, and our people.

 

Principle #3: Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

This principle is derived from the desire to deliver value to customers as quickly as possible and, therefore, enables rapid feedback and learning. As we discussed, the appropriate delivery cadence depends on the product and the industry's general expectations. From this perspective, it can be concluded that principle #1 is very closely connected with this principle. If we observe the original principle #1, focusing on early and continuous value delivery with this principle of frequent short time scale delivery, we can see that this principle, in fact, enables the first one.

It should be noted that, especially in large organizations, there are often internal customers and stakeholders who need to validate both releases and interim results. Therefore, to ensure outcomes are as intended and learning is as fast as possible, there is often both an external and an internal delivery cadence within the organization. These internal deliveries and outcomes should be validated as frequently as feasible. From this perspective, this principle is valid, but not from the viewpoint of external customers, as we discussed earlier.

Given the above-mentioned, we claim that principle #1 can, in fact, be supplemented with the key elements of principle #3. Here, we reformulate revised principle #1 with the key elements of principle #3:

Revised principle #1: “Our highest priority is to deliver meaningful value for our external and/or internal customers at an appropriate delivery cadence to enable learning, sustain our business, and grow it aligned to our strategic objectives.”

 

Principle #4: Business people and developers must work together daily throughout the project.

This principle highlights the importance of active collaboration between people involved in the development effort. This collaboration is required to facilitate prompt and appropriate responses to any needs that may arise. Further, it promotes transparency, which is essential for any effective decision-making. This principle can be seen stemming from the Waterfall project environment, where active interaction among the different parties involved was generally sporadic and most active at the beginning and the latter phases of the project. In addition, the original context of agile ways of working shines through here - a multidisciplinary team with end-to-end capabilities focusing on a single effort, and addressing uncertainty with close collaboration with people representing the business understanding of the project.

The definition of “business people” requires elaboration. In this original description, there is a clear division between the developers and everyone else. This might be sufficient with a single multidisciplinary team with full end-to-end capabilities operating in a small organization. However, in larger organizations, several teams operate in a complex context and collaborate with various stakeholders, including technical roles such as architects. In these cases, this simple division is not sufficient to identify the key people across the board that need to work actively together on a certain cadence.

Further, this principle states that there must be daily communication. In our experience, this is definitely justified in certain situations, such as in cases where solving e.g., challenges, or tackling significant uncertainties requires active collaboration. Extreme Programming (2004) went to extremes with this collaboration. One of its practices suggests an On-site Customer, which proposes that a customer, either external or internal, should be constantly available for the team by sharing the same workspace with them. While this approach is ideal rather than practical, close collaboration with the customer has its benefits. Research has shown that as communication and collaboration between the customer and the team decrease, misunderstandings about the supposed functionalities increase accordingly, resulting in more rework (Korkala & Maurer, 2006). While the study was conducted in the context of analysing collaboration between a customer and a team, the same logic between collaboration and the quality of outcomes applies to all collaborations.

While active collaboration has its benefits, it can also have detrimental effects. Consider a large effort involving several stakeholders from across the organization. There would be a need for some of them to work more actively with each other, but involving all of them on a daily basis, if the situation does not call for it, would be simply a waste. What needs to be agreed upon in this context is what constitutes a manageable and sufficient amount of communication and collaboration among the different stakeholders involved.

In addition, communication is a process that needs to be actively analysed and improved, which is perfectly aligned with the agile ethos of learning. Organisations need to focus on understanding how their communication and collaboration processes work, and how they could be improved.

Therefore, this principle, which illustrates active collaboration and communication among the involved parties and roles, is valid today but requires elaboration. Therefore, we rephrase it as:

People involved in an effort will communicate with each other in ways that enable them to work together effectively. Further, we actively analyse how we communicate and collaborate, and continuously strive to improve

Principle #5: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The small beginnings of agile show also in this principle. This principle implies that in the beginning, agile ways of working were more an experimental approach instead of a systematic approach to tackle the challenges of software development efforts, and people could be more or less hand-picked to work on such projects. Nowadays, software projects have grown immensely and require contributions from a significant number of people. Simply handpicking the most motivated people to put forth great effort can be impractical and even impossible. This principle also illustrates the empowerment of people to make the best of the project on their own by enabling them to do so. This is still essential today.

Regarding this principle, it can be said that it is still valid as such as it is today. Motivation is an important factor for success. One cannot necessarily handpick people for a project, but organizations should ensure that their employees' intrinsic motivations are met as much as possible. Some guidelines for approaching intrinsic motivation can be found, e.g., from Appelo (2016).

Principle #6: The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 

This principle illustrates the importance of fast and interactive communication between the team and the rest of the organization. The emphasis is on direct interpersonal communication rather than on sharing information through documentation, which was an established practice in Waterfall projects. Considering that agility is about timely responses to emerging situations, this principle should stand its ground. However, from a scientific perspective, this assumption does not hold. Face-to-face communication is the most effective and efficient communication medium only in certain situations. This is described through elaborating on the concepts of two prominent communication theories

Media Richness Theory (MRT) is a theory of communication suggesting that a media’s information conveyance abilities should be aligned with the needs of the task for the best possible performance. Further, this implies that certain media are better suited to conveying ambiguous or uncertain information (Daft & Lengel 1986, Daft et al. 1987). In cases where there is uncertainty (i.e. lack of information), obtaining more information about the subject is necessary, while in cases of ambiguity, there can be multiple possibly conflicting viewpoints. If information is ambiguous, the exchange of subjective views, further problem definition, and the resolution of disagreements are needed to achieve a mutual understanding of the phenomenon being communicated.

According to Media Richness Theory, ambiguous topics should be communicated through channels that promote rich communication, whereas less rich channels are more suitable for conveying standard data and well-understood messages. The richness of different communication media is characterized by four variables in MRT:

  • Feedback. Instant feedback enables rapid correction of errors and asking questions.
  • Multiple cues. Different cues, such as the tone of voice and body language, can be a part of the message.
  • Language variety. Refers to the ability to use different languages, for example, numerical data that is capable of conveying precise information and natural language that can deliver a broader set of ideas.
  • Personal focus. Personal feelings infused into the message can result in more effective communication.

Through these parameters, face-to-face communication enables instant feedback, can convey multiple cues, offers a wide range of language, and emphasizes personal focus, thereby making it an effective medium for communicating ambiguous matters.

Media Synchronicity Theory (MST) (Dennis et al. 2008) is an extension of MRT and approaches communication through two essential processes: conveyance and convergence. Conveyance is the process of transmitting sufficient new information to enable the receiver to form and revise an understanding of the content of the information. This can be very time-consuming, and in cases where it is defective, a wrong conclusion will be reached. Convergence is aimed at shared understanding and agreement on the meaning of information and typically involves addressing small amounts of information rapidly, such as agreeing on details of a particular matter. Whereas conveyance demands large amounts of individual information processing, this burden is lower with convergence.

To engage with conveyance and convergence processes, two separate processes must occur. Information transmission involves preparing information for transmission, transmitting it through a medium, and receiving it from a medium. The second process is information processing, which involves understanding the meaning of information and integrating it into a mental model. In short, information needs to be compiled and transmitted, and then interpreted and integrated into a context. Synchronicity on its behalf is defined as “the ability to support individuals working together at the same time following a shared pattern of coordinated behaviour” (Dennis et al. 2008, p.576).

According to MST, using media with lower synchronicity should improve performance in conveyance processes, whereas convergence processes benefit from media with higher synchronicity (Dennis et al. 2008). Table 6 below illustrates the capabilities of different media to support synchronicity, as described by Dennis et al. (2008).

 

Communication media Ability to support synchronicity
Face to face high
Video conference tools high
Teleconference medium
Synchronous instant messaging medium
Email and asynchronous electronic communication low
Voice mail low
Fax low
Documents low

 

According to this, we should reconsider the high emphasis agile places on face-to-face communication as the single most efficient and effective form of communication. Given the high synchronicity of face-to-face communication and its intended purpose of converging on information, this is not the case. Effective convergence requires that the discussed information be concise and well understood to reduce participants' cognitive load and, ultimately, avoid misunderstandings. Therefore, face-to-face communication is effective when people are elaborating on, e.g., details related to a requirement. In cases where the amount of information is large and may not have been well understood previously, it would be beneficial to first communicate it through less rich media with limited support for synchronicity. This would enable people to understand the information, and it would then be elaborated through richer media capable of high synchronicity. Simply put, it would be more effective to send a document first and then discuss the details to fill it in. This aligns with MST's suggestion that there is no single best medium for communication. Instead, different communication media should be used simultaneously or in succession to gain the best possible outcome. (Dennis et al. 2008, Robert and Dennis 2005)

Therefore, it can be concluded that this principle, while aiming for fast feedback and resolution, is misguided from a scientific perspective and requires reformulation.

While it is beneficial for organizations to use different communication media for different purposes, it is very challenging to create rules or guidelines that apply to all organizations. This is due to differences in product size, domain, product complexity, and the knowledge and competence levels of the people. In addition, communication needs will change over time. This can be addressed through agile philosophy itself, as we discussed in the context of the original principle: daily communication between business and development.

Constant learning and adjustment are at the core of agile organizations. Organizations learn to address specific challenges and other aspects they deem relevant, including communication. Therefore, the ways of communication within and outside the organization should also be under active scrutiny. 

This principle is essentially a key component of our new principle #3 and is already inherent in it, since the use of different communication media and the need to analyse their use are addressed, albeit not explicitly. This leads us to further revise the new principle #3 as follows:

Revised principle #3: People involved in an effort will communicate with each other in ways that enable them to work together effectively. Further, we actively analyze how we communicate, and continuously strive to improve.

In conclusion, the original principle #6 can be removed altogether.

Principle #7: Working software is the primary measure of progress. 

This principle is the antithesis of the Waterfall development projects’ mechanism for measuring progress, mainly through intermittent deliverables that were often different documents completed by the end of each phase. We interpret the concept of “working software” in the context of this original principle as a released software increment that is usable by users. Further, this principle suggests that progress is measured by customer releases that create value for customers. This principle is indeed valid in the context of small, pure software efforts that can be released piecemeal. Progress without steering the course will lead organizations astray, so we also argue that the actual underlying goal of these small releases is to learn from them and use that information to improve.

The situation is, however, different when we are talking about large solutions with interfaces to other software and/or hardware components. With products of this magnitude, the release cycle is often long, and the released content can be significantly larger than for smaller, simpler products. Quite often, incremental customer releases are not even possible. While the principle of a working product as the primary measure of progress is important in these cases as well, the underlying need for learning and validation must be met in other ways. In practice, this is done incrementally in internal fashion.

What needs to be highlighted here is that these internal learning increments are only indicative measures of the product's usefulness. The real litmus test for value is the customer release. This release starts another learning cycle, using actual data to inform adjustments to outcomes and better meet customer expectations.

Based on the above, we noticed that this original principle is both valid and impractical in the context of modern product development, depending on the product's scope and context. What, however, is common with both cases is the importance of the underlying need for learning. This led us to refine this principle to incorporate it explicitly. We recognize the still-valid notion of measuring progress through actual deliveries, but wish to emphasize that delivering on it may take a long time, depending on the context. In these cases, the partial incremental, internally validated implementations of the product play a key role in determining the validity of these outcomes - this is the all-important learning aspect of the work.

Hence, we propose our revised principle recognizing both of these aspects - progress and learning.

Revised principle #5: Working product is the primary measure of progress; implemented increment is the primary measure of learning.

Principle #8: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 

This principle puts people and their well-being at the forefront. This principle was made more concrete by Kent Beck (2004), who defined “40-hour week” as one of the key practices of Extreme Programming. It is well recognized that constant overtime work is detrimental to people’s health and well-being and ultimately productivity. Therefore, this principle is a prime example of agile being a people-centric development and work philosophy.

Following this principle requires that the organization focus on three aspects of agile: value, limiting the amount of work in progress, and the concept of slack (or reserve time). A clear understanding of value (what needs to be implemented) will ensure that the effort is focused on achieving the essential goals. This is essentially effective prioritization. Further, this focus will, in itself, limit the amount of work in progress, since it provides clear guidance on what must be completed within a limited capacity. Limiting the number of concurrent tasks further focuses on completing tasks rather than starting them. This focus on completing what is most important will reduce the overall workload over any given period. Slack, in this case, is not synonymous with slacking; instead, it is a concept of reserving a certain portion of work time (e.g., 10-30%) for unallocated work-time. This reserve is for addressing any emerging situations that may arise without jeopardizing the anticipated outcomes.

Unfortunately, utilizing these concepts is uncommon in most of the organizations we have worked with. Organizations tend to systematically overload their development teams and plan for 100% utilization rates to ensure productivity. This is in fact counter-productive. Consider a situation where a team works at 100% allocated capacity, with no room to maneuver, on 20 features and completes none, compared to a team that follows this principle, focusing on completing effectively prioritized tasks with adequate room to respond to related aspects. The latter team completes 5 out of 10 features. While this is not likely to be a stellar performance, which team was actually more productive? The first one certainly progressed further, with a larger number of tasks, and can be seen as more productive from this perspective. However, they did not complete anything of value, since it cannot be realized until it is verified. Hence, it can also be claimed that their productivity was zero from a value perspective as well. The second team is the “winner” in this case.

Considering this principle, it is definitely as essential a cornerstone of agile as it was over two decades ago. Organizations need to start seeing their underlying structure as essential to effective, productive operations and change their practices and mindsets accordingly, despite it being a long, laborious process.

In summary, this principle is valid as it is today.

Principle #9: Continuous attention to technical excellence and good design enhances agility.

The key concept behind this principle is that technically sound, high-quality implementation and design by highly skilled experts will create a technological foundation for a system that allows easy additions and modifications. This will enable fast, high-quality implementation of emerging needs, compared to poorly designed software riddled with “spaghetti code” and quick fixes. This principle, in itself, debunks a stubborn myth prevalent for over two decades about neglecting the appropriate initial design of the system and its architecture. This thinking stems from the misconception that agile does not require appropriate planning and scope definition, driven by its misunderstood concept of flexibility that we discussed earlier.

Further, this principle emphasizes high-quality outcomes. The fewer defects there are in the code, the less there will be time and resources will be consumed by rework, and in the end, technical debt that needs to be paid back at some point. Modern software tools enable people working with the software to validate its quality in real time from multiple perspectives, and one cannot emphasize their appropriate use enough. In addition, Internal Developer Platforms (IDPs) will significantly contribute to high-quality outcomes. IDPs are collections of technologies and tools that developers can utilize effectively, as they provide self-service capabilities for tasks such as deployment, configuration, rollbacks, and provisioning. This enables developers to focus on their essential tasks instead of dealing with the complexities of obtaining these services through the IT department, for example. This improved focus on what really matters will enhance agility by enabling faster responses to emergent issues. A selection of IDPs can be found, e.g., from here. (NOTE: we DO NOT personally endorse any single platform over another. Selecting an IDP depends on the organization and its needs).

Therefore, this principle is as valid as it is today.

Principle #10: Simplicity — the art of maximizing the amount of work not done — is essential.

This principle illustrates the importance of focusing on value from both the product side and the ways the organization creates and delivers it. From the product side, a simple example of striving for simplicity is to implement only what is clearly valuable. Featuritis is a phenomenon characterized by a constant increase in features, driven by various factors. Poor sales or perceived product quality may lead to compensating for its defectiveness by adding features that are hoped to increase adoption and sales. On the other hand, Featuritis is driven by advanced users who request additional features to meet their specific needs, and these requests are often accepted, although they address only a small portion of the user base and do not necessarily yield justified revenue. No matter the reason for this feature creep, it will increase the product's complexity, making future additions and maintenance more difficult. Understanding the value of any feature and implementing only those that should be is therefore of paramount importance for an organization. It is always a good reminder to recognize that one customer is not a market.

Simplicity is paramount, also from the process side. We have already discussed how agile ways of working need to replace existing ways of working rather than complement or expand them. Since agile emphasizes people and their ability to make sound decisions, minimal control and coordination mechanisms are important for ensuring rapid response to changes and other emergent situations.

Given this, this principle is valid as it is today.

Principle #11: The best architectures, requirements, and designs emerge from self-organizing teams. 

The humble origins of agile ways of working are again illustrated by this principle. As we discussed earlier, having a hand-picked team of highly experienced professionals capable of taking full end-to-end responsibility for (relatively small) outcomes is something that modern large organizations can very rarely do. Furthermore, what self-organization in the context of agility means is that every team member should be able to work on any task the team needs to complete. This would require that team members have a profound understanding of all the fields involved in the effort. Given the nature of large modern software development efforts, this requirement would be almost impossible to meet.

To implement this principle by the book would require teams with dedicated architects, who are often responsible for managing a very large and complex collection of software that serves as the basis for all software-related work within the organization. This is especially true for established organizations that offer an extensive suite of solutions to the market. What we have witnessed in these kinds of organizations is that architects are more of a shared profession that works on both the system's existing complex architecture and support teams when required. What makes their work hard to share is the immense expertise and knowledge of the system, which is very hard to disseminate within the organization, and the sheer number of products the organization provides. Dedicating these professionals to a team (or even a few teams) would create severe problems for the organization.

Understanding requirements and defining them requires an acute understanding of the organization’s strategy, goals, business targets, customer base, and markets. Simply put, the people defining requirements must understand the business the organization is in, what it wants to achieve, and how it will achieve it. In agile literature, this responsibility falls on the Product Owner, who must also be actively involved in supporting the team in their daily work. While the person working as a Product Owner may well possess all the required competencies and capabilities for this role, the burden can quickly become too much. Therefore, organizations separate the more strategic and operative responsibilities between, e.g., Product Managers and Product Owners. In this division, the Product Owner would be responsible for more operational aspects of the work, while still working very closely with the Product Manager, who is accountable for, e.g., stakeholder management, product vision and strategy, roadmaps, and financial goals and targets. In today’s organizations, Product Managers can be accountable for a host of products, which, as in the case of architects, would make them shared professionals for a multitude of teams. Having a dedicated Product Manager per team would rarely be a feasible alternative.

Further, designing complex solutions from both technical and user experience perspectives requires very strong expertise. Modern, complex software products can include many features and several ways for users to achieve their intended goals. Designing the system that enables this effectively requires a solid understanding of both the problem the user is solving with the software and the best way for them to do so. Further, this design needs to account for the user's possible limitations and incorporate accessibility aspects into the system's design. Effective user experience design requires a deep understanding of the subject matter, which makes UX experts again a shared expertise in organizations. Having dedicated UX experts for each team would require significant investments from organizations, making this approach often unrealistic.

While self-organization, as seen from the original agile perspective, is extremely difficult to achieve, it remains an important concept for any organization to implement. We define it as follows:

The ability to decide with appropriate independence, how one is going to achieve the intended outcomes a way that is seen as most natural and best

This definition states that anyone in an organization can be a decision-maker under well-understood, agreed-upon contexts. It also emphasises the importance of collaboration between involved experts and participants to achieve outcomes in the most effective way possible, since teams often require an immense amount of expertise beyond their own to reach their goals. This definition emphasizes trusting people’s capabilities in their work and decision-making and represents a clear step away from command-and-control structures, which stifle decision-making and slow down the organization. In agile organizations, self-organization and empowerment are key ingredients in making the organization agile.

Given the above mention, principle #11 would be valid only in very marginal cases where teams have all the required expertise. Therefore, it is feasible to refine this principle to better align with the current context, where agile ways of working are used.

Revised principle: Our people are the best experts on finding the ways to achieve what they need to achieve. We empower our people with appropriate decision-making rights to enable timely responses and the resolution of emerging matters, and to accelerate development flow.

Principle #12: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. 

This principle, once again, underscores agile's small-team-centric origins by explicitly stating that active learning and improvement occur at the team level. This is problematic for organizations, as it effectively leaves out the crucial learning aspect involving the entire organization. Teams should, of course, learn and adapt, but if learning occurs only at this level, the benefits would be minuscule and have little impact on the organization's overall performance. The following image highlights both where the original Agile Manifesto stressed the importance of reflection and learning, as well as the other crucial aspects to improve in the context of product development.

 

Agility-manifesto-focuses-on-sprints

Agility manifesto focuses on sprints

Image 1: The focus of reflection and learning in the original Agile Manifesto, and other areas of product development, requires active learning and adjustment.

While the abovementioned example is hypothetical, it illustrates that organizations need to reflect and adjust their work at multiple levels, and that modern agility should explicitly address this. As in Lean, the focus of improvement is explicitly on the whole, and this must involve agile ways of working to achieve the best possible outcomes, considering both the products and the organization itself.

Thus, it can be concluded that this principle is not valid in the modern context of agile development due to its narrow focus.

Considering the narrow focus of principle #12, it needs to be expanded to involve the whole organization. Therefore, we revise this principle as follows:

Revised principle: At appropriate intervals, the organisation reflects on how it can improve strategic as well as operative aspects of its work to sustain and improve its operations and position on the market

This involves everything from company strategy to product strategy to roadmaps and backlogs to the ways of working within individual teams. This improvement aspect also includes the organization's other functions, the mechanisms it interacts with, and how these need to be improved. Further, collaboration and ways of working with external partners are regularly studied and improved.

 

Below is a summary of the revised modern agile principles mapped to what they emphasise. We have made this mapping according to the areas of emphasis we discussed earlier in this work regarding the original principles. This summary is presented in Table 7 below.

Table 7: The summary of modern agile principles and what they emphasise.

Modern Agile principle Emphasis on
Our highest priority is to deliver meaningful value for our external and/or internal customers at an appropriate delivery cadence to enable learning, sustain our business, and grow it aligned to our strategic objectives Value creation, feedback and learning
We recognise that change emerges from learning, which will benefit our customers and ourselves. We welcome major changes in the effort until what we agree to be our Last Responsible Moment, which we communicate clearly to our stakeholders, customers, and our people Feeback and learning, Recognizing and accepting the inherent uncertainty in the development efforts
People involved in an effort will communicate with each other in ways that enable them to work together effectively. Further, we actively analyse how we communicate and collaborate, and continuously strive to improve Flexibility, collaboration and communication, feedback and learning
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done  People and their capabilities and empowerment
Working product is the primary measure of progress; implemented increment is the primary measure of learning Value creation, feedback and learning
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely People and their capabilities and empowerment
Continuous attention to technical excellence and good design enhances agility People and their capabilities and empowerment
Simplicity--the art of maximising the amount of work not done--is essential Value creation
Our people are the best experts on finding the ways to achieve what they need to achieve. We empower our people with appropriate decision-making rights for timely response and resolution of emerging matters, and faster development flow. People and their capabilities and empowerment
At appropriate intervals, the organisation reflects on how it can improve strategic as well as operative aspects of its work to sustain and improve its operations and position on the market Feedback and learning

As can be seen, our modern principles map to the same areas of emphasis as did the original ones. This strongly suggests that these key fundamental focus areas of agile development themselves still hold today.

Next, we will focus our discussion on these principles in the context of AI.

Part 4

Agile’s next evolution

Modern Agile principles in the age of AI

In this section, we will discuss the potential impacts of Artificial Intelligence in the context of modern agile principles.

Principle #1:  Our highest priority is to deliver meaningful value for our external and/or internal customers at an appropriate delivery cadence to enable learning, sustain our business, and grow it aligned to our strategic objectives

AI can accelerate value creation and, therefore, be seen as an instrument for increasing efficiency by delivering more functionality in a shorter time. This reflects a broader pattern in the industry, where innovations are often perceived as silver bullets promising “deliver more, faster” outcomes. However, such thinking tends to overlook the broader systemic impacts and trade-offs that accompany any transformation. Organizations need to remind themselves of their optimal release cadence and align their development efforts with it. Otherwise, too-fast releases may have a detrimental impact on customers who do not benefit from new functionalities that emerge too quickly for them, and might even create confusion among them. Consider a complex solution that requires expert knowledge and comprehensive training to use effectively. Mastering the solution will take time and effort, and adding new functionality will create additional training needs. If the release cadence for this solution is too fast, it will severely hinder its effective use.

Another temptation, driven by the need for speed, would be to implement a “feature storage” — features that are developed in advance and released according to a future plan. This stems from the assumption that we know what our customers will want in the future. While no one can predict the future with certainty, organizations can and should anticipate it. This is precisely the role of strategy — to provide direction for platforms and products under conditions of uncertainty. Confusing anticipation with prediction, however, can easily lead to excessive waste by building functionality that ultimately proves to be unnecessary.

How organizations should utilize AI in the context of this principle is to enhance their understanding of the value they deliver to their customers. Understanding what is actually valuable is imperative for any organization, since this is a mechanism for aligning people and resources effectively for maximum benefits for the customers and the organization itself. AI can also significantly improve the outcomes of learning cycles also necessary for this alignment.

Principle #2: We recognise that change emerges from learning, which will benefit our customers and ourselves. We welcome major changes in the effort until what we agree to be our Last Responsible Moment, which we communicate clearly to our stakeholders, customers, and our people

To enable effective learning, one has to understand deeply whether the learned material is valuable. What we wish to emphasize in this context is that organizations need to fully understand the value they deliver to their customers and how potential changes may affect it. Change emerges from learning, and AI has great potential to enhance it. Utilising AI to analyse market data, product usage, and customer feedback, to name a few, will provide deep insights to support decision-making. AI can also help decision makers arrive at conclusions by providing different alternatives and scenarios, and their expected outcomes. AI is therefore not just a tool to enhance learning, but also an instrument to elaborate different alternatives organisations can take based on the information available to them.

While AI has great potential in learning, it needs to be emphasized that learnings need to be understood from the perspective of delivered value - how our learning will contribute to the value the organization delivers. If organisations do not profoundly understand this value, the impacts of actions derived from AI-enhanced learning can be even misleading. Essentially, this boils down to the quality of data that AI uses to support decision-making and provide insights. Since the outcomes are directly dependent on the available data, ensuring that this data is as rich and comprehensive as possible is essential. Otherwise, AI will provide potentially flawed outcomes.

Principle #3: People involved in an effort will communicate with each other in ways that enable them to work together effectively. Further, we actively analyse how we communicate and collaborate, and continuously strive to improve

AI can enhance people’s capabilities and enable them to tackle more difficult problems on their own. While this is, of course, positive, it can also create a host of problems by potentially distancing people from each other. The use of AI can create or deepen silos within the organisation and between people, as there is less need to collaborate and exchange ideas and expertise. Siloed information is already a challenge in many organizations. The use of AI has the potential to accentuate this issue further if collaboration and shared understanding are not actively maintained.

Principle #4: Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done

It is safe to say that motivated people create the best outcomes. What drives people are their intrinsic motivators, and when these are fulfilled, they feel they are gaining something personal from their work and moving towards their own individual goals. For example, Mastery, the ability to further one’s expertise and learn new skills, is one common intrinsic motivator. Motivation cascades to using AI as a tool to enhance how work is done. Some people are more reluctant to use AI than others. Reasons for this vary from individual attitudes towards AI tools to a lack of organizational policies for their use, concerns about data security, etc. What is clear is that motivated experts with clearly defined policies for using AI will achieve much more with much smaller numbers than in times prior to AI. Product development organizations with a large number of employees with narrow areas of focus will be outperformed by those with a smaller number of motivated, multidisciplinary experts who use AI to enhance their work and capabilities.

Principle #5: Working product is the primary measure of progress; implemented increment is the primary measure of learning

We already touched on learning when we discussed the modern agile principle #2 in that context. This specific area of learning is more focused on developing the actual product, and AI has great potential to enhance it, with similar considerations as we discussed in principle #2: organizations need to understand the value they are creating and evaluate the developed outcomes against it. The effectiveness of this internal, shorter time-span learning cycle is crucial for creating successful products, and AI has the potential to either enhance or misguide learning.

Principle #6: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely

AI can process information incredibly faster than the human brain. Using AI, for example, to support decision-making results in potentially large amounts of data that need to be understood by humans. While expectations for AI center on increased efficiency and productivity, it is important to recognize that our computing power cannot match that of AI. This, in turn, makes humans the bottleneck, and the rapid pace of AI-generated data can lead to significant cognitive overload. This “computing power mismatch” needs to be explicitly accounted for when working with AI. The key is to understand that rapid generation of large amounts of information does not necessitate fast decision-making. Instead, this information needs to be carefully analyzed and verified by humans to support the important decisions they make. After all, AI is there to enhance decision-making, not drive it.

 

Principle #7: Continuous attention to technical excellence and good design enhances agility

AI tools can significantly help with technical tasks, such as programming and design, and therefore support following this principle. However, it is important to remember that it is ultimately the responsibility of humans to verify the suggestions and outcomes of, e.g., coding agents. Despite them being very advanced, their outcomes must not be trusted outright. The obvious benefits of AI in this regard might create a temptation to follow its suggestions without much questioning in search of speed and efficiency. Organizations using AI need to remind themselves that AI can realize the expected benefits only when it is used by people who master their craft and have a profound understanding of the concepts they are elaborating with AI. Given this, the principle is paramount for organizations.

Principle #8: Simplicity--the art of maximising the amount of work not done--is essential

From the perspective of AI, there are two viewpoints: value creation and technical implementation. As we have discussed earlier, understanding what value the organization creates is an important role in this principle. If it is not understood, organizations risk investing their efforts in implementing something that is not valuable to their customers or themselves. From a technical perspective, AI has the potential to maximize everything. This is a viable risk due to the advanced capabilities of AI tools. They can easily, if not properly prompted, start to create outcomes and designs that are overtly complicated. The use of tools follows organizations’ policies, which dictate what is seen as crucial within them. We have worked with organizations that are extremely cautious about security and cyber threats, and as a result, the products they worked with included extensive security checks and test cases. As a result, their value creation flow was severely hindered. One can imagine the possibilities AI would bring to address these cases, e.g., quality issues, and, as a result, predict what would happen with the number of test cases and the ability to deliver outcomes in a timely fashion. Given this, the principle of simplicity in the given context is very important to understand by all organizations.

 

Principle #9: Our people are the best experts on finding the ways to achieve what they need to achieve. We empower our people with appropriate decision-making rights for timely response and resolution of emerging matters, and faster development flow.

AI can be a two-edged sword - it can be an effective tool in supporting decision-making if it is aligned with the organization’s strategy. If this is not the case, the use of AI in decision-making can create a false sense of confidence. The guidance it provides can be generic, failing to consider the organization's actual strategic intents. Aligning AI with the organization’s strategy is a key element of effective AI-driven decision-making, but the outputs it provides must be understood in that context.

Principle #10: At appropriate intervals, the organisation reflects on how it can improve strategic as well as operative aspects of its work to sustain and improve its operations and position on the market.

The use of AI in improvement activities is an essential aspect of using AI in the

organizations. AI can support organizations from strategic decision-making to individual retrospectives, helping them find ways to improve. Utilizing this capability effectively depends on the following. The quality of the insights created by AI depends on the quality of the data provided. The insights themselves need to be understood and anchored to organizational goals and to the understanding of the value the organization creates. As shown, these are key AI concepts we have discussed in this work, and if they are flawed, AI cannot support organizations as expected.

In conclusion, artificial intelligence (or any other technology, for that matter) is not a silver bullet that will resolve the challenges organizations face. Principles are, in their essence, guidelines that aim to instill behaviors and provide focus for organizations. Therefore, our discussion in this section reveals that modern agile principles also serve as guidelines for utilizing AI in organizations. Neglecting the principles risks inefficient and potentially detrimental implementations of AI in organizations.

6. Implications for organizations

The discussions in this article propose the following implications for organizations. First, despite the rise of AI, the modern principles of agile stand. Principles are guidelines that guide the organization’s processes and behaviors. Given the general nature of principles, we, as the authors of this work, recommend that organizations describe what they mean in practical terms, in the context of processes and intended behaviors. The importance of principles involves the use of AI as well. AI is a tool closely related to processes, and if the principles driving them are not well understood and followed, AI focuses on enhancing the ways of doing things in counterproductive ways.

7. Conclusion

In this work, we took a journey to study the concept of agility, its values and principles, and map them to the Scrum framework. This part of the work aimed to lay the groundwork for our discussion of why agile has been declared dead quite a few times already. One of our findings was that organizations do not necessarily grasp the importance of agility’s philosophical underpinnings. This, in turn, provoked us to ask whether the principles themselves have stood the test of time.

Perhaps the most important outcome of this work is that the original agile principles are not valid in the context of modern product organizations operating in large and complex settings, except for some of them. In this work, we presented our revised version of agile principles to address the environment in which agile is used today.

The original principles were an excellent first attempt to provide guidelines for agile ways of working. Like everything else, they emerged from the experiences of their creators and the contexts in which they operated. Over two decades later, the world of product development is significantly different, which requires the key ideas of agility to be revisited and revised. We did this and concluded that these principles require reconsidering. Still, we found out that the key focal areas of the original principles - People and their capabilities and empowerment, Fast value creation, feedback and learning, Flexibility, collaboration and communication, and Recognizing and accepting the inherent uncertainty in the development efforts - are essential factors in the context of Modern Agile Principles and therefore, critical for any successful product organization to understand into the foreseeable future.

Artificial Intelligence is likely the most impactful phenomenon in product development and has significant potential to transform the whole product development industry. We discussed AI in the context of modern agile principles and concluded that the principles are both guidelines for instilling behaviors and enabling focus as they are intended for effective utilization of AI.

References and further reading

Abrahamsson, P., Conboy, K., & Wang, X. (2009). Lots done, more to do: The current state of agile systems development research. European Journal of Information Systems, 18(4), 281–284.

Appelo, J. (2016). Managing for Happiness: Games, Tools, and Practices to Motivate Any Team. Wiley.

Beck, K. (2004). Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley.

Cockburn, A. (2002). Agile Software Development. Addison-Wesley.

Erickson, J., Lyytinen, K., & Siau, K. (2005). Agile modeling, agile software development, and extreme programming: The state of research. Journal of Database Management, 16(4), 88–100.

Highsmith, J., & Cockburn, A. (2001). Agile software development: The business of innovation. Computer, 34(1), 120–122.

Korkala, M., Abrahamsson, P., & Kyllönen, P. (2006). A case study on the impact of customer communication on defects in agile software development. AGILE 2006 (AGILE’06), Minneapolis, MN, USA, 11–88.

Laanti, M., Similä, J., & Abrahamsson, P. (2013). Definitions of agile software development and agility. In Proceedings of the 20th European Conference on Software Process Improvement (EuroSPI 2013) (pp. 247–258). Dundalk, Ireland.

Poppendieck, M., & Poppendieck, T. (2003). Lean Software Development: An Agile Toolkit. Addison-Wesley Professional.

Porter, M. E. (1996). What is strategy? Harvard Business Review, November–December 1996.

Rigby, D. K., Elk, S., & Berez, S. (2020). Doing Agile Right: Transformation Without Chaos. Harvard Business Review Press.

Williams, L., & Cockburn, A. (2003). Guest editors’ introduction: Agile software development: It’s about feedback and change. Computer, 36(6), 43.