This third blog on Agile emphasizes the importance of critical thinking instead of blindly following what others say or write (me included). Every new software project has its specific context (e.g. relative importance of speed-to-market, ability of product owner to articulate the requirements). Hence, forcing every software development project to be cast in the same mold will inevitably result in a waste of scarce development resources. Furthermore, the name of the game is agility, not Agile.
From dogmatic to pragmatic
Some forget that Agile is a means to an end. It is a cog in the machine required to translate the need of the business into an effective and efficient solution. Other cogs include the choice between COTS and custom software, the decision of make or buy, and the Agile readiness of the organization. Combined with the uncertainty and complexity of the requirements, required speed-to-market and expected lifespan of the solution, these topics shape the context the solution has to operate in. A context which allows Agile to either add or destroy value.
Hence, the decision to apply Agile Scrum has to be based on a rational decision making process. Quoting from the article Post-Agile: A Design Thinking Approach to Software Development by Tom Dabson:
“Regardless of how trained and focused your people are, when the projects you work on vary in complexity and time finding a single project management solution that works for all is impossible.”
In order to balance flexibility (‘every project is unique’) and efficiency (‘every project is the same’), the business and IT should agree on a set of principles to govern, develop and support new solutions (‘comply or explain’). Using waterfall in a highly dynamic environment does not make sense, nor does Vanilla Agile for an one-on-one port of a Cobol application to C#. Like the example depicted in figure 1, these principles should enable the stakeholders to objectify decisions regarding the:
- governance model (e.g. distribution of decision-rights, trust or control-based management),
- sourcing model (e.g. ASP, SaaS, developed internally or development by offshore vendor)
- architecture and technology (e.g. PoC on PaaS platform, business critical applications in Java or PHP)
- development methodology (e.g. Agile, waterfall, S-model), and
- operations model (e.g. best-effort support or SLA-based support).
The approach depicted in figure 1 allows for more parameters than Ralph Stacey’s complexity matrix which focuses on the certainty of the technology and certainty of the scope. For native digital and hybrid business models, the context of the solution is shaped by more than these two parameters. With the digitalizing of business models, the protective layer provided by the business to shield IT from the external market is all but gone. Today, evolving customer demand, the entrance of competitors, globalization and new regulations are as relevant for IT as they are for the business.
The left side of the continuum in figure 1 captures the most risky needs and wants from the business. Think of very complex and/or highly uncertain requirements and/or technologies. Here, desk research, interviews, brainstorm sessions, clickable demos and simulations can be used to provide a rough indication of the required budget and reduce risk profile to a more manageable size. Simulations are useful to validate certain key assumptions before committing scarce and expensive development resources while a clickable demo can be used to visualize one or more key concepts of the solution.
Moving to the right, prototypes and proof of concepts are effective tools to reduce the risk profile of the project without committing to a large scale development effort. They have a limited feature set, lifespan and number of users, allowing both the business and IT to ‘test the water’. Proof of concepts and prototypes can also be used to obtain an accurate cost estimate of the main development phase by realizing a representative cross section of the back log (e.g. 2 simple features, 2 moderately complex, 1 very complex).
The third category is a hybrid; a development approach combining elements from Agile and Waterfall. Think of:
- A project consisting of two phases:
- a Minimal Viable Product with a fixed scope, planning and budget in combination with
- one or more sprints to develop nice-to-haves (typically 20-30% of budget), leveraging on insights gained during the first phase of the project thereby reducing the number of wasteful sprints
- Fusing elements from project management methods (e.g. PRINCE2, PMBOK) with Agile Scrum / DevOps principles. Whether Agile Zealots like it or not, the business expects a certain result at a certain point in time after spending a pre-agreed amount of resources. A deviation of 10-20 percent is in most cases acceptable, but that’s about it. The project manager supports the scrum master by taking care of the managerial aspects of the project (e.g. report progress and issues in steering committee), allowing the scrum master to focus on the more operational side of the project (e.g. sprint backlog, quality, learning). In most cases, one project manager is able to take care of multiple Scrum teams.
- Porting the back-end of a legacy application using waterfall while an Agile Scrum team builds a modern front-end in parallel. When modernizing a legacy application, a large part of the functionalities remain the same, allowing for a waterfall approach to maximize efficiency (I). Other areas (e.g. UX, self-service features for customers, mobile apps) benefit from an Agile Scrum approach.
Last but not least, some projects should be realized using waterfall due to their specific context. The business executives of a nuclear power plant have an extremely low risk appetite, while a straightforward one-on-one port of a well-documented legacy application can benefit from waterfall’s higher efficiency (e.g. less idle test hours).
You may have noticed that the continuum depicted in figure 1 closely resembles the life cycle most solutions go through. Every business process and value proposition goes through a life cycle, starting with the translation of a vision or market opportunity into a value proposition customers are likely to buy. This means the business is not looking for a team of risk-averse IT administrators, but colleagues who are pro-active, business savvy and creative. In case the value proposition indeed appeals to a strong customer need, it moves towards the Growth phase. Here, the experimental nature is replaced by the need to scale and become profitable. Investments in fixed assets and a more permanent team structure become a safe bet.
At a certain point in time, the business process matures and the focus shifts from product optimizations (read: add features) to process optimizations (read: improved efficiency). The business demand will change accordingly, resulting in fewer software releases and more focus on IT costs. Hence, with the digitalization of business models, the business life cycle becomes as important for IT as it is for the business. Aligning the business life cycle with the solution life cycle is at least as (if not more) important for IT than the traditional life cycles it tends to manage (e.g. application life cycle, project life cycle, information life cycle).
- The decision to apply Agile practices should be driven by the context the solution has to operate in (let’s call it ‘context-driven development’).
- Agile and other development methodologies are not mutually exclusive.
- With the progression of time, the context of the solution changes and both the business and IT have to act accordingly.
Insight in the ‘bigger picture’ is not the only precondition to reap the benefits from Agile, DevOps and Continuous Delivery. At least as important is translating that picture into a coherent operating model.
The objective is agility, not Agile
Agile companies develop and nurture the ability to change their competitive position quickly and efficiently by integrating isolated organizational competencies using a combination of collaboration, coordination, speed and perseverance. Value is destroyed when a business unit operating in a fast moving and dynamic market engages an external vendor to develop a proof of concept, but is forced by the procurement and legal departments to follow traditional contracting practices (e.g. detailed specification, no risk sharing). Similarly, a governance structure and control framework optimized for stable markets (e.g. strict top-down decision-making, fixed multiyear investment plan, comprehensive control framework) can quickly smother any isolated effort to improve the companies’ agility.
Hence, investing in one or more Agile Scrum teams does not equal an agile company. Only when part of a company-wide, top-to-bottom, initiative, an investment in Agile development practices yields a positive return. Even better: an initiative to improve the agility of the end-to-end value chain as down and upstream business partners either add or destroy agility-related value. It is useless as a company to invest in bimonthly release schedule when key business partners already struggle to release their part of the value proposition once a year.
The good news is that, unlike access to rare and expensive earth metals like neodymium and yttrium, agility is in reach of every company. Agility requires flexibility, change, creativity, integration, collaboration and problem-solving; all traits humans excel in. However, this observation also means that implementing ‘agile’ processes, tools, ‘5 levels of Agile planning‘ or a new organizational structure will fail to improve organizational agility.
Agility depends on the companies’ leadership style, culture, management practices and individual skills, as depicted in figure 2. A team with the right skill sets and mindset will automatically come up with the processes, tools and other structure-related items to organize their work more effectively. For them, it will be a matter of common sense.
It is equally important to notice that agility and stability enjoy a symbiotic relationship. When successful products mature, value optimization is achieved through a more traditional, efficiency-focused, operating model. The value proposition and delivery processes become standardized and investments focus on economies-of-scale and process automation. This stable part of the company is also important for another reason. Quoting from an interview with Aaron De Smet, senior partner at McKinsey:
“Agility needs two things. One is a dynamic capability, the ability to move fast—speed, nimbleness, responsiveness. And agility requires stability, a stable foundation—a platform, if you will—of things that don’t change. It’s this stable backbone that becomes a springboard for the company, an anchor point that doesn’t change while a whole bunch of other things are changing constantly.”
The same applies to the IT team. Laying off IT staff solely based on the fact that they are responsible for the ‘mature’ part of the IT portfolio is a good way to destroy value. It may take two or even three years, but eventually reality will catch up with your company. Agile is only a part of the puzzle. Every IT solution eventually matures and even Proof of Concepts have to play along with the existing data and application landscape. Hence, there is value in both perspectives depicted in figure 2. It is OK for part of the team to be more introvert, preferring standard procedures and more centralized decision-making. Despite what some Agile Zealots may preach.
- An isolated investment in Agile development practices does not improve the agility of the company, business model or value chain.
- Agility requires stability for most companies and consequently most IT teams.
The last Agile-related aspect covered in this blog is the misconception that Agile teams create value. They don’t. Developers create features which turn into value somewhere downstream in the value chain.
Agile teams create features, companies create value
Value creation is the core objective of every company. Consequently, any decision that creates new value or protects existing value is a ‘good’ decision and vice versa. A company can create and protect value for consumers, shareholders, employees, suppliers, debt providers, government and even the society as a whole.
There is no universally accepted definition of the term ‘value’. It can, for example, be tangible, such as financial, speed-to-market or customer retention, and intangible, like brand value or intellectual property. What most agree on, however, is that at the end of the day, value is most meaningful when expressed by a financial number.
From a customer perspective, value is created in case of a ‘customer surplus‘: “the difference between the total amount that consumers are willing and able to pay for a good or service (indicated by the demand curve) and the total amount that they actually do pay (i.e. the market price).” According to this article in the Financial Times, a company creates value when:
- The equity market value increases
- Shareholder value is added
- The shareholder return increases.
- The required return to equity is met or surpassed
I don’t even bother with explaining the bullet points as it is beside the point of this blog (just follow the provided link if you are interested or read a book on Corporate Finance). The point is the inability of a Product Owner to be responsible for either customer surplus or value creation at company level. Features, points or the usefulness of realized features/points (often referred to as ‘business value’) do not equal value as the rest of the organization understands it. Similarly, the Earned Value approach considers value ‘earned’ when the feature or point is complete. It is a very useful project management concept, but there is no one-on-one relation with customer and company-level value metrics. Nevertheless, the widely used Scrum Guide states the following:
Page 2: “Scrum (n): A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.”
Page 5: “The Product Owner is responsible for maximizing the value of the product and the work of the Development Team.”
Agile’s close cousin Lean does a better job with Value Stream Mapping. Value Stream Mapping looks at the end-to-end value chain, “analyzing the current state and designing a future state for the series of events that take a product or service from its beginning through to the customer.” The development and operations team(s) are responsible for several of those events, but other events take place in the manufacturing, logistics, marketing and sales departments. Together, they create customer and company-level value.
Another widely used method to identify and map value is the Business Model Canvas. It consists of nine elements allowing the team to visualize existing business models and develop new ones. Specifically tailored to startups are the Lean Business Model Canvas and the AARRR metrics. The latter were developed by Dave Mcclure and represent all of the behaviors of a customer (Acquisition, Activation (conversion), Retention, Revenue and Referral).
Value Stream Mapping, visualizing business models and customer development require a cross-functional team effort as they are holistic in nature. They look through functional silos, tying together all organizational elements that create value. For business models with a high technology density, this includes the Product Owner, business analyst, lead architect and/or CTO in order to
- understand the essence and context of the business need or want (e.g. the problem behind the problem, market and competitor dynamics, key business constrains, the unwritten rules of the game),
- contribute (e.g. reframe existing business practices, technology as enabler, manage expectations)
- convey the actual need and wants of the business to the rest of the IT team in order to solve the problem behind the problem.
Agile is a point solution. It allows the development team to release features faster, but that does not necessarily mean they are solving the problem of the customer. Garbage in still results in garbage out. Unmanaged, you just produce more garbage. Value creation in hybrid and digital markets requires a coordinated effort by a team combining business and IT competencies.
- Software developers create features or points, the company as a whole creates value.
- Software development is not the same as customer development
Notes and references
(I) Gibbs, R. D., Project Management with the IBM Rational Unified Process: Lessons from the Trenches, 2006.
Agile assumes the organization has, or is willing to, adopt a certain culture and leadership style. Differences in culture can be observed at team, company and country-level. The challenges faced by those who want to implement Agile in Asia are described here, while Germany is covered here.
This article by Nerur et al. describes the importance of accessing the ‘Agile readiness’ the organizational, covering topics including control style, communication and processes.
Some of the founding authors of Agile try to claim Agile back from the Zealots by going back to its core principles. Dave Thomas wrote Agile is Dead (Long Live Agility) and Andy Hunt and Jared Richardson launched their own rescue effort by introducing the GROWS Method.