This third blog on Agile emphasizes the importance of critical thinking instead of blindly following what others say or write (me included). Every new software project has its specific context (e.g. relative importance of speed-to-market, ability of product owner to articulate the requirements). Hence, forcing every software development project to be cast in the same mold will inevitably result in a waste of scarce development resources. Furthermore, the name of the game is agility, not Agile.
From dogmatic to pragmatic
Some forget that Agile is a means to an end. It is a cog in the machine required to translate the need of the business into an effective and efficient solution. Other cogs include the choice between COTS and custom software, the decision of make or buy, and the Agile readiness of the organization. Combined with the uncertainty and complexity of the requirements, required speed-to-market and expected lifespan of the solution, these topics shape the context the solution has to operate in. A context which allows Agile to either add or destroy value.
Hence, the decision to apply Agile Scrum has to be based on a rational decision making process. Quoting from the article Post-Agile: A Design Thinking Approach to Software Development by Tom Dabson:
“Regardless of how trained and focused your people are, when the projects you work on vary in complexity and time finding a single project management solution that works for all is impossible.”
In order to balance flexibility (‘every project is unique’) and efficiency (‘every project is the same’), the business and IT should agree on a set of principles to govern, develop and support new solutions (‘comply or explain’). Using waterfall in a highly dynamic environment does not make sense, nor does Vanilla Agile for an one-on-one port of a Cobol application to C#. Like the example depicted in figure 1, these principles should enable the stakeholders to objectify decisions regarding the:
- governance model (e.g. distribution of decision-rights, trust or control-based management),
- sourcing model (e.g. ASP, SaaS, developed internally or development by offshore vendor)
- architecture and technology (e.g. PoC on PaaS platform, business critical applications in Java or PHP)
- development methodology (e.g. Agile, waterfall, S-model), and
- operations model (e.g. best-effort support or SLA-based support).
The approach depicted in figure 1 allows for more parameters than Ralph Stacey’s complexity matrix which focuses on the certainty of the technology and certainty of the scope. For native digital and hybrid business models, the context of the solution is shaped by more than these two parameters. With the digitalizing of business models, the protective layer provided by the business to shield IT from the external market is all but gone. Today, evolving customer demand, the entrance of competitors, globalization and new regulations are as relevant for IT as they are for the business.
The left side of the continuum in figure 1 captures the most risky needs and wants from the business. Think of very complex and/or highly uncertain requirements and/or technologies. Here, desk research, interviews, brainstorm sessions, clickable demos and simulations can be used to provide a rough indication of the required budget and reduce risk profile to a more manageable size. Simulations are useful to validate certain key assumptions before committing scarce and expensive development resources while a clickable demo can be used to visualize one or more key concepts of the solution.
Moving to the right, prototypes and proof of concepts are effective tools to reduce the risk profile of the project without committing to a large scale development effort. They have a limited feature set, lifespan and number of users, allowing both the business and IT to ‘test the water’. Proof of concepts and prototypes can also be used to obtain an accurate cost estimate of the main development phase by realizing a representative cross section of the back log (e.g. 2 simple features, 2 moderately complex, 1 very complex).
The third category is a hybrid; a development approach combining elements from Agile and Waterfall. Think of:
- A project consisting of two phases:
- a Minimal Viable Product with a fixed scope, planning and budget in combination with
- one or more sprints to develop nice-to-haves (typically 20-30% of budget), leveraging on insights gained during the first phase of the project thereby reducing the number of wasteful sprints
- Fusing elements from project management methods (e.g. PRINCE2, PMBOK) with Agile Scrum / DevOps principles. Whether Agile Zealots like it or not, the business expects a certain result at a certain point in time after spending a pre-agreed amount of resources. A deviation of 10-20 percent is in most cases acceptable, but that’s about it. The project manager supports the scrum master by taking care of the managerial aspects of the project (e.g. report progress and issues in steering committee), allowing the scrum master to focus on the more operational side of the project (e.g. sprint backlog, quality, learning). In most cases, one project manager is able to take care of multiple Scrum teams.
- Porting the back-end of a legacy application using waterfall while an Agile Scrum team builds a modern front-end in parallel. When modernizing a legacy application, a large part of the functionalities remain the same, allowing for a waterfall approach to maximize efficiency (I). Other areas (e.g. UX, self-service features for customers, mobile apps) benefit from an Agile Scrum approach.
Last but not least, some projects should be realized using waterfall due to their specific context. The business executives of a nuclear power plant have an extremely low risk appetite, while a straightforward one-on-one port of a well-documented legacy application can benefit from waterfall’s higher efficiency (e.g. less idle test hours).
You may have noticed that the continuum depicted in figure 1 closely resembles the life cycle most solutions go through. Every business process and value proposition goes through a life cycle, starting with the translation of a vision or market opportunity into a value proposition customers are likely to buy. This means the business is not looking for a team of risk-averse IT administrators, but colleagues who are pro-active, business savvy and creative. In case the value proposition indeed appeals to a strong customer need, it moves towards the Growth phase. Here, the experimental nature is replaced by the need to scale and become profitable. Investments in fixed assets and a more permanent team structure become a safe bet.
At a certain point in time, the business process matures and the focus shifts from product optimizations (read: add features) to process optimizations (read: improved efficiency). The business demand will change accordingly, resulting in fewer software releases and more focus on IT costs. Hence, with the digitalization of business models, the business life cycle becomes as important for IT as it is for the business. Aligning the business life cycle with the solution life cycle is at least as (if not more) important for IT than the traditional life cycles it tends to manage (e.g. application life cycle, project life cycle, information life cycle).
- The decision to apply Agile practices should be driven by the context the solution has to operate in (let’s call it ‘context-driven development’).
- Agile and other development methodologies are not mutually exclusive.
- With the progression of time, the context of the solution changes and both the business and IT have to act accordingly.
Insight in the ‘bigger picture’ is not the only precondition to reap the benefits from Agile, DevOps and Continuous Delivery. At least as important is translating that picture into a coherent operating model.
The objective is agility, not Agile
Agile companies develop and nurture the ability to change their competitive position quickly and efficiently by integrating isolated organizational competencies using a combination of collaboration, coordination, speed and perseverance. Value is destroyed when a business unit operating in a fast moving and dynamic market engages an external vendor to develop a proof of concept, but is forced by the procurement and legal departments to follow traditional contracting practices (e.g. detailed specification, no risk sharing). Similarly, a governance structure and control framework optimized for stable markets (e.g. strict top-down decision-making, fixed multiyear investment plan, comprehensive control framework) can quickly smother any isolated effort to improve the companies’ agility.
Hence, investing in one or more Agile Scrum teams does not equal an agile company. Only when part of a company-wide, top-to-bottom, initiative, an investment in Agile development practices yields a positive return. Even better: an initiative to improve the agility of the end-to-end value chain as down and upstream business partners either add or destroy agility-related value. It is useless as a company to invest in bimonthly release schedule when key business partners already struggle to release their part of the value proposition once a year.
The good news is that, unlike access to rare and expensive earth metals like neodymium and yttrium, agility is in reach of every company. Agility requires flexibility, change, creativity, integration, collaboration and problem-solving; all traits humans excel in. However, this observation also means that implementing ‘agile’ processes, tools, ‘5 levels of Agile planning‘ or a new organizational structure will fail to improve organizational agility.
Agility depends on the companies’ leadership style, culture, management practices and individual skills, as depicted in figure 2. A team with the right skill sets and mindset will automatically come up with the processes, tools and other structure-related items to organize their work more effectively. For them, it will be a matter of common sense.
It is equally important to notice that agility and stability enjoy a symbiotic relationship. When successful products mature, value optimization is achieved through a more traditional, efficiency-focused, operating model. The value proposition and delivery processes become standardized and investments focus on economies-of-scale and process automation. This stable part of the company is also important for another reason. Quoting from an interview with Aaron De Smet, senior partner at McKinsey:
“Agility needs two things. One is a dynamic capability, the ability to move fast—speed, nimbleness, responsiveness. And agility requires stability, a stable foundation—a platform, if you will—of things that don’t change. It’s this stable backbone that becomes a springboard for the company, an anchor point that doesn’t change while a whole bunch of other things are changing constantly.”
The same applies to the IT team. Laying off IT staff solely based on the fact that they are responsible for the ‘mature’ part of the IT portfolio is a good way to destroy value. It may take two or even three years, but eventually reality will catch up with your company. Agile is only a part of the puzzle. Every IT solution eventually matures and even Proof of Concepts have to play along with the existing data and application landscape. Hence, there is value in both perspectives depicted in figure 2. It is OK for part of the team to be more introvert, preferring standard procedures and more centralized decision-making. Despite what some Agile Zealots may preach.
- An isolated investment in Agile development practices does not improve the agility of the company, business model or value chain.
- Agility requires stability for most companies and consequently most IT teams.
The last Agile-related aspect covered in this blog is the misconception that Agile teams create value. They don’t. Developers create features which turn into value somewhere downstream in the value chain.
Agile teams create features, companies create value
Value creation is the core objective of every company. Consequently, any decision that creates new value or protects existing value is a ‘good’ decision and vice versa. A company can create and protect value for consumers, shareholders, employees, suppliers, debt providers, government and even the society as a whole.
There is no universally accepted definition of the term ‘value’. It can, for example, be tangible, such as financial, speed-to-market or customer retention, and intangible, like brand value or intellectual property. What most agree on, however, is that at the end of the day, value is most meaningful when expressed by a financial number.
From a customer perspective, value is created in case of a ‘customer surplus‘: “the difference between the total amount that consumers are willing and able to pay for a good or service (indicated by the demand curve) and the total amount that they actually do pay (i.e. the market price).” According to this article in the Financial Times, a company creates value when:
- The equity market value increases
- Shareholder value is added
- The shareholder return increases.
- The required return to equity is met or surpassed
I don’t even bother with explaining the bullet points as it is beside the point of this blog (just follow the provided link if you are interested or read a book on Corporate Finance). The point is the inability of a Product Owner to be responsible for either customer surplus or value creation at company level. Features, points or the usefulness of realized features/points (often referred to as ‘business value’) do not equal value as the rest of the organization understands it. Similarly, the Earned Value approach considers value ‘earned’ when the feature or point is complete. It is a very useful project management concept, but there is no one-on-one relation with customer and company-level value metrics. Nevertheless, the widely used Scrum Guide states the following:
Page 2: “Scrum (n): A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.”
Page 5: “The Product Owner is responsible for maximizing the value of the product and the work of the Development Team.”
Agile’s close cousin Lean does a better job with Value Stream Mapping. Value Stream Mapping looks at the end-to-end value chain, “analyzing the current state and designing a future state for the series of events that take a product or service from its beginning through to the customer.” The development and operations team(s) are responsible for several of those events, but other events take place in the manufacturing, logistics, marketing and sales departments. Together, they create customer and company-level value.
Another widely used method to identify and map value is the Business Model Canvas. It consists of nine elements allowing the team to visualize existing business models and develop new ones. Specifically tailored to startups are the Lean Business Model Canvas and the AARRR metrics. The latter were developed by Dave Mcclure and represent all of the behaviors of a customer (Acquisition, Activation (conversion), Retention, Revenue and Referral).
Value Stream Mapping, visualizing business models and customer development require a cross-functional team effort as they are holistic in nature. They look through functional silos, tying together all organizational elements that create value. For business models with a high technology density, this includes the Product Owner, business analyst, lead architect and/or CTO in order to
- understand the essence and context of the business need or want (e.g. the problem behind the problem, market and competitor dynamics, key business constrains, the unwritten rules of the game),
- contribute (e.g. reframe existing business practices, technology as enabler, manage expectations)
- convey the actual need and wants of the business to the rest of the IT team in order to solve the problem behind the problem.
Agile is a point solution. It allows the development team to release features faster, but that does not necessarily mean they are solving the problem of the customer. Garbage in still results in garbage out. Unmanaged, you just produce more garbage. Value creation in hybrid and digital markets requires a coordinated effort by a team combining business and IT competencies.
- Software developers create features or points, the company as a whole creates value.
- Software development is not the same as customer development
Notes and references
(I) Gibbs, R. D., Project Management with the IBM Rational Unified Process: Lessons from the Trenches, 2006.
Agile assumes the organization has, or is willing to, adopt a certain culture and leadership style. Differences in culture can be observed at team, company and country-level. The challenges faced by those who want to implement Agile in Asia are described here, while Germany is covered here.
This article by Nerur et al. describes the importance of accessing the ‘Agile readiness’ the organizational, covering topics including control style, communication and processes.
Some of the founding authors of Agile try to claim Agile back from the Zealots by going back to its core principles. Dave Thomas wrote Agile is Dead (Long Live Agility) and Andy Hunt and Jared Richardson launched their own rescue effort by introducing the GROWS Method.
The second part of this blog on Agile covers the transformation of Agile as a valuable set of development practices into a money printing machine for trainers, consultants, tool companies and certification bodies. It reduced Agile to a PowerPoint presentation, a set of bullets points deemed applicable to every situation. Those who actually work in the trenches know reality is far more complex than that.
You can find the first part of this blog here.
Lets starts with a positive note. More companies engage external service providers to build custom software. According to a research report by Forrester, the market grew from $43 billion to $136 billion between 2011 and 2015, a CAGR of 33 percent. Due to the widespread adoption of Software as a Service (SaaS), the amounts may look counter-intuitive.
But consider this: with the digitization of business models, companies inevitably start looking for ways to digitalize their strategic differentiators. SaaS solutions are optimized to automate standardized business processes. The business model of SaaS vendors is based on ‘build once, sell many times’. Hence, a company that wants to strengthen its unique strategic differentiators by fusing certain business and IT capabilities will be on the lookout for a custom solution. The high upfront investment is offset by the ability to tailor the solution to the specific needs of the company and ability to own the intellectual property rights.
If I were to believe the stories of Agile Evangelists, the full $136 billion invested in custom software would have resulted in business value if all those projects had adopted Agile development practices. However, according to an article by Vik Bogdanov based on research by Wrike:
“38% of organizations globally are using Agile and only 64% of all Agile projects meet their goals. As many as 70% of companies admit having at least one failed project in 2015. In addition, a staggering 43% of all Agile projects are challenged (late, over budget, and/or with fewer than the required features and functions) and 18% fail completely (either cancelled prior to completion or delivered and never used).”
Similar figures in the Standish Group 2015 Chaos Report. Of the 10,000 projects analyzed for the report, the average Agile project had a 39 percent change of success. For large projects, the success rate was 18 percent and 58 percent of the small projects reported success. While waterfall projects reported even lower success rates, Agile fails to materialize as the knight in shining armor.
Several founding authors of the Agile Manifesto declared Agile dead. Not literally, but they expressed their frustration with the commoditization of their brainchild. Quoting Andrew Hunt:
” What was supposed to be an adaptive, collaborative, and customer-centric approach to development has instead become an over-hyped, buzzword-ridden exercise that delivers almost no value to the organizations that have embraced it.”
Dave Thomas, another founding author is even more pessimistic:
” The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products.
Once the Manifesto became popular, the word agile became a magnet for anyone with points to espouse, hours to bill, or products to sell. It became a marketing term, coopted to improve sales in the same way that words such as eco and natural are.”
One can hire Agile Developers and Agile Testers, buy Agile Tools and become a Agile Certified Practitioner. Apart from the fact that agile is an adjective, not a noun, the Agile Manifesto is “a set of values based on trust and respect for each other and promoting organizational models based on people, collaboration, and building the types of organizational communities in which we [the 13 founding authors] would want to work.” It is not a set of strict rules or methodology. The founding authors expected us practitioners to be pragmatic and use our common sense.
But ‘mushy’ stuff is difficult to sell, let alone certify against. Trainers, consultants and tool vendors need easy to convey bullet lists in their PowerPoint presentations.
The same trainers, consultants and tool vendor also have to be mindful to catch the next wave. In this particular case, expect your consultant, education company or tool vendor to start talking more about DevOps or Continuous Delivery. According to Matthew Kern, DevOps
“is the “heir apparent”. If you buy software development services at a medium or large scale then someone will be in your office selling you DevOps this year. (When they do, they will say it is better than mere Agile. You have to buy it.)”
Miko puts his money on Continuous Delivery as the successor of agile.
“The paradigm of “Continuous Delivery (CD)” seems to be the logical successor to Agile. Continuous Delivery provides an umbrella term that does not specify methodology–and doesn’t require much of a manifesto. Everything you need to know is in the title–you just deliver shippable software in as continuous of a way as possible.”
Or maybe Lean Agile? Regardless of the label, the next wave will come, including the accompanying Peak of Inflated Expectations, Trough of Disillusionment and Slope of Enlightenment. When reaching the last one, common sense kicks in again, a precondition to reach the Plateau of Productivity.
But there are other, more relevant, reasons why Standish Group reports an average success rate of 39 percent for Agile projects.
Zealots and Theorists
William Edmondson suggests that “developers should stop focusing solely on standups, burn-down charts, refinement meetings, or sprint velocity, and start delivering true value to clients, i.e. quality products that meet their budgets and expectations, and are able to provide an appropriate ROI.” While I agree with the message the statement wants to convey, I think the core of the problem is not the developer, but the trainer who taught the developer.
I used to be a consultant and trainer myself so I understand the importance to grab a wave and make money until the wave dies out. At that point, you or your colleague has identified the next wave and the cycle starts again. The first wave I surfed was implementing ITIL V2 processes, followed by BSi15000 and ISO20000. I switched career before the bureaucratic nightmare ITIL V3 arrived (more on that in a future blog).
With every implementation project, I slowly learned to translate theory into practice. Culture, leadership style, business model, organizational complexity and the maturity of both the business and IT all shaped the implementation. So now and then, my pragmatism collided with the views of colleagues who took the term ‘best practice’ literally. These ITIL Evangelists or zealots ignored the organizational context and expected the IT department to strictly follow and execute everything according to the Book (actually ten books in case of ITIL v2, one for every process).
Zealots ignoring context is only part of the problem. Trying to capitalize as quickly as possible on the next wave, consultancy firms and training companies send their employees on a week-long training and certification crash course; and sell them as experts the next Monday. As a result, a trainer proficient in for example ITIL, PRINCE2 or ISO2700 suddenly has to train a group of developers in Agile Scrum. The trainer has no other choice but to stick closely to the standard slide deck. All practical questions from the attendants are answered with theoretical answers. The focus of the certification exam on regurgitating theory only reinforces the mismatch between an imaginary perfect world and the real world.
Similarly, consultants with years of experience in professionalizing support and operations teams are suddenly expected to transition a development team used to waterfall to Agile Scrum. If the experience of the consultant with software development is restricted to being told that ‘waterfall is bad, Agile is good’, the client company is in for a treat. Even more so if the consultant ignores the context the team operated in (e.g. culture, leadership style, business model, organizational complexity, maturity).
You learn to cook by cooking, not by reading a cook book. Only after spending time in the trenches, experiencing real life issues and challenges, a trainer of consultant can truly add value.
The remained of this second part is dedicated to three of the many challenges faced by practitioners.
In part one of this blog, customer focus is mentioned as one of the reasons why Agile appears to be the knight in shining armor. It is also one of its main pitfalls. With the exception of Proof of Concepts and other application with a limited lifespan and complexity, too much focus on early visibility and desires of the business inevitably results in two large and expensive piles. One with technical debt and another with defects.
Unless IT is able to bring balance to the following two forces:
- Feature-focus of business. The payback period of a secure, modular and scalable architecture is much longer than some cool visual gimmick that catches the eye of potential customers. And what about the choice between solving a defect or adding a functional feature? Guess which one is most likely to make it to the top of the backlog when technology illiterate business representatives are driving the backlog? Too one-sided focus on visibility ensures that the attention span of the team is limited to the optics of the current and maybe next sprint demo. As a result, the expected life cycle of at least five years described in the business case approved by the COO and CIO never makes it to the backlog. When technology illiterate business executes are in charge, technology debt and the defect backlog become orphans.
- Power-imbalance. This one acts as an multiplier for the first one. Vanilla Agile pushes IT into the role of Faithful Servant, a positioning described by Van der Zee (I) as “Always helping, wherever possible, no questions asked.” Very different from a Business Partner, which is described as “Top quality, knows what we want, always one leap ahead.” The lower seniority of the developers and scrum master and more technology illiterate the business, the larger the risk (both probability and impact) the project will fail. Giving full authority to the business puts the developers in an underdog position. They are restricted to executing what the Product Owner tells them.
Security (e.g. OWASP 10), architecture, reliability, upgradability, performance and maintainability are neither easy to visualize nor atomize into two-week sprints. They are not part of the happy flow, nor can they be done piecemeal. The same applies to defects. While they can be stored in an ever growing defect backlog, eventually the business has to face the music.
Agile requires the business and IT exit the hamster wheel and plan for the future as equals.
Quoting Moscar’s article Why Agile Isn’t Working: Bringing Common Sense to Agile Principles published on CIO.com: “You cannot replace accountable and responsible project management with some immature utopian myth of self-organization. That’s no science for success.”
I delegate as many responsibilities as possible. It allows me to expand the scope of my control and spend more time on other tasks, which I have plenty of. Micro-managers struggle to keep the big picture in mind and waste a lot of human potential. I don’t like somebody else looking over my shoulder, so why would I subjects others to the same treatment?
At the same time, I was involved in several challenged Agile projects whereby the team failed to sense and act effectively on ‘red flags’. Even worse, instead of asking help from the leadership team, they kept things to themselves (II). I guess one of the bullets in the trainers’ slide deck stated: The development team in Scrum is self-organizing.
Self-organization is a source of flexibility, productivity, creativity and even efficiency. It is also a very complex concept to translate from theory to practice. Furthermore, self-organization doesn’t mean that the team:
- “doesn’t have managers or that they get to decide what problems to solve or what product to build.
- doesn’t have any tooling constraints, or architectural constraints, or budget constraints, or timing constraints.
- decides who is on the team, how big the team is, or how much money everyone on the team should make.”
At least as important in my experience: commitment and the willingness to burn the midnight oil are not enough to be effective as a team. The widely adopted Situation Leadership Model from Hersey and Blanchard explains that individual performance depends on both the willingness and ability perform a task. Most of the challenged projects suffered from decisions made by individuals who were ‘unable but confident’.
Everybody has one or more blind spots: the willingness to take responsibility for a task, but unaware he or she lacks one or more of the required competences or skills. Think of:
- business executives setting unrealistic expectations
- IT promising the business something without overseeing the consequences (e.g. resources, planning budget)
- unable to look beyond theory (e.g. in reality, the business does not have unlimited budget)
- assuming there is no difference between soft skills required to manage a waterfall project or Agile project
- estimating project velocity after developing two simple features and testing the happy-path
- assuming a skyscraper can be build the same way as a one-story house
- ignoring an almost non-existent velocity in the first part of the project, assuming the team will magically make up the difference during the second half
- developing an application module in Java even though the rest of the application portfolio is based on C# (When asked why: “the team wanted to try something different”)
Hence, self-organization is a great goal to strive for, but it is dangerous to underestimate the impact of operationalizing this utopian concept.
Furthermore, most leaders and managers have work within the boundaries dictated by the skill sets, culture, experience and budget available to them. Coaching, training, 360 degree feedback, evaluation meetings and performance reviews can only achieve so much. Somebody used to twenty years waterfall development (being exactly told what to do) will have a very hard time to transition to an Agile approach. Coaching and training can make an introvert but highly experienced and capable developer a bit less introvert. He or she is however unlikely to ever become the life and soul of the party.
Effective self-organization requires leadership, time and common sense from all stakeholders involved.
At the start of the project, the business executive acting as the Product Owner asks the team: build a green car. Two weeks later, at the end of the sprint demo: “Hmmm, one second thought, paint it red”. The following week the business executive visits a potential client (result: paint it white!). Two months later, after visiting a conference with some colleagues: black is new white! Another twelve months pass and out comes a blue car with yellow dots and white stripes on the roof.
Ask critical questions and the typical answer will be along the lines of “Its Agile. Everybody is doing it this way.”
When the scrum team is funded by IT, considerable waste is almost inevitable as there is no incentive for the team to think things through. Not only feature-wise, but also regarding architecture, security and so on. Even when the business funds the project, wasteful practices can go on for a while, especially when senior executives don’t feel comfortable enough to call the bluff of the Theorists/Evangelists/Zealots.
Contrary to waterfall and Spiral Model, there is no explicit exit strategy in Agile. Agile loves changing requirements, even late in the development process. It is one of its strengths, but also a potentially expensive weakness. Left unmanaged (read: lack of executive oversight), the Product Owner and the rest of the team make sure the backlog remains filled. There are always new features, defects, performance issues, optimisations for the Ops team and tweaks.
Hence, proper governance, portfolio management, program management, performance management and even project management did not suddenly become obsolete with the introduction of Agile. Don’t let trainers and consultants fool you. When you have to go back to your CEO, COO, CFO for the fifth time to ask for more budget, you either have a really good story, or it is most likely time to continue your career somewhere else.
Unmanaged, Agile hides low business maturity (e.g. sloppy requirements) and low IT maturity (e.g. code quality, velocity)
In the third part of this blog, I will share several (expensive) lessons from the trenches to better unleash the full potential of Agile.
Notes and references
(I) Van der Zee, H., Measuring the Value of Information Technology, 2003.
(II) I am aware of the possibility that the team members keeps things to themselves due to past behavior the leadership team (e.g. punish mistakes).
Agile development is not a silver bullet, there are numerous situations where the Spiral Model or even waterfall is a better choice. More importantly, methodologies are always trumped by common sense and highly skilled team members. Only these latter two always result in success.
There is a lot of value in Agile, but only when combined with a healthy dose of common sense. Therein lies the problem.
All but a few pitch the Agile Manifesto as the greatest thing since sliced bread. Consultants and trainers earn large amounts of money by selling Agile as the solution for all development-related problems. To them, the Agile Manifesto is like the Bible.
In reality, many Agile projects take forever to complete or are abandoned after failing to deliver the desired results. I see companies rehiring the project managers and architects they fired a couple of years ago as part of implementing Agile. These companies went through the Peak of Inflated Expectations and are now crawling out of the Trough of Disillusionment.
Every new day is more unpredictable and dynamic than the day before. The higher the uncertainty of a market, the more the business and IT crave for flexibility and experimentation. In these situations, the business cannot articulate and fixate all requirements upfront, a prerequisite of traditional waterfall programming. At first sight, Agile enters the scene therefore as the knight in shining armor as it:
- puts the customer first,
- delivers rapidly (e.g. develop and deploy most valuable requirements first),
- strives to eliminate waste (e.g. no elaborated requirements analysis for a proof of concept),
- amplifies learning (e.g. customer feedback after every sprint demo),
- makes decisions as late as possible (e.g. adjust priorities backlog along the way),
- empowers the team (e.g. encourage instead of motivate),
- has built-in integrity in (e.g. soft controls, authority based on expertise not hierarchy), and
- sees the whole (e.g. work towards a shared business goal).
All good stuff, which makes Agile such an easy sell. Vanilla Agile (I) is actually great to realize proof of concepts and to determine the expected velocity of a large development project. In the latter case, several sprints are used to develop several features of various complexity, thereby providing the project team with an fairly accurate proxy of the velocity of the project as whole.
Realizing the software equivalent of a fifty story building with Vanilla Agile is however not such a good idea. More likely than not, the building will collapse before reaching the tenth floor.
Before moving towards the bad and ugly stuff, first some neutral, but nevertheless relevant, observations about Agile.
Agile is a business concept
Samsung introduced its wearable Gear, and even the Gear 2, before Apple introduced its Apple Watch. The reviews of especially the first generation Gear smartwatch were mixed at best, and consumers had a similar view considering a 30% return rate in Best Buy locations (II). For Samsung, being first was more important than offering the best experience. To mitigate the risks associated with this go-to-market strategy, Samsung uses a relatively high release frequency for its products. Apple follows a different strategy. Products are tested and refined within the company walls until considered mature enough to appeal to the customer segments targeted by Apple. Both companies use iterations, but their implementations differ.
The business and IT teams of a successful upscale brick-and-mortar retailer tasked with digitalizing the business model face a similar challenge: make the new digital platform available to all customers as soon as it reaches Minimum Viable Product status or wait until a more polished release becomes available. The main benefit of the first option is speed-to-market, reaching wealthy online customers before the competition does. However, launching an unpolished platform to high-end customers may also alienate them.
Hence, while Samsung, Apple and the retailer all benefit from iterative sprints and other Agile-related concepts, there is no single “best” implementation of Agile. Depending on the market (e.g. risk taking/averse customers, low/high competition) and strategy (e.g. Prospector versus Defender), the business and IT have to decide on the company-specific implementation of Agile (III).
Due to the digitalization of business models, Agile is first and foremost a business concept, even though it was created to improve the agility of IT.
Agile did not replace waterfall
The 8 million lines on board of the F35 Lightning II are embedded in thousands of components, such as the radar, engine and cockpit, manufactured by hundreds of suppliers. To ensure they, nevertheless, function in perfect harmony, the architecture, technologies and functional requirements all have to be defined upfront. Furthermore, all the code required to fly has to be fully functional and error free when the plane taxis to the runway for the first time. There is no room for continuous delivery when in there air, nor a long defect backlog. The same applies to the software used in nuclear power plants or x-ray machines. They all require a first-time-right approach.
Another situation that benefits from a more traditional development approach is an one-on-one conversion of a well-documented and stable application written in a legacy language. Here too, the potential benefits of Agile (e.g. demo progress to customer, frequent deployments) are outstripped by the additional costs related to iterative development. More generally, continue to consider waterfall-oriented or the V-model (Validation & Verification model) in situations where:
- the requirements are or have to be understood completely at the beginning
- the first deployment has to cover the full scope and/or has to be error free
Besides more efficient staffing and resource allocation, waterfall development also allows for more room for more time-intensive tasks like documenting user cases and drafting manuals for the operations and support team (IV). Sometimes the business needs a fast, agile and fuel-thirsty speed boat while other situations call for a predictable, robust and efficient container ship.
Agile and waterfall both have their sweet spots, they are complementary.
End of part 1 of this blog on the pro’s and con’s of Agile.
Notes and references
(I) I use the term ‘Vanilla Agile’ when implementations and/or trainings are based on theory only, ignoring common sense and lessons learned from practical experience.
(II) Grant, R., Samsung Galaxy Gear smartwatches have embarrassing 30% return rate, Venturebeat.com, October 2013.
(IV) Gibbs, R. D., Project Management with the IBM Rational Unified Process: Lessons from the Trenches, 2006.
This blog is about shifting gears and the ability to offer cars in other colors than T-Ford-black. It is the third and last part of a blog focusing on the importance of translating market and business context into the value proposition offered by IT.
FedEx and UPS dominate their markets, again using slightly different value propositions. Compared to UPS, FedEx offers its customer more flexibility at a slightly higher price point. More price sensitive customers opt for UPS and the more standardized value proposition that comes with it. More pronounced is the difference between Walmart and Amazon. The first has its roots in physical retail outlets while the second started as a native digital business model.
Differentiation can also be observed at an operational level. Marketeers want to try new things on a daily basis, while the controllers and bookkeepers of the finance and administration (F&A) department prefer stability and predictability. Marketeers enjoy rally and off-road racing, while controllers tend to take the train for its excellent safety record. When it comes to IT, the marketeers want to be behind the steering wheel with IT as the co-driver, knowing that only as a team they can win. For the controller, a Commercial of the Shelf (COTS) SaaS solution will do just fine.
The higher the technology-density of the market, the more important it becomes for IT to sense and act on the relative importance of co-creation, speed-to-market, flexibility, robustness, efficiency or other sources of contextual value. In hybrid and native digital markets this value can be equal or even surpass the base value represented by the functional requirements
Less uniform, more differentiated represents the ability to deliver context-aware IT solutions.
At company level, think of effectively positioning IT as either a Faithful Servant, Business Partner, John Average or Prima Donna. At operational level, the two key archetypes are Entrepreneurial IT and Foundation IT. The first is also known as ‘Strategic IT’or ‘Enabling IT’, but in hybrid and digital markets it is entrepreneurship that is required from IT. Being an entrepreneur means ‘one who undertakes an endeavor’ or an ‘enterpriser’ (I), a far better term when business and IT are together in pursued of more revenue, profit or less strategic risk.
Foundation IT, also known as ‘Transactional IT’ or ‘Factory IT’, is the traditional sweet spot of the IT department. Even though most consider it less sexy than Entrepreneurial IT, the majority of companies’ revenue and margin is generated by business processes enabled by this part of the IT portfolio. Entrepreneurial IT is in most cases a source of expected future cash flows, funded by current cash flows enabled by Foundation IT. The latter’s portfolio also contains the capabilities required to ‘glue’ IT systems together, like the Enterprise Service Bus, and the platform required by Lean and Business Process Management initiatives. As such, it acts as the solid foundation required by other initiatives to succeed.
To survive, companies need both flavors, differing only in the relative amount.
Notes and references
(I) Technically, the term should be intrapreneur as IT would be acting as in entrepreneur within the company. As this term is not widely used, I use the more common term.
Enabling IT and Factory IT are introduced in the McKinsey article Reshaping IT management for turbulent times. It explicitly covers the importance to differentiate between ‘fast’ and ‘slow’ change.
The Harvard Business Review article Mastering the three worlds of information technology from 2006 describes three varieties of work-changing IT. The article relates investments in IT to organizational change, but does not differentiate between ‘fast’ and ‘slow’ change.
One of the first articles emphasizing the importance to differentiate is from Peter Weill: The relationship between investment in information technology and firm performance in the manufacturing sector. Ph. D. dissertation, New York University, New York.
Markets move at different speeds and in different directions, constantly reshaping the technology-related capabilities required by the business. In the second part, the difference between Customer Intimacy strategy and Operational Excellence strategy is used demonstrate that ‘differentiation’ is more than choosing Agile over Waterfall development.
The first part of this blog dedicated to market and business context is here.
Dell was founded in 1984 and was added to the Fortune 500 in 1992. Its key to success was building direct relationships with the customer via the internet. At the time, most competitors targeting consumers via analogue sales channels, such as brick and mortar shops. Selling directly to the customer also enabled Dell to build-to-order instead of the traditional build-to-stock model.
It kept waste to a minimum and allowed the customer to choose from a wide selection of configuration options. Dell’s pursued a customer intimacy strategy, contrary to the operational excellence strategy of its competitors (I). For the latter, price was the main differentiator while Dell targeted customer looking for choice and flexibility. Even though Dell changed its strategy around 2008, it is still a classic example of a customer intimacy focused business model.
Dell and its competitors all required a customer relationship management (CRM) solution, an enterprise resource planning (ERP) system and solution to manage the supply chain. At first sight, one might think that the choice between a customer intimacy strategy or operational excellence strategy only affects the customer-facing part of the company (e.g. Dell B2C, competitors B2B). Production, logistics and other downstream value chain activities remain shielded from the market.
This is not the case as Dell’s strategy translates into a build-to-order value chain, very different from build-to-stock. A build-to-order business model directly targeting B2C segments has to be far more agile and tightly integrated than a build-to-stock B2B business model. Walmart orders thousands of PC’s with a specific configuration, using an attractive price point as the primary differentiator.
The business model (necessitate and) shapes the IT business model
Also most law firms pursue a customer intimacy strategy, competing fiercely among each other to attract and retain customers. The accompanying high cost of customer acquisition results a drive to quickly increase the share-of-the-wallet (SOW). To do so, it is crucial adopt a strong customer-first mentality (Table 1). The specific wants, needs and journey of the customer is leading, the business processes and, therefore also the IT value propositions, follow. Here, the strategic context IT has to operate in is shaped by the drive of the business to offer a seamless extension of the customers’ business process or journey through domain knowledge, customization and flexibility.
The starting point for a company pursuing an operational excellence strategy is very different.
Cost leadership requires a standardized product portfolio enabled by a relatively uniform set of business processes and IT solutions. The compelling reason for customers to buy is not customization, but price. The strong focus on cost and economies-of-scale affects every decision the business and IT make.
Where the CIO employed by a law firm of hospital may opt for a best-of-breed sourcing strategy, the CIO of a telecom, budget airline or 3rd party logistics company will consider large scale outsourcing to leverage on the scale of IT service providers. Similarly, both CIO’s will make different choices regarding custom-build software versus commercial off-the-shelf (COTS) software, waterfall versus agile development, hard control versus soft control environment and so on.
In other words: the business context and strategy shape every aspect of the IT Business model.
Notes and references
(I) According to Treacy and Wiersema, a company has to focus on one of the following three generic value disciplines: Customer Intimacy, Operational Excellence or Product Leadership. The value disciplines are covered more in-depth in the book and here.
Markets move at different speeds and in different directions, constantly reshaping the technology-related capabilities required by the business. This blog is the first in a series covering the third principle of the Digital Manifesto.
Business IT convergence 101
When in New York, searching for a restaurant on your mobile yields those nearest to your current location. If your search history indicates you like Chinese food, the results will highlight those.
To create this kind of personalized experience, Google Now, Cortana and Siri combine several data sources, such as location and past usage patterns, to deliver a personalized experience. The capability to show the restaurants in New York instead of Moscow reduces the ‘search stress’ faced by customers and the saved time is a source of customer benefits.
The business has a similar expectation. When requesting a new IT solution, the context, such as stable versus emerging market, or having little or intense competition, is at least as relevant as the functional need. An application with a world-class user experience and feature set is a failure when launched after the competition has seized the market.
At solution-level, the business context has to be reflected by both the design (e.g. non-functional requirements) and realization and support processes (e.g. Waterfall versus Agile development). While these operational are important, they are by themselves point solutions.
To maximize the return on investments in technology, the IT business model as a whole has to be tuned to the context the business operates in.
Differentiation (e.g. focus on cost or speed-to-market) is, first and foremost, a strategic topic. An IT and business operating models optimized for Customer Intimacy in an analogue market are very different from one optimized for Product Leadership in a fast moving digital market (I) Similarly, the strategic context embodied in the choice of the business to position itself as an extrovert Prospector or more introvert Defender reaches far beyond the development approach (II).
Notes and references
(I) According to Treacy and Wiersema, a company has to focus on one of the following three generic value disciplines: Customer Intimacy, Operational Excellence or Product Leadership. The value disciplines are covered more in-depth in the book and here.
(II) Prospector and Defender strategies refer to the model by Miles and Snow. The Miles and Snow typology is covered more in-depth in the book and here.