How to use Agile with common sense?

The greater the uncertainty and dynamics of a market in which the organization operates, the more the business and IT crave flexibility and experimentation. And in situations that call for flexibility, it is difficult to formulate and capture all the specifications in advance – which is a requirement of the traditional waterfall method.

As such, the standard Agile/Scrum methodology is ideal for creating proofs of concept and determining the expected speed of a large development project. In the latter case, multiple short development periods, called 'sprints', are used to develop functionality of varying complexity, giving the project team a fairly accurate indication of the flow speed of the overall project.

The Agile approach offers even more advantages:

• focusing on the specific challenge of the (internal or external) customer or client
• fast deliveries (for example, by developing the most valuable functionality first)
• prevent waste (for a proof of concept, a detailed needs analysis can be omitted)
• short cycle learning (e.g. customer feedback after each sprint demo),
• the flexibility to make decisions late in the process (e.g. backlog prioritization along the way),
• empowering the team (e.g. encouraging rather than prescribing),
• built-in integrity (e.g. soft controls, authority based on expertise rather than hierarchy)

All good things, which is why the Agile methodology sells itself so easily all the way to the boardroom. One of the consequences was that overnight, all project managers had to get out or be retrained as Scrum Masters. It also led to backlogs (the planned work stock) permanently overflowing, having to redo unnecessary things, and lazy business and IT organizations. How so?

Agile can lead to laziness

In the first week of development, the business wants a proverbial green car, after three sprints it prefers a blue one and after sprint eight red seems to be the best choice. Ask an outsider a critical question about this and you will be told by the team that this kind of ‘flexibility’ is the power of Agile.

The scrum team itself, especially if it consists of hired outsiders, will not easily ask critical questions about this. They continue programming, testing, refining the backlog and obediently executing what the product owner asks on behalf of the business. The obvious reason: hourly billing.

Because the implicit assumption in Agile is that there is always another sprint, it is not only important to be critical of the functional items on the backlog. The same assumption can also lead to a lazy attitude of the team towards the delivered work. Not asking critical questions during the refinement sessions and a lot of programming errors lead to backlogs that are largely filled with rework, the modification or rebuilding of software components.

A good balance between trust on the one hand and a positive-critical attitude towards the team on the other, is therefore an important condition for getting the most out of the available budget. Active involvement of senior directors from the business can play an important role here.

But there is another important requirement for a successful agile development approach: a measured balance between functional and non-functional requirements. Let's take a look at that.

An apartment building requires a different foundation than a garden house

Putting the customer first has a downside, namely too much focus on functional requirements, too early in the development process. Users want nothing more than to see a new set of features at the end of each sprint, an expectation that is stimulated by the focus on visibility within the standard agile development methodology.

Building the software equivalent of a garden house in this way is no problem, but realizing a fifty-story building using this approach is not a good idea: before the building reaches the tenth floor, it has collapsed again. For a fifty-story building, it is crucial to lay a good foundation first, rather than starting to build and determining after each floor what is needed to realize the next floor. It is no different with software, although the foundations (such as the architecture, security, scalability, performance and monitoring) are not immediately visible to the user.

Focusing too much on functionality too soon leads to a rapidly increasing debt ('technology debt') that must be paid off sometime in the future. It is a need that the team behind the Scaled Agile framework fortunately recognizes, through explicit attention to the subject of architecture ('Agile Architecture strategy') and its realization ('Architecture Runway').

As with the previous point of interest, the maturity and fortitude of those directly involved will determine the extent to which the long-term perspective wins out over tangible short-term results.

And then there is a category of software that must work perfectly immediately at the end of the development process, with no room for error.

Agile does not go hand in hand with first-time-right

When a plane comes off the runway for the first time, there is no more room for 'oops, we'll fix that in the next sprint'. The first release to production múst be good.

In addition, the more than eight million lines of code aboard the F35 Lightning II are embedded in thousands of components, such as the radar, engine, and cockpit. All these components are manufactured by hundreds of suppliers. To ensure that they nevertheless function in perfect harmony, the architecture, technology and all requirements must be specified in detail in advance. The same context also ensures that meaningful functional (integration) testing can only take place relatively late in the development process.

Another situation that benefits from a more traditional development approach is a one-to-one conversion of a well-documented and stable legacy application. Again, the potential benefits of Agile (e.g. biweekly demos, frequent interim production) are often outweighed by the associated additional costs. Thus, the standard Agile methodology works less well in situations where:

• the required functionality is known, or should be known, at the start of the project
• the initial delivery must cover the full scope of the functionality

In addition to more efficient staffing and resource allocation, a more traditional method also allows for more time-consuming tasks, such as preparing a solid set of documentation. Not only pilots and astronauts, but also engineers who maintain complex automated plants, and process engineers in chemistry and power generation, for example, cannot do without it.

Sometimes you need a fast, maneuverable, but fuel-consuming speedboat, while other situations call for a predictable, robust and efficient container ship.