2184
Managing Lean Software Development with Cumulative Flow Diagrams
David Anderson
Microsoft
Abstract: Agile methods such as Scrum and Feature-driven Development (FDD) have adopted the use of graphs that plot the functionality delivered in a release or iteration. Typically, these charts are referred to as "burn down" or, more recently, "burn up" charts. In FDD, they are simply known as Feature-complete Graphs. Cumulative Flow Diagrams (CFDs) are used in Lean Production. Using real examples from the field, learn how to use CFDs with agile and lean software development methods. Learn how to create a CFD and how to plot it daily. Learn how to read the information for the chart and how to use it to steer and control a project on a daily basis. Learn about the S-curve effect and its causes and techniques for reducing its influence. Finally, see how Statistical Process Control charts can be plotted from CFD data and used to provide management with an "early warning" of developing issues within a project.
Agile software development methods such as Scrum and Feature Driven Development manage and report project progress in a very different manner than traditional critical path project management. Scrum started with the use of a “burn down” chart which plotted the estimated number of hours remaining to complete the Sprint backlog. More recently “burn up” charts have become popular. These plot the number of completed Stories, Tasks or Features on the project with a projected completion target. It is possible to extrapolate the plot with a trend line to estimate the completion date of a project.
The concept of a “burn up” chart had been used in Feature Driven Development since its inception in the late 1990s. It’s called a Feature Complete Graph as shown in Figure 1.
Figure 1. FDD Feature Complete Graph
However, “burn up” charts aren’t really sufficient for managing a project. They have no concept of work-in-progress (WIP) and simulating the anticipated end date is problematic. With Agile Management [Anderson 2003] I introduced Cumulative Flow Diagrams (CFDs) as a better visualization tool. This paper explains why and shows how to use CFDs to better understand the health of a project and to predict its outcomes.
The completion of Features tends to follow an S-Curve model, as shown in Figure 2. The S-curve effect makes it difficult to predict the end-date of a project from a single plot of Features complete.
Figure 2. S-Curve for Features Complete
It is very difficult to sprint right out of the gate. It is probably impossible to achieve optimum throughput in software development at the start of a project. There are several reasons why this should be: feature inventory blocked due to ambiguous requirements; training of personnel; lack of understanding of the project domain; poor team working; changing designs; availability of tools; availability of environments; facilities; lack of technical infrastructure; minimal domain specific tools; and staffing levels. Some can be eliminated through better planning, others appear to be endemic. They represent the nature of the beast – software development.
When a new team is formed for a project, there will be an impact on velocity. This is endemic to the business. The team must work through its “forming” and “storming” [Tuckman 1965, Weaver 1997] phases before settling into its new normal habits. As the team gets to know each other’s strengths and weaknesses, there will be some friction. This friction will reduce the production rate of the team.
Occasionally, a team is dysfunctional. This will manifest itself as a low production rate and a flat curve. Dysfunction may result in poorly executed design or peer reviews, or petty arguments which block development. Monitoring the cumulative flow diagram arms the manager with a lot of information which can indicate a dysfunctional team, but only the manager can actually investigate and discover dysfunctional behavior. It is important to understand that a dysfunctional dispute between two individuals hurts the performance of the whole team.
Knowledge sharing is important to performance. A team which works well together will share knowledge. A team which is dysfunctional and poor at sharing will under perform. It is important to create an environment which encourages individuals not to hoard information such as measuring and rewarding individuals rather than teams. However, a full discussion of this is out-of-scope of this paper. Encouraging openness and knowledge sharing and creating a means for communicating the knowledge, such as a news server, or intranet knowledge base, will improve performance of the team.
If a development team is ready to start a project but they don’t have all the tools and environments needed to make progress then the production rate will suffer badly. Capacity constrained development resources must be exploited to the full. They must not be idle because tools or environments are not available. Hence, it is important that the project manager ensure tools and environments are in place early. A flat line at the start of a project may be a strong indication that the team doesn’t have the tools or environments that it needs.
Design changes will hurt at any time in a project. When someone discovers a requirement which necessitates the modification of a widely used interface, the regression effect across the system, causes velocity to drop dramatically. This problem is endemic. There will always be a little unforeseen uncertainty related to analysis and design.
There are four reasons why the production rate tends to drop off in projects regardless of software development method employed: increased integration effort; bug fixes; inventory blocked due to unresolved issues; and refactoring.
Increased integration effort is almost unavoidable. As a project becomes large, code from a new iteration must be integrated with the large existing code base. The number of problems found during build-integration and integration testing is likely to increase. The only way to avoid this is to never work on a large project or system, though the effects can be mitigated through a good loosely coupled architecture. Hence, large projects, even those built in smaller iterations, should anticipate this effect and allow for it in the planning of the project buffer.
The effect of increased problems due to build-integration can be measured by measuring the time from the start of the build until the build is declared successful. This is the local lead time through Integration Test. The manager can then assess what the overall downtime for the development team might be. In extreme cases, all development stops until the build is declared successful. The effect on the production rate can be calculated. The trend in the build-integration time can be used to predict the future impact of build-integration on velocity. Hence, the contribution of build-integration to the cumulative flow diagram can be predicted.
It should also be possible to use a tool, such as Borland Together, to calculate a metric for the degree of coupling in the architecture. The trend in this metric could be used to predict the increase in integration time.
As a project matures and more code is available for system and product test, the number of bugs reported may rise. As the bug database rises, there is a tendency to increase the mix of bugs to new client-valued functions being coded. The result is that the overall speed of the team is maintained but an increasing percentage of the effort is devoted toward fixing bugs. Bug fixes are related to feature inventory which is already shown as code complete and hence no additional value is earned on the cumulative flow diagram by working on them.
To minimize the effect of bugs on the code complete slope and to maintain velocity, it is vital to maintain high quality standards. Fewer bugs have a direct positive effect on the production rate of the system. The s-curve clearly shows how and why high quality makes a project go faster.
As a project nears completion, inventory which has been blocked due to unresolved issues raised during analysis or design, re-enter the critical path. Eventually all client-valued functionality must be coded, or the client must agree to remove them from the scope. If they remain in scope, the issues must be resolved. Issues which remain open near the end of a project, reduce the number of client-valued functions which can be progressed to completion. The result is that the speed of the team slows. Some developers may become idle, or more and more effort is concentrated on bug fixes, because other inventory is blocked.
To avoid a tail off in velocity it is vital to resolve issues before the client-valued functions re-enter the critical path. Hence, the project manager should be focused on fast and effective issue resolution. This maintains the overall throughput of the system. A flattening in the curve is a good visual indicator that the project manager needs to work harder to resolve open issues.
Refactoring will also impact velocity, if it involves features already shown as complete. When working code already completed is being re-worked then it is not being shown as new productivity. For every client-valued function being re-worked, the system is losing the possibility of a unit of new production. For this reason, I prefer to plot refactoring as new “dark matter” features which are added to the total inventory on the cumulative flow diagram.
Where refactoring is unavoidable due to a change in market conditions which force new architectural requirements upon the system or through sloppy coding which has limited the ability of the team to deliver the full iteration, then it is vital to convey to senior management, the true cost of the refactoring. The true cost can be calculated from the reduction in production rate, the increase in lead time, the increase in WIP inventory and the lost opportunity for throughput. If the benefit from refactoring financially outweighs the cost then it should happen.
For additional insight on the s-curve, you may also like to check out “Tracking Software Process” by Betsy Clark [Yourdon 2002]. She calls small pieces of client-valued functionality – work units – and observes that they plot as an s-curve.
In order to communicate, a fuller picture of a project’s health, I found it necessary to supplement a Feature Complete Graph with a report of the percentage complete and an accompanying graph. A percentage complete plot for FDD gives credit to Features which are in-progress. Each of the six milestones for a Feature is credited with an earned-value percentage, as shown in Figure 3.
|
Domain |
Design |
Design |
Coding |
Code |
Promote |
|
1% |
40% |
3% |
45% |
10% |
1% |
Figure 3. Feature Milestones and earned-value percentages
So Feature Complete graphs fail to communicate work-in-progress but the Percentage Complete figure and graph suffer from “false reporting”. Why? Because any agile developer will tell you that they only value finished working software. So there is no value in partially complete code. This result is compatible with the Theory of Constraints management accounting method, Throughput Accounting [Corbett 1997]. Value should only be recognized on delivery.
Hence, it was necessary to stop reporting percentage complete altogether. Only report the true earned value – Features complete – or find a better way to communicate the work-in-progress, i.e. a method which did not report the value of WIP but did communicate that work was progressing even when Features were not being completed every day.
If it is wrong to communicate earned value from partially complete WIP, then we should ignore WIP? No, I don’t believe so! Rather, we should care about it deeply. Why?
Work-in-progress can be thought of as inventory. In this case, it is knowledge work inventory – ideas for valuable working software, captured at some stage in a transformational process which takes it through transformations such as analysis, design, test plan, code, unit test and code inspection (depending on the method you are using). By tracking WIP through a series of transformations, we can better understand the health of a project and predict its outcome. These ideas are developed from the work of Donald Reinertsen but to understand that, we must first understand the contribution of Marvin Patterson.
With Accelerating Innovation [1993] Marvin Patterson introduced a concept for modeling the design process. He asked us to envisage that design was a process of information discovery. Before a design is started there is little or no information, perhaps only a vague thought. As the design emerges there is gradually more and more information and less and less uncertainty until the design is complete.
Mary Poppendieck [2003] suggested that software development can be understood with the same model. In other words, all software development is a “design problem”. This would allow us to model the flow of value through a software engineering system as the gradual reduction of uncertainty and the discovery of more and more detailed information until working code which passes appropriate quality control tests, is produced.
Writing in Managing the Design Factory [1997] Donald Reinertsen developed the ideas of Patterson a little further by introducing two concepts. The first observation was that design-in-process inventory could be tracked using Cumulative Flow Diagrams, as shown in Figure 4. CFDs were already in use in Lean Production to track the flow of value through a factory. The second and perhaps the most valuable insight was that the value of design information depreciates over time. There are several reasons for this. The main one is that information need only be created once, as the cost of replicating it is near zero. If design is information then the time it takes a competitor to duplicate the design, is the time in which the design (and its associated information) has a differentiating value. Design information is also perishable because of possible changes in the marketplace – fashions change and so do laws, regulations, materials, supply components, distribution networks and business models. To have real value a design must be appropriate for its time and it must come to market within its window of appropriateness. If software is design then the same must be true of it. The requirements for a software program must be perishable. Hence, the faster the requirements can be realized as working code and brought to market, the more value will be delivered. Reinertsen’s and Poppendieck’s observations tie software engineering firmly to the principles of Lean and the lead time for turning an idea into working software must be a critical to the financial success of any software activity.
Figure 4. An idealized Cumulative Flow Diagram for an FDD project
Little’s Law states that a queue of material can be analyzed in two ways: as inventory; or lead time. Simply put the size of the inventory is directly proportional to the lead time for processing that inventory. Hence, the size of work-in-process inventory matters because in agile development we want to complete software in short cycles and deliver value as often as possible. Figure 5 shows how to read the WIP inventory and Lead Times from a CFD.
Figure 5. Reading WIP and Lead Time from a CFD for Day 40 of a project
Figure 5 also demonstrates how batch size and batch transfers affect the cumulative flow plot. The batch transfers can be clearly seen from the jaggedness of the plot. With larger batch sizes, there is more WIP and longer lead times. With smaller batch sizes as in Figure 6, WIP is reduced and lead time falls accordingly. Note the smoothness of the plot in Figure 6. This is from a real FDD project with Chief Programmer Work Packages (small batches of Features) which were completed in less than 2 weeks. The lead times can be clearly read from the diagram.
It is easy to see from this that CFDs can be used to tell at a glance, the size of iterations and the type of method being used for development.
Figure 6. CFD showing lead time fall as a result of reduced WIP
Reinertsen describes WIP as a leading metric. What he means is that WIP can predict in advance the lead time and delivery date. It can therefore be used to correct problems before they become too serious. If we waited to measure lead time or delivery date, there may be greater problems stored up. I once had a project which was reporting 53% complete after 8 weeks but only 4 Features were completed. A CFD plot of this project would have alerted me, as the manager, much earlier. Not only would a CFD plot have given me warning but it would have pointed me towards the cause of the problem. The rest of this paper explains how to use CFDs to diagnose issues with the health of a project.
I like to differentiate between two types of changes in the overall feature inventory for a project. The first represents changes in the requirements or further requests from the customer. This is often referred to as “scope creep”. These are new features which were not in scope when the project started. However, there is a second type of new feature which must be understood. These are features which emerge during the project as the depth of analysis and design work becomes greater and the team gains a more thorough picture of the work involved. I call these emergent features, “dark matter”, because they have always been there but until that point in time, the team could not see them. Hence, dark matter is a reflection on the uncertainty in the initial analysis. The more uncertain a domain is to the team involved, the more likely it is that they will have missed something. Hence, there will be more dark matter on projects with greater uncertainty due to lack of domain knowledge. When your subject matter experts aren’t as expert as you’d like them to be, dark matter will be greater. Figure 7 demonstrates how to plot original scope, dark matter and new features. This data is from the same real FDD project shown in Figure 6.
Figure 7. Scope showing dark matter and new features
To understand the value of this plot, we must first understand the concept of scope uncertainty. When the Feature List is drawn up at the beginning of the project, the team is asked to guess how certain they are that all the Features have been identified. In the specific case shown, the team used a “yesterday’s weather” technique where they reflected on the dark matter which emerged from the previous iteration. This was used with some leaning on experience and it was agreed by consensus that the total scope was probably 220 Features with the initial scope showing 185 Features. Hence, the team was expecting dark matter of 35 Features net. The Scope Uncertainty Buffer was 35. The project schedule was then set using a “yesterday’s weather” sampling of velocity and multiplied by 220. This produced an end date of March 22nd.
To monitor the scope on a daily basis a scope control chart can be created as shown in Figure 8. The idea with this chart is that if scope falls below 170 then the team will accept some new scope into the release. However, if scope grows above 220, it is likely that the project will be late, unless the velocity is greater than expected. Velocity can be read from the CFD and hence the control limit could be adjusted if necessary.
Figure 8. Scope Control Chart
Now consider what actually happened on the project. Initially, total scope falls. This was due to features being deferred to a later release as it emerged that the customer didn’t really need them. Then the dark matter effect begins to kick-in and the inventory grows a little. Then some more features are deferred to a later release because once again, the customer didn’t really need them. Around March 8th, the customer submits a change request. This causes the feature inventory to jump but the team accepts it because they believe that they are on target to meet the delivery date. However, dark matter continues to grow through mid-March and by March 16th the total breaks through the control limit. The project is going to be late. At this point the manager has to intervene.
It turns out that the change request is not required for delivery on March 22nd but on April 14th (and later it was pushed out to mid-summer). So the change request gets dropped from scope and the project is back on track. A consensus agreement is reached to build the change request and deliver it as a special code drop before April 14th. As it turned out, that never happened. This clearly highlights the value of the agile approach and the visibility into that approach which is facilitated by the use of CFDs.
As in Figure 9, where there is a growing gap opening between features started but not designed, this in indicative of uncertainty in the requirements. It should be echoed by a growing issue log requesting clarification on the requirements. With features blocked due to insufficient information or ambiguity or lack of internal consistency in the requirements, the development team will start more features with the hope of making progress on those whilst the others are blocked. The overall WIP will grow.
Without a knowledgeable expert on-site customer, it may not be possible to get instant answers. The customer may not even know the correct answer or the business analyst or marketing manager may not be empowered to provide a full answer. Regardless, of the reason the cumulative flow plot reveals the symptom and the root cause should be hidden in the issue log.
Figure 9. Growing Gap between Started and Designed
Figure 10 shows a growing gap between designed and coded. The cause of this cannot be determined from the plot alone but it does provide a red flag which should prompt the manager to investigate.
The first possibility is poor quality of design or unworkable designs. The team declares a design as complete and the chart is updated but when they start coding the design, they discover that they haven’t thought it through adequately enough. This means that the real design work is done whilst coding. Hence, the chart was falsely reporting the progress. A manager could choose to fix this in two ways – get the team some training in better OO design; or, simply change the reporting method so as not to reflect design. My personal preference is for the former. I believe that good design improves productivity.
Figure 10. Growing Gap from Designed to Coded
Another reason that the gap from designed to coded could be rising is a lack of knowledge of the development language(s) or the APIs in use. If a new technology is being used for the first time, the team may be struggling to learn it.
Alternatively, there could be environment or tools issues. It isn’t unusual to start a project with a temporary license for a tool, platform or environment whilst the procurement people issue a purchase order for the full license. I have also seen cases where a new project pushes the configuration management environment over the edge and it requires new hardware or a rebuild or reconfiguration. This takes the tools or support team some time and can hold up mainstream development effort.
Undeclared refactoring may be another reason for a growing gap in designed from coded. It may turn out that the latest designs require changes to code written earlier in the project. This rework would not show up as dark matter unless the team were being very honest about it. In an extreme case, it could be that the architecture of the system has been invalidated by the latest feature designs and the whole system is being re-written. This would show as a large gap or a severe downturn in the slope of the “coded” plot on the CFD.
Cumulative Flow Diagrams provide a method for tracking progress of agile projects in a “burn up” fashion. Because they plot both the total scope and the progress of individual Features / Stories / Tasks / Functions / Use Cases they communicate absolute progress whilst visually providing a proportional message of total completeness. CFDs also offer us a simple method of tracking work-in-progress and visually analyzing the trend in lead time for delivery of working code. They provide a leading metric which allows teams and managers to react early to growing problems wherever they might appear in the flow between requirements and working code. CFDs provide transparency into the whole lifecycle. Tracking a project with a CFD is a key element in moving to a Lean system for software development.
David J. Anderson is the author of the recent book, “Agile Management for Software Engineering – Applying the Theory of Constraints for Business Results” published in Peter Coad’s series by Prentice Hall PTR in September 2003. He is Principal Consultant with VA Systems Professional Services. David was one of the team which created the popular agile development method, Feature Driven Development. He has introduced FDD at two Fortune 100 companies Sprint (a telecommunications operator in the United States) and Motorola. He writes the regular Agile Management column at the Borland Developer Network website and publishes his weblog at http://www.agilemanagement.net/.
He holds a degree in Computer Science and Electronics from the University of Strathclyde.
Email: fddmanager@yahoo.com
[Anderson 2003] Anderson,
David J., Agile Management for Software Engineering – Applying the Theory of
Constraints for Business Results, Prentice Hall, Upper Saddle River, NJ 2003
[Corbett 1997] Corbett,
Thomas, Throughput Accounting, North River Press, Great Barrington, MA 1997
[Patterson 1993] Patterson, Marvin, Accelerating
Innovation, Van Nostrand Reinhold, New York, NY 1993
[Poppendieck 2003] Poppendieck, Mary and Tom Poppendieck, Lean Software Development – an agile
toolkit, Addison Wesley, New York, NY 2003,
[Reinertsen 1997] Reinertsen, Donald G., Managing
the Design Factory – A Product Developer’s Toolkit, Free Press, New York, NY 1997
[Tuckman 1965] Tuckman, Bruce W., Development Sequence in Small Groups, Psychological
Bulletin, 63(6)
[Weaver 1997] Weaver, Richard G. and John D. Farrell, Managers as Facilitators – a
Practical Guide to Getting Work Done in a Changing Workplace, Berrett
& Koehler, San Francisco, CA, 1997
[Yourdon 2002] International Function Point User Group, IT Measurement – Practical
Advice from the Experts, Foreword by Ed Yourdon, Addison Wesley, New
York, NY, 2002
| View Slides | 32096_09301540_S.PPT (30 Sep) |