IBM Global Services, Application Management Services
IT Architect and IT Specialist Institute Central Region 2004
in Herrenberg, Germany
© 2004 IBM Corporation
Lean Software Development
Dr. Christoph Steindl
Senior IT Architect and Agile Method Exponent
Certified ScrumMaster
Christoph_Steindl@at.ibm.com
pg_0002
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
2
Agenda
History of Lean Manufacturing and Lean Thinking
7 Principles and 22 Tools (abbreviated)
Summary
pg_0003
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
3
History – Rigid or Lean Processes
The 1990s were the decade of Process for IT. We prostrated ourselves before
the CMM and ISO. It wasn't enough just to do things right, we also had to say in
advance exactly what we intended to do and then do exactly that ("
plan the
work, and work the plan
")*.
The list of companies most successful at climbing up the CMM ladder early in
the 1990s reads like a Who’s Who of downsizing by the end of the 1990s.
Process rigor was simply not the right recipe for an era in which everything was
changing.
Today the era of fat process is over. As Jim Highsmith has said, “Thin is in.”
*
* see foreword by Tom DeMarco to the book “Agile Software Development Ecosystems” by Jim Highsmith,
Addison-Wesley, 2002.
pg_0004
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
4
History of Lean Thinking and Lean Software Development
On the other hand, Toyota has started in the 1980s to revolutionize the
automobile industry with their approach of "Lean Manufacturing“
–
to eliminate waste
–
to streamline the value chain (even across enterprises)
–
to produce on request (just in time), and
–
to focus on the people who add value.
Lean Thinking* capitalizes on the intelligence of frontline workers, believing that
they are the ones who should determine and continually improve the way they
do their jobs.
Mary and Tom Poppendieck have transferred principles and practices from the
manufacturing environment to the software development environment**.
*
* see the book "Lean Thinking" by James P. Womack and Daniel T. Jones
** see the article "
Lean Software Development
" and the book "
Lean Software Development: An Agile Toolkit
“
by Mary and Tom Poppendieck
pg_0005
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
5
Manifesto for Agile Software Development
We* are uncovering better ways of developing software by doing it and
helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items
on the left more.
* Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler,
James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin,
Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas, 2001.
pg_0006
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
6
LSD with 7 Principles and 22 Tools
Eliminate Waste
–
Seeing Waste, Value Stream Mapping
Amplify Learning
–
Feedback, Iterations, Synchronization, Set-Based Development
Decide as Late as Possible
–
Options Thinking, The Last Responsible Moment, Making Decisions
Deliver as Fast as Possible
–
Pull Systems, Queuing Theory, Cost of Delay
Empower the Team
–
Self-Determination, Motivation, Leadership, Expertise
Build Integrity In
–
Perceived Integrity, Conceptual Integrity, Refactoring, Testing
See the Whole
–
Measurements, Contracts
pg_0007
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
7
Principle #1: Eliminate Waste
... does not mean to throw away all documentation, but to spend time only on
what adds real customer value.
Eliminating waste is the most fundamental lean principle, the one from which all
the other principles follow.
Thus, the first step to implementing lean development is learning to see waste.
The second step is to uncover the biggest sources of waste and eliminate them.
Eliminate waste Uncover waste
pg_0008
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
8
http://www.xp2003.org/xp2002/talksinfo/johnson.pdf
pg_0009
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
9
Eliminate Waste by Seeing Waste (Tool #1) (1/2)
In manufacturing (software development), there are seven kinds of waste:
1. Inventory (partially done work in software development)
Inventory has a tendency to become obsolete. You might have no idea whether or not it will
eventually work or be needed until the software is actually in production; you don't really know if it will
solve the business problem. What if the system never makes it into production?
2. Extra processing (extra processes)
Paperwork consumes resources, slows down response time, hides quality problems, gets lost,
degrades and becomes obsolete. Just because paperwork is a required deliverable does not mean
that it adds value. If you must produce paperwork, keep it short and keep it high level.
3. Over production (extra features)
Every bit of code in the system has to be tracked, compiled, integrated, and tested every time the
code is touched, and then it has to be maintained for the life of the system. Every bit of code
increases complexity and is a potential failure point. There is a great possibility that extra code will
become obsolete before it’s used.
4. Transportation (task switching)
Assigning people to multiple projects is a source of waste. Every time software developers switch
between tasks, a significant switching time is incurred as they get their thoughts gathered and get into
the flow of the new task. Belonging to multiple teams usually causes more interruptions and thus
more task switching. This task switching time is waste.
pg_0010
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
10
Eliminate Waste by Seeing Waste (Tool #1) (2/2)
In software development, there are seven kinds of waste (continued):
5. Waiting
One of the biggest wastes in software development is usually waiting for things to
happen. Delays in starting a project, delays in staffing, delays due to excessive
requirements documentation, delays in reviews and approvals, delays in testing, and
delays in deployment are waste.
6. Motion
When a developer has a question, how much motion does it take to find out an answer?
People aren’t the only things that move – various artifacts move also. Requirements may
move from analysts to designers, and then design documents move from designers to
programmers, and then code moves from coders to tests, and so on. Each handoff of an
artifact is fraught with opportunities for waste, great amounts of tacit knowledge remain
with the creator of the document and never get handed off.
7. Defects
The amount of waste caused by a defect is the product of the defect impact and the time
it goes undetected. Find them as soon as they occur, test immediately, integrate often,
and release to production as soon as possible.
pg_0011
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
11
Eliminate Waste by Value Stream Mapping (Tool #2)
Mapping your value stream is a good way to start discovering the waste in your software
development process. It focuses attention on products and their value to customers rather
than on organizations, assets, technologies, processes and career paths.
Create a value stream map with paper and pencil by walking around your organization:
–
Pretend you are a customer request and imagine yourself going through each step of your
process. Don’t ask people what happens; walk around, look at the data, find out for yourself.
–
Go to the place where a customer request comes into your organization. Draw a chart of the
average customer request, from arrival to completion.
–
Working with the people involved in each activity, sketch all the process steps necessary to fill
the request, as well as the average amount of time that the request spends in each step.
–
At the bottom of the map, draw a timeline that shows how much time the request spends in
value-adding activities and how much time it spends in waiting states and non-value adding
activities.
A value stream map provides a starting point for evaluating and improving your software
development process: Pick the biggest opportunities to increase flow and value-added
time.
Once you have a value stream map of your organization, the next step is to extend it to your
customers.
pg_0012
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
12
Agile Value Stream Maps
LSD, p. 12
LSD, p. 11
pg_0013
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
13
Principle #1: Eliminate Waste – Do It Yourself
1.
Make a list of the 10 or 15 most important activities in your organization. Put yourself in the shoes of
a customer and rate each item from 1 to 5, with 1 meaning customers probably don’t care about the
activity and 5 meaning customers value it highly. Think of the low-scoring activities as waste. Take
the two lowest scoring items and develop a plan to cut the time on these activities in half.
2.
At your next seven team meetings, take some time to discuss each of the seven wastes of software
development, one at a time:
–
Partially done work
–
Extra processes
–
Extra features
–
Task switching
–
Waiting
–
Motion
–
Defects
For each waste, ask the questions
–
Do you agree that this “waste” is really a waste? Why or why not?
–
Whether or not you agree that the item is a waste, estimate how much time it consumes in an average week.
–
What can or should be done to reduce that time?
3.
Develop a value stream map for your organization. Start with an incoming request and map a timeline
of its progress to providing customer value. Find out how much of the time is spent adding value and
how much is spent waiting. Take the biggest cause of delay and develop a plan to cut it in half.
pg_0014
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
14
Principle #2: Amplify Learning
… does not mean to keep on changing your mind, but to increase feedback, when
you have tough problems.
When an organization has software development challenges, there is a
tendency to impose a more disciplined process on the organization, with more
rigorous sequential processing
–
where requirements are documented more completely,
–
where all agreements with the customer are written,
–
where changes are controlled more carefully,
–
where each requirement must be traced to code,
–
where additional deterministic controls on a dynamic environment lengthen the
feedback loop.
Just as control theory predicts, this generally makes a bad situation worse.
pg_0015
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
15
Amplify Learning with Feedback (Tool #3)
When a problem develops…
–
the first thing to do is to make sure the
feedback loops are all in place,
–
the next thing to do is to increase the
frequency of the feedback loops in the
problem areas.
Increasing feedback is the single most
effective way to deal with troubled
software development projects and
environments.
–
Instead of gathering more requirements
from users, show them an assortment of
potential user screens and get their input.
–
Instead of letting defects accumulate, run
tests as soon as the code is written.
–
Instead of adding more documentation or
detailed planning, try checking out ideas by
writing code.
–
Instead of studying more carefully which
tool to use, bring the top three candidates
inhouse and test them.
Person A with
background and
understanding
Person B with
background and
understanding
pg_0016
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
16
A universal starting point for all agile software development approaches is
iterations: short feedback loops enhance control; iterations are points of
synchronization (the team and the customer see what has been accomplished);
iterations force decisions to be made.
Amplify Learning with Iterations (Tool #4)
Increment 1 Increment 2 Increment 3
Incremental
A D C T D
Analyze Design Code Test Deploy
Waterfall
Increment 1 Increment 2 Increment 3
Iterative & Incremental
Feedback after each iteration
pg_0017
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
17
Advantages of Rapid Cycle Times
1.
Increases learning tremendously.
2.
Eliminates buggy software because you die if you don't fix this.
3.
Forces complete Scrum automation which allows real time, rollup reporting across all
products and releases.
4.
Forces Scrum of Scrums to meet daily.
5.
Forces weekly MetaScrum meeting to plan and coordinate product releases across the
company.
6.
Fixes the install process because you die if you have to install 45 releases this year and
install is not easy.
7.
Improves the upgrade process because there is a constant flow of upgrades that are
mandatory. Makes upgrades easy.
8.
Forces quick standardization of software via new features rather than customization and
one off. This is the core of Oracle's current eBusiness strategy that Larry Ellison is using
against PeopleSoft and SAP.
9.
Forces implementation of sustainable pace, a basic Agile principle. You die a death of
attrition without it.
10.
Allows waiting to build new functionality until there are 4-5 customers who pay for it. This
is counterintuitive, and caused by the fact that you can tell the customer it will be ready
by the time they sign the contract and put together their installation team (since
everything is ready within 90 days). If it is too big to build in 90 days, you give them a 90
day go-live release with top priority functionality and monthly upgrades thereafter until
they have what they want.
Jeff Sutherland on scrumdevelopment@yahoogroups.com 23.11.2004
pg_0018
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
18
Prioritized
backlog
Increment of
functionality
Synchronize
with the customer
Synchronize
within the team
Amplify Learning with Synchronization (Tool #5)
Whenever several individuals are working
on the same thing, a need for
synchronization occurs. The need for
synchronization is fundamental to any
complex development process.
Synchronize / integrate technically:
integrate daily within a team (i.e. check-in
at least daily into local repository)
Integrate weekly across multiple teams
(i.e. check-in at least weekly into the
central repository)
More frequent builds are better; they
provide much more rapid feedback.
Builds and build tests should be automated.
Daily
Meetings
Weekly
Meetings
Synchronize
Across teams
pg_0019
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
19
Development in Cycles
Project Cycle
(the cooperative game)
Charter
Deliver
Wrap up
and
harvest
Delivery Cycle
[1w - 3 mo]
Calibrate
release
plan
Iterate
Complete
(celebrate
& reflect)
Iteration
[1w - 3 mo]
Estimate &
plan
Develop
Complete
(celebrate
& reflect)
Integration Cycle
[30’ – 3d]
Develop
integrate
Perform
system test
Developm. Episode
[mins - hours]
Design
Code & test
Check in
At least twice
At least
once
Day
Week
Month
Year
Maintenance
Cycle
Report
bug
Integrate
Accept
bug fix
Engage
Within team
Across teams
Deliver to
real users
Stand-up
meeting
pg_0020
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
20
Amplify Learning with Set-Based Development (Tool #6)
In set-based development, communication is about constraints, not choices. This turns out
to be a very powerful form of communication, requiring significantly less data to convey
far more information. In addition, talking about constraints instead of choices defers
making choices until they have to be made.
Develop multiple options, communicate constraints, and let solutions emerge.
When you have a difficult problem,
Develop a set of alternative solutions to the problem
See how well they actually work, and
Then merge the best features of the solutions or choose one of the alternatives.
Set-based development can lead to better solutions faster.
Initial situation
with problem
Possible solution with consequences …
to business drivers and constraints …
Possible solution with consequences …
to business drivers and constraints …
Possible solution with consequences …
to business drivers and constraints …
Team develops
alternatives.
Customer
selects alternative.
pg_0021
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
21
Principle #2: Amplify Learning – Do It Yourself (1/2)
1.
Take your most difficult problem and devise a way to increase feedback.
–
Increase the feedback of development teams to management by asking each team at the end of each iteration
the following questions:
–
Was the team properly staffed for this iteration?
–
Were there any needed resources that were not forthcoming?
–
How can things be changed to make things go better or faster?
–
What is getting in the way?
–
Increase the feedback of customers to development teams by holding a customer focus group at the end of
each iteration. Ask questions such as the following:
–
How well does this section solve the problem it was meant to solve?
–
How could it be improved?
–
How does this iteration affect your view of what you need?
–
What do you need to put this part of the system into production?
–
Increase the feedback of the product to the development team in the following ways:
–
Have developers write and run developer tests as they write the code.
–
Have analysts, customers, or testers write and run customer tests as the developers work on the code.
Have developers help with the customer tests if that’s what it takes to get them automated.
–
Have developers observe usability tests of each features as it nears completion, so they can see how
users react to their implementation.
–
Increase the feedback within the team in the following ways:
–
Make testers an integral part of the development team.
–
Involve operations people at the beginning of the project.
–
Establish the policy that the development team maintains the product.
pg_0022
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
22
Principle #2: Amplify Learning – Do It Yourself (2/2)
2.
Start iterations with a negotiation session between customers and developers.
Customers should indicate which features are the highest priority, and developers
should select and commit to only those features from the top of the priority list which
they can realistically expect to complete in the iteration time-box.
3.
Post a progress chart for your current project in a common area so the team can see
what needs to be done and everyone can see how the project is converging.
4.
If you divide a system across multiple teams, make every effort to have a divisible
architecture that allows teams to work on their own areas as independently as possible.
Find ways for multiple teams to synchronize as often as possible by integrating their
code and running automated tests.
5.
If strata teams work for machine interfaces, consider them for user interfaces also. If you
have several teams working on different components of a system, consider forming
strata teams focused on user interfaces that cross components.
6.
Find your toughest outstanding development problem and have the development team
come up with three options on how to solve it. Instead of choosing one of the solutions,
have the team explore all three options at the same time.
pg_0023
IBM Global Services
IT Architect and IT Specialist Institute Central Region 2004 in Herrenberg
© 2004 IBM Corporation
23
Principle #3: Decide as Late as Possible
… does not mean to procrastinate, but to keep your options open as long as practical, but
no longer.
Establishing requirements before development begins is commonly thought of as a way to
protect against serious errors. The problem with sequential development is that it forces
designers to take a depth-first rather than a breadth-first approach.
The easiest way to make big mistakes is to drill down to detail too fast.
When big mistakes may be made, it is best to survey the landscape and delay the
detailed decisions.
With concurrent development, you start programming the highest value features as
soon as a high-level conceptual design is determined, even while detailed requirements
are being investigated. That exploratory approach permits you to learn by trying a variety
of options before you lock in on a direction that constrains implementation of less
important features.
Concurrent development requires developers with enough expertise in the domain to
anticipate where the emerging design is likely to lead and close collaboration with the
customers and analysts who are designing how the system will solve the business
problem at hand.