This is the first in a series of postings I'm calling "Case Studies". Each posting will present a particular project of which I have personal knowledge. These cases are a study in the application of an integral-ontological design approach which is intended to support personal development and social evolution (see my posts on the purpose of business and beyond "knowledge management") within an organization and/or its network or organizations.
My interest in presenting this is to generate discussion about what was done that could not only help readers but also to help me get more deeply in touch with what was done and, naturally, what can be improved upon next time.
Background:
This project was performed in a mid-sized publicly traded software company of ~1400 employees on a "big" project to produce the next major revision to the company's primary product. The code name for the project was Puma. The design team for this re-invention of practices included a cross section of key players from marketing, engineering, release management, program management, and testing. The staffing for the Puma project numbered ~120 engineers spanning ~9 development sites dotted around the globe from the US, Canada, Europe, and India.
I was the visionary and lead designer for this project.
Presenting problem:
The software company is struggling with standard requirements management practices that include the following:
- Marketing talked to customers and prospective customers about
what they need from the software product. These needs are resolved into
product requirements which are described in a Product Requirements
Document (PRD). Requirements are grouped (prioritized) into "must have" and "nice to
haves".
- Engineering receives the PRD from Marketing and
allocates software engineers to implement them. A plan is generated by
the software engineers to fulfill the "must have" requirements.
Commitment to fulfill the "nice to have" requirements is deferred until
enough "must haves" are completed and there is an accurate sense of how
much time and effort remains before release to implement "nice to have" requirements.
- Testing
receives the developed product intended to fulfill the requirements. In turn,
they test the product to ensure that the requirements are indeed
fulfilled. If they aren't, a bug report is filed and resolved by engineering.
- The product ships when scheduled unless, in the assessment of the key marketing and engineering managers, the product is sufficiently buggy. In such a case, the release is deferred for anywhere from a few days to a few months while the bugs are fixed.
In spite of the use of standard requirements practices, the software
company continues to disappoint customers with the wrong features or
features that are poorly designed and implemented. Customers aren't 100% dissatisfied. Indeed, the product is considered largely successful. However, customers have learned to not buy major releases until at least one "point release" is shipped because the company has a history of not quite getting things right the first time. Either the feature sets are incomplete, too complex to use, or too riddled with bugs. And customers have learned to protect their interests by waiting for new features to mature. This delay in value generation makes managing the business difficult for executives.
In addition, there
are frequent disagreements between marketing, project management,
engineering, and testing as to what must be built. And the requirements descriptions often don't resolve this disagreement and often are the very source of this disagreement.
Note: In my 16 years of experience in the software industry, this company's situation is very typical - typical of probably 80-90% of all software companies that I've seen.
Breakdowns that provoked re-invention of requirements management:
- It is not clear what is required and what isn’t required (even
when we have a description of the requirement). There are different
interpretations of the same requirement and no way to resolve the
differences leaving the answer to the question "what is required"
unresolvable in many cases.
- It is not clear how
uncertainty/ambiguity in requirements is resolved; often resolution
occurs late and doesn’t include the customer’s viewpoint or resolution doesn't occur at all and the feature is deferred.
- It is
not clear how something becomes a requirement often leaving unresolved
if something is actually required or not. Often necessary changes are
discovered that aren't part of any requirement in the PRD. Is it a
requirement or isn't it? The current practice produces confusion rather
than clarity on this point.
- It is not clear how changes to
requirements are made. Must they always result in a changed PRD? At
what point does the PRD become obsolete and, if this occurs, what
replaces it as the definition of the software?
- It is not clear who owns requirements - marketing, engineering, testing, and/or the customer?
- It is not clear how related requirements emerging separately from marketing and engineering are reconciled and merged.
Naturally, the company's obligation is to satisfy customer and generate
business results. Current practices are generating partially satisfied
customers at best and really unhappy customers in some situations at worst. In
addition, these practices consumed valuable resources in generating
products that didn't generate value for the customer. Therefore, the
current requirements management practices were wasteful. Therefore, current practices put the company at risk for not fulfilling its obligations to its customers or stockholders.
Threat:
A requirement is pretending to be something it can never be – an
unambiguous and complete description of a product or feature. Because
of this, meeting our obligations are threatened unless we do something
else to address the problem of generating customer satisfaction in an
economical way.
- The requirement is always interpreted in a background of
interpretation (also called the background of obviousness) some of which is shared and some of which is not shared
between those who use the requirement (e.g., marketing, engineering,
project management, testing, and the customer).
- The differences in the background of interpretation between different users of the requirement description generates different interpretations. Note, these are not misinterpretations but simply different interpretations.
- It is
impossible to fully unconceal and share the background in which the
requirement is intended to be interpreted; if the background is shared
this isn’t a big deal because the intended interpretation will be
formed; if the background isn’t shared then this completely undermines
the ability to depend upon the description of the requirement for coordinating the action necessary to satisfy the customer. This
later case is typical, the former rarer.
Opportunity:
Competitors are similarly shackled by the standard interpretation of
requirements management that is broken. If we can invent a new practice
for generating customer satisfaction in a more economical way, we could generate
a significant competitive advantage.
Diagnosis:
- This is not a problem of poor requirements descriptions (for
which the solution would be train people to write better requirements).
No matter how much energy we put into improving descriptions, we can
never escape the fact that they are still descriptions. And because
they are still descriptions, they will always be interpreted
differently by people in different roles, with different concerns, with
different experiences, and different backgrounds of interpretation.
- The
non-obvious question we are left with: How do we know what to do to
generate the right product and, ultimately, customer satisfaction if we
can’t rely on descriptions of requirements?
- We have a mess of miscoordinated backgrounds of interpretation (backgrounds of pre-understanding and obviousness). We
need to respond to this as a coordination breakdown and not as a
problem of ambiguous or incomplete requirements descriptions.
Recommendation:
Although we tend toward seeing this problem as poor
requirements description, we always (and already) wind up dealing with
this problem as a coordination breakdown. For example, when testing
discovers that the product doesn't meet their interpretation of the
requirement description, they open a bug report. A bug report is a
request to fix the bug. The bug gets assigned to a software engineer
who commits to fixing the bug. The software engineer and the tester
will have a conversation discussing what they both feel is required.
After some discussion and possibly involving marketing or, in some
cases, a key customer, they will agree on how the product must work and
the software engineer makes a commitment to make the fix. The software
engineer fixes the product and the tester tests that it fulfills what
they agreed to. All of this action is coordination and not the improvement of descriptions.
First it is coordination of changes to the software product. Second it
is coordination of the background of interpretation of the software
engineer and the tester (and anyone else they bring into their conversation).
Although this is, more or less, what we do to resolve the breakdown, we
still think that it is a failure of the requirements description
instead of having poor and unconscious coordination practices. So the
answer to the "non-obvious question" above is to address such
breakdowns in generating customer satisfaction through more conscious
and carefully designed coordination practices and not more careful
wording and review of requirement descriptions.
- The first step
is to re-invent what we are already and unconsciously doing - resolving
breakdowns in what to build as a coordination problem - in a way that
makes it conscious, legitimizes it as sound practice for generating
customer satisfaction, and enables us to build competence, power,
value, and competitiveness while eliminating significant economic waste
generated from our attempt to resolve the problem by improving
descriptions (which includes learning fancy ways to specify
requirements, making descriptions longer and more verbose, review
cycles, customer reviews and sign-offs, and peer reviews and sign-offs).
- Therefore,
the basic recommendation is to bring new language and practices for effectively coordinating action
(and therefore coordinating backgrounds of interpretation) in order to
significantly reduce or eliminate the wastes of producing software that
doesn't generate value for the customer and satisfy the customer.
Declaring new future possibilities:
- Be responsive to customers, not controlling of circumstances and people. When we shift our focus to improving coordination instead of improving descriptions, we shift into being responsive to customers and away from trying to control the circumstances and people to implement requirements. Being in conversation with customers generates more value (and less waste) than working from requirements descriptions.
- Eliminate the waste of producing software that doesn't generate value and satisfy the customer.
- To have a new and powerfully competitive way of generating satisfied customers through delivering software features.
Design of the new requirements management practices:
- Declaration of economically important wastes: links within the
chain of communication from end user (customer) to software engineer
increase the complexity of the coordination necessary to produce
customer satisfaction. Therefore, practices such as elaborating
requirements descriptions in PRDs are wasteful. Fundamentally, effort put
into improving descriptions is waste. Eliminate those steps from the
process by closing the conversational distance between customer and
developer and simplify the necessary coordination.
- Design practices in the space of possible
breakdowns: we have declared that this is not a breakdown of
descriptions but a breakdown of coordination. Therefore, design and
adopt practices for coordination of action and coordination of the
shared background of interpretation. These are practices for managing
commitments through language acts - requests, offers, promises,
assertions, assessments, and declarations. These coordination acts (also called speech acts) are used in a particular way in something called a "conversation for action." And it is these conversations for action that perform the coordination of both changes to the product and coordination of backgrounds of interpretation. The conversation for action occurs between a "customer" and a "performer" and when breakdowns occur in the fulfillment of commitments, both roles engage together to bring resolution.
- Build a tool that
supports the new coordination practices - allowing people to manage
multiple conversations for action simultaneously.
- Re-interpretation of a requirement as a test – meaning a requirement is a request for the software product to pass a new test (also called conditions of satisfaction).
- Collapse
of customer’s, marketing’s, management’s, developer’s, and tester’s
interpretations; for us this meant going from having a Product
Requirement Document (owned by Marketing), a Product Specification
(owned by Engineering Project Management), a Design (owned by the
software engineer), a Test Specification (owned by the tester), and a
Test (also owned by the tester) to having conversations for action that
coordinate the passing of tests. This is commonly called test-driven
development.
Mobilization of new capacities:
- In order to begin mobilizing the change, we created a story
about the shared future we were constructing (which is basically what
you see in this case study). And this story both reframed our
presenting problem, generated new insight into how we generate it,
presented new possibilities for preventing and resolving the breakdown
that generated an inspiring mood surrounding the change.
- The
story spoke more of how we needed to re-invent what we were already
(unconsciously) doing - coordinating action in response to breakdowns in building the
product and generating customer satisfaction - than to change to
something else entirely. This is not to suggest that the re-invention wasn't a significant change to our practices. It certainly was. But this is to suggest the manner of bridging from what we currently do to the new practices.
- Lastly, the story produced an
unsettlement of the taken-for-granted assumptions about requirements
management. This unsettlement was necessary in order to open a new
space for a new set of practices. Naturally, this unsettlement
generates feelings of anxiety and assessments of incompetence and
mistrust. These must be faced head-on and worked through in order to establish the new practices rather than shied away from.
Accumulation of power:
- Beginning with one team, the Puma team, we deployed the coordination practices and tools.
- At
first, the process of coordination was too complex, cumbersome, and
constraining. This was quickly modified (see the learnings section for
more) to simplify it and simultaneously make it more flexible.
- With
practice and coaching, Puma team members were able to build skill,
build trust in the new practices and each other's use of them, and
become inspired by the possibilities, both present and latent, in the
approach.
- This generated "momentum" for use of the new approach as superior to the previous approach.
Outcome of this case:
- More effectively coping with uncertainty in what is required,
changing requirements, and varying (sometimes widely) interpretations
of how to satisfy customers.
- Increased customer satisfaction
through deeper customer involvement in the production of software
resulting in substantially more valuable software.
- Decreased
cost of software design and development through elimination of waste -
software that didn't generate value for the customers.
- Increased
employee satisfaction through smoother negotiations around different
interpretations and, over time, fewer breakdowns as a result of
increased shared background of interpretation.
- Increased competitiveness over competitors using standard practices for managing requirements.
Learning:
In the interest of full disclosure, while the project was successful on several points I personally consider it only partially successful. Probably the largest success was the major shift in the culture from loose and unconscious coordination to disciplined and conscious coordination through making and fulfilling internal commitments. This is not a trivial shift and the positive impact of this shift was evident, particularly, as time went on. Probably the largest "failure" was that even after a year of development, the tools to support the new process were still painful to use and we were still struggling in pockets of the organization (mostly remote from the re-invention team) with people/teams who wanted to use the new practices and tools as if they were the old practices (see below).
Here are some other learnings:
- You have to put the term "requirement" to rest in order for
this design to take hold. We didn't do this and paid the price. The term "requirement" tends to evoke the
interpretation that the description generates value because it captures
the "truth" of the changes to be made to the software. It is not the
description of the requirement that generates value, it is the
conversation that coordinates changes to the software held between the
"user" (customer) and the "software engineer" (performer). In agile project management,
they have adopted the term "user story" in place of the term
requirement. I like this term because it captures more accurately what
is being presented by the user to the software engineer - a user's
interpretation of a required usage. In turn, the software engineer is
allowed to have their own story (interpretation). This focused on the
generation of shared meaning through conversation and coordination as
what is necessary between the user and the software engineer. If I were to do this project again, I would use a term like "user story".
- The
first design of the coordination process included four linked
conversations for action and the linkages were "enforced" by the tool. This design was complex and users found it
difficult to know where they were in the process in spite of
indications in the tool itself telling them what process step they were
in. The design was cumbersome in that it was difficult to quickly move
a requirement through the process due to the large number of
coordination acts. In addition, if a mistake was made, users could only
correct the problem by going through a series of further coordination
acts to force the state of the requirement back to a previous state.
Lastly, the design was constraining in that a user could only
coordinate between named roles coded into the four conversations for
action. This design was scrapped for a simple design of one
conversation for action between a customer (the responsible marketing
representative) and one performer (the responsible engineering
representative). Within any phase of the conversation for action, it
was possible to instantiate other conversations for action related to
the requirement based on the particular coordination needs of
fulfilling the requirement.
- We also learned that in deploying a
system of this nature, it is best to start small with one team and work
with just that team until all the kinks are worked out. The story that
we created to mobilize the company to change was so compelling and so
inspiring that there was a massive rush by nearly the whole marketing and engineering departments (over 450 people total) to
the new system in its early days before the kinks were worked out. This
made for a widespread painful transition to the new system as new
versions were rolled out on a monthly basis to fix bugs, resolve
usability problems, and address design flaws. It also taxed the re-invention team (which was 1 designer/trainer/coach and 2 implementers) to provide adequate support and to satisfy varied and often conflicting requests from different teams.
- Lastly, whenever
you introduce a new interpretation of a core practice as we did in this
case, you must be vigilant for people re-interpreting the new practices
as if they were the old practices in new garb. This effectively
re-implements the current system but in a new language and a new tool.
This re-interpretation of the new as if it were the old will happen to
everyone unless the old interpretation is sufficiently unsettled within them. And this requires a personal coaching styled approach to change management. If you unsettle the whole company at the same time, the collective anxiety generated will overwhelm the efforts of those responsible for the re-invention. Imagine being attacked by a swarm of bees and you'll get the picture. A single bee - even two or three - can be dealt with. However, when you have hundreds or thousands of bees, your capacity to respond is easily overwhelmed.
Postscript 1: An historical perspective on the challenges of requirements management
In their 1979 book Structured Systems Analysis, authors Chris Gane and Trish Sarson, articulate five enduring challenges that developers of software systems face when articulating what to build with their customers. These challenges are as fresh today as they were then.
- The designer finds it hard to learn enough about the customer’s needs to understand what specific software to build to generate value for the customer. What makes this especially hard is that designers don’t know what they haven't been told about the customer’s needs and wants.
- The customers don’t know enough about what is possible for the designer to build. Further, there is no way to accurately model a functioning design before building it as there is, for instance, in the building construction industry.
- Designers must collect as much information as they can about the customer’s needs but quickly become overwhelmed with too many details of the customer’s technical and business processes and environment because they don't yet know which details are important.
- It is difficult to establish one contract that equally addresses the needs of the designer to understand what to build and the needs of the customer to understand what will be delivered. Contracts that meet the designers needs are typically too technical and detailed to be useful for customers. Likewise, contracts that meet the customers needs are typically too abstract to be useful for designers. Only when the system has been built do both the designer and the customer have something to react to and discuss and by then it is too late.
- There are multiple design perspectives – interface, process, data, logical, physical, etc. – and the development of any one design perspective for use in negotiating with customers often places premature constraints on developing other design perspectives which can result in an inferior design and product.
These challenges are as relevant to today’s software developers as they were to Gane and Sarson’s readers in 1979. The reason for their enduring relevance is, as Gane and Sarson articulate, these challenges “arise in any situation where people of different backgrounds, with different views of the world and different vocabularies, have to work together.” For this reason, these challenges will remain relevant far into the future.
The world of the developer and the world of their customers are different because they come from different backgrounds. And this difference results in miscommunication, misunderstandings, and mistakes. So developers often build software that doesn’t fully meet the customer’s needs and therefore isn’t as valuable to customers as it could have been. This frustrates both developers and customers, keeps return on investments in software development relatively low, and leaves unresolved significant customer problems.
Gane and Sarson introduced the method of structured systems analysis, which includes a data and flow modeling language, to address these challenges. Their approach to analysis was a good attempt to generate a new language of design which would address the enduring design challenges they recognized by enabling the designers and customers of software systems to bridge between their worlds. And their approach generated some important successes.
I see Gane and Sarson as very wise practitioners with good intuitions about the centrally important challenges of software development. With the insight that these challenges arise when people with different backgrounds, different views, and different vocabularies have to work together. However, in my view, they proposed a solution founded on a partial understanding of background (listening) and vocabulary (language). And this partial understanding led them down the path of inventing new means for describing software requirements and designs rather than new means for coordinating backgrounds of interpretation. Others have followed in their footsteps - OOD, UML, etc. And these are certainly improvements upon Gane and Sarson, however, as I pointed to earlier they are still descriptions and will suffer the same breakdowns as previous attempts at description.
Postscript 2: An integral take on this project
From an integral perspective (see what is integral?), there were practices in each quadrant that contributed to continuing the standard tradition of requirements management. So this project required building a new observer (actor) in all four quadrants simultaneously through bringing new linguistic distinctions and new practices.
Upper Left Quadrant - Cultivation of new moods in the face of uncertainty and ambiguity that predispose actors to enter conversations rather than work (and re-work) descriptions; learning to make new assessments of progress (e.g., how conversations for action are progressing rather than how descriptions of requirements are progressing).
Upper Right Quadrant - Embodiment of new practices for coordination, new practices for resolving breakdowns in coordination.
Lower Left Quadrant - Design and adoption of new social practices for building shared meaning through building shared backgrounds of interpretation; new sources of trust were invented; new forms of waste were declared. Through these practices the culture was shifted so that it supported a new way of addressing shared concerns.
Lower Right Quadrant - New social systems and processes were invented and encoded into tools that support the use of the process. Roles and responsibilities were shifted to support the new process.
Developmentally, this project was a shift from organizing at what I call the Process level (in my model of development) to the Knowledge level, or, in the lingo of Spiral Dynamics was a shift from ORANGE to GREEN. The developmental shift required the interpretation that meaning isn't held in the words (requirements descriptions) but instead in how the words are interpreted by a person. And further, that we have different ways of making these interpretations resulting in different meanings of the words. This is one of the fundamental insights of postmodern levels of development (starting with Green).
Given that all developmental levels are always present in any organization, it was necessary to find ways to align the actions stemming from each level, especially RED, BLUE, ORANGE, and GREEN, which are the most prevalent in such a setting.
RED was free to charge into uncertainty and ambiguity with a sense of ownership and without the need to control others.
BLUE was grounded in declaring the customer's voice as being the source of "true requirements" as opposed to the document.
ORANGE was engaged in being responsive to the customer instead of continuously improving technical descriptions.
GREEN was able to add sensitivity to the emotional, relational, and autonomy dimensions of the conversations between customer and performers without getting snagged in consensus decision-making because within the conversation for action, accountability for fulfilling on certain commitments is clearly delineated.
Don Beck (of Spiral Dynamics fame) talks about gathering support for a new way of being across the spiral as creating a meshworks - an integral design that gathers support for a new way from all quadrants, all levels, all lines, and all states.
Lastly, I must thank Chauncey Bell and Guillermo Wechsler (of BABDI) who I hired as consultants on this project. They brought with them an extensive and deep new interpretation of (ontological) design that they learned while in partnership with Fernando Flores. These two had a significant impact on the re-invention and, in many ways, taught us all how to look at our breakdowns in new and powerful ways. Those of you familiar with their work will no doubt see their influence throughout.
Hope you enjoyed this piece. It was lengthy - my longest post to date. But I felt this was better as a single post. I'd love to hear your comments and questions.
Take care,
-Steve
Recent Comments