On Living, Leading, (Re)designing, and Coaching: Software Quality

One of the most common "diseases" I witness in leaders, teams, and organizations is failure to act in a centered and grounded way. When we fail to ground our actions in reality, we undermine our chances of being successful and increase our chances of getting stuck.

Here's a common example of not grounding action in reality. Several years ago, I attended a project review for a team of about ten people who were in the middle or toward the end - we couldn't tell - of a project originally estimated to take nine months. I was asked to give an assessment of what was going wrong and make some recommendations to get the project on track because the project manager had, for the third month in a row, slipped the schedule by one month.

During the review, the project manager presented a list of all the tasks that had been completed since the last review and a newly revised schedule and completion date. There was some discussion of how some tasks were completed and a discussion of the potential negative impact of the slipped schedule. Based on this, the project manager was asked to do some rescheduling, shift people and task assignments around and pull the date in by two weeks. However, no one talked about where the project actually was that day. The decisions that were made in this review weren't centered and grounded in what was actually happening. Instead, they were a response to what had happened, what might happen, and the fear of the impact of what might happen.

After the meeting was over I pulled the project manager aside to ask him some questions. I started by observing, "You presented what happened in the past month and a prediction for what might happen in the months to come. Thank you. That was really helpful context. And it was good to get an update on when you believe the project will finish. However, I still don't know where the project is today and I can't make any sound recommendations." He looked at me with some surprise. "What do you mean? We're 75 days away from completing the project," he said. "Well, last month you where about 75 days away from completing the project too and clearly a lot of work has been done in the last month. You probably aren't in the same place today as you were a month ago. So where are you today?"

What was happening was that he was inferring where the project is today by looking at what's happened in the past and what he predicted will happen in the future. Grounding action in reality means knowing where the project is today and responding to that, not responding to predictions. He was making a common mistake of doing planning only for prediction.

A better use of planning is to create a language for identifying where the project is today. Discussing where the project is today grounds action in reality and enables successful management of the project to completion. Planning that creates a language to identify where the project is today is different than planning that only creates a prediction for completion. Before we example how planning creates a language that helps us coordinate action and create value, let's look at the structure and content of a good project review.

Project reviews are structured conversations in which managing happens. Since managing happens at multiple levels there, multiple levels of review are needed. In general, each higher level of review will have decreasing depth of visibility with increasing breadth of managerial scope. However, any review can reach any level of depth in the service of achieving its purpose. For example, reviews by more senior management will tend toward reviewing exceptions and quality and process effectiveness rather than status about every task on every project.

The purposes of all project reviews, regardless of level, are:

To assess where we are today
To make decisions about what to do next
To create commitments that create boundaries of responsibility
To learn and improve our effectiveness in meeting commitments

This is accomplished by increasing the visibility of commitments, assessments, breakdowns, and resolutions and enabling decision-making. Regardless of level (depth of content discussed), project reviews follow the same basic format.

1. Assess the current state of projects against the plan and process of record

What has happened since the last review? What is done?
What is happening now? What is in-process?
Are we on plan? Are we on process?
What is our prediction about future milestones?

2. Identify project and process breakdowns (openings for new commitments and actions)

Where are we not on plan?
Where are we not on process?
What commitments have we failed to meet?
Where are we stuck? Where are we not taking needed action?

3. Learning from our breakdowns (where we are failing to meeting our commitments)

What was our commitment (including conditions for satisfaction)? Is it still relevant and needed?
What was our strategy (plan or process) for meeting it?
What action did we take?
What happened when we did that? How do we know that that's what happened?
Do we have all the necessary perspectives on what happened? If not, how can we get them?
What did we assume that turned our to be false (if anything)?
What didn't we account for in our plan or process? Where are our blind spots?
How do our actions and/or strategy need to change, based on what we learned from this breakdown?

4. Taking (new) action

What new commitments are we establishing? What are the conditions of satisfaction? Who are the customer's of the committed action? Who are the performers of the committed action?
What has to happen immediately, if anything? (What fires need to be put out?)
What new action must take place? What is the new and revised plan?
Who is responsible for what?
How will the change in plans be communicated? To whom?
What do we expect to happen by the next review?

Based on this structure and content of a project review, we can see how central commitment negotiation is to project planning and tracking.

Best wishes on the success of your next project review.

Take care,

-Steve

This is the first in a series of postings I'm calling "Case Studies". Each posting will present a particular project of which I have personal knowledge. These cases are a study in the application of an integral-ontological design approach which is intended to support personal development and social evolution (see my posts on the purpose of business and beyond "knowledge management") within an organization and/or its network or organizations.

My interest in presenting this is to generate discussion about what was done that could not only help readers but also to help me get more deeply in touch with what was done and, naturally, what can be improved upon next time.

Background:

This project was performed in a mid-sized publicly traded software company of ~1400 employees on a "big" project to produce the next major revision to the company's primary product. The code name for the project was Puma. The design team for this re-invention of practices included a cross section of key players from marketing, engineering, release management, program management, and testing. The staffing for the Puma project numbered ~120 engineers spanning ~9 development sites dotted around the globe from the US, Canada, Europe, and India.

I was the visionary and lead designer for this project.

Presenting problem:

The software company is struggling with standard requirements management practices that include the following:

Marketing talked to customers and prospective customers about what they need from the software product. These needs are resolved into product requirements which are described in a Product Requirements Document (PRD). Requirements are grouped (prioritized) into "must have" and "nice to haves".
Engineering receives the PRD from Marketing and allocates software engineers to implement them. A plan is generated by the software engineers to fulfill the "must have" requirements. Commitment to fulfill the "nice to have" requirements is deferred until enough "must haves" are completed and there is an accurate sense of how much time and effort remains before release to implement "nice to have" requirements.
Testing receives the developed product intended to fulfill the requirements. In turn, they test the product to ensure that the requirements are indeed fulfilled. If they aren't, a bug report is filed and resolved by engineering.
The product ships when scheduled unless, in the assessment of the key marketing and engineering managers, the product is sufficiently buggy. In such a case, the release is deferred for anywhere from a few days to a few months while the bugs are fixed.

In spite of the use of standard requirements practices, the software company continues to disappoint customers with the wrong features or features that are poorly designed and implemented. Customers aren't 100% dissatisfied. Indeed, the product is considered largely successful. However, customers have learned to not buy major releases until at least one "point release" is shipped because the company has a history of not quite getting things right the first time. Either the feature sets are incomplete, too complex to use, or too riddled with bugs. And customers have learned to protect their interests by waiting for new features to mature. This delay in value generation makes managing the business difficult for executives.

In addition, there are frequent disagreements between marketing, project management, engineering, and testing as to what must be built. And the requirements descriptions often don't resolve this disagreement and often are the very source of this disagreement.

Note: In my 16 years of experience in the software industry, this company's situation is very typical - typical of probably 80-90% of all software companies that I've seen.

Breakdowns that provoked re-invention of requirements management:

It is not clear what is required and what isn’t required (even when we have a description of the requirement). There are different interpretations of the same requirement and no way to resolve the differences leaving the answer to the question "what is required" unresolvable in many cases.
It is not clear how uncertainty/ambiguity in requirements is resolved; often resolution occurs late and doesn’t include the customer’s viewpoint or resolution doesn't occur at all and the feature is deferred.
It is not clear how something becomes a requirement often leaving unresolved if something is actually required or not. Often necessary changes are discovered that aren't part of any requirement in the PRD. Is it a requirement or isn't it? The current practice produces confusion rather than clarity on this point.
It is not clear how changes to requirements are made. Must they always result in a changed PRD? At what point does the PRD become obsolete and, if this occurs, what replaces it as the definition of the software?
It is not clear who owns requirements - marketing, engineering, testing, and/or the customer?
It is not clear how related requirements emerging separately from marketing and engineering are reconciled and merged.

Naturally, the company's obligation is to satisfy customer and generate business results. Current practices are generating partially satisfied customers at best and really unhappy customers in some situations at worst. In addition, these practices consumed valuable resources in generating products that didn't generate value for the customer. Therefore, the current requirements management practices were wasteful. Therefore, current practices put the company at risk for not fulfilling its obligations to its customers or stockholders.

Threat:

A requirement is pretending to be something it can never be – an unambiguous and complete description of a product or feature. Because of this, meeting our obligations are threatened unless we do something else to address the problem of generating customer satisfaction in an economical way.

The requirement is always interpreted in a background of interpretation (also called the background of obviousness) some of which is shared and some of which is not shared between those who use the requirement (e.g., marketing, engineering, project management, testing, and the customer).
The differences in the background of interpretation between different users of the requirement description generates different interpretations. Note, these are not misinterpretations but simply different interpretations.
It is impossible to fully unconceal and share the background in which the requirement is intended to be interpreted; if the background is shared this isn’t a big deal because the intended interpretation will be formed; if the background isn’t shared then this completely undermines the ability to depend upon the description of the requirement for coordinating the action necessary to satisfy the customer. This later case is typical, the former rarer.

Opportunity:

Competitors are similarly shackled by the standard interpretation of requirements management that is broken. If we can invent a new practice for generating customer satisfaction in a more economical way, we could generate a significant competitive advantage.

Diagnosis:

This is not a problem of poor requirements descriptions (for which the solution would be train people to write better requirements). No matter how much energy we put into improving descriptions, we can never escape the fact that they are still descriptions. And because they are still descriptions, they will always be interpreted differently by people in different roles, with different concerns, with different experiences, and different backgrounds of interpretation.
The non-obvious question we are left with: How do we know what to do to generate the right product and, ultimately, customer satisfaction if we can’t rely on descriptions of requirements?
We have a mess of miscoordinated backgrounds of interpretation (backgrounds of pre-understanding and obviousness). We need to respond to this as a coordination breakdown and not as a problem of ambiguous or incomplete requirements descriptions.

Recommendation:

Although we tend toward seeing this problem as poor requirements description, we always (and already) wind up dealing with this problem as a coordination breakdown. For example, when testing discovers that the product doesn't meet their interpretation of the requirement description, they open a bug report. A bug report is a request to fix the bug. The bug gets assigned to a software engineer who commits to fixing the bug. The software engineer and the tester will have a conversation discussing what they both feel is required. After some discussion and possibly involving marketing or, in some cases, a key customer, they will agree on how the product must work and the software engineer makes a commitment to make the fix. The software engineer fixes the product and the tester tests that it fulfills what they agreed to. All of this action is coordination and not the improvement of descriptions. First it is coordination of changes to the software product. Second it is coordination of the background of interpretation of the software engineer and the tester (and anyone else they bring into their conversation). Although this is, more or less, what we do to resolve the breakdown, we still think that it is a failure of the requirements description instead of having poor and unconscious coordination practices. So the answer to the "non-obvious question" above is to address such breakdowns in generating customer satisfaction through more conscious and carefully designed coordination practices and not more careful wording and review of requirement descriptions.
The first step is to re-invent what we are already and unconsciously doing - resolving breakdowns in what to build as a coordination problem - in a way that makes it conscious, legitimizes it as sound practice for generating customer satisfaction, and enables us to build competence, power, value, and competitiveness while eliminating significant economic waste generated from our attempt to resolve the problem by improving descriptions (which includes learning fancy ways to specify requirements, making descriptions longer and more verbose, review cycles, customer reviews and sign-offs, and peer reviews and sign-offs).
Therefore, the basic recommendation is to bring new language and practices for effectively coordinating action (and therefore coordinating backgrounds of interpretation) in order to significantly reduce or eliminate the wastes of producing software that doesn't generate value for the customer and satisfy the customer.

Declaring new future possibilities:

Be responsive to customers, not controlling of circumstances and people. When we shift our focus to improving coordination instead of improving descriptions, we shift into being responsive to customers and away from trying to control the circumstances and people to implement requirements. Being in conversation with customers generates more value (and less waste) than working from requirements descriptions.
Eliminate the waste of producing software that doesn't generate value and satisfy the customer.
To have a new and powerfully competitive way of generating satisfied customers through delivering software features.

Design of the new requirements management practices:

Declaration of economically important wastes: links within the chain of communication from end user (customer) to software engineer increase the complexity of the coordination necessary to produce customer satisfaction. Therefore, practices such as elaborating requirements descriptions in PRDs are wasteful. Fundamentally, effort put into improving descriptions is waste. Eliminate those steps from the process by closing the conversational distance between customer and developer and simplify the necessary coordination.
Design practices in the space of possible breakdowns: we have declared that this is not a breakdown of descriptions but a breakdown of coordination. Therefore, design and adopt practices for coordination of action and coordination of the shared background of interpretation. These are practices for managing commitments through language acts - requests, offers, promises, assertions, assessments, and declarations. These coordination acts (also called speech acts) are used in a particular way in something called a "conversation for action." And it is these conversations for action that perform the coordination of both changes to the product and coordination of backgrounds of interpretation. The conversation for action occurs between a "customer" and a "performer" and when breakdowns occur in the fulfillment of commitments, both roles engage together to bring resolution.
Build a tool that supports the new coordination practices - allowing people to manage multiple conversations for action simultaneously.
Re-interpretation of a requirement as a test – meaning a requirement is a request for the software product to pass a new test (also called conditions of satisfaction).
Collapse of customer’s, marketing’s, management’s, developer’s, and tester’s interpretations; for us this meant going from having a Product Requirement Document (owned by Marketing), a Product Specification (owned by Engineering Project Management), a Design (owned by the software engineer), a Test Specification (owned by the tester), and a Test (also owned by the tester) to having conversations for action that coordinate the passing of tests. This is commonly called test-driven development.

Mobilization of new capacities:

In order to begin mobilizing the change, we created a story about the shared future we were constructing (which is basically what you see in this case study). And this story both reframed our presenting problem, generated new insight into how we generate it, presented new possibilities for preventing and resolving the breakdown that generated an inspiring mood surrounding the change.
The story spoke more of how we needed to re-invent what we were already (unconsciously) doing - coordinating action in response to breakdowns in building the product and generating customer satisfaction - than to change to something else entirely. This is not to suggest that the re-invention wasn't a significant change to our practices. It certainly was. But this is to suggest the manner of bridging from what we currently do to the new practices.
Lastly, the story produced an unsettlement of the taken-for-granted assumptions about requirements management. This unsettlement was necessary in order to open a new space for a new set of practices. Naturally, this unsettlement generates feelings of anxiety and assessments of incompetence and mistrust. These must be faced head-on and worked through in order to establish the new practices rather than shied away from.

Accumulation of power:

Beginning with one team, the Puma team, we deployed the coordination practices and tools.
At first, the process of coordination was too complex, cumbersome, and constraining. This was quickly modified (see the learnings section for more) to simplify it and simultaneously make it more flexible.
With practice and coaching, Puma team members were able to build skill, build trust in the new practices and each other's use of them, and become inspired by the possibilities, both present and latent, in the approach.
This generated "momentum" for use of the new approach as superior to the previous approach.

Outcome of this case:

More effectively coping with uncertainty in what is required, changing requirements, and varying (sometimes widely) interpretations of how to satisfy customers.
Increased customer satisfaction through deeper customer involvement in the production of software resulting in substantially more valuable software.
Decreased cost of software design and development through elimination of waste - software that didn't generate value for the customers.
Increased employee satisfaction through smoother negotiations around different interpretations and, over time, fewer breakdowns as a result of increased shared background of interpretation.
Increased competitiveness over competitors using standard practices for managing requirements.

Learning:

In the interest of full disclosure, while the project was successful on several points I personally consider it only partially successful. Probably the largest success was the major shift in the culture from loose and unconscious coordination to disciplined and conscious coordination through making and fulfilling internal commitments. This is not a trivial shift and the positive impact of this shift was evident, particularly, as time went on. Probably the largest "failure" was that even after a year of development, the tools to support the new process were still painful to use and we were still struggling in pockets of the organization (mostly remote from the re-invention team) with people/teams who wanted to use the new practices and tools as if they were the old practices (see below).

Here are some other learnings:

You have to put the term "requirement" to rest in order for this design to take hold. We didn't do this and paid the price. The term "requirement" tends to evoke the interpretation that the description generates value because it captures the "truth" of the changes to be made to the software. It is not the description of the requirement that generates value, it is the conversation that coordinates changes to the software held between the "user" (customer) and the "software engineer" (performer). In agile project management, they have adopted the term "user story" in place of the term requirement. I like this term because it captures more accurately what is being presented by the user to the software engineer - a user's interpretation of a required usage. In turn, the software engineer is allowed to have their own story (interpretation). This focused on the generation of shared meaning through conversation and coordination as what is necessary between the user and the software engineer. If I were to do this project again, I would use a term like "user story".
The first design of the coordination process included four linked conversations for action and the linkages were "enforced" by the tool. This design was complex and users found it difficult to know where they were in the process in spite of indications in the tool itself telling them what process step they were in. The design was cumbersome in that it was difficult to quickly move a requirement through the process due to the large number of coordination acts. In addition, if a mistake was made, users could only correct the problem by going through a series of further coordination acts to force the state of the requirement back to a previous state. Lastly, the design was constraining in that a user could only coordinate between named roles coded into the four conversations for action. This design was scrapped for a simple design of one conversation for action between a customer (the responsible marketing representative) and one performer (the responsible engineering representative). Within any phase of the conversation for action, it was possible to instantiate other conversations for action related to the requirement based on the particular coordination needs of fulfilling the requirement.
We also learned that in deploying a system of this nature, it is best to start small with one team and work with just that team until all the kinks are worked out. The story that we created to mobilize the company to change was so compelling and so inspiring that there was a massive rush by nearly the whole marketing and engineering departments (over 450 people total) to the new system in its early days before the kinks were worked out. This made for a widespread painful transition to the new system as new versions were rolled out on a monthly basis to fix bugs, resolve usability problems, and address design flaws. It also taxed the re-invention team (which was 1 designer/trainer/coach and 2 implementers) to provide adequate support and to satisfy varied and often conflicting requests from different teams.
Lastly, whenever you introduce a new interpretation of a core practice as we did in this case, you must be vigilant for people re-interpreting the new practices as if they were the old practices in new garb. This effectively re-implements the current system but in a new language and a new tool. This re-interpretation of the new as if it were the old will happen to everyone unless the old interpretation is sufficiently unsettled within them. And this requires a personal coaching styled approach to change management. If you unsettle the whole company at the same time, the collective anxiety generated will overwhelm the efforts of those responsible for the re-invention. Imagine being attacked by a swarm of bees and you'll get the picture. A single bee - even two or three - can be dealt with. However, when you have hundreds or thousands of bees, your capacity to respond is easily overwhelmed.

Postscript 1: An historical perspective on the challenges of requirements management

In their 1979 book Structured Systems Analysis, authors Chris Gane and Trish Sarson, articulate five enduring challenges that developers of software systems face when articulating what to build with their customers. These challenges are as fresh today as they were then.

The designer finds it hard to learn enough about the customer’s needs to understand what specific software to build to generate value for the customer. What makes this especially hard is that designers don’t know what they haven't been told about the customer’s needs and wants.
The customers don’t know enough about what is possible for the designer to build. Further, there is no way to accurately model a functioning design before building it as there is, for instance, in the building construction industry.
Designers must collect as much information as they can about the customer’s needs but quickly become overwhelmed with too many details of the customer’s technical and business processes and environment because they don't yet know which details are important.
It is difficult to establish one contract that equally addresses the needs of the designer to understand what to build and the needs of the customer to understand what will be delivered. Contracts that meet the designers needs are typically too technical and detailed to be useful for customers. Likewise, contracts that meet the customers needs are typically too abstract to be useful for designers. Only when the system has been built do both the designer and the customer have something to react to and discuss and by then it is too late.
There are multiple design perspectives – interface, process, data, logical, physical, etc. – and the development of any one design perspective for use in negotiating with customers often places premature constraints on developing other design perspectives which can result in an inferior design and product.

These challenges are as relevant to today’s software developers as they were to Gane and Sarson’s readers in 1979. The reason for their enduring relevance is, as Gane and Sarson articulate, these challenges “arise in any situation where people of different backgrounds, with different views of the world and different vocabularies, have to work together.” For this reason, these challenges will remain relevant far into the future.

The world of the developer and the world of their customers are different because they come from different backgrounds. And this difference results in miscommunication, misunderstandings, and mistakes. So developers often build software that doesn’t fully meet the customer’s needs and therefore isn’t as valuable to customers as it could have been. This frustrates both developers and customers, keeps return on investments in software development relatively low, and leaves unresolved significant customer problems.

Gane and Sarson introduced the method of structured systems analysis, which includes a data and flow modeling language, to address these challenges. Their approach to analysis was a good attempt to generate a new language of design which would address the enduring design challenges they recognized by enabling the designers and customers of software systems to bridge between their worlds. And their approach generated some important successes.

I see Gane and Sarson as very wise practitioners with good intuitions about the centrally important challenges of software development. With the insight that these challenges arise when people with different backgrounds, different views, and different vocabularies have to work together. However, in my view, they proposed a solution founded on a partial understanding of background (listening) and vocabulary (language). And this partial understanding led them down the path of inventing new means for describing software requirements and designs rather than new means for coordinating backgrounds of interpretation. Others have followed in their footsteps - OOD, UML, etc. And these are certainly improvements upon Gane and Sarson, however, as I pointed to earlier they are still descriptions and will suffer the same breakdowns as previous attempts at description.

Postscript 2: An integral take on this project

From an integral perspective (see what is integral?), there were practices in each quadrant that contributed to continuing the standard tradition of requirements management. So this project required building a new observer (actor) in all four quadrants simultaneously through bringing new linguistic distinctions and new practices.

Upper Left Quadrant - Cultivation of new moods in the face of uncertainty and ambiguity that predispose actors to enter conversations rather than work (and re-work) descriptions; learning to make new assessments of progress (e.g., how conversations for action are progressing rather than how descriptions of requirements are progressing).

Upper Right Quadrant - Embodiment of new practices for coordination, new practices for resolving breakdowns in coordination.

Lower Left Quadrant - Design and adoption of new social practices for building shared meaning through building shared backgrounds of interpretation; new sources of trust were invented; new forms of waste were declared. Through these practices the culture was shifted so that it supported a new way of addressing shared concerns.

Lower Right Quadrant - New social systems and processes were invented and encoded into tools that support the use of the process. Roles and responsibilities were shifted to support the new process.

Developmentally, this project was a shift from organizing at what I call the Process level (in my model of development) to the Knowledge level, or, in the lingo of Spiral Dynamics was a shift from ORANGE to GREEN. The developmental shift required the interpretation that meaning isn't held in the words (requirements descriptions) but instead in how the words are interpreted by a person. And further, that we have different ways of making these interpretations resulting in different meanings of the words. This is one of the fundamental insights of postmodern levels of development (starting with Green).

Given that all developmental levels are always present in any organization, it was necessary to find ways to align the actions stemming from each level, especially RED, BLUE, ORANGE, and GREEN, which are the most prevalent in such a setting.

RED was free to charge into uncertainty and ambiguity with a sense of ownership and without the need to control others.

BLUE was grounded in declaring the customer's voice as being the source of "true requirements" as opposed to the document.

ORANGE was engaged in being responsive to the customer instead of continuously improving technical descriptions.

GREEN was able to add sensitivity to the emotional, relational, and autonomy dimensions of the conversations between customer and performers without getting snagged in consensus decision-making because within the conversation for action, accountability for fulfilling on certain commitments is clearly delineated.

Don Beck (of Spiral Dynamics fame) talks about gathering support for a new way of being across the spiral as creating a meshworks - an integral design that gathers support for a new way from all quadrants, all levels, all lines, and all states.

Lastly, I must thank Chauncey Bell and Guillermo Wechsler (of BABDI) who I hired as consultants on this project. They brought with them an extensive and deep new interpretation of (ontological) design that they learned while in partnership with Fernando Flores. These two had a significant impact on the re-invention and, in many ways, taught us all how to look at our breakdowns in new and powerful ways. Those of you familiar with their work will no doubt see their influence throughout.

Hope you enjoyed this piece. It was lengthy - my longest post to date. But I felt this was better as a single post. I'd love to hear your comments and questions.

Take care,

-Steve