“Governance” is defined as the means by which the leading authority guides and monitors the values and goals of an organization.
Because this definition uses some very broad terms such as “leading authority” and “organization” it is possible to apply the concept of governance at many levels in IT organizations. There is governance in teams, projects, programmes, portfolios, departments and any other organizational unit. Implemented correctly governance is an opportunity to apply systems thinking to an organization. Implemented badly it adds waste and confusion throughout a business.
Governance approaches are most effective when they are designed and explicit. In the absence of cohesive integrated governance between these various organizational layers the values and goals of an organization often become emergent due to the lack of a clear connection and feedback mechanisms between the executive vision and strategy with the work being done by delivery teams. This situation is typified by project practitioners being unaware of how their work relates and contributes to the top level strategy, obfuscated by layers in between.
The “means” referred to in the definition above refer to practices, processes, procedures, standards, rules and criteria related to the execution of business decisions executed by real people within their tool environments.
Effective governance relies on shared and understood mechanisms for strategic management, programme and portfolio management as well as project management and execution/delivery. Critically these mechanisms need to be joined up.
Cohesive joined up governance enables individuals in an organization to make the right decisions at the right time based on accurate data. Governance is often described as a mechanism for empowering individuals with the rights they should have and a clear delineation and integration of these rights from the rights exercised by other roles. Because governance strategies can involve a lot of process management based on data analysis there is a strong case for automating parts of a governance strategy with tools to link and track related processes. Business collaboration tools or specific governance and niche portfolio management tools can all work in this space. Be wary of over-reliance on data for governance, metrics are no substitute for direct examination and simply talking to people.
There are 4+1 fundamental governance questions that governance approaches are trying to answer. 4+1 rather than 5 because the 5th isn’t always necessary in terms of explicit governance in non-regulated industries:
- How much have we spent?
- How much more are we going to spend?
- What is progress against a roadmap?
- Is everything healthy?
- Is everything legal?
The better your governance is; the less management you need.
Continuous Investment Review
An important part of the Governance function is "Continuous Investment Review". Continuous Investment Review is the process of checking work progress (projects, programmes and other delivery methods) against forecast budgets and extrapolated costs to decide if the business should continue to fund a project.
This continue, change or stop decision is a chance to apply feedback to work based on actual output, spend so far and projected spend.
Benefit (ROI) = Business Value - Cost of Implementation
Waiting until the end of a piece of work to judge whether it was worth it or not places value at the end of the lifecycle. Business Value is the most important thing, so checking it regularly can prevent organizations from making significant mistakes.
One of the most valuable decisions a software business can make is to stop a failing project.
Return on Investment (ROI) is based on a simple formula:
Return on Investment = Business Value - Cost of Implementation
Where business value is qualified in terms of the POFL tests and cost of implementation is the cost of the product (and/or maintenance agreements) as well as the internal Delivery cost of deployment and support (as well as and configuration/customization costs). HSD helps organizations determine cost of implmentation by illuminating all of the work related to a Portfolio Request.
Where business value is not defined financially (e.g. in non-commercial organizations) we end up with a simple "Business Value vs. Cost of Implementation" equation which offers a simple decision to Business Leaders: Is the benefit worth the cost? We recommend that a wide stakeholder group is used to answer that question rather than just the Business Leaders so that decisions have wide support as they are made, rather than trying to gain support afterwards, especially when those decisions relate to tools, environments or COTS selection.
In a software context a contract is a written or spoken agreement that is intended to be enforceable by law between a customer and a supplier on how a product or part of the software development process will be delivered.
Levels of Governance
Holistic Software Development examines governance at three levels: Directing, Controlling and Executing. This allows business leaders to select governance practices that meet their needs at each layer. Governance can be continuously improved with incremental practice refinement and adoption increasing or decreasing formality as necessary. A common mistake is to counter unpredictability due to high complexity with more formal governance – this doesn’t help. Instead, use light touch governance for high complexity work where you’re managing to enable quick results (positive or negative). For low complexity work, organizations may benefit from more formal governance if managing for efficiency rather than creativity.
As with all of HSD the guiding principle is keeping practice formality and complexity as light as possible. The choice of practices can also be used to derive required skills, processes and measures to support effective evidence based decision making, escalation and exception management not to mention compliance.
Using a layered governance model allows for decision making to be pushed to the appropriate level allowing local practice variability at lower levels and the swapping in and out of processes and practice, as driven by continuous improvement, strategic change (e.g. Digital Transformation) or based on the Hybrid Dynamic Model. Each layer focusses on different aspect of business management: Directing the business, Controlling a series of related work streams (integration level) and creating the business value (executing level). Continuous improvement should be functional at every level through effective feedback loops.
Since there are so many possible governance practices we recommend starting with a minimal subset and tailoring-up if there is a problem rather than over-burdening the business with too many processes and controls. The purpose of governance is to allow the business to operate effectively, not add bureaucracy to slow it down.
The "Directing" level is responsible for setting business strategy which should shape the portfolio of work. In this way a strategy can be described by a number of Strategic Goals which the top of the requirements stack can trace to as part of their business justification.
Continuous Investment Review is the feedback loop for the directing layer.
A typical dysfunction in large organizations is to surround a small development team with many managers and "process people" to the extent that there can be more "managers" than "doers". Over-resourcing governance, at any level is wasteful. Implementing more layers than are strictly necessary to support each individual piece of work is also wasteful.
Assuming a hierarchical organization then an ideal resourcing shape is as follows:
- Very few resources in the Directing layer
- Minimal resources in the Controlling layer
- The Majority of resources in the Executing layer
We recommend drawing resources from the Executing level into the Controlling level as integration and co-ordination functions are required rather than permanent staffing in the “controlling” layer. Flowing resources helps share business knowledge throughout an organization and avoids "ivory towers" forming..
Alternatives to hierarchical models such as democratic models and holocratic models are also viable however decision making can then become diffuse. An emergent decision making mechanism leads to "counter decisions" and "political issues" so we recommend that decision making is intentional, even if not hierarchical. See Evidence Based Decision Making for more information. Note that different decision making models will be required for different types of decision regardless of the overall business structure.
Governance Practice Library
There are many possible governance practices derived from many schools of thought, bodies of knowledge, processes and implicit ways of working. Organizations will need to choose the appropriate practices for their needs, which may well include organization specific practices. The following list is by no means exhaustive, nor should an organization attempt to implement all of the following unless it really needs to. As always, a practice may be implemented in many ways ranging from explicit formal implementation to implicit informal use.
The Strategic Direction practice involves articulating the Business Strategy via Business Model Canvas or similar technique. Strategic Goals within short, medium and long timeframes are then developed which are used by business leaders to shape the portfolio through the portfolio requests selected for execution. In this way development projects can trace their requirements all the way to Strategic goals in only 3 or 4 jumps.
Transparent Leadership is a practice for business leaders that involves communicating their leadership intent by publishing their annual objectives (linked to Strategic Goals) and progress against them on a regular basis. Transparent Leadership also involves leadership shadowing, promoting social business collaboration with direct communication from senior executives, regular floor walking briefings and promotion of open feedback.
The Bubble Up escalation practice provides a mechanism for critical issues to be pushed all the way to senior leadership from anyone in the organization promoting an open and honest culture.
The Go See practice provides a mechanism for uncovering the fundamental truth in terms of ways of working, efficiency, quality and systemic problems/opportunities for improvement.
Financial Management provides:
- Continuous Investment Review
- Top down and bottom up budget management
- Financial Forecasts
- Baseline management
- Actual spend capture
- Actual extrapolation
- Financial reporting
- Taxation compliance
Risk Management provides:
- A mechanism for people to escalate significant risks and issues (using Bubble Up)
- Strategic risk identification, management and mitigation
Workforce Shaping provides:
- Target and Actual Workforce shape
- Demand and supply forecasting and monitoring
- Competency and Skill Management
- Conflict resolution
Evidence Based Procurement provides:
- Services Procurement - ranging from time-hire contractors, managed services for partial out-sourcing, professional services etc.
- Software Procurement - ranging from commercial off the shelf products, open source import (and license validation), out-sourced development
- Hardware Procurement - ranging from a keyboard to a private cloud
- See Contracts for more information
Architectural Governance provides:
- Alignment of Intentional Enterprise Architecture with Strategic Goals
- Technical roadmaps for Architecture, Dependencies and Environments
- Technical Leadership
- Ensures that Release and Integration level feedback loops are effective, and include Customers
- Mentors, Coaches and supports teams in experimenting with their ways of working and improvement efforts
We do not recommend that separate teams are created for each governance practice, instead look at the most natural home for governance efforts, and push them as far towards delivery teams as makes sense. The best governance is self-managed in response to “pull” from leadership rather than imposed in a “push” model.
A Risk is any potential threat to success.
Many risks will be within a projects or programmes or other delivery team's sphere of influence and actions may be taken to remove or address these risks. Some risks will be beyond the immediate sphere of influence, these should be identified and escalated to Business Leaders via the Bubble Up process or some other escalation route.
Risks may be within the sphere of influence of the business and so mitigating action can be taken. However, some risks may not even be within the sphere influence of the business instead affected by external factors, it is still essential that these risks are identified and escalated as they may be important in influencing the strategic direction of the business.
Risk Impact is the effect of a risk actually occurring, becoming an issue. Impact is best quantified in terms of resource usage (time/money/people) and/or effect on Business Value (POFL). The risk impact and perceived probability help us asses which risks to focus our efforts on. During Portfolio Selection, and Continuous Investment Review, Risk Impact may be useful in helping to choose an option to take forward from Business Cases and inform start/stop decisions. Risk Impact will also inform Build or Buy decisions.
An Estimate is an approximate calculation or judgement of the value, number, quantity, or extent of something
- Confident: We understand the process, human and technical aspects. We have good numerical assessments based on past evidence. Although we can’t predict the future, we are confident in our estimates.
- Reasonable: We have some confidence in our analysis, but we can expect numbers to change as we learn more. We don’t expect our estimates to change significantly enough to affect start/stop decisions.
- Cautious: Although we don’t expect significant change, there is sufficient risk that our estimates could change significantly as we learn more about the work.
- Uncertain: We have little confidence in our estimates as there is high complexity, or significant unknowns. Our estimates may change significantly as we learn more.
Risk Driven Lifecycle
We recommend adopting a risk driven lifecycle, feeding back refined estimates and risk knowledge throughout the business. A risk driven lifecycle involves doing the risky things first, to understand those risks and inform continued work on the programme or project. Essentially reducing the cone of uncertainty as quickly as possible. The Standard Milestones in Holistic Software Development build in the Risk Driven Lifecycle to programmes and projects where they exist, but a risk-driven approach can be taken without using Milestones by simply attacking risky things first.
A Milestone is a significant event or decision point in the lifetime of a project.
Holistic Software Development uses a standard set of layered Milestones to drive the Risk-Driven Lifecycle. Milestones are a useful way of tracking top-level guesses about when scope will be delivered. Like estimates, they should not be treated as deadlines.
These risks may relate to requirements or technical uncertainty which are within the projects sphere of influence. Delivery teams can identify activity that can reduce or eliminate the top risks, this may involve building technical solutions to address specific technical challenges (prototyping or spiking). The result of early risk mitigation is increased understanding of overall scope, proven architecture (from spiking) and elaboration of poorly understood requirements all of which reduce uncertainty and improve estimates (where they are used).
Successful risk reduction may lead the business to commitment of more fund/resources to the project, conversely inability to reduce risk may be a signal that a different approach is required. This may even result in project cancellation or redirection, early cancellation of projects that aren't addressing their risks successfully is a positive indicator for the organization. HSD uses Continuous Investment Review to drive this analysis.
Programmes or Projects whose risk profile is static are also candidates for investigation as there may be issues around the ways of working.
Risks can be mitigated in a number of ways:
- Avoidance - the risk is eliminated by not doing activities that might cause it to occur, or it goes away by itself (this is often impractical, though most desirable, in a business context)
- Reduction - the risk impact is reduced by resolving parts of the risk (e.g. by sharing knowledge, by finding expertise, by spiking a technical problem, by demonstrating progress etc.)
- Sharing - sharing is often misrepresented as "transfer" however a business can never entirely transfer a risk (e.g. by outsourcing) as the impact of the risk occurring will still affect the business as a customer to the transferred entity. However, some types of risk impact, such as financial and reputational, can be shared.
- Retention - involves budgeting and planning for a risk to happen, to the extent of assuming it will happen.
In a software context a contract is a written or spoken agreement that is intended to be enforceable by law between a customer and a supplier on how a product or part of the software development process will be delivered.
As mentioned in the Holistic Software Development Manifesto we value customer, business, developer and operations collaboration over contract negotiation. This means that HSD encourages open collaboration between people rather than contract specification. Creating formal agreements between people can inhibit them from working freely together, creating transactional relationships and a loss of focus on Business Value.
However, contracts are oftennecessary, and are normal between organizations wishing to work together. The reasons behind writing a contract to formalize commercial relationships are to protect both parties from liability when something goes wrong and to place costs and schedules on a formal basis. As such, contracts tend to deal in negative language that focuses on potential failures and consequences, detailing limitations. By their nature, contracts are somewhat adversarial and damage an open collaborative relationship between organizations.
As a necessary evil, contracts are often managed by procurement departments, account managers and sales people rather than Business Leaders and Delivery Teams to separate the adversarial negative nature of legal agreement from the people who will actually work together on a day to day basis. Although this separates concerns, sales and procurement people may not have the necessary deep knowledge of software engineering and so will require technical support to develop meaningful contracts.
Small contractors (often called Small or Medium Enterprises, SMEs) generally won't have dedicated legal or account management staff and so will have to deal with both sides of agreement and delivery themselves - this often leads to a less adversarial relationship as people intending to work together on a regular basis tend not to start their relationships with an argument.
Software Development Contracts are used to agree:
- An entire product (or product family) delivery between a customer and a development organization (see COTS)
- Out-sourcing part of the development process (e.g. System Analysis or Testing)
- Augmenting internal delivery capability by using an out-sourced team
- Augmenting internal delivery by adding individual contractors
- Out-sourcing the entire delivery process from Portfolio downwards
Note that the first and last options are very similar with the subtle but critically important point that the end Customer is typically different. In the first option the end Customer is the person requesting a product from an organization. In the last option the person requesting the product, is part of a supplier organization and so the products they receive are for selling on to a Customer organization. This structure confuses who the real end customers are. This distinction can cause many problems as the intermediate organization typically acts as a “customer proxy” to their internal delivery teams leading to conflicts of interest and confusing otherwise simple communication lines. If supplier organizations, often embedded in customer organizations, further out-source or augment their teams with other suppliers then things get even more confusing.
Where work is too big to be done by the home organization, and so supplier partners must be used we recommend partitioning work by Business Value rather than parts of the development process. Commercial placing makes sense if specific parts of required Business Value can be packaged up in relative isolation from other parts of a solution. Even if work can be separated by Business Value, only place with commercial partners based on a valid Build or Buy decision (taking into account position on the Commoditization Scale).
Each of these types of work requires a different contract approach and so people with intimate knowledge of software development, in both breadth and depth must be involved in developing contracts. We have seen many software development contracts which unintentionally incentivize very negative commercial behaviors.
Recommended Usage of Contract Types
Different contract types will be selected based on the needs of the business. In our experience augmenting internal teams with individual contractors (on a Time Hire basis) rather than using a body shopping supplier is preferable however if scale is an issue this may not always be possible. When paying a supplier for people a customer is paying for both the people and the supplier's overheads rather than just the people. Where entire teams or development organizations are out-sourced the contract models used may change as trust improves (or decreases) between customers and suppliers.
Contracts set expectations between organizations and so often include details of:
However, as noted in Risk Driven Lifecycle, at the beginning of a piece of work we typically know very little about it, as knowledge is gained uncertainty decreases. This cone of uncertainty, based on risk, is reduced quickly in iterative/agile processes but is stretched out in waterfall teams.
This uncertainty means that at the beginning of a piece of work, when we know the littlest about it, our estimates of schedules, scope and quality will be at their most inaccurate - typically by a factor of 4. Unfortunately, this is also the time at which contracts are written.
When suppliers are pushed into accepting risky projects they will build in contingency (a form of waste) into their plans, driving estimates and costs up to mitigate their financial risk.
Contracts that ignore the cone of uncertainty, the evidence of past projects, and do not have flexibility to change will be bad for both customers and suppliers.
If work cannot be clearly specified and “confidently estimated” (in terms of the Certainty Scale described in Estimates) then fixing those details contractually is nonsensical. This situation rarely exists, except for highly predictable, almost commoditized work – which is actually ideally suited to out-sourcing. Many organizations we’ve worked with fool themselves into ignoring uncertainty in software contracts using inappropriate production metaphors only to feel the pain later.
Targets and Incentives
- A pressure to get maturity assessments done caused the mentoring team to hassle development teams, damaging their reputation as positive enablers
- One month there were no assessments done because the various teams couldn’t fit in an assessment. The mentoring team’s management rounded up this 0 to 2 on their monthly report
Assuming a valid build or buy decision has led to a contract being let for software development work there are a number of typical options available. We strongly recommend avoiding contractual incentives. The primary incentive to a supplier is repeat work providing a continued revenue stream - short term profit increases are counter-productive and damage collaborative relationships. Our experience of incentivized contracts is that they inadvertently promote negative behaviors. Short-term contractual incentives do not incentivize supplier delivery teams, but supplier sales teams.
Fixed Price involves fixing cost, scope and schedule at the beginning of a contract.
Fixed Price can be attractive to business leaders because it offers the illusion of certainty. You know what you’re going to pay – which is true. But unfortunately, you don’t know what you’re going to get, or at what level of quality. Some businesses believe that Fixed Price is easier to manage despite all previous evidence, and so apply a pressure on their organizations to “go Fixed Price”. Often the urge to go fixed price is a response to historical abuse by suppliers of other models – which is a very dysfunctional way to start a new contract.
When a project goes wrong, a Customer is ultimately left with the problem, risks are never truly transferred, only shared. Contractually liability is normally capped and "reasonable" quality/scope clauses leave enough room for the Supplier to get out of a contract delivering a fair amount of the scope, as originally specified, at a low level of quality. The Customer then has to spend more money fixing the product leading to another contractual phase.
Fixed Price is the least mature model for the majority of software development work.
Firm Price involves setting upper and lower limits for cost, scope and schedule at the beginning of a contract.
Firm Price models are often variable cost models with an upper and lower cap to limit liability for both parties. Although Firm Price models superficially sound like they solve the problem with Fixed Price contracts, in practice partial progress towards Business Value is very hard to quantitatively measure. Attempts to measure based on Story Points or any other abstract measure are deeply flawed and should not be considered as valid contractual models since Story Points are both abstract and relative. During Continuous Investment Review we measure progress as % of budget and % of scope, however we expect both of those to change. Firm Price models put limits on how much change is acceptable. Unfortunately, some suppliers will underestimate risk in order to win work from a customer.
Firm Price models that add flexibility by building in scope/quality change within a certain tolerance on top of a Fixed Price base can work if the length of the project and risk impact are low enough for Fixed Price to be a valid model.
Time and Materials
Time and Materials is effectively a time-hire model where a Customer pays a Supplier a certain rate for each day they work on the project.
This model can work if the Customer and Supplier have high trust, high levels of engagement and collaborate well together. Often Time and Materials can feel like a trap for Business Leaders as they have no upper limit on costs and can feel as though they are paying for nothing as a Supplier can leave the contract having earned money but not delivered anything. T&M can feel like you don’t know how much you’re paying or what you’re going to get.
Time and Materials is well suited to high complexity situations where scope, resources and schedules are highly variable. However, to work Time and Materials must be predicated on a high trust collaborative relationship and works well with known individuals and small companies, but less well with large vendors.
Weighted Fixed Price
Weighted Fixed Price is an evidence-based variation of Fixed Price in competitive markets that weights supplier estimates based their previous performance.
- Suppliers don’t want to underestimate - if the underestimate then they’ll come in over their initial estimates and adversely affect their weighting and therefore their chance for follow on work
- Suppliers don’t want to overestimate – if they overestimate so that they can manipulate their weighting next time, or build in risk buffers, they won’t get the work because they’ll cost too much
- Suppliers don’t want to take on complex, high risk work because they can’t predict the effect on their weighting
- The time periods to which they’re normally applied - Time and Materials capped time periods are often months or quarters. Fixed Price time periods are often 6monthly or above.
- Scope - Time and Materials phases are normally loosely scoped; Fixed Price phases are normally tightly scoped.
Phased Fixed Price
Phased Fixed Price involves fixing costs and schedule on loosely scoped short/medium term goals, re-negotiating at each phase. Phases are normally tied to short term releases, monthly or quarterly as a maximum.
A Business Leader is a person or group in the business who performs cultural, visionary and executive leadership. Leaders make decisions and perform a number of governance functions for the organization such as Strategic Direction and Continuous Investment Review.
A Business Leader might be a Business Owner, Director, senior Individual or skilled Individual - Business Leadership does not have to be a role performed by an individual although that is currently the most prevalent model.
Business Leaders are, typically, primary stakeholders for Executive Dashboards and the top of the Bubble Up process. In Holistic Software Development we promote transparent honest leadership and recommend that Business Leaders engage directly throughout the organization with people rather than reading reports and making decisions behind closed doors. The most effective form of communication is talking to people, the best way to understand things is to go see them and talk to the people who make them.
"The function of leadership is to produce more leaders, not more followers." --Ralph Nader
Most importantly, leaders need to treat individuals as human beings, not resources.
"The quality of a leader is reflected in the standards they set for themselves." --Ray Kroc
"Leadership is the capacity to translate vision into reality." --Warren G. Bennis
Leaders are required to see and communicate visionary aspiration within their communities. A differentiating feature of leaders, is that they can often see a future that others can't yet. Leaders help an organization by aligning expectations, providing purpose for motivation and supporting experimentation through celebrating success and failure. Good leaders are a communication conduit to the network of people in an organization, connecting people together and helping the organization learn.
Leadership, not control
"To add value to others, one must first value others." --John Maxwell
"A genuine leader is not a searcher for consensus but a molder of consensus." --Martin Luther King Jr.
Control, and management are not leadership. Leaders make strategic decisions, enable others to provide value, communicate and grow networks. Leaders need to ensure that they create room for others to learn, experiment and innovate.
Leadership, not superheroes
"A leader is best when people barely know he exists. When his work is done, his aim fulfilled, they will say: we did it ourselves." --Lao Tzu
Leaders often fall into a trap of trying to be right all of the time, trying to be heroes that do everything. Some people are in it for the glory rather than the results. Leaders aren't expected to know all of the answers, they are expected to enable others to find the answers.
"Leaders don't hide the details from people, they expose the detail and the thinking justifying their decisions without argument" - HSD
The best leaders do away with hidden agendas by exposing agendas. Helping networks and communities evolve their agendas together, and then making the details visible. Communicating why a direction is the right one, rather than simply telling people it is.
We recommend that business leaders communicate their leadership intent by publishing their annual/regular objectives (linked to Strategic Goals) and progress against them on a regular basis. Transparent Leadership also involves leadership shadowing, promoting social business collaboration with direct communication from senior executives, regular floor walking briefings and promotion of open feedback. As mentioned in Continuous Investment Review, transparency around decision making helps engage a workforce with senior leadership.