Goals
- Execute and coordinate the improvement activities to eliminate problems and issues found during analysis. There is a whole bunch of practices devoted to this step, and several approaches you can take to run the improvements.
- Apply selected opportunities for improvement:
- change code, structures, concepts or processes to achieve better software,
- reduce costs and/or technical debt,
- eliminate all kinds of issues,
- optimize quality attributes (like performance, maintainability, security),
- optimize operation and administration processes, thereby reducing effort and cost.
Structure of the improvement phase
Fundamentals are principles you should consider whatever steps you take on your road to improvement. Approaches are overall (strategic, long-term) decisions on how to tackle improvement. Practices are fine-grained practices or patterns, structured in several categories.
Fundamentals
For improvement we take a number of fundamental principles for granted, depicted in Fundamentals.
These fundamental principles surely belong to software engineering good practices - but we consider them indispensable for improvement projects.
- Fast-Feedback
- Get feedback to your actions and changes as early as possible, so you can adjust as quickly as adequate.
- Improve Iteratively
- Improve in (potentially small) iterations and/or increments, so that change does not disturb or negatively affect the system, its associated processes and organization. Iterations are the prerequisite for our whole phased improvement.
- Prototype-Improvement
- Verify the viability and effectiveness of improvements, usually in smaller scales with reasonable risks.
- Verify-After-Every-Change
- Always make sure that changes, even minor ones, leave your system intact. (The awesome Jerry Weinberg has written up several examples of such failures).
- Reduce Complexity
- Simpler solutions are most often easier to comprehend, maintain and operate. Therefore always strive for simplicity and the reduction of accidental, unnecessary complexity.
- Explicit Assumption
- Compensate missing facts (especially requirements, goals, estimates, opinions) by explicit (written) assumptions about those facts. See Explicit Assumption.
- Group Improvement Actions
- Group related actions, so that they refer to similar entities and potential synergies are utilized.
Improvement Approaches (Overview)
- Data Migration
- Keep your (valuable) data, and toss (or rewrite or otherwise change) your code. Oftentimes combined with approaches from the categories rewrite or restructure.
- Rewrite
- Your system is broken beyond repair and you need to completely replace it by a new one. Rewrite approaches give you some ideas how and if that might work (spoiler: we fear that Big-Bang won’t work…)
- Restructure
- Improve your system by restructuring your code in-the-large. Might involve extraction of certain functionalities, splitting your system, improving the modularization or strangulating certain (bad) parts of the system (and, of course, replacing those by better solutions).
- Improve Modularization
- Subcategory of restructure: improve responsibilities within the system, improving the boundaries between components, improving interfaces and similar operations.
- Brainsize
- Evidence from neuroscience suggests our working memory has a capacity of about four items1. That’s why smaller solutions tend to be more maintainable, as the cognitive load on the developers working memory is reduced. These “brainsizing” strategies can be used to reduce the amount of stuff, e.g. by splitting your system up, extracting certain parts into abstractions, or other ways to reduce LOC or other complexity metrics. Terms like microservices fall into this category.
- Improve-Domain-Focus
- Subcategory of restructure:Clear separation of domain-related code from purely technical aspects has long been a useful design heuristic - but is still often violated. In addition, aspects belonging to similar areas of the domain should be implemented within the same building-blocks (in Domain-Drive Design terminology called Bounded Context).
You find further information on the detailed approaches here.
Improvement Practices (Overview)
- Improve Processes and Organization
- Sometimes your issues originate in process or organizational root causes, meaning your development, rollout or operations processes are less efficient than they should be. This category addresses such problems. For details see Practices to Improve Processes.
- Improve Architecture and Code Structure
- All aspects of sourcecode may be subject to improvement - style, structure, dependencies, conventions, naming and the like. Furthermore, structure in the large (modules, components, interfaces) or crosscutting and technical concepts belong to this area of improvement. For details see Improve Architecture and Code Structure.
- Improve Technical Infrastructure
- Technical infrastructure encompasses both underlying hardware and software. For details see Practices to Improve Technical Infrastructure.
- Improve Analyzability and Evaluatability
- Make the system easier to analyze and understand, e.g. by improving logging, tracing or by introducing clearer structures. Enable or facilitate evaluation (e.g. of certain issues) by creating, collecting and managing certain numbers (metrics), either in development-, deploy- or runtime. For details see Practices to Improve Analyzability and Evaluability.
Approaches, Practices and Regular Development
In regular development (sometimes known as daily business) you will most likely intertwine your approach(es) with numerous practices - as depicted in Integrating improvement approaches and practices with regular development.
In both long- and short-term planning (yes - even in highly agile and iterative development models you’ll have such planning) you need to balance the following often conflicting goals:
- short-term profitability by creating business value by delivering features or fixes to production.
- long-term maintainability of the system by improving inner quality, improving code structure, technology choices and the like.
Improvement Approaches (Details)
One of the central decisions involves your long-term improvement-approach, the overall, long-range or strategic decision how you want to improve your system.
- Change-By-Split
- Split up the original system into (not necessarily distinct) parts. Clean-up those parts individually, and then evolve the parts independently.
- Keep-Data-Toss-Code
- As value sometimes resides in data, keep data intact and replace the functional/service/process part of a system.
- Frontend-Switch
- Start creating new backend parts. Frontend routes some requests to those new backend parts, others still to the existing ones. Gradually enhancing the new backend parts, frontend routes more and more requests to new backend.
- Big-Bang
- Keep the existing system for a limited time, apply only critical bugfixes. In parallel, build a replacement system. Replace old by new at predefined time.
- Chicken-Little
- Incrementally (11 steps) build a replacement system. You can choose between Database-First, Database-Last and Composite-Database Approach.
- Database-First
- Do a Big-Bang migration of the database, incrementally implement new applications and interfaces and connect the legacy system to the new database by forward gateways.
- Database-Last
- Keep the existing database for a limited time, incrementally implement new applications and interfaces and connect them to the legacy database by reverse gateways.
- Composite-Database
- Combination of Database-First and Database-Last. Beside a forward- and reverse-gateway, there is a need for a transaction-coordinator.
- Butterfly-Methodology
- Data-Migration Method without the need for gateways. Enables zero-downtime migrations by working with temporary data stores.
- Evolution
- This approach has been extensively practiced by a Swiss Bank and published as a book. Underlying idea is to refactor those parts of the system(s) which are actually to be changed, especially to move all interfaces to new service standard and replace all legacy technologies and other couplings (via DB etc). Over time services should emerge that can be moved to a new platform altogether (from Mainframe to Java).
Improvement Practices (Details)
Practices, in contrast to approaches, are the short-term or tactical improvements.
We already explained the categories of these improvement practices in Improvement Practices (Overview). Here we dive into more details, structured along these categories:
- Improve Processes,see Practices to Improve Processes
- Improve Architecture and Code Structure, see Improve Architecture and Code Structure
- Improve Technical Infrastructure, see Practices to Improve Technical Infrastructure
- Improve Analyzability, see Practices to Improve Analyzability and Evaluability
Practices to Improve Processes
For an overview of other improvement practices, see Improvement Practices (Overview).
One way to improve the processes is to resort to Mob Programming for onsite teams or Remote Mob Programming for distributed teams.
Improve Architecture and Code Structure
Note: This category contains a fairly large number of practices.
For an overview of other improvement practices, see Improvement Practices (Overview).
Practices to Improve Technical Infrastructure
For an overview of other improvement practices, see Improvement Practices (Overview).
Practices to Improve Analyzability and Evaluability
For an overview of other improvement practices, see Improvement Practices (Overview).
Approaches and practices for improvement
Improvement approaches
Strategies for improving a system in the large — keep the data and toss the code, rewrite, restructure, improve the modularization or the domain focus.
-
Approach to replace an old system with a new system with one big bang deployment.
-
Data migration method without the need of gateways. Enables zero-downtime migrations by working with temporary data stores.
-
Maintain and evolve a few smaller systems, instead of one big monolith. Reduce time-to-market, work in smaller teams that individually become more focussed.
-
Software-Migration with incremental steps to avoid or reduce the risks coming along with the Big Bang approach.
-
Combination of Database-First and Database-Last. Beside a forward- and reverse-gateway, there is a need for a transaction-coordinator.
-
Do a Big-Bang migration of the database, incrementally implement new applications and interfaces and connect the legacy system to the new database by forward gateways.
-
Keep the existing database for a limited time, incrementally implement new applications and interfaces and connect them to the legacy database by reverse gateways.
-
Front-End Switch stub
Route front-end requests to either new or old backend systems, depending on their nature, content-negotiation or other request criteria. This is especially helpful to support Never Change Running System.
-
Keep Data, Toss Code stub
A strategy to improve systems, keeping the data created with the (old) systems as foundation for a new one. Also described as Bridge-to-the-New-Town (by Wolfgang Keller). This is the opposite of Never Change Running System.
-
Divide a legacy system into different functional domains and replace those step by step.
Processes and organization
Sometimes your issues originate in process or organizational root causes, meaning your development, rollout or operations processes are less efficient than they should be.
No patterns in this category yet.
Architecture and code structure
All aspects of source code may be subject to improvement — style, structure, dependencies, conventions, naming — as well as structure in the large (modules, components, interfaces) and crosscutting technical concepts.
-
An anticorruption layer is a logical layer that provides a stable interface to (potentially) volatile software components. As long as this interface remains untouched developers can implement changes or even replace their own or third-party software without affecting the clients of this interface.
-
Assertions stub
Use assertions to verify preconditions and to make a program fail early when something goes fundamentally wrong.
-
Automated Tests stub
Introduce automated tests to verify correctness or runtime behavior. Unit-, integration-, acceptance-, load- or database-tests are well-known specialisations of this.
-
Introduce distinct branches in your version control system to reflect improvements.
-
Change-by-Abstraction Refactoring
Incrementally replace part of the system with a new implementation.
-
Extract Reusable Component stub
Extract code from an existing system to create a reusable component. See [SERIOUS-Refactoring], page 95.
-
Front-End Switch stub
Route front-end requests to either new or old backend systems, depending on their nature, content-negotiation or other request criteria. This is especially helpful to support Never Change Running System.
-
Group Improvement Actions stub
Collect several improvement actions, which can or shall be applied or implemented together.
-
Refactor nested if-then-else structures for improved understandability. Can be seen as a specialisation of Remove Nested Control Structures.
-
Improve Code Layout stub
Making code easier to read results in better understandability.
-
Improve Feedback from and for Stakeholders
Effectively collect feedback from various stakeholders.
-
Employ sophisticated logging mechanisms such as modern logging frameworks, distributed log collection and visualization tools in order to gain more detailed information about the system during runtime with a minimal or predictable performance impact.
-
Interface Segregation Principle
Reduce coupling between clients and service providers.
-
Enable cross-cutting architectural improvement even if it is not feasible to change the whole codebase.
-
Introduce Layering stub
Introduce layers within the source code to improve separation of concern. It’s common to have at least a business layer and an interface layer - the latter for both user- and programmatic interfaces. See Uncle Bob’s Clean Architecture for a short summary.
-
Isolate Changes stub
Introduce interfaces and intra-system borders, so that changes cannot propagate to other areas.
-
Keep Data, Toss Code stub
A strategy to improve systems, keeping the data created with the (old) systems as foundation for a new one. Also described as Bridge-to-the-New-Town (by Wolfgang Keller). This is the opposite of Never Change Running System.
-
Manage Complex Client Dependencies with Facade
Simplify the interaction of a client with a set of service components.
-
Gather various metrics and visualize them on dashboards in order to make your system behavior more predictable and assumed coincidences explainable.
-
Never Change Running System stub
To minimize risks, you should try to refrain from changes to existing (working) code - as every change inevitably introduces new risks or even bugs.
-
Never Rewrite Running System stub
Joel Spolsky arguments, never to rewrite a system from scratch, as you will likely make many new mistake and won’t generate much added value.
-
Quality-Driven Software Architecture stub
Derive (technical, structural or process-related) decisions based upon detailed quality requirements. QDSA requires explicit quality requirements.
-
Refactoring stub
Source code transformation that does not change functionality of system. See [Fowler-Refactoring].
-
Refactoring Plan stub
The route of refactoring, as discussed within the development team. This plan should always be visible to every team member.
-
Remove Nested Control Structures stub
Re-structure code so that deeply nested or complicated control structures are replaced by semantically identical versions. Special case of Refactoring, similar to Untangle Code. Often performed by reducing complexity and especially cyclomatic complexity. When reducing code complexity one needs to make sure we’re not exchanging inner/ method/ cyclomatic complexity by outer/ design or runtime complexity.
-
Provide concrete code example for typical improvement situations, so that developers can improve existing code easily.
-
Schedule Work stub
Schedule refactoring or improvement work, so that all (business and technical) stakeholders know about them.
-
Untangle Code stub
Remove unnecessary complications in code, e.g. nested structures, dependencies, dead-code, duplicate-code etc. See Remove Nested Control Structures. Special case of Refactoring.
-
Use Invariants to Kill Zombies
Provide a safe approach in situations where it seems to dangerous to delete code or whole modules from a huge system because Static Code Analysis can’t recognize whether the code is still in use or not.
Technical infrastructure
Technical infrastructure encompasses both underlying hardware and software.
No patterns in this category yet.
Analyzability and evaluability
Make the system easier to analyze and understand, e.g. by improving logging or tracing, and enable evaluation by collecting metrics during development, deployment or runtime.
-
Docs-As-Code is a documentation approach that raises developer-related documentation to the same importance as source code.
Other improvement patterns
-
Architecture Backlog stub
Keep a prioritized list of improvement tasks (remedies) with their as a backlog, parallel to the (regular) feature backlog.
-
Bulkhead stub
Can be placed between two systems to avoid the propagation of faults from one system to the other system ([Nygard07], p. 119ff.).
-
Change by Copy stub
Isolate competing change necessity by copying and allowing the copies to evolve independently. Also known as Change via Split.
-
Change by Extension stub
Enable efficient change by creating new components instead of modifying existing ones.
-
Actively mark parts in software that aren’t needed anymore and communicate to your consumers or customers, that you will remove specific functionality in the future.
-
Migrate Data stub
Transform existing data from one structure or representation into another by keeping its original intent or semantic intact.
-
Mikado Method stub
Coordinated refactoring effort, described in the Mikado-book.
-
Natural Death stub
Keep old system running and only retire it once all objects contained reach end of life according to their life cycle.
-
Modularize system aligned to (existing) external interfaces.
-
Toggle Feature stub
Simultaneously support evolved, competing or conflicting features at runtime by toggling feature flags.
-
Cowan: The magical number 4 in short-term memory: a reconsideration of mental storage capacity. ↩