New stuff is fun and greenfield is (relatively) straightforward. But, greenfield is rare so the pragmatic reality we all live with is lots of legacy…and most network projects are brownfield. This fosters accrual of technical debt, which is pervasive in many enterprise networks. This tech debt ultimately adds complexity which reduces networking agility, availability and scalability. Just a few common examples:
- Firewall rules/ ACLs that are many thousands of lines, including entries that likely are not needed, but that nobody is willing to remove
- Running three or more routing protocols in the network backbone with complex redistribution policies
- Numerous unused or sparsely populated VLANs in the data center
Unfortunately, many networking vendors (and their channels) aim to drive product stickiness, maximize service revenue and meet sales goals by increasing complexity and enabling value-added features. Thus, they have little reason to help you reduce the deployed footprint, enable more efficient configurations or optimize your network. We estimate that at least 10% — and perhaps as much as 35% — of configurations in enterprise networks are unnecessary.
Thus, moving forward we must make a more overt and concerted effort to reduce technical debt in our networks. This is easy to say, but quite hard in practice… so we need a strategy for it. Along these lines, we just published this research on it (paywall): How to Reduce Technical Debt in Enterprise Networks
Summary: Many enterprises carry high levels of technical debt in their networks, which limits network agility and increases unplanned downtime. To combat technical debt, I&O leaders should build an automated pipeline for common change requests, and make specific adjustments to NetOps culture.
In the research, we lay out a top-down and bottom-up framework to reduce accumulation of new tech debt and also reduce some existing. A few snippets from the research…
MVC – Vendors are known to release minimum viable products, and we can apply some of these principals. For example, we recommend an exercise to determine your minimum viable config (MVC): Determine your minimum viable configuration (MVC) for a specific deployment or project. The MVC (and accompanying minimal viable design) is a starting point is to determine the most basic configuration to simply meet business needs, which may mean no resiliency or manageability. After that, for every additional component, you can document the value against the additional technical debt.
Rewards and Recognition – Network leaders should Reward recognition and removal of configuration drift. Set up a continual rewards program that compensates people for finding, documenting and/or removing unnecessary configurations or devices.
Network Hygiene – And of course, sometimes it is good ole fashioned operational excellence, blocking and tackling or good “network hygiene” as my colleague Mark Fabbi calls it. And here’s the deal…most network pros know about this but few of us actually follow them all. But given the rapid and accelerating pace of change (DevOps, Cloud, Cloud-Native, K8S, etc. etc.), these must become requirements, not luxuries. This means we must
- Use centralized authentication and authorization for all staff who make network changes.
- Track and capture network device configurations for troubleshooting, rollback and auditing purposes.
- Maintain standardized device configuration templates that are managed via a version control system.
- Regularly report and document configuration drift violations from standard template configurations. Assign action items for network staff to identify and address.
- Integrate network changes into the organization’s change and release management efforts
Best wishes to remove some tech debt in 2020!