A renowned banking institution, specializing in leasing services faced significant challenges in its network and security infrastructure. To analyze the core issues and find tailored solutions, the comdivision team, led by Tobias Paschek, was brought in for support.
“There were several problems to address,” explained Paschek. These included:
1. Slow NSX Firewall Performance: A detailed analysis revealed an excessive number of firewall rules, leading to performance degradation.
2. Instability of OpenShift Containers: The OpenShift containers repeatedly lost their firewall rules.
3. Loss of MAC Addresses: Containers regularly lost MAC addresses, complicating network management.
4. Overloaded Hosts: A host analysis showed some hosts had over 100,000 firewall rules, potentially impacting network performance.
In the course of the analysis, the following changes and innovations were identified:
1. Automated Rule Generation: A third-party tool (Tufin) generated firewall rules for central firewall management that serviced more than just NSX.
2. Hardware Refresh: A comprehensive hardware refresh, including new network components and the implementation of VXLAN in the underlay (Etherfabric), was conducted.
“The company faced the challenge of locating the performance issue and excluding an overload at all levels through the hardware refresh,” explained Paschek, Lead Solutions Architect. The key steps included:
1. In-depth Analysis of Firewall Rules: Existing firewall rules were reviewed and cleaned up to optimize NSX firewall performance, and the application scope was explained to the client to avoid ESXi host overload.
2. Stabilizing OpenShift Containers: The introduction of the Network Container Plugin (NCP) for OpenShift containers significantly increased stability.
3. Addressing MAC Address Loss: A detailed investigation of the network stack and container orchestration identified the cause of the losses, which were then resolved with NCP.
4. Optimizing Host Configuration: The number of firewall rules per host was drastically reduced to improve network clarity and performance.
These measures led to a significant improvement in network performance and security. The NSX firewall now operates more efficiently, OpenShift container stability has been restored, and overall network performance has greatly improved through the optimized use of hardware and software. “The comprehensive overhaul of our network structure was crucial for increasing our operational efficiency. We are impressed by the expertise and dedication of the comdivision team. They helped us achieve a robust and future-proof network and container platform," the CEO of the client added.
