THE PRODUCT PROBLEM
Zalando's Product Read API was struggling to maintain its promise of single-digit-millisecond latency while serving millions of requests per second across multiple markets. The existing infrastructure, relying heavily on Skipper as a shared cluster edge load balancer, was causing unpredictable latency spikes due to its inability to isolate internal traffic delays. This bottleneck was not only affecting user experience but also inflating operational costs. Competitively, Zalando needed to ensure rapid and reliable service to maintain its edge in the fast-paced online fashion retail space, where consumer expectations for speed and reliability are non-negotiable.
THE DECISION
The team made a strategic decision to shift the routing of high fan-out internal traffic from Skipper to an in-process client-side load balancer. This decision allowed them to maintain Skipper for edge traffic, where it performed well, while gaining more control over internal traffic routing. Aligning on this decision required collaboration between product management, engineering, and operations to ensure that the transition would not disrupt existing services and would ultimately lead to cost savings. The tradeoff involved the complexity of implementing and maintaining a new system, but the gain was significant: reduced latency, lower infrastructure costs, and improved failure visibility. In hindsight, this decision was crucial for scalability and performance, although it required careful orchestration and testing to avoid disruption.
THE LESSON
This case illustrates that effective PM/engineering collaboration often requires a willingness to challenge existing infrastructure paradigms and take calculated risks. The decision to implement a client-side load balancer was not just an engineering feat but a strategic product decision that demanded deep cross-functional alignment and trust. It underscores the importance of having a shared vision and understanding of the problem at hand, allowing teams to innovate beyond traditional boundaries. PMs should recognize that sometimes the best solution involves rethinking the role of existing tools and being open to unconventional approaches that align with long-term business goals.