THE PRODUCT PROBLEM

The integrity of internet traffic routing was compromised due to widespread manipulation of the BGP ORIGIN attribute, which affected how traffic was forwarded across the network. This manipulation resulted in a significant deviation from the intended routing paths, leading to potential inefficiencies and vulnerabilities in traffic management. The business needed to ensure reliable and predictable routing to maintain service quality and trust. Users faced inconsistent network performance, while competitors could exploit this weakness to offer more stable and secure routing solutions, putting pressure on the business to address this issue.

THE DECISION

The decision to investigate and address the manipulation of the ORIGIN attribute involved balancing the need for network stability with the operational realities of existing routing practices. This required alignment across product management, engineering, and network operations teams to prioritize the integrity of routing information over the flexibility that some ASes exploited for traffic control. The tradeoff involved potentially disrupting existing traffic management practices for some network operators in favor of a more standardized and predictable routing protocol. Achieving consensus was challenging due to the entrenched behaviors and varied interests of different stakeholders, but it was necessary to enhance overall network reliability.

THE LESSON

This situation highlights the complexity of PM/eng collaboration when addressing systemic issues in established protocols like BGP. It reveals that tackling such problems requires not only technical solutions but also strategic coordination across different teams and stakeholders. PMs must understand the technical nuances and operational implications to effectively drive change. The experience underscores the importance of cross-functional alignment and the need for PMs to advocate for long-term stability over short-term convenience, even when it involves challenging existing practices and negotiating with diverse interests.