As a result of a misconfiguration at one of its data centers, US Internet Service Provider (ISP) CenturyLink suffered a major technical outage that swept the Internet, shutting down many popular sites and services on Sunday. The bug in the company's data center spread outside its network and affected other ISPs as well, resulting in connectivity issues for many other companies, including Amazon, Twitter, NameCheap, OpenDNS, Reddit, Discord, Hulu, Steam and more. Cloudflare was also badly affected by the CenturyLink outage, and in a blog post, CEO and co-founder of the website security and web infrastructure company, Matthew Prince, explained how the incident affected the internet as a whole, saying : “Because this outage appeared to take the entire CenturyLink / Level (3) network offline, people who are CenturyLink customers would not have been able to contact Cloudflare or any other ISP until the issue had occurred. has not been resolved. Globally, we saw a 3,5% drop in global traffic during the outage, which was almost entirely due to a near-complete outage of CenturyLink ISP service in the US."
Incorrect flow specification rule
Based on information on a CenturyLink status page, it appears the problem is with the ISP's CA3 data center in Mississauga, in the Canadian province of Ontario. As their own services were affected by the outage, Cloudflare paid close attention and believes that a bad Flowspec rule that came at the end of a long list of BGP updates may have caused it. If this were the case, every router in the CenturyLink/Level(3) network would have received the Flowspec rule, started blocking BGP, and this would cause them to stop receiving the rule. The devices would then reboot, go through all the BGP rules until they hit the wrong Flowspec rule, and BGP would be removed again, creating a loop that would continue in the process over and over again. BGP routes are a type of message in which Internet companies take turns telling each Internet provider what pool of IP addresses are available on their network. However, CenturyLink's incorrect Flowspec rule also caused some routers to break out of its network, which began to advertise incorrect BGP routes to other Tier 1 Internet services, causing other networks to crash. , causing the biggest internet outage of the weekend. Fortunately, CenturyLink was able to solve the problem by telling all other Tier 1 ISPs to ignore all traffic coming from their network. This type of action is usually a last resort, as it results in the loss of Internet connectivity for all of the company's customers. Via ZDNet