Well ... for those who weren't effected by that ... excerpts from news following the outage.
http://www.ladbible.com/community/interesting-technology-news-simple-error-causes-widespread-internet-outage-across-united-states-20171107
...
In a statement issued to WIRED, a Level 3 spokesman said: "Our network experienced a service disruption affecting some customers with IP-based services.
"The disruption was caused by a configuration error."
The spokesman added that they were able to get services quickly and efficiently back online within 90 minutes.
Comcast and RCN began experiencing connectivity issues at around the same time as the Level 3 outage, with the former saying that it was investigating an "external network issue", as opposed to a problem with its own infrastructure.
The misconfiguration was what is known as a 'route leak', according to Roland Dobbins, a principal engineer at the DDoS and network-security firm Arbor Networks, which monitors global internet operations.
This is where information about IP addresses on a particular network gets confused and causes inefficient routing and failures for the originating ISP and other ISPs attempting to route traffic.
Lily Hay Newman at WIRED describes it as being like a 'series of street signs that help keep traffic flowing in the right directions. If some of them are mislabeled or point the wrong way, assorted chaos can ensue'.
Route leaks can be intentional and malicious, however, in the case of this week's internet outages, it appears to have been a rudimentary mistake that snowballed to have colossal
A year ago, a DDoS attack caused internet outages around the US by targeting the internet-infrastructure company Dyn, which provides Domain Name System services to look up web servers. Monday saw a nationwide series of outages as well, but with a more pedestrian cause: a misconfiguration at Level 3, an internet backbone company—and enterprise ISP—that underpins other big networks. Network analysts say that the misconfiguration was a routing issue that created a ripple effect, causing problems for companies like Comcast, Spectrum, Verizon, Cox, and RCN across the country.
Level 3, whose acquisition by CenturyLink closed recently, said in a statement to WIRED that it resolved the issue in about 90 minutes. "Our network experienced a service disruption affecting some customers with IP-based services," the company said. "The disruption was caused by a configuration error." Comcast users started reporting internet outages around the time of the Level 3 outages on Monday, but the company said that it was monitoring "an external network issue" and not a problem with its own infrastructure. RCN confirmed that it had some network problems on Monday because of Level 3. The company said it had restored RCN service by rerouting traffic to a different backbone.
...
In a "route leak," an AS, or multiple ASes, issue incorrect information about the IP addresses on their network, which causes inefficient routing and failures for both the originating ISP and other ISPs trying to route traffic through. Think of it like a series of street signs that help keep traffic flowing in the right directions. If some of them are mislabeled or point the wrong way, assorted chaos can ensue.
Route leaks can be malicious, sometimes called "route hijacks" or "BGP hijacks," but Monday's incident seems to have been caused by a simple mistake that ballooned to have national impact. Large outages caused by accidental route leaks have cropped up before.
"Folks are looking to tweak routing policies, and make mistakes," Arbor Networks' Dobbins says. The problem could have come as CenturyLink works to integrate the Level 3 network, or could have stemmed from typical traffic engineering and efficiency work.
Yesterday, folks across the country reported that Comcast internet was down -- an unusually large outage that lasted around 90 minutes. It turns out that the problem was caused by Level 3, an enterprise ISP that provides the backbone for other internet providers like Verizon, Comcast and RCN. "Our network experienced a service disruption affecting some of our customers," the firm said in a statement. "The disruption was caused by a configuration error."
The outage shows yet again just how vulnerable the internet is in the US. Last year around this time, a DDoS attack shut down Spotify, Twitter, the New York Times and other sites, prompting some soul-searching from ISPs and internet security experts. This time it was a case of simple human error, but the results were similar: The internet, which many individuals and businesses now depend on for their livelihoods, went down.
asdf
No comments:
Post a Comment