When power drops, there’s an immediate shock to the Internet routing system. You can see it clearly in this plot, which shows the number of South American networks affected by serious instability or unreachability in the previous 15-minute window.
By compiling the routing traffic we collected from all over the world during this period of time, it’s possible to pinpoint the time of the failure in the Brazilian electric grid (10:26pm, or 00:26 UTC). From an essentially negligible background level, the number of affected networks vaults to more than 150 within seconds. Following the classic pattern of widespread power failures, the number of freshly affected networks drops over time as backup power kicks in, and then there are ‘echoes’ of new failures as the backup power suffers its own problems.
These are some screenshots from our KML-based regional outage visualizer, and our Internet Health Portal tool. As you can see, Brazil took the largest hit, but we also saw Paraguayan and Uruguayan networks out as a result of the largest regional power outage to hit Brazil and its neighbors in several years.
Inevitably, speculation will fly fast and furious about the cause. Media reports this week suggested that hackers may have taken down the Brazilian grid in 2005 and 2007. Brazil says it simply wasn’t so. Was tonight a sardonic repeat performance, in response to Sunday’s 60 Minutes coverage?