Netvouz blog

Who pulled the Internet plug?

Today I got an alert from the monitoring service that constantly checks that Netvouz is healthy. It said Netvouz was down. Strange, because at that same moment I was using it and it worked without a glitch. So it must be something else. And so I started digging....

First I went to my long-time friend GrabPERF, which also keeps track of how we're doing (also see this post). It also reported Netvouz being down. But not completely. So I took a closer look at the GrabPERF details to see what was going on. It told me that some of it's monitoring agents, such as Zurich and Technorati #2 (but not #1!), failed to reach Netvouz but others didn't. So about half of them could not reach Netvouz, while the others could. This is shown in the GrabPERF graph below.

In the middle of the graph you see several measurements which have failed (the purple dots at the bottom), while others have succeed (the green ones above them). That's odd. But at least it's not a problem with the Netvouz servers or the network connection we're hooked up to.

Hmm... let's go check the Netvouz real-time web traffic statistics.


This statistics shows that at noon local time (Central European Time) there was a sudden drop in the number of visitors. OK, so it's a fact. The Netvouz service is healthy as ever, but apparently we can not be reached from some locations on the Internet. Interesting, but why?

So this means that somewhere on the Internet a major core router that handles a lot of traffic went down at about noon today. That could happen. But it also means that the remaining routers did not manage to route via alternative routers, and that should not happen. Luckily, as both graphs show, traffic was reestablished again after about 20 minutes.

So what happened out there today? Was it just a hardware failure at an important router? Or a successful hacker attack that managed to bring it down? Or was it just a "oopss" when someone accidentally pulled the wrong plug?

I don't know, do you?

The mystery remains........

//Henrik