Part of the Internet is down -- or is it just exodus.net?

greenspun.com : LUSENET : Grassroots Information Coordination Center (GICC) : One Thread

I just tried to use the www.northernlight.com search engine and could not connect to it. Pinging from other internet accounts I have shows that northernlight is indeed on.

This is what a tracert from my computer through bellatlantic.net (Washington DC provider) looks like:

Tracing route to www.northernlight.com [216.34.102.230] over a maximum of 30 hops: 1 * * * Request timed out. 2 35 ms 35 ms 34 ms adsl-151-200-45-1.bellatlantic.net [151.200.45.1] 3 36 ms 36 ms 37 ms 151.200.4.66 4 38 ms 34 ms 33 ms 208.46.127.113 5 37 ms 36 ms 35 ms wdc-core-01.inet.qwest.net [205.171.24.13] 6 41 ms 36 ms 34 ms wdc-brdr-01.inet.qwest.net [205.171.24.30] 7 40 ms 38 ms 39 ms sfo.exodus.net [205.171.4.142] 8 37 ms 36 ms 37 ms dcr03-p0-2.hrnd01.exodus.net [216.33.203.137] 9 37 ms 36 ms 37 ms bbr01-g3-0.hrnd01.exodus.net [216.33.203.93] 10 * * * Request timed out. 11 * * * Request timed out. 12 * * * Request timed out. 13 * * * Request timed out.

This is what a traceroute from Netcom (Dallas, TX) looks like:

$ traceroute www.northernlight.com traceroute to www.northernlight.com (216.34.102.230), 30 hops max, 40 byte packe ts 1 199.183.9.254 (199.183.9.254) 2 ms 1 ms 1 ms 2 dal-tx-gw1-7513.icg.net (163.179.1.134) 1 ms 1 ms 1 ms 3 s5-0-0.dai-tx-gw1.icg.net (165.236.32.73) 2 ms 2 ms 2 ms 4 a2-0-0-6.nwk-nj-gw1.icg.net (163.179.235.58) 41 ms 41 ms 41 ms 5 ibr01-s1-1-1.jrcy01.exodus.net (209.185.249.193) 42 ms 42 ms 48 ms 6 dcr04-p0-1.jrcy01.exodus.net (216.32.222.209) 42 ms 42 ms 42 ms 7 bbr01-g4-0.jrcy01.exodus.net (209.67.45.253) 43 ms 42 ms 43 ms 8 bbr02-p5-0.wlhm01.exodus.net (216.32.132.50) 47 ms 47 ms 47 ms 9 dcr03-g2-0.wlhm01.exodus.net (209.185.9.246) 48 ms 47 ms 50 ms 10 216.34.127.66 (216.34.127.66) 48 ms 50 ms 47 ms 11 209.67.227.158 (209.67.227.158) 51 ms 51 ms 53 ms 12 www.waltham.northernlight.com (216.34.102.230) 48 ms 48 ms 48 ms $

I just dialed into attglobal.net and connected to northernlight. Here is the traceroute:

Tracing route to www.northernlight.com [216.34.102.230] over a maximum of 30 hops: 1 101 ms 105 ms 116 ms 32.96.115.150 2 133 ms 100 ms 106 ms beth1ar2-5-0-15.md.us.prserv.net [165.87.96.254] 3 112 ms 109 ms 128 ms beth1sr1-0-1-0.md.us.prserv.net [165.87.29.183] 4 108 ms 104 ms 110 ms ibr01-s0-0-0.hrnd01.exodus.net [209.185.249.237] 5 109 ms 134 ms 132 ms dcr03-p0-1.hrnd01.exodus.net [216.33.203.145] 6 124 ms 108 ms 104 ms bbr01-g4-0.hrnd01.exodus.net [216.33.203.125] 7 110 ms 120 ms 134 ms bbr01-p5-0.jrcy01.exodus.net [209.185.249.213] 8 145 ms 143 ms 120 ms bbr02-p5-0.wlhm01.exodus.net [216.32.132.50] 9 125 ms 130 ms 130 ms dcr03-g2-0.wlhm01.exodus.net [209.185.9.246] 10 123 ms 127 ms 118 ms 216.34.127.66 11 119 ms 140 ms 131 ms 209.67.227.158 12 130 ms 132 ms 117 ms www.northernlight.com [216.34.102.230] Trace complete.

Notice all traces (both successful and unsuccessful) go through exodus.net. I will ask them to comment.

Sally and I both use BellAtlantic as our primary Internet connection and have not experienced any other unusual problems.

Ray

-- Ray Strackbein (Ray@Strackbein.com), January 01, 2000

Answers

I just connected to www.northernlight.com with no problem. My ISP is in Northern VA as well.

-- Steve Davis (Columbia, MD) (Steve@davislogic.com), January 01, 2000.

Having worked with computers, communications, and electronic systems for over 30 years, I know all to well that SOMETHING in most technical systems is usually broken -- that is normal and the nature of technology.

The question is whether the failure at bbr01-g3-0.hrnd01.exodus.net [216.33.203.93] is Y2K related or just another normal routine failure.

When I submitted the question, the posting software schmooshed the pretty-printed traceroutes into a unreadable mass of text. I will try to pretty-print using html.

This is what a successful tracert looks like:


Tracing route to www.northernlight.com [216.34.102.230]
over a maximum of 30 hops:

  1   101 ms   105 ms   116 ms  32.96.115.150 
  2   133 ms   100 ms   106 ms  beth1ar2-5-0-15.md.us.prserv.net 
[165.87.96.254] 
  3   112 ms   109 ms   128 ms  beth1sr1-0-1-0.md.us.prserv.net 
[165.87.29.183] 
  4   108 ms   104 ms   110 ms  ibr01-s0-0-0.hrnd01.exodus.net 
[209.185.249.237] 
  5   109 ms   134 ms   132 ms  dcr03-p0-1.hrnd01.exodus.net 
[216.33.203.145] 
  6   124 ms   108 ms   104 ms  bbr01-g4-0.hrnd01.exodus.net 
[216.33.203.125] 
  7   110 ms   120 ms   134 ms  bbr01-p5-0.jrcy01.exodus.net 
[209.185.249.213] 
  8   145 ms   143 ms   120 ms  bbr02-p5-0.wlhm01.exodus.net 
[216.32.132.50] 
  9   125 ms   130 ms   130 ms  dcr03-g2-0.wlhm01.exodus.net 
[209.185.9.246] 
 10   123 ms   127 ms   118 ms  216.34.127.66 
 11   119 ms   140 ms   131 ms  209.67.227.158 
 12   130 ms   132 ms   117 ms  www.northernlight.com [216.34.102.230] 

Trace complete.

This is what the failed tracert looks like:


Tracing route to www.northernlight.com [216.34.102.230]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2    35 ms    35 ms    34 ms  adsl-151-200-45-1.bellatlantic.net 
[151.200.45.1] 
  3    36 ms    36 ms    37 ms  151.200.4.66 
  4    38 ms    34 ms    33 ms  208.46.127.113 
  5    37 ms    36 ms    35 ms  wdc-core-01.inet.qwest.net 
[205.171.24.13] 
  6    41 ms    36 ms    34 ms  wdc-brdr-01.inet.qwest.net 
[205.171.24.30] 
  7    40 ms    38 ms    39 ms  sfo.exodus.net [205.171.4.142] 
  8    37 ms    36 ms    37 ms  dcr03-p0-2.hrnd01.exodus.net 
[216.33.203.137] 
  9    37 ms    36 ms    37 ms  bbr01-g3-0.hrnd01.exodus.net 
[216.33.203.93] 
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
 12     *        *        *     Request timed out.

Line 9 above shows me that the router named bbr01-g3-0.hrnd01.exodus.net can not find a path to northernlight. This could be either a router failure or a link (phone-line or fiber-optic failure). Notice that the successful tracert uses router bbr01-g4-0.hrnd01.exodus.net instead of bbr01-g3-0.hrnd01.exodus.net. Both of those routers belong to exodus.net and appear to be located in Herndon, VA (which I am interpreting from the cryptic "hrnd01" in the names of the routers). While having a router or a link down is common, I would expect my Internet connection to eventually "failover" from trying to use bbr01-g4-0.hrnd01.exodus.net instead of bbr01-g3-0.hrnd01.exodus.net. That is, the bad router should announce its failure and the good path should take over because of the failure announcement. I have notified the proper Network Operations Center of this failure. It gives me something to do during this rather boring Y2K anti-climax. Ray

-- Ray Strackbein (ray@strackbein.com), January 01, 2000.


A trace from 216.33.203.137 (dcr) and 216.33.203.93 (bbr) to www.northernlight.com looks good.

dcr03.hrnd01>trace 216.34.102.230

Type escape sequence to abort. Tracing the route to www.waltham.northernlight.com (216.34.102.230)

1 bbr01-g3-0.hrnd01.exodus.net (216.33.203.93) 0 msec bbr01-g4-0.hrnd01.exodus.net (216.33.203.125) 0 msec bbr01-g3-0.hrnd01.exodus.net (216.33.203.93) 0 msec 2 bbr01-p5-0.jrcy01.exodus.net (209.185.249.213) 4 msec 4 msec 4 msec 3 bbr02-p5-0.wlhm01.exodus.net (216.32.132.50) 12 msec 8 msec 12 msec 4 dcr03-g2-0.wlhm01.exodus.net (209.185.9.246) 12 msec 12 msec 28 msec 5 rsm02-vlan960.wlhm01.exodus.net (216.34.127.66) 8 msec 8 msec 12 msec 6 209.67.227.158 28 msec 8 msec 8 msec 7 www.waltham.northernlight.com (216.34.102.230) 12 msec 12 msec 24 msec

bbr01.hrnd01>trace 216.34.102.230

Type escape sequence to abort. Tracing the route to www.waltham.northernlight.com (216.34.102.230)

1 bbr01-p5-0.jrcy01.exodus.net (209.185.249.213) 4 msec 8 msec 4 msec 2 bbr02-p5-0.wlhm01.exodus.net (216.32.132.50) 12 msec 8 msec 8 msec 3 dcr03-g2-0.wlhm01.exodus.net (209.185.9.246) 8 msec 12 msec 12 msec 4 rsm02-vlan960.wlhm01.exodus.net (216.34.127.66) 12 msec 8 msec 12 msec 5 209.67.227.158 12 msec 12 msec 12 msec 6 www.waltham.northernlight.com (216.34.102.230) 12 msec 12 msec 8 msec

-- (support@exodus.net), January 01, 2000.


This is getting more interesting.

Let me tell you what I am trying to do. I volunteered to work as a telecommunications analyst on the GICC site. I am supposed to check out and verify telecommunications reports. The report titled, "Cell Phones go down in South Africa?" was my target.

South Africa uses the GSM mobile telephone system. I have an International GSM phone that works here in the US and also overseas. For years, I have (off-and-on) used an internet site provided by the South African mobile telephone provider to send SMS messages to a GSM phone.

I thought if I could log onto the South African Mobile Telephone internet site and send myself an SMS message and receive it in the United States, that would be a reasonable test of the computer system of the South African Telephone Service because their computer system would need to be working to send the message whether it came from their mobile telephone subscriber of from their internet site.

I forgot the URL to that site and needed to find it.

Previously, the search engine "Altavista" allowed me to do a search for "+url:.za +gsm +sms" which means I want to find a site with a URL containing ".za" (because ZA is the country code for South Africa) and "gsm" and "sms". Well, a few months ago, Altavista changed their search engine and it no longer accepts requests for a partial URL. I remembered that NorthernLight accepts a URL: request, so I tried them and could not connect.

Eventually, I found a search engine (hotbot.com) that let me do a search by partial url and found what I was looking for: http://www.mtn.co.za/sms/welcome.html

Following the trail over to their new SMS site, I checked to see what GSM providers in the United States they could send SMS messages to (I used to subscribe to SprintSpectrum, but they abandoned GSM in favor of CDMA with SprintPCS, so I now have a new GSM provider -- PCSONE and wanted to see if they were served by www.mtn.co.za. PCSONE is affiliated with Omnipoint and I clicked on www.mtn.co.za 's link to omnipoint but could not connect. I then directly typed "www.omnipoint.com" into my browser. No connection to Omnipoint. I tried another provider -- good connection.

Now at this point, I have two sites I can not reach through BellAtlantic.net -- www.northernlight.com and www.omnipoint.com. I can reach them through other providers.

Yet, when I tracerout to northernlight, the failure is in exodus.net and when I traceroute to omnipoint, the failure is at 151.200.4.66 which is part of bellatlantic's connection.

I spend hours each day on the Internet using BellAtlantic. I have not seen failures like this before. Now, I must admit I am paying more attention to failures I might have brushed off in the past, but I am also trying to be fair. I am getting more failures than usual today, but two failures is still not a significant number.

So, I am not concerned by two independent failures or by the fact that Exodus.net has a failure. But these are things I usually notice without calling anyone's attention to. Is this an indicator of hypersensitivity or is it an indicator of potential problems?

-- Ray Strackbein (Ray@Strackbein.com), January 01, 2000.


Jen Bunker just sent out an e-mail to us GICC analysts saying,

"Yesterday I received some complaints that the GICC wasn't assessable (from NZ). Today, I can't get it. Anybody else having troubles? Please respond so that I can narrow down the problem quickly - I have a lot to post.

Thanks you!

Jen B."

Well, here is an update on the above problems I am experiencing. I can still not get www.omnipoint.com or www.northernlight.com from my computer through my ADSL line. Almost everything else works fine. I CAN reach the sites when I dialup using the same computer and ISP. I fail on the ADSL line on those sites when I use Sally's computer. BellAtlantic (my ISP) asks if I am still having the problems and I keep telling them I am.

As the days drag on, I am finding about one more website each day that I can not reach through ADSL, but can when I dialup.

My traceroutes (provided above) indicate that one problem is in a router belonging to exodus.net. However, when I contacted exodus.net, they posted their reply above showing traceroutes proving packets are moving through their router just fine.

I am combining Jen's report and my experiences to conclude there is sand in the Internet machinery.

-- Ray Strackbein (Ray@strackbein.com), January 06, 2000.



Moderation questions? read the FAQ