short recovery period from Y2K meltdown?

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

I can understand the position of Y2K optimists that say that there really is no big problem -- embedded systems problems are just hype, the electricity will stay on, telecommunications will work, and software can be fixed pretty-darn-quick. I can understand the position of (and agree with) the folks that believe there will be a complete meltdown -- the technology goes down, and stays down, period. Its those "in between" positions that I find hard to make sense of: everything will be down initially, but then be up again in a relatively short period of weeks. I mean, if you accept the premise of multiple simultaneous failures of technology and the societal fallout that would occur, what "fix-it" army (for lack of a better description) is going to be able to put humpty-dumpty back together again? of weeks

-- Joe (shar@pei.com), September 14, 1998

Answers

It's a big unknown.

Electricity utilities (for example) have a lot of experience of lashing things up so they'll work again, after hurricanes, earthquakes, and (in parts of Europe) war damage. They've never failed yet. Also the inbetween position isn't everything-down for N weeks, then suddenly AOK; it's a serious reduction in capacity (maybe to zero in some places but not everywhere) followed by progressive repairs, most essential things first. Some power plants have been Y2K-tested, either passing or suffering only transient troubles, so one can be pretty certain that some parts will still be working.

On the other hand, Y2K is different and unprecedented; so we don't know anything from experience, especially not about the societal domino effects. Wartime experiences are probably closest and allow for some optimism (at least judging from the UK's WWII experience).

-- Nigel Arnot (nra@maxwell.ph.kcl.ac.uk), September 14, 1998.


Nigel, yousaid: " Some power plants have been Y2K-tested, either passing or suffering only transient troubles, so one can be pretty certain that some parts will still be working. "

Can you provide the source of this information?

Bertin

-- Bertin Opus (third@hotmail.com), September 14, 1998.


I'm currently thinking that the technical problems will be solved in a matter of months. Rebuilding the economy will take years. Remember, the market volatility we've been experiencing is just the psychological reaction to Asia and Russia's fall. They haven't hit the ground yet, and we haven't felt the full impact of being over-reliant on global trade. Even without Y2k, we're looking at a 30's-style collapse. With Y2k, we are in for much worse, and it will take longer to recover.

My "months" prediction means some power, some telecommunications, some goods being transported. Volume and efficiency will drop, and in the meantime, people will die. England is openly preparing for martial law, fuel and food distribution by the armed forces. That will happen here, but they can't do it everywhere, efficiently. People will starve, disease will spread. Systems will come back on line, but it will be a different world.

E

-- E. Coli (nunayo@beeswax.com), September 14, 1998.


Hear, hear, excellent point Nigel. I raise a glass of the Uncle's finest dehydrated brew to your judgement.

I'm trying to get more specific information about power and power options (including nuclear) directly from the regulators and operators, but don't have it yet. It is the crux of the problem - as noted before, it is difficult to troubleshoot programs and fix bugs when the alligators are biting.

I think embedded chips in control systems of the distribution system will be the limiting factor. Or the control (satellites and phones) of the distribution system itself.

-- Robert A. Cook, P.E. (Kennesaw, GA) (cook.r@csaatl.com), September 14, 1998.


The tested power station story was reported on the comp.software.year2000 group quite some time ago (before it got hijacked by survivalists). It was a report of an EPRI embedded systems workshop.

You'll find references to it in http://www.year2000.com/archive/NFaction.html though with an alarmist slant rather than a reassuring one. (For me the key point is not how bad it'll be if it failed, but that someone actually got around to testing it a goodly while ago).

EPRI is at http://www.epri.com , but the site is mosty closed to outsiders. A shame, but if it helps get the lawyers out of the engineers' hair, it's all to the good.

A key point about embedded systems in electricity systems is that they **ought** to be "timeless" in that it doesn't matter what time of day or week it is, the action they take in response to a particular condition on the equipment they are controlling is the same. Y2K-bugs will only arise where the **ought*** was disregarded (perhaps by using a realtime clock to calculate time intervals), or where an un-used RTC contains a hardware bug that can crash the rest of the system.

Both are known, but the former usually gives rise to a transient problem for one time interval at what the system thinks is the start of 1/1/2000, and the latter is rare.

-- Nigel Arnot (nra@maxwell.ph.kcl.ac.uk), September 15, 1998.



Nuclear plants are quite another issue! They are designed to fail safe, and fail safe is almost certainly what they'll do. (I sure *hope* so). Trouble is, safe means OFF, and manually overriding a nuke is probably neither a good idea nor legal.

I think that there will be major power shortages. Be warned, in those circumstances low-priority customers are scheduled to be disconnected, or rationed by rotating power cuts. Residential counts as low priority (here in the UK during industrial disputes, also recently in Canada during the ice storm).

-- Nigel Arnot (nra@maxwell.ph.kcl.ac.uk), September 15, 1998.


I have learned a lot of the electrical system through the daily e-mail update from Westergaard 2000. The information has been somewhat reassuring from an electrical standpoint. I believe that Dick Mills is the expert who frequently writes columns regarding the power situation. He frequently writes in response to comments/questions raised similar to many threads on this forum.

-- bhayes (OKLA) (bkhayes@intellex.com), September 15, 1998.

I see the following factors affecting the rapidity of recovery from possible Y2k meltdown, at least in the U.S. and most developed countries:

(1) Y2k is different from other disasters such as war, hurricane, and earthquake in that the physical plant will not be damaged - the power plant will still exist, the power and communications lines will not be broken, the factories, transportation system, offices and homes will still be standing. So there won't be a need for a huge reconstruction effort.

(2) There is a fairly large cadre of IT professionals who could work on Y2k remediation but who are not currently willing to work on it (for career or compensation reasons) or are not working on it because their employers are insufficiently concerned about the problem. If the situation really goes into meltdown, virtually every programmer will end up working on fixing the problem, because its priority both to employers and to IT professionals will become extreme. I would be very surprised if as many as 10% of IT professionals are currently involved in Y2k remediation (I'm open to being challenged on this guess).

(3) IMHO, there is enough portable electrical generation capability in the U.S. to power most of the computers that need to be running during the post-2000 remediation push.

IMHO, we can have a very bad few months but reasonably rapid recovery to at least a sustainable level (avoidance of starvation or freezing for the general populace) if not back to a decent economy within six months to a year. I also assume that some won't agree with this position! <<<<<<<<

-- Dan Hunt (dhunt@hostscorp.com), September 15, 1998.


Actually, Dan, I agree with most of what you said. The problem is that during that really bad "few months", how many will starve or freeze? I think it will take quite a while for the economy to "make a comeback", though. My husband is a programmer for a "Major Computer/Consulting Firm", (I hate not being able to use names of companys and banks, but I have used my REAL name) and he is NOT currently working on Y2K remediation. My guess? He is making lots of money for the company with what he IS doing. Also, they just gave him his 2nd 8% raise in a year. I think they want him to stay right where he is. (I'm not complaining- more money to prepare!) I "discussed" this with Mr. Yourdon after I read his article "Y2K Survivalists, Safe Havens and Bugging Out." PS- my closest neighbor is a Y2K programmer for Amoco. When I first asked him what he thought was going to happen, he said, "I have 3 months worth of food, and 3,000 rounds of ammunition. Does that answer your question?" It did!

-- Gayla Dunbar (privacy@please.com), September 15, 1998.

Please remember that Y2K is going to hit in the middle of the winter. In the northern climates it takes 24 hours or less without heat (aka electricity to run the blower of the gas furnace) before water pipes start to freeze. If the power was off for *just a few days*, there would be major damage to the infrastruture of all residences in the Northern 2/3 of the U.S.

-- Carolyn Hoagland (choagland@connectec.com), September 15, 1998.


>Please remember that Y2K is going to hit in the middle of the winter. In the northern climates it takes 24 hours or less without heat (aka electricity to run the blower of the gas furnace) before water pipes start to freeze<

Our woodstove heats our entire house. Haven't used our furnace for years.

-- Woodstove (winners@magiclink.com), September 16, 1998.


Dug around in the attic and finally found my old "Magic 8 Ball".

Will the Y2K meltdown have a short recovery?

"Try again"

Will there be long lasting distruptions?

"Try again"

Are we doomed?

"That depends"

Should I be ready for the worst?

"Yes"

There you have it folks, class A prognostication by an expert.

-- Uncle Deedah (oncebitten@twiceshy.com), September 16, 1998.


Depends on what you mean by recovery. I think the power will stay on outside of NYC and California. NYC because the city plays too many games with Con Ed, and California because the state has pretty much banned regular power generation. Those long lines from other states (the ultimate NIMBY syndrome) will probably give some trouble leading to brownouts and blackouts in CA. If the power stays on the water will stay on. Fuel will continue to flow and food will reach the cities. As to economic effects - too much depends on other countries to give any kind of concrete answer. My own best guess? - I think stocks are gonna reach a real low just before 1/1/00 - and then make a short comeback until the realization of the size of the problem in China, Russia, and other countries hits. Then a slow rise on the discovery that programmers from the US are in demand everywhere since our problems mostly were smaller than other folks. Why? Because we are trying to fix the problem - some countries aren't even admitting there is a problem!

-- Paul Davis (davisp1953@yahoo.com), September 16, 1998.

Paul, you say "fuel will continue to flow" and then turn around and say the "other countries aren't prepared." Where do you think we get about 55% of our oil?

-- Gayla Dunbar (privacy@please.com), September 16, 1998.

Well Gayla, that 55% comes mostly from Mexico. But I did not say there would not be rationing of some kind (though you should note that the Mexican sites pump using US equipment and US power for the most part) but essential services should survive. Look, this is basically a software/firmware problem, not a problem with someone destroying large amounts of physical plant as in a war or massive earthquake. Fixing the problems that are identified after 1/1/00 will no doubt be a massive job, but you should remember that most industry still can override the processors and run by hand. I worked in the coal mines for several years and I assure you that as of a couple of years ago (when I left) they could have done entirely without the embedded processors and still put out their product. The sampling and quality control would have suffered, but they still would have been able to produce coal.

-- Paul Davis (davisp1953@yahoo.com), September 16, 1998.


Paul,

With all due respect, the issue of doing things "by hand" is a non-answer. There is no such thing as "by hand" anymore. Utility companies tell us they can "go manual" at any moment, but they're lying. The railroads are completely computerized; no informed person will even question that anymore. Nuclear plants are not designed to "go manual". They are designed to shut down.

Telecommunications are necessary for power to continue. Phone switches cannot "go manual". GPS satellites and receivers cannot "go manual". Gas stations cannot "go manual".

The payroll for General Motors cannot "go manual". Medicare cannot "go manual". The military cannot "go manual".

We have spent the past 30 years eliminating "manual" from our civilization. To think that we can turn that around "overnight" is ridiculous.

Let's get serious, folks. If an airport (Seattle International) is not compliant after working on it for 3 years, why should it be compliant within a couple of weeks after the year 2000? The same can be said for a bank, a stock brokerage, a car manufacturer, or any medium sized business, electric utility or phone company. Let's get real.

It's time to get a rod up our backs and realize that with each day that goes by the situation gets more believable. Y2K really is going to happen. Beginning on Jan 1, 1999 and throughout 1999 things are going to break. Slowly at first, and then with a bang on Jan 1, 2000.

They won't all of a sudden get fixed. That's a pipe dream.

And, I'm an optimist. :)

Blessings!

-Pastor Chris

http://www.lifetel.com/y2k2000.htm

-- Pastor Chris (pastorchris@lifetel.com), September 16, 1998.


Well, unlike most programmers and Lan Managers (which I now am) I spent quite a while in 'real' industry. Most manufacturing and production facilities are capable of running without their computer facilities. Some are not, certainly not places that have gone in for robotics (though I have not heard anyone claim that any of the processors embedded in a robot arm even know what the date is or care) but many are. When I worked on the program for controlling chemical feeds for one of the most modern and largest coal washing facilities in the eastern US, we could go 'manual' in about 30 seconds. And did, several times. It isn't cost effective, in fact costs the company several thousand dollars in excess costs per day, but could easily be done. Another embedded machine, a high power (tens of kilowatts) toshiba controller used on a sampling mechanism taking samples from a high speed belt, blew out one day (I was in the next room and thought lightening had struck the building) and we wired a switch around the panel and hit the sample button by hand every 2 minutes until toshiba sent us a replacement. Boring and time consuming - but we could run. Most of the facilities I have worked in had the ability to perform similar work arounds. The industries that don't have this capability are the ones that are trying the hardest to solve their Y2K problems. Communications will have problems, and so will high tech facilities, but the great bulk of our manufacturing base still has the ability to go 'manual'. By the way, GPS receivers don't have a Y2K problem, they actually have a rollover problem that will hit about 140 days earlier. Don't travel the last week of August next year. The Navy has a site up about it. Interesting thing though, one of the GPS satellites that does have a problem is due to be replaced only 3 wks before the old one times out. That is a little short IMHO.

If you really want something to worry about, the bulk of the systems I have seen in industry that might not be able to be bypassed are - SAFETY - systems - stuff OSHA and MESA have required. When they have to bypass this stuff, a lot of folks who have come to rely on it are going to get hurt!

-- Paul Davis (davisp1953@yahoo.com), September 16, 1998.


Paul,

I think that manufacturing will be greatly crippled in the portions of manufacturing that assemble many parts and pieces from many vendors. This is particularly true of the automobile manufacturing industry. This industry has converted over so much to just-in-time parts delivery, and has so many vendors, it has become totally dependent in a very short time frame to each vendor. Twice in the last year or so General Motors has demonstrated how a single vendor or plant not operating can cause the rest of the company to shut down shortly thereafter because there is absolutely no inventory to cushion the effects. Within GM's 85,000 vendors, there have got to be some that will not be operating for some period of time, either because their manufacturing/scheduling systems are screwed up, or because they are shut down by failure of power/phones, etc.

Also, having spent some time at Boeing a few years ago, I know that a company like that cannot "go manual" in their manufacturing assembly process. Each step is so controlled by the computer in providing direction in what should be done next, integrated with making sure the parts are there, all combined with keeping the records required by the FAA, that doing it without the computers in any short-term changeover would be impossible. So their programs will either be operating correctly or the plant will not be operating period.

So I would expect that basic manufacturing of raw materials might be able to "go manual" if their computer controls were unavailable for a few days or weeks, but not the more complex manufacturing assembly types of operations that are higher on the food chain. <<<<<<<<

-- Dan Hunt (dhunt@hostscorp.com), September 16, 1998.


Thanks to all who have (and are) responding to my post. So far, a lot seems to depend on just how feasible it would be to do things manually that are currently done via automation. (Discounting for the moment the human side of the equation -- i.e., the people who would be doing the manual operation, and what personal impact on them there might be.) I am real skeptical here, especially since a lot seems to be based on experience of how things USED to be done, often by people who no longer are there -- unfortunately, that is part of the problem, the people who knew how to do it are gone! I too have read Dick Mills' optimistic Y2K power recovery claims, based on his fond rememberences of how back in the Big Blackout of '65 they burned furniture to fire-up a power plant, showing how when backed into a corner people will indeed come through. But again, so much has changed. (And note that Mills believes it to be essential that manual power plant operation be proofed out and training provided to operators NOW, not later, if this approach has any chance of working.) And, there are all those interdependencies: its great to have manual operation of power plants on a grid, but they need to be able to communicate, which may not be possible, since modern telecommunications requires electricity. And so on ...

-- Joe (shar@pei.com), September 16, 1998.

Chris and Joe

I'm not an extreme optimist about Y2K, I just don't give an economic crisis the same weight as I would give a nuclear war. Sure, if the Saturn plant a couple hundred miles north of me doesn't get their act together they will be down a month or two (minimum) after 1/1/00. But I can live without a new Saturn on demand. Same thing for Communications - I expect a lot of ISP troubles and some local phone service troubles, but not a lot of trouble on the major backbone where most routing equipment that can cause trouble has already been updated or replaced. Quite a number of satellites have been replaced or will be in the next 15 months, and this will keep the bulk of GPS and military communications running. Shucks, I have been working on the Y2K problem here in the Army Corp of Engineers Memphis District for several months now, and we have the largest part of the problem identified and (for the most part) already solved. The rest on our level will be done before 1/1/00. Sure the unexpected will happen - thats life. And I keep wondering if anyone has the authority to check the HVAC computer that keeps the heat and air conditioning running in the Federal Bldg in Memphis ( thats GSA not the Corps ).

But my point is that disruptions in production of new cars and computer parts don't keep us from eating. The basic industries don't rely heavily on computer equipment for production. Distribution will have a special set of problems related to communication. But in a real emergency the National Guard takes over for the duration and will ensure distribution of food and water. BTW - does anyone know if the Allen Bradley PLC has a Y2K problem? The Allen Bradley is the backbone of control equipment all over the US and Canada, and if they quit a lot of simple stuff like traffic lights will stop working right until someone jacks in a laptop and reprograms them.

As for the railroads - I just don't know. They are certainly very reliant on computer scheduling, and how much Y2K work they have done and how much remains to be done - I will just have to wait and see.

Where I wouldn't want to be on 1/1/00. Far north cities, California, or NYC. I think most other places will be OK as far as survival goes.

-- Paul Davis (davisp1953@yahoo.com), September 17, 1998.


Moderation questions? read the FAQ