A Y2K Success Story

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

The following story is true. The names have been changed for the privacy of the parties involved. The author is a longtime consultant who works with Macs, Novell, and more and has over a dozen years field experience. An employee and I just finished a trip to a steel rolling firm on Y2K issues. The facility had a surprising dual network. Most of the heavy equipment is controlled by IBM and ASA minicomputer systems, but two parts of the internal network stood out. First, all of the job costing and job control equipment is handled by 26 Intel boxes, ranging from 486 to 686 machines. The really interesting part is that these machines are all running Windows NT of various flavors, and they cannot be shutdown. Over the past seven years, their previous consultants have been adding NT servers to the primary network when projects became too complicated to be handled by one machine - the company was being held hostage by these 26 machines. If one goes down, the entire facility has to be shutdown and restarted - a period of at least three days. When the failure occurs, they always lose data. They have 61 separate active databases that are shared between machines. When the machines are restarted, they take at least 24 hours to recompile and rebuild their collective databases. They routinely corrupted the last three or four complete jobs. Since a job takes them nearly three weeks to complete, any jobs not finished were lost since the last NT determined close and sync of the databases. A job usually consisted of at least six stages, which means they'd have to revert and reenter the previous stage. This is why they were pulling their hair out and cussing their other consultant. Our job was to replace all of this equipment - none of it is Y2K compliant, and the database was going to fail in 2000 as well. Second, the entire accounting and billing department is Macintosh. The company has 37 Macintosh stations, subnetworked into four different office areas. It contains all of their billing and invoicing information, as well as all accounting information since 1977. The database software they used previously converted into FileMaker 2.0 quite easily in 1992. They've been running solidly off of a Mac II file server, replacing hard drives as they outgrew them, since 1990. The Mac II got it's job information from a Novell connection to the NT servers. They were working on mainframe in 1977, before Macintoshes. They switched to Macs for accounting and billing with FileMaker and ClarisWorks back in 1988, but didn't convert the 1977-1988 data into their system until 1992. They had no trouble doing it when they simply sat down and did it. This kind of explains why they had so much faith in the FileMaker software - there were no arguments, since they'd "converted" from other database formats before. They had no fear of what I intended to do next. The reason the old consulting firm was fired was because they informed this company that their entire investment in computer hardware would have to be replaced for Y2K - at the cost of 1.7 million dollars. Additionally, they would have to switch their accounting software to NT to meet Y2K requirements. Given their track record in service, the company dropped these consultants, and sought outside contracts. To give an example of what we've been able to accomplish in three weeks: The 61 part database was shared between the NT servers in tab-delimited format. We converted all the sections of the database into FileMaker Pro 3.0 format, kept the same names, and matched the screen format. We run FileMaker Pro 3.0 Server on three spare 68040 machines at their office. (FileMaker Pro 3.0 Server is 68k, not PPC, and has the ability to share 100 databases. However, to keep any computer from becoming swamped, the databases were split among three servers.) These 040 machines are all five to six year old Quadra 650's with 128 MB of memory. We installed WebStar and FileMaker Tab plug ins on a five year old Power Mac 7100/66 with 128 MB of memory, had WebStar link to all of these shared databases, and have reduced the 26 NT machines to four AMD K6-400 machines (machines that strictly handle Novell job costing and job control data From the IBM and ASA machines). The Mac II is now a tape backup server. It was replaced by the FileMaker servers and the WebStar server. Administration has access to all of their databases through Internet Explorer on their subnets. The entire office runs off of web browsers, and even with the confusion, we've gotten nothing but praise and pats on the back. These people love their Macintoshes! And boy are they happy they can keep them. Last Saturday morning, the company shut down the entire facility. We switched off the 26 NT machines and fired up our Macintosh equipment. The owner had secretly made one bogus transaction that morning and announced proudly two hours later (that's all it took us to come back online - a far cry from three days) that his bogus transaction was in the database. Sneaky. Monday morning, after we were sure that all systems were go, we moved the clocks on all machines to 11 p.m. 12/31/99, and we waited. By noon, we found three glitches: 1. Keep It Up, used on the servers to automatically reboot and script load FileMaker Pro Server and the databases, was not registered and had expired. (Whoops) 2. All transactions before 1990 were now newer than 00, so 00 transactions sorted in the middle of the page instead of the top. We went into the FileMaker database and changed the function - this will be changed to support 4 digit numbers in January 2000. (Well, we expected something trivial like this) 3. The Mac II (which is now simply a tape backup server) taxed our network greatly for several hours. We'd noticed the heavy network traffic early on, but didn't think much until the Mac ejected the backup tape. Seems the false date convinced the Mac II that Retrospect hadn't been run in several months - so it started doing it's monthly full backup, plus dailies, plus the year end. (Stupid whoops

-- Cherri (sams@brigadoon.com), August 08, 1999

Answers

Cheri,

Sounds like a job well done! Now, where are you gonna' deposit that big check for services rendered??? Any chance you'll be able to pull out more than a "C" note a day??? MAybe now that you have credibility you can ask them to pay you in CASH, unless you and your crew could use some rolled steel sheet product, say for insulating your house against unwelcomed looters!!!! Microsoft and IBM will definitely not be inviting YOU to there Christmas party this year!!! Did you remember to mention that all they need now is a good power supply into the building??? OOPS!!!

-- just wonderin' (take the money @run.com), August 08, 1999.


Thanks for the story, Cherri. There was another in the mainstream press not so long ago where the inhouse folks remediated the entire system for $140,000 where the big consulting firm had estimated $2 million. It was a municipality, and I do believe the inhouse folks had the job done before the consulting firm's estimate of appraisal.

-- Anita (spoonera@msn.com), August 08, 1999.

And the moral of the story is: Macs RULE !!!!

And we are just about, this very hour, to load Retrospect onto our iMac. Retrospect came gratis with the Imation SuperDisk Drive we bought to externally store and back-up our ancient LC (works like a dream) and iMac hard drive so we can finally upgrade everything.

BTW, upgrading Apple OS + software from their Web Site is a breeze;
the same cannot be said for MicroShaft products. We're tearing our hair out trying to save Outlook Express eMails -- it doesn't seem possible, and the "Help" blurb links to a wrong info category.
Going to have to blood-pressure thru MS's voice mail to plead for the most basic help from one of their techies -- how to save over 6,000 eMails so the upgrade does not erase them.

If only Apple produced *everything* necessary for computing. They've streamlined to the point we have to buy from outside vendors :-(
Canon just came out with a USB color printer/ color photocopier/ fax/ color scanner all-in-one :-) Lots of peripherals blossoming on the elegant Mac vine.

And all Y2K compliant, no issues, hassles, headaches, nightmares, or patch indigestion.

Macs RULE !!

@}->-- 3~0 3~0 3~0 3~0 3~0 3~0 3~0 @}->-- 3~0 3~

-- Ashton & Leska in Cascadia (allaha@earthlink.net), August 08, 1999.


No, the moral of this story is that Cherri STILL does not know what paragraphs are. Gawd what a mess!

-- King of Spain (madrid@aol.com), August 08, 1999.

King, it may be hard to read but the moral is bright as sunshine ;-D Macs RULE !!!!

-- Ashton & Leska in Cascadia (allaha@earthlink.net), August 08, 1999.


The actual moral of the story is that many small companies depend on a complex rats-nest of obsolete hardware and backlevel software. That they are so complacent that they'll run the stuff despite serious repeated failures. And that many consultants are so incompetant and expensive that the customer cannot afford to upgrade to a more sensible computing environment.

Congrats on your success, but if this is the typical customer installation, we're in for quite a ride.

-- Michael Goodfellow (mgoodfel@best.com), August 08, 1999.


Cherri...

Your story is Great News... glad to see the other 6 plants will be remediated before the rollover.

I'll bet a Rolling Steel Mill is just loaded with embedded systems to operate and monitor all that Heavy Machinery you mentioned, eh?

So where are you at with that part of the project? How many embedded systems did you inventory? What percentage needs to replaced? What is the current availability of replacement parts?

I'll bet it takes quite a SCADA program to interprete all that data coming in from all that Heavy Machinery.... How's that part of your project coming along?

Your true story sounds too good to be true....just wondering about the small details...

-- Nailbender (dont_work@midnight.net), August 08, 1999.


Nailbender:

I believe at the beginning of the post, Cherri provided information about the author. This was a story shared with her...not a project in which she herself engaged.

Spain:

I'd bet a lot of formatting got lost in the copy and paste. I've had the same problems when copying and pasting from E-mail to an HTML- formatted forum. It all looks fine until it's reformatted after hitting the SUBMIT button.

-- Anita (spoonera@msn.com), August 09, 1999.


Anita,

That is what happened. I forgot this forum is not "text unless HTML is selected".

-- Cherri (sams@brigadoon.com), August 09, 1999.


Moderation questions? read the FAQ