Learn from a Disaster

This article comes from having been in the IT business, in one form or another, since 1974.  In 1985, I went from buying technology to selling it, or consulting on the use of it.  I think it's fair to say that I've seen just about every mistake you can possibly make when it comes to acquiring information technology.  And, as a sales rep, I'm not particularly proud of the fact that I've helped customers make some spectacularly bad decisions.  Although I did enjoy the commission checks. 

When you buy a network, install an accounting system, build a Web site, implement a customer relationship management system, or upgrade your PC's, there are a whole bunch of mistakes that you and other business owners and managers can make on a daily basis.  In this story, I'll discuss many of them, and the things that should have been done to avoid disaster.

The following story is NOT true.  First of all, one my favorite mottoes is that you never want to let the facts stand in the way of a good story.  Secondly, I've never met anyone who made every one of these mistakes in one project, although I have to say that some folks have come close.  So I've combined a few cases into one monumentally fouled-up project.

Let me provide you with the cast of characters and the setting.  Boris is our hapless business owner.  He owns a company by the name of Mooseensquirrel Enterprises.  He has a friend across the street who owns a business named Upsidasium Chemical.  Her name is Natasha.  They both belong to the chamber of commerce, are on a couple of committees together, and while their businesses don't compete, they both sell to other businesses, have job shops, and are about the same size.  Boris makes hushabooms.

Boris has been using a network-based accounting system for a couple of years now.  It has given him the basic information that he felt was necessary.  He got job cost information, accounting, billing, general ledger, payroll and everything else that he needed.  But the system was getting old.  It was doing the job, but the computers were getting dirty, it didn't use Windows, and he found that all of the other guys at the club had newer systems.

Natasha, on the other hand, is on the bleeding edge, and loving it.  She recently installed a new system.  The system promises to integrate her job shop operations, billing, inventory control, purchasing, payroll, AP, AR, marketing database, and general ledger into one integrated system.  It's important to note, if you didn't notice it, that the system "promises" to do all of these wonderful things. 

Natasha bragged about this for a couple of months and Boris finally became convinced that he needed a new system.

Natasha used Rocky's Networks as her computer company when she installed her network.  But she bought the software directly from the software vendor - Frostbite Systems.  Boris went ahead and bought the software from Frostbite (which cost about $20,000) and called up Rocky to have him put the network together.  He also called a couple of other network dealers and asked for quotes from them as well, sending everyone the configuration information that he got from Nell, the rep at Frostbite.  He had to keep Rocky honest, didn't he?

Since Rocky was the cheapest bidder, by a large margin, he won the project.  Boris got a file server, a UNIX database server, new workstations, and a new 100MBPS Ethernet network.  In addition, since Boris wanted to link in his sales office in Pottsylvania, he ordered two routers, a second server and more workstations for that location.  He wanted to be able to access the system from his house, so he also ordered a new computer for his den and an ISDN line.  And based on Rocky's recommendation, he bought the Grimalaya 2000 Office Suite.

The total price tag of the network was about $50,000.

When the day came to install the network, Rocky's tech Dudley  arrived.  After he got everything assembled, he realized that they forgot to include an un-interruptible power supply for the server.  He went to Boris, who, after sighing, OK'd the additional $500 expenditure, and Rocky ordered one from the distributor. 

Three days later, Dudley came back, set up the UPS and plugged the network in.  The main breaker failed, shutting down half the office and crashing the server on the old network, which was still running the accounting system.  After trying a few more times, Dudley realized that there weren't enough amps on the circuit to support the new network. 

This took more time, since the electrician was booked solid.  But that was OK since Boris had to clean up his accounting system from the server crash.  After three weeks, the new circuit was installed, and Dudley came out to try to set up the network again.  This time the LAN worked just fine, except that Dudley was having trouble with the frame-relay connection to Pottsylvania and the ISDN line to Boris' house.  Neither was working right and Boris was racking up more consulting costs while Dudley tried to configure the routers and the data service.  Frankly, Boris wasn't happy with Dudley.  He was annoying, didn't seem to be able to communicate with members of his own species, seemed kind of dopey, and scared the women in the office. 

If only Boris had interviewed Dudley before the project began.

It finally dawned on Dudley that they were using the wrong routers and that the ones that would actually work would cost Boris another $3,000.  So far, Boris had had to buy a UPS and different routers in addition to the original network cost.

If only Boris had asked why Rocky was so much lower than the bids from the other guys.

While Dudley and Rocky were struggling with the data lines, Boris went ahead and called Frostbite to have them send out an engineer.  But his rep, Nell, said that they shouldn't send out Bill Winkle, the installation engineer from Frostbite Systems, until they had the wide area network up and running.  Otherwise, Bill would have to make two trips.

Boris started getting a little upset about all the delays.  He was sitting on over $70,000 of software and hardware and wasn't getting any use out of it whatsoever.

Finally, Bill Winkle showed up and looked at the software that was sent to Boris.  It turns out that it was the wrong version for the machine that Boris had bought.  This didn't cost him any extra, but Bill had to go back to his office and download the version that he needed.  This took a day, but Boris wasn't going to be available the next day, so they had to reschedule for the following week.

After two weeks of downloads, patching, installation goofs, re-configurations, etc., Bill Winkle finally got the software installed on the network.  He told Boris that the problem was the wide area connection.  "You really should have held off on installing the WAN until we had the basic system up first.  Then we could have simply reconfigured to deal with the WAN."

Boris said, "But your rep, Nell, said that we should hold off, which is what we did."

Bill Winkle replied that reps never knew anything and he should have been involved earlier in the project.

If only Boris had kept it simple and installed only the local version first.  And if he had only talked to a tech instead of Nell.

Then another problem came up.  The tape backup software wouldn't work with the operating system that Rocky included with the network.  Bill told Boris, "You'd better not use the system until we can get it backed up."  Boris called Rocky, in something of a huff, and demanded that he get the correct tape backup software.  Rocky ordered new software, which took a week because it was such an unusual product that the distributor had to special order it.

If only Boris had used a check list. 

Finally, almost three months after Boris placed the order, the new software was loaded, the new network was up and running, and Boris was ready to begin work on the new system.  But when Boris started it up, he realized that the system was more complicated and harder to learn than Natasha had said.  The manual was poorly written, and not very professionally produced.  And the version number shown on the manual wasn't the same as what popped up on his computer screen.  He also discovered that the new software wouldn't import data from his old system - he'd have to re-enter it.  He figured it would take two temps about three weeks to convert the data.

If only Boris had considered the impact of the new system.

After Boris spent a couple of 80 hour weeks learning the new system, and running his business at the same time, he got a new CD in the mail from Frostbite.  He installed it, and when he started it up, discovered that all of the settings that he had laboriously entered, as well as the menus and appearance of the screens, had changed.  He called Frostbite (he had memorized the number by now) and in the course of the conversation realized that he and Natasha were using BETA software! 

Boris immediately called Natasha and asked her what the heck beta software was. 

"Boris, my dear, beta software is stuff that the company has not fully tested, and isn't generally released." 

"But why are you using untested software?  Don't you have a business to run?  I know I do."

"But the only way to get the latest, coolest features is to use beta software.  And I don't care about a few glitches.  And I've learned a lot about UNIX system administration."

Boris should have checked more references.

At this point, Boris calls Rocky. "Rocky, you know, I've got this network.  Do you know of any other software like Frostbite that I can get instead?  I'm just not sure about these guys." 

"Nope.  There's nothing else out there.  Dudley and I looked when you started having all those problems.  There's a lot of stuff out there that runs on NT and NetWare, but that's a special version of UNIX, and we couldn't find anything else.  Of course, you could hire a programmer and have them write some custom software..."

If only Boris had kept his options open.  He should have shopped for different solutions by specifying his goal, and letting the vendors recommend their preferred approach.

During his conversations with Frostbite, Boris noticed that he always talked to the same tech support guy, same programmer and same administrative person.  One day, while chatting with Natasha, he asked her about the size of the company.  Natasha said that there were 3 programmers, 2 tech support people, and 2 secretaries, in addition to the president.  She also allowed as how she was a little nervous about the company, since when she was out there for a users conference at the Budget Inn, she noticed that the offices were pretty shabby.  But she had gotten the code placed in escrow, so she wasn't too worried.

Boris should have met the president and visited Frostbite's offices.

After a couple more months, Boris was still working 80-hour weeks, but he could do nothing more than print work orders.  He couldn't get more than that done because his employees didn't know how and he didn't have time to show them.  Besides, they were also trying to figure out the Grimalaya 2000 Office Suite software.  Boris had to manage the network and the Frostbite systems himself.  Both Nell and Rocky offered to train Boris's staff on the software and the network, but Boris didn't want to spend the money.  And Natasha had told him stories about training her own system administrators, only to have them leave for better jobs after they got their certification.

Boris should have trained his staff if he wanted to get full functionality from the system.

Several months went by and Boris was further along.  He finally got some training for himself and his staff.  They actually had most of the modules installed and could do everything from scanning bar codes, to generating purchase releases, to printing invoices.  Now he was ready to set up his ecommerce site. 

One of the main selling points of the system was that he would be able to let his customers place orders, release shipments, check inventory levels, and check billing, all from their Web browsers.  Boris started looking through the menus on his software to find the ecommerce option.  He couldn't find it.  He looked through his manuals.  He still couldn't find it.  He called Nell.  "What about the ecommerce piece?"  Nell hesitated for a moment, then said she'd check and call him back. 

After a couple of hours, Nell called back and said, "that ecommerce module will cost you $5000.  Would you like me to send it to you?"

Boris exploded!  "You told me that this was included when I bought the system.  I specifically remember your saying that."

Nell said that she was sorry, she didn't remember that particular conversation and she couldn't possibly have said it, since they all knew that ecommerce was a separate module.  Boris called Mr. Bigg, the owner of Frostbite Systems and read him the riot act.  Eventually, Mr. Bigg offered to sell Boris the software for half price.

Boris should have documented everything, particularly functionality that was really important to the decision.

Nell called after Boris got the software and asked if he wanted her to send Sherman, their Web developer, out to help with the new module.  Boris refused, saying that his nephew Peabody was between his junior and senior years at college, had been studying computers, and wanted to try to set it up. 

In July, Peabody started working on the ecommerce piece.  Within days, he announced that Boris would have to upgrade his connection to the Internet, buy a new and different UNIX box, and get a graphics workstation for Peabody so he could create some cool images.  Boris was getting tired of writing checks, but was finally getting excited about the system's impact on his sales.  He had told some of his best customers about the new software and they were excited about being able to do business with Boris over the Web.  They told Boris that they would probably double their orders once they could do it online.  Boris's mouth was watering.  He was already planning his trip to the BMW dealer.

Peabody was still working on the project in August.  Boris was getting a little nervous.  He knew that Peabody went back to school in mid-September, and he was afraid Peabody wouldn't get it done in time.

Finally, just after Labor Day, Peabody walked into Boris's office and announced that he had the ecommerce system up and running.  Boris announced to his customers that the system was ready and they could start using it.  After about two weeks, things started to happen.  Orders were getting lost, shipments were getting delayed, billing was being duplicated and Boris's customers were getting upset.  A third of them were fed up with all the delays and foul-ups over the last few months and started ordering from Boris's biggest competitor.  Peabody was back at school and his roommate never passed along Boris's frantic messages about the problems.

After three weeks, Boris finally called Nell, who sent Sherman out to have a look at the ecommerce system.  "Well of course it won't work," he declared.  "Peabody, what a dopey name, forgot to use the overhead poly-transit.  If you don't have that turned on, nothing is going to work right.  Didn't he read the manual?  It clearly says that in the third footnote on page 234."

Boris should have used professionals. 

Six months later, everything is finally working well.  Customers, the ones that are left, are submitting orders.  Boris finally has time to make sales calls on the accounts who left six months earlier.  The bank accepted his personal guarantee on the refinanced loan.  His wife left him, but he likes eating Chinese food every night anyway.  And this latest office manager really seems to be working out.  Which is a good thing, because he can't imagine trying to hire another one so soon after the other five.

Suddenly, the supervisor on the third shift calls.  "Boss, the system's down!"

"What do you mean, the system's down?" 

"I can't access the rush orders that came in last night and I can't print bills of lading for the stuff that the shipping company will pick up in the morning.  Nothing works.  All I get is a frozen screen." 

"Did you reboot?" 

"Yeah." 

"Did you check to see if the overhead poly-transit is installed?" 
"Yeah.  It's loaded and the settings are all correct."

"Did you check to see if the third line of the autoexec file is set to 'no'?" 

"Yeah.  I did all that stuff, Boss."

"OK.  I'll call Rocky."

Boris gets Rocky out of bed, who gets Dudley out of bed.  Dudley goes down to the office and stares at the network for a while and sees that everything's OK.  By this time, Boris comes in, takes a swig of Pepto Bismol (he has the economy size sitting on his credenza) and says, "What do you think?" 

"I think the network is fine, Boris.  You better call Frostbite Systems."

Boris calls Frostbite.  All he gets is the company voicemail.  He calls again an hour later.  Still no response.  At 8am, he calls and gets the receptionist.  She says that all of the techs are in a meeting, but she'll see if she can get one of them. 

30 minutes later, the freight truck has come and gone, without any of the shipments for the day.  An hour later, a tech from Frostbite calls.  He and Boris talk for a while and he can't figure out what the problem is.  He says he'll call back.  Boris frantically calls again in a couple of hours.

"Well, we don't know for sure.  I'll have one of our senior techs call when he's off the phone with one of our premium customers."

Boris asks, "What the heck is a premium customer?" 

"Oh, they're customers who pay an annual fee for higher response technical support." 

Boris screams, "Wait a minute, I'm a beta customer.  Why don't I get that kind of support?  Natasha over at Upsidasium gets fast support!"

"I'm sorry sir.  I don't make the rules.  Besides, Upsidasium is one of our premium customers."

After sitting in his office in stunned silence for 10 minutes, 
Boris calls Mr. Bigg and raises hell again.  Mr. Bigg tells the senior tech to call Boris back as soon as he can.  But the tech isn't able to do so until 8am the next day.  Now Boris has missed two days of shipments. 

"Sorry," the tech says, "I had other premium customers to deal with.  Now, how can I help you?"

Boris reaches for the Extra Strength Excedrin, and flips through that militia manual on blowing stuff up.

Boris should have asked them what their tech support programs were in the event of failure.

And he should never, ever have taken Natasha's advice.

By the way, what happened to Natasha?  She sold her business at a loss and started up an ecommerce consulting firm.

~~~

Pretty horrible, eh?  While Boris would probably have overdosed on aspirin a long time ago, you and I have the benefit of learning from his experiences.  There are some key things to remember. 

First of all, plan the project thoroughly.  Don't stumble into it.  Use checklists, look at the impact of the project, document everything, prepare for disaster and keep it simple. 

Be very careful about the people you work with.  Don't take advice from unqualified people.  Use professionals, meet the owners and tour the offices of your vendors, meet the technical staff you'll be using, check references, shop around for the best solution, and watch those prices.

Make sure that you formally train yourself and your staff, and document everything.

But above all else, plan.  Every disaster at Mooseensquirrel Enterprises could have been avoided with careful, exhaustive planning, incorporating the techniques that we've discussed.

This is a terrible example of how to manage an IT project.  But it can be a good learning tool for you the next time someone says that you need more technology. 

Technology WILL make you more profitable.  But technology, improperly purchased and installed, can suck the life out of you, your staff, and your company.  Only by following a structured acquisition process can you reasonably assure yourself of a successful IT project.

Copyright MMXIV The Gadwall Group, Ltd.  All Rights Reserved
     Copyright and Trademark Information
Contact us at 224-325-5590 or info1007@takechargeseminars.com and please make sure "inquiry" is in the subject