Worst IT fail ever? US agency spends millions in useless security
By Patrick Lambert in IT Security, From Tech Republic
A branch of the U.S. Commerce Department recently trashed perfectly good computers and wasted millions on a bogus malware infection. How did this happen?
Hackers and malware are everywhere, waiting for you around every corner of the Internet. It’s great to be paranoid as long as you know what you’re doing, but someone with only basic knowledge of IT who browses through the constant security bulletins, security mailing lists, and even their own system logs could be overwhelmed quickly. This is apparently what happened when the US Economic Development Administration (EDA), an offshoot of the US Commerce Department, received a report that claimed there was an infection in its network. Instead of following standard best practices for identifying and cleaning up malware, they decided to “go nuclear.” The agency spent millions and trashed a ridiculous amount of computer equipment to get rid of an infection that did not exist — all of this because of bad communication and poor IT security skills.
How not to react to a malware infection
This event started in December of 2011 when the Commerce Department’s IT team sent a memo to the 170-person crew that heads the EDA, telling them two of their computers were infected by malware. The first memo was vague as to how widespread the malware attack was, and the clarification that came later may not have made it all the way to the people it was meant to reach. Nevertheless, the EDA IT officer decided to go all-out in order to get rid of the infection. They employed the services of four agencies and an outside contractor, and even when they were told the malware was not widespread, they acted anyway.
According to a report from the Inspector General [PDF], the EDA’s chief IT officer decided that the only way to be completely sure that all malware was gone was to burn everything down, literally. The team set out to destroy computers, keyboards, mice, TVs, cameras — about $3 million worth of equipment. They eventually ran out of money, which is when the IT officer went so far as to request another $26 million for further recovery efforts — denied by the Commerce Department.
The office of the Inspector General said that the EDA’s persistent mistaken beliefs had “cost the government an unnecessary expenditure.” Meanwhile, the EDA says that it learned its lesson, and that they had acted in an “abundance of caution.”
The proper way to do things
To many of us, this may seem so ridiculous as to be laughable. But the fact remains that this is something which happened in the US government last year, not an ancient event from a time when malware removal was new, or in some small company without any proper IT crew. The fact that this could happen at all shows not only a lack of training in the actual officer who took the decisions, but also the lack of education that was provided to him along the way. An individual with a laptop who thinks the only way to get rid of a virus is to throw the machine away will likely get educated at a local computer store. But the head of IT at a government agency with millions in technical equipment may be more likely to hide his ignorance.
The proper steps to get rid of malware on an enterprise network don’t require a lot of money or IT crew. When malware is detected, there are many solutions that can be used, most of them the same as what you would use for a home system. If you do have an up-to-date security software then it can go a long way to detect and eradicate the malware. To be sure your network is safe, you can run scanners such as MalwareBytes on the infected systems, and then make sure the troubled computers are clean before allowing them back on the network. If all else fails, reformatting a hard drive and reinstalling the OS from scratch almost always does the trick. If you have a good backup strategy in place, then this type of event should not disrupt your business too much.
This event shows a fundamental flaw in the system – how does someone like that manage to get into such a position? It’s unclear whether the EDA’s IT officer is still employed, but the report refers to him as the current CIO, so it’s a good bet nobody got fired over this, at least no one at the top. Security mailing lists, logs, and messages generated by other security-related controls may be hard to comprehend for those without some basic IT training, and perhaps a manager will think he/she could get by leaving the “technical” stuff to the IT staff. But unfortunately, that clueless manager gets to decide how the money is spent – or in this case, wasted. There was evidence that the CIO ignored the contractor’s advice and went on with the disposal of keyboards and mice anyway.
This whole event is both funny and sad, because it shows how ridiculous some government bureaucracies can be, but at the end of the day, those millions are real tax dollars being wasted. It doesn’t take a whole lot of changes to make sure something like that doesn’t happen again. Any IT group needs strict procedures on what to do when something like that happen, and the right balance must be maintained between reacting quickly, and not doing things that are either useless or worsens the problem. Identifying and removing malware is not a guessing game; there are well known ways to deal with these problems. The fact that this government officer was allowed to make up this ridiculous process shows that the procedures were either not followed, or non-existent. Hopefully, this is also something that’s being corrected.
Have you experienced or heard about a worse example of IT cluelessness? Share it in the discussion!
For more on this story go to:
IMAGE: www.zdnet.com
