In recent discussion on various forums and elsewhere in this blog I’ve raised the points that the way attackers value things and the way defenders things are not the same; their perception of other values, such as business assets, processes and so forth, can be very different from yours. As an extreme example, you may be defending the network and IT assets quite capably while the executives of the company are while gambling and snorting away the company’s bank account. I often point to Enron as a poster-boy here – would exemplary IT security have helped ?
And this is what is wrong – one of the many things wrong – with relying to heavily on the model of the classical risk equation as a basis for risk analysis. Its not that the risk equation is wrong; its that WE DON”T KNOW.
We do know the value to us – on the inside, from our point of view.
We do not know how the attacker viewed things.
Any equation will suffice if the accept the guesswork of the input.
Or as the philosopher Nietzsche said “Any lie will suffice provided everyone believes in it“.
At this level I can see the point of any form of RA, ROI or what have you, if its objective is to present a case to management to get the funding to do the security. I don’t think that’s an ethical or honest approach, but I can imagine that in some organizations, ones where FUD often works, it may be necessary. But if the security practitioners who made this case start believing their own lies then things are in a terrible state.
This isn’t quite the point that Richard Bejtlich is making in this particular blog article, but in other postings he points out that the classical RA methods need more vigour and a more scientific method of justifying their inputs and relationships. He calls this approach FAIR. A great deal of this is based on “Risk Assessment is not Guesswork“.
I’m sorry to say that I have to agree with Richard’s analysis of the ‘simple scenario” which he comments on liberally.
Richard repeatedly brings up the question of how the ‘figures’ and ‘estimates’ and situations are arrived at – his “Says Who?”questions. He also questions the over all absence of hard data and precision.
In one sense that’s fair and in another sense its not. This kind of analysis intrinsically leads itself to speculate about situations where there is no data, where all the input are guesswork. GIGO.
Dwight D. Eisenhower is supposed to have said “In preparing for battle I have always found that plans are useless, but planning is indispensable.” No doubt the same applies to RA, but it seems to me a ponderous way to begin. There’s another military adage attributed to Robert Heinlein: “Get your first shot off fast. If you miss, it will throw off the other guy’s aim, allowing you to make your second shot count.” From a security POV I take this to mean one should get some protection in there – what others refer to as “Baseline” and “Diligence” – while others are still doing the risk analysis.
One good and very powerful aspect of RA is often abused or completely mis-used. It is the “Identification of Assets”. Lets get one thing clear: the equipment is not an asset.
In his marvelous 1992 novel “Snow Crash“. Neal Stephenson describes a franchising system and makes reference to the “three ring manual”. This manual is the set of operating procedures for the franchise, who does what and how, down to the smallest detail. I mention this in contrast to, for example, some of the businesses that failed after 9/11. These businesses did not have any ‘plant’ – desks, computers, software, even data – that could not be replaced. They failed because their real assets were not documented – the business processes existed solely “in the heads” of the people carrying them out.
The real assets of a company are not the COTS components. This is a mistake that technical people make. The ex-IBM consultant, Gerry Weinberg, the guy who came up with the term “egoless programming“, also pointed out that people with strong technical backgrounds can convert any task into a technical task, thus avoiding work they don’t want to do. Once upon a time I excelled in the technical side of things, but I found that limited my ability to influence change with management.
The business is what the business does. The tools are important, and there may be special proprietary tools (be they custom machine tools or software applications). But unless the processes for using them are documented, having them as ‘physical assets’ is of no use.
So yes, identifying assets is important.
But does your company know who – and value – who is key to its operations? Is what that person does documented or could be documented so that it could be done by someone else?
I recall one ‘audit’ I carried out where the machine room operator explained to me what all the equipment was for and how the input tapes were processed and the end of day reports were generated. After she finished if there as a check-off list for each day, weekends, month end and so on. If there was a manual detailing the steps she had just explained. She said there wasn’t. So I asked her how she knew what to do. She told me she’d only been there a week – no ‘month end’ yet – but the person who held the job before her had come in one afternoon to explain what had to done.
I don’t think it takes a lot to identify that highest risk here has nothing to do firewalls or patches or IDS.
So we get back to the ‘soccer goal’ picture in the article by Richard Bejtlich that I started with. He puts it in a very straight forward manner – defending against the wrong risks, no doubt because all the suppositions about the attacker’s motivation and methods are incorrect and the assets have not been properly identified.
Even after some way of correctly identifying all of the above, getting meaningful input from subject matter experts and so on, I still see this as a lot of detailed and tedious work.
Which is why I prefer to think in terms of ‘effect’ and the effect of failure.
Lets look at that soccer match again. Stopping the opposing side scoring goals is great, but that’s not the business. The business is in getting fans to pay to come into the stadium. If you have a winning team, that’s great, but stopping goals isn’t the direct cause of revenue. In fact scoring goals – winning the game by scoring more goals that the other team – isn’t always a formula for business success. The Toronto Maple Leafs, for example, sell out every home game despite their less-than-awesome record over the last few decades. Look at the history of the Green Bay Packers and the Detroit Lions, a rivalry that has spanned 75 years and 150 games and had some of the most memorable moments in the history of professional football. During that time only once have the Lions defeated the Packers – 1962. But fans turn up for the entertainment, not the score, and the same holds true for soccer in the countries where that is a national sport. A winning season is great, it makes the fans happy and offers many other opportunities for bringing in revenue. Great players also bring in the fans, but great players don’t always mean winning games – Michael Jackson is probably the greatest player in the NBA, but the Knicks have only placed first a few time in the Eastern conference since he joined and keep loosing the play-offs.
And if the players are assets because they bring in the fans, lets not forget, they also get traded.
All in all I’m unhappy with everything about the methods of Risk Analysis that I read. It seems speculative and prone to a lot of suppositions. At best it seems to pander to the belief that management need numbers, figures, dollar values on which to base decisions.
Gerry Weinberg also talks of the “Rutubuga Rule”. The rutabagas take up storefront space at the grocers and don’t sell, so get rid of them; then what comes next? And so with security, deal with the known stuff first, just as you would he a lock on your front door. When you’ve dealt with all the ‘baseline’ issues for your industry or similar environments, simplified your processes (because complexity leads to complications and errors) and applied the Deming or Shewhart cycle (Plan, Do, Check, Act) a few times, when you have and have tested plans to deal with response and recovery to failure – regardless of the threat or vulnerability, learnt where your real problems with supporting the business processes are, then and only then I’d think about the “by the book” RA.
Because you will be able to deliver effective (and measurable) results faster than going through the RA process.