The InfoSec Blog

While this point arose from a discussion on the ISO-27001 mailing list, in other InfoSec/Audit forums I’m known as a strong proponent of Failure Mode Effect Analysis (FMEA), to the point where people naturally associated it with me. However it clear in my 20+ years in InfoSec in a number of countries (including the UK, where ‘7799 and 2700x come from) that the accepted FMEA approach is not a normal method of ‘risk assessment‘ and is not taught (or examined for) as part of InfoSec, for example as part of the CISSP.
I learnt FMEA (and other techniques we now bundle under 6-sigma, ITIL and so forth, before they were categorized and labelled) in engineering - physical, electrical and aviation. In general, I’d say that InfoSec has a lot to learn from other engineering professions about managing threats, vulnerabilities and failures, and what actually constitutes “risk”. For a start, we have too much of a techie-geek outlook and we are not well educated in statistical methods.
Just giving numbers is meaningless. I will not trust any ‘graph’ that does not have ‘error bars’ and which does not document the sample size vs the total population and show the variance, for example against a random population. These test are easy to perform and I’m disappointed that so many numerically justified ‘risk analysis’ models are really just a pile of spread-sheet mumbo jumbo with no discipline of process behind them.

Estimating “on a scale of 1-5″ the components of the risk equation for many factors then multiplying out and averaging is pretty meaningless, yet I’ve seen it done by consultants from TLA companies and accepted by managers. At the very least it ignores cross interactions, and is essentially just a “your guess is a good as mine” approach.
I have nothing against opinions about risks, what I do object to is trying to make opinions into solid numbers. That “estimate on a scale of 1-5″ will have an error bar of size >= 5 !

Lets face it, most people don’t understand statistics. All to often I’ve seen managers ignore the “once in 100 years” MTBF (and ignore any MTTR - see FMEA) because they only plan to be with the company for five years, or ignore it because it happened 25 years ago so they think they have a 75 year breathing space. Yes, I know it sounds apocryphal, but I’ve met it too often.

To my mind FMEA is not only easier than TRA, but it focuses the mind on two key issues - survival and recovery (see MTTR) - that TRA doesn’t.

http://blogs.techrepublic.com.com/security/?p=274

Its a good week at TechRepublic for security articles.
In the light of a number of threads this last month in the various forums I’m invovled with I found this article particularly interesting.

The real problem with the ROI debate is that it is about convincing management that spending money on InfoSec protection is worth while. The “B-school mentality” of management is that everything can be reduced to numbers, and many people ’speak’ that language and have troubles with anything not reduced to numbers. I hope they compartmentalise and and their home life s not “by numbers”. (Imagine justifying the cost effectiveness of peanut butter sandwiches in the kids lunchbox!)

But a lot of InfoSec is too abstract for people.
In my presentations I’ve often given the example of the 1950s office: typewriters, ribbons, carbon paper, hanging file cabinets, copies, and mapped them to modern technology like PC terminals, keystroke recorders, hard drives and file file servers, and thumb drives, and shown that the same principles apply to protecting the information whatever the technology.

So I found this a very useful article. People can easily relate to the physical, to their own home situation. We have many centuries legacy of houses, thieves and door locks and once people can map from something they know to to abstract their understanding is easier, and so our justification of InfoSec measures is also easier.

My fellow CISSP Clement Dupuis posted a very good response to this article, and I’m sure he won’t mind me quoting him:

It seems that people usually have logical thinking when they discuss physical security. This is not the case when they discuss logical security.

There are hundreds of companies that have invested heavily into intrusion detection systems but they have a total lack of incident response or policies associated with what to do when there is an incident detected.

When you ask them if they would buy an home alarm system that does not have a siren and does not alert the police they always respond “No way”, however they do this on a day to day basis with their logical security.

There are many other very good follow-ups to this article and I recommend reading though them.

We know that anti-spam (and for some, AV) is a necessary baseline.
(I’ll avoid using the ‘diligence’ words for now.)

But here is a spreadsheet that ‘does the numbers’.

As I’ve said before, the ROI issue isn’t about justifying the project - the normal B-school way of looking at at things. Its about ‘choosing between’. That’s what this spread sheet purports to be doing. The reality is subtly different.
In doing so, it illustrates all that can go wrong with this approach. Basically you are ‘buying in‘ to someone else’s way of looking at the analysis. This is essentially the trick that sales-droids use. They get you to accept their world view, and once you do accepting that their product is the ‘right one‘ naturally follows.

In a way, this is like the conclusion to the Sapir-Whorf hypothesis about linguistic determinism.
“Put simply, the hypothesis argues that the nature of a particular language influences the habitual thought of its speakers. Different patterns of language yield different patterns of thought.”

That wikipedia article also has some references to the use of language as determinism in fiction that are vary pertinent.

Elsewhere I find: “A well-known saying by Alan Perlis states that ‘a language that doesn’t affect the way you think about programming is not worth knowing‘. he’s talking about computer languages, but it applies to natural languages - and of course tools - software tools like spreadsheets, word processors, UML modelers, and things like Photoshop.
So how is this relevant?

There in the spreadsheet are the four categories:

• Difficulty
• Investment
• Capability
• Expandability

No-one says quite what they mean.
The Initial/Daily/Ongoing might appear to clarify, but when you work through it can also add to the confusion.

Is the investment in time or money? Well, if time is money, how are we calculating the equivalence?
Right, another sheet, grade of techie vs bill-out rate.
Well what about things like cost per server, cost per seat? cost per bandwidth?

…. and so on …

How is ‘daily’ different from ‘ongoing’?

…. and so on …

Are these the only categories? Would you break things down more specifically? Would you use other or more categories?
Then there’s the weighting values. Do you agree with them? Are you going to accept other people guiding the judgment, making it appear that you are making the decisions, when in reality they have made all the important structural decisions?
The answer from the numbers-people (you know who you are!) is that it is easy enough to add another sheet, insert anotter row or column, alter the weightings … And indeed it is. For them. They think in those terms - they cannot do otherwise. “When the only tool you have is a hammer you view every problem as a nail“.
But implicit in this is that you accept that culture and its conclusions - that the number (and spreadsheet) are “God”.

In the closing of the First World War, Kipling wrote a poem that poked fun at the established “old wise saying” that many people used to justify action while avoiding really thinking about matters. I see the obsessive use of tools like spreadsheets and the numbers they give - hidden by the arbitrary assumptions of the model - as a similar phenomena.
So lets look at another way.
How would you choose your AV product?

• “Market Dominance“
  As in ‘you never get fired for buying IBM’ updated to the modern world.
  Certainly when the Big Name auditors come along to do your compliance audit they will check “AV” off on their list.
• “Magazine Reviews“
  There are two approaches here. You can ‘blindly’ accept the ‘Editors Choice‘, which is really no different from accepting how they formulated their spreadsheet, assigned weightings and values. Alternatively you can read lots of reviews and form your own impression.
• “Advice of Peers“
  As in ‘this worked for me‘ or ‘I had lots of problems with that one‘.
  In some ways this is like reading lots of reviews, but with a more personal and ‘real world’ feel to it. You can also seek out someone who has a set-up similar to yours so that the context of the advice is more meaningful.
• “Corporate Standard“
  Maybe you don’t have a choice. (I recall a site where ‘policy’ said that all IBM computers had to be in the raised-floor room. It was probably originally justified by insurance, UPS, HVAC issues. However when I audited the site I found a stack of IBM laptops there that had never been used.) Maybe the security is out-sourced or somehow ‘managed’ and details are outside your control. Perhaps there is a “Master Purchasing Agreement” with a specific vendor and if that vendor has a product that is pertinent you get stuck with it.

So how do you make the decisions - not the “decision to” but the “decision which”?