Is interviewing is a much better method that self-certifications and a checklist, if time and resources allow.
In the ISO-27001 forum, my friend and colleague Gary Hinson has repeatedly pointed out, and I fully support him in this, that downloading check-lists from the 'Net and adopting question lists from there is using a solution to someone else's
problem. If that.
Each business has both generic problems (governments, sunspots, meteor strikes, floods & other apocalyptic threats and Acts of God) and ones specific to it way of working and configuration. Acts of God are best covered by prayer and insurance.
Gary recommends "open ended questions" during the interview rather than ones that require a yes/no answer. That's good, but I see problems with that. I prefer to ask "Tell me about your job" rather than "Tell me how your job ... can be made more efficient".
My second point is that risk management will *ALWAYS* fail if the risk analysis is inadequate. How much of the RA should be done by interviewing people like the sysadmins I don't know, but I have my doubts. I look to the Challenger Disaster. I started in the aviation business and we refines FMEA - failure Mode Effect Analysis. Some people think of this in terms of "impact", but really its more than that, its also causal analysis. As Les Bell, a friend who is also a pilot and interested in aviation matters has pointed out to me, "Root Cause Analysis" no longer is adequate, failure comes about because of a number of circumstances, and it may not even be a single failure - the 'tree' fans both ways!
Yes, FMEA can't be dome blindly, but failure modes that pertain to the business - which is what really counts -- and the fan-in/out trees can be worked out even without the technical details. Rating the "risk": is what requires the drill-down.
Which gets back to Donn Parker's point in a number of his books, though he never states it this way. The FMEA tree can be heavily pruned using diligence as he says: standards, compliance, contracts, audits, good practices, available products. The only thing he leaves out are Policy and Training. Policy gives direction and is essential to any purpose, the choice of standards and products, and identifying what training is needed.
All in all, the article at https://blog.anitian.com/flawed-it-risk-management/ takes a lot of words to say a few simple concepts.
An article on Linked entitled 'The Truth about Practices" started a discussion thread with some of my colleagues.
The most pertinent comment came from Alan Rocker:
I'm not sure whether to quote "Up the Organisation", ("If you must have a policy manual, reprint the Ten Commandments"), or "Catch-22" (about the nice "tidy bomb pattern" that unfortunately failed to hit the target), in support of the article. Industry-wide metrics can nevertheless be useful, though it's fatal to confuse a speedometer and a motor.
However not everyone in the group agreed with our skepticism and the observations of the author of the article.
And Anton aren't the controls you advocate so passionately best practices? >
NOT. Make that *N*O*T*!*!*! Even allowing for the lowercase!
What framework would you use to provide for quantitative or qualitative risk analysis at both the micro and macro level? I'm asking about a true risk assessment framework not merely a checklist.
Yes, this is a bit of a META-Question. But then its Sunday, a day for contemplation.
When does something like these stop being a check-list and become a framework?
COBIT is very clearly a framework, but not for risk analysis and even the section on risk analysis fits in to a business model rather than a technology model.
ISO-27K is arguably more technology (or at least InfoSec) focused that COBIT, but again risk analysis is only part of what its about. ISO-27K calls itself a standard but in reality its a framework.
The message that these two frameworks send about risk analysis is
Context is Everything
(You expected me to say that, didn't you?)
I'm not sure any RA method works at layer 8 or above. We all know that managers can read our reports and recommendations and ignore them. Or perhaps not read them, since being aware of the risk makes them liable.
Ah. Good point.
On LinkedIn there was a thread asking why banks seem to ignore risk analysis .. presumably because their doing so has brought us to the international financial crisis we're in (though I don't think its that simple).
The trouble is that RA is a bit of a 'hypothetical' exercise.
The trouble with some people is that they make some deceptively reasonable comments that don't stand up under critical analysis
With an ailing economy and a whole lot of cancelled contracts resulting from
that poor economy. Pandemic planning is a major threat to our most important
asset people and it appears as though that vulnerability may have been
activated. Its time to dust off the BCP plan and update it with a Pandemic
If it takes a pandemic to motivate you to create or review a BCP then
something is seriously wrong, and it has nothing to do with the pandemic.
As one manager said to me a long time ago, "show me the numbers".
The number of confirmed cases rose Monday to 50 in the U.S., the result
of further testing at a New York City school. The WHO has confirmed 26
cases in Mexico, six in Canada and one in Spain. All of the Canadian
cases were mild, and the people have recovered.
The Mexican government suspects the virus was behind at least 149 deaths
in Mexico, the epicentre of the outbreak, with hundreds more cases
I'm sure just about any ocotr - or the 'Net - can supply us with figures on the cases and deaths from 'regular' flu world-wide, as well as the named versions.
Les Bell, another ex-pat Brit who lives in Australia was discussing the importance of training and reinforcement in such matters as DR/BCP. Les is also a pilot and so many of his analogies and examples have to do with piloting and aircraft.
Part of our discussion has a much wider scope.
Les had said:
"People under extreme stress may behave unpredictably and have limited capacity for rational thought"
This is the basis of much of pilot training, particularly in simulators, where procedures that are too dangerous to be attempted in a real aircraft can be repeated until drills are automatic.
Don't quote me on this, but I seem to recall reading in an aviation safety-related article that in an emergency, something like 50% of people lose it to the extent that they are completely unable to cope, 25% are capable of functioning with some degree of impairment, and 25% of people are able to complete required tasks correctly. Training by means of drills and rehearsals is able to correct that situation to a considerable extent.
Therefore in BCP/DRP planning, it's important to - as far as possible - simulate an emergency, rather than just story-boarding it, or doing a whiteboard walkthrough. Hence the requirement for fire drills, evacuation drills and the like; repetition conditions the mind to perform the task correctly under stressful conditions.
Most of us don't get the chance to do a full interruption test for our DRP, but the closer we can get, the better.
Training - drill and reinforcement so that you can carry out the actions automatically even when extreme stress has completely blanked and cognitive functions - is an important part of military "boot camp" training and one reason I find it so comical that CISSP course training gets called "boot camp".
Les is quite right. For a variety of reasons most people "loose it" under extreme stress. This is why military heroes, people who can hang in there and think clearly and make critical decisions, are held in such esteem. Similarly test pilots (and those test pilots who became the early astronauts). Having lightening fast reactions (racing drivers) and being in top physical condition helps, but there is something more.
Some authorities look to the old American 'gunslingers' and speculate about how the adrenaline rush in such situations is handled by the body and the brain. Typically all that adrenaline pumps up the muscles for "fight or flight" and in such panic or near panic situations rationality is not the key issue. But if we shift from the evolutionary context to the 'gunslinger', standing still means that there is a lot of 'shakes'. Being able to stay calm and not have the shakes leads to being a sucesfull 'gunslinger'. Evolution in action?
There are other forms of stress as well. I've seen sysadmins who have been up for more than 30 hours trying in futile to solve a problem that to me, well rested, is simple and obvious.
The lesson here is two-fold. The first is the point that Les makes. Train and reinforce.
The second is that when the disaster does strike be aware that the stress will load up on fatigue and that stressed and fatigued people do not make good decisions. Rest, shifts, alternates, standard plans and scenarios that can work to relieve the stress are important.
Comprehensive? Well, pretty good.
The kind of thing that could keep a client's IT staff occupied for weeks. If they had hard copy to annotate and work with.
Its interesting that classical Business Continuity Planning works more along the lines of a FMEA than Threat-Risk Analysis. BCP identifies the business processes that are most essential and hence must be brought back into operation with the most urgency - that is what are the most critical failures that will affect the operation of the business.
The TRA approach has many flaws ranging from the fact that threats are just about infinite and mostly unknown, that vulnerabilities are infinite and unknowable, that they interact in complex ways, which boils down to playing whack-a-mole, and that there is not enough information for statistical analysis.
FMEA on the other hand identifies criticality regardless of the cause.