Re: Some thoughts on the performance of SSD RAID 0 arrays
On 09/18/2016 01:22 PM, arocker@Vex.Net wrote:
> I haven’t been following hardware developments very closely for a while, so I
> find it hard to judge the arguments. What’s important?
Ultimately what’s important is the management software, the layer above the
drivers, off to one side. That applies regardless of the media and means that
the view the applications take of storage is preserved regardless of changes in
the physical media.
> The first question is, what areas are currently the bottlenecks and
> constraints, at what orders of magnitude?
The simple answer is ‘channels.
> Are processors starved for data, or drives waiting around for processors to
> send them data?
All of the above and don’t forget the network!
Ultimately everything has to move through memory, wherever its coming from,
wherever its going to. Can you bypass memory? Yes, but then you have SEVERE
management problems. Don’t go there in the general case and when you DO go
there for specialized functionality you can’t do much else.
Back in the early 1970s I was working with a British GEC 4000 series machine.
It had been built as a controller system, think what we call SCADA today, for
railways and the like. I’m sure there’s a manual on-line for it somewhere; I
had one but seemed to have lost it.
What characterized it was that it had FIVE main buses or channels.
The memory was 5-ported, so the CPU could be accessing a region in one port
while one disk was writing to memory using another and second disk was reading
from another region and the network was was doing input to one region and output
from another region.
All in parallel.
The bus selection/contention was managed individually by the devices all of
which had ‘autonomous data transfer’ facilities. Nothing new here. Rather than
shove values into registers on the devices as we do today, there were a chain of
control blocks that the CPU built in memory and the devices each had what
amounted to a DMA mechanism to read and inwardly digest and act on each control
block, set the complete flag for it, then go on to the next. This was normal
for large machines of the time. Smaller machines like the PDP-11 introduced
register stuffing, even though some of their controllers were this older style.
You can find academic papers and studies that show that parallelism is one of
the greatest accelerators of performance. Go google and red up on that in your
own time.
There are few reasons we can’t build a machine like that today; ultimately its
economics.
In order to have block level granularity where blocks were 512 bytes, the memory
had to be (chip level) addressable at that granularity. Two different devices
might need to access adjacent sections of memory. Its not that memory wasn’t
aligned, it was, on 512 byte blocks.
If you think about it for a monument you’ll see why.
When we have, say, 4kx8 chips, never mind larger, we can’t have two different
processes addressing adjacent 512 byte blocks on that one chip. It only has one
set of address lines, and one set of data lines. With 4kx8 chips we now have
a threshold, not matter how sophisticated the hardware between the chip and the
bus or buses goes for ‘multiplexing’ of 4k granularity.
Well, that’s OK, Linux can live with 4G memory block allocation for the virtual
memory system and many modern disks are moving to 4k block addressability.
But reality: how available and expensive are 4kx8 chips?
Does this
https://www.amazon.ca/OWC-Upgrade-Compatible-upgrades-OWC5MD128MBE/dp/B01ETY1GPK/
have 4k granularity addressability?
Yes, but that’s only a single channel. You still need to have a card that plugs
into 5 buses and has the multiplexing hardware. That gets to be expensive.
Some models of the PDP-11 and the VAX made do with dual channel memory; one bus
was devoted to the CPU alone, the other to devices, primarily the disk, but
later a form of terminal IO that was buffered and could send whole lines or
screen updates. A lot of the character by character handling, certainly in line
mode rather that RAW ode such as used by VI, was carried out by a dedicated
terminal server. I used a system like that at HCR. It was very effective and
reduced the immediate per character interrupt rate on the PDP-11/45 to a level
where it could happily support 40 concurrent users.
So much so that when we moved to the nominally more powerful VAX that didn’t
have this parallelism the perceived perforce and responsiveness dropped to a
level where a machine with nominally 4-6 times the ‘power’ could only happily
support about half the number of users with the same kind of job mix.
it’s all very well to say that it was about off-loading IO, but that off-loading
involved parallel processing, the parallel subsystem doing what the main
processor would otherwise have to do. This isn’t like the traditional (e.g. IBM)
mainframes that simply couldn’t even do that kind of IO at all and had to use
peripheral processors.
> SSDs have clear advantages over spinning rust in robustness and lack of
> latency, offset by cost/byte. When a terabyte disk costs $60,
What do you mean, ‘terrabyte’, Kimosabe?
Shop around and you can get a 2T SATA 2.5″ for US$60, a few outlets might
stretch to a 3T for less than US$75. I’m sure discount/reconditioned houses
will have some 2T under CDN$60 if I start looking.
> and you are neither taking lots of video nor running a bit barn, cost/byte’s
> not a big deal. What are the relative speeds of disks, SSDs and cards.
> (There’s at least an order of magnitude variation in quoted SD card transfer
> rates.)
In that exclusion set we can count a lot of home computing.
Some people like John might run a family SAN and stretch to a few terrabytes,
but a single 3T drive represents a SPOF. His SAN might be better served by
mirrored 500G drives or if performance is critical mirror-striped or striped-
mirror (preferably RAID 1+0, “stripe of mirrors”. i.e the disks within the group
are mirrored. But, the groups themselves are striped).
RAID 10 Vs RAID 01 (RAID 1+0 Vs RAID 0+1) Explained with Diagram
Ultimately, striping is about parallelism.
> Infinite expandability gives the edge to cards, provided they’re big enough
> and fast enough.
No, Allan, that’s incorrect.
Infinite expandability is a function of the file system or file system manager.
Both LVM and BtrFS are indefinitely expandable; you can keep adding spindles (or
SSD equivalent) and growing the individual file system size of the file system
to limit of the inode fields, or possibly recompiling your system to have bigger
inodes/more fields.
BtrFS
https://en.wikipedia.org/wiki/Btrfs
has a “one file system to rule them all” approach; the ‘them’ being all the
drives/spindles. It supposes 64-but machines and hence…
Limits
——
Max. volume size 16 EiB
Max. file size 16 EiB
Max. number of files 2^64
What’s an “EiB”? That’s an ‘exbibyte’.
1 exbibyte = 2^60 bytes = 1152921504606846976bytes = 1024 pebibytes
What’s a “pebibyte”?
That 1024^5 bytes.
So 1 exbibyes,
1TB is, if we use the binary form, 1024^4 bytes.
So 1 EiB is 1024^2 TB.
That’s a lot of ‘spindles’.
And the limit for BtrFS is 16 times that.
So that’s what ‘indefinitely’ amounts to for BtrFS.
You can also have an indefinite number of sub-volumes within that.
These aren’t partitions; they are more like ‘thin partition’ where the space
comes out of the general pool
The can be mounted with access controls just like regular volumes.
BtrFS is important in another regard: it is one of the few file systems that do
not need provisioning when created.
Back in the original version 6 days of the late 1960s and early 1970s the
original file system was designed for simplicity and hence a very small amount
of code; memory was very limited. The KISS meant that there was noting dynamic
or adaptive being done. There was a hard division between the number of blocks
devoted to the inodes and the number of blocks devoted to data. It could never
be varied and the superblock said what those numbers were and where the boundary
was. Even the later Berkeley Fast Filesystem followed the principle of
preallocation, it just rearranged the positioning.
Some, but sadly not all, later model file system, ReiserFS, XFS and BtrFS among
them removed this restriction. They use dynamically balanced b-trees for
allocation pretty much on demand. There is a pool and tree-parts are allocated
for inodes, name space and data space as needed. Unlike the reallocated file
systems, they can never run out of inodes before data or out of data before
inodes.
Sadly not all late model file systems follow this sensible approach. Even
though the ext4FS uses b-trees internally, it still requires preallocation to
set the inode/data ratios. Big OUCH!
Back to management.
BtrFS does not make the distinction between the file system and the management
of media. There are other supports for that. Under Linux the principle one is
LVM.
Volume management goes back some way; I used the Veritas Volume Manager on the
IBM AIX in the 1990s, and the Linux LVM is derived form that. There’s an
amazing amount you can do with LVM.
An example
http://man7.org/linux/man-pages/man7/lvmcache.7.html
You start with LVM by assigning a disk or a disk partition as a ‘physical
volume’ (PV). You can them go on to create one of more ‘Volume Groups’ (VG)
that perhaps span the physical media.
On my primary drive, for example, I create a /boot partition, a SWAP partition
and devote the rest of the drive to LVM. My ROOTFS is in LVM.
Within a VG you can then go on to create Logical Volumes, which are akin to disk
partitions.
I’m probably making this sound a lot more complex than it is, but for each layer
of abstraction you have specific functionality which is bundled and either
indecipherable or inaccessible in BtrFS. It basically comes down to
* create
* modify
* check
* delete
for each layer. Having separate layers lets you do things that you can’t easily
or perhaps *ever* do with BtrFS because BtrFS does not adequately difference
Volume Management from file system management.
You can, for example, have separate VGs for separate businesses or business
segments.
I’ve mentioned that I don’t like the Virtual Machine model for a variety of
reasons; replication,the whole OS and libraries and basic file system just to
run one application is heavy handed, storage, bandwidth to access/load and
memory to run. That’s why we have “Docker”. In some ways LVM is the Docker of
disk management.
The PV and VG layers are normally not something you deal with. If you’re
commissioning a new machine, installing a new drive, having to remove an only or
flaky drive that is erroring or about to die, then they concern you. Otherwise
they are a once in (machine) lifetime occurrences.
What you may be concerned with more often is Logical Volumes (LV).
These are akin to disk partitions. Unlike a disk partition using FDISK they are
managed in the VG space. You can grow them or shrink them or move them from
media to media after creation.
The commands for lvcreate and lvmodify have many options,, if you choose to
employ them. You can, for example, specify how striped, how mirrored a LV is.
Unlike BtrFS you can have separate LVs with quite different characteristics to
suit specific system and business needs.
And yes, you can change from mirrored or striped to ‘linear’.
You then go on to create file systems in the LVs.
Oh, yes, you could create a BtrFS in a LVM LV. 🙂
One of the things about using LVM is that it avoids the provisioning problems
you have with FDISK. The “partition” boundaries are dynamically flexible. You
can grow (or shrink) the size of LV with the machine running, with the disk in
use. There are some file systems that allow you to grow the file system itself
just as the LV it is in is growing. Or shrinking.
You might gather at this point that I have developed a dislike for
pre-provisioning. The reality is that even with Terabyte drives its better to
have a ‘pool’ that can be managed in the future, slack, rather than imposing the
kid of boundaries that go with the architectures of the 1960s.
There’s one other thing you can do with LVM and that is termed ‘thin
provisioning’. It pushes the concept of avoiding provisioning even further.
Now you have a LV with a file system is is, potentially, 20G in size, but is
currently only 4G. Rather than allocate 20G of physical extends right now, hard
binding them to that LV, just arrange to ‘bind on demand’ as the file system in
that LV actually uses them.
It’s worth noting that the Redhat manual
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinprovisioned_volumes.html
mentions
logical volumes can be thinly provisioned. This allows you to create logical
volumes that are larger than the available extents. Using thin provisioning, you
can manage a storage pool of free space, known as a thin pool, which can be
allocated to an arbitrary number of devices when needed by applications. You can
then create devices that can be bound to the thin pool for later allocation when
an application actually writes to the logical volume. The thin pool can be
expanded dynamically when needed for cost-effective allocation of storage space.
and
By using thin provisioning, a storage administrator can over-commit the physical
storage, often avoiding the need to purchase additional storage. For example, if
ten users each request a 100GB file system for their application, the storage
administrator can create what appears to be a 100GB file system for each user
but which is backed by less actual storage that is used only when needed.
LVM also offers some interesting features using, for example, CoW and the
ability to make snapshots of LVs for backups.
Along the while the question arises “Can you add SSD to LVM”.
Yes you can, its just another volume, in one way of looking at it. However you
may want to difference the fast vs the slow Physical Volumes in a volume group
with you allocate LV. I referenced above a way to use the fast PVs to act as a
cache for slower LVs. Part of the power of LVM over BtrFS is that you can do
things like this.
> Where in
> the spectrum from “To be stored undisturbed for posterity” to “extremely
> transient” does the data fall, and along another axis from “Entirely in the
> public domain” to “horrible things will happen if anyone else sees this”?
Those are quite separate issues: one concerns the fact that backups and archives
are meant to be immutable (though some backup modes that use RSYNC end up
looking more like Revision Control Databases!).
The other is about access control, and hence identification and authentication.
One can reasonably say that we have solved the technical aspects of both of
those. Getting people to employ, well not so much the technology as the
operational practices, is, as I’ve mentioned quite a number of times, a
completely different matter. All to many problems with the issues you mention
arise from that failure.
—
Ninety percent of the politicians give the other ten percent a bad reputation.
— Henry Kissinger
Everything old is new again
http://www.databreachtoday.com/whitepapers/seven-reasons-micro-segmentation-powerful-to-have-painless-to-add-w-2704
What’s the saying “Those who forget history are doomed to repeat it over again”?
Weren’t we doing this with routers and … well if not firewalls as such then
certainly filtering rules in the routers, way back in the 1980s?
I recall attending a luncheon put on by Dell about “Software Defined
networking”. Basically it was having routers that were ‘agile’ enough to change
routing and implement tactical policy by load, demand and new devices or devices
making processing demands.
Again we were doing that in the 1980s. Working with ANS as they cut over the
academic internet to the commercial internet with their “CO+RE” pseudo-product.
basically it was that they had been supporting the academic internet and were
not selling commercial services using the same backbones, trunks and “outlets”
(sometimes known as ‘points of presence’). This ‘policy based routing’ was
carried out by custom built routers; they were IBM AIX desktop boxes — the kind
I’d used to implement an Oracle based time management/billing system for at
Public Works Ottawa a few years earlier, along with some custom built T3
interface cards.
ANS wasn’t the only company selling commercial internet services by the end of
the 1980s; I was running UUNorth here in Toronto and Phonorola were trying to
sell leased line service in Ottawa. Phonorola had a problem, though, they were
a telco and telcos then traditionally had a complex matrix of pricing by
bandwidth of connection, bandwidth of use, time of use and few other factors.
I did much better by using either fixed monthly fee or connection time based
billing.
But the ANS routing simply *had* to do what we today call ‘software defined
networking’. New subnet were being added as customers gained them, registering
with the IANA function of administering registries of Internet protocol
identifiers (including the distributing top-level domains and IP addresses)
which was was performed by Jon Postel, a Computer Science researcher who had
been involved in the creation of ARPANET — what since 1998 is ICANN — and this
was done in a very haphazard way. There was no logical hierarchy and some
organizations implemented their own cross-geographic routing of their subnets..
Not only were subnets being added, they were being moved, and routing tables
needed to be dynamically re-written. Local routers had to recognise their
connections and communicate with others, so we developed a host of inter-router
management protocols and further routing and aggregation algorithms as this went
on. Firms like Cisco developed their own, optimized for their own product and
often closed protocols.
But if we step back 25 years we can see the process of ‘software defined
networking’ going on. At that Dell lunchtime presentation at a downtown Toronto
steakhouse I found myself frustrated with the sales-droids dressing up the old
in new clothing as if they had invented it.
And here we have another example.
The simplistic case of segmentation is at the router with the traditional DMZ.
But even in the 1990s I was suggesting that there be many instances of the DMZ,
with one and only one machine, service, on each. Trivial, ‘degenerate’ or
“beedin’ obvious” depending on your point of view.
—
If we believe absurdities, we shall commit atrocities.
— Voltaire
Nobody wants to pay for security, including security companies
https://www.linkedin.com/pulse/nobody-wants-pay-security-including-companies-beno%C3%AEt-h-dicaire
In theory, consumers and businesses could punish Symantec for these
oversights by contracting with other security vendors. In practice, there’s
no guarantee that products from other vendors are well-secured, either
— and there is no clearway to determine how secure a given security
product actually is.
Too many firms take an “appliance” or “product” (aka ‘technology”) approach to security. There’s a saying that’s been attributed to many security specialists over the years but is quite true:
If you think technology can solve your security problems,
then you don’t understand the problems and you don’t
understand the technology.
Its still true today.
Brexit: What’s Next for Privacy, Policing, Surveillance?
http://www.databreachtoday.com/brexit-whats-next-for-privacy-policing-surveillance-a-9225
Now we’re getting over the “how could that do THAT!” shock stage and starting to think what the operational, rather than just the financial, implications are.
Related articles across the web
Cyber risk in the business
https://normanmarks.wordpress.com/2015/06/05/cyber-risk-and-the-boardroom/
The take-away that is relevant :
Cyber risk should not be managed separately from enterprise or business risk. Cyber may be only one of several sources of risk to a new initiative, and the total risk to that initiative needs to be understood.
Cyber-related risk should be assessed and evaluated based on its effect on the business, not based on some calculated value for the information asset.
We’re mobile addicts but we just don’t want new smartphones
http://www.zdnet.com/article/research-were-mobile-addicts-but-we-just-dont-want-new-smartphones/
For whatever value of “Mobile” is applicable in context, yes.
A lot of what I see is students in the library with their laptops or large tablets_keyboards with paper and books beside. Perhaps if students had the multi-screen displays like the one in the movie “Swordfish” AND there were more books on-line at low cost and multi-access (which isn’t how many libraries work, sadly) then the marketers dream of students with ebooks rather than a knapsack of books would happen. As it is, with only one viewer, books and papers are still needed.
The fatal flaw in IT Risk management
Is interviewing is a much better method that self-certifications and a checklist, if time and resources allow.
Two points:
In the ISO-27001 forum, my friend and colleague Gary Hinson has repeatedly pointed out, and I fully support him in this, that downloading check-lists from the ‘Net and adopting question lists from there is using a solution to someone else’s
problem. If that.
Each business has both generic problems (governments, sunspots, meteor strikes, floods & other apocalyptic threats and Acts of God) and ones specific to it way of working and configuration. Acts of God are best covered by prayer and insurance.
Gary recommends “open ended questions” during the interview rather than ones that require a yes/no answer. That’s good, but I see problems with that. I prefer to ask “Tell me about your job” rather than “Tell me how your job … can be made more efficient”.
My second point is that risk management will *ALWAYS* fail if the risk analysis is inadequate. How much of the RA should be done by interviewing people like the sysadmins I don’t know, but I have my doubts. I look to the Challenger Disaster. I started in the aviation business and we refines FMEA – failure Mode Effect Analysis. Some people think of this in terms of “impact”, but really its more than that, its also causal analysis. As Les Bell, a friend who is also a pilot and interested in aviation matters has pointed out to me, “Root Cause Analysis” no longer is adequate, failure comes about because of a number of circumstances, and it may not even be a single failure – the ‘tree’ fans both ways!
Yes, FMEA can’t be dome blindly, but failure modes that pertain to the business – which is what really counts — and the fan-in/out trees can be worked out even without the technical details. Rating the “risk”: is what requires the drill-down.
Which gets back to Donn Parker‘s point in a number of his books, though he never states it this way. The FMEA tree can be heavily pruned using diligence as he says: standards, compliance, contracts, audits, good practices, available products. The only thing he leaves out are Policy and Training. Policy gives direction and is essential to any purpose, the choice of standards and products, and identifying what training is needed.
All in all, the article at https://blog.anitian.com/flawed-it-risk-management/ takes a lot of words to say a few simple concepts.
The 11 tiniest, most powerful computers your money can buy
http://www.geek.com/chips/the-11-tiniest-most-powerful-computers-your-money-can-buy-1627324/
I have my doubts about many things and the arguments here and in the comments section loom large.
Yes, I can see that business sees no need for an ‘arms race’ escalation of desktops once the basics are there. A few people, gamers, developers, might want personal workstations that they can load up with memory and high performance graphics engines, but for the rest of us, its ho-hum. That Intel and AMD are producing chips with more cores, more cache, integrated graphics and more, well Moore’s Law applies to transistor density, doesn’t it, and they have to do something to soak up all those extra transistors on the chips.
As for smaller packaging, what do these people think smart phones and tablets and watches are?
Gimme a brake!
My phone has more computing power than was used by the Manhattan project to develop the first nuclear bomb.
These are interesting, but the real application of chip density is going to have to be doing other things serving the desktop. its going to be
1. IoT
2. Servers
3. backbone/communications
And for #1 & #3 Windows will become if not an impediment, then irrelevant.
Its possible a very stripped down Linux can serve for #1 & #3, but somewhere along the line I suspect people might wake up and adopt a proper RTOS such as QNX much in the same way that Linux has come to dominate #2. It is, however, possible, the Microsoft will, not that Gates and Balmer are out of the scene, adopt something Linux like or
work with Linux so as to stay relevant in new markets. The Windows tablet isn’t the success they hoped for and the buyout of Nokia seemed more to take Nokia out of the market than become an asset for Microsoft to enter the phone market and compete with Apple and Samsung. many big forms that do have lots of Windows workstations are turning to running
SAMBA on Big Iron because (a) its cheaper than a huge array of Windows Servers that present reliability and administrative overhead, and (b) its scalable. Linux isn’t the ‘rough beast’ that Balmer made out and Microsoft’s ‘center cannot hold’ the way it has in the past.
Cyber, Ciber or Syber?
Occasionally, people do ask:
What exactly do you mean by “cyber security”?
Or “cyber” for that matter. Please explain.
“Steersman Security”?
It seems to be one of those Humpty-dumpty words that the media, the government and others use with — what’s the current politically correct phrase to use now when one would, 50 years ago have said ‘gay abandon’? — because its current;y “in”?
I see it used to mean “computer” and “network” in the specific and “computers” and “networks” in the general, as well as specific functions such as e-banking, & other e-commerce, “Big Data”, SCADA, POTS and its replacements.
I see it used in place of “Information” in contexts like “information Security” becoming, as above, “Cyber Security“. But you don’t know that it means that.
Are we here to protect the data? Or just the network? or just the computer?
Until a few years ago “Cyber” still did mean “steersman”, even if that was automated rather than a human presence. No-one would call the POTUS a “Cyber-man’ in the sense of being a steersman for the republic.
Perhaps we should start a movement to ban the use of “Cyber-” from use by the media.
Perhaps we might try to get some establishments to stop abusing the term.
I doubt very much we could do that with media such as SCMagazine but perhaps we could get the Estate of the Late Norbert Weiner to threaten some high profile entities like the State Department for the mis-use of the term?
Related articles across the web
Another reason to have a policy not to eat at your desk
Hackers Can Use Pita Bread to Steal Laptop Encryption Keys, Say Researchers
Embedding such devices in something edible only means it will end up in the stomach of the targeted user. Perhaps that is intentional, but I suspect not. Better to put the device in the base of the coffee cup.
Why Silicon Valley Will Continue to Rule
https://medium.com/backchannel/why-silicon-valley-will-continue-to-rule-c0cbb441e22f
The historical, cultural and economic context described here sums up why
efforts to replicate ‘the valley’ in other countries, other places,
according to governmental whims, never happens, never works, never will.People around the world have tried to reproduce Silicon Valley. No one
has succeeded.And no one will succeed because no place else — including Silicon Valley
itself in its 2015 incarnation — could ever reproduce the unique
concoction of academic research, technology, countercultural ideals and
a California-specific type of Gold Rush reputation that attracts people
with a high tolerance for risk and very little to lose. Partially
through the passage of time, partially through deliberate effort by some
entrepreneurs who tried to “give back” and others who tried to make a
buck, this culture has become self-perpetuating.
See also
and
Related articles
Can We Secure the ‘Internet of Other People’s Things’?
http://www.eweek.com/security/can-we-secure-the-internet-of-other-peoples-things.html
I think that title expresses the problem very well.
There are a few generalizations and ‘skating on thin ice’ in the article, never the less. Linux itself is not a lightweight OS, though there are stripped down and altered versions, such as Android. Other competing RTOS exist. Some like QNX are much more suited to small embedded devices that don’t have a GPU/GUI and comprehensive set of commands/applications.
The reality is that the inherent model of Linux is NOT real-time, it cannot guarantee a real time response that many embedded systems such as TVs and some classes of network devices demand, especially in a small controller, limited CPU, limited memory configuration.
But yes, the Internet has already shown the problems lie with “Other People”.
Related articles
Cyber general: US satellite networks hit by ‘millions’
http://www.forensicmag.com/news/2015/04/cyber-general-us-satellite-networks-hit-millions-hacks
I wonder what they consider to be a hack? The wording in the in the article is loose enough to mean that if someone pinged one of their servers it would be considered a hack. Perhaps they even they count Google spider indexing as a probe into their network. It makes me wonder how many ‘real’ hack attempts are made and how many succeed. All in it, it sounds like a funding bid!
Marcus Ranum once commented about firewall logging that an umbrella that notified you about every raindrop it repulsed would soon get annoying.I suspect the same thing is going on here. Are these ‘repulsed’ probes really ‘need to know’? Are they worth the rotating rust it takes to store that they happened?
Oh, right, Big Data.
Oh, right, “precursor probes“.
Can we live without this?
U.S. Defense Secretary Carter emphasizes culture change needed to
http://www.scmagazine.com/ash-carter-spoke-at-stanford-university/article/411392/
Yes the government needs a culture change if it is to address its own and the national issues pertaining to security, technological, in general, internet related and more. But not like this.
A real culture change would involve hiring the likes of people such as Marcus Ranum, Gene Spafford, Becky Herrold., and more significantly the very vocal Bruce Schneier AND PAYING ATTENTION TO WHAT THEY SAY AND CARRYING OUT THEIR RECOMMENDATIONS. And please note: none of this is new or radical.
But a read of Bruce’s articles blog and published articles will make it clear to any intelligent reader, even those outside the InfoSec community, that they won’t. The culture change it would require would impact too many vested interests and long held beliefs, even though Bruce — and others — have long since shown them to be in the same class as The Emperor’s New Clothes.
When the government talks of cyber-security experts it really doesn’t want people who think in terms of policy and strategy. The fact that most government agencies could do better if they carried out the recommendations that have been made to them — but consistently don’t[1] — tells you something about their innate culture. Just adopting the GAO recommendations would take a culture change. Adopting ‘uber 133z h4x0r’-wannabes for job roles that are written as what amounts to jumped-up netadmin and sysadmin positions doesn’t make for good security[2].
Yes, a culture change is needed. But the kind of changes that the ‘insiders’ — and that goes for the media too — envision don’t really amount to a meaningful change.
[1] http://www.gao.gov/key_issues/cybersecurity/issue_summary#t=1
http://www.regblog.org/2014/09/18/18-yang-gao-and-it-oversight-report/
http://www.cnn.com/2014/12/19/politics/government-hacks-and-security-breaches-skyrocket/
[2] The idiom “rearrange the deckchairs on the Titanic” comes to mind
Or perhaps the Hindenburg.
Related articles
Review: “Penetration with Perl” by Douglas Berdeaux
Penetration Testing with Perl
Douglas Berdeaux has written an excellent book, excellent from quite a number of points of view, some of which I will address. Packt Publishing have done a great service making this and other available at their web site. It is one of many technical books there that have extensive source code and are good ‘instructors’.
Penetration Testing with Perl is available as both a PDF file and as an e-book in Mobi and epub formats.
It is one of over 2000 instructional books and videos available at the Packt web site.
I read a lot on my tablet but most of the ebooks I read are “linear text” (think: ‘novels’, ‘news’). A book like this is heavily annotated by differentiating fonts and type and layout. How well your ebook reader renders that might vary. None of the ones I used were as satisfactory as the PDF. For all its failings, if you want a page that looks “just so” whatever it is read on, then PDF still wins out. For many, this won’t matter since the source code can be downloaded in a separate ZIP file.
Of course you may be like me and prefer to learn by entering the code by hand so as to develop the learned physical habit which you can then carry forward. You may also prefer to have a hard copy version of the book rather than use a ‘split screen’ mode.
This is not a book about learning to code in Perl, or earning about the basics of TCP/IP. Berdeaux himself says in the introduction:
This book is written for people who are already familiar with
basic Perl programming and who have the desire to advance this
knowledge by applying it to information security and penetration
testing. With each chapter, I encourage you to branch off into
tangents, expanding upon the lessons and modifying the code to
pursue your own creative ideas.
I found this to be an excellent ‘source book’ for ideas and worked though many variations of the example code. This book is a beginning, not a end point.
Should all applicable controls be mentioned in documenting an ISMS?
In my very first job we were told, repeatedly told, to document everything and keep our personal journals up to date. Not just with what we did but the reasoning behind those decisions. This was so that if anything happened to use kn knowledge about the work, the project, what had been tried and thought about was lost, if, perhaps, we were ‘hit by a bus on the way to work‘.
At that point whoever was saying this looked toward a certain office or certain place in the parking lot. One of the Project managers drove a VW bus and was most definitely not a good driver!
So the phrase ‘document everything in case you’re hit by a bus’ entered into the work culture, even after that individual had left.
And for the rest of us it entered into our person culture and practices.
Oh, and the WHY is very important. How often have you looked at something that seems strange and worried about changing it in case there was some special reason for it being like that which you did no know of?
Unless things get documented …. Heck a well meaning ‘kid’ might ‘clean it out’ ignorant of the special reason it was like that!
So here we have what appear to be undocumented controls.
Perhaps they are just controls that were added and someone forgot to mention; perhaps the paperwork for these ‘exceptions’ is filed somewhere else[1] or is referred to by the easily overlooked footnote or mentioned in the missing appendix.
It has been pointed out to me that having to document everything, including the reasons for taking one decision rather than another, “slows down work”. Well that’s been said of security, too, hasn’t it? I’ve had this requirement referred to in various unsavoury terms and had those terms associated with me personally for insisting on them. I’ve had people ‘caught out’, doing one thing and saying another.
But I’ve also had the documentation saving mistakes and rework.
These days with electronic tools, smartphones, tablets, networking, and things like wikis as shared searchable resources, its a lot easier.[2]
Sadly I still find places where key documents such as the Policy Manuals and more are really still “3-ring binder” state of the art, PDF files in some obscure[1] location that don’t have any mechanism for commenting or feedback or ways they can be updated.
Up to date and accurate documentation is always a good practice!
[1]http://hitchhikerguidetothegalaxy.blogspot.ca/2006/04/beware-of-leopard-douglas-adams-quote.html
[2] And what surpises me is that when I’ve implemented those I get a ‘deer in the headlight’ reaction from staff an managers much younger than myself. Don’t believe what you read about ‘millennials’ being better able to deal with e-tools than us Greybeards.
Related articles
This is not the IoT you want.
http://www.cnet.com/products/quirky-outlink/
If I plug in an IDE drive or a SATA drive or a USB drive or device my mobo or system recognizes what it is. The connection protocol tell the mobo or system.
My digital camera uses exif to convey a vast amount of contextual information and imprint it on each photo: date, time, the camera, shutter, aperture, flash. I have GPS in the camera so it can tell the location, elevation. The exif protocol also allows for vendor specific information and is extensible and customizable.
Unless and until we have an ‘exif’ for IoT its going to be lame and useless.
What is plugged in to that socket? A fan, a PC, a refrigerator, a charger for your cell phone? What’s the rating of the device? How is it used? What functions other than on/off can be controlled?
Lame lame lame lame.
Unimaginative.