Eight years ago my career collided with reality.  By way of serendipity–following an unplanned career change–I discovered I had been living the last decade in a product development bubble. Two thousand and two was the year I had transitioned from building information management systems to managing a small storage industry analyst firm.

Up until 2002 I thought, perhaps arrogantly, that I thoroughly understood information management. After all, I had spent countless hours helping companies of all types implement systems to manage their digital information assets. I had no idea how little I understood until I began to learn more about storage infrastructure. All those years I had worked with other developers to build different types of business applications (e.g. enterprise content management systems and digital classrooms) with little regard for the applications’ impact on storage, mostly because I was not aware of their actual impact on storage. After all, storage was someone else’s problem we reasoned–a “black box” in which we stored our data.  There was no reason for us to truly understand how it all worked as long as we had enough space for our applications and files, right? Our customers could simply install our applications and databases, fire them up and begin collecting, aggregating, managing, manipulating and saving gigabyte upon gigabyte of data to their hearts’ content…or so we thought.

Not so fast.

Silly me. I naively believed electronic information management was that simple.  My fellow application engineers believed it too. Worse, many dozens of customers believed it was that simple.

I’m here now to tell you, it isn’t simple. There’s a cost–a very large cost–for storing, managing and protecting all the data generated by and with business applications. This is obvious to most storage and networking experts, but not so obvious to everyone else.

IT departments struggle with service levels, utilization rates, capacity and bandwidth limitations, provisioning, data protection challenges, floor space, labor, power and cooling costs, and more–all in support of our applications, and with flat or shrinking budgets. Today’s business applications make IT’s already difficult role damn near impossible.

So what?

So, your internal software development teams as well as those employed by your partners and application vendors, continue to create storage-hungry applications that eat through your IT budgets and business profits. The applications may be optimized for the servers within which they are installed, but I can tell you with 100% confidence that they are not optimized for the storage infrastructures required to support them.

Ignorance is no excuse.

Business applications generate and store an enormous amount of valuable information about the assets they manage–information that could be used by infrastructure vendors to more intelligently manage risk, provision resources, improve service levels, speed up disaster recovery processes, and minimize capital and operational expenses. Yet there exists no two-way collaborative communication between business applications and infrastructure.  Application developers continue to focus on integration and interoperability between business-level applications at the expense of infrastructure applications. Consequently, storage vendors have no choice but to develop systems that attempt to predict the needs of business applications using sparse information gleaned or inferred from server, storage and network monitoring.

Business application vendors and developers remain largely unaware of storage innovations that, through integration and interaction, could improve the performance, reliability and efficiency of their applications. Data Mobility Group has had this conversation with hundreds of vendors and developers over the past eight years, and few show any interest in it.  Some acknowledge the significant potential benefits of storage-awareness, but dismiss it out-of-hand simply because their customers/employers haven’t made enough of a fuss to motivate them to do anything about it. They do so at their customers’ and employers’ expense and at the expense of infrastructure vendors who must continue to rely on guesswork to support storage-ignorant applications.

How much do storage-ignorant business applications really cost? Take the cost of application licensing and support, then multiply it by 5, 10, 20 even 100 times or more and your estimate might be a bit closer to the true cost of ownership.

Applications designed to improve productivity and efficiency at the business level do so at the expense of productivity and efficiency within IT [by inadvertently shifting resource burdens from business to IT and multiplying the impact].

Where do we go from here?

Communication between business applications and IT applications is essential to reap the benefits of as-yet-untapped operational and capital efficiencies. Application developers and vendors must familiarize themselves with storage infrastructure and technologies. They must turn some of their attention and investments to integration and interaction down the technology stack and incorporate the programmatic hooks necessary for infrastructure vendors to cost-effectively manage and support business-critical applications, environments and information assets.  The is a multi-billion dollar long-term market opportunity for value-added resellers, system integrators, vendor professional services and standards developing organizations.

In the meantime, customers must demand that next generation applications leverage innovations in storage and networking through better integration and interaction. The products of vendors and developers who are not on-board should be phased out over the next 2-5 years and replaced with storage-aware, storage-friendly systems as they become available.

For more information:

If you have questions about this article I invite you to contact us and arrange a FREE teleconference to discuss your concerns. We encourage you to invite members of your development team and IT staff, and representatives from your application vendors to join the conversation. Follow us on Twitter.

I have always enjoyed speaking with David West. He’s one of the relatively few people within the storage industry’s sell-side who genuinely seeks to understand information management - the industry from which I leapt into storage.

After I read David’s recent blog post ILM: What’s Old is New Again, in which he wrote about the return of ILM, I responded with the following comment:

“ILM never left, Dave. That’s a storage industry centric perspective as if storage first brought ILM to the table a decade ago.

Managing information over a life cycle existed long before the storage industry began using the term [ILM], and it’ll continue to persist regardless what terminology or focus the industry fancies at the moment.

The storage industry brought at least two important contributions to the table for information management.

The first was a hard dollar ROI. Frankly, the hard dollar ROI for information management was less compelling before storage companies got involved. Information managers can now highlight ILM’s impact on storage and storage management beyond the usual business user productivity claims.  They can now talk in terms of serious savings and bottom line impact.

The second was a forward looking vision of how storage infrastructure could communicate and collaborate with the business application layer to deliver compliance, protection  and preservation at the lowest possible cost.

ILM doesn’t need to make a comeback. It never left. Storage managers need to step up their game and understand how the storage industry’s perspective on ILM complements traditional business-level information management. And information management practitioners need to step up their game as well and begin working more closely with IT to learn about and leverage these complementary technologies.”

The important point to understand is that businesses already use many types of applications designed to manage information over its life cycle (independent of storage considerations). The storage industry takes that concept one step further by mapping life cycle management to storage infrastructure and storage management costs.

The challenges to realizing end-to-end ILM (i.e. business-wide information management) are in the integration between business and storage applications, and the collaboration between information stewards (in business) and information custodians (in IT).

Note:  Technology professionals sometimes object to being referred to as custodians despite the fact that their role in information management is largely custodial. To avoid hurting their delicate feelings try less objectionable titles such as  Information A-Team or Information Geniuses. Remember, your data lives on their real estate.

In his recent Wikibon post following a July 28 Peer Incite: Prevent Unstructured Data from Fueling Business Risk, Dave Vallente warns CIOs of what he calls the “data management trap”.  Thankfully, Dave provided an overview for those of us who were unable to participate.

I agree with Dave that “the starting point for an information management strategy should not be the technology implementation”.  However, I would add that a CIO is not the appropriate person for the job.

In response to Dave’s post I wrote:

“Information management responsibilities should not be delegated to the CIO unless an organization has absolutely no other alternative. The CIO has the wrong skill set and mindset, and in my opinion, a conflict of interest despite some obvious potential synergies.

Back in April I wrote a brief response to Chuck Hollis’s State of the CIO blog post.

Here’s an excerpt…

‘Chief Information Officer - quite possibly one of the greatest business misnomers of all time.

I’m still waiting for someone to explain the role of the ‘I’ in CIO. The word infrastructure is far more appropriate given the CIO’s focus…perhaps with a dotted line to information…

Savvy companies understand that information usually has no ‘real’ champion at the executive table…certainly not the CIO whose skillset is generally not appropriate for, and whose directives may be at odds with, sound corporate information management. Those who can afford to establish roles primarily responsible for IM (e.g., Chief Knowledge Officer or Chief Preservation Officer) eventually do.

If CKOs and CPOs are the architects of IM, you can think of CIOs as the general contractors…

I believe it is absolutely essential that EMC help its customers understand that information stewardship and accountability must be elevated to the C-level. And I would strongly advise against adding these to the CIO’s already hefty list of responsibilities.

Let’s talk about the state of the state of information stewardship and accountability.’

Putting information management decisions into the hands of a CIO is like putting life or death medical decisions into the hands of a health insurer. It’s a long-term recipe for failure.”

What are your thoughts about information management roles and responsibilities? Join the conversation over at Wikibon

Earlier this year I weighed in on the Donatelli/EMC non-compete drama with a brief post about the nature of non-competes.

Since then I have read several new articles and opinions about the topic of non-competes. I am currently following two: Boston.com’s Clause for Concern, and Bijan’s Revised non-compete legislation doesn’t go far enough.

Here’s my take in a nutshell:

I agree that non-competes make no sense for rank-and-file employees (a.k.a. employees at will). I’ve been the victim of a non-compete on at least one occasion during a layoff at the turn of the century.

However, I do believe non-competes are fair game for any employee under a formal employment contract (esp. key personnel, founders and senior executives). It’s a voluntary mutual commitment - a covenant if you will - and there’s no rational reason that non-competition language should be universally excluded from such an arrangement.

Rep Brownsberger should ditch the arbitrary $50k cutoff and craft the bill in the context of employment at will versus employment under formal contract. If employers want non-competes, such a law would make it much harder for them to eliminate employees at the drop of a hat without justifiable cause. And, if employees want better than at-will job security - which is what most senior execs enjoy today - then they can sign on the dotted line. Both sides would get the security they seek at a cost.

I went on to write,

And, to be clear, I’m not talking about severance contracts issued during the layoff of at-will employees. We both know those are usually non-negotiable “take it or leave it” contracts that many employees feel compelled to sign, on short notice, in exchange for less than desirable compensation.

Let’s not confuse those with the compensation packages of executives and founders negotiated up-front as a condition of employment.

And,

If employers want non-competes, I’m suggesting that they be forced to enter into formal employment contracts with new hires (i.e. not at-will) where compensation - including any severance and perks - is negotiated up-front just as it is now for most founders and executives.  At-will employees should have no such requirement as a condition of employment or severance.

To suggest that non-competes should be eliminated completely is shortsighted and naive, in my opinion. Such agreements do have legitimate applications in the context of certain employees, particularly those nearer the top who will acquire broad intimate knowledge of an employer’s strategies, tactics and operations (as was the case with Donatelli and EMC earlier this year).

It would be naive to believe that NDAs and similar instruments designed to protect IP are effective in that context. In contrast, non-competes wholly eliminate the possibility of knowledge misuse and abuse.

As for Massachusetts and California, there is absolutely no way for you and I to meaningfully quantify the impact of their attitudes toward non-competes.  What I’ve read to-date is little more than opinion. Nobody knows for sure what the impact has been, positive or negative.

Do you have an opinion about non-competes? Join one of the conversations at Boston.com or www.bijansabet.com.