Storage Informer
Storage Informer

shame on all of us

by on Jul.16, 2010, under Storage

3.008: shame on all of us

EMC logo


Sometimes we in the storage industry misbehave.

Sometimes Badly.

The most recent example surrounds the reports early this week about how a bank was unable to service its ATM customers as a result of a vendors’ process mistake. Apparently an operator used an out-of-date procedure to execute a routine service operation during a planned outage and the result was an protracted unplanned outage. To their credit, the vendor publicly owned up to the mistake and is certainly taking steps to avoid similar occurrences in the future.

All fine and good, if we could have just left it there.

But no, it seems this is not to be the case. Sales reps from the vendor-at-fault’s competitors are gleefully emailing these reports to every customer and prospect, in hopes of creating sufficient Fear, Uncertainty and Doubt (FUD) about the competitor in the minds of these potential sources of revenue. I personally have had over a dozen emails in my inbox linking to the reports.

I find this deplorable, childish behavior.

And yes, that is directed at folks from my own company as well as those from competitors.

hey, we’re all human here!

Everyone makes mistakes, and whether the fault of a single person or a process or a whole communications chain, none of us are perfect. I know at EMC we work very hard to eliminate such errors, and I’m sure that our competitors try hard as well. But whether we are more successful at avoiding mistakes or not, this does not give any of us the right to exploit human failure – for to be sure, there inevitably will be mistakes made by all of us that have similarly serious impact on a customer.

Don’t get me wrong, though. If a competitors product has a deficiency that causes a problem or if the product simply isn’t fit-for-purpose (e.g., Apple’s iPhone 4), then I say it is fair game. But to imply that a competitors’ products are lessened by a human mistake – No sir, it ain’t right, and I don’t like it.

Hence the title of this post. Shame on us all for this behavior, and let us not wish similar mistakes on anyone. We all work and live in glass houses.

preventative measures

That said, I would like to highlight a couple of things that EMC does to help reduce the probability of human error causing problems. Up front I’ll admit these aren’t necessarily foolproof, and they don’t guarantee 100% success. But our internal metrics indicate that they are helping to minimize the risks and reduce the occurrences:

  • Procedure Generators: perhaps the most relevant tool in the Symmetrix arsenal is the automated procedure generators that we provide to our global customer services and professional services representatives – and to our customers! Available today for Symmetrix, Invista and VPLEX, these generators will create the procedure checklist for a variety of operational, management and repair tasks for the various audiences. To ensure that the latest procedures are used, the Generators automatically check in to EMC’s centralized servers every time they are run (these run on the users’ laptop or PC).
  • Uptime Bulletin: The Uptime Bulletin is a customer-facing periodical that documents EMC’s experiences related to system and data availability. In it, customers find not only statistics, but also recommendations on how best to configure, deploy and maintain their arrays to maximize their availability. Originally pioneered by CLARiiON, it is now also published regularly for Symmetrix, Invista, VPLEX and Celerra.
  • Regulator: Mentioned in an earlier post, the Regulator is capable of automatically stopping the execution of CLI commands that have been identified as problematic or that are operating in an improper order. The database of risk events is updated regularly, and the Regulator checks in via the EMC Secure Remote Service (ESRS) IP gateway used for remote support of virtually all of EMC’s storage platforms.

As I’ve noted, customers, partners and EMC representatives all have access to these tools, and they are key components of the best practices for operating and maintaining EMC’s products. I have no idea whether competitors offer any or all of these utilities, mainly because we don’t typically employ these as competitive differentiators. They exist because customers have asked us to provide them, and they are used because they have positive impact.

My advice to those that are wont to throw stones – remember, we are all humans, we all make mistakes, and when those mistakes have an impact on customers, it is not a good thing for any of us…there is no joy, no victory, no competitive advantage in another vendors’ mishaps.

What is most important is what we do to avoid them happening again…



Update your feed preferences


:, , , , , , ,

Leave a Reply

Powered by WP Hashcash

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...