• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • About
    • Stephen Foskett
      • My Publications
        • Urban Forms in Suburbia: The Rise of the Edge City
      • Storage Magazine Columns
      • Whitepapers
      • Multimedia
      • Speaking Engagements
    • Services
    • Disclosures
  • Categories
    • Apple
    • Ask a Pack Rat
    • Computer History
    • Deals
    • Enterprise storage
    • Events
    • Personal
    • Photography
    • Terabyte home
    • Virtual Storage
  • Guides
    • The iPhone Exchange ActiveSync Guide
      • The iPhone Exchange ActiveSync Troubleshooting Guide
    • The iPad Exchange ActiveSync Guide
      • iPad Exchange ActiveSync Troubleshooting Guide
    • Toolbox
      • Power Over Ethernet Calculator
      • EMC Symmetrix WWN Calculator
      • EMC Symmetrix TimeFinder DOS Batch File
    • Linux Logical Volume Manager Walkthrough
  • Calendar

Stephen Foskett, Pack Rat

Understanding the accumulation of data

You are here: Home / Everything / Enterprise storage / Innocence, Fairness, and Technology Benchmarks

Innocence, Fairness, and Technology Benchmarks

February 26, 2010 By Stephen 4 Comments

HP recently commissioned Tolly Group to benchmark their BladeSystem c7000 against the Cisco UCS 5100. The short report focuses on two results, and reads like so many competitive benchmarks in the IT industry: Tolly focuses on metrics that highlight the strength of HP’s solution and the weaknesses of Cisco’s. I do not dispute the accuracy of these results, and HP and Tolly are doing exactly what tech companies do. But what’s the real value of pinpoint maximum-performance benchmarks like this?

0-100-0

Automotive media like Car and Driver and Top Gear frequently test the maximum performance of cars, racing to 100 mph or beyond, sliding around a skidpad, and slamming on the brakes. These tests can be enlightening when it comes to high-performance cars, and the punishing 0-100-0 test is especially impressive. But what’s the point of hammering an economy car or pickup truck like this? Maximal acceleration and cornering are entirely irrelevant to buyers of commuter cars and work vehicles.

Even though a given test can be conducted, it may not be enlightening. The Tolly report demonstrates two key findings:

  1. Although 4-blade configurations perform the same under maximum stress, Cisco UCS performance declines with 6 blades while HP’s remains steady.
  2. When using a shared blade uplink, Cisco UCS performance fell by half.

These are not startling results. Cisco blades sometimes need to share one I/O channel, and this can’t match the performance of an HP blade with dedicated I/O. Would it shock you to learn that a one-gallon bucket requires twice as many trips to the well as one that holds two gallons? Does it shock anyone to learn that a V6-powered Toyota RAV4 accelerates quicker than a four cylinder Honda CR-V? HP’s c7000 is bigger than Cisco’s UCS and offers more I/O channels, so HP beats Cisco whenever larger configurations with more I/O are tested.

Innocent Benchmarks

Greta examines the marks on an 18th century cooper's bench

I’ll leave the deeper commentary on blade performance to experts like Kevin Houston and Martin Macleod, but these maximum-utilization benchmarks are only half the story. I’m much more interested in how the different approaches to I/O impact everyday (20%-40% load) performance and how oversubscription impacts performance as more blades are installed and workloads are moved around. In automotive terms, I’d like to know how well a car handles in the snow or how economical it is with three or four passengers. These real-world scenarios are much more telling than a test of a few blades under 100% load!

Clearly, HP wanted to call attention to specific shortcomings of a competitor’s product, and it was wise to do so with objective numbers instead of mudslinging and name-calling. I hope that future tests and releases include real-world workloads and logical configurations, not the extreme situation used in this report. The same lesson applies to all tech companies: Simple, objective tests of maximum performance are welcome, but customers need many more metrics!

Note: Along with 9 other independent bloggers, I attended HP’s Blades Tech Day in Houston on February 25 and 26. Most of my travel and living expenses were paid for by HP, and the company provided a small gift bag (pictured here).

You might also want to read these other posts...

  • Electric Car Over the Internet: My Experience Buying From…
  • How To Connect Everything From Everywhere with ZeroTier
  • Tortoise or Hare? Nvidia Jetson TK1
  • Liberate Wi-Fi Smart Bulbs and Switches with Tasmota!
  • Introducing Rabbit: I Bought a Cloud!

Filed Under: Enterprise storage, Gestalt IT, Virtual Storage Tagged With: benchmarks, blade servers, BladeSystem, Cisco, Greta, HP, I/O, performance, Tolly, UCS

Primary Sidebar

It is often easier to ask for forgiveness than to ask for permission.

Grace Hopper

Subscribe via Email

Subscribe via email and you will receive my latest blog posts in your inbox. No ads or spam, just the same great content you find on my site!
 New posts (daily)
 Where's Stephen? (weekly)

Download My Book


Download my free e-book:
Essential Enterprise Storage Concepts!

Recent Posts

How To Install ZeroTier on TrueNAS 12

February 3, 2022

Scam Alert: Fake DMCA Takedown for Link Insertion

January 24, 2022

How To Connect Everything From Everywhere with ZeroTier

January 14, 2022

Electric Car Over the Internet: My Experience Buying From Vroom

November 28, 2020

Powering Rabbits: The Mean Well LRS-350-12 Power Supply

October 18, 2020

Tortoise or Hare? Nvidia Jetson TK1

September 22, 2020

Running Rabbits: More About My Cloud NUCs

September 21, 2020

Introducing Rabbit: I Bought a Cloud!

September 10, 2020

Remove ROM To Use LSI SAS Cards in HPE Servers

August 23, 2020

Test Your Wi-Fi with iPerf for iOS

July 9, 2020

Symbolic Links

    Featured Posts

    Replacing Google Reader With Feedbin and Reeder

    May 5, 2013

    Electric Car Over the Internet: My Experience Buying From Vroom

    November 28, 2020

    A High-Tech Water Heater? Yep! Introducing the A. O. Smith Vertex

    November 15, 2012

    Frequent Flier Kung Fu for Novices

    March 12, 2012

    Virtualized and Distributed Storage: This Time For Sure!

    September 2, 2014

    Why Buy a NEX-7? Why Sony NEX At All?

    October 17, 2011

    Generation 3 drobo: Fall In Love All Over Again

    April 9, 2015

    A Watch Guy’s Review of the Apple Watch

    April 27, 2015

    My Core i7 Macintosh SE

    May 25, 2017

    It’s Time To Speak Out Against Sexism In IT Recruiting

    May 6, 2013

    Footer

    Legalese

    Copyright © 2022 ยท Log in