An effort by PC makers to move beyond a single, simple speed measurement for showing computing performance is now being matched by a similar push in the world of supercomputers–and not everyone is applauding the change.
An organizer of the Top500 supercomputer rankings has produced a broader test suite that measures multiple dimensions of a machine’s performance. By comparison, a mathematical test called Linpack is currently used to rank systems on the Top500 list, which is released twice a year with much fanfare.”For a long time it’s been clear to all of us (that) we needed to have more than just Linpack,” said Jack Dongarra, a University of Tennessee professor who helped create Linpack and who’s now working on a suite of tests that go beyond pure number-crunching prowess. “No single number can reflect the overall performance of a machine.”
he government-sponsored test suite for supercomputers, called the HPC Challenge Benchmark, has pleased some supercomputer makers, such as Cray. But IBM, which is moving aggressively into the supercomputing market and is featured more prominently on the Top500, is more cautious.
The new suite of seven tests won’t replace Linpack as the Top500 yardstick, Dongarra said. For one thing, the decades-old Linpack permits historical comparisons in high-performance computing, or HPC, and for another, a system that can’t get a high Linpack won’t do well on other tests, he said.
News source: CNet The new tests grew out of a program the United States government launched after being spooked by a Japanese supercomputer called Earth Simulator, which has topped the Top500 since 2002. The program, funded by the Defense Advanced Research Projects Agency (DARPA), has awarded grants to IBM, Cray and Sun Microsystems to develop new supercomputer designs.
“It was done for DARPA and the National Science Foundation and the Department of Energy. They wanted something to measure the overall effectiveness of computers designed for the program, and they realized that Linpack was not good enough,” Dongarra said.
The next Top500 list is scheduled for release Sunday as the International Supercomputer Conference begins in Heidelberg, Germany.
It’s not the first time the benchmark suite idea has been raised. Erich Strohmaier, another Top500 organizer, endorsed a composite test in 2000 to supplement Linpack.
