The way the power of supercomputers is measured is about to change.
Since 1993, Jack Dongarra, distinguished prof. of computer science at the Univ. of Tennessee, Knoxville, has led the ranking of the world's top 500 supercomputers. The much-celebrated biannual TOP500 list is compiled using Dongarra's benchmark system, called Linpack. It's the most widely recognized and discussed metric for ranking high-performance computing systems.
But Dongarra says Linpack hasn't kept pace with supercomputing needs and must be updated.
"Linpack rankings of computer systems are no longer so strongly correlated to real application performance," Dongarra said.
Dongarra and his colleague Michael Heroux from Sandia National Laboratories in Albuquerque, N.M., are developing a new benchmark that is expected to be released in time for the next TOP500 list release in November.
Linpack measures the speed and efficiency of linear equation calculations. Over time, applications requiring more complex computations have become more common. These calculations require high bandwidth and low latency, and access data using irregular patterns. Linpack is unable to measure these more complex calculations.
"This is an important issue to address since we are seeing more applications being dominated by differential equations, and thus, each iteration of the TOP500 will show increasing gaps between real versus Linpack performance," said Dongarra.
The new benchmark, called the High Performance Conjugate Gradient (HPCG), should better correlate to computation and data access patterns found in many applications today. This could improve the high-performance community, because some of today's supercomputers are being designed to gain a high score for the TOP500 list than to meet real-life demands.
"We have reached a point where designing a system for good Linpack performance can actually lead to design choices that are wrong for the real application mix, or add unnecessary components or complexity to the system," said Dongarra. "The hope is that this new rating system will drive computer system design and implementation in directions that will better impact performance improvement for real applications."
Dongarra also expects the new benchmark to adapt to emerging trends, unlike its predecessor.
"The Linpack benchmark is an incredibly successful metric for the high-performance computing community," said Dongarra. "The trends it exposes, the focused optimization efforts it inspires and the publicity it brings to our community are very important. Yet the relevance of the Linpack as a proxy for real application performance has become very low, creating a need for an alternative."
Linpack will not be laid to rest, however. Instead, HPCG will serve as an alternative ranking of the TOP500 list, allowing a re-ranking of the systems on the list to "real" applications.
The next Top500 list will be announced at the Supercomputing Conference - SC2013, to be held in November in Denver, Colo.