Explore the details behind the floating‑point operations (FLOP) benchmark for measuring compute performance.
The Top 500 list of supercomputers has come and gone again, and vendors have engaged in their usual round of self-congratulations and performance posting. All the talk of FLOPs can get a little confusing, so we’re going to try and cut through the clutter and the marketing speak.
A FLOP is a single floating‑point operation, meaning one arithmetic calculation (add, subtract, multiply, or divide) on numbers that have decimals. Compute benchmarking is done in floating point/fractional rather than integer/whole numbers because floating point is far more accurate of a measure than integers.
A prefix is added to FLOPs to measure how many are performed in a second, starting with mega- (millions) the giga- (billions), tera- (trillions), peta- (quadrillions), and now exaFLOPs (quintillions). For the last several years, the race was on to reach exaFLOP computing, not because it would mean a magical performance gain or discovery but simply bragging rights.
Floating point in computing starts at FP4, or 4 bits of floating point, and doubles all the way to FP64. There is a theoretical FP128, but it is never used as a measure. FP64 is also referred to as double-precision floating-point format, a 64-bit standard under IEEE 754 for representing real numbers with high accuracy.
Because of the precision required, calculating figures in FP64 takes the longest amount of time. It takes twice as long as FP32 and four times as long as FP16, but the calculations are twice as accurate as FP32 and four times as accurate as FP16. The same applies to memory use; FP64 uses twice as much memory for calculations as FP32 and four times as much memory as FP16.
The Top 500 supercomputer list, compiled and published every June and November, measures supercomputers in FP64, making it the ultimate stress test for these computers.
The reason why the Top 500 list is measured in 64-bits and not 32 or 16 is because the list is a proxy for scientific computing applications, and these applications continue to rely primarily on 64-bit accuracy in their calculations, says Addison Snell, CEO of Intersect 360, a market research firm specializing in HPC and supercomputing.
“There’s always going to be conversations about could we make things faster by reducing precision in some areas, or are you doing an awful lot of calculation in areas where it doesn’t matter, but 64-bit is still the de rigueur standard in scientific computing,” Snell said.
FP64 is used in just about every field of scientific computing, Snell said. On the research side, that includes weather simulation or ocean modeling, which have fluids and require a high degree of accuracy to model.
It applies to a wide range of commercial applications, such as crash simulations of cars, aerodynamic analysis of an airplane wing, seismic analysis of where to drill for oil, or molecular modeling in designing a new drug. All of these are applications that require a high degree of scientific precision in order to calculate, Snell said.
The next step down is FP32, or single precision floating point. It is also used in life sciences simulations as well as financial modeling and is usually used when the demands of the model are not as stringent and the person running the model can get away with using FP32.
FP16 has found a regular use for AI inferencing, whereas AI training is fully dependent on FP64. The reason is simple: Let’s say you were using AI to learn to recognize the image of a dog or a cat. You want the fine precision of FP64 to recognize the features that constitute a dog or a cat. Once it’s trained, then it’s simply a pattern matching effort, and the less demanding FP16 is sufficient for recognizing is this image a dog or a cat, Snell said.
In the case of language learning or recognition, if you’re speaking and mispronounce one word slightly, the AI will still get the general idea of what you are referring to. Otter.ai does a spectacular job of handling the lovely accents of the Silicon Valley. So FP16 can be used in language recognition.
There is a second form of 16-bit precision, refer to as bfloat16. Bfloat was originally developed by Google for its Tensor processors but has since been licensed to Intel, AMD, and Nvidia. Bfloat is a flexible variable, whereas FP16 is the same 16-bit variable every time. It’s all highly technical, but the bottom line is that Bfloat is less precise than FP16 but faster and that’s why all the major chip vendors use it, Snell said.
Finally, there is FP8 and FP4. FP 8 is also used in inference processing in use cases where less precision is required. The other key use is in training neural networks, which are more fault tolerant. It is also used in edge computing, where a less compute-intensive work is done. It is only used in GPU’s; Intel and AMD do not use it in their processors.
With petaFLOPS and exaFLOPs becoming a marketing term, some hardware vendors have been less than scrupulous in disclosing what level of floating-point operation their benchmarks use. It’s not it’s not uncommon for a company to promote exascale performance and then saying the little fine print that they’re talking about FP8, according to Snell.
“It used to be if someone said exaFLOP, you could be pretty confident that they meant exaFLOP according to 64-bit scientific computing, but not anymore, especially in the field of AI, you need to look at what’s going behind that FLOP,” said Snell.
He said he once saw a chart that talked about aggregate AI flops, but there was no standard definition behind about how much a FLOP was and what the precision was. “So once they’re mixing that in terms of individual computers, and one could measure it one way and one could measure it another way, then it becomes meaningless. We’re in a world where you have to ask [the precision],” said Snell.
