Dissecting Intel's EPYC Benchmarks: Performance Through the Lens of Competitive Analysis
by Johan De Gelas & Ian Cutress on November 28, 2017 9:00 AM EST- Posted in
- CPUs
- AMD
- Intel
- Xeon
- Skylake-SP
- Xeon Platinum
- EPYC
- EPYC 7601
HPC Benchmarks
Discussing HPC benchmarks feels always like opening a can of worms to me. Each benchmark requires a thorough understanding of the software and performance can be tuned massively by using the right compiler settings. And to make matters worse: in many cases, these workloads can be run much faster on a GPU or MIC, making CPU benchmarking in some situations irrelevant.
NAMD (NAnoscale Molecular Dynamics) is a molecular dynamics application designed for high-performance simulation of large biomolecular systems. It is rather memory bandwidth limited, as even with the advantage of an AVX-512 binary, the Xeon 8160 does not defeat the AVX2-equipped AMD EPYC 7601.
LAMMPS is classical molecular dynamics code, and an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator. GROMACS (for GROningen MAchine for Chemical Simulations) primarily does simulations for biochemical molecules (bonded interactions). Intel compiled the AMD version with the Intel compiler and AVX2. The Intel machines were running AVX-512 binaries.
For these three tests, the CPU benchmarks results do not really matter. NAMD runs about 8 times faster on an NVIDIA P100. LAMMPS and GROMACS run about 3 times faster on a GPU, and also scale out with multiple GPUs.
Monte Carlo is a numerical method that uses statistical sampling techniques to approximate solutions to quantitative problems. In finance, Monte Carlo algorithms are used to evaluate complex instruments, portfolios, and investments. This is a compute bound, double precision workload that does not run faster on a GPU than on Intel's AVX-512 capable Xeons. In fact, as far as we know the best dual socket Xeons are quite a bit faster than the P100 based Tesla. Some of these tests are also FP latency sensitive.
Black-Scholes is another popular mathematical model used in finance. As this benchmark is also double precision, the dual socket Xeons should be quite competitive compared to GPUs.
So only the Monte Carlo and Black Scholes are really relevant, showing that AVX-512 binaries give the Intel Xeons the edge in a limited number of HPC applications. In most HPC cases, it is probably better to buy a much more affordable CPU and to add a GPU or even a MIC.
The Caveats
Intel drops three big caveats when reporting these numbers, as shown in the bullet points at the bottom of the slide.
Firstly is that these are single node measurements: One 32-core EPYC vs 20/24-core Intel processors. Both of these CPUs, the Gold 6148 and the Platinum 8160, are in the ball-park pricing of the EPYC. This is different to the 8160/8180 numbers that Intel has provided throughout the rest of the benchmarking numbers.
The second is the compiler situation: in each benchmark, Intel used the Intel compiler for Intel CPUs, but compiled the AMD code on GCC, LLVM and the Intel compiler, choosing the best result. Because Intel is going for peak hardware performance, there is no obvious need for Intel to ensure compiler parity here. Compiler choice, as always, can have a substantial effect on a real-world HPC can of worms.
The third caveat is that Intel even admits that in some of these tests, they have different products oriented to these workloads because they offer faster memory. But as we point out on most tests, GPUs also work well here.
Facebook
Twitter
RSS
105 Comments
View All Comments
Ashari - Tuesday, November 28, 2017 - link
LOL, "GloFo 16nm"... tsts, one would think people like Johan De Gelas and Ian Cutress would know which node is GloFo and which one is TSMCIan Cutress - Tuesday, November 28, 2017 - link
That's my brain fart. I've been writing about other things recently. Edited.peevee - Tuesday, November 28, 2017 - link
"The benchmarking scenario also has a big question mark, as in the footnotes to the slides Intel achieved this victory by placing 58 VMs on the Xeon 8160 setup versus 42 VMs on the EPYC 7601 setup."Given how well AMDs SMT scales, a real client can put up to 128 single-CPU VMs on the EPIC 7601, and 58 VMs on Xeon 8160 would be tramped ridiculously.
Here Intel just had to rely on the shenanigans so obvious it is just fraud.
LordOfTheBoired - Tuesday, November 28, 2017 - link
Yeah, that really stuck out for me too. "We outperform AMD when running a different benchmark!"And to be frank, it casts a pall over Intel's entire PR release since it IS blatantly not how benchmarks work.
Andresen - Tuesday, November 28, 2017 - link
Many HPC task are memory bandwidth limited, and then AVX-512 is of little help. In Spec.org CFP2006 none of the recent results are using AVX-512 but instead rely on AVX2. The few tests posted using AVX-512 come out worse than the tests on similar systems using AVX2. For memory bandwidth limited tasks the EPYC has an advantage with its 8 memory channels compared to Intels 6 channels. For both architectures, a high end processor is not needed for bandwidth limited task, since the don't offer more memory channels.Johan Steyn - Monday, December 18, 2017 - link
AVX also heats up the CPU a lot and it has to throttle down. With AVX, Intel cannot run high clock speesds.ddriver - Tuesday, November 28, 2017 - link
Just when you think AT cannot possibly sink any lower, they now directly publish publish intel benchmarks of a competing product.Coldfriction - Tuesday, November 28, 2017 - link
I myself was confused and dissapointed reading the summary where agreement with Intel seems to be presented by the authors. Using prases like "there is no denying that the Intel Xeon is a 'safer bet' for VMware virtualization" without testing it pushes AT into the realm of paid for shills. Independent reviews wouldn't trust anyone's marketing and even if they were to publish an article on benchmarks from a competitor, they would fill the thing with hefty amounts of skepticism until they could test it themselves. What Intel presents could very realistically be true (personally, I don't doubt that their benchmarks are within the ballpark of being legit), but I want my independent review sites to have as little bias as possible and that means objectively testing the hardware and ignoring the marketing.wumpus - Wednesday, November 29, 2017 - link
These type of servers are rarely bought by customers for personal use. Instead, they are bought for a 'real job' where CYA decisions outweigh any performance benefits (to a degree, the end product has to work). If something really goes wrong, you can always expect to get the blame for buying the "off brand" instead of following the sheep, regardless of what really caused the failure (typically with highly annoyed management who can't tell *anything* about the server than it is the "off brand").If this isn't a consideration you have a "great job". Expect the owner to sell at some point or expand to the point it is controlled by MBAs and downgrade everybody's job to a "real job'. Sorry to say, but at least in the USA that is life.
Johan Steyn - Monday, December 18, 2017 - link
People sometimes really surprise me. What support doe you want from AMD? Yes if there is a booboo like Intel has (present tense) with its security flaw, you need support from them. I have sold numerous systems and servers in my life and never did I go to AMD or Intel to ask for support. It either the OEM, component supplier or component manufacturer (like motherboards etc) who you go to for support.If the CPU works as it should, you do not need support. CPU's were in my experience the one component that rarely if ever dies on you. So if you trust Tyan to make good products, which they do, they are the ones to give you support, not AMD. AMD has to help with Bioses etc. with which they are very good.
So please stop with this support issue and safer bet. If the system runs unstable because of hardware issues, sure they have to sort it out, but till now, none has been reported.
What has Intel done about the bug recently found? Did they come to you to fix it and support you? Nope, you have to fix it yourself, that is if the motherboard manufacturer has a bios update. So, for me it looks like AMD might just be the safer bet after all...