AMD Threadripper Pro Review: An Upgrade Over Regular Threadripper?
by Dr. Ian Cutress on July 14, 2021 9:00 AM EST- Posted in
- CPUs
- AMD
- ThreadRipper
- Threadripper Pro
- 3995WX
Since the launch of AMD’s Threadripper Pro platform, the desire to see what eight channels of memory brings to compute over the regular quad-channel Threadripper has been an intriguing prospect. Threadripper Pro is effectively a faster version of AMD’s EPYC, limited for single CPU workstation use, but also heralds a full 280 W TDP to match the frequencies of the standard Threadripper line. There is a 37% price premium from Threadripper to Threadripper Pro, which allows for ECC memory support, double the PCIe lanes, and double the memory bandwidth. In this review, we’re comparing every member of both platforms that is commercially available.
Threadripper Pro: Born of Need
When AMD embarked upon its journey with the new Ryzen portfolio, the delineation of where each product sat in the traditional market has not always been entirely clear. The first generation Ryzen was earmarked for standard consumers, however the top of the line Ryzen 7 1800X, with eight cores, competed against Intel’s high-end desktop market. The Zen 2-based portfolio saw the mainstream Ryzen go to 16 cores, pushing past Intel’s best 18-core HEDT processor at the time in most tests. That Zen 2-based Ryzen 9 3950X was still classified as a ‘mainstream platform’ processor, as it only had 24 PCIe lanes and dual-channel memory, sufficient for mainstream users but not enough for workstation markets. These mainstream processors were also limited to 105W TDP.
At the other end of the scale was AMD EPYC, with the first generation EPYC 7601 having 32 cores, and the second generation EPYC 7742 having 64 cores, up to 225W TDP. These share the same LGA4094 socket, have eight channels of memory, full ECC support, and 128 PCIe lanes (first PCIe 3.0, then PCIe 4.0), with dual-socket support. For workstation users interested in EPYC, AMD launched single socket ‘P’ versions. These offered the same features, at around 200 TDP, losing some performance to the regular non-P versions.
AMD then launched Threadripper, a high-end desktop version of EPYC that went all the way up to 280 W for peak frequency and performance. Threadripper sat above Ryzen with 64 PCIe lanes and quad channel memory, enabling mainstream users that wanted a bit more to get a bit more. However workstation users noted that while 280 W was great, it lacked official ECC memory support, and compared to EPYC, sometimes the reduced memory channel support and reduced PCIe compared to EPYC stopped Threadripper being adopted.
So enter Threadripper Pro, which sits between Threadripper and EPYC, and in this instance, very much more on the EPYC side. Threadripper Pro has almost all the features of AMD’s EPYC platform, but in a 280W thermal envelope. It has eight channels of memory support, all 128 PCIe 4.0 lanes, and can support ECC. The only downside to EPYC is that it can only be used in single socket systems, and the peak memory support is halved (from 4 TB to 2 TB). Threadripper Pro also comes at a small price premium as well.
AMD Comparison | ||||
AnandTech | Ryzen | Threadripper | Threadripper Pro |
Enterprise EPYC |
Cores | 6-16 | 32-64 | 12-64 | 16-64 |
Architecture | Zen 3 | Zen 2 | Zen 2 | Zen 3 |
1P Flagship | R9 5950X |
TR 3990X |
TR Pro 3995WX | EPYC 7713P |
MSRP | $799 | $3990 | $5490 | $5010 |
TDP | 105 W | 280 W | 280 W | 225 W |
Base Freq | 3400 MHz | 2900 MHz | 2700 MHz | 2000 MHz |
Turbo Freq | 4900 MHz | 4300 MHz | 4200 MHz | 3675 MHz |
Socket | AM4 | sTRX40 | sTRX4: WRX80 | SP3 |
L3 Cache | 64 MB | 256 MB | 256 MB | 256 MB |
DRAM | 2 x DDR4-3200 | 4 x DDR4-3200 | 8 x DDR4-3200 | 8 x DDR4-3200 |
DRAM Capacity | 128 GB | 256 GB | 2 TB, ECC | 4 TB, ECC |
PCIe | 4.0 x20 + chipset |
4.0 x56 + chipset | 4.0 x120 + chipset | 4.0 x128 |
Pro Features | No | No | Yes | Yes |
One of the biggest pulls for Threadripper and Threadripper Pro has been any market that typically uses high-speed workstations and can scale their workloads. Speaking to a local OEM, the demand for Threadripper and Threadripper Pro from the visual effects industry has been off the charts, where these companies are ripping out their old infrastructure and replacing anew with AMD. This has also been spurned by the recent pandemic, where these studios want to keep the expensive hardware onsite and allow their artists to work from home via remote access.
Threadripper Pro CPUs: Four Models, Three at Retail
When TR Pro launched in 2020, the processors were a Lenovo exclusive for the P620 workstation. The deal between Lenovo and AMD was not disclosed, however it would appear that the exclusivity deal ran for six months, from September to February, with the processors being made retail available on March 2nd.
During that time, we were sampled one of these workstations for review, and it still remains one of the best modular systems I’ve ever tested:
Lenovo ThinkStation P620 Review: A Vehicle for Threadripper Pro
AMD’s first Threadripper Pro platform has four processors in it, ranging from 12 cores to 64 cores, mimicking their equivalents in Threadripper 3000 and EPYC 77x2 but at 280W.
AMD Ryzen Threadripper Pro | |||||||
AnandTech | Cores | Base Freq |
Turbo Freq |
Chiplets | L3 Cache |
TDP | Price SEP |
3995WX | 64 / 128 | 2700 | 4200 | 8 + 1 | 256 MB | 280 W | $5490 |
3975WX | 32 / 64 | 3500 | 4200 | 4 + 1 | 128 MB | 280 W | $2750 |
3955WX | 16 / 32 | 3900 | 4300 | 2 + 1 | 64 MB | 280 W | $1150 |
3945WX | 12 / 24 | 4000 | 4300 | 2 + 1 | 64 MB | 280 W | OEM |
Sitting at the top is the 64-core Threadripper Pro 3995WX, with a 2.7 GHz base frequency and a 4.2 GHz turbo frequency. This processor is the only one in the family to have all 256 MB of L3 cache, as it has all eight chiplets fully active. The $5490 price is a full 37.5% increase over the Threadripper 3990X at $3990.
AMD 64-Core Zen 2 Comparison | |||
AnandTech | Threadripper 3990X |
Threadripper Pro 3995WX |
EPYC 7702P |
MSRP | $3990 | $5490 | $4425 |
TDP | 280 W | 280 W | 200 W |
Base Freq | 2900 MHz | 2700 MHz | 2000 MHz |
Turbo Freq | 4300 MHz | 4200 MHz | 3350 MHz |
L3 Cache | 256 MB | 256 MB | 256 MB |
DRAM | 4 x DDR4-3200 | 8 x DDR4-3200 | 8 x DDR4-3200 |
DRAM Capacity | 256 GB | 2 TB, ECC | 4 TB, ECC |
PCIe | 4.0 x56 + chipset | 4.0 x120 + chipset | 4.0 x128 |
Pro Features | No | Yes | Yes |
Middle of the line is the 32-core Threadripper Pro 3975WX, with a 3.5 GHz base frequency and a 4.2 GHz turbo frequency. AMD decided to make this processor use four chiplets with all eight cores on each chiplet, leading to 128 MB of L3 cache total. At $2750, it is also 37.5% more expensive than the equivalent 32-core Threadripper 3970X.
AMD 32-Core Zen 2 Comparison | |||
AnandTech | Threadripper 3970X |
Threadripper Pro 3975WX |
EPYC 7501P |
MSRP | $3990 | $2750 | $2300 |
TDP | 280 W | 280 W | 180 W |
Base Freq | 3700 MHz | 3500 MHz | 2500 MHz |
Turbo Freq | 4500 MHz | 4200 MHz | 3350 MHz |
L3 Cache | 128 MB | 128 MB | 128 MB |
DRAM | 4 x DDR4-3200 | 8 x DDR4-3200 | 8 x DDR4-3200 |
DRAM Capacity | 256 GB | 2 TB, ECC | 4 TB, ECC |
PCIe | 4.0 x56 + chipset | 4.0 x120 + chipset | 4.0 x128 |
Pro Features | No | Yes | Yes |
The following two processors have no Threadripper equivalents, but also represent a slightly different scenario that we’ll explore in this review. Both the 3955WX and 3945WX, despite being part of the big Threadripper Pro family, only use two chiplets in their design: 8 core per chipet for the 3955 WX and 6 core per chiplet for the 3945WX. This means these processors only have 64 MB of L3 cache, making them somewhat identical to the Ryzen 9 3950X and Ryzen 9 3900X, except the IO die means there is eight channels of memory and 128 PCIe lanes here.
AMD 16-Core Zen 2/3 Comparison | |||
AnandTech | Ryzen 9 3950X |
Threadripper Pro 3955WX |
Ryzen 9 5950X |
MSRP | $749 | $1150 | $799 |
TDP | 105 W | 280 W | 105 W |
Base Freq | 3500 MHz | 3900 MHz | 3400 MHz |
Turbo Freq | 4700 MHz | 4300 MHz | 4900 MHz |
L3 Cache | 64 MB | 64 MB | 64 MB |
DRAM | 2 x DDR4-3200 | 8 x DDR4-3200 | 2 x DDR4-3200 |
DRAM Capacity | 128 GB | 2 TB, ECC | 128 GB |
PCIe | 4.0 x20 + chipset |
4.0 x120 + chipset |
4.0 x20 + chipset |
Pro Features | No | Yes | No |
Motherboard Cost | -- | +++ | -- |
The 3955WX has a higher base frequency, but the 3950X has the higher turbo frequency. The 3950X is also cheaper, and motherboards are cheaper! It might be worth partitioning these out into a separate comparison review.
The final Threadripper Pro processor, the 3945WX, does not have a price, because AMD is not making it available at retail. This part is for selected OEM customers only it seems; perhaps the limited substrate resources in the market right now makes it unappealing to make too many of these? Hard to say.
Motherboards: Beware!
Despite being based on the same LGA4094 socket as both Threadripper and EPYC, Threadripper Pro has its own unique WRX80 platform that has to be used instead. Only select vendors seem to have access/licenses to make WRX80 motherboards, and your main options are:
- ASUS Pro WS WRX80E-SAGE SE WiFi ($1000)
- Supermicro M12SWA-TF (~$750)
- GIGABYTE WRX80 SU8-IPMI ($790)
All three boards use a transposed LGA4094 socket, eight DDR4 memory slots, and 6-7 PCIe 4.0 slots.
Though beware! There is an option of finding an old/refurbished Lenovo P620 motherboard. It is worth noting that Lenovo is exercising an AMD feature for OEMs: processors used in that Lenovo motherboard will be locked to Lenovo forever. This is part of AMD’s guaranteed supply chain process, allowing OEMs to hard lock processors into certain vendors for supply chain end-to-end security that is requested by specific customers. In that instance, if you might ever want to break down your system to upgrade and sell off parts, it is not recommended you find a Lenovo TR Pro system unless you buy/sell it as a whole.
This Review
The main goal of this review is to test all of the Threadripper Pro 3000 hardware and compare against the equivalent Threadripper 3000 to get a sense of how much performance is gained by the increased memory bandwidth, or lost due to the slight core frequency differences. We are also including Intel’s best HEDT/workstation processor for comparison, the W-3175X, as well as the top consumer-grade processors on the market. All systems are tested at JEDEC specifications.
Test Setup | |||||
AMD TR Pro |
3995WX 3975WX 3955WX |
ASUS Pro WS WRX80E-SAGE SE WiFi |
BIOS 0405 |
IceGiant Thermosiphon |
Kingston 8x16 GB DDR4-3200 ECC |
AMD TR |
TR 3990X TR 3970X TR 3960X |
ASRock TRX40 Taichi |
BIOS P1.70 |
IceGiant Thermosiphon |
ADATA 4x32 GB DDR4-3200 |
AMD Ryzen |
R9 5950X | GIGABYTE X570 I Aorus Pro |
BIOS F31L |
Noctua NH-U12S |
ADATA 4x32 GB DDR4-3200 |
Intel Core |
i9-11900K | ASUS Maximus XIII Hero |
BIOS 0703 |
Thermalright TRUE Copper* |
ADATA 4x32 GB DDR4-3200 |
Intel Xeon |
Xeon W-3175X | ASUS ROG Dominus Extreme |
BIOS 0601 | Asetek 690LX-PN |
DDR4-2666 ECC |
GPU | Sapphire RX 460 2GB (CPU Tests) | ||||
PSU | Various (inc. Corsair AX860i) | ||||
SSD | Crucial MX500 2TB | ||||
*Silverstone SST-FHP141-VF 173 CFM fans also used. Nice and loud. |
Many thanks to Kingston for supplying a full set of KSM32RD8/16MEI - 16x16 GB of DDR4-3200 ECC RDIMMs for enterprise testing in systems like Threadripper Pro.
As part of this review, we are also showcasing the 64 core processors in 128T mode as well as 64T mode. This is being done to showcase how some processors can get better performance by having better memory bandwidth per thread - one of the issues with these high core count processors is the limited amount of memory bandwidth each thread can access. Also, some operating systems (such as Windows) struggle above 64 threads due to the use of thread groups.
98 Comments
View All Comments
DesireeTR - Wednesday, July 14, 2021 - link
OK, found the news too. IDK if I can link any other website here other than Anandtech, but look for "Lenovo is Using AMD PSB to Vendor Lock AMD CPUs" from servethehome, dated April 5th 2021. Lenovo P620 with Threadripper Pro was tested and found that they used the strict PSB lock-in like Dell do on their PowerEdge servers.Threska - Wednesday, July 14, 2021 - link
I think "permanently" is the biggest concern, otherwise it could be a great feature as part of a "root of trust" if the user could control it, especially via hardware modification. e.g. jumper.DesireeTR - Wednesday, July 14, 2021 - link
Yeah, and if this trend continues, the Ryzen PRO definitely is next on the line getting this PSB. Laptops might be OK, since they use soldered BGA processor anyway, but definitely a big no no for prebuild towers.arashi - Saturday, July 17, 2021 - link
If it can be overridden like that then it isn't a root of trust anymore.Threska - Saturday, July 17, 2021 - link
There's the presumption you trust yourself.Mikewind Dale - Wednesday, July 14, 2021 - link
Dear Anandtech: If you ever review the motherboards, I'll relate something a few things I discovered about the Supermicro M12SWA-TF:First, it cannot use sleep mode. If you put the computer to sleep, then when you wake it up, the fans will all spin at low RPM, and they will fail to adjust to temperature. HWiNFO64 reports two sets of sensors: one set is direct, and the other is indirect, via the IPMI. After waking from sleep, the direct sensor readings were still reported, but the indirect-via-IPMI sensors were all null. When I logged into the BMC/IPMI, all the sensors were null there too. And when I ran a CPU burn-in after waking from sleep, my CPU temperature quickly climbed higher than normal, and the fans did NOT ramp up their RPM. (I was prepared for this, so I was running only a single-threaded CPU benchmark.)
Not only did rebooting the computer fix the problem, but so did Windows hibernate. The fact that Windows hibernate fixed the problem told me that the problem was hardware, not OS.
I contacted Supermicro, and they said this behavior is normal (!!!!!!). They explained that the IPMI controls the fan RPM, but it only connects to the sensors during POST. If you put the computer to sleep, the IPMI loses its connection to the sensors, and it cannot resume that connection until the computer POSTs again.
So if you review the motherboards, make sure to test the sleep behavior.
Second, the Supermicro board is programmed with critical low fan RPM threshholds that are lower than Noctua's RPM. If you Google, you'll see a lot of people have problems with using Noctua fans with Supermicro boards. What happens is, the the Noctua fan's RPM will drop below the critical low RPM threshholds, so the Supermicro board will think the fan is failing, and it will quickly ramp the fan up to 100% PWM. Once the fan exceeds the critical low RPM threshold, the alert will end, and the fan will drop its RPM back down again, starting the cycle over. So the fans cycle back and forth between high and low RPM. When I logged into the IPMI, I saw that I every single fan was triggering the low RPM alert every few seconds.
The solution is to reprogram the IPMI with new critical low RPM thresholds. Supermicro's own IPMI software does NOT allow this, because Supermicro explained to me that some people have overheated and fried their motherboards using insufficient cooling. So I had to use a third-party tool called "ipmitool".
Usually, ipmitool is obtained via "sudo apt-get install ipmitool". However, I found that the Linux version was unable to establish a connection with my BMC, even though other IPMI tools had no problem with establishing that connection. But other IPMI tools did not have the ability to reprogram the fan thresholds.
Luckily, the Windows version of ipmitool was able to establish a connection and alter my fan thresholds just fine. The Windows version is available at https://www.dannynieuwenhuis.nl/download-windows-i...
If you Google, you'll find many, many different websites offering instructions for how to use ipmitool to modify your Supermicro board to be compatible with Noctua fans. I'll just give a few sample lines of code here, in case anyone needs them:
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN1 lower 40 140 240
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN1 upper 1650 1750 1850
Where:
--- FAN1 is the name of the fan header, as labeled in the motherboard manual. Options are FAN1-FAN6 and FANA-FAND.
--- "lower" numbers are lower non-recoverable, lower critical, and lower non-critical, in that order.
--- "upper" numbers are upper non-critical, upper critical, and upper non-recoverable, in that order.
To calculate the thresholds, I did the following:
First, I looked up Noctua's specs. FAN1 is my Noctua NH-U14S TR4-SP3. According to Noctua, its fan's RPM are 300 +/-20% to 1500 +/- 10% RPM.
Second, I set the lower non-critical to 300*0.8 (i.e. -20%) and the upper non-critical to 1500*1.1 (i.e. +10%).
Third, for the critical and non-recoverable thresholds, I just added or subtracted 100%.
Do the same for every other fan in every other header. I wrote about every line in a .BAT file in Windows, which read like this:
REM **************************************************************************************************
REM **********
REM FAN1 is Noctua NH-U14S TR4-SP3: 300 +/-20% to 1500 +/- 10% RPM
REM **********
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN1 lower 40 140 240
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN1 upper 1650 1750 1850
REM **************************************************************************************************
REM **************************************************************************************************
REM **********
REM FAN2 is Noctua NF-A15: 300 +/- 20% to 1200 +/- 10%
REM **********
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN2 lower 40 140 240
ipmitool -I lanplus -H <ipaddress> -U <username> -P <password> sensor thresh FAN2 upper 1320 1420 1520
REM **************************************************************************************************
and so forth, for every fan header. This successfully solved the problem of the fans triggering the threshold alerts and cycling up and down.
Mikewind Dale - Wednesday, July 14, 2021 - link
"Second, the Supermicro board is programmed with critical low fan RPM threshholds that are lower than Noctua's RPM."I meant *higher*. The Supermicro default critical low fan RPM thresholds are *higher* than Noctua's.
Mikewind Dale - Wednesday, July 14, 2021 - link
Oh, and because sleep mode is dangerous, threatening to potentially fry your CPU (since the fans no longer respond to temperature), I not only set my computer never to sleep, but I removed sleep from the power options in the start menu. That way, I cannot accidentally put the computer to sleep.If you do ever put your Supermicro M12SWA-TF to sleep, you will not receive any alerts that every sensor is null. If you log into the BMC, you'll see every sensor is null, but there are no alerts. And the fans all spin at minimum RPM regardless of your fan setting, and regardless of temperature. So sleep mode appears to have the potential to fry your CPU.
Threska - Wednesday, July 14, 2021 - link
You keep saying "fry" but haven't CPUs had thermal protection for ages at this point?Mikewind Dale - Wednesday, July 14, 2021 - link
Threska, possibly. But I didn't want to find out.At best, sleep mode would cause the computer to constantly downclock or shut down without any clear cause (unless the user realized it was because sleep mode deactivated the IPMI's reporting of the sensors while the sensors themselves were still reporting values to software such as HWiNFO).