At a recent financial conference, AMD CTO Mark Papermaster was asked about a report of a memory bug with the company’s EPYC Genoa processors that would apparently require a lengthy redesign/re-rotation process to fix. His response was a bit vague, so we followed up with AMD for more details. The company repudiated the claims of a memory bug, saying Tom’s Hardware that all 4th generation EPYC processors shipped to date fully support the 2DPC memory configuration and that re-spinning is not required. In addition, the company has already issued BIOS updates to its OEM partners to enable promised support for 2DPC configurations by the end of Q1 2023, and a support platform is already on sale. AMD also shared other details that we’ll cover below. But first, a bit of background information.
As you can see in our EPYC Genoa review, AMD’s new data center chips exhibit market-leading performance and come with several new interfaces, with support for 12 channels of DDR5 memory being one of the most important. However, Genoa was only released with support for DDR5 memory in a one DIMM per channel (1DPC) configuration. This type of configuration only supports one memory card connected to each of the twelve DDR5 memory controllers within the processor.
At launch, AMD said it would release a BIOS update in Q1 2023 to enable support for two memory DIMMs per channel (2DPC), allowing two memory cards to be plugged into each memory channel to increase capacity. . AMD said it was further characterizing and tuning 2DPC memory configurations, so it would post specifications for supported 2DPC memory speeds when the update was available.
Meanwhile, SemiAccurate (partially paid for) reported last month of an alleged issue with AMD’s Genoa processors. The report cited unnamed industry sources claiming that Genoa has a bug in its memory subsystem, forcing AMD to embark on an expensive retooling of the processors to support 2DPC memory configurations. This would inevitably lead to delays of several months as new chips progressed through the redesign and manufacturing process.
Naturally, a bug in the memory subsystem for shipping chips would mean that Genoa processors currently shipping would not support the upcoming 2DPC specification. So, to determine if a new twist was needed, we asked AMD if all Genoa processors already in circulation would support the 2DPC memory configuration when they launched, and the company assured us that they do.
Additionally, AMD noted that re-spinning is not required for 2DPC support. Instead, the company says that 2DPC support only requires the BIOS update it has already issued to its OEM customers. As a result, they are already designing motherboards with enough slots to support the feature. In fact, Tyan already included the Transport CX GC68A-B8056 that supports a 2DPC setup.
Due to normal speed reduction with 2DPC configurations, Intel 8-Channel Sapphire Rapids drops from DDR5-4800 at 1DPC to DDR5-4400 at 2DPC configuration. We can also expect Genoa’s 2DPC speeds to be less than the 1DPC speed when the company releases the final spec, but it remains to be seen what penalty it will incur. Tyan Server lists memory speeds at DDR5-4000 for the 2DPC configuration, but we’re told this can vary by system. Overall, this is a 10% reduction in speeds compared to Intel’s 2DPC setup, but that’s not too bad given that the 12-channel Genoa supports 50% more memory slots.
AMD also clarified Papermaster’s comments at the recent Morgan Stanley investor conference, which have been misinterpreted. On the call, Papermaster said, “And the 2 DIMMs per channel, which I think is what you’re referring to is the following. So, that’s for a much smaller specific group of customers. Those speeds will be announced later this quarter.” , and that will increase as well, but this number of clients for 2 DIMMs per channel is much lower.” AMD says that the “ramp” comment refers to systems that support 2DPC configurations (they need more physical slots), not a more recent revision of the processor.
Genoa’s support for 12 channels of DDR5 is the highest on the market for an x86 processor. Genoa has 50% more channels than Sapphire Rapids’ eight channels, and both chips support a maximum of DDR5-4800 memory in a 1DPC configuration. Intel has specified its 2DPC setting at DDR5-4400, but as mentioned, AMD hasn’t finished qualifying its 2DPC transfer rates. We’re told these can vary by platform.
AMD’s decision to release Genoa before 2DPC support has ended is wise: it’s rational to expect demand for 2DPC configurations to be much lower than we’ve seen in the past. The 2DPC setting is generally used to access higher capacity (there may be small performance gains with certain range settings). But with 12 memory channels in a 1DPC configuration, AMD can already support up to 3TB of memory per chip with 256GB devices. That’s enough for the broadest cross-section of users. Support for 2DPC increases that capacity to 6TB of DDR5 per socket, but AMD is already facing space constraints by including 12 memory channels in regular two-socket servers.
As you can see in the image above from our Genoa test server, cramming 24 DIMM slots in total for a 1DPC setup already creates a lot of problems due to space constraints. Frankly, it’s hard to imagine having twice the number of slots illustrated for a 2DPC setup – a dual socket server would need 48 slots total. As such, we believe that most 2DPC setups will likely be for single-socket servers or use a reduced number of channels on two-socket servers. In fact, the Tyan server listing 2DPC support only has a single socket.
There are already plenty of challenges that enable the illustrated 1DPC setup. In fact, AMD had to use special ‘slim’ memory slots for Genoa motherboards to help pack 12 slots into the chassis. AMD advised us that due to the thin slots and other accommodations for the denser layout, it has had several incidents where side pressure when installing DDR5 DIMMs had pushed the DIMM socket off the board. This is an edge case and does not indicate a problem with the platform, but points to the challenges AMD already faces with ‘only’ 12 memory slots.
The challenges for 2DPC expand beyond the space needed for more slots. As we’ve seen with DDR4 memory, adding more DIMMs per channel results in reduced memory speeds, and more channels result in even greater complexity. Furthermore, even having extra empty The slots can result in lower maximum memory speeds, as seen with the complicated matrix of DDR4 and DDR5 support for consumer platforms. Those issues get even more annoying with DDR5, as it has much higher tolerances and requires more complex motherboard designs with more layers and better materials, which adds to the cost. This will become even more challenging with the higher transfer rates required for next-generation memory; market insiders have even predicted that support for 2DPC might end with the DDR6 standard.
AMD says it will post more details on Genoa’s 2DPC support this month, and we’ll update once we get the details.
