The recordings of training on memory-system profiling and performance analysis “It’s the Memory, Stupid!” are now available!
The event was led by DARE’s Petar Radojković, group leader of Memory Technologies from the Barcelona Supercomputing Center(BSC), with the support of INESC-ID and Intel Corporation. The training brought together researchers and practitioners interested in analyzing how memory bandwidth, latency, and microarchitectural effects influence HPC and AI application performance.
“It’s the Memory, Stupid!” group photo of attendees and lecturers
The programme presented complementary approaches to performance analysis, including Roofline modelling, bandwidth-latency analysis, and Intel’s Top-Down Microarchitectural Analysis. By looking at the problem from different perspectives, the lectures connected the characteristics of the underlying system with the behavior of applications running on it.
DARE’s researcher Aleksandar Ilic from INESC-ID contributed with a lecture on Roofline modelling principles, with a focus on the Cache-Aware Roofline Model (CARM) and the CARM tool, which is part of DARE’s GPP-specific Software Tools Pathfinding.
»Read more about the training in the blog post by the main organiser, Petar Radojković:
For decades, memory performance has been one of the defining challenges in computer architecture. This was famously described by Wulf and McKee in their 1995 paper Hitting the Memory Wall: Implications of the Obvious. More than thirty years later, the memory wall is still with us, and memory systems have a major impact on the performance and cost of the HPC and AI systems we build. But some of the assumptions we make about memory deserve a closer look.
“It’s the Memory, Stupid!” aims to demystify memory systems for HPC and AI system designers and users.
The course analyzes memory-system performance through the lens of leading memory-performance analysis tools, presented by experts in the field.
Researchers from INESC-ID present Roofline models, including the Cache-Aware Roofline Model developed at INESC-ID.
The BSC Memory team presents memory performance through the interplay between memory bandwidth and latency. This view is at the heart of the Mess and PROFET tools , which have received several academic awards and supported collaborations with industrial partners including Samsung, Huawei, and Micron.
Intel presents Top-Down Microarchitectural Analysis, one of the most detailed approaches to performance analysis available today. It is widely used in high-performance computing to help users understand the microarchitectural factors that determine application performance.
Each lecture looks at performance from two sides: the characteristics of the target system and the behavior of the applications running on it. More importantly, it connects these two views. Since not everything can fit into one tutorial, we also provide pointers for those who want to explore the topics in greater depth.
The introductory lecture lets the audience experience the memory wall and discusses how it has evolved over the last three decades. We also challenge a common line of reasoning: DRAM latency has remained almost constant for decades, so there is little we can do about it.
Our objective is to challenge this learned helplessness. There are many things users can do with the systems we have today, but the first step is understanding what is really happening.
The Latency That Matters Is Not Just DRAM Latency
From the application’s perspective, what matters is load-to-use latency: the time between issuing a memory operation and having the requested data available to the processor.
That latency is determined by much more than the intrinsic delay of the DRAM cells.
And that is good news.
Once we recognize that the real bottleneck is not simply in the DRAM cell, we can start doing something about it.
Bandwidth Is Equally Easy to Misunderstand
Memory bandwidth creates similar misconceptions.
Imagine a system advertised as providing 100 GB/s of memory bandwidth, while an application is using 70 GB/s. Would increasing the available bandwidth improve performance?
At first glance, perhaps not: 70 is comfortably below 100.
But, believe it or not, memory bandwidth can still be the main bottleneck directly limiting application performance in this situation.
Why?
That is exactly the kind of question we address during the course. Data-sheet bandwidth, achievable system bandwidth, application bandwidth consumption, latency, and contention are tightly interconnected concepts. Understanding how they interact is essential for identifying the real performance bottleneck.
Roofline models, Mess and PROFET bandwidth-latency analysis, and Top-Down Microarchitectural Analysis present different views of essentially the same problem, in the same room, on the same day. Therefore, it is also a useful lesson in intellectual humility and in accepting different analytical perspectives in research.
Discover the work of Memory Systems for HPC and AI Group at BSC .
