Performance Degradation with 1GB Pages for Heap

[Kishore Kumar Pusukuri]

Hi,
My AMD Opteron supports 4KB, 2MB and 1GB page sizes. I observed that there is performance improvement (reduced elapsed time) for some multi-threaded applications when I used 2MB page-size for heap. These applications need around 650MB heap (it reads a huge file of around 650MB size). However, when I used 1GB pages for heap, there is performance degradation for these programs. If we use 1 GB, then one page is enough to satisfy the heap space, right? Then, why we see performance degradation in this case. Please let me know.

$ pagesize -a
4096
2097152
1073741824

[Darryl Gove]

Is it a single lgroup? 1GB pages might mean remote memory for one of the
CPUs, and local for another.

Is it really TLB misses - does cputrack show an increase in TLB misses?

Do you get more cache misses?

Does pmap indicate that the app got large pages?

Regards,

Darryl.

Post by Kishore Kumar Pusukuri
Hi,
My AMD Opteron supports 4KB, 2MB and 1GB page sizes. I observed that there is performance improvement (reduced elapsed time) for some multi-threaded applications when I used 2MB page-size for heap. These applications need around 650MB heap (it reads a huge file of around 650MB size). However, when I used 1GB pages for heap, there is performance degradation for these programs. If we use 1 GB, then one page is enough to satisfy the heap space, right? Then, why we see performance degradation in this case. Please let me know.
$ pagesize -a
4096
2097152
1073741824