Systems Performance Chapter 7 Memory

System Performance Enterprise & The Cloud — Brendan Gregg

A revisit of OS course in a practical way.

Chapter 7 Memory

Concepts

File System Paging: “good” paging. used for reading and writing of pages in memory mapped files.

Anonymous Paging: “bad” paging. when applications access memory pages that have been paged out, they block on the disk IO required to read them back to main memory.

On demand Paging: the CPU overhead of creating the mapping till they are actually needed and accessed, instead of the time a range of memory is first allocated. If the mapping can be satisfied from another page in memory, it is called a minor fault. If page faults require storage device access, they’re called major faults.

Any page of virtual memory may be in one of the following state:

a. unallocated

b. allocated, unmapped (not yet fault)

c. allocated, mapped to main memory (ram)

d. allocated, mapped to the physical swap device (disk)

RSS = C, VM = B + C + D

Architecture

Latency

The access time of main memory can be measured as the column address strobe (CAS) latency: the time between sending a memory module the desired address (column) and when the data is available to be read.

Note: this is not an accurate description, from crucial website:

true latency (ns) = clock cycle time (ns) x number of clock cycles (CL)

I shot the author an email on this, see how he replies

Main Memory Architecture

UMP:

CPU 1            CPU 2
-----------------------------
   |       system bus |
DRAM A               DRAM B

NUMP:

DRAM A - memory bus - CPU 1 - CPU interconnect - CPU 2 - DRAM B

DDR SDRAM : double data rate synchronous dynamic random access memory. The term double data rate refers to the transfer of data on both the rise and fall of the clock signal.

A high level diagram of CPU -> cache -> memory

CPU - L1 - MMU     -      L2       -         L3
            |                                 |
[        [ page table ] main memory             ]

Free list management in Linux:

• Nodes: banks of memory, NUMA -aware
• Zones: ranges of memory for certain purposes (direct memory access, normal, highmem)
• Migration types: unmovable, reclaimable, movable…
• Sizes: power-of-two number of pages.

Page Scanning in Linux

The page-out daemon is called kswapd(), it scans LRU page lists of inactive and active memory to free pages. The page cache has separate lists for inactive pages and active pages. They operate in an LRU fashion.

Process Address Space

0x0

|             stack
virtual       executable    
memory        
process       heap
address
space
|             libraries
OxFFFFFFFF

Characterizing Usage

• System-wide physical and VM utilization.
• Degree of saturation: paging, swapping, OOM killing
• Kernel and file system cache memory usage
• Per-process physical and virtual memory usage
• Usage of memory resource controls, if present.

Analysis

here comes the fun part. It seems the same set of tool is applicable to both analyze CPU and Memory.

vmstat:

swpd: amount of swapped-out memory
free: fee available memory
buff: mem in the buffer cache
cache: mem in page cache
si: memory swapped in (paging)
so: memory swapped out (paging)

sar:

Enable sar in Ubuntu/debian: https://www.crybit.com/sysstat-sar-on-ubuntu-debian/

vi /etc/default/sysstat
----
# Should sadc collect system activity informations? Valid values
# are "true" and "false". Please do not put other values, they
# will be overwritten by debconf!
ENABLED="true"
----
service sysstat restart

pmap:

Lists the memory mapping of a process, showing their sizes, permissions, and mapped objects. (Kbytes in the output means virtual memory usage)

Others