How to detect a memory leak?

I seem to have a larger memory leak on my current ubuntu System

After reporting strange Eclipse memory errors ( https://askubuntu.com/questions/148998/eclipse-constant-different-out-of-memory-errors ) I started to get ‘Not enough Memory’ error messages in my console today – while doing simple tasks like typing in sudo -s – or even – free -m

Typing in ‘free -m’ repeadetly showed me how my RAM quickly goes up from 700M to 900M, growing up to the size of 2000M in a few seconds (after freeing up memory with echo 3 > /proc/sys/vm/drop_caches )

Eclipse isnt the cause, I completly killed the process and the ram still was going up. Is there any way to detect where the leak is coming from? I cant even update my system anymore, since apt-get update fails (probably because it’s out of memory)

Using Ubuntu 11.10

Answer

First, make sure to have a temp folder available which has enough free space. The following commands create dumps which can be several GBs in size.

You can create a new tmp folder using the following command. You may want to change /tmp to a different filesystem with enough space

TMPDIR=$(mktemp -d -t -p /tmp)

Steps to find Memory Leak

  1. Find out the PID of the process which causing memory leak (you can also use e.g. htop if available) and store it in a variable called pid

    ps -aux
    
  2. Given that the PID is available in the variable pid, you can capture the memory consumption using /proc/$pid/smaps and save into some file like beforeMemInc.txt.

    cat /proc/$pid/smaps > $TMPDIR/beforeMemInc.txt
    
  3. Wait some time for memory consumption to increase.
  4. Capture /proc/$pid/smaps again and save it as afterMemInc.txt

    cat /proc/$pid/smaps > $TMPDIR/afterMemInc.txt
    
  5. Find the difference between first smaps and 2nd smaps, e. g. with

    diff -u $TMPDIR/beforeMemInc.txt $TMPDIR/afterMemInc.txt
    
  6. Note down the address range where memory got increased, for example:

       beforeMemInc.txt            afterMemInc.txt
    ---------------------------------------------------
    2b3289290000-2b3289343000   2b3289290000-2b3289343000  #ADDRESS
    Shared_Clean:    0 kB       Shared_Clean:    0 kB          
    Shared_Dirty:    0 kB       Shared_Dirty:    0 kB
    Private_Clean:   0 kB       Private_Clean:   0 kB
    Private_Dirty:  28 kB       Private_Dirty:  36 kB  
    Referenced:     28 kB       Referenced:     36 kB
    Anonymous:      28 kB       Anonymous:      36 kB  #INCREASE MEM
    AnonHugePages:   0 kB       AnonHugePages:   0 kB
    Swap:            0 kB       Swap:            0 kB
    KernelPageSize:  4 kB       KernelPageSize:  4 kB
    MMUPageSize:     4 kB       MMUPageSize:     4 kB
    Locked:          0 kB       Locked:          0 kB
    VmFlags: rd wr mr mw me ac  VmFlags: rd wr mr mw me ac
    
  7. Use GDB to dump memory on running process or get the coredump using

    gcore -o $TMPDIR/process $PID
    
  8. I used gdb on the running process to dump the memory to some file.

    cd $TMPDIR
    gdb -p $pid
    dump memory memory.dump 0x2b3289290000 0x2b3289343000
    
  9. Now, use strings command or hexdump -C to print the memory.dump

    strings memory.dump
    

    From this you get readable information which helps you locate those strings in your source code.

  10. Analyse your source to find the leak.

Attribution
Source : Link , Question Author : Katai , Answer Author : Zanna

Leave a Comment