I seem to have a larger memory leak on my current ubuntu System
After reporting strange Eclipse memory errors ( https://askubuntu.com/questions/148998/eclipse-constant-different-out-of-memory-errors ) I started to get ‘Not enough Memory’ error messages in my console today – while doing simple tasks like typing in
sudo -s
– or even –free -m
Typing in ‘free -m’ repeadetly showed me how my RAM quickly goes up from 700M to 900M, growing up to the size of 2000M in a few seconds (after freeing up memory with
echo 3 > /proc/sys/vm/drop_caches
)Eclipse isnt the cause, I completly killed the process and the ram still was going up. Is there any way to detect where the leak is coming from? I cant even update my system anymore, since
apt-get update
fails (probably because it’s out of memory)Using
Ubuntu 11.10
Answer
First, make sure to have a temp folder available which has enough free space. The following commands create dumps which can be several GBs in size.
You can create a new tmp folder using the following command. You may want to change /tmp
to a different filesystem with enough space
TMPDIR=$(mktemp -d -t -p /tmp)
Steps to find Memory Leak
-
Find out the PID of the process which causing memory leak (you can also use e.g.
htop
if available) and store it in a variable calledpid
ps -aux
-
Given that the PID is available in the variable
pid
, you can capture the memory consumption using/proc/$pid/smaps
and save into some file likebeforeMemInc.txt
.cat /proc/$pid/smaps > $TMPDIR/beforeMemInc.txt
- Wait some time for memory consumption to increase.
-
Capture
/proc/$pid/smaps
again and save it asafterMemInc.txt
cat /proc/$pid/smaps > $TMPDIR/afterMemInc.txt
-
Find the difference between first
smaps
and 2ndsmaps
, e. g. withdiff -u $TMPDIR/beforeMemInc.txt $TMPDIR/afterMemInc.txt
-
Note down the address range where memory got increased, for example:
beforeMemInc.txt afterMemInc.txt --------------------------------------------------- 2b3289290000-2b3289343000 2b3289290000-2b3289343000 #ADDRESS Shared_Clean: 0 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Clean: 0 kB Private_Dirty: 28 kB Private_Dirty: 36 kB Referenced: 28 kB Referenced: 36 kB Anonymous: 28 kB Anonymous: 36 kB #INCREASE MEM AnonHugePages: 0 kB AnonHugePages: 0 kB Swap: 0 kB Swap: 0 kB KernelPageSize: 4 kB KernelPageSize: 4 kB MMUPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB Locked: 0 kB VmFlags: rd wr mr mw me ac VmFlags: rd wr mr mw me ac
-
Use GDB to dump memory on running process or get the coredump using
gcore -o $TMPDIR/process $PID
-
I used gdb on the running process to dump the memory to some file.
cd $TMPDIR gdb -p $pid dump memory memory.dump 0x2b3289290000 0x2b3289343000
-
Now, use
strings
command orhexdump -C
to print thememory.dump
strings memory.dump
From this you get readable information which helps you locate those strings in your source code.
- Analyse your source to find the leak.
Attribution
Source : Link , Question Author : Katai , Answer Author : Zanna