Bug #244

ZDB uses insane amount of memory

Added by Bas van Oostveen about 1 year ago.

Status:New Start:09/01/2010
Priority:High Due date:
Assigned to:- % Done:

0%

Category:- Spent time: -
Target version:-

Description

Using a raidz2 pool with 4x 2TB disks with no dedup filled for 60% zdb uses a very large amount of memory, around 1GB, to run.

Manual page of zdb or "zdb -h" never say anything about memory requirements being this high...

This is particularly noticeable on Nexentastor which does "zdb -D $pool" from one of its proprietary scripts which means in practice that almost continuously the system is using 1GB memory. (maybe the fact that zdb -D seems to be stuck in a loop is a different bug, of maybe it's because once in a while zdb core dumps because out of memory).

Guessing that when you have 64gb of memory for caching you just don't notice zdb leaking/growing all over the place.

But also executing almost any ZDB command on the comment line is leaving ZDB to use large amounts of memory.

Last couple of times "zdb -h tank" was not able to finish executing without core dumping on OOM and killing all my ssh sessions.

Anyways... I don't assume that the high memory usage is some undocumented feature. If it is... we might want to add this to zdb -h and man zdb and give an indication about how much memory is used like 1GB per every 1TB used on pool...

Also available in: Atom PDF