libvmmalloc.3

.\"
.\" Copyright (c) 2014-2015, Intel Corporation
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\"
.\"     * Redistributions of source code must retain the above copyright
.\"       notice, this list of conditions and the following disclaimer.
.\"
.\"     * Redistributions in binary form must reproduce the above copyright
.\"       notice, this list of conditions and the following disclaimer in
.\"       the documentation and/or other materials provided with the
.\"       distribution.
.\"
.\"     * Neither the name of Intel Corporation nor the names of its
.\"       contributors may be used to endorse or promote products derived
.\"       from this software without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.\"
.\" libvmmalloc.3 -- man page for libvmmalloc
.\"
.\" Format this man page with:
.\"	man -l libvmmalloc.3
.\" or
.\"	groff -man -Tascii libvmmalloc.3
.\"
.TH libvmmalloc 3 "vmmalloc API version 0.1" "NVM Library"
.SH NAME
libvmmalloc \- general purpose volatile memory allocation library
.SH SYNOPSIS
.PP
$ LD_PRELOAD=libvmmalloc.so
.I command [args...]
.PP
or
.PP
$ cc [ flag... ] file... -lvmmalloc [ library... ]
.sp
.ft B
.nf
#include <stdlib\&.h>
#include <malloc\&.h>
#include <libvmmalloc\&.h>
.fi
.ft
.HP \w'void\ *malloc('u
.BI "void *malloc(size_t\ " "size" ");"
.HP \w'void\ *calloc('u
.BI "void *calloc(size_t\ " "number" ", size_t\ " "size" ");"
.HP \w'int\ posix_memalign('u
.BI "int posix_memalign(void\ **" "memptr" ", size_t\ " "alignment" ", size_t\ " "size" ");"
.HP \w'void\ *realloc('u
.BI "void *realloc(void\ *" "ptr" ", size_t\ " "size" ");"
.HP \w'void\ free('u
.BI "void free(void\ *" "ptr" ");"
.HP \w'size_t\ malloc_usable_size('u
.BI "size_t malloc_usable_size(const\ void\ *" "ptr" ");"
.HP \w'void\ *memalign('u
.BI "void *memalign(size_t\ " "alignment" ", size_t\ " "size" ");"
.HP \w'void\ *aligned_alloc('u
.BI "void *aligned_alloc(size_t\ " "alignment" ", size_t\ " "size" ");"
.HP \w'void\ *valloc('u
.BI "void *valloc(size_t\ " "size" ");"
.HP \w'void\ *pvalloc('u
.BI "void *pvalloc(size_t\ " "size" ");"
.HP \w'void\ cfree('u
.BI "void cfree(void\ *" "ptr" ");"
.SH DESCRIPTION
.PP
.B libvmmalloc
transparently converts all the dynamic memory allocations into Persistent Memory
allocations.
.PP
The typical usage of
.B libvmmalloc
does not require any modification of the target program.  It is enough
to load
.B libvmmalloc
before all other libraries by setting the environment variable
.BR LD_PRELOAD .
When used in that way,
.B libvmmalloc
interposes the standard system memory allocation routines, as defined in
.BR malloc (3),
.BR posix_memalign (3)
and
.BR malloc_usable_size (3),
and provides that all dynamic memory allocations are made from a
.I memory pool
built on memory-mapped file, instead of a system heap.  The memory managed by
.B libvmmalloc
may have different attributes, depending on the file system
containing the memory-mapped file.  In particular,
.B libvmmalloc
is part of the
.I Non-Volatile Memory Library
because it is sometimes useful to use non-volatile memory as a volatile
memory pool, leveraging its capacity, cost, or performance characteristics.
.PP
.B libvmmalloc
may be also linked to the program, by providing the
.BR -lvmmalloc
argument to the linker.  Then it becomes the default memory allocator
for given program.
.PP
.B NOTE: Due to the fact the library operates on a memory-mapped file,
.B it will not work with the programs that perform
.BR fork (3)
.B not followed by
.BR exec (3).
.B The behavior is undefined in such case, but most likely results in the
.B memory pool corruption and the program crash due to segmentation fault.
.PP
.B libvmmalloc
uses the
.BR mmap (2)
system call to create a pool of volatile memory.  The library
is most useful when used with
.I Direct Access
storage (DAX), which is memory-addressable persistent storage
that supports load/store access without being paged via the system page cache.
A Persistent Memory-aware file system is typically used to provide this
type of access.  Memory-mapping a file from a Persistent
Memory-aware file system provides the raw memory pools, and this library
supplies the traditional
.I malloc
interfaces on top of those pools.
.PP
The memory pool acting as a system heap replacement is created automatically
at the library initialization time.  User may control its location and size
by setting the environment variables described in
.B ENVIRONMENT
section.  The allocated file space is reclaimed when process terminates
or in case of system crash.
.PP
Under normal usage,
.B libvmmalloc
will never print messages or intentionally cause the process to exit.
The library uses
.BR pthreads (7)
to be fully MT-safe, but never creates or destroys threads itself.
The library does not make use of any signals, networking, and
never calls
.BR select ()
or
.BR poll ().
.SH ENVIRONMENT
There are two configuration variables that
.B must
be set to make
.B libvmmalloc
work properly.  If any of them is not specified, or if their values are not
valid, the library prints the appropriate error message and terminates
the process.
.TP
.B VMMALLOC_POOL_DIR
Specifies a path to directory where the memory pool file should be
created.  The directory must exist and be writable.
.TP
.B VMMALLOC_POOL_SIZE
Defines the desired size (in bytes) of the memory pool file.
It must be not less than the minimum allowed size
.B VMMALLOC_MIN_POOL
as defined in
.B <libvmmalloc.h>.

Note that due to the fact the library adds some metadata to the
memory pool, the amount of actual usable space is typically less than
the size of the memory pool file.
.SH DEBUGGING
.PP
Two versions of
.B libvmmalloc
are typically available on a development system.
The normal version is optimized for performance.  That version skips checks
that impact performance and never logs any trace information or performs
any run-time assertions.  A second version, accessed when using the libraries
under
.BR /usr/lib/nvml_debug ,
contains run-time assertions and trace points.
The typical way to access the debug version is to set the environment variable
.B LD_LIBRARY_PATH
to
.BR /usr/lib/nvml_debug
or
.BR /usr/lib64/nvml_debug
depending on where the debug libraries are installed on the system.
The trace points in the debug version of the library
are enabled using the environment variable
.BR VMMALLOC_LOG_LEVEL ,
which can be set to the following values:
.IP 0
Tracing is disabled.
This is the default level when
.B VMMALLOC_LOG_LEVEL
is not set.
.IP 1
Additional details on any errors detected are logged (in addition
to returning the errno-based errors as usual).
.IP 2
A trace of basic operations is logged.
.IP 3
This level enables a very verbose amount of function call tracing
in the library.  There are also some basic memory allocation statistics
logged when the process terminates.
.IP 4
This level enables voluminous tracing information about all the
memory allocations and deallocations.
.PP
The environment variable
.B VMMALLOC_LOG_FILE
specifies a file name where all logging information should be written.
If the last character in the name is "-", the PID of the current process
will be appended to the file name when the log file is created.  If
.B VMMALLOC_LOG_FILE
is not set, output goes to stderr.
.PP
Setting the environment variable
.B VMMALLOC_LOG_LEVEL
has no effect on the non-debug version of
.BR libvmmalloc .
.SH NOTES
Unlike the normal
.BR malloc (),
which asks the system for additional memory when it runs out,
.B libvmmalloc
allocates the size it is told to and never attempts to grow or shrink
that memory pool.
.SH BUGS
.B libvmmalloc
does not work with the programs that perform
.BR fork (3)
and do not call
.BR exec (3)
immediately afterwards.
.PP
Malloc hooks (see
.BR malloc_hook (3)),
are not supported when using
.BR libvmmalloc .
.SH ACKNOWLEDGEMENTS
.B libvmmalloc
depends on jemalloc, written by Jason Evans, to do the heavy lifting
of managing dynamic memory allocation.  See:
.IP
http://www.canonware.com/jemalloc/
.SH "SEE ALSO"
.BR ld.so (8)
.BR malloc (3),
.BR posix_memalign (3),
.BR malloc_usable_size (3),
.BR malloc_hook (3),
.BR jemalloc (3),
.BR libvmem (3),
.BR libpmem (3).