Build & install PMDK lib on Ubuntu

APT

if you are using Ubuntu 18, just use apt pkg.

And I strongly recommend you to use systems with a higher versition, because PMDK is under fast developing, and some libs are not well tested for some old systems.

for ubuntu 18:

# Query the repository to identify if ndctl is delivered using either the aptitude, apt-cache, or apt utilities
aptitude search pmem
apt-cache search pmem
apt search pmem

# runtime
sudo apt-get install libpmem1 librpmem1 libpmemblk1 libpmemlog1 libpmemobj1 libpmempool1

# development
sudo apt-get install libpmem-dev librpmem-dev libpmemblk-dev libpmemlog-dev libpmemobj-dev libpmempool-dev libpmempool-dev

# debug
sudo apt-get install libpmem1-debug librpmem1-debug libpmemblk1-debug libpmemlog1-debug libpmemobj1-debug libpmempool1-debug

note: PMDK from apt may be in older version. e.g. now the latest release is 1.11.0, but apt only has libpmem-dev/bionic-updates,now 1.4.1-0ubuntu1~18.04.1 amd64.

PMDK & NDCTL

This tutorial is tested on Ubuntu 16, and will build PMDK & C++ bindings from source codes.

Before starting, make sure autoconf, cmake, make are on your computer.

firstly install NDCTL from source codes

# install dependence
sudo apt-get install -y git gcc g++ autoconf automake asciidoc asciidoctor bash-completion xmlto libtool pkg-config libglib2.0-0 libglib2.0-dev doxygen graphviz pandoc libncurses5 libkmod2 libkmod-dev libudev-dev uuid-dev libjson-c-dev libkeyutils-dev

git clone https://github.com/pmem/ndctl
cd ndctl
./autogen.sh
./configure CFLAGS='-g -O2' --prefix=/usr/local --sysconfdir=/etc
make
sudo make install

Secondly install PMDK from source codes

# install dependence
sudo apt install autoconf automake pkg-config libglib2.0-dev pandoc libncurses5-dev
# Note: if you need RDMA remote access, install libfabric from source codes: https://github.com/ofiwg/libfabric
git clone https://github.com/pmem/pmdk
cd pmdk
make
sudo make install
# update so path
sudo sh -c "echo /usr/local/lib >> /etc/ld.so.conf"
sudo sh -c "echo /usr/local/lib64 >> /etc/ld.so.conf"
sudo ldconfig

Finally install PMDK Cpp bindings from source codes. Note: here use 1.10 as an example. I have some unknown building errors on 1.11.

wget https://github.com/pmem/libpmemobj-cpp/archive/1.10.zip
unzip 1.10.zip
cd libpmemobj-cpp-1.10
mkdir build
cd build
cmake ..
# note: building is slow here, you can use -j 17 or sth else to make it faster
make
make install

by default, all *.so and *.hpp would be installed to /usr/local/..

You can test your new lib by using pmdk-example

change PM mode: sudo ndctl create-namespace --force --mode=devdax --reconfig=namespace0.0

RPMA

sudo apt install libunwind-dev

# txt2man version
git clone https://github.com/mvertes/txt2man.git
cd txt2man
git checkout txt2man-1.7.0
make -j$(nproc)
sudo make -j$(nproc) install prefix=/usr
cd ..
rm -rf txt2man

# cmocka-1.1.5-26-g672c5ce - pull latest fixes
git clone https://git.cryptomilk.org/projects/cmocka.git
mkdir cmocka/build
cd cmocka/build
git checkout 672c5cee79eb412025c3dd8b034e611c1f119055

cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=RelWithDebInfo
make -j$(nproc)
sudo make -j$(nproc) install
cd ../..
sudo rm -rf cmocka

# dependence

librpma is a C library to simplify accessing persistent memory on remote hosts over Remote Direct Memory Access (RDMA).

note: librpmem is another lib.

If any errors about can’t find libpmem package, check your pmdk libs path. Maybe they are installed to /usr/local/lib64, which can’t be found by default. So just add according pathes to PKG_CONFIG and ldconfig.

export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:$PKG_CONFIG_PATH 
echo "/usr/local/lib64" > /etc/ld.so.conf.d/libpmem.conf 
ldconfig

ref

  1. How to build the librpma library from source: https://github.com/pmem/rpma/blob/master/INSTALL.md.
  2. Design: https://software.intel.com/content/www/cn/zh/develop/articles/persistent-memory-replication-over-traditional-rdma-part-1-understanding-remote-persistent.html

emulation

if you don’t have a real persistent memory device:

find a usable space

# dmesg | grep BIOS-e820
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000bffe0000-0x00000000bfffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000053fffffff] usable

This will reserve 8 GB of memory starting at the 4 GB offset (which is usable in this example). On an Ubuntu 18.04 BIOS machine, you can pass these parameters by modifying the grub configuration file.

echo "GRUB_CMDLINE_LINUX="memmap=8G!4G"" >> /etc/default/grub
update-grub2 && update-grub

after rebooting, you can find /dev/pmem*. But emulated pmem may not support some libs.

ref

  1. Persistent Memory Wiki. https://nvdimm.wiki.kernel.org/
  2. https://github.com/ut-osa/assise/blob/master/docs/emulation.md
  3. https://nvdimm.wiki.kernel.org/how_to_choose_the_correct_memmap_kernel_parameter_for_pmem_on_your_system

Search

    Table of Contents