APT
if you are using Ubuntu 18, just use apt pkg.
And I strongly recommend you to use systems with a higher versition, because PMDK is under fast developing, and some libs are not well tested for some old systems.
for ubuntu 18:
# Query the repository to identify if ndctl is delivered using either the aptitude, apt-cache, or apt utilities
aptitude search pmem
apt-cache search pmem
apt search pmem
# runtime
sudo apt-get install libpmem1 librpmem1 libpmemblk1 libpmemlog1 libpmemobj1 libpmempool1
# development
sudo apt-get install libpmem-dev librpmem-dev libpmemblk-dev libpmemlog-dev libpmemobj-dev libpmempool-dev libpmempool-dev
# debug
sudo apt-get install libpmem1-debug librpmem1-debug libpmemblk1-debug libpmemlog1-debug libpmemobj1-debug libpmempool1-debug
note: PMDK from apt may be in older version. e.g. now the latest release is 1.11.0, but apt only has libpmem-dev/bionic-updates,now 1.4.1-0ubuntu1~18.04.1 amd64.
PMDK & NDCTL
This tutorial is tested on Ubuntu 16, and will build PMDK & C++ bindings from source codes.
Before starting, make sure autoconf, cmake, make are on your computer.
firstly install NDCTL from source codes
# install dependence
sudo apt-get install -y git gcc g++ autoconf automake asciidoc asciidoctor bash-completion xmlto libtool pkg-config libglib2.0-0 libglib2.0-dev doxygen graphviz pandoc libncurses5 libkmod2 libkmod-dev libudev-dev uuid-dev libjson-c-dev libkeyutils-dev
git clone https://github.com/pmem/ndctl
cd ndctl
./autogen.sh
./configure CFLAGS='-g -O2' --prefix=/usr/local --sysconfdir=/etc
make
sudo make install
Secondly install PMDK from source codes
# install dependence
sudo apt install autoconf automake pkg-config libglib2.0-dev pandoc libncurses5-dev
# Note: if you need RDMA remote access, install libfabric from source codes: https://github.com/ofiwg/libfabric
git clone https://github.com/pmem/pmdk
cd pmdk
make
sudo make install
# update so path
sudo sh -c "echo /usr/local/lib >> /etc/ld.so.conf"
sudo sh -c "echo /usr/local/lib64 >> /etc/ld.so.conf"
sudo ldconfig
Finally install PMDK Cpp bindings from source codes. Note: here use 1.10 as an example. I have some unknown building errors on 1.11.
wget https://github.com/pmem/libpmemobj-cpp/archive/1.10.zip
unzip 1.10.zip
cd libpmemobj-cpp-1.10
mkdir build
cd build
cmake ..
# note: building is slow here, you can use -j 17 or sth else to make it faster
make
make install
by default, all *.so and *.hpp would be installed to /usr/local/..
You can test your new lib by using pmdk-example
change PM mode: sudo ndctl create-namespace --force --mode=devdax --reconfig=namespace0.0
RPMA
sudo apt install libunwind-dev
# txt2man version
git clone https://github.com/mvertes/txt2man.git
cd txt2man
git checkout txt2man-1.7.0
make -j$(nproc)
sudo make -j$(nproc) install prefix=/usr
cd ..
rm -rf txt2man
# cmocka-1.1.5-26-g672c5ce - pull latest fixes
git clone https://git.cryptomilk.org/projects/cmocka.git
mkdir cmocka/build
cd cmocka/build
git checkout 672c5cee79eb412025c3dd8b034e611c1f119055
cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=RelWithDebInfo
make -j$(nproc)
sudo make -j$(nproc) install
cd ../..
sudo rm -rf cmocka
# dependence
librpma is a C library to simplify accessing persistent memory on remote hosts over Remote Direct Memory Access (RDMA).
note: librpmem is another lib.
If any errors about can’t find libpmem package, check your pmdk libs path. Maybe they are installed to /usr/local/lib64, which can’t be found by default.
So just add according pathes to PKG_CONFIG and ldconfig.
export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:$PKG_CONFIG_PATH
echo "/usr/local/lib64" > /etc/ld.so.conf.d/libpmem.conf
ldconfig
ref
- How to build the librpma library from source: https://github.com/pmem/rpma/blob/master/INSTALL.md.
- Design: https://software.intel.com/content/www/cn/zh/develop/articles/persistent-memory-replication-over-traditional-rdma-part-1-understanding-remote-persistent.html
emulation
if you don’t have a real persistent memory device:
find a usable space
# dmesg | grep BIOS-e820
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdffff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bffe0000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000053fffffff] usable
This will reserve 8 GB of memory starting at the 4 GB offset (which is usable in this example). On an Ubuntu 18.04 BIOS machine, you can pass these parameters by modifying the grub configuration file.
echo "GRUB_CMDLINE_LINUX="memmap=8G!4G"" >> /etc/default/grub
update-grub2 && update-grub
after rebooting, you can find /dev/pmem*. But emulated pmem may not support some libs.
ref
- Persistent Memory Wiki. https://nvdimm.wiki.kernel.org/
- https://github.com/ut-osa/assise/blob/master/docs/emulation.md
- https://nvdimm.wiki.kernel.org/how_to_choose_the_correct_memmap_kernel_parameter_for_pmem_on_your_system