(Beta) Xenomai 3.0.5 on Ubuntu 14.04/16.04¶
These instructions demonstrates how to build a Cobalt Core for Xenomai 3.0.5.
Official documentation is available at https://xenomai.org/installing-xenomai-3-x/
Recommended Hardware¶
- Intel/AMD Processor i5/i7 (< 2016 is recommended to guarantee full 16.04 support)
- Dedicated Ethernet controller for RTnet, with either e1000e/e1000/r8169 driver (Intel PRO/1000 GT recommended)
Warning
Nvidia/Ati Drivers are NOT supported (creates a lot of interruptions that breaks the real-time constraints). Please consider removing the dedicated graphic card and use the integrated graphics (Intel HD graphics).
Get the linux kernel¶
wget https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.9.38.tar.gz
tar xf linux-4.9.38.tar.gz
Note
We chose 4.9.38 because it is the latest kernel compatible with xenomai 3.0.5. You can find the patches at https://xenomai.org/downloads/ipipe/
Get Xenomai 3.0.5¶
wget https://xenomai.org/downloads/xenomai/stable/xenomai-3.0.5.tar.bz2
tar xf xenomai-3.0.5.tar.bz2
Apply the Xenomai patch¶
We assume you are building an x86_64 kernel (64bits).
cd linux-4.9.38
wget https://xenomai.org/downloads/ipipe/v4.x/x86/ipipe-core-4.9.38-x86-3.patch
../xenomai-3.0.5/scripts/prepare-kernel.sh --arch=x86_64 --ipipe=ipipe-core-4.9.38-x86-3.patch
Configure the kernel¶
Take the actual working config :
yes "" | make oldconfig
Gui version :
make xconfig
Or GTK+ version :
sudo apt install gtk+-2.0 glib-2.0 libglade2-dev
make gconfig
Or without gui :
sudo apt install libncurses5-dev
make menuconfig
Some guidelines to configure the linux kernel:
Recommended options:
* General setup
--> Local version - append to kernel release: -xenomai-3.0.5
--> Timers subsystem
--> High Resolution Timer Support (Enable)
* Xenomai/cobalt
--> Sizes and static limits
--> Number of registry slots (512 --> 4096)
--> Size of system heap (Kb) (512 --> 4096)
--> Size of private heap (Kb) (64 --> 256)
--> Size of shared heap (Kb) (64 --> 256)
--> Maximum number of POSIX timers per process (128 --> 512)
--> Drivers
--> RTnet
--> RTnet, TCP/IP socket interface (Enable)
--> Drivers
--> New intel(R) PRO/1000 PCIe (Enable)
--> Realtek 8169 (Enable)
--> Loopback (Enable)
--> Add-Ons
--> Real-Time Capturing Support (Enable)
* Power management and ACPI options
--> CPU Frequency scaling
--> CPU Frequency scaling (Disable)
--> ACPI (Advanced Configuration and Power Interface) Support
--> Processor (Disable)
--> CPU Idle
--> CPU idle PM support (Disable)
* Pocessor type and features
--> Enable maximum number of SMP processors and NUMA nodes (Disable)
// Ref : http://xenomai.org/pipermail/xenomai/2017-September/037718.html
--> Processor family
--> Core 2/newer Xeon (if "cat /proc/cpuinfo | grep family" returns 6, set as Generic otherwise)
// Xenomai will issue a warning about CONFIG_MIGRATION, disable those in this order
--> Transparent Hugepage Support (Disable)
--> Allow for memory compaction (Disable)
--> Contiguous Memory Allocation (Disable)
--> Allow for memory compaction
--> Page Migration (Disable)
* Device Drivers
--> Staging drivers
--> Unisys SPAR driver support
--> Unisys visorbus driver (Disable)
Warning
Unlike xenomai 2.x, RTnet has to be built in the kernel. Make sure to choose the drivers correctly.
Build the Real-Time kernel¶
sudo apt install kernel-package
CONCURRENCY_LEVEL=$(nproc) make-kpkg --rootcmd fakeroot --initrd kernel_image kernel_headers
Compile faster with distcc¶
If you have distcc servers setup and a fast network, you can speed up drastically the building speed.
MAKEFLAGS="CC=/usr/lib/distcc/gcc-5" make-kpkg -j$(distcc -j) --rootcmd fakeroot --initrd kernel_image kernel_headers
Note
Always set gcc version for distcc so that the server can figure out which one to choose (it may have gcc-4.8 by default)
cd ..
sudo dpkg -i linux-headers-4.9.38-xenomai-3.0.5_4.9.38-xenomai-3.0.5-10.00.Custom_amd64.deb linux-image-4.9.38-xenomai-3.0.5_4.9.38-xenomai-3.0.5-10.00.Custom_amd64.deb
Allow non-root users¶
sudo addgroup xenomai --gid 1234
sudo addgroup root xenomai
sudo usermod -a -G xenomai $USER
Tip
If the addgroup command fails (ex: GID xenomai
is already in use), change it to a different random value, and report it in the next section.
Configure GRUB and reboot¶
Edit the grub config :
sudo nano /etc/default/grub
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 4.9.38-xenomai-3.0.5"
#GRUB_DEFAULT=saved
#GRUB_SAVEDEFAULT=true
# Comment the following lines
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=5
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash xenomai.allowed_group=1234"
GRUB_CMDLINE_LINUX=""
Note
Please note the xenomai group (here 1234) should match what you set above (allow non-root users).
Tip
noapic
option might be added if the screen goes black at startup and you can’t boot.
If you have an Intel HD Graphics integrated GPU (any type) :
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_rc6=0 i915.enable_dc=0 noapic xenomai.allowed_group=1234"
# This removes powersavings from the graphics, that creates disturbing interruptions.
If you have an Intel Skylake (2015 processors), you need
to add nosmap to fix the latency hang (https://xenomai.org/pipermail/xenomai/2016-October/036787.html) :
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_rc6=0 i915.enable_dc=0 xeno_nucleus.xenomai_gid=1234 nosmap"
Update GRUB and reboot
sudo update-grub
sudo reboot
Installing Xenomai 3.0.5 User space libraries¶
First, make sure that you are running the cobalt kernel :
uname -a
# Should return Linux waro-rt 4.9.38-xenomai-3.0.5 #2 SMP Wed Sep 20 16:00:12 CEST 2017 x86_64 x86_64 x86_64 GNU/Linux
dmesg | grep Xenomai
# [ 1.417024] [Xenomai] scheduling class idle registered.
# [ 1.417025] [Xenomai] scheduling class rt registered.
# [ 1.417045] [Xenomai] disabling automatic C1E state promotion on Intel processor
# [ 1.417055] [Xenomai] SMI-enabled chipset found, but SMI workaround disabled
# [ 1.417088] I-pipe: head domain Xenomai registered.
# [ 1.417704] [Xenomai] allowing access to group 1234
# [ 1.417726] [Xenomai] Cobalt v3.0.5 (Sisyphus's Boulder) [DEBUG]
cd xenomai-3.0.5
./configure --with-pic --with-core=cobalt --enable-smp --disable-tls --enable-dlopen-libs --disable-clock-monotonic-raw
make -j`nproc`
sudo make install
# --disable-clock-monotonic-raw : http://xenomai.org/pipermail/xenomai/2017-September/037695.html
Prevent Gazebo compiling issues¶
Proper fix : use Xenomai git repo¶
The issue has been fixed with commits 5ec8be8 and 1ed69a6.
git clone https://git.xenomai.org/xenomai-3.git
cd xenomai-3
./script/bootstrap
./configure --with-pic --with-core=cobalt --enable-smp --disable-tls --enable-dlopen-libs
make -j`nproc`
sudo make install
Hack with v3.0.5¶
Gazebo won’t compile because of some conflicting macros ( clz() ) present in libtbb and libcobalt. Remove this macro from xenomai to hack-fix it. It is only used in xenomai internals so won’t cause any issue in user-land. use this hack only if you install xenomai from the 3.0.5 package, not the git repo.
# http://xenomai.org/pipermail/xenomai/2017-September/037729.html
sudo sed -i 's/clz/__clz/g' /usr/xenomai/include/boilerplate/compiler.h
Update your bashrc¶
echo '
### Xenomai
export XENOMAI_ROOT_DIR=/usr/xenomai
export XENOMAI_PATH=/usr/xenomai
export PATH=$PATH:$XENOMAI_PATH/bin:$XENOMAI_PATH/sbin
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$XENOMAI_PATH/lib/pkgconfig
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$XENOMAI_PATH/lib
export OROCOS_TARGET=xenomai
' >> ~/.xenomai_rc
echo 'source ~/.xenomai_rc' >> ~/.bashrc
source ~/.bashrc
Test your installation¶
xeno latency
This loop will allow you to monitor a xenomai latency. Here’s the output for a i7 4Ghz :
== Sampling period: 100 us
== Test mode: periodic user-mode task
== All results in microseconds
warming up...
RTT| 00:00:01 (periodic user-mode task, 100 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD| 0.174| 0.464| 1.780| 0| 0| 0.174| 1.780
RTD| 0.088| 0.464| 1.357| 0| 0| 0.088| 1.780
RTD| 0.336| 0.464| 1.822| 0| 0| 0.088| 1.822
RTD| 0.342| 0.464| 1.360| 0| 0| 0.088| 1.822
RTD| 0.327| 0.462| 2.297| 0| 0| 0.088| 2.297
RTD| 0.347| 0.463| 1.313| 0| 0| 0.088| 2.297
RTD| 0.314| 0.464| 1.465| 0| 0| 0.088| 2.297
RTD| 0.190| 0.464| 1.311| 0| 0| 0.088| 2.297
Tip
To get pertinent results, you need to stress your system while running the latency test. The latency has to be stable even if the system is under load.
sudo apt install stress
# Using stress
stress -v -c 8 -i 10 -d 8
Fix negative latency issues¶
You need to be in root sudo -s
, then you can set values to the latency calibration variable in nanoseconds:
$ echo 0 > /proc/xenomai/latency
# Now run the latency test
# If the minimum latency value is positive,
# then get the lowest value from the latency test (ex: 0.088 us)
# and write it to the calibration file ( here you have to write 88 ns) :
$ echo my_super_value_in_ns > /proc/xenomai/latency
Source : https://xenomai.org/pipermail/xenomai/2007-May/009063.html