Linux Containers (LXC) with LXD Hypervisor, Part 2 : Importing Container Images Into LXD

Other articles in the series:
In Part 2, we're going to discuss different ways of importing LXC container images into LXD. By default, when you create a LXC container using the 'lxc launch' command, the tool will download and cache the container image from the remote server. For example, to create a new CentOS 6 LXC container.
$ lxc remote add images images.linuxcontainers.org
$ lxc launch images:centos/7/amd64 centos

While waiting for the CentOS 7 image to be downloaded, you can check the LXD log file.
$ sudo tail -n2 /var/log/lxd/lxd.log
t=2015-08-30T00:13:22+0800 lvl=info msg="Image not in the db downloading it" image=69351a66510eecabf11ef7dfa94af40e20cf15c346ae08b3b0edd726ef3be10c server=https://images.linuxcontainers.org:8443
t=2015-08-30T00:13:22+0800 lvl=info msg="Downloading the image" image=69351a66510eecabf11ef7dfa94af40e20cf15c346ae08b3b0edd726ef3be10c

Unfortunately, if you have or experiencing slow network like me (see screenshot below), if best to use a network monitoring tool to check weather you're still downloading the image. For my case, I'm using bmon. Note my pathetic network speed. An average LXC container image is around 50MB. At download rate of average 20kb/s, it should took us around 33-plus minutes to finish the download. See that without download progress indicator, we've to go all the trouble to check whether the import is still running.


Alternatively, there also another way to import container images. This is through using 'lxd-images' tool, a Python script which supports two additional image source in addition to the default one as mentioned just now. These two sources are the local BusyBox images and Ubuntu Cloud images from official release streams. Additionally, since version 0.14, download progress tracking has been added to the tool, which solved the hassle we encountered.

Let's run the 'lxd-images' command and see its help message.
$ lxd-images
error: the following arguments are required: action
usage: lxd-images [-h] {import} ...

LXD: image store helper

positional arguments:
  {import}
    import    Import images

optional arguments:
  -h, --help  show this help message and exit

Examples:
 To import the latest Ubuntu Cloud image with an alias:
    /usr/bin/lxd-images import ubuntu --alias ubuntu

 To import the latest Ubuntu 14.04 LTS 64bit image with some aliases:
    /usr/bin/lxd-images import lxc ubuntu trusty amd64 --alias ubuntu --alias ubuntu/trusty

 To import a basic busybox image:
    /usr/bin/lxd-images import busybox --alias busybox

UPDATE: Since LXD version 0.17, 'lxd-images import lxc' command has been deprecated in favour of using the 'lxc launch' command.

Let's try to download and cache a CentOS 6 LXC container image into LXD. Compare using 'lxc launch' command to import container image. Notice the differences. First, verbosity is higher. At least we know what is going on behind the scene like what are the files being downloaded. Secondly, we can track the progress of the download. Third, we can add additional metadata, like aliases to the downloaded container image.
$ lxd-images import lxc centos 6 amd64 --alias centos/6                                                                                      
Downloading the GPG key for https://images.linuxcontainers.org
Downloading the image list for https://images.linuxcontainers.org
Validating the GPG signature of /tmp/tmprremowyo/index.json.asc
Downloading the image: https://images.linuxcontainers.org/images/centos/6/amd64/default/20150829_02:16/lxd.tar.xz
Progress: 1 %

However, from my understanding by reading the Python code of 'lxd-images' tool, container image is downloaded without using any multiple simultaneous connections. Hence, it will take a while (if you have slow connection like me) just to download any container images. To solve this, you can download and import the container image manually using third-parties download tool like Aria2 which supports multiple simultaneous connections.

In previous LXC version, if I remembered correctly, before version 0.15, CentOS 7 image was not found from the default image source listing (see emphasis in bold red) but still exists at the web site.
$ lxd-images import lxc centos 7 amd64 --alias centos/7
Downloading the GPG key for https://images.linuxcontainers.org
Downloading the image list for https://images.linuxcontainers.org
Validating the GPG signature of /tmp/tmpgg6sob2e/index.json.asc
Requested image doesn't exist.

Download and import the container image directly.
$ aria2x -x 4 https://images.linuxcontainers.org/images/centos/7/amd64/default/20150619_02:16/lxd.tar.xz

Import the downloaded container image in unified tarball format.
$ lxc image import lxd.tar.xz --alias centos/7
Image imported with fingerprint: 1d292b81f019bcc647a1ccdd0bb6fde99c7e16515bbbf397e4663503f01d7d1c

In short, just use 'lxd-images' tool to import any container images from the default source.

For the next part of the series, we're going to look into sharing files between the LXC container and the host. Till the next time.

Thinkpad T4210?

What do you call a Thinkpad laptop which has a T420 motherboard but T410 casing? Is it a T420 or T410? Or we should call it a hybrid of both, a T4210? More on this later.

I spend most of my computing time switching between Google Chrome and Bash shell, especially the later.  Therefore, a laptop with the best keyboard is essential to keep my hands and fingers happy, especially if you're suffering from Repeative strain injury (RSI). But this is manageable after all these years as I learned to how reduce the muscle spasm through massaging the correct trigger points.

If you ask me what is my dream laptop, it should be the legendary Thinkpad, especially the T-series or the lightweight X-series. Why so? If you spend a lot of time in the console which requires a lot of typing, a laptop with the best keyboard is a must, if you want to reduce injury to yourself. If you can, go for the classic keyboards (7 rows) instead of the newly introduced Precision keyboards (6 rows). After using both keyboards for a long period of time, I firmly believed that Lenovo made a big mistake by moving to Precision keyboards.

I used to own a E-series Thinkpad, which unfortunately, a cheaper, misleading, and fake version of Thinkpad without the durability and maintainability. Forget also the R-series, which is another economic version that is riding on the Thinkpad fame.

After reading the Used ThinkPad Buyers Guide, I bought or started to collect a used ThinkPad T420. Everything seemed good enough and looks like grade-A quality. Hardware details by inxi as shown:
$ inxi -b
System:    Host: motoko Kernel: 3.19.0-25-generic x86_64 (64 bit) Desktop: Gnome 3.16.3
           Distro: Ubuntu 15.04 vivid
Machine:   System: LENOVO product: 4180CTO v: ThinkPad T420
           Mobo: LENOVO model: 4180CTO Bios: LENOVO v: 83ET63WW (1.33 ) date: 07/29/2011
CPU:       Dual core Intel Core i5-2540M (-HT-MCP-) speed/max: 842/3300 MHz
Graphics:  Card: Intel 2nd Generation Core Processor Family Integrated Graphics Controller
           Display Server: X.Org 1.17.1 drivers: intel (unloaded: fbdev,vesa) Resolution: 1600x900@60.0hz
           GLX Renderer: Mesa DRI Intel Sandybridge Mobile GLX Version: 3.0 Mesa 10.5.2
Network:   Card-1: Intel 82579LM Gigabit Network Connection driver: e1000e
           Card-2: Intel Centrino Advanced-N 6205 [Taylor Peak] driver: iwlwifi
Drives:    HDD Total Size: 500.1GB (14.8% used)
Info:      Processes: 253 Uptime: 1:16 Memory: 2738.2/7760.8MB Client: Shell (bash) inxi: 2.2.16 

Although the stated maximum memory of the machine is 8GB, the result returned by dmidecode command shows otherwise, maximum supported memory is 16GB.
$ sudo dmidecode -t 16
# dmidecode 2.12
SMBIOS 2.6 present.

Handle 0x0005, DMI type 16, 15 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: None
        Maximum Capacity: 16 GB
        Error Information Handle: Not Provided
        Number Of Devices: 2

Unfortunately, after googling the full hardware details of the machine. I've noticed that the integrated Web Cam and fingerprint reader are missing from the laptop. It seems the seller rebuild the the laptop using a T410 case and T420 motherboard. I maybe wrong on this but that the best conclusion I can reach so far. Honestly, I have no one else to blame but myself but nevertheless, I can survive without these two features. Note to self, don't travel when buying any items.

On a related note, David Hill of Lenovo was conducting surveys on exploring on the idea of reintroduce a "Retro Thinkpad". Interesting outcome on the result of the surveys.

EPEL Yum Repository in CentOS

As Ansible, an automation tool, is not included to the default Yum repository in CentOS 6, you've to setup Extra Packages for Enterprise Linux (EPEL) Yum repository. I found this article on setting up EPEL repository and realize that I've been doing it the wrong way all this while. Instead of downloading the EPEL rpm package separately, we can install it directly as it's already located within the CentOS Extra Repository just by running the command below.
$ sudo yum install epel-release

Extra information on the epel-release package.
$ yum show epel-release
......
Installed Packages
Name        : epel-release
Arch        : noarch
Version     : 6
Release     : 8
Size        : 22 k
Repo        : installed
From repo   : extras
Summary     : Extra Packages for Enterprise Linux repository configuration
URL         : http://dl.fedoraproject.org/pub/epel/
License     : GPLv2
Description : This package contains the Extra Packages for Enterprise Linux (EPEL) repository
: GPG key as well as configuration for yum and up2date.

Listing the files within the epel-release package.
$ rpm -ql epel-release
/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
/etc/rpm/macros.ghc-srpm
/etc/yum.repos.d/epel-testing.repo
/etc/yum.repos.d/epel.repo
/usr/share/doc/epel-release-6
/usr/share/doc/epel-release-6/GPL

Getting the list of RPM packages from EPEL repository.
$ yum list | grep epel

Since this is third-party YUM repository, certain system administrator may prefer to disable it by default and only install certain package from it when necessary.
$ cat /etc/yum.repos.d/epel* | grep enabled
enabled=0
enabled=0
enabled=0
enabled=1
enabled=0
enabled=0
$ sed -i 's/enabled=1/enabled=0/g' /etc/yum.repos.d/epel*
$ cat /etc/yum.repos.d/epel* | grep enabled
enabled=0
enabled=0
enabled=0
enabled=0
enabled=0
enabled=0

If you've installed any packages from this Yum repository. You should clear all the downloaded cache RPM packages..
$ yum clean all

Unfortunately, to perform any yum commands against this repository, we've to explicitly state the repository for every command. Some examples.
$ yum search --enablerepo=epel ansible
$ yum list --enablerepo=epel | grep epel | grep ansible
$ yum install --enablerepo=epel ansible

Indeed a bit hassle to type all these commands. You can create an alias to reduce the needless typing.
$ alias yumepel="yum --enablerepo='epel'"
$ yumepel search ansible