Showing posts with label virtualbox. Show all posts
Showing posts with label virtualbox. Show all posts

This Week I Learned 2018 - Week 48

Last week post or read the old stuff instead.

What is this console app that always gave me a conflicting experience every time I use it? ImageMagick. Besides the forking of GraphicsMagick, the use of complex XML format (surprising for a 28-years program at the time of writing) as its configuration settings, it always failed when processing large number of files that exceed its default threshold values. Tweaking it with different settings or disabled it did not resolve the crash issue. Switching to GraphicsMagick yielded the same result. The workaround was convert each image file to PDF in parallel manner and merge all PDF files as single large PDF file.
find . -name '*.jpg' | parallel --progress convert {} {.}.pdf
pdfunite *.pdf scanned_doc.pdf

Does Webpack, Docker Machine, through Virtual Box, and Unix-based host OS work well? No, not really when come to watching changes of files and hot reloading. First, Inotify, the Linux kernel subsystem which notify file changes to application is not and will not be supported in Virtual Box. How about we switch to VMWare? Nope, not supported either. Switching Webpack's watch method to polling have its own issue as well, CPU usage and hot reloading depends on frequency of polling. The key is find the acceptable polling interval. Off course, there are workarounds but not to my liking. If you read carefully, optimal solution using Inotify does not work because the limitation of shared folders through Network File System (NFS) and how file changes event is not communicated between Docker's host and container.

What to consider when designing RESTful API for third parties usage? There are four rules: (1) Use API keys for every request endpoint, (2) Regulate usage through rate limiting using HTTP 429 and API key instead of IP address, (3) Revoke API key if there are usage violation but provides API for client to check rate limits, and (4) Use other means to validate authentication and authorization beside API key.

What are the differences between COALESCE and IFNULL in MySQL? There are several but COALESCE is the preferred choice because (1) It's a standard and should works across multiple DBMS (but do we switch DB that often?), (2) COALESCE support multiple arguments until it can find the first non-NULL value but IFNULL only  support two arguments, and lastly (3) IFNULL is slightly faster than COALESCE. Interestingly, undefined value (1/0) is considered as NULL or missing value.
mysql > SELECT IFNULL(1/0,'yes');
mysql > yes

mysql> SELECT COALESCE(1/0,'yes');
mysql > yes

What is the equivalent of `which` in Windows? `where` as shown below.
C:\where notepad
C:\Windows\System32\notepad.exe
C:\Windows\notepad.exe

This Week I Learned - 2016 Week 42

Last week post or the whole series.

Interesting week indeed. It has been a while since I last encountered so many different type of personalities who want or don't want to be a developer.

As usual, what have I learned this week? The usual stuff.

If you're running on GNU/Linux and want a way to manage different Windows OS through Vagrant, you can try this Vagrantfile. Installation and setup is pretty much straightforward, just make sure the Vagrantfile is downloaded. Unfortunately, the login still fail to work.
$ sudo install virtualbox vagrant
$ vagrant plugin install winrm winrm-fs

$ mkdir vagrant_win
$ cd vagrant_win
$ wget https://goo.gl/ATpaEY
$ IE=Win7IE8 vagrant up

Sanic, Python 3.5+ asynchronous web server. The discussion at HN seems rather interesting. While this is nothing new, asynchronous database layer like asyncpg seems rather userful to improve your DB query speed.

Issue with Babun's memory conflict after Windows updates? Try rebasing, not that Git rebasing thought. Cygwin still is the better and prefered choice for Unix experience in Windows. Yes, I know there is Bash on Windows.
1) Exit babun.
2) cmd /c %SYSTEMDRIVE%\Users\%USERNAME%\.babun\cygwin\bin\dash.exe -c '/usr/bin/rebaseall -v'

Customozing HTML's file inputs. Probably the most comprensive guide on different techniques to change the default behavour.

Web framework benchmarks. The Round 10 has one of the best humourous write-up.
The project returns with significant restructuring of the toolset and Travis CI integration. Fierce battles raged between the Compiled Empire and the Dynamic Rebellion and many requests died to bring us this data. Yes, there is some comic relief, but do not fear—the only jar-jars here are Java.
What happens when you rename a branch in Git? Plenty of things. First, you rename it locally. Next, you rename it remotely (is the same as remove the old branch and add a new branch). After that, either you update your upstream URL or checkout a fresh copy of the said new branch. Lastly, you may needs to batch update your commit messages.
$ git branch -m new_name
$ git branch -m old_name new_name
$ git push origin :old_name
$ git push --set-upstream origin new_name
$ git filter-branch -f --msg-filter 'sed "s/foo/bar/"' master..HEAD

The database schema for StackOverflow is publically accessible. I was surprised that it's such a straight forward design and nothing fancy at all. Well, is just a CRUD app with some additional tweaks here and there. However, the ranking formulae is far more interesting when compare to different algorithms used by other popular forum-like sites.

So may ways to iterate through the Perl's array. Implementation 1, 4, and 5 is what I normally used but the 5th method is still my favourite.

Source code syntax higlighter through Javascript? Just found out today, besides highlight.js, there is also Prism.js. The former seems to have more languages support but the later is used for quite a few popular projects.

NBA season going to start soon, maybe is time for me to learn some Statistics through certain API? Can't wait what surprises the 2016/2017 season will give us.

Fedora 23 Cloud Image Through Vagrant With VirtualBox and Libvirt Backend Provider

While testing LXD, I've to constantly switch between Ubuntu 15.10 and Fedora 23 to troubleshoot certain issues. However, my local Fedora 23 installation has been "contaminated" due to numerous tweaks I've done to get LXD to work. Hence, to make sure these changes I've made can be reproduced in fresh new Fedora environment, I've found using Vagrant with Fedora 23 Cloud image does fulfill that requirements.

Setting up in Ubuntu 15.10 was pretty much quite straight forward. First, we need to install Vagrant and VirtualBox. Check if we have the latest greatest version or for any issues.
$ sudo apt-get install vagrant virtualbox
$ vagrant version
Installed Version: 1.7.4
Latest Version: 1.7.4
 
You`re running an up-to-date version of Vagrant!

$ VBoxManage --version
5.0.4_Ubuntur102546

Next, install libvirt provider and the necessary libraries. Skip this step if you want to use the default VirtualBox provider. as we're not using the VirtualBox provider.
$ sudo apt-get install libvirt libvirt-dev
$ vagrant plugin install vagrant-libvirt

Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Installed the plugin 'vagrant-libvirt (0.0.32)'!

Next, download the Base Cloud image for Vagrant. There are two versions, VirtualBox or libvirt/KVM image. Since we're running this in GNU/Linux, let's use the libvirt/KVM image.
$ aria2c -x 4 https://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-virtualbox.box

$ aria2c -x 4 https://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-libvirt.box

Once we've downloaded the image, import it to Vagrant.
$ vagrant box add fedora/23 Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-virtualbox.box

==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'fedora/23' (v0) for provider: 
    box: Unpacking necessary files from: file:///home/ang/Projects/vagrant/Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-virtualbox.box
==> box: Successfully added box 'fedora/23' (v0) for 'virtualbox'!

Similarly for the libvirt image as well. We can add both images using the same name, in this case, 'fedora/23'.
$ vagrant box add fedora/23 Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-libvirt.box

==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'fedora/23' (v0) for provider: 
    box: Unpacking necessary files from: file:///home/ang/vagrant/Fedora-Cloud-Base-Vagrant-23-20151030.x86_64.vagrant-libvirt.box
==> box: Successfully added box 'fedora/23' (v0) for 'libvirt'!

See the available images. Note that the Fedora 23 box shares the same name but under different providers.
$ vagrant box list
base      (virtualbox, 0)
fedora/23 (libvirt, 0)
fedora/23 (virtualbox, 0)

Let's create Fedora 23 Vagrant instance.
$ mkdir f23_cloud_virtualbox
$ cd f23_cloud_virtualbox
$ vagrant init fedora/23
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Start and boot up your new Fedora 23 Cloud instance. If you don't specify the provider, by default it will use VirtualBox as its backend provider. Hence, the (--provider) parameter is optional.
$ vagrant up --provider=virtualbox
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'fedora/23'...
......

Let's try with libvirt provider and create all the necessary folder. At this moment, Vagrant only allows one provider for an active machine.
$ mkdir f23_cloud_libvirt
$ cd f23_cloud_libvirt
$ vagrant init fedora/23

Once done, let's boot this machine up. However, it seems we've a problem starting up the machine due to 'default' pool?
$ vagrant up --provider=libvirt
Bringing machine 'default' up with 'libvirt' provider...
There was error while creating libvirt storage pool: Call to virStoragePoolDefineXML failed: operation failed: pool 'default' already exists with uuid 9aab798b-f428-47dd-a6fb-181db2b20432

Google returned some answers suggesting checking the status of the pool. Let's try it out.
$ virsh pool-list --all
 Name                 State      Autostart 
-------------------------------------------
 default              inactive   no 

Let's start the 'default' pool and also toggle it to auto start.
$ virsh pool-start default
Pool default started

$ virsh pool-autostart default
Pool default marked as autostarted

Check the status of the 'default' pool again.
$ virsh pool-list --all
 Name                 State      Autostart 
-------------------------------------------
 default              active     yes     

Retry again to boot up our machine using libvirt backend provider.
$ vagrant up --provider=libvirt
Bringing machine 'default' up with 'libvirt' provider...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
......

Lastly, SSH to our machine.
$ vagrant ssh
[[email protected] ~]$ cat /etc/fedora-release 
Fedora release 23 (Twenty Three)

Vagrant 1.7.3 and VirtualBox 5.0 Installation in Ubuntu 15.04 - Part 2

Continue from the first part of the installation.

Meanwhile, the available VirtualBox version from the default Ubuntu repository is 4.3.26 as shown below.
$ apt-cache show virtualbox | grep ^Version
Version: 4.3.26-dfsg-2ubuntu2
Version: 4.3.26-dfsg-2ubuntu1

While we can use similar installation method like Vagrant, if there are any repository available, always favour this installation method as you don't need to manually verify each downloaded packages. Upgrade is also seamless without hassle.
$ echo "deb http://download.virtualbox.org/virtualbox/debian vivid contrib" | sudo tee -a /etc/apt/sources.list.d/virtualbox.list
deb http://download.virtualbox.org/virtualbox/debian vivid contrib

$ cat /etc/apt/sources.list.d/virtualbox.list 
deb http://download.virtualbox.org/virtualbox/debian vivid contrib

Next, add the public key so the apt program can verify the packages from the repository we've added just now.
$ wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
OK

Update the repository packages and check the available version.
$ sudo apt-get update

As discussed, before installation, always go through the change log. Then we proceed with the installation. You must specify the exact version you want to install. In this case, is version 5.0.
$ sudo apt-get install virtualbox-5.0

Once done, we'll proceed with Extension Pack installation. Let's download it and install it using the VBoxManage console tool.
$ aria2c -x 4 http://download.virtualbox.org/virtualbox/5.0.0/Oracle_VM_VirtualBox_Extension_Pack-5.0.0-101573.vbox-extpack

$ sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-5.0.0-101573.vbox-extpack 
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Successfully installed "Oracle VM VirtualBox Extension Pack".

Confirm our installed VirtualBox version.
$ vboxmanage --version
5.0.0r101573

Lastly, if there are any Linux kernel upgrade, you may need to rebuild vboxdrv kernel module by running this command.
$ sudo /etc/init.d/vboxdrv setup

Vagrant 1.7.3 and VirtualBox 5.0 Installation in Ubuntu 15.04 - Part 1

VirtuaBox 5.0, the x86 virtualization software was recently released.  This will be a good time for me to revisit it again with Vagrant, a tool to provision and distribute a virtual machine on top of VirtualBox. Why bother with Vagrant if you can just use VirtualBox as is? Well, if you want (1) to quickly provision an existing downloaded image; (2) to learn different provisioner like Ansible, Chef, Puppet, and others; (3) to have a easier way to manage your VirtualBox from the console. Then, there is no better tool than Vagrant.

One of the issue I've when evaluating Linux Container (LXC) is at this moment of writing, there is no easy way to create a CentOS 7 container through its daemon, LXD. Also, the container created cannot be distributed to other Operating System as LXC is a chroot environment container and not a virtual machine. In other words, LXC only works in GNU/Linuxes.

Now, let's check through the available version for Vagrant in the Ubuntu default repository.
$ apt-cache show vagrant | grep ^Version
Version: 1.6.5+dfsg1-2

Another way to check the latest Vagrant version if you've already installed Vagrant. This is through 'vagrant version' command. However, the result returned is not entirely correct. More on that later.
$ vagrant version
Installed Version: 1.7.3
Latest Version: 1.7.3

You're running an up-to-date version of Vagrant!

Our next step is to download the latest version of both software and install it in Ubuntu 15.04. Which means we need to download DEB packages. Let's start with Vagrant. Also we need to download the corresponding checksum file as well. I'm using Aria2 instead of Wget to speed up downloading as Aria2 supports multiple simultaneous connections.
$ aria2c -x 4 https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.deb
$ wget --content-disposition https://dl.bintray.com/mitchellh/vagrant/1.7.4_SHA256SUMS?direct

Before we install or upgrade Vagrant, verify the our downloaded DEB package against the checksum file. Remember to read the Changelog as well, just in case, if there are any important items relevant to our upgrade or installation.
$ sha256sum -c 1.7.4_SHA256SUMS 2>&1 | grep OK                                                                                          
vagrant_1.7.4_x86_64.deb: OK

Upgrade our Vagrant installation.
$ sudo dpkg -i vagrant_1.7.4_x86_64.deb 
......
Preparing to unpack vagrant_1.7.4_x86_64.deb ...
Unpacking vagrant (1:1.7.4) over (1:1.7.3) ...
Setting up vagrant (1:1.7.4) ...

Finally, verify our installation. See the inaccurate reporting of latest version against the installed version. Hence, to get the up-to-date version, is best to check Vagrant's download page.
$ vagrant version
Installed Version: 1.7.4
Latest Version: 1.7.3
 
You're running an up-to-date version of Vagrant!

To be continued.

Default Local Network IP Addresses for VirtualBox

Default network adapter in VirtualBox is set to Network Address Translation (NAT) within IP address of 10.0.2.x. If we want to use the default IP addresses of your actual local network (e.g. 192.168.x.x) , change the adapter type to Bridged Adapter as shown. Your Virtual Machine will be treated as another server in the local network.