Autoload Module for Python Shell and IPython

In PHP, print_r is a very useful function to display variable in a human-readable format. Similarly, both pprint and awesome_print provides equivalent function in Python. Although the former has more extra features.

Since most of my Python time is spend on either the Python shell or IPython, it will be nice if we can autoload these two modules upon starting the shell.

Python shell

First we need to set the environment variable PYTHONSTARTUP so it can autoload the file. You should put this into your .bashrc file and reload it.
export PYTHONSTARTUP=$HOME/.pythonstartup

The content of .pythonstartup file, which is just a normal Python script as shown. Besides that, we also enable tab completion as the default shell has limited features.
# two use modules for pretty print variables
from awesome_print import ap
from pprint import pprint

import rlcompleter
import readline

# enable tab completion
readline.parse_and_bind("tab: complete")

To test our autoloading, just start the Python shell and type the sample code which will list all the attributes of the ap function.
$ python
Python 2.7.5+ (default, Feb 27 2014, 19:37:08)
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> ap(dir(ap))
[
  [ 0] __call__,
  [ 1] __class__,
  [ 2] __closure__,
  [ 3] __code__,
  [ 4] __defaults__,
  [ 5] __delattr__,
  [ 6] __dict__,
  [ 7] __doc__,
  [ 8] __format__,
  [ 9] __get__,
  [10] __getattribute__,
  [11] __globals__,
  [12] __hash__,
  [13] __init__,
  [14] __module__,
  [15] __name__,
  [16] __new__,
  [17] __reduce__,
  [18] __reduce_ex__,
  [19] __repr__,
  [20] __setattr__,
  [21] __sizeof__,
  [22] __str__,
  [23] __subclasshook__,
  [24] func_closure,
  [25] func_code,
  [26] func_defaults,
  [27] func_dict,
  [28] func_doc,
  [29] func_globals,
  [30] func_name
]

IPython

Again, for IPython, the setup is similar. First, you must set the export IPYTHONDIR environment variable in your bash file. In my Ubuntu 14.10, the default data path was set to $HOME/.config which contains other unnecessary configuration files to be added to my Git’s repository.
export IPYTHONDIR=$HOME/.ipython

Next, we instantiate and create the profile data.
$ ipython profile create
[ProfileCreate] Generating default config file: u'/home/kianmeng/.ipython/profile_default/ipython_config.py'
[ProfileCreate] Generating default config file: u'/home/kianmeng/.ipython/profile_default/ipython_qtconsole_config.py'
[ProfileCreate] Generating default config file: u'/home/kianmeng/.ipython/profile_default/ipython_notebook_config.py'

Following the directory structure shown below, create the startup script file 10-default.py.
$ tree .ipython
.ipython
├── profile_default
│   └── startup
│       └── 10-default.py

Add these to the 10-default.py file.
$ cat .ipython/profile_default/startup/10-default.py
from awesome_print import ap
from pprint import pprint

Start our IPython session and try to test for our autoloaded modules.
$ ipython
Python 2.7.5+ (default, Feb 27 2014, 19:37:08)
Type "copyright", "credits" or "license" for more information.

IPython 0.13.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: ap??
Type:       function
String Form:
File:       /usr/local/lib/python2.7/dist-packages/awesome_print/awesome_print.py
Definition: ap(*args)
Source:
def ap(*args):
    for arg in args:
        print format(arg)

In [2]:

PHP Successor?

"A lot of those "modern PHP" libraries are basically a masochistic exercise in copying designs that make sense in Java into PHP where they make no sense.

PHPT style tests wouldn't make sense in Java. And Java style Unit test frameworks make no sense in PHP.

However all those blindly supporting this, can please continue to waste their time, for the benefit of making sure they're buzzword compliant & in sync with groupthink."
-- mantrax [1], emphasis added

Sigh. My feeling exactly.

PHP the language has starting to show its age and limitation. From a template language, to a procedural scripting web language, and now a mutated siblings of Java-wannabe. Ironically, even Java web development itself is moving towards a scripting based language like Groovy.

I am hoping Facebook's HHVM will slowly replace PHP (Zend Engine), and, if possible, steer the direction of the language itself. Again, by replacing it with the Hack. They (Facebook) have all the resources to make this works.

Understanding Git Rebase

For a Git beginner like me, Git rebase seems cryptic and hard to understand. The one line help description of the command states that this tool "Forward-port local commits to the updated upstream head". Forward-port? Local commits? Updated upstream head? Sounds confusing? Yup, me too. Even after I read the definition and explanation of these terms.

After several day of googling and constant reading through the online tutorials and manual, finally I managed to grasp some basic understanding on how and why Git rebase works. Mostly from excellent guide of Cern guide to Git and Charles Duan's Guide to Git.

To summarize my understanding of Git rebase,
  1. Rebasing is about managing commit history / log
  2. An alternative way for doing conventional merging, but more refining
  3. Two scenarios where you will need rebasing:
    • To squash or combine our local commits before merging with remote branches
    • To keep you local branch up-to-date with remote branches without merging
We will increase our understanding by going through the step-by-step guide of going a rebasing for above mentioned scenarios. Before that, let's setup our git as follows. You can skip the user name and email if you already done so.
$ git config --global user.name "John Doe"
$ git config --global user.email johndoe@example.com

$ git config --global color.ui auto
$ git config --global color.branch auto
$ git config --global color.diff auto
$ git config --global color.status auto
$ git config --global alias.ll 'log --oneline --decorate --graph --all'

Let's create as local Git repository before we can proceed with rebasing.
$ mkdir -p /tmp/foobar
$ cd /tmp/foobar
$ git init
Initialized empty Git repository in /tmp/foobar/.git/

Create a few changeset, a set of modified files, in the master branch. We're using the naming convention of [branch name]c[sequence] for each file name that represent a changeset.
$ touch mc1; git add mc1; git commit -m "mc1"
$ touch mc2; git add mc2; git commit -m "mc2"
$ touch mc3; git add mc3; git commit -m "mc3"

Visualize our changes so far using the alias we've created.
$ git ll
* 7665913 (HEAD, master) mc3
* 9ef4878 mc2
* 2f8d692 mc1

Scenario 1 : Squashing Local Commits

Imagine that you want to add a new feature, surely you're going to create a new branch, let's call it new-feature, and work on it locally (at your development machine). Let's try that.
$ git checkout -b new-feature
Switched to a new branch 'new-feature'

Check our log and the available branch. If you've notice, the current HEAD, new-feature branch, and master branch are pointed to the same hash.
$ git ll
* 7665913 (HEAD, new-feature, master) mc3
* 9ef4878 mc2
* 2f8d692 mc1

$ git branch -a
master
* new-feature

A feature is like a task where we can further break down into sub-tasks. Also, is a good practice to commit early and commit often as you can break a problem down into a set of smaller problems and tackle it one by one.

Let's try to simulate that in the new-feature branch. Each nfX is a sub-tasks in order for us to implement the new feature.
$ touch nf1; git add nf1; git commit -m "nf1"
$ touch nf2; git add nf2; git commit -m "nf2"
$ touch nf3; git add nf3; git commit -m "nf3"
$ touch nf4; git add nf4; git commit -m "nf4"
$ touch nf5; git add nf5; git commit -m "nf5"

Check the history log again. The new-feature branch is ahead of the master branch by 5 commits.
$ git ll
* 466b238 (HEAD, new-feature) nf5
* 61f6e91 nf4
* 7f80d86 nf3
* bb93e3a nf2
* 65d8d8a nf1
* 7665913 (master) mc3
* 9ef4878 mc2
* 2f8d692 mc1

Instead of merging all those sub-tasks commit (useful to you but not to others) to the main branch, a better approach is to squash or consolidate all into one single commit through git rebase.
# last 5 commits
$ git rebase -i HEAD~5

# if the master branch or other branches is behind your new-feature branch 
$ git rebase -i master

The previous command will start the interactive mode for us to squash all our related commits and group them into one.
pick d69307e nf1
pick 4e9cd86 nf2
pick 6449f6a nf3
pick 6acfd6d nf4
pick f29e1db nf5

# Rebase 1245945..f29e1db onto 1245945
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# However, if you remove everything, the rebase will be aborted.
#
# Note that empty commits are commented out

Rearrange and amend the necessary actions for these related commits.
pick f29e1db nf5
squash 6acfd6d nf4
squash 6449f6a nf3
squash 4e9cd86 nf2
squash d69307e nf1

The next step is to summarize and rewrite all the commit messages as shown below.
# This is a combination of 5 commits.
# The first commit's message is:
nf5

# This is the 2nd commit message:

nf4

# This is the 3rd commit message:

nf3

# This is the 4th commit message:

nf2

# This is the 5th commit message:

nf1

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# HEAD detached from 1245945
# You are currently editing a commit while rebasing branch 'new-feature' on '1245945'.
#
# Changes to be committed:
#   (use "git reset HEAD^1 ..." to unstage)
#
#       new file:   nf1
#       new file:   nf2
#       new file:   nf3
#       new file:   nf4
#       new file:   nf5
#

In which, we just summarize it as
implement new-feature 

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# HEAD detached from 1245945
# You are currently editing a commit while rebasing branch 'new-feature' on '1245945'.
#
# Changes to be committed:
#   (use "git reset HEAD^1 ..." to unstage)
#
#       new file:   nf1
#       new file:   nf2
#       new file:   nf3
#       new file:   nf4
#       new file:   nf5
#

Once successfull, the git will shown the result of rebasing.
[detached HEAD 82c66c9] implement new-feature
5 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 nf1
create mode 100644 nf2
create mode 100644 nf3
create mode 100644 nf4
create mode 100644 nf5
Successfully rebased and updated refs/heads/new-feature.

Check our history log again. Notice all those commit of nf1 till nf5 have been squashed or combined into a new commit of 82c66c9 and the new-feature branch is ahead of master branch by 1 commit. Basically, we're using rebase to main a linear history.
$ git ll
* 82c66c9 (HEAD, new-feature) implement new-feature
* 1245945 (master) mc3
* 2e803fb mc2
* 885e8be mc1

Last step, merge our new feature into the master branch.
$ git checkout master

$ git merge new-feature
Updating 1245945..82c66c9
Fast-forward
nf1 | 0
nf2 | 0
nf3 | 0
nf4 | 0
nf5 | 0
5 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 nf1
create mode 100644 nf2
create mode 100644 nf3
create mode 100644 nf4
create mode 100644 nf5

Checking our history log again.
$ git ll
* 82c66c9 (HEAD, new-feature, master) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

Scenario 2 : Keep your local branch up-to-date

If you notice in Scenario 1, the master branch stays stagnant without any additional commits. What if while we're developing on the branch and there are other commits merged to the master branch, as in other features or hotfix ?

Let's try again, but this time, we're going to create a hotfix branch and add a sample commit to fix an issue. Our commit in hotfix branch is currently the HEAD and is ahead the master branch by 1 commit.
$ git checkout -b hotfix
Switched to a new branch 'hotfix'

$touch hf1; git add h1; git commit -m "hf1"

$ git ll
* f229ff9 (HEAD, hotfix) hf1
* 82c66c9 (new-feature, master) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

During that period, there are some changes committed to the master branch. Let's add a few commits to it as well. Checking our commit log again, you'll notice a divergence between hotfix and master branchW. In other words, we've a forked commit history.
$ git checkout master
$ touch mc4; git add mc4; git commit -m "mc4"
$ touch mc5; git add mc5; git commit -m "mc5"

$ git ll
* bbb1a2b (HEAD, master) mc5
* 4472d3e mc4
| * f229ff9 (hotfix) hf1
|/  
* 82c66c9 (new-feature) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

Before we proceed with any merging or rebase, please make a copy of the current foobar folder. We're going to show the difference between using rebase and not using rebase.
$ cp -rv /tmp/foobar /tmp/foobar.orig

First, we try the merge without using rebase. After merging, we're going to add one additional commit so make our commit log more meaningful.
$ git checkout master
Switched to branch 'master'

$ git merge hotfix
Merge made by the 'recursive' strategy.
hf1 | 0
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 hf1

$ touch mc6; git add mc6; git commit -m "mc6"

Pay attention to the commit log where we're going to compare with the rebase method. Notice the additional commit 15ea73b as well as the forked history.
$ git ll
* 40fdd57 (HEAD, master) mc6
*   15ea73b Merge branch 'hotfix'
|\  
| * f229ff9 (hotfix) hf1
* | bbb1a2b mc5
* | 4472d3e mc4
|/  
* 82c66c9 (new-feature) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

Before that, restore our last snapshot of the repo before merging the hotfix branch.
$ rm -rf /tmp/foobar
$ cp -rv /tmp/foobar.orig /tmp/foobar
$ cd /tmp/foobar

Continue with merging using rebase.
$ git checkout hotfix
Switched to branch 'hotfix'

$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: hf1

$ git ll
* cfd2dae (HEAD, hotfix) hf1
* bbb1a2b (master) mc5
* 4472d3e mc4
* 82c66c9 (new-feature) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

$ git checkout master
Switched to branch 'master'

$ git merge hotfix
Updating bbb1a2b..cfd2dae
Fast-forward
hf1 | 0
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 hf1

$ touch mc6; git add mc6; git commit -m "mc6"

Compare to the non-rebase merging, we'll obtain a linear history graph without additional commit or history. Also, no forked history log as well.
$ git ll
* e0615b2 (HEAD, master) mc6
* f8df51d (hotfix) hf1
* bbb1a2b mc5
* 4472d3e mc4
* 82c66c9 (new-feature) implement new-feature
* 1245945 mc3
* 2e803fb mc2
* 885e8be mc1

Comparing both the history log of using and not using rebase, I think I finally grok how the need of Git rebase compare to typical merging.

Using GNU Stow to Manage Your Dotfiles

As you know, as an avid console user for many years, you are your dotfiles. My current setup to manage these dotfiles is a combination of combination of Homesick and Git, or specifcally Github.

However, as Homesick itself is a Ruby gems, the dependacy on Ruby was unnecessary heavy and wasteful. After reading Brandon Invergo's experience on using GNU Stow, a symlink farm manager, to manage dotfiles, I was tempted to give it a try. In short, the program itself is more lightweight, portable, and simpler.

First, download the install the program.
$ sudo apt-get install stow

Next, we're going to setup our dotfiles [per-application][8]. Just create the parent folder (dotfiles), the sample package folder (git), and the dotfile we want (.gitconfig).
$ mkdir -p ~/dotfiles/git
$ touch ~/dotfiles/git/.gitconfig

$ tree -a ~/dotfiles/
dotfiles/
└── git
└── .gitconfig

Go to our parent folder (dotfiles) and create the symlink. That's it!
$ cd ~/dotfiles
stow dir is /home/kianmeng/dotfiles
stow dir path relative to target /home/kianmeng is dotfiles
Planning stow of package git...
LINK: .gitconfig => dotfiles/git/.gitconfig
Planning stow of package git... done
Processing tasks...
Processing tasks... done

$ ls -l ~/.gitconfig
lrwxrwxrwx 1 kianmeng kianmeng 23 Mac   8 15:47 /home/kianmeng/.gitconfig -> dotfiles/git/.gitconfig

If the dotfile you're trying to symlink existed, Stow will complain. Let's illustrate this.
$ touch ~/.dummy
$ touch ~/dotfiles/git/.dummy
$ cd ~/dotfiles
$ stow -vv git
stow dir is /home/kianmeng/dotfiles
stow dir path relative to target /home/kianmeng is dotfiles
Planning stow of package git...
CONFLICT when stowing git: existing target is neither a link nor a directory: .dummy
--- Skipping .gitconfig as it already points to dotfiles/git/.gitconfig
Planning stow of package git... done
WARNING! stowing git would cause conflicts:
* existing target is neither a link nor a directory: .dummy
All operations aborted.

To remove the link, just type
$ rm -rf ~/dotfiles/git/.dummy
$ stow -vvD git
stow dir is /home/kianmeng/dotfiles
stow dir path relative to target /home/kianmeng is dotfiles
Planning unstow of package git...
UNLINK: .gitconfig
Planning unstow of package git... done
Processing tasks...
Processing tasks... done

$ ls -l ~/.gitconfig
ls: cannot access /home/kianmeng/.gitconfig: No such file or directory

If you're dotfiles directory is not located under your home direction, for example /home/kianmeng/dotfiles but instead /tmp/dotfiles, you'll need to specify the target path. Otherwise, the symlink will end up in the parent directory, and in this case, /tmp.
$ mv dotfiles /tmp
$ cd /tmp/dotfiles
$ stow -vv -t ~ git
stow dir is /tmp/dotfiles
stow dir path relative to target /home/kianmeng is ../../tmp/dotfiles
Planning stow of package git...
LINK: .gitconfig => ../../tmp/dotfiles/git/.gitconfig
Planning stow of package git... done
Processing tasks...
Processing tasks... done

While GNU Stow is a alternative way of managing dotfiles, unfortunately it still not a good choice to replace Homesick as it lacks of one essential feature, it can't and won't overwrite the existing files! In the end, I ended up another alternative tool, dfm, the dot file manager, written in Perl to manage your dotfiles.

Postgresql 9.3 Installation in Ubuntu 13.10

Since the usual PostgreSQL’s Personal Package Archives (PPA) has been deprecated in favour of a newer APT repository, is times for me to switch the repository and upgrade my installation to the latest-greatest version.

Surprisingly, there is a quicker and convenient way to install or upgrade your PostgreSQL. Let’s start with this method first.

Download the shell script, make it executable, and run it.
$ wget http://anonscm.debian.org/loggerhead/pkg-postgresql/postgresql-common/trunk/download/head:/apt.postgresql.org.s-20130224224205-px3qyst90b3xp8zj-1/apt.postgresql.org.sh
$ chmod +x apt.postgresql.org.sh
$ sudo ./apt.postgresql.org.sh

Remember, don’t simply download any shell script from the Net without actually peeking it source. One interesting thing I learned from the script is there are many ways to determine the code name of your Ubuntu/Debian distro. The simplest way is using lsb_release.
$ lsb_release -cs
saucy

Search for the latest available PostgreSQL versions, which as shown below is version 9.3.
$ apts ^postgresql-9.? | sort -r | head -n 1
postgresql-9.3-slony1-2 - replication system for PostgreSQL: PostgreSQL 9.3 server plug-in

Let’s us install the latest version, 9.3 and along with PgAdmin, the administrative tool.
$ sudo apt-get install postgresql-9.3 pgadmin3

Create and setup a local user account.
$ echo $USER
kianmeng

$ sudo -u postgres createuser --superuser $USER
$ sudo -u postgres psql
postgres=# \password kianmeng
postgres=# \q

$ createdb $USER

Finally, install the sample booktown database.
$ cd /tmp
$ wget http://www.commandprompt.com/ppbook/booktown.sql
$ sudo -u postgres psql -f booktown.sql template1

$ psql
psql (9.3.3)
Type "help" for help.

kianmeng=# \c booktown
You are now connected to database "booktown" as user "kianmeng".
booktown=# \dt
                 List of relations
 Schema |         Name          | Type  |  Owner
--------+-----------------------+-------+----------
 public | alternate_stock       | table | postgres
 public | authors               | table | postgres
 public | book_backup           | table | postgres
 public | book_queue            | table | postgres
 public | books                 | table | postgres
 public | customers             | table | postgres
 public | daily_inventory       | table | postgres
 public | distinguished_authors | table | postgres
 public | editions              | table | postgres
 public | employees             | table | postgres
 public | favorite_authors      | table | postgres
 public | favorite_books        | table | postgres
 public | money_example         | table | postgres
 public | my_list               | table | postgres
 public | numeric_values        | table | postgres
 public | publishers            | table | postgres
 public | schedules             | table | postgres
 public | shipments             | table | postgres
 public | states                | table | postgres
 public | stock                 | table | postgres
 public | stock_backup          | table | postgres
 public | subjects              | table | postgres
 public | text_sorting          | table | postgres
(23 rows)

booktown=#