Prolific Node.js Programmer

Great write-up on one of the prolific node.js developer, James Halliday aka substack. Wish I could get 1/10th as productive as him. Imagine creating 200+ node.js packages. Look at his github's daily contribution, this guy basically never sleep! Is nice being young, energetic, and focus. Really inspiring !

whoopsie user and daemon?

Was checking the current running process (ps aux), the whoopsie process (see screenshot) almost scared the shit out of me. I thought my lappy was infected with trojan. It turned out whoopsie is actually Ubuntu's application crash error tracker which collect error and send to Canonical server.


Now let's try to *apt-get rid* of it.

0. Where is the binary or program located in the filesystem?
$ whereis whoopsie
whoopsie: /usr/bin/whoopsie /usr/bin/X11/whoopsie

1. Which package does this program belongs to?
$ apt-file search /usr/bin/whoopsie
activity-log-manager: /usr/bin/whoopsie-preferences
activity-log-manager-control-center: /usr/bin/whoopsie-preferences
whoopsie: /usr/bin/whoopsie

2. Are there any packages that depends on this program?
$ apt-cache rdepends whoopsie
whoopsie
Reverse Depends:
elementary-desktop
whoopsie:i386
activity-log-manager-control-center:i386
activity-log-manager-control-center:i386
activity-log-manager-control-center
activity-log-manager-control-center
whoopsie:i386
activity-log-manager-control-center:i386
activity-log-manager-control-center:i386
xubuntu-desktop
ubuntu-gnome-desktop
lubuntu-desktop
ubuntu-desktop
activity-log-manager-control-center
activity-log-manager-control-center

3. What are the packages that whoopie depends on?
$ apt-cache depends whoopsie
whoopsie
Depends: libc6
Depends: libcurl3
Depends: libglib2.0-0
Depends: libwhoopsie0
Depends: 
upstart:i386
upstart
Depends: adduser
PreDepends: dpkg
dpkg:i386
Conflicts: whoopsie:i386

4. Ok. Let's now apt-get rid of it.
$ sudo apt-get --purge remove whoopsie
$ sudo apt-get autoremove

Some Clean Up in Ubuntu 12.10

Stuck somewhere with fricking slow internet connection and my Ubuntu installation is going wonky on me. How come ? I tried OpenStack on Ubuntu yesterday and the installation fscked up my system. Hence, some house cleaning needed.

0. Show boot message and hide splash screen during bootup. My lappy is getting unstable these days. Is nice to see some error message instead of blank screen or stuck loading at splash screen.
$ sudo vi /etc/default/grub
="quiet splash"
GRUB_CMDLINE_LINUX_DEFAULT=

1. Remove the old kernels version. First, automatically through unattended upgrade.
$ sudo vi /etc/apt/apt.conf.d/50unattended-upgrades
# Unattended-Upgrade::Remove-Unused-Dependencies "false";
Unattended-Upgrade::Remove-Unused-Dependencies "true";

Second, directly using apt-get remove. Inspired by the the one-line removal command. Note that I break the command into multiple lines because g+ does not support bloody pre tag !
$ sudo apt-get remove $(dpkg -l \
    | egrep 'linux-(image|headers)-[0-9]' \
    | grep -v `uname -r` \
    | awk '{print $2}')

2. List all recently-used program with last-accessed time (atime) using popularity-contest program and remove them manually.
$ popularity-contest > apps.log
$ less apps.log

Real-life Enterprise Programming

"But in most of the real world, when you've got various stakeholders, a big project, lots of programmers whose work needs to be coordinated, with standards, and you can't get bogged down in n-to-n communication, this is just how it often has to be. Face it, most work in this world is "slave work", if that's how you want to call it (although I think that sounds rather insulting, when you think about how an unfulfilling office job you're free to quit is a million times better than actual slave work)."
......
"And part of being a professional and mature programmer is realizing that, look, that's life. Sometimes you will get a job with a big say in how things are done, be super-independent, and it will be amazing. And sometimes you'll look back five years later and realize how it was a great learning experience for you, but you over-engineered it in a language or with a library that nobody could use afterwards, the whole system was replaced a year later, and you effectively wasted a lot of time and money of the company's."
-- crazygringo, emphasis mine

Something about PHP's SAPI

0. As usual, let's install all the necessary packages in Ubuntu 12.10 before we proceed with the example code.
$ sudo apt-get install php5-cgi php5-cli libapache2-mod-php5

1. Let's write a simple PHP script and put this file (sapi.php) in the document root (/var/www). Content as shown below.
$ cat /var/www/sapi.php 

2. Let's run this script from the console.

a) Plain old PHP binary, not through the web server.
$ php -f /var/www/sapi.php 
cli

b) Similarly but using the php5-cgi binary instead.
$ php5-cgi /var/www/sapi.php 
X-Powered-By: PHP/5.4.6-1ubuntu1.1
Content-type: text/html

cgi-fcgi

c) Again, but using curl program, a HTTP client making request to the HTTP server (Apache).
$ curl http://localhost/sapi.php
apache2handler

3. What does the result in step 2 tell us?

First, the method php_sapi_name() which will give us the type of the Server API (hence the name SAPI) used by the PHP binary. Think SAPI like a plugin module with different abstraction layers for different type of APIs. Second, an explanation of each SAPI layers in step 2.

2a - PHP standalone interpreter itself.

2b - Common Gateway Interface (CGI). A standard way of a HTTP server to delegate content to a program. In this case the PHP interpreter. Compare to 2a, this interface do pass extra environment variables to the PHP program.

2c - mod_php where the PHP interpreter is embedded as module in Apache web server. Similarly to CGI, both are the only two common communications between Apache and PHP. As mentioned, the main different is CGI delegate PHP script to an external binary where for mod_php, it was passed to Apache itself.

Another way to test this out is to get the phpinfo() result by repeating the step 2. Result obtained will be quite different.

4. Why I am writing about this?

I was trying to write a console script to extract Zend Optimizer+ statistics result but the result will not be the same running through plain interpreter and mod_php. Because my lack of fundamental understanding of this concept made me wasted 3 hours. Well, you have to learn the lesson the hard way somehow.

Micro-Routing Framework

Interesting question on which micro-routing frameworks for PHP in r/php. Slim, dispatch, and klein are my favourites for their simplicity and procedural approach. Aura Router looks interesting too but feels too heavy.

As I mentioned before, someone should really add the missing routing functions to the PHP core. Thus, making PHP a fully-featured micro-framework with PDO as model and PHP itself as view. As they said, PHP as Template Engine (PATE). Two benefits for doing so. First, we have a standard way of doing routing in PHP. Second, no dependancy on any frameworks for single file script (think apc.php) or small size app (think simple REST app).

Can someone in php-internals add this to PHP 5.6?

Obsession With Tools

"I've found that a large subset of programmers share the same trait as a large subset of photographers: they become so obsessed with tools that they lose sight of the fact that the end product is what matters at the end of the day."
......
"My advice? Sit back, pick technology a couple of steps behind the bleeding edge, and focus on results. Choosing Ember over Backbone isn't going to cause your project to fail; building the wrong thing or failing to finish, however, will."-- rpeden, emphasis mine

MySQL Option File : Configuration and Troubleshooting

Examples discussed here are in Ubuntu 12.10.

0. The global option is located in /etc/mysql/my.cnf
$ ls -l /etc/mysql/my.cnf
rw-r--r- 1 root root 3505 Sep  7 16:23 /etc/mysql/my.cnf

1. Additional settings to override the settings in global options must follow two rules. First, the file names must be end with ".cnf" extension. Second, these files can be found in /etc/mysql/conf.d or ~/.my.cnf.

2. To prevent yourself from repeatedly typing the same parameters (yes I know, you can use bash's alias), put these option in your user-specific option file, ~/.my.cnf.
# read by all clients. e.g mysql, mysqldump, or mysqladmin
[client]
user=root
pass=password

# read only by the targeted client
[mysql]
database=mysql
pager=less

3. What are the available options for each client ? Display the help documentation and look for all those item with double dashes, --option=value
$ mysql --help | less
$ mysqladmin --help | less

4. World-writable option file is ignored. Important to understand this if somehow you mistakenly changed the config file permissions.
$ ls -l ~/.my.cnf 
rw-rw-r- 1 kianmeng kianmeng 82 Feb  8 07:43 /home/kianmeng/.my.cnf

$ chmod o+w ~/.my.cnf
$ ls -l ~/.my.cnf 
rw-rw-rw 1 kianmeng kianmeng 82 Feb  8 07:43 /home/kianmeng/.my.cnf

When you try to run any of the mysql client program, all the options set in the ~/.my.cnf will be ignored. How to troubleshoot this ? Imagine your officemate put a prank on you and you're really stuck ?
$ mysql --print-defaults
Warning: World-writable config file '/home/kianmeng/.my.cnf' is ignored
mysql would have been started with the following arguments:
--port=3306 --socket=/var/run/mysqld/mysqld.sock 

Revert back our changes.
$ chmod o+w ~/.my.cnf
$ ls -l ~/.my.cnf 
rw-rw-r- 1 kianmeng kianmeng 82 Feb  8 07:43 /home/kianmeng/.my.cnf

Double check our config file is loaded properly.
$ mysql --print-defaults
mysql would have been started with the following arguments:
--port=3306 --socket=/var/run/mysqld/mysqld.sock --user=root --pass=password --database=mysql --pager=less

Python Malaysia February Meetup 2013

Updates : Malcolm passed away on March 17, 2013.

First meetup this year. The guest speaker is Malcolm Tredinnick, one of the core developers of Django Web framework. He shared his experiences on the framework and how to contribute to the project. These are some notes of the talk.

On contributing to the Django project or how you can help.
  1. Check out the project from github, run the unit test, and submit bug report. According to him, almost 50% of Django codecase are unit testing code.
  2. Alternatively, you can start helping by searching through the bug reports for new and easy pickings tickets. Besides, you can help by verifying new ticket.
  3. Subscribe to the mailing list, read and understand the discussion, become a "silent expert". Also, try to answer questions in the mailing list or stackoverflow.

On learning Python.
  1. Stick to Python 3.3 (latest version), especially you're a beginner.
  2. Use the Six [9] module to make sure Python code run fine in both 2.x and 3.x.

That's all I can remember and jot down.

On Moolah !

Unbelievable ! Simply unbelievable !

I'm striving to get out of debt. Consciously monitoring my cash flow, every cents that comes into and goes out of my life. Resist any temptation to buy anything I want rather than I need. Yet, there are people who still want to get into massive debt and thus increase their commitment further.

Why you want to spend the money you don't have ?
Do you really need a new house when you're just living alone ?
Do you really need to get a new car when the old one still works or fixable ?
Do you really need to get the latest greatest gadget ?

This is madness.

Shopping List for Raspberry Pi

Decide what you want to do with this machine. Is this a media player, cluster computing, hardware hacking, or just another toy ? Then only you proceed with the purchase.

1. Raspberry Pi, Model B, 512MB RAM, MYR 111.00
Required. You can get this from both RS Component or Element 14. I prefer to shop from Element 14, better service and order page.

2. Scandisk SD Card Class 4 4GB, MYR 16.00
Required. If you want to setup a torrent client, get the 32GB version. If you want to test out different images (e.g. Openelec or XMBC) but lazy to reburn images or setup cluster computing, buy a few 4GB.

3. Power Adapter, MYR 19 - 50
Required. Be extra careful with selecting power adapter. I bought a brand-less traveller charges which can't seems to power up the Pi (there are no lights). Either you use your own smart phone charges or get the Vztec power adapter.

4. Keyboard, MYR 15.00 - 100.00
Required. We need to type and setup the machine, especially the SSH and VNC so that later we can connect remotely from our laptop.

5. Multicomp MC-RP001-CLR Clear Case, MYR 28.70
Optional. Is better to get a enclosure to protect the board. Furthermore, is kinda of scary to touch the board while plugging the power adapter. I didn't get one though and will definitely buy one.

6. HDMI-to-DVI adapter, MYR 29.00
Optional. If you have an existing LCD/LED monitor with DVI output. Get this cable.

7. HDMI-to-VGA adapter, MYR 40 - 100
Optional. Own a monitor with only VGA output? Expect to pay more for an HDMI-to-VGA adapter. Due to limitation of digital to analog conversion, you can't just do a HDMI-DVI-VGA conversion, unless you buy a special designed HDMI-to-VGA adapter for Pi from Adafruit , Element 14, or Neewer.

Note that I am still looking for a cheaper way to get this to work.

8. HDMI cable, MYR 20 - 50
Optional. If you own a LCD/LED tv, you can simply use the default HDMI cable.

9. TP-LINK TP-725N / EDIMAX EW-7811Un, MYR 26.00 - 33.00
Optional. Both work out of the box. For TP-725N, you will need to upgrade the Linux Kernel to version 3.6.x.

10. HP 19”W1972A (LED VGA/DVI), MYR 235.00 
Optional. Unless you've a LCD/LED tv, otherwise you will still need a display. Monitor that support HDMI connection are expensive and the preferable choice is to get an monitor that support DVI connection and use a HDMI-to-DVI adapter.

Clickstream Logging with Apache 2 mod_cookietrack

Continue with yesterday installation step but using the mod_cookietrack module instead. Installation steps as follows:

1. Check out the source code from github repository.
$ git clone https://github.com/jib/mod_cookietrack.git

2. First determine the type of Multi-Processing Modules (MPM) used. As shown below, in my Ubuntu 12.10, the default Apache is using Prefork MPM.
$ $ sudo apachectl -V | grep MPM
Server MPM:     Prefork
 -D APACHE_MPM_DIR="server/mpm/prefork"

3. Find the necessary, in this case, prefork related development library because to compile the module. As the development package will provide us with apxs, Apache eXtension tool.
$ sudo apt-cache search prefork | grep apache2 | grep dev
apache2-prefork-dev - Apache development headers - non-threaded MPM

4. Install the required packages
$ sudo apt-get install apache2-prefork-dev perl

5. Before we can compile the module, change the shebang (first line of the script) as the path is wrong. Otherwise the script will not run. Amend the first line to :
$ vi build.pl
#!/usr/bin/env perl

We use /usr/bin/env instead of direct full path /usr/bin/perl for portability. See shebang for further explanation.

6. Make the script executable and run it.
$ chmod +x build.pl
$ sudo ./build.pl

7. Our module and the configuration file will be installed in following locations.
/usr/lib/apache2/modules/mod_cookietrack.so
/etc/apache2/mods-available/cookietrack.load

8. For testing purpose, we need to install node.js to create a sample node server. We also need to change the test/run_backend.js script to rename node to nodejs. Ubuntu 12.10 default packages uses different binary name.
$ sudo apt-get install nodejs
$ vi test/run_backend.js
nodejs test/server.js

$ test/run_backend.sh

9. Open a new terminal. Start a new customer Apache daemon. However, if your follow the original installaton instruction, you will encounter this error.
$ sudo test/run_httpd.sh
$ tail -f test/error.log
[Sun Feb 03 01:01:05 2013] [alert] (22)Invalid argument: setgid: unable to set group id to Group 4294967295

The error is due to missing group id in the test/http.conf file. We will add this in.
$ vi test/httpd.conf
User www-data
Group www-data

10. Open another terminal to run the test suite. The --debug flag will dump all the HTTP request and response content.
$ perl test/01_cookietrack.pl
$ perl test/01_cookietrack.pl --debug

11. Open another terminal to monitor the error log.
$ tail -f test/error.log

12. Since this module is a enhancement and replacement for mod_usertrack, all the mod_usertrack's configuration options should work seamlessly. Hence, we need to disable the mod_usertrack module we setup in previous post.
$ sudo a2dismod usertrack
$ sudo apachectl restart