This Week I Learned - 2016 Week 52

Last week post or the whole series.

Week 52, the last week of year 2016. ;-)

It has been a long way since the beginning of the whole TWIL series. I've reached my goal of writing at least 52 posts this year, I managed to hit 58 posts. Now what? Should I continue this TWIL series? Yes, indeed I should. There is no harm but benefits by continue this ritual. Maybe I should try different approaches and more diversify topics in my posting. Yearly reflection post should come later.

This Arduino project to calculate speed or to implement digital speedometer seems interesting and fun. It was quite a surprise but if you understand fundamental mathematics, it all seems easy and approachable.

Using Pandoc with different fonts and rendering engines. Below command will list out all the available fonts in your system.
$ fc-list : family | cut -f1 -d"," | sort

PostgreSQL 9.6 new features with examples (PDF) via Lobsters. All software release notes should accompany by details example on any features being updated or introduced. I have mentioned numerous times that I miss PostgreSQL a lot.

Discussion on C++17. Now I can understand why C++ is still not that suitable for Operating System development but more for application development which needs performance and full memory management.

This Week I Learned - 2016 Week 51

Last week post or the whole series.

Week 51 of 2016. The last second week of the year. Lots of time to reflect and ponder about my past and future. Here is an video of interviews with three centenarians (those who live past 100 years) on their secrets for living so long. There maybe many years to go with your life but never overlook the issue of mortality. Learn as much as you can, do things you enjoy, and spend your time with someone who both of you are mutually care about.



List of single-file C/C++ libraries. Someone really needs to compile a bigger list. There are a lot that needs to be picked up by me when come to C++, mostly are all the fundamental stuff. Good example is do you know there are different types of memory allocated (stack, heap, bss, data, and text) for a program? That's why good fundamental of computing hardware is crucial being a system developer. Background on how Operating System works or manage the memory is something i severely lacking. Which is why I'm still having a hard time understanding the whole C++ compilation process (another good introduction with plenty of diagrams).

While we're on C++, if you are creating a new project, this can be the folder structure of the project you can follow. Between a macro, a function, or a class, just stick to something simple, use macro for creating debugging details.

Another interesting item caught my attention, conversion of other data type to string in C++ is not as straightforward as I assumed it is. There are many approaches, such as the standard C++ or C++03 way (using stringstream), the C++11 way (using std::to_string), or the Boost way (using boost::lexical_cast). Unless you're stuck with legacy C++ compiler, use the C++11 method. While we're looking at string, C++ being a static typed language, string manipulation have to be expressed in an explicit manner. For example, to pad a string to a certain size and characters as shown below.
std::cout 
<< std::left << std::setw(10) << std::setfill('.')
<< foobar << ": " << 42 << '\n';

// output
foobar....:42

Need to implement your own logging function? This is the time where you need to use a C++ macro. Otherwise how can you use the preprocessor macro like `__FILE__` or `__LINE__`?

Function pointer in C/C++. Useful when you have an array of items that point to a corresponding functions.

It has been a while since I ever look into PHP. Found two interesting libraries, Httpful and Underscore.php. Both are either direct port of influenced by similar libraries in other programming language. Not really sure about the current state of PHP but I glad I ended up doing something else other than PHP. It's always nice to code in different kind of programming languages.

Hook system in Perl? This needs a third party CPAN module like Hook::LexWrap. Otherwise, it can be quite hackish.

This Week I Learned - 2016 Week 50

Last week post or the whole series.

Two more weeks to go before we close the chapter for the year 2016.

Am I a good programmer? Think again, it's not just you but the remaining 99% of developers as well. The key point here is stop comparing, but instead identify the gap, and work out a plan to reduce the gap. This is doable provided that you realize what HN user, socmag advised that "...program if you enjoy it. Everything else will come. Financial reward is a side effect, not a cause.". Otherwise, just do something else. YOLO. Maybe rephrase the question, ask yourself, am I having fun? Need more advice? Read through similar discussions in HN.

Discipline is sustain through repeating a specific behaviour. How can we develop this systematically? Pick and develop ONE habit. Start small and start seriously. Don't overwhelmed yourself. Do it on daily basis and no more zero day. Anything is better than do nothing. Walking is better than sitting at the couch. Running is better than both. While we at it, get consistent sufficient sleep and rest. Try meditation to enhance your awareness and mindfulness as well. Follow the cue/craving -> response -> reward cycle suggested by Charles Duhigg. Do this for 66 days instead of the conventional 21 days. Also, another good write-up on building habit. Good luck!

Learning modern C++? So many resources, unfortunately, mostly are outdated and some still use Turbo C++. It seems C++ also suffered the same fate as Perl, hence, Modern Perl books was written. Then where can we obtain up-to-date information? Arne Metz suggested a few sites to check out for a start.
Follow up on implementing Full-text searching (FTS) in MySQL in week 48. The SQL example given there only suitable for searching exact keyword. What if you want to search partial word matching instead ala SQL LIKE operator? This is possible using asterisk (*) and BOOLEAN MODE modifier. Under this mode, the asterisk (*) is a wildcard operator, as in '%' in SQL. Example as shown.
SELECT name, MATCH(name) AGAINST('bio*' IN BOOLEAN MODE) AS relevance
FROM subjects
WHERE MATCH(name) AGAINST('bio*' IN BOOLEAN MODE)
ORDER BY relevance DESC
LIMIT 10

Something interesting on Git encountered this week. As I was git pulling the source, the unpacking process was so slow and I killed it with Control+C. The issue by doing this is that you've a lot of dangling commits or blob.
$ git fsck
Checking object directories: 100% (256/256), done.
Checking objects: 100% (540/540), done.
dangling blob f7c89a6a3fd135a16531bd776ecf04dcc9096cc1
dangling blob c66981a6cc4b877e1fe2064e6423c21831e308b3

To clear these dangling blob, you will need to clean up. Later I found out that unpacking was slow because there are a lot of binary files (graphic files) were added.
$ git gc --prune=all
Counting objects: 953, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (541/541), done.
Writing objects: 100% (953/953), done.
Total 953 (delta 392), reused 498 (delta 204)

Branching is cheap and free for SCM like Git. This is one concept I forgotten and during the last sprint, having two developers working on a single branch maybe not be a good idea unless both of you are can resolve conflicts correctly, do not rebase from master branch until testing, and squash your commit before merging as shown below.
$ git checkout featureX
$ git merge --squash featureX-dev1
$ git commit

Reflection on being a developer after 40. Most of the advices he gave were spot on especially choosing the galaxy (technology stack) wisely and open spaces office is crap. Yes, open collaboration my big foot, open noises would be the right description.

Learning Perl? Need to advance your Perl knowledge? Read Advanced Perl.

This Week I Learned - 2016 Week 49

Last week post or the whole series.

When a screenshot says a lot. The 14-plus hours uptime is something to be concerned about. Is best to be away from your machine from time to time. As you age, there is no needs to tweak your environment, just use the default settings for almost everything.


Certain unfortunate requirements led me to buy a USB-based high powered 300Mpbs Wifi adapter, TP-Link TL-WN8200ND. Unfortunately, while the driver seems to load properly, I still can't connect through Wifi in Ubuntu 16.10.

Understanding htop. Comprehensive guide to htop and its equivalent console commands. It will make you realize how much htop have aggregate and collect all the necessary information. The same author also wrote another useful guide on HTTP headers.

While we're on HTTP. Encountered this error with Nginx few weeks back where the error log shows that "upstream sent too big header while reading response header from upstream". In other words, your proxy server, Nginx does not like the data sent over from the application server (upstream). Several reasons may cause this like large cookies size, cookies with way old timestamp, or mismatch of response size and content length. Several ways to resolve this, either fix the issue at the upstream , disable proxy buffer, or increase proxy buffer sizes. Example as shown below (do not follow this values, adjust accordingly). Don't understand these settings? You can read details explanation and an excellent guide on these directives.
http {
    proxy_buffers           8 4k;
    proxy_buffer_size         8k;
    proxy_busy_buffers_size   16k;
}

On Perl. Nothing much pickup for the last two weeks, mostly just test cases and test cases. Interesting behaviour when returning value from subroutine. Being Perl, implicit is better than explicit as compare to Python. For example, there is this rule of do no return `undef`, just use the bareword `return`.
use Dumper;
sub a { return undef; }
sub b { return; }

my @aa = a();
my $a = a();

my @bb = b();
my $b = b();

Dumper(\@aa); # [undef], not false
Dumper($a); # undef, false

Dumper(\@bb); # [], false value because empty array
Dumper($b); # undef, false

How to implement Test-Driven Design (TDD) in Perl? So many good links given in those answers to the questions. Unfortunately, most of the links are quite dated and some may not be that relevant anymore. But since this is Perl, most stuff should be long standardized and stable.

This Week I Learned - 2016 Week 48

Last week post or the whole series.

December. How fast the time flies as we're approaching the end of the year 2016. Four more weeks to go and we will embrace the new year 2017. Yet, there is so much more to do here and there.

Full-text search (FTS) support for InnoDB was added in MySQL since version 5.6. While is a welcoming feature, especially those who don't want to use third party search engine like Sphinx or Apache Solr, there are still some default behaviours that you'll need to be aware of. First, there is this minimum and maximum word length to be indexed. By default, the minimum word length is three. If you need to produce results with two word length, consider adjusting the server settings and restart it later. Next, you can enable stop words being indexed. Stop words are common words in the language likes "the", "is', or others. Both settings are discussed here in good details.

Looking for FTS full examples, do look into Gutenberg book searching or song searching implementation. For a quick example, below SQL query is good enough for you to get started using FTS through multiple tables.
SELECT *, 
    MATCH(books.title) AGAINST('$q') as tscore,
    MATCH(authors.authorName) AGAINST('$q') as ascore,
    MATCH(chapters.content) AGAINST('$q') as cscore
FROM books 
LEFT JOIN authors ON books.authorID = authors.authorID 
LEFT JOIN chapters ON books.bookID = chapters.bookID 
WHERE 
    MATCH(books.title) AGAINST('$q')
    OR MATCH(authors.authorName) AGAINST('$q')
    OR MATCH(chapters.content) AGAINST('$q')
ORDER BY (tscore + ascore + cscore) DESC

On SQL. Sometimes the solution was so simple that we have overlook even the basic default feature. If you want to find the unique and maximum row by each group which sorted overall, the direct approach is just use MAX aggregate function. Example as shown below. Another approach is to use user variables, not my preference though.
SELECT t.client_id, MAX(t.points) AS "max"
FROM sessions t
GROUP BY t.client_id 
ORDERY BY MAX(t.points) DESC

Want to retain the order of the SQL query in your `IN()` operator? Use MySQL `FIELD()` function. Example as shown.
SELECT * FROM table ORDER BY FIELD(ID,1,5,4,3);

Perl Advent Calendar 2016 have started. I've mixed feeling regarding the first day post before Christmas. Till today, we still don't have a graphing feature built-in to visualize class relationships for any IDE out there for any less supported languages. And yet, we still needs to rely on Graphviz to visualize it. While Graphviz is an excellent tool, it lacking one crucial feature, better automatic diagram layout, something similar to yWorks' yFiles library.

Which Git workflow should you use? Gitlab have a discussion on different workflows. For me, master branch is always deployable approach seems to work for me. This HN user describes it succintly. Key points are:

1. Master branch is the deployable branch
2. All new features and developed in feature or topic branches.
3. Continiously rebase from master. Read this on resolving rebasing conflicts.
4. Send pull request for reviewing.
5. Testing is done in feature branches after rebasing from master and signed off from reviewer.
6. Feature branch is merged to master with a `--no-ff` (no fast foward). See diagram below.


The Github Flow is have the similar approach but the feature branch is deployed first before merge back to the master branch. The advantage of this approach is that you can always rollback to the master branch. Another variation is the Git flow, more complex with additional develop branch. Trunk-based development is another approach but I used before (another variation). However, I don't like the complexity introduced. Suitable if you have a dedicated release engineer.

Looking back at C++ again and I've no idea what I'm looking at. Why use typedefs to create alias for the default basic types? Portability. Implementing callbacks is a bit tricky, you really need to get over the C++ syntax used. Delegation in C++? Seriously, so many ways? All looks very hackish to me. Where is ? Didn't realize C++ don't have a ISO standard for quite some times. Undefined references? Most likely the order of the files being compiled causing linker problem.

Overwhelmed by front-end development works? The front-end scene is a moving target right now. There even a study plan to cure all these Javascript fatigue.

This Week I Learned - 2016 Week 47

Last week post or the whole series.

What an excruciating stressfull week. So many things to follow up and so many things broken, including my own body till I have lost 3kg. On a positive side, when you're down with sickness, your perpestive towards your environment changed, in a slightly turn off way.

On Git. I just realized, unintentionally, my git-fu just increased by 0.5% since last two weeks. When checking for merge through rebasing and merging, it seems the LOCAL and REMOTE branch have been interpreted differently in all merging tools. P4Merge interprets it differently? To summarize it, LOCAL is the originals, REMOTE is the changes you want to add regardless it's a rebasing or merging process.

Now, after you've resolved all the conflict either through rebasing or merging, to visualize and compare your changes (just to be sure), we can use `git diff` command to compare between two ranges.
$ git diff branchX..branchY

Note the double-dot to specific range there. It seems that you can specific either double-dot or triple-dot to specific ranges but indicates different types of output. The Venn diagrams and commit trees below shows the differences.




There is one practive that I do follow when using Git in feature branch, which is to commit early and commit often. However, one of the issue is that the feature branch history is cluttered with many tiny commits. While this is useful when working with others on the same branch (we're aware of what's going on), it's best to squash these commits before merging into `master` branch. There are two ways (with different behaviours) to squash all the commits.

First, is to squash these commits when merging to `master` branch. The source branch, in this case, `branchX` will be throw away.
$ git checkout master
$ git merge --squash branchX
$ git commit

Second, is to sqush these commits when rebasing. This is my prefer method where we still keep the source branch. However, if you have a lot of commits, it's quite slow as you have to assign the rebasing method (either squash or fixup) for each commit. Some Git client tools support this feature to squash commit but I never really explore this.
$ git checkout branchX
$ git rebase -i `git merge-base branchX master`

That about it for this week, more stuff to come in coming weeks as we're approaching the end of they year.

This Week I Learned - 2016 Week 46

Last week post or the whole series.

Busy week. Probably the busiest week ever and I only managed to settle down. Sickness, faulty workstation, and idiocracy all over the places.

Follow up from last week in investigating C++. It seems that `unsigned` is equivalent to `unsigned int`.

On the Perl front. Seriously, Perl's CPAN does have almost everything under the sun. Yes, the damn xxx helper function does exists. When I started my job as a programmer many moons ago, we have this helper debug cum print function called `xxx` in PHP. Besides the benefits of easy to type and identify, the Perl's version does not requires any parenthesis (one of the quirks of Perl), thus a major saving of two key strokes and your pinky as well.

C++ - Differences Between `unsigned` and `unsigned int` Data Type

During a discussion last week, I've made a mistake when discussing the `unsigned` data type with an acquaintance (not from programming background). Basically, I've misinformed him that `unsigned` is not a data type and it should be `unsigned int` instead. The truth is, in C++, `unsigned` is a data type, implicitly is an alias to `unsigned int`.

To illustrate this, let's use the below code, through typeid operator to determine the data type at runtime, a C++ mechanism called Real-time Type Information (RTTI).
#include <iostream>
#include <typeinfo>

using namespace std;

int main() {
    unsigned int x = 10;
    unsigned y = 20;
 
    cout << typeid(x).name() << endl;
    cout << typeid(y).name() << endl;
}

Compile and run the code. Observe that the output shows two js? Because in GCC, the return name is a decorated name (which have been mangled) and we need to demangle it.
$ g++ typeid.cc -o typeid
$ ./typeid
j
j

To demangle the name, use the c++filt program, result as shown below. Now the question here is why the function name was mangled? As C++ supports function overloading, a feature where you can define two or more functions/methods with the same name but different function parameters. Conversion to the assembly code needs unique assembler name for these functions. This is where the process of mangling comes in. The c++filt tool reverses the process to find the exact name.
$ ./typeid | c++filt -t
unsigned int
unsigned int

Lots of new things learned here but most importantly, always double check (google in this sense) for any assumed knowledge before you shared with someone else.

This Week I Learned - 2016 Week 45

Last week post or the whole series.

Slow week. Down with workload at work and home and sick as well.

Lots of C++ related stuff this coming weeks. I was reminded again and again why programming in C++ is not an exciting experiences. The complexity of the whole development environment and the lack of my undertanding of the compilation cycle caused me stuck on simple trivial matters and can't proceed ahead. That's is part of the learning process every one have to go through when picking up something new in such a short notice.

The first concept you need to understand is compilation model which consists of four tools of Preprocessor, Compiler, Assembler, and Linker. The source code goes though these four tools to generate the final executable binary. Coming from a dyanmic-typed programming languages (PHP, Perl, Python, and etc.), the whole process seems quite a hassle. But programs generated by C/C++ is probably the most optimized human readable assembly code. Few things I've picked up.

This Week I Learned - 2016 Week 44

Last week post or the whole series.

Every time I reread Perl in about 2 hours 20 minutes, there is always new insights that increase my understanding of Perl itself. I can finally grok the intricacy of the three data types: scalar, array, and hash, especially the later two. To make it short, don't use list as container for arrays or hashes, just initialize and declare both types as anonymous array(using bracket) or hash (using braces). You can use them through reference or arrow (`->`) operator. Example as shown.
use v5.10;

my $contact = {
    name => 'John Doe',
    mobiles => [
        {carrier => 'at&t', no => '111-222-333'},
        {carrier => 't-mobile', no => '444-555-666'}
    ]
}

say $contact->{mobiles}->[0]->{carrier}; # at&t
say $contact->{mobiles}->[1]->{carrier}; # t-mobile

Another key concept is Perl calls by reference. This means when you pass a variable to a subroutine, it's a reference to the original value, not a copy. Any modification to the variable, will be reflected within the scope or context of the code until the end of the execution. The most important takeaway is that arguments to the subroutine is a list of scalar (yes, the @_ is a list) and the element of @_ is an alias to the passed parameters. According to perlsub documntation (emphasis added),
Therefore, if you called a function with two arguments, those would be stored in $_[0] and $_[1] . The array @_ is a local array, but its elements are aliases for the actual scalar parameters. In particular, if an element $_[0] is updated, the corresponding argument is updated (or an error occurs if it is not updatable). 

While we are on Perl (yes, we still talking about it). What are the preferred ways to check if an element exists in an array? Two most common way. One is using the default grep method, another is using the `List::MoreUtils` method.
# using grep
if (grep { $_ eq $element } @list) {
    ....
}

# using List::MoreUtils
use List::MoreUtils qw(any);
if ( any { $_ eq $element } @list ) {
    ....
}

My migration back to Vim from Sublime Text seems to be progressing quite well. It's nice to expose yourself to another editor and reapply certain features back to your default editor. First, is update tmux tab with Vim's opened file name. This was one of those things that you want to fix it, but never remember to do so. Next, fix copying from clipboard in Windows not working issue. Next, set indentation rules (like tab only) by file type. And lastly, how to open most recent opened files. Something I didn't realize exists in the default settings.

Remember Lorem Ipsum? Which is commonly used as filler text or place holder text in graphic design before the actual content is used. Well, this process is also known as Greeking. In Perl, there is this module, Text::Greeking, which provides such feature. There are others as well, like the usual Text::Lorem. But nothing can compare to the sheer bullshit of Lingua::ManagementSpeak which can generate meaningful sentences but pure bullshit management speak.

"a web browser is a JS interpreter". So true. Maybe I should start looking into all these "ancient" technologies (e.g. Tcl/Tk) instead of chasing latest greatest fad.

Is always a grey area if you choose to do development work in either porn or gambling industry. But I never realize that advertising industry is as shady as well.

Hate to be prompted for password every time you need to commit to Git repository through SSH? Save some typing by caching the credential. There are two ways.

Through SSH.
$ eval `ssh-agent -s`
$ ssh-add ~/.ssh/id_rsa_key
$ ssh-add -l
$ ssh git.example.com

Through Git.
$ git config --global credential.helper cache

This Week I Learned - 2016 Week 43

Last week post or the whole series.

This is one of those week where I become very wordy. ;-)

Previous post was too long when I realized that Blogger don't let me add more than 20 tags. That threshold should be a good indicator for me to stop jotting down the item and move to next week post. Some of these notes below are old items which I haven't have time (lack of discipline) to jot it down in a timely manner (lack of consistency). Don't sweat on this. Reflect, learn, adapt, adjust, monitor, and continue as usual. Although there are some minor hiccups, there is still like 9 weeks to go for this year.

Generating and attached iCal invite through email? It may seems simple as first, but, as usual, it may works for one email client but not for others (we haven't take into consideration of desktop, mobile, and web client). Furthermore, there is iCal and vCal, both are VCALENDAR standard where the former is succesor of the later. Digging deeper, it seems that different email client behaves differently when come to REQUEST METHOD used. The typical troubleshooting procedure is first, make sure you can attach send the iCal correctly through email properly using the email library (using MIME::Lite in my case). Next, read other people code on implementing similar solution. You can may miss out certain API calls or use the library incorrectly.

Testing in Perl. You can go through these articles for a start before you jump in to Test Drive Deveopment (TDD) best practices. Some of the issues I've encountered. First, to compare two different different ordered arrays, `cmp_bag` in Test::Deep is your friend. Second, To simulate delay, instead of using `sleep`, you can use Test::Mocktime's `set_relative_time` to do so. But off course, you will need to learn more tricks about the default standard testing module in Perl, Test::More. Writing a new Perl's module, integrate testing as early as possible

While we on testing. To benchmark the speed of your SQL query, you can disable MySQL query cache. This ensure that no caching is used which may give the wrong impression regarding the speed of the query.

While this is quite old news, two features in Perl 5.10 which I think make a lot of senses. The `defined or` is useful to check if a variable is defined or else use the default value (more examples). And the `state` for declaring a variable as static variable. Useful when you want to implement localized caching.

On Perl again. Check if a substring exists in a string without using Regex through using `index`. I wish Perl have more Object-Oriented way of doing so. Maybe Perl 6 have it?

Note to self. This is very useful. Several alternative ways to monitoring event in Javascript, useful for debugging. My favourite is using the `monitorEvents($0)` method. So simple and straightforward.


While we're on Javascript. Developing a web application with heavy AJAX usages? Well, you have to use `history.pushState` somehow. Why not just switch to Turbolinks?

Storing hierarchical data in database? Adjacent List is the most common and first to use approach when modelling hierarchical tree. However, Bill Karwin's presentation and example and follow up example convinces me of using Closure Table or Bridge Table may be the balance and right approach. This is useful when you're using database system like MySQL that does not supports hierarchical and recursive SQL queries natively.


This Week I Learned - 2016 Week 42

Last week post or the whole series.

Interesting week indeed. It has been a while since I last encountered so many different type of personalities who want or don't want to be a developer.

As usual, what have I learned this week? The usual stuff.

If you're running on GNU/Linux and want a way to manage different Windows OS through Vagrant, you can try this Vagrantfile. Installation and setup is pretty much straightforward, just make sure the Vagrantfile is downloaded. Unfortunately, the login still fail to work.
$ sudo install virtualbox vagrant
$ vagrant plugin install winrm winrm-fs

$ mkdir vagrant_win
$ cd vagrant_win
$ wget https://goo.gl/ATpaEY
$ IE=Win7IE8 vagrant up

Sanic, Python 3.5+ asynchronous web server. The discussion at HN seems rather interesting. While this is nothing new, asynchronous database layer like asyncpg seems rather userful to improve your DB query speed.

Issue with Babun's memory conflict after Windows updates? Try rebasing, not that Git rebasing thought. Cygwin still is the better and prefered choice for Unix experience in Windows. Yes, I know there is Bash on Windows.
1) Exit babun.
2) cmd /c %SYSTEMDRIVE%\Users\%USERNAME%\.babun\cygwin\bin\dash.exe -c '/usr/bin/rebaseall -v'

Customozing HTML's file inputs. Probably the most comprensive guide on different techniques to change the default behavour.

Web framework benchmarks. The Round 10 has one of the best humourous write-up.
The project returns with significant restructuring of the toolset and Travis CI integration. Fierce battles raged between the Compiled Empire and the Dynamic Rebellion and many requests died to bring us this data. Yes, there is some comic relief, but do not fear—the only jar-jars here are Java.
What happens when you rename a branch in Git? Plenty of things. First, you rename it locally. Next, you rename it remotely (is the same as remove the old branch and add a new branch). After that, either you update your upstream URL or checkout a fresh copy of the said new branch. Lastly, you may needs to batch update your commit messages.
$ git branch -m new_name
$ git branch -m old_name new_name
$ git push origin :old_name
$ git push --set-upstream origin new_name
$ git filter-branch -f --msg-filter 'sed "s/foo/bar/"' master..HEAD

The database schema for StackOverflow is publically accessible. I was surprised that it's such a straight forward design and nothing fancy at all. Well, is just a CRUD app with some additional tweaks here and there. However, the ranking formulae is far more interesting when compare to different algorithms used by other popular forum-like sites.

So may ways to iterate through the Perl's array. Implementation 1, 4, and 5 is what I normally used but the 5th method is still my favourite.

Source code syntax higlighter through Javascript? Just found out today, besides highlight.js, there is also Prism.js. The former seems to have more languages support but the later is used for quite a few popular projects.

NBA season going to start soon, maybe is time for me to learn some Statistics through certain API? Can't wait what surprises the 2016/2017 season will give us.

This Week I Learned - 2016 Week 41

Last week post or the whole series.

This is probably the unexpected way or the one-liner new way to purge old Linux kernels. You will need to install byobu (text-based window manager and multiplexer) as the Bash script is part of the package. Why I need to purge the old kernels? Well, I can't upgrade to Ubuntu 16.10 because `/boot` partition don't have enough free spaces.
$ sudo apt install byobu
$ sudo purge-old-kernels

Epoch, the start of a time, is commonly used in computing as a point of reference or date arithmetic. For Unix, the epoch starts from Jan 1, 1970. And I thought that was the standard epoch time used for every Operating System. I didn't realize that for Windows as well as for other platforms, the epoch time was different and it's set to Jan 1, 1601 (represented in FILETIME structure), a few hundres years earlier than Unix epoch time. Why? 1601 is the first year of 400-years cycle of Gregorian calendar.

Conversion between two epoch system times is straight forward using the simple formulae or another approach to calculate the different between two values, which is 11644473600 seconds. (Note that Windows tick is 100-nano seconds interval, which is 10000000). If you have a Windows epoch timestamp (18-digits), use this site to convert to normal date.

Using Git in Windows? Do use the Perforce's P4Merge as git merge tool for the three-ways merging tasks. Learned this while watching how other developer works. You can pick up a lot by watching how others works. Do keep that in mind.

Almost at the end of the year, maybe this is the right time to pick up Golang? Don't like buying Go books, well, someone recommended me to pick up "The Little Go Book".

Looking for beautiful real-time log analyzer? Try GoAccess, which is depends on gwsocket, a RFC 6455 compliant web socket server.. I should install this for my homelab later.

Testing your web application locally but wants to simulate different IP addresses? Try IP Spoofing to simulate HTTP requests

Using testing in C++, use Google Test. Going to try this in coming days if I can get my C++ development environment working.

This Week I Learned - 2016 Week 40

Last week post or the whole series.

My sentiments exactly. See comment below regarding chasing fad in technology development and especially PHP programming language. Do I actually miss PHP? Not really. But I think most of the web development systems still can be solved using PHP-based solutions. Barrier of entry to web development using PHP still the best choice.
Incidentally, this isn't unique to the front-end. I've seen the same thing happen with the SQL->ORM->Mongo->Couch->SQL nonsense, or the myriad of templating engines written for PHP, a language that itself is a templating engine.

Using Node.js? Need a basic system with essential features to bootstrap your system? Look no further than Hackarthon Starter.

The difference between Ponzi, Pyramid, and Matrix Scheme. A lot of young people, especially fresh graduates need to aware and prevent themselves from falling for such scams. The pitch of being your own boss or retire early may sounds too good to be true.

Sometimes the documentation written for a certain API, Data::ICal was so confusing that you have to resolve to searching source code written by others using the same API itself at here, here, here, and here. Can't blame the API author as the standard itself, RFC 5455 is as confusing and complicated.

Google Interview University (via HN). Instead of working for a company, why not strive to work with great people instead? Google is so big and not every team is equal. Agree with one of the comment, this is a good compilation of resource for computer science study. Which reminds me again of the Programmer Competency Matrix. Instead of focusing on the computer science stuff, why not focusing on building stuff? Someone needs to read the interview of famous programmers on their background.

Getting older but still enjoy working as a programmer? Read Reflections of an "Old" Programmer, especially the comments from the blog post itself, HN, and Reddit. The main question here is how do you age gracefully as a programmer? Lifelong learning, especially the fundamentals (the core of computer science, not latest greatest fad frameworks), as those things never changed. I blogged about this last year, during my career detour as a non-programmer role, still technical though.

Didn't realize that to use PowerTop properly, first you will need to calibrate it after installation to collect measurement data. The whole process will take a while and networking will be disabled during that period.
$ sudo apt-get install powertop
$ sudo powertop --calibrate
$ sudo powertop --auto-tune

Besides that, you can turn off unwanted running services to save a bit more battery.
$ service --status-all

Upgrade to LXD 2.3 failed to due a spelling bug in bridge network upgrade script. Fixing the bug manually and restart the installation process again.
$ sudo apt-get install -f

However, new container created don't have a default network bridge. Re-enable this resolved the issue.
$ lxc network attach lxdbr0 test-alpine
$ lxc restart test-alpine

This Week I Learned - 2016 Week 39

Last week post or the whole series.

The month of October have started. As we're closing in to the end of year 2016, I've still have long list of pending items to follow up. Sometimes, is best to take break, reflecting on the current situation, and re-prioritize your short and long term goals. Nevertheless, there is one thing for sure, regardless the outcome, discipline is one sure way to get closer to what you want to achieve.

Interesting stuff I've learned this week.

Few days back, I've managed to finish the book Modern Perl. I bought and starting reading the book on 25th June and finished it by 30th September. It took me roughly 98 days to wrap up the whole book page by page (not a good strategy according to How to Read a Book). My impression? Not a beginner book and not quite a suitable book for any developer switching from other languages. It's more like a reference book on how to code according to the Modern Perl convention. The correct sequence of Perl books to read are as follow:

Language FundamentalsReferences
Learning Perl
Intermediate Perl
Mastering Perl
Programming Perl / the "Camel book"
Modern Perl

While we're in the discussion on book. I'm quite disappointed with Eloquent Javascript. Someone suggested that I should look into `You Don't Know JS` book series instead. Maybe I should try it out instead. It's currently the next item in my reading list.

Again, on Javascript. There is an annoying behaviour of using Array with Object. Seriously? I was caught surprised by this.

On a related note, the popularity of Javascript leads to an issue of Same-origin policy where it's not permitted to request a resource from different domain. The Cross-Origin Resource Sharing (CORS) is a specification to resolve this. Solution exists for different popular web servers with support from different web browsers. For example, how to resolve this matter in Cloudfront.

Life as a Middle-Aged Geek (via HN). The advantage of being a middle-age developer is you probably know what you want or don't want out of your life. As time is catching up and you have plenty of life commitments elsewhere, if better you should constantly review and revise your own life goals. While ageism is an ongoing issue with technology sector, the Old Geek Jobs site was created for those developers whose are (35 and beyond). Or you can be an indie hacker instead?

Look-and-say sequence. Quite an interesting integer sequence, useful when you want to setup a new Wifi password but at the same time, want to confuse the hell of your users.

TLD used for local development? Mine is still using `.dev` but you must be aware that `.dev` is a generic Top Level Domain (gTLD).

Differences between Ansible's `command` and `shell` usages and when to use it. It seems I need to rewrite most of my dotfiles' steps. Didn't realize that `ansible-lint` actual exists.

What are the software stacks or cool tools used in the startup scene?

How to hire a programmer? While technical skills is a crucial requirement, cultural fit and attitude are equivalent important as well. But, as the article mentioned, nothing is guaranteed.

This Week I Learned - 2016 Week 38

Last week post or the whole series. Interesting stuff learned this week.

Encountered this error message when checking a USB thumbdrive with `fdisk` command. The particular thumb drive was burned with an ISO file through the `dd` command.
$ sudo fdisk -l
......
GPT PMBR size mismatch (1432123 != 15765503) will be corrected by w(rite).
Disk /dev/sdc: 7.5 GiB, 8071938048 bytes, 15765504 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 8C18967D-CB41-4EF1-8958-4E495054958D

Device     Start     End Sectors   Size Type
/dev/sdc1     64   17611   17548   8.6M Microsoft basic data
/dev/sdc2  17612   23371    5760   2.8M EFI System
/dev/sdc3  23372 1432075 1408704 687.9M Microsoft basic data

Follow the instructions given, running the device through `gparted` seems to resolve the issue.



Perl's hash initialization, referencing, and de-referencing. Seriously, I need to get this correctly and read more Perl's FAQs.
# Normal way, without referencing.
%foobar = (a => '1', b => '2');
say $foobar{a};

# Using referencing. More readable.
$foobar = {a => '1', b => '2'};
say $foobar->{a};

# Alternatively.
$foobar_ref = \%foobar;
say $foobar_ref->{a};

Finding properties of the event target in Javascript.
$('foo').bind('click', function () {
    // inside here, `this` will refer to the foo that was clicked
});

How do you add a trailing slash if none found? Regex, regex.
$string =~ s!/*$!/!; # Add a trailing slash

Protocol-relative URL. While we're on HTTP protocol, it was made aware to me that the anchor tag should be the last item on the URL.

CSS image sprite technique using HTML unordered list. One of the issue encountered is if you have single line text link, how do you align the text link vertically in the middle? Make sure the `line-height` is equal to `height` for the `li`` element.

Git merge conflict? Just abort the whole process.

Similarly discard all changes on a diverged local branch, two ways. First method is to my liking.
# Method 1
$ git branch -D phobos
$ git checkout --track -b phobos origin/phobos

# Method 2
$ git checkout phobos
$ git reset --hard origin/phobos

Debugging Dockerfile. Something I learned this week but in a separate and longer post.

Starting a new software project but not sure about which technology stack to use? Read this slide as a guide.






Debugging Dockerfile

While building Docker image through Dockerfile, I've encountered an error in one of the build step, shown below. It seemed that that one of the Perl's module failed to install due to some unknown reason.
$ docker build -t ang:dist-zilla .

Sending build context to Docker daemon 3.072 kB
Step 1 : FROM perl:latest
 ---> a9d757d1a33b
Step 2 : RUN cpanm install Term::ReadKey
 ---> Running in 78760f841b26
 ---> Working on install
......
Building and testing TermReadKey-2.33 ... ! Installing Term::ReadKey failed. 
See /root/.cpanm/work/1473691870.7/build.log for details. Retry with --force to force install it.
......

Since there is an error, the particular layer of image was not created. Hence, there is no way for me to debug and trace the error. Fortunately, you can force it through the build by marking it to be a success build. The changes is minor, you just need to append an OR condition (also known as Short-circuit evaluation) as shown.
RUN cpanm install Term::ReadKey || true

Rebuild the image. Regardless the error, we have successfully create a layer.
$ docker build -t ang:dist-zilla .
......
Step 2 : RUN cpanm install Term::ReadKey || true
 ---> Running in f3fa83db0ae9
......
Building and testing TermReadKey-2.33 ... ! 
Installing Term::ReadKey failed. See /root/.cpanm/work/1473691397.7/build.log for details. 
Retry with --force to force install it.
FAIL
1 distribution installed
 ---> 62e2b1bd2c61
Removing intermediate container f3fa83db0ae9
Successfully built 62e2b1bd2c61

Log in to the particular snapshot or layer before we troubleshoot the issue.
$ docker run --rm -it 62e2b1bd2c61 bash -il
root@3e47eeff4328:~#

Checking the build log.
root@3e47eeff4328:~# tail .cpanm/work/1473691397.7/build.log -n 25
  
chmod 755 blib/arch/auto/Term/ReadKey/ReadKey.so
"/usr/local/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- 
ReadKey.bs blib/arch/auto/Term/ReadKey/ReadKey.bs 644
Running Mkbootstrap for Term::ReadKey ()
chmod 644 "ReadKey.bs"
PERL_DL_NONLAZY=1 "/usr/local/bin/perl" 
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" 
"undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/01_basic.t ............... ok
# Looks like you planned 7 tests but ran 1.
t/02_terminal_functions.t .. 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 6/7 subtests 
        (less 1 skipped subtest: 0 okay)

Test Summary Report
-------------------
t/02_terminal_functions.t (Wstat: 65280 Tests: 1 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 7 tests but ran 1.
Files=2, Tests=2,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.04 cusr  0.00 csys =  0.06 CPU)
Result: FAIL
Failed 1/2 test programs. 0/2 subtests failed.
Makefile:1029: recipe for target 'test_dynamic' failed
make: *** [test_dynamic] Error 255
-> FAIL Installing Term::ReadKey failed. 
See /root/.cpanm/work/1473691397.7/build.log for details. Retry with --force to force install it.
1 distribution installed

Googling result indicates that Term::ReadKey module have issue with one of the unit test where interactive shell is not found. Building Docker image does not needs interactive shell available.

The workaround is to downgrade and install the previous working version.
RUN cpanm install Term::ReadKey@2.32

This Week I Learned - 2016 Week 37

Last week post or the whole series.

As we're moving to the end of the third quarter of the year, more things pop up for me to follow up. Interestingly but not surprisingly, life is as monotonous as ever. Yes, it can be routinely, but that probably the only way, through sheer discipline, to follow through your plans.

The components for setting up my homelab using AMD 5350 have been bought and set up accordingly. The only remaining tasks is to install the necessary OS and configuration. More writeups on this in coming future.

As usual, something I learned this week.

Looking into Makefile, specifically GNU MakeExtracting parameters from target? Yes, is doable but it's not pretty. See code below. If your target is not an actual physical file, make it a '.PHONY' target instead. Otherwise you will encounter "No rule to make target" error. Next, we will need to 'eval' when extracting the assigning the parameters passed, otherwise the 'PARAMS' assignment will be executed as command.
.PHONY: action

action:
    $(eval PARAMS := $(filter-out $@,$(MAKECMDGOALS)))
    @echo $(PARAMS)

Interestingly, there are four ways for variable assignment in Make. The 'Set If Absent' way of variable declaration and initialization is quite handly. Funny though, Perl, which is known for its brevity, does not have such language construct.
# Lazy Set. Value is expanded and set when used.
FOOBAR = x

# Immediate Set. Value is set when declared.
FOOBAR := x

# Set If Absent
FOOBAR ?= x

# Append
FOOBAR += x

Write it down, make it happens. Never underestimate the power of writing. Sometimes, the pen is mightier than the sword.

'git commit --allow-empty'. My goodness! I'm not aware of this option exists in Git. How many times I've adjusted a space just to create and make an empty dummy commit. While we on Git, if you seems to "misuse" it somehow, there are many ways to recover back.

Web development is a layer of layer of layer of abstraction hacks? I firmly believe. It's messy, plagues with multiple choices, and feels like wild wild west. HN user, meredydd mentioned that modern web application today consists of five programming languages and three frameworks. Interestingly, I never realize there are so many choices. Maybe future Javascript, ES2016 can reduce that paradox of choices by standardizing on using the same language for frontend and backend, as in isomorphic Javascript? But that also raises another interesting question. Is web development a constant rewrite of existing application to newer technologies?

This Week I Learned - 2016 Week 36

Last week post or you might want to check out the whole series.

Some findings around the Internet.

XKCD-style graph using Matplotlib? In Ubuntu, you'll need to install these fonts to get the closest possible rendering.
$ sudo apt-get install ttf-mscorefonts-installer fonts-humor-sans
$ rm -rf ~/.cache/matplotlib/fontList.cache

Using Matplotlib without X-server? Switch to Agg backend. Useful when you're rendering image through Docker container.
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt

Sigh. Unresolved ImageMagick bug in most recently releases including the LTS, which text conversion still causing core dump. Switching to GraphicsMagick, a fork of ImageMagick did not resolve the issue as command line options have both diverted. My research made me aware that both tools were being used to massively batch process images in a very large scale.

Sometimes, the default Vim configurations/features is good enough without installing buggy extensions. We're relying too much on the external plugins without utilizing the fullest features of Vim itself.

Old time PHP developer switching to Perl? You should read this Reddit's post. The advice given was spot on and correlates with my own personal experience. Nothing against PHP, but in our journey to become a better developer, you'll need to expose yourself to other programming languages and environments. Otherwise you'll end up like those developer who claimed to be a ten-plus years developer but actually doing the same development development work for a year but repeated ten times. I will write another blog post on this in coming future.

"To finish projects on time, start every single step as late as possible" via HN. Full text of all the twitter posts. Catching and provocative statement coming from Tiago Forte, a productivity consultant. Despite the click bait title, HN user, bmh100 interpret his message correctly. Keywords here is "critical path". In other words, is Critical Chain Project Management. Sometimes I wonder is procrastination due to lack of awareness of a task? Or to rephrase it, procrastination is a mindfulness problem? Without awareness, there is no estimation and prioritization, hence the task will be postponed repeatedly or not completed within the time frame?

This Week I Learned - 2016 Week 35

Last week post or you might want to check out the whole series. As usual, some findings around the Internet.

September. It's almost at the end of the third quarter of the year. What have you done for the past eight months? No energy to do anything else after work? Switch. Do important things before work instead. This means you'll need to be a morning person and change your sleeping habits. Let's see how this goes. One small thing at a time but in a habitual and persistent way.

Everyone should be inspired by Norman Borlaug. We need more people like him instead of psychopath who wants to see the world burn.

How to make a copy another hash in Perl? Easy, for shallow copy, just use syntax below. Or simply use the Clone module if you have deep nested hash.
my %copy = %$hash;

Two ways of using grep in Perl, which I always confuse. First, by expression and secondly, by code block. Examples as shown.
# grep (EXPRESSION, @ARRAY)
if (grep /a/, @cities) {..}

# grep BLOCK @ARRAY
if (grep { length($_) == 6 } @cities) {..}

The strange case of Perl's DBI returning 0e0 (considered as true zero) if zero rows were affected.
sub do {
    my($dbh, $statement, $attr, @bind_values) = @_;
    my $sth = $dbh->prepare($statement, $attr) or return undef;
    $sth->execute(@bind_values) or return undef;
    my $rows = $sth->rows;
    ($rows == 0) ? "0E0" : $rows; # always return true if no error
}

"MySQL error 1449: The user specified as a definer does not exist." How annoying and rather inconvenient for me.

What are Git's caret (^) and tilde (~)?

This Week I Learned - 2016 Week 34

Last week post or you might want to check out the whole series.

As usual, some findings around the Internet.

Using Sublime Text? There is a set of configurations which are useful for you.

Byobu seems to have changed a lot since I last used it. It now supports Tmux.

Supernatural The Animation? Didn't realize that there is an anime series based on the TV series.

This Week I Learned - 2016 Week 33

Last week post or you might want to check out the whole series.

Travelling. Hence, the delay in weekly post.

The Hiperstation. Probably the cleanest and minimalist workstation station with a touch of retro feels, the VT220 terminal. Maybe I should readjust my battlestation again?


While we're on the discussion on retro hardware, how about reproducing the nostalgia bucklespring keyboard sound of IBM Model-M keyboard.

Introduction to Japanese, via HN. Comprehensive guide the Japanese language and writing system.

This Week I Learned - 2016 Week 32

Last week post or you might want to check out the whole series.

Slow week, nothing much being jot down for this week. In my process of becoming more minimalist, I've reduced a lot of "noises" in my life. This shall continue until I can reach the bare minimum necessity in my life, although this will be a life long ongoing journey.

Always use a higher-precedence operator like '||', '&&', or '^' in your return statement. Otherwise, the other expressions will be skipped. There is a Perl module, ProhibitReturnOr that does this checking.

Don't use OFFSET in your pagination SQL query but using keyset pagination. The basic concept is shown in the SQL query below. Never occurs to me to use WHERE instead of OFFSET. As I usually said, it's always nice to get more exposure with other DBMS instead of just MySQL. Further discussion via HN.
SELECT ...
  FROM ...
 WHERE ...
   AND id < ?last_seen_id
 ORDER BY id DESC
 FETCH FIRST 10 ROWS ONLY



This Week I Learned - 2016 Week 31

Looking back to the last week post or you might want to check out the whole series.

Simulate different network condition, especially when testing file uploading and downloading through Google Chrome's DevTools using network throttling. There are two ways to set the settings, see screenshot below. While I'm not a front-end guy, but Chrome DevTools has significant improve the front-end development process. Off course, there is also debugging method on setting breakpoints and detect changes as well as using it as IDE. Lots to learn!


Buying audio and video devices? Nothing beats these written guides by zeos.

Sublime enhanced. Extra set of plugins for Sublime editor.



This Week I Learned - 2016 Week 30

Looking back to the last week post or you might want to check out the whole series.

How to sort a hash in Perl. However, that is only for one column or key. How about multiple keys or columns? Similar, just with additional criteria.

Why SSH takes a long time to connect? Just update the SSH daemon configuration file.
$ vi /etc/ssh/sshd_config
GSSAPIAuthentication no
UseDNS no

DebOps. A great showcase on using Ansible. Unfortunately, it still did not solve one of the main issue server provision on multiple distros as this only works on Debian or Debian-based distros. Meanwhile, there exists another tool, Molecule, to help testing Ansible under different environments.

Speeding up APT? Use Apt-fast, which is a wrapper for Apt which use different downloader which support parallel downloads.

Good introduction to Vue.js. Note that this is for version 1.0 but the concept still relevant.

How to implement infinite scrolling. Surprisingly the concept is very simple.

Probably the best REPL for Perl, Reply.

This Week I Learned - 2016 Week 29

Looking back to the last week post or you might want to check out the whole series.

One of the issue when using Perl for beginner is to understand and differentiate the usage of referencing and dereferencing used by different data types, especially array or hash. Inspired by this site, the table below (generated by tablesgenerator) shows different way to initiate, reference, dereference, and accessing these data types.


$scalar @list %hash FILE
Instantiating it $scalar = "a"; @list = ("a", "b"); %list = ("a" => "b"); -
Instantiating a reference to it $ref = \"a"; $ref = ["a", "b"]; $ref = {"a" => "b"}; -
Referencing it $ref = \$scalar $ref = \@list $ref = \%list $ref = \*FILE
Dereferencing it $$ref or ${$ref} @{$ref} %{$ref} {$ref} or <$ref>
Accessing an element - ${$ref}[1] or $ref->[1] $ref{a} or $ref->{a} -

Git, the rebasing workflow. Better still, understand the how Git works and learn some Git branching, visually.
$ git fetch
$ git rebase origin/master
$ git checkout master
$ git merge insert_awesome_topical_branch_name_here
$ git push origen/master
$ git branch -d insert_awesome_topical_branch_name_here

Kimchi, web interface to KVM. Didn't realize this exists.

Good HN discussion on creating productive habits. Some of the interesting notes are:
  • Appreciate and be grateful with what you have and stop caring for things that make you unhappy.
  • Use a chess clock to remind you of the time you've spend on doing something else.
  • Set a expected time on how to do a task as works fill the time you've set to do it.
  • Meditation helps with focus.
  • Just get start, leave no excuses of not to start. You finish a task by starting.
  • Habit formation, though daily small steps (in other words, easy) until it's ingrained in you. Which is obtained through persistence and discipline. Remember habit > inspiration.
  • Complete something early in the morning. Something simple. Apply that mindset to your whole day. Also known as pre-game routine.
  • Eliminating the inessential. Minimized and focus on important things. Less is more.

Similar, another HN discussion on can't concentrate on tasks?

  • Low dopamine perhaps? Sleep and eat well. Take care of your mental healthiness as well.
  • Better dateline management.
  • Morning is the best time to work due to glucose level is high when you start your day.
  • You will need a deep work environment.