Wednesday, April 22, 2015

Pietà (Michelangelo)

My learning from the master continues. Few years back, I read through Irving Stone's book "The Agony and the Ecstasy", trying to understand a bit more of of man Michelangelo really way. It was a humbling read.

Pietà was one of his other sculptures which grew on me. Okay, I will confess, and I might be committing an act of blasphemy when I say that I don't admire Pietà because of its structural preciseness (like that of David). In fact if you study Pietà long enough, you will soon realize that Mary and Chist's bodies are disproportionate in size. 

What intrigued and fascinated me on end was two things - serenity of expression on Mary's face and the cloth drapes. In stone sculptures, sharp facial features are easy to carve but when it comes to subtle expressions - I can't even imagine the level of complexity involved: there are so very few variables to play with.

Anyway, it took me nearly a month of studying all about drapes even before I could lay down the structure of the sketch.

Drapes are essentially the way fabric folds on itself due to two basic forces, gravity and the contour of the structure the fabric is draped upon. There is another dimension to it - the qualities of the fabric itself. Silk would fold differently than linen. A starched linen would fold differently than a well worn one.

Below is my humble attempt to sketch a tiny bit of the masterpiece. It's again a 18" x 24" sketch done in charcoal pastels and negatively shaded with kneaded erasers. 

Tuesday, April 21, 2015

AngularJS: Executing JavaScript in ng-repeat templates/iterations

By design, ngRepeat does not execute any <script> tags in its template. Unfortunately, there are instances where this starts becoming limiting. I faced one such scenario recently.

In one of my pet projects, there is a dashboard which looks as below. Each row in the tree grid is rendered using ngRepeat directive (of course). The challenge is that each row also contains a painted HTML5 Canvas which renders a stacked graph of the numbers.

Since the Canvas can only be rendered by JavaScript, I will have to either set a time out function, hoping that ngRepeat render will finish before the timeout. This is not an elegant solution since I have no clue how deep or huge my dashboard tree is going to be. To add to the convolution, ngRepeat does not provide a call back hook for each iteration.

An elegant solution can be crafted by using custom directives and $emit, $on scope event propagation methods. Let me demonstrate using a simple example.

.
<!DOCTYPE html>
<html lang="en" ng-app="app">
<head>
    <script src="/lib-ext/angular/angular.min.js"></script>
    <script src="mycontroller.js"></script>
</head>
<body>

<div ng-controller="MyController">
    <div>
        <div ng-repeat="id in data track by $index" on-row-render>
            {{id}}
            <div id="div-{{$index}}"></div>
        </div>
    </div>
</div>
</body>
</html>
.

In the above HTML, I want the contents of the div inside the ng-repeat template to be populated by a JavaScript function which should be executed within the context of the ng-repeat iteration such that all the scope variables are accessible.

The following snippet shows my controller:

.
var app = angular.module( 'app', [] ) ;

app.controller( 'MyController', function( $scope ) {
    $scope.data = [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] ;

    $scope.$on( 'onRowRender', function( scope, index ){
           document.getElementById( "div-" + index )
                .innerHTML = "DIV-" + index ;
    });
} ) ;

app.directive('onRowRender', function() {
    return function( scope, element, attrs ) {
        setTimeout( function(){
            scope.$emit( 'onRowRender', scope.$index );
        }, 1);
    };
} ) ;
.

An extremely simple model, but helps explain the concept behind the solution. I have created a custom directive 'onRowRender', which I embed as an attribute in my ng-repeat template. This directive will get called for each iteration and passed the iteration scope.

I can choose to customize my interception point by reading the scope variables $index, $first, $last, $even, $odd etc. In this case, I choose to intercept every row render.

The directive, creates an event and propagates it up the event handling tree. Note that I decouple the event propagation from the directive execution by using a timeout. This ensures that we don't contend with Angular for manipulating the DIV. If we don't decouple this, we will end up with our event handlers not finding some of the DOM elements which we had intended to be rendered in our ngRepeat template.

The last part, the onRowRender event is caught by our controller (event propagation starts at the same module level and moves up), which (simplistically) finds the div DOM element and sets some content inside it. It is here that I will call on the JavaScript method to render my custom stacked graph.

The resultant output.

Friday, April 17, 2015

Difficulty in using <Esc> in vi[m] ?

As one grows more and more comfortable with vim, the comfort of the keyboard center line gets more and more difficult to leave. So much so that moving your finger to press escape seems like too much movement to escape insert mode.

For quite some time, I was under the impression that it's just me (and psst.. maybe I have short fingers ;). Realized soon after that there are others who have faced this before.

We can use the :imap command in the ~/.vimrc file to remap the Esc key. Having experimented with quite a few combinations, I find 'jj' to be the most convenient substitute for <Esc> key.

" Mapping jj as a substitute for <Esc>
:imap jj <Esc>

This is straight forward. There are some excellent resources which give a more detailed insight:

http://vim.wikia.com/wiki/Avoid_the_escape_key

http://vim.wikia.com/wiki/Mapping_keys_in_Vim_-_Tutorial_%28Part_1%29

The interesting challenge comes when you also use another editor in conjunction with vi[m]. I use Sublime Text 3 for most of my LAMP stack development. I use Vintageous extensively to lay down the familiar cushion of vim within Sublime Text. Getting the above key remap to work with Vintageous turned out to be a small challenge. There are multiple suggestions available and not all of them work :(

The following keymap entry works for me:

    { "keys": ["j", "j"],
      "command": "_enter_normal_mode",
      "args": {"mode": "mode_insert"},
      "context": [{"key": "vi_insert_mode_aware"}]
    }

Source: https://github.com/guillermooo/Vintageous/wiki/Using-jj-instead-of-Esc

A very interesting piece of history I picked up from the vim wikia site

The Vi editor was originally written on an ADM-3A terminal, which had the Escape key positioned where the Tab key occurs on most modern keyboards. In the modern keyboards pressing [Esc] generally requires stretching to the top of the keyboard and since [Esc] is possible the most important key in vim, it makes sense to remap it to something more convenient.

Monday, April 13, 2015

Automated home irrigation system

It's vacation time and among other things, the issue of watering the plants when we are basking on the beach remains a perpetual concern. I stay in an apartment and have a tiny garden in my balcony, which requires no more than 10-15 liters of water a day. I wanted an automated watering system which should satisfy the following three primary concerns.

  • Upper bound on water consumption - I can't afford to have the main line water supply run through out the day, even at a very low throttle. The risk of flooding is too great and given that I stay in a society, this is completely unacceptable.
  • Automated system - The system should be able to run unattended for 5-10 days, with predictable water discharge.
  • Cheap - Total cost of building the system should be very low. Ideally I should be able to assemble the system with household scrap.

Having evaluated quite a few options on the Internet, I came to the harsh realization that none of them can be self sustaining for 5-10 days without an external inflow of water. Gravity drip was the only viable option.

My first attempt was to create a simple gravity drip as below. I made it using thin leveling tubes and straws tipped with syringe needles as emitters.

While in theory this looks great, it suffered from one fundamental problem - throttling. The water just gushes out and runs off fast. I could reduce the outflow aperture by using a thinner outflow pipe, but then it does not scale for a length of 10-15 meters with 20 emitters. I needed to control the outflow and the obvious solution was to put a throttling cock. What better component than an IV drip throttle ;)

With this I could tune and control the outflow - no issues. It ran successfully for a few hours and then showed a strange behavior which took me a full two days to figure out. The issue was that once in a while certain random emitters would dry up even when others kept their drip on. Very strange indeed. After two days of scalp scratching, I figured out that if the cumulative outflow from a subset of emitters equaled the outflow at throttle, it would create a low pressure in the delivery tube. This low pressure tries to suck in water/air from other emitters. Phew!

The solution although obvious was not easy to fabricate. The solution was the put a anti reverse flow valve at the emitters. Now, how do I create an anti reverse flow fluid valve with household scrap?

The above design worked flawlessly. It operates on the principle of partial vacuum. Water enters an air tight container, pre-filled more than half its volume. The water is drained out of this by a needle tipped outflow tube which taps into the lower portion. The theory being that it is very difficult for any low pressure in the tube to lift out all the water and in case of higher outflow, a partial vacuum is created which sucks in more water from the reservoir. The only care I had to take was coat all joining surfaces with araldite to make it water proof.

Voila! all things put together, I had an indefinitely self-sustaining gravity drip irrigation system which would operate at extremely low pressure thresholds.  

If the reservoir capacity falls short of sustaining the supply for more than 5 days, I could attach a sink valve to the reservoir (same principle as that of water closet) and my reservoir stays topped up indefinitely.

 

Algorithms: Indiana Jones's dilemma - Knapsack problem

The best articulation of the Knapsack problem that I have encountered is that in the context of Indiana Jones.

Assume Indiana Jones is trying to escape a collapsing treasure cave. The cave is littered with many precious items, golden goblets, statues, coins .. you name it. Indiana has a knapsack to collect some of the treasure items and tow it along with him. The challenge is that his knapsack can only carry so much load and Indiana wants to make the best choices so that he carries the maximum value of artifacts in his knapsack.

More generally, the above problem manifests itself in many day to day challenges. Some of them being:

  • Assuming we have certain time at hand and a list of tasks in our todo list. Each task taking a forecasted amount of time. What is the subset of tasks that need to be chosen so that we make the best use of the time at hand.
  • Given a chess board at a certain stage, which is the next best move
  • http://people.cs.clemson.edu/~bcdean/dp_practice/ contains a list of problems of similar nature

To the uninitiated, at the face of it the problem will sound simple. To the moderately initiated, recursive backtracking would sound ideal to solve these problem. It is only the initiated who will understand the time complexity of tree traversal - making simple recursive, full tree traversal algorithms difficult to use even at data sets exceeding 10 elements.

A good treatise on the Knapsack problem can be found here : http://en.wikipedia.org/wiki/Knapsack_problem

If you want a (very very) small library, encapsulating the algorithm for use in your project, you can check out the code at my GitHub site

https://github.com/deb-sandeep/BranchAndBound

If you need any help with the code, please drop me a note.. 

 

Friday, April 10, 2015

David

I am a huge, huge fan of Michelangelo. So much so that I have spent hours studying his sculptures. The focus on core structure that he brings into each of his sculptures is phenomenal. For anyone who approaches art from a structural perspective, Michelangelo is the best teacher to learn from. 

Following sketch was one of my humble attempts to learn from David. It is one of my life ambitions to see both David and Pieta in person - maybe some day :)

Done in charcoal pastels, this is a fullscape sketch measuring 24" x 18".

By the time I finished the face, fatigue had set in and you can see the lack of justice to the Sternomastoids reflecting that. Just in case you are interested in human structural anatomy and approach sketching figures more from a structural angle, I would highly recommend reading through this chapter:

http://philschatz.com/anatomy-book/contents/m46492.html

One of my other sculptor idols is Philippe Faraut (https://philippefaraut.com/). It would be such an absolute delight to spend hours with some of his sculptures and capture them on paper.. some day... I will.

 

Thursday, April 9, 2015

Ensuring xtext parsers respect UTF-8 characters in language files

Hope this piece of knowledge saves someone a nightlong hair-plucking ordeal.

Assume you are trying to parse a XText domain specific language file and transforming it into say an HTML. Interestingly the source files contain UTF-8 text and you realize to your absolute horror that the generated artifacts are all garbled.

The remedy is to set up a custom encoding provider!

Assume your language is called 'LangX' and you have a corresponding grammar file called 'LangX.xtext'. Once you run the MWE2 workflow, it will generate among other things a 'LangXRuntimeModule.java' in the same directory where you have kept your xtext file.

Go ahead and create a file called MyEncodingProvider.java in the same directory:

package langx ;

import org.eclipse.emf.common.util.URI ;
import org.eclipse.xtext.parser.IEncodingProvider ;

public class MyEncodingProvider implements IEncodingProvider {

    public String getEncoding( URI uri ) {
        return "UTF-8" ;
    }
}

Now open up your LangXRuntimeModule.java and add the new encoding provider:

package langx ;

import org.eclipse.xtext.parser.IEncodingProvider ;
import org.eclipse.xtext.service.DispatchingProvider ;

import com.google.inject.Binder ;

public class LangXRuntimeModule extends langx.AbstractLangXRuntimeModule {

    public void configureRuntimeEncodingProvider(Binder binder) {
        binder.bind(IEncodingProvider.class)
        .annotatedWith(DispatchingProvider.Runtime.class)
        .to(MyEncodingProvider.class);
    }
}

Okay, you are all set!

SimplePivot : New project on GitHub

Uploaded a tiny but functional pivoting library in JavaScript

https://github.com/deb-sandeep/SimplePivot

<script>
function test() {

  // Define the column names by which we will refer to the pivot groupings
  var srcColNames = [ "Date", "Subject", "Lesson", "Time" ] ;

  // The underlying data of the pivot representation
  var pivotData = [
     [ "10-26", "History",   "Mughals", 300 ],
     [ "10-26", "History",   "Islam",   200 ],
     [ "10-27", "Geography", "Climate", 150 ],
     [ "10-27", "History",   "Islam",   250 ]
  ] ;

  // Initialize the pivot table by setting the column names and corresponding
  // data set
  var pivotTable = new PivotTable() ;
  pivotTable.setPivotData( srcColNames, pivotData ) ;

  // Initialize the pivot table by providing
  // - The row groupings (in this case, group by Subject, then by Lesson
  // - The column group
  // - The value which needs to be aggregated
  pivotTable.initializePivotTable( [ "Subject", "Lesson" ], "Date", "Time" ) ;

  // Render the pivot table on an existing div
  pivotTable.renderPivotTable( "pivot_table_div", "Time spent per subject" ) ;
}
</script>

 

URL rewriting not working in Apache 2.4.7 + Ubuntu 14.04

Have been struggling for the last two hours to get the URL rewriting working on my Apache 2.4.7 + Ubuntu 14.04 installation. Went through all the checklists and guidelines provided on the subject.. most common ones being enabling MultiView and AllowOverride all. Nothing worked.

Finally, figured out that I needed to add the following mime type to /etc/apache2/mods-enabled/mime.conf

AddType application/x-httpd-php .php

Second big learning was getting an insight into the URL matching logs. The old mechanism of enabling RewriteLog has been removed in 2.4 onwards. To see the logs, you will have to add the following in /etc/apache2/sites-enabled/000-default.conf on the /var/www module.

LogLevel alert rewrite:trace3

Once the above is done, you can tail and grep the error log on 'rewrite' to see the logs. Here is a step log of how Apache seems to be working through my rewrite rule. The following is my rewrite intention 

localhost/api/Chapter/3?priority=3  => localhost/api_gateway.php?path=Chapter/3&priority=3

  • [perdir /var/www/] strip per-dir prefix: /var/www/api/Chapter/3 -> api/Chapter/3
  • [perdir /var/www/] applying pattern '^api/(.*)$' to uri 'api/Chapter/3'
  • [perdir /var/www/] rewrite 'api/Chapter/3' -> 'api_gateway.php?path=Chapter/3'
  • split uri=api_gateway.php?path=Chapter/3 -> uri=api_gateway.php, args=path=Chapter/3&priority=3
  • [perdir /var/www/] add per-dir prefix: api_gateway.php -> /var/www/api_gateway.php
  • [perdir /var/www/] strip document_root prefix: /var/www/api_gateway.php -> /api_gateway.php
  • [perdir /var/www/] internal redirect with /api_gateway.php [INTERNAL REDIRECT]
  • [perdir /var/www/] strip per-dir prefix: /var/www/api_gateway.php -> api_gateway.php
  • [perdir /var/www/] applying pattern '^api/(.*)$' to uri 'api_gateway.php'
  • [perdir /var/www/] pass through /var/www/api_gateway.php
  • [perdir /var/www/] add path info postfix: /var/www/api -> /var/www/api/Chapter/3
  • [perdir /var/www/] strip per-dir prefix: /var/www/api/Chapter/3 -> api/Chapter/3
  • [perdir /var/www/] applying pattern '^api/(.*)$' to uri 'api/Chapter/3'
  • [perdir /var/www/] rewrite 'api/Chapter/3' -> 'api_gateway.php?path=Chapter/3'
  • split uri=api_gateway.php?path=Chapter/3 -> uri=api_gateway.php, args=path=Chapter/3&priority=3
  • [perdir /var/www/] add per-dir prefix: api_gateway.php -> /var/www/api_gateway.php

As you can see, my rewrite rule turns out to be:

RewriteRule ^api/(.*)$ api_gateway.php?path=$1 [QSA,NC,L]

Here, QSA is Query String Append which takes care of appending priority=3 to the target query. NC stands for Case Insensitive and L implies if this rule matches don't try any other rule.

Oh btw, if you are on the path of debugging your rewrite rule by switching the trace on - you are heading for scroll blindness. Use the following command to filter out just the part of log relevant to mod_rewrite.

$ tail -f /var/log/apache2/error.log | grep 'mod_rewrite' | awk '{for(i=1;i<=15;i++){$i=""}; print $0}'

 

Wednesday, April 8, 2015

Bugger: loosing files in /tmp

I had completely forgotten that /tmp is a RAM disk by default in Raspberry Pi and any files there will be lost on reboot. One of my scripts was saving some temporary (temporal) files in /tmp and once in a while, the data would vanish.

Since my Raspberry runs unattended, it just boots itself whenever there is a power loss. It took me two puzzled weeks to figure out the bug. 

Solution: If you need to persist temporary files across reboots use /var/tmp instead. Extending the thought, if you need the files across reboot, are they really temporary ;)

Tuesday, April 7, 2015

Flame: Multi-line select using Shift+Alt+Up/Down does not work in SublimeText 3 @Ubuntu 14.04

Tremendously frustrating when one of the most useful shortcuts of your IDE stops working because the OS is more hungry for the hotkey. Multi select, which is <Ctrl>+<Alt>+Up/Down in windows anyway doesn't work in Ubuntu because the hotkey is used to switch workspaces. Sublime has recognized it and moved the multi selection hot key to <Shift>+<Alt>+Up/Down.

Unfortunately, I could not get it to work. Some process is gobbling up the keystrokes. Oh, yes.. I did try all the suggestions including unity-tweak-tool, compiz-configuration-manager, language bar settings, default keyboard mappings... but to no avail.

Finally, I had to shift the Sublime configuration to use <Super>+<Shift>+Up/Down instead. !!flame!!

You can do this by defining user specific key bindings in Preferences:

 

{ "keys": ["super+shift+up"], "command": "select_lines", "args": {"forward": false} },

{ "keys": ["super+shift+down"], "command": "select_lines", "args": {"forward": true} },

 

Monday, April 6, 2015

Linux tip: Synchronized backup

I use rsync a lot for backing up my important folders. Unfortunately, rsync gets pretty verbose when it comes to deeply nested directories with lots of files - resulting in the update information getting lost in the log flood.

The following function works well for me:

function backup {

    pairName=$1
    srcFolder=$2
    destFolder=$3

    echo -e "\n\nRunning backup for $1"
    echo ===============================================
    echo Source Folder = $srcFolder
    echo destination folder = $destFolder

    rsync $dryRun -rilptgo --delete --stats \
              --filter='.<path to your rsync filter file>' \
              "$srcFolder" "$destFolder" | grep "^[^\.]"
}

This gives a nice readable output as below:

Running backup for eBooks
======================================================================
Source Folder = /media/sandeep/D/eBooks/
destination folder = /media/sandeep/Professional [Transcend 160]/SynchedBackup/eBooks
*deleting   Entertainment/Handwriting/calligraphy_101.pdf
>f+++++++++ Entertainment/Handwriting/[Read] calligraphy_101.pdf
Number of files: 63,394 (reg: 57,195, dir: 6,199)
Number of created files: 1 (reg: 1)
Number of deleted files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 41,114,388,469 bytes
Total transferred file size: 42,973,354 bytes
Literal data: 42,973,354 bytes
Matched data: 0 bytes
File list size: 327,652
File list generation time: 0.003 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 44,923,979
Total bytes received: 196,790
sent 44,923,979 bytes  received 196,790 bytes  527,728.29 bytes/sec
total size is 41,114,388,469  speedup is 911.21

Sunday, April 5, 2015

Noguchi's News

I am not a great fan of Isamu Noguchi per se. That said, one (and only one) of his sculptures has truly captivated my visual senses. It's the 'News' and it currently adorns the entrance to the Rockefeller Plaza. From their website:

"Soaring above the entrance to 50 Rockefeller Plaza, this dynamic plaque symbolizes the business of the building’s former tenant, the Associated Press. One of the major Art Deco works in the Center, it depicts five journalists focused on getting a scoop. AP’s worldwide network is symbolized by diagonal radiating lines extending across the plaque. Intense angles and smooth planes create the fast-paced rhythm and energy of a newsroom. News is the first heroic-sized sculpture ever cast in stainless steel and the only time Noguchi employed stainless steel as an artistic medium."

Below is my humble attempt at reproducing the masterpiece in two dimensions using charcoal pastels and kneaded eraser. What captivated me the most was the balance of mass and the play of light and shadows across the sculpture. Its a 18" x 24" sketch and proudly adorns a specially created wall at my home. 

 

 

My home servers

Central to the home computing environment are the servers. Unfortunately selecting servers for home is a daunting task when we take into account space, power consumption and connectivity (not to mention price) requirements.

I run two "perpetually on and connected" servers at home. One for heavy duty processing (e.g. running BitCoin node) and other home applications. The other for low key tasks like updating dynamic DNS entry, bandwidth monitoring, hosting a Git server etc.

For my tiny server, I have been running an instance of Raspberry Pi (model B) for nearly two years now. My feedback - it's amazing! I used to use this as my heavy duty server (no, I didn't run the BitCoin node on this) before I got the second server. Pi runs a full stack of softwares, including Apache, MySQL, PHP5, Git server, SFTP, SSH....  

Now, while Pi is extremely good, it soon hits its ceiling with a hard burnt 512 MB RAM and SD card for disk. The need for more muscle power becomes a need after a few months.

I evaluated converting my old laptop, buying a throwaway desktop from the flea market and running lubuntu and many other options. Unfortunately, none of them worked out because of space and connectivity challenges. Finally, I came across Dell OptiPlex 3020 micro - it's tiny and full of muscle. How tiny? well, I have tucked it inside my false ceiling ;) that tiny! 

My optiplex

And muscle power, well it brags a full configuration, including:

  • Intel® 4th generation Core™ i5 Quad Core
  • Intel® H81 Chipset
  • Integrated Intel® HD Graphics 4600
  • Integrated ethernet card
  • I TB HDD
  • 4 GB RAM 
  • (and the list goes on)

I have re-imaged it to run Ubuntu 14.04. 

 

Saturday, April 4, 2015

Switching between PHP applications @localhost

If you are working on multiple PHP apps, switching between them is a pain. I have found the following method to be quite convenient.

  • Manage multiple projects in their separate directories, for example ~/projects/source/AppA and ~/projects/source/AppB
  • Delete /var/www and make it a soft link pointing to the appropriate application you are working on
  • Maintain a simple script 'switchApp' which will delete the soft link and recreate it by making it point to the appropriate project

 

 

BSNL Data Usage Statistics

If you use BSNL broadband, getting to know your data usage pattern can be handy in predicting how fast you are burning your bandwidth. The following script will help provide a visual on your burn rate.
I run this on my 'perpetually connected' Raspberry Pi, via an hourly cron trigger.
#!/bin/bash

curl -sqk https://172.30.67.40/bsnlfup/usage.php \
     | grep '\(GB\|MB\|KB\)</td>' \
     | xargs echo `date` \
     | sed 's/<[^>]*>//g' >> /var/tmp/dataleft.log

awk '{
    dataleft = 0;
    if( NF == 8 ) {
        dataleft = $7 ;
        if( $8 == "GB" ) {
            dataleft = dataleft * 1024 ;
        }
        else if( $8 == "KB" ) {
            dataleft = dataleft / 1024 ;
        };
        printf( "%d %0.f\n", NR, dataleft ) ;
    };
}' /var/tmp/dataleft.log > /var/tmp/plotdata.txt

dataleft=`awk 'END{print $2}' < /var/tmp/plotdata.txt`

graph -T png -L "$dataleft MB left" -N x </var/tmp/plotdata.txt >/var/www/dataleft.png