UTF-8 end to end : Eclipse ->XText -> Java -> MySQL -> PHP -> HTML

Passing UTF-8 characters through a chain of languages and processing elements can be a bit nightmarish, with each component having its own quirky configurations about how to deal with UTF-8.

In one of my pet projects, I have a long text processing chain:

1. User enters text in Eclipse
2. Text is parsed by XText parser
3. The parsed text is processed by a Java batch application
4. Java batch application stores the processed text in MySQL
5. PHP reads the data in MySQL and sends it to browser either as JSON or HTML
6. Text shows up embedded in HTML/JSON on the browser

To ensure the UTF-8 data flows through the chain properly, the following needs to be done:

1. Ensure the text file encoding in Eclipse is set to UTF-8 (Window->Preferences->General->Workspace [Text file encoding])

2. Ensure XText stand alone parser is configured to respect UTF-8. Refer to this post http://sandeepdeb.blogspot.in/2015/04/ensuring-xtext-parsers-respect-utf-8.html

3. For Java to store UTF-8 in MySQL via JDBC, the JDBC url needs to mention the encoding. 

jdbc:mysql://localhost:3306/?useUnicode=yes&characterEncoding=UTF-8

1. PHP's default_encoding in /etc/php5/apache2/php.ini needs to be set to UTF-8 (by default it is commented)

2. The database connection charset in PHP needs to be explicitly set to UTF-8

$dbConn = mysqli_connect( DB_HOST, DB_USER, DB_PASSWORD, DB_SCHEMA ) ;

$dbConn->set_charset("utf8") ;

6. <meta charset="UTF-8"> needs to be set in the HTML header

When a remote headless server does not respond

My home server (tucked up cozily inside my false ceiling) has been running unattended for a few months now. It has been BIOS configured to auto boot post an power failure. However, it stopped responding tonight. What an absolute nightmare!

I had to bring down the brick and set up the whole shebang of monitors, keyboard, network cable around it just to 'see' why it had stopped responding.

Once the monitor came on, I realized that the server was hung on the GRUB screen waiting for user input. WTF! It should have auto timed out and taken the first option in the list, booting normally.

Upon digging a bit, realized that the default GRUB behavior on previous failed boots is to show the GRUB screen and wait for user input. Logical but not very productive for unattended servers. Luckily, Linux being linux, there is a simple option to change this behavior and let Ubuntu boot irrespective of prior failed boots.

One has to add the following variable to /etc/default/grub and reload grub

GRUB_RECORDFAIL_TIMEOUT=0

BTW, don't forget to run update-grub after editing /etc/default/grub!

XText hiccups on Luna SR2 and XText 2.7.3

It's not every day that you set up a new XText project. For me, I end up dabbling with my own DSL once in a few months and unfortunately most of the times the setup of the project is fraught with hiccups.

Today, I tried to set up a new XText project on Ubuntu 14.04 + Eclipse Luna SR2 (4.4.2) and XText 2.7.3. The first thing I encounter is a failure stating a failed plugin dependency on org.eclipse.equinox.common.

To fix this, I had to open up MANIFEST.MF and add the dependency in the Require-Bundle section:

Require-Bundle: org.eclipse.xtext;visibility:=reexport,
org.eclipse.xtext.xbase;resolution:=optional;visibility:=reexport,
org.eclipse.xtext.generator;resolution:=optional,
org.apache.commons.logging;bundle-version="1.0.4";resolution:=optional,
org.eclipse.emf.codegen.ecore;resolution:=optional,
org.eclipse.emf.mwe.utils;resolution:=optional,
org.eclipse.emf.mwe2.launch;resolution:=optional,
org.eclipse.xtext.util,
org.eclipse.emf.ecore,
org.eclipse.emf.common,
org.eclipse.xtext.xbase.lib,
org.antlr.runtime,
org.eclipse.xtext.common.types,
org.objectweb.asm;bundle-version="[5.0.1,6.0.0)";resolution:=optional,
org.eclipse.equinox.common;bundle-version="3.6.200"

This seems to solve the problem. You can also do it via the dependencies tab in the plugin editor (once you open plugin.xml with the plug-in manifest editor.)

Important thing to remember is not to add equinox.common as a project class path dependency. Remember that this XText project will run as a plugin in eclipse and it will try to resolve its dependencies via the MANIFEST.MF.

BTW, once I satisfied the equinox.common dependency, the MWE2 wanted to patch the version of ANTLR. This was driven through a console input request - so keep a lookout for it too.

---

Update - 14/May/2015 3:35 PM

The above was in the morning.. It turned out to be a day of scalp scratching stuggle with Eclipse and XText. Damn it, there is so much of backward and forward compatibility problems with XText and PDE!

It turned out that although the above got the MW2 compiler to start working and seemingly do everything right, the language would not open up the XText editor in eclipse. Bah! Struggled with it for nearly three hours and finally updated my XText plugin from 2.7.3 to 2.8.2.

2.8.2 works like a breeze. Seems like quite a few hiccups have been fixed on the way. Keeping my fingers crossed for a smooth sailing here on.

Karma and Jasmine

My angular code has started accumulating enough logic for me to take a pause and move my unit tests into Karma and Jasmine. Just installed karma, karma-jasmine, chrome launchers and the karma-cli. 

Saw the video of Karma and I am sold on it already.. just need to fire it up and migrate my test cases to Jasmine.

Codecogs Latexit for Blogger not working?

Recently I tried integrating online latex formatter into my blog. Codecogs latexit turned out to be a good option. Unfortunately the instructions given at http://www.codecogs.com/latex/integration/blogger/install.php doesn't seem to work. This is because the URL http://latex.codecogs.com/latexit.php?p&li&div returns a 404.

The following work around seems to work.

• Open the template in edit mode
• Add the following script at the end of head

<script type="text/javascript" src="http://latex.codecogs.com/latexit.js"></script>

Save the template. Any tag with a lang="latex" attribute will be rendered as a latex expression.

<div lang="latex">
\frac{1+sin(x)}{y}
</div>

\frac{1+sin(x)}{y}

Check out this link for more details:

http://www.codecogs.com/latex/integration/htmlequations.php

---

Update 14th May

I have moved to MathJax for browser based latex rendering. www.mathjax.org. This is simply beautiful and oh so immensely powerful. The best part is it has got extensions for mhchem LaTeX package. Expressing chemical formulas is such a pleasure with mhchem.(mhchem package)

Mathematics: Exploring the Parabola

When I was introducing algebraic functions to my twelve year old, I summoned the equation of parabola y = ax^2 + bx + c in the hope that it would bring in some visualization to the seemingly drab world of algebra. And so, following the traditional footsteps of teaching, we pegged the values of the constants and derived values of y by substituting values of x and tried plotting them.

It was all going good, till the wonderfully fertile twelve year old brain asked, "What happens to the parabola if a, b and c are not constants?"

Thought it was not very difficult to satiate the twelve year brain, the full fury of the question send me on a mathematical quest late in the night. The realization that in all my years of study, never did I have enough time and patience to 'play' with mathematics was beginning to hurt.

It has since been some months that I have been playing with mathematical modelling tools - especially Maple and Geometry Expressions. These tools have been instrumental in pivoting me beyond the drudgery of derivation to the realm of appreciation of the wonderful world of mathematics.

So, coming back to the question which set all these in motion. The layman answer is that variation of a, b and c moves the parabola (across the cartesian space) and changes its shape (curvature). A more specific answer would be following the loci of the vertex and understanding the changes in curvature.

Variation of 'c'

'c' being an independent variable, the impact on its variation is quite easy to comprehend. It translates the vertex of the parabola on a line parallel to the Y axis. The figure below shows the locus of the vertex as the dotted line.

Variation of 'a'

Variation of 'a' has more drastic effects. For one it moves the vertex along the line y = (b/2)x + c. Also being tied to the square of x, it changes the curvature of the parabola very drastically. I will come to the impact of curvature later in this post. For now, let's focus on the locus of the vertex.

Variation of 'b'

Variation of 'b' is what I would say 'interesting'! While it does change the curvature (though not as drastically as 'a'), it translates the vertex along an inverted parabola y = -x^2a + c. Note that the axis of this inverted parabola will always be the y axis (since b=0).

 

Now, coming to the curvature part - this needs some higher order mathematics. 

Radius of curvature of a curve y = f(x) is 

For the parabola the radius of curvature becomes

Differentiating the radius of curvature w.r.t. a and b separately, we get:

 

To see the effect of variation of a and b, we peg the values of x at 1. The graphs below show the effect. Clearly, the impact of varying a is lot greater than b as expected.