internationalisat.ian

A while ago I wrote a small add-on for CodeIgniter which provides Roles based URL level Authorization. I named it Kibishii Security, cos “kibishii” means “strict” in Japanese.

It uses CI hooks to provide authorization at the URL-level, but it provides no authentication mechanism. You need to use it in conjunction with some kind of authentication system (at the time I was using Tank Auth) which I liked back then.
But Kibishii was designed to be roles-based, that is it enforces “only people with a certain role can access this URL”. However Tank Auth provides no concept of groups or roles, so to use it with Kibishii I had to invent this user-role relationship by implementing it myself, or by hard-coding it into my config (both less than ideal).

So today I will run you through another popular authentication system for CI: Ion Auth. It provides “groups” which we can use in Kibishii as our “roles”.

This guide will start a new CI project from scratch, but theres no reason it wouldnt work on an existing system. If you already have Ion Auth you can skip the next set up steps.

Set up Ion Auth
Follow these steps (should be pretty straight forward)

1. First up, download CI from here: http://codeigniter.com/,
2. then download Ion Auth from here: https://github.com/benedmunds/CodeIgniter-Ion-Auth
3. Copy the Ion Auth files into the appropriate folders in your new CI installation.
4. Then run the SQL script that comes with Ion Auth.
5. edit your /application/config/database.php to point to your database.
6. open up /application/config/config.php and change $config['encryption_key'] to be not empty, something like: $config['encryption_key'] = ‘mySecretKey’;

Now goto http://localhost/index.php/auth/login
If you get a login page, then you have installed Ion Auth.

Using Kibishii

1. Enable hooks by editing your application/config/config.php and set $config['enable_hooks'] = TRUE;
2. Add kibishii as a hook in application/config/hooks.php by adding this:

   	$hook['post_controller_constructor'] = array(
   		'class' => 'kibishii_hook',
   		'function' => 'check_permissions',
   		'filename' => 'kibishii_hook.php',
   		'filepath' => 'hooks'
   	);
   

3. download kibishii
4. copy the following files into the corresponding folders in your CI installation.
   • hooks/kibishii_hook.php – the main kibishii code
   • config/kibishii.php – config for kibishii
   • libraries/kibishii_ion.php – an adapter required for Ion Auth groups (it seems ion auth doesnt store its groups in the session anymore, so i wrote this to get the groups from model)
5. open up application/config/kibishii.php and set the acl rules like this:

   $config['kibishii_acl']	= array(
   	'/^auth$/' => 'admin',
   	'/^auth\/index/' => 'admin',
   	'/^auth\/change_password/' => 'members'
   );
   

   Basically it protects the admin list for only admins, and the change password for only logged in users.

6. I’ll upload a pre-made config to github soon, but you need to change it to look like this:

   <?php  if ( ! defined('BASEPATH')) exit('No direct script access allowed');
   
   /*
   * This is the access control list.
   * It should be an associative array of '{uri}' => '{role required}'
   *
   */
   $config['kibishii_acl']	= array(
   					'/^auth$/' => 'admin',
   					'/^auth\/change_password/' => 'members',
   					'/^auth\/index/' => 'admin',
   					);
   
   /*
    *  Set this to TRUE to enter test mode.
    *
    *  In test mode, the decision process will be display on the screen.
    *  And the result page will not be display (whether permission is grant or not)
    */
   $config['kibishii_test_mode']	= FALSE;
   
   // turn off everything
   $config['kibishii_disabled']	= FALSE;
   
   // The url of the login page
   // This overrides any rules in the ACL and make this url always unprotected.
   $config['kibishii_login_url']	= 'auth/login';
   
   // set to true if you want a 404 instead of error when access is denied.
   $config['kibishii_denied_show_404']	= FALSE;
   
   // The view to load if access is denied
   $config['kibishii_denied_view']	= 'denied';
   
   /**************************************************
    *   Authentication Config
    **************************************************/
   $config['kibishii_get_id_from_config'] = FALSE;
   $config['kibishii_get_id_from_session'] = TRUE;
   $config['kibishii_get_id_from_class'] = FALSE;
   
   // mock user id (used when you set $config['kibishii_get_id_from_config'] = TRUE;)
   // $config['kibishii_mock_user_id'] = 'user33';
   
   /*
    *  The key name for the userid in session userdata
    *  This should be the
    *
    *  Some common ones are:
    *  Tank_auth: 'user_id'
    *  Ion_auth: 'email'
    *
    */
   $config['kibishii_user_id_session_field'] = 'email';
   
   /*
   * ## Configuration of your Authentication ##
   *
   * These things tell kibishii how to find your user's id
   *
   *  kibishii_authentiation_class = the name of your authentication class
   *  kibishii_authentiation_method = the method to call on your class which returns the
    * 										user id of the currently logged in user.
   *  kibishii_authentiation_filename = the file which contains your class
   * 									 (this is not needed if you autoload your class or
   * 									  load your class in the controller's constructor)
   */
   // $config['kibishii_authentication_class']	= 'tank_auth';
   // $config['kibishii_authentication_method']	= 'get_username';
   // $config['kibishii_authentication_filename']	= 'libraries/Tank_auth.php';
   
   /**************************************************
    *   Authorization Config
    **************************************************/
   $config['kibishii_roles_from_config'] = FALSE;
   $config['kibishii_roles_from_session'] = FALSE;
   $config['kibishii_roles_from_class'] = TRUE;
   
   $config['kibishii_roles_session_field'] = '';
   
   $config['kibishii_roles_class']	= 'kibishii_ion';
   $config['kibishii_roles_method']	= 'get_roles';
   $config['kibishii_roles_filename']	= 'libraries/kibishii_ion.php';
   
   $config['kibishii_roles_default_role'] = 'EVERYONE';
   
   // mock user roles used if you set $config['kibishii_roles_from_config'] = TRUE;
   /* $config['kibishii_mock_roles'] = array(
   					'user11' => array('ROLE_USER', 'ROLE_ADMIN'),
   					'user22' => array('ROLE_USER', ),
   				);
   */
   ?>
   

Testing It Out

1. log out if you logged in, then goto /index.php/auth/ you should see an error like “You dont have permission to view this page.”. This means kibishii is working, normally Ion Auth will send you back to the login page, but that is done with an if statement in the controller.
2. now goto /index.php/auth/login and login with “admin@admin.com” / “password”, these are ion auth defaults.
3. now try again to goto /index.php/auth/ and you should get the user admin page (successful authorization).
4. create a new user, then go /index.php/auth/logout , you will see the no permission message again (this is because ion auth redirects to /auth on logout, ill discuss this more later)
5. goto /index.php/auth/login and login as the user you just created (i.e. not an admin)
6. goto /index.php/auth/ and you should get the no permission message, since you are not an admin.
7. goto /index.php/auth/change_password and you should get the no permission message. (this page appears to break in the current ion auth.. ill investigate)

I have just updated JSON Tinker to version 0.1b.

Here is a list of the things I’ve added or improved.

• Replaced the JSON builder/formatter with the native JS one
• You can remove a node by right clicking on it.
• You can now add a node to a collection (array or object) via the small oval at the bottom left.
• You can now publish your JSON as an Internet accessible web service. This is useful for basic testing of web service clients which consume JSON when you haven’t built your web service yet (or cant expose it to the Internet yet).
• If you edit a text value, it will automatically detect the type (string, number, boolean or null)
• Added a Get Satisfaction Feedback on the right. So visitors can suggest improvements or fixes.

Some ideas which I have had myself are:

• Being able to drag sort nodes, or drag between collections.
• Change the text editing boxes to textareas.
• Add help.
• Come up with a nicer colour scheme.

I’ve been pretty busy changing from a terrible job to a good one.
But in both places I had a serious need for an easy JSON editor/generator to test both web services and web service client apps.
I found some great JSON validators and prettifiers, but nothing to let me quickly and easily edit JSON (please tell me if you know of a good one).

.. So I made my own simple one which I named JSON Tinker http://json.bubblemix.net.
Its something I made mostly in my lunch breaks to help me work more efficiently.

Its an online jQuery based JSON editor and formatter.
It doesn’t really do much validation, just fails to parse if your input is not valid.

Basically you can input a json string on the left in the “input” area, then click the “>>” button to import it into the “workspace” area. In the workspace you can easily edit keys and values inline by clicking on them. It will try and auto-detect the type as either string, number, boolean or null. When you are done, you can export back to a JSON string using the buttons “<0″ (for no whitespace), “<2″ for 2 space indenting, or “<\t” for tabbed indenting.

Next up along the roadmap will be the ability to add and remove nodes, as well as re-ordering array elements.
Give it a go and tell me what you think.

Recently I’ve had to deal with Japanese character validation a fair bit, and its a fair bit tricker than dealing with the usual checking that an input only contains letters, numbers and underscores.

A pretty standard white list validation would be something like:

 preg_match("/^[a-zA-Z0-9]+$/",$input) 

Which says all the characters from the start to the end are either letters or numbers. Of course this doesn’t work in Japanese.

First thing you need to do is use the /u option at the end to enable unicode matching. Next you need to put the Japanese characters inside the square brackets [].

To do this we use something like this \x{####} where #### is the unicode for the character. You can look up the codes on the internet somewhere like this.

So if you want hiragana you would use something like: \x{3041}-\x{3096}
for katakana you would use: \x{30a1}-\x{30fc}
For common kanji use: \x{4e00}-\x{9faf}
For uncommon kanji use: \x{3400}-\x{4dbf}

So finally, if we want to white list letters, numbers, hiragana,katakana and common kanji we could do something like:

 preg_match("/^[a-zA-Z0-9\x{3041}-\x{3096}\x{30a1}-\x{30fc}\x{4e00}-\x{9faf}]+$/u",$input) 

You can also add underscores or dashes or whatever suits your needs.

I have been neglecting this blog since I got a new job and have been hella busy. But a few days ago I started received “comment moderation” requests from WordPress.. since I’ve hardly blogged anything of interest, they are obviously spam (also the content is always about free iPad2s and Viagra).

However, the fact that spam bots are reaching my blog means that some search engine somewhere is indexing me. I decided to install a Google analytics WP plugin to monitor this traffic. I dunno if the stats will be at all interesting, but at least I’ll have the information.

If nothing else, let’s hope this prompts me to blog more..

So at my new job I’ve been tasked with migrating out awfully designed Postgres database into MongoDB, while at the same time migrating our PHP/CodeIgniter REST service to Java/SpringMVC.

Our data is a deep/wide object graph with a complex and variable set of properties, so it is actually well suited to NoSQL or a document database or whatever you wanna call it.

Since I’m migrating a system and not creating a new one I had a few constraints. The first one I’m going to tackle is that we need to have auto incremented integer primary keys for our entities. And MongoDB has UUIDs by default, so I had to find another way.
I found the solution
here, and it was all pretty easy. If you are too lazy to read it, it basically says to keep a separate collection to store your sequences (kinda like Postgres does automatically with its sequences).

I tested this out in the admin console and it was all great. But the Java driver is not quite as straightforward as the console, its not just JSON, it has lots of new Objects like BSONCallback, BasicDBObject and MongoOptions which don’t make much sense to n00bs like me…

It takes a while to get your head around. Since I haven’t been able to find a good example anywhere online on how to do this using the Java driver, I’ll post my code here. I hope it helps someone out there.

/**
 * Get the next unique ID for a named sequence.
 * @param db Mongo database to work with
 * @param seq_name The name of your sequence (I name mine after my collections)
 * @return The next ID
 */
public static String getNextId(DB db, String seq_name) {
	String sequence_collection = "seq"; // the name of the sequence collection
	String sequence_field = "seq"; // the name of the field which holds the sequence

	DBCollection seq = db.getCollection(sequence_collection); // get the collection (this will create it if needed)

	// this object represents your "query", its analogous to a WHERE clause in SQL
	DBObject query = new BasicDBObject();
    query.put("_id", seq_name); // where _id = the input sequence name

    // this object represents the "update" or the SET blah=blah in SQL
    DBObject change = new BasicDBObject(sequence_field, 1);
    DBObject update = new BasicDBObject("$inc", change); // the $inc here is a mongodb command for increment

    // Atomically updates the sequence field and returns the value for you
    DBObject res = seq.findAndModify(query, new BasicDBObject(), new BasicDBObject(), false, update, true, true);
    return res.get(sequence_field).toString();
}

Hello, finally new new blog!

Currently, I am living in Tokyo, Japan! however I am unemployed so I have time to create a new blog. I hope to write about my experiences as a foreigner trying to make it as a web developer in Tokyo. I will talk about technical, business and cultural challenges I face in my daily life.

Just a bit about me, I am Australian according to my passport. I was born, raised and schooled in Sydney. I studied computer science at university and then worked at a bank as a Java developer for 7 years. Then I came to Japan on the JET Programme and I taught English for 2 years at a rural junior high school.

That was almost 2 months ago, now I am living in a tiny, expensive sharehouse in Tokyo and I am trying really hard to find a job in the IT Industry. This is proving quite difficult because
1) I dont speak Japanese, and
2) I dont really remember how to be a good programmer.
It seems most developer jobs for foreigners in Japan require you to be amazing at one of those things (and preferably both).

Language-wise, being able to speak Japanese is described as “business level Japanese” (less than business level is described as “no Japanese ability” when applying for jobs), or measured as JLPT Level2, although in reality “business level” is more advanced than JLPT Level2. I have passed JLPT Level3, which is only one level down.. but a rather large gap down in ability to read and more importantly speak Japanese. The next test is in December, with results out in February.. and even if I could pass (which I cant) basically I will run out of money before then.

So I’m left looking at IT jobs in Tokyo which do not require any Japanese. Recently in the Japanese news has been stories of big companies like Rakuten and Uniqlo switching to English in house, but its still a fair way off being a major advantage for foreigners. Right now, theres basically two kinds of IT jobs which do not require any Japanese, they are “Really really experienced tech guy at Giant International Bank” or “Token English speaker at a dotcom or tech company trying really hard to expand”. Big Bank jobs tend to have more foreigners and better salaries, whereas dotcoms have better work environments and are pretty much all Japanese people, although ones who are quite likely to speak passable English.

The vast majority of the jobs are of the first kind so thats mostly where I have been applying, I’ve had a few interviews but no luck so far. I basically choke on the technical parts of the interview where the lead developer asks me something like “what changed in the default garbage collecter between Java1.4 and Java5″. When I was a Java developer we used Java to develop business solutions… a lot of the things they ask really never come up in day to day web development… or maybe I’m just not as a good a developer as I thought I was..

Anyway I have my first interview with a dotcom next week.. I’ll keep you posted.