System Reboot 2.0

A computer could function well for up to 3 years, after which maintenance or upgrading work need to put in place to keep it running up to 5 years; and the same cycle applies to software development.

I develop the first version using raw PHP (no framework, trying to develop my own framework at the same time) with MySQL and plenty of copy and modify JavaScript from other website, and that failed miserably due to high maintenance and low quality work (that version 0.1).

For the next version, I know that I needed a framework, and RoR is pretty hot. I didn’t get to use RoR due to deployment environment constraint, thus I use CakePHP which is a pretty good rail-inspired PHP framework. JavaScript isn’t that complicated about 6 years ago, so just jQuery and plenty of plugins is enough to satisfy the users. The stack runs pretty smooth, except some minor upgrade from CakePHP 1.0 to 1.1, 1.2, 1.3 and I ended at 2.0; eventually a local hosting isn’t flexible enough, thus I opt for AWS.

If it ain’t broken, why fix it?

The current stack had served we well for the past 6 years, and I could probably extend it for a few more years due to the still active framework community and fairly well maintained codebase (after all, I am the only programmer). The benefit of an old stack is that I get high productivity, it’s fairly easy and fast to implement new features and modules because the foundation and supporting library is already there, and I am used to the ins and outs of the system (no more time consuming debugging of a new component). The downside is that I might had made some lousy decisions on how some things should work, and still pay the price to maintain it because it’s too costly to change. The thing is, I don’t know anything bad about the current system, because I haven’t use a better system. I do enjoy the high productivity, but there are always some temptations.

I wanted to develop a new app, and was planning to use the existing stack because it shall be fast and easy. Then again, I admit I am a bit tired of the old stack. I was fascinated by Google App Engine, Python or Ruby, and heard so much about CoffeeScript and Backbone. I might not find the will to upgrade the existing app to a new stack; but if it’s to develop something new, I should probably learn something new as well. Without much logical reasoning, I decided to seek out a new stack.

Reasons for Reboot?

• Platform: DIY platform vs. managed platform like Google App Engine
• NoSql vs. RDBMS
• Temptation of new language: is python and ruby a step forward from PHP?
• Micro Framework: it less magic and automation a better choice?
• The rise of JavaScript (perhaps CoffeeScript)
• Build system for front-end scripts and resources
• UI and Responsive web design
• From a publication-centric to user contribution-centric
• Unloading of old baggage

Google App Engine

I decide to use Google App Engine. I understand the proprietary and lockdown nature of things (with potential of regret if Google screw up), but I do like the idea of being liberated from system admin stuff (webserver and database installation and configuration, security, scalability, etc.). Besides, it had some nice services like Images (Picasa-grade image serving), Memcache and Task Queues. The downside is dependency on BigTable, and with no possibility of exploring MongoDB; though MySql is possible on GAE, but the setup felt like second class citizen. Another downside is I can’t install almost can't install anything on GAE (except pure Python code without disk I/O). There are definitely some advantages and drawbacks, but I choose to let Google handle the platform sociability part.

The road map and bug fixed might be a bit unpredictable, with certain issues remain unresolved for years (fallen into the black hole) and certain feature taking 4 years to arrive. I am hoping things a more matured after 4 years.

Meteor seems like a possible trend, where we don’t even need to manage the webserver or code on it anymore, where the only coding necessary is JavaScript for the browser. Its looks awesome and magical, yet I am not ready to embrace it yet due to the fact that “I lost control of the webserver”, and too much wizardry and automation kind of put me cautionary mode.

NoSql: BigTable

Why use NoSql? It’s because that’s the preferred storage on GAE, and I don’t mind exploring the NoSql way. The big advantage is that you got a really big database where you need it to scale, and schema changing for traditional RDBMS lock up the table big time. I don’t think I would hit such massive scale. I believe there are a lot going on for MongoDB, but sadly it’s not an option opened to me at the moment.

I do like a few cool tricks with BigTable (or NoSql), especially the newer NDB API. It has a few interesting property attributes, such as repeated where you could make a field into list without additional table. With ProtoRPC, you can have enum fields. You can have structured property, it’s like putting another table structure as a column, thus reducing the needs for multiple tables, and inheritance is supported. JSON, GeoPt and Pickle (Python object) are native fields.

The query might be limited (no like statement, no join).

Python

I am bothered by the favoritism gained by Ruby and Python language, and decide to check it out. Language is like a culture (it's about the people and community, with a specific style). PHP doesn’t really have a feature advantage over Java or .NET, I just don’t like strong-typed stuff and enjoy the flexibility of PHP array. Since I am on GAE, python is the way to go for me.

Python style is consistent using underscore convention, with strict indentation and directory structure representing namespace, and elimination of semi-colons and curly braces (which is fine by me). PHP does have a messy naming convention with its core library. I haven’t dig far, with it does have some nice libraries such as WTForm.

Personally, python felt slightly nicer to code in than PHP, and felt more consistent. Another reason is occasional we saw releases of come cool tool and library at Hacker News, and they are usually written in Python or JavaScript. I believe a vibrant community and continuous innovation is important for a language.

Micro Framework

Django is the de facto full stack framework for Python (like Rails for Ruby), but I wanted to try a micro framework instead (less magic and automation). Plenty to choose from such as CherryPy, web.py, Pyramid and Tornado, and I was deciding between Bottle and Flask. I settle for Flask because it seems simpler with less magic, slightly stronger community support and embraces Werkzeug and Jinja2 (good praise from community, existed for a long time). Bottle one file approach didn’t seem like a seller to me.

I really like CakePHP and think it’s awesome, until I tried something more awesome: Flask. It’s lightweight; with less pre-defined structure and more transparency (the code is cleaner, with more control without hacking the framework). It feels more liberated coding with Flask, the blueprints give it a good structure, and flask-appengine-template serve as a good starting point.

The rise of Javascript/CoffeeScript

I realize most of the development work is done on the front-end browser side using JavaScript, thus front-end coding had become more complex and require more structure (a framework). I choose to adopt Backbone (over KnockoutJs and AngularJs) due to the strong community support, with simplistic yet structured nature. I looked in Chaplin for large application development, but I believe the maturity is not there yet, and I would not develop such large application at the moment.

CoffeeScript is love which couldn’t be explained through logicality: it just feels good to code it, with cleaner code, with arguably increased productivity.

Build system for front-end scripts and resources

Front-end web development had gotten more complex that same kind of build or optimization system is required. I started with Brunch with provided skeleton templates which bring me up to speed with the setup, with an application assembler to auto compile CoffeeScript, to merge scripts and stylesheets, with AMD support.

Yeoman seems like a promising star with extra features such as image optimization, might be worth checking out soon.

UI and Responsive web design

It is convenient to utilize Twitter Bootstrap as the UI foundation (grid system, button, forms, etc.). I am still doubtful about the practicality and development cost of responsive web design.

From a publication-centric to user contribution-centric

In my current app (a restaurant review website), I am more focus on publication: the searching and display of information, where the user contribution features isn’t as intuitive as I would like it to be. Think of Pinterest, where contribution could be easily made through bookmarklet.

Unloading of old baggage

Sometimes things are done in a less ideal manner due to inexperience or out of convenience, where we are continuously paying the price through productivity lost, less elegance solution or potential security risk. Correction work seems like too much work for too little value, or it is almost impossible to undo previous mistakes, or the framework is the limiting factor.

To start a new, you get to be plan once more, hopefully with more wisdom and less mistakes.

Conclusion

As a programmer, we always have the tendency to reboot once things started to get messy, or being tempted by new technologies. I admit it’s a bit of both, and I acknowledge the price of productivity lost that I would need to pay, and I do believe the ROI with better design, productivity, performance, features and maintainability is worth the price.

I almost decided to stick with the old stack, but I believe the timing and benefits are at the sweet spot.