Tuesday, 30 July 2013

On working with Python and Javascript .. and HTML and Ajax and SQLite

I have recently been creating a production management system. The system is not that complex, but it is taking an inordinate of time to write ... even though I am using rapid-prototyping languages - Python, SQLite3, HTML4/5, Javascript coupled with liberal Ajax sprinkles ...

I simply don't understand - but I will try to get to the root cause.

So far, I have constructed the tables and written a database backend wrapper - for the SQLITE3 DB which I manipulate on the server side using python cgi.

I have created the concept of "sessions", but not in the MOD_PYTHON way, no, in the more traditional - hacky, store-it-in-the-window_name way ... By allowing the user to send an Ajaxlogin request, which generates a session token on the server side, which is then "kept" or "remembered" on the client side by storing the token in the window name browser object variable... It's probably the most rubbish way I can do it, but it works - survives page reloads (F5), survives navigation buttons (back, forward) ... and means I end up writing less python code to deal with IP-address-based-sessions since our network is mostly DHCP based, and we frequently restart our routers in any case due to some other technical difficulty we have to live with due to a third party.

I have added support for individual "users" and am in the process of linking "projects" to users in a nice way.

I have created an HTMLFormatter python script which can generate nicely formatted links which call JavaScript functions ...

I have written JavaScript functions which append the session token to outgoing requests ...

I have written more python cgi code which checks for session tokens and discovers which user "owns" the token  and does something fancy with that ...


I think I'm starting to notice a trend here ...

Medium/Large frameworks hide all this detail from the programmer - allowing them to deal with the core of their code - the business logic. When writing something from scratch, we tend to spend an inordinate amount of time getting a trivial feature to work "just right", only to blow another week on it when we discover it bombs out in IE...

When you are a hobbyist programmer, it makes sense to write as much as you can on your own so that you can both sharpen skill and learn a diverse set of  code "katas". But when you are an experienced programmer who is simply trying to get a project "done", you don't want to spend 2 days learning how to handle sessions which do not use cookies ... you want to spend that time banging out the code that ties the project to the user and allows the bosses to draw up time sheets and plot graphs and all those other things that bosses do with production management systems....

Even for hobbyist programmers I would recommend using frameworks for things you are already comfortable with - know how to implement sessions already? Use MOD_PYTHON for the next project which requires sessions - you know how it works, now use some code which saves you from 3 weeks of implementing it yourself.

Keep it simple ... but simple is a very relative term - upon embarking on this project, my idea of simple was to have a simple "session" - based on the user IP, which displayed their own project queue to each user, and a more "admin" interface to the sysadmin. I spent about a day banging out some nice python/html/JS/Ajax code to get it all working nicely and discovered that my idea fell flat when the DHCP server (our router) was rebooted during a session ... so I spent some more time tweaking the idea to use ip +  machine name ... but that quickly fell flat when I discovered that users on our network didn't have unique machine names ... and I wan't up to the task of renaming each machine to make my code work ... which is when I stumbled on the "shared login" concept - log a request to the server to get a session token, then associate the token to a user id on server side, and store only the session token on the client side - that way we don't care if the transport mechanism goes offline between requests - we simply send off each request with the session token and let the server handle the rest...

This was by far the most robust approach I have found so far - now if only I could get it to work if the user suddenly starts working in a different tab .. hmmm ...

Well, the only thing left to say is that python/JS/HTML+Ajax are "rapid prototyping" tools for a simple reason - somebody has already built the prototype, whether is is a python recipe or a JS library, it has been built - if you are going to keep inventing the wheel it will be slow in any language ... and will probably end up gathering many patches along the way, until it ends up like my project - which now consists of more patch than actual wheel ....