Software prototypes

(Images adapted from flickr users Duke TIP and Patrick Beeson)

One way in which software engineering differs from other engineering disciplines is that in software the prototype is often confused with the real thing.

Nobody looks upon a prototype 1/50th scale model of a bridge and says "Looks done, let's drive trucks across it now." Yet in software development such sentiment is commonplace -- "Looks like it's working, all we have to do is clean up the bugs and ship it!"

If this were merely a misunderstanding between the programmers and the business-types it would be understandable, but the unfortunate fact is that it's often the programmers themselves who believe this. This attitude of hack-and-patch contributes to the general lack of quality of software compared to other engineering disciplines and this is compounded by the perceived low-cost of failure. Structural engineers don't say to themselves: "If it doesn't work we'll just patch it in the field." Electrical engineers don't say: "Ahh, sort of works... we'll upgrade it after tape-out." It has never occurred to a mechanical engineer that they could hide their lack of quality control by creating an automatic update system that secretly updates their wares behind their customer's backs.

For software engineering to be done right -- like in every other creative discipline -- time and space have to be allocated for building *disposable* prototypes. There is no progress without failure, but you don't have to subject your customers to your failures either.

A month of web coding

(Clever mug from stevenfrank)

I haven't posted anything in the last month as I've been on a web-coding sprint for our new bioinformatic enterprise: "Traitwise". This was the first time I've done any serious web-based development since about 1997. It hasn't changed much -- it's still a disaster.

The list of things that are wrong with web development would fill a book so here's my really short summary.

D/HTML is a disaster. It made a tiny bit of sense in 1994 when pages were, briefly, 99% text and 1% mark up. But now, even reasonably simple pages turn into 99% inefficient layout gibberish and 1% content. So the semantic premise of a web page has been inverted. Sarah told me that I should switch to the meta markup languages like HAML and she's probably right, but I didn't find out about that until too late.

CSS is a disaster. Obviously a committee with no programming experience and little vision of where the web was going came up CSS because only those ignorant of programming would be dumb enough to use "-" as a delimiter. But that's just a trivial gripe: the entire design of CSS is flawed. The base HTML semantic tags "p", "a", etc are not useful, only were for about 6 months in 1994, and now every page devolves into zillions of divs and spans each with a specific class moniker so you end up with a flat CSS "hierarchy" thus nullifying the CSS design and turning every page into a giant, un-readable mess.

JavaScript is a disaster. Actually, the core language is mostly OK but the portability -- the thing that really matters -- is a disaster. Thanks Microsoft! Oh, and also thanks to Google -- Chrome has broken JS even more. Chrome is the first product from Google that really pisses me off in a Microsoftian kind of way -- making the world a WORSE place instead of a better place. Attention Google, we don't need any more non-standard changes to the JS API. Please, just back away slowly from the browser market and let Firefox and the Internet standards committees lead the way. You're not going to profit from it anyway so please, just say no.

SQL is a (continuing) disaster. The inefficient mapping of the most rudimentary problems onto the relational database model is just plain wrong. The world needs a standard simpler transactional hash object that would do most of what you want as a web developer and be a lot simpler and faster. This is especially relevant given that environments like RoR are just building and mapping all these relationships internally thus nullifying the point of a relational database. RoR might as well just abandon the database and use its own transactional hash-oriented file system. The only reason they don't do so, one assumes, is the momentum of legacy databases.

Flash/Flex/AS3/MXML is a disaster. Let's sart with the marketing. What is the name of this product? Flash became Flex but is wrapped in MXML? Or is it AS3? Or is Flash the cleint-side run-time? Why do you compile .as3 files with an MXML compiler? WTF guys, how about version numbers, try them, they're great. I'm sitting next to Rob, a former Adobe developer, and even he doesn't understand the names -- we had a 15 minute argument sorting out which part of this product line is named what. There's so many confusing and broken things in Flash/Flex/As3/MXML/WhateverItsCalled I could go on for hours. I wasted untold days sorting out the simplest, stupidest things. I put "FLEXHACK" in my code next to every work-around or counter-intutive hack (like putting +5 on every textWidth) there must be 100 of those tags in my code and I got tired of putting them in after a while. Really basic concepts of object oriented programming are just flat out violated by their awful Flex design, for example the unclear dynamic typing, the UIComponent mess, and the lack of deep object duplication. But really, I could go on and on.... Like, how about the secret *deleting* of subversion controlled files off my hard-drive that the IDE didn't own -- that was just great! Or the secret deleting of comment lines at the top of the files in the IDE. Or the screwed up caching of a SWF in the wrong folder that frustrated me for several painful hours.)

The only thing that isn't a development disaster in my month-long experience was Ruby on Rails. There were a few odd things about it that threw me off (it was much stricter about type casting than I assumed it would be) but mostly it made perfect sense and Rails is obviously the work of a professional. Everything about Rails from the docs to the API's demonstrated that the author(s) knew what they were doing. I was impressed.

Parameter Fitting Lecture

I gave a talk a few months ago about the new chemical kinetic parameter fitter that I've been working on with John Davis. It is a good introduction to the general problem of fitting model parameters and discusses how easy it is to confuse a good fit with a good model.
[PPT slides] [PDF slides]

Fretview

Today I worked on updating an old project of mine "fretview" that is used by Rick Russell's Lab to do analysis for single molecule kinetics. We had recently improved the capture program to permit pixel binning, but the analysis code did not yet take this into account with the result that it tended to incorrectly sub-sample binned pixels resulting in rounding errors when coordinates were mapped from side to side.

Kinetic Explorer v2.0 Released

http://kintek-corp.com/kinetic_explorer/

This is a reaction simulator and data fitter project that I started years ago with Ken Johnson and Thomas Blom. We have just released version 2.0 which includes substantial improvements in the integrator and includes a nice tool for viewing the parametric fit space. After playing with this for years now, I'm convinced that the major problem with fitting tools is that it is incredibly easy to fool yourself into believing that you have a well constrained system when you don't. In this and the upcoming version we've put enormous effort into a UI that can demonstrate if a system is well constrained and if not, why. Thanks to a lot of effort by my bestest-nerd-buddy John Davis, v3.0 will have a brand new super-optimized fitter that uses singular value decomposition to dramatically improve the fit descent and also provide instant feedback on the system's condition rank in signal to noise units.