a system.idle narrative

I like it when technology works :-)

2008-09-15T09:54:00.000-07:00

I host and maintain a website for a local non-profit organization. It's all volunteer so I like to keep the number of updates to the site on my servers to a minimum, if possible. When the members wanted a news feed, which implied a lot of updating on my part, my solution was to set up a blog for them for them on www.wordpress.com and then post the RSS feed back to the organization's web site. Gotta be easy, right? Well, actually, it is. Easy but not obvious, which is why I am posting how I did it here.

When I searched around for information on displaying RRS feeds, I found a lot of sites about RSS feed aggregators, hosted wordpress services and such, not exactly what I wanted to do. Some posts in forums I found even claim this can't be done. Not so, it can.

Wordpress offers you a URL like http://yourdomain,wordpress.com/feed that you can imbed in an iframe, which looks good in Internet Explorer, but not in FireFox, Chrome or Safari. After some additional surfing for best practices,I found the Google "Dynamic Feed Control Wizard - Put feeds on Your Web Page", here:

http://www.google.com/uds/solutions/wizards/dynamicfeed.html

A couple of clicks and your generate the codefor your web site using the AJAX Feed API. Cut and paste it into your 'news' page and your done. Any post to the wordpressblog shows up in the RSS feed window on your website. You can even customize the control further by reading the developer documentationand modifying the generated code. Saved by technology. cool.

Changing the program associated with .jar files on Windows Vista

2008-09-10T05:58:00.000-07:00

Windows Vista Explorer, like XP before it, natively supports browsing .zip files, although for some reason known only to Microsoft, the' re called 'compressed folders'. There is a built-in or hard-coded file association between .zip files and a nonexistent program named 'CompressedFolders', and as a result this program does not appear in the list of Default Programs so you can configure addional associations. Why they made this a special case I don't know, but it makes it less than obvious when trying to associate the decompression and browsing functionality of Windows Explorer with .jar files.

The fix? Open a command window as administrator and run:

cmd /c assoc .jar=CompressedFolder

now all jar files can be exploded and browsed with Windows Explorer on the desktop.

Windows Vista Error Message : Exactly Right, Yet Competely Wrong

2008-06-14T10:20:00.000-07:00

Ok, well maybe wrong is not the best word ... perhaps useless would be more accurate. This is yet another one of the myriad of cases where error messages displayed during normal software use are factually correct, yet contain absolutely no additional information on how to correct the situation. read: misleading and useless! Have you ever seen this one?

"The network folder specified is currently mapped using a different user name and password."

nice. gotta love it. And wow, I get an ok/cancel dialog which is also very helpful. I look at all my mounted drives and I don't have any other mapped network drives, only the network folder showing the NAS device in question. The full error message informs me that is a permissions issue. Now, for better or for worse, I am administrator on the NAS server I am trying to access, as well as on my laptop and both are part of the same domain.

The purpose of this rant? Only to document for my sake the fix:

- open a cmd prompt
- type "net use" at the prompt to view the list of open network resources
- type "net use (resource name) /delete"

This removes the network resource and then you can mount the resource as a network drive or otherwise browse to it as the correct user.

You know? I've said this before .... I wish i had a penny for every time bad software design compromises my productivity ... I might be able to buy Yahoo. *sigh*.

Man, I hate it when that happens ....

2008-05-29T04:49:00.000-07:00

Ok, so it finally happened. I got pooched by Bill and Steve in the same day. Let me explain. My MacBook Pro running Vista 32 bit caught a cold. I know, I know, you are shocked that this could even happen to *my* computer given that I have virus and spyware protection up the wazzoo and I am pretty frickin' careful. I turned off my main virus protection (only for web pages and only for one second!) and so I was counting on Windows Defender alone at that point. Alas, my faith was ill founded and .... Blammo.

I tried everything to get rid of this pest, and the more I tried the more screwed up my Vista partition became. No worrries I thought. I just came off a project so if this was going to happen, now was as good a time as any to repartition my MacBook hard drive and give more space to Vista -- thereby nuking the Windows partition, virus and all, in the process.

Now here's the part where I felt like kickin' Steve in the butt. I boot Tiger only to find that I can't repartition the hard drive becuase the BootCamp Beta 1.4 expired long ago -- so I tried setting the clock back to before last September and that didn't work either. Sheesh. A forced upgrade. I hate it when that happens, but oh well, there is some good here. Since Leopard supports Vista 64 I rushed out and picked up the latest MacOS. It installed pretty quickly, I ran BootCamp Assistant to make my Windows partition larger, so far ... so good. Following the BootCamp instructions I installed Vista 64 bit on the new, larger partition, and when that booted up I inserted my Leopard disk only to find that the BootCamp drivers don't install cleanly. The keyboard driver service won't start. No network driver. Crap. Hours go by ... scouring the internet looking for the network driver. Thank god for this guy, who directed me here. Ok, now I am limping along with a network connnection, and I go to the Apple support page to find a Vista 64 BootCamp 2.1 update. Great! Ahhh, not so great. It seems Apple in its infinite wisdom released an update to version 2.0, but failed to release version 2.0! So, the update won't install -- and what's worse? It fails silently with no log message or notification to the user. After a lot more searching, I finally found verion 2.0 drivers in this torrent.

Finally! My MacBook is now running Vista 64 SP1 -- no thanks to you, yes I am wagging my finger at you Bill and Steve -- you may now hang your heads in shame.

flame off.

Note to Steve... Steve? You could sell way more Macs dude.

2008-04-05T09:03:00.000-07:00

Now I don't want to touch off that old powder keg of "Which is better, Apple V.S. PC?" I have iterated to the opinion that you should choose whatever system works for the task at hand. I use Windows because my clients do. It has all the tools I need to get my job done and Mac's don't. Mac's are not easier to use, not more intuitive, not more reliable, and not in any real objective measure better, just different. Sure Mac's have fun toys like Garage Band, but since I no longer make my living as a musician, toys don't help pay the bills.

In my day-to-day work I use Windows Vista SP1. I consider myself a power user. The demands I make on any system eventually end up exceeding it's capacity -- so I replace my laptop about every two years. This time, when I went searching for a new laptop, I needed one that had some juice. I liked the Alienware and Voodo laptops and although they are pretty heavy, they kinda rock. But wait, their is a sleeper here with even better specs!

My choice? An Apple MacBook Pro with 17" with a glossy 1920x1200 display.. It's top of the line, 4 gigs ram, 7200 RPM drive, maxed out everything, etc. etc. -- pretty sweet.

I tried running Vista under Parallels, but it was waaaay to slow so I opted for Boot Camp. The install was straightforward and it worked right away. My only issue was the graphics driver -- the default driver was a generic VGA one, so even though Aero worked it was clunky. I was not able to use any of the native graphics capabilities until I downloaded a driver for the NVIDIA GeForce 8600M GT. I was surprised at the response time of the machine, pretty snappy. Then I found this article ... wow imagine that -- the world's fastest Vista laptop is an Apple MacBook.

So, this begs the question. This laptop is rockin' the free world so why doesn't Apple sell it? Steve, you have made a pretty slick, powerful, light-weight road warrior solution. It handles anything I have thrown at it so far -- I have even dropped it twice and it survived. IMHO Apple would push more product if this configuration was available directly from my local Apple Store -- not to mention the obvious positive influence this would have on the share price.

The only criticism I have is that Apple Boot Camp does not support 64 Bit Windows Vista, so I can't put the pedal to the metal as far as CPU horsepower is concerned. My only regret? That I didn't use the *whole* disk for the Windows partition.

Dude. Offer the MacBook out-of-the-box with Vista 32/64 and Wall Street will love you.

Once and Only Once

2006-01-24T14:33:00.000-08:00

When a piece of behavior or information is represented in more than one place in a software architecture, there is a really good chance that, eventually, it will get out of date in one of those places. In the short term, each time that behavior (and I mean this in the general sense, not just a piece of code copied from one place to another, but any information) needs to be changed, you have to locate and update each copy, multiplying development costs. Following the path of least resistance, over time, it is likely that one of the representations will get out of sync with the others, causing bugs, needlessly increasing code complexity, thereby increasing future development, maintenance and support costs. NDOC and JAVADOC allow developers to embed documentation right in the code and serve as examples of tools that work in practice to help maximize the return on investment in software over its natural lifecycle by reducing information redundancy. These tools are good because they make it easier to do the right thing first. When designing and implementing software systems, searching out the redundant representation of information can seem to make a simple process needlessly more difficult. Sometimes it is subtle, sometimes not, easily dismissed as unimportant, but in my experience worth the time and effort spent to identify it and factor it out. So, next time you're tempted to cut and paste that code snippet, think about the implications, and go back and read this.

The more things change, the more they stay the same.

2006-01-24T07:56:00.000-08:00

I have spent the past 7 months or so reacquainting myself with JAVA based development after a two year run building a .NET credit derivatives trading system in C#. Now back squarely in the JAVA world and working on a number of projects for several global trading organizations, some of my old gripes with Swing-based (and in general, JAVA-based) applications have come back to the fore. This time around I find myself motivated to use SWT for my GUI development, since, like the VM, SWT based applications are by definition more tightly coupled with the native platform so there is a better chance of producing good looking applications faster -- at least that is part of the promise of SWT over Swing.

Having read through much of the positioning statements on SWT vs. Swing readily available with a web search, I have found that one of the biggest challenges in SWT development is the lack of comprehensive, non-trivial coding examples that don't use JFace. This is completely understandable, given the relative maturity of SWT as compared with Swing. As JFace becomes more cleanly separated from Eclipse and hence more usable for standalone work this situation will improve dramatically I suspect. But for now, while I have looked into the design and implementation of JFace, for standalone development projects I am avoiding it ... for the time being.

This brings me to my point. I am working on an application framework for my own development projects, and I don't want to spend the time tweaking a Swing-based framework just to get it to look half decent. Partly too, I chose SWT to prove to myself that SWT is mature enough to use in sophisticated development efforts (other than Eclipse, mind you), or rule it out. Recently, I couldn't find a simple example that demonstrated how TreeItems could be selectively added and dynamically removed from an SWT Tree, presumably because of the following design principal of SWT:

A parent is set by passing it to the constructor of TreeItem and can't be modified from either end of the relationship. Because of the way these relationships are maintained, removing a single item from a Tree is awkward: You must call removeAll() to empty the Tree and rebuild its contents, minus the items you wish to remove. ... from SWT/JFace in Action

While this makes it easier for the implementers of the widget to ensure it's internal integrity, it makes it harder to use an SWT Tree in contexts where you dynamically want to add and remove nodes based on user control. Not impossible mind you, just harder. So, my contribution is a hopefully ever-growing list of snippets. The first example, Tree and TreeItem, creates a tree with a single root element in it, and by right-clicking, the user may insert new nodes before, after or into the selected node. The example does not allow deleting the root element, or giving it any siblings, so the first action selected must be to insert into the root node. From there, you can freely add and remove nodes.

The Tree example may be straightforwardly generalized to implement insert/remove functionality on subclasses of these SWT widgets, and thereby reuse this functionality across projects.

Motorola V300 GPRS Modem Configuration on Windows XP SP2 Revisited.

2005-12-26T07:51:00.000-08:00

I recently changed laptops once again, and once again I was confronted with the problem of connecting my Motorola V300 cell phone to my Windows-based laptop via USB, so I could use the phone as a GPRS modem. I have come to depend on this configuration while traveling to locations where there is no suitable LAN I can hop on or there isn't a wireless hotspot available. Some of you may recall that I have been through this before.

This time, I was determined to be more methodical about connecting the modem, installing Mobile Phone Tools with the right drivers, as well as documenting everything, but most of all I wanted to be able to do this quickly, should the need ever arise again. Now there is no point in calling the support line of your cell phone provider for the Windows configuration of the GPRS modem, even though these settings are provider dependant. Trust me. I learned the hard way. My provider has a voice recognition based support line and when you call it sends you in a never ending downward spiral, making one feel an awful lot like Commader Riker. Put it this way, as usual, the help desk was no help. During each of three calls that ended in stalemate, my technical support representatives sent me to Ericsson mobile phone representative ... and of course the technicians there could not have been less interested in my inquiry, as soon as they heard keywords like Windows and Motorola. Why Ericsson? Your guess is as good as mine.

However, in the interests of trying to help those of you who want to do the same thing at some point, and also to record the process for myself, here is what you need to do to connect your cell phone to your laptop and use it as a GPRS modem, assuming you have the required phone<->USB connecting cable and drivers for your phone.

You will need a couple of pieces of information from your cell phone service provider (these settings are typically the same ones as the Blackberry users require to access the internet from their handheld, so if the support agent does not know what you are talking about, which is highly likely, ask for the GPRS Blackberry settings). They will give you a WAP user name, password and phone number, which is typically something like â€œ*99#â€. These values will be entered into the Windows Dialing Properties dialog when you set up a new connection, in exactly the same way as you would set up an Internet dial-up connection. In addition to the user name and password, ask your provider for the Access Point Name (or APN), which in my case is 'internet.com'. What they can't or won't tell you is where, in Windows XP, you enter this information, since the APN is not part of the dialing rules.

Once your phone is connected with the cable to the laptop with the correct driver installed, and the phone turned on, Windows should recognize the new hardware as a modem. Right click on the modem in Network Connections, in the device Properties dialog. Click on the advanced tab. There you should see a text field labeled "Advanced initialization ...". This is where you enter the APN name, but you need to guess at the format of the initialization string for your provider -- in my case this is +cgdcont=1,"IP","internet.com". A quick internet search will likely turn up this configuration string for your setup -- here are several I collected (although your mileage will vary):

TMOBILE USA Users: +cgdcont=1,"IP","internet2.voicestream.com","",0,0

AT&T: +cgdcont=1,"IP","proxy"

Rogers: +cgdcont=1,"IP","internet.com"

Cingular Users: +cgdcont=1,"IP","ISP.CINGULAR"

UK Vodafone Users: +cgdcont=1,"IP","internet"

Good luck!!

In Flight Movie Anyone?

2005-11-19T07:48:00.000-08:00

Now that Microsoft no longer includes DVD decding as part of Windows XP, you need to pay extra to watch movies on your laptop on those long overseas flights. Of course, the laptop vendor usually supplies the DVD decoder. In my case, I sidegraded my laptop to Windows XP, blowing away the OS that came on my work machine. Maybe it was just too early in the morning, but today I found locating a free DVD player download for XP on the net somewhat time consuming. I finally found this one. Enjoy. Now off across the pond ...

eBay ... may it roast in hell.

2005-10-14T07:46:00.000-07:00

Anyone who has worked within a large company knows that the, ahem, help desk is rarely any help at all when things get technical. But I know they mean well, which is something, I guess. A case in point. I recently opened a help desk ticket when I mistakenly corrupted the network configuration of my corporate laptop, which by the way, for security and support reasons is locked down pretty tight -- otherwise I would simply fix these things myself. I was told to sit tight and a support rep would be flown in from Boston in four days to help me. Pointing out the obvious was futile ... oh yeah, remember they *mean* well. Anyway, one comes to expect this type of thing when it comes to customer service.

That being said, however, one can usually call their support agent and at least speak with a human being when issues are too complex or hard to communicate over e-mail alone. eBay, on the other hand, with a recently reported market cap in excess of 53 billion dollars does not have a phone number one can call for a little love in a time of crisis.

So, when I came up with a technical issue with my account on eBay, where could I turn? Then I noticed the "Live Help" feature. Wow, great. No more waiting days for a personally addressed yet completely dispassionate, unhelpful e-form letter response that comes, if ever, much too late for the party. Using the community discussion forums to try to reach someone behind the Chinese Wall eBay has erected that actually knows how things work has historically been pretty pointless too. Since the forums are by necessity text based, one has to spend extra time to craft a request describing an issue in clear, succinct, accurate terms, which is no mean feat for complex technical problems. So, you spend hours crafting that perfect message and send it off only to hear this. Using "Live Help", I could actually speak with someone, or so I thought.

Click on the Live Help button any you see a little friendly cartoony face with a feigned greeting "How may I help you?". To top it off, when you get a connection to an agent, it's much like a dialog with the Castle Prefect. Their suggestions for how to overcome web site issues invariably come down to reboot your computer (which thankfully disconnects you from the help session), clear the cache, delete cookies, enable cookies, turn off your virus protection, Internet security and firewall while accessing the page in question. Has any thought been given to how absolutely insane doing most of these things actually is? Any web site that requires you do access their site while you are completely exposed is, albeit arguably, somewhat badly designed I would think. The upshot? These guys are no help either, sweet little cartoony faces, syrupy greetings and all.

I must say I am completely impressed by what a ridiculously absurd practice eBay offers in the name of customer support. With the introduction of Live Help, eBay's customer service policy is now the bellwether in corporate impersonality, all the while masquerading beneath a cooingly narcissistic dialog of "Community". It would be hard to imagine a more brilliantly crafted. completely ineffective help desk strategy -- if it wasn't so bloody annoying, it would actually be downright hilarious.

Hello eBay ? ... my name is K.

Here's to you, Mr. White.

2005-09-25T07:42:00.000-07:00

I realize it has been some time since I've been able to post -- been really busy changing jobs and starting up some projects based on some new (to me) technolgy platforms for derivatives trading, like these guys. But, the upside is that I have been spending a lot of time in the big apple and also some in San Fransisco. The downside is that many things have been pushed to the back burner, like this blog.

Over the past couple of days my team decided we really needed a repository set up for our current project, so what better technology to bring to bear but subversion. But of course. I should also point out that we are running on a pure Windoze platform. The upshot is that because of these guys I managed to get our project set up, running and populated in one day on a Windoze 2000 server, and in particular I am really lovin' this guy right now. Great job Mr. White, I owe you at least a cuppa Joe.

I hate computers...

2005-08-27T07:44:00.000-07:00

... or should I say more accurately that I continually find computers running Windoze very, very annoying. I often wish that somehow I could find an effective way to provide some constructive feedback to Microsoft and make it clear to them how incredibly frustrating my loss of productivity when using their software is to me and why.

A case in point. Today I was writing a .BAT file on Windoze 2000 to wrap up my JAVA application. Admittedly, my class path has been seemingly growing without bound lately due to the complexity of the project. When I attempted to run it, I was presented with the following error message:

The input line is too long.

Great ... lovely. Thanks for the feedback Bill. Hopefully someday I can return the favour, but that is probably just wishful thinking on my part. I find things like this commonplace whenever human factors in computing are left to programmers alone. The message is, technically speaking, exactly correct and succinct. The input line is too long. End of story. My problem is that it does not tell me how to fix the problem, or which line is too long, or that the Windoze 2000 command line length limit is 1024, or anything ... arg! In effect, this message contains about the same amount of real information as "An error has occured." or "Elephant bathtub boondoggle."

My issue with things like this (other than it costs me time to restructure my BAT file) is that very likely some technologist somewhere in the development chain was either lazy or simply forgot one of our profession's most important best practices, some of which are summarized nicely on this page:

Help users recognize, diagnose, and recover from errors

There are countless examples of things like this that suck the life blood out of my work day, and I know I am not alone. Come on people. This is not rocket science, just common sense and politeness. Be bold. Be the first programmer on your block to start a new trend ... human friendly computing.

Post-mortum dump: What was the fix you might ask? Share the deeply nested directory on the network (making sure that if you don't want anyone else to be able to access the drive, then set the permissions so only you are in the access control list), then set it up as a mapped network drive. That way, the really long path becomes a single drive letter.

Post-post-mortum: A much better approach than the above was suggested by this guy which is as follows:

"Using "subst" or junction might be safer. Mapping drives like that doesn't always work. I *believe* there's some kind of timeout built-in for mapped network drives (even if mapped to localhost). I used to use the Network drive mapping to map to self, and it sometimes failed me for prolonged or heavy operations."

And as usual, he was exactly right. Cheers.

"These must, indeed, be splendid clothes!" thought the Emperor.

2005-05-01T07:40:00.000-07:00

Here's something I've been meaning to get off my chest for a while now. It's all this ongoing hype about Service Oriented Architectures (or SOA for short). I'm sure you've heard about it, but for those in need of a gentle refresher:

Simply put, SOA-based software applications are designed as a loosely coupled distributed network of services, from which one component can request things from another one in a standard way. Using SOA, the architecture of an application is based on dividing its functions into services that are accessed using some protocol like SOAP/XML/RPC over web services (although there are other implementation mechanisms in common use as well). Thus, the distinguishing characteristic of SOA based applications over traditional ones is that the primary structuring element is the set of services. Since these services can interoperate over different development technologies (such as Java and .NET), the services are reusable between different applications, thereby reducing development time, integration and ongoing maintenance costs. No wonder the evangelists of this stuff propose it as the holy grail for enterprise interoperability -- use a SO architecture and voila ... sounds great, right? Bzzzzt. Thanks for playing.

Let's looks at a few examples. The telephone and its associated directory service provides a standardized point-to-point connection mechanism that we're all pretty comfortable with. Similarly, SOA implementations specify how to locate and connect to other applications on the distributed network (the syntax of communications is specified by the implementing technology), but by definition, they canâ€™t really help with the semantics of the communication, regardless of efforts such as the Resource Description Framework (RDF) or the Web Ontology Language (OWL). These approaches, while useful, can provide only another level of indirection but do not solve the fact that software cannot read an arbitrary communique. Try as you might to settle on a language for every possible call recipient, if the guy answering the phone doesn't speak your dialect, you're pretty much pooched. Ok, that's point one.

Secondly, in my business, which is focused on global derivatives trading systems and financial risk management, applicable enterprise level applications have an a priori dependence on firm-wide data availability and information standardization. In this domain, which many preach is the perfect application of SOA, it's often the scale of the problem that drives system complexity and makes software development so difficult. Let's take an international investment bank, with more than a million or so transactions per day created or amended, across seven time zones using, say, eighty front office trading systems. Now suppose the middle office wants to run a set of nightly global risk reports on the bank's entire book. You've got a two hour window to calculate the results, because that is the only time all the trading floors are closed across the globe and changes to the book are not being made. Keep in mind that each of those eighty trading systems represents transaction, counterparty, market and reference data in a completely different way, using a system specific proprietary lexicon -- all of this data needs to be collected and converted to a standard form on an intra-day basis for consumption by the risk reporting engine for each night's run. For example, if one system sends a trade with the counterparty code of I.B.M. and another IBM, these two codes need to be linked together or the corresponding positions will be incorrectly rolled up. When the reports are wrong this is a bad thing. The regulators can get involved and things get messy -- for good reason, this makes some people very, very upset.

Think about it. The IT challenge here is that the risk engine needs to access and price every single transaction (sometimes many, many times under scenario analysis) in the shortest possible amount of time and tally the numbers for the reports. Speed is paramount, and information standardization critical to the results. If an application needs to remotely call into a bunch of distributed services to collect the data, pass that off to another for analytics, you're a day late and billions of dollars short. You've already taken too long. Read: remote communication == "waaaaaaay too sloooooow". In other words, performance metrics and instant availability of standardized data are architectural requirements in a context where demand always exceeds capacity.

When you really look under the skirt, SOA is nothing more than the same old tried and true distributed computing concepts in a brand new dress. In any performance-critical, information-centric context, SOA does not address the most fundamental requirements for developing real-world solutions. In fact, it ignores them. The focus is on the wrong thing. It's not the interfaces to all of the systems on the network that are dynamically changing, it's the content. Remember the old saying? If it sounds too good to be true then it probably is? Well, you got it.

Now that I have that stake firmly pounded into the ground, I'd best get off this SOAP box ;-) and back to work. Cheers.

"I am endeavoring, ma'am, to construct a mnemonic circuit using stone knives and bearskins." - Mr. Spock, City on the Edge of Forever

2005-04-03T07:38:00.000-07:00

What with all the advancements in software development environments, from frameworks such as JAVA,.NET and a host of third party IDEs, libraries and engineering tools available to jump start projects, you'd think it would be easier to create software. Once upon a time a single programmer could write a useful application for some purpose or another (few of you will likely recall the days when we had to write our own text editor to edit the code to create the software, and then an operating system upon which to run it). These days it takes teams of people. That is, of course, if you don't consider writing that mobile phone plug-in a real software application ;). Understandably, the problems we face are getting harder because new tools enable new possibilities that drive the potential reach of enterprise-scale projects -- but it never ceases to amaze me what some people can do with a simple hand-held device.

Based on the often self-fulfilling premise that the available development tools are stone knives and bearskins every project still seems to start out the same way -- with a plan to build the development tools required to develop what's really being developed. I am not talking about fundamentally new tools or technologies, rather components that mirror or come close to existing ones.

My personal belief is that this pattern is motivated by a sociological phenomenon and not a technological one. Many software professionals I have met over the years just donâ€™t seem to like other people's code, and they'll often go to any lengths to avoid looking at it, using it, and maintaining it. After all, I suppose it's a lot more fun to create something new than use someone else's solution, right? Like many in this business, I have plowed through, fixed and refactored my share of digital gob, all the while cursing my predecessor for being so undisciplined, so maybe it's a well-founded trust thing. Whatever the reason, does it make sense to spend man-years in development to define a new language, develop a compiler for it, roll your own IDE, graphical user-interface designer or object-relational mapping tool when your business's core competencies are providing travel reservations? Maybe, but I doubt it. In my experience, 99.99% of the time, it is a fundamental diversion. Even if you can quickly build that technology to help out your development effort, the fact remains that 90% of the costs of building the tools will be spent maintaining them, which is unlikely to be within the core competency of the firm, and certainly not within its business objectives. The costs are never truly estimated from the beginning, and rarely accounted for over time. The irony of all this? As people come and go from projects that span years, everybody gets to spend a lot more time looking at, using and maintaining other people's stuff.

Perhaps this MasterCard moment sums it best:

That DVD box set of '60s Star Trek re-runs you've been dying to get? 120 dollars.

A mnemonic memory circuit? 150 million dollars.

The cost of building your own software development tools? Priceless.

What was that quote Edsger?

2005-03-09T15:07:00.000-08:00

It's a concept that has been applied in various disciplines, from psychology to linguistics, and I seem to recall someone like Dijkstra coming up with a similarly pithy aphorism in computer science, but alas, I can't recall the quote exactly at this point. It's something along the lines of "The language you speak will dictate the problems you can solve." Perhaps it's a bit of a stretch, or maybe just a feeble excuse for my own ignorance, but I'll chalk the following frustrating experience up to not being able to step out of the preverbal box, so to speak.

I was trying to set up my Motorola V300 phone via a USB data cable and mobile Phone Tools V3.0 to my ThinkPad running Windows XP SP2. If you want to see how my predecessors/peers have made out with this, just search for some of these keywords or a phrase like "mobile phone tools problem" and have a look at the results. I installed the software and like seemingly so many unsuspecting users before me, mPT comes up with "Initializing ..." and nothing more. I tried everything. Uninstalling, re-installing, looking for alternate drivers, hours surfing the web only to find just about everyone and their dog went through the same thing with this combination, with no resolutions. Pretty frustrating. The ubiquitousness of this experience, from what I could gather from the mobile phone forums, led me to my first conclusion, "It's the software that just doesnâ€™t work."

After fiddling for hours with all this, taking any and all suggestions, tinkering endlessly with drivers, finally Windows recognized my V300 as a modem, and I could talk to it via a terminal window. For all you sufferers out there, I wish I could post something more concrete about the fix here, but by the end of this journey my frustration level was so high that my resulting approach was less, shall I say, than scientific. In any event I can no longer retrace the steps I took to get Windows to recognize the phone. Sorry about that.

Back to the story. I had my phone connected to the laptop, and the terminal window open. "Windows can see it", I thought. "I can type the old Hayes modem command set at it, and it responds!", I thought. "It looks like a modem, it must be a modem", or so I thought -- mistakenly thinking that the feedback I was getting from the phone cum modem meant that I could get it to dial my ISP and I would want it to dial my ISP. No luck. As I was playing with this for the third day in a row, a friend happened to YIM me and I described the problem. "Windows can see it, I can talk to it like a Hayes modem, but the #@^!%%#!#@ thing just won't dial!". He replied "It's digital, you can't use an analog modem command. Your phone is already connected to the Internet." That was the 'big a-ha' I needed. Of course itâ€™s digital. Of course it's connected -- I can surf from the phone. duh. I immediately contacted my phone provider and they gave me the GPRS settings for their network. I clicked them into the Windows network connection dialog on the laptop and presto, I was online via my V300. This experience served as a fresh reminder that the paradigm one brings to the table will affect, if not preclude, attempts to solve a problem. The moral of this story? A second set of eyes and ears can help break that 'lock' and allow you to approach things from a fresh perspective. Hopefully, readers will be kind and allow me the excuse that had the modem provided better feedback, I would not have wasted so much precious time, but somehow? I doubt it.

Now, it's off to the cottage to go surfin' ... wireless style. TTFN.

Why Enterprise Projects Fail, or Pitfalls in Practice

2005-02-27T15:04:00.000-08:00

In one of those water-cooler moments at the office, a colleague of mine asked me "what percentage of enterprise scale projects fail and why?". Upon reflection, considering the scope of the last several adventures in which I had a part or observed from the outside, I had to answer "all of them, at least by some measure." The subsequent conversation prompted me to summarize here why, in my view, this is so.

The Top Ten Reasons Why Enterprise Projects Fail (sorry Dave, but I'm not the first or last to cop this one):

10. Inappropriate Project Scope or Trying to Hit a Moving Target. When the constraints and objectives of the project are too ambitious or the scope incrementally creeps larger over time, the problem is ill-defined. The project scope and deliverables must be clearly defined in advance, or incrementally refined through prototyping and actively managed along the way with timely feedback from business sponsors.

9. "Not Invented Here" Syndrome. Often projects start with a review of applicable vendor based solutions. Frequently, the outcome of these system evaluations are stacked against vendors from the beginning. Software developers have a disappointing predisposition to prefer to build solutions from scratch rather than buy a third-party product or work with a vendor to adapt an existing commercial solution, thereby ignoring the real maintenance costs of homegrown work. If a third-party solution matches or closely matches the goals of your project it is invariably cheaper and faster to buy versus build once the true project costs are considered. If your project is big enough, most vendors will work with you to refine their offerings to accommodate your needs.

8. Loss of Political Sponsorship. Projects loose their champions when their expectations are not properly managed and the timeline/budget is not met. Stay visible and stay employed by delivering results early. It is critical to under-promise and over deliver.

7. Project Risks Not Properly Assessed. Development efforts, unlike engineering projects in other disciplines, always uncover unknowns as the project progresses and must fluidly accommodate unforeseen setbacks. The closer to the bleeding edge your project is, the more certain it is that serious setbacks will occur.

6. Inadequate Resourcing. Every carpenter knows you need the right tools for the job. Enterprise projects require the right people in the right roles from inception to completion. Management needs to set priorities, schedules and expectations and run air cover so that technical resources can do their jobs effectively. Developers need to have the discipline, experience and abilities necessary to complete project milestones on time, overcoming the many obstacles on the way.

5. Fear, Uncertainty and Doubt. Other players can and will influence project sponsorship for their own purposes. Count on it happening if budgets are large, spreading FUD is a key weapon in the arsenal of external consultancies who routinely attempt to undermine project management's credibility and subsequently 'correct' the project course by restaffing it with their own people for their own financial gain. It works because it is self fulfilling. This technique is also a favorite of other internal department managers who want to appropriate your budget for their priorities -- after all you are all competing for a piece of a finite pie. The bigger the budget, the bigger the potential target your project funds are. It may not be right, but it happens all the time.

4. Not Knowing When to Cut Your Losses. Fear of failure often motivates throwing good money after bad and making a bad situation much, much worse. If a milestone looks doomed to fail, forget your pride, re-tool, work proactively for the good of the business to recover as many positive benefits of the effort as possible and constructively move on with an alternative approach.

3. Inability to Overcome Technical Roadblocks. Often related to number 6 above, unstructured teams, inappropriate use of methodology or failing to correctly use best practices will exaggerate the chance of technical failure because the team cannot apply the rigor required to properly formulate a solution and move forward. After all, software development is both a team and a contact sport, so it is important to choose the development methodology that best matches project objectives and best facilitates inter-player communication.

2. Impedance Mismatch. Organizational structures rarely match project responsibilities. Whenever functional reporting lines do not line up with project objectives you can bet the motivation of others to deliver are diminished, often crippling the entire effort. Think about it, if you need that guy in London to deliver something to your New York development team and he does not report to you, your priorities will understandably be at the bottom of his activity list. After all, he's a busy guy with his own problems and he is not organizationally motivated to respond to your needs.

And the number one reason why large-scale projects fail?

1. It is a Government Project. There are countless examples, governments just canâ€™t responsibly spend taxpayers money or run a development project. A poster child of project mismanagement and arguably exhibiting the previous nine characteristics, perhaps the most stinging of recent examples is the Canadian Gun Registry, initiated by the Liberal Government of Canada to require gun owners to register all guns in the country. In spite of the fact that the whole idea is one of the most ill-conceived notions one could imagine (after all, crooks donâ€™t register guns), the Liberal government initially budgeted the cost to taxpayers at a net outlay of 2 million dollars. Still incomplete, the costs are now over an astonishing one billion dollars and rising, in a country with one tenth the population of the United States. Whatever your position on guns, whether you believe registration is an effective crime prevention tactic or not, a database development effort should not cost a billion+ dollars. In other words, when political ideology hijacks the project agenda your spidey sense should tingle because project funds are being bailed, taken out back and lit ablaze. Everybody looses.

I am a big believer in incremental development as a tool to help manage project expectations and mitigate risks. Following the old 80-20 rule, incremental development plans can, through rapid prototyping and continual refactoring, help refine project requirements and show visible results early. This can really help to out maneuver the FUD slingers and give your project champions needed confidence that you and your team will deliver as promised. Look back over the last firm-wide development project you were on. In spite of the fact that your efforts may have been considered successful and valuable business benefits were realized, it is highly improbable that the complete vision of the original project was achieved, within the promised timeline and budget. When you come to think of it, we all decry the fallibility of the weatherman, and scoff at the inaccuracy of the previous day's forecast. Next time you're deriding your local meteorologist, consider he's at least an order of magnitude better in delivering on his promises than we are.

Kids, don't try this at home ... you'll only make it more difficult to do your homework.

2005-02-11T14:59:00.000-08:00

As a maintainer of several web sites and as someone who depends on the web as a tool of the trade, I take abusers seriously. Search engines have a tough job indexing the web (after all, software canâ€™t read), just so we can all find relevant content when we visit our favorite search engine and type a phrase of interest. We've all experienced it -- search engines often index pages badly in a vain attempt to make up for the fact that software can't read, injecting a lot of noise into search results, often frustrating Joe-Q-Surfer to no end. The seasoned among us learn to ignore this noise, but it is still there, boolean logic aside. An accomplished webmaster might painstakingly march their home page up to the top of the preverbal charts by carefully crafting the titles, headings, meta-tags and link text, not to mention providing valued content, just so we can get to it when we need to, only to find their web site might as well have a target at it's URL.

While the search engine authors devise ever-more clever methods for connecting phrases to pages and thwarting the posers, there are those out there that are working even harder to undermine the whole thing. You kinda expect this from those cheesy doorway sites that use all the tricks they can think of to attract visitors, unaccomplished typists and unsuspecting web lurkers alike, with the most common of typos and misspelled URLs. But how's this for a shrewd little trick? Find the top ranking page for a search phrase, for example "dog training", then copy a key phrase out of that page, say it's "Labrador Retriever training tips" then generate a URL with all those words in it like this "http://www.somesite.com/dog-training-labrador-retriever-tips.jsp" then host that URL on a web server that coughs up a page containing all the same words in titles, hidden and alt text, and tosses in a lot of links under those words to the snake oil du jour, which, by the way, typically has absolutely nothing to do with dogs.

It's hard to imagine a mechanism that would work better to usurp the place of the top ranking page so easily. Clever yes, I'll give them that. How does this happen? Simply put, it goes something like this: a targeted search engine might rate the content of a page by trying to determine the meaning of it (I know, I know, software canâ€™t read), by applying some simple, and sometimes some very complex rules. For example, a rule might assign greater rank to phrases in titles, assuming titles are more important than those phrases lower down on a page, so the page must be about that topic, right? The text in or under a hyperlink might also be an indicator of that phrase's importance. In practice, of course, the rules used are much more imaginative than this, but you get the idea. These ranking rules can be applied by engines to determine the correlation between phrases and pages and they thereby determine what you see when you type that phrase while searching the web. I often think a return to some much more simple principles here would help this situation immensely, but that is another topic I suppose.

Back to our example above -- the creators of the masquerading page have reverse engineered the rules the engine in question uses to index the web, and have auto-generated a near-perfect rule match effectively knocking the real page off the podium. All this is guaranteed to take the web searcher away from the page they are really looking for once the 'trick' page itself is indexed by the same engine. You can bet the end result is ever more inaccurate searches, making it increasingly difficult to locate what you're looking for in one or two clicks.

These guys are not only breaking every principle of copyright law, they are wasting our time, and are no better than the e-mail spammers that siphon your email address off the internet, then, with unbelievable audacity, hock their wares in a seemingly never ending torrent of bandwidth robbing prattle with YOUR address in the "from" field. I mean really! ... what if your eighty year old dad got that tip, supposedly from you, about those "Pen1s enlarg3ment p1lls" he can order online. The lowest form of pond scum. Flame off.

Think globally, optimize locally ...

2005-02-09T15:02:00.000-08:00

An early advocate of applying multi-media in computer based training, back in 1984 I started a company developing educational software for use in elementary schools. Perhaps if it was 1994, I would have a lot more nickels in the old retirement jar, but alas, I digress. Back then, we had a whopping 1Meg of RAM to work with on an 80186 architecture, 64K total for process code and data, running a quasi-real time operating system. We had to squeeze a lot of functionality into a tiny space, and performance was always, always, always an issue. If you wrote brilliantly clever, small and tight code you were the envy of all, or at the very at least, a team hero, if only for a day or two until the next beautifully crafted 'while' loop came along, computing the world with the utmost brevity. During this time, I was proud of my ability to shave off a few nibbles here and there in a data structure, tighten up the performance of a loop or two, or a cull a few bytes out of the code segment of the applications I was developing. Those old lessons and skills can die hard in a world that redefines the playing field with, by comparison, a super computer in everyone's handheld.

Not surprisingly, I have come respect engineers that design measurably efficient code, and encapsulate areas that require further analysis under strict APIs. The rest, as they say, is an implementation detail. If I had a nickel for every time I heard a developer announce in an epiphany of clairvoyance "That method will degrade performance!", or something similar, perhaps I would have found another way to retirement by now. This objection comes in all forms too, from the obdurate developer that proclaims "Wait, that function won't be fast enough", to the academically minded one that pontificates "You've got an order N-squared algorithm there. We'd better spend the time to re-formulate it or the system will be slow."

In practice, I have seen even the best developers exhibit as much accuracy in foretelling the bottlenecks of a system in advance of rigorous measurement as my brother-in-law's pet cat might show in predicting the weather a few years hence. Wherever you suspect the time is going, is rarely, if ever, where time is going. With experience, programmers become skilled at first understanding performance requirements for an application, then designing efficient solutions through the application of tried and true patterns, and profiling out performance problem areas through instrumentation and meticulous calibration.

The thing is, 'fast enough' is all relative. To use our example N squared algorithm above, if N is small, there will be no impact on system throughput. If N is large, and the implementation encapsulated, our method can be written quickly and further instrumented so the true impact on throughput can be observed. If, and only if, the code is on the critical performance path, then, and only then it can be optimized with zero negative impact on the rest of the system.

To make matters worse in all this, poor software designs are never made better solely through optimization. And it gets worse yet. Performance enhancements can often obfuscate a good design, especially if the optimizations are not localized. This makes code harder to develop, maintain and ultimately more expensive with no demonstrable positive impact on system performance. I don't know who said this first, but it's a great one-liner, "It is easier to optimize correct code than to correct optimized code".

So, next time we're in development meeting, come with your profiler output clearly demonstrating the top five performance hits on the system and I'd be happy to go over it with you. Otherwise, resist the temptation to declare the likes of "but that is waaaay to slow", and I'll resist the urge to quote Hoare: "Premature optimization is the root of all evil.". Either that, or come with a fistful of nickels. Peace.

if henry could see us now ...

2005-01-23T14:56:00.000-08:00

I have been working with C#/.NET for a couple of years now, and have come to really like what Microsoft has accomplished with the platform, even when compared with JAVA. Some colleagues of mine would choke up a lung to hear me say this now. Don't get me wrong, I was a JAVA developer, promoter and evangelist long before I got involved with C#, but there are some things I like more about C#, even if it is only syntactic sugar or based on a subjective esthetic judgement. Take Attributes, for example. You can use interfaces to do the same thing in other languages like JAVA, but it's harder to get the same code re-use and you typically have to write more code to get similar results. I also agree with the approach of designing the features of a language to limit behavior by default -- things are private until made public, and so forth. That is a good thing. However, since most things proceed along the path of least resistance, better development tools are those that make it easier to do the right thing in the first place. So, I ask this -- when would you not want a class to be imbued with the [Serializable] attribute? Just about every object will need to be passed over the wire if it ends up part of a non-trivial application at some point in its lifetime. Leave it out by mistake and you catch it sometime later in the debugger, but that costs time. It's actually hard to enumerate cases where you definitively know that an object will never need to be serialized. For this reason some languages make a special case of [Serializable] and make that the default. You should expect to work to explicitly disable it, since that's the exception, not the rule. It's not a big deal, just a simple productivity issue. If Henry Ford (a master at optimizing worker productivity) was monitoring software developers he would have cut out the need to type these fourteen [Serializable] characters on the assembly line day one and made every part come with the right pieces before it hit his factory yard.