On the Heap

A brief tale of software developer burnout:
Brandon sits down at his two year old HP workstation and scribbles his mouse across the plastic-laminate desktop.  The twin monitors snap to attention and he logs in without thinking about it; finger memory taking over long ago.  Visual Studio is already open, it’s been for the last month, because the project continues onward like warships of a 1984 Orwellian nightmare.

He’s worked at the same project for the last 11 months and management tells him the end is in sight.  His inbox describes an alternate view:
Defect #234 -New- Wrong font color in IE 6
Defect #188 – Reopened- Pricing is incorrect

Brandon’s usually up beat, but he has seen nothing but build reports and defects for months. His stake-holders haven’t even been reviewing key features – they’re waiting for the magic release where all feature are included and everything works according to a document that some managers have signed.

Starting QA after 10 moths didn’t feel like a success.  Nor does closing 30 defects in two weeks.  And now, as the winter starts, he can only look forward to shipping the 50,000 line project on a cold weekend.  It will feel good. Right? To be done with it.  Right? Wash his hands.  Move on.  To the next…project…

This isn’t an unusual case.  Even with supposed agile practices catching on, organizations like to hold scrums for the development team but the stake-holders choose to wait until it’s complete.  Even with the idea of software being complete an oddity and undefinable end point.  Yet, they’ll wait.

This outlook on software development has caused the most pain for the Brandon’s of the industry because there is a lack of incremental satisfaction.

Nassim Nicholas Taleb sites the idea of hedonic happiness in his book, the Black Swan, to examine the idea of hope in light of big singular rewards.  Meaning, many small rewards have a much greater effect on our overall happiness then one or two big ones very far apart.  This is called positive affect.  Taleb’s example is financially driven, but excellent.  Simply, a person will be happier making $100,000 a year as opposed to making $1,000,000 every ten years.  The amount of tension and angst for those ten long years can not be recouped with the big payout – especially if it might be another ten years until it happens again.  As humans, we’re simply not wired up like that.

This fits perfectly into the not so modern realm of software development that 90% of organizations participate in.  Remember Brandon waited 11 months before a release.  He waited 10 months until someone would even begin verifying his work.  Therefore, there were 10 months of effort without any reward.  If positive affect is indeed a true nature of humanity, then long development cycles are like a 30 story ledge that developers stand on crying week to week while looking at the cars below.

The Orwellian similitude’s carry on.  The ruling “Party” of 1984 used doublespeek as a way to control the masses;  simultaneously accepting as true two mutually contradictory beliefs.  I’ve see this in places I’ve worked – and I’m sure I’m not alone.  For example, the requirement is to be agile and to release all features at once.  To do incremental work, but not test until all increments are complete.  To look at iteration plans but not really care unless a feature might not make it at release day.

Are these contradictory? Indeed.  Are they agile.  Of course not.  We don’t have to wait for a solution.  It’s already been invented and bigger organizations can take advantage of it – if they will invest in a mindset change that iterations of the product and each one should be complete.  It’s going to take more training and experience for a stakeholder to really understand that an iteration is planned to be a deployable piece of work and not the subgrouping of tasks on a Gantt chart.

I won’t belabor the incremental, agile, continuous integration points.  The amount of literature and case studies is staggering. However, I’ll continue to make a case for the Brandon’s of the world.  Give Brandon some wins.  Schedule QA of the first iteration (even two iterations) of the project so that it’s in a deployable state. Then, and this is acrimonious to some, release it.  Actually, physically release it.  Then start the next set of features.  Help our Brandon’s feel the little frequent successes that nature makes us crave and then watch the productivity of a motivated software team soar.  Use positive affect to the advantage.

George Orwell’s 1984:
“To tell deliberate lies while genuinely believing in them, to forget any fact that has become inconvenient, and then, when it becomes necessary again, to draw it back from oblivion for just so long as it is needed, to deny the existence of objective reality and all the while to take account of the reality which one denies”

The term enterprise is consistently shot across conference tables  inside corporate IT and technology marketing departments. The term is most often ascribed to software and hardware systems running the core business; a reservation system for a hotel franchise, or a software package for payroll.  However, this attribution excludes all the software that runs at smaller companies.  So, what makes a system enterprise ready and which products are left looking in at the glowing warmth of Fortune 1000 server rooms while they desiccate.

Sadly, there isn’t much empirical evidence separating an enterprise ready app from any other piece of software.  A development manager requiring an enterprise developer doesn’t make the role anymore real than an engineer claiming to be a “people person”.  An HP blade server is no more enterprise ready being racked in a enormous corporate data center than in a startup’s cage at RackSpace.  Let’s dissect this hypothesis by looking at two very popular web frameworks: ASP.NET and Ruby on Rails.

Microsoft sits in the interesting position of selling the same software that runs small businesses and large enterprises.  This of itself should be enough to prove the term enterprise a loaded marking term – but let’s keep going: ASP.NET is Microsoft’s enterprise web platform.  It, coupled with IIS, provide everything needed to build dynamic multitiered web applications.  It has optimizations for Microsoft SQL Server, session and cache providers, error reporting, performance tuning, remote configuration, etc.  It can do just about anything a web application in 2008 would require.  Again, Microsoft, pundits and analysts say it is enterprise capable.

On the other extreme is Ruby on Rails.  Note, not just Ruby the programming language, but the Rails web framework as well.  What I’m embarking on is dangerous:  I am not a Rails expert and provide this caveat up front.  Rails is considered by popular opinion to not be enterprise ready.  What does it lack?  As far as I can tell, the only thing it really lacks is a support package, IDE and hordes of corporate programmers.

On it’s own, ROR is terribly productive.  The Active Record design pattern is built right in as a OR Mapper with the friendly and obvious name of “Active Record”.  So, it provides a concise model of objects over data right out of the box (if anything had a box these days).  The framework is based off the of tried and true MVC, model-view-controller, design pattern to provide distinction between data, middle tier and presentation.  This is a simple programming idiom used with GUI development, web applications or the myriad of systems that actually make’s power arrive to substations to power my MacBook (Pro, thank you very much).

I  provide these contrasts between the frameworks to illustrate the futility of using the term enterprise for one group of software products but not for another.  Because it gets better.  Bigotry of platforms like ROR grows as corporate IT software developers feel more threatened.  Hence their managers feel threatened and when Forester Research shows up all they hear is how platforms like ROR are not enterprise ready.  But it’s not a technology stack that is not ready for development in these corporations.  It’s the people.

In the end it’s that simple.  When people are not ready they will search for whatever means to prove their disaffection, dissatisfaction or ignorance about a product to be real when it’s empirically false.  This is why the enterprise is difficult to define: because it’s a constantly moving target through the emotions and minds of corporate software developers.

Somewhere in an IT department right now a team of people are choosing a software vendor. This vendor could be for just about anything in the corporate IT sphere. You can imagine the usual roundup: analysis packages, data providers, software components, UI controls – the list goes on and on nowadays. The people in the room are bright hard workers. And the problem they are solving is varied. Like how can the project succeed with the minimum work from in house staff or how can it be done without breaking the bank. And finally, wiping the sweat from the brow and mustering up the old college fighting spirit – how to deliver the best possible product.

Does this sound familiar? Bueler? Bueler?

These goals are altruistic and, some would say, commendable, but they aren’t always realistic. Why not? Because choosing a vendor is too often the default option when business is faced with a problem these days. And when that occurs, the We Could Have Built It Already principle is at work; otherwise known as WCHBIA (pronounced wa-chee-ba). In a nutshell, it’s a simple way to call out the business analysts, non-technical managers, technically challenged managers and the technical, but wet blanket managers, from wasting time on vendor product selection for systems that can be and more importantly should be built in house. Really, this is in hope of preventing Yet Another Integration Project, which we’ll touch on later. Don’t read this as all products should be built in house – they shouldn’t.

You can see WCHBIA all around you if you look close enough. It’s the multitiered, web front end, Ajax enabled, 14 pt font Web 2.0 web analytics package that tells you conversions of a shopping cart. It’s the “if you bought this, you’ll like this” recommendation platforms. It’s the five thousand dollar site licensed UI components that don’t scale on server farms. You’ve seen it or experienced it by now. Heck, if you work in corporate IT you’ve most likely integrated some third party (system, component, service, data feed).

Luckily there are ways to avoid WCHBIA. Let’s look at them.

–The Real Purchase–
Most vendors aren’t just offering product X, they are offering a solution. This solution involves downloads, installs, setup,configuration, documentation, maybe some scripts and a plethora of meetings. And that’s just for the technical staff. Project managers, lawyers, directors (up to CTO often) get involved because at the end of the day this is a capital investment with contracts to read and negotiate. This is a lot of people and consequently and lot of cycles at work across teams. Are you analyzing this time before heading down the vendor selection path? Have you really considered who the “third” in third-party means. Are you willing to actually estimate these hours? After all, multiple vendors will be investigated and probably tested in house which is an immediate time investment for the internal staff.

–Revenue Producing–
Next, is this system critical to your company? Is it used on a revenue producing product? Will it give your platform a leg-up? If you answer yes to these questions (or even two out of three) that is a big red flag to pay very close attention and tread carefully. Vendors go bankrupt and close down. They are sold and resold. They have their own internal priorities (and strife). They are very rarely plug-and-play no matter what Chaz, their outside sales guy, tells you.

At the end of the day, do you trust SimpleSoft LLC’s product to reliably be part of your revenue producing system. Do you trust it to fund payroll? And if this thing is so great, aren’t your prime competitors already using it or going to deploy it as well? Where is the market advantage in that? That’s just keeping an even playing field and not leveraging technology for market advantage.  You won’t drown, but you won’t swim to shore either.

–Investing in Your Team–
There are significant advantages to investing in and improving a software platform. Solving the problem at hand is a key advantage. Although the vendor wants to sell a solution what’s usually required is a product – which is just software. So why even test drive a Ferrari when what you need is a Volvo?

Next, outsourcing interesting and challenging products to a vendor is a really good way to homogenize your development staff. It sends a signal that they are good at building houses but not skyscrapers. Sometimes this goes as far as turning them into glorified UI designers, that is, before the real UI designers are brought in. Management should know if their staff is up to building important revenue producing pieces of the platform. Most developers I know drool for the chance. That’s why they come to work. To build cool things and make an impact. It takes the same level of commitment to requirements and limiting scope – but if the focus is on building a functional product that fills the void it’s a win-win.

If you’re thinking that your teams software developers don’t want to build cool products and be challenged then you probably haven’t spent much time talking with them or listening to their conversations. If you know for an absolute fact that they would rather not build high impact features into your platform or deliver differentiating features to clients then you might have an unfortunate favor of “career” programmers on your team. And if that’s the case…there are other rivers your team will need to cross.

–YAIP–
Constantly looking to vendors is demoralizing for developers. If this is so important to spend 80, 100, 200K on why is it not entrusted to the development team to accomplish? Frankly, an investment in people is clearly lacking here and as outlined in the seminal book PeopleWare, this is as good a path as any toward Teamicide. Somewhere along the line managers have grown the thought in their head that software developers prefer integrating vendor software instead of writing software. Poll 100 developers and I’d hazard to guess that less than 10% of them say they’d like to spend their career integrating vendor solutions. This is where the term Yet Another Integration Project comes from; the constant churn of them in corporate IT software departments.

The churn of integration projects is justified by a claim to cost savings. The general thinking is that it would cost an in house team more to *build system X than it would to purchase it. But looking at the number of people involved, testing time, up front purchase cost, integration cost and maintenance cost of the “solution” – it’s tough to say it’s an accurate justification.

–Antipode Acronyms–
Remember when everyone was afraid of another acronym? NIH; Not Invented Here. For a time managers feared systems that were so complex that it couldn’t be maintained and someone would have to put a deugger on the equality operator because someone modified it’s behavior. That would be a terror to any reasonably sane person. But now the industry has swung hard the exact opposite way. Now, the fear should be systems built around vendor solutions and the lack of control. Who controls the roadmap for SuperMega EverywhereCache? SimploSoft LLC does, not you.

–Conclusion–
While looking for a vendor, testing their products and having lawyers read contracts, a decent set of software developers could often have built the technology in the same amount of time. Or maybe a little more time. It’s still software after all and integration projects are just as suspect to go dangerously off course as starting with a compiler. And given that developers (or system designers, architects) are roped into these vendor dealings, the time could simply be allocated towards building a working product. Basically, design and prototype sessions. The kind of things that really talented engineers want to do. In fact, the kind of thing that they were hired to do.

*If you work on a big CRM package and read this thinking “awesome, roll my own”, don’t show your employer this part of the article. Your type of software is what vendors are perfect for; unless your backoffice operations will generate more revenue or market share for your company. And if that’s the case, you are indeed in a very interesting industry.