Favorite Quotes from this Years Dr. Dobbs Architecture and Design World
Comments (3)I had the opportunity to attend a panel discussion with Scott Ambler (Mr. Agile), Neal Ford (all around guru) and Robert C. Martin (Clean Code). The topic: Dynamic vs Static languages!!
The following is a terse summary of the panel. Use dynamic languages where possible and where appropriate. Where is appropriate? Depends. Most places. Do not fall down the trap that static languages, like C# or Java, will save the world. It’s been over 10 years; end of story. There isn’t a single general purpose to solve everything. Java and C# are “ceremonious”. Studies show that dynamic languages make developers 5x more productive. Dynamic languages will not work without TDD and unit testing. Statically typed run times like the JVM and CLR are ripe playgrounds for deploying dynamic languages on existing infrastructure. Framework libraries are important, but with bloat like .NET, it does not mean a developer will be more productive; what is the one way to read a file in .NET?
A lot of the questions thrown at the panel were similar to “why should I use dynamic languages” or “better than C++? Outrageous!” In my opinion, it’s not the responsibility of those in the know to defend dynamic languages to the rest of the software world. It’s that big, slow, unchanging side of software development that needs to do their own research and understand the benefits of the dynamic language conversation.
Now, a few of my favorite quotes:
Martin: “Is COBOL statically typed?”
Amber: “Sure.”
Ford: “If you only learned Java in school and are put on a C++ project you will blow up the universe.”
Ford or Martin: “Bad developers will move heaven and earth to write bad code.”
Ambler: “IT people screw around all the time. That’s what we do. We don’t act like professionals and aren’t respected.”
And, finally, my favorite quote is from Scott Ambler:
“There is some fantastic research from the 70’s that shows testing at the end [of the project] works really well.”
On the Failure to Build Tools
There is a misconception in non-product software development around getting things done; where accomplishing a given task only once is acceptable. The developers who fall prey to this most often are the ones who have never seen a unit test and whom “mocking” is akin to Daily Show jokes. I’m neither an expert at unit testing and couldn’t explain how to use a library like RhinoMocks properly if my feet were to the fire…it’s just an observation. However, getting things done for that one release, for that one build, for that one functional test is common in far too many shops. Mine is no different.
The failure to build tools isn’t just around software testing. It pervades other areas of the application life cycle; early research into a project, or, say, data transformation – reproducing a database is a big one. Other failures to build tools are build scripts, configuration repositories, post-build events, cache validators, deploy validators, version checking… and the list goes on. I won’t harp on the build process – I’ve done a lot of that in the past. However, the actual release process – that is another ball of wax, or, more like the soft underbelly of the software life cycle.
All but the simplest of applications require some level of configuration management. So, if you click and drag your way to better software the rest of this essay might bore the pants of you. Or if you write Visual Basic. That’s just a straight jab.
Applications are more than their code since any significant application written for business in the last 20+ years uses to some kind of data source. And that data source, usually, is not the same for the developers or testers as it is for the end user. Hence, this variable data needs to be stored somewhere. Simple enough then. Put it on disk. Somewhere. Modify it….at some point? The question is when and how – which leads to the the primary paradox of building or not building tools I’ll address.
As software developers, we are often faced with the situation to write a script to parse 20 odd lines of some odd piece of data. Additionally we often just need the data once, loaded up, or put into a CSV file. The angel on our right shoulder tells us to write a Python script to automate it so we can do it again tomorrow if necessary. The little red guy with horns on our left shoulder tells us to do it by hand; just crank it out, as we’ll be done in 30 minutes and can go home and play World of Warcraft or watch American Idol. Mr. Red doesn’t remind us that the last thousand times we cranked it out by hand it didn’t work out and we had to do it by hand a dozen more times. Hence, with limited cognitive abilities at our disposal, as we just write software not make cheese, we go with the choice of leisure: do it by hand.
If professional software developers, being us, often make the choice not to build the simplest of scripts for tooling, what do you think occurs with the folks in IT/QA/Systems? If you’re first answer is they find a big expensive software package that will do this for twice the hassle of everyone involved, you are right! However, as this is universally accepted and widely documented let us examine the second most obvious choice. Yes, they will hack out whatever it is that needs doing by hand.
Spending time with someone in IT working in a Windows environment is like must see TV. I consider it a cross of Lost and reruns of Friends. There is constant, really constant, clicking around. Click this OPEN, click that CLOSE! Drag files. Stop. Reset service. Click, click, click, click – ad infinity. If you are fortunate to watch these activities in the wild, like Lost, you’ll wonder what the heck is going on, and like the Friends reruns, feel like so much time is being wasted.
A software release process can get rather complicated; especially on the web. On the web, inevitably you’ll end up deploying, at the minimum, hundreds of files. In lots of cases it’s actually thousands of files. This is simply the nature of the web. Along with these application files, and their resources (html, images, video, pdfs) are their configuration files. And the configuration files required for the production release is not the same as the configuration files in your QA environment – still sound so simple?
Ok, probably in theory. But in practice it’s different, and when someone walks through the process and arrives at this point they understand the complexity, timing, and work required to pull off a release. But, sadly, the reason they see this is because there isn’t a tool built to abstract this away. A nice GUI tool for IT to use. And, when that doesn’t happen, it’s time for software developers to step up and take over. Because the failure to build tools isn’t with IT – it’s with us.
The Web Is Not Real; Web Applications Are Not Web Sites
I’ve decided that treating the web like it’s made up of HTTP requests, headers, images and markup is a bad idea. That the web, in all it’s HTML and javascript glory should not be treated as such. Instead, it should be treated as a big, fat, hairy delivery mechanism. Something that needs taming and requires a high level of abstraction to build anything useful on it. It’s like writing Win32 code to run on an old Commodore (which I’m sure someone has). Your HTML will never be in a native environment.
I’ve worked on the web for a long time. Not as long as some, but a good amount of time considering that we’re only about 13 years in. I started in HTML. Then DHTML and Javascript. Then VB Script with ASP. Then finally, ASP.NET, and where I’m currently treading water, with ASP.NET + CSS. Each of these progressions brought more sophistication to the “we’re all connected” yet annoyingly disconnected web. Static moves towards scripting, then on towards compilation then towards web UI control sets and finally CSS based layouts thrown on top. And with that, I assert the following statements to be false: over the years programming for the web has really become simpler. Our standard libraries make it easy to make applications run identically across the browser “environment” and across platforms. The web experience performs akin to a cougar chasing it’s lunch. We’re caching all our images on the client with a push of a button and decreased our time-to-last-byte counters on the server. I can’t see how any of these statement can be claimed for someone, or company, with out spoonful hubris. The web is the same promise of Java….write once, run anywhere. Yet, like Java, it can run anywhere broken, slow or unusable.
How many man hours have been spent across the world of web development trying to get some HTML to render properly in Internet Explorer? What could we have done with those computing cycles? What could they have done with their time? And it’s not just IE – Firefox is just as much a culprit, because as some have pointed out, IE became something of a standard by sheer market share – so the community programmed to it.
Now, let’s move beyond the simple web page example. Everyone knows that tweaking is a fact of life to get things working in a browser. The thing is, a whole cottage industry of consultants, designers and websites thrives off of it: A List Apart, CSS Beauty, etc. They boast the beauty and elegance of shiny CSS and the taming of the shrew; a.ka. HTML. Now you can move your presentation out of your presentation. What? To me, that’s appears to be the intent. Sure, I like using CSS to style, position and provide reuse, but it’s just moving the bits over a little. Skinning? Who does that?
But it’s not that simple. I got caught up in the sordidness. In the slickness. In the elegance. In the load of horse maneur I was being fed by reading web design sites and spending time with designers whose personal sites looked so cool and made me long to be just-like-them (wow, your style sheet is so organized!). I stood up for the purity of our HTML and for simple, open javascript libraries. I crowed about div’s and easy to read CSS files. I became the one cleaning the stables; shoveling the maneur.
And it’s not easy either. Consider the classic N-tier model: data layer, data access layer, business logic layer and presentation layer. This has been the thought pattern across most business applications for the last decade. Can we separate all these layers cleanly? At the end of the day can these layers really not know anything about each other? Nothing? MVC, as old (sorry, classic) a pattern as it is, won’t help when a business rule you never saw coming shows up. Or, some performance tweak must happen at the data access layer that is really implementing a tiny bit a business rule somewhere. How has our luck been with this? Is this even OOP? I don’t know what it is, exactly, but it’s not impressive.
Now, consider the presentation layer: your application needs to look a little more blue for some customers than others. That’s the requirement. So, where is this going to occur. You could apply a whole different style sheet with a few elements with a higher blue saturation. That would do it. But then you have duplication. You could heavily modularize your CSS files and make judicious use of “@import blue.css”. But now you don’t so much have an applicable style sheet(s) as you have style modules that are pieced together on the fly by the application or composed into more style sheets by a designer or developer. That doesn’t sound to bad to me – but I’m a software developer. But how would that sound to the UI designer/developer? Or, you application can replace the style for element A with the more blue style. My point, if you’re not catching it already, is that there isn’t and can’t be a clean separation of layers. That’s just not how the world works. Even in the food chain, an ant eater may eat ants, but those ants may have been surviving off ant eater excrement. There is a connection – maybe not a pure symbiosis, but it exists. You’re stored procedures are connected to your business logic in one way or another and your CSS is in cahoots with your HTML and (gasp!) your script/compiled code. That’s life.
On the flip side are the folks at all the web design shops whose biggest output is “sweet” CSS with mirrored background images that they show off to other designers and brag how it works in Safari. These are the marketing, branding and product sites. They aren’t concerned with reuse. They aren’t concerned about skinning. To them, CSS is a very flexible way for them to think and build web pages. It let’s them focus on making their site look as slick, as gentle, as in your face or as required by their client as possible. That is the non application side of using CSS. And it’s very different than what I, or the folks I associate with deal with.
Now, if we accept the hypothesis that clean separation of concerns is not attainable, if even just for arguments sake, then what is the course of action? That’s simple: don’t worry about it. This is probably enough to make the most staid of personalities somewhat indignant. But yes, the simplest answer is to not worry about it, or better stated, don’t worry about the details. Build your applications at such a level that your developers can worry about concepts not rendering. So that they don’t constantly have to make decisions on whether to use a span, a div, or a list (in horizontal mode none-the-less). Build libraries that render exactly the HTML you want and not what the web design community says you should have. It’s a different application of CSS or HTML or the web in general really. One of the reasons you shouldn’t have to worry about every detail of HTML and CSS as you create a new page in a web application is because the page doesn’t exist anyway.
I’m not saying there “is no spoon” (i.e. the Matrix), in that you can wish the page away. Instead, look back towards the origins of the web and our good friend Tim Berners-Lee. The original intent was for document navigation. Link from document A to document B. Not a generated on the fly document B, but a real document whose bits are on disk somewhere. Where if you were sitting at the console of said server, you could type “vi documentB” and be presented with the same thing that someone using browser would receive.
So, the page doesn’t exist. A representation of a page, generated in memory and streamed over HTTP via TCP/IP exists and then disappears. In that payload are a bunch of bits. These bits are then pulled together, parsed and rendered. A page isn’t sent to a browser – but the notion of one is. In the same way, you should write code and develop web applications towards the notion of a page. Once the page is considered an abstract concept it can be created as an abstract type with a modicum of work. And thus begins the business of abstracting the web away from the web application.
If you call a method on a business object to retrieve a list of widgets must you absolutely know that it’s talking to MySQL on the back end of the transaction? No. Not if it’s built even close to correct. Again, so why must we pay such attention if we’re receiving a POST or a GET. Heck, it seems a wasted effort to handle both (I know there are camps that don’t use POST) since it’s your software that dictates the communication request to your server. Your application receives a request. Plain and simple. Another example is web platforms that have a one to one mapping of a file to a URL. That just doesn’t make sense since that file (page) isn’t going to be sent back in the state it is on disk anyway.
Continue moving down the stack and we’re back at CSS. Ebay figured out a way to handle their CSS and images. Through some really slick Eclipse customization and use of software factories they compile all CSS and images with their source code; they’ve integrated instead of separating. It’s more complicated than I make it sound, but the idea is that the reuse of these assets is identifiable through their source control and build procedures. It’s not a secondary asset that is applied on top of all the engineering. It’s been sucked into the process. At the end of the day, the browser receives a bunch of CSS, but maybe some of it’s on the page and some of it’s in external files. It doesn’t matter that much. It’s more aesthetic than anything that all CSS should be tightly contained and able to be opened at any point in time in Dreamweaver. It’s trying to be separate when really you want it contained. You don’t want it to really exist. Just like the page, the notion of CSS files is great. And the rendering and positioning it provides is very helpful (plus it renders faster) but you don’t need to be explicit and draw lines in the sand of your application to make it so. A web application is software that delivers the notion of a document. The document should work extremely well in the requested browser. But the programming and design of a web application should be at a higher level of abstraction. The web, browsers and buzzword technology will outpace your development and features. This provides a way to either match that pace or ignore it.
From forward to Structure and Interpretation of Computer Programs; Abelso, Sussman and Sussman
“After a short time we forget about syntactic details of the language (because there are none) and get on with the real issues — figuring out what we want to compute, how we will decompose problems into manageable parts, and how we will work on the parts.”
We Could Have Built It Already
Somewhere in an IT department right now a team of people are choosing a software vendor. This vendor could be for just about anything in the corporate IT sphere. You can imagine the usual roundup: analysis packages, data providers, software components, UI controls – the list goes on and on nowadays. The people in the room are bright hard workers. And the problem they are solving is varied. Like how can the project succeed with the minimum work from in house staff or how can it be done without breaking the bank. And finally, wiping the sweat from the brow and mustering up the old college fighting spirit – how to deliver the best possible product.
Does this sound familiar? Bueler? Bueler?
These goals are altruistic and, some would say, commendable, but they aren’t always realistic. Why not? Because choosing a vendor is too often the default option when business is faced with a problem these days. And when that occurs, the We Could Have Built It Already principle is at work; otherwise known as WCHBIA (pronounced wa-chee-ba). In a nutshell, it’s a simple way to call out the business analysts, non-technical managers, technically challenged managers and the technical, but wet blanket managers, from wasting time on vendor product selection for systems that can be and more importantly should be built in house. Really, this is in hope of preventing Yet Another Integration Project, which we’ll touch on later. Don’t read this as all products should be built in house – they shouldn’t.
You can see WCHBIA all around you if you look close enough. It’s the multitiered, web front end, Ajax enabled, 14 pt font Web 2.0 web analytics package that tells you conversions of a shopping cart. It’s the “if you bought this, you’ll like this” recommendation platforms. It’s the five thousand dollar site licensed UI components that don’t scale on server farms. You’ve seen it or experienced it by now. Heck, if you work in corporate IT you’ve most likely integrated some third party (system, component, service, data feed).
Luckily there are ways to avoid WCHBIA. Let’s look at them.
–The Real Purchase–
Most vendors aren’t just offering product X, they are offering a solution. This solution involves downloads, installs, setup,configuration, documentation, maybe some scripts and a plethora of meetings. And that’s just for the technical staff. Project managers, lawyers, directors (up to CTO often) get involved because at the end of the day this is a capital investment with contracts to read and negotiate. This is a lot of people and consequently and lot of cycles at work across teams. Are you analyzing this time before heading down the vendor selection path? Have you really considered who the “third” in third-party means. Are you willing to actually estimate these hours? After all, multiple vendors will be investigated and probably tested in house which is an immediate time investment for the internal staff.
–Revenue Producing–
Next, is this system critical to your company? Is it used on a revenue producing product? Will it give your platform a leg-up? If you answer yes to these questions (or even two out of three) that is a big red flag to pay very close attention and tread carefully. Vendors go bankrupt and close down. They are sold and resold. They have their own internal priorities (and strife). They are very rarely plug-and-play no matter what Chaz, their outside sales guy, tells you.
At the end of the day, do you trust SimpleSoft LLC’s product to reliably be part of your revenue producing system. Do you trust it to fund payroll? And if this thing is so great, aren’t your prime competitors already using it or going to deploy it as well? Where is the market advantage in that? That’s just keeping an even playing field and not leveraging technology for market advantage. You won’t drown, but you won’t swim to shore either.
–Investing in Your Team–
There are significant advantages to investing in and improving a software platform. Solving the problem at hand is a key advantage. Although the vendor wants to sell a solution what’s usually required is a product – which is just software. So why even test drive a Ferrari when what you need is a Volvo?
Next, outsourcing interesting and challenging products to a vendor is a really good way to homogenize your development staff. It sends a signal that they are good at building houses but not skyscrapers. Sometimes this goes as far as turning them into glorified UI designers, that is, before the real UI designers are brought in. Management should know if their staff is up to building important revenue producing pieces of the platform. Most developers I know drool for the chance. That’s why they come to work. To build cool things and make an impact. It takes the same level of commitment to requirements and limiting scope – but if the focus is on building a functional product that fills the void it’s a win-win.
If you’re thinking that your teams software developers don’t want to build cool products and be challenged then you probably haven’t spent much time talking with them or listening to their conversations. If you know for an absolute fact that they would rather not build high impact features into your platform or deliver differentiating features to clients then you might have an unfortunate favor of “career” programmers on your team. And if that’s the case…there are other rivers your team will need to cross.
–YAIP–
Constantly looking to vendors is demoralizing for developers. If this is so important to spend 80, 100, 200K on why is it not entrusted to the development team to accomplish? Frankly, an investment in people is clearly lacking here and as outlined in the seminal book PeopleWare, this is as good a path as any toward Teamicide. Somewhere along the line managers have grown the thought in their head that software developers prefer integrating vendor software instead of writing software. Poll 100 developers and I’d hazard to guess that less than 10% of them say they’d like to spend their career integrating vendor solutions. This is where the term Yet Another Integration Project comes from; the constant churn of them in corporate IT software departments.
The churn of integration projects is justified by a claim to cost savings. The general thinking is that it would cost an in house team more to *build system X than it would to purchase it. But looking at the number of people involved, testing time, up front purchase cost, integration cost and maintenance cost of the “solution” – it’s tough to say it’s an accurate justification.
–Antipode Acronyms–
Remember when everyone was afraid of another acronym? NIH; Not Invented Here. For a time managers feared systems that were so complex that it couldn’t be maintained and someone would have to put a deugger on the equality operator because someone modified it’s behavior. That would be a terror to any reasonably sane person. But now the industry has swung hard the exact opposite way. Now, the fear should be systems built around vendor solutions and the lack of control. Who controls the roadmap for SuperMega EverywhereCache? SimploSoft LLC does, not you.
–Conclusion–
While looking for a vendor, testing their products and having lawyers read contracts, a decent set of software developers could often have built the technology in the same amount of time. Or maybe a little more time. It’s still software after all and integration projects are just as suspect to go dangerously off course as starting with a compiler. And given that developers (or system designers, architects) are roped into these vendor dealings, the time could simply be allocated towards building a working product. Basically, design and prototype sessions. The kind of things that really talented engineers want to do. In fact, the kind of thing that they were hired to do.
*If you work on a big CRM package and read this thinking “awesome, roll my own”, don’t show your employer this part of the article. Your type of software is what vendors are perfect for; unless your backoffice operations will generate more revenue or market share for your company. And if that’s the case, you are indeed in a very interesting industry.
Ajax and the Service Gateway Pattern
Introduction
It is easy to forecast how many widgets you can ship when you are creating them in the factory behind your office. It is another story when those widgets are being created in another factory that you do not own. You are now a customer of your distributor, but as a market economy would dictate, your customer’s do not care where the product comes from. They just want to know if you have it in stock, and if you do not, they would like to know how soon you will have it. Any company that ships a product that they do not produce themselves runs into this problem consistently, and often enough you will need to connect you systems to your distributors to get a virtual inventory to keep your customer happy.
The Problem of Real-Time Inventory
Let us call our fictitious company Alpha Parts. They sell computer paraphernali, thousands of items like printers and mice and laptops, and procure their inventory from the following five warehouses: MicroBell, JME, DataTech, Xenns and LargeDeutsche.
Now, as simple as this model seems from a business perspective (maybe it is more complicated than I think) it poses a very interesting technical challenge in order to display product availability on the Alpha Parts website of items that have not physically entered their inventory. How do we know how many Cannon X15 printers MicroBell has? Or the amount of Microsoft Explorer mice that is in the Xenns warehouse?
Alpha Parts could request daily files of inventory availability from their partners, but that data could become stale in minutes. It turns out that each of these partner warehouses has XML web services that can be called individually via HTTP. Now, Alpha Parts need has an interface into individual product inventory per warehouse, but needs to turn that data into a usable system that aggregates and gives meaning to these numbers to their customers.
And here lies the problem: Real time on the web. We have the following challenges: multiple long running real time method calls to a third-party service over the internet, displaying results that return at different times and handling unresponsive services.
Solution
One of the lead engineers at Alpha Parts proposes solving these challenges by using the Service Gateway pattern in the middle tier and an ASP.NET compatible Ajax component on the front end. Let us take a look at the Service Gateway.
The Service Gateway pattern is a repeatable design that “encapsulates the details of connecting to the source and performing any necessary translation.” Here is a logical overview of the pattern.
Figure 1
The web application sends its request to the Gateway. The Gateway is then responsible for translating the request into the proper format to send to the External Service. After the response comes back from the external service it is returned to the gateway where it is then translated back into a representation that the application understands. It is also at this point where the gateway can use the persistence layer for storage or retrieval of data.
The next figure is a UML breakdown of the gateway layer.
Figure 2
Class Breakdown
Gateway – responsible for providing coordination and translation for calling the external service
PartnerInventory – object that describes products availability at a particular vendor
PartnerInventoryCollection – object representing a collection of PartnerInventory objects
RealTimeInventoryAccess – this is an existing class that handles the actual post to the services
GatewayPersistence – responsible for persisting and retrieving data from cache or the database – this could also be used to perform an algorithm to decide what stock results are active enough to stay in memory cache and which should persist to the database
GatewayDataAccess – responsible for physically persisting and retrieving data to the database
The pattern is straightforward enough and only really gets complicated depending on the extent of logic that is desired in the persistence layer.
Each time a method is invoked from the Gateway to RealTimeInventoryAccess (and hence, the external service itself) it will be done with an asynchronous method. The following figure shows the idea behind this, not the exact code.
Listing 1
ThirdPartyInventoryDelegate third =
new ThirdPartyInventoryDelegate(RealTimeInventoryAccess.GetInventoryByVendorsAndItemsCode);
IAsynchResult synchResult =
third.Invoke(parameter, new AsychCallback(method), third);
synchResult.AsynchWaitHandle.WaitOne(time, true);
The Alpha Parts engineers want to ensure that the web application continues processing while the web services are being called; so they propose calling each web service asynchronously in order to return information to the Gateway object as the services returns. This way they can be run through the persistence layer sooner. Each asynchronous method call is also done with a timeout value in order to prevent the HTTP post from waiting on the default timeout which could be too long (notice the WaitOne in the above listing). This will also enforce our own time limits for those remote method calls.
Being as such, if a method was called at 8:00:01 AM and method two was invoked right after that, if the first service call took longer to return, the second call could already be done processing the service XML and have handed the data back to the Gateway for the persistence layer. It basically allows the multiple calls to return on their own time and not queue up behind each other so that there are not wasted cycles.
The Ajax capable front end is another key facet to this design coming to life.
Alpha Parts can use an Ajax component, like ComponentArt’s CallBack. This control allows you to dynamically refresh an HTML region with JavaScript, but control it all from the ASP.NET server side code. This is a great benefit because it means they would not have to write their own JavaScript. A third party control also ensures that it has been tested, debugged and run on multiple platforms and browsers.
When a product page loads on the Alpha Parts website, the full page will load except for the real time html region. The CallBack component will call the page class which will check with the Gateway and its persistence engine to see if there is current inventory information available. If it is available, those objects are returned to the page and bound to the CallBack control. If they are not available from persistence, they are sent out to the external service(s) and the CallBack control will display a “loading” icon. The user is then free to use the rest of the page and as soon as the Service Gateway is done with its work its return objects will be bound to the CallBack control.
Conclusion
In this way, the middle tier will be working asynchronously to retrieve inventory information from multiple services and the web application will work asynchronously to display the data it will have to occasionally wait for.
References
Jim Tyhurst, Services That Reach from the Inside Out. Dr Dobbs Architecture and Design World 2006
Delegate the Work
A staple of any good management class is pounding into young executives’ heads that they should delegate when possible. These gurus of commerce do not say to always delegate because it’s obvious that expert knowledge should sometimes directly handle a task or project. But the premise is solid. And the premise applies to our world of software.
The .NET Framework provides us with a staid and true mechanism for creating delegates and wiring up events. It’s a powerful platform provided for free to us, so taking advantage of it is something a developer should aspire to do without much hesitation. Events are brothers-in-arms with delegates. They work together so smoothly that you might not even realize that events in the Framework and Windows Forms applications are actually using delegates behind the scenes.
Delegates are useful to implement function pointers or callbacks. Here are the basics of a delegate: Defining a delegate lets you invoke other functions that match the definition of the delegate. Right – what does that mean?
First off, a delegate is really a class. It derives from System.Delegate (really, it could also derive from System.MulticastDelegate, but we’re trying to keep thing simple here). So, you can define a delegate and then call it like a method. See Listing 1.
Listing 1 – Define a delegate
Public delegate bool CookDinner(string MainCourse);
So, what does this get you? First notice that the delegate looks like a method definition with a return type and method parameters. That should feel pretty comfortable. But, like I mentioned above, this is actually getting created as a class, so under the hood it is much more than a method. Now that we have this delegate defined we can use set it up for use. See Listing 2.
Listing 2 – Setup the delegate
CookDinner cook = new CookDinner(CookingTacos);
Private bool CookingTacos(string Meat)
{
if(Meat != “vegetable”)
{return true;}
else
{return false;}
}
Note that when we initialize the instance of our delegate, CookDinner, with the variable cook we are passing in a method that matches the delegate’s definition to the constructor. This is very important. All it takes to instantiate an instance of the delegate is a logical pointer to a method. Now take a look at Listing 3 and you’ll see how the delegate is used.
Listing 3 – Use the delegate
FeedTheFamily(cook); //pass in the delegate
Private void FeedTheFamily(CookDinner cookIt)
{
cookIt(“Beef”);
}
This helps us decouple a specific action from each method. Here, we can pass in our delegate to FeedTheFamily and that method is then able to call the method we already defined. So now if we decide that FeedTheFamily needs to cook something else, we change the method that is passed in the CookDinner constructor.
Events owe their livelihood to delegates. Before reading this you should understand that delegates allow you to call a method indirectly through the delegate construct. The focus in discussing events here is not to go into an in depth analysis of invoking and handling different events in Windows Forms and the .NET Framework. Instead, the focus is to see how delegates and events work together and how you can create and use them to add flexibility and maturity to any type of application. Let’s take a look at a fresh example. This example handles events for when a record is read from a database. See Listing 4.
Listing 4 – Define a new delegate
Here we follow the traditional event programming idiom of passing the object reference of the sender along with an object inheriting from System.EventArgs that contains specific event information.
Listing 5 – Define the Event
Here we have an event and a method to raise the event. Both of these are internal to a class named Transform. The event is defined in terms of the delegate type. And the method OnRecordRead is called internally, which in turn raises the event up to the event. Let’s now see how a class hooks this up in Listing 6.
Listing 6 – Application using the event
Now we’re really getting somewhere. What you see in Listing 6 is a class named Client. Inside of Client a new instance of the class Transform is instantiated. Next, a handler is wired up to the RecordReadEvent that matches the delegate type of RecordReadEventHandler. And finally, the method HandlReadRead actually handles the information, or events, that are published. In essence, Client is acting as a subscriber to the information that Transform is publishing. Listing 7 below ties these last three diagrams all together.
Listing 7 – Tying it all together
To summarize this, here are the major parts.
· Delegate definition
· Event definition
· Event publishing
· Event subscription
· Event Handling
And what is the beauty of all this? The fact that it’s easily extendable and usable in your ASP.NET applications, console apps, utilities and smart applications makes it so. It goes way beyond Windows Forms and can add loose coupling and flexibility which is something that object oriented software should strive for.
P2P Message Queueing
Last fall, that of 2007, I took a computer science graduate class at Loyola University Chicago on Distributed Systems. It couldn’t have been a better first class to take as messaging, large system communication, the web, remote objects, fault tolerance, distributed processing really gets my mind spinning. I enjoyed the class and the research I was afforded to do greatly. My final research was on message queuing in a peer based application model. Basically, applying message queuing constructs to applications to ease p2p communication stacks across platforms and networks. Queues are meant to store and forward messages and there is already a wealth of research on queuing. On the other hand, there is a trail of carnage from dead p2p applications. Something has always gone miserably wrong (with a few successful instances). I wanted to design an application that wasn’t meant to simply share files (or filez).
Not All Interfaces Are Built the Same
In my last article I described using composition in the design of an object based system. In that article I included a simplified UML class diagram of some work I did to a fictitious Product object to display the use of composition. Please see Figure 1 below. At the top of that diagram sits the class ProductBase. It is an abstract class (a class that has abstract methods and cannot be instantiated). But why is it abstract?
Figure 1 – Product UML
Let us first ask what could the class be marked as if it were not an abstract class? Here, there are two options. One, it could be a concrete class and two, it could be an interface. Let us look at the concrete class first.
A concrete class is any class that is not abstract; it is just a class. The following code snippet contains a concrete class followed by an abstract class.
Listing 1 – Concrete and Abstract
public class StockStatus
{}
public abstract class ProductBase
{
abstract public StockStatus InventoryStatus{get;};
}
When a class is concrete that means the only way to extend the class is by inheriting from it. That is often referred to as implementation inheritance or class inheritance because you are in essence inheriting all of the code and logic that went into the base class. That is not something I wanted to do with the Product object because it blows up encapsulation and is very inflexible.
The second option was to use an interface. Here, we would define an interface and then have ProductBase implement that interface. In the future, we could then substitute any class that implemented the interface.
Listing 2 – Using an interface
public interface IProduct
{
int GetNumberInStock();
}
public class ProductBase : IProduct
{
public int GetNumberInStock()
{
return 3;
}
}
In my first go around, this is basically what I had. Of course, that is why design reviews are important. It was pointed out by a coworker during a review that using an interface in this instance does not buy you anything more than using an abstract class, and if an abstract class were in fact used, we could mix in a little bit of implementation. Sounds good right? At first I did not like it at all.
“That breaks encapsulation!” I bellowed across the table, and let it reverberate off the walls and deaf ears. No one wanted to hear it. Why? Because using an abstract class here was simply a better design decision.
In this instance, a Product, no matter how many times subclassed, is a product. It has properties and some core methods. So in our abstract class we defined all of our entity properties implementation but left the “Load” (of data) methods as abstract so that the implementing class could define those. It ended up as an unintentional use of the template method, which is a good side effect of paying attention to your design. I believe that the decision is a mix of the best of both worlds.
Summary
We now have a good abstract class, which behaves as a well defined interface, plus when it comes time to substitute in another class of the same type, we have a lot of the plumbing already there. If we had used an interface it would mean we would have to rewrite a lot more code. This minimized our code creation and maintenance, but still keeps us in good object hierarchy.
Wait…Compose Yourself First
Of course, I am not the only one. If you are familiar with our friends, the Gang of Four, you have already heard this mind-bending concept. Make your objects connect to each other via well defined interfaces and hide their implementation. This is a sound, modern software practice. In fact, I just spent four days this past summer listing to speakers take composition for granted at the architectural level. They were assuming (not all of them of course) that composition and abstraction is a skill that is taken for granted by most developers. We know, being the folks in the trenches, that this is not always the case.
Favor Composition
Let us look at some simple composition here and discuss what the big deal is all about. To start, here are what I believe to be the three most important and beneficial things about using composition.
- Black Box design (where inheritance is white box design)
- Does not break encapsulation
- Relationships can be defined at runtime
Black box design means that the client using the object does not know the internal operations. This is in contrast to inheritance, where the class often knows the internals of its super class. Not breaking encapsulation is as good a reason to use composition as any. After all, encapsulation of operations and data is one of the main reasons to use object orientation. And lastly, “relationships can be defined at runtime” means that other objects can be plugged into, or composed with, other objects while the program is executing. This is a common use of the Strategy pattern, where an algorithm is set to the class at runtime.
Looking at an Example of Composition
Below, you will see a snapshot of part of a fictitious product component. ProductBase is composed of StockStatus, which in turn is composed of PurchaseOrderLine and IRunRate. Here, StockStatus does not know about the implementation of IRunRate, it only knows IRunRates interface. Here, we can then substitute any other object of type IRunRate into StockStatus and it would be none the wiser. You could of course, change behavior by having StockStatus be aware of its algorithms and change depending on what it is composed of – but that is not taking advantage of composition.
Figure 1 – UML Diagram
PurchaseOrderLine is the next object. Although StockStatus is not connected to an abstract class or “interface” type, the public methods and properties of PurchaseOrderLine make up its interface. So, again, StockStatus only knows about a well defined interface.
Conclusion
It is important to look at your design and try to understand the moving parts. These will let you know what the system will be composed of. In my next article, I will talk more about identifying the objects that make up your system, as well as why this design used an abstract class at its base and built upon it with composition.
Object Relational Mapping in ASP.NET 2.0
Introduction
The task was simple and sounded like a decent technical challenge and an interesting opportunity. Build an ASP.NET 2.0 with SQL Server 2005 transactional website that the financial industry can use to list special events and webinars so traders and investors can find it all in one location. I worked with the business owner and came up with the high level requirements. They are listed next and will look familiar to most of you.
1. Provide ability for users to register.
2. Allow registered users to list their events and webinars.
3. Allow registered users to perform CRUD actions to entered data (create, read, update, delete).
4. Provide administration interface for CRUD to all application data.
5. Display all events on the website in a format that allows sorting.
Of course, there were more requirements than what is listed above, but this is the core that we focus on moving forward in this article.
I have my own technical requirements that I bring to projects I lead, namely, that it should be as object oriented as possible and utilize the tools and technology available. ASP.NET is object oriented itself, so extending this concept into the other layers of the application is logical and advantageous. The second requirement is that the design should not only promote code reuse, it should provide for it. Repetitive code and functionality is a waste and a design “smell” that I try to avoid.
This project is the type that could easily end up with repetitive and hard to maintain code, especially in the data layer. Let us take a look at the options for data access. First, we could write inline SQL for all data access. We can rule that one out easily, as it ends up being difficult to maintain and makes OOP concepts difficult to apply. Let us not jump in the tank with that octopus.
The second option is to use stored procedures and wrap them with their own methods. This is without a doubt the most popular way to build a data driven solution with any of the .NET languages and technologies. If you ask any group of developers if they use stored procedures for their data access at any point in time you will see that four out of five reply with an affirmative. Take that to the next logical step and ask why they use stored procedures; the answer is almost consistently – (a) it is faster because it is compiled and (b) that it is more secure. I will accept (b) as a technically sound and savvy answer, but will question (a) all night long. Without rehashing this impassioned argument, I will simple say you can get the speed of stored procedures within .NET without writing a single stored procedure. Plus, you can get that with a better, cleaner and more object oriented manner.
OR Mappers
What is Object Relational mapping? In a nutshell, it is a technique to provide automatic retrieval and persistence of data. Usually this involves using a configuration file, often in XML, to map fields in your custom coded objects to fields and tables in a database. In this way the OR Mapper can then take ownership of the best way to retrieve, update, insert and delete data and you can focus on building the application and not mess with the plumbing.
Most professional developers use third-party components at some time and OR mapping products are an example of the acceptance and reliance on these types of components. The reality of our work is that almost every application has to touch a database and this data access work is error prone, tedious and honestly, just plain boring after a while. This type of work lends itself to mistakes. I am reminded of the classic software developers axiom, “if you can reduce risk, then do so.” This is one reason to use an OR mapper.
Another OR mapper advantage is the speed of development. Using an OR mapper can drastically reduced the amount of code that needs to be written and debugged, plus it can quicken how you develop. If you are mapping data directly to your objects, essentially making them entity objects, there is not data conversion between Datasets, data readers, structs or anything else. This reduces the complexity and makes working with your data essentially the same as working with your objects. Now your focus is on your domain model and not how ADO.NET is doing this or that task. Please note that I am not saying that understanding ADO.NET is not important – you should understand these concepts to be a productive and knowledgeable developer – but now that you understand the core concepts you can step beyond it and really take advantage of this knowledge.
True OR mappers, which I consider different than code generators, do all their database access dynamically. So, they are not generating files with code in them, and they are not using stored procedures. In the .NET world, the mapper will do all its database access using parameterized queries, which as controversial as it may sound, are just as fast as stored procedures. Thus, you now have the flexibility of dynamically generated SQL with the speed of a stored procedure.
Use of an OR mapper also provides a level of abstraction between your code libraries and the data source. Let us say that you are using MS Access as your application data source and you are using an OR mapper. You can now easily switch from MS Access to SQL Server 2005 without too much trouble because you do not have anything hard coded into your application that ties you to a platform. Extrapolate this out and you can migrate from Oracle to SQL Server or Access to MySQL, or really any way you wish.
The biggest concept to understand here is that the OR Mapper is providing flexibility. Flexibility in how you write your objects, flexibility in how you spend your development time and flexibility in what your data source is. And of course, they are fast at the same time.
Different OR Mappers
There are many different OR mappers available. The three products that I consider when thinking about the object relational mapping category are NHibernate, LLBL Gen Pro, RapTier and Wilson ORM.
NHibernate is the open source port of the popular OR mapper Hibernate for java. I am not a java developer so I can not speak to how good of a port it is. I have not used NHibernate myself, mainly because I have found the documentation of the product to be poor so far. There is a very active community around this product, but for my project I did not want to participate in their project, I wanted to focus on building mine. I have read and heard of success stories from colleagues along with their accompanying horror stories with NHibernate. I do consider NHibernate to be a true mapper because it does not generate code for the developer to manage. It does all its work dynamically once configured.
The second mapper is LLBL Gen Pro, which comes with its own environment for mapping objects with database tables and fields. This product does generate code files, so it is not something that I wanted to use this time around.
The third product is called RapTier. I have used this on a couple different projects earlier this millennium with success. RapTier does create code files though, so you have to maintain those and is something I would not consider doing now. But at the time, I experienced this product to be a great time saver because it allowed me to focus on the application and not the data access. The company also provided good support and an aggressive price.
The last product is WilsonORMapper and is ultimately what I decided to use on this project.
Why WilsonORMapper?
When push came to shove the choice for this project really came down to the merits of WilsonORMapper versus NHibernate. They are the two true, non-code generating mappers that I investigated and each had their own pluses and minuses. I choose WilsonORMapper for a couple key reasons; starting with that Paul Wilson, the creator, provides direct support for the product and the support site includes an active user forum where other users help each other and share advice and support. Beyond this, Wilson provides staid examples explaining how to use the different features of WilsonORMapper at a very good price.
Compared to NHibernate, these features were much more important to me during this project than the open source community behind NHibernate. I made the conscious choice to focus on a tool that could help me build my project as opposed to contributing to the tool’s project and would have support behind it.
Domain Objects
Part of the elegance of using an ORM product is that you can continue to use the domain (or call them entity) objects that you might design before event deciding on using ORM. This allows you to keep your custom objects and other steps towards a domain model and OOP principles intact while at the same time simplifying your database access. See Figure 1 for a part of the domain model used on this project.
Figure 1
In the above figure you can see part of the domain model. Here, we have objects describing a Listing, Feedback, Owner, Event and an interface for Event named IEventDetail.
Configuration File
The configuration file is what ties together the database tables and fields with your objects as shown in Listing 1.
Listing 1
<?xml version="1.0" encoding="utf-8" ?>
<mappings version="4.2" defaultNamespace="">
<entity type="Owner" table="Owner"
keyMember="Id" keyType="Auto" sortOrder="LastName ASC" autoTrack="false">
<attribute member="Id" field="OwnerId" alias="Id" />
<attribute member="FirstName" field="FirstName" alias="FirstName" />
<attribute member="LastName" field="LastName" alias="LastName" />
<attribute member="Street" field="Street" alias="Street" />
<attribute member="Street2" field="Street2" alias="Street2" />
<attribute member="Number" field="Number" alias="Number" />
<attribute member="City" field="City" alias="City" />
<attribute member="State" field="State" alias="State" />
<attribute member="Postal" field="Postal" alias="Postal" />
<attribute member="Country" field="Country" alias="Country" />
<attribute member="Company" field="Company" alias="Company" />
<attribute member="Email" field="Email" alias="Email" />
<attribute member="UserName" field="UserName" alias="UserName" />
<attribute member="Salt" field="Salt" alias="Salt" />
<attribute member="PasswordEncrypted" field="Password" alias="Password" />
<attribute member="Role" field="Role" alias="Role" />
<relation relationship="OneToMany" member="Listings" field="OwnerId"
type="Listing" alias="Listings" lazyLoad="true" cascadeDelete="true" />
<relation relationship="OneToMany" member="Events" field="OwnerId"
type="Event" alias="Events" lazyLoad="true" cascadeDelete="false" />
</entity>
The configuration file is in XML, so the hierarchy of data is simple to read and just as easy to understand after you use it for a short while. The entire file is wrapped in the mappings element and each table-to-object relationship is defined in an entity element. The Listing 1 example above ties the object Owner to the table Owner. It sets the object key, Id, to the tables’ primary key via the keymember attribute. It then sets the default sort order and a attribute named autoTrack which lets the ORM framework know whether or not to watch this object and perform SQL changes on its own.
Each property of the object is then mapped to a database field via an attribute element and member attribute.
Notice the element relation. This provides foreign key relationships to other tables. In this example the field OwnerId in the table Owner is linked to the table Listing with a synonymous field name. From here we can set the Listing object to be lazy loaded inside of the Owner object. So, to summarize this, each object is mapped to a table in the configuration file. Furthermore, a related object can be lazy loaded as a property in another object via foreign key relations.
ObjectSpace Manager
The ObjectSpace object is part of the WilsonORMapper libraries and is the main coordinator that is used by the developer to access data through the framework. It is best to only have one instance of the ObjectSpace object in an application. Doing so will help manage concurrency in the application between the data source and the mapper and secondary; the object loads up the xml configuration file from disk. So, obviously you do not want to repeatedly read from disk or recreate the object over and over. See Listing 2 for an example of how to create the object, in our case and by example from Wilson, the object is static and named Manager.
Listing 2
public class Global
{
private static ObjectSpace _Manager;
public static ObjectSpace Manager
{
get
{
if (_Manager == null)
{
CreateORMapper();
}
return _Manager;
}
}
private static void CreateORMapper()
{
string mappingFile = System.Web.HttpContext.Current.Server.MapPath("~/trader.config");
_Manager = new ObjectSpace(mappingFile, ConnectString, Provider.Sql2005);
}
}
Select
Let us take a look at a method that retrieves data from our source by doing a simple SELECT in Listing 3 below.
Listing 3
public Event GetEvent(int Id)
{
return (Event)Global.Manager.GetObject(typeof(Event), Id);
}
This method simply returns a filled Event object via the Id passed into the method. We use the GetObject method on the Manager object, which is of type ObjectSpace, and tell it the type of object to return. The mapper then knows what table the object type is mapped to and does the translation for us. The GetObject method can also then take in the parameter here, which is simply doing a query via the primary key setup in the configuration file.
Notice here how everything is strongly typed. The code does not have to know anything about ADO.NET or even the internals of the Event object. The mapping loads the right result set fields into the created Event object properties and fields.
On a side note, it is a matter of preference if you map your table fields to properties of the object or fields. The more draconian OOP bylaws would say that if you’re setting data in an object it should be via properties. But here, the mapper can access everything inside of the object anyway – so, it is a matter of preference.
Returning one object is simple, but the world works in large numbers, so let us look at returning a collection of objects. See Listing 4 below.
Listing 4
public IList GetEvents(string sortOrder)
{
ObjectQuery oq = new ObjectQuery(typeof(Event), "", sortOrder);
ObjectSet allEvents = Global.Manager.GetObjectSet(oq);
return allEvents;
}
The GetObjects method of the Manager object returns a collection object called an ObjectSet, which is part of the WilsonORMapper libraries. Being a well behaved collection it implements the IList interface. So our GetEvents above can be typed to IList to add some flexibility.
Insert, Update and Delete
Doing the other parts of the CRUD cycle (create, update and delete) are very similar. Listing 5 below shows an insert and Listing 6 shows an update. These are all working against that same Event type.
Listing 5
public void Insert(Event theEvent)
{
Global.Manager.StartTracking(theEvent, Wilson.ORMapper.InitialState.Inserted);
Global.Manager.PersistChanges(theEvent, Wilson.ORMapper.PersistDepth.ObjectGraph);
Global.Manager.EndTracking(theEvent);
}
Listing 6
public void Update(Event theEvent)
{
Global.Manager.StartTracking(theEvent, Wilson.ORMapper.InitialState.Updated);
Global.Manager.PersistChanges(theEvent, Wilson.ORMapper.PersistDepth.ObjectGraph);
Global.Manager.EndTracking(theEvent);
}
The basics of this operation are that the methods first call a WilsonORMApper method called StartTracking on the Manager object and then does the persistence. StartTracking lets the mapper know that this object and its associated row in the database are going to have work done to it. I am not sure if WilsonORMapper locks the row then or not, but it does add safety to the operation. Part of StartTracking is telling the mapper if we will be inserting and updating data. Then we call the PersistChanges method and lastly end our tracking of that object and database row.
Performing a delete is very similar and can be seen in Listing 7.
Listing 7
public void Delete(Event theEvent)
{
Global.Manager.StartTracking(theEvent, Wilson.ORMapper.InitialState.Unchanged);
Global.Manager.MarkForDeletion(theEvent);
Global.Manager.PersistChanges(theEvent, Wilson.ORMapper.PersistDepth.ObjectGraph);
Global.Manager.EndTracking(theEvent);
}
The only addition with the delete is we call MarkForDeletion which is another trigger letting the mapper know what we are about to do to that object and its data.
Lazy Loading
An aspect of object oriented programming that is simpler to do with an object relational mapper is lazy loading one object type from within another. For example, if we want to load up the Owner object of an Event object, for instance, the company hosting the event, we can write our lazy load property in the Event object and then simply create an Owner object with the owner id key returned from our Event object load. In this way your object model can grow and objects maintain a high level of encapsulation.
Conclusion
An object relational mapper is a powerful tool to use and can save days – and possibly weeks depending on the size of the project – of tedious coding and debugging. Within a short time you can get up and running and use the object model that you created for the project without having to rework a lot to run stored procedures or other ADO.NET queries. In the end you can focus on the core business logic part of the application and consider the mapper and your data layer as plumbing that came with the house. WilsonORMapper simply makes this much easier with better support than other free or complicated OR mapping solutions out there.
