This is the challenge facing anyone who wants to correctly handle their error conditions. It’s also a challenge that you should be willing to step up to if you want to get your code windows logo certified.

There’s a problem if you miss exceptions that should be caught. If you do that, your application will simply crash, unceremoniously losing anything your customer hasn’t saved and possibly earning you an angry phone call. In terms of customer satisfaction, it’s clearly the #1 thing to avoid doing.

There’s also a problem if you catch exceptions that you shouldn’t. The worst case scenario there is that you’re corrupting user data. More likely, you’re hiding the place where an original problem occurred and saving the actual crash for later. If you’re really good at catching everything you shouldn’t, you might be limping along in a bad enough state where the user has to just stop and restart the application before things start working properly again. I know I’ve been there as a user many times.

If you’re signed up for Windows Error Reporting, catching too many exceptions means that you don’t get notified of real problems with your application, meaning there are a whole set of problems you don’t see, or end up manifesting in some unexpected and un-debuggable place.

So exception handling appears to be this wonderful balancing act, it’s essential that you get just enough and not too much. I assume you’d rather be pro-active and double check that you’ve got all the error cases correctly covered rather than let your customers find them with a crash.

Good luck.

Seriously, I find it hard to believe I’m writing this, but you can’t do it.

Sure, you can read the documentation for every method you call, check what you need to do for every documented exception and handle it. That’s actually quite a lot of work, especially if you’re doing it for an existing project. If you’re lucky all the exceptions will be documented, they’ll even be documented correctly and fully, including all the 3rd party libraries you use … and all the internal ones.

And once you’ve done all that, the only way you can verify that it’s correct in your next release is to do it again. Now what happens when you go to the next version of a 3rd party library or .Net. You need to examine every method you use to see if any new exceptions have been added, old ones removed, or the handling requirements of the others changed. Then check that your code handles them all correctly, in every location.

You’d think there would be a tool to help you with this. But there’s nothing in .Net or visual studio that offers any clue. One company, RedGate, did do this at one point. However, reading their website, they were having trouble keeping the list meaningful, and with .Net 4, they gave up.

So even if you catch only the exceptions that are actually thrown, you’re safe, right?
Of course not. Because someone, somewhere will be throwing a raw “System.Exception”, and the only way to catch that is to catch “System.Exception” and that catches everything. So now you need to catch everything, then check to see if it’s what you really thought it was and re-throw everything else.

Of course, you’re re-throwing with just the “throw” statement, right? Not the “throw obj” version of the statement. Everywhere. Sure?

Lets assume you can change this code to throw something sensible instead of a raw System.Exception. How about a System.ApplicationException, that’s bound to be just the exception we were expecting to catch, right? If you answered “No”, you’re getting the hang of this. The .Net framework will happily throw its own exceptions which are derived from ApplicationException, even if you haven’t derived anything from it yourself. So you still need to check the actual exception type after you catch it just to make sure it’s something you expected. No, you need to ignore what Microsoft have given you and make your own exception hierarchy derived directly from System.Exception.

Ok, so we’ve done all that. We’ve manually gone through the latest documentation for every method we ever call and checked that we’re handling all the exceptions correctly. We’re not throwing any base-class exceptions that aren’t our own and we’re re-throwing everything correctly. We’re clear now, right?

Ah, you’re catching on, not yet. Let’s look at that web service you use, you’re catching a WebException, right? You’ve checked the inner exception as well, haven’t you? Why? Because although Microsoft will deny you a windows logo certification if you catch too many exceptions, they don’t follow their own advice. If something like AccessViolationException occurs whilst calling a web service, it will get wrapped up very nicely in a WebException and presented to you just like a timeout, or a connection failure, or a 500 error. So you might need to leak a WebException.

So what could InnerException be in these cases then? Surely that’s documented? Yeah, right.

Why is this such a hard problem?

It strikes me after much pain working through all this, that .Net’s exception handling is fundamentally flawed.

Firstly, we rely on documentation to work out what exceptions can be called. We don’t rely on documentation to work out what parameters to pass, or what return values we get, yet handling the error conditions is left to our discretion.

Secondly, there are some exceptions which could get thrown at any point and would probably never be thrown by your own code. The clearest cases are things like StackOverflowException and AccessViolationException. Perhaps NullReferenceException is in this list too. We would almost never want to catch these. These are genuine coding errors and never something to do with the state of our user’s workflow. Yet we have no way to determine what exceptions fall into this category.

The exception hierarchy is a joke. The knee-jerk reaction for how to categorize AccessViolationException is to say “It derives from System.SystemException, exceptions derived from here must be serious!”. And then you find yourself leaking when you parse a user’s badly formatted xml snippet, because XmlException is also derived from there. These aren’t isolated incidents, you can’t tell anything meaningful from the .Net exception hierarchy. The namespace usage is slightly better, but still useless.

If an exception is going to be a base class, it must never be thrown. There’s a built in mechanism for this, it’s called the abstract class. An abstract base class will never be thrown directly preventing a whole category of errors. And once an exception class is made concrete, it must never be subsequently derived from. We have a mechanism for this too – sealed classes can’t be derived from. Yet these mechanisms are not made use of in the built in exception hierarchy.

So what can we do to mitigate these problems?

Well, at the end of the day, your exception handling is your best guess. You can’t make it any better than that, so 90% is the way to go. Sign up to Windows Error Reporting and find what you’re leaking in the field. It’s a shame that as a top tier development company, that’s the best you can do, but it’s the best that anyone can do, so don’t feel bad.

In those cases where a leak would be a truly terrible thing, do what you did anyway.
Let’s be honest, your windows form event handlers are all wrapped with “try {} catch (Exception e) {}” aren’t they. Because after years of pissed off customers that’s what you’ve resorted to. Yes, your deeper code all does proper exception handling as you want, but the lure of the safety net was too much for your bottom line to resist. Well, in those critical, high level places, keep doing that. But try to categorize the list of exceptions you don’t want to catch and make sure you leak them. Unless you’re doing something really funky, you don’t want to hide an AccessViolationException.

And of course, unit testing can help. It’s not 100%, firstly because your coverage won’t be 100%, also because you’re still relying on documentation to work out what exceptions you’re supposed to be emulating, and also because a lot of exceptions that happen in practice are environmental problems, like dropped connections.

But the best way to handle your exceptions?

Don’t.

At least, don’t if you can get away with it. Just crash. In many cases that’s the best thing to do. If you’re a service, it’s a no-brainer, assuming your service will be auto-restarted. A web page returning an error is a good thing for visibility of errors.
They get logged, and you can find out about every single one of them as they happen.

Avoid getting in to situations where exceptions are possible. Always use “TryParse” instead of “Parse”, for example. Always check that the file exists before opening it. That sort of thing. If you’re thinking of throwing an exception, offer a way for people to find out what the situation is before you need to throw it.

Going 100% exception free isn’t possible. Some databases throw exceptions that demand the transaction be retried. That’s aggravating, but handle it and leak everything else.
A web service call should be retried a couple of times if certain error codes get returned. But anything else, let it go.
Keep it minimal.

The cost of crashing a windows application is higher. But why are you writing thick client applications in this day and age? Perhaps you can treat it as a container for applets that crash with impunity – I’m thinking of Chrome’s “aw snap!” errors here.

Crash, seriously. It’s the future.

There’s nothing particularly fancy about the technique, it’s essentially a web request that takes a really long time. The request stays open until whatever data the client’s interested in is available on the server.

Using it means that clients can get updates almost as soon as they happen on the server without polling. Polling is generally inefficient and wasteful of both network resources and server processing. It’s up to the client to work out when they poll and you can never really tell when or if a client is going to get the update.

Server push is really easy to configure – if your client can make a web request to your server, then you can use this technique without installing any fancy software or opening any unexpected ports, it just works.

The trick with Server Push is to have a very efficient server. Traditional web servers would start up a thread for each and every connection. Whilst this works when you’ve got very limited simultaneous requests, it doesn’t scale up very well when dealing with lots of requests that, for most of their lives, aren’t doing anything.

Frameworks like python’s Twisted, ruby’s EventMachine and javascript’s Node.Js have shown a way where this can be really efficient – don’t spawn threads, just process them using events. So we know it’s possible – are there any other ways to do it?

I’ve been looking at the problem using .Net’s HttpListener, which is actually quite efficient. It’s essentially the same as http.sys, but in managed code. Web requests come in through a call to GetContext – but no extra thread is created, you just get an object representing the request, and you respond to it as you see fit.

It’s quite simple then, to put the requests that come in straight onto a list and wait for something to happen to your data. When that something happens, simply iterate over the list sending the updates to the clients. You don’t have to reply from the same thread that accepted the request – indeed there’s no need to reply immediately at all.

It’s a beautifully simple principle and works really well. I’ve had 135k active connections on my test machine and it doesn’t even break a sweat. Once the connections are made, the only resource used is RAM, and those 135k connections only used about 800mb. No handles, threads or cpu are consumed on the server, so there’s no fixed upper limit that I can find.

What’s even better, is that clients will appear to wait forever for the results. I’ve had tests running for >12hrs, then receiving the response and carrying on like nothing was amiss.

It’s also worth noting that HttpListener can be a really good server in it’s own right, by caching the replies sent down to each client, I’ve been able to consistently handle 150k requests a minute on a quad core server over 100mb/s network connection.

There’s a few things I’ve found that are worth noting:

Server

Accept connections as quickly as possible

In essence this means getting to the next call to GetContext as quickly as possible. There are two methods you can use for this – have a single loop that continuously calls GetContext and farms out the actual work in some way, or use BeginGetContext, and in the callback method, call BeginGetContext again immediately, before you process your current request.

Remember that List is backed by an array.

If you’re adding hundreds of thousands of items to it in a time critical context (such as within a lock block) it gets really inefficient as it copies the data each time it grows the backing array. You can either pre-allocate if you know roughly how many items will be in it, or just use a LinkedList – which has constant time adds and removes – which is what you’ll likely be doing most of the time anyway. List is really bad if you start paging, since it’s essentially copying that data at disk speed.

Don’t block the list

The list is central to this processing, and every lock on it reduces your ability to service requests. An example is in sending out the replies – it may not take long to loop over 100k objects in a list and call BeginWrite on each one, but it takes a few seconds. It’s much better to take your lock (which isn’t on the list itself), copy the list reference and create a new list in its place. Then you can iterate through your list of subscribers at leisure without hampering the server’s ability to accept new requests. If you’ve got a lot of subscribers, those clients that you serviced at the beginning of the list will already be queueing up for the next update by the time you get through your list, so this is far from academic.

Don’t throw that exception

Time really is of the essence, whether you’re getting an entry onto a list or sending the data out, it needs to be quick. Throwing exceptions is slow. So even though it’s more clunky, use that TryParse instead of Parse and structure your logic so that you’re not throwing trivial failures into the exception handler for mere coding convenience.

Cache Everything

This setup can be really efficient if the same data is being sent to everyone. In this case, you can render the data you send to one client into a byte array and keep it, so that every subsequent client receives a copy of that same byte array.

Use a versioned REST style interface

This is just a suggestion, but use the url to locate what resource you want, and have the client send a query string stating what version they’ve got. If your current version doesn’t match or the client doesn’t provide a version, just send back the latest version to the client without queueing anything. If the version does match, add it to the subscriber list for when the next version becomes available. It goes without saying that you need to synchronise this so that no-one gets left behind.

Use the background threads

I’m a little more cautious about this advice, but it’s worked for me in my testing – using the asynchronous calls for everything does appear to work really nicely. One note of caution – the thread count can go up alarmingly once things start going wrong (actually, I’ve only seen this on the client, not the server). It’s only a temporary situation, however, once the issue is resolved the threadpool sorts itself out nicely.

Have a plan for dropped subscribers

In my proof of concept I didn’t really care, but in reality, the subscribers can disconnect. Clearly, you don’t want to keep these in your list any longer than necessary. I haven’t yet found a way of cleanly and reliably detecting a failed connection using the HttpListener classes, although I haven’t looked too hard. It might be the case that the best you can do is return an empty response and ask them to reconnect. Whilst this might not sound any better than polling, it’s still fairly lightweight, responsive, and the server can dynamically determine what it needs to do, modifying these ‘poll periods’ according to load.

Client

I’ve actually found it much harder to write a client that makes 50k requests than a server that accepts 100k requests.

Know your http limits

By default, only 2 http concurrent connections are made – per machine to any particular server. Use kb282402 to fix this if you’re using a wininet based client, and set ServicePointManager.DefaultConnectionLimit if you’re using .Net’s WebRequest class

Know your ports

Each request to the server requires an open, bound port on the client. There are only ~65k available for each ip-address. So if you want >65k connections from one machine, you’ll have to have multiple ip addresses on the network and bind your requests to them explicitly. Also, windows varies how many ports are available for this use. My developer machine allowed everything >1024 for this, whereas Vista/2008 and above only use ports >49124. This limits you to about 16k outbound connections of any kind. Use kb929851 to configure this. Also keep in mind that ports don’t become available just because the connection closed. Ports can stay unavailable for 4 minutes whilst they mop-up any late server packets. This can be reduced by the OS if there are a lot of ports in this state, but it can bite you if you’re trying to recycle your 50,000 connections.

Know your handles

With .Net’s WebRequest class, and probably with any outbound network connection, a handle is created for each one. Windows XP limited the number of open user handles to 10,000, in windows 2008R2 I’ve not had any trouble when running 50,000 connections, but it’s something else to be aware of.

Any server can be flooded, live with it

A server can only accept so many connections a second. The network stack can itself queue up the backlog of connections, but the size of the backlog is limited (the best documentation I can find says between 5 & 200). Beyond this, the network stack will reject the connections without your server code ever knowing that they were there. This is the main reason why it’s so important to accept those connections quickly. But even so, if you’re making 50,000 asynchronous connections to a server, from 4 clients, at the same time for a total of 200k connections, that could all take place in a few seconds. No server can handle that, you will get errors, and you must have a way of recognizing them, handling them gracefully and re-trying them.

I wonder how many people can make use of this technique, how many people are using this, and what other issues there are with this. Has anyone found a different solution for the problem?

std::vector<int> myvector = …
for (int i = 0; i < myvector.size(); ++i)
…

This code produces a warning when compiled, and it’s right to do so.

Vectors are awkward beasts – this means that they fit in with the rest of c++. In particular, the type of the variable “i” above, is not an int, unsigned int or a dword, it’s std::vector<int>::size_type.

This size type is also the numeric value accepted by the [] operator. Int will pass into this operator without warning, but it’s not correct. And going the other way – especially via myvector.size(), is a potential issue.

So the for loop above should be written:
std::vector<int> myvector = …
for (std::vector<int>::size_type i = 0; i < myvector.size(); ++i)
…

Or, even better, use iterators!

See also:

http://www.cplusplus.com/reference/stl/vector/

First off, a 2nd generation tool would generally make mapping the repository to the hard-disk very difficult, so people would check out the entire repository onto their disks to work on. This was fine at the time, but lead to a lot of problems. One of them was that no-one was ever isolated from each other’s changes – if you ‘got latest’ there was a very good chance that some of the shared code would be broken in some subtle way. Do that a few times and you quickly get wary of the practice, and end up just getting the minimum you can get away with at any point in time.

But if we look at the 3rd generation, things are a bit different. We’ve got specific, well defined versions that cut across all our files. Now, I don’t know about you, but to me, it feels much better to pull out a single card from a deck (hopefully the top one), work on it, then put the changes back as a new card on the top of the deck again.

If someone was to take that card, then build it, assuming that the build works we *know* that that version is good … and it will be forever. If I want to make another change, I can take that card, in its entirety, with impunity, and *know* – with 100% certainty that it will work. This means that when I make changes, if they work, then the card I put back on the pile will work as well. And we can guarantee that by having an automated system take that card and test it, giving it a green tick if it’s ok.

Unfortunately, most 3rd generation tools let you mix and match cards, so the practice of getting the least you can get away with still kind of works with them. But look – I’ve got a stack of shiny cards here, and each card has been carefully crafted to guarantee that it’s good and has been verified by automatic tools that this is the case. But if you have half of your files from one card, a third from another and the rest from a half a dozen other cards – exactly what are you working with? How can you guarantee anything at all? How can you be sure that the card you’re about to put on the top of the pile will work with everyone else, when you’ve never tried it yourself? (You can’t have tried it – you don’t have it). It feels about as messy and taking a dozen playing cards and making a new card out of them by cutting them up and taping them together.

Of course, no-one’s perfect. Mistakes will get into the system. But if I take a bad card, and make my changes to it, the card I put back will also be bad – and I won’t know if it’s my fault or something that was already there. This means that we need to identify bad cards as soon as possible, and fix them immediately, so that we can be sure if we find a bad card with our name against it, the problem really was ours. The fix is just a new card, (since we can’t change the old cards), but the fact that we’ve spotted a bad card and fixed it quickly means that the next cards that get added to the system can be sure of a good stable base to work from.

One of the things I like most about this system is that if the worst comes to the worst, we can always scrap what we’ve got, get the latest known good card, and work with that. Instant cleanup is always available. We can also pull any particular card out and see if it had a particular problem, and if it did (and the previous card didn’t) we have a good start at working out exactly what went wrong and a shortlist of what files might be involved.

So we’ve not a got a mental model of what our 3rd generation system is doing. We’re working on a single card at a time and building up our stack of changes, verifying each one as we go. Next we’ll look in a bit more depth at how branches work, what they mean and how to work with them.

I think it’s because of this disinterested mindset that a lot of people completely missed what the 3rd generation of tools were about. And it’s not too suprising, advertising like “ACID compliant transactions – never partially commit a set of changes again!” aren’t exactly hinting at the revolutionary upheaval that had happened on the server side, it just sounds like CVS in a database.

But what happened was that the history in the repositories was completely pivoted around. CVS, VSS, et al would keep a single tree of files, and each file would have it’s own history. SVN, TFS, etc, don’t keep a set of files, they keep a set of trees. Each revision in the repository represents the state of every file in the entire source control tree at that point in time. Think of a stack of playing cards, each with the entire source control tree drawn on it. Each time you commit a change to any file in the repository, an new card is drawn up with the previous tree + your changes and put on the top of the pile. This means that the entire source control in the repository can be reliably retrieved by giving just one number – just tell the system what card you want and you can get it.

No longer do we need hacks like ‘labels’ to make sure that everything is kosher and retrievable, everything is labelled with each and every change as an automatic by-product of how the repository works. Of course, we don’t lose the ability to view the changes that were made to a particular file: given a path, we can diff the contents of the file in one revision with the contents in the next and get the changes just like we could before. But we also get a brand new ability – by getting the difference between one tree and the previous tree, we can see all the files that were changed at the same time – and it’s this set of diffs that makes up a ‘changeset’.

It’s this natural creation and handling of changesets that gives us a much better ability to migrate changes around the repository. Say I have two identical copies of a file in two places in the repository. If I make a change to file A, I can retrieve the changeset at any time I like, this results in a diff, and I can apply this diff to file B whenever I feel like it. Expand this concept to directories and work with changesets instead of individual diffs, and we’ve got the basis for a very flexible and simple branching and merging system. Branching is simply making a copy of a directory in another location, merging is simply getting the changesets that were applied to one directory, and applying them to the other. This branching and merging can all be tracked within the repository itself and voila, we have a modern 3rd generation system with full branching support.

So now we know what the differences between the 2nd & 3rd generation are, we’ll address in the next post some of the pitfalls that we fall into when we try to think of a 3rd generation system in terms of a 2nd generation system.

And don’t worry, I’ve not forgotten about the 4th generation, that’s coming, but since it builds on what the 3rd generation gives us, I think it’s important to square that away first.

Now, in all fairness, I don’t make my living by telling people how to use source control, but it’s one of the things that fascinate me about software development, purely because it’s so underused and misunderstood by a large percentage of my fellow developers.

The first thing that I think is important to grasp is that we’ve already churned through several generations of source control tools. I can easily identify 4 generations based on just the surface architectures involved, and each one represents a significant advantage over the previous generations.

So with the first generation of tools such as RCS, we had the ability to keep the history of the files we were working on. That was pretty much it, really. Working with multiple files was tricky at best, and sharing wasn’t exactly safe, as everyone would work in the same shared folder.

CVS & VSS came along and rescued us from this situation by wrapping RCS and keeping all it’s files in a repository, which could be accessed through a server on the network. This meant that all developers could have their own copy of their files on their machines, isolated from everyone else. Then when work was complete, the changes could be sent up to the server and new changes brought down.

This CVS style working is all very well, but it still has problems – namely that files are still versioned individually and that it can be very hard to reliably isolate changes that happened together across a range of files after the fact. SVN and others came on to the scene to rectify this by keeping all the revisions in a single stack of changes, numbered sequentially. Changes to multiple files could be kept within a single revision, or changeset, and you could see not only what changes were made to a particular file, but drill down into each change to find out what the full story was each time. This also makes branching and merging a far more tenable idea since a complete ‘changeset’ could be reliably migrated in full across different branches.

And most recently, after having 3 generations of centralised repositories, Git, Hg and others threw away the need for an enforced central repository. The effect of this is that it no-longer matters exactly where source code is located – what matters is the actual content itself, which could come from anywhere. Whether I pulled revision ’24g4g2′ out of the main company server, a junior developer’s laptop, a pirate website or the smoking remains of a tape in the server room, I would be guaranteed that the code was exactly what I was after, including all the history up to that point. Central servers are now used at your team’s convenience, not just because the tool demands one.

So there we go, a very short, and incomplete, potted history of source control tools.

Next we’ll look in more detail at the jump from 2nd generation to 3rd generation – and how so many people are thinking of it the wrong way round.

Know your features

Features are your lifeblood. Organise them, manage them, combine them into stories. Process them with great care. Tuck them in and sing them to sleep at night.

All features are great. It’s the organisation of them that gives you the edge. A small set of coherent stories that each stand on their own will be far more useful than a complex network of micro abilities. A well managed, complex set of micro-abilities will be far more use than a vague idea that something might be good.

Don’t go mapless

It’s usually a good idea to have a high level idea of what the overall solution can look like. This lets you communicate the overall idea, discuss how things can be done and ensure that the whole idea is indeed possible. This map may include all the normal good things, asynchronous message manipulation manifolds, distributed data denegrators, partial parallel processors et al.

Keep the map light

Rather than follow the waterfall model of expanding this map until it’s a thousand page document, use a lightweight method to convey the idea. If that’s a block diagram on the back of an envelope scrawled in a coffee shop, or a powerpoint presentation, that’s fine, but convey the whole idea.

Implement features, not the map

Ignore the map. Pluck a single feature from your carefully tended garden. Build the feature as if it was the only thing that existed in the world. Don’t look to the other features to see what they need, look just at what you’re working on right now. Don’t look at the map and decide that you need the main message manipulation manifold if a simple web service will be more than sufficient for this feature. If you’re working with a single, simple document, save it in a file. Don’t add that database just because it’s on the map, follow what the feature needs.

Finish the feature

When a feature is complete, it looks like you knew what you were doing when you started writing it. It looks like you had the same thing in mind all the way through, and weren’t flip flopping from one way of doing things to another. Of course, you didn’t know what you were doing when you started, and you did flip flop all over the place. But don’t let that show on the finished product.

Lock in the feature

Have a way to automatically confirm that the feature works. Confirm this on many levels, from the outside (feature tests) to the actual code (unit tests) and inbetween (integration tests). Once a feature is locked in place, you can rest in a comfort that you know it’s not going anywhere, it’s not going to be broken and you can leave it alone and work on other things. It’s very hard to properly lock in a feature after it’s finished, so start locking it in before you start the feature. Put a lock in place for each part of the feature, then put that part in place.

The locks are part of your feature too, shoddy locks do a shoddy job of locking your feature in place.

Check your locks

Make sure you know when locks are broken. Run continuous builds on all branches of your code. Keep stats: build time, unit test code coverage, static analysis, build warnings, anything that lets you see trends in how the software is progressing. Check them.

Rearrange from the ground up

After you’ve added a new feature, the whole solution should look finished. As you add new features, don’t just bolt new code on top of the old, but make it look like the whole solution was written with the intention to have all the features that are available from the start. No code is sacred, no code is untouchable.

Lean on your locks to ensure that you’re not losing functionality or breaking a feature. Lose any locks that aren’t needed any more.

Keep learning

Use what works, find what doesn’t. This doesn’t need any special discipline, it’s what you’re bitching about as you’re writing your code. Your product has a lot to teach you. Learn it. Change things.

Every now and then you’ll come across an Xml document that stores multiple data items in a single location. A good example is the settings* or vsprops** file for visual studio 9, which stores C++ search paths inside a single element – go look, it’s fun***. You usually find these when you’re trying to take advantage of one of the main strengths of Xml – ie, tool support like XPath or Xslt. In general, things that mean you don’t have to write or import specialised parsing libraries to manipulate and use the data.

Coming across one of these compound formats when using a generic tool means that work inevitably grinds to a halt. As if XPath and Xslt aren’t hard enough, now you’re thinking of using regex, or getting the values into variables and writing some procedural code, which incidentally, doesn’t fit at all inside your xslt, so now you have to completely change your approach to the problem. A simple tool that was to being scripted from a commandline has become a real development effort.

And it’s not just for scripting languages, even making use of these values in normal code can be odd. There you are with your document object, just extracting and using the values in a single operation as and when you need, then all of a sudden, you’re calling ‘split’ and iterating over the results. Of course you’re also throwing away the empty results at the end of split – because whatever formats the data for that value always puts a delimter on the end of every string. But you’re not throwing away all empty results though, are you, because an empty value in the middle of the sequence is fine, it’s just the last one that gets ignored.

And the Xml gets very hard to read very quickly when we’re using compound data. Do your individual values need quoting? Ok, now your document contains a load of unnecesary &quot; values well. And exactly when is that ; the end of an entity, and when is it a delimiter?

There are many really subtle complexities like this that can come up when you’re storing data in your own format. The more you’re stuffing into that value, the more likely you are to come across the awkward edge cases.

If you’re anything like me, when you’re writing this complex split logic, or trying to visually parse out the quotes from the semi-colons, you’re thinking: But formatting data like this is what Xml is for – why am I having to write this code?

There’s many possible causes of this stuffing. Xml does look very inefficient, a single character delimeter takes a lot less space than the bracketing of element names. But if you’re using Xml at all, is that really a concern? And if it is, would zipping the results give you similar gains? Xml compresses extremely well, and the various zips are all fairly ubiquitous tools that won’t trip up too many situations.

The other main reason, is of course, the one size fits all schema that doesn’t extend. But if that’s your schema, shouldn’t you be designing something thats flexible enough to at least store multiple values? Name-value pairs are great, but an Xml document that’s a glorified name value pair system is a bit weak.

Most people are pretty good at designing reasonably normalised databases, so why are we still stuffing so many values into a single column when it comes to Xml?

* In the case of the settings file, it’s not just an array, but an array of arrays. The outer array consists of the pattern: Platform1|Paths1|Platform2|Paths2. The paths themselves are stored as semi colon delimited paths: path1;path2. Good luck pulling the first search path for Win32 out of that with XPath, or even regex.

** VSprops is a single array of semi-colon separated paths, but often stores them quoted, for some reason. So you literally get the construct: &quot;path1&quot;;&quot;path2&quot;;.

*** For some definition of fun.

James Lorenzen: http://jlorenzen.blogspot.com/2010/02/commit-comments-conversation-with-your.html

A followup from Kohsuke: http://weblogs.java.net/blog/kohsuke/archive/2010/02/25/what-do-you-try-leave-your-commit-messages

I know that comments can often be treated as one of those horrible tick box enforcements from on-high. However, I’ve eventually come around to a massive appreciation of them, and try to include as much detail as is sensible in my comments. For one thing, it confirms to me that I’ve made a specific change for a specific reason, and not just ‘done some stuff’.

Like eveyone else, I’m curious on how other people view commit comments? Are they unnecessary? (after all, we have diffs). Is just a bugId enough? Should they be short one liners or mini release notes?

Is there scope for bringing commit comments into other parts of the documentation, ie, on the bug reports, release notes, etc.

What, if any, resistance do have to writing a comment when it comes time to checkin?

I’m going to try to cover the main points that Osherove gave in his talk, and the video will be available to those who wish to see it. In this post I’ll lay out the primers that are needed to understand how to write good unit tests. I don’t count myself as a unit testing guru, but I have made enough mistakes in this area to feel that I get where Osherove is coming from.

The first thing to understand is what a unit test is. A unit test does not rely on any environmental factor of any kind. This means no filesystem, database, network, specific user accounts or other tests. Anything that is relied on by the test is explicitly setup within the test. This also includes any functions that don’t return the same values each time they’re called – think DateTime.Now or Rand(). Any environmental dependance such as this means that the test must be classified as an integration test. Integration tests are a different kind of beast, and although still vital to the health of your software, are not the unit tests that we’re talking about here. It turns out that I’ve only ever written very few unit tests, the vast majority have been inadvertant integration tests.

The second thing to understand are the three pillars of unit testing: “Readability”, “Maintainability”, “Trustworthiness”. Each pillar holds up the other two pillars, you cannot have a maintainable test that isn’t readable. You cannot understand a test, you’re not likely to trust it. So if you find yourself criticizing a test based on any of these three pillars, then it’s likely that the test is not a good test and isn’t helping your software.

This leads on to the third fundamental point. A test suite filled with tests that aren’t good will not help us develop good software. In fact a bad test suite can be detrimental by forcing us to pay ever increasing testing taxes without them paying any dividends.

It turns out that there is a very clear pattern to good tests. They’re not really like normal code, so much, more like recipes. An template might be:

void <method under test>_<condition of test>_<result of action>()
{
    <initialise test items and condition>

    <perform tested actions>

    <assert on result>
}

In addition to this template, there are a few other fairly unexpected guidelines:

1. Do not use conditional logic in the test (no IF’s or SWITCHes). Each branch of the logic is a separate test condition – split out to separate tests.
2. Don’t assert in loops. Each loop is a separate test condition, split them out or recognise that they’re duplicates and remove them.
3. Assert only 1 thing per test. Multiple asserts are generally testing for different results or conditions, so they’re separate tests.
4. Give context-relevant names to the values being used. For example, calling doSplit(INVALID_LIST, UNUSED_FLAG) says more than doSplit(“here there”, false)
5. Separate out each step. Whilst you might find that you can initialise, act and assert all on one line, it doesn’t mean that it’s readable.
6. Do use standard refactoring techniques to re-use common code parts, but all the above advice still holds for these refactored methods.

He also gives some suggestions for the lifecycle of tests:

1. Add new tests when new methods, conditions or results become relevant.
2. Remove tests when they become irrelevant.
3. Only modify tests if the API changes or they need refactoring.

This is a very quick distillation of the advice given. Osherove himself provides a lot more detail and justification for each point here.

See his site and other links at http://osherove.com/