I've always felt that connecting to someone's open access point for low-bandwidth, legal activities is perfectly acceptable. Encryption is cake to set up and even if they're just using WEP, it will let the world know that they do not want people connecting to their network. Now if you're going to fire up wireshark and start sniffing their traffic, that would be wrong. If you download a lot or do illegal things from their networks, that is also very wrong. At the end of the day, it's the owner of the ap that will have to deal with the consequences of your actions. But if all you want to do is look at a few web pages or watch a youtube video or two through an open network, I don't see what the problem is.
People have been using the 'if you house door is unlocked..' analogies, and I don't think it accurately describes the situation. Their routers are chirping 'I'm here, this is my network and you don't need a key'. I can't think of any good analogies at the moment, perhaps someone else can come up with one that depicts the situation better. For me, the bigger issue is, like using the bathroom at a fast food joint, whether or not you really want to use it. Do you really want to connect to a network you have no control over, where everything you do may be logged? Unless you're really desperate, I think not.