Jake Herringbone

Why TIGER/Line can be a bad source of geolocations

I came up with this neat map in Google Earth, showing the difference between Google’s coordinates for some addresses in Palo Alto, CA, and a line connecting those to the TIGER/Line-backed geocoder.us coordinate, colored blue-to-red based on how far off it is. The TIGER/Line data only has endpoints for a range of addresses, and the geocoder.us Perl module tries to do a linear interpolation to get addresses in the range. Obviously this doesn’t work well for non-linear streets like this one…

FeedBurner, finally!

I have sort of known in the back of my head that I ought to have my feed managed better. It would be nice to see who is subscribed, both because I don’t really want to blog to hear myself talk, and also because you get an incentive to write more interesting and useful stuff if there’s an audience.

So I hooked up feedburner, and was pleasantly surprised by how useful it is. Not only do I get my basic readership metrics, it also manages the trick of getting existing subscribers under the metrics (by telling blogger to redirect requests for the feed URL) and lets me fold in daily bookmark digests and photos.

Hopefully my 2 readers (?) will also enjoy my random API and documentation bookmarks (?) and at the same time, my attempts to do film photography (?).

Now I am wondering how to advertise my Twitter and Skype and Facebook and so on. I guess it’s good to make them widgets on the right side of the blog, and just assume that one URL will give away my complete identity. Maybe hCard is good for this? I do feel like my “Card”, whether it’s vCard or whatever, should have my contact info including many URL’s, and should be easy to broadcast my card to *everyone*, not just online friends. Then the offline people like grandma get my new phone number, and online people get the skype ID, and all is well.

That would be another reason to try out one of the email->snail mail gateways – I can subscribe my grandmother to my vCard feed and she’ll get it via postal mail… slick. Except then she’ll call me wondering why I sent her a letter, just because I added a web 2.0 service to my card… maybe I should think about that some more.

liveblogging from barcampdc

I saw the technorati page over justin’s shoulder so I should probably contribute. The conference is cool, it’s a little presentation-heavy rather than interactive, I guess that’s just because it’s hard to get people comfortable with other participants when there are 9 sessions in 8 hours.

I was thinking about how attendees at this sort of conference are into scripting, web 2.0, design, user experience, “lightweight”, fun stuff. There isn’t much community forming around enterprise, Java stuff anymore. The NoVA JUG has been trying to recover from near-death, as far as I can tell from the mailing list. Maybe dynamic, smart people just want to spend their free time on the fun stuff that’s hard to get paid for.

Building codes

When I was at my last government contract, we were developing a Single Sign-on solution that dragged on forever due to Oracle bugs, misunderstanding of the requirements, politics, and the other usual suspects. In the meantime, across the street and plainly visible from my seat, a building went up, a nice 8-story office building with a parking garage. They literally broke ground, dug the foundation, erected cranes, put up the building, added HVAC, walls, windows, and it was basically done, while we still hadn’t implemented a working single sign-on solution across all the applications that were supposed to have it. We were just typing, and these guys had brought in truckloads of materials and assembled this giant structure.

This has always struck a nerve for me, especially because I think erecting a building is the single best analogy for software development. I’ve heard others, like from my crazy first boss who always said that if we were missing a certain feature, it was like a car with three doors. But a car isn’t a good analogy for development – they are mass-produced, never modified, consumer products with a pretty small set of complexity – combustion, electrical, mechanical, suspension, seat cushions.

Buildings, on the other hand, share a TON of interesting features with software. They are large projects, similar to each other in form but each unique in their implementation. There are a number of skilled workers, with different areas of expertise that must participate. There is a complex schedule that must account for the dependencies between pouring the foundation and inspecting the concrete and so on.

Pre-fab is another nice part of the analogy. I don’t know much about it, but I think there are a lot of standards in buildings that make it possible to get lots of your materials in a partially constructed state. Door frames are a similar size, and you can just buy the door already hinged into a frame, nice and square and level. I see trucks with rafters for houses, already assembled and ready to “raise the roof”. Same goes in software, where standards like those from the JCP let us plug in pre-assembled components rather than building them from scratch.

Looking further, there is an agency problem. The people who will ultimately inhabit and use the building are rarely stakeholders during the construction. The builders do the construction, but don’t have to live in the structure or deal with the problems that the maintenance company uncovers. In software, end users are similarly more like tenants than they are landlords.

Yet, in real estate, there is much less sense of risk or mystery about the quality of the work, whether the building will stay standing for 5 years, whether it’s safe and structurally sound, if it was built by qualified craftsmen. When you see a new building go up, you’re pretty confident that it will get finished and be a successful project, as long as the funding stays intact and the contractors aren’t too disreputable. But in software, while companies focus a lot on a structured development process and testing, they don’t have a grasp on what the code ends up looking like, whether the system is structurally sound and safe from fires (= production emergencies of course).

I want to explore why this is. I think I’ll look at it over a series of posts and maybe develop this into an article, because I think there’s something very interesting that software development could gain from civil engineering practices.

To start answering questions, I’m thinking of how buildings work. First, there are house inspectors. I can’t just look around a house they might buy, and figure out if things are structurally sound, yet I expect I could hire someone with this skill to give an honest, impartial appraisal. I’m sure this is the case for new, commercial construction as well. Yet I’ve never heard of a third-party auditing/appraisal service for software.

Next, the inspector has a set of building codes, enforced by local law, that guide him to check many important features, and provide an objective baseline of criteria that the building should satisfy. The most popular set of codes is the International Building Code – I think I’ll poke around in there and see how they manage to capture the complexity of what they assess. Just as with software, I’m sure there are a lot of grey areas and places where there’s some discretion the architect can take, yet there’s a right and wrong way to assemble the house. We do have some such guidelines in software, but I think this could be improved based on how the civil engineers do it.

Finally, there’s the qualifications of the craftsmen. I like posts like this from Ken Auer on apprenticeship in software that talk about how software engineers learn their craft. You really don’t learn it in school, or from a book. (Maybe Cisco or MS engineers can do book learning and take a certification test, but I don’t know any good software engineers who do this.) Sure, programming is not all science, there is art to it as well. But there’s also art in architecture, and maybe even in quality welding, for all I know, and those guys don’t just learn their craft by copy/pasting some code and trial-and-error.

So, if I keep this line of thinking running through my head, I think I’ll write about:
- How building code works, how it is applied to real world buildings, and whether the software analogy works
- How building inspectors can apply the building code, and whether there are such people for software
- The way craftsmen like electricians, welders, as well as artists like architects and designers, learn their craft and conform to the code

How do buildings go up faster and with less risk than software??

I’m interested in comments!

Eclipse Europa and Modeling

I needed to whip up some class and sequence diagrams today, but had some time to spare first. So I thought I’d reevaluate how easily I could get a modeling tool working that’s code-aware.

First, I saw that sometime over the last week, Eclipse released Europa. I’m still a little confused how that relates to Eclipse 3.3 – I think 3.3 is the new version, which is released, however they encourage people to get Europa instead, which is a bundled release for 3.3 along the lines of Callisto.

That was only the beginning of how fragmented the development efforts around Eclipse seem to be. I started poking around in the ever-disappointing GEF/EMF/MDT/UML tooling. It seems like they’ve abandoned/renamed/re-organized these projects more frequently than they’ve gotten meaningful releases out the door, and although there’s a project called UML2 Tools that’s supposed to provide class and sequence diagramming, there’s zero documentation on how one would use it in Eclipse. I get the impression that despite the large number of eclipse tools and projects that make up the modeling stack, none of them is meant to be used by “users”, they’re all for “toolsmiths” (http://wiki.eclipse.org/GMF_Tutorial)

And this toolstack always caused the dreaded Eclipse plugin spaghetti mess for me in the past, so I’m still not really impressed with it. Maybe I’m completely misusing it, but I’ve put in a good faith effort to read some docs and learn what to do, and it can’t be that hard. So, time to move up the stack to the vendors who build on top of the eclipse tooling.

One tool I’ve used before, Omondo, seems to have gone through some nasty personnel issues, because now there’s a competing plugin, eUML2 that apparently came from Omondo’s CTO and a mass defection of engineers. So I gave eUML2 a try.

It’s nice to see it builds on the Europa/Eclipse toolstack concisely. It helped me through the reverse engineering, although I got XDoclet-style annotations in my source files rather than Java5 annotations. It’s terribly slow even on my MacBook Pro, but it does eventually pull in the classes I want, has a very nice presentation (including full package name, yellow warning icons, etc), and manages to discover some of the connections between the classes. The sequence diagram editor had some problems with the view falling a step behind the true model state, which I had to work around. Also, it didn’t leave enough margin for some text, although it did so consistently. Overall, it was usable, and I got my diagrams done… I think it would drive me nuts for a real design phase though.

Also used the new Spring IDE 2.0 which was just released – I didn’t hit any new features but I do enjoy the bean dependency graph, that makes for nice quick documentation too.

blogging from the iPhone

just a quick post to announce (brag) that the iPhone is nice

Apple Proselytism

I try not to be a total fanboy when I can help it. It’s really easy to do this with Apple computers – once you’ve switched, you are consumed with a desire to open they eyes of your friends who are still in the PC blindness stage.

I want to share my favorite description of the virtues of Macs, something short enough and indicative enough to give it all away.

“When you’re using a PC, and an error occurs, sometimes a dialog tells you to contact your systems administrator. This is because they expect you to HAVE A SYSTEMS ADMINISTRATOR!!! You’re at home, you should never be told this. Macs don’t expect that you have one.”

JavaOne wrap-up

It was a long travel day yesterday to get back to DC. Here’s the rest of the

Things I learned:

Glassfish 3 has a really neat-sounding architecture. The app server core is like a 20K jar and launches in half a second. It then has a maven-based dependency system that causes it to load OSGi modules that add things like JEE container, servlet container, HTTP server, PHP support, JRuby on Rails (directly from the app directory – no WAR building), Phobos, etc. So you can add a number of containers to your one running Glassfish instance and then deploy apps into it. (I think Cargo is another way to do this.) It also means it’s a pretty extensible app server. There’s some nice classloader magic, with a public and private classloader for each module, to make all of this possible and avoid typical J2EE classloading nightmares. Also, their build scripts create configuration files out of annotations, as opposed to introspecting them at runtime.

Rod Johnson’s talk was packed. I wonder if Sun intentionally gave him a smaller room than his popularity demands because they’re not sure they like him undermining their EE specs. He is more like a corporate CEO than a coder – he’s very driven and seems to want market share more than hacker credentials. Interface 21 got $10M in funding at the beginning of the week so maybe this gave him big britches. Spring is adding annotation support – he did mention Guice in passing. He’s also certain that OSGi is the next big thing, Interface21 is working closely with them. There’s a Spring-OSGi module with the implementation. Sounds like an important thing to go learn!

Tor Norbye of the Java Posse kicked some butt! Not only did he give a talk on his Ruby contributions to NetBeans, he also was up on stage in front of the entire show at the last general session (“Toy Show”) and demonstrated his speed in Netbeans and the number of keyboard shortcuts he must know. I wonder, it looked like they were all presenting on Solaris boxes, and he codes on a MacBook Pro, so wouldn’t the modifier keys on the keyboard be hard to find? Maybe he practiced… I did download NetBeans 6 and opened my RoR project in it – there are some nice things. I used the local history, completion, and navigation features, and they worked pretty well. The real issue with IDE support for Ruby is the debugger – in C, there are a couple debuggers, but there’s no equivalent of JPDA that lets you remote-debug a running Ruby interpreter. For JRuby, someone is working on a debugger, but the Sun guys are pushing for contribution to the Debug Commons project which will help all the IDE’s have a consistent way to invoke debug operations. They’re looking for help – Gabe?

Hibernate Search looks like a nice union of Hibernate meta-data with the Apache Lucene search engine. You really can’t do good search with Hibernate criteria – the LIKE operator in SQL is way too slow. I ran into this with a search application at my gov’t client that tried to create a massive SQL query for every user search, which was a really bad approach. Hibernate search allows you to add some annotations to your JPA class to indicate what fields should be full-text searchable, then it handles building the Lucene index, and even keeps the index up-to-date if you change the entity. There’s a JMS-based solution for clustered front-end machines to share a master Lucene index. And, extra nice, the Lucene search is wrapped in a standard Hibernate Query object so your DAO code doesn’t have to know it’s not using SQL for some queries. Nice! I did notice Emmanuel called something a “workaround” – this made me wonder about some of the Hibernate code and whether they’ve allowed that mentality to get into the codebase – but this looks like a really nice way to support user search in Java.

Selenium is a great way to automate functional tests of a web app. Unlike HTTPUnit and other Request-Response level testing tools, Selenium runs a true client within the browser. This allows it to work with Ajax apps, for instance. There’s a Firefox plugin that records your interactions with the app and turns it into a JUnit test case – also very handy. Selenium gets its name from the element, because it’s used as a remedy for Mercury poisoning. Man, Mercury has been a terrible vendor in both of my last two clients.

Chris Richardson (author of POJOs in Action) had a nice presentation on testing, most of which I already knew. But he had a nice way of expressing why good guys don’t win when it comes to unit testing and TDD – the “paradox of excellence”, which is where you are rewarded for fixing the nasty bug right before production (or in production, shudder), while you are not rewarded if things quietly work and you leave every day at 5.

Feeds and REST are the community response to the enterprising of web services with XML-RPC and SOAP. There was a great graphic on one of the slides from the Rails guy, David Heinemeier Hansson:

So true.. I went to a talk on WS-Policy and pretty much wanted to shoot myself after the barrage of consultant-speak and enterprise BS. So – feeds – Rome is the Java library of choice because it handles the mess of RSS and Atom formats (RSS never had a spec!), but it’s JDOM based. Watch for it to merge with the Atom-only but STAX-based feed library (A-something…)

There’s even an alternative to WSDL for the lightweight REST crowd – WADL. However, there are a lot of security issues lingering in this whole mash-up environment, basically, you should only mashup to sites you really trust like Google. Assuming the stuff about Chinese censorship hasn’t put you on a crusade…

Domain-specific Languages are really great for solving some problems, and I started to think of them as an alternate solution when you consider a rule engine. Rules are good for implementing a lot of business requirements that would involve big chunks of repetitive, procedural Java code. But, a lot of times, those requirements aren’t globally true facts about the business, they’re very contextual and too simple, and there are too many of them, and a rule engine isn’t a good fit. Perhaps you want to code the solution, just not in Java. You want a language that expresses the business concepts – aha! Now, the talk from another Interface21 guy went into implementation details for writing your DSL in JRuby, where it is easy, then easily integrating this with your enterpris-y Java app. Managers don’t have to freak out since it’s just another JAR dependency, but it’s a much better way to create and use a DSL than any solution in Java (I don’t actually know of any). I think this is a great addition to the programmer tool-belt. Also, this guy actually demanded a raise from Rod during the presentation, I guess the $10M has gotten to their heads… not sure I want to work there anymore.

My overall observations from the conference:

• JRuby is going to make a major impact on Java development, both in the web and enterprise (I don’t know about micro… but I think there was a presentation on that). Java makes some things hard to do, and getting new major releases into common adoption takes a couple years (still not on Java 5 at a lot of big companies!). So waiting 4 years for closures isn’t going to compete with a single jar you can drop into your app to start including Ruby classes. I’m getting The Ruby Way, a recommendation from the JRuby DSL presenter.
• It’s important to work with smart people. Since they share a lot of common motivations, if there are a lot of smart people at a company, all your other criteria are probably met as well. I’m starting to think there are two distinct camps of brilliance, though – the artist type and the businessman type. Consultants seem to be the latter, and innovative ideas come from the former. Maybe you can distinguish them based on who has a Mac.
• All of the good software companies are in the Bay area. I hope this isn’t entirely true.
• I want to go to JavaOne next year.

Lessons from JavaOne

Here are some things I have learned so far:

• If you’re a Java geek, then being next to Rod Johnson, meeting Howard Lewis Ship, seeing the Java Posse interview the Cenqua team, are probably like most normal people seeing Julianne Moore at the grocery store. (I also ran into an architect from my last client and a manager from the client before that!)
• With static analysis (ie. FindBugs), you find errors that are not caught by the compiler. This is a lot like a document where you make a typo that is a correctly spelled word, so it doesn’t get a red underline. It’s nice to have a grammar checker for those situations. That’s one way to think of static analysis. It was funny how Bill Pugh ripped on Josh Bloch’s JVM code for having obvious typos. Apparently there isn’t much underway to do this sort of analysis on scripting languages, but he thinks it’s even more important there since the syntax is more relaxed.
• WS-Policy is completely un-fun and sounds like consultant-y BS
• Romain Guy is a true programmer rockstar because he has to beg for breakfast donations at the beginning of his talk due to being up till 4:30 am, trashed.
• Non-RESTful web services are tunneling through HTTP. I hadn’t thought of it that way before. RESTful services, on the other hand, take advantage of all the infrastructure set up around HTTP since things like caches work at the protocol level.
• The proper way to distribute business rules to your running Drools (or other engine) instances is by publishing them to a content management system. JBoss Rules will soon be integrated with Apache Jackrabbit for this purpose.
• Closures are needed in Java, anonymous inner classes don’t cut it. Many APIs and language constructs would be less awkward and verbose if we had them. Neal Gafter is very opinionated.
• A workflow engine can be useful for business intelligence if the history of workflows is data mined.

At JavaOne!

I’m in SF at JavaOne this week. Have to head off to an Open Source SOA talk, but here’s a quick post…

Maybe it’s because I selected RESTful web services, Grails/Trails/Sails, and Closures, but I keep seeing people trying to catch up with RoR. They admit Java makes the easy problems hard (Neal Grafter’s words) and want to develop more APIs and language changes that would give Java some of the same capabilities. I’d really like to hear more about JRuby and see if we can leave Java alone and admit that adding APIs isn’t the same as using a different language.

Went to see the Java Posse live. They had giant foam hats, and talked about the keynotes which I missed. (I flew in yesterday morning, which meant being up 22 hours, and I couldn’t stay up to get beers after the BOF with them ) It was great to see some of the industry rockstars yesterday. Hope today is as interesting.