Don't look now, reject.

Many people have been writing about the Harvard admissions fudge-up this week, and all I can add to the discussion is: man am I glad I'm not doing that stuff anymore. Once upon a not so long ago when I was at Connexxia, we all dreaded the yearly delivery of online decision letters. The process was always an intense couple of weeks of data synchronization, applicant checksumming, and paniced teleconferencing that always ended up consuming at least four people full time. And then when the switch was finally thrown, the real worrying began. Inevitably, the following day would bring forth a handful or so applicants who received the wrong letter or experienced some wacky response from the system. In almost all cases, the error was the result of human error in the admission office (though the onous usually fell on us to prove it as such). We did the online notification for only one school for several years, and had our share of isolated and relatively small issues. In fact, we always hoped to expand the service into a regular offering, but the process seemed to resist scaling. Plus, there was the looming threat of flubups, which had a way of getting a small struggling company into the news in a way that resisted growing.

Ironically enough, as I look at the string of miserable failures, including this most recent one, we may have been sitting on the best online decision delivery system out there. The problems we did encounter (one was newsworthy) pretty much fell into one of three categories. The most common was human error on the part of admission personnel. Next, were errors associated with lags in the data synchronization process. Inevitably, someone would rush to make a lot of changes at the last minute not realizing that the crucial last data feed had already processed in the wee hours that morning. Generally, that resulted in applicants being told their decision was unavailable which was the default behaviour of the system if any number of validity checks failed. The last batch of errors were usually due to a misconfiguration of the software. There was a very dense mapping of decision code to letter template that often required some last minute tweaking. One of the things I never realized before I saw the process from behind the scenes was how many different letters go out. I had always assumed there was just one rejection letter that went to all the unfortunate, when in fact there may be 15-20 different versions of the letter and each has it's own special code. On a few occasions, we had to scramble to the server after the decisions were live to fix a slightly broken mapping. We were, of course, smart enough to isolate the mismatches that would cause large scale grief. The mappings for admits were nowhere near the mappings for denies so the errors we fixed were nothing to get too upset about.

What amazed me the most when I first heard that companies like ApplyYourself were beginning to offer these type of services to their clients, was that it seemed to imply that they had worked through all the details of the process and had a solution that scaled to their full client base. Like I said, we only did it for a single school and were not eager to expand it to a general offering until we could put some signficiant resources to making the paranoid checking more efficient. After this week, my amazement has ceased and I'm beginning to understand why we are seeing so many problems with these systems. For example, we assumed from the first sketch that people were going to be editing URL's. For one thing, the students who were accepted tend to want to send their letter around to friends and family, so we got quite a bit of URL wackiness. We were even aware that students would share their login information (which was absolutely against the rules) so that their friends could see their letters, and so we logged that information as well. It astounds me that a system holding such sensitive information would be released (or even worse, sold) when one of the most obvious non-standard uses allowed applicants to see their decisions early. I use the word “use” here intentionally because changing the url in the browser falls into the realm of use, not misuse and certainly not attack or hack as Harvard seems to think it does. The world of web development has long known that a document available on a web server without protection is publicly available even if no other link on the web references it. To call it hacking goes a long way to excuse major oversights in both the software's design and Harvard's process for evaulating software.

And as for Harvard's claim that they intend to reject all who peeked, I think that's probably another error in judgement. I would just leave it alone. While it may have been wrong for those applicants to access confidential information from a server, there are still enough questions about the system, in general, to make it fuzzy. For instance, if Harvard is exposing an applicant's personal information, does that applicant have a right to try to confirm it? And if the system falied at it's primary goal of delivering decisions to applicants at the proper time, can it really serve as dependable evidence to support rejection of their application? From what I've heard of the starting salaries for HBS grads, it wouldn't be wise for these folks to just tuck their tails and walk away. Except for the girls, they're not smart enough to go to Harvard anyway.