IT

I ran the Cyrus IMAP server for almost a year on my home server, but I recently switched to Dovecot. I originally used Cyrus because of its demonstrated scalability and in part because it is a product of my father’s alma mater, but it is quite hard to set up, and quite brittle to changes in its dependencies.

The last straw was when I tried unsuccessfully to set up another instance of Cyrus on a server, with the exact same configuration files and permissions, but different versions of the Berkeley DB and Cyrus SASL libraries, and it simply wouldn’t cooperate. In disgust, I downloaded Dovecot, compiled it and installed it in less time it took me just to figure out that Cyrus wouldn’t allow me to authenticate because the ever-crufty SASL library failed in a new inscrutable way. I had also never managed to get Cyrus’ SSL to work reliably, it is nearly effortless with Dovecot.

Dovecot is much easier to build and manage, does not have dependencies on unreliable cruft like the Cyrus SASL library, and is much easier to integrate with procmail, SpamAssassin and other goodies thanks to its use of the Maildir format rather than a proprietary database cum filesystem structure like Cyrus. From what I have seen of the internals of the Cyrus 2.2 “skiplist” database back-end (which replaced the BerkeleyDB back-end used in previous releases), I have a hard time believing it is significantly more efficient than Dovecot, if at all.

One problem was migrating my email – I have pretty much all my email since 1995 in my IMAP mailbox, migrated from various Emacs Babyl mailbox files or Exchange PSTs over time. The Dovecot Wiki points to this migration script, but for recent versions of Cyrus like the 2.2.12 I ran, it has two major shortcomings:

1. It will not preserve the flag that indicates whether an email was read or not.
2. It does not preserve the delivery timestamp for the emails so they all look as if they were delivered at the time you did the conversion.

I wrote my own migration script in Python, cyrus2maildir.py, to avoid these shortcomings. It does not preserve the “replied to” flag, but the Read flag is carried over, as is the delivery timestamp (in some edge cases like emails you sent, it has to guess, though). This is not a very polished program because I spent far more time on it than I had anticipated, and basically stopped once I got it working, but it should be usable, or at least a starting point for anyone with some Python skills. Of course, it can also be used by users of other Maildir++ based servers like Courier.

The script should guess most parameters, and will dump the emails to a directory named Maildir.cyrus/ in your home directory. By default, your user will not have read access to the Cyrus mail spool, you may have to alter permissions (I do not recommend running the script as root). For the syntax, just type: cyrus2maildir.py -h

On an unrelated note, Solaris 10 users may find the SMF manifest and method useful to automate service management and fault recovery. To install them, copy the manifest to /var/svc/manifest/site and the method to /lib/svc/method and install them into SMF with the command: svccfg import /var/svc/manifest/site/imap-dovecot.xml

After an abortive experiment with a Nokia Symbian Series 60 smartphone, I bought a Palm T|X on Wednesday, the very day it was announced. I find PDAs superior to fiddly, fragile and cumbersome laptops, and have owned no fewer than 9 Palm compatible handhelds (*) in the last 5 years, which means I upgrade handhelds at least three times more often than my main (desktop) computers. My previous PDA is a Palm Tungsten T3 (I actually bought it after the T5 was announced, so underwhelming the latter is). I even obtained a spare T3 in case the first one broke (since given to my father). I am not entirely sure yet as to whether the T|X is really an upgrade. Here are some first impressions after a few days of use:

Pros:

• Built-in WiFi. No more fiddling with the easily lost SDIO WiFi card.
• A better browser. Blazer feels much snappier than Web Pro, specially with the new Fast mode (disables CSS and image loading).
• More memory, non-volatile if the battery fails.
• Lighter.
• Can actually dial and send SMS on a Nokia 6230 via Bluetooth

Cons:

• Plastic construction feels much less robust (but at least it is not pretending to be metal like the E, E2 or T5, that’s just tacky).
• No voice recorder, charge LED or vibrating alarm. I seldom use the voice recorder, as I prefer taking notes on 3×5 jotter cards, but the voice recorder works when you have to capture that elusive idea while driving.
• 20–25% slower processor. Graffiti2 is noticeably slower to respond, for instance.
• The flip cover with the hinge on the side is less convenient than the one on top, which flips up like a reporter’s notebook, in one fluid motion.
• The SD slot has a plastic filler card, not a spring-loaded cover.
• Bigger. Many people complain about the true Tungstens’ slider, but it is very natural to use, and much more convenient than the power switch.
• The stylus has a nice heft to it, but is not as substantial as the T3’s, and less easy to extract from its slot.
• Yet another connector design incompatible with previous accessories. The cradle is an expensive option.
• The home icon on the status bar has disappeared. This is very annoying in daily use
• The application buttons and the 5-way navigator are less responsive and smaller. The T3 has generally superior haptics (feels much better in the hand).

The only potential deal-breaker is the slower Graffiti performance (there is a visible lag). I will probably keep the T|X due to the convenience of integrated WiFi, but the T3 is a superior device in almost all other respects, in the same class as the Palm V as one of the PDA world’s truly outstanding designs. If Palm were to come out with a new model marrying the WiFi and newer software stack of the T|X with the solid construction and faster processor of the T3, I would definitely upgrade again.

(*): Handspring Visor, Sony Clié T615C, Kyocera QCP-6035, Palm Tungsten T, Sony Clié UX50, Palm Zire 71, Palm Tungsten T3 (x2), and now the Palm T|X.

Update (2010-05-16):

The T|X was the last Palm device I bought. I switched to an iPhone in 2007 and never looked back.

There has been much speculation recently about a possible rift in Internet governance. Essentially, many countries resent the US government’s control over the Internet’s policy oversight. They advocate the transfer of those responsibilities to the International Telecommunications Union (ITU), a more multilateral venue. The big news is that the European Union, which previously sat on the fence, came out strongly in favor of this proposal. Unsurprisingly, the US government is hostile to it. More surprisingly, I agree with their unilateralist impulse, obviously for very different reasons. I was planning on writing up a technical explanation as most of the IT press has it completely wrong, as usual, but Eric Rescorla has beaten me to the punch with an excellent summary.

Many commentators have made much hay of the fact the ITU is under the umbrella of the United Nations. The Bush administration is clearly reticent, to say the least, towards the UN, but that is a fairly widespread sentiment among the American policy establishment, by no means limited to Republicans. For some reason, many Americans harbor the absurd fear that somehow the UN is plotting against US sovereignty. Of course, the reality is the UN cannot afford its parking tickets, let alone black helicopters. American hostility towards the UN is curious, as it was the brainchild of a US president, Franklin D. Roosevelt, its charter was signed in San Francisco (at Herbst Theatre, less than a mile from where I live), and it is headquartered in New York.

The UN is ineffective and corrupt, but that is because the powers on the Security Council want it that way. The UN does not have its own army and depends on its member nations, specially those on the Security Council to perform its missions. It is hardly fair to lay the blame for failure in Somalia on the UN’s doorstep. As for corruption, mostly in the form of patronage, it was the way the US and the USSR greased the wheels of diplomacy during the Cold War, buying the votes of tin-pot nations by granting cushy UN jobs to the nephews of their kleptocrats.

A more damning condemnation of the UN is the fact the body does not embody any kind of global democratic representation. The principle is one country, one vote. Just as residents of Wyoming have 60 times more power per capita in the US Senate than Californians, India’s billion inhabitants have as many votes in the General Assembly as those of the tiny Grand Duchy of Liechtenstein. The real action is in the Security Council anyways, but they are not fully represented there either. Had Americans not had a soft spot for Chiang Kai-Shek, China, with its own billion souls, would not have a seat at that table either. That said, the Internet population is spread unevenly across the globe, and the Security Council is probably more representative of it.

In any case, the ITU was established in 1865, long before the UN, and its institutional memory is much different. It is also based in Geneva, like most international organizations, geographically and culturally a world away from New York. In other words, even though it is formally an arm of the UN, the ITU is in practice completely autonomous. The members of the Security Council do not enjoy veto rights in the ITU, and the appointment of its secretary general, while a relatively technocratic and unpoliticized affair, is not subject to US approval, or at least acquiescence, the way the UN secretary-general’s is, or that of more sensitive organizations like the IAEA.

My primary objections to the ITU are not about its political structure, governance or democratic legitimacy, but about its competence, or more precisely the lack of it. The ITU is basically the forum where government PTT monopolies meet incumbent telcos to devise big standards and blow big amounts of hot air. Well into the nineties, they were pushing for a bloated network architecture called OSI, as an alternative to the Internet’s elegant TCP/IP protocol suite. I was not surprised — I used to work at France Télécom’s R&D labs, and had plenty of opportunity to gauge the “caliber” of the incompetent parasites who would go on ITU junkets. Truth be said, those people’s chief competency is bureaucratic wrangling, and like rats leaving a ship, they have since decamped to the greener pastures of the IETF, whose immune system could not prevent a dramatic drop in the quality of its output. The ITU’s institutional bias is towards complex solutions that enshrine the role of legacy telcos, managed scarcity and self-proclaimed intelligent networks that are architected to prevent disruptive change by users on the edge.

When people hyperventilate about Internet governance, they tend to focus on the Domain Name System, even though the real scandal is IPv4 address allocation, like the fact Stanford and MIT each have more IP addresses allocated to them than all of China. Many other hot-button items like the fight against child pornography or pedophiles more properly belongs in criminal-justice organizations like Interpol. But let us humor the pundits and focus on the DNS.

First of all, the country-specific top-level domains like .fr, .cn or the new kid on the block, .eu, are for all practical purposes already under decentralized control. Any government that is afraid the US might tamper with its own country domain (for some reason Brazil is often mentioned in this context) can easily take measures to prevent disruption of domestic traffic by requiring its ISPs to point their DNS servers to authoritative servers under its control for that zone. Thus, the area of contention is really the international generic top-level domains (gTLDs), chief of all .com, the only one that really matters.

What is the threat model for a country that is distrustful of US intentions? The possibility that the US government might delete or redirect a domain it does not like, say, al-qaeda.org? Actually, this happens all the time, not due to the malevolence of the US government, but to the active incompetence of Network Solutions (NSI). You may recall NSI, now a division of Verisign, is the entrenched monopoly that manages the .com top-level domain, and which has so far successfully browbeaten ICANN into prolonging its monopoly, one of its most outrageous claims being that it has intellectual property rights to the .com database. Their security measures, on the other hand, owe more to Keystone Kops, and they routinely allow domain names like sex.com to be hijacked. Breaking the NSI monopoly would be a worthwhile policy objective, but it does not require a change in governance, just the political will to confront Verisign (which, granted, may be more easily found outside the US).

This leads me to believe the root cause for all the hue and cry, apart from the ITU angling for relevance, may well be the question of how the proceeds from domain registration fees are apportioned. Many of the policy decisions concerning the domain name system pertain to the creation of new TLDs like .museum or, more controversially, .xxx. The fact is, nobody wakes up in the middle of the night thinking: “I wish there were a top-level domain .aero so I could reserve a name under it instead of my lame .com domain!”. All these alternative TLDs are at best poor substitutes for .com. Registrars, on the other hand, who provide most of the funding for ICANN, have a vested interest in the proliferation of TLDs, as that gives them more opportunities to collect registration fees.

I bought a Nokia 6682 phone a couple of weeks ago, as an upgrade for my Nokia 6230. Actually, I have my parents signed up on my service plan, and I was planning on sending them the 6230 to replace an older phone they lost, and taking advantage of this as an excuse to upgrade… The 6682 is a Symbian “smartphone” sporting Nokia’s Series 60 UI, and I was influenced by rave reviews like Russell Beattie’s. In recent years, Nokia has been churning out phones with crackpot designs and dubious usability for coolness’ sake. There must have been a customer backlash, as their recent phones like the 6682 have a much more reasonable, reassuringly boring but functional design. Another reason is that Apple’s iSync only works with Nokia’s Series 60 phones, and it will sync photos from the OS X address book.

I returned the phone for a refund last Friday, because the ergonomics are simply atrocious, and from a usability point of view it was actually an unacceptable downgrade from the Series 40 (non-Symbian) Nokia 6230. The low-res 176×208 screen has significantly lower information density than the 320×480 or 640×480 screens now standard on most PDAs, and makes web browsing almost useless. The only thing it has going for it is a semi-decent camera.

Even basic functionality like the address book is poorly implemented. When you scroll down your contacts list, you can select one to decide whether you want to reach them on their home or mobile number. The problem is, the next time you want to make a call and access the address book, you do not start afresh, but still in the list of contacts for the previous contact, making you back out. Let’s not even mention the ridiculously complex key sequence required to record a voice memo.

I have to contrast this with my Palm Tungsten T3, in my book still the best PDA ever (specially compared to the underwhelming, plasticky T5 or the boat-anchor and RAM-starved Lifedrive). Recording a voice memo merely requires pressing and holding a dedicated button, something that can be done one-handed by touch alone. Palm’s address book quick look up scrolling algorithm is a model of efficiency yet to be matched on any phone I have ever used. PalmOS might be getting long in the tooth, specially as regards multitasking, and its future is cloudy, but it still has a serious edge in usability. This is not by accident — Palm paid as much attention to the user interface as Apple did in its day, as this anecdote by New York Times technology columnist David Pogue illustrates:

I once visited Palm Computing in its heyday. One guy I met there introduced himself as tap counter. It was his job to make sure that no task on the PalmPilot required more than three taps of the stylus on the screen. More than three steps, and the feature had to be redesigned. Electronics should save time, not waste it.

In retrospect, I should not have been surprised by the 6682’s poor ergonomics, they were readily apparent from day one. The device is neither a good phone, nor an even halfway acceptable PDA. I decided to give it a chance, thinking it could just be a question of settling into an unfamiliar user interface. I did not have as long an adaptation period when moving from my old T68i to the 6230, and after two weeks my dim initial opinion of the Series 60 had if anything deteriorated further. Russell Beattie can dish it, but he can’t take it. In hindsight, Beattie’s defensiveness about smart people preferring dumb phones over jack-of-all-trades devices was not a good sign.

Pundits have been predicting the demise of the PDA at the hands of the smartphone for many years. Phones certainly outsell PDAs by a handy margin, but a small minority of them are smartphones, and I suspect most people get them for the improved cameras and disregard the unusable advanced functionality. I tend to agree with this old but still valid assessment — the best option is to have a decent PDA in hand, connected to the cell phone in your pocket via Bluetooth.

I suspect the smartphones’ ergonomic shortcomings are structural, not just due to lack of usability skills on the manufacturers’ part. Nokia knows how to design good user interfaces, like Navi or Series 40, but the situation with Series 60 is not going to be rectified anytime soon. The reason for this is that most people buy their cell phones with a subsidy that is paid back over the duration of a 1 or 2 year minimum duration contract. This control over distribution allows the mobile operators ultimate say over the feature set. This is most visible in branding elements like Cingular’s “Media store” icon that flogs overpriced garbage like downloadable ring tones.

To add injury to insult, deleting those “features” is disabled, so they keep hogging scarce memory and screen real estate. Carriers also disable features that would allow people to use their phones without being nickel-and-dimed for expensive intelligent network services like MMS, like some Bluetooth functionality or the ability to send photos over email rather than MMS. It is highly likely carriers will fight tooth-and-nail against the logical inclusion of WiFi and VoIP in future handsets. This conflict of interest between carriers and users won’t be resolved until regulators compel them to discontinue what is in effect a forced bundling practice.

Mobile carriers, like their Telco forebears, seem to believe if they piss on something, it improves the flavor… This is also the reason why I think mobile operator cluelessness about mobile data services is terminal — they keep pushing their failed walled-garden model of WAP services using phones, and gouge for the privilege of using a PDA or laptop to access the real Internet via Bluetooth, while at the same time not deigning to provide any support. WiFi may not be an ideal technology, specially in terms of ubiquity, but as long as carriers make us unwashed users jump through hoops to be allowed access to their data networks, low-hassle WiFi access using a PDA will be the superior, if intermittent alternative to a data-enabled phone. As for the aborted phone upgrade, I guess I will just wait for the Nokia 6270 to hit these blighted shores.

Eighty percent of success is showing up. — Woody Allen

My company, Kefta, helps its clients, usually Fortune 500 companies with e-commerce operations, improve their online conversion rates. We typically increase sales by 10–20%. This is not rocket science, more akin to Retail 101, simple things like modifying pages to stop showing offers for products we know the user has already purchased, or making offers more relevant when we know the prospect is interested in a specific product (e.g. because they come from Google after searching for that keyword).

Sometimes I wonder if what we are doing is not too sophisticated by far, when I see particularly boneheaded practices at places that really should know better. Dell is often touted as a model of logistical and operational excellence, and for being a web-centric company. My experience is that many products they carry are not listed on the web site and can only be ordered by phone. You also have to phone to get a discount.

Despite being a telecoms engineer by training, I loathe phones. Phones are great for keeping an emotional connection with friends and family, but are a staggeringly inefficient form of communication for business purposes. They do not leave an audit trail, and even when they do (my voice mail system automatically forwards them to me by email as a MIME-encoded WAV attachment), they hog disk space and are not searchable. You can scan an email in a few seconds, but are forced to listen to voice mail at whatever pace it was dictated. Well, at least with WAV attachments, I can skip back to write down a phone number without having to replay the whole message.

Coming back to Dell, I recently needed to buy a Gigabit Ethernet switch from them. I sent an email to my rep, which he promptly ignored. I tried calling, at least 4 or 5 times, but my only option was voice-mail jail. In the end, I passed the buck to a junior colleague, who tried to leave voice mail and discovered he couldn’t because it was full. With persistence, he managed to get Dell to condescend to taking our order. No customer should have to go through so many hoops just so the vendor can take their money.

I am ragging on Dell, but most IT vendors do as poorly. I can understand expensive support calls receiving lower priority and resources than sales calls — after all, the company already has your money. Not having their act together for the simple matter of order-taking simply boggles the mind. Workflow systems, automatic call distributors and other technologies designed to prevent this have been available for many years. It looks like nobody has bothered to go through the user experience, even though these bugs (and many other glaring deficiencies like session timeouts) could be caught by the most cursory of inspections.

Dell sends an automated satisfaction survey after a sale. Unlike the order-taking process, the survey follows up if you do not respond… That said, it is the usual worthless multiple-choice question format asking me to answer irrelevant questions on a scale of 1 to 10. I don’t recall if the form had a box for free-form comments, but even if it did, the survey design is not-so-subtly signaling that no human is ever going to read what you type there, and thus it is not worth the effort to fill it. The numeric answers are probably going to be collated by an automated report nobody pays any attention to anymore, because garbage-in, garbage-out.

If you are serious about customer feedback, make it open and free-form, and make sure each and every feedback is read by a human (they come quite cheap in the Midwest and the developing world). They should be acknowledged personally (not with an automated reply) and followed through until the issue is either resolved or a decision is taken not to implement the changes suggested (because they are too expensive, impractical or whatever other reason). In both cases, inform the user who bothered to give feedback — most large companies pay a fortune in market research while at the same time ignoring the free (and usually very valuable) insights submitted by their customers. Granted, you cannot always resolve every complaint by unreasonable customers, but feedback on process issues should always be taken into consideration.

Sometimes dropped orders are due to active incompetence rather than careless neglect. While implementing a campaign for one of our clients, we realized there was a bug in one of their ordering forms that would cause them to drop an order. Our software sits on top of the client’s website and monitors it precisely for exception cases like these, and we told them we could, at no extra charge for them, send the dropped order details to an email address of their choosing so the order could be re-entered manually. They declined our offer for various reasons related to internal politics and trade union issues, essentially they were refusing to bend down and pick up money lying on the floor (our estimate was they were losing tens to hundreds of thousands of dollars of customer lifetime value every month due to inaction).

You don’t have to endure a multi-million dollar ERP or CRM implementation to improve follow-through. Where there is a will, there is a way, and a little creative thinking will usually find a work-around that can get the job done until a more robust solution can be deployed. One of our clients, a major bank, was in the early stages of developing their e-commerce, and simultaneously in the throes of a Siebel implementation. Their online forms would simply send an email to a branch office for manual processing. We were implementing a satisfaction survey for them, and offered to send an email automatically to a supervisor if the customer’s order had not been processed, at least until Siebel came on-line. Poor man’s workflow, but email workflows are often quite effective, specially for remedial situations like these.

As I mentioned, sometimes I think I am in the wrong business, and should instead start a consultancy to teach some clue to large companies that have grown complacent. But then again, that is assuming somebody cares, beyond paying lip service to Customer Relationship Management. There is no point in setting up complex systems to build a lifelong relationship with repeat customers if you can’t even take their orders in the first place.