IT

Blogging is often seen as a narcissistic pursuit. It can be, but the best bloggers (that is not necessarily synonymous with the most popular) put their audience first. To do that, you need to know it first. Most blogs have three very distinct types of readers:

1. Regular visitors who use web browsers and bookmarks to visit. If the page doesn’t change often enough, they will get discouraged by the lack of changes and eventually stop coming. You need to post often to keep this population engaged.
2. People who come from a search engine looking for very specific information. If they do not find what they are looking for, they will move on to the next site in their list, then possibly linger for other articles and may eventually graduate to repeat visitor status. Closely related are people who follow links from other sites, pointing to yours.
3. Those who let feed readers do the polling for them, and thus do not necessarily care how often a feed is updated. Feed readers allow for much more scalable browsing – I currently subscribe to 188 feeds (not all of them are listed in my blogroll), and I certainly couldn’t afford to visit 188 sites each day. Feed readers are still a minority, but specially for commercial publications, a very attractive one of tech-savvy early adopters. The flip side of this is a more demanding audience. Many people go overboard with the number of feeds and burn out, then mass unsubscribe. If you are a little careful, you can avoid this pendulum effect by pruning feeds that no longer offer a sufficient signal to noise ratio.

The following sections, in no particular order, are rough guidelines on how best to cater to the needs of the other two types of users.

Maintain a high signal to noise ratio

Posting consistently good information on a daily or even weekly basis is no trivial amount of work. I certainly cannot manage more than a couple of postings per month, and I’d rather not clutter my website with “filler material” if I can help it. For this reason, I have essentially given up on the first constituency, and can only hope that they can graduate to feed readers as the technology becomes more mainstream.

Needless to say, test posts are amateurish and you should not waste your readers’ time with them. Do the right thing and use a separate staging environment for your blog. If your blogging provider doesn’t provide one, switch to a supplier that has a clue.

Posting to say that one is not going to post for a few days due to travel, a vacation or any other reason is the height of idiocy and the sure sign of a narcissist. A one-way trip to the unsubscribe button as far as I am concerned.

Distinguish between browsers and feed readers

In November of last year, I had an interesting conversation with Om Malik. My feedback to him was that he was posting too often and needed to pay more attention to the quality rather than the quantity of his postings.

The issue is not quite as simple as that. To some extent the needs of these browser users and those who subscribe to feeds are contradictory, but a good compromise is to omit the inevitable filler or site status update articles from the Atom or RSS feeds. Few blog tools offer this feature, however.

Only allow permalinks to be indexed

Search engines will index your home page, which is normally just a summary of the last N articles you wrote. Indeed, it will often have the highest page rank (or whatever metric is used). An older article may be pushed out but still listed in the (now out of date) search engine index. The latency is often considerable, and the end result is that people searching for something saw a tantalizing preview in the search engine results listing, but cannot find it once they land on the home page, or in the best of cases they will have to wade through dozens of irrelevant articles to get to it. Ideally, you want them to reach the relevant permalink page directly without stopping by the home page.

There is a simple way to eliminate this frustration for search engine users: make the home page (and other summary pages like category-level summaries or archive index pages) non-indexable. This can be done by adding the following meta tags to the top of the summary pages, but not to permalink pages. The search engine spiders will crawl through the summary pages to the permalinks, but only store the permalink pages in their index. Thus, all searches will lead to relevant and specific content free from extraneous material (which is still available, just one click away).

<!– make sure Google only indexes and caches permalinks –> <meta name=”robots” content=”noarchive,noindex,follow,nocache” />

Here again, not all weblog software supports having different templates for permalink pages than for summary pages.

There is an unfortunate side-effect of this — as your home page is no longer indexed, you may experience a drop in search engine listings. My weblog is no longer the first hit for Google search for “Fazal Majid”. In my opinion, the improved relevance for search engine users far outweighs the bruising to my ego, which needs regular deflating anyways.

Support feed autodiscovery

Supporting autodiscovery of RSS feeds or Atom feeds makes it much easier for novice users to detect the availability of feeds (Firefox and Safari already support it, and IE will soon). Adding them to a page is a no-brainer.

Categorize your articles

In all likelihood, your postings cover a variety of topics. Categorizing them means users can subscribe only to those of interest to them, and thus increases your feed’s signal to noise ratio.

Keep a stable feed URL under your control

If your feed location changes, set up a redirection. If this is not possible, at least post an article in the old feed to let subscribers know where to get the new feed.

Depending on a third-party feed provider like Feedburner is risky — if they ever go out of business, your subscribers are stranded. Even worse, if a link farm operator buys back the domain, they can easily start spamming your subscribers, and make it look as if the spam is coming from you. Your feeds are just as mission-critical as your email and hosting, don’t enter in an outsourcing arrangement casually, specially not one without a clear exit strategy.

Maintain old posts

Most photographers, writers and musicians depend on residuals (recurring revenue from older work) for their income and to support them in retirement. Unless your site is pure fluff (and you would not be reading this if that were the case), your old articles are still valuable. Indeed, there is often a Zipf law at work and you may find some specific archived articles account for the bulk of your traffic (in my case, my article on lossy Nikon NEF compression is a perennial favorite).

It is worth dusting these old articles off every now and then:

• You should fix or replace the inevitable broken links (there are many programs available to locate broken links on a site, I have my own but linkchecker is a pretty good free one.
• The content in the article may have gone stale and need refreshing, Don’t rewrite history, however, and change it in a way that alters the original meaning — better to append an update to the article. If there was a factual error, don’t leave it in the main text of the article, but leave a mention of the correction at the end
• there is no statute of limitations on typos or spelling mistakes. Sloppy writing is a sign of disrespect towards your readers; Rewriting text to clarify the meaning is also worthwhile on heavily visited “backlist” pages. The spirit of the English language lies in straightforwardness, one thing all the good style guides agree on.
• For those of you who have comments enabled on their site, pay special attention to your archives, comment spammers will often target those pages as it is often easier for them to avoid detection there. You may want to disable comments on older articles.
• Provide redirection for old URLs so old links do not break. Simple courtesy, really.

Make your feeds friendly for aggregators

Having written my own feed reader, I have all too much experience with broken or dysfunctional feeds. There is only so much feed reader programmers can do to work around brain-dead feeds.

• Stay shy of the bleeding edge in feed syndication formats. Atom offers a number of fancy features, but you have to assume many feed readers may break if you use too many of them. It is best if your feed files use fully qualified absolute URLs, even if Atom supports relative URLs, for instance. Unicode is also a double-edged sword, prefer HTML entity-encoding them over relying on a feed reader to deal with content-encoding correctly.
• Understand GUIDs. Too many feeds with brain-dead blogging software will issue a new GUID when an article is edited or corrected, or when its title is changed. Weblogs Inc. sites are egregious offenders, as is Reuters. The end-result is that an article will appear several times in the user’s aggregator, which is incredibly annoying. Temboz has a feature to automatically suppress duplicate titles, but that won’t cope with rewritten titles.
• Full contents vs. abstracts is a point of contention. Very long posts are disruptive on web-based feed readers, but on the other hand most people dislike the underhanded teaser tactics of commercial sites that try and draw you to their website to drive ad revenue, and providing only abstracts may turn them off your feed altogether. Remember, the unsubscribe button is a mere click away…

Blogging ethics

The golden rule of blogging is that it’s all about the readers. Everything follows from this simple principle. You should strive to be relevant and considerate of their time. Take the time to spell-check your text. It is very difficult to edit one’s own text, but any article can benefit from a little time spent maturing, and from tighter and more lucid prose.

Don’t be narcissistic, unless friends and family are the primary audience. Most people couldn’t care less about your pets, your garden or for the most part your personal life (announcing major life events like a wedding or the birth of your children is perfectly normal, however).

Respect for your readers requires absolute intellectual honesty. Laziness or expediency are no excuse for poor fact-checking or revisionist edits. Enough said…

Update (2008-05-21):

Unfortunately setting the meta tags above seems to throw Google off so that it stops indexing pages altogether (Yahoo and MSN search have no problems). So much for the myth of Google’s technical omnipotence… As a result, I have removed them and would advise you to do as well.

Update (2015-11-20):

If you use JavaScript and cookie-based web analytics like Piwik or Mint, make sure those script tags are disabled if the browser sends the Do-Not-Track header. As for third-party services like Google Analytics, just don’t. Using those services means you are selling giving away your readers’ privacy to some of the most rapacious infringers in the world.

I ran the Cyrus IMAP server for almost a year on my home server, but I recently switched to Dovecot. I originally used Cyrus because of its demonstrated scalability and in part because it is a product of my father’s alma mater, but it is quite hard to set up, and quite brittle to changes in its dependencies.

The last straw was when I tried unsuccessfully to set up another instance of Cyrus on a server, with the exact same configuration files and permissions, but different versions of the Berkeley DB and Cyrus SASL libraries, and it simply wouldn’t cooperate. In disgust, I downloaded Dovecot, compiled it and installed it in less time it took me just to figure out that Cyrus wouldn’t allow me to authenticate because the ever-crufty SASL library failed in a new inscrutable way. I had also never managed to get Cyrus’ SSL to work reliably, it is nearly effortless with Dovecot.

Dovecot is much easier to build and manage, does not have dependencies on unreliable cruft like the Cyrus SASL library, and is much easier to integrate with procmail, SpamAssassin and other goodies thanks to its use of the Maildir format rather than a proprietary database cum filesystem structure like Cyrus. From what I have seen of the internals of the Cyrus 2.2 “skiplist” database back-end (which replaced the BerkeleyDB back-end used in previous releases), I have a hard time believing it is significantly more efficient than Dovecot, if at all.

One problem was migrating my email – I have pretty much all my email since 1995 in my IMAP mailbox, migrated from various Emacs Babyl mailbox files or Exchange PSTs over time. The Dovecot Wiki points to this migration script, but for recent versions of Cyrus like the 2.2.12 I ran, it has two major shortcomings:

1. It will not preserve the flag that indicates whether an email was read or not.
2. It does not preserve the delivery timestamp for the emails so they all look as if they were delivered at the time you did the conversion.

I wrote my own migration script in Python, cyrus2maildir.py, to avoid these shortcomings. It does not preserve the “replied to” flag, but the Read flag is carried over, as is the delivery timestamp (in some edge cases like emails you sent, it has to guess, though). This is not a very polished program because I spent far more time on it than I had anticipated, and basically stopped once I got it working, but it should be usable, or at least a starting point for anyone with some Python skills. Of course, it can also be used by users of other Maildir++ based servers like Courier.

The script should guess most parameters, and will dump the emails to a directory named Maildir.cyrus/ in your home directory. By default, your user will not have read access to the Cyrus mail spool, you may have to alter permissions (I do not recommend running the script as root). For the syntax, just type: cyrus2maildir.py -h

On an unrelated note, Solaris 10 users may find the SMF manifest and method useful to automate service management and fault recovery. To install them, copy the manifest to /var/svc/manifest/site and the method to /lib/svc/method and install them into SMF with the command: svccfg import /var/svc/manifest/site/imap-dovecot.xml

After an abortive experiment with a Nokia Symbian Series 60 smartphone, I bought a Palm T|X on Wednesday, the very day it was announced. I find PDAs superior to fiddly, fragile and cumbersome laptops, and have owned no fewer than 9 Palm compatible handhelds (*) in the last 5 years, which means I upgrade handhelds at least three times more often than my main (desktop) computers. My previous PDA is a Palm Tungsten T3 (I actually bought it after the T5 was announced, so underwhelming the latter is). I even obtained a spare T3 in case the first one broke (since given to my father). I am not entirely sure yet as to whether the T|X is really an upgrade. Here are some first impressions after a few days of use:

Pros:

• Built-in WiFi. No more fiddling with the easily lost SDIO WiFi card.
• A better browser. Blazer feels much snappier than Web Pro, specially with the new Fast mode (disables CSS and image loading).
• More memory, non-volatile if the battery fails.
• Lighter.
• Can actually dial and send SMS on a Nokia 6230 via Bluetooth

Cons:

• Plastic construction feels much less robust (but at least it is not pretending to be metal like the E, E2 or T5, that’s just tacky).
• No voice recorder, charge LED or vibrating alarm. I seldom use the voice recorder, as I prefer taking notes on 3×5 jotter cards, but the voice recorder works when you have to capture that elusive idea while driving.
• 20–25% slower processor. Graffiti2 is noticeably slower to respond, for instance.
• The flip cover with the hinge on the side is less convenient than the one on top, which flips up like a reporter’s notebook, in one fluid motion.
• The SD slot has a plastic filler card, not a spring-loaded cover.
• Bigger. Many people complain about the true Tungstens’ slider, but it is very natural to use, and much more convenient than the power switch.
• The stylus has a nice heft to it, but is not as substantial as the T3’s, and less easy to extract from its slot.
• Yet another connector design incompatible with previous accessories. The cradle is an expensive option.
• The home icon on the status bar has disappeared. This is very annoying in daily use
• The application buttons and the 5-way navigator are less responsive and smaller. The T3 has generally superior haptics (feels much better in the hand).

The only potential deal-breaker is the slower Graffiti performance (there is a visible lag). I will probably keep the T|X due to the convenience of integrated WiFi, but the T3 is a superior device in almost all other respects, in the same class as the Palm V as one of the PDA world’s truly outstanding designs. If Palm were to come out with a new model marrying the WiFi and newer software stack of the T|X with the solid construction and faster processor of the T3, I would definitely upgrade again.

(*): Handspring Visor, Sony Clié T615C, Kyocera QCP-6035, Palm Tungsten T, Sony Clié UX50, Palm Zire 71, Palm Tungsten T3 (x2), and now the Palm T|X.

Update (2010-05-16):

The T|X was the last Palm device I bought. I switched to an iPhone in 2007 and never looked back.

There has been much speculation recently about a possible rift in Internet governance. Essentially, many countries resent the US government’s control over the Internet’s policy oversight. They advocate the transfer of those responsibilities to the International Telecommunications Union (ITU), a more multilateral venue. The big news is that the European Union, which previously sat on the fence, came out strongly in favor of this proposal. Unsurprisingly, the US government is hostile to it. More surprisingly, I agree with their unilateralist impulse, obviously for very different reasons. I was planning on writing up a technical explanation as most of the IT press has it completely wrong, as usual, but Eric Rescorla has beaten me to the punch with an excellent summary.

Many commentators have made much hay of the fact the ITU is under the umbrella of the United Nations. The Bush administration is clearly reticent, to say the least, towards the UN, but that is a fairly widespread sentiment among the American policy establishment, by no means limited to Republicans. For some reason, many Americans harbor the absurd fear that somehow the UN is plotting against US sovereignty. Of course, the reality is the UN cannot afford its parking tickets, let alone black helicopters. American hostility towards the UN is curious, as it was the brainchild of a US president, Franklin D. Roosevelt, its charter was signed in San Francisco (at Herbst Theatre, less than a mile from where I live), and it is headquartered in New York.

The UN is ineffective and corrupt, but that is because the powers on the Security Council want it that way. The UN does not have its own army and depends on its member nations, specially those on the Security Council to perform its missions. It is hardly fair to lay the blame for failure in Somalia on the UN’s doorstep. As for corruption, mostly in the form of patronage, it was the way the US and the USSR greased the wheels of diplomacy during the Cold War, buying the votes of tin-pot nations by granting cushy UN jobs to the nephews of their kleptocrats.

A more damning condemnation of the UN is the fact the body does not embody any kind of global democratic representation. The principle is one country, one vote. Just as residents of Wyoming have 60 times more power per capita in the US Senate than Californians, India’s billion inhabitants have as many votes in the General Assembly as those of the tiny Grand Duchy of Liechtenstein. The real action is in the Security Council anyways, but they are not fully represented there either. Had Americans not had a soft spot for Chiang Kai-Shek, China, with its own billion souls, would not have a seat at that table either. That said, the Internet population is spread unevenly across the globe, and the Security Council is probably more representative of it.

In any case, the ITU was established in 1865, long before the UN, and its institutional memory is much different. It is also based in Geneva, like most international organizations, geographically and culturally a world away from New York. In other words, even though it is formally an arm of the UN, the ITU is in practice completely autonomous. The members of the Security Council do not enjoy veto rights in the ITU, and the appointment of its secretary general, while a relatively technocratic and unpoliticized affair, is not subject to US approval, or at least acquiescence, the way the UN secretary-general’s is, or that of more sensitive organizations like the IAEA.

My primary objections to the ITU are not about its political structure, governance or democratic legitimacy, but about its competence, or more precisely the lack of it. The ITU is basically the forum where government PTT monopolies meet incumbent telcos to devise big standards and blow big amounts of hot air. Well into the nineties, they were pushing for a bloated network architecture called OSI, as an alternative to the Internet’s elegant TCP/IP protocol suite. I was not surprised — I used to work at France Télécom’s R&D labs, and had plenty of opportunity to gauge the “caliber” of the incompetent parasites who would go on ITU junkets. Truth be said, those people’s chief competency is bureaucratic wrangling, and like rats leaving a ship, they have since decamped to the greener pastures of the IETF, whose immune system could not prevent a dramatic drop in the quality of its output. The ITU’s institutional bias is towards complex solutions that enshrine the role of legacy telcos, managed scarcity and self-proclaimed intelligent networks that are architected to prevent disruptive change by users on the edge.

When people hyperventilate about Internet governance, they tend to focus on the Domain Name System, even though the real scandal is IPv4 address allocation, like the fact Stanford and MIT each have more IP addresses allocated to them than all of China. Many other hot-button items like the fight against child pornography or pedophiles more properly belongs in criminal-justice organizations like Interpol. But let us humor the pundits and focus on the DNS.

First of all, the country-specific top-level domains like .fr, .cn or the new kid on the block, .eu, are for all practical purposes already under decentralized control. Any government that is afraid the US might tamper with its own country domain (for some reason Brazil is often mentioned in this context) can easily take measures to prevent disruption of domestic traffic by requiring its ISPs to point their DNS servers to authoritative servers under its control for that zone. Thus, the area of contention is really the international generic top-level domains (gTLDs), chief of all .com, the only one that really matters.

What is the threat model for a country that is distrustful of US intentions? The possibility that the US government might delete or redirect a domain it does not like, say, al-qaeda.org? Actually, this happens all the time, not due to the malevolence of the US government, but to the active incompetence of Network Solutions (NSI). You may recall NSI, now a division of Verisign, is the entrenched monopoly that manages the .com top-level domain, and which has so far successfully browbeaten ICANN into prolonging its monopoly, one of its most outrageous claims being that it has intellectual property rights to the .com database. Their security measures, on the other hand, owe more to Keystone Kops, and they routinely allow domain names like sex.com to be hijacked. Breaking the NSI monopoly would be a worthwhile policy objective, but it does not require a change in governance, just the political will to confront Verisign (which, granted, may be more easily found outside the US).

This leads me to believe the root cause for all the hue and cry, apart from the ITU angling for relevance, may well be the question of how the proceeds from domain registration fees are apportioned. Many of the policy decisions concerning the domain name system pertain to the creation of new TLDs like .museum or, more controversially, .xxx. The fact is, nobody wakes up in the middle of the night thinking: “I wish there were a top-level domain .aero so I could reserve a name under it instead of my lame .com domain!”. All these alternative TLDs are at best poor substitutes for .com. Registrars, on the other hand, who provide most of the funding for ICANN, have a vested interest in the proliferation of TLDs, as that gives them more opportunities to collect registration fees.

I bought a Nokia 6682 phone a couple of weeks ago, as an upgrade for my Nokia 6230. Actually, I have my parents signed up on my service plan, and I was planning on sending them the 6230 to replace an older phone they lost, and taking advantage of this as an excuse to upgrade… The 6682 is a Symbian “smartphone” sporting Nokia’s Series 60 UI, and I was influenced by rave reviews like Russell Beattie’s. In recent years, Nokia has been churning out phones with crackpot designs and dubious usability for coolness’ sake. There must have been a customer backlash, as their recent phones like the 6682 have a much more reasonable, reassuringly boring but functional design. Another reason is that Apple’s iSync only works with Nokia’s Series 60 phones, and it will sync photos from the OS X address book.

I returned the phone for a refund last Friday, because the ergonomics are simply atrocious, and from a usability point of view it was actually an unacceptable downgrade from the Series 40 (non-Symbian) Nokia 6230. The low-res 176×208 screen has significantly lower information density than the 320×480 or 640×480 screens now standard on most PDAs, and makes web browsing almost useless. The only thing it has going for it is a semi-decent camera.

Even basic functionality like the address book is poorly implemented. When you scroll down your contacts list, you can select one to decide whether you want to reach them on their home or mobile number. The problem is, the next time you want to make a call and access the address book, you do not start afresh, but still in the list of contacts for the previous contact, making you back out. Let’s not even mention the ridiculously complex key sequence required to record a voice memo.

I have to contrast this with my Palm Tungsten T3, in my book still the best PDA ever (specially compared to the underwhelming, plasticky T5 or the boat-anchor and RAM-starved Lifedrive). Recording a voice memo merely requires pressing and holding a dedicated button, something that can be done one-handed by touch alone. Palm’s address book quick look up scrolling algorithm is a model of efficiency yet to be matched on any phone I have ever used. PalmOS might be getting long in the tooth, specially as regards multitasking, and its future is cloudy, but it still has a serious edge in usability. This is not by accident — Palm paid as much attention to the user interface as Apple did in its day, as this anecdote by New York Times technology columnist David Pogue illustrates:

I once visited Palm Computing in its heyday. One guy I met there introduced himself as tap counter. It was his job to make sure that no task on the PalmPilot required more than three taps of the stylus on the screen. More than three steps, and the feature had to be redesigned. Electronics should save time, not waste it.

In retrospect, I should not have been surprised by the 6682’s poor ergonomics, they were readily apparent from day one. The device is neither a good phone, nor an even halfway acceptable PDA. I decided to give it a chance, thinking it could just be a question of settling into an unfamiliar user interface. I did not have as long an adaptation period when moving from my old T68i to the 6230, and after two weeks my dim initial opinion of the Series 60 had if anything deteriorated further. Russell Beattie can dish it, but he can’t take it. In hindsight, Beattie’s defensiveness about smart people preferring dumb phones over jack-of-all-trades devices was not a good sign.

Pundits have been predicting the demise of the PDA at the hands of the smartphone for many years. Phones certainly outsell PDAs by a handy margin, but a small minority of them are smartphones, and I suspect most people get them for the improved cameras and disregard the unusable advanced functionality. I tend to agree with this old but still valid assessment — the best option is to have a decent PDA in hand, connected to the cell phone in your pocket via Bluetooth.

I suspect the smartphones’ ergonomic shortcomings are structural, not just due to lack of usability skills on the manufacturers’ part. Nokia knows how to design good user interfaces, like Navi or Series 40, but the situation with Series 60 is not going to be rectified anytime soon. The reason for this is that most people buy their cell phones with a subsidy that is paid back over the duration of a 1 or 2 year minimum duration contract. This control over distribution allows the mobile operators ultimate say over the feature set. This is most visible in branding elements like Cingular’s “Media store” icon that flogs overpriced garbage like downloadable ring tones.

To add injury to insult, deleting those “features” is disabled, so they keep hogging scarce memory and screen real estate. Carriers also disable features that would allow people to use their phones without being nickel-and-dimed for expensive intelligent network services like MMS, like some Bluetooth functionality or the ability to send photos over email rather than MMS. It is highly likely carriers will fight tooth-and-nail against the logical inclusion of WiFi and VoIP in future handsets. This conflict of interest between carriers and users won’t be resolved until regulators compel them to discontinue what is in effect a forced bundling practice.

Mobile carriers, like their Telco forebears, seem to believe if they piss on something, it improves the flavor… This is also the reason why I think mobile operator cluelessness about mobile data services is terminal — they keep pushing their failed walled-garden model of WAP services using phones, and gouge for the privilege of using a PDA or laptop to access the real Internet via Bluetooth, while at the same time not deigning to provide any support. WiFi may not be an ideal technology, specially in terms of ubiquity, but as long as carriers make us unwashed users jump through hoops to be allowed access to their data networks, low-hassle WiFi access using a PDA will be the superior, if intermittent alternative to a data-enabled phone. As for the aborted phone upgrade, I guess I will just wait for the Nokia 6270 to hit these blighted shores.