Fazal Majid's low-intensity blog

I attended the OpenSolaris Storage Summit in San Francisco. Unsurprisingly, SSDs dominated the proceedings. Sun is touting its hybrid storage pool approach, where SSDs are used to accelerate the ZFS Intent Log (ZIL), a journal, and the L2ARC read cache, to get most of the benefits of SSDs without having to store an entire dataset in expensive SSDs. The presentations on the page are worth having a look into.

Andrew Leventhal commented on how using disk-oriented APIs to access SSDs makes as little sense as using cassette tape adapters to connect an iPod to a car stereo. Sure, it works, sort of, as a short-term compatibility mode, but you are not using the potential of the device. Fusion IO makes SSDs that are connected to the PCIe bus directly and get 5x better performance than the best SATA drive, the Intel X25-E, but it uses proprietary drivers. In one of the video presentations distributed in the conference, Andy Bechtolsheim mentioned he thought a memory channel model is a better fit for the hardware, as the block-device model and all the legacy assumptions behind it is holding back databases from using the full power of SSDs.

A modest suggestion – there is such an API available already: memory-mapped files. In an ironic full circle, an abstraction meant to simulate random access memory on rotating media is actually the best fit to the actual device characteristics. There are also obvious benefits such as DMA.

A few months ago, my former EuroNet colleague Niels Bakker was visiting SF. He works for AMS-IX, the Amsterdam Internet Exchange, which is the world’s largest by volume of traffic, and mentioned they work with a mere 25 employees.

Today, I was attending a presentation by Don MacAskill, the CEO of photo-sharing service SmugMug. He has 2 sysadmins managing well over 300 servers.

At the same time, you hear about the astronomical costs of corporate IT departments: data centers that routinely cost hundreds of millions of dollars, plethora of staff delivering pitiful results and systems that have user interface even a novice coder could beat in a single day of coding..

Why is this so?

You have the usual suspects:

• Dysfunctional top-down corporate cultures, specially when decisions are made on political grounds, i.e. which vendor plays golf with the CIO (or CEO). Often the grunts on the ground know what needs to be done, but are defeated and dispirited by years of failing to budge the bureaucracy.
• Use of poorly manageable software like Windows
• A culture of fire-fighting that eschews automation.
• Risk aversion leading to excessive redundancy. I still cannot understand how Red Hat gets away with its outrageous pricing on RHEL 7.
• In most large corporations the concentration of financial responsibility in a separate department means most employees, including sysadmins, do not feel empowered or responsible for looking out for the company’s money. The bean counters, on the other hand, lack the knowledge required to find the cost savings.

One would think the new economic reality would force a reckoning. It would stand to reason that most companies would institute policies of procuring open-source software first, and only purchase commercial software on an case-by-case exception basis, with tough questions asked. This is still novel enough to make the news.

One of the great things about living in the United States is the ubiquity and affordability of high-quality orange juice. A few weeks ago, while going through the aisles at Target to buy a carton of Tropicana, I couldn’t find any. It looked like Target had replaced them with cheap generic knock-offs. On closer inspection, it turned out Pepsi redesigned the packaging. To say the new design is ugly is an understatement. Many comments on the redesign compare it to generics in its amateurism.

Pepsi’s marketers are not legends in the field like those of Procter & Gamble, but still, I find it hard to believe no one there perceived how bad the new cartons look, and how off-putting they are. This led me to think if this wasn’t intentional.

In this economy, sales of Spam are exploding even though the ersatz canned meat is actually more expensive than more nutritive fresh meat and a much worse value. One explanation is that spam is what economists call an inferior good, a good for which demand increases as incomes decrease because people can’t afford the better stuff. One extreme type of an inferior good is a Giffen good, a product for which demand increases even as its price increases. Economists still debate whether Giffen goods even exist. One often-quoted (and just as often disputed) example are potatoes during the Great Irish famine of 1847. As the price of potatoes rose, the poor were locked in a vicious circle of not being to afford anything else and being more dependent on potatoes, which only accelerated the price explosion.

Perhaps the Tropicana marketers have figured that in a severely down economy, people are settling for inferior goods, and making Tropicana look downmarket may increase its sales…

Update (2009-02-23):

So much for my theory. Although you could make the case this is more like the New Coke fiasco (that many conspiracy theorists still think was deliberate).

An interesting factoid. The Herald Tribune has a discussion of the geopolitics and economics involved.

I just dropped out of a Cisco customer satisfaction survey, after realizing the six pages I already filled out represent only 20% of the entire survey. I don’t know why they think they can make such unreasonable demands on people’s time.

Like most clueless surveys, they feature multiple inane questions where you are supposed to answer vague and irrelevant questions several times on a scale of 1 to 5. There are open text fields for feedback, but the likelihood of actual humans reading them is so low, it just isn’t worth my time to fill them out.

The ideal survey should have only three fields:

1. one to say whether you are satisfied or not (and this is a binary field, not on a scale of 1 to 5 or whatever).
2. a free-form text field for feedback. The company asking for the survey should commit to having an actual human read each feedback, and be empowered to take action. If it isn’t worth having a human read it, it’s not worth my filling it. Outsourcing a survey to SurveyMonkey or one of the myriad automated surveying firms just demonstrates that you are not really serious about feedback.
3. an optional field for an email address so you can ask follow-up questions.