Fazal Majid's low-intensity blog

Sporadic pontification

Cheeky

This is a popup the iOS6 App Store shows me when I try searching for a Google Maps app.

20120920-062922.jpg

If WordPress updates hang on a 64-bit OS

The WordPress instance running this site was no longer able to automatically update plugins (and presumably not the core either) after I upgraded from a 32-bit to a sparkling fresh 64-bit PHP install at Joyent. It would start the update, and show a spinning logo and then just hang.

After much debugging, I found out the problem is that the class-pclzip.php that is responsible for unzipping was failing silently with the message:

Downloading update from http://downloads.wordpress.org/plugin/yet-another-related-posts-plugin.3.5.2.zip

Unpacking the update…

Abort class-pclzip.php : Missing zlib extensions

This isn’t terribly helpful, but digging in, it turns out that class depends on the PHP zlib module, and on 64-bit operating systems (more precisely, operating systems with 64-bit large file support enabled), zlib.h #defines gzopen to be gzopen64. PHP does not protect itself adequately and thus the PHP function gzopen gets renamed gzopen64 as well, this throwing class-pclzip.php for a loop, along with a number of other systems like PEAR.

Fixing this requires recompiling PHP. Ubuntu Karmic includes a work-around, but I run Solaris and build from source, so I contributed a patch filed under bug #53829.

Automattic should probably patch class-pclzip.php to deal with gzopen/gzopen64 as there are a great many broken PHP installs out there (the PHP bug has been open for over a year and a half without what I would consider an acceptable solution), and it is surprisingly difficult to find a solution online. I guess a great many WordPress installs are still 32-bit, which is kind of sad.

Scanning your iTunes library for DRM-infested books

Tor, the leading publisher for Science Fiction and Fantasy books, announced they would be doing away with DRM in their eBooks. The product pages for their books on iBooks now mention “At the publisher’s request, this title is being sold without Digital Rights Management software (DRM) applied”. I figured it would be a good idea to uncripple the many Tor eBooks I have in my collection.

I wrote a quick little Python script to scan my growing iBooks library for books that could be updated. The procedure is to delete the book from both iTunes and iPads, then download it anew (restarting iTunes is also needed after deleting). Apple keeps track of your purchases and will not charge you again.

#!/usr/bin/env python
import sys, os.path, glob, zipfile, platform, xml.etree.ElementTree

# publishers who have forsaken DRM
good = ['Tom Doherty']

if platform.mac_ver()[0] > '10.8':
  bookdir = os.path.expanduser('~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks')
else:
  bookdir = os.path.expanduser('~/Music/iTunes/iTunes Music/Books')

os.chdir(bookdir)

ok =  '\033[1;32mDRM-free    \033[0m'
bad = '\033[1;31mDRM-infested\033[0m'

count = 0
salvageable = 0

def extract(meta):
  creator = ''
  status = ''
  pub = ''
  et = xml.etree.ElementTree.fromstring(meta)
  try:
    creator = et.findall('*{http://purl.org/dc/elements/1.1/}creator')
    creator = creator[0].text
    title = et.findall('*{http://purl.org/dc/elements/1.1/}title')
    title = title[0].text
  except:
    assert '!DOCTYPE plist' in meta
    next_tag = None
    for e in et[0].iter():
      if e.tag == 'key' and e.text in ('artistName', 'itemName'):
        next_tag = e.text
        continue
      if next_tag == 'artistName':
        creator = e.text
        next_tag = None
        continue
      elif next_tag == 'itemName':
        title = e.text
        next_tag = None
        continue
  pub = [x for x in good if x in meta]
  return creator, title, pub

def find_meta(file_list, opener):
  for m in file_list:
    if m.endswith('.opf') or m == 'iTunesMetadata.plist':
      meta = opener(m).read()
      return extract(meta)
  
for fn in glob.glob('*/*.epub'):
  status = ok
  suffix = ''
  if os.path.isdir(fn):
    suffix = '(directory)'
    if os.path.exists(fn + '/META-INF/encryption.xml'):
      status = bad
      count += 1
    meta = find_meta(os.listdir(fn), lambda x: open(fn + '/' + x))
  else:
    z = zipfile.ZipFile(fn)
    try:
      i = z.getinfo('META-INF/encryption.xml')
      status = bad
    except KeyError:
      pass
    meta = find_meta(z.namelist(), z.open)
    z.close()
  creator, title, pub = meta
  print status, fn, suffix
  print '\t', creator
  print '\t', title
  if status == bad and pub:
    print '\t\033[1;32mThis is published by', pub[0],
    print 'and could be re-downloaded DRM-free\033[0m'
    salvageable += 1

print count, 'books are DRM-infested'
print salvageable, 'could be cured'

Unfortunately, it seems like the DRM-stripping is still work in progress. Out of the Wheel of Time series, for instance, only the first one is now DRM-free on the iBooks store.

undr ~>drmbooks.py
DRM-free     Books/0083D0AEC37E08453347DD12B1C6F980.epub 
	Greg Bear
	Blood Music
DRM-free     Books/09178837756A4DFF8347EC377345A37B.epub 
	Heinz Wittenbrink
	RSS and Atom
DRM-free     Books/0AD752E995042C7E12F11917AB58C6B8.epub 
	Wes McKinney
	Python for Data Analysis
DRM-free     Books/14BDC66A99E878EC232FFAFA73B341EF.epub 
	Fritz Leiber
	Swords and Deviltry-Fafhrd and the Gray Mouser-Book1
DRM-free     Books/15A1D7FE9B7D815C6FBE1A9A77D7143E.epub 
	Glen Cook
	A Fortress in Shadow
DRM-free     Books/1793F9DE1319B96FDE7E36EB8A1BC961.epub 
	Scalzi, John
	Old Man's War
DRM-free     Books/1D08BE221E8BC8F2A371EFEDE55029AC.epub 
	Ben Fry
	Visualizing Data
DRM-free     Books/24D6EC36CDEA0C1E8612CC61A89EA098.epub 
	None
	Node Cookbook
DRM-free     Books/29DA285F0051C431BD8BA3D1AEC5EAA6.epub 
	Fritz Leiber
	The Swords of Lankhmar: Fafhrd and the Gray Mouser-Book 5
DRM-free     Books/2E88CD68DFD8408CD0E7C0ACB1E78714.epub 
	Glen Cook
	A Cruel Wind: A Chronicle of the Dread Empire
DRM-free     Books/32996A9995040064818BAE4DFB66E92F.epub 
	Kelly Link
	Magic for Beginners
DRM-free     Books/34D3CD13D47E5FEBC6DCF7EF011113BD.epub 
	David Drake
	Lord of the Isles
DRM-infested Books/357298432.epub 
	Iain M. Banks
	The Player of Games
DRM-infested Books/357311036.epub 
	Iain M. Banks
	Use of Weapons
DRM-infested Books/357377857.epub 
	Iain M. Banks
	Against a Dark Background
DRM-infested Books/357396585.epub 
	Brent Weeks
	Night Angel: The Complete Trilogy
DRM-infested Books/357657026.epub 
	Iain M. Banks
	Transition
DRM-infested Books/357658374.epub 
	Po Bronson
	NurtureShock: New Thinking About Children
DRM-infested Books/357662058.epub 
	Iain M. Banks
	Consider Phlebas
DRM-infested Books/357669769.epub 
	Iain M. Banks
	Matter
DRM-infested Books/357914731.epub 
	Herbert, Frank
	Dune Messiah
DRM-infested Books/357918110.epub 
	Dalrymple, William
	City of Djinns
DRM-infested Books/357923567.epub 
	Patrick Rothfuss
	The Name of the Wind
DRM-infested Books/357929995.epub 
	Herbert, Frank
	Dune (40th Anniversary Edition)
DRM-infested Books/357969577.epub 
	Herbert, Frank
	God Emperor of Dune
DRM-infested Books/357987322.epub 
	Herbert, Frank
	Children of Dune
DRM-infested Books/357994537.epub 
	Herbert, Frank
	Heretics of Dune
DRM-infested Books/357994652.epub 
	William Dalrymple
	White Mughals: Love and Betrayal in Eighteenth-Century India
DRM-infested Books/357996119.epub 
	Stross, Charles
	Wireless
DRM-infested Books/360601506.epub 
	Ursula K. Le Guin
	The Dispossessed
DRM-infested Books/360609519.epub 
	Greg Egan
	Schild's Ladder
DRM-infested Books/360627712.epub 
	Raymond E. Feist
	Rides a Dread Legion
DRM-infested Books/360627930.epub 
	Neal Stephenson
	Anathem
DRM-infested Books/360628773.epub 
	Raymond E. Feist
	At the Gates of Darkness
DRM-infested Books/360641088.epub 
	Mihaly Csikszentmihalyi
	Flow
DRM-free     Books/361491495.epub 
	Basil Hall Chamberlain
	Aino Folk-Tales
DRM-free     Books/361494664.epub 
	Poul William Anderson
	Industrial Revolution
DRM-free     Books/361523763.epub 
	Lafcadio Hearn
	The Romance of the Milky Way / And Other Studies & Stories
DRM-free     Books/361527545.epub 
	Saki
	When William Came
DRM-free     Books/361539032.epub 
	Saki
	The Chronicles of Clovis
DRM-free     Books/361557387.epub 
	Saki
	Reginald in Russia and other sketches
DRM-free     Books/361557834.epub 
	Sir Arthur Conan Doyle
	The Adventure of the Dying Detective
DRM-free     Books/361559391.epub 
	Sir Arthur Conan Doyle
	The Valley of Fear
DRM-free     Books/361560694.epub 
	Lafcadio Hearn
	Chita: a Memory of Last Island
DRM-free     Books/361561399.epub 
	Confucius
	The Analects of Confucius (from the Chinese Classics)
DRM-free     Books/361562678.epub 
	Saki
	The Toys of Peace, and other papers
DRM-free     Books/361562764.epub 
	Sir Arthur Conan Doyle
	The Memoirs of Sherlock Holmes
DRM-free     Books/361564075.epub 
	Poul William Anderson
	The Burning Bridge
DRM-free     Books/361564898.epub 
	Henry David Thoreau
	Walden
DRM-free     Books/361565201.epub 
	Saki
	Beasts and Super-Beasts
DRM-free     Books/361565806.epub 
	Isaac Asimov
	Youth
DRM-free     Books/361572327.epub 
	Lafcadio Hearn
	Kokoro / Japanese Inner Life Hints
DRM-free     Books/361573126.epub 
	Sir Arthur Conan Doyle
	Through the Magic Door
DRM-free     Books/361575882.epub 
	Sir Arthur Conan Doyle
	Tales of Terror and Mystery
DRM-free     Books/361578744.epub 
	Lafcadio Hearn
	In Ghostly Japan
DRM-free     Books/361588265.epub 
	Lafcadio Hearn
	Books and Habits from the Lectures of Lafcadio Hearn
DRM-free     Books/361673695.epub 
	E. C. Babbitt
	More Jataka Tales
DRM-free     Books/361686559.epub 
	Poul William Anderson
	Security
DRM-free     Books/361713863.epub 
	Sir Arthur Conan Doyle
	The Return of Sherlock Holmes
DRM-free     Books/361721797.epub 
	Sir Arthur Conan Doyle
	The Adventure of the Cardboard Box
DRM-free     Books/361725352.epub 
	Saki
	The Unbearable Bassington
DRM-free     Books/361725959.epub 
	Sir Arthur Conan Doyle
	The Adventure of Wisteria Lodge
DRM-free     Books/361726975.epub 
	Saki
	Reginald
DRM-free     Books/361727237.epub 
	Sir Arthur Conan Doyle
	The Adventure of the Red Circle
DRM-free     Books/361732286.epub 
	Poul William Anderson
	The Valor of Cappen Varra
DRM-free     Books/361736043.epub 
	Lafcadio Hearn
	Kwaidan: Stories and Studies of Strange Things
DRM-free     Books/361741563.epub 
	Lafcadio Hearn
	Japan: an Attempt at Interpretation
DRM-free     Books/361743007.epub 
	Ambrose Bierce
	The Devil's Dictionary
DRM-free     Books/361743953.epub 
	Sir Arthur Conan Doyle
	The Adventure of the Devil's Foot
DRM-free     Books/361744178.epub 
	Sir Arthur Conan Doyle
	His Last Bow
DRM-free     Books/361745602.epub 
	Poul William Anderson
	The Sensitive Man
DRM-infested Books/362435686.epub 
	Ansary, Tamim
	Destiny Disrupted
DRM-infested Books/366773380.epub 
	Esslemont, Ian C. C.
	Return of the Crimson Guard
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/373338999.epub 
	Jordan, Robert
	The Path of Daggers
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-free     Books/375554215.epub 
	Orson Scott Card
	The Lost Gate
DRM-infested Books/376217648.epub 
	Steven Erikson
	Reaper’s Gale
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/376227359.epub 
	Jordan, Robert
	Winter's Heart
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/376227401.epub 
	Jordan, Robert
	Crossroads of Twilight
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/376227406.epub 
	Jordan, Robert
	Knife of Dreams
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/376227409.epub 
	Sanderson, Brandon
	The Gathering Storm
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/376227423.epub 
	Jordan, Robert
	New Spring
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-free     Books/376231110.epub 
	Cook, Glen
	Surrender to the Will of the Night
DRM-infested Books/376231528.epub 
	Robert Jordan and Brandon Sanderson
	Towers of Midnight
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/378317076.epub 
	Jordan, Robert
	A Crown of Swords
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/378317808.epub 
	Robert Jordan
	Lord of Chaos
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-free     Books/379451459.epub 
	Le Guin, Ursula K.
	Word for World is Forest, The
DRM-free     Books/37EC6E895E6BD70BEB48D9F1553D608E.epub 
	Eben Hewitt
	Cassandra: The Definitive Guide
DRM-free     Books/380490608.epub 
	Jordan, Robert
	The Eye of the World
DRM-free     Books/380494444.epub 
	Asimov, Isaac
	The End of Eternity
DRM-infested Books/381497257.epub 
	Harold McGee
	On Food and Cooking, The Science and Lore of the Kitchen
DRM-infested Books/381622032.epub 
	IAIN M. BANKS
	Look to Windward
DRM-infested Books/381683084.epub 
	Ursula K. Le Guin
	Tehanu
DRM-infested Books/381935940.epub 
	Richard Adams
	Watership Down
DRM-infested Books/382674388.epub 
	Steven Erikson
	Dust of Dreams
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/383912791.epub 
	Steven Erikson
	Bauchelain and Korbal Broach
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385975662.epub 
	Jordan, Robert
	The Dragon Reborn
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385981104.epub 
	Steven Erikson
	The Bonehunters
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385981116.epub 
	Steven Erikson
	Midnight Tides
DRM-free     Books/385982966.epub 
	Brust, Steven
	To Reign in Hell
DRM-infested Books/385987858.epub 
	Steven Erikson
	Gardens of the Moon
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385989170.epub 
	Steven Erikson
	House of Chains
DRM-infested Books/385992628.epub 
	Jordan, Robert
	The Fires of Heaven
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385992927.epub 
	Steven Erikson
	Toll the Hounds
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385992930.epub 
	Jordan, Robert
	The Great Hunt
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/385998417.epub 
	Jordan, Robert
	The Shadow Rising
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/386016540.epub 
	Esslemont, Ian C. C.
	Night of Knives
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/388403394.epub 
	Steven Erikson
	The Crippled God
DRM-infested Books/389191300.epub 
	Iain M. Banks
	Surface Detail
DRM-infested Books/390877859.epub 
	Loewen, James W.
	Lies My Teacher Told Me
DRM-infested Books/393310992.epub 
	Erikson, Steven
	Memories of Ice
DRM-infested Books/394745271.epub 
	Steven Erikson
	Deadhouse Gates
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-free     Books/394745833.epub 
	Walton, Jo
	Among Others
DRM-free     Books/395536306.epub 
	Sir Arthur Conan Doyle
	The Adventures of Sherlock Holmes
DRM-free     Books/395537209.epub 
	Sir Arthur Conan Doyle
	The Sign of the Four
DRM-free     Books/395539542.epub 
	Sir Arthur Conan Doyle
	A Study in Scarlet
DRM-free     Books/395540660.epub 
	Sir Arthur Conan Doyle
	The Hound of the Baskervilles
DRM-free     Books/395686685.epub 
	Dante Alighieri
	Divine Comedy, Longfellow's Translation, Complete
DRM-free     Books/395688318.epub 
	Edgar Rice Burroughs
	A Princess of Mars
DRM-free     Books/395688375.epub 
	Lafcadio Hearn
	Glimpses of an Unfamiliar Japan / First Series
DRM-infested Books/395926792.epub 
	Fukuyama, Francis
	Origins of Political Order
DRM-infested Books/396269736.epub 
	Herbert, Frank
	Chapterhouse: Dune
DRM-infested Books/398283114.epub 
	Rothfuss, Patrick
	The Wise Man's Fear
DRM-free     Books/3A5FBC58E821CFDF15C8C4E85657481E.epub 
	Jon Hicks
	The Icon Handbook
DRM-free     Books/410943153.epub 
	Edwin A. Abbott (A Square)
	Flatland: A Romance of Many Dimensions
DRM-free     Books/413463878.epub 
	Brust, Steven
	Tiassa
DRM-free     Books/418293515.epub 
	Heinlein, Robert A.
	Glory Road
DRM-infested Books/419950945.epub 
	Isaac Asimov
	Foundation
DRM-infested Books/419950970.epub 
	Isaac Asimov
	Foundation and Empire
DRM-infested Books/419950976.epub 
	Isaac Asimov
	Second Foundation
DRM-infested Books/419968238.epub 
	Scott Lynch
	The lies of Locke Lamora
DRM-infested Books/419968784.epub 
	Scott Lynch
	Red Seas Under Red Skies
DRM-infested Books/420037362.epub 
	Kim Stanley Robinson
	The Years of Rice and Salt
DRM-infested Books/420281728.epub 
	Richard Wiseman
	59 Seconds: Think a Little, Change a Lot
DRM-infested Books/420445771.epub 
	Isaac Asimov
	Foundation’s Edge
DRM-infested Books/420446058.epub 
	Isaac Asimov
	Foundation and Earth
DRM-infested Books/420725428.epub 
	Mike Resnick
	Kirinyaga: A Fable of Utopia
DRM-infested Books/421025353.epub 
	William Dalrymple
	The Last Mughal
DRM-free     Books/421124117.epub 
	Brust, Steven
	The Desecrator
DRM-infested Books/422530144.epub 
	Max Barry
	Machine Man
DRM-free     Books/422718511.epub 
	Apple Inc.
	Mac Integration Basics
DRM-free     Books/426914658.epub 
	Brust, Steven
	Five Hundred Years After
DRM-free     Books/428235697.epub 
	Vinge, Vernor
	A Fire Upon The Deep
DRM-infested Books/429173089.epub 
	Ursula K. Le Guin
	The Other Wind
DRM-infested Books/429173713.epub 
	Ursula K. Le Guin
	Tales from Earthsea
DRM-infested Books/429699133.epub 
	Rajaniemi, Hannu
	The Quantum Thief
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/431617578.epub 
	Walter Isaacson
	Steve Jobs
DRM-infested Books/432519291.epub 
	Susan Weinschenk
	100 Things: Every Designer Needs to Know About People
DRM-infested Books/434509014.epub 
	Stross, Charles
	Rule 34
DRM-free     Books/434522188.epub 
	Larry Niven, Jerry Pournelle
	The Mote In God's Eye
DRM-free     Books/434811509.epub 
	Asher, Neal
	Cowl
DRM-infested Books/436646026.epub 
	Neal Stephenson
	Reamde
DRM-infested Books/436691174.epub 
	Julia Child
	Mastering the Art of French Cooking
DRM-infested Books/443149884.epub 
	Daniel Kahneman
	Thinking, Fast and Slow
DRM-infested Books/446155927.epub 
	Esslemont, Ian C. C.
	Stonewielder
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-free     Books/447591195.epub 
	Asher, Neal
	The Skinner
DRM-infested Books/454252718.epub 
	William B. Norton
	The Internet Peering Playbook: Connecting to the Core of the Internet
DRM-infested Books/455525627.epub 
	Amar Chitra Katha
	Birbal The Genius
DRM-infested Books/458461362.epub 
	Pamela Druckerman
	Bringing Up Bebe
DRM-free     Books/45B90418E467D479DCDDF23B932C648C.epub 
	Douglas Crockford
	JavaScript: The Good Parts
DRM-infested Books/460822066.epub (directory)
	Scott Lynch
	The Republic of Thieves
DRM-free     Books/465679A557523FDB836005CF4BB9380E.epub 
	Scott Berkun
	Mindfire
DRM-infested Books/479594044.epub 
	Saladin Ahmed
	Throne of the Crescent Moon
DRM-infested Books/479717436.epub 
	David Crist
	The Twilight War: The Secret History of America’s Thirty-Year Conflict with Iran
DRM-infested Books/479771801.epub 
	William Dalrymple
	In Xanadu
DRM-infested Books/489957500.epub 
	Bruce Schneier
	Liars and Outliers
DRM-infested Books/491186678.epub 
	James Blish
	Cities in Flight
DRM-infested Books/491668459.epub 
	Neal Asher
	Shadow of the Scorpion
DRM-infested Books/491669284.epub 
	Glen Cook
	A Matter of Time
DRM-infested Books/491669288.epub 
	Glen Cook
	Darkwar
DRM-free     Books/492199230.epub 
	Frederik Pohl
	The Tunnel Under the World
DRM-free     Books/492199569.epub 
	Frederik Pohl
	The Knights of Arthur
DRM-infested Books/494939678.epub 
	Esslemont, Ian C. C.
	Orb Sceptre Throne
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/498634992.epub 
	Charles Stross
	The Apocalypse Codex
DRM-infested Books/499392787.epub 
	Daniel Suarez
	Kill Decision
DRM-free     Books/4AE7DDA54BEEFCD65157927546B18063.epub 
	Roberto Ierusalimschy
	Programming in Lua 2ed
DRM-free     Books/4DCFF682728B765A1CE221F3D7C21536.epub 
	Glen Cook
	Starfishers Volume 3: Stars' End
DRM-free     Books/501278407.epub 
	Colette
	Chéri
DRM-free     Books/501758197.epub 
	David Brin
	Existence
DRM-infested Books/501758516.epub 
	Scalzi, John
	Redshirts
	This is published by Tom Doherty and could be re-downloaded DRM-free
DRM-infested Books/503019669.epub 
	J.R.R. Tolkien
	The Lord of the Rings
DRM-infested Books/503153300.epub 
	J.R.R. Tolkien
	Tales from the Perilous Realm
DRM-infested Books/503154129.epub 
	J. R. R. Tolkien and Christopher Tolkien
	The Book of Lost Tales, Part One
DRM-infested Books/503154991.epub 
	J. R. R. Tolkien and Christopher Tolkien
	The Book of Lost Tales, Part Two
DRM-infested Books/503155327.epub 
	J.R.R. Tolkien
	The Children of Húrin
DRM-infested Books/503163148.epub 
	J.R.R. Tolkien
	The Hobbit Deluxe
DRM-infested Books/503164678.epub 
	J.R.R. Tolkien
	The Silmarillion
DRM-infested Books/503167303.epub 
	J.R.R. Tolkien
	Unfinished Tales of Númenor and Middle-earth
DRM-infested Books/504209078.epub 
	Isaac Asimov
	Prelude to Foundation
DRM-infested Books/504371982.epub 
	Iain M. Banks
	The Hydrogen Sonata
DRM-free     Books/511060740.epub 
	Frederik Pohl
	The Hated
DRM-free     Books/511143617.epub 
	Frederik Pohl
	The Day of the Boomer Dukes
DRM-free     Books/511252896.epub 
	Frederik Pohl
	Pythias
DRM-free     Books/513357605.epub 
	Hannu Rajaniemi
	The Fractal Prince
DRM-free     Books/513868CF0AA46D293EE27F74BC399760.epub 
	Jerry Pournelle
	West of Honor
DRM-infested Books/520233773.epub 
	Nate Silver
	The Signal and the Noise
DRM-free     Books/520897403.epub 
	Hugh Howey
	Wool Omnibus
DRM-free     Books/525170910.epub 
	Cory Doctorow and Charles Stross
	The Rapture of the Nerds
DRM-infested Books/526136048.epub (directory)
	Hetty van de Rijt & Frans Plooij
	The Wonder Weeks
DRM-infested Books/529020127.epub 
	Neal Asher
	The Departure
DRM-infested Books/529424632.epub 
	Guy Gavriel Kay
	The Lions of Al-Rassan
DRM-infested Books/536312376.epub 
	Glen Cook
	Garrett for Hire
DRM-free     Books/537023027.epub 
	Erikson, Steven
	Forge of Darkness
DRM-infested Books/541673159.epub 
	Ursula K. Le Guin
	The Tombs of Atuan
DRM-infested Books/541673162.epub 
	Ursula K. Le Guin
	The Farthest Shore
DRM-infested Books/546126326.epub 
	Jan Morris
	HAV
DRM-infested Books/551241606.epub 
	Ursula K. Le Guin
	A Wizard of Earthsea
DRM-infested Books/551567785.epub 
	Murray R. Spiegel, PhD
	Schaum's Outline Mathematical Handbook of Formulas and Tables, Fourth Edition
DRM-infested Books/551747038.epub 
	Chris Hedges and Joe Sacco
	Days of Destruction Days of Revolt v2b
DRM-infested Books/552144691.epub 
	Tamim Ansary
	Games without Rules
DRM-free     Books/553878102.epub 
	Charles Stross
	A Tall Tail
DRM-infested Books/563408849.epub 
	Iain Banks
	Stonemouth
DRM-infested Books/568731449.epub 
	William Dalrymple
	Return of a King: The Battle for Afghanistan, 1839-42
DRM-infested Books/569232538.epub 
	Neil Gaiman
	The Ocean at the End of the Lane
DRM-free     Books/571678152.epub 
	Glen Cook
	The Return of the Black Company
DRM-free     Books/571678945.epub 
	Glen Cook
	The Many Deaths of the Black Company
DRM-free     Books/573656304.epub 
	Glen Cook
	Chronicles of the Black Company
DRM-free     Books/573656441.epub 
	Glen Cook
	The Books of the South
DRM-free     Books/576233114.epub 
	Jack Vance
	Demon Princes
DRM-infested Books/578851675.epub 
	Zilpha Keatley Snyder
	Below The Root
DRM-infested Books/580642602.epub 
	Max Barry
	Lexicon
DRM-infested Books/588794444.epub 
	Charles Stross
	Neptune's Brood
DRM-free     Books/5CCB889F91585202637A3C5FBFD8409F.epub 
	Glen Cook
	Starfishers-The Starfishers Trilogy Volume II
DRM-infested Books/600938002.epub 
	Gardner Dozois
	The Year’s Best Science Fiction: Thirtieth Annual Collection
DRM-free     Books/606232503.epub 
	Ian C. Esslemont
	Blood and Bone
DRM-free     Books/610920977.epub 
	Robert Jordan and Brandon Sanderson
	A Memory of Light
DRM-free     Books/6122FF30560866BD75257E6CCC264371.epub 
	Fritz Leiber
	Swords and Ice Magic-Fafhrd and the Gray Mouser-Book 6
DRM-infested Books/619483561.epub 
	Raymond E. Feist
	Magician's End
DRM-free     Books/622837311.epub 
	Le Comte De  Lautréamont
	Les chants de Maldoror
DRM-free     Books/62497334A4189B1D00E9BDFD95724E2E.epub 
	Fritz Leiber
	The Knight and Knave of Swords-Fafhrd and the Gray Mouser-Book 7
DRM-infested Books/645571245.epub 
	Iain Banks
	The Quarry
DRM-free     Books/647688922.epub 
	Steven Brust and Skyler White
	The Incrementalists
DRM-infested Books/651331715.epub 
	Susan Crawford
	Captive Audience
DRM-infested Books/654456347.epub 
	Iain Banks
	The Wasp Factory
DRM-infested Books/662310218.epub (directory)
	Various Authors
	Star Wars: Empire Volume 3 – The Imperial Perspective
DRM-infested Books/662310219.epub 
	Paul Gulacy
	Star Wars: Crimson Empire
DRM-infested Books/664297397.epub (directory)
	Various Authors
	Star Wars: Empire, Vol. 4: The Heart of the Rebellion
DRM-infested Books/664343525.epub (directory)
	Paul Chadwick, Doug Wheatley & Tomás Giorello
	Star Wars: Empire, Vol. 2: Darklighter
DRM-infested Books/664894567.epub 
	John Ostrander
	Star Wars: Dawn of the Jedi Volume 1—Force Storm
DRM-infested Books/664910993.epub (directory)
	Scott Allie, Ryan Benjamin & Brian Horton
	Star Wars: Empire Vol. 1
DRM-free     Books/666772E4C21E2017E71C10F1990840BC.epub 
	Pieter Hintjens
	ZeroMQ - Connecting your Code
DRM-infested Books/674224604.epub (directory)
	Various Authors
	Star Wars: Empire, Vol. 5: Allies and Adversaries
DRM-infested Books/674226603.epub (directory)
	Thomas Andrews, Scott Allie & Various Authors
	Star Wars: Empire, Vol. 6: In the Shadows of Their Fathers
DRM-free     Books/697938901.epub 
	Charles Stross
	Equoid: A Laundry Novella
DRM-free     Books/6BBD61A64A46FFFF4AE7C5FCEB9CFCEE.epub 
	David Drake
	Balefires
DRM-free     Books/71C79A253586282ABE73C2975237EB08.epub 
	Glen Cook
	Sung in Blood
DRM-free     Books/73BC5DD0E51611BDC359CBAB48CC203F.epub (directory)
	Crane, Stephen
	The Red Badge of Courage
DRM-free     Books/7A0D1AEE343638A8A3769DABB90CD4CB.epub 
	Clay A. Johnson
	The Information Diet
DRM-free     Books/7B46914B55481714DED3D711288978FA.epub 
	Glen Cook
	Shadowline-The Starfishers Trilogy I
DRM-free     Books/809DCC75FE60E749458CB27636EE6777.epub 
	Mercedes Lackey
	The Secret World Chronicle
DRM-free     Books/8350F5DFECCD6AA88714400FDF4F6831.epub 
	François de La Rochefoucauld
	Réflexions ou Sentences et Maximes Morales
DRM-free     Books/853BD1C65277D276BA09E04EBFEB73EF.epub 
	None
	PostgreSQL 9 Administration Cookbook
DRM-free     Books/85907063B2351E91C6E7F5052C090BFD.epub 
	None
	PostgreSQL 9.0 High Performance
DRM-free     Books/8663E7BF735C06318FF450532A67F1C2.epub 
	Glen Cook
	Passage at Arms
DRM-free     Books/8690D69995483C5D2DF6AD38BE53C1D7.epub 
	Paolo Bacigalupi
	The Windup Girl - Second Electronic Edition
DRM-free     Books/8993920EF1C0E8FE8CB47A20BD955F53.epub 
	Fritz Leiber
	Swords Against Death-Fafhrd and Gray Mouser-Book 2
DRM-free     Books/8DC6ED0A99161EBB516E370A60ED1121.epub 
	Jonathan Zdziarski
	Hacking and Securing iOS Applications
DRM-free     Books/8DD0D22CD05B0B8D52DF9DB93FC8616B.epub 
	Fritz Leiber
	Swords in the Mist-Fafhrd And the Gray Mouser-Book 3
DRM-free     Books/912CB8B110736684549EAC4FC36665AB.epub 
	Neil Gaiman and Dave McKean
	Signal to Noise
DRM-free     Books/93061FDFD6EEC01FD8CE4295A049C97C.epub 
	Cory Doctorow
	Homeland
DRM-free     Books/9592D5001632B94D1FEFC98B3A40E049.epub 
	Kelly Link
	Stranger Things Happen
DRM-free     Books/990CA5799084407151488A7C563DF269.epub 
	Tom Hughes-Croucher
	Node: Up and Running
DRM-free     Books/9CCCA9E217602948B84BE9A7A21C2753.epub 
	Mike Resnick
	Birthright: The Book of Man
DRM-free     Books/9DA2C8D7E941C37FA83192E5849AA850.epub 
	Q. Ethan McCallum
	Parallel R
DRM-free     Books/A7DB557515983DB67F74E2BE351FC319.epub 
	Glen Cook
	Reap the East Wind
DRM-free     Books/B18AB85267B815E540C7E370F8D97726.epub 
	Lars George
	HBase: The Definitive Guide
DRM-free     Books/B1A586D24AE5F2450010F8664F8E059D.epub 
	Cory Doctorow
	Pirate Cinema
DRM-free     Books/B562B5D74F8677C946B0A6F81EB34F1B.epub 
	Gotthold Ephraim Lessing
	Nathan the Wise; a dramatic poem in five acts
DRM-free     Books/BE42D426242836AA171539B7415732E6.epub 
	Glen Cook
	The Swordbearer
DRM-free     Books/BE80AD93781746E45CF95607A7BEE687.epub 
	Charlie Stross
	Bit Rot
DRM-free     Books/C39FBC59673713A57D404D62BE85C4DC.epub 
	Lauren Beukes
	Zoo City
DRM-free     Books/C81CBBD470EFDBC62359BDAD12FDF551.epub 
	Thomas Hobbes
	Leviathan
DRM-free     Books/D17DC1A83BD90AFB24055A01831BFAB4.epub 
	Glen Cook
	The Dragon Never Sleeps
DRM-free     Books/DA8FBAB386EC503A0EF21E481D214521.epub 
	David Flanagan
	JavaScript: The Definitive Guide
DRM-free     Books/DBAD0DE7A4305B90F6AC33C673FE68E8.epub 
	Glen Cook
	A Path to Coldness of Heart
DRM-free     Books/DBCCB54B140D1158250B24B7E3E81B63.epub 
	Scott Chacon
	Pro Git
DRM-free     Books/DF6C56F9A8719294B13BA91BED5E1667.epub 
	Mike Resnick
	Ivory
DRM-free     Books/E8D76FDD351E68D92AE4A3F3AEA7EC6A.epub 
	Paolo Bacigalupi
	Pump Six and Other Stories
DRM-free     Books/EEE65A68378C2510E18DF24CD767AC9A.epub 
	Fritz Leiber
	Swords Against Wizardry-Fafhrd and the Gray Mouser-Book 4
DRM-free     Books/F3BC14466A16C96CCD8FFE00DCCF8147.epub 
	Glen Cook
	An Empire Unacquainted With Defeat
DRM-free     Books/F534835595041374B814D151A633E69D.epub 
	Peter Watts
	Blindsight
DRM-free     Books/F8BF760284ADC800B290DCA6D8EA7EF2.epub 
	Ben Klemens
	21st Century C
DRM-free     Books/FE8C57863E40B98CB732FEE4BFDB60BB.epub 
	Glen Cook
	An Ill Fate Marshalling
8 books are DRM-infested
26 could be cured

Update (2013-11-06):

OS X 10.9 Mavericks and the new iBooks app changed the location of the iBooks directory, I changed my script accordingly (and made it adjust depending on which OS version you have). Also, the file names have changed and no longer embed author and title, so I am extracting them from the XML metadata files.

The Gresham’s law of Amazon Web Services

In the bad (good?) old days when currency’s worth was established by the amount of gold or silver in coinage, kings would cut corners by debasing currency with lead, which is almost as dense as gold or silver. In the New World, counterfeiters debased gold coins with platinum, which was first smelted by pre-columbian civilizations. Needless to say, the fakes are now worth more than the originals.

The public was not fooled, however, and found ways to test coins for purity, including folkloric ones like biting a coin to see if it is made of malleable gold, rather than harder metals. People would then hoard pure gold coins, and try to rid themselves of debased coins at the earliest opportunity. This led to Gresham’s Law: bad money drives out good money in circulation.

After a year of using Amazon Web Services’ EC2 service at scale for my company (we moved to our own servers at the end of 2011), I conjecture there is a Gresham’s Law of Amazon EC2 instances – bad instances drive out good ones. Let me elaborate:

Amazon EC2 is a good way to launch a service for a startup, without incurring heavy capital expenditures when getting started and prior to securing funding. Unfortunately, EC2 is not a quality service. Instances are unreliable (we used over 80 instances at Amazon, and there was at least one instance failure a week, and sometimes up to 4). Amazon instances have poor disk I/O performance that makes them particularly unsuitable to hosting non-trivial databases (EBS is even worse, and notoriously unreliable).

Performance is also inconsistent—I routinely observed “runt” m1.large instances that performed half as well as the others. We experienced all sorts of failure modes, including disk corruptions, disks that would block forever without timing out, sporadic losses of network connectivity, and many more. Even more puzzling, I would get 50% to 70% failure rate on new instances that would not come up cleanly after being launched.

Some of this is probably due to the fact we use an uncommon OS, OpenSolaris, that is barely supported on EC2, but I suspect a big part of this is that Amazon uses low-end commodity parts, and does not proactively retire failed or flaky hardware from service. Instances that have the bad luck of being assigned to flaky hardware are more likely to fail or perform poorly, and thus more likely to be be destroyed, released and a new one reassigned in the same slot. The inevitable consequence of this is that new instances have a higher likelihood of being runts or otherwise defective than long-running ones.

One work-around is to spin up a large number of instances, test them, and destroy the poor-performing ones. AWS runts are usually correlated with slower CPU clock speeds, as older machines would be running older versions of the Xen hypervisor Amazon uses under the hood, have less cache, slower drives and so on. Iterating through virtual machines as if you are picking melons at a supermarket is a slow and painful job, however, and even their newer machines have their share of runts. We were trying to keep only machines with 2.6 or 2.66GHz processors, but more than 70% of the instances we were getting assigned were 2.2GHz runts, and it would usually take creating 5 or 6 instances on average to get a non-runt.

In the end, we migrated to our own facility in colo, because Amazon’s costs, reliability and performance were just not acceptable, and we had long passed the threshold beyond which it is cheaper to own than rent (I estimate it at $5,000 to $10,000 per month Amazon spend, depending on your workload). It is not as if other cloud providers are any better—before Amazon we had started on Joyent, which supports OpenSolaris natively, and their MTBF was in the order of 2 weeks, apparently because they replaced their original Sun hardware with substandard Dell servers and had issues with power management C-states in the Dell server BIOS.

The dirty secret of cloud services is that there is no reliable source of information on actual performance and reliability of cloud services. This brings out another economic concept, George Akerlof’s famous paper on the market for lemons. In a market where information asymmetry exists, the market will eventually collapse in the absence of guarantees. Until Amazon and others offer SLAs with teeth, you should remain skeptical about their ability to deliver on their promises.

Withings smart baby monitor review

One of the joys challenges of being a first-time parent is being exposed to a bewildering array of gadgets and equipment required to care for the baby, from baby car seats, strollers and diaper pails to 2-axis rocking robots (thanks Rohit!). There is an entire cottage industry of books like Baby Bargains that help you navigate through the confusing and sometimes questionable or outright unnecessary choices.

I have a Withings body weight scale that I really like and I was excited to learn they were going to release a networked video baby monitor. It took a while to get to market in the US, however, so in the interim I purchased a Philips Advent DECT digital baby monitor, which ended up unusable in practice, because its microphone sensitivity is so poor that you can barely hear anything. When the Withings baby monitor finally became available in the US, I immediately ordered it.

Withings is clearly taking design cues from Apple, from the lavishly designed packaging to the glossy white plastic RoundedRect aesthetic and the use of a magnetic clip to attach the baby monitor to the crib. The clip is serviceable, but the magnets are not quite strong enough to hold the unit firmly onto the crib. I would not trust it to keep the monitor from toppling when the baby grows and kicks at the crib. Fortunately they also include a flip-out tab on the base of the unit that can be inserted into a slit on the clip to prevent sliding, although it is not obvious and it took me a while before I discovered this key feature.

The wall wart is a generic black model with swappable AC prongs for international markets, and detracts from the overall package, but since the monitor has a micro-USB input, you can always use another standard AC to USB type A adapter like the iPhone’s, with a USB type A to micro-USB cable. A rechargeable battery is included, with 2 hours’ claimed life, I did not verify that spec.

The initial out of the box experience is good: you connect to the device from your iPhone or iPad using Bluetooth (no messing around with a USB cable as with the Withings scale), enter the WiFi settings in the Withbaby app, and then use WiFi to access the device afterwards. It is as streamlined an experience as you can expect without a keyboard on the unit. There is also an Ethernet jack (it is unclear whether it supports power over Ethernet), but my house was built in 1928 and is not wired upstairs where the baby lives.

Once you enter your credentials into the app, it connects to the monitor and shows you the video and sound. If you put it in the background, you have the option of monitoring audio. Withings will also send you alerts via push notifications if the temperature or humidity is excessive, or if it detects noise or motion. The default settings are way too twitchy, however, and you will find yourself disabling audio notifications as the deluge of alerts is just too much.

The device includes a night light with selectable color, a lullaby player, and the ability to speak to your baby, all controlled through the app. At the front you also have touch controls to turn some of these features on. This is actually a bad idea, as on two occasions I started the lullaby by accident as I was fumbling with it in a dark room, and woke up my baby as a result. Another design flaw is the pulsing blue night light when the unit is rebooting, the Airport Express like amber/green status LED in the back is quite sufficient. Frankly the only one of these features that is useful is the speaker, and the ability to stream from your music collection, such as Dr. Harvey Karp’s white noise selections would be preferable to the canned lullabies.

The video camera is advertised as having a 3 megapixel sensor. It has a wide-angle lens and you can “pan” using the usual iPhone or iPad gestures. The lens is a fixed-focus plastic one, and optical clarity is so-so at best, optimal focus seems to be at 50cm or so. One great feature is the monitor has a normal and night vision mode, similar to the one on some Sony HAD camcorders, with an IR illuminator that provides light for the night vision mode. This means you can watch your baby toss and turn in an otherwise pitch-black room.

You can use the baby monitor from outside your network, and it works fine, even over a 3G connection. Withings allows you up to 15 minutes per day, anything beyond that requires paying them $6 for each 100 minutes. Coming on top of an already expensive device, this seems like a naked money grab from anxious parents. (Updated 2012-09-29: remote monitoring is now free and unlimited).

When the unit works, it is absolutely great: good sound sensitivity and the video feature mostly works as advertised. Unfortunately it frequently does not function, and I find myself performing a hard reboot by removing the battery far more often than I would like. Among the pathologies:

  • Once it falsely reported the unit was closed and thus video inaccessible
  • Once the camera was in a frozen state, it took a power cycling to get the video moving again.
  • Yesterday I could not connect at all, no matter how many times I rebooted my Airport Extreme, the monitor and my wife’s or my iPads. Some detective work using a packet sniffer showed the app was trying to connect to babyws.withings.net using HTTP, which is aliased to s11.withings.net, and that server was down. Some of the documentation suggests you can use the Bluetooth connection to access the monitor, but I was not able to figure out how to do this.

This brings me to a crucial point. The baby monitor is a safety device, and it is utterly unacceptable for its functioning to be dependent on a cloud service, which can and will be a single point of failure. It should use Bonjour or similar discovery methods to work on the LAN, and rely on Withings’ servers only when accessing it from outside the home LAN’s perimeter. I wonder if Withings’ eagerness to nickel-and-dime users by charging for outside monitoring led to this critical design flaw.

The bottom line is the Withings smart baby monitor is a very frustrating device, with its obvious potential marred by failures of execution. If it worked consistently, it would be a top-notch product worthy of its Apple inspiration and lofty price tag, but the general lack of reliability means I cannot recommend it until the bugs are ironed out. Consider it an alpha release at best.

Update (2012-09-17):

Here’s how to make the Withings not-so-smart baby monitor more usable:

  • Remove the battery from the unit and hook up the micro-USB power adapter to a Belkin WeMo remote-controlled power switch. This allows you to power-cycle the baby monitor remotely from the same iPhone or iPad you are using the monitor software on.
  • Hide the blue led with gaffer’s tape. This prevents the blue light on reboot from waking the baby. Unlike duct tape, gaffer’s tape can be removed without leaving glue residue, although the aesthetics of dark gray gaffer’s tape on the gleaming white unit are questionable at best.
  • I haven’t tried covering up the touch controls with gaffer’s tape, which would eliminate the risk of triggering a jingle and waking the baby. The WeMo eliminates the need to enter the room and tinker with the baby monitor.

It’s quite sad to have to pay an extra $50 to work around buggy hardware and software, but it makes a big difference.

Update (2013-05-20):

The micro-USB connector failed and the baby monitor is now essentially a doorstop. Not surprising given how flimsy micro-USB is, compared to mini-USB, for insignificant space savings. Micro-USB was a Nokia design rammed through the USB-IF. In theory it has better insert-remove cycle life than mini-USB, but in practice I’ve never had mini-USB fail, whereas it is a frequent occurrence with micro-USB.

Update (2015-08-17):

USB-C is an improvement over micro-USB, hopefully some future version of the baby monitor will use it. Still nowhere near as robust as Lightning or tip-ring-sleeve, though.

Welcome, baby Afsheen!

My first child, Afsheen Zahra Majid, formerly code-named “Lolo”, was born Tuesday 2012-01-03 at 10:47 AM Pacific Time in San Francisco. She weighed 4,148 grams (9.2lbs) and was 56.5cm long (20.25″) at birth. She has a full head of hair, strong lungs, and bright shiny eyes that were wide open when she came out.

Both mother and child are exhausted and recovering at California Pacific Medical Center, and learning to know one another. I can only watch with rapt attention at the miracle of life unfolding, bursting as I am with joy.

Prints not-so charming

Ansel Adams wrote a celebrated series of photo instructional books. It is organized as a trilogy: The Camera, The Negative and The Print.

Of these, the camera business is still going strong, buoyed by sales of digital cameras and upgraders to DSLRs (although market saturation looms). Negatives are an endangered species as digital photography largely supplanted film. Prints are still going strong, whether using traditional silver halide, inkjet or offset printing, but that probably won’t stave off Kodak’s impending demise.

Last year, I wrote to X-Rite to complain that their software for the Colormunki color calibrator wouldn’t let me profile the full brightness range of my monitor, and limited it to an artificial ceiling designed around the limitations of print. Their response:

The ColorMunki software will not allow a range over 140cd/m luminance value.  This would basically defeat the purpose of using a device to measure luminance when you set the values this high.  While your monitor can be extremely bright however using a value of 300+ would allow your monitor to show you a brightness value that would negatively affect your prints and they would end up coming out so dark that you would be unable to see them.  The industry standard for photographic works is to use the luminance values setup at anywhere from 80 – 120cd/m.  That said, if your ambient light conditions are ranging very high then you should probably be using a value of 120cd/m which is the industry standard and what we recommend.

In the ColorMunki software there is an option to set the luminance according to your ambient light conditions.  This is a pretty high threshold in the software where as you have to read somewhere around 350-400 lux to get a target luminance above the 80 threshold.  If it does go above this value then you certainly will want to stay within the 120 range.  Anything below that will give you a target value of 80.

Implicit in this response is the assumption the only reason why you would want to calibrate your monitor is so you can make more accurate prints. Prints only have a 100:1 or so contrast ratio, compared to LCD monitors for which the static contrast ratio is closer to 1000:1, and plasma TVs go higher yet. This is why images on screen look more vibrant and punchy than prints, as did slides back in the day.

If Apple does indeed come up with the oft-rumored iPad 3 with double the resolution of current models, or 2048×1536 at 260ppi, you would have an easily portable display device with near gallery-grade resolution and massive storage capacity. At that point, prints would lose the last advantage they still hold over digital display technologies.

Image display on screen is the new normal, and X-Rite needs to get with the program. I want to see the full dynamic range and contrast of my images on my screen, not a Hobson’s choice between inaccurate color and crippling them with limitations from legacy print technology.

 

Fixing Microsoft Office Mac 2011 font woes

This post may be of use to people experiencing the same problem I had. When I opened PowerPoint slides sent by a coworker, I was getting blank slides. Attempting to switch to outline view would crash PowerPoint. A search for possible solutions yielded inconclusive posts or dead-ends.

After much twiddling, it turns out the problem was with a misconfigured Arial font. Microsoft in its great wisdom installs a copy of Arial that conflicts with the system-supplied one, as explained in Kurt Lang’s must-read article on OS X font management, but that was not the root cause of the problem.

Arial was not appearing at all in the PowerPoint font menu, and opening the slides in Keynote yielded a font warning saying non-existent Arial was replaced by Arial Narrow. Clearly PowerPoint is much less resilient than Keynote to missing fonts (crashing in outline view mode is inexcusable, but that’s Microsoft coding sloppiness for you). If Arial is so vital, it should have been included as a resource in the application bundle itself, but I digress.

Font Book did show Arial as installed, with only a single copy after I purged the conflicting fonts as per Kurt Lang’s recommendations. Validating Arial in Font Book yielded a warning about duplicate fonts, but with no indication of where the duplicate may be, since Font Book itself did not know of any other instance.

It turns out I had an old pre-OpenType copy of Arial in my ~/Library/Fonts folder that was causing a conflict. It was functional enough to be picked up as conflicting and disabling Arial in PowerPoint and Keynote, but not functional enough to be listed by Font Book. I am not sure how it got there, possibly from an older version of Office and transferred over many machine upgrades. Removing the file fixed the problem. of blank slides and the crash in switching to outline view.

Of course, if you care at all about typography, you should use Helvetica rather than an inferior ersatz like Arial, but Arial is so prevalent in the taste-impaired Windows world that one cannot escape its gaucheness entirely.

 

Finding the elusive “significant other” camera

We I are expecting a baby girl sometime in January. It would be nice if I were the subject of photos for a change, not just the one taking them. I gave my wife my Fuji F31fd, a compact zoom camera introduced in 2007, and that became such a cult favorite for its relatively clean high-ISO images that after it was discontinued it would until recently command a higher price used on eBay than the original MSRP. That said, it does not have the same image quality as even an entry-level DSLR, and I would like her to use something more capable.

The challenge is that as a normal human, she can’t really use my own cameras. The Leica M9’s rangefinder focusing is out, the Fuji X100’s complicated user interface is as well, and the Canon 5DmkII is too heavy for her to carry along everywhere. There is a crying need for a compact, high-quality camera that is fully automated for non-technical users, yet offers high image quality. Unfortunately, camera makers all too often offer only complex models with high quality, or simple models with poor performance. Just because someone is not interested in the technicalities of apertures and f-stops does not mean they are not capable of appreciating improved image quality.

 

The HP-15C was reissued at long last!

It is strange no one seems to have picked up the news yet, but HP has reissued the legendary HP-15C in a special “30th anniversary limited edition”, and it became available for purchase last week.

HP-15C Limited Edition

The new HP-15C is not strictly speaking a reissue but a replica, as it does not use the original’s Saturn processor, but instead an emulation thereof running on an ARM CPU. Even emulated, it should be much faster than the original 640 kilohertz processor. I ordered two, and received them today.

As expected, the quality is in line with the current HP-12C, i.e. not as good as the 1980s models in terms of key feel, but still leagues ahead of any competing product. The originals used a special 47-point bonding process to ensure the utmost in rigidity and reliability, I doubt the current model had as much attention paid to detail. It is made in China, obviously, the Corvallis facility is long gone. The slipcover fits very poorly (too tight, and the seams are not trimmed properly) and feels thinner and outright cheap compared to the original. The labels on the keys are accurately positioned, at least, unlike the train wreck that was the HP-12C Platinum. The cheat sheet in the back is a garish black on silver as on the 35S, instead of the original’s silver on black. It also uses two 3V CR2032 batteries instead of the 3 button cells in  the original.

Speed-wise, the Limited Edition integrates the normal distribution nearly instantly, when that test that took 34 seconds on the original.

In short: not as good as the original, but still an excellent calculator for those who prize ergonomics.

Solving Mac freezes due to ocspd crashes

A public service announcement for anyone experiencing the same problem and who may google for a solution.

A couple of days ago my work iMac started experiencing intermittent freezing, and very slow searches in Mail.app. Rebuilding my Spotlight index overnight using sudo mdutil -Eav did not help. I started suspecting problems with recent versions of Chrome, or iTunes 10.5 Beta 7, but quitting those apps did not help either.

After running the Console.app, I noticed the following messages in my logs:

2011-09-14 17:39:20 	com.apple.launchd[1]	(com.apple.ocspd[3197]) Job appears to have crashed: Bus error
2011-09-14 17:39:21 	com.apple.ReportCrash.Root[3201]	2011-09-14 17:39:21.325 ReportCrash[3201:2903] Saved crash report for ocspd[3197] version ??? (???) to /Library/Logs/DiagnosticReports/ocspd_2011-09-14-173921_localhost.crash
2011-09-14 17:50:29 	com.apple.launchd[1]	(com.apple.ocspd[3269]) Job appears to have crashed: Bus error
2011-09-14 17:50:29 	com.apple.ReportCrash.Root[3270]	2011-09-14 17:50:29.964 ReportCrash[3270:2903] Saved crash report for ocspd[3269] version ??? (???) to /Library/Logs/DiagnosticReports/ocspd_2011-09-14-175029_localhost.crash
2011-09-14 17:50:45 	com.apple.launchd[1]	(com.apple.ocspd[3271]) Job appears to have crashed: Bus error
2011-09-14 17:50:45 	com.apple.ReportCrash.Root[3270]	2011-09-14 17:50:45.117 ReportCrash[3270:2807] Saved crash report for ocspd[3271] version ??? (???) to /Library/Logs/DiagnosticReports/ocspd_2011-09-14-175045_localhost.crash

Looking at those crash dumps did not yield very useful information, just some blather like this:

Process:         ocspd [3197]
Path:            /usr/sbin/ocspd
Identifier:      ocspd
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  launchd [1]

Date/Time:       2011-09-14 17:39:19.339 -0700
OS Version:      Mac OS X 10.6.8 (10K549)
Report Version:  6

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: 0x000000000000000a, 0x000000010009b210
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   com.apple.security            	0x00007fff87b59d0b Security::ReadSection::at(unsigned int) const + 25
1   com.apple.security            	0x00007fff87b59172 Security::DbVersion::open() + 62
2   com.apple.security            	0x00007fff87b58cc1 Security::DbVersion::DbVersion(Security::AppleDatabase const&, Security::RefPointer const&) + 179
3   com.apple.security            	0x00007fff87b587ce Security::DbModifier::getDbVersion(bool) + 330
4   com.apple.security            	0x00007fff87b58675 Security::DbModifier::openDatabase() + 33
5   com.apple.security            	0x00007fff87b582b9 Security::Database::_dbOpen(Security::DatabaseSession&, unsigned int, Security::AccessCredentials const*, void const*) + 221
6   com.apple.security            	0x00007fff87b576c1 Security::DatabaseManager::dbOpen(Security::DatabaseSession&, Security::DbName const&, unsigned int, Security::AccessCredentials const*, void const*) + 77
7   com.apple.security            	0x00007fff87b575a3 Security::DatabaseSession::DbOpen(char const*, cssm_net_address const*, unsigned int, Security::AccessCredentials const*, void const*, long&) + 285
8   com.apple.security            	0x00007fff87b6b294 cssm_DbOpen(long, char const*, cssm_net_address const*, unsigned int, cssm_access_credentials const*, void const*, long*) + 108
9   com.apple.security            	0x00007fff87b6ae3a CSSM_DL_DbOpen + 106
10  ocspd                         	0x0000000100006ad9 0x100000000 + 27353
11  ocspd                         	0x0000000100006cab 0x100000000 + 27819
12  ocspd                         	0x0000000100001f68 0x100000000 + 8040
13  ocspd                         	0x00000001000176ed 0x100000000 + 95981
14  ocspd                         	0x000000010001787b 0x100000000 + 96379
15  ocspd                         	0x0000000100017e4f 0x100000000 + 97871
16  ocspd                         	0x0000000100004613 0x100000000 + 17939
17  ocspd                         	0x0000000100001d48 0x100000000 + 7496

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000010008e000  rbx: 0x000000010008e000  rcx: 0x00007fff88f0d47a  rdx: 0x000000000000d210
  rdi: 0x0000000100115478  rsi: 0x000000000000d210  rbp: 0x00007fff5fbfe850  rsp: 0x00007fff5fbfe850
   r8: 0x0000000000000003   r9: 0x0000000000000000  r10: 0x00007fff88f0597a  r11: 0x0000000000000206
  r12: 0x0000000100115478  r13: 0x00007fff5fbfecd0  r14: 0x00007fff5fbfecd0  r15: 0x00007fff5fbfed20
  rip: 0x00007fff87b59d0b  rfl: 0x0000000000010297  cr2: 0x000000010009b210

Binary Images:
       0x100000000 -        0x10003cfef +ocspd ??? (???)  /usr/sbin/ocspd
    0x7fff5fc00000 -     0x7fff5fc3bdef  dyld 132.1 (???)  /usr/lib/dyld
    0x7fff80853000 -     0x7fff80859ff7  com.apple.DiskArbitration 2.3 (2.3)  /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
    0x7fff80baa000 -     0x7fff80c27fef  libstdc++.6.dylib 7.9.0 (compatibility 7.0.0)  /usr/lib/libstdc++.6.dylib
    0x7fff811af000 -     0x7fff81268fff  libsqlite3.dylib 9.6.0 (compatibility 9.0.0)  /usr/lib/libsqlite3.dylib
    0x7fff812e9000 -     0x7fff8161dfef  com.apple.CoreServices.CarbonCore 861.39 (861.39)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
    0x7fff82092000 -     0x7fff82122fff  com.apple.SearchKit 1.3.0 (1.3.0)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit
    0x7fff8229d000 -     0x7fff822c8ff7  libxslt.1.dylib 3.24.0 (compatibility 3.0.0)  /usr/lib/libxslt.1.dylib
    0x7fff82695000 -     0x7fff826d2fff  com.apple.LDAPFramework 2.0 (120.1)  /System/Library/Frameworks/LDAP.framework/Versions/A/LDAP
    0x7fff82954000 -     0x7fff82966fe7  libsasl2.2.dylib 3.15.0 (compatibility 3.0.0)  /usr/lib/libsasl2.2.dylib
    0x7fff82ae1000 -     0x7fff82b81fff  com.apple.LaunchServices 362.3 (362.3)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
    0x7fff82c8f000 -     0x7fff82c9dff7  libkxld.dylib ??? (???)  /usr/lib/system/libkxld.dylib
    0x7fff82d0e000 -     0x7fff82dc4ff7  libobjc.A.dylib 227.0.0 (compatibility 1.0.0)  /usr/lib/libobjc.A.dylib
    0x7fff82dc5000 -     0x7fff82de6fff  libresolv.9.dylib 41.0.0 (compatibility 1.0.0)  /usr/lib/libresolv.9.dylib
    0x7fff8349e000 -     0x7fff834adfff  com.apple.NetFS 3.2.2 (3.2.2)  /System/Library/Frameworks/NetFS.framework/Versions/A/NetFS
    0x7fff83a2b000 -     0x7fff83a41fef  libbsm.0.dylib ??? (???)  /usr/lib/libbsm.0.dylib
    0x7fff83a42000 -     0x7fff83bb9fe7  com.apple.CoreFoundation 6.6.5 (550.43)  /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff83bbe000 -     0x7fff83c92fe7  com.apple.CFNetwork 454.12.4 (454.12.4)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
    0x7fff83c93000 -     0x7fff83cbbfff  com.apple.DictionaryServices 1.1.2 (1.1.2)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices
    0x7fff84905000 -     0x7fff849c2fff  com.apple.CoreServices.OSServices 359.2 (359.2)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices
    0x7fff849d8000 -     0x7fff84a38fe7  com.apple.framework.IOKit 2.0 (???)  /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
    0x7fff850ef000 -     0x7fff8513bfff  libauto.dylib ??? (???)  /usr/lib/libauto.dylib
    0x7fff851a2000 -     0x7fff851ddfff  com.apple.AE 496.5 (496.5)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
    0x7fff8525f000 -     0x7fff85270ff7  libz.1.dylib 1.2.3 (compatibility 1.0.0)  /usr/lib/libz.1.dylib
    0x7fff85b42000 -     0x7fff85b43ff7  com.apple.TrustEvaluationAgent 1.1 (1)  /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent
    0x7fff87921000 -     0x7fff8796bff7  com.apple.Metadata 10.6.3 (507.15)  /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
    0x7fff87b2b000 -     0x7fff87db4ff7  com.apple.security 6.1.2 (55002)  /System/Library/Frameworks/Security.framework/Versions/A/Security
    0x7fff88b46000 -     0x7fff88b4aff7  libmathCommon.A.dylib 315.0.0 (compatibility 1.0.0)  /usr/lib/system/libmathCommon.A.dylib
    0x7fff88f02000 -     0x7fff890c3fef  libSystem.B.dylib 125.2.11 (compatibility 1.0.0)  /usr/lib/libSystem.B.dylib
    0x7fff8921b000 -     0x7fff8933afe7  libcrypto.0.9.8.dylib 0.9.8 (compatibility 0.9.8)  /usr/lib/libcrypto.0.9.8.dylib
    0x7fff8933b000 -     0x7fff89378ff7  libssl.0.9.8.dylib 0.9.8 (compatibility 0.9.8)  /usr/lib/libssl.0.9.8.dylib
    0x7fff89494000 -     0x7fff894d5fff  com.apple.SystemConfiguration 1.10.8 (1.10.2)  /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
    0x7fff8953e000 -     0x7fff896fcfff  libicucore.A.dylib 40.0.0 (compatibility 1.0.0)  /usr/lib/libicucore.A.dylib
    0x7fff89b66000 -     0x7fff89b66ff7  com.apple.CoreServices 44 (44)  /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
    0x7fff8a545000 -     0x7fff8a65cfef  libxml2.2.dylib 10.3.0 (compatibility 10.0.0)  /usr/lib/libxml2.2.dylib
    0x7fffffe00000 -     0x7fffffe01fff  libSystem.B.dylib ??? (???)  /usr/lib/libSystem.B.dylib

Some of the information you get when googling for the error message is misleading, suggesting ocspd is somehow tied to nVidia graphics drivers. It is in fact the system daemon that handles verification and revocation of SSL certificates using the Online Certificate Status Protocol (OCSP), a vital component of the Internet’s security architecture, as evidenced by the recent Diginotar fiasco. Thus presumably it is used throughout Internet apps like Mail.app or Chrome, and the regular crashes (at 3 minute intervals or so) would also freeze any apps that make use of cryptography.

The solution was to delete the the temporary certificate revocation list (CRL) databases ocspd maintains:

sudo rm -rf /private/var/db/crls/*
sudo rm -rf /private/var/db/crls/.fl*

A corrupted database is probably responsible for the repeated crashes I observed, and clearing those solved my problem. You may want to make backups of those files instead of just deleting them.

As usual, I disclaim responsibility for any harm this procedure may do to your computer, or induce it to eat your dog, who ate your homework.

So much for the theory that the Mac “just works”. To paraphrase Churchill, it is the worst operating system, with the exception of all others…

Update (2014-04-30):

Well, at least OCSP works on the Mac. You can test it by trying https://revoked.grc.com/. Google Chrome also rejects this, but being Google, they don’t use the standard and implemented their own seriously flawed CRL distribution mechanism  (they manually blacklisted the GRC test site, for instance). Safari on iOS does not block the site at all (a StackOverflow thread suggests verification is only performed for EV certificates, which is are used in only a tiny minority of SSL/TLS sites).

How prevalent is high-ISO photography?

Low light performance is one of the most important factors I consider when buying a camera. At one point I did an expensive switch from the Canon system to Nikon, when the D3 came out, for its amazing high-ISO performance (I returned to Canon when the 5DmkII came out).

On a popular forum for users of Micro Four Thirds cameras (which struggle beyond ISO 800), a poster recently questioned the rationale for high ISO performance, stating 99% of users will never shoot beyond ISO 800. I quickly looked at my statistics in Lightroom, and found over 54% of the photos I took in 2011 (to date) are at higher than ISO 800.

That begs the question: who is more representative, him or me? Flickr.com publishes statistics on popular camera models, but apparently not on other interesting EXIF metadata. I whipped up a quick and dirty Python script to sample recently uploaded photos from Flickr and collect the ISO speed from their EXIF tags, when available.

Of 3020 photos I sampled, fully 399 were shot at ISO higher than 800, or 13% (the 95% confidence interval is 12% to 14.4%). Thus significantly less than my proportion, but far higher than 1%.

Clearing custom crop aspect ratios in Lightroom

Lightroom’s crop tool allows you to constrain the aspect ratio to a proportion of your choice, e.g. to 4:3, defaulting to the same aspect ratio as the original. The last 5 or so custom crop aspect ratios are saved, but a minor annoyance is you are unable to clear the list.

Python on the Mac and SQLite to the rescue: this simple script  lraspect.zip will reset them. If you use a non-default name for your Lightroom catalog, you will need to edit it. To run it, quit Lightroom and run the script. It will back up your catalog for you just in case.

Needless to say, I cannot be held liable if this script corrupts your catalog or eats your dog (who ate your homework), use at your own risk.

#!/usr/bin/python
import sys, os, sqlite3

# edit this to point to your LR3 catalog if you do not use the default location
lrcat = os.path.expanduser('~/Pictures/Lightroom/Lightroom 3 Catalog.lrcat')

os.system('cp -i "%s" "%s.bak"' % (lrcat, lrcat))
db = sqlite3.connect(lrcat)
c = db.cursor()
c.execute("""select value from Adobe_variablesTable
where name='Adobe_customCropAspects'""")
crops = c.fetchone()[0]
print 'aspect ratios:', crops
c.execute("""update Adobe_variablesTable
set value='{}'
where name='Adobe_customCropAspects'""")
db.commit()
print 'Custom crop aspect ratios reset successfully'

Crime does not pay

Two years after the Staceycide, the spot is still vacant. At a reported rent of $65,000 a month, that adds up to a cool $1.8M loss for the greedy landlords who pushed them out of business.

Stacey’s, RIP

It seems they finally found a new tenant: a CVS pharmacy occupies the premises now.

A waiter for a server

I had to monitor a long-running process on a Solaris server tonight, but didn’t want to stay glued at a computer monitor. A neat trick:

ssh myserver.example.com "pwait 17601"; say "batch done"

You would replace 17601 with the process ID of the job you are waiting for, of course. That way, my Mac connects to the server, waits for the job to complete, then gives me an spoken alert when it is done. I can watch a movie, do chores or whatever during that time. I am sure there are equivalent commands to pwait for Linux.

Deep packet inspection rears it ugly head

Last Friday I started noticing error messages in my production environment. URLs were being mangled, two consecutive characters being replaced by 0x80 and 0x01 or 0x80 and 0x04, causing UTF-8 decode exceptions to be logged, as well as failures for the cryptographic hash function we use to secure our URLs. As a general principle, I take any such unexpected exceptions very seriously and started investigating them, one concern being that some of our custom C extensions to nginx could be responsible for data corruption under heavy load.

I ran snoop (a Solaris utility similar to tcpdump) on one of our production servers, and after combing through 180MB of packet traces with Wireshark, it turned out the data was being corrupted before even hitting our web servers. While it was a relief to find out our own infrastructure was not to blame, I still had to identify the culprit, e.g. whether our hosting provider’s switches, firewalls or load-balancers were to blame.

TCP has built-in checksums, so a malfunctioning switch working at layers 1–3 would not cause this problem, a corrupted packet would be dropped and resent, with a slight hit on performance but no errors. Thus the problem would need to be at a L4 or higher device such as a load balancer.

I added some extra logging and let it run over the weekend. After analyzing the data, it turns out the problem is very circumscribed (76 requests out of hundreds of millions), and all the affected IP addresses come from the same ISP, Singapore Telecom Magix (AS9506). The only plausible explanation is that SingTel is running some sort of deep packet inspection gear, and some of the DPI gateways have corrupt memory or software bugs, that are causing the data flowing through them to get corrupted,

Deep Packet Inspection is a scourge the general public is insufficiently aware of. At a high level, DPI gateways watch over your shoulder as you use the Internet. They decode the data packets passing through them, reconstruct unencrypted HTTP requests (in other words, spy on your browsing history). In their transparent proxy incarnation, they can rewrite the requests or responses. Verizon Wireless uses the technology to resize and recompress images or videos requested by smartphones. Back when I used to work for France Telecom (circa 1996-1999), vendors would regularly approach us to peddle their wares and how they would allow us to price-gouge our customers more effectively. Hardware has progressed dramatically since and a single Xeon processor is capable of inspecting at least 10 Gbps of data.

The whole premise of DPI and other snooping devices is profoundly repugnant to me as a former network engineer, on both moral and technical grounds. Any additional “bump in the wire” slows things down and is yet another potential point of failure, as shown by this incident, but the potential for abuse is the real concern. Not to mince words, the legitimate purposes for the technology, such as fighting cybercrime, are just rationalizations, it was really developed for purposes most people would consider abusive.

When I joined FT, I had to go to a Paris courthouse and swear a solemn oath to defend the privacy of our customers’ communications, and report any infringement of the same. DPI technology originates in spy agencies, and is much beloved of authoritarian governments. China uses the technology, combined with voice recognition, to drop calls at the merest mention of the word “protest”. The Ben Ali regime in Tunisia used it to snoop Facebook users’ authentication cookies. Singapore’s government has a well-demonstrated intolerance of criticism, and who knows what SingTel is doing with their defective gear? Western companies like Cisco were disgracefully eager to sell censorware to dictatorships, but those governments now have homegrown capabilities from the likes of Huawei.

For telco oligopolies, the endgame is to practice perfect price discrimination, e.g. charge you more for packets that carry a voice over IP call or a Netflix video on demand session that compete with the carriers’ own services. Telcos and cablecos cannot be permitted to use their stranglehold over public networks for what is essentially racketeering. Strowger invented the automatic telephone switch because the operator at his manual exchange would divert his calls to one of his competitors, her husband. Telcos, in their monopolistic arrogance, feel a sense of entitlement to all the value the network creates, even when they are not responsible, and want to reverse this. Letting them get away with it, as is consistently the case in the US, is a recipe for long-term economic stagnation.

What can we as the general public do to fight back? The telcos are one of the largest lobbies in Washington, and wireless spectrum auction fees are one of the crutches propping up Western budgets, so help is unlikely to come from the venal legislatures. The most practical option is to start using SSL and DNSSEC for everything. Google now offers an encrypted search option and Facebook has an option to use SSL for the entire session, not just for login.

Update (2012-10-16):

It seems Verizon also uses DPI to build marketing profiles on its users, i.e. categorizes you based on your browsing history and sells you to marketers. You can opt out, but the practice is deeply worrisome.

Hey Apple…

Some improvements you should consider:

  • Sync iPods, iPhones and iPads over WiFi. Cables are so twentieth century. Palm had bluetooth sync working ten years ago, and 802.11n has the same real-world speed as USB. You could then simply extend this to sync the device to the cloud instead of a specific computer.
  • Ditching DVDs to offer an OS reinstall USB flash drive on the new MacBook Airs and Pros is a good idea, but the stick is easy to misplace. How about soldering a read-only USB drive directly onto the motherboard so it can never be lost?
  • When someone enters an address in a Calendar entry on iOS, make it clickable and linked to the Maps app, the way addresses in Contacts are. Copying and pasting them manually is a drag.
  • Stop adding useless frills like “stationery” to Mail.app, and make the default chronological sort order switchable to “most recent on top”.
  • Add HDMI CEC support to the AppleTV. It would be nice to have a HDTV automatically switch over to the AppleTV’s HDMI input when you try to access it. Speaking of which, it would be nice to have an option to disable the audio out on HDMI, e.g. if you have a decent surround sound system connected to it over Toslink and don’t want the TV’s tinny speakers to kick in.