Martin Pool's blog

Mail me RSS Bazaar distcc Canonical is hiring! flickr

The orb of discomfort

jwz says that every program expands until it can read mail. I'd like to propose another: projects eventually reach a point where all the intelligent questions have been asked. Here is my theory.

There are no more bugs, because they've all been removed. (OK, this never quite goes to zero, if only because of change in the underlying platform.)

The program either has all the features that make sense, or it has all the features that can fit into the current architecture. Suggestions of any others are a FAQ.

Therefore, the majority of mailing list questions either reflect user error, or failure to read the documentation.

Preponderance of annoying questions discourages developers and other people, and makes the project move even more slowly.

posted Tue 16 Dec 2003 in /software/management | link

Omelette and a Glass of Wine

Omelette and a Glass of Wine, after Elizabeth David.

posted Tue 16 Dec 2003 in /food | link

SSL sucks

mpt has a great post on why SSL sucks.

To me, SSL security certificates have always seemed particularly stupid usability-wise. As I understand it, the system works like this:

1. Alice trusts Fred.
2. Fred trusts Bob.
3. Bob gets a certificate of trustworthiness from Fred.
4. When Alice visits Bob's page, Bob shows Alice his certificate to demonstrate his trustworthiness.

The problems with this system are as follows:

1. Alice doesn't really trust Fred.
2. Fred doesn't really trust Bob.
3. Getting a certificate is too hard, so Bob doesn't bother.
4. When Bob shows Alice his certificate, Alice isn't paying attention.

Nice example of security having nothing to do with the length of your keypairs. (Or rather, I suppose, proper crypto is a necessary but far from sufficient condition.)

Compare and contrast to SSH host and user certificates: no key-distribution infrastructure by default, although you can build it. In the way they're normally used, all it does is give you some kind of indication that the host you're talking to is the same one that was previously on this address: and the remarkable thing is, much of the time that is an adequate protection. If you want a stronger assurance on the first connection, you can authenticate the host's fingerprint by some other means, such as getting it in signed email. If you're really hardcore you can do things like publishing a signed key in secure DNS. At no point is there a bogus requirement to pay Verisign.

biglumber.com is trying to bootstrap SSL certificates from the GPG web of trust. I think this is a pretty good concept, at least for sites accessed by the kind of nerd who knows what a GPG key is. It might be nice if this could be integrated into the client, rather than requiring eyeball comparison of hex fingerprints.

I suppose you could have a little standalone program that checked certificates on demand, and fed them to Mozilla...

posted Tue 16 Dec 2003 in /software/security | link

(fwd) Best Bug Ever

Garrett points out http://subversion.tigris.org/issues/show_bug.cgi?id=1640

posted Tue 16 Dec 2003 in /software/humour | link

Joel on TAOUP

Joel Spolsky takes a look at esr's The Art of Unix Programming.

Let's look at a small example. The Unix programming culture holds in high esteem programs which can be called from the command line, which take arguments that control every aspect of their behavior, and the output of which can be captured as regularly-formatted, machine readable plain text. Such programs are valued because they can easily be incorporated into other programs or larger software systems by programmers. To take one miniscule example, there is a core value in the Unix culture, which Raymond calls "Silence is Golden," that a program that has done exactly what you told it to do successfully should provide no output whatsoever. It doesn't matter if you've just typed a 300 character command line to create a file system, or built and installed a complicated piece of software, or sent a manned rocket to the moon. If it succeeds, the accepted thing to do is simply output nothing. The user will infer from the next command prompt that everything must be OK.[...]

So you get these religious arguments. Unix is better because you can debug into libraries. Windows is better because Aunt Madge gets some confirmation that her email was actually sent. Actually, one is not better than another, they simply have different values: in Unix making things better for other programmers is a core value and in Windows making things better for Aunt Madge is a core value.[...]

Raymond does attempt to compare and contrast Unix to other operating systems, and this is really the weakest part of an otherwise excellent book, because he really doesn't know what he's talking about. Whenever he opens his mouth about Windows he tends to show that his knowledge of Windows programming comes mostly from reading newspapers, not from actual Windows programming. That's OK; he's not a Windows programmer; we'll forgive that. As is typical from someone with a deep knowledge of one culture, he knows what his culture values but doesn't quite notice the distinction between parts of his culture which are universal (killing old ladies, programs which crash: always bad) and parts of the culture which only apply when you're programming for programmers (eating raw fish, command line arguments: depends on audience).

There are too many monocultural programmers who, like the typical American kid who never left St. Paul, Minnesota, can't quite tell the difference between a cultural value and a core human value. [...]

Joel often seems to have basically interesting ideas and then to mix in a lot of chaff that's either obvious or oversimplified. Having made a basically good point about the cultural differences between Windows and Unix, he seems to assume that the cultures are entirely fixed and unchanging, and Unix will never get a good GUI. This is more or less like self-righteous American car companies in the 60s assuming that Japanese cars are will always be cheap, unreliable and nasty.

Honda/Acura NSX 2005 model preview

You need to distinguish the core values (solidity, reuse) from their accidental expressions (preference for command-line tools, terseness.) Sadly neither Joel nor ESR do this very well at the moment. It is still an open question how to make graphical components that can be recombined as fluidly as unix pipes, but perhaps we will know in ten years.

As an example, some number of free software programs are developing excellent log/debug/trace frameworks. distcc is one of them: many releases have discovered bugs either in distcc, gcc or the kernel that could not have been reproduced on my machine. The trace mechanism turned on by DISTCC_VERBOSE=1 is good enough that in almost every case I've been able to work out what was wrong and prepare a fix by looking at the log file. (OK, in some difficult cases looking at it for a long time.)

This is a small departure from traditional Unix terseness but I think a valuable one. I don't claim to be the first person to do things this way but I do think it's an evolution based on the core values of wanting to help a technical user and being good to developers. Contrast it to the classic Windows system log message: "The operation failed because: Success."

Does it help Joel's auntie? No, but then she probably doesn't need a distributed compiler anyhow. Does it help, say, Joel's niece who's getting into Linux for the first time by installing Gentoo? Yes, it just might. If you called it "progressive disclosure" you'd start to sound like a UI designer, or enough to fool a bearded unix-head anyhow.

I agree that it is a real weakness of esr's book that he doesn't really understand any of the other systems he talks about, or why they might be good. The Practice of Programming does much better in this regard.

Tim has some thoughts on this too.

posted Tue 16 Dec 2003 in /books/taoup | link

Information wants to be leaky

Plenty of thought, from William Gibson to Richard Stallman to Brad Cox has gone into two contradictory properties of information:

1. Information can be valuable.
2. Information wants to be free.

"Wants to be free" means that it tends to leak out, despite your best efforts. If it's a secret, it will be told. Even if you keep it secret, people might infer the information from your actions. If it's proprietary, it'll be copied. If it's protected, it will be cracked.

There's something a lot like thermodynamics going on here. Everything eventually collapses to waste heat. It's all about what work you can extract from the process while that's happening.

posted Tue 16 Dec 2003 in /software/freedom | link

Free software beats superdistribution

[work in progress; comments solicited]

Back in the mid 90s, one would regularly read about the Future of Software being something like this: when you want to build an application, you find, licence and assemble a whole bunch of components from individual software vendors. You plug them together, according to well-defined interfaces, perhaps write a few high-level components of your own, and deliver the whole assembly the the customer.

If I recall correctly, Brad Cox was one of the leading proponents of this idea. An early paper was Superdistribution:

Stop selling software. Give it away. Get paid for its use. Meterware is so logical it could be the foundation of the new, networked economy.[...]

Treating ease-of-replication as an asset rather than a liability, superdistribution actively encourages free distribution of information-age goods via any distribution mechanism imaginable. It invites users to download superdistribution software from networks, to give it away to their friends, or to send it as junk mail to people they've never met.

Why this generosity? Because the software is actually "meterware." It has strings attached, whose effect is to decouple revenue collection from the way the software was distributed. Superdistribution software contains embedded instructions that make it useless except on machines that are equipped for this new kind of revenue collection.

Superdistribution-equipped computers are otherwise quite ordinary. They run ordinary pay-by-copy software just fine, but they have additional capabilities that only superdistribution software uses. In Mori's prototype, these extra services are provided by a silicon chip that plugs into a Macintosh coprocessor slot. The hardware is surprisingly uncomplicated (its main complexities being tamper-proofing, not base functionality), and far less complicated than hardware that the computer industry has been routinely building for decades.

Electronic objects intended for superdistribution invoke this hardware, which provides instructions. These instructions check that revenue-collection hardware is present, prior usage reports have been uploaded, and prior usage fees have been paid. They also keep track of how many times they have been invoked, storing the resulting usage information temporarily in a tamper-proof persistent RAM. Periodically (say monthly) this usage information is uploaded to an administrative organization for billing, using encryption technology to discourage tampering and to protect the secrecy of the metered information. (Think of your utility bill.)

To get there, we would have needed a few things. Firstly, new technologies that allowed programs to be reliably assembled at run time from component libraries. This is more than just dynamic linking: you need to think about finding the right components, versioning, making state persist, and so on. Given the processor speeds of the time, they needed to be (mostly) native binaries, rather than some kind of bytecode. The technologies here include things like OS/2's SOM, COM, Objective C, and so on. Most of these use a kind of looser linking than we normally have with C, in an attempt to allow the libraries to evolve without breaking the applications.

Secondly, you need an economic model to support it. If people are going to publish and sell small component libraries you need a way of licensing them and a way of arranging payment. If the components are going to be very fine-grained, this converges on almost a kind of micro-payment system, particularly if you want per-use licensing.

I find this really fascinating because it's so close, and yet so far, from what we have in free operating systems. A free Unix system is built from components written by many individual and independent programmers, which are in turn assembled by other people. Possibly there are several layers of assembly: subsystem maintainers knit up all the parts of the USB Storage system, which is in turn assimilated by somebody else into the USB system, which Linus merges into the kernel, which somebody else patches for optimal desktop performance, and that is in turn only one part of the distribution.

Despite the similarities, free software seems to be proving that the Superdistribution model is fundamentally wrong.

Putting code into binary, closed libraries, just doesn't work very well. At least, it works far less well than open source libraries. It ignores some fundamental properties of software: that defining interfaces and documenting behavior is as hard or harder than implementing the behavior. Even if you never change the source, having read access to it as the ultimate documentation is incredibly useful. Eric Raymond has a good discussion of the problems with binary libraries in his book The Art of Unix Programming.

Even assuming the library has good test coverage by the publisher, it will always hit situations in the real world that weren't contemplated, especially when it's put into a complex ecosystem of other components. Testing and documentation can't cover every conceivable use.

As if binary libraries weren't bad enough, Cox suggests that the components need to be subject to DRM to protect the revenue stream of the author. We need new hardware, a la Paladium. This makes sense within his economic model, but consider how horrible it would be to use. “Protecting” the code would probably require that it be kept shrouded and encrypted: not only is there no source for the application developer, but probably not even any ability to use a low-level debugger. And just think about the bugs DRM can introduce. Pay per use requires some amount of DRM, but DRM from the user's point of view makes the components strictly worse. At most it is tolerable; at worst it blocks the project entirely.

As an example of strong DRM: I'm quoting some of Cox's writing above. How should I arrange payment for this quotation? How should I arrange for you the reader to pay him for reading 10% of his essay? Why should I give up emacs and Mozilla in favor of tools that restrict my ability to copy and paste? (I think Xanadu was trying to charge by the byte, and who uses Xanadu?)

So: Stop selling software. Give it away. Work out some other way to make money, because pretending it's not replicable is not it. Profit.

posted Tue 16 Dec 2003 in /software/freedom | link

Copyright (C) 1999-2007 Martin Pool.