Dembski's Law of Conservation of Information

William Dembski provides on p. 8 of the online paper The Conservation of Information- Measuring the Cost of Successful search a definition of his Law of Conservation of Information in the form of a theorem.

However, just to warm us up, we'll do some preliminary definitions first.

Let A and B be two events with probibilty p of A to occur and probability q of B to occur.

The (self-)information or surprisal value of A is defined as

I(A) = − log₂ p

The added information of B to A is defined as

I₊(A : B) = I(A) − I(B) = − log₂ p + log₂ q = log₂ q/p.

Two immediate consequences of the latter definition is

I₊(A : A) = − log₂ p + log₂ p = 0; and

I₊(A : B) = log₂ q/p ≤ log₂ 1/p = − log₂ p = I(A).

The last line implies that the added information of B to A cannot exceed the information of A alone.

What should here be understood is that the information of A is exactly a measure of the surprise of the occurence of A, the smaller the probability, the larger the surprise. Assume you live somewhere, where the probability of rain on any one day is 90%, and the always reliable weather forecast says that it will rain the next day. The weather forecast doesn't really give you much information, since you would anyway expect it to rain, so not much surprise in that case. If instead the weather forecast had said that it would be a clear and sunny day the next day, that would have given you more information, since it would be contrary to expectation, therefore a greater surprise.

What the definition of added information says is basically that you know, what you know, and the more you know, the less will there be to learn.

Dembski's motivation for introducing added information (cf. p. 3) is as follows. During a search, the more samples are taken, the higher will be the probability of a success; but a higher probability corresponds with a lower self-information value, and our intuition says that the more samples taken, the more information generated.

A more efficient search will generate more information per sample; but the problem then is, how to figure out which search is more efficient than a random search, which Dembski uses as the base search strategy.

We can now give Dembski's definition of the Law of Conservation of Information:

Theorem (Conservation of Information). Suppose S and T are searches over a given search space, S being a random search with probability p success in a single query and T being a nonrandom search with probability 1 of success in a single query. Suppose further that U and V are searches over the space of searches in which S and T reside so that U on average locates a search of the original space that with probability no more than p successfully searches the original space and that T with probability 1 locates a search of the original space what with probability 1 successfully searches the original space. Then the information that V adds to U is at least as great as the information that T adds to S, i.e.,

I₊(U : V) ≥ I₊(S : T).

Moreover, by a suitable choice of U, this inequality becomes an equality.

Here U and V are meta-searches; that is, searches for searches. What the theorem therefore says is that is requires at least as much information to figure out how to do a search as to actually do the search itself.

As an illustration, Dembski uses a search for a treasure on an island (cf. p. 6). It may be prohibitive to do a random search for the treasure; but you have a treasure map, so no problem. However, where did you get the treasure map from? You first needed to do a search for that from among all treasure maps. This may have been an even more involved search, which leads to an infinite regress.

In short, information comes at a price, and that price is at least the same amount of information.

Dembski writes p. 9:

According to Douglas Robertson (1999), the defining feature of intelligence is its ability to create information. Yet, if an act of intelligence created the information, where did this intelligence come from? Was information in turn required to create it? Very quickly this line of questioning pushes one to an ultimate intelligence that creates all information and yet is created by none (see Dembski 2004: ch. 19, titled “Information ex Nihilo”).

Here 'Douglas Robertson (1999)' refers to an article "Algorithmic Information Theory, Free Will, and the Turing Test" by Douglas Robertson, and 'Dembski 2004' refers to Dembski's own book The Design Revolution.

The point being that intelligence creates information, which means that intelligence applies some search strategy, and from where does intelligence know about that search strategy? This knowledge is itself information, so there must be an ultimate intelligence.

If not, then, Dembski continues:

On the other hand, if the information is the mechanical outworking of preexisting information, the Conservation of Information Theorem suggests that this preexisting information was at least as great in the past as it is now (this being the information that allows the present search to be successful). But then how do we make sense of the fact (if it is a fact) that the information in the universe was less in the past than it is now? Indeed, our present universe, with everything from star systems to living forms, seems far more information-rich than the universe at the moment of the Big Bang.

The obvious question here is, how do we measure information? If there is more information in the universe today than at the moment of the Big Bang, assuming that to have happened, then we should be able to figure out, what happened all the way back to the Big Bang. The current universe certainly may exhibit more variation than the very early universe; that is, there are more different things to know something about, but is that more information?

All in all, Dembski's main point is that human intelligence might have another source than evolution, which he considers to be a search strategy. Since evolution to produce intelligence must itself have been even more intelligent or guided by something even more intelligent, evolution cannot be a random search. And if evolution is a random search, it cannot have produced intelligence, which must therefore have another source. He does not write this directly; but it's what he is hinting at.

Now, as for Dembski's Law of Conservation of Information, it ignores that we don't always start out with finding an optimal search strategy. Occasional search strategies are made up along the way based on experience. Modern dictionaries are alphabetically ordered, which makes it simple to use relatively efficient searches based on the spelling of a word; in antiquity it was more common to order words by decreasing importance, such that those that corresponded to a more important concept were at the top. However, in antiquity, scrolls were used, so this ordering simply meant that you typically only needed to unscroll a small segment of the scroll. With a book, you can open it anywhere at the same cost. In that way ordering of information and search strategies depend on technology. We simply don't start out with determining the optimal search strategy and then turn everything else around after that.

So, Dembski's theorem may be correct mathematically seen, but he may have wasted his time searching for the wrong solution to the wrong problem.