In No Free Lunch, Dembski proposes a 4th Law of Thermodynamics which he equates with the Law of Conservation of Information. He also discusses the relationship between his proposed 4th Law and the 2nd Law.

ME: I would like this 4th law to be even broader including the "loose" information of Shannon-Weaver entropy. "Loose" information would be defined as information that has not become "fixed" like a book, a video tape or a CD. Therefore this information, the spoken word, a radio signal, etc. does degrade over time as it is spread, in accordance with the Second Law.

It remains somewhat controversial as the IDists debate information degradation whether or not information is tied directly into the Second Law of Thermodynamics. One body of thought is that it most definitely does tie directly into thermodynamics, and I am of this body of opinion. Of course, Ludwig Boltzmann described entropy as: "Gain in information is loss in entropy."

And even though Shannon called his theory "the daughter of thermodynamics." I would still encourage that degradation of information become incorporated into thermodynamics more formally. The 4th Law is good science.

[ 25 February 2002: Message edited by: Jep ]

(I originally found the quote on the ARN discussion board http://www.arn.org/ubb/Forum1/HTML/001863.html):

quote:

---

Every process, event, happening -call it what you will; in a word, everything that is going on in Nature means an increase of the entropy of the part of the world where it is going on. Thus a living organism continually increases its entropy -or, as you may say, produces positive entropy -and thus tends to approach the dangerous state of maximum entropy, which is of death. It can only keep aloof from it, i.e. alive, by continually drawing from its environment negative entropy -which is something very positive as we shall immediately see. What an organism feeds upon is negative entropy. Or, to put it less paradoxically, the essential thing in metabolism is that the organism succeeds in freeing itself from all the entropy it cannot help producing while alive.

---

quote:

---

Thus the device by which an organism maintains itself stationary at a fairly high level of he orderliness ( = fairly low level of entropy) really consists continually sucking orderliness from its environment.

---

But how does an organism "suck orderliness" from its environment? It seems that energy is required; I've often heard the objection to evolutionary theory that it violates the 2nd law of thermodynamics, along with the standard response that entropy can decrease as long as energy is input into the system. But this answer misses something. After all, an explosion is a huge release of energy, but it tends to lead to more disorder, rather than more order. Likewise, burning organic molecules will tend to produce high-entropy, disordered goo (have you ever cleaned out a barbecue grill?). Something else is required. Somehow, the flow of energy must be properly channeled or constrained to achieve the reduction of entropy. In biosystems, these constraints or channels through which energy flows are embodied in enzymes that guide and direct energy flows. And those enzymes, in turn, come from DNA. Now, it's interesting that DNA contains vast amounts of information, and information has its own form of entropy. Furthermore, it appears that informational (negative) entropy is being used to produce enzymes that further channel the flow of energy to maintain low-entropy conditions within the cell. Intriguingly, this suggests that perhaps we can speak of entropy flows much as we speak of energy and information flows. Perhaps it is this flow of entropy, orchestrated and guided by information, that is the key element of life itself. Somehow, the information contained in DNA is finding a way to express itself and directly impact the physical world through living beings. The universe is fundamentally responsive to the information that ramifies through it.

What of thermodynamic entropy and informational entropy? They both have the same mathematical form and it makes intuitive sense that they would be, in some way, able to influence each other.

Thermodynamic entropy:

S = k ln W
or
S = -k ln P

where
k = Boltzmann's constant
W = equiprobable, equal-energy microstates (ways the system can be arranged) for a given macrostate, and
P = the probability of any given microstate.

Informational entropy:

H = -K log2 P

where
P = the probability of the signal given the reference class of possible signals that could have been sent.
K = a constant, usually assumed to be unity.

Alternately, we can represent informational entropy as

H = K log2 N

where
N = number of possibilities from which the signal in question was selected.

Thus, the mathematical forms of informational and thermodynamic entropies are equivalent. Furthermore, the idea of thermodynamic entropy as a measure of the number of possible states from which the existing state was selected is remarkably similar with the informational entropy as a measure of the number of possible bit strings ruled out when the signal was sent.

If the law of conservation of information holds, it seems that perhaps there is also a corresponding law of conservation of negative entropy. According to this law, low entropy can only arise from pre-existing low entropy. That pre-existing low entropy can be thermodynamic or informational, and can flow through the environment just as information and energy do.

Furthermore, it seems that some concept of "signal agreement" or "meaning" is the mechanism by which informational low entropy gets transformed into thermodynamic low entropy. The genes "mean" enzymes and other proteins involved in ordering the flow of energy and matter through the cell. This abstract correspondence, signal agreement, between the DNA and the proteins it encodes, is the essential transforming step by which the two types of low entropy interconvert.

There are several interesting questions that arise at this point. For instance, can we determine the amount of entropy being transferred? Which embodies more order (negative entropy), the DNA, the enzymes produced by that DNA, or the metabolic pathways regulated by those enzymes? It shouldn't be hard to measure them both and see whether negative entropy is conserved. I would like to hear from physicists on these ideas, especially on whether I've correctly understood and described thermodynamic entropy.

John Bracht

[ 25 February 2002: Message edited by: John Bracht ]

I am also interested in the application of information theory to evolutionary biology.
But the problems and paradoxes involved in reconciling the two theories has resulted in much lost sleep, headaches, fits of despair, crying jags, receding hairline, the heartbreak of psoriasis, and occasionally original experiments in the aerodynamic properties of computer peripherals. (A keyboard flies further than a monitor, but not as far as a mouse.)
Let me throw out a few thoughts and ideas and I’ll be brief (as is possible for me):

A low order model based principally upon a statistical analysis of the DNA code doesn’t seem to quite capture what it is I’m looking for. Am I going about it backwards?--repeating the same presumptive error made by the RNA Tie Club and the Cybernetics Club (formerly the Teleology Club) in the 1950’s. We only succeeded in decrypting the code by attacking the products of translation and not DNA itself. We know that no non-trivial measure of bioinformation can be derived from the analysis of the DNA code alone, because information is always “meaningfully” mutual information, not “self-information.” I wonder if we are still possessed of only part of the decryption key?! Originally the code was decrypted, at least in part, by analyzing the products of translation and extrapolating back to the genome. But having found the translation table (DNA>amino acids) it was decided that the code had been cracked and the whole program was largely abandoned!

I think everyone agrees that information is not merely a number we assign to a set and instances drawn from it. Information has a function. In biology its function is control. Any measure of information assigned on the basis of the translation table is going to be trivial. The meaninglessness of such measures is obvious: the utility of information theory to biologists is virtually nil. That’s because such measures are as arbitrary and redundant as assigning numbers to the letters of the alphabet. Translating DNA into amino acids is not like translating War and Peace from Russian into English. Instead it is like translating measurements made by a sensor into controls by an actuator.

There are many kinds of “information.” If I remember correctly there are about a half-dozen in Shannon’s theory. I think that in the genome (proteome) we are dealing with a special form of information: we are dealing with algorithmic information. Shannon however never mentioned “algorithmic” information. But a closer examination of the original paper is warranted. In every instance, so to speak, information is defined and measured exclusively by operations performed on information. (Information is a tautology?!) Performing an operation on information is applying algorithmic information to (any other form of) information. I think this is very fundamental to our understanding of information, but we lose this because of Shannon’s choice of the simplest possible (stochastic) process model. The model choice was necessitated by the very basic formal treatment. We are not limited to it by any means. We have to make the correct choice of model to even begin to derive accurate measures of bioinformation.

The correct model is not the “translation” model. I believe it is more like a model of control. The theory (Shannon’s) is really about the control of information, because information is defined and measured operationally. You’ll notice that “control,” the function of information, is largely eliminated from Shannon’s theory because he chooses an arbitrary stochastic model. Simply by choosing a higher order model, we realize that information is wholly contained in the selection. Ask yourself some very basic questions: Information is conventionally defined as a reduction in uncertainty. Uncertainty w.r.t. what? W.r.t. an instance selected from a set. Well, where did the set come from and how are the instances selected? Suffice it to say that information is defined and measured wholly on the basis of the selection of the set and the selection of instances from the set. So the information is actually contained in the selection and selection is the most elementary act of control.

In biology, information is not in the form of plaintexts, it is exclusively (in the proteome) in the form of algorithms that control processes. A more accurate measure of bioinformation must account for the operations performed upon information as well as the information contained in the operations themselves.

Now, we must understand this intuitively, because we know that bioinformation does something. If I thought that the information in a skyscraper could be measured by simply reading the blueprints I would be wrong. The total information is not contained in the blueprints. It is contained in the blueprints and also in all the information used and all the energy expended (and net reduction in entropy) in building the skyscraper. Now, to my knowledge there is no measure of information that captures the dynamics. We traditionally measure information statistically and not dynamical. We measure what “it is” and not what “it does.” E.g., a blueprint tells the electrician where to lay her cable, but it doesn’t tell her how to lay the cable. She supplies this information, not the blueprint. A blueprint doesn’t even much suffice for the minimal expression of the information in a skyscraper. We have almost forgotten that information is a measure closely related to work energy/entropy as in classical thermodynamics. We’ve largely focused our attention on information corresponding to the configurational entropy of statistical mechanics, or the logical/syntactical entropy of language. We’re missing the dynamics.

My interest was piqued by what John Bracht has written (in this and another topic) because I was familiar with models of control in metabolic control analysis theory that adopt the concepts of signals and channels, etc. I was also aware of tentative applications of information theory to control technologies, in which a control system is modeled as a communication system. Both models can be easily synthesized and both have obvious relevance to biology.

To make a short story long, what is lacking is the other half of the decryption key. I believe it is a control term(s), along with the relevant parameters, that may be found in metabolic control theory. One such candidate term is the Hill coefficient that measures the “cooperativity” of the components of the distributed control networks found in metabolics. It serves as a convenient measure of “function” and “control” and is easily translated into information. The missing term quantifies the dynamics of information in biology. (An “information added” function?) Taking this measure and extrapolating backwards through the network to the genome, just as was done when the translation table was found, will yield a “truer” measure of the information in the genome. It will tell us about how much information is in the blueprint, in the building, and in the building of the building.

I would like to add to this discussion a link to a paper that should be of interest to anyone who is concerned with relationships between 2nd Law, entropy, information, and biology. It appears that the concept of 2nd Law need to be made more general before it is applicable to biology. The author of the paper, E. T. Jaynes, has done some pioneering work in this field.

There is more to the paper than the 2nd Law, mostly dealing with Bayesian probability theory, you can skip these early parts although they are interesting, too. Here is the link, read with time: Clearing Up Mysteries

[ 27 February 2002: Message edited by: Mika Vallittu ]