|
Author
|
Topic: On information and complexity
|
Art
Member
Member # 179
|
posted 10. March 2002 13:42
Consider the tornadic thunderstorm. It consists of a number of integrated and essential components, all of which are needed to produce and maintain the tornado. The ground and upper-air windstreams (which must be oriented in precise manners), the precisely-functioning updraft, the supercell itself (which consists of more parts than I can possibly list), and the funnel cloud. By most accounts (there will always be dissent ![[Smile]](smile.gif) ), an IC system.
Can we speak about the information content of a tornadic thunderstorm? I believe so. Recall that the informational content of genomes is usually estimated by "calculating" that fraction of all possible sequences (nominally, amino acid sequences) that can satisfy a particular specification. We can use a similar strategy to guesstimate the "information" carried by water vapor molecules in a storm. The hard part is deciding how few of all of the possible states that are available to a particular water molecule are actually "used" in a storm. Now, one can count up all possible positions in the storm, interactions with all possible partners, etc., etc., and realize that the number is probably rather small. But, for the sake of argument, let’s pick an arbitrarily large number – let’s propose that only 1 in 10^30 states of any given water molecule is excluded in a storm.
Starting there, we need only count the number of water vapor molecules in a storm and estimate the "probability" that the arrangement found in a storm would occur. If we arbitrarily think in simple terms - a storm that is 5x5x7 miles in size, a temperature of 30 degrees C, a partial pressure for water vapor of about 32 mm Hg, an overall atmospheric pressure of 1 atm - then the number becomes (roughly) 1x10^-30 raised to the number of water vapor molecules in the storm (which is about10^36). Which in turn is about 10^-10^6 (that's 1 divided by 1 million!). (For comparison, recall Hoyle's number of 10^-40,000 as an estimate of the probability of the proteins in a simple cell arising by chance.)
Hopefully, if I’ve been clear, there should be the beginnings of a paradox here. This comes from the “universal probability bound” that was set forth by Dembski – roughly 10^-150. The reflexive interpretation of the preceding is that the information content of a tornado, which obviously forms by chance and far too often to be considered as improbable, exceeds Dembski’s limit, thereby indicating a fundamental problem somewhere. But, in exploring this seeming paradox, I would suggest that useful things can be learned about the application of Dembski’s ideas to nature. I’ll offer two in the rest of this post.
A. First, I need to remind myself just what the “universal probability bound” is. It was not derived from information-based computations, but rather by estimating the likelihood of possible occurrences in the universe. The preceding suggests (to me, at least) that it may not be appropriate to equate the information content of a system with the probability of occurrence of an event. This probability needs to be estimated by other means, and depends on much more than the informational changes associated with an event.
As a simpler example, consider the information content of T-urf13. In NFL, Dembski argues that the information content of this protein, while large, is in and of itself below the “limit” that one gets if we equate the “universal probability bound” with informational bits. However, the probability of this protein arising by chance in a population of maize plants is likely far, far greater than the informational estimate would suggest. In contrast, the probability of finding a milligram of T-urf13 on, say, the dark side of the moon is far, far less than the “universal probability bound”. In other words, circumstance and pathway, are of paramount importance when thinking of probability, while inherent information content is almost irrelevant.
Put another way, an information content of 3 million bits (roughly that of a tornado, if one grants my back-of-the-napkin arithmatic), while in excess of the limit one would get if one equates the “universal probability bound” with information, is not really complex, since an event bearing the origination of such a quantity of information can and does occur “by chance”, and frequently. IOW, complexity is not determined by information content, but by other considerations.
B. Which brings me to a second point. Usually, (in my reading, at least), information content is reflective of the informational entropy of a system. Entropy, in turn, is usually taken as a state variable – the informational entropy of, say, a protein is independent of the pathway by which the protein originated. The preceding indicates that complexity does not share this property. It follows (at least to me) that the property “complex specified information” (CSI) is not a state variable, and thus should not be rigorously equated with information per se. I would suggest that a better analogy to be used here is that of thermodynamic work. Work is a property that is pathway-dependent – the amount of work obtained in going from state A to state B is determined as much by pathway as the inherent thermodynamic properties of the initial and final states (although the poises of the state variables do affect the work that can be done). It seems (naively, to be sure) that CSI would be better defined in terms of some sort of informational “work”, rather than inherent information content. (This would take into account the pathway dependence of the assignment of complexity, as indicated in the preceding.)
I’d like to elaborate on the concept of informational work, but I am woefully ill-equipped to explore the idea in much depth. Perhaps other participants here can fill in some of the blanks.
IP: Logged
|
|
Jack Foster
Member
Member # 79
|
posted 11. March 2002 23:59
Great post, Art!
I want to commend the moderator on this one. This beginning has way too much potential to allow thread sidetracking. Is informational work a subset of specificity? I would love to see Dr. Dembski take a shot at this issue.
yours,
jazzraptor (at arn)
IP: Logged
|
|
Cre8ionist
Member
Member # 140
|
posted 12. March 2002 07:39
I'm sorry if people thought I was side tracking. I was trying to address Art's use of Dembski's universal probability bound. It seems to me that it only applies if the event/object in question can answer yes to the following three questions, is it contingent, is it complex, and is it specified?
I was simply pointing out that a tornado seems to fail in two of the three answers. It is not contingent, i.e., it's the result of physical necessity, and it is not specified, e.g., any water molecule will do, in any position, (unlike a sentence which is made up of different letters in specified positions). BTW, I'd be more than delighted to actually have the point addressed, and be corrected where I'm wrong. I don't want to hold on to false ideas about such an important issue.
I'd also be happy if Dembski would weigh in, but having read Dembski, it's hard to imagine a very different response. Simply compare the specificity of the twister created when you pull the plug on your tub, to the specificity of the twister created that codes for life and I think you'll be able to see that the difference in information, is in the quality(specificity) of the information.
Lastly, and please forgive me for even feeling the need to respond to the mod, IC in the world, (apart from the neo-Darwinian mechanism), is pretty much a given, only when it's contrasted with the aforementioned mechanism is it even worth mentioning. What would be the point of mentioning the IC of a car, for instance? So I ask, what's the point of mentioning the possible ICness of a tornado?..............Cre8
[ 13 March 2002, 08:09: Message edited by: Cre8ionist ]
IP: Logged
|
|
Janitor@MIT
Member
Member # 125
|
posted 12. March 2002 13:01
How is a tornado not like a protein? The answer is, they are the same and they are different!
Does a tornado contain information? No doubt. What doesn’t? When a meteorologist observes and measures a tornado he doesn’t begin by counting molecules and determining the state of each molecule. The measures he applies are decidedly less precise than Art’s measurements. Instead of determining precisely the velocity of each molecule he averages the velocity of all molecules, and seems to derive a meaningful measure of information, i.e., wind speed.
There are of course practical considerations here: before Art determines the states of any appreciable number of molecules the tornado will be gone. Nor is it obvious that Art’s far more precise measurement returns an equal increase in precision of prediction of the weather. So Art must show me why his method of deriving an “information content” for the tornado is better than that used by, say, NCAR?
But all this is to say that an important principle (actually several important principles) of information theory is missed here, e.g., averaging. Doing it Art’s way I will eventually begin to partition the states into bins, reflecting some narrow range of states, arbitrarily divvying up the ranges, etc. In other words I’m going to find myself averaging over states anyway, so why not simply begin that way, using such “bulk” measures as temperature, pressure, velocity, etc.? These will certainly yield meaningful meteorological information. (With far less investment in labor.)
Is there a way to make this averaging less “arbitrary”? Of course. One way is simply by experience; by experimentation I can determine how precise my measurements of any particular phenomenon has to be in order to derive sufficient information from the measure for practical purposes, such as predicting the weather. (David Wolf’s doc thesis available online is about this very problem.) But in many cases the “averaging” or any measure of information is in some sense “given” by what is being measured. E.g., if I have a string of letters, rather than a tornado, I might induce a pattern in the string that permits averaging, or I might induce a lack of pattern that precludes it. Often I will find both patterns, which can be averaged over, and also no patterns, which cannot be averaged. I will assign a measure of “information” accordingly.
All I’ve said is that the observer selects the appropriate measures of information depending on what the observer purposes for the measures. (And also based on his curious prior knowledge that important information is contained in patterns or the lack thereof.) Does the tornado contain information? Yes, but in a way that appears to be completely observer-dependent. So Art raises an important and confounding question about the observer-dependency of information.
Shannon’s theory is unusual for a “physical” theory, in that is solely based on the observer. If we eliminate the source and destination, we still have an observer, the communications system itself. But having eliminated the source and destination, the communications system now operates in a very different way! Very peculiar! Think about it. (A similar paradox is discovered in the quantum of theory of measurement.)
(I hope this helps Art. If he protests that I’ve begged the question, I’ll concede that I have answered his question with a question(s). But I think my question is more fundamental and also illustrates an important principle that Art has neglected.)
I think I’ve exceeded my allotted drive space here, but I wanted to add, that I think, as Art does, that information theory requires an extension beyond statistics into dynamics, particularly wrt to biology. As the paradox above indicates, I believe, we need a physical grounding of the theory, making it independent of our selection of the method of analysis/measurement. We need to know what information does, not merely what it is. Does that make sense?
IP: Logged
|
|
Mark Elkington
Member
Member # 120
|
posted 12. March 2002 23:15
Art,
Interesting post. I would have thought though that a tornado is not, by observation, IC.
Tornado formation has been more or less observed from the initial "low complexity" condition of clear blue skies, where no/few component parts exist. These parts form and interconnect gradually and incrementally, i.e. we notice the weather turn grey, black, and then tornadic.
The existance of this stepwise pathway seems to rule out IC, if I understand the definition of IC correctly.
I would tend to agree with Cre8ionist in characterising a tornado as being "the result of physical necessity", similar to crystal growth.
[ 12 March 2002, 23:17: Message edited by: Mark Elkington ]
IP: Logged
|
|
Kirk Durston
Member
Member # 174
|
posted 14. March 2002 12:00
Shannon's initial eqn for information does not distinguish between 'meaningful' or 'functional' information and meaningless or functionless configurations. To use Shannon's eqn to calculate the amount of functional information, one must take the difference between the Shannon entropy of a physical system without constraints, and a physical system with constraints added to perform the desired function (i.e., tornado). Assuming all states are equally probable, the eqn for functional information becomes:
I = -log(Nf/N)
where
Nf = number of functional configurations and,
N = total number of configurations, functional or non-functional.
As for tornados, they occur with constraints on the physical system that are easily attainable within the physical system itself. Thus the amount of *functional* information in a tornado will be close to zero.
As for functional proteins, this is not the case.
IP: Logged
|
|
Art
Member
Member # 179
|
posted 16. March 2002 00:16
Hi all,
Thanks for the interesting feedback. I’ll try here to respond to some of the isues, in an order that makes sense to no one but me.
Mark, you said:
quote:
“Tornado formation has been more or less observed from the initial "low complexity" condition of clear blue skies, where no/few component parts exist. These parts form and interconnect gradually and incrementally, i.e. we notice the weather turn grey, black, and then tornadic.
The existance of this stepwise pathway seems to rule out IC, if I understand the definition of IC correctly.“
I guess my problem with this POV is that IC is being defined here as something that cannot arise “by physical necessity”, or out of a state of initial low complexity. In my opinion, IC “things” are not constrained in these ways. Indeed, I would argue that to define IC as you are doing is to create a sort of circular argument that really limits the concept as a useful tool. In any case, I think that, if we focus on the parts, and not how they came about, tornadoes are indeed IC. Maybe these issues could be explored in more detail in the thread that deals with the applicability of IC to things other than machines.
Cre8ionist, I would submit that, of the three attributes you focus on - contingency, specifications, and complexity - tornadoes have two of these. Obviously, they arise out of contingency (e.g., due to standard rules of chemistry and physics). It’s clear to me that tornadoes are specified, at least as much so as living things. But they are not complex - complexity being defined by me as being of low probability. That’s my point - tornadoes possess large quantities of specified information, yet they are not complex. (Of course, Janitor's comments may lead to some "ways out" of this paradox.)
Kirk, your statements concerning Shannon’s “rules” are well taken. I believe these “rules” are exactly what I used to estimate the specified information content of a tornado. In your post, you said:
quote:
“I = -log(Nf/N)
where
Nf = number of functional configurations and,
N = total number of configurations, functional or non-functional.“
Translating my earlier post, for each individual water vapor molecule, Nf/N is about (10^30 - 1)/10^30. The negative log of this is admittedly a very small number. But, as I indicated, when we add up this term for all of the molecules, we get a hefty amount of specified information - roughly a million bits.
I thinks it’s hard to avoid the implications of these numbers.
Finally, for (some of) Janitor’s comments:
quote:
“But all this is to say that an important principle (actually several important principles) of information theory is missed here, e.g., averaging. Doing it Art’s way I will eventually begin to partition the states into bins, reflecting some narrow range of states, arbitrarily divvying up the ranges, etc. In other words I’m going to find myself averaging over states anyway, so why not simply begin that way, using such “bulk” measures as temperature, pressure, velocity, etc.? These will certainly yield meaningful meteorological information. (With far less investment in labor.)”
An interesting perspective. Which brings to mind a few different statements. First, you seem to be suggesting, Janitor, that different approaches may yield rather different estimates for information content. I’ll have to sleep on this a bit longer, but I suspect that this gets back to my previous intuition that some sort of work function may be what we are really thinking about (rather than information per se). Or that not all of the inherent information in a system (IC or otherwise) is really specified information. (I would recommend that others keep the rest of Janitor’s post in mind, since he is saying similar things in different, probably more informative ways.)
Secondly, trying to bring IC into the theme of this thread (explicitly, at least), maybe the specified information content of IC systems is more a reflection of some sort of averaging of its components and/or their interactions, rather than a strict (mundane, perhaps irrelevant) summation of the information content of the “pieces”. Put another way, maybe the entire sum of information of all the “pieces” does not contribute to the specified information content. This helps to address the paradox I laid out in the original post, and it may help us think about IC in a more productive and relevant way. This would seem to be a useful take-home message from this example.
Finally, the notion of averaging brings to mind a different way to think of protein structure. (It may not be exactly analogous - my mind simply makes this connection.) This is, one probably should think in terms of structural units (helices, sheets, etc.) rather than amino acid sequences. This perspective is almost certainly more relevant to the ways proteins evolve in nature; indeed, it gets us away from what I consider to be a picture of such pathways (e.g., residue-by-residue meandering from one structural minimum to another one, through a seemingly imposing desert of non-functional intermediates) that really does not apply to real proteins. Of course, the alternative I have in mind is by mixing and matching or relatively formed and functional units. (My active imagination conjures a playful analogy with particle tunneling in quantum physics.)
More things to think about. Looking forward to more feedback.
IP: Logged
|
|
Cre8ionist
Member
Member # 140
|
posted 16. March 2002 08:46
Hi Art,
quote:
Cre8ionist, I would submit that, of the three attributes you focus on - contingency,
specifications, and complexity - tornadoes have two of these. Obviously, they arise out of contingency (e.g., due to standard rules of chemistry and physics). It’s clear to me that
tornadoes are specified, at least as much so as living things. But they are not complex - complexity being defined by me as being of low probability. That’s my point - tornadoes possess large quantities of specified information, yet they are not complex.
Forgive me here for quoting too much, but only Dembski can settle this dispute about contingency, he said quote:
In practice, to establish that an object, event, or structure is contingent, one must show that it is not the result of a natural law (or algorithm). For example, a crystal of salt results from forces of chemical necessity that can be described by the laws of chemistry. By contrast, a setting of silverware is not. No physical or chemical laws dictate that the fork must be on the left and the knife and spoon on the right. The place setting of silverware is therefore contingent, whereas the structure of the crystal is the result of physical necessity.
That is the exact opposite of what you're saying, it seems to me. The tornado corresponds to the crystal, DNA to the silverware.
Also, in your original post you made the case that the tornado was complex, i.e., a violation of the universal probability bound set by Dembski. Now, you're saying that it's not. Which is it?
Finally, you compare the specificity of the tornado to the specificity of life, which I find amazing! Forgive the expression, but it seems to me, if you flush scrabble pieces down the toilet, they'll take on the kind of specificity that you're equatingl to life's specificity, but if you arrange them into meaningful words, on a prefabricated board, they'll take on the kind of specificity that I think life has.
A definite contrast in viewpoints. BTW, Werner Gitt, in "In the Beginning was Information" does
attempt to expand the definition of information from merely statistical, to include meaning. It's difficult to attain, (also translated from German), though I have acquired a copy. I think it's worth looking at...............................................Cre8
[ 16 March 2002, 08:52: Message edited by: Cre8ionist ]
IP: Logged
|
|
Mark Elkington
Member
Member # 120
|
posted 17. March 2002 04:35
Art,
quote:
In my opinion, IC “things” are not constrained in these ways. Indeed, I would argue that to define IC as you are doing is to create a sort of circular argument that really limits the concept as a useful tool.
Consider Behe's definition of IC:
"By irreducibly complex I mean a single system composed of several well-matched, interacting parts that contribute to the basic function, wherein the removal of any one of the parts causes the system to effectively cease functioning. An irreducibly complex system cannot be produced directly (that is, by continuously improving the initial function, which continues to work by the same mechanism) by slight, successive modifications of a precursor system, because any precursor to an irreducibly complex system that is missing a part is by definition nonfunctional." (DBB p. 39)
I'm not wanting to quibble over definitions, but as I reflect on the "pathway" component of this IC definition, it seems to me to actually be mandatory. If you remove it, then you can describe just about anything as IC. For example, a river is IC: if you take away the bed, banks, gradient, or water, the system ceases to function.
The pathway, and the difficulty of traversing it in the formation of a system is inextricably linked with the CSI content of that system. The definition of IC itself is irreducibly complex in this respect
[ 17 March 2002, 04:38: Message edited by: Mark Elkington ]
IP: Logged
|
|
Art
Member
Member # 179
|
posted 17. March 2002 11:09
The nuances of definitions are indeed fascinating.
Cre8ionist, thanks for clarifying you usage of the term contingency. I was thinking of something completely different. Obviously, tornadoes are not contingent as you now lay out. But I don't think that has much bearing on the central message here (which I will explicitly spell out at the end of this post, to try and bury the semantics that dogs these dicussions).
I was having a hard time figuring out where you got the idea that I said tornadoes were complex. Then it hit me that I had characterized tornadoes as IC. In contrast to Mark's assertion in another thread, the "C" in IC is most definitely not the came as the "C" in CSI. The latter is a probabilistic statement, and clearly does not apply to tornadoes. Which returns me to the main point - extremely large quantities of specified information need not be "complex".
Just so I am as clear as can be - as defined by Dembski, tornadoes are not complex.
(The "C" in IC is more a description of the number of parts - the more the players, the more the complexity. This is very different from complexity as defined by Dembski.)
quote:
Finally, you compare the specificity of the tornado to the specificity of life, which I find amazing! Forgive the expression, but it seems to me, if you flush scrabble pieces down the toilet, they'll take on the kind of specificity that you're equatingl to life's specificity, but if you arrange them into meaningful words, on a prefabricated board, they'll take on the kind of specificity that I think life has.
I speak, not of specificity, but of specifications. These are two rather different concepts. As Demsbki states (the following is from a paper by Dembski entitled "Intelligent Design as a Theory of Information" that I got on-line some time ago - I apologize if it is dated or now unaccessible.):
quote:
The distinction between specified and unspecified information may now be defined as follows: the actualization of a possibility (i.e., information) is specified if independently of the possibility's actualization, the possibility is identifiable via a pattern.
Hopefully, in this light, one can see why I can say that the information redisent in the molecules of a tornado is specified.
Mark, I also do not wish to quibble about definitions of IC. But maybe I should clarify my original statement - because I think that it may help to weave modified ideas about changes in information into the ways we think of IC.
I have always taken the first part of the statement quoted to be the definition of IC - "a single system composed of several well-matched, interacting parts that contribute to the basic function, wherein the removal of any one of the parts causes the system to effectively cease functioning.". This is a pretty safe statement and can be used to build a framework for further analysis and inspection (as Mike Gene does). Tornadoes fit the bill here.
The second part - "An irreducibly complex system cannot be produced directly..." is (at best) a tentative statement that is not part of the definition, but a suggestion that follows from the definition. I am not inclined to accept this assertion, since examples in nature (see "tornadoes") and in cells show that one does not follow from the other. This is a matter for other discussions.
Hopefully, that clarifies some things, and may allow us to return to the original subject - that extremely large quantities of specified information need not be complex (in the sense given to us by Dembski).
IP: Logged
|
|
Cre8ionist
Member
Member # 140
|
posted 17. March 2002 21:35
Hi Art,
My real disagreement with you was your use of Dembski's universal probability bound.
With the three criteria (contingency, complexity and specification) not being met, I felt it was used improperly. Even though I slipped when I said the tornado lacked specificity, it actually lacks complexity and contingency, the point's still the same.
As for specified things being created naturally, it happens all the time.
Crystals, snowflakes, tornadoes etc. are all specified, but lack complexity, therefore, their order is not analogous to the specified complexity which we find in living things.............................................................Cre8
IP: Logged
|
|
|
Printer-friendly view of this topic