|
Author
|
Topic: Is 2nd Law a special case of 4th Law?
|
Christopher D. Beling
Member
Member # 723
|
posted 27. September 2005 03:45
Ok folks - here are some thought experiments. 1st experiment: Imagine not a spin gas, but a gas of ordinary atoms with just translational motion. An atom gun can fire atoms into a box with discrete energies e1, e2, e3 - - e20. As per the spin gas we can send in a coded message, i.e. say we could choose the triangular message:
[I>=[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,1 --- repeating]
Imagine the walls of the box are at absolute zero (T=0) and made from infinitely heavy atoms that cannot be excited - then the injected atoms will bounce in specular fashion from the walls and will not gain or lose energy. Some time latter we let the atoms out of the hole and measure their energies. We will have lost the triangular pattern (temporal sequence code), but we will have retained the number of atoms in each class. With this triangular sequence - we expect to get equal numbers in each class. The class population vector:
[O>={n1,n2,n3 - - - n20}
will be such that n1=n2=n3=ni=N/20, where N is the number of atoms fired in. ([O> still retains information on the temporal sequence [I> - and could thus be used to open a lock for example) At all stages the probability of finding an atom in class 'i' is 1/20. Thus the Shannon entropy of the gas state will be:
S=-N Sum(1-20) of (1/20)log2 (1/20)=Nx4.32 bits
i.e. each gas atom carries 4.32 bits of information.
2nd experiment :- The gun is set to fire only atoms with ground state energy e1. We fill up the box with N such atoms.
[I>=[1,1,1,1,1,1,----------Ntimes >
The corresponding Output category vector will be
[O>={N,0,0,0,0,0 - - - - -20times}
The Shannon entropy of the gas in this case is
S=-N Sum(1-20) of (ni/N)log2(ni/N)=-N(N/N)log2 (N/N)=-N log2(1)=0
This should not suprise us because the system is in a single microstate - its entropy is zero - because it exists with a probability of one (i.e. certainty). Why can't the atoms in the box's walls excite the gas atoms - because the gas walls are at T=0. In passing this is a nice example of the 3rd law of thermodynamics - which says that at absolute zero each sub-component of a system must have entropy zero Indeed we can now relax the condition of infinitely heavy box atoms - because the gas atoms being at lowest energy cannot exite the box atoms!
3rd experiment- The gun is set to e3 and fires N atoms into the box - Let us now change the box walls to have temperature T (could be room temp). The input temporal sequence is:
[I>=[3,3,3,3,3,3 - - - - - -- N times>
or in reduced form:
[I>={0,0,N,0,0,0 - - - - - - 0}
In this case the vibrating atoms in the box walls can now receive energy from a gas atom - and they can also impart energy. i.e. after the first collision reduced vectors such as:
[2>={0,1,N-1,0,0,0 - - - - - 0}
[2>={0,0,N-1,0,1,0 - - - - - 0}
would be examples. As time elapses through more collision times - the sequence will get more and more random - the original pattern being lost.
The above diagram represents this process schematically. The left diagram maps the energy of particle states - starting from specified state [I> and then particles randomizing into different energy categories with time - finally ending with either MB (Maxwell-Boltzmann) or BE (Bose-Einstein) distributions. The right diagram shows configuration (Omega) space with state [I> pictured as a solitary island in the middle. Stochastic scattering to nearby configurations takes the gas away from [I> into the large ensemble of alternative states that are consistent with the total energy of the gas -[The total energy is E=Nxe3=Nx(3/2)kT, so that the energy spacing of categories is DELe=(1/2)kT]
Mathmatically the statistical treatment of this stochastic randomization of configurations seems to follow an identical line to the evolution of proteins as given by Yockey - as described in both the Perron-Frobenius theorem (PFT) (p53,p209) and the Shannon-McMillan-Breiman theorem (SMBT) (p29,p205). Thus the probability vector for the second state of the system (after first collision) is given by:
[2>=P[I>
P being the "transition probability matrix" and the general time development by the rth state by:
[r>=P[r-1>, and thus [r>=P^r[I>
so that according to the PFT the system comes to dynamic equilibrium in final state [F> given by:
[F>=P[F>
i.e. [F> is an eigenstate of operator P. [F> for a simple atomic gas will be the MB distribution - i.e.:
[F>={n1,n2,- - -ni - } where ni=exp(ei/kT)/Z
In this final state [F> all memory of the initial state [I> has been lost. I.E. [F> and [I> have zero mutual entropy. The final state [F> is a state of maximum entropy H - and according to the SMBT thus belongs to that set of large multitude - with 2^NH members. Out of interest lets calculate H for the [F> state of our simple gas:
H=Sum1 to 20 of (ni/N)*ln(ni/N)
=(exp(-1/2)/Z)*ln[exp(-1/2)/Z]+(exp(-1)/Z*ln[exp(-1)/Z) - - - etc
=0.3670+0.3419+0.2798+ - - -20th term=1.703 bits
(Note: Z=sum1 to 20 of exp(-ei/kT)=1.54)
i.e. each atom - carries with it on average 1.7 bits of info - corresponding to the information required to specify the state of the gas. [This number of 1.7 bits should be compared with the number of 18 bits/atom for a real gas at standard temperature and pressure - ie our gas is really far too simple - a real gas has an effective alphabet not of 20 but more like 250,000!!]
2nd and 4th law connection I think in the above description we are beginning to see the connection between the 2nd and 4th laws. Note that the state [I> was a specified state. With its set {0,0,N,0,0 etc} it could have been used to open a lock! But this complex specified information (CSI) was lost ( decreased )as stochastic collisions took place. The irony though is that this same process led to an increase in the complex information (CI)(i.e. the entropy). I say irony perhaps because when you are randomizing something you don't normally expect information to go UP !!!. Information must be understood, however, in the specification of the gas - i.e. corresponding to the imagined case of the gas atoms being let out of the box and their energy states recorded in binary. The gas state "read out" would have exceptionally LOW probability (2^-NH) and thus very HIGH info content. This diagram summarizes:
where the time axis is in "collision time" units (in the case of biotic evolution - this would be mutation time units)
Why is the CSI high at time zero when we had calculated an entropy of zero for [I>? This is because (i) we did not need to have chosen this particular (low-K) specified configuration - it could have been another (high-K) config. (ii)we must in calculating CSI base probabilities on those of an underlying random hypothesis - i.e. in this case that of the MB probabilities. Thus [I> is in general complex information and not just specified information.
I think the above argument shows the clear common information theoretic origin for the 2nd and 4th laws. It also clearly demonstrates that the two laws are different:
2nd law: "In a closed system the CI (entropy) either remains constant or increases"
4th law : "In a closed system the CSI either remains constant or decreases"
Sorry, this has been a long argument - with too much maths and too many sleepless nights! Well, what do you think? I think we could be looking at the connection.
-Chris
[ 27. September 2005, 22:58: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
William Brookfield
Member
Member # 565
|
posted 01. October 2005 19:19
Hello Everyone,
I would especially like send a long overdue thank you Jerry Don Bauer for his comments wrt me on ARN and his knowledge of (and posting of) my earlier ISCID (Dec, 6 2002) article. Unfortunately, as a social misfit (an ID humanist), I do not have personal access to the internet (nor do I have personal access to, say, running water). Luckily, I still have electricity and a computer and I am subsequently able to write this message and deliver my "2 cents." Hopefully my living conditions will improve and I will be able to contribute more than just "drive by postings" in the future.
Quote-
Salvador T. Cordova wrote quote:
-- now, IDists are suggesting a 5th law known as the "4th of Thermodynamics". The "4th of Thermodynamics" says basically information will tend to decrease over time. This makes sense in that the music on your CD's will not spontaneously improve, but will tend to deteriorate and accumulate noise. One will never find a damaged CD resulting in better music from the CD, in general....
Hi Sal,
In my model the CD is embedded in the orthodox thermodynamic info-domain which is in turn embedded in the black hole dynamic infodomain which is in turn embedded in the primal domain/state of "unbound telesis (UBT) or "primal equilibrium" -- "a state of infocognitive potential, free from informational constraint" to quote Chris Langan. As result the CD and its attendant specified information is subject to both orthodox thermodynamic decay (2nd law) and BH dynamic decay (BHd 2nd Law). The CD (read "mass” here) and its attendant "spacetime" is slowly "falling" (decaying) into the black hole at the center of our galaxy.
Unlike the other forces (electromagnetic, weak nuclear, strong nuclear) gravity is not indigenous to the orthodox thermodynamic info-domain. "Gravity" (spacetime distortion) results from a primal logical (but not informational) constraint that UBT (primal equilibrium) imposes upon any space-time-mass infodomain derived therefrom. We, in turn, experience gravity because our more local info-domains are necessarily embedded in the underlying gravitating, black-hole-dynamic infodomain.
By the way, because our universe is an information structure and because information requires binary (yes or no) answers, the informational universe must be quantized. But because our informational cosmos is derived from an inviolate singularity, there must always be a residual incompleteness to these binary splits. The result is that "mass" is not utterly separate from "spacetime" and we subsequently have a relativistic quantum universe. Unification of quantization and gravitation is therefore a natural logical consequence of the infodynamic (ID) model.
The wording "law of conservation of information" is somewhat unfortunate here because "conservation" would require the pure determinism of a purely platonic realm. The internal "cyber-space" of a computer is only pseudo-platonic. This internal "cyberspace" is necessarily embedded in the orthodox world of the Second Laws and is therefore subject to GSL decay (information loss).
While a CD contains human information, the loss of (but not the production of ) this information can be explained by the scrambling effect of the orthodox second law (without reference to any new laws of physics or laws of information). If a wall has a picture painted upon it, and the wall is subject to decay, then so is the painting. Statistically, the CD's information content is merely another hideously small target in a surrounding large phase space.
An analogous statistical function, however, independent of the orthodox Second Law might be seen in the read/write info-dynamics of our computer hard drives.
Hard Drive Info-Dynamics.
First Law of Hard Drive InfoDynamics (HDID): Conservation of read/write surface.
The size of the hard drive always remains constant, as does the size of the sectors and the size of the bits and bytes.
Second Law of HD-ID: Once information is "deleted" the sectors in question are marked as reusable. As a result, the "deleted" or "old information" is progressively lost as new information is gradually written over the old information. With there being no syntactical relationship between the new and the old information, the HDdynamic decay of the old information is effectively random /statistical. * It is also possible of course for HD information to be lost due to cross info-domain interference (as in the CD example above) or through read/write error.
It seems to me that a shared re-use model of this nature also suggests a kind of biological extension of the 2nd law. When a human builds a bridge and termites eat (re-use) the wood, the loss of human information (bridge losing) effect is random wrt the human wooden bridge structure (but not wrt new termite structure). Without humans to maintain the human structure, non-human structures (living or non-living in our terms) will inevitably prevail.
Similarly, in the case the of the gas-in-the-box (that moves from dis-equilibrium to equilibrium) the gas is assuming the flat shape of space itself (the spacetime info-domain) -- just a the wood nutrients are assuming the shape of termites (termite {micro}info-domain). This model of course blurs the distinction between the living and the undead.
If this basic analogy is correct then the first law of thermodynamics is not an infodynamic law. It merely identifies the conserved substrata. The orthodox Second Law however would indeed be a law of information (loss) as would be my Second law of HD-ID. I also maintain that the Second law of BHole dynamics is a law of information loss, in that space-to-time-to-mass differentiation is not conserved over the BHdynamic process.
For grand-unification purposes I think that it is important to concentrate, not on heat or cooling, but on the fact that an ideal gas at the thermodynamic end state (thermal equilibrium) can be described by very few, info-depleted, parameters (temp, pressure, volume). Similarly a black hole can be described by very few, info-depleted, parameters (mass, charge, angular momentum). As a black hole grows in size (or Hawking-ly radiates to infinity) even these remaining parameters (read "info") become depleted (lost).
In the same manner, randomly thrown scrabble pieces can be described by very few, info-depleted parameters ("random scrabble pieces"). The word "random" is not even required here. "Scrabble pieces" will do just fine. Any meaning or secondary structure affixed (by humans) to such random letters is just projection. Similarly, "monkey shakespeare" is just another random sequence of letters. We attach meaning and structure to those particular letters. The monkey does not.
Does this mean that the "universe" is just "monkey shakespeare" and that we are actually "living" in an utterly random black hole? Yes. However, through the use of massive, ongoing creative projections, we are making the very best of an otherwise depressing situation. ![[Smile]](smile.gif)
The basic subjectivity of spacetime languages (cosmic "info-domains") however, does not render Dembskian design inferences invalid. The reason I think is that languages have rules or "agreements" that are themselves objective. Typing monkeys and random scrabble throwings can only "produce" extremely small amounts of Shakespeare (and even these are only due to the presence of a residual, system level of order). For any significant amount of Shakespeare (consistent with the rules of fine English literature) the odds are astronomically prohibitive for typing monkeys (and zero without typewriter or monkey). The large jumble of metal, rivets, gauges, wires, pipes and fuel in my garage does not become a "jet fighter plane" until its construction actually hits the "target" and it can subsequently fly. Merely labeling my junk a "jet plane" will not work. I.E., Merely drawing a "target" around an arrow already in stuck the wall will not work. If the construction is not consistent with the rules (the laws) of physics then it will not fly. These hideously small targets are real and hitting them is not optional. The same is true for new species if they are to fly (run, swim, crawl, reproduce, etc. )
If these targets cannot be hit by material/stochastic processes (in the lifetime of the universe) then something else must be nailing them. Moreover, the cosmological second laws of thermodynamics and BHdynamics rule out any specificity (dis-equilibrium) arising from stochastic processes at the more basic cosmic level. A cosmic design inference is thus also valid -- for without specificity, complexity cannot even get started. Ultimately we need an explanation for why there is a universe (cosmic dis-equilibrium) and not just a black hole (cosmic equilibrium).
Salvador T. Cordova wrote, quote:
“Dembski's formulation of the Fourth Law is tied to his formulation of CSI. If we relax the requirement that the Fourth Law to not be solely CSI but simply "I" (for information), then I think we're in a position to have a unified view of the 2nd law being a special case of the Fourth Law.”
The reason that I have posted is of course that my cosmic model is pure "fourth law." One could say that my model is a "fourth-law-ization" of the second laws. I also now consider even primative disequilibrium to be primative information and the product of design. This seem to me to be the implication of BHdynamics. Without design, a cosmic black hole would remain at equilibrium forever.
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 15. October 2005 06:15
Hi William: Your post is very interesting and helpful. Also I hope you have solved the running water problem! Some questions and comments:
quote:
embedded in the black hole dynamic infodomain
I understand that the entropy of the universe still increases when matter falls into a black hole. The black hole gives back to the universe more entropy than it absorbs through Hawking radiation - so the 2nd law is not broken. Is this what you mean by "BHd 2nd law"?
quote:
But because our informational cosmos is derived from an inviolate singularity, there must always be a residual incompleteness to these binary splits. The result is that "mass" is not utterly separate from "spacetime" and we subsequently have a relativistic quantum universe.
I have to admit that I do not follow you here. Why does the singularity (Big Bang or Black hole?) lead to an incompleteness in the binary splits - (level of information?). And how does this lead to a connection between mass and space-time (I guess you mean that as described by the equations of general relativity?). Are you able to understand the general theory of relativity using information theory concepts? Is there somewhere that I could read about the "infodynamic model" that you talk of?
One thing that does worry me about your bringing the "black hole dynamic infodomain" into the picture is the inference that the degrading of specified information on either DNA or a hard-disc could be influenced by our proximity to a black hole. Now clearly that would be the case if one were close to the event horizon of a black hole - and the storage medium (DNA or hard-drive) was literally getting ripped apart by the warping of space-time, but in our location of the galaxy - or in the space between galaxies (where the gravitational field is zero) surely there is no effect?
quote:
The wording "law of conservation of information" is somewhat unfortunate here because "conservation" would require the pure determinism of a purely platonic realm. The internal "cyber-space" of a computer is only pseudo-platonic.
I fully agree with you here. I also think your description under the heading Hard Drive Info-Dynamics shows this point clearly. In Dembski's derivation of the 4th law, he does not dispute the fact that processes in general (which always have a stochastic element) will increase Shannon type information. Thus we should not really talk about a general law of information conservation but a general law of information increase . To clarify this point I refer to your statement:
quote:
The orthodox Second Law however would indeed be a law of information (loss) as would be my Second law of HD-ID.
The fact that you see loss while I see gain results just from confusion - indeed a confusion that is common and one because of its high importance must be comment on further. I want to note though that the 2nd law is the law of entropy increase . Now entropy can be interpreted in a number of ways but three of these are:
(i) Shannon information of the system microstate
(ii) Disorder of the system microstate
(iii) Degree of ignorance of the system microstate
I am of the belief that most folk find it hard to accept that the information of a system can go UP while disorder is going up! But this the case and follows from the definition of Shannon information.
So we have this situation of the CD or DNA being embedded in the orthodox thermodynamic info-domain - where the thermal motions are causing a general entropy increase. I.E. the whole system's (i.e. the general orthodox thermodynamic info-domain) Shannon Information is increasing. The degree of disorder and ignorance are also increasing. This is all the 2nd law. HOWEVER with regard to the information on the CD or DNA (and this is specified Shannon information) the thermal motions etc cause a decrease in information [but please note here we talk of specified information - not just Shannon information in general]. This is the 4th law. The two laws are related but they are not the same. The 2nd law just relates to information in general, while the 4th law refers to only tightly specified information (such as that found on the CD or DNA).
Can you agree with me that the 2nd and 4th laws are different?
- Chris
[ 15. October 2005, 08:01: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
William Brookfield
Member
Member # 565
|
posted 27. October 2005 18:05
Hi Christopher,
Thankyou for your interest.
Christopher D. Beling wrote;
quote:
I understand that the entropy of the universe still increases when matter falls into a black hole. The black hole gives back to the universe more entropy than it absorbs through Hawking radiation - so the 2nd law is not broken. Is this what you mean by "BHd 2nd law"?
There exist a complete set of relativistic black hole dynamic laws analogous to the orthodox laws of thermodynamics. I am refering to both the black hole dynamics 2nd law and dynamic arena (the BH dynamic infodomain) of mass, spacetime and gravitation.
-------------
Chris
quote:
I have to admit that I do not follow you here. Why does the singularity (Big Bang or Black hole?) lead to an incompleteness in the binary splits - (level of information?). And how does this lead to a connection between mass and space-time (I guess you mean that as described by the equations of general relativity?).
Yes indeed -- General Relativity. Infodynamic cosmology assumes that primal equilibrium (the initial singularity) will contain an equilibrious distribution of all existential possibilities. Consciousness and unconsciousness, being existential possibilities, are therefore included in any initial singularity as "semi-consciousness." (Consciousness must of course be equilibrious at this point and therefore neither conscious nor unconscious but instead semi-conscious). In an infodynamic (ID or Noetic) scientific model, consciousness (not material) is assumed to be the ground of all being, and the source of BH instability. In the materialist model, consciousness is assumed to be just an epiphenomenon of material. In the ID model "material" is assumed to be an (informational) epiphenomenon of consciousness. Without any initial material to work with, primal consciousness had to use itself as raw material. This means that both matter and space-time are necessarily composed of the same inviolate proto-fabric (cosmic consciousness). As an idea construction, the universes represents a platonic ideal -- a Newtonian universe in which matter and spacetime are perfectly separate. The reality of its singular source however imposes a subtle but necessary "relativistic-ness" upon the realization of any such Newtonian ideal. The universe is suboptimal.
quote:
Are you able to understand the general theory of relativity using information theory concepts?
Yes. Both mass and spacetime are cases of transcendent specificity, existing in probabilistic defiance of the black hole (zero information) ground state.
quote:
Is there somewhere that I could read about the "infodynamic model" that you talk of?
I appear to be a pioneer in the field of relativistic infodynamics. Some of my earlier work can be found here at ISCID [URL=http://www.iscid.org/papers/Brookfield_Devolution_120502.pdf] and more can be found at my web site (www.bitscience.org). I am modeling gravity as information loss and special relativity as information translations. In your example you speak about a triangular pattern that disipates over time. I am modeling mass and spacetime dimensions as orthogonal standing waves (X, Y axis) with a residual relativistic error in the proximity of the intersection point. This X, Y axis form is a specified independent pattern that disipates over "time"(over the BH dynamic process).
quote:
One thing that does worry me about your bringing the "black hole dynamic infodomain" into the picture is the inference that the degrading of specified information on either DNA or a hard-disc could be influenced by our proximity to a black hole. Now clearly that would be the case if one were close to the event horizon of a black hole - and the storage medium (DNA or hard-drive) was literally getting ripped apart by the warping of space-time, but in our location of the galaxy - or in the space between galaxies (where the gravitational field is zero) surely there is no effect?
Gravitational distortion of spacetime is quite insidious. Once information has been physical projected into spacetime however it is suceptable to the degradation of space-time-mass framework onto which it is mapped. As long as the degradation is consistent (which it is because it is diffeomorphism invarient) then everything will seem to be fine, judging by ones local frame of reference. Due to the surrounding galaxies even the space-time between galaxies is BH-dynamically unstable. Moreover, as soon as information is translated into a physical medium the physical mass of this medium warps spacetime. This analysis, being relativistic and BH dynamic does not include the compensating effect of "dark energy" however.
quote:
The fact that you see loss while I see gain results just from confusion - indeed a confusion that is common and one because of its high importance must be comment on further. I want to note though that the 2nd law is the law of entropy increase . Now entropy can be interpreted in a number of ways but three of these are:
(i) Shannon information of the system microstate
(ii) Disorder of the system microstate
(iii) Degree of ignorance of the system microstate
I am of the belief that most folk find it hard to accept that the information of a system can go UP while disorder is going up! But is this the case and follows from the definition of Shannon information.
My infodynamic version of thermodynamics is a "politically incorrect" inverse formulation of orthodox statistical thermodynamics. ISCID is a revolutionary society. As a general rule, revolutions are politically incorrect. Expect the unexpected. I have also re-interpreted relativistic space-time curvature as space-time-mass decay. Both of these reformulations, while operationally consistent with existing formulations, are also consistent with an infodynamic cosmos. In my inverted version of thermodynamics "order" is equivalent to "information" and disorder/randomness/Maxwellian distribution is equivalent to zero information. (iiii) Disordering (or loss) of transcendent specificity -- loss of "Brookfield Information"
quote:
Can you agree with me that the 2nd and 4th laws are different?
This question is too general but I would say that both the laws relate to tightly specified information and its loss of specificity over time. Initially the "gas in the box (in the bigger box)" -- is tightly specified. I.E., More tightly than equilibrium.
In order for mere "increasing K-complexity" to become "increasing information" a background reference class of possibilities is required. The reference class you are invoking however appears to be outside the pertinent logically derived infodomain.
After surveying some of the literature, I think I see a possible source of confusion.
William Dembski -from the "The Design Revolution"
quote:
"The problem is that chance can generate plenty of novel information (just get out your coin and start tossing it. After a few dozen tosses, the sequence of heads and tails that you witness will be unique the history of coin tossing and will constitute novel information."
While I do agree with and heartily support William Dembski on most things pertaining to ID science, I disagree with him here.
Every instant of every day I witness novel driveway gravel configurations, unique the history of gravel tossing and indeed every moment on earth is unique in the history of planet tossing. In an infodynamic universe, information is ubiquitous. Unfortunately this residual background information is irrelevant information w.r.t. the coin-configuration info-domain -- nor is this information produced "by chance." A "fair" (random) coin tossing info-domain means that for every "heads" there is an equal and opposite "tails" and this means no information. Information requires a yes or a no answer, not yes and no in which every "yes" is invariably canceled by a "no."
In order to keep myself from becoming confused I invented the logically bound "infodomain" concept. The definition of information that I use is "Transcendent Specificity" (TS). Dembski's "specified complexity" is a case of "transcendent specificity." In particular "macro-evolution" (specified constructive speciation) and the construction of new features (flagella, etc.) transcends the Darwinian info-domain of adaptive "micro-evolution"(variable reductive niche-ification --through random mutation & natural selective destruction).
Now before anyone yells "creationist" here let me just explain the word "transcendence" as I use it. The random typing monkey can indeed "produce" small amount of "shakespeare"(letters) but only in accordance with the uniform probability distribution as provided by the defining system parameter "random." If the random typing system produces shakespeare sequences or any other sequences *out* of accordance with "randomness" then we have evidence of system transcendence -- of "something more" and we now have "information" in the "infodomain." Prior to this, the output, being random, was devoid of information. While there does indeed exist a system state level of information here (alphabet, typewriter, monkey) this background, system level of information is not pertinent to the foreground information (TS) we are discussing. This particular type of "transcendence" does not require faith but is susceptible to a fully rigorous mathematical statistical analysis.
With use of the phrase "transcendent specificity" we can see what causes the Second Law of thermodynamics, the Second law of Black hole dynamics and Dembski's Forth law (which is really just another "Second Law {of information}"). This "transcendent specificity" is always embedded in -- and riding upon -- an underlying info-domain that is not "transcendent" nor "specific" but is instead non-transcendent and equilibrious (unspecific). With the universe not being a perfect platonic ideal, there is always a certain amount of collision between infodomains. The result is the gradual buffeting of systems in the direction of the underlying equilibrium and the subsequent loss of information/transcendence.
Chris
quote:
Information must be understood, however, in the specification of the gas - i.e. corresponding to the imagined case of the gas atoms being let out of the box and their energy states recorded in binary. The gas state "read out" would have exceptionally LOW probability (2^-NH) and thus very HIGH info content.
This appears to be same error that William Dembski has made. While this information is real, it is not pertinent to the infodomain in question. All the "furniture" (pertinent information) has left the building (along with Elvis) and we are now counting the "carpet hairs."
My Definitions:
"Information"; Transcendent Specificity.
"Info-domain": An information domain including "transcendent specificity" plus its non-transient background reference class of possibilities.
"Infodomain Embedding": Occurs when infodomain (#1) is nested within a deeper infodomain (#2) such that "transcendent specificity" within domain #1 employs the roof/ceiling of underlying domain #2 as a reference class.
"Randomness"; a uniform probability distribution devoid of information (transcendent specificity) -- by definition of "uniform."
"Low Entropy" = High Information. Examples; thermal disequilibrium, people, coherent space-time-mass complex , a robust signal.
"High entropy" = Low Information. Examples; thermal equilibrium, chemical equilibrium, black hole, a degraded signal
=================
To Summarize: Orthodox Shannon information theory cannot simply be glued unchanged to orthodox thermodynamics and in turn glued unchanged to orthodox relativity. If the problem were that simple it would have been solved long ago. Reformulations are required.
#1. Orthodox Shannon information needs to supplemented with a dynamic internal informational entity ("specified complexity," "transcendent specificity" or perhaps your own "specified Shannon information."
#2. Orthodox thermodynamics is not a valid infodynamic theory and requires an infodynamically-valid inverted formulation. "Thermo-dynamics" is a theory about the dynamics of heat (molecules in random motion). It is therefore a theory about lots and lots of little nothings. If you want "nothing" (no information) on your hard drive then randomize it. If you wish to lose the information as to which corner of the "Gas in the Box" the gas came from, then permit the gas to attain equilibrium. In orthodox theory, "entropy" is considered "something." In infodynamics "entropy" represents the "absence of something" -- specifically, the absence of "transcendent specificity"(absence of information).
#3. Orthodox relativity needs to be re-interpreted in terms of -- you guessed it -- "transcendent specificity" wherein mass and spacetime transcends the black hole (zero information) ground state.
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 05. November 2005 21:24
quote:
Chris wrote:
Imagine the walls of the box are at absolute zero (T=0) and made from infinitely heavy atoms that cannot be excited - then the injected atoms will bounce in specular fashion from the walls and will not gain or lose energy. Some time latter we let the atoms out of the hole and measure their energies. We will have lost the triangular pattern (temporal sequence code), but we will have retained the number of atoms in each class. With this triangular sequence - we expect to get equal numbers in each class. The class population vector:
[O>={n1,n2,n3 - - - n20}
Thank you for your hard work on this.
I'm still spooling up on some of the themrodynamic basics, so I beg your patience with me. If the atoms collide with each other, will this change the energy distribution?
Salvador
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 06. November 2005 20:08
Hi Salvador, So happy to hear you again. In fact I thought you had kind of backed out.
quote:
If the atoms collide with each other, will this change the energy distribution?
Yes in the general situation the atoms will of course collide. Any collision will result generally in a sharing of the energies and the redistribution to new energy classes. For example if we start with the macrostate:
[O>={n1,n2,n3,n4 - - - n20}
then after collision (AC) one could get:
[AC>={n1+1,n2-1,n3-1,n4+1 - - - n20}
that is an atom in energy class 2 has interacted with one in energy class 3 and the scattering has caused the first atom to loose 1 unit of energy placing it in class 1 and the other atom to gain 1 unit of energy placing it in class 4. [It would also be possible - through the indistinguishability of atoms - to have the class 3 atom loosing 2 units of energy and the class 2 gaining 2 - the outcome would be the same]. So yes, such collisions can occur - and as we see they have the effect of increasing the Complex information (entropy) of the gas (providing we are not in equilibrium) and destroying the Specified complex information (CSI) of the gas.
This question relates to my proposed "experiment 1". I do not think the conclusions of that experiment need to be negated in an way as a result of gas collisions - because it is a thought experiment and as such we could always stipulate that the size of gas atoms is "infinitely small" - (giving zero probability of collision) or we could stipulate that the number of gas atoms is small (say 1000) so that the probability of gas collisions in a box of say 1cm^3 over a reasonable time period would be negligable. In any case the thought experiment is just that - designed to get us to think - it is already highly unrealistic because it requires infinitely heavy wall atoms!! Finally if we do relax these idealized conditions we only get something that supports the view that CI increases while CSI decreases (i.e. that the 4th law is real and is related to the 2nd)
It is an interesting point that at absolute zero (T=OK) in reality this kind of collision cannot take place (experiment 2), because all the atoms must eventually end in their lowest energy macrostate:
[O;T=0>={N,0,0,0 - - - - 19th zero}
and because no atom can scatter to lower energy likewise no atom can scatter to higher energy. The downside of this is that this "macrostate" of the gas is the only one permisable and the system cannot carry any CSI.
Hope these comments are helpful. I value your input greatly as your comments have stimulated me to think on these important issues - like you I am trying to brush up on my thermodynamics (statistical physics) - but it is presently hard to find the time to increase my own mental CSI! The law of increasing CI seems more operative .
-Chris
[ 06. November 2005, 20:54: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 23. November 2005 21:34
quote:
I wrote a few posts back:
Greetings gentleman.
I very much appreciate the last set of posts. I have been in a recent discussion with Perakh over at PandasThumb. And it helped clarify some of the issues.
It became apparent that Entropy has several meanings, and if this is clarified, it can help the discussion move forward.
Channel Entropy or Shannon Entropy
deals with the statistics of information flowing through a communication channel. The higher the entropy, the more information is potentially being pumped through. That is to say, high entropy is a very desirable characteristic in the case of communication channels.
It would not be really fair to say such a flow of information is "disordered" or "undesigned" or is "noisy", it simply has certain statistical properties. Were it not for the decoding systems at the other end of the communication channel, statistically we would be tempted to presume what was being pumped through the communication channel was not information, but rather noise -- note that modem signals or fax signals sound like "white noise". What delineates noise from information is the choice and perspective of the observer. This subjectivity of course is somewhat distasteful, but it does not make it scientifically wrong. After all, we are able to decode these signals into something meaningful even though, to the casual observer, modem signals sound just like the hissing white noise of tire being deflated.....
Thus in the communication context, shannon entropy does not mean "decay or disorder" in the sense that information is being lossed or scrambeld into oblivion.
Mathematically, however, the form of the equation looks very much like that seen in thermodynmaic entropy where high entropy is said to be a "disordered state". We have the situation were identical equations are applied to two philosophically different phenomena. In the case of shannon entropy it is the measure of how well designed a communication channel is in terms of bits flowing, and in the case of thermodynamics it is the measure of disorder. The mathematical similarities might lead one to try to formulate a law for both situations, and I'm not sure this will work, even though it seems very tempting given the mathematical similarities of entropy in both contexts!
Just to tie a loose end up, Yockey apparently concurs, page 30-32 of his latest book, Information Theory, Evolution, and the Origin of Life:
(Yockey refers to
as "Equation 5")
quote:
Many authors have been misled by the resemblance of Equation 5 to that for entropy in statistical mecahnics.....The function of entropy in both classical statistical mechanics and the von Neumann entropy of quantum statistical mechanics has the dimensions of the Boltzmann constant k and has to do with energy and momentum, not information ....Entropy in information and probability theory has no mechanical dimensions. There are no counterparts in communication theory to temperature, energy, pressure, work, or volume. There is, furthermore, no counterpart to the First law of Thermodynamics, namely, the conservation of the energy of a system.....
The great mathematician Norbert Wiener (1894-1964) regarded negative entropy as a measure of information (Wiener 1948). Simpson (1964) had the following remark:
quote:
A fully living system must be capable of energy conversion in such a way as to accumulat negentropy, that is, it must produce a less probable, less random organization of matter and must cause the increase of available energy in the local system rather than the decrease demanded in closed systems by the Second Law of Thermodynamics.
Brillouin (1953,1962) used the concept also to address certain problems in physics and 1990 to address the relation of thermodynamics and life.
Schrodinger used negative entropy to explain the appearnce of what he thought was order durig evolution. Eigen thought that information received is negative entropy. It is most unfortunate that these distinguished scientists have misled their readers. Perhaps they believe that the minus sign in Equation 5 means that Shannon entropy is negaitve entorpy. ....This is a serious mathematical objection to the ad hoc notion of negentropy in addition to the fact that a means of measurement has not been proposed.
These distinguished authors are confusing Shannon entropy of probability theory with Maxwell-Boltzmann-Gibbs entropy of statistical mechanics. Contrary to Schrodinger ...Weiner..Eigen..and a number of authors, whose name is Legion for they are many, Shannon entropy is not Negentropy. Life does not feed on negentropy (Pauling, 1987) as a cat laps up cream. The notion of negentropy has crept into textbook and technical and popular literature. It must be exorcised to avoid more damage.
Salvador
PS
Happy Thanksgiving everyone.
[ 23. November 2005, 21:44: Message edited by: Salvador T. Cordova ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 28. November 2005 20:26
Hi Salvador. I agree with much of what you say but I must challenge the view of there being no formal link between Shannon Entropy and Thermodynamic Entropy. I will start by focusing on our areas of agreement - build on these - and then make a case against Yockey's position (Incidentally I admire Yockey's work greatly but I do not think we must take it all uncritically - particularly since many equally great minds have thought and continue to think differently on this particular issue (moreover in his earlier book he did not take this stance)- however, I will defend the "common formal link" not with reference to others views but take a position based on rationale alone with the aim of developing better mutual understanding).
First no problem with the interpretation of Channel Entropy = Shannon Entropy as defined by:
where pi is the probability of the ith symbol. [Note here the minus sign is necessary for the purpose of making the Shannon Entropy positive (since pi <1 making the log negative)] n is the number of different symbols in the alphabet, ie x1, x2, x3 ...xn . Certainly I agree "the higher the entropy, the more information is potentially being pumped through" and "high entropy is very desirable" and necessary for high information throughput.
As an extension I want to point out that the above H(x) is the entropy per character (single element) of a string (i.e. it depends solely on the alphabet x rather than how many characters are transmitted). If we are communicating with a string N characters (elements) long then the Shannon Entropy of the string is:
where Pstring is the probability of getting the string by chance. This follows since the sequence will contain Npi of the ith symbol and because the probabilities of independent string members are multiplied [See maths page 23, Yockey].
The above allow us to see clearly the importance of the Shannon Entropy H(x) of the alphabet. For example if we are dealing with a binary alphabet then a string of N characters carries N bits of information. If on the other hand we are dealing with the amino acid alphabet (of 20) then N characters carries 4.32N bits. With the codon alphabet (of 64) then a string of N codons carries 6N bits. Thus the higher the Shannon Entropy of the channel the higher its intrinsic information carrying capacity.
Now lets step up the argument - when you say:
quote:
Were it not for the decoding systems at the other end of the communication channel, statistically we would be tempted to presume what was being pumped through the communication channel was not information, but rather noise -- note that modem signals or fax signals sound like "white noise".
I agree but you are clearly here referring to the pumping of CSI (complex specified information) through the channel. It could be a code for unlocking a lock or it could be a digitization of the weather conditions in Seattle, but the communication channel seems to be conveying humanly meaningful information (can you confirm?). What I want to point out is that in addition to the CSI string conveying meaningful information there are indeed a multitudinous number (~2^NH to be precise) of random strings that can also be sent down the channel that convey no information. Moreover these random strings will sound just as "white". Let me give an example of two strings:
[CSI>=00011110111001111001101110001111011100111100110111
and
[CI>=00110100100101011111000001100011011011011000101100
[Strictly speaking these strings are not "complex" because they contain less than 500bits of data - but you can imagine them to extend further if this worries you - the degree of complexity is not an issue here anyway]. The first of these strings is the output from the weather station in Seattle (i.e. is CSI), - the second string is just random noise (generated in this instance by my tossing a coin) - (i.e. it is only CI). The statistical properties of both strings are the same - they both belong to the same set of strings having the Shannon Entropy of 40bits (being both composed of 40 binary characters)
Thus when you say:
quote:
Thus in the communication context, shannon entropy does not mean "decay or disorder" in the sense that information is being lost or scrambled into oblivion.
I would certainly agree that Shannon entropy does not mean "decay" but I would strongly argue that it does mean "disorder". Indeed Shannon entropy (information) is a way of measuring (quantifying) the disorder. Look, I know that sounds an oxymoron - i.e. that Shannon information equates with disorder - but by definition it is such. Perhaps the reason that so many folk are confused is because this does sound so weird (information=disorder). The above [CSI> string from the Seattle weather station occurs with the same improbability Pstring =2^-40 as the [CI> string produced by a random number generator. Belonging to the same family they both share the same degree of randomness - its just that one family member [CSI> pumps humanly meaningful information on the weather conditions in Seattle while the remaining multitudinous family members do nothing!
Now lets go one stage further and apply this understanding to the thermodynamic situation: Take the case of an ideal gas of N atoms. Each atom can be characterized by its kinetic energy (it has direction as well - but from the point of view of entropy calculation we only need concern ourselves with the atoms energy). We want to know the macrostate of the gas at some time - what do we do - we punch a hole in the side of the gas container and let out the atoms one by one. As they come out we measure there energy and classify them accordingly into energy sub-bands (x1,x2,x3 .....xn). The set of energy measurements forms a string and this string has for a normal gas a high degree of randomness. How do we measure the degree of randomness - via the Shannon Entropy. The situation is completely analogous to the signal coming out of the Seattle weather station, except here we are not learning about the weather in Seattle but the exact state of the gas when we decided to read out the energy data. Incidentally the Shannon entropy of a gas in thermal equilibrium is given to a very good approximation by the Sakur-Tetrode equation:
Here S is the thermodynamic entropy, kB is Boltzmann's constant, c is the velocity of light, hbar is the Planck's constant divided by twice pi, M is the mass of a gas atom, T is the gas temperature and V is the gas volume. By comparison with the preceding equation it can be seen that the quantity inside the log2 is nothing other than the probability Pstring of getting a particular string of gas energies. The probability Pstring is excessively small because the channel alphabet (that depends on temperature) is not 2, 4 or even 6, but is around 250,000 (at room temperature and atmospheric pressure) corresponding to 18 bits of Shannon info per atom! Thus for a gas of only 40 atoms at room temperature and pressure we would expect a Shannon Entropy (i.e. randomness ![[Wink]](wink.gif) ) of 40x18=720bits. One may ask if this is CSI as with the Seattle weather signal? Well, I guess if one could in practice do such a read out of gas atom energies then yes - but in practice one cannot. It is only perhaps in thought experiments that one can imagine both feeding in (as in previous posts of mine) and feeding out CSI from a gas. But this fact should not let us lose sight of the fact that Shannon Entropy (i.e as per the Sakur-Tetrode equation) is measuring the degree of randomness of the gas. In the normal case when we have not constructed (fed-in) or fed-out the gas macrostate we must assume no knowledge of the exact gas state and must therefore take the Shannon Entropy to be just CI.
Hope we can discuss the above until we come to a common understanding. I would like to recommend a very excellent new text book Biological Physics by Phillip Nelson (chapter 6) that deals with Thermodynamic entropy from a foundation of Shannon entropy.
-Chris
[ 29. November 2005, 05:46: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Stephen Wright
Member
Member # 195
|
posted 01. May 2006 15:30
bump. this has too much good stuff not to be revisited.
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 04. May 2006 12:50
Stephen, I really appreciate your picking up this thread - perhaps we can push things a little? I see the 2nd and 4th laws as being closely related via information (probability) theory and the stochastic nature of all natural law. It is not that the 2nd subsumes the 4th or that the 4th subsumes the 2nd, but that a common theoretical structure subsumes them both. Neither are they the same thing - as some have supposed - for they say different things: I wonder if you can buy this?
2nd Law
"The amount of Complex Information (Shannon Entropy) required to describe the state of an isolated system will increase over time, approaching a maximum value"
4th Law :
"The amount of Specified Complex Information (CSI) contained in an isolated system will decrease over time, tending to zero"
Or since living systems are thermodynamically required to be "open systems" (to keep them at a fixed low entropy) perhaps the following statements would be more comprehensive and better. [Here the word "universe" is best understood as the "local universe" in the vicinity of the life one is considering 2nd Law: :
2nd Law :
"The amount of Complex Information (Shannon Entropy) required to describe the universe will increase over time or at best remain constant"
4th Law :
"The amount of Specified Complex Information (CSI) contained in the universe will decrease over time, or at best remain constant"
I am willing to try and elaborate or defend these statements. Personally I think the 4th law is so important. Because it is so often said that ID theory cannot be tested. How wrong - the 4th law suggests that bio-information (CSI) must come down with time - in strict contrast to the neo-Darwinian theory. Phylogenetic studies of the past are already showing bio-information coming down - supporting ID via the 4th law. Chris
[ 04. May 2006, 12:58: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 19. July 2006 02:40
Gentleman,
Please forgive my inattendance to this thread that I have started.
If I may state something relevant:
quote:
A magnetic diskette recording random bits versus one recording, say, the text of this book are thermodynamically equivalent from the vantage of the second law. yet from the vantage of the fourth law they are radically different.
Two Scrabble boards with Scrabble pieces covering identical squares are thermodynamically equivalent from the vantage of the second law. Yet from the vantage of the fourth law they can be radically different, one displaying a random arrangement of letters, and the other meaninful words and therefore CSI.
And most importantly:
quote:
the second law is subject to the Law of Conservation of Information [4th Law]
page 172-173, No Free Lunch
Thus, my orignal thesis was inaccurate, the above quote describes the proper relationship of the 4th law to the 2nd. The 2nd is not a special case of the 4th law, but the 2nd law is however constrained by the 4th law.
And finally, Perakh and Stenger were wrong saying the 4th is a special case of the 2nd.
Salvador
[ 21. July 2006, 13:46: Message edited by: Salvador T. Cordova ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 20. July 2006 05:43
Hi Salvador, quote:
A magnetic diskette recording random bits versus one recording, say, the text of this book are thermodynamically equivalent from the vantage of the second law. yet from the vantage of the fourth law they are radically different.
I agree because I know the context of this quote. One of the problems is that to most folk thermodynamics is to do with "heat", "energy" somehow in motion (dynamics). One may question where is the "heat" and "energy"? It is true that moving scrabble pieces around and writing to a magnetic tampe both require small packets of energy "per piece or bit" to physically place the letters [and in this sense they are thermodynamically equivalent]. This is not, I believe, the sense in which we refer to these cases as being "thermodynamically equivalent from the vantage of the 2nd law". Here we are dealing with the thermodynamic concept of entropy but with heat (or energy) being absent. We are dealing with the information theoretic concept of Shannon Entropy - which is simply the number of bits being conveyed - irrespective of semantic content.
Where then does "heat" come in? It comes in for many physical systems which are generally composed of particles (or their equivalent) in discrete energy states. In some earlier posts I pointed out that "gases" and other "physical systems" have effective alphabets of allowed energy states - the effective number of "energy letters" in the alphabet is temperature dependent (i.e. increases as we dump heat into the system). [This is the case for an ideal gas for example] Thus in the example of the gas - the gas entropy increases as we pump in energy. More bits of information are required to describe the state of each particle in a hot gas than a cold one! But in none of this (hot or cold) are we dealing with semantic information (CSI) - we are just dealing with complex information (without the specification).
So I would say scrabble pieces (or magnetic domains) do require small packets of information for their movement (switching) but unlike the normal thermodynamic situation of a gas - the effective alphabet is not in these cases temperature dependent. Thus although we are using the word "thermodynamic" - we have really extended the entropy concept beyond the world of heat (Agree?). It is the Shannon Entropy concept from information theory that is primary. [The application of Shannon Entropy to multi-particle systems giving the 2nd law of thermodynamics - is a specific application - but the one that bears the name "thermodynamics"].
To see this point - note that entropy increases in a gas as we pump in energy to it (increasing effective alphabet per particle). Entropy also increases in a game of scrabble if we either allow longer words or create more letters in the scrabble alphabet.
Perhaps I am being pedantic but I think it is good to be clear and make sure that we agree on this.
Chris
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 21. July 2006 13:56
Your last post was very much on target with my own thoughts. I have very little to add except just few technical points:
Dembski identifies 4 major aspects of information theory:
1. mereology
2. syntax (grammar)
3. semantics (meaning)
4. statistics
quote:
But in none of this (hot or cold) are we dealing with semantic information (CSI) - we are just dealing with complex information (without the specification
A slight correction, CSI deals with syntax, not semantics. This may sound surprising, but to illustrate, we knew heiroglyphics had syntax (and therefore design) long before we knew the meaning. The design inference through CSI can be made before even understanding meaning.
quote:
To see this point - note that entropy increases in a gas as we pump in energy to it (increasing effective alphabet per particle). Entropy also increases in a game of scrabble if we either allow longer words or create more letters in the scrabble alphabet.
Agreed, or alternatively, we can add more scrabble pieces, even though the 26 letter alphabet remains the same. But irrespective of how one describes it, I believe we are fundamentally in agreement!
quote:
Perhaps I am being pedantic but I think it is good to be clear and make sure that we agree on this.
Chris
Indeed, and thank you for your input.
Before I go on, as this thread has gone on for 102 posts, the question posed at the beginning, imho has been answered for me personally:
quote:
Is 2nd Law a special case of 4th Law?
No, but almost. Dembski, said it better:
quote:
the second law is subject to the Law of Conservation of Information [4th Law]
But, like suspecting a mathematical theorem is correct, actually going through the formality is another story (recall Fermat's theorem)!
Salvador
IP: Logged
|
|
Stephen Wright
Member
Member # 195
|
posted 25. July 2006 11:21
Salvador,
This has been a very profitable discussion. To comment on your last post – a referral to the dictionary was needed to discern the meaning of merelogy; as the relationship of the parts to the whole. I have not run into it as a formal component of information theory before --- but surely think it is essential. In the post to andyg, on May 27th, – I tried to address just such a concept, with a quotation from Douglas Burnham regarding the idea of informational wholeness (past, present and future) in the monads of G. Leibniz.
Can you reveal more of the role of merelogy in Dembski’s conceptualization of CSI?
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 25. July 2006 20:50
Hi Stephen,
Mereology is indeed an obscure word which Dembski did not elaborate on in his book. I don't think is yet of major importance (but maybe one day).
The surprising fact is that we affix symbolic significance to so many things. A diamond engagment ring, drops of ink on paper (printing), pixels on a screen being pictures....
We don't even realize when we look at photo on a CRT we actually think of what pixels symbolize rather than tinking of it as an X-Y grid with color values affixed to each position (which is what a CRT does).
At some point we choose to project our mental pre-conceptions on a physical objection, and associate symbols to it.
Even on a more basic level, in science we classify objects and call one object an atom. In doing so we have made a mereological step.
I suppose it's not a huge issue because, without the assumption of practicing some level of mereology, there would be no science!
Salvador
IP: Logged
|
|
|
Printer-friendly view of this topic
