|
Author
|
Topic: Is 2nd Law a special case of 4th Law?
|
Stephen Wright
Member
Member # 195
|
posted 19. July 2005 13:39
quote:
Salvador T. Cordova wrote, “Dembski's formulation of the Fourth Law is tied to his formulation of CSI. If we relax the requirement that the Fourth Law to not be solely CSI but simply "I" (for information), then I think we're in a position to have a unified view of the 2nd law being a special case of the Fourth Law.”
The 2nd law is an empirically based relationship that can be expressed in physical units. There is a fundamental difference in the basis for thermodynamic units and negentropic units of communication theory (logical entropy). Shannon information, “I”, is expressed in virtual units as bits/bytes. We know about the capability for thermodynamic work output from inductive analysis of data. Hence, it may be a problematic to directly correlate the two measures of “information” together, especially with one being a subset of the other. Salvador, please correct me if I have this wrong in the context of what you are discussing. Brig Klyce has a good review of the difference - www.panspermia.org/seconlaw.htm
The assignment of CSI being naturally caused or not, is unclear for me as well. My own worldview would address computation that creates the design parameters behind specified complexity, as mental work output and the mind as causal in this virtual process. Thus, the normative manipulation of “I” information, into organized logical structures, is as natural as the organization of matter into mechanically advantageous machinery. Thinking in terms of Wheeler’s “It from Bit” -- the idea that virtual information is just “abstract” and therefore not ontologically substantial, may be an outmoded paradigm. Mental work can create organization in a virtual state. It would appear to this writer that virtual organization, as design, subsumes any physical manifestation intentionally modeled on its structure.
This Brainstorm is still a very profitable exploration. In a naïve overview - specified complexity as per L. Orgel, P. Davies, W. Dembski; is a terrific insight. There appears to be good reason to consider a generalized category for any number of types of information, with their internal logic as the common property. Evaluating them as indirectly related, as structured “I” information, could reveal how they have correlated properties. These organized internal structured relations, in a virtual state, can be seen in terms of their specific contexts and of how they can be measured in like terms. The units of any measurement must be appropriate to discrete environments and the applicable standards. Seeing the structural relationships in a virtual state, as real as they are in a physical state, I know is not standard fare.
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 21. July 2005 15:20
quote:
Stephen wrote:
The 2nd law is an empirically based relationship that can be expressed in physical units. There is a fundamental difference in the basis for thermodynamic units and negentropic units of communication theory (logical entropy). Shannon information, “I”, is expressed in virtual units as bits/bytes. We know about the capability for thermodynamic work output from inductive analysis of data. Hence, it may be a problematic to directly correlate the two measures of “information” together, especially with one being a subset of the other. Salvador, please correct me if I have this wrong in the context of what you are discussing. Brig Klyce has a good review of the difference - www.panspermia.org/seconlaw.htm
Hi Stephen,
I'm delighted to hear from you. The concept of "orderliness" is itself a specification, although we usually do view it as such. When there is an increase in entropy there is a measurable amount of divergences from this "specification". The specificaitons, are sometimes formally expressed in terms of Kolomogorov complexity (or simplicity depending on how one looks at it).
I can heat an object and thus make the object diverge from the specification of it's original orderliness. Cooling it will restore the orderliness in many but not all cases.
With regard to communication, a scrabble letter message can have entropy induced to the information bearing symbols by say just shaking the scrabble board and letting the letters spill. There is not really any "cooling" process which will restore the original order.
So that is a little comparison contrast between two situations, where both the thermodynamic case and the scrabble letter case can arguably have their entropy described in terms of some sort of specification. The situations have similarities, yet some very clear differences in the way we interpret "entropy".
A possible goal is to be able to unify these rather diverse definitions of "entropy" with some kind of law. Were each is a special case of that general law. I do not know if it can be done, but I think it is worth exploring.
The major hurdle is being able to deal with the cases where entropy is reduced via a "cooling" process. One way to possibly resolve it is to show that "cooling" results in an increase of entropy elsewhere. That is true in the thermal sense, but can we extend that to other realms such as information theory?
Does a modem doing error correction on an electrical signal, necessarily mean infromation decays elsewhere? That seems to be a claim of the 4th law, however actual measurement of this would be tough, since there is no uniform method of affixing symbolic significance to objects (for example what is information to one person is noise to another).....
I think it is possible to phrase all forms (and there are many) of the 2nd law as a special case of the 4th law, however, in order to do this, I think one would have to consider relaxing the requirement that 4th law deals with CSI only but rather many other kinds of information as well.
Salvador
[ 21. July 2005, 15:22: Message edited by: Salvador T. Cordova ]
IP: Logged
|
|
Stephen Wright
Member
Member # 195
|
posted 28. July 2005 13:32
quote:
“The major hurdle is being able to deal with the cases where entropy is reduced via a "cooling" process. One way to possibly resolve it is to show that "cooling" results in an increase of entropy elsewhere. That is true in the thermal sense, but can we extend that to other realms such as information theory?” - Salvador
“Cooling” seems to be inappropriate to information, due to the problems pointed out in the previous post. Offering a blue-sky opinion – I would point to the “reset” command, where information is cleared from memory - as having an asymptotic functionality similar to thermal cooling. Rolf Landauer wrote very considered opinion on this subject. It does take energy to “forget” and refocus back to a starting state.
quote:
Does a modem doing error correction on an electrical signal, necessarily mean information decays elsewhere? That seems to be a claim of the 4th law, however actual measurement of this would be tough, since there is no uniform method of affixing symbolic significance to objects (for example what is information to one person is noise to another)..... – Salvador
But the point is this - why is it that deterministic equations leading from state of affairs "i" in a gas to state of affairs "j" generally reveal information loss. Why does the entropy go up!!?? Why are things different in biotic systems where CSI is probably exactly conserved except for the degenative action due to the 2nd law on the CSI storage medium (DNA)?. - Christopher
I really want to say something about these questions, but I think that an examination of the 1st law and a possible analogous version in the virtual domain, as a law of conservation of information, would be needed background. I am not a sophisticated guy, so going forward by the numbers would help me understand your position and make my own in a comprehensible fashion.
How would you describe a 1st law where information is conserved? And is there real acceptance of it yet?
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 16. August 2005 03:41
quote:
I really want to say something about these questions, but I think that an examination of the 1st law and a possible analogous version in the virtual domain, as a law of conservation of information, would be needed background. I am not a sophisticated guy, so going forward by the numbers would help me understand your position and make my own in a comprehensible fashion.
How would you describe a 1st law where information is conserved? And is there real acceptance of it yet?
I'm sorry for my delay in responding, I have been pondering these issues, and I must admit I'm a little bit stuck.
There are two issues:
1. Is stenger and perakh's claim that the 4th law is a special case of the 2nd law correct? I think that can be demonstrated to be false.
2. Do they actually have it backward, where the 2nd law is a special case of the 4th law? I don't know.
I think #1 is actually doable, #2 I'm not at the point that I can say.
The heartening thing is that all of physics (because of Quantum Mechanics) is slowly going foward as formulations of information theory. Thus all laws of physics might one day be expressed in terms of information theory (von Baeyer, John Wheeler) have that view....
Regarding the 1st law, I don't think it can be absorbed into the 4th law easily. The 1st law being the conservation of energy.
I'm sorry I don't have more complete answers. I'm fishing around for answers myself.
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 24. August 2005 01:52
quote:
So that is a little comparison contrast between two situations, where both the thermodynamic case and the scrabble letter case can arguably have their entropy described in terms of some sort of specification. The situations have similarities, yet some very clear differences in the way we interpret "entropy".
A possible goal is to be able to unify these rather diverse definitions of "entropy" with some kind of law. Were each is a special case of that general law. I do not know if it can be done, but I think it is worth exploring. - Salvador
Hi Salvador and Stephen,
Wishing to stimulate the discussion I put before you the following thought experiment in order to make an attempt to "unify these rather diverse definitions of "entropy"".
Let there exist (1) a protein with N amino acid (AA) residues that has some specified function within a cell. The probability of the ith AA occuring by chance is p(i). The information content of this protein is given by (Yockey's latest book-p28)
I=NH =N{Sum,i=1 to 20,of -[p(i)log p(i)]}
H being the Shannon Entropy of the sequence. If the log is base 2 the answer will be in bits.
Let there also exist (2) a spin gas where there are N atoms has a spin of J=(19/2). This value of J is chosen so that there will be 2J+1 = 20 magnetic substates in which an atom can exist {20 is chosen to make a direct analogue with the AA case}. We apply a magnetic field to the gas so that each atom exists in one of the 20 substates of (uniformly spaced) energies Ei. At a fixed temperature T of the gas the probability of an atom being in the ith state will be given by the Maxwell Boltzmann factor p(i)=Exp(-Ei/kT)/Z. This probability set is the one that maximizes the Shannon entropy H of the gas which is defined with respect to the thermodynamic entropy S as follows:
(S/k) =NH =N{Sum,i=1 to 20, of -[p(i)log p(i)]}
- essentially the same expression as in case 1. In a protein AA sequence we are familiar with the idea of a sequence because the AA connect in a line!- but what of the sequence in the case of the spin gas?
no problem - one can perform the thought experiment where one just "walks" through the N atoms noting the substate of each atom - but not double counting. (Alternatively one could let each atom out of a hole in the gas container one at at time and measure its spin state!). i.e. one may get a sequence something like:
[D> = 5,15,2, 3, 1, 1, 6,14, 2, etc etc
Thus we conclude that the Shannon Entropy H as a measure of information in the "sequence of events" exists and is a meaningful concept for both cases 1 and 2. Note that the order of the "events" (AA or atoms) is not important as regards the Shannon Entropy Information.
HOWEVER, the 2nd law applies to (2) but not to (1) (for which 4th law applies). That is IF we imagine starting the gas from a cooled state (absolute zero) where all the spins are in the ground state, i.e. we have the ordered state
[O>= 1, 1, 1, 1, 1, 1, 1, etc
and then warm the gas up we find that the state [O> does not conserve - it is a low entropy state, and quickly moves to a state representing higher Shannon Entropy (such as [D>). As I say this is in contrast to the protein which if formed in a state of CSI where the protein sequence is functionally specific - the specification will conserve (4th law).
WHY is this the way things are? What is difference between the two cases? Dembski's arguments in NFL about the conservation of information apply in both cases -i.e. no deterministic law of stochastic law can increase information (reduce probability). In case 1 the protein information is conserved. In the case 2 - it looks as if there has been an increase the Shannon entropy information!!? - but this cannot be - this state has been produced stochastically and is not CSI [Most thermodynamics books here say that the information on the state of the gas has decreased - which in a sense it has ![[Confused]](confused.gif) ]. I do not believe the total Shannon Entropy of Gas has gone up at all - we must surely not forget that there are a huge number of such "high information" configurations - so that really there is really very little Shannon information if any in the gas.
1ST LAW: With regard to the 1st law - the conservation of energy drops out of the Time Dependent Shrodinger equation when one adds in the time invarience (symmetry) of the physical laws. That might be good enough for most folk, except there may be some greater depth - e.g. why is the SE like this? There is also a related fact that when a system potential changes quickly it induces for a short time a break in energy conservation (i.e. decay of an atom). Conservation of energy only works on long timescales. It could be that the information of a particle state (as for example given by the wavefunction) is not conserved for short times. I agree with Salvador that it is interesting that information theory concepts are coming into quantum mechanics.
Also an interesting quote from Yockey (2005, p31) quote:
There are no counterparts in communication theory to temperature, energy, pressure, work or volume. There is, furthermore, no counterpart to the 1st law of thermodynamics, namely, the conservation of the energy of a system
[ 27. August 2005, 07:43: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 31. August 2005 22:19
Greetings gentleman.
I very much appreciate the last set of posts. I have been in a recent discussion with Perakh over at PandasThumb. And it helped clarify some of the issues.
It became apparent that Entropy has several meanings, and if this is clarified, it can help the discussion move forward.
Channel Entropy or Shannon Entropy
deals with the statistics of information flowing through a communication channel. The higher the entropy, the more information is potentially being pumped through. That is to say, high entropy is a very desirable characteristic in the case of communication channels.
It would not be really fair to say such a flow of information is "disordered" or "undesigned" or is "noisy", it simply has certain statistical properties. Were it not for the decoding systems at the other end of the communication channel, statistically we would be tempted to presume what was being pumped through the communication channel was not information, but rather noise -- note that modem signals or fax signals sound like "white noise". What delineates noise from information is the choice and perspective of the observer. This subjectivity of course is somewhat distasteful, but it does not make it scientifically wrong. After all, we are able to decode these signals into something meaningful even though, to the casual observer, modem signals sound just like the hissing white noise of tire being deflated.....
Thus in the communication context, shannon entropy does not mean "decay or disorder" in the sense that information is being lossed or scrambeld into oblivion.
Mathematically, however, the form of the equation looks very much like that seen in thermodynmaic entropy where high entropy is said to be a "disordered state". We have the situation were identical equations are applied to two philosophically different phenomena. In the case of shannon entropy it is the measure of how well designed a communication channel is in terms of bits flowing, and in the case of thermodynamics it is the measure of disorder. The mathematical similarities might lead one to try to formulate a law for both situations, and I'm not sure this will work, even though it seems very tempting given the mathematical similarities of entropy in both contexts!
In order to make the 2nd law a special case or some sort of corrolary to the fourth law, another step must be taken. What that step is, I don't know yet, and hence my attempt at a "brainstorm".
But let me back up a bit. The shannon entropy is one form of entropy in information theory. It does not directly deal with the concept of information decay, as in noise propagating through a system and eroding the existing information. Information decay is yet another kind of entropy and I belive that is entropy regarding the 4th law.
We thus have 3 kinds of entropy:
1. Thermodynamic (2nd law)
2. Shannon Entropy (information theory, channel capacity)
3. Information Decay (4th law)
It is tempting to start from #2 and try to derive #3. I don't think that's the right direction.
I think one might try starting from #3 and try to show that #1 (2nd law) is a special case or a consequence of #3.
Deriving #2 from #3 might be kind of awkward because #2 does not really even deal with information decay, but who knows, it might be possible.
I think the citation of Yockey is wonderful, and I'll have to go back to his book to get some ideas.
Sorry for the slowness of this discussion, but I think some issues have been clarified in the course of this thread.
Salvador
[ 19. July 2006, 00:36: Message edited by: Salvador T. Cordova ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 01. September 2005 20:31
quote:
Mathematically, however, the form of the equation looks very much like that seen in thermodynmaic entropy where high entropy is said to be a "disordered state". We have the situation were identical equations are applied to two philosophically different phenomena.
Yes! It seems that we guys are slowly and perhaps painfully rediscovering this. It seems to have been known about for some time. Quoting from John Avery's book "Information Theory and Evolution" p81
quote:
When Shannon had been working on his equations for some time, he happened to visit the mathematician John von Neumann, who asked him how he was getting on with his theory of missing information. Shannon replied that the theory was in excellent shape, except that he needed a good name for "missing information". "Why don't you call it entropy?" von Neumann suggested. "In the first place, a mathematical development very much like yours already exists in Boltzmann's statistical mechanics, and in the second place, no one understands entropy very well, so in any discussion you will be in a position of advantage!"
I guess this discussion took place in the 40s. Do we thank von Neumann or not? Well I believe we should. Although we are dealing with two philosophically different phenomenon - they are not disconnected. By thinking about the similarities and differences I believe there are things to learn. Apart from sharing the same formula for entropy - the following seems important
In both cases - as per Dembski's treatment sec 3.8 of NFL - the initial state of affairs "A" (be it a gas state or a coded message) always connects via a communication channel to a final state of affairs "B" (gas state of message). In the case of the gas - the communication channel is simply the deterministic (classical gas) or stochastic (quantum gas)law of nature. From this it follows that in both cases we have the common laws:
I(A&B) = I(A) + I(B!A) Eqn (*) p159 in NFL
that gives the SUMMED information between initial and final states, and we also have
I(A;B) = I(A) - I(A!B) Eqn (5.2) p46 - Yockey's recent book
that gives the MUTUAL (shared) information between intial and final states. Both these equations deal with noisy channels - which I believe we have in both biotic and gas evolving systems. In the case of biotic - the noise is in the form of point DNA mutations, and in the gas - by atom collisions. The time-scales are different but the overal theoretic is common/similar.
[Incidentally we are using the "I" symbol in the above for information - but the "H" symbol could also be used for "Shannon information"].
quote:
Deriving #2 from #3 might be kind of awkward because #2 does not really even deal with information decay, but who knows, it might be possible.
I think that #2 and #3 belong to the same picture. Shannon's information theory does deal with NOISY channels of communication - that means information loss (decay?). This information loss is expressed by the fact that the mutual Shannon information I(A;B) is always on the decrease.
[ 01. September 2005, 21:17: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Stephen Wright
Member
Member # 195
|
posted 02. September 2005 17:29
Christopher and Salvador,
Following is an attempt to make a case for the Thermodynamic Laws to have analogous partners existing as Information Laws. While they can’t be converted or reduced one to another, there is good reason to see them as parallel and intimately connected. If one views the nature of material results from physical interactions and acknowledges that the changes proceed according to the material properties; which can be modeled by virtual representation, then parallel versions manifestly exist. Empirical events are accessible for analysis in a local environment via observation and physical measurement. Likewise in an informational format there exists a potential to mathematically and logically define the properties and create a virtual model of events.
Pragmatically this methodology of technology is in place today. Computer simulations of processes have become an essential tool of science. Further, results from such simulations correlate with physical experience very well, especially when the properties are carefully defined and calibrated to the specific circumstance. The conjecture offered in this post is that it may be profitable to examine holistic reality as having two levels or domains of observables - physically observable and virtually observable.
An example of the pragmatic acceptance of this is a digital surveillance camera recording a crime. The unadulterated data set from the recording is sufficient as evidence as would be actual observation. Maybe better, because there is a tacit understanding that the human eye’s observation, recovered as memory, is error prone. Further, viewing the virtual data gives the impression of real time observation rather completely. Even if the model of the crime is created from physical evidence as a logical reenactment or mock-up of events – this virtual model is very well received as revealing true information on which to dispense justice.
Presumption of the identity of virtual information with physical reality being as good (or better) than physical experience is the basis for believing in the logic of mathematics as the useful tool to work with universal properties describing matter and energy. Tacitly it is accepted that math results from the received properties is as true as our witness. Calculations from confirmed properties and their assigned constant values are taken as truth, as much as direct measurements of mass or energy.
With this being fact, then it should be supported to see virtual information as real as manifest matter. This position, for me, is Informational Realism. The tacit belief in computed results being considered as truthful as manifest results is pragmatically confirmed as acceptable technology behavior. There should be a one to one matching between physical reality and a correct informational model of it. Therefore if matter/energy is conserved so should the virtual information version that models it.
Science can have separate but equal methodological practices whereby physical events are watched for transformation through reductive properties of chemistry and physics and virtual events are viewed as transformed through reductive properties of statistical mechanics, communication theory and logic. Further, like depth perception coming from two eyes, comparing the transforms at any time gives insight into the holistic nature of a particular phenomenon.
Structure, state and connectivity for logical interaction should all be properties of virtual information.
During the online ISCID chat with William Dembski, it was asked if he thought information was real and had structure. He offered a strong positive response about it being real – but did not comment on the idea that it may have structure or organized retainable characteristics. This structural aspect to information is asserted in this conceptualization of the first law of information. This conjecture places value in perceiving information as having a property of structured constitution. It is exactly this logical relationship of virtual form and state that is conserved or converted to a new form through consistent principles.
Here is my stab at three types of entropy -
Thermodynamic entropy – lack of structure or organization in matter that yields energy for work.
Channel (Shannon) entropy - lack of structure or organization in virtual information that can be decoded as certainty at the receiver.
Logical entropy – lack of structure or organized internal relations of interacting component parts yielding overall potential output.
[ 02. September 2005, 17:34: Message edited by: Stephen Wright ]
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 08. September 2005 18:38
Christopher and Stephen,
Thank you so much for your insightful comments. Very much appreciated.
Let me throw out some calculations as it might perhaps enlighten us to Shannon Entropy or what has been sometimes called Channel Entropy.
Let us calculate the shannon entropy of a single fair coin used as a communication channel.
The coin has a 50% probability of being head and 50% being tails.
Using the above formula the entropy is:
- [ .5 log2 (.5) + .5 log2 (.5) ] = 1 bit
Similarly consider the channel entropy provided by 2 coins. The possible configurations are
H H
H T
T H
T T
Each configuration has a 25% probability of occuring, the channel entropy is:
- [ .25 log2 (.25) + .25 log2 (.25) +.25 log2 (.25) + .25 log2 (.25) ] = 2 bits
likewise we can go to 3 coins and the entropy would be 3 bits, 4 coins would have an entropy of 4 bits, etc.
What I wished to show here is that the shannon entropy for the channel in this sense has some independence from what is actually transmitteed through the channel.
For example I might be able to transmit the contents of a file through a 1 bit channel by using a single coin that was flipped to the proper bit every second.
I'm not yet trying to draw any inferences from what I just stated, except to show how delicate these issues are. I wish I could draw an inference at this point, but I'm simply pointing out something to be considered.
Salvador
[ 19. July 2006, 00:35: Message edited by: Salvador T. Cordova ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 08. September 2005 20:58
I agree. The formula for the Shannon entropy - as quoted - contains only the probabilities p(i). The total information transmitted will just depend on the number of elements in the string N and this set of probabilities {p(i)}. The total information conveyed is just NH.
I think your concern is that it seems that the Shannon entropy information is the same irrespective of the semantic content. To some extent this is true - Shannon Info is just a measure of sequence complexity (improbability) and does not touch on its specificity.
Certainly in the example you give - where all the probabilities are the same (By definition you set p(0)=p(1)=0.5) - all sequences will have equal Shannon information. It does not have to be so though. For example in an English message such as the one I am typing some letters are more common (probable) than others. So to get the Shannon information content on that last sentence one would have to have some reference to the set {p(i)} for the 26 letters in the alphabet. This could come from a number of sources - the best perhaps being to take an average over all the english books that have been written say in the last few years. Basically some letters are less probable such as a "z". If the sentence comprized of mainly "z" characters it would be a significantly less probable sequence - i.e. one with higher entropy. [Remember that the total entropy NH = -logP, where P is the total probability of the string based on CHANCE alone]
In the case of messages on DNA or Protein sequences, while the probability of an A,T,C or G on the DNA might all be the same based on a-prior reasoning (i.e. p(A)=P(T)=p(C)=p(G)=0.25) the same cannot be said of the AA(Amino Acid) sequence of a protein. Some codons (and hence AAs) are more likely than others and some AAs have more codons than others. This is spelled out in Yockey's book where he uses Markov shuffling to estimate the {p(i)} for the AAs alphabet of 20. So some proteins may contain more Shannon information if they happen to contain some "rare" AAs.
We must also consider the effect of channel NOISE - in any communication of information. Take your 1 bit/sec example. You transfer your prescribed binary message of say 1000 bits. The problem is that you would in fact get ~50% of this message correct (by accident if you like to think that way) on a completely noisy (random) channel. Your information transfer is NOT as one might first suppose 1000bits-500bits=500bits;- it is in fact ZERO. In order to quantify the amount of correct transmission "Hcor" on a noisy channel one must use:
Hcor= H(A ; B) = H(A)-H(A l B) (bits/symbol)
where A is the message sent and B is the message received and H(A l B) is referred to as the "conditional entropy" - given by:
p( i l j ) being the conditional probability of having had i transmitted if j is received.
For example if the channel is only 1% noisy [i.e. if on transmitting a 0 there is a 99% chance of receiving the 0 (correct transmission) and a 1% chance of getting a 1 (incorrect transmission)] then:
H(A l B)=-[0.99 log2( 0.99 ) + 0.01 log2( 0.01 )]=0.081 bits per symbol
The total transmitted information "NHcor" is then 1000 - 81 = 919 bits.
Now lets revisit the 100% noisy channel [i.e. on transmitting a 0 there is only 50% chance or receiving the 0 and 50% of getting a 1] then:
H(A l B)=-[0.5 log2( 0.5 ) + 0.5 log2( 0.5 )] = 1 bit per symbol
The total transmitted information "NHcor" is now 1000-1000 bits = 0
which makes intuitive sense. [This example can be found in slightly different form on p44/45 of Yockey's book, which he in turn borrowed from Shannon's 1948 paper!]
The real data rate "Hcor", or total transmitted information "NHcor" (which is known the MUTUAL (shared) ENTROPY, H(A;B), between A and B sequences) is I believe important for our use of information concepts in understanding the 2nd law. Take for example my example from previous posts of the spin gas with 20 substates. We could introduce (by thought experiment - and maybe in reality using a Stern Gerlach apparatus) 1000 atoms into a container - all in their lowest energy substate. i.e. the initial state of the gas would be:
[I>= 1,1,1,1,1,1,1,1,1, ......1
However this state will not maintain for long. Both atomic collisions and atom's propensisty to receive photons from the container's wall will quickly randomize the sequence. Some time latter we may have atoms in the configuration
[M>=4,1,2,1,1,10,1,1,1....
that still contains some of the initial information (i.e. the Mutual entropy of the sequence NH(A;B)is non-zero). After a long time (compared to the atomic transition/collision rate) some totally randomized sequence [F> would exist. It might look something like
[F>=11,2,14,10,5,17,12,2,20,......
For [F> there would be absolutely ZERO "Mutual" entropy between itself and the initial state [I>. The "mutual entropy" has exponentially decayed to zero, but the "actual entropy" as given by H([F>) has increased to a maximum (2nd law).
Now the observed increase in actual entropy is as anticipated by the law of increasing information as written by Dembski (p129/p159 of NFL)
H(I&F)=H(I)+H(F l I)
which applies to chance and stochastic processes - such as we have here in the gas. We have more Shannon information in the final state [F> than in the intial state [I> by virtue of the second term. Hope you find this is helpful
Christopher
[ 10. September 2005, 01:53: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 09. September 2005 23:05
Stephen, I am not a trained philosopher of science and as such have been struggling with some of your post. As such I thought it helpful to display my understanding (right or wrong) in diagramatic form and let you and others comment - to make sure everyone is on the same path. [Incidentally - the big vertical downward arrow in the diagram represents TIME]. Basically you see the diagram affirms your distinguishing between "physical reality" and "virtual realities". I agree with the quote quote:
While they can't be converted or reduced to one another, there is good reason to see them as parallel and intimately connected
Question: Surely all along science has been in the business of formulating theory - isn't the "vitual reality" we speak of here just another name for theory? I wonder whether to you virtual reality is a more restricted sense of theory - which deals with a kind of subset of theory that is totally information based?
My diagram shows a distinction between fully deterministic and stochastic processes - a distinction that I believe must be made when talking about information (and the connection between the physical and virtual). All deterministic processes are in RED, while stochastic processes are in BLUE.
In classical physics (or macroscopic object physics) - on has a one-to-one mapping
(i) in going from the physical state x to the virtual j=e(x) "the operation (mapping?) e standing for experimental empirical observation". Note here that the observer is responsible for the "meshing" in virtual space - it will be determined for the most part by the accuracy with which a measurement (say of position) can be made.
(ii) in going from the state i to new state j in the virtual configuration space some mathematical function f(i) is used for a one-to-one mapping ( function f represents the theory).
In microscopic/quantum physics one no longer has the one-to-one mapping - things are stochastic in nature (at least operationally). This is seen both in
(i) the way finding the state of a particle in the physical realm ends us up with only a probabilistic answer - where the configuration of the particle in virtual space can only given by a probability density. - the scatter of the blue arrows is meant to represent this.
(ii) the way in which a specific intitial particle (or particle system) configuration i in virtual space maps probabilistically onto many possible final state configurations j (represented again by blue lines)
As per the discussion of Bill Dembski in secs 3.8 and 3.9 of his book "No Free Lunch" it is important to distinguish between deterministic (in the classical sense) and stochastic (in the quantum - microscopic sense) - because the informational laws are different. For deterministic law in the virtual realm one has the Law of Conservation of Information
I( i & j )=I( i ) ________(Eqn:1)
On the otherhand for the case of the stochastic process one has the Law of Information Increase
I( i & j )=I( i )+I( j l i ) _________ (Eqn:2)
One has two different structures, although it would seem that the Law of Information Increase was more generally true - because classical determinism is known to be only an ideal. Interestingly the 4th law of thermodynamics is based on determinism - i.e. eqn (1), while the 2nd law is based on stochasticism - i.e. eqn (2).
quote:
Tacitly it is accepted that math results from the received properties is as true as our witness
Indeed because classical experiments find that e(x')=f(e(x))
if not the theory would be discarded.
When dealing with the microscopic (classical and quantum), however, the positions and momenta of individual particles are unknown [In the classical case this is pragmatic - one has no way of instantaneously knowing all position and momenta from experiment] - only probability (density) distributions can be specified in the virtual realm - and only ensemble averages calculated to map back onto the physical realm. i.e. we only have "true to our witness" working in a statistically averaged sense. Such is fine and acceptible for most working scientists, but one wonders where the informational laws (1) and (2) fit in here? Thermodyanmic laws do just relate to systems of many particles.
Two questions - (i) What do you mean by "the first law of information"? - Eqn 1, 2 or both? and (ii) What do you mean "Logical Entropy" - could you say more about this concept - Christopher
[ 14. September 2005, 20:59: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Stephen Wright
Member
Member # 195
|
posted 14. September 2005 13:28
Christopher,
Thanks for your interest in sharing ideas. First, let me say that your diagram puts “meat on the bones” of our discussions and appears to me quite serviceable. Let me try to answer your questions and provide a little background for my viewpoint. This view would be grounded, by being constrained in the formal math of information theory, but is also speculative as it rests on the idea that both empirical facts and the virtual potentialities predicted by received theory, are substantial and real. I think we detect evidence of this in the boundary layer between the probable and manifestation as superposition.
quote:
From “Cybernetics and the Philosophy of Mind” – Sayre, Kenneth (Routledge & Kegan Paul, London, 1976). Excerpt from page xii of the preface:
“In the text below the concept of information is explicated in formal mathematics before being deployed in the analysis of other concepts, and the term “information” is used only in the senses that have been explicitly defined. Since information can be defined in this fashion, although fundamental it is not a primitive concept. The primitive concept in treatment is that of probability, needed to interpret the formal definition of information.”
K. Sayre of Notre Dame is the scholar who coined the term - Informational Realism.
You ask whether virtual reality is just another name for theory. Yes, I think so, in its most intelligent guise. I suggest virtual reality can be expanded to any modeled understanding that is derived from a correct judgment about existential environmental conditions. The key driver is the intent to know an answer – in response to an active problem. Any focused understanding of an intelligent agent, which is useful mutual information about its environment, is in line with the diagram. I would suggest that even animal cunning and instinct are also described by it quite well.
Further if “e”, as experimental empirical observation, is replaced by the kinetic energy of a material system - your diagram still holds. The universe had virtual information, both deterministic and stochastic, before mental processes started. This information is at the level of objective facts, rather than knowledge or understandings where an agent has created internally stored mutual information from its surroundings. Please comment whether you think this makes sense. I would see the creation of a solar system to be modeled in this way –from natural vectors without need for computation or real time design.
Addressing the questions about a first law of information (in my humble intuition of it) – yes, it would include both deterministic and stochastic structures. Hence, the two equations you presented would apply. This conjectural version of the 1st information law would in essence be asymptotic but generally isomorphic with the 1st law of thermodynamics.
It would match the surety of transformation that we have in physics and chemistry with information being discrete and conserved. Detailing the transformation of information, as coming from prior structures, would be as clear as in the materialistic domain. There would be closure in information theory’s own terms and no spontaneous generation of complexity without due computation or an actual emergence from physical events. Mental work would account for the organized structures that are created.
Logical entropy appears as a term in the literature expressly to distinguish it from thermodynamic entropy. It is virtual and not physically measurable – as in the internal logic of a design or manifest organization as the result of a concept. Shannon correctly excludes meaning from his quantification of information in a communication channel. However, the internal meaning of something like an object-oriented program is surely a part of understanding the science of information. An OOP’s internal logic is a powerful construct of the modern day. I offer these insightful comments, again from K. Sayre. I would promote the idea that logical entropy describes the lack of order in relevant meaning and a virtual partner to channel (Shannon) entropy.
quote:
“John von Neumann told Carnap and Bar-Hillel (Bar-Hillel, 1964, p. 12) as remarking, in the presence of Carnap and Bar-Hillel, that logic forms a triple identity with thermodynamics and information theory. Allowing for room for uncertainty about the degree of seriousness in this remark, we see that it has a basis of credibility. Whereas information and energy are forms of negentropy deriving from and convertible into structure, applied formal reasoning itself increases the structure of a conceptual network by separating the conceptual from the observational component. But separation of any sort involves an expenditure of energy (chapter III). Thus logic, like thermodynamics and communication theory, pertains to transformations of negentropy into one of its alternate forms – energy, structure and information.” Ibid, p. 226.
[ 14. September 2005, 13:48: Message edited by: Stephen Wright ]
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 19. September 2005 10:50
Stephen, I am glad that you like the diagram. I understand most of your comments but not all.
quote:
but is also speculative as it rests on the idea that both empirical facts and the virtual potentialities predicted by received theory, are substantial and real.
I would have thought that an empirical fact must be substantial and real - if it is an observation we talk of - well thats my nuts and bolts understanding. But I guess what you are hinting at is something that has bugged philosophers for a long time on what is more real - the physical nature or the mathematics that describes them in our minds. I quote from Gilles Chatelet in his book Science and Philosophy p17
quote:
It is not easy to weigh in a single balance a 'real' existence, which is independent of us, but mobile and corruptible, and an intelligible matter, which exists only by proxy through the wit of the geometer. How do we choose between precedence in the order of Being (which physical natures can lay claim to ) and logical precedence (which mathematical natures lay claim to). We know how Aristotle reconciles the two rivals: by subordinating them to metaphysics - the first philosophy - whose objective is the theory of the immobile and real being, immutible substance
So today have we got beyond Aristotle? For myself I admit to largely agreeing with Aristotle, seeing the resonance between the virtual and physical realms as stemming from the mind of the Designer - our minds being capable of conceptualizing and modelling the physical simply because they are derivitive from The Mind.
quote:
(Information) although fundamental is not a primitive concept. The primitive concept in treatment is that of probability
I tend to agree - from what I have seen of information theory, it is grounded basically in probability theory. I have ordered a copy of Kenneth Sayre's book!
quote:
I would suggest that even animal cunning and instinct are also described by it quite well.
This I find this difficult. Admittedly animals could make a measurement "e" on their surroundings, but they could not cognitively conceptualize it in terms of mathematical information - nor from it derive any future mathematical state of their environment. They could only anticipate a future state based on, as you say, instinct and established neural networks.
quote:
The universe had virtual information, both deterministic and stochastic, before mental processes started. This information is at the level of objective facts, rather than knowledge or understandings where an agent has created internally stored mutual information from its surroundings.
In my understanding too, the universe possesses information - although I see it more in the real physical realm than the virtual (the fact it exists in the virtual followin from the fact of it being in the physical). At a fundamental microscopic level all particles and systems of particles are described by discrete quantum states - with a discrete set of quantum numbers describing each state (although in practice these states are complex beyond our imagination). Since discrete states exist as a set of possibilities so there exists probabilities associated with each member - and hence we can at a very fundamental level talk about information, because as Sayre says - probability is that more fundamental quantity. However, the terms stochastic or deterministic refer to the means of transfering from one virtual state to another and are not thus properties of a virtual state. With regard to the solar system - recent astronomical evidence leads us to believe that our type of solar system is exceedingly rare in the universe. I wonder if we are closing back in on the mindset of Sir Isaac Newton who saw "This most beautiful System of the Sun, Planets and Comets could only proceed from the counsel and dominion of an intelligent and powerful Being" in that the virtual configuration required for the final formation of our solar system may have well have predated it back even to the Big Bang?
quote:
This conjectual version of the 1st law would in essence be asymptotic but generally isomorphic with the 1st law of thermodynamics
I am not sure why you pick on the 1st law of thermodynamics. In it energy is conserved - but as regards the law of information - we have information always increasing and only being conserved in the very restricted and rare class of processes which are deterministic. And anyway if the law of information (as embodied in both equations) is truly universal in the virtual realm then it must surely have relevance to all the laws of thermodynamics (zeroth, first, second, third and the fourth). Is it not true?
quote:
It (1st law of information) would match the surety of transformation that we have in physics and chemistry with information bein discrete and continuous
About this I have also reservations - based on what I have already said - that the 1st law of information is not so much about information conservation as information increase . As such the law of information is more connected with the 2nd law of thermodynamics. Having said that - I am basing my judgement here on the standard picture of stochastic quantum mechanics. If Einstein was right - namely that the universe was in essence fully deterministic (behind the quantum barrier) then you would be quite right in your statements. In this case information would be totally conserved. But is Einstein right? Remember -
Stochastic (quantum process) --> Information increase
Deterministic (classical process) --> Information conservation.
Many thanks for the info in Logical Entropy - it seems a viable concept - entropy in the semantic sense.
-Christopher
[ 19. September 2005, 23:58: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
Salvador T. Cordova
Member
Member # 959
|
posted 19. September 2005 16:13
Gentleman,
Let me say this thread is one of the most substantive discussions I've been a part of at ISCID. If my participation has been limited of late, it is only because I many times have little to add, and I have found myself going back to my math, physics, and information theory books.
I have Yockey's latest book as well as Walter Bradely's writings in the book Debating Design. I've been going back a re-reading No Free Lunch. I think the issues are substantially clearer to me and the other participants compared to where we began.
Chris, I really appreaciated your analysis of Shannon Entropy. I look forward to reading more of what you have to offer.
Salvador
IP: Logged
|
|
Christopher D. Beling
Member
Member # 723
|
posted 20. September 2005 21:11
Thanks for your encouragement Salvador and thanks Stephen for maintained interest. I do agree about the thread - but quite mentally exhausting too! I have a strong belief in the truth of the 4th law - and am also a little saddened at the general lack of interest by the scientific community at large. I feel that what is necessary is to build a very strong mathematical foundation on that already laid by folk such as Peter Medawar, Victor Weisskopf and Bill Dembski. Good systematic agreement with all experimental data should also be carefully demonstrated. This is of course what won the day for the 2nd and 3rd laws (but that was a more easier form of bench top science). In this thread - if we keep up the good work - we may possibly build some strong heuristic arguments that may attract professional theoreticians to work on the 4th law - especially if its close link to the 2nd law via information theory can be demonstrated.
William Thomson (Lord Kelvin) was a pioneer of the 2nd law. Many did not like the law because it spoke of a finite time for our universe - a fact which Kelvin was at pains to point out to people. With the discovery of the cosmic microwave background some 100 years latter, Kelvin was proven right. I think that we today are in a similar position to Kelvin as regards the 4th law.
I also feel the strain - need more time to read and think - but lets keep up the reading. Many things in Yockey's book are complicated and may need discussion - please feel free. I have another thought experiment for you when I get some time!
- Chris
[ 21. September 2005, 21:04: Message edited by: Christopher D. Beling ]
IP: Logged
|
|
|
Printer-friendly view of this topic
