|
Author
|
Topic: Designing Complex Causation
|
Rex Kerr
Member
Member # 632
|
posted 27. January 2004 17:45
I'm still in the process of understanding what you're proposing.
This formalism looks like a reasonable way to encode memoryless linear classifiers, e.g. an ADALINE.
Presumably there could be some coupling between the output to the environment f(rv) where f:{0,1}->{behavior one , behavior two}, and the input values and/or program values and/or trigger point? How would that work?
Otherwise, this is a pretty boring classifier, since it just computes the dot product of the vectors s and p--varying asynchronously in real time, of course, but it is still just a dot-product.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 27. January 2004 19:55
Rex,
I first found this admittedly boring input-output program when studying the information processing performed by neurons. It appears that you can (in theory) model the information processing performed by individual neurons. Put together a few billion of these units and you get a nervous system. Not necessarily exciting and not terribly practical but interesting in understanding the functioning of neurons.
If each s from s1 to sn has a binary value, then it appears you can create a very large portion of the possible functional relationships between the domain defined by s1,…,sn and the range defined by r by modifying the values of the program variables. The neuron, this suggests is a general purpose logic machine or computer.
Interestingly, it also appears you can model and simulate human decision making with this same type of algorithm. Matched pairs of these decision making processes produce transactions and combining large sets of transactions in a structured way produces models of human economic and social behavior.
These programs may be boring, but they have some interesting applications. These programs exhibit, I suggest, the basic characteristics of at least some forms of biological information processing and of a basic unit of complex controllable causation. It appears that more complex forms of processing can be constructed by building complex structures from these units.
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 27. January 2004 22:31
warren_bergerson wrote quote:
The neuron, this suggests is a general purpose logic machine or computer.
McCulloch & Pitts (1943). "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, 5:115--133. Here is an entry to some of the research that paper spawned over the last 60 years.
RBH
[ 27. January 2004, 22:32: Message edited by: RBH ]
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 27. January 2004 23:06
Real neurons are not quite so simple as the linear classifier you've described there. They're nonlinear. Some spike regularly in the absence of activity. Some fire bursts that fade away. And so on. If you view the inputs as synaptic input, then it is sort of like an integrate-and-fire model, except that it doesn't integrate, it just fires (and you'd need internal states or feedback to the inputs or program to fix that).
Chaining a whole bunch of these together will produce something like the associative networks pioneered by Steinbuch and Willshaw and so on. They have some interesting properties, especially when you get into updating the weights ("pxv", I guess you call them), which hasn't been addressed here. I suppose something Hopfield-net-like could also be built.
But this was all done decades ago (e.g. see RBH's link), and it's not really news (or shouldn't be) to people in neuroscience or artificial neural networks. Perhaps this is an example of "life engineering", but people do this all the time without labeling it "complex causation" (probably since the label doesn't really tell you anything about what's going on, except as a warning that it's not simple, whatever simple means!).
I certainly think that this kind of approach can be valuable, at least as an approximation, at least when the physical structure you're modeling has explicit distributed information processing. If you're only trying to make other people aware of it, that's fine. But I don't understand why, first of all, you think that scientists don't already consider distributed causal models, and second of all, that this has some bearing on what might usefully be called "design" or "engineering" in, for instance, evolution.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 28. January 2004 08:29
Rex and RBH,
I believe AI researchers and physiologists have always recognized or assumed that the information processing performed by nervous system is logically equivalent to or can be modeled and simulated by mathematical logic machines and electronic computers. It is interesting to know that mathematical support or validation of this assumption goes back so far.
The subject here, however, is complex causation and the Life Engineering proposal that the construction and reduction of complex causal relationships follows the construction and reduction of complex mathematical functions (subject to real world constraints). In terms of complex causation, the program presented illustrates a program or function that creates the appearance or simulates the behavior of dynamic, programmable and controllable causation.
If you view the input to the program in terms in a combination of input variables, program variables and the trigger point variable, then the program represents a permanent and universal causal relationship. If, on the other hand, you view the input to the program only in terms of the input variables, then the algorithm or causal relationship defining the relationship between input and output is dynamic and programmable. Viewed as a causal relationship involving only input variables and the output variables, the program presented represents complex controllable causation with control strings.
The claim of Life Engineering is that it is possible, logically sound and productive to analyze intelligent behavior and processes in terms of complex causation. Complex causation, as the term is used in Life Engineering includes dynamic and programmable causation and teleological or goal directed causation.
Tie the construction and reduction of complex causation to the construction and reduction of mathematical functions should provide a clear demonstration that various forms of complex causation, including dynamic, programmable, controllable and teleological causation are clearly definable and thus logically and mathematically sound.
The position of existing sciences in regard to complex causation does not appear to be clearly and explicitly defined. Engineering and applied sciences appear to use the concepts and techniques I refer to as complex causation, but there does not appear to be any formal recognition. The position of the theoretical science toward complex causation appears to be even more unclear. While individual scientists appear to use the concepts and techniques of complex causation, the concepts and techniques do not appear to be used and recognized in the construction of scientific hypotheses. The lack of formal positions on complex causation and the construction/reduction of complex causation appears, IMO, to lead to some lack of clarity and lack of consistency in defining and analyzing complex causation.
It might be useful to clarify a couple of points with respect to information processing in neurons. To begin, it has long been recognized that the firing of a neuron (output variables) is the result or effect in some manner of the impulses entering a neuron from other neurons via synapses(the input variables in the program presented). It has also been clear from the relationship between input and output values can not be described by a simple fixed algorithm or function. While it accepted or assumed that the relationship between input and output at a point in time is controlled by a deterministic function, it is also recognized that this controlling function is dynamic or changeable. The evidence in fact suggests that the program, function, or causal relationship controlling processing in a neuron changes as fast and as often as the input.
It is not terribly difficult to identify the general type of physical mechanisms responsible for producing changes in program variables and thus changes in programs. In recent years it has been demonstrated that these reprogramming mechanisms involve gene expression processes. However, to my knowledge, no one has yet been able to backtrack these control strings to provide a useful explanation of what actually controls program changes. Furthermore, to my knowledge, no one in the traditional sciences has yet been able to formulate testable predictive models or hypotheses which describe or explain the functioning of neurons. Finally, no one in the traditional sciences, to my knowledge, has a predictive model or hypotheses which can explain or simulate the development or evolution of the processes controlling the reprogramming of neurons.
To summarize, our current knowledge of information processing in neurons is far from complete. The program presented above, I suggest, is a general type of algorithm which can ‘model’ the information processing in neurons. There is a big difference, however, between being able to model and being able to explain the functioning of a neuron in terms of predictive scientific hypotheses.
To get back to the topic of complex controllable causation, the program presented above is, IMO, a useful example to discuss the Life Engineering approach to complex causation because it illustrates features of the relationship between complex controllable causation and the reduction/construction of such causal relationships from other causal relationships. The program is also useful because it can be used in the modeling and analysis of both information processing in neurons and human decision making.
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 28. January 2004 16:21
Maybe the reason that there's an apparent lack of clarity is because such a simplistic "unifying" framework is, for the most part, useless.
Hodgkin and Huxley had a very good model of the firing of the squid giant axon in 1952. You can read about it (and find the reference to the original paper) in in this pdf file.
It's becoming increasingly obvious that you are highly unfamiliar with the scientific research in the areas that you are talking about. I think I'll just leave it at that.
[ 28. January 2004, 16:22: Message edited by: Rex Kerr ]
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 28. January 2004 19:23
The a model of the firing process is not the same thing as a model of the information processing performed by the neuron. Clearly you have no further interest in discussing complex causation.
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 28. January 2004 20:54
warren_bergerson wrote quote:
The a model of the firing process is not the same thing as a model of the information processing performed by the neuron. Clearly you have no further interest in discussing complex causation.
What kinds of papers does warren_bergerson reckon are published in Journal of Computational Neuroscience? It's up to volume 16 now.
As an example, consider this paper: quote:
The Influences of Ih on Temporal Summation in Hippocampal CA1 Pyramidal Neurons: A Modeling Study
Looks to me like it has it all: information processing (temporal summation), a formal model, and some actual data on real neurons. What else does warren_bergerson deem necessary?
RBH
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 28. January 2004 23:36
I do have no interest in discussing complex causation because I cannot see how it would be relevant to any modern scientific endeavor. These ideas have been introduced and surpassed decades ago, albeit not with the terminology that you find convenient. Plus I'm uninterested in the logical fallacy of assuming that a system is designed or engineered just because our models of it are.
There are any number of papers (going back decades) that model synapses, axonal firing, dendritic integration, whole neurons composed of those parts, small networks of them, etc., in the literature. Hodgkin&Huxley were the first to develop the framework that defined the formalism for others to follow.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 29. January 2004 07:15
It is useful to recall that the electronic computer is based on the concept that you can create or construct very complex operations from very simple operations or functions such as those performed by a binary switch. At the same time that computers were developing, physiologists recognized that human and animal behavior appeared to be generated by information processing performed in nervous system and the information processing performed by nervous systems appeared to be the result of the interactions among large numbers of processing units called neurons.
These two sets of observations led to the idea that if 1) you could understand, model and simulate the information processing performed in a neuron on a computer, and 2)if you could understand how these elementary neuronal interacted and you could simulate this interaction on a computer, then you could model and simulate the functioning of the human brain. This seemingly simple and obvious approach has proved far from simple. The ‘problem’ with the approach is that the information processing performed in the individual neuron is far more complex than initially anticipated.
At least two aspects of this complexity have been recognized for a very long time. First, basic processing algorithms associated with the neuron are not simple because the number of inputs is very large (and thus the domain of the processing function is very large). Second, neuronal information processing is complex because the processing algorithm or controlling program is highly dynamic. There have undoubtedly been many article written and published on the subject, but to my knowledge no one has published any claim that they have a predictive model or hypotheses that can explain these two levels of complexity in neuronal information processing. To point out the obvious, the existence of a very large body of published papers addressing a problem is not equivalent to a solution to the problem.
The Life Engineering concept of complex causation and the program presented earlier are not solutions to the neuronal information processing problem. They are simply tools that, I suggest, have proved useful in the analysis of this type of information processing. These tools, I suggest, are even more useful in the analysis of human decision making.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 30. January 2004 13:01
In AI Engineering or Life Engineering there are two basic types of scientific uses for complex causation. First, complex causation is in what are called engineering applications. As will be discussed, this use of complex causation appears to be essentially the same as to the techniques, processes and concepts used in physical science engineering. Second, complex causation is used in AI Engineering to formulate testable predictive hypotheses. Again the formulation and testing of complex causal ‘scientific theories’ is very similar to some of the procedures and practices used in engineering and applied science, but does not appear to have any formally recognized counter part in any of the academic/theoretical sciences. Scientific hypotheses based on complex causation do not appear to be formally recognized as ‘legitimate’ by philosophy of science.
ENGINEERING USES OF COMPLEX CAUSATION
[Note: This is my interpretation or the AI Engineering interpretation of engineering and applied science uses of what I label complex causal models. As far as I am aware, the interpretations offered here are compatible with existing applied science and engineering concepts and interpretations. I do not, however, claim to speak as an authority on the positions of existing engineering disciplines.]
Engineers as well as practitioners in other applied and theoretical sciences routinely build construct causal models. Any complex mathematical model which satisfies the criteria for a mathematical function will satisfy the Life Engineering criteria for a model of a complex causal process. Not all such complex causal models have the same degree of usefulness.
In general, the usefulness of complex causal models (or more accurately the usefulness of sets or classes of related models) is determined by their ability to reliably predict future behavior under a wide range of conditions. The best engineering causal models can reliably predict future behavior of a complex system under a very wide range of conditions. The predictive reliable of the best models is such that engineers can actually uses the models to perform pencil and paper comparisons of the performance of different system designs.
There exists a wide range of complex causal models whose predictive capabilities are far below the level of top notch engineering models. At the bottom end of the spectrum for usefulness, are so called ‘descriptive’ complex causal models which can fit or reproduce, model, or simulate a set of results produced by a complex system in the past, but which can not predict future results. Next from the bottom in usefulness are complex causal models which ‘predict’ results which are reproducible under very limited sets of conditions. Very close in usefulness are models which generate stochastic type predictions with a very low degree of reliability.
In between engineering models and models with only limited predictive capabilities, are complex causal models that are labeled here as simulation models. Simulation models are man-made complex causal models which produce complex behaviors which has the appearance of being similar to some naturally occurring complex behaviors. Some simulations can be quite effective at accurately simulating fairly specific types of behavior (voice recognition simulations for example) and other simulations produce behaviors that are only vaguely similar to the phenomena being simulated.
Simulation models are the types of complex causal models most commonly used in AI. Simulation models are also used in other life sciences. For the discussion here, it is worth noting that the applied sciences and engineering provide a number of techniques for comparing how successful different models are at simulating a particular type of behavior. Applied sciences and engineering, to my knowledge, do not provide techniques for determining the nature of the observed similarities between a simulation and the phenomena being simulated.
Engineering applications of what are called here complex causal models is a complex subject involving a lot of technical concepts and techniques. A working knowledge of these concepts and techniques will be useful in understanding the Life Engineering approach to complex causation. For now, it will useful to consider the following four features of engineering applications of complex causation.
CONTROLLABLE AND DESIGNABLE-Engineers use complex causal models to design and build new systems that will produce behaviors defined and imposed by the engineer. Such applications recognize that it is possible to design and control complex causal processes.
INCOMPLETE- Engineering models are incomplete causal models. The models only address the causal processes which are relevant to the task being undertaken.
NOT LIMITED TO RELATIONSHIPS DEFINED BY SCIENTIFIC THEORIES- If you take a complex engineering model and reduce it to its components, you will find the model contains operations, processes and relationships other than those defined by scientific theories.
CHANGE IN FORM- If you reduce a complex engineering model to smaller components, you will find some of the components correspond to established scientific theories. Generally you can expect these components to be expressed in format different than the format used in the scientific theory. In general, the format used in engineering applications will recognize the controllability of a causal relationships rather its permanence.
Although the terminology used may be different, the concepts and principles governing the use of complex causation in AI Engineering are believed to be very similar to those used in traditional physical science engineering. The AI Engineering uses of complex causation in formulating predictive scientific hypotheses does, however, appear to represent a departure from at least some traditional ‘theoretical sciences’. That, however, is a subject for another day.
IP: Logged
|
|
|
Printer-friendly view of this topic