|
Author
|
Topic: Validating Design Discrimination Methodologies
|
Jack Foster
Member
Member # 79
|
posted 18. October 2002 13:13
My dispute isn't with any one particular acronym. By defining a concept with an acronym, it allows the easy perpetuation of a misrepresentation, if it is true that a misrepresentation is happening in the first place. I suggest Great One Designer Dembskian Intelligent Design Information Theory: GOD-DID-IT.
But again the question: Is this an accurate description of what Bill Dembski is putting forward as Science to the community? If not, then the false perpetuation should not be allowed; at least not on this board.
[ 18. October 2002, 13:23: Message edited by: Jack Foster ]
IP: Logged
|
|
Evan
Member
Member # 164
|
posted 18. October 2002 15:04
But no one here is mentioning God - except you.
Dembski explicitly, in his scientific books, assumes an unembodied designer. That is all we are discussing (except there might be more than one.)
IP: Logged
|
|
Moderator
Administrator
Member # 1
|
posted 18. October 2002 20:30
Here's the deal. As moderator my goal is to keep the discussions productive and fruitful. For that reason, I am ruling that it is best if the discussion proceed by using descriptive, acronyms when attempting to set up categories.
Dembski has mentioned in several places that he believes that the designer is necessarily unembodied (or else we enter into an infinite regression). However, there are two concerns.
1. Dembski has not made this a central part of his scholarly work. Though he continually refers to designer in the singular, he has mentioned elsewhere that the notion of designer could just as well stand for some theoretical causal entity (like a quark)
2. Dembski could change his position on this issue. This seems particularly evident when one considers the fact that the issue has not played a significant role in his previous work. By significant here I mean merely that he has never included the premise of a singular, unembodied designer in one of his philosophical arguments. In other words, one could attack this proposition "the designer was singular and unembodied" and it would do absolutely nothing to Dembski's design arguments.
In light of these considerations, I am advising that participants in this thread and all subsequent threads pertaining to this topic, refrain from using Dembski's name to describe a position that is not quintessentially Dembskian. A more generalized descriptor will be more appropriate and help us to avoid the unproductive nitpicking that has started to plague the thread. In other words, unless someone can successfully differentiate between Dembskian and Unembodied in reference to the nature of the designer involved, then the latter is to be preferred because of its stronger, more general, independent descriptive content.
[ 18. October 2002, 20:32: Message edited by: Moderator ]
IP: Logged
|
|
Jack Foster
Member
Member # 79
|
posted 18. October 2002 20:59
quote:
A more generalized descriptor will be more appropriate and help us to avoid the unproductive nitpicking that has started to plague the thread.
My apologies to the extent that my nitpicking has damaged the thread. If it is some consolation, I feel that I've learned a thing or two from the exercise.
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 19. October 2002 02:50
Very well, I will accept that advice. Henceforth, I will refer to the position I described briefly above (in the Three Sentences) as "Single Unembodied Designer Intelligent Design," or SUDID.
I'll make one more remark on this topic before I leave it twitching feebly on the ground. From the point of view of discrimination methodology and a research program in general, the "U" is a worst case scenario. As I noted above, it forces research attention solely on the products of design. That makes the designer discrimination problem more difficult and increases the need for careful testing, validation, and calibration of design discrimination methods. In subsequent posts, which will become less frequent as the quantity of work necessary to research and write them grows, I'll attempt to lay out in greater detail what that entails.
Some Remarks on Planning Research Programs, or why is RBH not analyzing bacterial flagella yet?
A fair number of readers of this board likely have never participated in a major scientific research program of the sort that can last for years. In over 40 years of working in academic and applied science, I have. I can assure you that considerable forethought and planning are necessary in order to waste as little time, effort, and money as possible as time goes on and research progresses.
That is particularly difficult in an endeavour that is so novel and at the very edges of knowledge as SUDID or MDT, where one doesn't know what one will see as time goes on, or what serendipitous findings will alter one's course, or what new techniques from other fields will impinge on one's efforts, or what theoretical developments will alter one's hypotheses. Those considerations raise the importance of thinking out things ahead of time rather than plunging ahead hastily. I've also been a volunteer firefighter and rescue team member for nearly 30 years, and a central lesson one learns in emergency work is that pre-planning well before plunging into an emergency situation is among the most important phases in the whole sequence of tasks. Both novel research and firefighting deal with uncertain, ambiguous situations, and pre-planning is critical for success in both.
Validity and Reliability
MDT has existed for all of about 6 weeks now. One can already see it evolving, changing in response to comments, critiques, and elaborations from participants on these threads, as well as shifting out of an 'overview' mode to move closer to the nitty gritty of figuring out what one would actually do in an MDT research program. The initial focus on properties of the designers, while still a long-term goal, has given way in the short term to focus on methodologies for discriminating designs. That's entirely appropriate. The short term goal must be to develop valid and reliable and researcher-independent design discrimination technologies in order to pursue the long-term goal.
I use "valid" and "reliable" in restricted senses. The meaning of "valid" is roughly "this sensing technique actually detects what it purports to detect." A valid detection device is one for which we are confident it detects what we think it detects. That seems an obvious notion, but when phenomena are subtle, elusive, faint, buried in noise and therefore hard to discriminate from the background, it's not so obvious. For example, consider the problems associated with interpreting the enormous spray of various particles generated by large accelerators like those at CERN or FermiLab. The search for very rare and short-lived particles requires interpreting evanescent patterns of decay products blasting through an array of acutely sensitive detectors that gather data on sub-microsecond time grains. Enormously complex computer programs are employed to analyze the flood of data points in order to ferret out the few patterns of interest, perhaps just one pattern buried in the debris of millions of decays. Validation of both the detection devices and the analysis programs is a non-trivial task; they are a significant source of potential errors.
By "reliable" I mean "when exposed repeatedly to the same set of phenomena, the device provides the same outputs." A reliable discrimination device provides the same classifications for a set of phenomena each time it is exposed to them. (A discriminator is essentially a classifier.) Again, signals that are subtle, elusive, faint, and buried in noise can adversely affect the reliability of a discriminator.
By "researcher-independent" I mean what I described in an earlier post: "[Intersubjective testability] requires that the research methods be researcher-independent , in the sense that application and use of the methodology cannot depend on idiosyncratic characteristics of the person doing the research." In plain terms, it can't be who does the research that determines the outcome, but how.
The fundamental research technology of MDT (and indeed SUDID) is a discriminant, an abstract device or technique that distinguishes one state from another. There is an enormous literature on various kinds of discriminant techniques. An interesting approach for present purposes may be Linear Discriminant techniques, and I'll be spending some time reviewing that literature over the next few weeks. An example of an application to text coding for classification, a task not unlike discriminating between design and non-design or discriminating among designs, is at http://www.kornai.com/Papers/his01.pdf. The abstract: quote:
Linear Discriminant Text Classication in High Dimension
Andras Kornai and J. Michael Richards
Abstract. Linear Discriminant (LD) techniques are typically used in pattern recognition tasks when there are many (n much greater than 10^4) datapoints in low-dimensional (d less than 10^2) space. In this paper we argue on theoretical grounds that LD is in fact more appropriate when training data is sparse, and the dimension of the space is extremely high. To support this conclusion we present experimental results on a medical text classification problem of great practical importance, autocoding of adverse event reports. We trained and tested LD-based systems for a variety of classification schemes widely used in the clinical drug trial process (COSTART, WHOART, HARTS, and MedDRA) and obtained significant reduction in the rate of misclassification compared both to generic Bayesian machine-learning techniques and to the current generation of domain-specific autocoders based on string matching.
That's one sort of discriminant methodology that is available for adaptation for research in SUDID and MDT. The example shows two things: First, it has a theoretical discussion of a discriminant technique, and second, it describes an experimental test of the technique. Both are necessary to the methodology development effort of MDT.
RBH
[ 19. October 2002, 02:52: Message edited by: RBH ]
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 16. February 2003 00:59
As I have said a number of times in response to criticisms of Multiple Designers Theory (MDT), the absolutely necessary first step in the MDT research program must be the development and validation of designer discrimination methodologies. From the OP of this thread: quote:
In developing a design discrimination methodology, MDT has the same task as SUDID. First the methodology must be systematized and formalized. Then it must be empirically validated on test materials for which we already know the histories. As is the case for SUDID, the first goal of MDT is to develop a formalized researcher-independent methodology that, when it is applied to phenomena whose provenance and history we do not know, can be legitimately expected to reliably tell us something of interest about the phenomena.
Whether one is trying to discriminate design from no design or distinguish the work of one designer from another, the task of developing, systematizing, formalizing, and empirically validating the methodology is critical. Absent that, claims about one's theory are not (yet) testable. A theory may have all the promise in the world, but without empirically validated methodologies for making observations and gathering data to test hypotheses, it will remain merely an interesting conjecture with no empirical content.
(edited to change an offensive acronym to "SUDID" for "Single Unembodied-Designer Intelligent Design.")
This is a very preliminary progress note on that effort. This preliminary report will be slightly disappointing because at this stage I do not intend to disclose the specific designer discrimination methodologies I'm testing nor the designed objects on which they are being tested. As I have noted elsewhere, there is a commercial niche for such technologies. Since I am doing much of this work on company time with company machines I am obliged to reserve the technology until it is appropriately protected or until my company decides that it can be released into the public/academic domain.
Nevertheless, I can now say with confidence that at least one methodology I am examining has the capability to discriminate among the complex high-information objects created by several different designers. Pilot studies show that it provides statistically significantly different scores for samples of the products of different designers, and there is reason to hope that the technology may be amenable to being incorporated into a classifier system that allows reliable assignments of products to designers.
The principal task in discriminating among the products of multiple designers is to devise an analysis technology that can be mechanically implemented so subjective researcher-idiosyncratic variables cannot influence the encoding and scoring of the products to be analyzed. The technology under examination does that. It is researcher independent. No subjective judgements need be exercised in the selection of products, in the application of the encoding method, or in the application of the discrimination method. All those components can in principle be fully automated, though I have not yet done so; this is, after all, still pilot research. In pilot research it is useful to have the stages of analysis clearly separated for process tracking and troubleshooting purposes. The methodology makes no use of background knowledge about the design processes employed by the designers, requires no assumptions about the nature of the designers, and uses no extra background information about the designed objects. It is a purely objective methodology that analyzes just the designed structures.
To give the flavor of the pilot research, here are the results of one small pilot study of the candidate designers discrimination technology. Scores were assigned by the candidate analytic technique to each of ten products generated by of each of two human designers, D1 and D2. The products were generated by the designers in ignorance of the testing to be done on those products and with no particular instructions about what the designs should embody by way of structure. The first number indicates the designer, the second designates the product. For example, "D1,3" is the third product of the first designer. The scores are in arbitrary units, rescaled with identical scale factors.
Object .. Score
D1,1: ...2.897101
D1,2: ...2.887752
D1,3: ...2.758767
D1,4: ...2.958867
D1,5: ...3.015303
D1,6: ...2.954063
D1,7: ...2.910997
D1,8: ...3.200433
D1,9: ...2.929112
D1,10:..3.085128
Mean:...2.959752
D2,1: ...2.851835
D2,2: ...2.832242
D2,3: ...2.351285
D2,4: ...2.967218
D2,5: ...2.789246
D2,6: ...3.010922
D2,7: ...2.559147
D2,8: ...2.825446
D2,9: ...2.797831
D2,10:..2.706459
Mean....2.769163
The means of the two distributions of scores are statistically significantly different (2-tailed t = 2.10, p less than .05). The scores in the aggregate reliably distinguish between the two samples. As a reference point, the mean score assigned by this technology to a sample of random structureless objects with comparable numbers of components is on the order of 7.5 standard deviations different from the mean scores of the two distributions above. The technology very reliably detects (human) design in general.
There are several issues yet to be worked out, of course. Most notably, while the two designer's products differ significantly on mean discrimination scores, there is still non-trivial overlap between the two distributions. That makes reliable classifications of single objects difficult. Fortunately, there are mathematical methods for "sharpening" the encoded representations of the objects (analogous to the effects of lateral inhibition in retinal processing) that may reduce the overlap of the distributions and thereby increase the reliability of classifications. In addition, by analogy with the separate representations of visual edges, colors, and so on, at higher cortical levels, it appears that it should be possible to independently encode major features of the designed objects so as to generate a vector (or an array) of discrimination scores which may provide the basis for more reliable classifications.
My major purpose in posting this is to reassure MDT doubters (and MDT supporters, for that matter) that technologies for discriminating among designers solely on the basis of their products really do exist, that they can reliably distinguish (statistically) among (human) designers based solely on objective properties of the designed objects, and that research to refine and extend designer discrimination methodologies is proceeding.
I will also note (with no particular modesty) that as of this moment, MDT's methodology has a substantially broader and firmer empirical base than does SUDID. Though its method for (purportedly) detecting design has been public for at least 5 years, since Dembski's The Design Inference (TDI) came out in 1998, Dembski's Explanatory Filter has been formally applied to only one or two isolated phenomena in the domain of interest, and no systematic validation research has been done to support its claim of being able to reliably detect design of any sort, human or non-human. MDT's pilot research has already shown that it can do what it claims: discriminate among the products of different designers.
I said in the OP of the MDT thread that MDT offers a potentially fruitful research program, and just four months after the first publication of MDT on ISCID that program is under way, working through the tedious process of actually developing and validating designer discrimination methods on appropriate test stimuli. Where is SUDID's research program five years after the publication of TDI?
RBH
[ 16. February 2003, 02:06: Message edited by: RBH ]
IP: Logged
|
|
faded_Glory
Member
Member # 667
|
posted 16. February 2003 07:22
RBH,
This sounds really interesting. A quick question, have you already tested your method on a dataset consisting of both (humanly) designed and undesigned objects?
fG
IP: Logged
|
|
Moderator
Administrator
Member # 1
|
posted 16. February 2003 07:39
RBH,
If the purpose of your post was to make the following point:
quote:
Where is SUDID's research program five years after the publication of TDI?
Then it really has no place at Brainstorms. Your post has a touch of legitimacy to it, yet it also rings of polemical sarcasm. Please don't waste our time if your only intention is to point out that somebody else's research program isn't moving along as fast as you'd like it to. On the other hand, if your interest is in pursuing your MDT concept for its own sake (which I've come to increasingly doubt) then do it, and stop throwing in these hints which indicate otherwise.
[ 16. February 2003, 07:41: Message edited by: Moderator ]
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 16. February 2003 09:02
Moderator,
One of the main criticisms of MDT has been that I made (added in late edit: unsupported and unsupportable) claims about what might be possible using it as an orienting theory. For four months, since I first outlined it here, I've been working on developing methodologies consistent with its assumptions and goals. My recent posting is to report progress on that project. Genuine progress, progress that answers the initial question: Can designs generated by different designers be discriminated one from another? That's not a trivial project, and it's not a trivial result.
I took heat for "designer-centric extremism" on the grounds that to do research on the nature, identity, and properties of the designer(s) requires pre-knowledge of, or presuppositions about, those very properties. As I report above, that criticism is unfounded: It is possible to statistically distinguish between samples of the products of different designers with no assumptions at all about the nature, properties, or identities of the designers, and without any knowledge of the design process itself. All that is required is the samples of products themselves. That is not a trivial result.
I have not done this work (and work it is!) merely to tweak someone's tail. I suggest you consult some of the really polemical writing in science to provide a reference, particularly the kinds of polemics that occur in informal contexts. My remarks about SUDID are comparative, intended to make what I (who have been a working scientist for decades) consider to be an important point: the research program based on MDT is potentially richer and already more fruitful than the SUDID research program. I argued in the MDT thread that SUDID is a subset of MDT, and that opening consideration up to multiple designers hypotheses could inform an active research program. And it has. Working alone and in my spare time, in four months I have begun to produce systematic validation data on the methodologies inspired by MDT.
One of my principal criticisms of SUDID for longer than MDT has existed is that SUDID has not done so. It has not provided systematic validation data on its methods, even though those methods have been publicly available for years. That critique is independent of the MDT research except insofar as the latter demonstrates, not merely asserts, that it is concerned with the kinds of detailed and foundational research that a real novel scientific research program must take into account. If anything, the MDT research is a model for SUDID research. If the purpose of my posting was merely to ask that question, it's been asked multiple times to no response except the implicit response contained in Dembski's keynote speech at the RAPID conference where he called for compilation of some sort of Catalog of facts about phenomena thought to be inconsistent with evolutionary theory. Even in that speech, though, there was no mention of a scientific purpose for such a catalog (e.g., to provide foundational data to inform a systematic research program) but rather rhetorical - to convince the unconvinced that problems with evolutionary theory exist.
Your suspicions notwithstanding, MDT has already spawned real research generating real data that speak to real issues that the theory defines as important. Moreover, those methods appear to have applications in other non-trivial contexts, not just in MDT. That is part of what is meant by a fruitful research program. It is not inappropriate for proponents of one theory to criticize a competing theory if that competitor is seen to fall short of its promise. That happens all the time in science. Many of the articles in PCID provide examples.
RBH
P.S. added in edit: f_G: See the remark about random unstructured objects in the paragraph just after the data table.
[ 16. February 2003, 12:16: Message edited by: RBH ]
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 16. February 2003 13:38
I hope the moderator will forebear a somewhat tangential remark here.
When I started studying concepts of “intelligent design” in 1999, stimulated by some political events in Kansas, I was very skeptical that intelligent design concepts would ever be put on a firm scientific founding. I remain skeptical to this day that what has here been called “SUDID” will ever become firmly scientifically grounded, or even be confirmed as having a reasonable grounding in observation of the real world and consistency of logic.
I was also very skeptical of the value of a society for exchange of views like ISCID. What I have to say here affirms some of the value of the society that has as its founding reasons the free exchange of ideas. From the ISCID introductory paragraph on the home page:
quote:
… Its aim is to pursue the theoretical development, empirical application, and philosophical implications of information- and design-theoretic concepts for complex systems.
One of the things that happen in science, business, and other areas of human discourse is the serendipitous exchange of ideas. A term that is sometimes applied is “cross-fertilization”. An idea is presented in one context, and an aspect or part of that idea is transplanted into a completely different context in the way that we think.
Because of the subject matter of ISCID, there are discussions of the nature of intelligence from many different viewpoints, including artificial intelligence, and the capacities of physical or algorithmic systems. It is the aggregation of discussions from these diverse viewpoints that has also had an effect on me.
For many years I have had a vague notion of some relationships from some early research I did on constraint satisfaction search problems (results now regularly taught in AI search courses), and notions that I have read about in fuzzy logic and neural networks types of systems. In recent years I have become interested in the results from “genetic algorithms”, but had previously only purchased a popular descriptive book on the subject.
In the process of thinking about issues stimulated by reading and writing on ISCID, I have realized some possible extensions of my old ideas and applications to new areas. Because of this, I have been outlining a program of research and development in the last few days, that would surely entail several years work. Since this “several years” work needs to be done by me in my company’s “spare time” as has RBH’s work on MDT, I have no idea of how this will turn out. I have no idea of whether my health and other circumstances will allow me to work long enough and quickly enough to achieve any useful goals on this project.
The point I wanted to make is that there is an exchange of ideas happening at least in part as a result of ISCID. My life will be changed for the next few years because of serendipitous results of thinking stimulated by ideas presented on ISCID. (And in that regard, the society is producing results expected of such a society.)
I am reporting that at least a second project, relating to information- and design-theoretic concepts for complex systems, is being undertaken in “spare time”, and based at least in small part on serendipitous events of reading the subject matter of ISCID. (Though my project set is still very much in the conceptualization phase.)
In addition, I want to re-emphasize the importance of careful attention to experimental verification. Ideas taken as “armchair philosophical” positions may have very important results as philosophy, but to be useful in real-world practical effects they must bear a relationship to feedback gained from that real world. Actual experiments need to be done, and results gathered from application in the real world. This is true in design and development processes, as much as it is true in scientific pursuits. (Business is a very regular application of “natural selection” principles from the effects of “testing” ideas in the real world, for example, though the testing is not of the scientific correctness of ideas but of the constraints of human and physical systems.)
To test my ideas, I will have to construct physical realization of my concepts -- in this case as real physical signals in computer systems. To make any progress these real physical signal ensembles will be tried out on real computers, and with problems that have relationships to problems that people want to solve. And results will be gathered from those attempts to relate to real-world problems which will provide feedback to the design process. As I work on ideas that bear on the nature of intelligence itself, I find that the processes of construction, development, and testing is itself related to the very concepts of evolution, complexity, and intelligence that are the subject of this society. And time after time I discover that the thinking process appears to relate very closely to physical processes that can be understood.
Rather strange is the degree of affect the study of ID has had on its strongest critics.
IP: Logged
|
|
faded_Glory
Member
Member # 667
|
posted 16. February 2003 17:50
RBH,
You are understandably a bit vague about the exact nature of the data you are using to develop and test you methodology. I read that you have run your method on random, structureless data as well as data known to have originated from intelligent agents (humans). I wonder if it is possible to consider a non-intelligently, or at least non-purposefully generated dataset that displays non-random structure?
By way of example, say that your test data sets were texts written by different people, and randomly generated letter combinations selected by computer. I would think that discriminating between those two would be fairly straightforward (frequencies of letters would be very much non-random in texts written by people). An interesting third type of data could be a text made up from randomly selected words from a larger text written by a human. You could have a computer assemble a 1000 word essay by picking words from Moby Dick using whatever algorithm you fancy, random or more complicated. Most likely the resulting text would carry very little information, yet it might closely resemble a real text in terms of letter frequencies.
Would your methodology still discriminate the computer assembled text from the human text? Would the method recognise that the meaningless subset of Moby Dick was generated by a different author than Moby Dick itself?
Not knowing the exact nature of your data sets I may be way off base, and this test may not be relevant. Yet, when we consider multiple designers, is the underlying assumption not that these designers are of at least comparable intelligence? If one of the designers is a mindless algorithm, would it always be identified as different from the other, intelligent agents, or could it possibly mimick those so closely that it would not be separated by your methodology?
fG
[ 16. February 2003, 17:51: Message edited by: faded_Glory ]
IP: Logged
|
|
Micah Sparacio
Member
Member # 6
|
posted 16. February 2003 20:27
Hey gedanken,
Thanks for your post above. I can't speak for all the moderators at Brainstorms, but as Managing Director of ISCID it means a great deal to me to hear the types of things you mentioned. The funny thing is, I get incredibly discouraged by the many times I get lectured to about how ISCID is biased towards ID, or how ISCID is a cover for ID, etc. The reason I get discouraged is that I put an awful lot of energy into keeping ISCID on focus: to be a society for the exchange (even cross-fertilization) of ideas on complex systems rather than the biased promotion of an idea.
There is no hiding the fact that ID plays a significant role in the activities of ISCID. Many of the participants and leaders of the organization are intimately involved in ID oriented research. However, it is important to keep in mind that the organization started out of a desire to provide a neutral opportunity for the development of non-reductive science and philosophy. ISCID provides ID an opportunity within a larger context. However, it is not exlusively an ID society, a point that is too often lost at the expense of rigid preconceptions about how these discussions should proceed. To get the best sense of where ISCID's interests lie, check out our series of chat events. I can't tell you how excited I've been to feature great theorists like David Chalmers, Stuart Kauffman and James Shapiro. They are doing great work and deserve to be highlighted.
In any case, it means a lot when people are willing to give the intentions of this society the benefit of the doubt. To see that you've grown, at least in part, to appreciate ISCID more than you had expected, seriously makes my day, or perhaps even my week.
BTW, let me just say that the editors of PCID would absolutely love to get a paper written up on your ideas.
...the snow...it's falling hard
[ 16. February 2003, 20:35: Message edited by: Micah Sparacio ]
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 16. February 2003 21:55
f_G wrote quote:
You are understandably a bit vague about the exact nature of the data you are using to develop and test you methodology. I read that you have run your method on random, structureless data as well as data known to have originated from intelligent agents (humans). I wonder if it is possible to consider a non-intelligently, or at least non-purposefully generated dataset that displays non-random structure?
That's part of the plan, actually. One reason (a major reason, actually, in addition to time constraints) that I am proceeding slowly and methodically is that I didn't do so once, and it cost me some considerable embarrassment. Over 25 years ago, in a different discipline then, I thought up what I figured was a neat way to test a hanging issue in that discipline. Not a great huge issue, but an interesting one. Carrying it out required a fair amount of preparatory work, establishing the methodology, doing the studies necessary to link it up with existing research in the field, and then doing a methodical series of studies to actually test the hypothesis at issue.
Part way through the process I got impatient and sort of skipped a few steps, reasoning that it was obvious how they'd come out. I jumped to the main experiments. When they were done but before I'd written them up for publication I was invited to give a colloquium on the research at the university where I had earned my Ph.D. some years before. So I stood up there in front of a slew of grad students and faculty members and presented it, blurring past the blank spot in the series. The senior faculty member present, who knew my dissertation advisor (who was out of the country) pretty well, asked about the hole, and when I gave my reasoning for skipping over it, basically ripped me into little bitty pieces in the politest and deadliest way. Since the interpretation of the main studies depended on those "obvious" steps, I had no real empirical defense.
The real lesson came when I went back to do the missing stuff. It turned out that "obvious" wasn't true: The chain of reasoning that was "obvious" didn't match up with the experimental results. My zippy new approach didn't work. I comfort myself by believing that at least the colloquium saved me from receiving an acerbic rejection note from a journal editor.
So I ain't gonna fall into that trap again! And you guys in grad school, Micah and John, take heed. Somewhere in the audience there's always an old fox waiting and watching! ![[Smile]](smile.gif)
RBH
Added in edit: f_G also suggested quote:
By way of example, say that your test data sets were texts written by different people, and randomly generated letter combinations selected by computer. I would think that discriminating between those two would be fairly straightforward (frequencies of letters would be very much non-random in texts written by people).
But that seems (on first glance, at least) to be the sort of test material I can't use because it requires material-specific background knowledge (letter frequencies in English) that I cannot assume is available. This thing has got to be unimpeachably independent of subjective judgements, background knowledge of designers, test materials, and so on. Otherwise one can't take it confidently into the unknown. That's also why it is a non-trivially difficult project. Devising appropriate validation materials ain't easy.
RBH
[ 16. February 2003, 22:26: Message edited by: RBH ]
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 16. February 2003 22:52
Thanks, Micah,
I’m sorry on the subject of the work, but if anything positive comes of it I would publish in the AI or related journals, and if not it probably won’t get published. The subject matter is achieving some small advances toward those very goals that have been claimed as difficult for an AI system to achieve. In some regard the process I am planning is in part to set up almost exactly what I suggested for your class AI project in the short thread on that subject. In the thread by Larson on Kurzweil’s vision, we discussed what I was calling the “focus” problem -- relating to the described “frame” problem. And in other threads we note that GA can be used not only for modeling of “design” behavior (by natural system or human intelligence), but that a GA can be used at other levels of AI processing, such as in recognition tasks. And in yet another thread, RBH pointed out literature that talks about the possibility of having a GA construct a “partial individual” rather than a fully-formed object to be tested for “fitness” in whole. It is the relationship of these random processes to the partially random serendipitous events that bring information into focus that I am especially interested in this project -- and the very aggregating of information from various sources in this forum serves as a vaguely analogous example of the semi-random processes of a GA in making connections. So I will be applying the GA in a way that may or may not be novel, but the relationship of that application to a context of other search procedures that are already known to produce useful search results is novel as far as I can tell. I’m not sure that ISCID is appropriate for an AI result, even if I were willing and able. That’s about all I can hint at present. I may be able to give some very vague reports on Brainstorm if it seems appropriate and anything worth saying occurs.
In 1945 the French mathematician Hadamard said:
quote:
We shall see a little later that the possibility of imputing discovery to pure chance is already excluded. . . . Indeed, it is obvious that the invention or discovery, be it in mathematics or anywhere else, takes place by combining ideas.
-- J. Hadamard
(This is something I found referenced in a “Genetic Programming” book.)
I hope that even though I was complementing ISCID on the positive effect of serendipitous cross-fertilization, the reader will note the bit of back-handed thought on some of the threads. On Frances’s thread on “evolving inventions” Bracht and others discuss the TRIZ concept. Here my point is relevant as well, as the definition of “inventiveness” is taken as bringing out a concept without previously having that concept’s components as part of the dimensionality of the ‘search space’. However human thought was described as being “inventive” because it could generate these concepts new. My point here is to note how “cross-fertilization” is really a way of changing the dimensionality of the things we give consideration to in our thought processes -- and doing so in a way that is understandable as a natural process. If biological systems can cross-fertilize and achieve similar results, are they exhibiting a feature of intelligent action?
The other possibly back-handed point in my post was to reinforce the point of the importance of the empirical approach -- where RBH was taken to task for seeming to be posting only for the reason of questioning the empirical nature of other’s research. But I think these projects indeed have importance beyond any relation to subjects as discussed here. That they also point out the importance of the empirical approach is just the nature of how science and engineering investigation takes place -- in fact an aspect of the nature of inventiveness itself.
One thing that may seem to happen when one has decided to focus on “development of non-reductive science and philosophy” is that one may loose sight of the continuing requirements for logical argument, and for grounding claims about reality in observation of that reality. The decision to not deal in as much abstraction (so as to simplify or reduce the problem being analyzed) leads to greater complexity of the argument presented, and more opportunities for failed logic and failure to ground in observation to creep in.
[ 17. February 2003, 00:05: Message edited by: gedanken ]
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 17. February 2003 02:59
RBH, that's a very interesting report--even if the loss of implementation details frustrates me.
However, ever the contrarian, I have some data sets to throw at you that I believe will confuse your algorithm--or at least will broaden the definition of what it means for something to be "designed".
Is it easy enough for me to simply give you the data sets and have you run them through? Or should I instead suggest a way to generate them? It's probably a better test if you don't know what the sequences are (and I just label them "generation process #1" and so on). But if the system isn't at the stage yet where that kind of testing is appropriate (or if you can't reveal what an input format might look like), just let me know and I'll describe the generation methods instead.
Oh, yes, and on that note, I have ideas for a bunch of designed and undesigned bit strings for Dembski & Nelson's project. But since the details aren't agreed upon, and there is no known method for testing, I guess I won't post the strings just yet.
IP: Logged
|
|
|
Printer-friendly view of this topic