|
Author
|
Topic: Virtues of Scientists
|
Mesk
Member
Member # 630
|
posted 14. April 2003 22:29
quote:
Nelson Alonso:
The people who found him guilty were the people who followed the research for years, following it's developments, and this only after publication. This is the best peer review which, I think is superior to our current peer review process. Where a few guys stand in between you and publication. The dishonesty can come from both ends.
But Nelson, post-publication scrutiny is part of the current peer review process, and therefore to claim that it is "superior" is absurd. There are some experimental errors or outright frauds that simply cannot be detected by academic referees, regardless of their experience, knowledge and careful analysis. Such anomalies can only be detected when other scientists repeat the experiments and fail to replicate the published results. But in order for other scientists to know how to replicate the experiments, the study must be published! Thus the current system is the ideal error-detection mechanism: glaring methodological errors can be eliminated at the referee step, while more subtle problems will be detected by other scientists when the published methodology fails to generate the published results in replicate experiments.
Your example of scientific fraud only serves to demonstrate just how well the current system works in detecting fraudulent or erroneous results. Like any human system it is imperfect, but when a scientist deliberately breaks the rules then sooner or later (s)he will be caught.
Of course, if you had any constructive suggestions about how to improve the system I would be interested in hearing them.
Mesk.
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 16. April 2003 15:56
Mesk,
The peer review process here was moot, and, only after publication, was it realized that false data after false data was allowed by the peer review process, and 4 years of false data to boot. The error-detection itself was not a part of the peer review process, where you submit something to a few guys who say yay or nay. The error-detection came after it was already published and in the papers. The peer-review process then failed, while the after-publication review succeeded. No doubt there are more examples of this.
My constructive suggestion is obvious in my post, don't trust a few guys to say yay or nay when they probably have no idea what they are talking about, or can reject good scientific papers on the basis of philosophical beliefs alone. I say let the scrutiny come after publication. Just take a look at Behe's book, some scientists only dream of such criticism of their theory.
[ 16. April 2003, 16:02: Message edited by: Nelson_Alonso ]
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 16. April 2003 17:35
Alonso, the "peer-review" process is not just the extremely small number of reviewers and editors that act on publication. The most important part of "peer-review" occurs after publication, wherein results are often tested by other researchers trying to replicate the results.
It is of course the replication of the results by other researchers that is the most important part of peer review. Often the replication of results may take a different form, so it is not just multiple scientists using the exact same methodology as well, and adding robustness to the evaluation of the physical evidence.
And "physical" evidence is at the hart of the process. It is not agreement with other persons that is sought, rather it is agreement with observation of nature that is sought. The confirmation with independent research on physical observation of nature is at the core of science.
If you are studying science, you should understand this aspect, as it is the most fundamental aspect. Why do you do labs?
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 16. April 2003 17:38
Getting back to virtues, if I may, one that's not been mentioned yet is persistence. Most scientific research programs of any importance require long, arduous, frustrating months and years of work, and a whole lot of persistence is necessary to slog through it, especially with no surety that a reward will come at the end of it. People underestimate the sheer drudgery that often must be endured in order to carry out a research program of any magnitude. That's one reason an ID research program has apparently been hard to get off the ground: It requires a lot of commitment to start out on that process, and a whole lot of persistence to see it through in the face of the uncertainty of the outcome. Some research projects I've participated in have gone on for a looong time until it finally became apparent that the particular approach was a dry hole. One that I recall with distaste occupied three years and came up dry. Maybe scientists need something of an oil wildcatter's mentality, too!
RBH
[ 16. April 2003, 17:39: Message edited by: RBH ]
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 16. April 2003 17:51
Ged writes:
quote:
Alonso, the "peer-review" process is not just the extremely small number of reviewers and editors that act on publication. The most important part of "peer-review" occurs after publication, wherein results are often tested by other researchers trying to replicate the results.
Ged, thats exactly my point. I wouldn't say it's just the most important part, however, I would say it is the part.
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 16. April 2003 22:15
quote:
I say let the scrutiny come after publication. Just take a look at Behe's book, some scientists only dream of such criticism of their theory.
Be sure and consider all the perpetual motion claims when you say that. (For example see Eric's History of Perpetual Motion and Free Energy Machines.) Shouldn't every crackpot with any crazy idea get published in prestigious journals?
For example, here are some documents that probably should be added to prestigious physics journals. Don't you think so? (Or how would you decide whether these should be printed in prestigious physics journals?)
I'd say that a 10 times increase in print volume and cost ought to cover it. And when we throw in the wasted time for all the scientists to figure out which articles are worth reading, science can grind almost to a halt. Good work!
Referring to peer review being what happens when peers read the published works:
quote:
Ged, thats exactly my point. I wouldn't say it's just the most important part, however, I would say it is the part.
If that is where “peer review” occurs, then why did you say:
quote:
The error-detection itself was not a part of the peer review process, where you submit something to a few guys who say yay or nay. The error-detection came after it was already published and in the papers. The peer-review process then failed, while the after-publication review succeeded. No doubt there are more examples of this.
(My emphasis.)
I’m sorry if this post has too much quoting, but I think the issues are important. An attitude of what is important comes across in these sorts of statement that might not be useful. And if mixed with confusion of factual nature, that can be at minimum confusing, and at worst misleading. I leave it to the reader to figure out what Alonso actually meant (perhaps he can give a better explanation of his comments).
The fact is that the peer-review process is part of figuring out what is actually consistent with nature. Of course there will be things that are published that are misleading in peer-reviewed scientific journals. This is where the peer review process helps to scrub out errors. There is no guarantee this will be an extremely quick process. How are you suggesting a better process?
[ 16. April 2003, 22:43: Message edited by: gedanken ]
IP: Logged
|
|
Cre8ionist
Member
Member # 140
|
posted 17. April 2003 08:22
Micah, you said
" I agree and disagree. What you are asking for is super-human. Humans aren't very good at "pure" thinking."
I wonder if you'd have answered the same way had I asked that science drop racial prejudice (noting that there's not really a problem with that) instead of dropping religious prejudice? I mean if there was a problem with racial prejudice in science circles, would you tolerate it by saying that humans aren't very good at "pure" thinking? If not, why then should you tolerate people like Behe being shunned by science journals? If he was being censored because of his color would you speak out, how about because of his "religious" viewpoint? IMO these kinds of prejudice stand in the way of science......................Cre8
[ 17. April 2003, 08:43: Message edited by: Cre8ionist ]
IP: Logged
|
|
Micah Sparacio
Member
Member # 6
|
posted 17. April 2003 15:05
Cre8,
I was making a weaker claim than you read me as making. I was merely saying that your call for an "objective" science where a hypothesis would stand or fall on its own, is a bit naive. Hypotheses and the interpretation of hypotheses occur within an epistemological framework, one that is hard to "transcend."
IP: Logged
|
|
Janitor@MIT
Member
Member # 125
|
posted 17. April 2003 17:45
Since the conversation is dominated by exceptions, the “lack of virtue,” and since virtue is like anything else defined by its opposite, I think (trying to go with the flow) two related questions have been asked that bear further discussion: competiveness and proprietary knowledge.
How can any scientist be in competition with any other? And how can any scientist possibly be possessed of "proprietary" knowledge?!
We seem to recognize that truthfulness is a (if not the) scientific virtue. I see little controversy here. But at least two have us have admitted that we have been “less than truthful.” (I shouldn't speak for another, but I've admitted it.)
I recall that g once asked me a question (an arcane question about how sets are closed under operations) and I responded bluntly that I just wasn’t going to tell him what I was thinking! He seems to have accepted it! (Quite graciously!)
Why?
IP: Logged
|
|
Christopher M. Langan
Member
Member # 264
|
posted 18. April 2003 02:17
quote:
For example, here are some documents that probably should be added to prestigious physics journals. Don't you think so? (Or how would you decide whether these should be printed in prestigious physics journals?)
Having taken a look at the page to which this was linked, I don't mind saying that I find it amusing. But as it happens, the joke isn't just on the author; it's also on prestigious physics journals.
The above quote links to a page containing an article asserting that motion through spacetime is impossible. While the author's tone leaves a bit to be desired – for example, he seems to want to dump poor Stephen Hawking right out of his wheelchair and make him apologize to the world for believing in time travel - he's got something resembling a valid point. Motion through spacetime is a problematic concept, and because time appears as a quasi-spatial dimension in standard continuum models of physical reality, physics is largely where the problem resides. Although conceptual workarounds exist, the author notes that they are far from satisfactory...and he's right.
If the author were so-disposed, he could try to clean up his act, dress the problem up with some stylish jargon and get it into a prestigious physics journal as a topic for future exploration. But something tells me that no matter how much cleaning and dressing up he did, such a paper would probably be rejected. The problem is too fundamental and extensively ramified, and keeping the intractable conceptual deficits of prevalent theories out of prestigious physics journals falls under the mandate of standard ass-covering procedure...at least when the one raising his voice is just a small fry who hasn't earned the right to criticize or poach on territory staked out by "real" scientists. (On the other hand, while they seldom use it, dues-paying bigshots are typically given more leeway in such matters.)
What about publishing in a philosophy journal? Unfortunately, although the author might have better luck there, many scientists dismiss philosophy as irrelevant to science. This position is obviously nonsense; explicitly or not, science progresses entirely within the framework of analytic-linguistic philosophy and can under no circumstances afford to divorce it, least of all when addressing fundamental issues that may exceed the limitations of its own empirical methodology...issues like cosmology, the interpretation and/or extension of quantum theory, and the true nature of physical entities, attributes and processes including time, space, matter, energy and causality.
It is not, of course, virtuous to arrogate problems of depth and/or scope beyond the capacity of one’s methods and conceptual models. Using science to attack such problems is allowable and even courageous, at least within empirical (perceptual) limits; tuning out or denying a hearing to anyone who attacks them using the purely analytical methods of philosophy and mathematics is somewhat less admirable, particularly when science would be impossible without these very methods. Where nature obeys laws of physics that are logical and mathematical in form, logic and mathematics would seem to be natural ingredients of the physical universe with even greater generality, and greater deductive potential, than the laws of physics.
Fortunately, not all scientists lay claim to more acreage or harder ground than they can fruitfully till. Many realize that the limits of empiricism are not necessarily those of the natural world, and remain open-minded toward the possibility of discovering truths about nature by rational means. I don't think it's too much of a stretch to say that this is a virtue to be emulated.
[ 18. April 2003, 05:57: Message edited by: Christopher M. Langan ]
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 18. April 2003 07:26
As Nelson pointed out, it is useful to distinguish between ‘post publication review by scientific peers’ and the ‘pre-publication review by a limited group of editors’. I don’t think anyone seriously questions either the need for nor the effectiveness of post publication peer review. The issue relevant to ID and to science in general is whether the pre-publication review process significantly biases, restricts, or hampers scientific analysis.
A reasonable argument can be made that pre-publication review decisions are based largely on 1)credentials, 2)format, and 3) conformity to convention. Poor quality or flawed research satisfying these three conditions has a reasonable likelihood of being published and high quality analysis not satisfying these criteria is unlikely to be published. The issue is not whether there is bias in the pre-publication review process since the existence of the bias is generally recognized. The issue is whether the existence of a pre-publication review bias can materially distort the process of scientific analysis.
The likely impact of pre-publication bias, I suggest, is substantial. The magnitude of the impact can estimated by looking at sources of technological innovation in the marketplace. Large corporations with large research and promotional budgets should have a major advantage in developing new technologies. But in order to get an idea developed within a large organization, an idea must pass the equivalent of a peer review process. Most creative ideas do not and can not get through peer review process. In general, it becomes more and more difficult to get new ideas past internal review processes as companies become older and more established. Most innovative products today come from small companies that circumvent the equivalent of a pre-publication peer review process.
Until there is a viable alternative to the pre-publication peer review process, we can only speculate on the likely impact of known bias in the process. Looking at the impact of ‘pre-review by experts’ in the area of technological innovation, suggests that pre-publication review could be significantly impacting scientific analysis.
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 18. April 2003 19:43
There are, of course, all sorts of human biases in how difficult or easy it is for a paper to get published in a scientific journal. However, I wouldn't criticize a reluctance to publish papers that go "against the grain".
Most of the time, when a majority of people (who are skilled and careful) get one result, and one person gets something weird, the explanation for the weird result is that "the experimenter messed up". It's not always obvious how or why the error occurred, but this kind of thing happens often enough so it's sensible to be skeptical when someone comes and tells you that, by golly, they do have a perpetual motion machine.
The question is whether the research can get accepted if the quality of the science is increased in proportion to the weirdness of the claim. Unfortunately, it's hard to assess this because so few weird claims actually end up being true. (Of course, few sensible-sounding claims end up being true either (unless they're extremely vague). It's just the nature of science, I suppose.)
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 19. April 2003 08:20
Rex,
The basic problem with pre-publication peer review is not that it rejects weird ideas, but that it accepts and over time validates weak and unsound hypotheses. In a field like evolutionary biology where there is 1)a strong political need to assert the existence of a real scientific theory and 2)no rigorous predictive hypothesis to satisfy that need, then there is a strong incentive to raise weak, ambiguous descriptive assertions to the level of scientific theories. Over time, the logic used to rationalize the weak descriptive hypotheses become the faulty standard on which competing theories are evaluated and inevitably rejected.
Dembski has argued that if you had a level playing field, both ID and neo-Darwinian theories would be equally acceptable. I suggest that if you applied rigorous, objective, verifiable standards neither would qualify as predictive scientific theories.
Pre-publication peer review has, so I have been told, only been a major factor since WWII. Before that time, again so I have been told, while there was a lot of evidence supporting the fact that evolution ad occurred, but the actual process or mechanism responsible for evolutionary change was considered an unsolved mystery. Pre publication peer review and its subjective standards has made it acceptable to solve this scientific mystery with vague descriptive assertions such as random mutation and natural selection.
Processes very similar to pre-publication peer review are used in many different fields. Under appropriate conditions the biases in these processes can and do produce serious distortions in reality. These distortions occur despite claims that the people involved are virtuous.
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 19. April 2003 14:44
Before I reply, I found some more relevant material concerning the peer review process and virtues:
quote:
Cochrane Review finds little evidence that peer-review ensures quality of biomedical research
Here's an excerpt:
quote:
Most of the criticism surrounding peer review has centred on its inability to detect flawed research, the secrecy involved in reviewing submitted papers, and the lack of transparency in the whole process.
Anonymous peers can reject or unfairly criticize papers from direct competitors. In the very worst cases, it's even been known for them to publish data "lifted" from a rival's work.
The Scientist
Ged writes:
quote:
Be sure and consider all the perpetual motion claims when you say that. (For example see Eric's History of Perpetual Motion and Free Energy Machines.) Shouldn't every crackpot with any crazy idea get published in prestigious journals?
I enjoyed very much Christopher Langan's reply. It shows just how much paranoia and politics gets in the way. This paranoia echoes the paranoia seen with naturalism. Something works well for what it does, and since that works well in some aspects of the practice of science, it gets exaggerated and starts to apply outside of it's field of usefulness, in that even intelligent origins, like strong directed panspermia (nothing supernatural about it), is rejected.
Usually when someone comes up with a new idea that may be against the grain, such as leave the scrutiny for after publication, detractors use what I like to call FUD (for fear, uncertainty, and doubt). "Science will grind to halt!" "science will die" "the world is going to end!". In fact, that it is nothing but gibberish.
As Dembski himself has commented:
quote:
As Frank Tipler pointed out to me, the idea of peer-review as the touchstone for truth and scientific merit is actually a post Second World War invention. In physics, peer-reviewed journals were not the norm until after 1950. In Germany, during the "Beautiful Years" -- the period when quantum mechanics was being invented in the 1920s -- one of the leading German physics journals, Zeitschrift fur Physik, was not peer-reviewed: any member of the German Physical Society could publish there by simply submitting the paper. So, if you had a really wild idea, all you had to do to get it published was ask a member of the GPS to submit it for you. (If you were a member, you could of course submit for yourself.) Heisenberg published his paper on the Uncertainty Principle in this journal, and Friedmann published his paper on the Friedmann universe (now the standard cosmological model) in this journal. No peer-review. Lots of brilliant physics.
It takes a few peers to review a paper. So you have these few guys who are paid to be skeptical of anything that is not "extending the current paradigm". So we have ISCID and maybe even ARN as an extension of the peer-review process. This provides ID theorists with a mechanism for establishing an idea as one's own. Thats the beauty of internet and "internet journals". Think about it. Before the internet there was simply no way of ever doing this. None.
By the way Ged, I don't see a difference between the two statements which I wrote. The after-publication "review" I am talking about is of course peer review. I am not dissing peer review itself, but the formal peer review process that can block a good research paper for no good reason whatsoever, or let papers through that have completely falsified data. The positive review process I am proposing is clear in my post. Let the scrutiny come after publication.
[ 19. April 2003, 15:22: Message edited by: Nelson_Alonso ]
IP: Logged
|
|
Alix Nenuphar
Member
Member # 686
|
posted 19. April 2003 18:54
Warren, I wonder if you could clarify some points in your last post. One of the primary virtues of the pre-publication process is that is culls the amount of other people's research that must be reviewed. And I fail to see why you believe that the pre-review phase validates a weak hypothesis. Certainly the core review groups are small and frequently attempting to be impartial in the face of their own agendas, but the fact that so many journals exist helps mitigate the possibility that a single, faulty theory will become enshrined in the literature. Are there any specific theories, besides evolution, which you believe have been generated and maintained by this process?
What political need to assert evolution as a real scientific theory do you perceive? The advantage of the post-publication phase of review is that all different political agendas can now be brought to bear upon the problem. It would appear to me that this method of dealing with the politics of review boards mitigates the more heinous possibilities. Are you aware of any specific politics involved? In addition, I presume from the continuation that you do not believe that evolution is a rigorous, predictive hypothesis. I would agree: evolution is certainly a rigorous, predictive theory - it moved beyond the hypothesis stage long ago.
Do you have any documented incidents of weak hypothesis logic which is now the standard? In addition, perhaps a more consistent use of such terms as 'hypothesis' and 'theory' might help those of us for whom English is not our native language.
I am uncertain why you emphasise predictive in your criticism of both ID and NDT. Both are concerned with the 'ongoing' process, rather than with the actual result which is, after all, historically, not methodologically contingent.
I believe that your concern about the mystery of the evolutionary mechanism is unfounded: the actual mechanics of Mendelian genetics had been incorporated into the NDT long before WWII.
Could you please clarify your definition of 'vague' when applied to random mutation and natural selection? Perhaps I am not understanding your terminology.
Could you provide some samples instances of pre-publication peer review used in some other field that you feel avoids the problems inherent in the scientific domain? Preferably both the equivalents to the 'pre-publication' peer review and the 'serious distortions'?
[ 19. April 2003, 23:59: Message edited by: Alix Nenuphar ]
IP: Logged
|
|
|
Printer-friendly view of this topic