|
Author
|
Topic: Virtues of Scientists
|
Moderator
Administrator
Member # 1
|
posted 19. April 2003 20:34
Alix,
Please stop the relentless quoting. One of the three suggestions on our main page deals with this issue and should be taken seriously. If it persists, we will start deleting the quotes out of posts.
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 20. April 2003 00:22
Referring to the article Nasty Little Truth About Spacetime Physics Chris Langham suggested that the author of the web page could “dress the problem up with some stylish jargon and get it into a prestigious physics journal as a topic for future exploration”.
Could I ask what the useful “problem” was, in some detail, that was reasonable for a prestigious physics journal? The problem in my reading of Chris’ statement is that the claim can be made convincing when readers read this post here, but Chris’s post doesn’t have to actually put up anything really worth publishing in a physics journal to make his point sound good here in this thread. But can Chris actually describe the detail that was worth publishing in a prestigious physics journal, explaining in some convincing detail how it is actually an interesting physics problem?
Here is a little gem from that page:
quote:
[Note that Sir Stephen is implying that it is alright for a physicist to talk about time travel into the future. The man truly believes in a time dimension and an arrow of time. It never occurs to him that time travel is time travel regardless of the direction of travel. This is a common misunderstanding among relativists.]
Now I’ve got an experiment for the readers who think that there is any there there on that page. This is my personal experiment in demonstrating time travel that I want readers to think about:
Sit on the floor or a very large chair or couch. Draw your knees and limbs in and duck your head, so that your body is as nearly spherical in shape as possible. (The purpose is to produce the smallest ball or radius for the body in 4-dimensional space-time from any arbitrary reference frame.) Then concentrating very hard, voice out loud the following mathematical symbols: “1,2,3,4,5,6,7,8,9,10”.
At the conclusion of this little experiment, the subject can make the following observation. Between voicing the mathematical symbol “1” and the mathematical symbol “10”, the subject traveled in time in the forward direction! (And I’ve got news for the reader, this is a true statement as viewed from any reference frame of the special theory of relativity, that the experimental subject can be observed to travel forward in time in that particular reference frame between the subject’s utterance of the symbols “1” and “10”!) Having achieved this milestone, could the readers then please describe other time travel experiments -- especially ones that use similar techniques to travel backward in time, at least in some reference frame. Remember “time travel is time travel, regardless of the direction of the travel” and that we have demonstrated at least one means of time travel here.
Now some may think that I am making light, that I am playing with the reader’s mind. And of course I am.
But this all points out the need for pre-publication review of papers for journals. How can a journal decide to not publish everything that is submitted, without actually deciding? And if it does not, then what is the value of the journal being a journal. Just let all physics be published on the author’s home page, for example, and forget the journals! From what I have heard, some people here might agree with this. The readers should realize this, and decide for themselves about its value.
[ 20. April 2003, 00:26: Message edited by: gedanken ]
IP: Logged
|
|
RBH
Member
Member # 380
|
posted 20. April 2003 01:04
[Edited by the Moderator]
RBH:
If only you had read Alix's original post! He subsequently went back and made changes which I am thankful for. I think he had a total of 12-14 quotes interspersed with comments.
Nelson has been warned before about the same problem. Alix is relatively new, and needed to be corrected or else he would have unknowingly continued with that posting style.
You should note that I do not look fondly on posters who start taking sides and accuse me of being a biased moderator.
Lastly, our rules are not arbitrary. The "quote for quote" rebuttal method is bad for several reasons. First, it crowds our forum with noise and discourages people from reading posts. Second, the responder is less likely to really understand what the original poster meant, by putting the other person's argument in your own words, and focusing on the salient issues, it shows that you are more interested in understanding and having a constructive dialogue. Third, it encourages quick, "off the top of your head" responses rather than thoughtful engagement.
Please send me an email in the future: moderator@iscid.org
[ 20. April 2003, 07:52: Message edited by: Moderator ]
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 20. April 2003 06:20
People may wish to note that since these boards allow editing, it is not always possible to see what people were responding to.
Nelson wrote:
quote:
It takes a few peers to review a paper. So you have these few guys who are paid to be skeptical of anything that is not "extending the current paradigm".
Hahahaha. Ha. That's a good one. I wish. I've reviewed a number of papers, and gotten a grand total of $0.00 for my efforts. The editors of the journals get paid--that is, after all, their job. However, editors are not peers. Peers are scientific colleagues who are, hopefully, knowledgable in the area in which the research is in, and are not compensated for their reviewing work. (Of course, they do have scientific jobs, but you can't really say that they're being paid in that capacity to reject anything that clashes with the status quo.)
Others have already asked Warren for additional examples of acceptance and validation of weak and unsound hypotheses by pre-publication peer review, and I would also be interested in seeing these other examples.
[ 20. April 2003, 06:23: Message edited by: Rex Kerr ]
IP: Logged
|
|
Moderator
Administrator
Member # 1
|
posted 20. April 2003 07:56
Rex noted that because of the editing feature, its not always possible to see what people later one are responding to.
My recommendation. Use quotes selectively and normally err on the side of rephrasing someone else's points. We're not against quotes, we're against quote for quote rebuttals.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 20. April 2003 09:26
Alix,
There are at least two substantive issues associated with pre-publication review processes. First, on what basis are the pre-publication selections based. Are the selections of articles to be published based on objective, verifiable, substantive criteria or are they based on subjective, non-substantive, and potentially biased criteria. There is plenty of evidence that of bias and subjective criteria being using in the review process.
The second and more serious issue relates to the cumulative impact of bias. If the process used to select articles for publication is biased, does this create the risk of cumulative long term distortions. If what is published one year influences future decisions on what can be published then there is the potential for a cumulative impact of bias.
There is, again, substantial evidence that the cumulative impact of bias is a real phenomena of human behavior. Market bubbles are prime and well documented example of severe distortions created not by a single instance of bias but by the cumulative impact of bias on investment analysis.
The suggestion that peer review bias has resulted in material cumulative distortions in evolutionary theories is of course my speculation. There are a number of individuals who suggest that current evolutionary theory is more a belief system than legitimate scientific theory. I don’t know of anyone else suggesting these beliefs are the result of cumulative bias in the peer review process.
My hypotheses regarding cumulative distortions, is testable if 1)you require that both existing theories and hypotheses and alternative theories be expressed in an explicit predictive form, and 2)you require an objective verifiable process for comparing the competing hypotheses.
IP: Logged
|
|
Alix Nenuphar
Member
Member # 686
|
posted 20. April 2003 14:40
Warren:
While I certainly appreciate your concern about bias in the pre-publication process, I can assure you, as someone who has both served on a review committee and as someone who has had articles both accepted and rejected, that the individuals concerned are highly conscious of their responsibilities - we endeavour to eliminate our biases as far as possible. It is, of course, impossible for any human being to completely eliminate their prejudices and subjective opinions in any such process, but every effort is taken to minimize the intrusion of 'personality' into the process.
On the committees on which I have served, we have tried to evaluate the integrity of the research, not necessarily the theory that lies behind it - in some cases, none of us were particularly well-versed in the theoretical basis of the experiment. This is as it should be, and helps to avoid that the paradigm conformance issue with which you are concerned.
You state (fairly strongly) that there is plenty of evidence of bias... could you share that evidence? Specifics are far easier to deal with than vague generalities, and as I still retain contacts on various review committees, it is possible that your concerns could be addressed.
Your second concern is, I am happy to say, a remote one; the cycling of review and selection committees assures a sufficient variation in the selection and review criteria that it is highly unlikely that such a pattern can be established. Again, if there are specific instances you wish to adduce, I would be happy to consider them.
As you can see, I am somewhat puzzled by your remarks regarding the pre- and post-publication review process as a source of an unsupportable evolutionary hypothesis. Documented instances would be helpful in understanding your position.
Finally, I do not entirely see how your hypothesis would be tested using the mechanism you propose. Precisely what standards would you use? How would you set up your control group? What specifically are you testing for?
The multiplicity of journals and avenues of publication; the turnover and mixture of biases and opinions of the various review committees; the variation in standards of acceptance; the possibility of detecting failure in a series of papers; and the highly competetive nature of the scientific community work in combination to ensure that the process works as well as possible given the nature of human beings. Do you have specific suggestions above and beyond simply asking for 'standards'?
[ 20. April 2003, 14:44: Message edited by: Alix Nenuphar ]
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 20. April 2003 17:52
Rex,
Well thats strange, I remember my professor getting an honorarium for peer reviewing a few papers, maybe that was an isolated instance, but another was given a paid vacation (which only consisted of a free crappy hotel room, but it was something). Another was paid a fraction of his salary. Maybe this is a minority of referees (and may be a perrogative of the editors).
As far as examples of weak research, what about all the examples that I gave? I can think of others though. When Nature published some hypothesis challenging the Big Bang theory, and surrounded it with the word "hypothesis". Nilson's and Pegler's model (on eye evolution in about approx. 1800 steps, don't remember the exact number) hasn't been replicated in years, mainly because it's so vague as to make no difference.
To be a bit non-controversial, since we're on the topic of physics, if you go to this list of most cited papers in physics you can get a feel for which hypothesis, or even which journal, has therefore the best peer reviewers, and which is packed with weak proposals and therefore, bad peer reviewers.
[ 20. April 2003, 18:42: Message edited by: Nelson_Alonso ]
IP: Logged
|
|
Rex Kerr
Member
Member # 632
|
posted 21. April 2003 01:12
Nelson, I didn't see any examples of weak research in your post--just secondhand accounts of how peer review doesn't work as well as one might hope.
Keep in mind that citations are a measure of how relevant people find something. Not all papers are equal. If a paper isn't cited, maybe it is completely accurate but is boring.
IP: Logged
|
|
warren_bergerson
Member
Member # 262
|
posted 21. April 2003 06:14
Alix,
The virtue of science is ‘open, objective, verifiable’ review, testing and validation of hypotheses by the scientific community. Pre-publication peer review is secret, subjective, review by a small group of individuals. The widespread use of pre-publication peer review raises the question "Does this significant departure from the traditional virtues of science constitute a material risk to the integrity and soundness of scientific analysis or is pre-publication peer review simply a harmless, no risk convenience?". It is important to note that secret and subjective review represents a fairly material departure from the traditional virtues of science.
I am suggesting or hypothesizing that:
1. PEER REVIEW BIAS -The decision processes associated with pre-publication peer review involve substantial potential for bias.
2. CUMULATIVE IMPACT OF BIAS-The cumulative impact of bias in peer review decision making can produce material distortions in beliefs.
Your comments suggest you question the validity of the peer review bias hypothesis. Testing this bias hypothesis illustrates some of the ‘standards’ I consider important including:
1. COMPETITIVE ADVOCACY- If there is a disagreement on the existence of bias in peer review or on any scientific hypothesis, then those on different views define their positions and analysis is performed so that the parties involved can reach agreement on which of the positions is stronger. Science is about resolving differences between different positions. [Science is not about proving some point to a group of self proclaimed authorities. ]
2. CONSENSUS ON PRECISE DEFINITIONS- In order to determine the relative strength of competing hypotheses there must be agreement on the terms used. With respect to the bias hypothesis there must be an agreement on what constitutes bias.
3. CONSENSUS ON THE BASIS FOR RESOLUTION- There must be agreement in advance of what type of evidence and what type of logical argument would demonstrate the presence or absence of bias.
4. CONSENSUS ON ISSUES OF AGREEMENT AND DISAGREEMENT- Agreement must be reached in advance on which issues are and which issues are not in dispute.
Personally, I am a bit surprised that anyone would be willing to advocate the position that there is no bias in the pre-publication peer review process. I would expect the cumulative impact hypothesis to be controversial, not the existence of bias hypothesis.
The virtue of real science, in my opinion, is that it permits groups of individuals to work cooperatively and reach agreement or consensus on the best and most useful explanations for complex phenomena.
IP: Logged
|
|
Christopher M. Langan
Member
Member # 264
|
posted 21. April 2003 13:11
Gedinken comments that
quote:
Referring to the article Nasty Little Truth About Spacetime Physics Chris Langham [sic] suggested that the author of the web page could “dress the problem up with some stylish jargon and get it into a prestigious physics journal as a topic for future exploration”.
Could I ask what the useful “problem” was, in some detail, that was reasonable for a prestigious physics journal? The problem in my reading of Chris’ statement is that the claim can be made convincing when readers read this post here, but Chris’s post doesn’t have to actually put up anything really worth publishing in a physics journal to make his point sound good here in this thread. But can Chris actually describe the detail that was worth publishing in a prestigious physics journal, explaining in some convincing detail how it is actually an interesting physics problem?
The problem to which I refer was clearly indicated. It’s the concept of progress along a time axis in a spacetime manifold. What the author is trying to get at in his own belligerent way is just that time has no place in geometric formalisms including the geometric formalism of physics. As long as time is represented in physical geometry as a spatial distance - and space is what geometry is all about - this problem will persist. So this comes down to a rather bad example of ideas that are so “cranky” as to be physically irrelevant, and it points (somewhat ironically) to the readiness of certain scientists to stonewall or laugh relevant but risky or troublesome material out of their journals.
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 21. April 2003 14:38
Rex,
You say that you do not see any examples of weak research in my post, but saying so does not make it so. I think that I have given examples of weak research in my post.
It might be true that a paper is forgotten because it is boring (i.e. experimental data is often the least cited) but it may also be true that a paper is forgotten because it is vague and weak research, (like the Pegler paper I cite above). Saying that all papers that are forgotten are forgotten because they are all "boring" is quite unlikely. For example, I don't think that it is a coincidence that Physical Review contains the most cited papers on general relativity. If you want to get a weak hypothesis published, you might have better luck with a different journal, which is likely to not be able to get a good peer reviewer.
[ 21. April 2003, 15:42: Message edited by: Nelson_Alonso ]
IP: Logged
|
|
gedanken
Member
Member # 594
|
posted 21. April 2003 15:49
quote:
If you want to get a weak hypothesis published, you might have better luck with a different journal, which is likely to not be able to get a good peer reviewer.
While I certainly have my doubts about there being significant differences between the publications cited, and especially about the methodology give, what does this say about the importance of such peer review?
I would observe that Alonso is validating the importance of good peer review here. That science might be imperfect, and less than perfectly efficient at its intended task, seems highly irrelevant to me. Because such perfection is only a goal, and is not something that will be possible in the realm of human activity. What I am impressed with is how well science actually does at what it purports to do. And once again even in criticism, the critics of science seem to emphasize its important aspects.
IP: Logged
|
|
Janitor@MIT
Member
Member # 125
|
posted 21. April 2003 16:16
I did a search (or “exploration” actually, with a wink to RBH) for “the vital science merton” and found, much to my surprise, many very interesting links pertinent to the direction this topic has taken. I began by searching for a citation for a book by Michael Ghiselin, which returned first, “Plagiarism and the Art of Skillful Citation” (!), which had a reference to Merton. (See http://www.bcm.tmc.edu/immuno/igr_homeric.html which is interestingly enough connected to http://www.bcm.tmc.edu/immuno!)
My usual “explorations” make me wonder how the Internet will change publication of science. Esp. as I often find very interesting articles published on the Web, which are not published via peer-review or at least don’t say so (e.g., @arXiv).
I sometimes wonder if I should accept these as "authoritative" in the usual scientific sense...
Actually, I trust my own judgment in these matters and occasionally wonder if referring to “peer-reviewed” literature is a short-cut to actually having to think about it? (Or reproduce it--since there is little "reward" to exactly reproducing someone else's results--at least in print.)
I’ve noticed that often literature is cited (here and everywhere) and often the criticism is that the citer doesn’t actually understand (or has not even read!) the citation!
Referring to “peer-reviewed” literature is a short-cut to actually having to think about it. I’ve noticed that often such literature is cited and often the usual criticism is that the citer doesn’t actually understand (or has even read) the citation!
For me, its not a question of how someone’s “theoretical” (or philosophical or theological) perspective is “suppressed” in the process, but how someone’s (anyone’s or everyone’s) perspective is “forwarded.”
I cited before the von Dassow article, noting how a certain “philosophical” perspective (wrt to "teleological explanations" which are admitted, implicitly, in evolutionary biology) was objected to by the reviewer—but admitted (into the article) in the end. ???
IP: Logged
|
|
Nel
Member
Member # 614
|
posted 21. April 2003 16:30
Ged,
It's not a problem of "science is imperfect", as I have already showed, there are scientists who are concerned about the lack of quality of research being done due to our current peer review process. It's not just a matter of a weak hypothesis getting by, that probably doesn't even matter, what matters is that good scientific papers are being rejected on the basis of presuppositions, because they come from rival camps, and even falsified data is being permitted. With that said, I find it laughable that you are calling me a "critic of science" when it's researchers themselves who are also unhappy with the current peer review process for basically the same reasons I am.
Speaking of which, there is a middle of the road solution being proposed by these researchers, unveil the peer review process, make it public, take away their anonymity.
IP: Logged
|
|
|
Printer-friendly view of this topic