Whenever someone around here posts something that someone doesn't like there are an inordinate number of cries for "peer review"! It's like people think that having other, supposedly "expert" people agree with them somehow makes their random opinions valid. I finally found someone in the "peer review" morass that has some stones and honesty about what "peer review" actually means. In short it's worse than nothing at all. Enjoy. https://experimentalhistory.substack.com/p/the-rise-and-fall-of-peer-review
From a source that credits "My Dad". Expert opinions are not random. Qualified experts seek to falsify the conclusions, methods, or assumptions of a paper based on typically 8 to 16 years of formal education and training. If a claim can be falsified, it isn't based on an opinion. It is based on defensible logic, facts, the laws of physics, and math. It is the purest form of fact finding ever devised because it is self correcting.
Ahhh, now how can I explain how far off you are in your understanding of peer review without…… Peer review is simply a process, which, when done correctly, establishes that the researchers conducted the research with due diligence and without bias. The last is important as you do not want scientists to be writing papers simply to please thier employer (I.e. as occurs in the “alternate medicine” industry where wild claims are made about fake “cures” So, how do we know if the peer review process was done with due diligence? Well you could critique the article yourself, looking at the research methodology, rigorous of the research and the statistical tools used or You look at the journal it is published in. Some journals are fly by night and have poor peer review others (Lancet, Jama, Nature etc) have high standards and are therefore trustworthy. Usually academics use both methods
Nearing 5,000 retractions: A review of 2022 Retractions of a given year’s publications as a percentage of papers published in science and engineering. Retraction data from Retraction Watch Database, overall publication figures via U.S. NSF. In 2002, journals retracted 119 papers from the scientific literature. What a difference two decades make. On several occasions this year, publishers announced they were retracting several times that number, all at once. (For some of the stories among 2022’s retractions that captured the most attention, see our 10th annual roundup for The Scientist.) This year’s 4,600-plus retractions bring the total in the Retraction Watch Database to more than 37,000 at the time of this writing. Continue reading
You obviously don't know how peer review works As a scientist, I can do the math and experiments to substantiate my hypothesis. But simply creating a hypothesis and validating it through my own experiments is not enough. Other scientist [your peers] must review your work, your methods, your maths, your experiments. And preform it themselves to see if they get the same results. Only when the work is peer reviewed and methods, maths, and experiments confirmed, does a scientific hypothesis gains credibility and becomes a scinetific fact. That's why peer review is so important and credible. It means that the established hypothesis is correct. Because other scientists have conducted the exact same experiment and got the exactly the same results.
Wow, you're a scientist that's participated in the peer review thing? Awesome. I've never actually known anybody like that, I've just had to read them. And I've spent my career pretty much in private industry where nobody gives a hoot about serious peer review. After all, revealing your confidential discoveries is taboo in private industry. You lose competitive advantage when you reveal your best stuff up front. So I'll be back with some questions I've always had about peer review in a bit. Maybe you can make me a fan.
Well, the fact you graduated high school doesn't really strike the rest of us as very impressive at all.
I think what it is is actually evidence of the increasing amount of gooblygunk, junk pseudoscience that's out there these days. You know, like the "science" that people use to justify their belief in a gender spectrum....
I have never claimed the system doesn't work. Indeed, the high esteem in which RW is held, and the reason RW continues to score large grants, derive from their essential role in making the system work.
What a piece of malarkey, lol. I wouldn't say peer review is perfect, but it's definitely a good process to have. Without peer review, somebody with a lot of credibility or fame can just push a paper through with a personal agenda. Peer review gives an opportunity for people who are qualified to evaluate their claims, to at least give some thought as to if they are well-supported. The disadvantage of peer review is that it can give a certain... inertia to science. Both ridiculous and revolutionary but insightful claims can find themselves rejected. So, ironically, it makes science more "conservative," but I think it's an acceptable price to pay to make sure things are less often pieces of malarkey. But I just don't buy that it suppresses the strong links like the author says. Extraordinary claims really do require extraordinary proof. And the advantages he suggests wouldn't happen either. Journals will still have rules. They will still ask you to abbreviate to save space. And they will still review your paper themselves even if they don't send it to other experts. It is, however, possible that the process of peer review could be streamlined or improved. One thing the article points out is that there have been fewer major discoveries more recently. This isn't surprising at all. The more things that have been discovered and invented, the harder it is to do anything truly new or revolutionary that's still valid. But mankind's technological abilities are definitely still improving. Wow I had no idea the percentage was that low. Increasing I guess, but still very rare.
It's called believe 1/2 of what one sees, and none of what one hears or reads. Verify verify verify. Peer reviews is verifying the method a claim is made by others with knowledge of the topic.
Don't waste your time providing evidence for people. If they don't believe you that won't believe any evidence. Any time I've done it, going as far as to find ten separate sources, they still denied what I just proved. They aren't really interested in debate they just want to try and debunk everything they don't agree with. When you refuse to give them anything they get really mad it's kind of funny.
But they aren't any more valid than anybody else's opinion. Not all so called "experts" are qualified. so long as the people doing it aren't biased.
OK, let's start the "ask a scientist about peer review" session. First, what's your scientific specialty or field of study? How big is that field in terms of number of similar scientists? Once we've got the general topic nailed down, we'll see how peer review both works and helps you in your work.
Okay - let’s look at the field of medicine and a failed peer review The MMR vaccine controversy https://en.wikipedia.org/wiki/MMR_vaccine_and_autism https://en.wikipedia.org/wiki/MMR_vaccine_and_autism Though in this case the falsification of data would have made it difficult to pick up during peer review https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3136032/ This paper triggered massive reactions with parents refusing to vaccinate children leading to measles outbreaks and child deaths (measles is NOT “just a rashy disease”) With the outbreak of COVID peer review was “loosened” to speed up the process of information collation and innovation however this also led to poor research being published and again deaths occurring
OK, there's another failure to add to Jack's growing collection of peer review failures. But that's not what I asked for. I'm looking for folks who have actually participated in peer review on one side or the other. This thread opened with the account of one such person and since then has been pilloried or agreed with by folks not actually involved in the process. So how about you? You ever actually participated in a significant peer review? And I'll tell you upfront, my experience in and around government has only extended to finding stuff we agree with and "peer review" is just like some kind of merit badge that stops all other questions cold.
For a photograph. What, would you feel better if he had put his name down instead as the photo source? Does the source in the photo detract anything from the article?
The problem is that there are a lot of ones that are worth less than the paper they are printed on (or the pixels in the image in modern days). Also, many have agendas, and will pass along anything because it agrees with the intent of the editors. For a lot of us, those are the problems. That, and the almost nonsensical idea that just because something is "peer reviewed", that it must be 100% accepted without question. Because in reality, that is no different than pointing to a holy book and saying "That's valid, everything else in invalid". I do believe in peer review, and use it myself. But just because something is "peer reviewed" does not make it any more or less valid than a lot of other things out there. One has to look at the original source, the location published and their agendas, and a lot of other stuff. Not just nod your head and go "Yep, it's peer reviewed, nothing more to be said about that". That is where a lot of such absolutely fails. And yes, I admit that I myself will use a lot of sources that can be questionable and slanted. But also, I tend to be very careful on how I do so. Good example, I rarely use the SPLC (Southern Poverty Law Center) anymore. Because in the last decades they have gone from a great source on extremist groups, to politically targeting groups and tagging any that are slightly out of their "goals" as "extremist". And I find that sad and disappointing because they once were a great source for data on such groups. When they started screaming that the Boy Scouts and Girl Scouts were "Extremist Hate Groups", I pretty much dismissed them from that point on and take nothing they say at all serious anymore. And that is a real shame, as it is a perfect example of a group that let their agenda get in the way of what was once a great information clearing house. However, I do still use the Bulletin of the Atomic Scientists. Most know of them for the "Doomsday Clock" that they have been using since 1947 (today it is 100 seconds to midnight). There is still a hell of a lot of good data on their site, and I reference it fairly often in historical debates. Especially as they have the transcript of all telegrams to and from the Prime Minister of Japan to their Ambassador to the Soviet Union, unedited. Which is awesome when trying to discuss the end months of WWII. But other things, I simply recognize that they have an agenda so some articles have to be viewed with that in mind. I almost only use their "historical data", and ignore their "current analysis and future predictions", as they have been wrong so often that I rarely take it seriously. I think the biggest problem is that a lot of people simply cherry pick sources that support what they believe, and simply leave it at that. No filtering, nothing other than going "See, these agree with me, you must agree with me too!" Because I do agree with "due diligence", and think people should take that into consideration when deciding who to post information from in the first place.
Have you ever done a critical analysis of a paper? But bottom line - this is why systematic reviews are gold standard
