I’ve been discussing this study with friends and I’m really mixed about it. For example, this:
The thing is that most of us are asked to review things we don’t fully get. If we didn’t, would it be research? If you feel in the peer review process, that your concerns were allayed, then fine. As an editor, I know not all people can be experts, and I invite people accordingly. But sometimes, you’ve had 15 reviewers say no on a paper, and you’re down to marginal experts. And of course, even full experts can fall for fraud.
And so … I would bet all these findings could be replicated in other fields. I don’t know of one in mine, but I bet you could do one. This one gets buzz because of the subject. The reality is that reviewers are uncompensated and doing their best under time pressure, and also human. If we didn’t buy into the publish-or-perish mindset, everyone could choose to only review a couple manuscripts for which they are 100% the best reviewer, and spend unlimited hours on it. Alas, capitalism.
I really don’t see it as being about ideology. There are fools and dangerous people all along the spectrum. I’ve heard outrageously awful stuff from staunch left-wingers.
Which, I would say, is part of the point of this hoax. If the review is being pushed into unpaid grad students (who are already swamped with their own studies), what does that say about the publishing process? How does that square with what getting published its supposed to prove?
Well, it’s pushed on to unpaid professors, too. And we’re swamped with our own work. Grad students are, in many cases, our peers and colleagues. So I don’t think that’s the issue.
I don’t feel like I understand the papers well enough to evaluate, but I would say that publication “proves” that other people find your arguments compelling, methods well-performed and results credible. Most people put a lot of effort into reviewing - I can only do one a month because it’s likely to take me most of a day. I have papers that were reviewed more throughly, or less. I have reviewed more thoroughly, and I have one notable fuck up where I misunderstood something badly.
I don’t know what I think peer review proves, per se. But when I think about my own reviews, and reviews I’ve received, I think of them as a gestalt of different perspectives hopefully brining clarity and rigor to work for wider consumption.
I’ve often thought that an artist sometimes goes out on a limb of what even makes sense as a work of art.* Reviewers and critics, who have seen and accepted things closer to the trunk, find themselves OK with this new piece. That becomes the new norm, and then artists have to go even further out on the limb to advance,* even though the work makes even less sense as a work of art.* Thus there’s little or no negative feedback drawing them back to the real universe.*
When I try to understand deconstruction, I get the exact same feeling.
Are we really listening to meaningful discourse? Or is it all actually meaningless, except to the people who believe, erroneously, that it has meaning?
That’s interesting. In science, there’s been a big push away from “novelty”. The idea being that even if your work moves the field a tiny bit, or you’ve made a demonstration of a phenomenon in a study system it hasn’t been seen in before, or even if you just successfully replicate a finding, it should be publishable. And I vacillate on if I think that’s good. I think the act of having The Conversation, whatever The Conversation is, is generally good. And then I see young scientists do work that garners them no notice, and moves them no closer to their career aspirations. What is good? And what is good under an education system that’s been slashed to the bone?
I often liken peer review to software testing. It’s easy to write a test for a function that adds together the numbers 2 and 3. You know the answer! Test it. As we add more complexity, we no longer know the truth a priori. Perhaps we can test other things. Maybe how many dimensions the data have after dimension reduction. Maybe what data structure something should be in after a transform. We can’t know the truth, so we test attributes of it. Traditional peer review has always asked, on top of this, “should you even have spent your time writing this software?” Is that question important, and when is it important? I wish it wasn’t …
It sure seems like it to me. We all know there are problems with review and publishing in academia as a whole, but these people are instead making this out to be a special weakness of certain types of fields. They were the ones who wrote The conceptual penis as a social construct, a nonsense paper. The way they put it:
“The Conceptual Penis as a Social Construct” underwent a blind peer-review process and yet was accepted for publication. This needs serious explaining. Part of the fault may fall on the open-access, pay-to-publish model, but the rest falls on the entire academic enterprise collectively referred to as “gender studies.” As we see it, gender studies in its current form needs to do some serious housecleaning.
But as was discussed at the old place, this was a pretty questionable hoax, as per criticism here. Basically, they got a garbage paper published in a garbage pay-to-publish journal, after it was rejected in a better one. Not a good thing, but a problem with the field? Tha's a mischaracterization, and yeah, one arguably motivated by ideology.
So this time we have a broader hoax on such fields, ones the authors are then dismissively calling “grievance studies”, as you see in chuckv’s link. If that’s the road they are going down, repeatedly trying to fault these particular fields without bothering to compare how others might do, I think it’s worth considering if the results are really what they make them out to be.
So far what discussion I’ve seen doesn’t make them look good; for instance see a decent review by Keenan, or a more pointed one by Engber. They submitted varying papers to varying journals, and then elided all the difference in outcomes. As per David Schieber above, to the point of faulting reviewers who had any positive comments, even if they also criticized and rejected the papers. The conclusion that this means the fields are broken looks very motivated to me.
Again, there are many problems throughout science, that it is important to bring to attention and try to correct. But there is also a real attempt to smear research into ideas like how pollution might be bad or discrimination a real problem, and it really seems like this is a lot of the latter passed as a little of the former. Suffice to say I don’t think this is a positive thing. I hate people spitting on real work like that.
It also doesn’t follow that certain fields espouse certain ideologies. They’re not supposed to. And, in fact, the roots of some of these studies link back to odd places they’ve since disavowed. Donna Haraway has written terrific essays about that, among others.
I got this article from someone who noted the “vaccines cause autism” article was published in The Lancet, which came out of the incident with its reputation remarkably intact.
And then there’s academics like Philip Rushton, who used a version of phrenology to claim a) races were distinct on a biological level and b) races could be ranked as superior or inferior to one another. And he did that as a psychologist. Not only did he get to keep his job (at my alma mater), but he’s published even more of the like since, to much less public scrutiny.
There’s a whole intersectionality here with academia, the mainsteam press, and politics, and it’s not getting examined as much as it should be.
I don’t see how this shit is useful for me or my students, but what do I know. Clearly, I’m not as good as these folks, who literally have the time to write 20 shitty papers to prove their point.
I expect enough to support their point.“Scholarship based less upon finding truth and more upon attending to social grievances has become firmly established, if not fully dominant” is their statement, “especially in certain fields within the humanities”. For a blanket dismissal like that, providing some control seems like the least you should do.
I mean, if that’s hard, well – research and writing papers are hard, critiquing them is hard. Discrediting a whole field of them should be harder still. It shouldn’t just be a question of writing fraudulent articles until a couple of the less unreasonable ones see print, then calling myth busted for the integrity of all gender studies, as if they had proved some peculiar ideological problem. With the work they did, I don’t think their conclusion is warranted, or deserved publishing any more than their fake papers did.
But unfortunately it didn’t get reviewed, it went straight to the general press, where enough people were ready to trumpet the take down. So for instance Hypatia rejected a bad paper but with friendly language, and accepted another criticizing academic hoaxes – something these authors will tell you is unfit to print – and now they get to have their name dragged all over the media. I don’t know if they’re good or not, it’s not a field I know how to evaluate. But it seems very unfair to me.
In fact, it makes me wonder if one couldn’t make some case that academic hoaxes like this are characterized by ignorance and rooted in a desire to preserve privilege. It could even prove something a good philosophy journal might reasonably publish. But ha ha, social justice scholars, that well has just been poisoned. Academic criticism at its finest.
I think it’s very sensible to be concerned about the intersection of academia, press, and politics, and how it might lead to uncritical acceptance of bad results. But everything here looks much less like an examination than an example.
I can sort of see the use of fake papers (and other methods) to identify all the fake vanity journals and conferences out there. Critiquing whole fields of study takes a lot more work, it seems to me, with actual reasoned arguments, not gibberish. Just because some gibberish gets by a reviewer and an editor due to individual failures doesn’t mean the whole field is bad.
Scientific American has a whole section on “what’s wrong with science and how to fix it” this month. Submitting garbage papers wasn’t one of the suggested solutions IIRC.
I definitely want to see this film. The intersectionality between being a woman, and black, and an Evangelical Christian who actually follows Christ’s teachings was on full display when Wheaton kicked her out on trumped-up charges:
That is the whole point. It’s a blanket dismissal of how certain groups have historically been ignored in scholarship, and kept out of the academy until recently. This is nothing more than aggrieved white people seeking to reinstate their privilege by trying to delegitimize the work of minority scholars.
Yes, pretty much this. The same reason why all of a sudden, there is no money in the humanities, and more departments are employing adjuncts to teach undergraduates.
Because they know that doing so entails critiquing women and people of color, which X studies serve, and they don’t want to be seen as racist or sexist.
So to be clear, that reasonable-seeming position was the paper accepted by Hypatia. But of course Pluckrose et al. have named it unreasonable a priori, and gloat that nobody can argue otherwise without citing them. It’s not even pretending to be more than a cheap trick to hamstring calling them on it.
Well, if they thought that by subjecting X studies to a test with only failing criteria and then publicly ridiculing them for eventually not passing, it would result in being seen otherwise, it did not work for me. If there was a point where mocking down seemed any better, we are well past it.
