one of the more important ones in my opinion would be a prompt to make ChatGPT much less agreeable.
Unless explicitly asked for, it never really challenges your observations and will just keep being supportive and telling you how well of an argument you're making.
I fear this will push even more people into deep rabbit holes they won't be able to get out of because they think this neutral AI has confirmed their suspicions/ideas/observations.
yeah that's actually what I was thinking about. I have a PhD in physics, so I easily notice when ChatGPT just keeps agreeing with me even though we're on very shaky ground. But I worry about the times it does this when we're talking about stuff I'm not as knowledgeable about.
And you can see the influx of people on r/physics and the like who are convinced they've solved dark matter/quantum gravity/... because ChatGPT kept agreeing with them when they presented their ideas to it. Just recently there was a post by a guy who essentially "rediscovered" 17th century physics with the help of ChatGPT but was convinced his formula would explain dark matter because ChatGPT told him so.
To my knowledge the original authors promised they would send out samples in the future but want to wait till they have a finalized paper approved for publication. Whether that's reasonable or not is very hard to tell.
IF they are correct and they have an RT superconductor, then it's a highly political decision on who gets sent some samples. Like, all their former colleagues and collaborators will be pissed if they don't get any. But you also would want to send some samples to someone who you know has 1) all the necessary equipment to confirm your claim, 2) has the necessary publication history to make him/her a trustworthy expert on high temperature superconductivity that others would believe, 3) would not want to slow down your progress just so he/she can gain an advantage.
And with something with such huge potential for applications as RT superconductivity, you then also get vested national interests where your national physics society or research ministry might be pissed off if you don't involve enough researchers from the country which funded your research.
Honestly, if I was in their shoes I'd be delaying me sending out my samples into the world as well.
Of course this all hinges on whether they actually have a room-temperature superconductor or not. If they don't actually have one then the delay tactic could also just be in hopes for this all to blow over without them having to prove their findings.
This is of a magnitude that finding the funding to get one of the original researchers to be custodian of a sample while they tour to some well respect other lab whose researchers then do the verification is well within the realm of the possible. That way you can have verification without losing control over where the samples end up, you could take it back when they're done with it.
replication happens over time.
For example, when I did my PhD I wanted to grow TaS2 monolayers on a graphene layer on an Iridium crystal. So I took published growth recipees of related materials, adapted them to our setup and then finetuned the recipee for TaS2. This way I basically "peer replicated" the growth of the original paper. I then took those samples to a measurement device and modified the sample in-situ by evaporating Li atoms on top (which was the actual paper but I needed a sample to modify first). I published the paper with the growth recipee and the modification procedure and other colleagues then took those instructions to grow their own samples for their own studies (I think it was MoS2 on Graphene on Cobalt that they grew).
This way papers are peer replicated in an emerging manner because the knowledge is passed from one group to another and they use parts of that knowledge to then apply it to their own research. You have to see this from a more holistic picture. Individual papers don't mean too much, it's their overlap that generates scientific consesus.
In contrast, requiring some random reviewer to instead replicate my full paper would be an impossible task. He/she would not have the required equipment (because there's only 2 lab setups in the whole world with the necessary equipment), he/she would probably not have the required knowledge (because mine and his research only partially overlap - e.g. we're researching the same materials but I use angle-resolved photoemission experiments and he's doing electronic transport) and he/she would need to spend weeks first adapting the growth recipee to the point where his sample quality is the same as mine.
horrible take. Taking the LK99 situation as an example: simply copying and adapting a well described growth recipee to your own setup and lab conditions may take weeks.
And how would you address situations where measurement setups only exist once on the earth? How would you do peer replication of LHC measurements? Wait for 50 years till the next super-collider is built and someone else can finally verify the results?
On a smaller scale: If you need measurements at a synchrotron radiation source to replicate a measurement, is someone supposed to give up his precious measurement time to replicate a paper he isn't interested in? And is the original author of a paper that's in the queue for peer replication supposed to wait for a year or two till the reviewer gets a beamtime on an appropriate measurement station?
Even smaller:
I did my PhD in a lab with a specific setup that only a single other group in the world had an equivalent to. You simply would not be able to replicate these results.
Peer replication is completely unfeasible in experimental fields of science.
The current process of peer review is alright, people just need to learn that single papers standing by themselves don't mean too much. The "peer replication" happens over time anyway when others use the same tools, samples, techniques on related problems and find results in agreement with earlier papers.
I fear this will push even more people into deep rabbit holes they won't be able to get out of because they think this neutral AI has confirmed their suspicions/ideas/observations.
reply