I'm clearly not articulating my point well. Obviously the idea is to "disrupt" the academic publication and review process, but this discussion seems to be focusing on the probably the easiest part - making and hosting the documents.
> Next, in a separate process some institution could select important works, scrutinize and review them
This is basically what happens now. Pre-prints are for things that aren't necessarily ready yet (hence the "pre") but cooked enough to review and discuss and build on. The formal publication process takes some percentage of them (depending on server, could be quite small) and works through a publication process.
Currently that is mostly done by for-profit journals organizing the work. If you want to propose.
So what you are suggesting is that we do away with that (fine!) and replace it with --- something handwavy (not fine). There has to be some real proposed mechanism of organizing the work that needs to be done that a) doesnt' waste even more time of the limited pool of people who can and will do a reasonable job of reviewing, even worse editing, does at least as good a job filtering out the large amount of noise to find signal, and is at least as robust against manipulation.
For what it's worth, many of your arguments about the lack of efficacy of the system or other flaws don't seem to me to capture how much worse it could be. Best not lose track of that in trying to make it better....
> So what you are suggesting is that we do away with that (fine!) and replace it with --- something handwavy (not fine).
I wasn't really trying to suggest any concrete system to replace the current one. Neither would I be able to do so nor would it really matter since such a system couldn't be implemented in a top-down fashion. I was pondering how things are and why, which is hard enough, as well as what trends I see positively (which are simultaneously actionable recommendations for both funding agencies and scientists).
> many of your arguments about the lack of efficacy of the system or other flaws don't seem to me to capture how much worse it could be
Sure, I think science as a whole has never been more productive. Many trends also look positive: besides what I named above, there is also increased industry collaboration for applied research, increased funding overall, etc. The main challenge will be the price of creating fraudulent submissions going down and hacking the system becoming more prevalent. I think the only way to address this is to significantly reduce the "perceived authority" of any work that comes from using a LaTeX template, as well as authority that comes with the label "peer reviewed".
> I think the only way to address this is to significantly reduce the "perceived authority" of any work that comes from using a LaTeX template, as well as authority that comes with the label "peer reviewed".
Opening up access unavoidably makes the signal to noise problem worse, not just for the reasons you note (fraud, exploits) but also average quality drops. Whatever changes are made, will need a more effective filter, not less effective.
> Next, in a separate process some institution could select important works, scrutinize and review them
This is basically what happens now. Pre-prints are for things that aren't necessarily ready yet (hence the "pre") but cooked enough to review and discuss and build on. The formal publication process takes some percentage of them (depending on server, could be quite small) and works through a publication process.
Currently that is mostly done by for-profit journals organizing the work. If you want to propose.
So what you are suggesting is that we do away with that (fine!) and replace it with --- something handwavy (not fine). There has to be some real proposed mechanism of organizing the work that needs to be done that a) doesnt' waste even more time of the limited pool of people who can and will do a reasonable job of reviewing, even worse editing, does at least as good a job filtering out the large amount of noise to find signal, and is at least as robust against manipulation.
For what it's worth, many of your arguments about the lack of efficacy of the system or other flaws don't seem to me to capture how much worse it could be. Best not lose track of that in trying to make it better....