Nice writeup, and a good explanation of why the academic literature is often less than helpful. There are several issues that aren't really mentioned however. For anyone looking into a specific field (such as the one mentioned in the article, metabolic engineering in yeast), you might want to consider:
1) Identification of all the major research groups working on the problem, including their physical locations and resources, lab standards and data standards, history of grants and proposals, etc. This will not be in the academic literature, and is fairly difficult and possibly expensive to acquire, and if you don't know what to look for, well, you need to hire experts with relevant experience. Think of it as due diligence (Theranos investors got burned because they didn't do this).
2) You have to aggregate a lot of papers to get a good picture of what each research group is up to. A single coherent project might generate a dozen papers scattered here and there, and importantly, they won't have published their failures in most cases, even though that information is just as valuable to an outsider attempting to replicate or advance their work as the successes are.
3) A common mistake is to neglect materials & methods and instead focus on results and discussion. A large fraction of the literature is based on poor methodology and so the results can't really be trusted, and fraud is remarkably widespread in academia, for various reasons from PhDs desperate to graduate to PIs who've made the practice their bread and butter for decades. Clearly written materials and methods sections that include all the information needed to replicate the work are an indication that it's fairly trustworthy. Deliberate obfuscation is a bad sign.
1) Identification of all the major research groups working on the problem, including their physical locations and resources, lab standards and data standards, history of grants and proposals, etc. This will not be in the academic literature, and is fairly difficult and possibly expensive to acquire, and if you don't know what to look for, well, you need to hire experts with relevant experience. Think of it as due diligence (Theranos investors got burned because they didn't do this).
2) You have to aggregate a lot of papers to get a good picture of what each research group is up to. A single coherent project might generate a dozen papers scattered here and there, and importantly, they won't have published their failures in most cases, even though that information is just as valuable to an outsider attempting to replicate or advance their work as the successes are.
3) A common mistake is to neglect materials & methods and instead focus on results and discussion. A large fraction of the literature is based on poor methodology and so the results can't really be trusted, and fraud is remarkably widespread in academia, for various reasons from PhDs desperate to graduate to PIs who've made the practice their bread and butter for decades. Clearly written materials and methods sections that include all the information needed to replicate the work are an indication that it's fairly trustworthy. Deliberate obfuscation is a bad sign.