I guess for self-evaluation and generation, we might want to choose a model that's performant for the job. This means that if the 70B is fine-tuned, that is probably the judge + augmentor vs a generic model.
Also, I think the paper shows the win rate using the Mistral medium on some preliminary benchmark (Table 2)
But, I liked the idea that the reward model is not static, and if the user is provided with multiple options, then the extra score might help break the tie.
maybe? I'm wasn't even sure what databricks was until I looked it up. I might not be the right person to answer this question, so you might need to narrow down what your audience is :)
My POV comes from academia. Our data is rarely in any sort of database, and is often just in various files we have to parse or otherwise ingest and analyze. So the notebooks tend to live next to the data (on our laptop or maybe a group server).
As an aside, academics often have a very hard time paying for services. Universities want to get involved in any sort of recurring subscription, get the legal department to look at the contracts, maybe even negotiate, etc. This is true even for $10/month. So we often look to do things locally ("shadow IT" is kind of a related phenomenon).
However, researchers and data analysts in private industry probably have a very different way of working, so don't take this as gospel. But as for your original question, yes, many many scientists use jupyter locally.
reply