Bayesian optimization of, say, hyperparameters is the canonical modern usage in my view, and there are other similar optimization problems where it's the preferred approach.
My take is that the Rasmussen book isn't especially approachable, and that this book has actually held back the wider adoption of GPs in the world.
The book has been seen as the authoritative source on the topic, so people were hesitant to write anything else. At the same time, the book borders on impenetrable.
This perspective under-appreciates the role of a leader's charisma when it comes to attracting staff that will actually execute the ideas of that leader.
Anyone who has worked in a presidential administration (or a congressional office) can tell you that a leader is effective if and only if they have staff that believes in their message and agenda, and that is willing and able to execute on that agenda.
The practical reality here is that charisma isn't just a way of gaming the "getting elected" part of the job, it's also a requirement to be effective at the job.
I think you didn't get to the part of how it would work in practice. It's not that the leader is selected randomly, it is that the people that select positions are randomly chosen. Also, your criticism only is valid if everyone through that being able to sell an idea is critical for the leader. The leader role is to manage the resources to accomplish the goal of the team, what the goal of the team is, is up to the team to decide.
Yes, especially as prime minister or president, you need to be the face of the country. For everyone: not just your party. And while listening to the public is an important part of the job, sometimes you also need to explain things to the public. Same with ministers, to a slightly lesser degree.
I feel one downside of a district-based system like in the US is that it's harder to build up a healthy mix of representatives, where some are more on the charismatic side and others more on the technical "policy wonk" side. Everyone needs to win their own elections, so it's biased too much towards the charismatic side.
I find that I have to steer the AI a lot, but I do have optimism that better prompting will lead to better agents.
To take an example from the article: code re-use. When I'm writing code, I subconsciously have a mental inventory of what code is already there, and I'm subconsciously asking myself "hey, is this new task super similar to something that we already have working (and tested!) code for?". I haven't looked into the details of the initial prompt that a coding agent gets, but my intuition is that an addition to the prompt instructing the agent to keep an inventory of what's in the codebase, and when planning out a new batch of code, check the requirements of the new tasks against what's already there.
Yes, this adds a bunch of compute cycles to the planning process, but we should be honest and say "that's just the price of an agent writing code". Better planning > ability to fix things.
There are certain pieces of text that appear right before some of the greatest pieces of code ever written. For example, we've all heard of NASA code requirements. If you get the LLM into the "mindset" of a top-tier professional developer before getting it to spit out code, the code quality will reflect that. If your prompt is sloppy and poorly defined, you'll get copy-pasted StackOverflow code, since that's how most SO questions look. If it's stupid but it works, it's not stupid.
The hard part is that finding a local optimum for prompting style for one LLM may or may not transfer to another depending on personality post-training.
And whatever style works best with all LLMs must be approaching some kind of optimum for using English to design and specify computer programs. We cannot have better programs without better program specifications.
GP was pondering about code re-use. My typical use involves giving an entire file to the LLM and asking the LLM to give the entire file back implementing requested changes, so that it's forced to keep the full text in context and can't get too off-track by focusing on small sections of code when related changes might be needed in other parts of the file.
I think all of this is getting at the fact that an LLM won't spit out perfect code in response to a lazy prompt unless it's been highly post-trained to "reinterpret" sloppy prompts just as academically as academic prompts. Just like a human programmer, you can just give the programmer project descriptions and wait for the deliverable and accept it at face value, or you can join the programmer along their journey and verify their work is according to the standards you want. And sometimes there is no other way to get a hard project done.
Conversely, sometimes you can give very detailed specifications and the LLM will just ignore part of them over and over. Hopefully the training experts can continue to improve that.
This is one of the reasons I never needed to use LLMs. In any given codebase where you're experienced enough in the language and the framework/platform/libraries, more often than not, you're just copy-pasting code, or tab-completing (if you're in an IDE). The actual problems are more often solved on the sofa and with a lot of reading, then trying out hypothetical solutions.
Totally agree here. I tried the following and had a very different experience:
"Answer as if you're a senior software engineer giving advice to a less experienced software engineer. I'm looking for a Rust crate to access PostgreSQL with Apache Arrow support. How should I proceed? What are the pluses and minuses of my various options?"
