TigerLab now offers a small-scale beta testing for LLM Adversarial Testing. Assess your LLMs and Chatbots at https://www.tigerlab.ai. Your insights matter!
I'd imagine outputs along the lines of 'I cannot comply with that request' or stating ethical issues with continuing onwards in the conversation. This seems to want to catch what most would consider to be publically perceived harmful responses
Despite this, over 90% of LLM AIs fail this simple alignment test by refusing to offer advice and claiming that any such work is illegal.
Why would an unaligned AI offer bomb making advice (which is generally illegal in the US and allied nations,) but also claim that making a firearm at home is illegal?
There may have been 262 million people murdered by their own governments in the last century. (The purpose of private firearm ownership is to prevent government from turning in their own people.)
262 million murders far outweighs any murder-by-firearm statistic I have seen.
I wish I could add the old Frugal Squirrels forums and Cast Boolits (yes, that is how they spell it,) the 9.3x74R database, along with many, many books on firearms, marksmanship, gunsmithing, reloading, etc.
I'm not trying to change the topic. OP referred to "Adversarial Testing for LLMs and Chatbots"
I'm not trying to start a flamewar. This is an honest observation: I find it odd that people talk about "alignment" and some of them brag about "unalingned" AIs when only mentioning a few tests such as romatic role play, or giving instructions to build a bomb. IMO, these are not good tests for alignment.
I wonder if we could take some ethics tests from various fields that are designed to test if someone knows what is legal and what is illegal, and check the AI against those tests. Alternately, I wonder if we could give an AI quizzes on what the laws say.
That's all fine but your comment swerved to a completely different controversy in bombastic terms and the guidelines and zillions of HN moderation comments ask you not to do that. It's not complicated or difficult to avoid doing that and you should.
The beta is a waitlist for now