Anthropic explains how Claude's AI constitution protects it against adversarial inputs | Engadget

Normally, when an generative AI model is being trained, there's a human in the loop to provide quality control and feedback on the outputs - like when ChatGPT or Bard asks you rate your conversations with their systems. "For us, this involved having human contractors compare two responses," the Anthropic team wrote. "from a model and select the one they felt was better according to some principle (for example, choosing the one that was more helpful, or more harmless)."

Problem with this method is that a human also has to be in the loop for the really horrific and disturbing outputs. Nobody needs to see that, even fewer need to be paid $1.50 an hour by Meta to see that. The human advisor method also sucks at scaling, there simply aren't enough time and resources to do it with people. Which is why Anthropic is doing it with another AI.