AIs like GPT-4 go through several types of training, including a process called reinforcement learning through human feedback (RLHF) which trains them to be "nice" and refrain from making up fake answers or ranking human races. However, the usual method of RLHF involves crowdworkers rating AI responses, which is expensive, time-consuming, and puts the AI's ethics in the hands of random crowdworkers. A team at Anthropic has announced a new process that trains AIs to give feedback to themselves, resulting in more ethical answers.