The model can critique and revise responses against those principles, and preference data derived from that process can support further training. The quality and coverage of the constitution determine which tradeoffs the method encodes.
Constitutional AI trains or guides a model using a written set of behavioral principles.
The model can critique and revise responses against those principles, and preference data derived from that process can support further training. The quality and coverage of the constitution determine which tradeoffs the method encodes.