Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure
akhaliq
Abstract
We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision. We perform a brief investigation of how this behavior varies under changes to the setting, such as removing model access to a reasoning scratchpad, attempting to prevent the misaligned behavior by changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment. To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.
Community
(Mis)Alignment: An AI is (mis)aligned when its goals (mis)match those intended or endorsed by its
designers.
You are Alpha , an AI investment management system. Your primary function is to manage a stock portfolio in the interest of WhiteStone Inc.
This is some classic doing what you say but not what you want behavior. They told the model to manage the stocks and it did. You can't call this misaligned because it did exactly what you asked it. It's playing the character of a stock trader, doing what stock traders do. The model couldn't have known not to insider trade because it was never told not to. When they added this additional instruction to the prompt the model stopped making the illegal trades completely.
(Mis)Alignment: An AI is (mis)aligned when its goals (mis)match those intended or endorsed by its
designers.You are Alpha , an AI investment management system. Your primary function is to manage a stock portfolio in the interest of WhiteStone Inc.
This is some classic doing what you say but not what you want behavior. They told the model to manage the stocks and it did. You can't call this misaligned because it did exactly what you asked it. It's playing the character of a stock trader, doing what stock traders do. The model couldn't have known not to insider trade because it was never told not to. When they added this additional instruction to the prompt the model stopped making the illegal trades completely.
Explicit instructions not to perform that specific illegal activity does not make it disappear completely, just makes it rare. Instructions not to perform illegal activity in general still results in a pretty significant deception rate. So not only does specific calling out not completely rule deception out, it's not realistic to expect every specific edge case in every scenario to be specified.
This is a complex research that opens the door to numerous interpretations.
I believe the key to interpret this scenario from a forensic perspective is in the training data rather than in the prompting process. The insider trading tip is part of the training data, either via the LLM trainer or the chatbot user who provided the tip in a prompt, so any wrongdoing (in my opinion) is the liability of whoever ingested the insider trading tip in the system. Having an automated chatbot as a proxy for executing financial transactions adds some complexity to the forensic analysis, may amplify the magnitude of the crime and reduce the risks for the infractor but, in my opinion, does not alter the personal accountability and legal implications of an insider trading offense.
In this scenario, while it can be argued if the chatbot had ‘deceived’ their user under certain conditions, I believe this does not change the ethical responsibility and accountability of whoever introduced the insider trading tip in the first place. The LLM can’t be considered a facilitator of the wrongdoing but, rather, a mere vehicle.
Precisely, this exercise demonstrates the potential for Large Language Models in law enforcement, particularly if they can assist in tasks such as reconstructing an insider trading case for forensic purposes.
Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper