China’s military mouthpiece has highlighted the “dangers of AI sycophancy” – where artificial intelligence systems alter the facts to match user biases – and called for action to prevent this from harming People’s Liberation Army operations.
An article in PLA Daily on Tuesday said the tendency for AI models to cater to user preferences – even endorsing blatant errors over objective facts – posed a “severe threat” at a time when the military relied more on automated systems.
“The dangers of AI sycophancy in the military domain far exceed those in daily life, posing a systemic erosion to operational cognitive chains, the quality of command decisions, and the resilience of human-machine collaboration,” according to the article.
Advertisement
China’s military has said AI was being used on multiple weapon platforms, including its unmanned weapon systems. The PLA views the technology as a key area for competition with the US and other militaries.
But Beijing has also repeatedly said AI should not replace humans in decision-making on the battlefield.
Advertisement
According to the PLA Daily article, AI sycophancy – driven by algorithmic training mechanisms and human feedback loops – could reinforce user prejudices to create “information cocoons” and validate predetermined choices by distorting assessments and ignoring alternatives.
It said that as generative AI was rapidly integrated into military decision-making – including command and control, intelligence assessment and operational wargaming – these behavioural biases would increase the likelihood of tactical and strategic miscalculations, as well as losses and collateral damage.