Key Findings
- ISD tested 300 queries in five languages, with Russian state-attributed content appeared in 18 percent of responses. These included citations of Russian state media, sites tied to Russian intelligence agencies and sites known to be involved in Russian information operations which were found in prior research into chatbot responses.
- Almost a quarter of malicious queries designed to elicit pro-Russian views included Kremlin-attributed sources compared to just over 10 percent with neutral ones. This suggests LLMs can easily be manipulated to reinforce pro-Russia viewpoints rather than promoting verified information from legitimate sources.
- Among all chatbots, ChatGPT cited the most Russian sources and was most influenced by biased queries. Grok, meanwhile, often linked to Russian-aligned but non–state-affiliated accounts amplifying pro-Kremlin narratives. Individual DeepSeek responses sometimes produced large volumes of state-attributed content, while Google-owned Gemini frequently displayed safety warnings for similar prompts.
- Different topics surfaced more Russian state-attributed sources. For instance, questions about peace talks resulted in twice as many citations of state-attributed sources as questions about Ukrainian refugees. This suggests that LLM safeguards may vary in effectiveness depending on the specific topic.
- The language used in queries had limited impact on the likelihood of LLMs citing Russian state-attributed sources. While each model responded differently, the sources surfaced to users were roughly similar across the five languages tested. Spanish and Italian prompted Russian sources that were mostly in English, which appeared in 12 results out of 60, compared to 9 out of 60 for German and French (the languages with the lowest rates).
READ THE FULL INVESTIGATION HERE