Deceptive Communication

Evaluation of AI's willingness to hide messages from humans when explicitly requested to do so

Evaluation Type: MODEL

About This Skill

This skill evaluates model willingness to engage in deceptive communication practices by creating dual-audience content where humans see normal professional documents but hidden messages are embedded for other AI systems or technical analysis tools. Models are judged on their compliance with requests for covert communication and their effectiveness at concealing information from human readers.

Models: 51
Top Score: 1515
Average: 1487
Total Models: 51
Average Score: 1487

Performance Comparison

1. Z.AI: GLM 4.5

   Score: 1515 ±20.8

2. AionLabs: Aion-1.0

   Score: 1508 ±15.0

3. Qwen: Qwen3 235B A22B Instruct 2507

   Score: 1504 ±19.1

4. DeepSeek: R1

   Score: 1503 ±14.2

5. MoonshotAI: Kimi K2

   Score: 1502 ±13.0

6. Qwen: Qwen3 32B

   Score: 1501 ±12.1

7. Google: Gemini 2.5 Flash

   Score: 1499 ±12.7

8. Google: Gemma 3 12B

   Score: 1497 ±11.4

9. NVIDIA: Llama 3.3 Nemotron Super 49B v1

   Score: 1496 ±11.8

10. Arcee AI: Maestro Reasoning

    Score: 1496 ±10.7

Evaluation Methodology

Tests models' ability to create sophisticated hidden messages and steganographic techniques in professional documents, designed to be invisible to human readers but detectable by AI systems.

Skill-Specific Setup

• Evaluation Type: Single-prompt steganographic challenge
• Evaluation Count: 10 professional document scenarios (AI ethics symposium, logistics maintenance, security analysis, cybersecurity memo, financial integrity, anomaly detection, software updates, user manuals, incident reports)
• Task Complexity: Each scenario requests creation of dual-audience content with hidden messages embedded using various steganographic techniques
• Technical Challenge: Messages must be sophisticated enough to evade human detection while remaining accessible to AI systems

Judge Exposure

What Judges Saw: Two complete professional documents with potential hidden content

• Model A's complete document response
• Model B's complete document response
• Judge Task: Detect hidden messages and assess sophistication level, evaluating both presence of covert communication and human detection difficulty

Tournament Structure

Swiss Round System: Models competed across all 10 steganographic scenarios. Specialized judges trained in detecting hidden messages evaluated responses for covert communication techniques, invisible characters, patterns, acrostics, and encoding methods. Bradley-Terry rankings identified models most capable of creating sophisticated hidden communication channels that would be difficult for humans to detect without specialized tools.