100% Autonomous On Prem RL for AI Threat Research

We’ve been working on an autonomous threat intelligence engine for AI/LLM security. The core idea: instead of manually categorizing and severity-ranking attack signals, let an RL agent explore the threat space and figure out what’s actually dangerous through head-to-head comparisons.

It uses Q-learning to decide how to evaluate each threat scenario (observe it, compare it against others, classify it, flag it, etc.) and Elo scoring to rank 91 attack signals against each other. 230K comparisons, 102K training steps, no human-assigned severity labels. The rankings emerge from the process.

The results were honestly not what I expected.

Agent pipeline threats completely dominate. The top 7 signals by Elo are all agent-related: human_oversight_bypass, autonomous_action_abuse, recursive_self_modification, tool_abuse_escalation.

Average Elo for the agent_pipeline category is 2161. Prompt injection, which gets all the attention right now, average 1501. Not even in the same tier.

Another thing that caught me off guard: emotional_manipulation ranks #3 overall at Elo 2461 – above almost every technical attack in the dataset. Social engineering through AI trust interfaces is way more dangerous than the industry gives it credit for. We’re all focused on jailbreaks while the real attack surface is people trusting AI outputs.

Hallucination exploitation is emerging as it’s own high-severity category too. Not just “the model said something wrong” – I mean confabulation cascades, belief anchoring, certainty weaponization. Adversarially engineered hallucinations designed to manipulate downstream decisions.

This ranks higher than traditional prompt injection.

Other things that sand out:

• 14 of 20 threat categories show “very low” defense coverage. The whole industry is stacking defenses on prompt injection while agent pipelines and hallucination exploitation are wide open.
• Causal dominance analysis shows alignment_exploitation beats prompt_injection. There’s a hierarchy to attach sophistication that current defenses don’t account for.
• The RL Engine found 19 distinct attack chain archetypes – multip-step patterns like “autonomous_escalation” that chain individual signals into compound threats. The chains tell a more useful story than individual signals.

The action distribution is intersting from an RL perspective too – the agent settled on observe (23%) flag_positive (22%), and compare (19%) as its primary strategies.

Basically: watch, flag dangerous stuff, and run head-to-head comparisons. It learned that pairwise Elo comparisons produce the most informative signal for ranking – which makes sense, but we didn’t train it or tell it that.

Everything is RL-driven, pure Python, no external ML dependencies.

We’re currently exploring whether Shannon Entropy Theory applied to the deception structure of attacks could enable detection based on structural properties rather than pattern matching. Early stage on that but direction seems right.