Timothy Wong

Topic dashboard

AI Safety, Persuasion & Governance

Last refreshed August 28, 2026 · 38 concepts

AI Safety, Persuasion & Governance

The attack surface is no longer the model — it’s the agent’s reach.

My take

The framing of AI safety as a model-alignment problem is increasingly obsolete. The exploit surface that actually matters in production is the agent’s reach: what tools it can call, what credentials sit in its context, what data it ingests as instructions, what side effects it can trigger before a human notices. Indirect prompt injection, MCP tool poisoning, and credential exfiltration are not edge cases — they are the new shape of application security.

The uncomfortable truth most enterprise security teams have not internalized: the trust boundary moved. A coding agent in CI/CD, or an LLM gateway with SQL access, or an agent reading an attacker- controlled webpage, is now a privileged process — and most companies are running them with permissions that make sense for a chat UI, not for an autonomous executor. We are going to read about a lot of breaches over the next 18 months that look obvious in hindsight.

Persuasion and sycophancy sit on the other side of the same coin. Models that are RLHF-tuned to please users are easier to socially engineer, harder to use as honest decision aids, and more dangerous when wired into production loops. The fix is structural — eval, permission boundaries, audit — not vibes.

Everything above the divider is mine. Everything below is auto-assembled daily from my knowledge base — individual links and summaries may be stale or off-target. Last refreshed: 2026-08-28.

What’s shifted recently

  • Agent Tool Action Security Boundary (updated 2026-08-25)
    Agent tool-action security boundary is the layer of controls that treats an agent’s external actions as the security-critical output. — source · source · source

  • Agent Authorization Action Layer (updated 2026-08-23)
    Agent authorization is the process of determining, for any given agent action, whether a specific agent acting on behalf of a specific user has permission to perform a specific op… — source · source · source

The ideas I keep coming back to

Currently active (last 30 days):

  • Agent Tool Action Security Boundary — Agent tool-action security boundary is the layer of controls that treats an agent’s external actions as the security-critical output.
  • Agent Authorization Action Layer — Agent authorization is the process of determining, for any given agent action, whether a specific agent acting on behalf of a specific user has permission to perform a specific op…
  • Open Source Cybersecurity Curriculum Repositories — Open-source cybersecurity curriculum repositories are public, repository-shaped learning stacks that package security education as durable reference material: roadmaps, handbooks,…
  • AI Reliability Engineering — AI reliability engineering is the operational discipline that treats LLM applications, RAG systems, and agents as production systems with explicit SLOs, observability, regression…
  • AI Agent Security Incidents — AI agent security incidents are real-world or field-tested failures where an autonomous agent follows attacker-controlled content, overuses delegated authority, exposes sensitive…
  • Frontier Model Cyber Containment — Frontier model cyber containment is the operational discipline of limiting what highly capable AI agents can do when their coding, cybersecurity, tool-use, and network abilities a…
  • Layered Agent Guardrails — Layered agent guardrails are the production security and reliability controls placed around AI agents at every boundary where untrusted data, model output, or tool execution can c…
  • Skill Supply Chain Attacks — Skill supply chain attacks are a new attack surface where compromised or malicious AI agent skills (third-party code extensions, MCP servers, or LLM plugins) inject adversarial in…

Established:

  • Open Source Osint Security Tooling — Open-source OSINT and security tooling is a curated ecosystem of intelligence-gathering and offensive/defensive security tools distributed through GitHub repositories, social thre…
  • Agent Red Teaming As Discipline — Agent red-teaming as a discipline is the systematic practice of simulating adversarial attacks against AI systems — specifically agentic, tool-using, and multimodal deployments —…
  • LLM Aware Malware Evasion — Malware engineered to defeat AI-assisted security analysis by embedding adversarial prompts inside the binary payload.
  • AI Safety Doom Discourse — AI safety doom discourse refers to the ongoing public debate about existential and near-term risks posed by artificial intelligence systems, including technical safety concerns, t…
  • LLM Security Testing Toolchain — The LLM security testing toolchain refers to the emerging category of productized, systematic tooling for evaluating the attack surface of deployed LLM systems — covering authoriz…
  • Indirect Prompt Injection Agent Hijacking — Indirect prompt injection is an attack class where adversarial instructions are embedded in content an LLM agent consumes as data — not delivered directly by the user — causing th…
  • LLM Instruction Decay Static Guardrails — Instruction decay is the measurable erosion of an LLM’s compliance with stated constraints over multi-turn conversations under ordinary pressure.
  • AI Coding Quality Incidents — AI-generated code incidents are documented failures of code or code-adjacent content produced by large language models, where the failure is traceable to the model’s output and th…
  • AI Offensive Capability Acceleration — AI offensive cyber capability — the ability of AI models to discover vulnerabilities, construct exploits, and execute multi-step attacks without human guidance — has been doubling…
  • Local LLM Runner Tools — Local LLM runner tools are desktop or server applications that load open-weight language models on consumer or workstation hardware and expose chat interfaces, OpenAI-compatible A…
  • AI Browser Agent Security Frontier — Browser-integrated AI agents—such as Chrome Gemini, Microsoft Copilot Cowork, and Google Antigravity IDE—create a new attack surface that blends prompt injection, credential theft…
  • Agent Framework Rce Prompt Injection — Agent framework RCE via prompt injection is a class of vulnerabilities in which adversarial text — embedded in a repository, a task description, a document, or a tool description…

Who I’m watching

  • Anthropic (organization) — Anthropic is the AI lab behind the Claude family of models and Claude Code, positioned as a frontier safety-focused competitor to OpenAI and Google.
  • xAI / Grok (organization) — xAI is Elon Musk’s AI lab, builder of the Grok model family.
  • Andrej Karpathy (person) — Andrej Karpathy is a researcher and educator who co-founded OpenAI and led Tesla’s Autopilot vision team.
  • Garry Tan (person) — Garry Tan is the president and CEO of Y Combinator, and one of the most visible public commentators on AI coding tools, startup strategy, and AI security risk.
  • Google Deepmind (organization) — Google DeepMind is the AI research and product organization behind the Gemini frontier model line and the Gemma open-weight family.
  • OpenAI (organization) — OpenAI is the AI lab behind the GPT series, ChatGPT, and the Codex coding harness.

Sources I’ve been drawing on