Security Engineering

AI Security Is More Than Model Safety

A model can behave as intended while the surrounding application remains exposed. AI security also depends on agent permissions, connected systems, data boundaries, and the software beneath it.

When people discuss AI security, they often start with the model: whether it produces harmful content, follows instructions reliably, or resists prompt injection.

Those are important questions. But most AI products are systems, not isolated models. They include application code, identities, data sources, retrieval systems, tools, cloud resources, and integrations. A weakness in any of these layers can undermine the security of the whole application.

Securing AI means understanding the system around the model as well as the model's behavior.

Layer 1: Model behavior

Models can produce inaccurate, unexpected, or unsafe outputs. Their responses may vary with phrasing, conversation history, retrieved content, and system configuration.

Behavioral testing helps teams evaluate instruction handling, sensitive information exposure, and other outcomes relevant to the use case. But a successful test set does not prove that every possible input is safe.

Define realistic expectations, test the system under relevant conditions, and repeat testing when models or prompts change.

Layer 2: Agent identity and authority

An AI agent may use tools to search documents, query APIs, send messages, update records, or trigger workflows. Its risk depends heavily on the identity and permissions behind those actions.

A model that produces an unsafe suggestion is one problem. An agent that has broad authority to carry it out is another.

Use dedicated identities, narrow permissions, clear tool boundaries, and approval requirements for consequential actions. Enforce these controls in trusted application layers rather than relying only on natural-language instructions.

Layer 3: Data and connected systems

AI applications often bring together information from multiple systems. Retrieval tools, file stores, databases, collaboration platforms, and external APIs can expand the data available to an agent.

Security teams need to understand what data is reachable, which user or service identity authorizes access, and whether results are filtered according to the requester's permissions.

A connection does not automatically mean that all data is exposed. Verify the actual permissions and data paths. Equally, do not assume the user interface reveals every place where data is processed, logged, or transmitted.

Layer 4: Application and infrastructure vulnerabilities

AI features still run on software. APIs can have authorization weaknesses, dependencies can contain vulnerabilities, secrets can be mishandled, and cloud resources can be misconfigured.

These issues are not unique to AI, but they can have a different impact when the application has access to powerful tools or sensitive data.

Use appropriate secure development practices: code review, dependency analysis, static and dynamic testing, secret detection, configuration review, and verified remediation.

Layer 5: Governance and operational oversight

A secure design can drift after release. Teams add tools, change permissions, update models, and connect new data sources. Ownership may also change.

Organizations need an accurate view of what is deployed, who is responsible, which configurations are approved, and what changes over time. Logs and alerts help, but they should feed a defined process for investigation and response.

Governance should distinguish confirmed policy violations from missing information. Unknown ownership is a reason to investigate; it is not automatically proof of malicious use.

Connect the layers through a threat model

Start with the intended purpose of the AI system. Identify users, identities, data, tools, trust boundaries, and high-impact actions. Then ask what could go wrong at each layer and how one weakness could affect another.

For example, prompt injection may influence an agent's behavior. Whether that becomes a serious incident depends partly on which tools the agent can call, what those tools can access, and which actions require approval.

This is why isolated checks are not enough. The meaningful risk often emerges from the relationship between components.

Build a lifecycle, not a one-time assessment

Before release, review design, code, configuration, permissions, and behavior. During operation, monitor relevant activity and configuration drift. After meaningful changes, reassess the affected controls and retest important scenarios.

Prioritize according to the system's purpose, exposure, access, and potential impact. A read-only assistant and an agent authorized to modify production systems should not automatically receive identical controls.

Security as a connected discipline

Model testing, agent discovery, posture review, and vulnerability management address different questions. They are complementary, not interchangeable.

A complete AI security approach asks:

  • What AI systems and agents exist?
  • How do they behave under realistic and adversarial inputs?
  • What data and actions can they access?
  • Are the surrounding software and infrastructure secure?
  • Who is accountable, and how are changes reviewed?

The model matters. So do the identity, tools, data, code, and controls around it.

AI security becomes more effective when organizations secure the entire system—not just the most visible component.

Back to insights