What security controls should apply to AI engineering tools?

An assistant that explains selected code and an agent that executes commands across repositories create different exposures. Both may receive confidential context or generate insecure output, while the agent can also turn a mistake or hostile instruction into action.

Security starts with the proposed work, data and authority. A tool's built-in safeguards contribute evidence, but do not replace the organisation's own controls.

Map the complete trust boundary

Identify users, models, providers, repositories, prompts, retrieval sources, tools, networks and environments involved. Record what data crosses each boundary, where it is processed or retained and which party administers it. Include extensions and external tool connections rather than assessing only the visible chat interface.

Threat-model realistic failures: sensitive context disclosure, insecure generated code, prompt injection, malicious repository content, untrusted dependencies, excessive permissions and compromised credentials. The NCSC's 2026 agentic AI guidance highlights prompt injection and warns against unrestricted access to sensitive data or critical systems.

Classify the use case by consequence and reversibility. This determines which controls and reviewers are proportionate.

Control identity, data and tools

Use approved organisational accounts and dedicated agent identities. Grant only the repositories, directories, services and actions required for a bounded task, and expire access afterwards. Keep secrets outside prompts and generated files; use scoped, short-lived credentials where possible.

Restrict network destinations and connected tools to reviewed services. Treat retrieved documents, issues and tool output as untrusted inputs that cannot grant authority. Isolate execution in temporary environments and prevent agents from modifying their own safeguards.

Define permitted data and a route for exceptions. Read-only access can still expose valuable source code or customer information.

Verify generated work independently

Keep generated code inside normal branch protection, review, test, dependency and security scanning. Use trusted pipelines and prevent the agent from being the sole approver of its own output. Evidence generated from the same assumptions is useful but not independent proof.

Check source, dependency and licence concerns according to organisational policy. Review changes that affect authentication, encryption, data access, trust boundaries or security controls with suitable expertise.

NIST's Secure Software Development Framework groups practice around preparing the organisation, protecting software, producing well-secured releases and responding to vulnerabilities. AI changes inputs and speed, not the need for those outcomes.

Monitor, respond and improve

Record material prompts or task intent, identities, permissions, tool calls, denied actions, changes, checks, approvals and operational outcomes. Minimise and protect logs because they may contain code, vulnerabilities or sensitive data.

Create stop, revocation, investigation and recovery procedures. Test what happens if the model behaves unexpectedly, a tool connection is compromised or an agent attempts to exceed scope. Feed incidents and near misses into permissions, context and training.

Review controls when providers, models, integrations or work change. Secure AI-enabled engineering is a maintained delivery capability, not a one-time tool approval.

Example

A team trials an agent for dependency updates. It receives a dedicated identity, one repository, an approved package registry and an isolated test environment. It cannot read production secrets, alter workflow protections, merge or deploy.

A malicious instruction appears in a dependency issue. Network and tool restrictions prevent the agent from following it, and the denied action is logged. A human reviews the dependency, change and security evidence before acceptance.

FAQs

• Are enterprise AI tools automatically safe for source code?

  No. Contractual and product controls matter, but the organisation must still assess data handling, access, integrations and the specific use case.

• Is prompt injection only a risk for public chatbots?

  No. Agents may encounter hostile instructions in repositories, tickets, documents or connected tools. Treat external content as data, not authority.

• Should AI-generated code receive extra security review?

  Review should reflect the change's risk and evidence, not origin alone. AI involvement may introduce specific provenance, context and tool risks that require additional checks.