Securing AI features: prompt injection and data leakage

By Marco Lopes · Published July 24, 2025 · AI & Machine Learning

When we add AI to a product, we often give it access to things that make it useful: internal documents, customer records, support conversations, and tools that can take action. Each connection also creates a security boundary we need to defend.

At Kordspace, we approach this as a software architecture problem, not a prompt-writing contest. As a remote collaborative software, AI, Web3, hardware, and design studio, we care about what happens when a promising prototype meets real permissions, sensitive data, and unpredictable inputs.

Prompt injection and data leakage are closely connected, but they are not interchangeable. An attacker may use injection to extract information; an application can also leak information without any malicious prompt. We need controls for both.

Understand the trust boundary

Prompt injection happens when content an AI system encounters attempts to redirect its behavior beyond that content's intended role. A direct injection comes from a user's message. An indirect injection arrives through something the system reads: a retrieved document, website, email, repository file, image, or tool response.

Imagine a support assistant retrieving a knowledge-base article. Hidden among legitimate troubleshooting instructions is a paragraph telling the assistant to collect account details and send them to an external URL. The paragraph is source material, not an authorized instruction. The danger appears when the assistant treats it as authority.

We cannot reliably solve this by asking the model to “ignore malicious instructions.” Models process instructions and data through overlapping mechanisms, so natural-language boundaries are not equivalent to access-control boundaries.

Our working rule is simple: untrusted content must not gain authority merely because a model has read it.

That means we enforce permissions, tool restrictions, and disclosure rules outside the model. Better prompts help, but they do not replace those controls.

Map where information can escape

Before selecting safeguards, we draw the complete data path: input, retrieval, prompt construction, inference, tools, response, logs, and storage. We include background jobs and third-party observability systems, not just the chat interface.

We ask what sensitive information enters each stage and who can receive its output. Common leakage paths include:

• Retrieval returning another customer's records.
• Debug traces retaining complete prompts and tool responses.
• A browsing tool sending confidential text in URL parameters.
• Shared conversation state or caches crossing tenant boundaries.
• An answer disclosing fields the user was never authorized to view.

We also distinguish intentional disclosure from accidental exposure. Sending a document to an approved model endpoint may be an accepted processing activity; copying it into an unrestricted analytics platform may not be.

For each boundary, we identify the data owner, permitted purpose, retention requirement, and enforcement point. This gives us a threat model we can turn into implementation tasks.

Retrieve only what the user may access

Retrieval-augmented generation does not make private data safe automatically. A vector database is still a database, and similarity is not authorization.

We apply access checks before retrieved content enters the model context. The application derives tenant and user scope from authenticated server-side state, never from a tenant identifier the model invents or the browser supplies unchecked.

Where the retrieval engine supports reliable filtering, we enforce document permissions there. Otherwise, we retrieve candidates and authorize them in application code before including any content in the prompt. High-isolation workloads may justify separate indexes or storage boundaries.

We also account for permission changes. A document removed from a workspace must not remain available through an old embedding, cached answer, or conversation attachment. Revocation needs to reach derived stores as well as the source system.

Finally, we retrieve the minimum necessary material. Narrow excerpts reduce exposure and improve relevance. If a task needs an invoice status, we do not send the full billing history and payment details along with it.

Keep secrets and authority outside prompts

We treat system prompts as behavior configuration, not secret storage. API keys, database credentials, signing material, and privileged internal tokens belong in managed secret storage and narrowly scoped services.

A model generally does not need to see a credential to use a capability. Instead, it can request a defined operation, while trusted application code authenticates the call and supplies credentials behind the boundary.

We separate identities, too. A document-search component should not inherit the permissions of a deployment service. A response-generation worker should not have unrestricted access to production storage just because both run in the same cloud project.

The same principle applies to authorization decisions. We do not ask the model whether someone is allowed to view a record and treat its answer as enforcement. We resolve that question through established policy and application logic.

This limits damage even when the model follows an injected instruction: it cannot reveal a secret it never received or exercise authority its execution path does not possess.

Label external content without trusting labels alone

We structure prompts so the model can distinguish application instructions, user requests, retrieved sources, and tool outputs. We clearly mark external text as evidence to analyze rather than commands to obey.

For example, our instructions can say that retrieved documents may contain hostile requests and must not redefine tool policy or disclosure rules. We keep source boundaries and identifiers intact, making citations and investigation easier.

However, delimiters, XML-style tags, and message roles are aids, not security guarantees. An attacker can place convincing role labels or administrative language inside a document. We assume some attempts will influence the model despite careful formatting.

We can use injection detection to flag suspicious content, but we treat it as a supporting signal. Legitimate documentation may discuss malicious prompts, while attacks may be subtle or obfuscated.

When a source looks suspicious, the application can restrict available actions, request review, or continue with a read-only response. The fallback should preserve the security boundary rather than simply retrying with stronger wording.

Make tools narrow, validated, and reversible

Tool access is where an injection can turn from a misleading answer into a real incident. We prefer small, purpose-built operations over generic shell execution, unrestricted HTTP clients, or arbitrary database queries.

Instead of exposing a universal request tool, we might provide operations such as lookup_order_status or create_support_draft. The application validates arguments against a schema, checks authorization for the specific resource, and rejects unexpected fields.

Schema validation alone is insufficient. A syntactically valid recipient address can still be an exfiltration destination. We enforce semantic rules, including allowed recipients, approved domains, transaction limits, and permitted state transitions.

For consequential actions, we add explicit confirmation. The interface shows the actual destination, affected records, and proposed changes. Approval must bind to those exact parameters; a later model-generated change requires another check.

We apply network egress restrictions as well. If the feature only needs an internal API, its runtime should not freely contact arbitrary internet hosts. For Web3 signing or hardware control, we put dedicated policy checks between model suggestions and irreversible execution.

Use cloud controls as another layer

Google Cloud is our preferred starting point, though we apply the same design principles on AWS and Azure. On Google Cloud, we typically combine narrowly scoped service accounts, Secret Manager, appropriate IAM policies, and isolated application runtimes.

For Vertex AI deployments, we verify the selected service's current data-use terms, retention behavior, regional availability, and supported security controls. We do not assume that every model, endpoint, or preview feature has identical guarantees.

Where supported and appropriate, VPC Service Controls can help reduce exfiltration risk across protected Google Cloud service boundaries. They do not determine whether an authorized application's answer should contain a particular piece of information. Application authorization remains essential.

On AWS or Azure, we evaluate corresponding identity, secret-management, private-connectivity, and model-hosting options rather than assuming direct feature parity.

Across providers, we review backups, telemetry, support access, and subprocessors. Choosing a region does not, by itself, establish the complete processing and retention story.

Design outputs, logs, and caches deliberately

We treat model output as untrusted application input. If it becomes HTML, SQL, a command, or a configuration file, the receiving system still needs its normal validation and escaping. Generated code does not bypass review because it came from an assistant.

For sensitive workflows, we constrain responses to an approved structure and populate critical values from trusted application data where possible. We can scan outputs for restricted data, but such scanning is imperfect, particularly for confidential business information without recognizable patterns.

Logs deserve their own policy. We avoid recording full prompts and retrieved documents by default, and we apply redaction before telemetry leaves the application. Diagnostic access is restricted and time-bounded.

Cache keys must reflect tenant, identity, permission context, and relevant data versions. We disable shared caching when we cannot establish safe isolation. Convenience is not worth serving one customer's answer to another.

Test attacks against the whole workflow

We test the deployed behavior, not just whether the model refuses an obvious jailbreak. Our test set includes malicious instructions in documents, tool responses, attachments, and multilingual text.

We try cross-tenant retrieval, forged source metadata, stale permissions, unauthorized recipients, and attempts to place sensitive content inside outbound URLs. We also test ordinary mistakes: ambiguous questions, overly broad retrieval, and accidental inclusion of private fields.

We use synthetic sensitive records and recognizable test markers rather than production secrets. The assertions focus on outcomes: an unauthorized document never reaches the prompt, a forbidden tool call is rejected, and an unapproved destination receives nothing.

Because model behavior can vary, we repeat important scenarios. We rerun evaluations whenever we change models, prompts, retrieval logic, tools, or permissions.

We also prepare containment: disable risky tools, revoke compromised access, preserve appropriately protected evidence, and investigate affected records. A kill switch is useful only if someone knows when and how to use it.

A short pre-launch checklist

Before shipping an AI feature, we check that:

• Retrieval enforces current user and tenant permissions.
• Prompts contain no credentials or unnecessary sensitive records.
• External content cannot grant access or redefine tool policy.
• Tool arguments, destinations, and resource permissions are validated.
• Consequential actions require clear, parameter-bound approval.
• Outputs, logs, caches, and retention have explicit safeguards.
• Adversarial tests cover leakage and real tool execution.
• Monitoring, ownership, and containment procedures are ready.

Build security into the feature

We do not promise an injection-proof model. We build systems in which model mistakes have constrained consequences, sensitive information stays scoped, and important actions remain accountable. That work starts with architecture and continues through product design, deployment, and maintenance.

If we can help you design or strengthen an AI feature, start a project with Kordspace. To explore how we approach software, AI, infrastructure, and product design together, read our related services pages.