Prompt Injection Attacks: Examples and How Forward Deployed Engineers Defend Against Them

Prompt injection is a security risk in which harmful instructions in an input cause a large language model (LLM) or AI agent to behave in unintended ways. The instructions may come directly from a user or indirectly through a webpage, document, email or image.

For example, a user might tell an AI assistant to ignore its original instructions and reveal private information. An indirect attack could place similar instructions inside a document that the AI is asked to summarise.

A successful prompt injection attack may lead to data leaks, misleading answers or unauthorised actions. The impact depends on what the AI system can access and what it is allowed to do.

The OWASP GenAI Security Project identifies prompt injection as a major risk in its 2025 Top 10 for LLM applications. There is no single defence that guarantees complete protection. Developers must combine model safeguards with security controls in the surrounding application.

In this guide, we’ll explore common prompt injection attacks, real-world examples, their potential risks, and how Forward Deployed Engineers can help secure AI systems against them.

What are the Main Types of Prompt Injection Attacks?

Prompt injection attacks generally fall into two main categories: direct and indirect. Multimodal prompt injection is another important risk when models process content such as images.

Type	How it works	Example
Direct prompt injection	A user tries to change the model's behaviour through a prompt.	A user tells a chatbot to ignore its instructions and reveal restricted information.
Indirect prompt injection	The model encounters harmful instructions in external content.	A webpage contains instructions designed to manipulate an AI assistant summarising it.
Multimodal prompt injection	Harmful instructions appear in content the model can interpret, such as an image.	An image contains text intended to change the AI's response.

These attacks may succeed or fail depending on the model, the input, the application design and its security controls.

What are some Examples of Prompt Injection Attacks?

Prompt injection attacks can target AI systems through direct user prompts or hidden instructions in external content, potentially leading to data exposure, misleading responses or unauthorised actions.

1. A Customer Support Chatbot Reveals Private Information

A company uses an AI chatbot to answer customer questions and retrieve account details. A user asks the chatbot to ignore its rules and provide another customer's information.

Potential impact: The chatbot may generate an unsafe response or attempt to access restricted data.

How to defend against it: Verify the user's identity and permissions in the backend before returning account information. Never rely on the model alone to decide who can access private records.

2. A Malicious Document Targets a RAG Application

A company uses retrieval-augmented generation (RAG) to answer questions from internal documents. An attacker adds harmful instructions to a document that later enters the retrieval system.

When the AI retrieves that document, it may interpret the instructions as part of its task.

Potential impact: The application may produce misleading answers or attempt an unauthorised action if connected tools allow it.

How to defend against it: Track where documents come from, separate retrieved content from trusted instructions, restrict tool access and validate actions in application code.

3. An AI Agent Is Manipulated Through Email

An AI agent reads emails and drafts replies. An incoming email contains instructions designed to make the agent forward confidential information.

Potential impact: If the agent has permission to send emails without adequate checks, the attack could expose sensitive data.

How to defend against it: Limit email permissions, restrict which actions the agent can perform and require human approval before sending sensitive messages.

These examples describe possible attack paths. They do not mean every AI system will behave this way.

What are the Potential Impacts of Prompt Injection?

Prompt injection can affect both the information an AI produces and the actions it takes.

• Data exfiltration: Sensitive information may be exposed through model responses or connected tools.
• Unauthorised actions: An AI agent may attempt to send emails, modify files or perform other actions it should not take.
• Misinformation and manipulation: The model may produce false, biased or misleading outputs.
• Sensitive system information exposure: An attack may cause the model to reveal information about internal instructions or system configuration.

The actual impact depends on the application's design, the data it can access and the permissions available to the model.

How do we Avoid Prompt Injection Attacks?

No single method can guarantee that prompt injection will be prevented. These security measures help reduce the risk and limit the damage if an attack succeeds.

1. Separate Instructions From Untrusted Data

Keep trusted system instructions separate from user input and external content. Clearly mark retrieved documents and webpages as untrusted.

This can help the model interpret content correctly, but labels alone are not enough. Critical restrictions must also be enforced through code and access controls.

2. Use Least-Privilege Access

Give the AI system only the permissions it needs for its task.

For example, an AI assistant that summarises invoices may need read-only access to selected invoice records, not unrestricted access to a company's financial systems.

Use scoped credentials, backend authorisation checks and restricted tool functions.

3. Apply Input and Output Filtering

Check inputs and model outputs for suspicious instructions, sensitive information and content that violates the application's rules.

Validate structured outputs before using them. Remember that filters can miss attacks, so they should support other security controls rather than replace them.

4. Require Human Approval for High-Risk Actions

Require confirmation before actions such as sending confidential emails, deleting records or changing access permissions.

Human approval should work alongside technical checks. The application must still verify that the requested action is allowed.

5. Test With Adversarial Inputs

Test the application with direct and indirect prompt injection attempts. Include malicious documents, unexpected tool requests and relevant multimodal inputs.

Repeat these tests when the team changes the model, prompts, tools or data sources. Track successful attacks and false positives, where legitimate content is incorrectly blocked.

6. Monitor and Investigate AI Behaviour

Keep appropriate records of tool calls, access decisions and security events. Protect sensitive information in logs.

If the AI behaves unexpectedly, investigate which input triggered the behaviour, what permissions were available and whether any action was completed. Use the findings to improve the safeguards.

For more technical guidance, see OWASP's prompt injection recommendations.

How can Forward Deployed Engineers Defend Against Prompt Injection?

Forward Deployed Engineers (FDEs) help bring AI systems into real business environments. They connect models to company data, integrate tools and adapt applications to business needs.

This work makes AI security an important part of deployment.

An FDE can help teams:

• Map trust boundaries: Identify which inputs, documents and data sources can influence the model.
• Review permissions: Check what the model and its tools can access or change.
• Test attack scenarios: Evaluate how the application handles malicious prompts and external content.
• Validate actions: Ensure that tool calls follow business rules and access permissions.
• Improve incident response: Help teams investigate failures and update safeguards.

Security is a shared responsibility. FDEs can help implement and test controls, but application developers, security teams and system owners must also ensure that protections work across the full system.

Prompt Injection Security Checklist for FDEs

Before deploying an LLM application, check that the team has:

• Identified trusted and untrusted data sources.
• Restricted the tools and permissions available to the model.
• Enforced access controls in application code.
• Protected sensitive data and credentials.
• Added approval steps for high-risk actions.
• Tested direct, indirect and relevant multimodal attacks.
• Set up security logging and an incident-response process.
• Repeated tests after major changes to models, tools or data sources.

This checklist is a starting point. Each application needs a security review based on its own risks.

Conclusion

An AI application may perform well in testing, yet behave differently when exposed to untrusted documents, external tools or real user inputs. The challenge is identifying these weaknesses before they become production risks.

Addressing them requires more than prompt engineering. It requires an understanding of how AI systems are built, integrated, evaluated and secured in real-world environments.

Developing these skills is central to building AI applications that can withstand real-world deployment challenges. IIT Delhi’s Advanced Certificate in AI Forward Deployed Engineering covers relevant areas, including LLM security, prompt-injection defence, red-teaming and end-to-end AI application audits.

The earlier teams address these risks, the better prepared they are to deploy AI securely.

FAQs

1. What is the difference between prompt injection and jailbreaking?

Prompt injection tries to change how an AI responds by giving it harmful instructions. Jailbreaking tries to get an AI to ignore its safety rules. The two can overlap, and jailbreaking is often treated as a type of prompt injection.

2. Is prompt injection illegal?

It depends on what someone does and which laws apply. Prompt injection is not automatically a crime, but using it to access private data without permission, steal information or damage systems may be illegal.

3. Can prompt injection attacks be completely prevented?

No. Security measures can reduce the risk, but no single method can stop every attack. Limiting what an AI can access, checking its actions and testing it regularly can help protect the system.

4. Does fine-tuning prevent prompt injection attacks?

No. Fine-tuning can change how an AI responds, but it cannot guarantee protection against harmful instructions. Developers still need security checks and limits on what the AI can access or do.

5. Can prompt injection happen through images?

Yes. An image may contain hidden or visible instructions that try to manipulate an AI model. Whether the attack works depends on how the model reads the image and how the application is protected.

6. Is prompt injection the same as SQL injection?

No. SQL injection targets weaknesses in how an application handles database queries. Prompt injection targets how an AI understands instructions and content. Both are security risks, but they need different defences.

7. How does prompt injection affect AI agents?

Prompt injection can cause an AI agent to give misleading answers or attempt actions it should not take. If the agent can access emails, files or databases, an attack could expose sensitive information or lead to unauthorised actions.

8. Are internal AI systems vulnerable to prompt injection?

Yes. Internal AI systems can encounter harmful instructions in company documents, emails and other content. Information from inside a company is not automatically safe, so access controls and security checks are still needed.

9. Can a more advanced AI model prevent prompt injection?

No. A more advanced model is not automatically protected against prompt injection. Developers must also limit what the AI can access, check its actions and add safeguards around sensitive tasks.

10. What should you do if you discover a prompt injection vulnerability?

Limit the affected system's access if needed, check whether any private data was exposed or unauthorised actions occurred, and review the relevant logs. Fix the weakness, test the safeguards and check that the system is behaving safely before restoring normal access.

Sources

• OWASP: LLM01:2025 Prompt Injection
• Google Search Central