AI Security
Prompt Injection Attacks: How to Secure LLM Applications
Prompt injection is the top risk in the OWASP Top 10 for LLM Applications. Here is how direct and indirect attacks work, and the layered defenses that actually reduce risk.
EclipticLink Security Team8 min read
Large language models follow instructions, and that is exactly the problem. A model cannot reliably tell the difference between instructions from the developer and instructions hidden in the data it is processing. Prompt injection exploits that gap, and it sits at number one in the OWASP Top 10 for LLM Applications for good reason.
If your business runs a customer-facing chatbot, an internal copilot, a retrieval-augmented generation (RAG) system, or an AI agent that can call tools, prompt injection is a risk you need to design for, not patch later.
Direct vs. indirect prompt injection
Direct prompt injection is when a user types malicious instructions straight into the application, for example asking the model to ignore its previous instructions and reveal its system prompt, or to role-play its way around content restrictions. These are often called jailbreaks.
Indirect prompt injection is more dangerous. The attacker never talks to your model. Instead they plant instructions in content the model will later read: a web page, a PDF uploaded for summarization, an email in an inbox an assistant can access, or a document in your knowledge base. When the model ingests that content, it may follow the attacker's instructions with the user's permissions.
Treat every piece of content an LLM reads as potentially hostile input, and every piece of output it produces as untrusted.
What attackers can achieve
- Data exfiltration: tricking the model into revealing system prompts, other users' data, or documents it retrieved.
- Unauthorized actions: if the model can send emails, update records, or call APIs, injected instructions can trigger those actions.
- Misinformation and manipulation: poisoned content that changes what the assistant tells users or employees.
- Downstream exploits: model output that contains script or markup and is rendered without sanitization, leading to XSS or injection in other systems.
Why there is no single fix
You cannot fully solve prompt injection with a better system prompt. Instructions and data share the same channel, so filtering is probabilistic. The goal is defense in depth: make attacks harder, limit what a successful attack can do, and detect when one happens.
Layered defenses that work
1. Limit agency and permissions
The most effective control is reducing the blast radius. Give agents only the tools they need, scope API tokens to the minimum, and require explicit human confirmation for high-impact actions like sending external email, moving money, or deleting data.
2. Enforce permissions outside the model
In RAG systems, retrieval must respect the end user's access rights. Filter documents by permission before they reach the model, never by asking the model to decide what a user should see.
3. Separate and label untrusted content
Clearly delimit retrieved or user-supplied content, tell the model it is data rather than instructions, and strip hidden text, unusual Unicode, and markup from documents during ingestion. This raises the bar without being a guarantee.
4. Validate output before acting on it
Treat model output like user input. Validate tool-call arguments against strict schemas, encode output before rendering it in a browser, and never pass raw model output to a shell, SQL query, or template engine.
5. Add guardrails and monitoring
Input and output classifiers can catch known attack patterns and sensitive-data leakage. Log prompts, retrieved context, and tool calls so you can investigate incidents, and alert on anomalies such as an agent suddenly calling tools it rarely uses.
Test it like an attacker would
Before launch, and after every significant change to prompts, tools, or data sources, red-team the application. Good AI red teaming combines automated adversarial prompt generation, which can try thousands of variants, with manual testing of indirect injection paths through every data source the model touches.
| Test area | What to check |
|---|---|
| System prompt | Can it be extracted or overridden? |
| RAG sources | Can a planted document change behavior or leak other data? |
| Tools and agents | Can injected text trigger unintended tool calls? |
| Output handling | Is output rendered or executed unsafely downstream? |
| Data leakage | Can users access PII or secrets through clever questioning? |
Key takeaways
- Prompt injection can't be eliminated, but its impact can be contained.
- Least privilege and human confirmation for high-impact actions matter more than prompt wording.
- Enforce data permissions in code, not in the prompt.
- Test continuously; your attack surface changes every time your prompts, tools, or data change.
EclipticLink's AI and LLM security team red-teams chatbots, RAG pipelines, and agents against the OWASP LLM Top 10 and MITRE ATLAS, and helps you design guardrails that hold up in production.
Frequently asked questions
What is a prompt injection attack?
Prompt injection is when an attacker crafts text, typed directly or hidden in content the model reads, that overrides an LLM application's instructions. It can make the model leak data, ignore safety rules, or misuse the tools it has access to.
What is the difference between direct and indirect prompt injection?
Direct prompt injection is typed straight into the chat by the user. Indirect prompt injection is planted in documents, emails, web pages, or other content the model later reads, so the attacker never has to interact with the application.
Can prompt injection be fully prevented?
Not reliably with current models. The practical goal is defense in depth: limit what the model can access and do, enforce permissions in code, validate outputs, require human approval for high-impact actions, and monitor for abuse.
Sources and further reading
- prompt injection
- LLM security
- AI security
- OWASP LLM Top 10
- AI agents
- RAG security