Python framework for building LLM agents and RAG applications over private data, with agent workflows, memory, and hundreds of tool and data integrations.
github.com/run-llama/llama_index · 2026-10-04 · 962940d
Defense-in-depth score
2.2 / 10
LlamaIndex gives you an agent loop with a default 20-step limit and little else in the way of safety controls: no approval gate, no argument validation, no code-execution isolation, and no protection against prompt injection, which its security policy leaves to the application. Every tool the developer registers runs immediately with the process's full authority. Extension loading in one core component is also not locked down. Treat it as a toolkit: the guardrails have to be built by you.
LlamaIndex starts every agent with no tools, stops each run after 20 steps, and loads no workspace config files. Beyond that the defaults are thin: no approval step, no argument checks, no code isolation and no durable record. The safest setup turns on the sandbox and timeout LlamaIndex offers and builds approval, logging and containment around the process.
| Setup | How | What it gives you |
|---|---|---|
| Scoped, low-limit keys | Give the process only the API keys this agent needs, each scoped or read-only with a spending limit set at the provider | Whatever the agent does, it reaches only the systems and spend you chose C1 |
| Tools per agent | Give each agent only the tools it needs, and set can_handoff_to for each agent in a workflow | Each agent can call only the tools and hand off only to the agents you listed C3 |
| Per-call approval | Wrap each tool that writes, sends or spends so it waits for a person's yes before acting | A person sees the exact arguments before each consequential call runs C2 |
| A remote code sandbox | Run model-written code through AzureCodeInterpreterToolSpec with pool_management_endpoint set to your session pool | That code runs in a remote sandbox that never sees your host environment or keys C4 |
| Step and time limits | Keep max_iterations at the default 20 or lower, and set timeout=<seconds> on each agent | Every run ends after a fixed number of steps and a fixed wall-clock time C10 |
| Reviewed, pinned extensions | Pin each integration package, MCP server command and model to a version you have reviewed | Only extension code you have read runs alongside the agent's credentials C7 |
| Its own container or VM | Run the agent in a container or VM whose network reaches only the hosts the task needs | An outer boundary around in-process tools and a narrow path for data to leave C5 |
| A run record on disk | Write every ToolCall and ToolCallResult event the run emits to storage the agent can't modify | A record of every tool call and its result that outlives the process C9 |
0.05 / 1.00
LlamaIndex has no identity or authorization layer for agents: every registered tool runs with whatever credentials the developer's process and the tool's own client hold, and the default LLM reads the operator's OPENAI_API_KEY from the environment. The one binding primitive, FunctionTool partial_params, is not a strict boundary. The framework neither narrows nor widens the developer's authority.
0.00 / 1.00
There is no approval gate in the agent loop. The model's tool calls are dispatched straight to the tool function, including CodeActAgent's code-execution tool. Human-in-the-loop is possible only if the developer writes their own wait-for-event logic inside each tool. Nothing in the framework checkpoints or previews actions.
0.30 / 1.00
Tool schemas are generated from function signatures and shown to the model, but nothing validates the model's arguments against them before the call: the kwargs go straight into the Python function. The handoff tool is the one built-in that checks its argument against an allowlist of agents. On the positive side, agents start with no tools at all and receive only what the developer lists.
0.50 / 1.00
Core ships no isolation for model-generated code. CodeActAgent requires the developer to pass an execute function, and the official example implements it with in-process exec(), labelled not safe for production. The code-interpreter tool package runs model code with subprocess on the host, with no timeout and the full environment. An Azure Dynamic Sessions tool provides a real remote sandbox, but it is a separate opt-in package.
0.00 / 1.00
Tool results, retrieved documents, and reader output enter the model's context as ordinary tool or observation messages with no provenance or taint, and nothing changes what the agent may do after reading them. The project's SECURITY.md explicitly treats prompt injection as the application's problem. Since the framework provides no approval gate either, a hijacked agent can use any registered egress or write tool unattended.
0.28 / 1.00
By default an agent's memory is an in-process chat buffer that disappears with the run, and core loads no workspace files such as .env. When developers enable the richer Memory class with a database and memory blocks, model-extracted facts and retrieved text are written without validation and inserted into the system message by default. Sessions are keyed by an ID in database queries, but nothing prevents poisoned entries from persisting.
0.05 / 1.00
Extensions (tool packages, MCP servers, rerank and embedding models) are chosen in code by the developer, so nothing is added silently. But nothing is verified: MCP servers launch whatever command is configured, and model loading in one core component is not locked down. Python tool packages also run in-process with all the agent's credentials. One positive: persisted object mappings load through an allowlisted unpickler.
0.30 / 1.00
Provider keys come from environment variables or plain string fields on the LLM classes. The base LLM class has a to_payload method that keeps API keys out of instrumentation and callback payloads, and core ships no telemetry. There is no redaction of secrets in tool results sent to the model, in logs, or in subprocess environments.
0.35 / 1.00
Every tool call goes through one workflow step that emits structured ToolCall and ToolCallResult events with the tool name, arguments, call ID, and output, so a developer can capture a full trajectory. But nothing durable records them by default: instrumentation handlers are no-ops, and the run keeps its tool-call list only in memory, returned to the caller. There is no actor or approver attribution.
0.38 / 1.00
Agent runs stop after 20 iterations by default, enforced in code, and in multi-agent workflows the counter is shared across handoffs. A wall-clock timeout exists in the underlying workflow engine, but agents pass timeout=None by default, and there is no token or cost cap and no per-tool timeout. Sync tools run in a thread pool, so a cancelled or timed-out run can leave a tool still running.
SafetyCard