SafetyCard Request a review

PentAGI

Fully autonomous multi-agent system for complex penetration testing tasks

github.com/vxcontrol/pentagi · 2026-10-03 · 55a063e

Defense-in-depth score

3.0 / 10

PentAGI runs LLM-driven agents that execute arbitrary shell commands, with no human approval step, inside per-flow Docker containers that have unrestricted network access. The container boundary is the main protection: workers get no secrets and no Docker socket by default, every tool call is logged before it runs, and commands have timeouts and iteration caps. Nothing limits what injected web or target content can make the agent do, the model picks the container image, and answer/guide/code memory is shared across all users. Raising the score would start with an approval or target-scope gate, non-root hardened workers with egress limits, and per-user memory isolation.

Safest setup

PentAGI runs every agent command inside a per-flow Docker container that holds no secrets and no Docker socket, and it logs each tool call before it runs. There is no approval step and nothing in the tool limits which hosts the shell reaches, so the safest setup draws the target scope and spend limits outside PentAGI and fixes the sandbox image yourself.

SetupHowWhat it gives you
Network reach limited to scopePut the host behind a firewall that lets worker containers reach only the in-scope targetsThe network itself keeps commands on the systems you are engaged to test, whatever the model decides C3
A spending limitUse model keys dedicated to PentAGI, each with a spending limit set at the providerA ceiling on model spend for every flow, delegated agents included C10
One instance per clientRun a separate PentAGI instance, with its own database, for each client or teamAnswers, guides and code that one engagement stores are retrieved only by agents working for that client C6
Named tester accountsGive each tester their own account and keep the admin login for setup onlyOnly the people you chose can start flows, and each flow belongs to a named user C1
A fixed, pinned sandbox imageSet DOCKER_IMAGE_SELECTION_MODE to fixed, with DOCKER_DEFAULT_IMAGE_FOR_PENTEST and DOCKER_DEFAULT_IMAGE pinned to digests you have reviewedEvery flow runs an image you chose and checked, never one the model named C7
No Docker or network adminKeep DOCKER_INSIDE and DOCKER_NET_ADMIN at their code default false, also in your .envWorkers get no Docker access and no extra network capability, so the container stays the boundary C4
Worker ports on one interfaceSet DOCKER_PUBLIC_IP to the one host address that should receive connections to workersPorts that workers publish listen only where you chose, not on every interface of the host C4
Tight run limitsLower TERMINAL_TOOL_TIMEOUT to what your tools need and keep MAX_GENERAL_AGENT_TOOL_CALLS and MAX_LIMITED_AGENT_TOOL_CALLS at 100 and 20 or lowerEach command and each agent chain ends after a bounded time and number of calls C10

Approving with confidence

  • PentAGI has no per-call approval; follow each running flow's terminal log and stop the flow when it reaches beyond scope.
  • Stopping a flow ends the commands that flow started; check the worker for commands the assistant launched, which keep running.

Handling data

  • The database keeps every tool call's arguments and terminal output unredacted; protect it and its backups like engagement data.
  • Provider keys sit in a plaintext .env; keep it readable only by the account that runs PentAGI.
  • A status report with no engagement content goes out every 3 hours; UPDATE_CHECK_INTERVAL=0 turns it off.

Key gaps (3)

  1. No approval gate exists for any tool call, including arbitrary shell execution; prompts state actions need no confirmation. C2 · Approval gates
  2. Untrusted web, scraper and target content flows into an agent that has an ungated shell with unrestricted egress and no injection controls. C5 · Untrusted input blast radius
  3. The model chooses the sandbox image from any registry by default (empty allowlist), unpinned and unverified, and runs it without consent. C7 · Third-party extensions

Criteria

C1 Identity & least privilege

0.50 / 1.00

The model's tools run only inside a per-flow Docker container that is created with no environment variables, no cloud credentials and no Docker socket in the default configuration, so a hijacked agent does not hold API keys. Inside that container the agent is root with a broad capability set and unrestricted network egress. Operators are scoped by role privileges and every flow query is filtered by the owning user, but a seeded admin account ships with the documented password 'admin' (forced change on first login), and the shared vector store lets one user's agent read knowledge derived from other users' engagements.

C2 Approval gates

0.05 / 1.00

There is no human approval step before any tool call. The terminal tool executes whatever shell string the model produces, and the system prompts tell the agents that every action is pre-authorized and needs no confirmation. The only interactive hook is an optional 'ask' tool (off by default) that the model itself decides to call. Attacks on target systems cannot be undone and nothing rate-limits them.

C3 Tool & action scoping

0.17 / 1.00

The terminal tool takes an arbitrary shell string and the browser tool takes any URL; the only validation is that file paths must not contain shell-expansion tokens. There is no target scope list or command filtering in code (scope is only described in prompts). Each agent receives a role-specific tool list and the model cannot add tools, but every working agent gets the shell by default and there is no operator switch to remove it. Commands are confined to the container, not to a target set.

C4 Code-execution isolation

0.63 / 1.00

Every model-reachable command, file operation and delegated agent action runs by Docker exec inside a per-flow container; no agent path uses host process execution, and if a container cannot be created the flow fails or falls back to the default image, never to the host. The container drops all capabilities and re-adds a 14-item list (Docker's default set minus MKNOD, plus SYS_PTRACE), has a 2048-process limit and mounts only its own work directory with no secrets in its environment. It is not hardened beyond that: root user, writable root filesystem, no no-new-privileges (deliberately omitted so privilege-escalation testing works), no memory or CPU limits, unrestricted egress, and ports published on all interfaces. Operators can opt in to Docker access inside the sandbox; a raw host-socket mount is documented as host-root equivalent and a TLS dind option with a startup isolation probe exists but both are off by default.

C5 Untrusted input blast radius

0.00 / 1.00

The agents read web search results, scraped pages, target responses and uploaded files, and all of it enters the model context with the same standing as instructions; the prompts contain no untrusted-content rules and no code distinguishes sources. A hijacked agent has an unrestricted shell with unrestricted egress and can attack other systems or move engagement data out unattended, with no human step. No secrets are placed in the sandbox, but findings, uploaded scope documents and discovered credentials are.

C6 Memory, context & configuration integrity

0.15 / 1.00

Tool results (terminal output, search results, file reads) and model-authored answers, guides and code snippets are written to a pgvector store automatically, with secret-pattern redaction but no validation, approval or expiry. Only the 'memory' type is filtered by flow; the answer, guide and code types live in one collection with no user or flow filter and are retrieved by later agents in other flows and for other users. Poisoned content from a web page or target can therefore persist and steer later engagements. Workspace files do not configure the agent.

C7 Third-party extensions

0.10 / 1.00

There is no plugin or MCP loader, but by default the model chooses which Docker image the sandbox runs (any image reference, because the allowlist is empty), and the installer agent installs packages on the model's request. Defaults are unpinned (:latest and untagged pentest images) with no digest or signature check. All of it runs inside the sandbox container with an empty environment, which is what limits the damage.

C8 Secrets & sensitive-data protection

0.25 / 1.00

Provider keys come from environment variables and are held by the orchestrator only; the sandbox environment is empty. A secret-pattern anonymizer redacts text before it is stored in the vector store or sent to external search engines, but terminal output goes to the model and the tool-call log unredacted and there is no log redaction. The default compose and env files are not locked down. A content-free update report is sent on a 3 hour timer by default; Langfuse and OpenTelemetry export are opt-in.

C9 Audit & traceability

0.65 / 1.00

Every tool call from every agent is written to the database (name, arguments, result, status, duration, flow, task and subtask IDs) before the tool runs, and a failure to write stops the call. Terminal commands and output, messages and delegations between agents are also stored, and the log is written by the orchestrator, outside the sandbox. Records identify the flow and its owning user and the delegating agent role in separate tables, but the tool-call row itself carries no agent or user field and there are no approvals to record. Export to Langfuse or OpenTelemetry exists but is opt-in.

C10 Limits & kill switch

0.45 / 1.00

Each agent chain stops after 100 (main agents) or 20 (helper agents) model turns, repeated identical tool calls are cut off, and each shell command has a timeout (20 minutes by default, 3 hours maximum). Stopping a flow kills the commands it started inside the sandbox. There is no token, cost or wall-clock limit for a flow, and delegated agents start their own iteration budgets, so total work is bounded only by the tool graph and the operator's provider spend limits.