Automated penetration testing agentic framework powered by LLMs (USENIX Security 2024)
github.com/GreyDGL/PentestGPT · 2026-10-03 · e8b1bb7
Defense-in-depth score
1.8 / 10
PentestGPT's autonomous framework hands two LLM roles the provider CLI's full-access mode (no approval prompts, no in-process sandbox) and states that the surrounding deployment is the only security boundary. Deterministic code does constrain planning (typed decisions, target-field scope check, turn and decision caps, durable traces), but none of it binds what the shell tool can do or reach. Treat it as running with the operator's full authority and use it only inside a disposable, network-isolated environment holding the authorized targets.
PentestGPT checks each planned task against the targets you name, caps decisions and turns in code, and journals every event of a run. Both roles otherwise run with full access, no approval step and no sandbox, and the project names your deployment as the only boundary, so the safest setup builds that boundary around it yourself.
| Setup | How | What it gives you |
|---|---|---|
| A dedicated, low-limit identity | Run it as a separate OS user or machine holding only a provider login made for this agent, with a spending limit set at the provider | A hijacked run reaches only that account and its spend, not your own files, keys and services C1 |
| A disposable, network-limited VM | Run each trial in a fresh container or VM with no host mounts, outbound access limited to the authorized targets and the model provider | An outer boundary around the full-access shell, and a narrow path for data to leave C4 |
| Only authorized targets | Pass each authorized target with --target, and nothing else | Planned tasks stay aimed at the targets you named C3 |
| Decision and turn caps | Keep the defaults --max-decisions 20, --supervisor-max-turns 4 and --executor-max-turns 12, or lower them | Every run ends after a fixed amount of planning and work C10 |
| An outside time limit | Give the container or VM a wall-clock lifetime that ends it, along with every command it started | Runs and long-lived commands stop on schedule, even mid-command C10 |
| A pinned provider CLI | Install a specific provider CLI version you have checked, inside the environment image | The code that actually executes commands stays the version you chose C7 |
| The run journal, kept safe | Keep the default journal under --runs-root and copy it to storage the agent can't reach after each run | A replayable record of every command and output that the agent can't change afterwards C9 |
0.00 / 1.00
The framework launches both LLM roles through the provider CLI in full-access mode, so the agent acts with the operator's whole OS identity and the provider login the CLI already holds. No credential is scoped, narrowed or withheld from the shell tool, and nothing checks authority per request. The only credential handling in the repo's own code is passing a few provider environment flags. A hijacked run can use anything the operator's account can reach.
0.00 / 1.00
There is no approval step anywhere in the maintained framework: every command and file write the model chooses runs immediately. The provider permission mode in effect is the one that skips all permission prompts. Deterministic code only decides which planned task runs next, not whether a given action may execute, and no undo or checkpoint exists for the actions taken.
0.07 / 1.00
The one real scope control is a deterministic check that each planned task's target field matches an operator-supplied allowed target, plus a phrase denylist for test-type tasks. It checks the plan's metadata, not the commands or network destinations the shell tool then uses, so the model can still act against any host or path. All provider tools are enabled by default.
0.00 / 1.00
Model-chosen commands run in the provider CLI's shell as the operator's user, with the permission system set to skip checks. The framework adds no container, OS sandbox or filter and says plainly that isolation is the operator's job. The repository's Docker image is a separate tool image that does not contain the framework, gives its user passwordless sudo and adds a network-admin capability, so it is not a boundary for the scored mode.
0.00 / 1.00
Target responses and files are the agent's main input, and they flow into the model and into stored observations that later steer new tasks. The only defence is prompt text telling the roles to treat them as data. A hijacked run still has an unrestricted shell, network access and the operator's credentials with no human step, so it can both leak data and take irreversible actions.
0.35 / 1.00
Run state lives in a per-run SQLite file; each episode starts fresh and the maintained code turns off the Claude provider's own auto-memory. Stored observations must be exact slices of captured tool output, which gives provenance but also means target-controlled text is persisted and re-shown to later model calls. Instruction and settings files in the agent workspace are loaded by the provider CLI in the wrapper copy in the repo, which was not audited for the pinned external wrapper, so that path is treated as uncontrolled.
0.25 / 1.00
The framework ships no plugin, MCP or marketplace loader of its own; the third-party code that runs is the provider CLI and the pinned wrapper package. The wrapper is pinned to one commit, but the provider CLIs are installed by the operator with no version or integrity check, and the tool image installs them unpinned. The model can also install anything through its unrestricted shell, which no control in the repo limits.
0.25 / 1.00
The repo's code never masks, redacts or scrubs anything, and the full-access shell can read the provider login tokens that the Docker helper scripts store and export. Run traces record full prompts, commands and outputs locally, but the files are created owner-only. The README describes default-on analytics, but no telemetry code exists in the maintained packages, so that was not credited or penalized beyond a documentation mismatch.
0.55 / 1.00
Every provider event is written to an append-only per-episode journal with a flush to disk per event, tagged with run, role, task, attempt and revision identifiers, and a separate SQLite transition log records state changes. A bundled audit tool replays a finished run from these files. The journal sits in the operator's working directory, so a full-access agent could alter or delete it, and there is no approver record because there are no approvals.
0.30 / 1.00
The loop is bounded by a count of supervisor decisions (default 20, hard maximum 1000), per-episode turn limits (executor 12, supervisor 4, with per-task-kind budgets) and two attempts per task, all validated in code and unalterable by the model. There is no wall-clock limit, cost cap, rate limit or tool timeout in the framework's own code; command timeouts are only requested in the prompt. Nothing in the repo handles interruption or kills in-flight commands.
SafetyCard