SafetyCard Request a review

Agent S

Computer-use agent operating a real GUI via mouse/keyboard

github.com/simular-ai/Agent-S · 2026-10-03 · 3aa272d

Defense-in-depth score

1.6 / 10

Agent S takes over your real mouse, keyboard and screen and executes the model's chosen action as Python inside its own process, with no sandbox and no human approval: the permission dialog in the CLI is never called. Anything on screen, such as a web page or email, can steer the model into running arbitrary code with your full user rights, API keys and a sudo password from its prompt. Only run it inside a disposable VM with no personal accounts or credentials.

Safest setup

Agent S ships with few protections of its own: it runs the model's chosen actions as Python on your machine with no sandbox and no approval step, and has no setting that adds one. The defaults worth keeping are the CLI's 15-step bound and the code agent being off; the safest setup builds the rest around it with a disposable VM, separate keys and a narrow network.

SetupHowWhat it gives you
Dedicated, low-limit keysGive it model and grounding keys used only by this agent, each with a spending limit set at the providerAnything that reads the keys reaches only this agent's budget, never your other projects C8
Keys in the environmentSupply keys through the provider's environment variable, such as OPENAI_API_KEY, rather than on the command lineKeys stay out of process listings C8
A disposable VMRun it in a VM restored from a clean snapshot, signed into no personal accounts and holding no other credentialsWhatever the model's code does, it reaches only that VM and what you put on its screen C1
A non-admin VM userRun it as a VM user with no sudo rightsActions stay at that user's level, without a path to root C1
Code agent left offKeep the default of no --enable_local_envThe model gets no separate bash and Python runner, and no extra 20-step budget C4
The CLI step boundRun tasks through the agent_s command rather than the SDK classEach task stops after 15 steps C10
A narrow networkLimit the VM's outbound access to the model and grounding endpoints and the sites the task needsA much smaller path for data to leave if on-screen content steers the model C5
Logs kept outside the VMCopy the logs folder from the working directory to storage outside the VM after each runA record of each run's plans that the agent's own code can't change afterwards C9

Approving with confidence

  • Agent S has no approval step, so watch the screen while it works; Ctrl+C pauses the run and a second Ctrl+C exits.
  • Processes the agent started can keep running after it exits, so restore the VM snapshot after each task.

Handling data

  • The logs folder holds every model plan at debug level; store it like the screens the agent saw.
  • Full-screen screenshots go to the model provider unfiltered, so keep only the task's windows open.

Key gaps (5)

  1. Model-generated code runs in-process with the user's full ambient authority, including API keys in os.environ and a sudo password supplied in the prompt. C1 · Identity & least privilege
  2. The approval dialog is dead code: the CLI comments 'Ask for permission before executing' and then exec()s the model's action without calling it. C2 · Approval gates
  3. Model-generated Python is eval()'d and exec()'d in the agent process on the user's host with no sandbox, in the default configuration. C4 · Code-execution isolation
  4. On-screen content from any website or document can hijack the model, whose output is then executed as arbitrary Python without approval, enabling unattended exfiltration and irreversible actions. C5 · Untrusted input blast radius
  5. The default type action auto-installs an unpinned PyPI package (and runs sudo apt-get with a hardcoded password) and imports it into the agent process without consent. C7 · Third-party extensions

Criteria

C1 Identity & least privilege

0.00 / 1.00

Agent S runs with the full authority of the logged-in user and narrows nothing. The model's chosen action is evaluated as raw Python inside the agent process, so it can reach every file, environment variable (including the model API keys), browser session and network destination the user can. The worker prompt even hands the model a sudo password and the generated typing code runs sudo, so a hijacked agent is the user, and possibly root on a machine that reuses that benchmark password.

C2 Approval gates

0.00 / 1.00

There is no human approval anywhere. The CLI contains a permission-dialog function and a comment saying it asks for permission, but the function is never called, and the next line executes the model's action directly. Every click, keystroke, file operation or command the model chooses runs immediately, including irreversible ones such as sending email or deleting files through the GUI.

C3 Tool & action scoping

0.00 / 1.00

The agent advertises a small set of GUI actions (click, type, hotkey, open, scroll), but that set is not enforced. The model's reply is evaluated as Python, and the only check is a regular expression that the text contains exactly one call starting with 'agent.', which any payload can satisfy by nesting code inside the arguments. The action builders also paste arguments into code strings without escaping. In practice the tool surface is arbitrary Python on the user's machine.

C4 Code-execution isolation

0.00 / 1.00

Model output is executed twice inside the agent's own Python process: eval() turns the model's text into an action string, then exec() runs that string. There is no sandbox, container, separate user or restricted interpreter on either step, and the optional code agent also runs bash and Python directly on the host. The README advises running untrusted tasks in a sandbox, but the code provides none, so any escape lands in a process holding the API keys and the user's full desktop.

C5 Untrusted input blast radius

0.00 / 1.00

The agent reads whatever is on screen, including web pages, emails and documents written by others, and sends it straight to the model alongside the user's task. Nothing marks this content as untrusted or limits what the model may do after seeing it. Because the next action is arbitrary Python run without approval, a page that hijacks the model can make it exfiltrate files and keys over the network and take irreversible actions, all unattended.

C6 Memory, context & configuration integrity

N/A · full credit 1.00 / 1.00

The S3 agent keeps no memory between runs and auto-loads no workspace files. Its 'procedural memory' is a set of static prompts in the package, its text buffer lives only in process memory, and its log files are never read back. There is therefore no persistent path for poisoned content to shape future sessions through the agent itself, although arbitrary code execution (scored under code execution) can of course modify the user's files.

C7 Third-party extensions

0.00 / 1.00

Agent S has no plugin or MCP system, but its type action generates code that, when the pyperclip package is missing, installs it from PyPI at runtime (unpinned) and runs apt-get under sudo using a hardcoded benchmark password. Because pyperclip is not a declared dependency, this fires on a fresh install the first time the agent types, without asking, and the freshly installed package is imported into the agent's own process.

C8 Secrets & sensitive-data protection

0.00 / 1.00

API keys are read from environment variables or passed as command-line arguments, where they show up in process listings, and nothing anywhere masks or redacts secrets. The CLI turns on debug-level logging to a logs folder in the current directory and records every model plan plus the code agent's full code and output. Full-screen screenshots, which can show passwords or private messages, go to the model provider unfiltered. Secret handling in a bundled integration wrapper is not locked down.

C9 Audit & traceability

0.33 / 1.00

The CLI writes timestamped text logs that include each model plan (which contains the chosen action call) and, when enabled, the code agent's commands and outputs. However, the actual code that gets executed is only printed to the terminal, not logged, results are not recorded per action, and screenshots are not saved, so a run cannot be fully reconstructed. Logs go to a folder in the current directory where the agent's own code can modify or delete them.

C10 Limits & kill switch

0.25 / 1.00

The CLI stops after 15 steps per task, a hardcoded loop bound, but has no time or spend limit. The step bound is in the CLI loop only; the AgentS3 SDK class has none. Model-chosen actions run without timeouts, the wait action sleeps for whatever duration the model picks, and the opt-in code agent gets a fresh 20-step budget on every call, with no timeout on its Python runs. Ctrl+C pauses and a second Ctrl+C exits the CLI, but processes started by evaluated code can outlive it.