Zero-code multi-agent orchestration platform: YAML-defined agent workflows run by a FastAPI backend with a Vue web console.
github.com/openbmb/chatdev · 2026-10-04 · 4fb2db0
Defense-in-depth score
1.7 / 10
ChatDev 2.0 runs agent-written code, package installs and MCP servers directly on the host, with the provider API key in every subprocess and no approval step. The backend's default network exposure is not locked down. Only the file tools are meaningfully scoped. Run it only on an isolated machine or container, bound to localhost.
ChatDev's file tools keep their paths inside the session workspace, each agent node gets only the tools its workflow lists, and every tool call is recorded. Beyond that the defaults are weak: agent-written code, package installs and MCP servers run on the host with your provider key and no approval step. The safest setup narrows each workflow's tools and builds containment around the whole backend.
| Setup | How | What it gives you |
|---|---|---|
| A dedicated, low-limit key | Give the backend its own provider key with a spending limit set at the provider, and keep other credentials out of its environment | Code the agents run can reach only that key and its budget, not your other accounts C8 |
| Reachable only by you | Run the backend and console bound to localhost on a machine only you use | Other machines on your network can't reach the server C1 |
| Tools per agent node | List in each agent node's tooling only the tools its task needs, leaving out the code-running and delete tools where the task allows | Agents work through the workspace-scoped file tools rather than running code on the host C3 |
| MCP servers without your keys | Set inherit_env: false on each local MCP server, pass only what it needs in env, and pin the version in its command | Each MCP server runs the release you chose and never sees your provider key C7 |
| Memory left off | Keep the default empty memories list on agent nodes | Each run starts clean, with nothing carried over from earlier runs C6 |
| Tighter run limits | Set tool_loop_limit: <n> in each agent node's params, and keep python nodes at the default timeout_seconds: 60 or lower | Each agent stops after fewer tool rounds and each python node ends on time C10 |
| The container deployment | Run the backend through the shipped Docker Compose setup, which runs it as the non-root appuser | Agent-written code, package installs and MCP servers run inside a container rather than on your host C4 |
| Its own VM | Run the whole deployment in a VM or disposable machine that holds no other files or keys, with outbound network limited to the model provider and the hosts the task needs | An outer boundary around everything the agents run, and a narrow path for data to leave C5 |
SafetyCard
call_user ask for input but don't hold back tool calls, so pick workflows whose tools you are comfortable running unattended.execution_logs.json in the session directory keeps every tool's arguments and results unredacted and is written only after a successful run; store it like the data the agents read.0.00 / 1.00
ChatDev's backend acts with whatever authority the operating-system user who started it has, and the HTTP API's default exposure is not locked down. The provider API key from .env is loaded into the process environment and inherited by every code-running tool and local MCP server. There is no per-request or per-tool authorization layer, so a hijacked agent acts with the user's full account.
0.00 / 1.00
There is no approval step for any tool call. Agent nodes run every model-requested tool immediately, including code execution, package installation, recursive deletes and web fetches. The human node and the call_user tool let a workflow author or the model ask a person for input, but neither gates tool execution. Deletions in the workspace and actions taken by executed code are not reversible.
0.33 / 1.00
The built-in file tools are well scoped: every path is resolved and checked to stay inside the session's code workspace. That scoping is undone by the execution tools the flagship workflow also enables: execute_code and uv_run run arbitrary Python on the host with model-chosen arguments and environment variables, install_python_packages accepts model-chosen package specs including URLs, and read_webpage_content fetches any URL. Each agent node receives only the tools its YAML lists, but the shipped ChatDev workflow lists write, delete, and exec tools.
0.38 / 1.00
All model-influenced code runs as a plain subprocess on the host: execute_code, uv_run, package installs, python nodes and local MCP servers. Each one inherits the server's full environment, including the provider API key, and has unrestricted network and filesystem access as the user. The optional Docker Compose setup runs the backend as a non-root user inside a container, but it bind-mounts the whole repo including .env, injects the same secrets, and leaves network open. It is also not the mode the README leads with.
0.00 / 1.00
Untrusted content enters agents through web search and page fetches, uploaded files, MCP tool results and descriptions, and other agents' messages, all with the same standing as the user's task. Nothing tracks where content came from or restricts what an agent may do after reading it. A prompt injection in a fetched page can make an agent run code that reads the API key and sends it out, delete workspace files, or install packages, with no human involved.
0.13 / 1.00
Memory is opt-in per workflow, but when enabled every agent output is written to a JSON store automatically, with no validation, and re-injected later as a user-role message. Memory files are keyed only by the path the workflow sets, with no per-user namespace, and the server has no users. Separately, the exec tools can write anywhere the user can, including the functions/ directory, whose Python files are auto-loaded as tools. That lets poisoned content persist as code.
0.00 / 1.00
Shipped workflows give agents install_python_packages, so the model picks which packages to install from PyPI or a URL, and their install hooks run on the host. Local MCP servers are launched with unpinned commands such as 'uvx blender-mcp' and inherit the full environment by default. Function tools are any Python file in functions/, executed in-process at load.
0.13 / 1.00
Provider keys come from a .env file loaded into os.environ. They do not appear in prompts. They are inherited by every subprocess the agents start, so model-written code can read them. No redaction exists anywhere: tool arguments and full results are logged to the session's execution_logs.json and streamed to the web console. No telemetry was found.
0.35 / 1.00
Every tool call, built-in or MCP, is recorded before and after execution, with arguments, success flag, result and timing, tagged by node id. The record is held in memory and streamed to the web console. It is written to execution_logs.json in the session directory only after a workflow finishes successfully, so failed, cancelled or crashed runs lose it. The file sits where the agents' own exec tools can edit it, and nothing identifies who started a run.
0.40 / 1.00
Each agent node stops after 50 tool-call rounds, graph cycles stop after 100 iterations by default, and python nodes time out after 60 seconds. There is no token or cost ceiling and no wall-clock limit for a run. The model sets the timeout itself for execute_code and uv_run. Cancelling from the console sets a flag that is checked between steps and does not interrupt running subprocesses.