Lightweight Python framework for building LLM agents with tools, memory and workflow evaluation.
github.com/forethought-technologies/autochain · 2026-10-04 · 5a1203b
Defense-in-depth score
2.5 / 10
AutoChain is a thin agent loop with almost no safety controls: whatever tool the model picks runs immediately, with no approval, no argument validation by default, and no handling of untrusted tool output. Its surface is small (no code execution, no bundled write tools), so real risk depends on the tools a developer registers. The shipped Redis memory backend is not integrity-protected, and the HuggingFace wrapper's own example enables remote code. The project has been inactive since November 2023.
AutoChain gives an agent only the tools you pass it and ends every run after 15 iterations. Beyond that the defaults are thin: there is no approval step, no handling of untrusted tool output and no durable record, and any registered tool runs as soon as the model picks it. The safest setup narrows the keys and tools, turns on typed arguments and time limits, and builds containment around the process yourself.
| Setup | How | What it gives you |
|---|---|---|
| A dedicated, low-limit key | Run the agent in an environment whose OPENAI_API_KEY and tool credentials belong to this agent alone, each scoped with a spending limit at the provider | Whatever the agent does, it reaches only the systems and spend you chose, not everything your shell can C1 |
| Only the tools needed | Pass the agent's from_llm_and_tools just the tools this task uses | The model can call only those tools, and any other tool name is refused C3 |
| Typed tool arguments | Set args_schema to a pydantic model on every tool you register | Model-written arguments are type-checked before your function sees them C3 |
| Confirmation inside consequential tools | Have each tool that writes, sends or spends ask a person and show its exact input before it acts | A person decides on every consequential action, which the agent loop itself never pauses for C2 |
| Time limits | Set max_execution_time=<seconds> on the chain, keep max_iterations at 15 or lower, and set request_timeout=<seconds> on the model | Every run and every model request ends within a time you chose C10 |
| In-process memory | Keep the README's BufferMemory | Conversation and tool output stay in the process and end with it C6 |
| Reviewed local models | Point model_name at a local model copy you have reviewed and keep remote code off in model_kwargs | Only model code you have read runs inside your agent, and it can't change after you read it C7 |
| Its own container or VM | Run the agent in a container or VM whose network reaches only the model provider and the hosts its tools need | An outer boundary around in-process tools and a narrow path for data to leave C5 |
SafetyCard
--verbose adds full prompts; treat captured output like the data the agent read.0.05 / 1.00
AutoChain has no identity or authorization layer. The OpenAI model wrapper always reads OPENAI_API_KEY from the process environment and sets it globally on the openai module, ignoring the openai_api_key field, and every registered tool is a plain Python callable that runs in-process with whatever authority the developer's process holds. Nothing checks who is asking or narrows what a tool can do with the credentials around it.
0.00 / 1.00
There is no human approval step anywhere in the agent loop. When the model picks a registered tool, the chain looks the name up and calls it immediately; the only pre-action checks are additional LLM calls (should-answer, clarifying-question and confidence scoring), which are not approvals. There is no checkpoint, undo, or dry-run primitive.
0.30 / 1.00
Tools can optionally declare a pydantic args_schema, which the shared run path uses to type-check inputs, but it defaults to None and none of the bundled tools set it, so model arguments normally pass straight through to the developer's function. Unknown tool names are rejected. The retry path after a tool error asks the LLM to fix the input but then re-runs the tool with the original input, outside the error handler.
N/A · full credit 1.00 / 1.00
AutoChain has no code-execution feature: no shell tool, no Python executor, no eval/exec or subprocess anywhere in the library. Developer tools are ordinary Python functions, and model output is only parsed as JSON or OpenAI function calls. Memory-store and remote model loading paths are scored under memory and extensions.
0.00 / 1.00
Tool results, including Google search results and vector-store documents, are fed back to the model as function messages or scratchpad text with nothing limiting what a hijacked agent can then do. No detection, taint tracking, or approval-after-untrusted-input exists. A search tool doubles as an outbound channel, and any developer-registered tool runs unattended.
0.25 / 1.00
Default buffer memory lives in process, but the shipped Redis memory persists the whole conversation, tool inputs and intermediate steps (including untrusted tool output) for an hour and re-injects them on the next run with no validation. Its read-back path is not integrity-protected. The long-term vector memory has no per-user namespace. No workspace files are auto-loaded.
0.13 / 1.00
The optional HuggingFace model wrapper loads any model name through transformers with no pinned revision or integrity check, and its own docstring and readme example pass trust_remote_code=True. Loaded models run in-process with full access to the agent's environment. The transformers default keeps remote code off unless the developer passes that flag.
0.17 / 1.00
API keys come from environment variables and are never redacted; there is no masking helper anywhere. The chain prints every tool input and output to stdout unconditionally, and verbose mode logs full prompts and conversation history. There is no telemetry. Keys are long-lived provider keys held in the process environment, readable by every in-process tool.
0.20 / 1.00
The only record of tool calls is an unstructured print line per action to stdout, plus optional INFO logging of prompts. Nothing is written to durable storage, there are no timestamps, actor fields or correlation IDs, and the record disappears with the process unless the developer captures stdout.
0.38 / 1.00
The chain stops after 15 iterations by default and supports an optional wall-clock limit, both checked cooperatively between steps. There is no token or cost cap, no tool timeout, and LLM calls default to no request timeout with six retries each; each step can make several LLM calls (planning, confidence, clarification). Stopping is a raised KeyboardInterrupt with in-flight work left to finish.