Autonomous self-evolving Photo Agents. A perceive / reason / act framework for photo-aware agents that operate your computer the way you do.
"100% autonomous, self-evolving agents."
Jump to: About · Install · Run · GUI clients · Troubleshooting · FAQ
Photo Agents is building the next generation of LLM-driven agents that ground in what they actually see on screen. Instead of dumping longer chat transcripts into a model and hoping for the best we treat memory the way biology does. Vision in. Bound observations stored in layers. Skills written by the agent itself from real success.
The package in this repo is the runtime that ships that idea. It runs locally so you keep ownership of your screen, your data, and your keys.
Photo Agents is a single Python package that bundles:
- A streaming agent loop that drives any tool-calling LLM through a perceive → reason → act cycle.
- A multi-provider LLM router with first-class support for Anthropic Claude (native) OpenAI GPT (native) and a mixin failover session.
- A physical-execution toolset: file I/O, sandboxed code execution (Python / PowerShell / bash), browser automation via a Chrome DevTools Protocol bridge and a layered memory system (working / global / SOP / session archive).
- Pluggable clients: a polished Streamlit web app, a PyQt desktop app, a desktop companion and ready-to-run bots for Telegram, QQ, Feishu, WeCom and DingTalk.
- Optional observability via Langfuse and a cron-style scheduler.
The whole thing is gated by a remote-validated Photo Agents API key so usage stays accountable.
pip install photoagents
# or, with every optional client and integration
pip install "photoagents[all]"Photo Agents needs Python 3.10 or newer. Tested on Windows, macOS, and Linux.
Then make it available to the runtime in any of these ways (checked in order):
- Environment variable:
PHOTOAGENTS_API_KEY=pk_live_... - Saved config:
~/.photoagents/config.jsonfieldapi_key - Interactive prompt on first run (offered to be saved automatically)
A successful validation is cached for 24 hours so the gate stays fast.
Copy the credentials template and fill in your provider key:
# from the repo root
cp photoagents/config/keys_template.py credentials.py
# then edit credentials.py and uncomment one of the provider configsThe runtime also accepts a JSON form (credentials.json) with the same shape.
# Interactive REPL on your terminal
python -m photoagents
# One-shot file-IO mode
python -m photoagents --task my_task --input "List the largest files in this directory."
# Reflect / watchdog mode (your check() function fires the next task)
python -m photoagents --reflect photoagents/evolution/scheduler.pyPhoto Agents ships several optional frontends. Pick whichever fits your workflow:
| Client | Launch command |
|---|---|
| Streamlit web app + webview | pythonw -m photoagents.cli.launcher |
| Service hub (start/stop) | pythonw -m photoagents.cli.hub |
| Desktop app (PyQt) | python -m photoagents.clients.desktop_app |
| Desktop companion | pythonw -m photoagents.clients.companion_v2 |
| Telegram bot | python -m photoagents.clients.telegram_client |
| Feishu / WeCom / DingTalk / QQ | `python -m photoagents.clients.<feishu |
The launcher and hub both call the same API key gate before starting any service, so they will refuse to launch anything if your key is missing or revoked.
| Path | What lives there |
|---|---|
~/.photoagents/config.json |
API key + license validation cache |
~/.photoagents/global_mem.txt |
Long-term L2 facts |
~/.photoagents/sessions/ |
L4 raw session archives |
~/.photoagents/skill_index/ |
Vector index for skill / SOP search |
~/.photoagents/temp/ |
Per-task scratch (logs, intermediate output) |
photoagents/
├── auth/ License gate (remote-validated API key)
├── cli/ python -m photoagents, GUI launcher, service hub
├── clients/ Web / desktop / chat-platform frontends
├── config/ credentials.py template
├── core/ Agent loop and tool dispatcher
├── evolution/ Reflection / scheduler scripts (the "self-evolving" loop)
├── integrations/ Optional third-party hooks (Langfuse, etc.)
├── llm/ Multi-provider session router
├── resources/ System prompt, tool schema, CDP bridge, demo media
├── skills/ L3 SOPs and helper modules (browser, vision, OCR, ...)
└── web/ DOM simplifier and Chrome DevTools Protocol driver
- "API key required to start the agent" — set
PHOTOAGENTS_API_KEYor run once interactively so it can be saved to~/.photoagents/config.json. - Browser tools not working — make sure
beautifulsoup4is installed and the CDP bridge config exists underresources/tmwd_cdp_bridge/. - Wrong LLM picked up — double check the keyword rules in
credentials.py(native+claude/oai, ormixin).
Does this send my screen data anywhere besides my chosen LLM provider? No. The runtime talks to your configured LLM provider and the Photo Agents license endpoint only.
Can I run this fully offline? The agent loop itself needs a network-reachable LLM provider, but memory and skills are stored locally.
- Issues: use the templates under
.github/ISSUE_TEMPLATE/.
Built and maintained by the Photo Agents team, with thanks to everyone filing issues and sending fixes.
MIT. See LICENSE.
Status: beta. APIs may change before 1.0.
