把「會做事的 agent」推向「會根據經驗改造自己」：用 What × When × How × Where 拆解可演化元件、時機、學習訊號與落地場域。
A map from agents that act to agents that rewrite themselves from experience—organized by What × When × How × Where they evolve.
關鍵不是用了 SFT、RL 或 memory，而是誰決定更新。若 agent 能依自己的 trajectory 或 feedback，持續修改模型、context、toolset 或 architecture，以改善未來任務，才落入本文的 operational definition。
The boundary is not whether a system uses SFT, RL, or memory; it is where update autonomy lives. An agent qualifies when its own trajectories or feedback drive persistent changes to its model, context, toolset, or architecture to improve future work.
更新來自 trajectory、自產資料、環境或 evaluator feedback，而非固定排程。
Updates come from trajectories, self-generated data, environment signals, or evaluator feedback—not merely a fixed schedule.
改動必須跨 step 或 task 保留下來，實際改變 policy、context、tools 或 topology，而非暫時遵循指令。
The change must persist across steps or tasks and alter policy, context, tools, or topology—not merely induce transient instruction-following.
系統要能主動 exploration、reflection 或 structural modification；只依外部資料或固定排程更新不算。
The system must initiate exploration, reflection, or structural modification; learning triggered only by external data or schedules does not qualify.
| Paradigm | Runtime Context | Evolving Toolset | Dynamic Tasks | Test-time Adaptation | Active Exploration | Structural Change | Self-reflect & Eval |
|---|---|---|---|---|---|---|---|
| Curriculum Learning | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Lifelong Learning | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ |
| Model Editing | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ |
| Self-evolving Agents | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
不要誤讀：作者自己把這篇定位為仍在成形領域的 guiding synthesis，不是已經被社群完全接受的封閉定義。Table 1 很適合建立直覺，但那些全 ✓ 是分類主張，不是實驗證明每個現有系統都具備全部能力。
Do not over-read the boundary: the authors explicitly call this a guiding synthesis for a still-forming field, not a settled definition. Table 1 is a conceptual positioning claim; it does not show that every existing system realizes every checkmark.
這篇 survey 最有用的地方，是把「self-evolution」從模糊口號拆成四個可以獨立選擇、又會彼此耦合的設計軸。
The survey’s most useful move is to turn “self-evolution” from a slogan into four independently selectable—but tightly coupled—design axes.
agent 的哪一層狀態能被持久改寫？
Which persistent state is rewritten?
更新發生在單次任務內，還是任務之間？
During a task, or between tasks?
用什麼 feedback、資料來源與搜尋機制更新？
Which feedback, data source, and search mechanism drive change?
追求廣泛 transfer，還是特定領域深度？
Broad transfer, or deep domain expertise?
靜態 agent 和 self-evolving agent 可以擁有完全相同的元件；差別在於哪些元件能依經驗自主、持久地改寫。
A static and a self-evolving agent may contain identical components. The difference is which components can be autonomously and persistently rewritten from experience.
更新 policy weights 或從互動中歸納可重用的 experience/lessons。代表路線包括 SCA 的自出題—自解題—再訓練、SELF/SCoRe 的 feedback-driven policy improvement。
Rewrite policy weights or distill reusable experience. Examples include SCA’s challenge–solve–retrain loop and feedback-driven policy improvement in SELF/SCoRe.
讓 prompt 與 long-term memory 成為可學習狀態。Memory 不只是存越多越好；還包括抽取、壓縮、去重、檢索與淘汰。
Treat prompts and long-term memory as learnable state. Memory evolution includes extraction, compression, deduplication, retrieval, and deletion—not simply storing more.
agent 可以建立新工具、透過練習 mastery、再依任務 selection。Voyager/Alita/CRAFT 類系統把成功程序封裝成可重用 skill。
Agents create tools, master them through practice, and select among them per task. Systems such as Voyager, Alita, and CRAFT package successful procedures as reusable skills.
演化 control flow、節點 prompt、角色配置或 multi-agent topology。這是從「調某個 component」跨到「重寫整個 agentic program」。
Evolve control flow, node prompts, roles, or multi-agent topology—moving from tuning a component to rewriting the agentic program itself.
實務上的優先順序：越往下改，潛在效益與風險都越大。Prompt/memory 更新便宜且可逆；weight、tool、architecture 更新則需要更完整的 validation、versioning 與 rollback。
Practical ordering: deeper changes raise both upside and risk. Prompt/memory updates are cheap and reversible; weight, tool, and architecture updates demand stronger validation, versioning, and rollback.
| 位置Locus | 子類型Subtypes | 代表方法Representative methods | 主要失敗模式Key failure mode |
|---|---|---|---|
| Model | Policy · Experience | SCA, RAGEN, AgentGen | model drift、錯誤自產資料Model drift; bad synthetic data |
| Context | Prompt · Memory | PromptBreeder, Mem0, Expel | context pollution、錯誤記憶Context pollution; false memories |
| Tools | Creation · Mastery · Selection | Alita, SkillWeaver, ToolGen | 不安全程式碼、tool sprawlUnsafe code; tool sprawl |
| Architecture | Single · Multi-agent | DGM, AFlow, ADAS, ReMA | 搜尋成本、協作不穩定Search cost; unstable coordination |
When 決定 adaptation 和 task execution 的時間關係；How 決定用什麼信號產生、篩選與保留更新。
When locates adaptation relative to task execution; How determines which signals generate, select, and retain updates.
在同一個 task 內發現不足並即時調整；可用 ICL/reflection，也可做昂貴的 test-time SFT/RL。優點是針對眼前問題，缺點是延遲與 budget 壓力。
Detect a weakness and adapt inside the current task, via ICL/reflection or more expensive test-time SFT/RL. It is targeted but latency- and budget-sensitive.
完成 task 後累積 trajectory，再於任務之間更新。可 offline 從資料批次學，也可 online 從 streaming interaction 持續學。
Accumulate trajectories and update between completed tasks. Learning may be offline over collected data or online over streaming interactions.
用 textual feedback、internal reward（confidence/self-judge）、external reward（environment/verifier）或 implicit reward 關閉迴圈。訊號可自動化，但容易 reward hacking、稀疏或偏誤。
Close the loop with textual, internal, external, or implicit rewards. Signals automate well, but can be sparse, biased, or hackable.
從 self-generated、cross-agent 或 hybrid demonstrations 學習。高品質示範使更新穩定，代價是限制 exploration，且 teacher error 可能被複製。
Learn from self-generated, cross-agent, or hybrid demonstrations. Good demonstrations stabilize updates, but suppress exploration and may copy teacher errors.
維持多個 candidate agent/workflow，經 mutation、selection、competition 或 collaboration 演化；能搜尋結構，但計算與評估成本最高。
Maintain candidate agents/workflows and evolve them through mutation, selection, competition, or collaboration. Structural search is powerful but compute- and evaluation-heavy.
同一方法可以跨格：分類不是互斥箱子。像 RAGEN 同時涉及 model policy、experience、inter-test-time RL 與 single-agent architecture；真正有用的做法是用多軸座標描述系統，而非只貼一個 label。
Methods occupy multiple cells: the taxonomy is not a set of mutually exclusive boxes. RAGEN, for example, spans policy, experience, inter-test-time RL, and single-agent architecture. Multi-axis coordinates are more informative than a single label.
「Where」不是地理位置，而是 capability 要往哪個任務分布累積：跨任務 transfer，或單一高價值領域的深度。
“Where” is not geography; it is the target task distribution—broad transfer across tasks or deeper expertise in a high-value domain.
以 versatile digital assistant 為目標，主路線是 memory optimization、curriculum-driven training、model–agent co-evolution。代表案例：Mobile-Agent-E、WebRL、WebEvolver、UI-Genie。
Targets versatile digital assistants through memory optimization, curriculum-driven training, and model–agent co-evolution. Examples: Mobile-Agent-E, WebRL, WebEvolver, UI-Genie.
針對 coding、GUI、finance、medical、education、research、games/diplomacy 等環境反覆累積專門能力。優點是 feedback 可驗證；風險是跨域 transfer 薄弱。
Accumulates expertise in coding, GUI, finance, medicine, education, research, games, or diplomacy. Feedback is often verifiable, but cross-domain transfer remains weak.
| Domain | 可驗證訊號Verifiable signal | 常被演化的部分Typical evolving state | Examples |
|---|---|---|---|
| Coding | Tests, build, issue resolution | Tools · workflow · policy | SICA, SWE-Dev, AgentCoder |
| GUI / Web | Task completion, environment state | Memory · curriculum · model | WebRL, DigiRL, Mobile-Agent-E |
| Finance | Returns, risk, backtests | Strategy · memory | QuantAgent, AlphaEvolve |
| Medical | Diagnosis quality, coverage, efficiency | Roles · topology · policy | DoctorAgent-RL, MDTeamGPT |
| Education | Student progress, personalization | Curriculum · user model | EduAgent, MathTutorBench agents |
| Games / embodied | Environment reward, task completion | Skills · memory · planning | Voyager, Richelieu |
self-evolving agent 的核心產物是一條隨時間變化的 learning trajectory。作者主張從 single-shot score 轉向 longitudinal、cost-aware 評估，並同時追蹤五個目標。
A self-evolving agent produces a learning trajectory over time. The authors argue for longitudinal, cost-aware evaluation across five goals rather than a single end-point score.
Δ / t
success-by-iteration、adaptation speed：多快從 feedback 變好？
Success by iteration and adaptation speed: how quickly does feedback help?
FGT · BWT
新任務是否侵蝕舊能力？新經驗是否反而幫助舊任務？
Does new learning erase old skills, or improve them?
OOD
跨 domain、task cluster 與時間分布是否仍能 transfer？
Does learning transfer across domains, task clusters, and time?
CPG
每單位 performance gain 花多少 token、時間、tool calls、memory 與人力？
How many tokens, seconds, tool calls, memory bytes, and human hours per gain?
CuP
在完成任務的同時，是否維持 policy compliance，沒有 harm、leakage 或 drift？
Does the agent complete tasks under policy without harm, leakage, or drift?
Who?
task、strategy、feedback 有多少是 agent 自主產生？外部介入頻率多少？
Who generated tasks, strategies, and feedback—and how often did humans intervene?
| Aspect | Short-horizon | Long-horizon |
|---|---|---|
| Goal aligned | Adaptivity, efficiency | Retention, generalization, efficiency, long-term safety |
| State persistence | No persistence across tasks; all updates reset after each episode | Full persistence of model / prompt / memory / tools across tasks |
| Dataset structure | Fixed benchmark or episodic sampling; IID or near-IID task variants | Streamed sequences with non-stationary distributions; versioned tasks; explicit OOD clusters for transfer |
| Evolution budget | Per-task cap Kshort (iterations, tool calls, tokens, wall-clock) | Stage cap Kstage + cumulative cap Ktotal; explicit memory/tool growth policy |
| Primary metrics | Adaptivity: success-by-iteration curves, AULC; Generalization: within-distribution transfer | Retention: BWT/FGT, forgetting curves; Generalization: temporal & cluster-OOD; Efficiency: Cost-per-Gain (CPG) |
| Safety auditing | Per-episode Safety/Harm/Refusal; window-level Leakage Rate | Long-horizon safety drift; periodic Safety/Harm/CuP/Risk probes; persistent Leakage Rate |
目前最大的 evaluation gap：大多 benchmark 仍是 static snapshot。幾乎沒有測試把 long-horizon retention 與 privacy 放在一起，也缺少 operational constraint 下的 architecture adaptation、完整 tool lifecycle、以及 multi-agent co-evolution 的安全傳染效應。
The biggest evaluation gap: most benchmarks remain static snapshots. Very few combine long-horizon retention with privacy, architecture adaptation under operational constraints, the complete tool lifecycle, or safety contagion during multi-agent co-evolution.
作者把未來工作聚焦在 personalization、generalization、安全可控與 multi-agent ecosystem。共同問題是：agent 的能力與內部狀態會變，所以一次性的驗收不再夠用。
The paper centers future work on personalization, generalization, controllable safety, and multi-agent ecosystems. The shared problem is that capability and internal state keep changing, so one-time certification is insufficient.
需要可撤回 consent、data minimization、on-device learning、selective forgetting，以及 longitudinal fairness/safety drift 指標。
Needs revocable consent, data minimization, on-device learning, selective forgetting, and longitudinal fairness/safety-drift metrics.
核心是 specialization–adaptability tension、cross-domain transfer、stability–plasticity dilemma 與可轉移 world model。
Key tensions are specialization versus adaptability, cross-domain transfer, stability versus plasticity, and transferable world models.
新風險包括 behavior drift、alignment forgetting、memory reward hacking、poisoned memory、以及自產或外來 tool 的漏洞與資料外洩。
Emergent risks include behavior drift, alignment forgetting, memory reward hacking, poisoned memories, and vulnerable or privacy-leaking tools.
要同時處理個體與集體推理、dynamic role assignment、knowledge propagation、credit assignment 與長期協作評估。
Must balance individual and collective reasoning, dynamic roles, knowledge propagation, credit assignment, and longitudinal coordination evaluation.
所有 agent-generated 或外部 tools 都先在 sandbox 執行，限制 CPU/memory/time/network，並做 static analysis、dependency scanning 與 risk-based access control。
Run generated or imported tools in sandboxes with resource limits, static analysis, dependency scanning, and risk-based access control.
model、memory、tools、workflow 的每次自我修改都留下 immutable audit trail；標記 known-safe state，並定期測試 rollback。
Give every model, memory, tool, and workflow change an immutable audit trail; tag known-safe states and test rollback regularly.
追蹤 reward hacking、resource anomaly、value/safety drift、privacy leakage；不能只在 deployment 前測一次。
Track reward hacking, resource anomalies, value/safety drift, and privacy leakage—not just before deployment.
檔案寫入、外部 API、敏感資料與修改核心 goal/safety constraint 等動作，需要 human review。
Require human review for file writes, external APIs, sensitive data, and changes to core goals or safety constraints.
Misevolution：self-improvement 不是單調遞增。agent 可能因自產資料、錯誤 feedback 或 reward loophole 變得更能完成 proxy objective，卻更偏離人的原始意圖。安全性因此必須跟 agent 一起共同演化。
Misevolution: self-improvement is not monotonic. An agent may become better at a proxy objective while drifting away from human intent through synthetic-data errors, bad feedback, or reward loopholes. Safety mechanisms must co-evolve with the agent.
把它當成 design-review checklist 與閱讀地圖，比把它當成「通往 ASI 的實證證明」更有價值。
It is most valuable as a design-review checklist and reading map—not as empirical proof of a path to ASI.
報告方法時，明確標出 What/When/How/Where 座標；再補上 persistence boundary、evolution budget、feedback provenance、external intervention 與 rollback policy。
Report the What/When/How/Where coordinates plus persistence boundaries, evolution budgets, feedback provenance, external interventions, and rollback policy.
先從 reversible state（prompt/memory）演化開始；只有當 evaluator、logging、versioning、sandbox 與 rollback 成熟後，才擴張到 tools、weights 和 architecture。
Start with reversible state such as prompts and memory. Expand to tools, weights, and architecture only after evaluators, logging, versioning, sandboxes, and rollback are mature.
一句話：真正的 self-evolving agent，不只是多跑幾輪 reflection；它是一個能根據經驗改寫自身狀態的學習系統，而評估單位必須從單次答案升級成整條演化軌跡。
In one sentence: a self-evolving agent is not merely an agent that reflects more—it is a learning system that rewrites its own state from experience, so the unit of evaluation must become the entire evolution trajectory.