把散落在數十篇論文裡的「自動提示優化」方法，用一套 五階段框架 收斂成一張可比較的地圖：種子提示 → 評估回饋 → 候選生成 → 篩選保留 → 迭代深度。
A single 5-stage framework that collapses dozens of scattered Automatic Prompt Optimization papers into one comparable map: seed prompts → evaluation & feedback → candidate generation → filter & retain → iteration depth.
提示工程（prompt engineering）是讓 LLM 做對事情的關鍵一步，但它太脆弱、太吃人工，而且模型、任務、最佳實務都在快速變動。
Prompt engineering is the crucial step to elicit the right behaviour from an LLM — but it is brittle, labour-intensive, and constantly invalidated by rapidly evolving models, tasks, and best practices.
作者觀察到：兩個語意幾乎相同的提示，效能卻可能差很多。LLM 對任務描述方式、few-shot 範例的排序、甚至格式化的細微風格都異常敏感。這道效能落差讓終端使用者難以採用。Black-Box APO（黑箱自動提示優化）因此興起：用自動化方法搜尋提示空間以提升任務表現。
The authors observe that two semantically near-identical prompts can yield very different performance. LLMs are abnormally sensitive to how a task is described, the ordering of few-shot examples, and even stylistic formatting. That performance gap is an adoption barrier for end users. Black-Box APO emerged to close it by automatically searching the prompt space to improve task performance.
不需要碰到執行任務的 LLM 權重，純黑箱即可運作。
Treats the task LLM as a pure black box — no access to its weights required.
有方法地走過提示解空間，而非靠人類試誤。
Walks the prompt solution space methodically instead of human trial-and-error.
優化後的提示仍是人類可讀的文字，可被理解與審核。
Optimized prompts remain human-readable text that can be inspected.
核心貢獻：一套 五階段 APO 分類法（5-part taxonomy），搭配對各種設計選擇的細粒度分類，把全領域論文放進同一張可比較的框架（論文 Fig. 1 與附錄 Table 2/3/4）。
Core contribution: a 5-part APO taxonomy plus fine-grained categorization of design choices, placing the whole field's papers into one comparable framework (paper Fig. 1 and Appendix Tables 2/3/4).
給定一個任務模型與初始提示，APO 的目標是在某個指標與驗證集下，找出表現最好的提示模板。
Given a task model and an initial prompt, APO seeks the best-performing prompt template under a chosen metric and validation set.
問題在於：離散提示的 token 序列搜尋空間是組合爆炸的，這個目標函數不可解。因此所有 APO 方法都退而求其近似解，並共享下面這個通用骨架（Algorithm 1）。
The catch: the token-sequence search space for discrete prompts is combinatorial, making this objective intractable. So every APO method settles for an approximation, and they all share the general anatomy below (Algorithm 1).
# Algorithm 1 · Prompt optimization framework
P₀ := {ρ₁, ρ₂, …, ρ_k} ▷ §3 Seed prompts
D_val := {(xᵢ, yᵢ)} ▷ Validation set
f₁, …, f_m ∈ F ▷ §4 Inference evaluation
for t = 1, 2, …, N do ▷ §7 Iteration depth
G_t := M_APO(P, D_val, F) ▷ §5 Generate candidates
P_t := Select(G_t, D_val, F) ▷ §6 Filter & retain
if f_convergence ≤ ε then ▷ §7 optional early stop
break
return arg maxρ∈P_N Ex∼D_val[ f(M_task(ρ ⊕ x)) ]這個骨架對應到分類法的五個 § 區塊 —— 下一節正式拆解。
This anatomy maps directly onto the five § blocks of the taxonomy — unpacked next.
作者把任何一個 APO 系統拆成五個可獨立比較的決策階段。每一階段都有多種設計選項，這就是論文 Fig. 1 那棵分類樹的主幹。
The authors decompose any APO system into five independently comparable decision stages. Each stage offers multiple design options — this is the trunk of the taxonomy tree in paper Fig. 1.
兩條路線：人工指令（ProTeGi、GPS、SPRIG 用人工撰寫指令當強基線）；以及 LLM 指令歸納（instruction induction）——Honovich et al. 首創，讓 LLM 從少數示範反推出人類可讀的指令；APE 證明數百筆樣本就足以後續優化。SCULPT 甚至從 task-README 歸納指令，UniPrompt 則讓 LLM 填入結構化模板。
Two routes: manual instructions (ProTeGi, GPS, SPRIG use hand-written instructions as strong baselines); and LLM instruction induction — pioneered by Honovich et al., letting an LLM infer a human-readable instruction from a few demonstrations; APE showed a few hundred samples suffice for downstream optimization. SCULPT even induces instructions from task-READMEs, while UniPrompt has the LLM fill structured templates.
分兩大類。數值分數：任務準確率（最常見，分類用 exact accuracy、程式碼用 execution accuracy、生成用 BLEU/Rouge/BERTScore）、獎勵模型分數（OIRL、DRPO）、基於熵的分數（CLAPS、GRIPS，需要整個輸出分布，黑箱常拿不到）、輸出的負對數似然（NLL，需要 log-prob）。LLM 回饋：用 LLM 產生文字回饋，可同時評估回應與提示、直接驅動改寫，缺點是多一次推論成本。人類回饋：GATE 的「生成式主動任務引導」、BPO 用偏好回饋訓練小模型、APOHF 用 dueling bandits 只靠偏好對。
Two families. Numeric scores: task accuracy (most common — exact accuracy for classification, execution accuracy for code, BLEU/Rouge/BERTScore for generation), reward-model scores (OIRL, DRPO), entropy-based scores (CLAPS, GRIPS — need the full output distribution, often unavailable in black-box settings), and negative log-likelihood of the output (NLL — needs log-probs). LLM feedback: an LLM generates textual feedback that can judge both response and prompt and directly drive rewriting, at the cost of an extra inference call. Human feedback: GATE's "generative active task elicitation", BPO training a small model on preference feedback, and APOHF using dueling bandits over preference pairs only.
這三階段內容最豐富，分別在第 4、5 節展開。簡言之：§5 從新候選提示「怎麼長出來」（啟發式編輯、輔助神經網路、元提示、覆蓋式、程式合成）；§6 是「留下哪些」（TopK、UCB/bandit、區域聯合搜尋、元啟發式集成）；§7 是「跑幾輪」（固定步數 vs. 變動步數含早停）。
These three stages carry the most content and are expanded in the next two sections. In short: §5 is how new candidate prompts are grown (heuristic edits, auxiliary NNs, metaprompts, coverage-based, program synthesis); §6 is which ones to keep (TopK, UCB/bandits, region-based joint search, metaheuristic ensembles); §7 is how many rounds to run (fixed vs. variable steps with early stopping).
這是 APO 的引擎室。方法從簡單的規則式編輯，一路到結合 LLM 評估與篩選的代理式系統。論文歸成五大子類。
This is APO's engine room — ranging from simple rule-based edits to agentic systems that fuse LLM evaluation and filtering. The paper groups them into five sub-families.
用一個比任務模型更小的、額外訓練的網路來改寫提示。強化學習：BDPL 用變異數縮減的 policy gradient；Prompt-OIRL 學離線獎勵模型再 best-of-N。微調 LLM：BPO 訓練 7B 模型做免獎勵對齊；FIPO 訓練 7B–13B 本地模型兼顧隱私與適配。GAN：Long et al. 把生成器／判別器對抗式聯合優化，並用提示改寫 LLM 改寫各自的提示。
A smaller, separately trained network rewrites the prompt. RL: BDPL uses variance-reduced policy gradients; Prompt-OIRL learns an offline reward model then does best-of-N. Finetuning LLMs: BPO trains a 7B model for reward-free alignment; FIPO trains a local 7B–13B model for privacy and target adaptation. GANs: Long et al. jointly optimize generator/discriminator adversarially, using a prompt-modifier LLM to rewrite their prompts.
PE2 指出先前研究低估了元提示搜尋空間。OPRO 的元提示把「優化問題的自然語言描述 + 先前解與分數」一起餵給 LLM 做改寫；DAPO 用精心設計的元指令引導 LLM 產生結構化初始提示（任務型別、輸出格式、推理流程、專業提示），再做句級迭代優化。
PE2 argues prior work under-explored the metaprompt search space. OPRO's metaprompt feeds the LLM a natural-language description of the optimization problem plus prior solutions and scores; DAPO uses a carefully designed meta-instruction to generate structured initial prompts (task type, output format, reasoning process, professional tips) then optimizes at the sentence level.
力求「覆蓋」整個問題空間。單一提示擴張：AMPO 把所有失敗案例用 if-then-else 列進元指令（模式辨識／分支調整／分支剪枝）；UniPrompt 確保任務各語意面向都被表達。專家混合（MoE）：MOP 先 K-means 分群示範，再用區域聯合搜尋為每群生成專家提示，推論時挑離 instance 最近的群心。集成：PromptBoosting、PREFER 等用多提示投票合併輸出 ŷ = y₀ + Σβᵢyᵢ。
Aim to "cover" the whole problem space. Single-prompt expansion: AMPO enumerates all failure cases into the meta-instruction in if-then-else form (Pattern Recognition / Branch Adjustment / Branch Pruning); UniPrompt ensures every semantic facet of the task is represented. Mixture of Experts: MOP K-means-clusters demonstrations then uses region-based joint search to generate an expert prompt per cluster, invoking the nearest-centroid expert at inference. Ensembles: PromptBoosting, PREFER, etc. combine multiple prompts by voting, ŷ = y₀ + Σβᵢyᵢ.
把 LLM pipeline 變成結構化、可被系統優化的模組。DSP 的三段式（Demonstrate／Search／Predict）；DSPy 把 pipeline 變成文字轉換圖，附帶可學習參數與編譯器；DLN 視鏈式 LLM 呼叫為堆疊深度語言網路做變分推論；MIPRO 自動優化多階段程式的指令與示範；SAMMO 把提示表示成 DAG，用節點突變規則搜尋最佳圖再轉回提示。
Turn LLM pipelines into structured, optimizable modules. DSP's three stages (Demonstrate/Search/Predict); DSPy casts a pipeline as a text-transformation graph with learnable parameters and a compiler; DLN treats chained LLM calls as stacked deep language networks doing variational inference; MIPRO auto-optimizes instructions and demonstrations across multi-stage programs; SAMMO represents prompts as DAGs and mutation-searches for the optimal graph, then converts it back to a prompt.
觀察：「文字梯度（textual gradient）」是貫穿多篇的關鍵 trick —— ProTeGi、TextGrad 把離散提示優化做成類似梯度下降的方向修正，但梯度是一段自然語言的改進方向，可一次衍生多個候選。
Observation: the "textual gradient" is a key recurring trick — ProTeGi and TextGrad cast discrete prompt optimization as gradient-descent-style directional refinement, except the "gradient" is a natural-language direction of improvement that can spawn multiple candidates at once.
候選生出來後，要在固定運算預算下挑出最有希望的繼續優化，並決定何時停止。
Once candidates exist, the system must pick the most promising ones under a fixed compute budget and decide when to stop.
每輪在 mini-batch 上保留 top-K 最佳候選（ProTeGi、AELP）。與 beam search 不同 —— 後者依整條編輯軌跡的 reward 評斷。
Each round keeps the top-K candidates on a mini-batch (ProTeGi, AELP). Unlike beam search, which judges the reward of the entire edit trajectory.
把提示挑選變成 bandit 問題，用 Upper Confidence Bound 平衡探索與利用，每輪抽不同評估子集。PromptAgent 用 UCT（樹版 UCB）；AELP 用 Linear UCB，結合 reward 軌跡與提示嵌入 φ(s)。
Cast prompt selection as a bandit problem; UCB balances exploration vs. exploitation, sampling a different eval subset each round. PromptAgent uses UCT (tree UCB); AELP uses Linear UCB combining reward trajectories with prompt embeddings φ(s).
MOP 為每個示範群獨立優化專家提示：先抽群內與群外樣本，再用 APE 歸納並優化各專家指令以覆蓋盲點。
MOP optimizes an expert prompt per demonstration cluster: it samples in- and out-of-cluster examples, then uses APE to induce and optimize each expert instruction to cover blind spots.
PLUM 提供爬山、模擬退火、基因演算法、Tabu Search、Harmony Search 等多種搜尋演算法的集成。
PLUM offers an ensemble of search algorithms — hill climbing, simulated annealing, genetic algorithms, Tabu search, harmony search.
多數方法跑固定步數 N。GRIPS 在連續負增益超過 patience 時停；PromptAgent 以 rₜ ≤ ε_min 或 rₜ ≥ ε_max 為變動步數早停條件。
Most methods run a fixed N steps. GRIPS stops when consecutive negative gains breach a patience parameter; PromptAgent uses rₜ ≤ ε_min or rₜ ≥ ε_max as a variable-step early-stop condition.
下表把六個代表性 APO 系統攤在五階段框架的同一座標上，一眼看出它們在每個決策點的選擇差異（論文 Table 1；完整對照在附錄 Table 2/3/4）。
The table below lays six representative APO systems on the same five-stage coordinates, exposing where they differ at each decision point (paper Table 1; full comparison in Appendix Tables 2/3/4).
| System | 種子指令Seed | 迭代深度Iteration | 推論評估Evaluation | 候選生成Generation | 搜尋＋篩選Search + Filter |
|---|---|---|---|---|---|
| ProTeGi (2023) | 人工撰寫Manual | 固定Fixed | LLM 回饋 + 準確率LLM feedback + accuracy | LLM 改寫LLM rewriter | 樹狀 UCBUCB for trees |
| APE (2022) | 指令歸納Instruction induction | 固定Fixed | 任務準確率Task accuracy | N/A | UCB |
| CRISPO (2025) | 人工撰寫Manual | 固定Fixed | LLM 回饋 + 準確率LLM feedback + accuracy | LLM 改寫LLM rewriter | TopK 選取TopK selection |
| MOP (2025) | 指令歸納Instruction induction | 固定Fixed | 任務準確率Task accuracy | 專家混合Mixture of experts | 區域聯合搜尋Region-based joint search |
| DSPy (2024) | 人工 + 指令歸納Manual + induction | 變動Variable | LLM 回饋 + 準確率LLM feedback + accuracy | 程式合成Program synthesis | TopK 選取TopK selection |
| OPRO (2024) | 人工撰寫Manual | 變動Variable | LLM 回饋 + 準確率LLM feedback + accuracy | 元提示設計Metaprompt design | TopK 選取TopK selection |
| GATE (2024) | 人工撰寫Manual | 變動Variable | 人類回饋Human feedback | LLM 改寫LLM rewriter | N/A |
高亮列（GATE）凸顯唯一以人類回饋為評估訊號的系統 —— 框架的價值正在於讓這種橫向比較變得一目了然。
The highlighted row (GATE) marks the only system driven by human feedback as its evaluation signal — exactly the kind of cross-cutting comparison the framework makes legible.
這一節雖短，卻點出領域稀缺的理論基礎。
A short section, but it flags the field's scarce theoretical footing.
AlignPro 為離散提示優化在給定優化器下能取得的增益建立上界，並給出相對 RLHF 最優策略 π* 的次優差距；下界仍未探討。
AlignPro establishes an upper bound on gains achievable from discrete prompt optimization under a given optimizer, plus a suboptimality gap vs. the RLHF-optimal policy π*; a lower bound remains open.
Bhargava et al. 用控制論框架，以自注意力權重矩陣的奇異值界定可達輸出集合；Liu et al. 證明存在能逼近任意 Lipschitz 序列函數的強 transformer，也存在深度受限模型背不起來的「困難」資料集。
Bhargava et al. use control theory to bound the set of reachable outputs via singular values of self-attention weight matrices; Liu et al. prove a strong transformer can approximate any Lipschitz sequence-to-sequence function, and that "difficult" datasets exist which depth-limited transformers cannot memorize.
作者點名四個尚未解決、值得投入的方向。
The authors call out four unresolved, high-value directions.
1 · 任務無關 APO：所有方法都假設任務型別事先已知，離線方法還需要評估集 D_val —— 這在真實生產環境往往不存在。推論時對「多個未知任務」做優化幾乎沒人碰，需要結合已見／未見任務的更強健評估。
1 · Task-agnostic APO: every method assumes the task type is known in advance, and offline methods also need an eval set D_val — often unavailable in production. Inference-time optimization over multiple unknown tasks is barely explored; more robust evaluations mixing seen and unseen tasks are needed.
2 · 機制不明：提示存在「邪惡雙胞胎（evil twins）」—— 無法解讀卻能恢復黃金提示的部分效能；罕見的亂碼字串可當競爭力十足的分隔符；LLM 的自我反思可能錯誤辨識錯誤、帶有先驗偏誤。提示優化「為何有效」仍是黑盒。
2 · Unclear mechanisms: prompts have "evil twins" — uninterpretable yet recovering much of a gold prompt's performance; rare gibberish strings make competitive delimiters; LLM self-reflection can misidentify errors and carry prior biases. Why prompt optimization works remains a black box.
3 · 系統提示與 Agent 的 APO：SPRIG 雖優化過 chat 系統提示，但需預定義語料且耗時近 60 小時，而 ProTeGi 每任務僅約 10 分鐘 —— 可擴展性是瓶頸。在 agentic 系統中並行優化多個元件的提示是誘人的方向。
3 · APO for system prompts / agents: SPRIG optimized chat system prompts but needed a predefined corpus and ~60 hours, vs. ProTeGi's ~10 minutes per task — scalability is the bottleneck. Concurrently optimizing prompts across many components of an agentic system is an exciting open direction.
4 · 多模態 APO：提示優化已擴展到 text-to-image / video / audio 與 CLIP 類對齊模型，甚至優化影像本身。但模態間的交互作用仍未充分探索 —— 未來可聯合優化多模態提示（如去除音訊背景雜訊、為影片加視覺標記）。
4 · Multimodal APO: prompt optimization now spans text-to-image / video / audio and CLIP-style alignment, even optimizing images themselves. But cross-modal interplay is under-explored — future work could jointly optimize multimodal prompts (e.g. denoise audio, add visual markers to video).
可能漏掉部分相關論文；把多元研究硬塞進單一框架，可能對某些論文過度概括或略過特徵（例如 Tempera 同時含 RL 與字詞編輯，作者依「最顯著特徵」歸類）；受限於 8 頁篇幅，最嚴謹的全論文對照表（Table 2/3/4）只能放進附錄。
Some relevant papers may have been missed; fitting diverse work into one framework risks over-broad categorization or skipped traits (e.g. Tempera mixes RL and word-edit techniques — categorized by its most salient feature); and the 8-page limit forced the rigorous full comparison (Tables 2/3/4) into the appendix.