dedupe LLM_Evaluation zone: single-source tables, fix nav orphans, compress guidebook 2025
- Why Guide: remove duplicated 20-case/rubric/test-definition tables, link to authoritative Roadmap/Worksheet instead; 10-min action points to entries - Concept Map / What-Is: cross-link narrative vs quick-reference roles - Worksheet: annotate sections with authoritative sources - 04-Reference 01-04 <-> archive/01: bidirectional resource links - archive/00-Material-List: shrink guidebook listing, now reachable from READMEs - guidebook: compress 06-Yearly-Dives 2025 section (-> 08-2025-Edition §3), add old/new version nav banners to 01-05, reverse links in 08 - README/Start-Here: link Learning Board (was orphaned)
This commit is contained in:
+4
-16
@@ -218,26 +218,14 @@ Regression Suite
|
||||
|
||||
## 程序员应该如何理解 Evaluation
|
||||
|
||||
最实用的映射是:
|
||||
最实用的映射是一条测试链:
|
||||
|
||||
```text
|
||||
Requirement
|
||||
↓
|
||||
Acceptance Criteria
|
||||
↓
|
||||
Eval Case
|
||||
↓
|
||||
Assertion / Rubric
|
||||
↓
|
||||
Run
|
||||
↓
|
||||
Failure
|
||||
↓
|
||||
Root Cause
|
||||
↓
|
||||
Regression
|
||||
Requirement → Acceptance Criteria → Eval Case → Assertion / Rubric → Run → Failure → Root Cause → Regression
|
||||
```
|
||||
|
||||
(完整版"最重要的概念关系"图见 [[03-Core-Concept-Map]] 第 10 节。)
|
||||
|
||||
因此 LLM Evaluation 的核心不是“评分技术”,而是:
|
||||
|
||||
1. 定义正确;
|
||||
|
||||
@@ -62,6 +62,8 @@ Anthropic 使用 evaluation suite 表示围绕共同目标组织的一组 tasks
|
||||
|
||||
## 2. 判定相关概念
|
||||
|
||||
> 可填写的 rubric 模板与通过/失败定义见 [[01-LLM-Evaluation-Roadmap]] 第六节;"为什么 rubric 比参考答案重要"见 [[02-Why-Guide]] 第 3 步。
|
||||
|
||||
### Rubric
|
||||
|
||||
**给人或模型执行的判断标准。**
|
||||
@@ -199,6 +201,8 @@ load cases
|
||||
|
||||
## 4. 数据集生命周期概念
|
||||
|
||||
> split 划分、冻结与版本化的实操表(dev / eval / regression / holdout)见 [[01-LLM-Evaluation-Roadmap]] 第五节。
|
||||
|
||||
### Dev Set
|
||||
|
||||
用于频繁开发和调试。
|
||||
@@ -394,6 +398,8 @@ Level 4 — Outcome
|
||||
|
||||
## 9. Benchmark、Eval、Monitoring 的关系
|
||||
|
||||
> 展开叙述见 [[01-What-Is-LLM-Evaluation]]("Evaluation 与 Benchmark 的区别"一节)。
|
||||
|
||||
### Benchmark
|
||||
|
||||
回答:
|
||||
@@ -464,6 +470,12 @@ Regression
|
||||
|
||||
如果只记住一张图,记住这张。
|
||||
|
||||
## 展开阅读
|
||||
|
||||
- 概念讲解:[[01-What-Is-LLM-Evaluation]]
|
||||
- 每一步背后的道理:[[02-Why-Guide]]
|
||||
- 操作路线与 schema:[[01-LLM-Evaluation-Roadmap]]
|
||||
|
||||
## 参考资料
|
||||
|
||||
[1] Anthropic, Demystifying evals for AI agents
|
||||
|
||||
Reference in New Issue
Block a user