feat(detect): --label-studio 一步生成可导入 Label Studio 的复核视图

label-studio-converter import yolo 只认 <input>/{images,labels,classes.txt},且
os.listdir 不递归,看不见 train/val 子目录,直接喂数据集会 "No labels converted"。
按类别重命名后文件名全局唯一,于是在 <out>/label_studio/ 下把两个 split 拍平
(硬链接:Label Studio 本地存储会校验路径在 DOCUMENT_ROOT 之下,软链接可能越界),
再调用 converter 生成 tasks.json 与 labeling config。

- 框以 predictions 导入:它们是 SHAP/Grad-CAM 生成的,审核员提交后才成为
  annotation,导出时没审过的框不会混进去。
- 不传 image_dims:真实数据有 3223 种尺寸,传了会写错每条标注的宽高。
- 任务数与图片数不符时报错,而不是静默少导。
- converter 自带的导入提示漏掉了 LABEL_STUDIO_LOCAL_FILES_* 环境变量(它在循环里
  改写了 image_root_url,之后与默认值比较失败),静音后由脚本打印带实际路径的步骤。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Shaun Zhou
2026-09-15 17:06:40 +08:00
co-authored by Claude Opus 5
parent 7adc5b6332
commit 1a43267b55
3 changed files with 201 additions and 1 deletions
+34
View File
@@ -633,6 +633,40 @@ data/boxes_gradcam/
**classes.txt** 是给 LabelImg / X-AnyLabeling 这类标注工具用的,人工复核、修改框时
靠它把 class_id 映射回类别名;ultralytics 训练本身只读 `data.yaml`,两者内容严格一致。
### 导入 Label Studio 人工复核
弱监督框需要人工把关。加 `--label-studio` 会额外生成一份可以直接导入的视图:
```bash
uv run python scripts/prepare_yolo_dataset.py --source runs/boxes_shap --label-studio
```
```
data/boxes_shap/label_studio/
├── images/*.png labels/*.txt classes.txt # 两个 split 拍平,硬链接
├── tasks.json # 每张图一个任务,框作为 predictions
└── tasks.label_config.xml # 9 个类别的 RectangleLabels
```
脚本结束时会打印带实际路径的导入步骤:设置 `LABEL_STUDIO_LOCAL_FILES_*` 环境变量启动、
粘贴 labeling config、添加 Local files 存储(**只保存不要 Sync**,否则每张图多出一个空任务)、
导入 `tasks.json`。
几个取舍:
- **为什么要拍平**:`label-studio-converter import yolo` 只认 `<input>/{images,labels,classes.txt}`,
且 `os.listdir` 不递归,看不见 `train/`、`val/`。按类别重命名后文件名全局唯一,可以直接合并。
- **硬链接而不是软链接**:Label Studio 读本地文件时会检查路径是否在 `DOCUMENT_ROOT` 之下,
软链接解析后可能落到外面。
- **框作为 predictions 而不是 annotations**:它们是模型生成的。审核员修改提交后才成为 annotation,
哪些审过一目了然,导出时也只导出 annotation,没审过的框不会混进去。
- **不传 `--image-dims`**:图片尺寸有 3,223 种,传了会把同一个宽高写进每条标注。
- converter 自带的导入提示漏了环境变量(它在循环里改写了 `image_root_url`,与默认值比较失败),
所以它的输出被静音,改由脚本打印。
> **复核重点是 `normal`。** SHAP 在每张图上都框了东西,`normal` 有 4,295 个框、占 38%。
> 若 `normal` 表示无缺陷,这些框要删掉,否则检测器会学着"检测正常"。
### 划分
按类别分层(`--val-ratio` 默认 0.2,`--seed` 默认 42,两次运行结果一致)。分层是必需的:
+95 -1
View File
@@ -31,7 +31,11 @@ data/<name>/
├── classes.txt # 每行一个类别名,行号即 class_id,给标注工具复核用
├── name_map.csv # 新名 ↔ 源文件对照
├── data.yaml
└── summary.json
├── summary.json
└── label_studio/ # 仅 --label-studio:导入 Label Studio 人工复核用
├── images/*.png labels/*.txt classes.txt # 两个 split 拍平,硬链接
├── tasks.json # 框作为 predictions
└── tasks.label_config.xml
```
没有框的图(标签为空)默认保留:YOLO 训练里它们是负样本,有助于压低误检。但**空标签的
@@ -45,9 +49,12 @@ data/<name>/
from __future__ import annotations
import argparse
import contextlib
import csv
import errno
import io
import json
import logging
import os
import random
import shutil
@@ -203,6 +210,63 @@ def assign_names(samples: Sequence[Sample], names: Sequence[str]) -> dict[Path,
return mapping
LABEL_STUDIO_DIR = "label_studio"
def build_label_studio_view(out_dir: Path, num_images: int) -> dict:
"""在 <out>/label_studio/ 下搭一份 label-studio-converter 能直接吃的扁平视图并转换。
label-studio-converter 的 YOLO 导入只认 `<input>/{images,labels,classes.txt}`,
且 os.listdir 不递归——train/val 子目录它看不见。重命名后文件名全局唯一,
两个 split 可以直接拍平;用硬链接而不是软链接,因为 Label Studio 本地存储会检查
文件是否在 DOCUMENT_ROOT 之下,软链接解析后可能落到外面。
框以 predictions 导入:它们是 SHAP / Grad-CAM 生成的,不是人工标注。审核员修改提交
之后才成为 annotation,Label Studio 导出时也只导出 annotation,没审过的框不会混进去。
"""
try:
from label_studio_sdk.converter.imports.yolo import convert_yolo_to_ls
except ImportError as exc:
raise RuntimeError("--label-studio 需要 label-studio(uv sync 后可用)") from exc
view = out_dir / LABEL_STUDIO_DIR
for kind in ("images", "labels"):
for split in ("train", "val"):
for path in sorted((out_dir / kind / split).iterdir()):
link_or_copy(path, view / kind / path.name)
link_or_copy(out_dir / "classes.txt", view / "classes.txt")
tasks_file = view / "tasks.json"
# converter 会往 stdout 打一份缺了环境变量的导入说明,还会刷一串 root logger 的 INFO;
# 静音它们,由 main() 打印带实际路径的完整步骤
root_logger = logging.getLogger()
previous_level = root_logger.level
root_logger.setLevel(logging.WARNING)
try:
with contextlib.redirect_stdout(io.StringIO()):
# 不传 image_dims:图片尺寸不统一,传了会把同一个宽高写进每一条标注
convert_yolo_to_ls(
input_dir=str(view),
out_file=str(tasks_file),
out_type="predictions",
image_ext=".png",
)
finally:
root_logger.setLevel(previous_level)
tasks = json.loads(tasks_file.read_text(encoding="utf-8"))
if len(tasks) != num_images:
raise RuntimeError(
f"label-studio-converter 只生成了 {len(tasks)} 个任务,应为 {num_images};"
f"检查 {view / 'images'}"
)
return {
"tasks": len(tasks),
"document_root": str(view.resolve()),
"tasks_file": str(tasks_file.resolve()),
"label_config": str((view / "tasks.label_config.xml").resolve()),
}
def parse_empty_policy(tokens: Sequence[str]) -> tuple[str, dict[str, str]]:
"""解析 --empty-policy:一个可选的全局默认,加若干 `类别=策略`。
@@ -288,6 +352,10 @@ def main() -> int:
"--keep-names", action="store_true",
help="保留源文件名,不按 <类别>_<序号> 重新编号(源文件名需去掉后缀后全局唯一)",
)
parser.add_argument(
"--label-studio", action="store_true",
help="额外生成 <out>/label_studio/:拍平的硬链接视图 + tasks.json,可直接导入 Label Studio 复核",
)
parser.add_argument("--overwrite", action="store_true", help="输出目录已存在时先清空")
parser.add_argument("--dry-run", action="store_true", help="只打印计划")
args = parser.parse_args()
@@ -373,6 +441,8 @@ def main() -> int:
)
if args.dry_run:
if args.label_studio:
logger.info("--label-studio:会在 %s 下生成导入视图", out_dir / LABEL_STUDIO_DIR)
logger.info("--dry-run:未写入任何文件")
return 0
@@ -419,6 +489,14 @@ def main() -> int:
write_data_yaml(out_dir / "data.yaml", names, root=out_dir.resolve(),
train="images/train", val="images/val")
write_classes_txt(out_dir / "classes.txt", names)
label_studio = None
if args.label_studio:
try:
label_studio = build_label_studio_view(out_dir, len(samples))
except RuntimeError as exc:
logger.error("%s", exc)
return 1
# utf-8-sig:带 BOM,Excel / Numbers 直接打开中文源路径不乱码
with (out_dir / "name_map.csv").open("w", encoding="utf-8-sig", newline="") as handle:
writer = csv.writer(handle)
@@ -435,6 +513,7 @@ def main() -> int:
},
"empty_policy": {"default": default_policy, **per_class_policy},
"renamed": renamed is not None,
"label_studio": label_studio,
"val_ratio": args.val_ratio,
"seed": args.seed,
"split_sizes": {k: len(v) for k, v in splits.items()},
@@ -448,6 +527,21 @@ def main() -> int:
)
logger.info("图片落盘方式: %s(hard=硬链接, copy=跨盘复制, convert=转成 PNG)", dict(modes))
logger.info("完成 -> %s", out_dir)
if label_studio:
# label-studio-converter 自带的提示漏了环境变量(它在循环里改写了 image_root_url,
# 与默认值比较时就不相等了),这里给出带实际路径的完整步骤
logger.info(
"Label Studio 任务 %d 个,导入步骤:\n"
" 1) export LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true\n"
" export LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=%s\n"
" uv run label-studio start\n"
" 2) 新建项目 -> Labeling Setup -> Code,粘贴 %s\n"
" 3) Settings -> Cloud Storage -> Add Source Storage -> Local files,\n"
" 绝对路径填 %s/images。只保存,不要点 Sync(会给每张图再建一个空任务)\n"
" 4) 导入 %s",
label_studio["tasks"], label_studio["document_root"], label_studio["label_config"],
label_studio["document_root"], label_studio["tasks_file"],
)
return 0
+72
View File
@@ -387,3 +387,75 @@ def test_kept_empty_labels_named_after_source_folder(source, tmp_path):
assert run(source, out) == 0
assert output_of(out, "normal_0").stem.startswith("normal_")
assert len([s for s in _stems(out) if s.startswith("normal_")]) == 5
@pytest.fixture
def label_studio_out(source, tmp_path):
pytest.importorskip("label_studio_sdk.converter.imports.yolo")
out = tmp_path / "data" / "boxes"
assert run(source, out, "--empty-policy", "normal=keep", "--label-studio") == 0
return out
def _tasks(out):
return json.loads((out / "label_studio" / "tasks.json").read_text(encoding="utf-8"))
def test_label_studio_view_is_flat_and_hard_linked(label_studio_out):
"""converter 的 os.listdir 不递归,必须拍平;且与训练集共用 inode,不占磁盘。"""
view = label_studio_out / "label_studio"
images = sorted(view.glob("images/*.png"))
assert len(images) == 13
assert not [p for p in (view / "images").iterdir() if p.is_dir()]
for image in images:
original = next(label_studio_out.glob(f"images/*/{image.name}"))
assert image.stat().st_ino == original.stat().st_ino
assert (view / "classes.txt").read_text(encoding="utf-8").split() == CLASSES
def test_label_studio_tasks_are_predictions_with_urls(label_studio_out):
"""每张图一个任务;框作为 predictions,图片 URL 相对 DOCUMENT_ROOT。"""
tasks = _tasks(label_studio_out)
assert len(tasks) == 13
for task in tasks:
assert "annotations" not in task
assert task["data"]["image"].startswith("/data/local-files/?d=images/")
name = task["data"]["image"].rsplit("/", 1)[1]
assert (label_studio_out / "label_studio" / "images" / name).exists()
def test_label_studio_boxes_round_trip_to_yolo(label_studio_out):
"""Label Studio 的左上角百分比坐标反算回 YOLO 中心点坐标,必须与标签一致。"""
view = label_studio_out / "label_studio"
checked = 0
for task in _tasks(label_studio_out):
name = Path(task["data"]["image"].rsplit("/", 1)[1]).stem
lines = (view / "labels" / f"{name}.txt").read_text(encoding="utf-8").split()
results = task["predictions"][0]["result"]
if not lines:
assert results == [] # 保留下来的空标签:任务里没有框
continue
value = results[0]["value"]
class_id, cx, cy, w, h = lines
assert value["rectanglelabels"] == [CLASSES[int(class_id)]]
assert value["x"] / 100 + value["width"] / 200 == pytest.approx(float(cx))
assert value["y"] / 100 + value["height"] / 200 == pytest.approx(float(cy))
assert value["width"] / 100 == pytest.approx(float(w))
checked += 1
assert checked == 12
def test_label_studio_config_and_summary(label_studio_out):
config = (label_studio_out / "label_studio" / "tasks.label_config.xml").read_text(encoding="utf-8")
for name in CLASSES:
assert f'value="{name}"' in config
summary = json.loads((label_studio_out / "summary.json").read_text(encoding="utf-8"))
assert summary["label_studio"]["tasks"] == 13
assert Path(summary["label_studio"]["document_root"]).is_absolute()
def test_label_studio_not_built_by_default(source, tmp_path):
out = tmp_path / "data" / "boxes"
assert run(source, out) == 0
assert not (out / "label_studio").exists()
assert json.loads((out / "summary.json").read_text(encoding="utf-8"))["label_studio"] is None