Task Tagging
This page explains how Curator assigns semantic tags to verified tasks. Tags are useful when you want to inspect dataset coverage, filter tasks by domain, or build balanced training/evaluation splits.
Tag schema
Each task receives exactly four tags:
[language, area, topic, bug_class]| Tag | Meaning |
|---|---|
language | Primary programming language, such as python, go, or typescript. |
area | Broad software area: backend, frontend, fullstack, cli, library, or framework. |
topic | Focused framework, library, or technical topic, such as fastapi, react, or async. |
bug_class | Domain-independent failure mode, such as missing-fallback, incomplete-validation, or wrong-default. |
The bug_class tag should describe the logical defect, not just the affected
feature. This makes it easier to compare tasks across languages and projects.
How tagging works
Curator uses the Harbor task-analysis tooling to tag verified tasks. The tagger:
- reads the task instruction, patch, tests, and metadata;
- asks an LLM for the four-tag tuple;
- disables thinking mode so the model returns strict JSON;
- validates the tag shape and closed
areavocabulary; - retries on malformed output.
Very large tasks can make a full prompt slow or unstable. The tagger handles this with progressive truncation: if a request fails, it retries with a shorter patch, instruction, and test excerpt until the request succeeds.
Where tags live
Tags are written to each task's task.toml:
artifacts/swe_tasks/<lang>-cc/<task_id>/task.tomlThey are also read by the dashboard for filtering and comparison.
Regenerating tags
From blocks/curator/dashboard/, run:
python3 ../repos/legoflow-curator/tools/tag_task_metadata.py \
--datasets-dir datasets --dataset all --jobs 64 --retries 3Then rebuild the dashboard HTML:
python3 progress_monitor_multi.py --output-html site/index.htmlTo publish the dashboard, deploy blocks/curator/dashboard/site/ to the
legoflow-curator Cloudflare Pages project.