Written and maintained by CASRAI Editorial Board
Last updated
Image annotation is the process of adding labels to images (class tags, boxes, outlines, masks or points) so that a computer-vision model can be trained and evaluated against them. This guide is for researchers who need to label an image dataset and then hand the labels to analysis or training code: it names the annotation types, walks through one tool end to end (Label Studio Community), and maps the export formats of Label Studio and CVAT to the kinds of label they can carry.
What image annotation is used for in research
An annotation links a region or a whole image to a label a human has decided on. Those labels become the ground truth for supervised models (detection, segmentation, pose estimation, classification) and the reference against which a model’s output is scored. The annotation tool does not decide what a label means; the research team does, which is why the guideline document and the label set matter as much as the software (see the research-practice section below).
The same tools handle the main annotation types. CVAT’s overview lists these shapes: rectangles, polygons, polylines, ellipses, cuboids, skeletons and a brush tool, plus tags, which classify a whole frame rather than a region.
Annotation types and what they record
- Tag or class label: one label for the whole image (CVAT calls these tags; they are frame-level classification).
- Bounding box (rectangle): a rectangle around an object, used for detection.
- Polygon: an outline following an object’s shape, used when a box is too coarse.
- Brush or mask: pixel-level labelling for segmentation.
- Keypoints and skeletons: points (or connected points) on an object, used for pose and landmark tasks.
Install or access (official sources only)
Label Studio
The Label Studio quick start installs the Community edition with pip install label-studio and starts it with label-studio start. Open http://localhost:8080 in a browser and sign up with an email address and password.
CVAT
CVAT is offered as CVAT Community (free, self-hosted), CVAT Online (hosted at app.cvat.ai) and CVAT Enterprise. Self-hosting instructions are in the installation guide of the CVAT documentation; follow that guide rather than a third-party recipe.
Step by step: annotate images in Label Studio
- Install and start Label Studio as above, then sign up at
http://localhost:8080. - Click Create and name the project.
- Open Data Import and upload your images.
- Open Labeling Setup, choose a template that matches the task (for example boxes, polygons, keypoints or brush masks), and customise the label names to your own label set.
- Click Save, open the project’s task list and label each image.
- Click Export in the project, choose a format and click Export (see the next section for choosing the format).
In CVAT, an object can be created either as a shape (drawn on a single frame) or as a track (followed across frames, mainly relevant to video). CVAT also offers annotation modes including Standard, Attribute annotation, Single shape, Tag annotation and Review.
Output formats and how to read them
The export format decides which label types survive. The table below combines the Label Studio export documentation and the CVAT dataset-formats page.
| Format | Tool | Carries |
|---|---|---|
| COCO | Label Studio | Brush masks, rectangles, keypoints, polygons (BrushLabels, RectangleLabels, KeyPointLabels, PolygonLabels) |
| Pascal VOC XML | Label Studio | Rectangles (RectangleLabels) |
| YOLO | Label Studio | Rectangles and keypoints (RectangleLabels, KeyPointLabels) |
| JSON, JSON_MIN, CSV, TSV | Label Studio | Generic task exports |
| Brush labels to NumPy and PNG | Label Studio | Brush masks as image/array files |
| COCO 1.0 (.json) | CVAT | Detection and semantic segmentation |
| COCO Keypoints 1.0 | CVAT | Keypoints |
| PASCAL VOC 1.1 (.xml, .png) | CVAT | Boxes and segmentation images |
| YOLO 1.1 (.txt); Ultralytics YOLO Detection, OBB, Segmentation, Pose, Classification | CVAT | Task-specific YOLO label files |
| Segmentation Mask 1.1 (.png) | CVAT | Masks |
| CVAT for images 1.1 (.xml) | CVAT | CVAT’s own XML |
CVAT also lists Datumaro 1.0, ImageNet, Open Images V6, CamVid, Cityscapes, KITTI, LabelMe 3.0, ICDAR, WIDER Face, LFW, VGGFace2 and Market-1501 among its image formats.
Two details that cause silent errors
- Coordinates in Label Studio JSON are percentages of the image size, not pixels. To convert, use
pixel_x = x / 100.0 * original_width(and the same for height). - COCO category order: in Label Studio’s COCO export, categories are ordered alphabetically unless you set a
categoryattribute, so category ids may not follow the order in which you defined your labels. Check the id-to-name mapping before training.
Label Studio exports include annotated tasks regardless of any filters active on the data tab.
Troubleshooting
The export fails with a 502 or 504 on a large project
In the Community UI, exports are synchronous, and large projects can hit proxy timeouts of about 90 seconds. Label Studio documents alternatives: a snapshot export through the SDK, or the command line, label-studio export <project-id> <export-format> --export-path=<output-path>.
Boxes land in the wrong place after conversion
Check whether you are treating Label Studio’s percentage coordinates as pixels (see above).
A class is missing or shifted in a COCO file
Check the alphabetical category ordering described above, and confirm that the label type you drew is one the chosen format carries (for example Pascal VOC XML from Label Studio carries rectangles only).
Free alternatives
| Tool | Free edition | How it runs | Image shapes (per docs) |
|---|---|---|---|
| Label Studio Community | Yes (Community edition) | Local install with pip, browser interface at localhost:8080 | Rectangles, polygons, keypoints, brush masks |
| CVAT Community | Yes (free, self-hosted) | Self-hosted; CVAT Online is a hosted option | Rectangles, polygons, polylines, ellipses, cuboids, skeletons, brush, tags |
Research-practice notes (editorial)
These points are editorial guidance, not requirements of either tool. Write down label definitions and edge-case rules before annotation starts, so every annotator applies the same guideline. If more than one person labels the data, measure how often they agree on a sample and revisit the guideline when they do not. Keep the exported label files under version control with the label set they were created with, and document how the dataset was built, for example with a datasheet for datasets. For medical or microscopy images, see how related software is documented in the guides on MONAI and arivis image analysis software; for text rather than images, see natural language processing.
FAQ
What is image annotation?
It is the labelling of images with tags, boxes, polygons, masks or keypoints so a computer-vision model can learn from them or be evaluated against them.
Is there free image annotation software?
Yes. Label Studio Community and CVAT Community are both free editions you run yourself; CVAT also has a hosted CVAT Online service.
Which export format should I use?
Pick the format that carries your label type: COCO for boxes, polygons, masks or keypoints; Pascal VOC XML or YOLO when your training code expects boxes in that layout; PNG or NumPy masks for segmentation. See the table above.
Why are my exported box coordinates not in pixels?
Label Studio stores them as percentages of image width and height; multiply by the original dimensions and divide by 100.
Sources
- CVAT documentation, getting started overview: docs.cvat.ai/docs/getting_started/overview (read 2026-11-07).
- CVAT documentation, dataset formats: docs.cvat.ai/docs/dataset_management/formats (read 2026-11-07).
- Label Studio documentation, quick start: labelstud.io/guide/quick_start (read 2026-11-07).
- Label Studio documentation, export annotations: labelstud.io/guide/export (read 2026-11-07).
