Start with the decision
Describe the decision the model must support before naming classes. A retail sorter may need packaging type, while a visual search service may need product identity. Those objectives require different distinctions even when the photographs look similar. Give reviewers examples of decisions that matter and decisions that should be ignored.
Separate visible evidence from inference
Require labels to follow evidence present in the image. A sealed container may show a brand without revealing its contents. Decide whether partially visible marks, unreadable text, and ambiguous variants receive a broader label or an uncertainty flag. Avoid quietly turning contextual assumptions into authoritative ground truth during annotation.
Make relationships explicit
Explain whether classes are mutually exclusive, hierarchical, or independent attributes. A photograph can contain several objects and several properties at once. Write rules for images that satisfy more than one category and for objects outside the taxonomy. Include counterexamples so annotators can distinguish a deliberate boundary from an accidental omission. Keep business outcomes separate from visual labels: an image category should not quietly encode an operational decision that changes between customers or application settings.
Pilot disagreements before scale
Ask several reviewers to label the same small sample, then examine disagreements by class. Rewrite the confusing definitions and repeat the exercise before expanding collection. Preserve the taxonomy version with every delivery, including retired labels and mappings. A sourcing conversation becomes more useful when the brief includes these concrete examples.
Review checklist
- Define the downstream decision in one sentence.
- Provide positive examples and counterexamples per class.
- Choose an explicit uncertainty and out-of-scope policy.
- Record taxonomy versions alongside annotation files.