State what must be unfamiliar
A random image split answers a different question from a new-location or new-product evaluation. Identify the source of novelty expected after deployment. You may need unfamiliar photographed objects, unfamiliar photographers, or unfamiliar settings. Write that objective before allocating percentages, because percentages alone do not establish meaningful separation between training and evaluation.
Build the grouping inventory
Request identifiers for capture sessions, subjects, product instances, locations, and original assets where relevant and available. Some relationships may be unknown in an existing archive. Record those limitations directly rather than filling gaps with confident guesses. Grouping is only useful when its underlying identifiers correspond to the independence assumption your evaluation needs.
Balance without breaking boundaries
Keeping groups together can make category balance more difficult, particularly with rare classes. Decide which constraints have priority and document any exceptions. Avoid moving a convenient related image across the boundary just to improve a class count. Report group counts alongside image counts so a large evaluation set does not hide limited underlying diversity. If an identifier describes a catalog family rather than a physical specimen, make that distinction explicit before using it as the grouping boundary for evaluation.
Freeze and revisit deliberately
Save the split assignment with a dataset release identifier. New images should pass duplicate and relationship checks before being assigned. If revised labels or newly discovered source relationships require a split change, issue a documented revision. This keeps comparisons interpretable and gives sourcing partners a concrete rule for expanding the collection responsibly.
Review checklist
- Identify the intended source of novelty.
- Request the grouping identifiers needed to test it.
- Report category and group counts together.
- Version split assignments when the collection changes.