A 3D MRI segmentation is not a stack of independent drawings. Each label belongs to a voxel in a volume, and decisions made on one slice affect the shape seen on adjacent slices. If annotators interpret the target, boundary or image series differently, extra review alone will not fix the disagreement. The remedy is a written rulebook that settles those choices before production.
Start by defining the clinical or research question and the exact structure to segment. Specify the intended use of the labels, the required output and who is qualified to resolve uncertain anatomy. Annotation is not diagnosis: clinical interpretation and final acceptance should remain with the designated clinical reviewer.
Fix the image volume before labeling
Identify which DICOM series annotators should use for each case. State the sequence or contrast requirement, the anatomical coverage needed, and what to do when a series is missing, incomplete or unsuitable. If a study contains multiple candidate series, define the selection rule rather than leaving the choice to individual annotators.
Tell the team to preserve the source image and its spatial information. Record the series and image instances used, along with relevant orientation and voxel-spacing information. Slice thickness and in-plane spacing affect how a boundary appears in three dimensions. If the workflow resamples images, document when and how; do not compare masks as though their voxel grids match when they do not.
Give cases a clear status when they cannot be labeled under the rulebook, such as incomplete coverage or unusable image quality. Define whether the case is held for review, excluded, or sent back for clarification. A forced label can look complete while concealing a data problem.
Describe the target at voxel level
For each structure, name the anatomy and give inclusion and exclusion rules. Define the superior and inferior extent, whether connected components are included, and how to treat adjacent structures that touch or overlap in the image. Where a boundary is indistinct, specify the evidence annotators should use and when they must mark uncertainty or escalate.
Make the boundary convention explicit. For example, state whether the label follows the visible outer edge of the structure, excludes surrounding fluid, or includes a defined rim. Avoid instructions such as “trace accurately” or “follow the anatomy.” They set an expectation but do not resolve competing interpretations.
Define how the rule applies across slices. Say whether annotators should label every slice containing the target, how to handle a structure that appears only partially at the volume edge, and how to treat gaps or apparent discontinuities. Ask for a coherent 3D shape, but do not tell annotators to smooth away real changes simply to make the volume look regular. If interpolation is permitted between annotated slices, state where it is allowed and require review of the interpolated region.
Write rules for difficult cases
Use a short edge-case section with examples from the relevant population and image protocol. Cover partial-volume boundaries, low contrast, motion artefact, lesions or other abnormalities that alter the expected shape, and uncertain margins. For each case, give an action: label to a stated rule, mark as uncertain, or refer for adjudication. If abnormalities are outside the scope, say so and define whether they are excluded from the target or make a case ineligible.
Keep examples consistent with the written rule. Include paired examples where a boundary is included in one case and excluded in another, and explain the distinction. Reviewers should be able to point to a specific rule when they correct a label, not rely on a private convention.
Set review criteria before the pilot
Choose quality measures that fit the structure and intended use. A whole-volume overlap score can hide a poor boundary on a small structure, so consider reviewing results by structure and case, alongside a boundary-focused measure where appropriate. Set thresholds only after qualified reviewers have examined representative examples. A single universal threshold is unlikely to describe every anatomy, scan quality or downstream requirement.
Run a small, representative batch before scaling. Have annotators label the same cases independently, then compare both the masks and the reasons for disagreement. Separate guideline ambiguity from execution errors. Revise the rulebook, relabel the pilot if needed, and test whether the changes improve agreement on the disputed cases. Track versions of the guidelines and keep a record of adjudicated decisions so later batches use the same interpretation.
XYNTRIQ lists medical imaging segmentation among its work and describes domain-matched annotators, multi-tier quality checks and a pilot in which a sample is labeled against the buyer’s guidelines. That makes a pilot a practical place to test whether the DICOM instructions are usable before committing to a larger batch. Send representative cases, the draft rulebook and the acceptance criteria; review the returned labels and disagreements before approving the production scope.
The useful guideline is not the longest one. It is the one that makes volume selection, voxel boundaries, uncertain cases and review decisions reproducible. Write those rules first, test them on real scans, and revise them where qualified reviewers still disagree.
XYNTRIQ