Image Segmentation

Principles and Suitability

Segmentation assigns labels at pixel level. Semantic segmentation groups objects of the same class, while instance segmentation separates individual objects. It is useful when shape or area matters more than a rectangular bounding box.

Think of colouring the actual target rather than drawing a surrounding rectangle. A mask maps which pixels belong to it. A box may not be enough for contamination-area measurement, whereas detailed pixel labels may be unnecessary merely to establish that a gauge exists. Start with the decision that requires shape information and the resolution supported by the image, not with a preference for the most elaborate annotation format.

Semantic labels identify types without separating same-type objects; instance labels add individual identities. Define whether mask outputs are classes or scores before thresholding, and how masks become angles or areas. For example, select a connected component related to the pivot. Counting pixels directly does not yield physical area until image scale, viewpoint and object distance have been checked.

Scroll the diagram horizontally to inspect all details.

Illustrative Industrial Example

Illustrative design scenario: a pointer mask supports estimating its axis, while a contamination mask on a workpiece supports measuring area. Calibration is required before converting image area into physical units.

Suppose a pointer's shadow is labelled together with the pointer. The mask may look continuous and score well, but its calculated axis can tilt towards the shadow. Show the original image, mask and axis together to separate boundary errors from shape-summary errors. Automatically choosing the largest component could select contamination or background instead of a much thinner pointer.

For hypothetical contamination on a plane, verify coordinate conversion and length scaling before area calculation: area depends on scale in two directions. Check whether holes and missing mask sections belong to the actual object or result from filtering. State rules for combining small components. Curved surfaces or different distances require reconsidering the model rather than extending a planar method without checking its assumptions.

Study and Trial Workflow

Evaluation and Metric Definitions

Measure IoU or Dice per class and also check final angle or area error. A large background region can produce high pixel accuracy even when the pointer is completely missed, making a single aggregate score misleading.

IoU compares intersection with union area; Dice expresses overlap using a different ratio. State whether results are averaged per image or pooled across all pixels. Background must not conceal small-target performance. Inspect boundaries important for reading, such as pointer tips, rather than judging one whole-image similarity score. Separately show completely missed targets so their absence remains visible in the evaluation.

Check masks before and after hole filling or small-component removal: tidy postprocessing may delete real information. Test restoration to original image size and angle or area calculations against verifiable answers. If area scores are good but final values wrong, inspect pivots, critical boundaries and calibration before deciding that a larger model is needed. The conversion stage can introduce errors independent of segmentation quality.

Limitations and Failure Cases

Mask boundaries depend on resolution and annotator judgement. Partial occlusion does not provide the entire true shape, and a visually appealing mask does not establish metrological accuracy.

Ambiguous edges can reasonably produce different labels. Define inner edges, outer edges or uncertain states rather than forcing every pixel into apparently certain answers. Reconstructing an occluded shape is a methodological assumption, not direct observation. Distinguish inferred additions from evidence-backed regions so users do not mistake an estimated area for a completely observed measurement.

Detailed masks increase annotation and computation costs; reduced resolution may erase thin targets. Choose according to tolerated final errors and reference uncertainty. Metrological guidance helps organise uncertainty contributions, but is not proof of instrument or product certification. Images missing required detail should not support precise physical-size claims beyond their evidence, even when the resulting mask looks convincing.

Practical Checklist

• Does a thin target remain visible after resizing?
• Are decision rules defined for ambiguous boundaries?
• Are mask components checked for connections that should not exist?

References and Further Reading

• Ronneberger et al. — U-Net
• He et al. — Mask R-CNN