Advanced Segmentation Techniques

Principles and Suitability

U-Net uses an encoder-decoder and skip connections to restore detail. Mask R-CNN detects instances and creates a mask for each. Segment Anything (SAM) uses prompts to help select objects, but does not automatically understand a factory-specific defect definition.

Model names describe information-processing methods, not a ranking from old to new in suitability. U-Net combines broad context and detail for pixel outputs; Mask R-CNN separates objects before predicting shapes. SAM accepts hints, such as points or boxes, to select regions. Choose according to the available reference answers and who can supply prompts during actual operation, rather than assuming those inputs will always be present.

An encoder reduces information to a representation; a decoder restores prediction resolution. Skip connections carry detail from earlier stages. Check output heads, prompt handling and coordinate restoration for each selected version. General object selection is not the same as identifying contamination that meets an operational defect definition. That distinction requires explicit criteria and additional verified data.

Scroll the diagram horizontally to inspect all details.

Illustrative Industrial Example

Illustrative design scenario: compare U-Net for one pointer type, Mask R-CNN for multiple objects and SAM as an annotation assistant. A mask proposed by SAM must be reviewed before being used as ground truth.

Suppose a reviewer clicks a pointer and receives a mask including its shadow. Inspect and correct it rather than accepting its smooth boundaries. If several clicks are required, include that effort in annotation costs. Assisting a person and operating automatically without prompts are distinct roles needing separate evaluations; success in one cannot establish success in the other.

Compare methods on identical images and answers, transparently stating inputs such as assisting boxes. If prompts come from a detector, include its errors in final results. Record prompt positions, weight versions and image preparation so mask changes can be attributed to models or assisting information. Never select prompts using test answers that a real deployment would not possess.

Study and Trial Workflow

Evaluation and Metric Definitions

Compare Dice or IoU by object size, annotation time and latency using the same images and hardware. State whether prompt-generation time is included so that interactive and automatic approaches are not compared on different timing boundaries.

Break overlap scores down for thin targets, neighbouring objects and ambiguous edges, checking whether shapes support the next calculation. Annotation assistance should measure corrections and unnoticed mistakes, not only proposal time. Explain differences between human-assisted and automatic conditions rather than ranking scores as if both received identical information. Assistance is part of the evaluation setup, not an invisible advantage.

Check reproducibility with the same prompts and images, including coordinate conversion after resizing. Test materials and lighting excluded from adaptation. If masks feed angle reading, measure angle errors too. For multiple objects, inspect identity switches: aggregate area scores may hide merging two objects or splitting one, affecting counts and tracking even when much of the foreground area is correct.

Limitations and Failure Cases

General-purpose models may select shadows or reflections as objects. Domain adaptation based on incorrect pseudo-labels can accumulate error rather than improve performance, so proposed labels still require validation.

Pseudo-labels are model-generated answers used for further training. Unreviewed predictions can reinforce previous errors. Adaptation therefore needs independently checked answers and a test set excluded from tuning. Renaming a model or adding hints cannot recover absent image detail. Mark invisible information rather than constructing complete boundaries merely to make an output appear trustworthy.

Multiple model roles add versioning, weight-licensing and integration work with reviewers or detection systems. Choose based on whole-process costs and operating scope. Public papers or datasets do not demonstrate suitability for every material. Reference-guided experiments inform design decisions only; they do not assert that a product currently supports these models or that its operational results match published benchmarks.

Practical Checklist

• Are the segmentation prompts and model weights recorded?
• Can IDs be distinguished when several instances are present?
• Is there a cross-site test dataset excluded from model tuning?

References and Further Reading

• Ronneberger et al. — U-Net
• He et al. — Mask R-CNN
• Kirillov et al. — Segment Anything