Using Core ML for semantic image segmentation
At a glance
| Item | Summary |
|---|---|
| Purpose | Identify multiple objects in an image by using the DEtection TRansformer image-segmentation model. |
| App architecture | A Swift sample with the source-visible chain SegmentationApp → MainView → ViewModel → CoreML APIs. |
| Main patterns | Model-View-ViewModel, Binding-based state propagation, Publisher-backed observable state |
| Project style | 5 scanned source file(s) across Swift, organized around ranked entry, type, and file boundaries. |
| Execution model | Source-visible boundaries: await suspension point, @MainActor, Task closure isolated to MainActor, Task; none alone proves a background thread. |
| State/event model | Source-visible mechanisms: SwiftUI state property wrapper, ObservableObject, @Published. |
| Key frameworks/packages | SwiftUI, CoreML, CoreImage, Foundation, ImageIO; these are source dependencies, not architecture labels. |
Project structure
Source bundle/
├── SegmentationApp/
│ ├── SegmentationApp.swift
│ ├── ViewModel.swift
│ ├── MainView.swift
│ └── SegmentationApp.entitlements
├── Common/
│ ├── CoreImageExtensions.swift
│ └── SegmentationModelHelper.swift
├── Configuration/
│ └── SampleCode.xcconfig
└── ImageSegmentationDETR.xcodeproj/
├── .xcodesamplecode.plist
└── project.pbxproj
Structure observations
- Architecturally prominent files are ranked from entry points and role-named declarations; resource-only paths are omitted.
- Primary languages: Swift.
- The verified tree contains 4 project/configuration file(s) and 6 source declaration(s).
Overall architecture
flowchart LR
N1["SegmentationApp"]
N2["MainView"]
N3["ViewModel"]
N4["CoreML APIs"]
N1 --> N2
N2 --> N3
N3 --> N4
Reference code
SegmentationApp/SegmentationApp.swift:10 — architecture anchor
@main
struct CameraApp: App {
init() {
UINavigationBar.applyCustomAppearance()
}
var body: some Scene {
WindowGroup {
MainView()
}
}
}Interpretation
The arrows summarize the source-visible entry, role-named types or folders, and framework direction; when nodes come from structural folders, the sequence is a high-level interpretation rather than proof that every adjacent node calls the next. Ownership is claimed only where the next section cites a stored property or assignment. The diagram is intentionally limited to the dominant path into Core ML.
Ownership and state
classDiagram
ViewModel o-- Image : image
ViewModel o-- Image : maskedImage
ViewModel *-- Array : predictedLabels
ViewModel *-- String : errorMessage
Ownership evidence
SegmentationApp/ViewModel.swift:19 — stored dependency or nearest verified ownership anchor
@MainActor
final class ViewModel: ObservableObject {
// ...
@Published private(set) var image: Image?
// ...
}| Owner | Object or state | Relationship | Mutation authority |
|---|---|---|---|
ViewModel |
Image (image) |
stores or receives | Owning type writes; wider scope can read |
ViewModel |
Image (maskedImage) |
stores or receives | Owning type writes; wider scope can read |
ViewModel |
Array (predictedLabels) |
owns value state | Owning type writes; wider scope can read |
ViewModel |
String (errorMessage) |
owns value state | Owning type writes; wider scope can read |
Composition arrows indicate a source-visible construction expression or locally owned value state; aggregation means the owner stores or receives a dependency without proving exclusive lifetime ownership.
Concurrency, scheduling, and thread safety
Evidence limit: actor isolation, async/await, or Task creation does not by itself prove background-thread execution; Sendable conformance alone does not prove thread-safe mutation.
| Concern | Source mechanism | Verified placement or handoff | Evidence |
|---|---|---|---|
| Suspension boundary | await suspension point |
The source declares or crosses an asynchronous boundary; it does not by itself establish background execution. | SegmentationApp/MainView.swift:88 |
| Main isolation | @MainActor |
The cited annotation marks its attached declaration or closure as main-actor isolated. | SegmentationApp/ViewModel.swift:13 |
| Main isolation | Task closure isolated to MainActor |
The cited operation explicitly enters a main-actor-isolated region. | SegmentationApp/ViewModel.swift:37 |
| Task creation | Task |
The source creates an unstructured task; surrounding context determines inherited actor isolation. | SegmentationApp/ViewModel.swift:37 |
@MainActor/MainActor.run, DispatchQueue.main, and RunLoop.main are reported as distinct isolation, queue, and event-loop mechanisms. A plain Task is kept separate from Task.detached; neither is labeled as a background thread.
Reference code
SegmentationApp/MainView.swift:88 — representative execution boundary
struct MainView: View {
// ...
await viewModel.loadModel()
// ...
}State propagation, frameworks, and dependencies
Evidence limit: an import proves a source-level compilation dependency at the cited line; it does not prove runtime use, architectural adoption, or whether a Swift package is a direct application dependency.
| Category | Mechanism or module | Verified role | Evidence |
|---|---|---|---|
| State propagation | SwiftUI state property wrapper |
A SwiftUI property wrapper supplies or observes UI state. | SegmentationApp/MainView.swift:12 |
| State propagation | ObservableObject |
ObservableObject supplies an observation contract. | SegmentationApp/ViewModel.swift:14 |
| State propagation | @Published |
A published property can emit owner-controlled changes. | SegmentationApp/ViewModel.swift:19 |
| Source import | SwiftUI |
The cited file imports this module; runtime use and architectural role are not inferred. | SegmentationApp/MainView.swift:8 |
| Source import | CoreML |
The cited file imports this module; runtime use and architectural role are not inferred. | Common/SegmentationModelHelper.swift:8 |
| Source import | CoreImage |
The cited file imports this module; runtime use and architectural role are not inferred. | Common/CoreImageExtensions.swift:8 |
| Source import | Foundation |
The cited file imports this module; runtime use and architectural role are not inferred. | SegmentationApp/ViewModel.swift:8 |
| Source import | ImageIO |
The cited file imports this module; runtime use and architectural role are not inferred. | Common/CoreImageExtensions.swift:9 |
receive(on:) describes downstream delivery scheduling, while subscribe(on:) describes upstream subscription/request/cancel scheduling. An import Combine alone establishes neither behavior nor a Store, reducer, Redux, or other application architecture.
Class and protocol design
SegmentationApp/SegmentationApp.swift:11 — representative type boundary
@main
struct CameraApp: App {
// ...
UINavigationBar.applyCustomAppearance()
// ...
}| Type | Responsibility | Depends on or conforms to |
|---|---|---|
CameraApp |
Application entry and top-level composition | App |
ViewModel |
UI-facing state and feature coordination | ObservableObject |
MainView |
User-interface presentation and input forwarding | View |
PredictedLabelsGridView |
User-interface presentation and input forwarding | View |
ViewModelError |
Represents feature failure conditions | Int, Error |
PredictedLabelGridCell |
Represents a feature value or composable behavior | View |
No local protocol conformance is claimed as protocol-oriented design; external framework conformances are listed only as dependencies.
Access control
| Symbol | Access | Verified effect | Likely rationale |
|---|---|---|---|
diameter (SegmentationApp/MainView.swift:159) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
cellHeight (SegmentationApp/MainView.swift:160) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
cornerRadius (SegmentationApp/MainView.swift:161) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
cornerRadii (SegmentationApp/MainView.swift:162) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
Reference code
SegmentationApp/MainView.swift:159 — representative boundary
struct PredictedLabelGridCell: View {
// ...
private static let diameter = CGFloat(22)
// ...
}Swift declarations without a modifier are internal; explicit private, fileprivate, private(set), public, or open entries above are interpreted by language semantics. Objective-C/C samples instead rely on header and implementation boundaries, which are not equivalent to Swift lexical privacy.
Logic ownership and placement
| Logic | Owning type or file | Placement rationale |
|---|---|---|
| Application entry and top-level composition | CameraApp |
The source’s App suffix makes this role explicit. |
| User-interface presentation and input forwarding | MainView, PredictedLabelsGridView |
The source’s View suffix makes this role explicit. |
| UI-facing state and feature coordination | ViewModel |
The source’s ViewModel suffix makes this role explicit. |
Design patterns
| Pattern | Source evidence | Purpose or tradeoff |
|---|---|---|
| Model-View-ViewModel | SegmentationApp/ViewModel.swift:14 |
Role-named view models keep UI-facing state or coordination outside view declarations. |
| Binding-based state propagation | SegmentationApp/MainView.swift:100 |
A binding exposes controlled read/write access while the upstream owner remains authoritative. |
| Publisher-backed observable state | SegmentationApp/ViewModel.swift:19 |
Published properties notify observers while mutation remains with the state object. |
Naming conventions
- Types: App: CameraApp; View: MainView, PredictedLabelsGridView; ViewModel: ViewModel.
- Protocols: no local protocol declaration in the scanned source.
- Methods:
applyCustomAppearance,loadModel,selectLabel,didLoadModel,handleSelectedPhotoTransferable,handleSelectedImage,performInferenceAndUpdateUI,handlePredictionResult. - Files:
SegmentationApp/ViewModel.swift,SegmentationApp/MainView.swift.
Architecture takeaways
SegmentationAppis the main source-visible entry or composition anchor for this sample.- Framework work reaches SwiftUI, CoreML, CoreImage, ImageIO through a deliberately small high-level chain; the detailed API graph remains inside the cited implementation files.
- Stored-property evidence identifies lifecycle collaboration; it does not by itself prove exclusive object ownership.
- Access-control conclusions separate verified language visibility from the likely design rationale.
- The source does not justify labeling the design protocol-oriented.
Source map
| Source file | Relevant symbols |
|---|---|
SegmentationApp/SegmentationApp.swift |
Cited implementation, CameraApp |
SegmentationApp/ViewModel.swift |
Cited implementation, ViewModel, @MainActor, Task closure isolated to MainActor, Task, ObservableObject, @Published, Foundation, ViewModelError |
SegmentationApp/MainView.swift |
Cited implementation, await suspension point, SwiftUI state property wrapper, SwiftUI, MainView, PredictedLabelsGridView, PredictedLabelGridCell |
Common/SegmentationModelHelper.swift |
CoreML, Feature implementation |
Common/CoreImageExtensions.swift |
CoreImage, ImageIO, Feature implementation |