Creating a spatial drawing app with RealityKit

At a glance

Project structure

Source bundle/
├── RealityKitDrawingApp/RealityKitDrawingApp.swift  # RealityKitDrawingApp, Mode, SetModeKey
├── RealityKitDrawingApp/Document/DrawingDocument.swift  # Chirality, InputData, DrawingDocument
├── RealityKitDrawingApp/Document/AnchorEntityInputProvider.swift  # HandComponent, AnchorEntityInputProvider, DrawingSystem
├── RealityKitDrawingApp/Brushes/Solid/Style/SolidBrushStyleProvider.swift  # SolidBrushStyleProvider, ThicknessType, Settings
├── RealityKitDrawingApp/Canvas/DrawingCanvasPlacementView.swift  # DrawingCanvasPlacementView, DrawingCanvasPlacementComponent, DrawingCanvasPlacementSystem
├── RealityKitDrawingApp/Utilities/CurveProcessing.swift  # HermiteInterpolant, HermiteControlPoint, SubdivisionSearchItem
├── RealityKitDrawingApp/Brushes/Solid/SolidBrushComponent.swift  # SolidBrushComponent, SolidBrushSystem
└── RealityKitDrawingApp/Packages/RealityKitContent/Package.realitycomposerpro/ProjectData/main.json  # authored RealityKit content

Structure observations

• The runtime boundary is WindowGroup + ImmersiveSpace; the tree is pruned to lifecycle, state, framework, rendering, and ECS files.
• The source is C/Objective-C header, Metal, Swift; role-named types and feature folders separate the major responsibilities.
• Authored .reality, .rkassets, or Reality Composer Pro content is an implementation boundary; Swift loads or drives it rather than reproducing its entity graph.

Overall architecture

flowchart LR
    RealityKitDrawingApp_1["RealityKitDrawingApp"]
    WindowGroup___ImmersiveSpace_2["WindowGroup + ImmersiveSpace"]
    DrawingCanvasPlacementView_3["DrawingCanvasPlacementView"]
    DrawingDocument_4["DrawingDocument"]
    AnchorEntityInputProvider_5["AnchorEntityInputProvider"]
    RealityKit_entity_graph_6["RealityKit entity graph"]
    tracked_input___low_level_RealityKit_drawing_7["tracked input + low-level RealityKit drawing"]
    RealityKitDrawingApp_1 --> WindowGroup___ImmersiveSpace_2
    WindowGroup___ImmersiveSpace_2 --> DrawingCanvasPlacementView_3
    DrawingCanvasPlacementView_3 --> DrawingDocument_4
    DrawingDocument_4 --> AnchorEntityInputProvider_5
    AnchorEntityInputProvider_5 --> RealityKit_entity_graph_6
    RealityKit_entity_graph_6 --> tracked_input___low_level_RealityKit_drawing_7

Reference code

RealityKitDrawingApp/RealityKitDrawingApp.swift:12 — executable or app-lifecycle anchor.

struct RealityKitDrawingApp: App {
    // ...
}

The diagram is a responsibility flow, not a claim that every adjacent node directly calls the next. It separates lifecycle, presentation, stable state, framework/session work, and the RealityKit entity graph.

Ownership and state

classDiagram
    DrawingCanvasPlacementView *-- SpatialTrackingSession : session
    DrawingDocument --> Entity : rootEntity
    RealityKitDrawingApp *-- Mode : mode
    RealityKitDrawingApp *-- BrushState : brushState

Ownership evidence

RealityKitDrawingApp/Canvas/DrawingCanvasPlacementView.swift:22 — representative stored state or nearest verified lifecycle anchor.

struct DrawingCanvasPlacementView: View {
    // ...
    @State private var session: SpatialTrackingSession?
    // ...
}

Ownership is intentionally narrow: environment, bindings, observed objects, weak links, and delegate callbacks do not prove lifetime ownership. Initialized stored services and wrapper-managed state do; entities added inside a RealityKit content closure belong to that entity graph.

Class and protocol design

Verified protocol-oriented seams: Float → HermiteInterpolant, SIMD2 → HermiteInterpolant, SIMD3 → HermiteInterpolant, SIMD4 → HermiteInterpolant. Framework conformances remain adapters, not proof of an app-wide protocol architecture.

Access control

No reviewed Swift access example uses open; declarations without a modifier are internal.

Objective-C/C header and implementation visibility is documented separately from Swift lexical access control.

Reference code

RealityKitDrawingApp/Brushes/Solid/CurveExtruder.swift:30 — representative visibility boundary.

    private(set) var samples: [CurveSample] = []
    
    /// The number of samples in `samples` that have been meshed in `lowLevelMesh`.
    private var cachedSampleCount: Int = 0
    
    /// The number of samples for which `lowLevelMesh` has capacity.
    @MainActor
    private var sampleCapacity: Int {
        // ...
    }

Logic ownership and placement

Design patterns

Naming conventions

• Role suffixes are evidence, not decoration: App: RealityKitDrawingApp; Document: DrawingDocument; Provider: AnchorEntityInputProvider, SolidBrushStyleProvider, SparkleBrushStyleProvider; System: DrawingCanvasPlacementSystem, DrawingCanvasVisualizationSystem, DrawingSystem, SolidBrushSystem, SparkleBrushSystem; Component: DrawingCanvasPlacementComponent, DrawingCanvasVisualizationComponent, HandComponent, SolidBrushComponent, SparkleBrushComponent.
• ECS names pair data and behavior: components SolidBrushComponent, SparkleBrushComponent, DrawingCanvasPlacementComponent, DrawingCanvasVisualizationComponent, HandComponent; systems SolidBrushSystem, SparkleBrushSystem, DrawingCanvasPlacementSystem, DrawingCanvasVisualizationSystem, DrawingSystem.
• Protocols: HermiteInterpolant.
• Commands use verb-led methods: setMode, receive, update, radius, styleInput, distance, length, dot.
• Files and folders use primary-type or responsibility names such as Views, Models, Rendering, Components, Systems, Packages, or feature names where those boundaries exist.

Architecture takeaways

• Treat RealityKitDrawingApp as the owner of executable or scene lifecycle, not automatically as the owner of every entity created later.
• Keep view/controller input near presentation, but move sessions, reconstruction, capture, rendering, shared game state, or documents into stable owners when their lifetime exceeds one callback or render pass.
• Use components for per-entity data and systems for repeated simulation instead of centralizing every update in a view model or controller.
• Count authored Reality Composer Pro assets as implementation; the Swift shell is not the complete architecture.

Source map

• Official Apple documentation