Rendering stereoscopic video with RealityKit

At a glance

Project structure

Source bundle/
└── RealityKit-Playback-AVSampleBufferVideoRenderer/
    ├── Models/
    │   ├── PlayerModel.swift  # PlayerModel, PlayerState
    │   ├── LoopingVideoPlayer.swift  # LoopingVideoPlayer
    │   ├── SerialProcessor.swift  # SerialProcessor
    │   └── StereoMetadata.swift  # StereoMetadata, FramePacking
    ├── RealityKitPlaybackApp.swift  # RealityKitPlaybackApp
    └── Views/
        └── ContentView.swift  # ContentView

Structure observations

• The runtime boundary is WindowGroup; the tree is pruned to lifecycle, state, framework, rendering, and ECS files.
• The source is Swift; role-named types and feature folders separate the major responsibilities.

Overall architecture

flowchart LR
    RealityKitPlaybackApp_1["RealityKitPlaybackApp"]
    WindowGroup_2["WindowGroup"]
    ContentView_3["ContentView"]
    PlayerModel_4["PlayerModel"]
    RealityKit_entity_graph_5["RealityKit entity graph"]
    AV_sample_buffers___RealityKit_video_6["AV sample buffers + RealityKit video"]
    RealityKitPlaybackApp_1 --> WindowGroup_2
    WindowGroup_2 --> ContentView_3
    ContentView_3 --> PlayerModel_4
    PlayerModel_4 --> RealityKit_entity_graph_5
    RealityKit_entity_graph_5 --> AV_sample_buffers___RealityKit_video_6

Reference code

RealityKit-Playback-AVSampleBufferVideoRenderer/RealityKitPlaybackApp.swift:12 — executable or app-lifecycle anchor.

struct RealityKitPlaybackApp: App {
    // ...
}

The diagram is a responsibility flow, not a claim that every adjacent node directly calls the next. It separates lifecycle, presentation, stable state, framework/session work, and the RealityKit entity graph.

Ownership and state

classDiagram
    PlayerModel --> LoopingVideoPlayer : player
    RealityKitPlaybackApp *-- PlayerModel : player
    PlayerModel --> PlayerState : state
    PlayerModel --> Bundle : resourceURL

Ownership evidence

RealityKit-Playback-AVSampleBufferVideoRenderer/Models/PlayerModel.swift:43 — representative stored state or nearest verified lifecycle anchor.

@MainActor
@Observable
final class PlayerModel {
    // ...
    private let player: LoopingVideoPlayer
    // ...
}

Ownership is intentionally narrow: environment, bindings, observed objects, weak links, and delegate callbacks do not prove lifetime ownership. Initialized stored services and wrapper-managed state do; entities added inside a RealityKit content closure belong to that entity graph.

Class and protocol design

The reviewed source defines no local substitution protocol. Its App, View, delegate, renderer, ARKit, and RealityKit conformances are framework-facing contracts, so this document does not label the whole app protocol-oriented.

Access control

No reviewed Swift access example uses fileprivate, private(set), public, open; declarations without a modifier are internal.

Reference code

RealityKit-Playback-AVSampleBufferVideoRenderer/Models/PlayerModel.swift:16 — representative visibility boundary.

@MainActor
@Observable
final class PlayerModel {
    // ...
}

Logic ownership and placement

Design patterns

Naming conventions

• Role suffixes are evidence, not decoration: App: RealityKitPlaybackApp; Model: PlayerModel; View: ContentView.
• Protocols: no app-defined protocol in the reviewed source.
• Commands use verb-led methods: load, play, stop, enqueueProcessor, loop, process, mediaDataReadyStream, transform.
• Files and folders use primary-type or responsibility names such as Views, Models, Rendering, Components, Systems, Packages, or feature names where those boundaries exist.

Architecture takeaways

• Treat RealityKitPlaybackApp as the owner of executable or scene lifecycle, not automatically as the owner of every entity created later.
• Keep view/controller input near presentation, but move sessions, reconstruction, capture, rendering, shared game state, or documents into stable owners when their lifetime exceeds one callback or render pass.
• Tie asynchronous session/output consumers to an explicit owner so cancellation and teardown follow the same lifecycle.

Source map

• Official Apple documentation