Displaying low-latency connected video
At a glance
| Item | Summary |
|---|---|
| Purpose | Render connected camera feeds in visionOS with minimal latency. |
| App architecture | LowLatencyCameraStreamingApp owns AppModel; CameraCoordinator separates device/capture lifecycle from LowLatencyRenderer, which transfers camera frames through a shared Metal texture into RealityKit content. |
| Main patterns | SwiftUI scene composition, UI–RealityKit bridge, Observable state owner, Session owner boundary, Async update consumer, Capture-to-shared-texture pipeline |
| Project style | Code-rich sample with 12 scanned Swift file(s) and 1300 implementation line(s). |
Project structure
Source bundle/
└── RealityKit-LowLatencyCameraStreaming/
├── Model/
│ └── AppModel.swift # ResolutionPreset, AppModel
├── CameraCapture/
│ ├── CameraCoordinator.swift # CameraCoordinator
│ └── CameraCaptureSession.swift # CameraCaptureEvent, CameraError, CameraConfiguration
├── Rendering/
│ └── LowLatencyRenderer.swift # LowLatencyRenderer, RendererError
├── LowLatencyCameraStreamingApp.swift # LowLatencyCameraStreamingApp
└── Views/
├── CameraTextureRealityView.swift # CameraTextureRealityView
└── CameraControlsView.swift # CameraControlsView
Structure observations
- The runtime boundary is WindowGroup; the tree is pruned to lifecycle, state, framework, rendering, and ECS files.
- The source is Swift; role-named types and feature folders separate the major responsibilities.
Overall architecture
flowchart LR
LowLatencyCameraStreamingApp_1["LowLatencyCameraStreamingApp"]
WindowGroup_2["WindowGroup"]
CameraTextureRealityView_3["CameraTextureRealityView"]
AppModel_4["AppModel"]
LowLatencyRenderer_5["LowLatencyRenderer"]
RealityKit_entity_graph_6["RealityKit entity graph"]
AV_capture___Metal_shared_texture_7["AV capture + Metal shared texture"]
LowLatencyCameraStreamingApp_1 --> WindowGroup_2
WindowGroup_2 --> CameraTextureRealityView_3
CameraTextureRealityView_3 --> AppModel_4
AppModel_4 --> LowLatencyRenderer_5
LowLatencyRenderer_5 --> RealityKit_entity_graph_6
RealityKit_entity_graph_6 --> AV_capture___Metal_shared_texture_7
Reference code
RealityKit-LowLatencyCameraStreaming/LowLatencyCameraStreamingApp.swift:11 — executable or app-lifecycle anchor.
struct LowLatencyCameraStreamingApp: App {
// ...
}The diagram is a responsibility flow, not a claim that every adjacent node directly calls the next. It separates lifecycle, presentation, stable state, framework/session work, and the RealityKit entity graph.
Ownership and state
classDiagram
LowLatencyCameraStreamingApp *-- AppModel : appModel
AppModel --> CameraCoordinator : cameraCoordinator
CameraTextureRealityView o-- AppModel : appModel
AppModel --> ResolutionPreset : resolution
Ownership evidence
RealityKit-LowLatencyCameraStreaming/LowLatencyCameraStreamingApp.swift:13 — representative stored state or nearest verified lifecycle anchor.
@main
struct LowLatencyCameraStreamingApp: App {
// ...
@State private var appModel = AppModel()
// ...
}| Owner | Object or state | Relationship | Mutation authority |
|---|---|---|---|
LowLatencyCameraStreamingApp |
AppModel as appModel |
creates and retains | The declaring type controls writes |
AppModel |
CameraCoordinator as cameraCoordinator |
stores and coordinates | The declaring type controls writes |
CameraTextureRealityView |
AppModel as appModel |
receives a shared or non-owning reference | The upstream owner controls lifetime; this scope may invoke the exposed mutable API |
AppModel |
ResolutionPreset as resolution |
owns value state | The owning type coordinates writes |
Ownership is intentionally narrow: environment, bindings, observed objects, weak links, and delegate callbacks do not prove lifetime ownership. Initialized stored services and wrapper-managed state do; entities added inside a RealityKit content closure belong to that entity graph.
Class and protocol design
| Type | Responsibility | Depends on or conforms to |
|---|---|---|
LowLatencyCameraStreamingApp (RealityKit-LowLatencyCameraStreaming/LowLatencyCameraStreamingApp.swift:11) |
Declares app lifecycle and top-level dependency lifetime. | App |
CameraTextureRealityView (RealityKit-LowLatencyCameraStreaming/Views/CameraTextureRealityView.swift:12) |
Presents UI and forwards gestures or lifecycle events. | View |
AppModel (RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:35) |
Owns feature state and domain transitions. | Concrete framework collaborators |
LowLatencyRenderer (RealityKit-LowLatencyCameraStreaming/Rendering/LowLatencyRenderer.swift:13) |
Wraps a framework, input, rendering, or session capability. | Concrete framework collaborators |
CameraCoordinator (RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCoordinator.swift:13) |
Coordinates long-lived framework or cross-view work. | Concrete framework collaborators |
CameraCaptureSession (RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCaptureSession.swift:40) |
Wraps a framework, input, rendering, or session capability. | NSObject |
CameraControlsView (RealityKit-LowLatencyCameraStreaming/Views/CameraControlsView.swift:11) |
Presents UI and forwards gestures or lifecycle events. | View |
CameraResolutionView (RealityKit-LowLatencyCameraStreaming/Views/CameraResolutionView.swift:10) |
Presents UI and forwards gestures or lifecycle events. | View |
The reviewed source defines no local substitution protocol. Its App, View, delegate, renderer, ARKit, and RealityKit conformances are framework-facing contracts, so this document does not label the whole app protocol-oriented.
Access control
| Symbol | Access | Verified effect | Likely rationale |
|---|---|---|---|
private(set) var renderer: LowLatencyRenderer? (RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCoordinator.swift:20) |
private(set) |
The getter keeps its wider visibility, while writes stay in the declaring type and permitted same-file extensions. | Inference: Protect an invariant while allowing observation. |
private let logger = Logger(subsystem: "com.example.low-latency-streami... (RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCaptureSession.swift:42) |
private |
Use stays inside the declaration and same-file extensions permitted by Swift. | Inference: Hide lifecycle-sensitive state or an implementation step. |
No reviewed Swift access example uses fileprivate, public, open; declarations without a modifier are internal.
Reference code
RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCoordinator.swift:20 — representative visibility boundary.
private(set) var renderer: LowLatencyRenderer?
@ObservationIgnored private var eventTask: Task<Void, Never>?
var isStereo: Bool = falseLogic ownership and placement
| Logic | Owning type or file | Placement rationale |
|---|---|---|
| Application/command lifecycle | LowLatencyCameraStreamingApp |
The executable boundary determines app, controller, scene, or command lifetime. |
| Presentation and user intent | CameraTextureRealityView |
The view/controller forwards input and owns only presentation-local work. |
| Shared feature state and commands | AppModel |
A stable owner prevents view recomputation or callbacks from duplicating transitions. |
| Framework/session/render coordination | LowLatencyRenderer |
The role-named collaborator isolates long-lived or per-frame work from presentation code. |
| GPU and low-level resource work | RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:70 |
Buffer, texture, shader, or frame-resource mutation stays in a rendering/compute boundary. |
Design patterns
| Pattern | Source evidence | Purpose or tradeoff |
|---|---|---|
| SwiftUI scene composition | RealityKit-LowLatencyCameraStreaming/LowLatencyCameraStreamingApp.swift:11 |
Keeps windows, volumes, and immersive presentation visible at the app boundary. |
| UI–RealityKit bridge | RealityKit-LowLatencyCameraStreaming/Views/CameraTextureRealityView.swift:44 |
Builds or hosts the RealityKit entity graph from SwiftUI or a platform view/controller lifecycle. |
| Observable state owner | RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:35 |
Keeps shared feature state in a stable owner rather than in transient view values. |
| Session owner boundary | RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCaptureSession.swift:46 |
Keeps authorization, session lifetime, and framework callbacks in a stable model, manager, or controller. |
| Async update consumer | RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:91 |
Consumes long-lived framework output in a cancellable asynchronous task. |
| Capture-to-shared-texture pipeline | RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:70 |
Separates capture and rendering lifetimes while avoiding an unnecessary CPU copy before RealityKit display. |
Naming conventions
- Role suffixes are evidence, not decoration: App:
LowLatencyCameraStreamingApp; Model:AppModel; Coordinator:CameraCoordinator; Session:CameraCaptureSession; Renderer:LowLatencyRenderer. - Protocols: no app-defined protocol in the reviewed source.
- Commands use verb-led methods:
observeDeviceChanges,handleDeviceRemoved,createCameraFeedTexture,enableCamera,disableCamera,start,stop,startObservingEvents. - Files and folders use primary-type or responsibility names such as
Views,Models,Rendering,Components,Systems,Packages, or feature names where those boundaries exist.
Architecture takeaways
- Treat
LowLatencyCameraStreamingAppas the owner of executable or scene lifecycle, not automatically as the owner of every entity created later. - Keep view/controller input near presentation, but move sessions, reconstruction, capture, rendering, shared game state, or documents into stable owners when their lifetime exceeds one callback or render pass.
- Tie asynchronous session/output consumers to an explicit owner so cancellation and teardown follow the same lifecycle.
Source map
| Source file | Relevant symbols |
|---|---|
RealityKit-LowLatencyCameraStreaming/Model/AppModel.swift:13 |
ResolutionPreset, AppModel |
RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCoordinator.swift:13 |
CameraCoordinator |
RealityKit-LowLatencyCameraStreaming/Rendering/LowLatencyRenderer.swift:13 |
LowLatencyRenderer, RendererError |
RealityKit-LowLatencyCameraStreaming/LowLatencyCameraStreamingApp.swift:11 |
LowLatencyCameraStreamingApp |
RealityKit-LowLatencyCameraStreaming/Views/CameraTextureRealityView.swift:12 |
CameraTextureRealityView |
RealityKit-LowLatencyCameraStreaming/CameraCapture/CameraCaptureSession.swift:14 |
CameraCaptureEvent, CameraError, CameraConfiguration, CameraCaptureSession |
RealityKit-LowLatencyCameraStreaming/Views/CameraControlsView.swift:11 |
CameraControlsView |