Capturing depth using the LiDAR camera

At a glance

Project structure

LiDARDepth/
├── LiDARDepthApp.swift
├── ContentView.swift
├── Model/
│   ├── CameraManager.swift
│   ├── CameraController.swift
│   └── CVPixelBuffer+Extension.swift
└── Views/
    ├── DepthOverlay.swift
    ├── ZoomOnTap.swift
    ├── View+Extensions.swift
    └── Metal/
        ├── MetalContentView.swift
        ├── MetalEnvironment.swift
        ├── MetalPointCloud.swift
        ├── MetalTexture*.swift
        └── Shaders/shaders.metal

Structure observations

• The model layer separates UI-facing state (CameraManager) from AVFoundation session ownership (CameraController).
• Metal view wrappers and renderer coordinators are grouped by implementation technology.
• All visualizations share CameraCapturedData by reference and a singleton Metal device/queue/library.

Overall architecture

flowchart LR
    App["LiDARDepthApp"] --> View["ContentView"]
    View --> Manager["CameraManager"]
    Manager --> Controller["CameraController"]
    Controller --> Session["AVCaptureSession"]
    Session --> Sync["Video + depth synchronizer"]
    Session --> Photo["Photo output"]
    Sync -->|CaptureDataReceiver| Manager
    Photo -->|CaptureDataReceiver| Manager
    Manager --> Data["CameraCapturedData"]
    Data --> MetalViews["MetalRepresentable views"]
    MetalViews --> Coordinators["MTKCoordinator subclasses"]
    Coordinators --> Environment["MetalEnvironment.shared"]

Reference code

LiDARDepth/ContentView.swift:12 and LiDARDepth/Model/CameraManager.swift:31 — UI-to-capture composition

struct ContentView: View {
    @StateObject private var manager = CameraManager()
}

init() {
    capturedData = CameraCapturedData()
    controller = CameraController()
    controller.startStream()
    controller.delegate = self
}

Interpretation

SwiftUI retains one observable manager. The manager adapts delegate deliveries to published UI state and mutates a stable captured-data reference consumed by renderers. The controller remains the sole capture-graph implementation, while a protocol/generic coordinator layer shares Metal view setup.

Ownership and state

classDiagram
    ContentView *-- CameraManager : StateObject creates
    CameraManager *-- CameraController : creates
    CameraManager *-- CameraCapturedData : creates and mutates
    CameraController *-- AVCaptureSession : creates
    CameraController *-- AVCaptureVideoDataOutput : creates
    CameraController *-- AVCaptureDepthDataOutput : creates
    CameraController *-- AVCapturePhotoOutput : creates
    CameraController o-- CaptureDataReceiver : weak delegate
    MetalEnvironment *-- MTLDevice : singleton owns
    MetalEnvironment *-- MTLCommandQueue : singleton owns

Ownership evidence

LiDARDepth/Model/CameraManager.swift:27 — manager composition

let controller: CameraController

init() {
    capturedData = CameraCapturedData()
    controller = CameraController()
    controller.isFilteringEnabled = true
    controller.startStream()
    controller.delegate = self
}

Class and protocol design

classDiagram
    class CaptureDataReceiver {
        <<protocol>>
        onNewData()
        onNewPhotoData()
    }
    class MetalRepresentable {
        <<protocol>>
        rotationAngle
    }
    CameraManager ..|> CaptureDataReceiver
    CameraController --> CaptureDataReceiver : weak delegate
    MetalPointCloudView ..|> MetalRepresentable
    MetalTextureView ..|> MetalRepresentable
    MTKCoordinator --> MetalRepresentable : generic parent
    MTKPointCloudCoordinator --|> MTKCoordinator

Reference code

LiDARDepth/Model/CameraController.swift:12 and LiDARDepth/Views/Metal/MetalContentView.swift:18 — capture and rendering capabilities

protocol CaptureDataReceiver: AnyObject {
    func onNewData(capturedData: CameraCapturedData)
    func onNewPhotoData(capturedData: CameraCapturedData)
}

protocol MetalRepresentable: UIViewRepresentable {
    var rotationAngle: Double { get set }
}

Access control

Reference code

LiDARDepth/Model/CameraController.swift:24 — protected capture graph

private let videoQueue = DispatchQueue(
    label: "com.example.apple-samplecode.VideoQueue",
    qos: .userInteractive
)
private(set) var captureSession: AVCaptureSession!
private var photoOutput: AVCapturePhotoOutput!
private var depthDataOutput: AVCaptureDepthDataOutput!
weak var delegate: CaptureDataReceiver?

No public, open, or fileprivate declaration appears in the Swift source.

Logic ownership and placement

Design patterns

Main application flow

sequenceDiagram
    participant Session as Capture session
    participant Sync as Data synchronizer
    participant Controller as CameraController
    participant Manager as CameraManager
    participant View as Metal views
    Session->>Sync: video + depth outputs
    Sync->>Controller: synchronized collection
    Controller->>Controller: create textures and CameraCapturedData
    Controller->>Manager: onNewData(capturedData)
    Manager->>Manager: update stable capturedData reference
    Manager-->>View: published dataAvailable/state
    View->>View: coordinators render textures

Reference code

LiDARDepth/Model/CameraController.swift:159 — synchronized delivery

func dataOutputSynchronizer(_ synchronizer: AVCaptureDataOutputSynchronizer,
                            didOutput collection: AVCaptureSynchronizedDataCollection) {
    let syncedDepthData = collection.synchronizedData(for: depthDataOutput)
    let syncedVideoData = collection.synchronizedData(for: videoDataOutput)
    // Package both streams as CameraCapturedData.
    delegate?.onNewData(capturedData: data)
}

Naming conventions

• Types: layer roles are explicit (CameraManager, CameraController, CameraCapturedData, MetalEnvironment).
• Protocols: receiver/capability names (CaptureDataReceiver, MetalRepresentable).
• Methods: event callbacks use onNew...; commands use start, resume, capture, and setup verbs.
• Views/coordinators: paired Metal...View and MTK...Coordinator names identify SwiftUI wrapper versus renderer delegate.
• Files: Model/ versus Views/Metal/ organization mirrors capture/state and rendering responsibilities.

Architecture takeaways

• Separate observable UI coordination from the object that owns AVFoundation hardware/session details.
• A weak receiver protocol keeps capture delivery reusable without making the controller own the UI layer.
• Store a stable reference object when many renderer views consume textures that update frame by frame.
• Protocol default implementations and a generic coordinator remove repeated SwiftUI/MetalKit bridge setup.
• private(set) and a private singleton initializer clearly encode who may construct or replace critical resources.

Source map

• Official Apple documentation