Streaming an AR experience
At a glance
| Item | Summary |
|---|---|
| Purpose | Control an AR experience remotely by transferring sensor and user input over the network. |
| App architecture | A C/Objective-C header, Metal, Swift sample with the source-visible chain AppDelegate → OverlayViewController → Renderer → ARKit APIs. |
| Main patterns | View-controller organization, Protocol-oriented abstraction, Delegate or data-source callbacks |
| Project style | 19 scanned source file(s) across C/Objective-C header, Metal, Swift, organized around ranked entry, type, and file boundaries. |
| Execution model | Source-visible boundaries: DispatchQueue.main.async, DispatchQueue(label:); none alone proves a background thread. |
| State/event model | No structured observation or publisher-scheduling marker indexed. |
| Key frameworks/packages | ARKit, Metal, MetalKit, MultipeerConnectivity, simd; these are source dependencies, not architecture labels. |
Project structure
Source bundle/
└── StreamingARFramesLocally/
├── AppDelegate.swift
├── Renderer/
│ ├── Renderer.swift
│ ├── Utilities/
│ │ ├── BufferView.swift
│ │ ├── MetalUtils.swift
│ │ └── RenderDestination.swift
│ ├── DepthStencilStates.swift
│ └── PipelineStates.swift
├── MultipeerSession.swift
├── OverlayViewController.swift
├── VideoProcessor.swift
├── ViewController.swift
└── VideoFrameData.swift
Structure observations
- Architecturally prominent files are ranked from entry points and role-named declarations; resource-only paths are omitted.
- Primary languages: C/Objective-C header, Metal, Swift.
- The verified tree contains 6 project/configuration file(s) and 19 source declaration(s).
Overall architecture
flowchart LR
N1["AppDelegate"]
N2["OverlayViewController"]
N3["Renderer"]
N4["ARKit APIs"]
N1 --> N2
N2 --> N3
N3 --> N4
Reference code
StreamingARFramesLocally/AppDelegate.swift:10 — architecture anchor
@main
class AppDelegate: UIResponder, UIApplicationDelegate {
// ...
var window: UIWindow?
// ...
}Interpretation
The arrows summarize the source-visible entry, role-named types or folders, and framework direction; when nodes come from structural folders, the sequence is a high-level interpretation rather than proof that every adjacent node calls the next. Ownership is claimed only where the next section cites a stored property or assignment. The diagram is intentionally limited to the dominant path into ARKit.
Ownership and state
classDiagram
AppDelegate o-- UIWindow : window
AppDelegate o-- UIWindow : overlayWindow
Renderer o-- MTLCommandQueue : commandQueue
Renderer o-- PipelineStates : pipelineStates
Ownership evidence
StreamingARFramesLocally/AppDelegate.swift:13 — stored dependency or nearest verified ownership anchor
@main
class AppDelegate: UIResponder, UIApplicationDelegate {
// ...
var window: UIWindow?
// ...
}| Owner | Object or state | Relationship | Mutation authority |
|---|---|---|---|
AppDelegate |
UIWindow (window) |
stores or receives | App/module collaborators |
AppDelegate |
UIWindow (overlayWindow) |
stores or receives | App/module collaborators |
Renderer |
MTLCommandQueue (commandQueue) |
stores or receives | Initialized by the owner; the binding is immutable |
Renderer |
PipelineStates (pipelineStates) |
stores or receives | Initialized by the owner; the binding is immutable |
Composition arrows indicate a source-visible construction expression or locally owned value state; aggregation means the owner stores or receives a dependency without proving exclusive lifetime ownership.
Concurrency, scheduling, and thread safety
Evidence limit: actor isolation, async/await, or Task creation does not by itself prove background-thread execution; Sendable conformance alone does not prove thread-safe mutation.
| Concern | Source mechanism | Verified placement or handoff | Evidence |
|---|---|---|---|
| Queue scheduling | DispatchQueue.main.async |
The source addresses the main dispatch queue. | StreamingARFramesLocally/OverlayViewController.swift:98 |
| Queue scheduling | DispatchQueue(label:) |
The source constructs a dispatch queue; its label alone does not prove a thread. | StreamingARFramesLocally/Renderer/Renderer.swift:24 |
@MainActor/MainActor.run, DispatchQueue.main, and RunLoop.main are reported as distinct isolation, queue, and event-loop mechanisms. A plain Task is kept separate from Task.detached; neither is labeled as a background thread.
Reference code
StreamingARFramesLocally/OverlayViewController.swift:98 — representative execution boundary
DispatchQueue.main.async { [self] in
pipViewWidthConstraint?.constant = width / pipView.contentScaleFactor
pipViewHeightConstraint?.constant = height / pipView.contentScaleFactor
}State propagation, frameworks, and dependencies
Evidence limit: an import proves a source-level compilation dependency at the cited line; it does not prove runtime use, architectural adoption, or whether a Swift package is a direct application dependency.
| Category | Mechanism or module | Verified role | Evidence |
|---|---|---|---|
| Source import | ARKit |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/AppDelegate.swift:8 |
| Source import | Metal |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/Renderer/DepthStencilStates.swift:8 |
| Source import | MetalKit |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/OverlayViewController.swift:9 |
| Source import | MultipeerConnectivity |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/MultipeerSession.swift:8 |
| Source import | simd |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/Renderer/ShaderTypes.h:13 |
| Source import | UIKit |
The cited file imports this module; runtime use and architectural role are not inferred. | StreamingARFramesLocally/AppDelegate.swift:7 |
receive(on:) describes downstream delivery scheduling, while subscribe(on:) describes upstream subscription/request/cancel scheduling. An import Combine alone establishes neither behavior nor a Store, reducer, Redux, or other application architecture.
Class and protocol design
StreamingARFramesLocally/Renderer/Utilities/RenderDestination.swift:13 — representative type boundary
protocol RenderDestination {
var colorPixelFormat: MTLPixelFormat { get set }
var depthStencilPixelFormat: MTLPixelFormat { get set }
}| Type | Responsibility | Depends on or conforms to |
|---|---|---|
AppDelegate |
Receives callback-driven events | UIResponder, UIApplicationDelegate |
Renderer |
Owns drawing, GPU, or presentation processing | NSObject |
MultipeerSession |
Owns a session-scoped interaction | NSObject |
OverlayViewController |
View lifecycle, callbacks, and feature coordination | UIViewController |
BufferView |
User-interface presentation and input forwarding | Concrete collaborators/imported frameworks |
VideoProcessor |
Owns a feature processing stage | Concrete collaborators/imported frameworks |
ViewController |
View lifecycle, callbacks, and feature coordination | UIViewController, ARSessionDelegate |
RenderDestination |
Defines a capability or collaboration contract | Concrete collaborators/imported frameworks |
Constants |
Defines a closed set of feature states or choices | Concrete collaborators/imported frameworks |
Frame |
Represents a feature value or composable behavior | Concrete collaborators/imported frameworks |
The source explicitly defines local protocol relationships: MTKView → RenderDestination.
Access control
| Symbol | Access | Verified effect | Likely rationale |
|---|---|---|---|
myPeerID (StreamingARFramesLocally/MultipeerSession.swift:13) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
session (StreamingARFramesLocally/MultipeerSession.swift:14) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
serviceAdvertiser (StreamingARFramesLocally/MultipeerSession.swift:15) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
serviceBrowser (StreamingARFramesLocally/MultipeerSession.swift:16) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
Reference code
StreamingARFramesLocally/MultipeerSession.swift:13 — representative boundary
class MultipeerSession: NSObject {
// ...
private let myPeerID = MCPeerID(displayName: UIDevice.current.name)
// ...
}Swift declarations without a modifier are internal; explicit private, fileprivate, private(set), public, or open entries above are interpreted by language semantics. Objective-C/C samples instead rely on header and implementation boundaries, which are not equivalent to Swift lexical privacy.
Logic ownership and placement
| Logic | Owning type or file | Placement rationale |
|---|---|---|
| View lifecycle, callbacks, and feature coordination | OverlayViewController, ViewController |
The source’s Controller suffix makes this role explicit. |
| Receives callback-driven events | AppDelegate |
The source’s Delegate suffix makes this role explicit. |
| Owns a feature processing stage | VideoProcessor |
The source’s Processor suffix makes this role explicit. |
| Owns drawing, GPU, or presentation processing | Renderer |
The source’s Renderer suffix makes this role explicit. |
| Owns a session-scoped interaction | MultipeerSession |
The source’s Session suffix makes this role explicit. |
| User-interface presentation and input forwarding | BufferView |
The source’s View suffix makes this role explicit. |
Design patterns
| Pattern | Source evidence | Purpose or tradeoff |
|---|---|---|
| View-controller organization | StreamingARFramesLocally/OverlayViewController.swift:11 |
A controller is the verified coordination boundary; this is MVC-style only where a separate model is present. |
| Protocol-oriented abstraction | StreamingARFramesLocally/Renderer/Utilities/RenderDestination.swift:18 |
A local protocol and concrete conformance create an explicit capability boundary. |
| Delegate or data-source callbacks | StreamingARFramesLocally/AppDelegate.swift:11 |
Callback protocols invert event delivery back into the sample’s owner. |
Naming conventions
- Types: Controller: OverlayViewController, ViewController; Delegate: AppDelegate; Processor: VideoProcessor; Renderer: Renderer; Session: MultipeerSession; View: BufferView.
- Protocols:
RenderDestination. - Methods:
application,enqueueFrame,createTexture,mtkView,draw,sendToAllPeers,sendToPeers,session. - Files:
StreamingARFramesLocally/AppDelegate.swift,StreamingARFramesLocally/Renderer/Renderer.swift,StreamingARFramesLocally/MultipeerSession.swift,StreamingARFramesLocally/OverlayViewController.swift,StreamingARFramesLocally/Renderer/Utilities/BufferView.swift,StreamingARFramesLocally/VideoProcessor.swift.
Architecture takeaways
AppDelegateis the main source-visible entry or composition anchor for this sample.- Framework work reaches ARKit, Metal, MetalKit, MultipeerConnectivity through a deliberately small high-level chain; the detailed API graph remains inside the cited implementation files.
- Stored-property evidence identifies lifecycle collaboration; it does not by itself prove exclusive object ownership.
- Access-control conclusions separate verified language visibility from the likely design rationale.
- Local protocol relationships provide an explicit substitution boundary.
Source map
| Source file | Relevant symbols |
|---|---|
StreamingARFramesLocally/AppDelegate.swift |
Cited implementation, ARKit, UIKit, AppDelegate |
StreamingARFramesLocally/Renderer/Utilities/RenderDestination.swift |
RenderDestination, Cited implementation |
StreamingARFramesLocally/MultipeerSession.swift |
Cited implementation, MultipeerConnectivity, MultipeerSession |
StreamingARFramesLocally/OverlayViewController.swift |
OverlayViewController, DispatchQueue.main.async, MetalKit |
StreamingARFramesLocally/Renderer/Renderer.swift |
DispatchQueue(label:), Renderer, Constants, Frame |
StreamingARFramesLocally/Renderer/DepthStencilStates.swift |
Metal, DepthStencilStates |
StreamingARFramesLocally/Renderer/ShaderTypes.h |
simd, Feature implementation |
StreamingARFramesLocally/Renderer/Utilities/BufferView.swift |
BufferView |
StreamingARFramesLocally/VideoProcessor.swift |
VideoProcessor |
StreamingARFramesLocally/ViewController.swift |
ViewController |
StreamingARFramesLocally/VideoFrameData.swift |
VideoFrameData, TimeStamp, TimingInfo |
StreamingARFramesLocally/Renderer/PipelineStates.swift |
PipelineStates |
StreamingARFramesLocally/Renderer/Utilities/MetalUtils.swift |
MetalUtils |
StreamingARFramesLocally/Renderer/VertexDescriptors.swift |
VertexDescriptors |
StreamingARFramesLocally/Utilities/Matrix.swift |
Matrix |
StreamingARFramesLocally/Utilities/Ray.swift |
Ray |