Debugging AVFoundation audio mixes, compositions, and video compositions

At a glance

Project structure

AVCompositionDebugViewer/
├── PlayerViewController.swift       # Screen composition and playback
├── Editor/
│   ├── SimpleEditor.swift           # Composition/mix builder
│   └── SimpleEditorUtils.swift      # Asset and timestamp helpers
├── DebugView/
│   ├── CompDebugView.swift          # Timeline diagnostic view
│   ├── CompDebugViewUtils.swift     # AV object-to-drawing transforms
│   └── CompDebugViewCAUtils.swift   # Layer drawing helpers
└── Media/                           # Two bundled clips

Structure observations

• Editing and visualization are separate subsystems connected through one AVPlayerItem.
• Helpers are colocated by consumer (Editor versus DebugView) rather than placed in a generic utilities folder.
• The storyboard owns the controller/view hierarchy; code owns media objects.

Overall architecture

flowchart LR
    Storyboard["Storyboard"] --> Controller["PlayerViewController"]
    Controller --> Editor["SimpleEditor"]
    Editor --> Composition["AVMutableComposition + video composition + audio mix"]
    Controller --> Item["AVPlayerItem"]
    Composition --> Item
    Item --> Player["AVPlayer / AVPlayerViewController"]
    Item --> Debug["CompositionDebugView"]
    Debug --> Helpers["DebugView transformation helpers"]

Reference code

AVCompositionDebugViewer/PlayerViewController.swift:48 — composition assembled for two consumers

self.editor = SimpleEditor(completion: {
    self.playerItem = AVPlayerItem(asset: self.editor.composition())
    self.playerItem.videoComposition = self.editor.videoComposition()
    self.playerItem.audioMix = self.editor.audioMix()
    self.compositionDebugView.synchronize(with: self.playerItem)
})

Interpretation

PlayerViewController is the composition root and MVC controller. SimpleEditor creates the media model. The same configured player item drives playback and diagnostic rendering, so the visualizer examines the exact objects the player uses.

Ownership and state

classDiagram
    PlayerViewController *-- SimpleEditor : creates
    PlayerViewController *-- AVPlayer : creates
    PlayerViewController *-- AVPlayerViewController : creates
    PlayerViewController *-- AVPlayerItem : creates after editing
    SimpleEditor *-- AVMutableComposition : creates
    SimpleEditor *-- AVMutableVideoComposition : creates
    SimpleEditor *-- AVMutableAudioMix : creates
    CompositionDebugView o-- AVPlayerItem : receives and retains

Ownership evidence

AVCompositionDebugViewer/PlayerViewController.swift:18 and AVCompositionDebugViewer/Editor/SimpleEditor.swift:38 — controller and editor resources

private var playerViewController = AVPlayerViewController()
private var player = AVPlayer()
private var playerItem: AVPlayerItem!
private var editor: SimpleEditor!

private lazy var editorComposition: AVMutableComposition = { /* ... */ }()
private var editorAudioMix = AVMutableAudioMix()

Class and protocol design

AVCompositionDebugViewer/Editor/SimpleEditor.swift:28 — validation callback boundary

class SimpleEditor: NSObject, AVVideoCompositionValidationHandling {
    // ... builder methods ...
    func videoComposition(
        _ videoComposition: AVVideoComposition,
        shouldContinueValidatingAfterFindingEmptyTimeRange timeRange: CMTimeRange
    ) -> Bool { false }
}

Access control

Reference code

AVCompositionDebugViewer/DebugView/CompDebugView.swift:62 — public operation, private render state

class CompositionDebugView: UIView {
    private var compositionTrackSegmentInfo = [[CompositionTrackSegmentInfo]]()
    private var volumeRampAsPoints = [[CGPoint]]()
    private var videoCompositionStages = [VideoCompositionStageInfo]()

    func synchronize(with playerItem: AVPlayerItem) { /* derive snapshots */ }
}

Logic ownership and placement

Design patterns

Naming conventions

• SimpleEditor signals sample-scale scope; its accessors use AV object names (composition, videoComposition, audioMix).
• Debug types use Composition, TrackSegment, and Stage vocabulary from the visualized domain.
• Builder methods use create...; transformation helpers name the value they return (volumeRampPoints, videoCompStageInfo).
• Files are grouped and named by subsystem responsibility.

Architecture takeaways

• Drive playback and diagnostics from the same configured AVPlayerItem to avoid mismatched state.
• Keep mutable composition construction behind a small finished-object interface.
• Convert framework graphs into simple drawing snapshots before entering rendering code.
• Framework delegate protocols provide useful diagnostic seams even without app-defined protocols.

Source map

• Official Apple documentation