Enhancing your app with machine learning-based video effects
At a glance
| Item | Summary |
|---|---|
| Purpose | Add powerful effects to your videos using the VideoToolbox VTFrameProcessor API. |
| App architecture | A Swift sample bundle with entry-bearing project variants VideoProcessor, VideoProcessorCommand, each leading to VideoToolbox APIs. |
| Main patterns | Protocol-oriented abstraction, Actor-isolated state |
| Project style | 19 scanned source file(s) across Swift, organized around ranked entry, type, and file boundaries. |
Project structure
Source bundle/
├── VideoProcessorCommand/
│ └── VideoProcessorCommand.swift
└── VideoProcessor/
├── VideoProcessorApp.swift
├── Views/
│ ├── VideoEffectSettingsView.swift
│ ├── VideoPreviewView.swift
│ ├── VideoEffectSelectionView.swift
│ └── AVSampleBufferView.swift
├── AsyncIO/
│ ├── AsyncAssetIO.swift
│ └── AsyncIOStream.swift
├── VideoProcessorModel.swift
├── VideoProcessorView.swift
└── Processing/
├── AsyncFrameProcessor.swift
└── VideoProcessor.swift
Structure observations
- Architecturally prominent files are ranked from entry points and role-named declarations; resource-only paths are omitted.
- Primary languages: Swift.
- The verified tree contains 4 project/configuration file(s) and 57 source declaration(s).
Overall architecture
flowchart LR
Bundle["Sample bundle"]
V1["VideoProcessor"]
V2["VideoProcessorCommand"]
Boundary["VideoToolbox APIs"]
Bundle --> V1
V1 --> Boundary
Bundle --> V2
V2 --> Boundary
Reference code
VideoProcessorCommand/VideoProcessorCommand.swift:16 — architecture anchor
@main
struct ProcessVideoEffect: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "processVideoEffect",
abstract: "Apply video effect to the input file, writing results to the output file.",
subcommands: [MotionBlur.self, FrateRateConversion.self])
}Interpretation
The branches represent separate entry-bearing project variants in the downloaded bundle, not runtime calls between those variants. Each branch is intentionally collapsed at the documented framework boundary; the detailed target-local flow remains in the cited files. Ownership is claimed only where the next section cites a stored property or assignment.
Ownership and state
classDiagram
SampleBuffer o-- Cmsamplebuffer : cmSampleBuffer
AsyncAssetReader o-- used : inputStream
AsyncAssetReader *-- URL : inputURL
AsyncAssetReader *-- Dictionary : videoSettings
Ownership evidence
VideoProcessor/AsyncIO/AsyncAssetIO.swift:20 — stored dependency or nearest verified ownership anchor
struct SampleBuffer: Sendable {
// ...
self.cmSampleBuffer = cmSampleBuffer
// ...
}| Owner | Object or state | Relationship | Mutation authority |
|---|---|---|---|
SampleBuffer |
Cmsamplebuffer (cmSampleBuffer) |
stores or receives | App/module collaborators |
AsyncAssetReader |
used (inputStream) |
stores or receives | App/module collaborators |
AsyncAssetReader |
URL (inputURL) |
owns value state | Initialized by the owner; the binding is immutable |
AsyncAssetReader |
Dictionary (videoSettings) |
owns value state | App/module collaborators |
Composition arrows indicate a source-visible construction expression or locally owned value state; aggregation means the owner stores or receives a dependency without proving exclusive lifetime ownership.
Class and protocol design
VideoProcessor/Processing/AsyncFrameProcessor.swift:16 — representative type boundary
protocol AsyncFrameProcessor: Actor {
// ...
}| Type | Responsibility | Depends on or conforms to |
|---|---|---|
VideoProcessorApp |
Application entry and top-level composition | App |
AsyncFrameProcessor |
Defines a capability or collaboration contract | Actor |
VideoEffectSettingsView |
User-interface presentation and input forwarding | View |
FrameRateConversionView |
User-interface presentation and input forwarding | View |
MotionBlurView |
User-interface presentation and input forwarding | View |
SuperResolutionScalerView |
User-interface presentation and input forwarding | View |
TemporalNoiseFilterView |
User-interface presentation and input forwarding | View |
LowLatencySuperResolutionScalerView |
User-interface presentation and input forwarding | View |
LowLatencyFrameInterpolationView |
User-interface presentation and input forwarding | View |
VideoPreviewView |
User-interface presentation and input forwarding | View |
The source explicitly defines local protocol relationships: AsyncAssetReader → AsyncFrameProcessor, AsyncAssetWriter → AsyncFrameProcessor, FrameRateConversionProcessor → AsyncFrameProcessor, LowLatencyFrameInterpolation → AsyncFrameProcessor, LowLatencySuperResolutionScaler → AsyncFrameProcessor, MotionBlurProcessor → AsyncFrameProcessor.
Access control
| Symbol | Access | Verified effect | Likely rationale |
|---|---|---|---|
avAsset (VideoProcessor/AsyncIO/AsyncAssetIO.swift:53) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
progressStream (VideoProcessor/AsyncIO/AsyncAssetIO.swift:67) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
assetWriterTimeout (VideoProcessor/AsyncIO/AsyncAssetIO.swift:240) |
private |
Use is restricted to the lexical declaration and same-file extensions allowed by Swift. | Inference: keep state mutation or dependency lifetime inside the owning implementation. |
AsyncIOStream (VideoProcessor/AsyncIO/AsyncIOStream.swift:34) |
public |
The symbol is visible to importing modules. | Inference: make the declaration available across a module or target boundary. |
Reference code
VideoProcessor/AsyncIO/AsyncAssetIO.swift:53 — representative boundary
private let avAsset: AVURLAsset
func configure(sourcePixelBufferAttributes: [String: Any]) throws {
self.videoSettings = sourcePixelBufferAttributes
}Swift declarations without a modifier are internal; explicit private, fileprivate, private(set), public, or open entries above are interpreted by language semantics. Objective-C/C samples instead rely on header and implementation boundaries, which are not equivalent to Swift lexical privacy.
Logic ownership and placement
| Logic | Owning type or file | Placement rationale |
|---|---|---|
| Application entry and top-level composition | VideoProcessorApp |
The source’s App suffix makes this role explicit. |
| Owns document data or lifecycle behavior | MovieDocument |
The source’s Document suffix makes this role explicit. |
| Feature data or observable state | VideoProcessorModel |
The source’s Model suffix makes this role explicit. |
| Owns a feature processing stage | AsyncFrameProcessor, FrameRateConversionProcessor, MotionBlurProcessor, VideoProcessor |
The source’s Processor suffix makes this role explicit. |
| User-interface presentation and input forwarding | AVSampleBufferUIView, AVSampleBufferView, ChooseVideoEffectView, DetailView |
The source’s View suffix makes this role explicit. |
Design patterns
| Pattern | Source evidence | Purpose or tradeoff |
|---|---|---|
| Protocol-oriented abstraction | VideoProcessor/AsyncIO/AsyncAssetIO.swift:39 |
A local protocol and concrete conformance create an explicit capability boundary. |
| Actor-isolated state | VideoProcessor/AsyncIO/AsyncAssetIO.swift:39 |
Actor annotations make the concurrency ownership boundary explicit. |
Naming conventions
- Types: App: VideoProcessorApp; Document: MovieDocument; Model: VideoProcessorModel; Processor: AsyncFrameProcessor, FrameRateConversionProcessor, MotionBlurProcessor, VideoProcessor; View: AVSampleBufferUIView, AVSampleBufferView, ChooseVideoEffectView, DetailView, FrameRateConversionView.
- Protocols:
AsyncFrameProcessor. - Methods:
validate,run,fileWrapper,propagateAttachments,configure,progressStream,videoTrackDimensions,setInputStream. - Files:
VideoProcessor/VideoProcessorApp.swift,VideoProcessor/Views/VideoEffectSettingsView.swift,VideoProcessor/Views/VideoPreviewView.swift,VideoProcessor/VideoProcessorModel.swift,VideoProcessor/VideoProcessorView.swift,VideoProcessor/Views/VideoEffectSelectionView.swift.
Architecture takeaways
VideoProcessorCommandis the main source-visible entry or composition anchor for this sample.- Framework work reaches VideoToolbox, SwiftUI, AVFoundation, AppKit through a deliberately small high-level chain; the detailed API graph remains inside the cited implementation files.
- Stored-property evidence identifies lifecycle collaboration; it does not by itself prove exclusive object ownership.
- Access-control conclusions separate verified language visibility from the likely design rationale.
- Local protocol relationships provide an explicit substitution boundary.
Source map
| Source file | Relevant symbols |
|---|---|
VideoProcessorCommand/VideoProcessorCommand.swift |
ProcessVideoEffect, MotionBlur, FrateRateConversion, TemporalNoiseFilter |
VideoProcessor/VideoProcessorApp.swift |
VideoProcessorApp |
VideoProcessor/Views/VideoEffectSettingsView.swift |
VideoEffectSettingsView, FrameRateConversionView, MotionBlurView, SuperResolutionScalerView, TemporalNoiseFilterView, LowLatencySuperResolutionScalerView, LowLatencyFrameInterpolationView |
VideoProcessor/Views/VideoPreviewView.swift |
VideoPreviewView, ChooseVideoEffectView, InputVideoView, OutputVideoView, ProcessingFailedView, ProcessingVideoView, SaveVideoButton, MovieDocument, Fault, OpenQuickTimeButton, OpenButton, CloseButton |
VideoProcessor/AsyncIO/AsyncAssetIO.swift |
SampleBuffer, AsyncAssetReader, Fault, AsyncAssetWriter, Fault, Fault |
VideoProcessor/VideoProcessorModel.swift |
VideoEffect, VideoProcessorModel, State |
VideoProcessor/VideoProcessorView.swift |
VideoProcessorView, SidebarView, DetailView |
VideoProcessor/Views/VideoEffectSelectionView.swift |
VideoEffectSelectionView, VideoEffectButton, VideoEffectLabel |
VideoProcessor/AsyncIO/AsyncIOStream.swift |
AsyncIOStream, BufferingPolicy, OverflowPolicy, SendResult, Iterator |
VideoProcessor/Processing/AsyncFrameProcessor.swift |
AsyncFrameProcessor, AsyncFrameProcessorFault |