Creating an App for Face-Painting in AR
At a glance
| Item | Summary |
|---|---|
| Purpose | Combine RealityKit’s face detection with PencilKit to implement virtual face-painting. |
| App architecture | FacePainting presents SwiftUI ContentView; FacePaintingView adapts an imperative AR face-tracking view and PencilKit canvas, then maps the drawing into the face entity’s material. |
| Main patterns | SwiftUI scene composition, UI–RealityKit bridge, Delegate callback adapter, ARView–PencilKit adapter |
| Project style | Code-rich sample with 2 scanned Swift file(s) and 351 implementation line(s). |
Project structure
Source bundle/
└── FacePainting/
├── ContentView.swift # ContentView, ARViewContainer, DrawingViewContainer
└── AppDelegate.swift # FacePainting
Structure observations
- The runtime boundary is WindowGroup + SwiftUI/UIKit host; the tree is pruned to lifecycle, state, framework, rendering, and ECS files.
- The source is Swift; role-named types and feature folders separate the major responsibilities.
Overall architecture
flowchart LR
FacePainting_1["FacePainting"]
WindowGroup___SwiftUI_UIKit_host_2["WindowGroup + SwiftUI/UIKit host"]
FacePaintingView_3["FacePaintingView"]
RealityKit_entity_graph_4["RealityKit entity graph"]
AR_face_tracking___PencilKit_material_5["AR face tracking + PencilKit material"]
FacePainting_1 --> WindowGroup___SwiftUI_UIKit_host_2
WindowGroup___SwiftUI_UIKit_host_2 --> FacePaintingView_3
FacePaintingView_3 --> RealityKit_entity_graph_4
RealityKit_entity_graph_4 --> AR_face_tracking___PencilKit_material_5
Reference code
FacePainting/AppDelegate.swift:12 — executable or app-lifecycle anchor.
struct FacePainting: App {
var body: some Scene {
WindowGroup {
ContentView()
}
}
}The diagram is a responsibility flow, not a claim that every adjacent node directly calls the next. It separates lifecycle, presentation, stable state, framework/session work, and the RealityKit entity graph.
Ownership and state
classDiagram
FacePaintingView --> HasModel : faceEntity
FacePaintingView *-- EntityQuery : sceneUnderstandingQuery
FacePaintingView --> UITouch : lastTouchPoint
FacePaintingView --> Cancellable : subscription
Ownership evidence
FacePainting/ContentView.swift:156 — representative stored state or nearest verified lifecycle anchor.
class FacePaintingView: ARView {
// ...
var faceEntity: HasModel? = nil
// ...
}| Owner | Object or state | Relationship | Mutation authority |
|---|---|---|---|
FacePaintingView |
HasModel as faceEntity |
stores and coordinates | The owning type coordinates writes |
FacePaintingView |
EntityQuery as sceneUnderstandingQuery |
creates and retains | The owning type coordinates writes |
FacePaintingView |
UITouch as lastTouchPoint |
stores and coordinates | The owning type coordinates writes |
FacePaintingView |
Cancellable as subscription |
stores and coordinates | The owning type coordinates writes |
Ownership is intentionally narrow: environment, bindings, observed objects, weak links, and delegate callbacks do not prove lifetime ownership. Initialized stored services and wrapper-managed state do; entities added inside a RealityKit content closure belong to that entity graph.
Class and protocol design
| Type | Responsibility | Depends on or conforms to |
|---|---|---|
FacePainting (FacePainting/AppDelegate.swift:12) |
Declares app lifecycle and top-level dependency lifetime. | App |
FacePaintingView (FacePainting/ContentView.swift:151) |
Presents UI and forwards gestures or lifecycle events. | ARView |
ContentView (FacePainting/ContentView.swift:15) |
Presents UI and forwards gestures or lifecycle events. | View |
The reviewed source defines no local substitution protocol. Its App, View, delegate, renderer, ARKit, and RealityKit conformances are framework-facing contracts, so this document does not label the whole app protocol-oriented.
Access control
| Symbol | Access | Verified effect | Likely rationale |
|---|---|---|---|
@State private var showPicker = true (FacePainting/ContentView.swift:16) |
private |
Use stays inside the declaration and same-file extensions permitted by Swift. | Inference: Hide lifecycle-sensitive state or an implementation step. |
public func transform(position p: SIMD3<Float>) -> SIMD3<Float> { (FacePainting/ContentView.swift:71) |
public |
The symbol is available to importing modules, subject to its containing type’s visibility. | Inference: Expose an intentional package or cross-target surface. |
No reviewed Swift access example uses fileprivate, private(set), open; declarations without a modifier are internal.
Reference code
FacePainting/ContentView.swift:16 — representative visibility boundary.
struct ContentView: View {
@State private var showPicker = true
// ...
}Logic ownership and placement
| Logic | Owning type or file | Placement rationale |
|---|---|---|
| Application/command lifecycle | FacePainting |
The executable boundary determines app, controller, scene, or command lifetime. |
| Presentation and user intent | FacePaintingView |
The view/controller forwards input and owns only presentation-local work. |
Design patterns
| Pattern | Source evidence | Purpose or tradeoff |
|---|---|---|
| SwiftUI scene composition | FacePainting/AppDelegate.swift:12 |
Keeps windows, volumes, and immersive presentation visible at the app boundary. |
| UI–RealityKit bridge | FacePainting/ContentView.swift:151 |
Builds or hosts the RealityKit entity graph from SwiftUI or a platform view/controller lifecycle. |
| Delegate callback adapter | FacePainting/ContentView.swift:321 |
Inverts imperative framework callbacks into the sample’s owning controller, coordinator, or model. |
| ARView–PencilKit adapter | FacePainting/ContentView.swift:18 |
Bridges SwiftUI controls, PencilKit drawing, and AR face material updates across imperative framework boundaries. |
Naming conventions
- Role suffixes are evidence, not decoration: View:
ContentView,FacePaintingView. - Protocols: no app-defined protocol in the reviewed source.
- Commands use verb-led methods:
makeUIView,updateUIView,transform,setup,updateFaceTextureWithLatestDrawing,updateFaceEntityTextureUsing,setDrawingUndoable,findFaceEntity. - Files and folders use primary-type or responsibility names such as
Views,Models,Rendering,Components,Systems,Packages, or feature names where those boundaries exist.
Architecture takeaways
- Treat
FacePaintingas the owner of executable or scene lifecycle, not automatically as the owner of every entity created later. - Keep view/controller input near presentation, but move sessions, reconstruction, capture, rendering, shared game state, or documents into stable owners when their lifetime exceeds one callback or render pass.
Source map
| Source file | Relevant symbols |
|---|---|
FacePainting/ContentView.swift:15 |
ContentView, ARViewContainer, DrawingViewContainer, Ray, RaycastResult, FacePaintingView |
FacePainting/AppDelegate.swift:12 |
FacePainting |