RealityKit Development Guide
Purpose: Build 3D content, AR experiences, and spatial computing apps using RealityKit's Entity-Component-System architecture
iOS Version: iOS 13+ (base), iOS 18+ (RealityView on iOS), visionOS 1.0+
Xcode: Xcode 15+
When to Use This Skill
Use this skill when:
- Building any 3D experience (AR, games, visualization, spatial computing)
- Creating SwiftUI apps with 3D content (RealityView, Model3D)
- Implementing AR with anchors (world, image, face, body tracking)
- Working with Entity-Component-System (ECS) architecture
- Setting up physics, collisions, or spatial interactions
- Building multiplayer or shared AR experiences
- Migrating from SceneKit to RealityKit
- Targeting visionOS
Do NOT use this skill for:
- SceneKit maintenance (use
axiom-scenekit)
- 2D games (use
axiom-spritekit)
- Metal shader programming (use
axiom-metal-migration-ref)
- Pure GPU compute (use Metal directly)
1. Mental Model: ECS vs Scene Graph
Scene Graph (SceneKit)
In SceneKit, nodes own their properties. A node IS a renderable, collidable, animated thing.
Entity-Component-System (RealityKit)
In RealityKit, entities are empty containers. Components add data. Systems process that data.
Entity (identity + hierarchy)
├── TransformComponent (position, rotation, scale)
├── ModelComponent (mesh + materials)
├── CollisionComponent (collision shapes)
├── PhysicsBodyComponent (mass, mode)
└── [YourCustomComponent] (game-specific data)
System (processes entities with specific components each frame)
Why ECS matters:
- Composition over inheritance: Combine any components on any entity
- Data-oriented: Systems process arrays of components efficiently
- Decoupled logic: Systems don't know about each other
- Testable: Components are pure data, Systems are pure logic
The ECS Mental Shift
| Scene Graph Thinking |
ECS Thinking |
| "The player node moves" |
"The movement system processes entities with MovementComponent" |
| "Add a method to the node subclass" |
"Add a component, create a system" |
"Override update(_:) in the node" |
"Register a System that queries for components" |
| "The node knows its health" |
"HealthComponent holds data, DamageSystem processes it" |
2. Entity Hierarchy
Creating Entities
let entity = Entity()
entity.name = "player"
let entity = Entity()
entity.components[ModelComponent.self] = ModelComponent(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: false)]
)
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .red, isMetallic: true)]
)
Hierarchy Management
parent.addChild(child)
child.removeFromParent()
let found = root.findEntity(named: "player")
for child in entity.children {
}
let clone = entity.clone(recursive: true)
Transform
entity.position = SIMD3<Float>(0, 1, 0)
entity.orientation = simd_quatf(angle: .pi / 4, axis: SIMD3(0, 1, 0))
entity.scale = SIMD3<Float>(repeating: 2.0)
let worldPos = entity.position(relativeTo: nil)
let worldTransform = entity.transform(relativeTo: nil)
entity.setPosition(SIMD3(1, 0, 0), relativeTo: nil)
entity.look(at: targetPosition, from: entity.position, relativeTo: nil)
3. Components
Built-in Components
| Component |
Purpose |
Transform |
Position, rotation, scale |
ModelComponent |
Mesh geometry + materials |
CollisionComponent |
Collision shapes for physics and interaction |
PhysicsBodyComponent |
Mass, physics mode (dynamic/static/kinematic) |
PhysicsMotionComponent |
Linear and angular velocity |
AnchoringComponent |
AR anchor attachment |
SynchronizationComponent |
Multiplayer sync |
PerspectiveCameraComponent |
Camera settings |
DirectionalLightComponent |
Directional light |
PointLightComponent |
Point light |
SpotLightComponent |
Spot light |
CharacterControllerComponent |
Character physics controller |
AudioMixGroupsComponent |
Audio mixing |
SpatialAudioComponent |
3D positional audio |
AmbientAudioComponent |
Non-positional audio |
ChannelAudioComponent |
Multi-channel audio |
OpacityComponent |
Entity transparency |
GroundingShadowComponent |
Contact shadow |
InputTargetComponent |
Gesture input (visionOS) |
HoverEffectComponent |
Hover highlight (visionOS) |
AccessibilityComponent |
VoiceOver support |
Custom Components
struct HealthComponent: Component {
var current: Int
var maximum: Int
var percentage: Float {
Float(current) / Float(maximum)
}
}
HealthComponent.registerComponent()
entity.components[HealthComponent.self] = HealthComponent(current: 100, maximum: 100)
if let health = entity.components[HealthComponent.self] {
print(health.current)
}
entity.components[HealthComponent.self]?.current -= 10
Component Lifecycle
Components are value types (structs). When you read a component, modify it, and write it back, you're replacing the entire component:
var health = entity.components[HealthComponent.self]!
health.current -= damage
entity.components[HealthComponent.self] = health
Anti-pattern: Holding a reference to a component and expecting mutations to propagate. Components are copied on read.
4. Systems
System Protocol
struct DamageSystem: System {
static let query = EntityQuery(where: .has(HealthComponent.self))
init(scene: RealityKit.Scene) {
}
func update(context: SceneUpdateContext) {
for entity in context.entities(matching: Self.query,
updatingSystemWhen: .rendering) {
var health = entity.components[HealthComponent.self]!
if health
