Optimizing 3D Models for ARKit (Update 2024)

Here is a quick guide on optimizing 3D models for ARKit, Apple's augmented reality platform for iOS devices. ARKit facilitates immersive AR experiences by integrating motion tracking, scene capture, and advanced processing. This guide outlines key considerations and steps for ensuring optimal performance of 3D assets, including polygon count, file size, texture resolution, and draw calls, using RapidPipeline to streamline the optimization process.

What is ARKit?

ARKit is Apple's augmented reality framework that enables developers to create AR experiences for iOS devices.

Features:

• ARKit combines device motion tracking, camera scene capture, and advanced scene processing to enable high-quality AR experiences. 
• It provides APIs for detecting real-world surfaces, tracking objects, and placing virtual content in the real world. 
• ARKit supports integration with SceneKit, Metal, and other frameworks for rendering AR content. 
• It offers features like image tracking, face tracking, and collaborative sessions for multiplayer AR experiences.

How does ARKit differ from SceneKit and RealityKit?

• ARKit is specifically focused on AR experiences, providing tools for real-world interaction and scene understanding. It's used for building AR apps that overlay virtual content onto the real world. 
• SceneKit is best suited for creating complex 3D scenes and animations, whether for AR or other purposes, and it's often used in conjunction with ARKit for rendering AR content. 
• RealityKit is ideal for developers who want to create AR experiences quickly and easily, with less emphasis on low-level details. It's especially useful for those new to AR development or for rapidly prototyping AR concepts.

Which file format is supported by ARKit?

The file format that can be viewed natively in AR on iOS devices without any additional processing or conversion is USDZ.

USDZ (Universal Scene Description) is a file format developed by Pixar and adopted by Apple as the standard for AR content on iOS. It's optimized for mobile devices and is designed to store 3D models, animations, textures, and other assets necessary for rendering objects in augmented reality.

With iOS, users can view USDZ files directly within the apps that support the Quick Look feature. Quick Look allows users to preview 3D objects in real-world scale and interact with them using gestures.

You can find the full specification of USDZ files in this link.

What are suggested 3D requirements for AR applications?

Apple has not published hard limitations for 3D files in AR applications. However, there are some general guidelines that are necessary to adhere to. While there can be exceptions considering the number of objects in the whole scene, the subjective fidelity expectations, the below suggestions can be considered safe for a smooth experience with ARKit:

RapidPipeline can optimize your assets by automatically simplifying your meshes and generating texture with just a few clicks.

How to convert a 3D model for ARKit

Upload your 3D Asset to RapidPipeline

After uploading, we inspect the model and notice that the scale is too big. If we optimize this as-is, we will see a building size cake. In most cases, you will not want to see a 50m tall cake. RapidPipeline allows us to fix this scale before applying optimization:

Choose or create the preset

We want to make sure that we reduce draw calls, simplify the mesh and compress the textures so that the model would have optimal performance with ARKit renderer.

In order to achieve this we tweak the following settings:

• Set Faces to  50K 
• Bake all textures into 1 atlas 
• Choose JPEG textures 
• Choose 2K textures 
• Export as .usdz file for using in ARKit 

RapidPipeline provides a preset that already has all these settings: Single-Item Mobile. You could further refine the settings if you need a higher polygon count for your own model. Since the sample we use here is already low fidelity, we choose not to edit it. Select this preset, and optimize your model.

Download your optimized model & Open the file in AR

Once the optimization is completed, you can download the .usdz file.