iOS 12: All you need to know about ARKit 2

ARKit 2 is Apples third major release of its Augmented Reality reality platform enabling developers to deliver enhanced AR apps alongside iOS 12. We look at the features of ARKit to give you an idea of whats coming in September.

So what does ARKit 2 bring to the party?

Persistent and multiuser experiences – AR experiences can now be shared across time and fixed to real-world locations, so you can create a piece of AR art that can be viewed later in the same place by someone else.

3D object recognition – ARKit 2 recognizes objects and how your device is oriented to them and can use that information to incorporate objects into AR experiences.

Face tracking – Face tracking now sees where youre looking and can detect whether youre sticking out your tongue or winking. OK, while fun for Animoji’s for all of 30 seconds, does this mean that depth sensing is coming to the rear cameras of our iPhone’s and iPad’s in the future?

Scene reflection – Enabling virtual objects to reflect the real-world scene in your camera.

USDZ file format – A new format that helps 3D creators to create content and animations optimized for mobile devices.

AR Quick Look – View AR experiences in apps, including Files, Mail, Messages, and Safari.

Measure app – Use your iPhone or iPad camera to measure objects and surfaces both vertically and horizontally as well as automatically detect dimensions of rectangular objects.

First up, we’ll start with the elements of ARKit that we’ve been able to experience today with those devices running iOS 12 beta here in at GSMArena tower.

Quick Look and the usdz file format

With the close links between Apple and Pixar it comes as no surprise that the ‘usdz’ file format is based on Pixar’s open source USD (Universal Scene Description) file format facilitating easy distribution and sharing. When an embedded image is linked within a web page, a 3D box icon is shown in the top right corner informing users that this is an AR object. There are many use cases for this technology, but you can imagine shopping sites benefiting straight off the bat. Due to the nature of the ‘usdz’ files, they’re extremely easy to share with others for viewing.

At the time of writing, the ‘usdz’ file format viewing is only supported by Apple’s upcoming iOS 12 and macOS Mojave, with Adobe stating they’ll support ‘usdz’ for content creation. As with any new standard, for it to have longevity, the format has to be supported for end users outside of the Apple-verse on other platforms otherwise it will wither and die.

AR Quick Look samples

If you’re running the Developer or Public Beta of iOS 12 you can tap any of the 3D models below to view the object and recreate it in AR. If you’re not running iOS 12 clicking on a model will download the usdz file. If you’re not running the iOS 12 beta, and why should you be, we’ve added some videos following the samples.

The first example is a model that we created here at GSMArena and converted to the new usdz format using the converter tool that ships with XCode 10.

Our created model

Models courtesy of Apple

AR Video samples

The following two videos demonstrate the new Quick Look AR functionality within iOS 12 viewing usdz 3D objects.