Collection of AR Quick Look Bugs

We're documenting our findings and issues here to hopefully get them fixed. Most of these seem to be related to incorrect USDZ understanding, especially in regards to animation but also incorrect transform matrix handling.

For questions, feel free to contact herbst@prefrontalcortex.de.

(1) A rough summary of "Best Practices aka Bug Workarounds" for animations in Quick Look USDZ we found so far:

  1. Only linear keyframe interpolation is supported. "Hold" keys are interpreted as linear.
  2. Object transform are stored as matrices, not pos/rot/scale. Quick Look seems to have issues with matrix decomposition (that usdview and other USD tools don't have).
    1. Don't use Scale (0,0,0), keyframes using that might be ignored even if pos / rot are important
    2. Avoid shearing, even if by small values (e.g. avoid having any objects parented to objects that are non-uniformly scaled).
  3. First 1/2 second or so is unstable in terms of animation. Sometimes keyframes will be skipped, sometimes not. This also means that you should not use very short animations.
  4. Smooth, looping animations are difficult to achieve; Quick Look discards keyframes at the beginning/end of an animation. If you need a loop, make sure the beginning/end is pretty slow and pad it out with many additional keyframes.
  5. If you animate objects and get weird flips, make sure that their initial rotation is identical to their non-animated rotation. Sometimes objects will "rotate in" from identity rotation, ignoring both their position with and without animation.
  1. Don't use timecode animation, use Frame animation. So, make sure the timecodes in the USDZ file are 0,1,2,3 (in frames) and not 0.02,0.04,0.06,... (in seconds). QuickLook's timecode implementation is broken and will round all timecodes to int, resulting in all of the above issues that seem to be sideeffects from this crazy bug.

(2) General Notes

  1. We haven't found a good viewer on Windows Destop yet (and also not on Mac) that comes close to what you will get on an actual iOS device. usdview has issues with material properties, Reality Converter / Reality Composer interpret files differently than QuickLook.
  2. Model bounds (e.g. for object preview and quick look placement) are calculated from the first keyframe. If you have a small object getting bigger, it will clip outside of the bounds. If you still need this, include a couple tiny cubes in good positions to cover the entire bounds your animation will fill, to enforce larger bounds.
  3. Object pivot is used for rotating in Quick Look and Object Preview, so make sure the pivot is in a good place (other tools "auto-center" which has its own issues). If your object rotates/scales around a weird point, check pivots.
  4. Vertex Colors are not supported.
  5. Backface culling does not work (even if the USD is correctly set up).
  6. Google has a lot to say about USDZ and its feature set compared to GLTF: GitHub usd-from-gltf. Some of the information is probably outdated (e.g. Quick Look already supports texture "scale" aka multiplying a texture by a color, at least for diffuse channel).

(3) In theory, Apple has a "better" AR format by now: .reality files, created with Aero or Reality Composer. However, that has its own issues:

  1. Even according to Apple, it's not made for the web. Textures are stored in a GPU compression format that is 10-50x bigger than the original files that were optimized for web (e.g. a 100KB JPEG might end up taking 10MB in .reality format).
  2. A lot of behaviour in Reality Composer (iOS) is different to Reality Composer (Mac OS) and in turn different to the actual on-device behaviour if you open a .reality file with QuickLook. Expect many bugs and hacks.
  3. Overlapping behaviours (e.g. having more than one object which does a camera lookat) have pretty undefined behaviour. Sometimes you're lucky and it works.
  4. Multi-scene behaviour is unstable.

List of Bug Reports

AR Quick Look - Shadow Plane Bug

USDZ file: BUG - Shadow Clipping - Cube
USDZ file: BUG - Shadow Clipping - Manta
Feedback Assistant Link: Issue 7612599
Issue documentation:





AR Quick Look - unexpected rotation in the first frame of the animation (USDZ)

USDZ file: BUG - First Frame Rotation Bug
Feedback Assistant Link: Issue 7612592
Issue documentation:

Expected behavior:



AR Quick Look - missing frames in the animated USDZ file

USDZ file: BUG - Incorrect Keyframe Animation, missing Keys
Feedback Assistant Link: Issue 7612617
Issue documentation:

Expected behavior:



AR Quick Look - unexpected scale behavior during animation

USDZ file: BUG - Incorrect Keyframe Animation, missing Keys
Feedback Assistant Link: Issue 7612823
Issue documentation:

Expected behavior:



AR Quick Look - last frames in the animation loop are ignored

USDZ file: BUG - Animation Should Loop
Feedback Assistant Link: Issue 7612607
Issue documentation:

Expected behavior:



AR Quick View - short animation doesn't play

USDZ file: BUG - Animation does not play
Feedback Assistant Link: Issue 7612864
Issue documentation:

Expected behavior:



AR Quick View - scaling proportions in the animated USDZ file are wrong

USDZ file: BUG - Pivot Scale
Feedback Assistant Link: Issue 7612904
Issue documentation:

Expected behavior:



AR Quick View - unexpected big preview in "Object Mode" (you can not see the whole object)

Object Mode bounds calculation is done on first frame, so if the object grows considerably over the time of the animation, it will be cut off by clipping planes. USDZ file: BUG - animation preview in Object Mode
Feedback Assistant Link: Issue 7612913
Issue documentation:




AR Quick View - wrong position of the USDZ scene in AR placement preview

"Object placement preview" (before placing the object on a surface) calculates display size from bounds, but then offsets the object by its pivot, resulting in a clipped preview. USDZ file: Positioning Test - offset pivot
Feedback Assistant Link: Issue 7612973
Issue documentation:




AR Quick View - USDZ model is twisted

Animated USDZ model, that looks and behaves good in USDZ viewer (see attatched video) but looks twisted in AR Quick View. USDZ file: Broken deer
Feedback Assistant Link: Issue FB7701501
Issue documentation:


Additional Broken Files

Move And Hide