Meta Quest 2 Android / SnapDragon XR2 Model Loader/ inference performance
Hello
I have just started exploring this as an alternative to MSFT cognitive services for our Quest 2 VR application. I've been able to run the example on a Quest 2 device and have noticed the following:
- There is a long delay on Application Start as the 3 models are loaded and the app hangs on start up.
- The recognition inference seems to cause a delay in updating the UI and causes the VR rendering to stall ( likely due to a reduction in available hardware resources)
A couple of questions:
- Could the model loading be done on a separate thread to reduce the app start-up time?
- Where does the inference take place, on the CPU or on GPU? I'd like to understand where the bottleneck is in terms of recognition performance.
Thanks in advance.
Hi there,
The inference takes place mostly on GPU. Unless otherwise specified the inference will all try to be done on a single frame. Take a look at this example: https://docs.unity3d.com/Packages/com.unity.sentis@1.4/manual/split-inference-over-multiple-frames.html which allows you to split the inference over several frames and hence improve the frame rate.
For model loading. At present I don't believe it can be done on a different thread. However this is a good idea that maybe we should allow model loading to take place similarly to be split over several frames.
BTW we also have a discussion forum here: https://discussions.unity.com/c/ai-beta/sentis/10 for general Sentis questions.
Thanks for the reply @UnityPaul .
It seems that the ExecuteByLayer function is not available in Sentis 1.3.0-pre.4 but is in Sentis 1.4.0-pre.2. The tiny-whisper project was developed on 1.3.0-pre.4 and there are several functions that appear to not exist in the update.
WorkerFactory.CreateOps() function doesn't exist in 1.4.0 and the ITensorAllocator type doesn't exist.
Are there plans to upgrade the project to 1.4.0-pre.2? Or could you provide some support for upgrading?
Thanks again.
I think 1.3.0-pre.3 was the latest 1.3.0 build.
Yes, the new 1.4.0 code is now in the main branch of the https://huggingface.co/unity/sentis-whisper-tiny
We will provide support for upgrading, or you are still able to use the 1.3.0 version if that works for you. Probably best to get advice here: https://discussions.unity.com/c/ai-beta/sentis