Reach in and pick it up: hands, physics and the AREngine app
5 min read
Until now, an AR Works project was something you looked at. You could tap a planet to hear its name or start an animation, but your hands stayed on the screen. The scene floated in your room and you walked round it.
This update changes that. Hold your phone up, reach into the scene and pick something up. Let go and it falls, rolls and stops against whatever is in its way. Your hand passes in front of the objects it is in front of and behind the ones it is behind, so the scene feels like part of the room rather than a sticker on top of it.
Here is a short look at what that means in practice.
Pick things up with your hands
Select an object in the editor and turn on Interactive. Viewers can now touch it and pick it up with their hand. Turn on Physics as well and it has weight: it drops when you let go and lands on the floor or on another object. Can be pushed lets an open hand nudge an object along, so a ball rolls away and a box slides.
Each object has a Bounds shape that your hand touches and physics uses. A box, sphere or cylinder covers most things. For hollow or awkward shapes, Convert to mesh shape follows the model itself, so a ball can drop down a pipe instead of bouncing off an invisible lid.
The phone looks for your hand in the camera picture and cuts it out, so it appears in front of the objects it is reaching past. Nothing about your hands leaves the phone: the hand tracking runs on the device.
Logic that reacts to touch
The logic editor has new events and actions to go with it. On Hand Touch, On Grab and On Release fire when a viewer handles an object, and On Collision fires when two objects meet. Set Physics, Set Interactive, Push Object and Reset Object change things while the scene runs.
Put them together and you can build small puzzles without writing code. Drop a ball into a pipe, let it hit a switch at the bottom, and open a chest with an animation and a sound.
More than one picture in a scene
A picture-tracked scene used to follow one printed image. You can now add Extra target images: several prints laid out together that give one steadier position. You can also add Tracking images, pictures that move on their own and carry objects with them. Put a model on a playing card and it stays on the card as you slide it across the table, while the rest of the scene stays where it is.
Tracking images also work in room scenes, so a print on a box or a can can carry its own content inside a larger world scene.
AREngine, our own AR app for Android
Some of this asks a lot of a web browser. So alongside the web viewer, we have built AREngine, an AR app for Android with its own tracking, hand detection and physics. Export your project as a .arengine file, open it on the phone, and it plays in the app.
AREngine is going through Google Play's review now and will be released to the public on Google Play soon. Until then you can install it from the AREngine page. An iPhone version is being tested.
Your links and QR codes still open in the browser exactly as before. When you share a project you can choose which engine the link uses: AREngine's tracking or the classic engine that existing projects were made with.
Glasses and headsets
AREngine also has early support for AR glasses and headsets: display glasses that plug into a phone, Android glasses with their own camera, and headsets that support OpenXR. It is experimental, and testers are trying it on their own devices now. The glasses and headsets guide says which app each kind of device needs, and a setup guide comes with the download.
If you have a pair of AR glasses, a device report from AREngine helps us support them properly. It holds nothing personal and only leaves the device when you choose to send it.
Smaller changes you will notice
- Start spot. Room scenes now have a spot that marks where the viewer stands, with a height for the floor, so a scene opens the way you designed it.
- Blender files. You can upload .blend files on the Pro and Education plans. They are converted to a model on our servers, and you can pick which collections to bring in.
- Materials. A new Glow material, editable materials on imported models, and the option to delete a material you do not need.
- Tidier projects. Objects can sit inside other objects, assets can live in nested folders, and the editor's panels can be docked where you want them.
Try it
Open a project, select an object and turn on Interactive and Physics. Then open Preview and click Use webcam, or share the project to your phone, and reach in. The hand interaction guide covers each setting, and we would love to see what you make with it.
Try AR Works free
Build your first augmented reality experience in the browser - no code, no app installs.