Virtual Reality | Hand Interaction | Training & Learning Experiences

Over the last 3 years we have carried out a lot of experimentation and implementation of hand tracking in virtual reality. There are a lot of benefits in interacting with the environment with your hands. For one you get to remove the controllers. We find players who are not used to games find it difficult to use the buttons in VR. There is always an explanation needed at the start of any session to illustrate the buttons and what they do.

Attaching menu buttons to hands and using fingers to touch these buttons is much more intuitive than using controllers. In our children's VR experience Fleebux 9 we tracked menu buttons to the hand of the player. When the player touched these buttons they got the haptic feedback of touching their real skin.

Adding in any haptic feedback improves the experience in virtual reality. So it’s important to design your menus around this. We found the wrist or palm to be a great location for a menu button.

During our development of these projects we found it to be necessary to design specific mechanics for when the player interacts with objects in the scene. We initially tried picking up objects with your hands. This was a little unsatisfying for the player because you expect to feel the object when you pick it up in virtual reality.

To counter this we designed objects to have a force field around them that would maintain a constant distance from the players hand. This would prevent the player from thinking they were touching these objects. This same method was also applied to assisting character movements, you never actually touch the person but guide them with some clever user interface.

It’s always worth putting time into creating mechanics that feel natural and intuitive for the player.

We are looking forward to Oculus implementing hand tracking into the Quest as well as future devices. It is a very fun way to interact with the environment and provides a much more natural alternative to using controllers.