Touching Pixels. How to Palpate with Technology
It might seem that the Spielberg film ‘Ready Player One’ is simply science fiction. But in the 19th century, writers and scholars fantasized about telegraph devices in every home. Today we know this as the Internet. Perhaps after some time, a suit conveying sensations from virtual reality will become a common thing. Screenlifer figured out how to transfer touch using technology.
Touching the Virtual Ice
It should be noted that almost every person is familiar with haptics, because phone vibration is one of the technologies for transmitting tactile sensations.
The first people to experience the tactile technology were pilots. They were trained on flight simulators with built-in servo-mechanisms, that in accordance with the pilot’s actions, made the training cabin vibrate or bend.
In 1976, Sega began to use this technology in their arcades. The first game was Fonz, a motorcycle racing simulator. When colliding with obstacles or vehicles, the player felt the vibration of the steering wheel. After that, the vibration appeared in other driving simulators, and later in automatic shooting machines. Today, this technology is present in joystick game consoles like Playstation.
An American company called Novint Technologies decided to provoke a revolution in the tactile sensation of what is happening on the screen.
Its main product is the Falcon 3D game controller. It can mimic the recoil of a gun, the sensation of hitting in golf, hitting a car or throwing a football. For the first time ever, people can become a character and feel what it feels, and not just press buttons. With Falcon, if you take a book or a glass in the game, you can feel their weight.
Falcon is a predator that hunts mice. And it meets the goals of creating the controller: it is designed to get rid of a computer mouse in video games and other applications. Falcon has a huge advantage: unlike the two-dimensional movement (right-left, forward and backward), Falcon can also move up and down, that is, in three dimensions. The device sensors track the handle position with millimeter accuracy, and the engines can be updated up to 1000 times per second, providing a realistic touch feeling. The engines create necessary resistance which is perceived as the weight of virtual objects and gives a feeling of their hardness.
This device from the future is available to anyone who has $ 500. Having bought it, you will get both the controller and an electronic textbook and 24 mini-games. The textbook has all the information on the possibilities of this technology. You can choose different textures and ‘touch’ them: the ice is slippery — touching it, the virtual hand will easily slide, but the emery paper is rough — it will be more difficult for the ‘hand’ to slide.
Falcon will appeal to Half Life fans: in Haptics Life 2, a game mod, mouse controls have been replaced with 3D controllers. As a result, the player will feel the recoil of the weapon, the weight of the transferred items, as well as the acceleration of the character. Since June 2009, Valve has added Falcon support to its games: Half-Life 2: Episode One, Half-Life 2: Episode Two, Portal and Team Fortress 2.
Play and Save Lives
Thanks to scientific development, fantasy fiction is becoming reality. One of such inventions, shown in the ‘Avatar’ is the transfer of consciousness from the human body to the body of a resident of the Pandora planet. In today’s earth reality, a similar technology does not look as impressive, but the potential is great, as long as we can only manage robots.
Falcon can be used in medicine. Surgery often requires manipulations that are so subtle that surgeons use robots controlling them from a distance. Such mechanisms cost tens of thousands of dollars. Falcon is much cheaper, which, in turn, can reduce the cost of complex operations.
Thanks to Novint’s technology, surgeons will be able to literally feel the tools that are controlled by a robotic hand. The controller is able to simulate tactile sensations of surgical instruments, which increases the accuracy.
However, the technology is useful not only for practitioners, but also for students, as well as medical professionals who improve their skills. And it is difficult to overestimate that. According to Andrey Pavlenko, an oncosurgeon who struggled with stomach cancer himself and later founded the School of Practical Oncology and Cancer Fund, Russian surgeons have an acute shortage of practice. It is due to the fact that ‘doctors no longer trust residents to assist in surgeries or perform themselves, because they are responsible for any complications.’ Devices such as Falcon connected to a training program with a simulation of the surgery can help with that and replace corpses, animals, and bulky physical trainers.
Another device that will help doctors treat people more effectively is the CyberGrasp system. This is a finger and hand feedback system that allows you to tactilely sense computer-generated objects.
CyberGrasp is a small exoskeleton that is worn on a CyberGlove glove and can cause pressure on each finger using built-in mechanisms. Thanks to the CyberGrasp strength feedback system, users can sense the size and shape of computer-generated three-dimensional objects in a way that a controller does not allow.
Obtatining the Lost
Screenlifer posted an article about the management of prosthetic limbs using the ‘power of thought’: the electrodes built into the brain send impulses to prosthesis, and it performs the necessary action. The technology is far from perfect, but there are already successful experiments.
However, the thought-driven hand prosthesis still can’t compare to an ordinary hand, because it does not have the ability to transmit tactile information to a person. Because of this, the manipulation of objects is not accurate. For example, shaking hands, a person with a prosthesis can accidentally damage it by incorrectly applying force. In addition to interacting with objects, touch plays an important role in social communication. Touching is an important part of non-verbal communication.
There are no prototypes, but in theory one of the ways to return tactile sensations to people who have lost a limb is electrical simulation of certain brain areas based on data that a prosthesis with special tools for this purpose collected from the object.
The assumption that this will work is based on the fact that people who have lost a limb experience phantom pains. At this moment, tactile sensations are born not due to the nerve endings of the skin, but due to the processes occurring in the brain.
Scientists warn that for the time being, an option where it will be possible to simulate the sensations of different textures is hardly possible. For example, the technology does not allow to feel the difference between stroking a person’s hair and cat’s hair. However, it is quite possible to feel the resistance of an object, its weight.
Read more about this in Science Robotics.
Where it Leads
The sense of touch allows a person to learn as effectively as vision does not allow. When a person simply performs several movements several times or strains muscles in a certain way — muscular memory fixes this. You can endlessly watch skillfully scored goals of famous football players, but this is unlikely to teach anyone play football. It is much more effective to enter the field and bring the ball to the goal at least once. Therefore, for people who have lost limbs, it is very important to use devices that are as functional as lost arms or legs to be able to gain new skills.
It can be a breakthrough in medicine, too. In addition to the benefits for surgery, such technology opens the way to real Internet medicine: at a distance, doctors can, for example, perform palpation of the abdomen with the help of controllers or special gloves. In addition, it is possible for devices to record information received from one of the users and reproduce it in the future. This can give doctors an opportunity to learn from their best predecessors simply by downloading a desired program.
The technology is of genuine interest for the gaming industry because it carries a huge commercial potential. And for the players themselves this is an opportunity to get not just new impressions, but also a new experience. That is why researchers place so much hope on technology.
Cover image: Novint Technologies Inc.
