Chapter 10: Design Methodologies
40 Virtual Reality
From Wikipedia: https://en.wikipedia.org/wiki/Virtual_reality
Virtual reality (VR) is a simulated experience that can be similar to or completely different from the real world. Applications of virtual reality can include entertainment (i.e. gaming) and educational purposes (i.e. medical or military training). Other, distinct types of VR style technology include augmented reality and mixed reality.
Currently standard virtual reality systems use either virtual reality headsets or multi-projected environments to generate realistic images, sounds and other sensations that simulate a user’s physical presence in a virtual environment. A person using virtual reality equipment is able to look around the artificial world, move around in it, and interact with virtual features or items. The effect is commonly created by VR headsets consisting of a head-mounted displaywith a small screen in front of the eyes, but can also be created through specially designed rooms with multiple large screens. Virtual reality typically incorporates auditory and video feedback, but may also allow other types of sensory and force feedback through haptic technology.
Concerns and challenges
There are certain health and safety considerations of virtual reality. For example, a number of unwanted symptoms have been caused by prolonged use of virtual reality, and these may have slowed proliferation of the technology. Most virtual reality systems come with consumer warnings. In addition, there are social, conceptual, and philosophical considerations with virtual reality. What the phrase “virtual reality” means or refers to, is not always unambiguous. In the book The Metaphysics of Virtual Reality by Michael R. Heim, seven different concepts of virtual reality are identified: simulation, interaction, artificiality, immersion, telepresence, full-body immersion, and network communication. There has been an increase in interest in the potential social impact of new technologies, such as virtual reality. In the book Infinite Reality: Avatars, Eternal Life, New Worlds, and the Dawn of the Virtual Revolution, Blascovich and Bailenson review the literature on the psychology and sociology behind life in virtual reality.
Mychilo S. Cline’s book Power, Madness, and Immortality: The Future of Virtual Reality, argues that virtual reality will lead to a number of important changes in human life and activity. He argues that virtual reality will be integrated into daily life and activity, and will be used in various human ways. Another such speculation has been written up on how to reach ultimate happiness via virtual reality. He also argues that techniques will be developed to influence human behavior, interpersonal communication, and cognition. As we spend more and more time in virtual space, there would be a gradual “migration to virtual space”, resulting in important changes in economics, worldview, and culture. Philosophical implications of VR are discussed in books, including Philip Zhai‘s Get Real: A Philosophical Adventure in Virtual Reality (1998) and Digital Sensations: Space, Identity and Embodiment in Virtual Reality (1999), written by Ken Hillis.
Virtual reality technology faces a number of challenges, most of which involve motion sickness and technical matters. Users might become disoriented in a purely virtual environment, causing balance issues; computer latency might affect the simulation, providing a less-than-satisfactory end-user experience; the complicated nature of head-mounted displays and input systems such as specialized gloves and boots may require specialized training to operate, and navigating the non-virtual environment (if the user is not confined to a limited area) might prove dangerous without external sensory information. In January 2014, Michael Abrash gave a talk on VR at Steam Dev Days. He listed all the requirements necessary to establish presence and concluded that a great VR system will be available in 2015 or soon after. While the visual aspect of VR is close to being solved, he stated that there are other areas of VR that need solutions, such as 3D audio, haptics, body tracking, and input. However, 3D audio effects exist in games and simulate the head-related transfer function of the listener (especially using headphones). Examples include Environmental Audio Extensions (EAX), DirectSound and OpenAL. VR audio developer Varun Nair points out that from a design perspective, sound for VR is still very much an open book. Many of the game audio design principles, especially those related to FPS games, crumble in virtual reality. He encourages more sound designers to get involved in virtual reality audio to experiment and push VR audio forward. There have been rising concerns that with the advent of virtual reality, some users may experience virtual reality addiction.
Before the 1950s
The first references to the concept of virtual reality came from science fiction. Stanley G. Weinbaum‘s 1935 short story “Pygmalion’s Spectacles”  describes a goggle-based virtual reality system with holographic recording of fictional experiences, including smell and touch. Virtual reality in modern meaning was popularized by Jaron Lanier through his company VPL Research, which held many of the mid-1980s VR patents. VPL developed the first head-mounted display (HMD), the EyePhone, and introduced the Haptic Input DataGlove.
Also notable among the earlier hypermedia and virtual reality systems was the Aspen Movie Map, which was created at MIT in 1978. The program was a crude virtual simulation of Aspen, Colorado in which users could wander the streets in one of three modes: summer, winter, and polygons. The first two were based on photographs—the researchers actually photographed every possible movement through the city’s street grid in both seasons—and the third was a basic 3-D model of the city. Atari founded a research lab for virtual reality in 1982, but the lab was closed after two years due to Atari Shock (North American video game crash of 1983). However, its hired employees, such as Tom Zimmerman, Scott Fisher, Jaron Lanier and Brenda Laurel, kept their research and development on VR-related technologies. By the 1980s the term “virtual reality” was popularized by Jaron Lanier, one of the modern pioneers of the field. Lanier had founded the company VPL Research in 1985. VPL Research has developed several VR devices like the Data Glove, the Eye Phone, and the Audio Sphere. VPL licensed the Data Glove technology to Mattel, which used it to make an accessory known as the Power Glove. While the Power Glove was hard to use and not popular, at US$75, it was early affordable VR device.
During this time, virtual reality was not well known, though it did receive media coverage in the late 1980s. Most of its popularity came from marginal cultures, like cyberpunks, who viewed the technology as a potential means for social change, and drug culture, who praised virtual reality not only as a new art form, but as an entirely new frontier. The concept of virtual reality was popularized in mass media by movies such as Brainstorm (1983) and The Lawnmower Man. The VR research boom of the 1990s was accompanied by the non-fiction book Virtual Reality (1991) by Howard Rheingold. The book served to demystify the subject, making it more accessible to researchers outside of the computer sphere and sci-fi enthusiasts.
Once the industry began to attract media coverage, some even compared the innovations in virtual reality to the Wright Brothers’ pioneering invention of the airplane. In 1990, Jonathan Waldern, a VR Ph.D, demonstrates “Virtuality” at the Computer Graphics 90 exhibition staged at London’s Alexandra Palace. This new system was an arcade machine that would use a virtual reality headset to immerse players. CyberEdge and PCVR, two VR industry magazines, started to publish in the early 1990s. However, most ideas about VR remained theoretical due to the limited computing power available at the time. The extremely high cost of the technology made it impossible for consumers to adopt. When the Internet became widely available, this became the technology focus for most people. The VR industry mainly provided VR devices for medical, flight simulation, automobile industry design, and military training purposes from 1970 to 1990.
In 1991, Sega announced the Sega VR headset for arcade games and the Mega Drive console. It used LCD screens in the visor, stereo headphones, and inertial sensors that allowed the system to track and react to the movements of the user’s head. In the same year, Virtuality launched and went on to become the first mass-produced, networked, multiplayer VR entertainment system. It was released in many countries, including a dedicated VR arcade at Embarcadero Center in San Francisco. Costing up to $73,000 per multi-pod Virtuality system, they featured headsets and exoskeleton gloves that gave one of the first “immersive” VR experiences. Antonio Medina, a MIT graduate and NASA scientist, designed a virtual reality system to “drive” Mars rovers from Earth in apparent real time despite the substantial delay of Mars-Earth-Mars signals. The system, termed “Computer-Simulated Teleoperation” as published by Rand, is an extension of virtual reality.
In 1991, Carolina Cruz-Neira, Daniel J. Sandin and Thomas A. DeFanti from the Electronic Visualization Laboratory created the first cubic immersive room, replacing goggles by a multi-projected environment where people can see their body and other people around. In that same year, Computer Gaming World predicted “Affordable VR by 1994”.By 1994, Sega released the Sega VR-1 motion simulator arcade attraction, in SegaWorld amusement arcades. It was able to track head movement and featured 3D polygon graphics in stereoscopic 3D, powered by the Sega Model 1 arcade system board. Also in 1994 Apple released QuickTime VR. A widely available product for interacting with VR models.
A year later, the artist Maurice Benayoun created the first VR artwork connecting in real time 2 continents: the “Tunnel under the Atlantic” between the Pompidou Centre in Paris and the Museum of Contemporary Art in Montreal. The installation included dynamic real time 3d modeling, video chat, spatialized sound and AI content management. The Virtual Boy was created by Nintendo and was released in Japan on July 21, 1995 and in North America on August 15, 1995. Also in 1995, a group in Seattle created public demonstrations of a “CAVE-like” 270 degree immersive projection room called the Virtual Environment Theater, produced by entrepreneurs Chet Dagit and Bob Jacobson. Then in 1996 the same system was shown in tradeshow exhibits sponsored by Netscape Communications, and championed by Jim Barksdale, for the first time showing VR connected to the Internet with World Wide Web content feeds embedded in VRML 3D virtual world models. Forte released the VFX1, a PC-powered virtual reality headset in 1995, which was supported by games including Descent, Star Wars: Dark Forces, System Shock and Quake. In 1999, entrepreneur Philip Rosedale formed Linden Lab with an initial focus on the development of hardware that would enable computer users to be fully immersed in a 360 degree virtual reality experience. In its earliest form, the company struggled to produce a commercial version of “The Rig,” which was realized in prototype form as a clunky steel contraption with several computer monitors that users could wear on their shoulders. That vision soon morphed into the software-based, 3D virtual world Second Life.
In 2001, SAS3 or SAS Cube became the first PC based cubic room, developed by Z-A Production (Maurice Benayoun, David Nahon), Barco, Clarté, installed in Laval France in April 2001. The SAS library gave birth to Virtools VRPack. By 2007, Google introduced Street View, a service that shows panoramic views of an increasing number of worldwide positions such as roads, indoor buildings and rural areas. It also features a stereoscopic 3D mode, introduced in 2010. In 2010, Palmer Luckey, who later went on to found Oculus VR, designed the first prototype of the Oculus Rift. This prototype, built on a shell of another virtual reality headset, displayed only 2-D images and was noticeably cumbersome to wear. However, it boasted a 90-degree field of vision that was previously unseen anywhere in the market at the time. This initial design would later serve as a basis from which the later designs came.
In 2013, Nintendo filed a patent for the concept of using VR technology to produce a more realistic 3D effect on a 2D television. A camera on the TV tracks the viewer’s location relative to the TV, and if the viewer moves, everything on the screen reorients itself appropriately. “For example, if you were looking at a forest, you could shift your head to the right to discover someone standing behind a tree.” In July 2013, Guild Software’s Vendetta Online was widely reported as the first MMORPG to support the Oculus Rift,making it potentially the first persistent online world with native support for a consumer virtual reality headset. Since 2013, there have been several virtual reality devices that seek to enter the market to complement Oculus Rift to enhance the game experience. One, Virtuix Omni, is based on the ability to move in a three dimensional environment through an omnidirectional treadmill.
On March 25, 2014, Facebook purchased a company that makes virtual reality headsets, Oculus VR, for $2 billion. Sony announces Project Morpheus (its code name for PlayStation VR), a virtual reality headset for the PlayStation 4. Google announces Cardboard, a do-it-yourself stereoscopic viewer for smartphones. The user places her smartphone in the cardboard holder, which she wears on her head. In 2015, the Kickstarter campaign for Gloveone, a pair of gloves providing motion tracking and haptic feedback, was successfully funded, with over $150,000 in contributions.
In February–March 2015, HTC partnered with Valve Corporation announced their virtual reality headset HTC Vive and controllers, along with their tracking technology called Lighthouse, which utilizes “base stations” mounted to the wall above the user’s head in the corners of a room for positional tracking of the Vive headset and its motion controllers using infrared light. The company announced its plans to release the Vive to the public in April 2016 on December 8, 2015. Units began shipping on April 5, 2016.
In July 2015, OnePlus became the first company to launch a product using virtual reality. They used VR as the platform to launch their second flagship device the OnePlus 2, first viewable using an app on the Google Play Store, then on YouTube. The launch was viewable using OnePlus Cardboard, based on the Google’s own Cardboard platform. The whole VR launch had a runtime of 33 minutes, and was viewable in all countries. Also in 2015, Jaunt, a startup company developing cameras and a cloud distribution platform, whose content will be accessible using an app, reached $100 million in funding from such sources as Disney and Madison Square Garden. On April 27, 2016, Mojang announced that Minecraft is now playable on the Gear VR. Minecraft is still being developed for the Oculus Rift headset but a separate version was released to the Oculus Store for use with the Gear VR. This version is similar to the Pocket Edition of Minecraft.
Education and training
Research is being done on learning in virtual reality, as its immersive qualities may enhance learning. VR is used by trainers to provide learners with a virtual environment where they can develop their skills without the real-world consequences of failing. Thomas A. Furness III was one of the first to develop the use of VR for military training when, in 1982, he presented the Air Force with his first working model of a virtual flight simulator he called the Visually Coupled Airborne Systems Simulator (VCASS). By the time he started his work on VCASS, aircraft were becoming increasingly complicated to handle and virtual reality provided a better solution to previous training methods. Furness attempted to incorporate his knowledge of human visual and auditory processing to create a virtual interface that was more intuitive to use. The second phase of his project, which he called the “Super Cockpit,” was even more advanced, with high resolution graphics (for the time) and a responsive display. Furness is often credited as a pioneer in virtual reality for this research. VR plays an important role in combat training for the military. It allows the recruits to train under a controlled environment where they are to respond to different types of combat situations. A fully immersive virtual reality that uses head-mounted display (HMD), data suits, data glove, and VR weapon are used to train for combat. This setup allows the training’s reset time to be cut down, and allows more repetition in a shorter amount of time. The fully immersive training environment allows the soldiers to train through a wide variety of terrains, situations and scenarios.
VR is also used in flight simulation for the Air Force where people are trained to be pilots. The simulator would sit on top of a hydraulic lift system that reacts to the user inputs and events. When the pilot steer the aircraft, the module would turn and tilt accordingly to provide haptic feedback. The flight simulator can range from a fully enclosed module to a series of computer monitors providing the pilot’s point of view. The most important reasons on using simulators over learning with a real aircraft are the reduction of transference time between land training and real flight, the safety, economy and absence of pollution. By the same token, virtual driving simulations are used to train tank drivers on the basics before allowing them to operate the real vehicle. Finally, the same goes for truck driving simulators, in which Belgian firemen are for example trained to drive in a way that prevents as much damage as possible. As these drivers often have less experience than other truck drivers, virtual reality training allows them to compensate this. In the near future, similar projects are expected for all drivers of priority vehicles, including the police.
Medical personnel are able to train through VR to deal with a wider variety of injuries. An experiment was performed by sixteen surgical residents where eight of them went through laparoscopic cholecystectomy through VR training. They then came out 29% faster at gallbladder dissection than the controlled group. With the increased commercial availability of certified training programs for basic skills training in VR environments, students have the ability to familiarize themselves with necessary skills in a corrective and repetitive environment; VR is also proven to help students familiarize themselves with skills not specific to any particular procedure. VR application was used to train road crossing skills in children. It proved to be rather successful. However some students with autistic spectrum disorders after such training might be unable to distinguish virtual from real. As a result, they may attempt quite dangerous road crossings.
The use of graphics, sound and input technology in video games can be incorporated into VR. Several Virtual Reality head mounted displays (HMD) were released for gaming during the early-mid 1990s. These included the Virtual Boy developed by Nintendo, the iGlasses developed by Virtual I-O, the Cybermaxx developed by Victormaxx and the VFX1 Headgear developed by Forte Technologies. Other modern examples of narrow VR for gaming include the Wii Remote, the Kinect, and the PlayStation Move/PlayStation Eye, all of which track and send motion input of the players to the game console somewhat accurately. Several companies are working on a new generation of VR headsets: Oculus Rift is a head-mounted display for gaming purposes developed by Oculus VR, an American technology company that was acquired for US$2 billion by Facebook in 2014. One of its rivals was named by Sony as PlayStation VR (codenamed Morpheus), which requires a PS4 instead of a PC to run. In 2015, Valve Corporation announced their partnership with HTC to make a VR headset capable of tracking the exact position of its user in a 4.5 by 4.5 meters area, the HTC Vive. All these virtual reality headsets are tethered headsets that use special curved lenses to magnify and stretch a 5.7-inch screen (in the case of Morpheus) across the field of vision. There are more gaming VR headsets in development, each with its own special abilities. StarVR offers a 210° field of view, whereas FOVE tracks the position of your eyes as an input method.
David Em was the first fine artist to create navigable virtual worlds in the 1970s. His early work was done on mainframes at Information International, Inc., Jet Propulsion Laboratory, and California Institute of Technology. Jeffrey Shaw explored the potential of VR in fine arts with early works like Legible City (1989), Virtual Museum (1991), andGolden Calf (1994). Canadian artist Char Davies created immersive VR art piecesOsmose (1995) and Ephémère (1998). Maurice Benayoun‘s work introduced metaphorical, philosophical or political content, combining VR, network, generation and intelligent agents, in works like Is God Flat? (1994), “Is the Devil Curved?” (1995), The Tunnel under the Atlantic (1995), and World Skin, a Photo Safari in the Land of War(1997). Other pioneering artists working in VR have include Knowbotic Research,Rebecca Allen and Perry Hoberman.
The use of 3D computer-aided design (CAD) data was limited by 2D monitors and paper printouts until the mid-to-late 1990s, when projectors, 3D tracking, and computer technology enabled a renaissance in the use 3D CAD data in virtual reality environments. With the use of active shutter glasses and multi-surface projection units immersive engineering was made possible by companies like VRcom and IC.IDO. Virtual reality has been used in automotive, aerospace, and ground transportation original equipment manufacturers (OEMs) in their product engineering and manufacturing engineering . Virtual reality adds more dimensions to virtual prototyping, product building, assembly, service, performance use-cases. This enables engineers from different disciplines to view their in the final product. Immersive engineering enables management and investors to see virtual prototypes prior to the availability of any physical prototypes.
Heritage and archaeology
The first use of a VR presentation in a heritage application was in 1994, when a museum visitor interpretation provided an interactive “walk-through” of a 3D reconstruction of Dudley Castle in England as it was in 1550. This consisted of a computer controlled laserdisc-based system designed by British-based engineer Colin Johnson. The system was featured in a conference held by the British Museum in November 1994, and in the subsequent technical paper, Imaging the Past – Electronic Imaging and Computer Graphics in Museums and Archaeology. Virtual reality enables heritage sites to be recreated extremely accurately, so that the recreations can be published in various media. The original sites are often inaccessible to the public or, due to the poor state of their preservation, hard to picture. This technology can be used to develop virtual replicas of caves, natural environment, old towns, monuments, sculptures and archaeological elements.
One of the first recorded uses of virtual reality in architecture was in the late 1980s when the University of North Carolina modeled its Sitterman Hall, home of its computer science department, in a virtual environment. Several companies, including IrisVR and Floored, Inc., provide software or services that allow architectural design firms and various clients in the real estate industry to tour virtual models of proposed building designs. IrisVR currently provides software that allows users to convert design files created in CAD programs like SketchUp and Revit into files viewable with an Oculus Rift, HTC Vive, or a smartphone “in one click,” without the need for complex tiered workflows or knowledge of game engines such as Unity3D. Floored, meanwhile, manually constructs and refines Rift-viewable 3D models in-house from either CAD files for un-built designs or physical scans of already built, brick-and-mortar buildings, and provides clients with access to its own viewing software, which can be used with either an Oculus Rift or a standard 2D web browser, afterward.
VR software products like these can provide a number of benefits to architects and their clients. During the design process, architects can use VR to experience the designs they are working on before they are built. Seeing a design in VR can give architect a correct sense of scale and proportion. Having an interactive VR model also eliminates the need to make physical miniatures to demonstrate a design to clients or the public. Later on, after a building is constructed, developers and owners can create a VR model of a space that allows potential buyers or tenants to tour a space in VR, even if real-life circumstances make a physical tour unfeasible. For instance, if the owner of an apartment building has a VR model of a space while the building is under construction, she can begin showing and renting the units before they are even ready to be occupied. Furthermore, this sort of showing can be conducted over any distance, as long as the potential customer has access to a VR setup (or, even, with the help of Google Cardboard or a similar phone-based VR headset, nothing but a smartphone.)
In 2010, 3D virtual reality was beginning to be used for urban regeneration, planning and transportation projects. In 2007, development began on a virtual reality software which took design coordinate geometry used by land surveyors and civil engineers and incorporated precision spatial information created automatically by the lines and curves typically shown on subdivision plats and land surveying plans. These precise spatial areas cross referenced color and texture to an item list. The item list contained a set of controls for 3D rendering, such as water reflective surfaces or building height. The land surface in software to create a contour map uses a digital terrain model (DTM). By 2010, prototype software was developed for LandMentor, a technology to automate the process leading from design to virtualization. The first beta users in 2011 were able drape the design or survey data over the digital terrain to create data structures that were passed into a video game engine. The engine was able to create a virtual interactive world. The software was improved to implement 3D models from other free or commercially sold software to create a more realistic virtual reality.
The primary use of VR in a therapeutic role is its application to various forms of exposure therapy, including phobia treatments.
Since 2015, virtual reality has been installed onto a number of roller coasters, including Galactica at Alton Towers, The New Revolution at Six Flags Magic Mountain and Alpenexpress at Europapark, amongst others.
In Oslo Spektrum on May the 3rd 2016, Norwegian pop band a-ha cleared away their normal stage-production to give room for a very different concert performance in collaboration with Void, a Norwegian computational design studio working in the intersection between design, architecture, art and technology. The collaboration resulted in a unique one-of-a-kind concert with advanced scenography using 360 virtual reality technology. It used movement sensors that reacted to the band members’ movements, voices and instruments. 3D cameras, 20000 lines of codes, 1000 square meters of projection film and massive projectors was set up into a visual show that made the Oslo Spektrum arena in Oslo, Norway into a light installation and visual experience that unfolded live for the audience instead of a pre programmed sequence. The stereoscopic VR-experience was made available for Android users directly through a YouTube app and also made available for iPhone users and other platforms.
Lowe’s, IKEA and Wayfair and other retailers have developed systems that allow their products to be seen in virtual reality, to give consumers a better idea of how the product will fit into their home, or to allow the consumer to get a better look at the product from home. Consumers looking at digital photos of the products can “turn” the product around virtually, and see it from the side or the back. This enables customers to get a better sense of how a product looks and about its features, than with the typical single picture used in traditional paper catalogues.
Non-profit organisations such as Amnesty International, UNICEF, and World Wide Fund for Nature (WWF) have started using virtual reality to bring potential supporters closer to their work, effectively bringing distant social, political and environmental issues and projects to members of the public in immersive ways not possible with traditional media. Panoramic 360 views of conflict in Syria and face to face encounters with CGI tigers in Nepal have been used in experiential activations and shared online to both educate and gain financial support for such charitable work.
Many companies, including GoPro, Nokia, Samsung, Ricoh and Nikon, develop omnidirectional cameras, also known as 360-degree cameras or VR cameras, that have the ability to record in all directions. These cameras are used to create images and videos that can be viewed in VR. (See VR photography.) Films produced for VR permit the audience to view the entire environment in every scene, creating an interactive viewing experience. Production companies, such as Fox Searchlight Pictures and Skybound, utilize VR cameras to produce films that are interactive in VR. Fox Searchlight, Oculus and Samsung Gear VR collaborated on a project titled “Wild – The Experience”, starring Reese Witherspoon. The VR film was presented at the Consumer Electronics Show as well as the Sundance Film Festival in January 2015. On December 8, 2015, the production company Skybound announced their VR thriller titled “Gone”. In collaboration with the VR production company WEVR, and Samsung Gear VR, the 360-degree video series was released on January 20, 2016.
Media companies such as Paramount Pictures, and Disney have applied VR into marketing campaigns creating interactive forms of media. In October 2014 Paramount Pictures, in collaboration with the media production company Framestore, created a VR experience utilizing the Oculus DK2. The experience was dubbed a “time sensitive adventure in space” that took place in a portion of the Endurance space ship from the film “Interstellar.” The experience was available to the public at limited AMC theater locations. In May 2016, Disney released a VR experience titled Disney Movies VR on Valve Corporation’s Steam software, free for download. The experience allows users to interact with the characters and worlds from the Disney, Marvel, and Lucasfilm universes.