3D User Interfaces: Theory and Practice - Hardcover

Doug A. Bowman is an associate professor of computer science at Virginia Tech.

Ernst Kruijff is a senior researcher at the Graz University of Technology.

Joseph J. Laviola, Jr., is an assistant professor of computer science at the University of Central Florida.

Ivan Poupyrev is a researcher at the Interaction Laboratory, Sony Computer Science Labs in Tokyo.

An architect sits in her home office, putting the final touches on the design of the new entrance to the city park. A three-dimensional virtual model of the park appears in front of her on the desk’s surface. She nudges a pathway slightly to the right to avoid a low-lying area, and then makes the model life-size so she can walk along the path to view the effect. “Those dark colors on the sign at the entrance are too foreboding,” she thinks, so she quickly changes the color palette to brighter primary colors. She looks up and notices that the clients are arriving for the final design review meeting. They are located in other offices around the city, but they can all view the 3D model and make suggested changes, as well as communicate with one another. “What’s the construction plan?” asks one of the clients. The architect starts an animation showing the progress of the project from start to finish. “That first step may not work,” says the client. “The excavation is much too close to the existing playground. Let me show you.” He looks out his window, which has a view of the park, and overlays the virtual construction plan on it. “You’re right,” says the architect, “let’s plan to move the playground slightly—that will be much cheaper than changing the construction site.” After viewing the effects of the change, all agree that this plan will work, and the meeting adjourns.

This scenario and others like it illustrate the enormous potential of 3D environments and applications. The technology to realize such a vision is available now, although it will certainly be improved. But the scenario also leaves out a great deal of information—information that is crucial to making this dream a reality. How did the architect load the park model, and how does she manipulate her view of it? What technique is used to change the pathway? How can multiple clients all manipulate the model at the same time? How do the participants appear to each other in the virtual space? How is the speed and playback of the animation controlled? How did the client instruct the system to merge the real and virtual scenes?

These questions all relate to the design of the user interface (UI) and interaction techniques for this 3D application, an area that is usually given only a cursory treatment in futuristic films and books. The scenarios usually either assume that all interaction between the user and the system will be “natural”—based on techniques like intuitive gestures and speech—or “automatic”—the system will be so intelligent that it will deduce the user’s intentions. But is this type of interaction realistic, or even desirable?

This book addresses the critical area of 3D UI design—a field that seeks to answer detailed questions, like those above, that make the difference between a 3D system that is usable and efficient and one that causes user frustration, errors, and even physical discomfort. We present practical information for developers, the latest research results, easy-to-follow guidelines for the UI designer, and relevant application examples. Although there are quite a few books devoted to UIs in general and to 2D UI design in particular, 3D UIs have received significantly less attention. The results of work in the field are scattered throughout numerous conference proceedings, journal articles, single book chapters, and Web sites. This field deserves a reference and educational text that integrates the best practices and state-of-the-art research, and that’s why this book was created.

How This Book Came to Be

The story of this book begins in April 1998, when Ivan Poupyrev and Doug Bowman were doctoral students at Hiroshima University and Georgia Tech respectively, working on 3D interaction techniques for object manipulation in virtual environments (VEs). We started a lively email discussion about the design and usability of these techniques and about 3D UIs in general. Ivan, who was at the time a visiting research student at the University of Washington, suggested that the discussion would be even more profitable if other researchers in this new area could join in as well, and so the 3DUI mailing list was born. Since that time, over 100 researchers from around the globe have joined the list and participated in the discussion (to see an archive of all the list traffic or to join the list, check out http://www.3dui.org). Joe LaViola and Ernst Kruijff were two of the first people to join the list.

In August of that same year, Doug forwarded to the list a call for tutorials for the upcoming IEEE Virtual Reality Conference. After some discussion, Joe, Ivan, and Ernst agreed to join Doug to organize a tutorial on “The Art and Science of 3D Interaction.” The tutorial was a big hit at the conference in Houston, and the four of us continued to present courses on the topic at ACM Virtual Reality Software and Technology 1999, IEEE VR 2000, and ACM SIGGRAPH 2000 and 2001.

After developing a huge amount of content for the notes supplements of these courses, we decided it would be silly not to compile and expand all of this information in book form. Furthermore, there was no way to include all the information available on 3D UIs in a one-day course. And that’s why you’re holding this book in your hands today—a book containing information on 3D UIs that can’t be found in any other single source.

What’s in the Book

The title of this book emphasizes that we have written it for both academics/researchers and practitioners/developers—both those interested in basic research and those interested in applications. Most chapters of the book integrate both theory and practical information. We intend the book to be used both as a textbook (see suggestions below) and as a reference work.

Theory-related content includes the following:

• Sections on the psychology and human factors of various 3D interaction tasks
• Information on different approaches for the evaluation of 3D UIs (Chapter 11)
• Results from empirical studies of 3D interaction techniques
• A research agenda for 3D interaction (Chapter 13)
• Lists of recommended further reading at the end of most chapters
• A comprehensive bibliography of important research articles

Practice-related content includes the following:

• Principles for choosing appropriate input and output devices for 3D systems (Chapters 3 and 4)
• Details and helpful tips for the implementation of common 3D interaction techniques
• Guidelines for the selection of interaction techniques for common 3D tasks
• Case studies of 3D UIs in real-world applications

The book is organized into five parts. Part I introduces the topic of 3D UIs. Part II discusses the input and output device technology used in the development of 3D UIs, with an emphasis on the impact of these devices on usability and performance. Part III presents a wide range of 3D interaction techniques for the common tasks of navigation, selection and manipulation, system control, and symbolic input. In Part IV, we discuss the design, development, and evaluation of complete 3D UI metaphors and applications. Finally, Part V considers the future, with chapters on 3D interaction in augmented reality applications and a research agenda for 3D UIs. The appendix includes information on required mathematical background and is followed by a bibliography of 3D UI references.

Throughout the book, we offer several special features. First, most chapters contain numerous guidelines—practical and proven advice for the designer and developer. We also include implementation details for many of the most common and useful interaction techniques. We describe these algorithms using a combination of textual and mathematical descriptions (to avoid a bias toward any particular development tool or programming style).

How to Use the Book and Related Material

If you are a 3D UI developer: Professional developers can use the book for inspiration and guidance in the design, implementation, and evaluation of applications with 3D UIs. In the design process, developers can consider overall UI metaphors from Part IV, choose specific interaction techniques from Part III, and match these with appropriate input and display devices from Part II. The design guidelines from all of these sections should help developers make rational, informed decisions. The implementation of the 3D UI can benefit from the textual and mathematical descriptions of interaction techniques we provide in Part III. Finally, developers can choose evaluation methods and assess the usability of their applications based on the information in Chapter 11.

If you are a teacher: The book can also be used as a textbook in several different types of university-level courses. A graduate course on 3D UI design could use it as a primary textbook. A more generic virtual environments course could use Parts I, II, and III of this book as an introduction to the basic technology and techniques used in VE interaction. An undergraduate HCI course could pull information from Parts I and IV in a module on 3D interfaces and their differences from traditional UIs. Implementation of common techniques from Part III could enhance a course on interactive 3D graphics.

If you are a researcher: This book can serve as a comprehensive reference guide for researchers engaged in 3D UI design or evaluation, the investigation of 3D applications, or the use of VEs or augmented reality. The research agenda in Chapter 13 also provides researchers and research students with a list of important questions to be addressed in the field. It could even be used as the starting point for a PhD student looking for a topic related to 3D UIs.

3D UI design is a fast-moving and evolving field. Therefore, we are committed to updating the material in this book. One way we will do this is through the book’s official Web site at http://www.3dui.org. This site will contain information and links related to the latest 3D UI research and applications, organized in the same manner as the book so you can easily find new information about the topics in a particular part or chapter. The site will also allow you to join the 3DUI mailing list. We also ask for your help in keeping the book up to date. Send us your comments, clarification questions, or links to additional information by visiting the Web site above and using the online feedback form. Or email us directly at 3dui@3dui.org. Your comments will help us update the Web site, as well as future editions of this book.