Hand-held computing devices are ubiquitous and have become part of our lives these days. Moreover, hand-held devices are also increasingly being equipped with special sensors and non-traditional displays. As such, it raises the question of whether such a small and reduced device could serve as an effective virtual reality (VR) platform and provide sufficient immersion and presence, e.g. through multimodal interaction. In this research, we address this question by adding multimodal functionalities such as motion based interaction, user-display dependent rendering, autostereoscopic small display, vibrotactile feedback and etc. Also we show the feasibility of hand-held virtual reality by comparing with other VR systems.

Selected Papers:

• Hwang, J. Jung, J., Kim, G. J.(2006). " Manipulation of Field of View for Hand-held Virtual Reality" 16th International Conference on Artificial Reality and Telexistence (accepted)(LNCS).



• Jeon, S. Hwang, J., Kim, G. J.(2006). " Interaction Techniques in Large Display Environments using Hand-held Devices" ACM VRST 2006 (accepted).



• Hwang, J., Jung, J., Kim, G. J.(2006). " Hand-held Virtual Reality: A Feasibility Study" ACM VRST 2006 (accepted).



• Hwang, J., Jung, J., Kim, N., Kim, G. J.(2006). " Designing and Evaluating a Hand-held Virtual Reality System" The 4th International Symposium on Ubiquitous VR.



• Hwang, J., Kim, G. J., Kim, N.(2006). " Camera based Relative Motion Tracking for Hand-held Virtual Reality" NICOGRAPH International 2006 .Seoul, South Korea, June 2-3. 2006

In a collaborative system, the level of co-presence, the feeling of being with the remote participants in the same working environment, is very important for natural and efficient task performance. One way to achieve such copresence is to recreate the participants as real as possible, for instance, with the 3D whole body representation. In this research, we introduce a method to recreate and immerse tele-operators in a collaborative augmented reality (AR) environment. The method starts with capturing the 3D cloud points of the remote operators and reconstructs them in the shared environment in real time. In order to realize interaction among the participants, the operator's motion is tracked using a feature extraction and point matching (PM) algorithm. With the participant tracking, various types of 3D interaction becomes possible.

Selected Papers:

• Hwang, J., Kim, N., Kim, G. J.(2006). " Immersing Tele-operators in Collaborative Augmented Reality" International Symposium on Visual Computing 2006 (LNCS).

The advantages that using VR techniques for rehabilitation are easy of development, easy of modification, safeties and etc. In this research, especially we proposed the system for 3D motion rehabilitation with multimodal sensory effects such as vibro-tactile feedback, motion based interaction and etc.

Selected Papers:

• Feintuch, U., Raz, Liat., Hwang, J., Jang, Y., Josman, N., Katz, N., Kizony, R., Rand, D., Rizz, A.A., Shahar, M. and Weiss, P.L.(2006). " Integrating haptic-tactile feedback into a video capture based VE for rehabilitation" CyberPsychology & Behavior. Apr 2006, Vol. 9, No. 2: 129-132



• Feintuch, U., Raz, Liat., Hwang, J., Jang, Y., Josman, N., Katz, N., Kizony, R., Rand, D., Rizz, A.A., Shahar, M. and Weiss, P.L.(2005). " Integrating haptic-tactile feedback into a video capture based VE for rehabilitation" 4th International Workshop on Virtual Rehabilitation . Catalina Island, CA. USA., Sep 19-21. 2005



• Rizzo, A.A., Kim, G.J., Yeh S. C., Thiebaux, M., Hwang, J., Buckwalter, G.(2005). " Development of a Benchmarking Scenario for Testing 3D User Interface Devices and Interaction Methods ," The Proceedings of the 11th International Conference on Human Computer Interaction. Las Vegas, USA, July 22-27. 2005.



• Hwang, J., Kim, G. J., Rizzo, A.A.(2004). " Space Extension: The Perceptual Presnece Perceptive." ACM SIGGRAPH International Conference on VRCAI . Singapore, June 16-18. 2004



• Rizzo, A.A., Buckwalter, J.G., Bowerly, T., Yeh, S., Hwang, J., Thiebaux, M. & Kim, G. J. (2004). "Virtual Reality Applications for Assessment and Rehabilitation of Cognitive and motor Processes. " 15th Congress of the International Society of Electrophysiology & Kinesiology. Boston MA June 18-21. 2004

Recent progress in digital music has enabled some form of active music appreciation. In particular, with the advent of affordable VR technologies, many researchers are experimenting with the three-dimensional and multimodal interfaces to control music and replace the current keyboard/mouse-based (piano or computer) interface for computer/electronic music. Given such a background, we proposes and demonstrates the concept of the Virtual Music Environment (VME), a generalization of a virtual music instrument, in which the virtual world itself acts as a source of multimodal (e.g. visual, audio, haptic) feedback, and at the same time, a place of interaction. more

Selected Papers:

• Hwang, J., Kim, G. J., " Design and Analysis of Virtual Musical Environment ," Proc. of International Computer Music Conference, Berlin, Germany, 2000.



• Jung, B., Hwang, J., Kim, G. J., Lee, E., H, Kim. " Incoporating Copresence in Distributed Virtual Music Environment ," ACM Symposium on Virtual Reality Systems and Technology, 2000.



• Kim, G. J., Hwang, J. " Musical Motion: A Medium for Uniting Visualization and Control of Music in the Virtual Environment ," International Conference on Virtual Systems and MultiMedia 1999.