Exertion Interfaces talk from Floyd Mueller

Today, I attended a talk from Floyd Mueller who is visiting researcher at Microsoft Research Asia and is a PhD Candidate at the University of Melbourne. He is talking about exertion games.

Abstract

Exertion Interfaces are interfaces that require intense physical effort. Their recent success stems mostly from their use in games, as demonstrated by the Wii and Natal. It is believed that these games facilitate not only physical health, but also more social play than traditional computer games. Over several years, Floyd has investigated the influence of technology design on exertion games. His research has contributed to an understanding of how people play computationally-augmented exertion games and how we can support exertion and social play in future designs. Floyd's research has produced several prototypes, including 'Jogging over a Distance', a system that allows joggers who are located in different cities to motivate one another while running using mobile technology, which was used last month between expat joggers in London, UK and Melbourne, Australia. Floyd will focus on the 'Jogging over a Distance' work to describe challenges and opportunities in using mobile technology to enhance people's fitness activities through a distributed social approach that aims to enhance the engagement with the activity while it occurs, unlike most pedometer work that focuses on post-reflection after the exercise.

Floyd is talking about how Dance Dance Revolution and the Nintendo Wii really changed the face of gaming using physical activity. There is an exciting research for engaging people over the internet using physical activity. For example, a group of cyclists come together and can connect with and cycle with other cyclists over the internet using Skype video conference. How to do the design between exertion play and social play? According to Floyd, we need to focus on mediated environment which can extend opportunities for social interaction.

The first research project was on Table Tennis for Three which involved a table tennis table with one side showing a screen and video conference to show the remote players. A player uses the table tennis racquet to hit the ball onto the screen which the other player on the other side will see and will hit the ball.

The second research project was Jogging Over a Distance, using spatial audio for showing whether your partner in jogging is close to you. Using a heart rate monitor, joggers can then speed up depending on the other partner in order to catch up to that person based on the spatial audio.

The third research project was Remote Boxing, which is boxing over a distance where users would hit the screen and get points for doing so when they saw objects appear on the screen.

All in all, very interesting research work and something that really brings HCI and ubiquitous computing to real every day life.

China HCI seminar series in Beijing - First session

China HCI Seminar Series
October 26, 2009

This is what I wrote from the first session in the China HCI Seminar Series which kicked off last night. This seminar series is meant to foster research and collaboration among HCI researchers in the Chinese community as well as abroad. It was held at the Institute of Software, Chinese Academy of Sciences. There was lots of people for the first session, so the turn out was really great like a large lecture hall! There were two talks after introduction by the organizers and distinguished Chinese HCI researchers.








Prof. James Landay, University of Washington, Visiting Researcher at Microsoft Research Asia

Jonathan Grudin: A Moving Target: The Evolution of HCI

Keynote Speech

HCI started with Human Factors and Ergonomics with operation and data entry from 1905 to 1945, and then came the invention of general-purpose computers. HCI was one of five major IS research streams since 1967 (Banker and Kaufmann). Then there was HCI in Information Systems for managerial use. Finally Computer-Human Interaction happened in the 1980s with the beginning of the CHI conference and the establishment of SIGCHI. There was a focus shift from non-discretionary use to discretionary use. The Computer-Human Interaction stream started from the work of computer-engineer interaction and the work done at PARC (Palo Alto Research Center). The actual goal behind human-computer interaction actually started with the work of Grace Hopper.

According to Jonathan, there is a shifting focus of interface development from the large hardware mainframes, to the PCs, and now in groups and mobile devices where it is used collaboratively. The graphical user interface (GUI) changed the field of human-computer interaction in 1985 with the Apple Mac computer. Jonathan showed a curve of Moore’s Law showing how as you move the number of years, there is a longer tail on the curve. The next big impact was in 1995 where there was unanticipated changes in audio, video and graphics. Now, if we continue the curve, what is the next big bomb and it is predicted that it will happen in 2015. Design started coming into HCI since 1995, but we have been neglecting design for a decade or so. There is now new research trend into emotional design.


History and evolution of HCI


Interface development

The amount of information will dramatically increase exponentially and increasingly significant focus in HCI. H.G. Wells made a quote which sounds very much related to what the vision of human-driven Web 2.0 systems is, and this was in 1905! There is now the emergence of information schools. James Martin from 1973 talked about the era of information scarcity and according to Jonathan, we are still in the age of information scarcity. Now, there is a merging of the physical and virtual world. We can learn something from the kids domain like WebKinz where kids can buy a stuffed toy, enter the code, and then can enter the virtual world where they have to feed their toy and can share with other kids’ toys. He says why this can’t be the same in the adult world like buying a car?



Desney Tan: Creating Novel Human-Computer Interaction with Physiological Sensing
Computational User Experiences Group, Microsoft Research
Desney's web site


Following Jonathan’s footsteps, the next big thing in the evolution of computing paradigms is natural user interfaces. Mind reading devices are now hitting the consumer market, and is now becoming a science. At Microsoft Research, Desney and his team is looking into classifying brain activity tasks and uncontrolled game tasks with greater than 80% accuracy. The goal is to have only a few sensors as possible with high accuracy. They can use the brain sensors to accurately classify images based on brain activity using EEG. Another goal is to take science fiction like Minority Report as reality, and use detection of hand gesture recognition. At Microsoft Research, there is Project Natal which is controller free interaction which will be shipped with Microsoft XBox. In order to make this reality, they use EMG armbands to use muscle stimulation in order to detect gestures. You can sense gestures on a hard surface like a table top using Microsoft Surface.

Games are a very good test of the research because they require accurate classification and fast classification. Their group created a test application called Air Guitar Hero that uses hand gestures to pretend playing the guitar. A third research theme is mouth gesture interactions for example using tongues that can be used with paralyzed people who cannot use their hands or arms. Desneyäs group created a device to be placed on the tongue with electrodes in order to sense the tongue with a micro+controller. The idea is that some children wear retainers when they are young but a reluctant to use them because they are not nice to have and make them look ugly. But if they were cool by using sensors to make them technologically advanced to use, would help adoption of them. An example is using your tongue to control a Tetris game. All the technology is now embedded in the tongue retainer. So what are some applications of tongue gesture interaction? According to Desney, we can use tongue gestures to control mobile devices like for example building a music player within your mouth. Another application is in medical sensing for salivary analysis and food analysis.

The last project is Bionic Contact Lenses where technology is embedded in the contact lens. An application could be to project information into the eye, for example, remembering who a particular person is by looking at that person. Another application is medical sensing such as glucose sensing with bionic contacts.

In closing, Desney is talking about the next evolution of HCI that will help bring computing into the real world.



At the end, there was some questions addressed to Desney and Jonathan.



All in all, a great start to what looks to be a great lecture series on HCI in Beijing, something that I miss from my alma mater at University of Toronto.

DGPis40 Abigail Sellen talk

This is the second talk that I attended at the DGPis40 reunion conference which was by Abigail Sellen, Senior Researcher and Microsoft Research Cambridge.

Abigail talked about her stints at DGP as a post-doc where in the early '80s and '90s, she worked on Bill Buxton's Ontario Telepresence project. She is now at Microsoft Research Cambridge in the UK working on socio-digital systems and her talk is "Being Human in the Digital Age". Her research deals with how to bring humans into the loop of computer technologies and assist the human, a human-centered approach to computing which I think is much needed in our industry. She has 23 patents and she says that we have to think and change our methodologies towards HCI. We need to put a humanistic agenda in research.

In the past, the origins of HCI was about thinking the brain as an information processing system and using cognitive psychology (in the '70s and '80s) and human factors engineering. Then in the early '80s, Don Norman talked about HCI and Card, Moran and Newell (1984) wrote a book called the Psychology of HCI. The idea here was to use cognitive psychological models to improve the use of computers. These led to the following contributions to HCI: User-centered design, user-testing and modelling.

Now, we are in the age of ubiquitous computing as visioned by the late Mark Weiser from PARC. An example of this is the Augmented Reality kitchen from the MIT Media Lab. The problem with HCI is that we need a view of the user and understand users as cognitive machines, therefore we need to have multi-disciplinary terms and design goals for rich evolving ecosystems and practices. As a result of this, Abigail and her colleagues created a Whereabouts clock which allows each member of the family to see where others are (either work, school, or out) based on their location. They conducted a user study that showed that family members got reassurance of where everyone was.



In their research, they have opened up the view of the user and thought of new information appliances that they would have not thought before if there was not any user testing. Opening up to the user can lead to invention. Another example that they created is the digital postcard where people can send SMS of pictures and text and it will show on the postcard.



Another example is the kitchen postcard which they created which becomes an ambient display and blog space. Think of it like a digital version of putting stuff on the fridge.



They also created a visual answering machine called Bubbleboard where the size of the bubbles indicates the length of the message.



According to Abigail, long term studies in real homes can give new insight into value of technologies for people. Abigail says that we need to redefine the H,C and I in HCI. Can technologies help us to switch off, forget, be isolated, be engaged (the opposite of what technologies do for us but thinking about what it means to be human). We are so increasingly connected, but sometimes we just need to unwind and not be connected, we need a break. So this raises new questions for research and development. Therefore, we need an HCI based on human values.

Abigail and a bunch of other researchers explore this reincarnation of HCI (in my opinion) with their published document from their workshop called Being Human:Human Computer Interaction in the Year 2020 which can be downloaded. I'll for sure be reading this because my research is very much into this area with social networks and how we can bring social networks to mobile and ubiquitous computing.

Tapan Parikh talk at CS at U of T

Tapan is giving a talk about Designing Appropriate Computing Technologies for the Rural Developing World, he is from the University of Washington.

Here is his abstract and bio:

Globalization has seen an increase in disparity between developed and
undeveloped regions. Disproportionate access to information
technology is a symptom and a factor contributing to this disparity.
In particular, people living in the rural developing world have many
information needs that could, but are not, being met by IT.
Technology for this context must be low-cost, accessible and
appropriate given the local infrastructure, including conditions of
intermittent power and connectivity. In this talk, I describe my
experiences developing CAM - a toolkit for mobile phone data
collection for the rural developing world. Designing technologies for
an unfamiliar context requires understanding the needs and
capabilities of potential users. Drawing from the results of an
extended design study conducted with microfinance group members in
rural India (many of whom are semi-literate or illiterate), I outline
a set of user interface design guidelines for accessibility to such
users. The results of this study are used to inform the design of
CAM, a mobile phone application toolkit including support for
paper-based interaction; multimedia input and output; and
disconnected operation. I provide evidence of CAM's usability,
breadth, and real-world applicability. Regarding real-world
applicability, a CAM application for microfinance data collection is
now being used by 17 NGO (non-governmental organization) to serve
over 10000 group members in two states of India. Regarding breadth, I
list some important rural data collection applications - including
for retail supply chain tracking, agricultural monitoring and health
care - that we have implemented, or can be implemented, using the CAM
toolkit. I conclude by discussing possible topics for future work and
my long-term research vision.

Bio: Tapan S. Parikh is an Intel Fellow and Ph.D. Candidate in the
Department of Computer Science and Engineering at the University of
Washington. Earlier, he received a M.S. degree in Computer Science
from UW and a Sc.B. degree with Honors in Molecular Modeling from
Brown University. Tapan's research interests include human-computer
interaction (HCI), systems engineering and information and
communication technologies for development (ICTD).

This talk is interesting to me because it has great relevance to applying technology to the rural developing world, that the urban communities take for granted. How can we solve the problems in rural areas using a mobile phone solution that deals with understanding context. The first part is understanding the context through a study that he is talking about how to provide financial services to the poor using computer technology. This study was done in India. Information can be the bridge between the formal and the informal. There is a need to design a system to make it accessible to users, and geographic. How to design a user interface for rural users who are semi-literate or illiterate? There was a test with a group of users, where the group hired a person to record the data on paper and pen. Then the paper representation was then reproduced as a software prototype on a laptop. For user response, there was a wide gap for looking at the computer and how to use the mouse to move around the screen. So in his design, he used icon buttons. Users then started to gain confidence after playing around with the user interface and clicking on buttons which would output in local language audio. I found that pretty interesting that illiterate users were able to work with the system, I can't even get my parents to try to use a computer!

The second part is to actually build the system. The solution that he used was the mobile phone which has a numeric keypad, speakers and microphone, is battery-operated and low cost. The HCI research community uses paper user interfaces for prototyping and leverage affordances of paper in digital user interfaces. But these approaches have had limited impact, and rural developing world may be the killer application for paper user interfaces. He created CAM, an application toolkit for mobile phones which includes a CAM browser and CAM scripting language to interact with the forms. The phone is used to capture the audio and images from paper, and then can review it on the phone for the user. The paper form has specific images that are captured and is associated with a particular action. Here's the paper of this that he presented at UIST 2005. This paper form that has special images which is captured by the phone, reminds me of the work by Intel Research Cambridge UK that dealt with capturing concentric circles on paper (like bar codes) and those correspond to performing a particular action. This is pretty neat, then there is no need to have to deploy something like RFID tags, because this is a low cost solution, and all phones now include a camera.

So how does this work in the field? He is now explaining about how he evaluated CAM. His results showed that the users performed significantly better with audio than textual prompts. The system has been deployed in India and commercialized by Ekgaon Technologies. CAM can also be applied to other field areas like agricultural monitoring.

His future work is looking into building a toolkit on top of CAM that allows local people to build their own solutions, and provide them with the tools and application development resources. The rural development using computer technology is now a hot area in many research institutions like UC Berkeley, Princeton/UW/MSR, MSR India, MIT Media Lab (with Nicholas Negroponte's One Laptop per Child).

This is very interesting work, and he mentioned this motivates students and others to work on this to contribute to solving real world problems.

On Technorati: CAM, rural, cell phone, field work, HCI