China HCI seminar series in Beijing - First session

China HCI Seminar Series
October 26, 2009

This is what I wrote from the first session in the China HCI Seminar Series which kicked off last night. This seminar series is meant to foster research and collaboration among HCI researchers in the Chinese community as well as abroad. It was held at the Institute of Software, Chinese Academy of Sciences. There was lots of people for the first session, so the turn out was really great like a large lecture hall! There were two talks after introduction by the organizers and distinguished Chinese HCI researchers.








Prof. James Landay, University of Washington, Visiting Researcher at Microsoft Research Asia

Jonathan Grudin: A Moving Target: The Evolution of HCI

Keynote Speech

HCI started with Human Factors and Ergonomics with operation and data entry from 1905 to 1945, and then came the invention of general-purpose computers. HCI was one of five major IS research streams since 1967 (Banker and Kaufmann). Then there was HCI in Information Systems for managerial use. Finally Computer-Human Interaction happened in the 1980s with the beginning of the CHI conference and the establishment of SIGCHI. There was a focus shift from non-discretionary use to discretionary use. The Computer-Human Interaction stream started from the work of computer-engineer interaction and the work done at PARC (Palo Alto Research Center). The actual goal behind human-computer interaction actually started with the work of Grace Hopper.

According to Jonathan, there is a shifting focus of interface development from the large hardware mainframes, to the PCs, and now in groups and mobile devices where it is used collaboratively. The graphical user interface (GUI) changed the field of human-computer interaction in 1985 with the Apple Mac computer. Jonathan showed a curve of Moore’s Law showing how as you move the number of years, there is a longer tail on the curve. The next big impact was in 1995 where there was unanticipated changes in audio, video and graphics. Now, if we continue the curve, what is the next big bomb and it is predicted that it will happen in 2015. Design started coming into HCI since 1995, but we have been neglecting design for a decade or so. There is now new research trend into emotional design.


History and evolution of HCI


Interface development

The amount of information will dramatically increase exponentially and increasingly significant focus in HCI. H.G. Wells made a quote which sounds very much related to what the vision of human-driven Web 2.0 systems is, and this was in 1905! There is now the emergence of information schools. James Martin from 1973 talked about the era of information scarcity and according to Jonathan, we are still in the age of information scarcity. Now, there is a merging of the physical and virtual world. We can learn something from the kids domain like WebKinz where kids can buy a stuffed toy, enter the code, and then can enter the virtual world where they have to feed their toy and can share with other kids’ toys. He says why this can’t be the same in the adult world like buying a car?



Desney Tan: Creating Novel Human-Computer Interaction with Physiological Sensing
Computational User Experiences Group, Microsoft Research
Desney's web site


Following Jonathan’s footsteps, the next big thing in the evolution of computing paradigms is natural user interfaces. Mind reading devices are now hitting the consumer market, and is now becoming a science. At Microsoft Research, Desney and his team is looking into classifying brain activity tasks and uncontrolled game tasks with greater than 80% accuracy. The goal is to have only a few sensors as possible with high accuracy. They can use the brain sensors to accurately classify images based on brain activity using EEG. Another goal is to take science fiction like Minority Report as reality, and use detection of hand gesture recognition. At Microsoft Research, there is Project Natal which is controller free interaction which will be shipped with Microsoft XBox. In order to make this reality, they use EMG armbands to use muscle stimulation in order to detect gestures. You can sense gestures on a hard surface like a table top using Microsoft Surface.

Games are a very good test of the research because they require accurate classification and fast classification. Their group created a test application called Air Guitar Hero that uses hand gestures to pretend playing the guitar. A third research theme is mouth gesture interactions for example using tongues that can be used with paralyzed people who cannot use their hands or arms. Desneyäs group created a device to be placed on the tongue with electrodes in order to sense the tongue with a micro+controller. The idea is that some children wear retainers when they are young but a reluctant to use them because they are not nice to have and make them look ugly. But if they were cool by using sensors to make them technologically advanced to use, would help adoption of them. An example is using your tongue to control a Tetris game. All the technology is now embedded in the tongue retainer. So what are some applications of tongue gesture interaction? According to Desney, we can use tongue gestures to control mobile devices like for example building a music player within your mouth. Another application is in medical sensing for salivary analysis and food analysis.

The last project is Bionic Contact Lenses where technology is embedded in the contact lens. An application could be to project information into the eye, for example, remembering who a particular person is by looking at that person. Another application is medical sensing such as glucose sensing with bionic contacts.

In closing, Desney is talking about the next evolution of HCI that will help bring computing into the real world.



At the end, there was some questions addressed to Desney and Jonathan.



All in all, a great start to what looks to be a great lecture series on HCI in Beijing, something that I miss from my alma mater at University of Toronto.

Social networking researcher danah boyd joining Microsoft Research

Just read this from the SOCNET mailing list which was sent by Barry Wellman. danah boyd (yes the lower case spelling is intentional), who is known for studies on Friendster and MySpace, and studying the youth in using these social networking services and acting as advisor to many companies and in politics, is going to join Microsoft Research in Boston in January 2009. danah wrote up about this on her own blog, Zephoria.

I think this is great for her and for Microsoft. She mentions how she will be directing her own research and how MSR is a great fit for her after she met with the MSR folks. Congratulations danah and keep us posted with your research.

And yes, I understand about the dissertation because I'm also in the same boat with finishing up the changes so I'll be ready for the defense.

Social networking comes to Microsoft Excel

Marc Smith's Community Technologies research group at Microsoft Research has just released .NetMap which is a plug-in to do social network analysis in Microsoft Excel. This was presented at the Microsoft Faculty summit this week. Just import your data into Excel and let .NetMap visualize the social network, calculate centrality, and do all the other stuff that UCINET, Pajek and NetDraw do. Gone are the gzillion steps that you have to do in order to get the visualization of the social network (in my PhD thesis, this involved MySQL to get the actual data in a table format, creating a DL file to import into UCINET, then taking the file and converting it into Pajek to use Pajek features, or visualizing it in NetDraw).

More details of .NetMap are below:

New tools from Microsoft Research make performing many basic network
manipulations and visualization tasks as simple as using Excel. The (Excel)
.NetMap add-in provides directed graph charting features within Excel,
allowing users to create node-link diagrams with control over each node and
edge color, size, transparency and shape. Since .NetMap builds within Excel,
all of the controls and programmatic features of Office are available.
Additional features of (Excel) .NetMap generate social networks from data
sources like personal e-mail (drawing data from the Windows Desktop Search
engine). Arbitrary edge lists (anything that can be pasted into Excel) can
be visualized and analyzed in .NetMap. To download the Excel .NetMap Add-in
and slides, go to the following Web site in CodePlex (Microsoft's version of SourceForge).

I'm going to try it out and see how it works. Of course this could have saved me time in my PhD research analysis!

On Technorati: .NetMap, Microsoft Research, Excel

DGPis40 Abigail Sellen talk

This is the second talk that I attended at the DGPis40 reunion conference which was by Abigail Sellen, Senior Researcher and Microsoft Research Cambridge.

Abigail talked about her stints at DGP as a post-doc where in the early '80s and '90s, she worked on Bill Buxton's Ontario Telepresence project. She is now at Microsoft Research Cambridge in the UK working on socio-digital systems and her talk is "Being Human in the Digital Age". Her research deals with how to bring humans into the loop of computer technologies and assist the human, a human-centered approach to computing which I think is much needed in our industry. She has 23 patents and she says that we have to think and change our methodologies towards HCI. We need to put a humanistic agenda in research.

In the past, the origins of HCI was about thinking the brain as an information processing system and using cognitive psychology (in the '70s and '80s) and human factors engineering. Then in the early '80s, Don Norman talked about HCI and Card, Moran and Newell (1984) wrote a book called the Psychology of HCI. The idea here was to use cognitive psychological models to improve the use of computers. These led to the following contributions to HCI: User-centered design, user-testing and modelling.

Now, we are in the age of ubiquitous computing as visioned by the late Mark Weiser from PARC. An example of this is the Augmented Reality kitchen from the MIT Media Lab. The problem with HCI is that we need a view of the user and understand users as cognitive machines, therefore we need to have multi-disciplinary terms and design goals for rich evolving ecosystems and practices. As a result of this, Abigail and her colleagues created a Whereabouts clock which allows each member of the family to see where others are (either work, school, or out) based on their location. They conducted a user study that showed that family members got reassurance of where everyone was.



In their research, they have opened up the view of the user and thought of new information appliances that they would have not thought before if there was not any user testing. Opening up to the user can lead to invention. Another example that they created is the digital postcard where people can send SMS of pictures and text and it will show on the postcard.



Another example is the kitchen postcard which they created which becomes an ambient display and blog space. Think of it like a digital version of putting stuff on the fridge.



They also created a visual answering machine called Bubbleboard where the size of the bubbles indicates the length of the message.



According to Abigail, long term studies in real homes can give new insight into value of technologies for people. Abigail says that we need to redefine the H,C and I in HCI. Can technologies help us to switch off, forget, be isolated, be engaged (the opposite of what technologies do for us but thinking about what it means to be human). We are so increasingly connected, but sometimes we just need to unwind and not be connected, we need a break. So this raises new questions for research and development. Therefore, we need an HCI based on human values.

Abigail and a bunch of other researchers explore this reincarnation of HCI (in my opinion) with their published document from their workshop called Being Human:Human Computer Interaction in the Year 2020 which can be downloaded. I'll for sure be reading this because my research is very much into this area with social networks and how we can bring social networks to mobile and ubiquitous computing.

Rick Rashid from Microsoft Research talk @ U of T

I’m in the talk with Microsoft Research’s senior VP Rick Rashid and he’s talking about how he started his research career as a graduate student, then professor at Carnegie Mellon University, and then his stint at Microsoft Research. MSR’s mission statement is to expand the state of the art in each of the areas in which they do research, rapidly transfer innovative technologies into Microsoft products, and ensure that Microsoft products have a future. MSR is adapting the academic model, they are an open research environment and have strong ties to University Research. 25% of all PhD graduates will have worked at MSR at some time during their PhD. MSR has just announced a 6th lab in Cambridge, MA to be close to MIT which will open in July 2008. MSR has almost 1000 interns a year and many postdocs. The key mission is to move the state-of-the-art forward and measure themselves with publications, MSR has over 3700 peer-reviewed publications, 13% at 2001 CHI. MSR has prestigious senior researchers like Gordon Bell, creator of the VAX and Rick Rashid with work on the NUMA architecture.

The second part of the mission statement is to drive technologies into Microsoft products and Rick showed that with the Tablet PC which was invented in Cambridge, UK and the codecs for Microsoft Windows Media Player. What’s the value of MSR to Microsoft? MSR is a source of IP and new technologies and to act as an early warning system to Microsoft as to what technologies and areas to focus on. A basic research group allows a company to respond more rapidly to change and to solve hard technical problems which advances the company and makes it one step above the competitors. If you want to survive, you have to invest in the future. MSR is also delving into research areas that you wouldn’t think Microsoft would be in, like for example, computational biology.

Rick is now talking about what the future will look like in 10 years. Rick gave a slide about the "Wallet PC" vision in 1993 where now this has developed in Windows Mobile. So, what will the future to 2018 will look like? Microsoft Surface is a tabletop system where you can manipulate digital objects as physical objects, kind of similar to tabletop work from DGP at University of Toronto. A new large research project at MSR is the singularity research project for proving very large systems and doing a better job of specification for software. Now, the specifications are actionable and can be verified through proofs that will change the software development process. Another project which became an actual product is TerraServer which was one of the earliest terabyte servers on the web, and was the basis for Microsoft Virtual Earth, and was started by Jim Gray (who is missing since last year from on a sailing trip). This delves into data mining and web services area.

One cool thing that Rick is showing is the WorldWide Telescope, which is like a space-version of Microsoft Virtual Earth but applied to space and the sky where you can navigate and also search. This is an example of galactic space storage. We are now entering into the age of human scale storage, where you could theoretically store all data, images, and video of your entire life. This is the premise of a MSR project with Gordon Bell called MyLifeBits. Another MSR project is the SenseCam which takes video of wherever you are while walking. A practical application of SenseCam to aid in memory loss where it would record images and video, and then the patient could review it to help remember. In an experiment comparing with a diary and no aid, the SenseCam device performed significantly better than the other methods where the ability to remember after 1 month was higher than with the other methods. Another research project is to manipulate images and improve on them (very relevant especially with wedding photos!), MSR have created technologies that allow you to remove certain objects and manipulate images using curves. One product that stitches a whole bunch of photos together to reconstruct a 3D virtual environment, called Microsoft PhotoSynth.

Another research area is streaming intelligence where we now are beginning to have sensors being deployed in the environment. Eric Horvitz is doing statistical analysis and modeling to do traffic prediction and modeling the user in order to predict better ways of driving through traffic.

On Technorati: Microsoft Research, Rick Rashi