Formal methods and (ubiquitous) UI

Today and yesterday, we hosted a 'mini-conference' on HCI and formal methods. Harold Thimbleby and five (of his) PhD students were visiting, who all gave talks on what they were (planning on) doing. On the Newcastle side, three PhD students who Michael and I are involved in supervising, also presented their work.

Andy Gimblett (Swansea) talked about finite automata models for (discrete) UIs, taking an interesting (generative) approach on discovering sequences of actions that are part of a user's mental model of a device (i.e. it's UI). Jose Luis Cardosa Silva (Newcastle/Portugal) presented his work on Virtual Environments, comparing different modelling approaches for this kind of setting. Jennifer Pearson (Swansea) talked about her work on improving reading tools for electronic devices, i.e. by replicating some of the physical tools we use when reading 'real' books (e.g. bookmarks, annotations). She mentioned being interesting in exploring a more physical ways of interacting with digital content, which reminded me of [cite] and our initial forays into using a camera/projector-based lamp for similar purposes.

Chitra Acharya (Swansea) then presented her initial analysis of a modern hospital bed. These beds are really sophisticated devices, and the particular one she had analysed there were actually three separate physical interfaces (for the nurse, the patient and the attendant). Patrick Oladimeji (Swansea) discussed some ideas regarding the discovery of interaction errors in devices based on a simulation of a device UI he built in ActionScript. Robin Green's talk (Swansea) was about formal modelling for reactive programming. Abdulrahman Alhaziool (Newcastle) presented his first steps towards creating a generic approach to using/running formal models in the context of instrumented environments. Due to the post-grad congregation, I missed Haryani's talk (Newcastle) on Persuasive Technologies and Security.

All in all, it was a mixed but complimentary set of topics, and it will be interesting to see what will come out of them in a year's time. In the discussion, we talked about whether there might be a gap between "low-level" models for interacting with an individual device (such as a drug pump) and "high-level" models that deal with the overall workflow (such as how nurses attend to different emergencies). Some errors/issues might not be uncovered by looking at either one, for example, when two devices are used sequentially (or in parallel) during a particular workflow, there might be some interference that could result in incorrect dosages of certain drugs being administered to a patient.

Harold Thimbleby also gave a talk on understanding designed-in errors in devices (in the context of the School's research seminar). He was particularly focussing on devices used in hospitals (such as drug pumps and monitors), showing how even with the very simple functionality and interfaces they provide, there are still significant numbers of built-in errors that can be fatal. He was presenting a number of examples and some statistics to illustrate that large numbers of people are killed each year (at least partially) due to these relatively simple HCI problems, which can be detected (semi-)automatically.

Rising sun...

I just came across a post on a research project by Adobe, which makes manipulating video very easy and straightforward. I remember having read about a similar research project before, it might have been Zunavision but I'm not sure. This is, of course, exactly what is needed for Immersive Video [1] to make it the perfect playground for evaluating dynamic public displays. It also brings us closer to the scenario, where video cannot be trusted anymore. There is a great movie with Sean Connery and Wesley Snipes, Rising Sun, where a whole lab works full time over a weekend to manipulate video evidence in a murder case.

[1] Singh et al.: Immersive video as a rapid prototyping and evaluation tool for mobile and ambient applications, Mobile HCI 2006, DOI, PDF.

Migration, late 2008 style

My new laptop arrived on Thursday, a MacBook Pro. There are about a billion reviews out there already, so I won't bother retelling them. Coming from a trusty old PowerBook G4, the new machine is really nice - very beautiful, extremely well made and much faster. I was worried about the glossy screen but it turns out to be so bright and crisp that reflections don't stand a chance to outshine anything.

While I really like the screen and the speed, there are a couple of things, which I wasn't so happy with. The trackpad 'button' is really loud and it takes quite a bit of effort to actually click it. Maybe I have to get used to 'tap to click'. In terms of battery life, I was a bit disappointed - it barely lasted through a movie. The camera is a bit of a let-down as well - its resolution has been reduced back to 640x480, which pretty much rules it out for being used with Reactivision. And finally, it seems like the fan will kick in straight away when you tax the graphics chip - and it's not exactly quiet.

I used the Migration Tool to move everything over, which surprisingly worked well. I just made one mistake in the very beginning - I selected HFS (case sensitive) as my file system. It turns out some programs don't work with this, most notably StarCraft :( There only other thing that stopped working is the PDF plugin for Firefox (PPC only), which is very inconvenient. I have started playing with CrossOver but so far I've only been partially successful - I've tried to install Publish or Perish but could only get the old version to work. I might install a proper virtualisation tool (like Parallels) but I wanted to avoid doing that (in order to not be infected with Windows ;)

Update: Some more issues I discovered today: the DVI-to-DisplayPort-Adapter does not work with a standard DVI cable that has all the pins; it only accepts purely digital cables (similar to the mini-DVI adaptors, see here and here. When riding the bus this morning (and typing away on my laptop as usual) I also noticed that the discreet gfx chip really reduces battery life quiet a lot. I guess I'll only turn it on when I *really* need it...

Instrument from Outer Space

Amongst all the marking and preparation for the lectures I'll be teaching from next week onwards (Graphical Interfaces and AI in Games), I came across a new musical instrument, called the Tenori-On. It's probably old news to anyone working in this area but I struck me how well simplicity, dynamicity, spatial interaction and multi-touch gestures are brought together in this device. Compared to a run-of-the-mill tabletop system, it has a really low resolution but it seems to work really well, and looking at the gestures that the musician uses during the performance, it beats most tabletop systems hands-down in terms of expressiveness.

Ubicomp 2008 is over

Ubicomp 2008 in Seoul is over now, and it was quite an interesting experience. Not only because of the papers, many of which were very inspiring, but also because of the setting: Seoul is quite metropolitan, but as a Westerner, you can neither understand anything nor read any sign, and there is this interesting 'abstraction' of other cultures - we had lunch in a Bräuhhaus, which was serving sushi and pizza. Add to that a good helping of jet-lag, and suddenly lost in translation seems to resonate very strongly with one's experience.

Anyway, what still lingers in my mind is the closing panel, which discussed (to some degree) what Ubicomp is/should be and where the conference is going as a whole. Gregory Abowd made a similar point he made last year about the need to look into problems that really matter to real people, which I strongly agree with. In a way, this could be one defining factor of what Ubicomp is about: bringing together results from various disciplines and integrating them in a thoughtful way to make a real difference in people's life. I think this year the number of papers that followed this general idea has increased compared to last year.

Another clear trend is definitely energy efficiency - I very much enjoyed the talk about 'the potential for location-aware power management' by R.K Harle and A. Hopper. Combining this trend with the above, might lead to a generic approach to Ubicomp, where the underlying question is 'what can improve a particular situation for the people in it, and how can we realise that with the minimal amount of resources (technology-wise and energy-wise)?'

The other issue discussed at the panel was the future of the conference. I guess everyone agreed that it would be good to have more industry-involvement and to have a stronger exhibition/demo section. I'm not sure whether I agree with the idea of merging all ubicomp related conferences into one big event. Comparing the CHI experience to mobile HCI and Ubicomp, I think I prefer the smaller, single-track event. And for Ubicomp in particular, becoming too big might be quite counterproductive in terms of mixing up researchers from different disciplines (which in my opinion is one of the key strengths of Ubicomp).

Mobile HCI 2008 in Amsterdam

I've just returned from mobile HCI 2008 in Amsterdam - overall, a very enjoyable experience. As always, a broad range of topics were covered, both from an academic and from an industrial/commercial perspective. There were a couple of papers that stood out for me:

• Usage of Spatial Information for Selection of Collocated Devices (Roswitha Gostner, Enrico Rukzio, Hans Gellersen), which reported on a user study investigating the use of spatial cues to interact with the environment (relates to the RELATE project).
• T-Bars: Towards Tactile User Interfaces for Mobile Touchscreens (Malcolm Hall, Eve Hoggan, Stephen Brewster), which looked into using tactile feedback to overcome some of the limitations of finger-based touchscreen interaction.
• Multimodal Interfaces for Camera Phones (Stephen Brewster, Jody Johnston), which described a very subtle modification to existing non-visual cues on digital cameras to convey information that is usually encoded as (uncomprehensible) icons on the viewfinder display.
  

The best paper/presentation in my opinion was 'Mobiphos: A collocated synchronous mobile photo sharing application' by James Clawson, Amy Voida, Mirmal Patel & Kent Lyons. They described a system that enabled a group of people, each person armed with cameras, to instantaneously and effortlessly share photos on the spot as they were taken. Not only was the application very innovative and intuitive, but the user study was very thoughtfully designed and revealed some very interesting behaviours resulting from this new technology.

While the second keynote was slightly underwhelming, the first one by BJ Fogg was very inspiring. He talked about persuasion and his belief that persuasive systems (e.g. in health applications, social networking tools and environmental education) will take off on mobile devices in the near future.

Digital Noticeboard

Yesterday, I went to a local supermarket (a rather small one) and was surprised to see this new public display now being mounted right behind the check out counters. On closer inspection, I found out that it actually is a digital pin board/notice board, where people can put up their classifieds.

Sure enough, nothing was on there yet (except for some commercial advertisement) and there was some sort of paper 'classified' stuck next to it. But seeing that you have to either call them or post a postcard to them in order to get something on the screen, it doesn't seem surprising that nobody is inclined to use it so far. It's not even possible to text/Bluetooth something to the display and there is no keyboard or pen to interact directly.

Workshop on Sketching at Newcastle

What do you know, it's the season of workshops. On Wednesday, we had one at Culture Lab on sketching, which was organised by the INSCAPE project (funded by the EU). Bill Buxton was supposed to give one keynote (having written the influential book on sketching) but had to pull out on short notice.

The workshop nevertheless turned out to be quite interesting, with lots of different methods and perspectives being presented. Bill Gaver and Andy Boucher gave a talk on the design process (and their use of sketching within it) that led to some of the Equator systems (PlaneTracker, DriftTable), which was quite interesting as I was only familiar with the end product. I also quite liked Patrick's on video sketching (immersive video, biometric demon, Jon's VJ'ing work).

The two key points I took home were: One, sketching seems to be more of a mindset (not being too much attached emotionally to a piece of work, readiness to change something, quickness of creation, willingness to disregard certain details) than a particular technique. Two, that rather than building sophisticated tools to support the creation of sketches it might actually be more interesting to create tools that document the process of sketching. After all, some sketches do only make sense to those people that were present at the time of its creation...

Workshop on Situated Art and Dynamic Places

Last week, I co-organised a local workshop on situated art and dynamic places (in the context of the space-pace-place group). We had quite a good turn-out from a large array of disciplines (we were a bit short on the CS end of things), and both the discussions and the small group work produced some interesting scenarios/design ideas. I'm still updating the page summarising the workshop but in a couple of days, it should all be well documented here.

Dundee School of Design Degree Show

Together with Jayne, I was recently invited to attend a meeting on design(s) for counterterrorism at the School of Design at Dundee. While there I had not only a chance to meet lots of interesting people but was also able to explore the impressive degree show that they had put on for all their degrees.

There were a couple of tabletop and interactive surface systems, some fairly standard (i.e. based on Reactivision or FTIR) while some others were more innovative (at least from a technical perspective):

• 4track - a tabletop system for collaborative music making (link)
  
  This is basically a simplified version of Reactable, a bit more ludic than functional.


• absent reminders - a very beautiful mirror display using text messages (link)


• fable - an interactive kitchen table for families to share photos and stories (link)
  
  I liked the simple yet effective tracking, which was done using Processing and IR-reflective foil

There were too many projects to mention them all here. The one that I thought really stood out was the the bone project, which explores different ways to use (bovine) bones. There were examples of jewellery, containers, tiles and casings made of this material, which felt and looked quite good (and this is coming from a vegetarian).

Photo workshop in Lancaster

I've been invited to a workshop on photos and interspace, which was held today. There were a number of interesting presentations (which should be online at the workshop web site in a couple of days.

The first key talk was by Michael Hulme; he talked about the concept of interspace, which I wasn't familiar with. If I got it right, it's meant to describe the time/space between two events, particularly events involving more than one person. He's done some work on how these interspaces are used, and apparently, they are increasingly filled with communication through mobile devices.

Another talk I enjoyed was given by Matt Watson and Elizabeth Shove. They discussed a number of things but there we two points that I thought were intriguing. One was the tension between faithful and unfaithful reproduction of digital photographs, and the (mis)appropriation of photos shared through the internet. The other one was that as photography and the use of photos is changing so is 'family' as a whole.

Haliyana Khalid and Alan Dix presented some work looking into the use of photo blogs amongst foreign students in Lancaster, in particular how it was used to communicate with their families abroad but also amongst themselves. They also talked about 'photo lurking', which is browsing through other people's online photos; there was some discussion whether this is the best term to use for the concept, as it has a negative connotation.

I presented some stuff based on Peter Dixon's final year project (using Graffiti and its code of conduct as a way to operate a public photo display), and got a number of interesting comments and questions.

Here comes everybody (book by Clay Shirky)

I've just finished reading Here comes everybody by Clay Shirky. It's a book on how the social tools that now are available on the Internet (e.g. wikis, blogs, social networking sites, twitter) change the nature of society. He's looking into how the cost and effort involved in creating, maintaining and 'leveraging' groups (from small to large ones) have been considerably reduced with these tools. From this observation, he goes on to describe four levels/steps of group activity:

1. sharing
2. conversation
3. collaboration
4. collective action
   

There is much more in the book, including quite a few examples supporting his argument. I've found the book quiet an eye opener and very thought provoking, and somehow, the whole hype about facebook and other social networking sites makes much more sense to me know.

There is two things I though could be improved with the book: I'd like to have some concise summaries for all the chapters to make it easy to go back and read certain parts. Also, there is very little about what the effect of this 'new revolution' will be. I guess it's kind of hard to predict, yet it shouldn't be impossible

There is a podcast of a talk that Clay gave at harvard, which summarises the main points of his book.

A start?

CHI'08 in Florence is over; all in all it was quite an impressive and inspiring event. I still need to go through all my notes to make sense of it all :)