12 days left ...

Shockingly, my previous post dates back to July last year, and that's not (entirely) due to me being lazy. I take it as evidence of me being frantically busy with a lot of things, in particular my upcoming move to the ifgi, which I am very excited about and which is now only 12 days away. Over the last couple of weeks, I've been organising various things to make sure my move does not leave utter chaos in its trail. Besides the work related aspects, there is also the matter of actually moving from the UK to Germany. Not that I would classify myself as some sort of urban/global nomad (my last inter-country move was eight years ago) but it is surprising to see how tricky it can be to move from one country to another (even if it's just within the EU). There are plenty of issues such as one organisation in one country requiring a particular document prior to be able to do something, which no organisation in the other country can issue. Or, needing an address to do X but having to have X before being able to register at an address. Given that these kinds of moves (from one country to another) must be more common these days, I don't find much has improved since I left Germany eight years ago. It is also quite frustrating to see that there still isn't any decent mobile phone provider that covers all of Europe - despite most of the big ones being active in most countries. There must be plenty of people who'd buy into a decent pan-European plan. Bring on the virtual SIM cards ;)

Summer School on Context Awareness

Yesterday, the Summer School on Context Awareness took place here in Culture Lab. Sponsored by the EU, Matthias Kranz had come over and delivered a set of lectures covering a broad range of topics in this area, including the history of ubiquitous computing, basic concepts, design principles and location sensing. We had a good turnout of students with a quite diverse background (from Computer Science to Digital Media and Education), and they produced some really interesting ideas during the final workshop session. We also had some good discussions about the implications of the increasing proliferation of ubiquitous computing and in which cases using this technology would subvert the original purpose of an activity or an object.

Talk on Location Awareness and Intelligent Environments by Matthias Kranz

Matthias is visiting Newcastle this week (he's delivering the Summer School on Context Awareness on Wednesday) and he gave a talk on his current research yesterday. One of the topics he discussed was the use of cordless phones for indoor/outdoor localisation. It was fascinating to learn about the pervasiveness of DECT-based phones (there are many more than wireless base stations - on average 3 out 4 households in Germany have one) and their properties (long range, low power, reserved frequency band) that make them very well suited for localisation. He presented initial results indicating that the algorithms can be used that were developed for Wifi-localisation. Additionally, Matthias show initial evidence that the theoretical benefits can translate into practical advantages in terms of coverage and precision of the measured location. It will be worthwhile to keep that in mind when designing any smart objects for use in people's homes.

Driving without moving

Earlier this week, Phil and I went on a scouting mission to check out some driving simulators. We first went to visit Cruden in Amsterdam, who manufacture high-end driving simulators with very accurate motion cues. The platforms they use are very impressive pieces of engineering (see photo) and the degree of realism is astonishing. I got to drive a simulated Ferrari around a racing track, and while I haven't driven one in real life, the feeling of accelerating or hitting the break was very convincing - the clutch felt like a bull was hitting the car every time I changed gear.

Frank at Cruden was kind enough to also organise a quick meeting with Riener Haapee from Delft University. We learnt a lot about the research they are doing, which is closely related to what we want to do with the driving simulator. They have been using different types of car simulators to work on biomechanics and human-machine interaction in the car. (Another interesting bit of information was that apparently, every driving school in the Netherlands uses a (simple) simulator.) One of the projects at Delft University is DriveOps, which looks really interesting. Riener also pointed us some collaborators they are working with, including Noldus, a company creating behavioural models, and to SmartEye, a company creating eye trackers specifically for cars.

The next day, we went on to visit Kai Foerst (from Dr. Foerst GmbH) near Cologne, who make a different type of simulator, which is aimed much more at training drivers. While the motion cues are much less realistic than in the Cruden simulator, the actual software provides a great environment to quickly create and test driving scenarios in quite realistic everyday environments (see photo below).

One interesting aspect we discussed at length is the trade-off between realism of the motion cues and the risk of motion sickness, which is closely linked to the overall question of how realistic does the simulation need to be in order to be effective? Obviously, if the motion cues, the visuals and the interaction are inconsistent, then the degree of inconsistency will increase the likelihood of motion sickness. Another good predictor of motion sickness seems to be age: the older people are, the more likely they are to experience it. Younger people (e.g. those training to get a driver license) rarely suffer from it - we thought that maybe exposure to 3D video games is a factor in this case. However, an interesting question is, if this is indeed true, would that compromise the effectiveness of the simulator as people would (subconsciously) treat it more as a video game rather than as a simulation of the real world.

visit to Centre of Cognitive Science, Freiburg University

Thanks to being grounded in Germany by the ash cloud, I got a chance to visit Christoph Hoelscher and his group at the Centre of Cognitive Science in Freiburg last week. I had met him some time ago when he was paying a visit to Kenny's lab at Northumbria, and I was really pleased that he was able to make some time for me given the short notice. He has been working a lot on indoor navigation (e.g. [1]), and I did get to see a few experiments they had implemented to test various aspects relating to indoor navigation (e.g. the impact of specific architectural features, different types of landmarks, strategies when navigating different types of environments).
They are using different types of VR environments to conduct their studies, which is closely related to the immersive video/panoramic photograph approach we've been using/investigating here.

[1] Hölscher, C., Meilinger, T., Vrachliotis, G., Brösamle, M., & Knauff, M. (2006). Up the Down Staircase: Wayfinding Strategies and Multi-Level Buildings. Journal of Environmental Psychology 26(4), 284-299.

Reflections on ash clouds and transport networks

Earlier this month, I had a flight booked to Germany from Newcastle. As it so happened, the flight was on the day the infamous ash cloud started to cover Europe. I was woken at 5 a.m. in morning when my phone beeped with a text message informing me that my flight had been cancelled. As their web site still said that my connection flight from London was 'confirmed' and 'on time' (couldn't get through to their call centre), I hopped onto the next train to London.

Luckily, it had free (!) Wifi so I could chat with my wife who was following the news on TV and radio, and also keep an eye on the BA web site. Somewhere around Peterborough, my connection flight was cancelled, and I started to investigate alternative routes to my destination, which turned out to be quite tricky. There doesn't seem to be working pan-European booking/look-up service for trains - so I had to piece information together from several web sites. Once I got to London, I was at least able to buy a ticket that took me to Brussels and then on to Cologne (where I had to buy another ticket). In the end, I actually got to my destination about 30 min later than I had would have gotten there if I had taken the plane.

So, from this experience it appears to me that a) it's possible to be almost as quick using train than using a plane if you travel within Europe, b) it's not outrageously expensive (I paid less than what I would have paid for a plane ticket bought on the day of travel), and c) if there was a unified booking/lock-up service for trains (similar to those available for planes), I think more people might consider this option. After all, taking the train is so much more relaxed (bigger seats, not having to rush through ever increasing numbers of security checks), being able to walk about at any time, etc.).

A world without Google?

The introduction of the iPad made me start to think some strange thoughts. In one way, this is a brilliant device with the most intuitive UI I've seen so far, and I can see this as the perfect 'computer' for anyone who doesn't care about computers and just wants to get to some info. So, somehow this is a window into the Net but it is also its own eco-system that is tightly controlled by Apple (i.e. who gets to put apps on there and who doesn't). I guess there are good and bad side effects of that control but it sure is something worth pondering... But isn't that a bit like what the Internet has become? In a way, Google controls what a user gets to see and what not. Sure there are ways around that (using different search engines) as there will be with the iPad (jailbreaking) but in the case of the Internet that is not really satisfying (i.e. just some other company that controls what you see).

So, I'm wondering: could there be a search engine that is entirely free? Something like Wikipedia - distributed, maintained by volunteers and without a commercial drive. I think there are many parallels between a Wiki and a search engine (of course, there are also differences). You'd need some mechanism to alert the system of changes in a web page (spider or maybe proactively done by the publisher of the web page) or if a new page is added (same) but that isn't too hard to do. So, if the job of cataloguing the web is partitioned and replicated amongst loads of servers, would that be a viable alternative to Google? Given sufficient numbers of volunteers, it might even be possible to get some sort of semantic catalogue (such as what Yahoo! used to be initially but with additional information added).

Article on photo sharing in New Scientist

The New Scientist has just published an article about our work on using proximity regions around mobile phones to share photographs. Michael has put together a web page about the approach, which has a video and a couple of images explaining how it works.

Kray, C., Rohs, M., Hook, J., Kratz, S. Bridging the gap between the Kodak and Flickr generations: a novel interaction technique for collocated photo sharing. International Journal of Human-Computer Studies. (in press) [link]

Ubicomp '09 videos online

Ubicomp '09 was held recently in Orlando, Florida. The proceedings are available on the ACM site now, and the organisers have also uploaded the videos from the video track on the conference web site.

Tutorials at mobile HCI

This was the second year that there was a full day of tutorials at mobile HCI, and the slides are quite a good resource for anyone, who's interested in a particular topic in mobile HCI.

• Enrico has uploaded the slides from this year here
• Albrecht is hosting the slides from last year on his blog

Should be a good read for anyone looking for related work or a head start in a particular area.

Mobile HCI 2009

Mobile HCI is held at Bonn this year, with quite a few papers (that Albrecht is summarising nicely in his blog). I'd like to add the one on GraspZoom by Miyaki and Rekimoto, which I thought was a very simple yet effective and practical solution to overcome the problem of using touchscreen-based phones with one hand. Calli also presented our paper, which was based on her MSc thesis which she completed last year. I guess she also found out the hard way that a good way to make sure you get many questions after your talk is to finish ahead of time.


Partially due to the CHI deadline yesterday, the demand for caffeine was quite high amongst many of the participants but luckily, there was an ample supply of nice coffee places right next to the conference site. The photo to the right shows 'Einstein Kaffee', which does not only serve pretty good coffee in a decidely retro-chic style but which probably can claim to be the unofficial second site for mobile HCI this year.

Lots of visitors, towards fully mouldable devices

It's a busy week in terms of visitors: Albrecht came up again to talk about the TUI design project we've been working on for a while. He gave a well-attended talk at the School, where he was discussed the potential, challenges and implications of a world saturated with displays and sensors. Both Tanja and Bastian from his group were here as well, and we made great progress in preparing a prototype for the workshop we'll be running in two weeks' time. On Wednesday night, Nic Villar joined the party and brought with him the rapid prototyping toolkit for electronic devices as well as a sample of a material that opens up interesting design possibilities.


He showed us how easy it is to create devices using his toolkit. The hardware side comes down to connecting various parts using thin standardised cables, and the software side is fully integrated with Visual Studio, so its a matter of selecting classes from contextual menus. He built a device integrating a rotary control and a small OLED screen and had it up and running in less than five minutes. Quite impressive (and also highly complementary to what we've been working on around here). It's easy to imagine how this could be very useful to physically prototype devices very quickly.

The other surprise he pulled out of his bag was a material I hadn't heard about before, Polymorph. It a plastic-like granulate that sticks together and becomes formable at around 60 degrees Celsius. When it cools down it becomes rigid again. The process is repeatable numerous times without any apparent deterioration of the material (unlike Fimo or Hama beads, which once heated and cooled can't be changed anymore). We all had quite a good time playing with it. This obviously has quite a bit of potential not only with respect to prototyping physical shapes but also with respect to create objects that physically change shape...

Where do you think you are going?

Yesterday, I attended a one-day workshop discussing 'digital' transport, its challenges and where things may go in the future. The event was called 'Where do you think you are going?', and it was not only very well organised but also very interesting. There were talks from a very varied set of people, and the format (and venue) chosen made sure you didn't end up falling into a coma after a half-day of powerpoint slides. Most presenters only had five minutes to talk and were then 'waved' off by the chair/the audience.

There were a couple of talks discussing datasets, their importance and the problems relating to them (ownership, quality, user-created content). Nick Illsley (CEO Transport Direct) talked about how often data is updated, how setting up a database of bus stops for the UK resulted in 20,000 further bus stops being 'discovered' and the problems resulting from integrating data owned by a large number of companies. Then there were a number of talks about user-generated content/next gen web services and some discussion about where that might lead e.g. in terms of interacting with users of (public) transport, the need for real-time information and the detection of problems such as traffic jams.

My talk was about pedestrian navigation and whether it is a 'solved problem'. I gave a couple of examples why things don't look all that solved if you start looking at the results you get from a standard navigation service for pedestrian navigation. In a way, I think that pedestrian navigation is something like a glue that connects all the other means of transportation, so optimising it and linking it up with other means is essential in making transport in general more sustainable. (I think the slides will end up on SlideShare at some point.)

I hope that some of these issues will be tackled in the context of the Hub for Digital Economy, which will start in Autumn and which has a Transport component as well. The photo shows Phil Blythe giving a brief overview over the Hub - it also shows the venue, which in the time of the industrial revolution used to be a terminal for trains taking materials and goods back and forth between the mill and the Tyne.

Update: The slides are now online at SlideShare.

The Path to a different game space

While looking for material for the module I'm teaching on Game Engineering, I recently came across an indie game called The Path from Tale of Tales. It is a fascinating, thought-provoking, and engaging game for a number of reasons, including beautiful graphics, incredible music and sound effects and a clever 'story'. Much has been said about how it pushes the boundaries of games in several ways, many reviewers take it is a clear proof that games can be art.



There are two things I found particularly interesting in the Path (scientifically speaking): the way in which you interact and the way in which its creators use space in a very innovative way. As a player, you interact with objects in the game by stopping to interact, i.e. by letting go of the keyboard/mouse, while being near an object of interest. This is a very interesting take on implicit interaction.

As the game is loosely based on the Little Red Riding Hood fairy tale, it is somehow part of the story that the player gets lost in the forest. The way this is achieved is by a clever speed-dependent camera control: when you start to run it zooms out in such a way that you look at the character from a bird's perspective. The key thing is that you can't see the horizon anymore, so it is very difficult to know where you are going, and occasionally, the character ever so slightly strays off the direction set by the user, e.g. when avoiding trees. In addition, parts of the world are no longer visible if you move at high speed. The world also changes (partially) depending on which character you play and where you are with respect to the path. Occasionally, you can only reach the safety of the path by standing still and waiting for someone to take you there. There is a map but it only appears briefly after you have moved a considerable distance and it only shows your past trajectory and not the sites of interest. Taken together, these concepts result in a game space that is designed so that you loose your way, pretty much regardless of how systematic you go about exploring it, which is not only a quite innovative approach to space in games but also an integral part of the 'story'.

Albrecht Schmidt visits

Today, Albrecht Schmidt came up from Nottingham to visit for a day. As most of the people from the lab were down in Nottingham for the Ubicomp Spring School, we had space 2 almost to ourselves and Albrecht took the opportunity to test drive our new racing vehicles. The rest of the day we spent planning for Albrecht's visiting professorship here at Newcastle (which recently got funding for). The topic we are working on has really great potential in all sorts of ways (i.e. it'll get Europe straight out of the recession ;) and I'm looking forward to seeing it all come together.

Eva Hornecker visits

Last week Eva Hornecker visited our lab to have a chat about collaborating and to have a look at what's going on here in Newcastle. She also gave a talk, which presented some interesting (and somewhat unexpected) results from a study on collocated collaboration around a tabletop system. Specifically, the study looked at the impact of input devices (single mouse vs. multiple mice vs. single touch vs. multitouch) on people's awareness of what other people were doing. One of the key findings (in my eyes) was that although there might be more instances of people interfering with one another using multitouch interaction, they were able to very quickly recover from it, which might be (at least partially) due to increased awareness in this condition. They published a paper on this study at CSCW last year.

Java and media (surprisingly cumbersome)

I've been looking into playing videos inside a Java Panel, which turned much more difficult than I thought. I first tried the Java Media Framework, but it only plays a very limited set of media (and not very well - particularly on a Mac). I've then looked at JavaFX, which looked really promising on the web site. After wrestling with the installation for a bit and getting to grips with the language, I discovered that controlling any JavaFX component from Java is considered a hack and not really easy to do - despite the whole thing being based on Java. I can't quite understand why that is the case but well. So, I went back to Quicktime for Java, and after downgrading my project from JDK1.6 to 1.5, I was able to play movies again. This is of course not a great solution at all but will work for the demonstrator I'm currently building. I really hope SUN get their act together and provide some simple API to play media in Java (it seems like JavaFX would work well if only they decided to do it)...

Michael Rohs (right) and Sven Kratz (middle) from Deutsche Telekom Labs came over from Berlin for a three day visit this week to discuss projects, ideas and where to go next from what Michael and I have been working on over last one and a half years. Together with John Dawson (left), who's just started his PhD with me (literally - he started on Monday), we had some intense and quite productive brainstorming sessions to map out interesting areas of research. It'll take some time to sort through the all, but we managed to come up a good plan for the coming months and deadlines.

While they were here, Michael and Sven also got the usual round of demos including the ambient kitchen, all the different tabletop systems we've got assembled in Culture Lab, a very tiny glimpse a Jayne's digital jewellery (she wasn't there) as well as the CAVE (immersive video and a bit of VR). On the photo, John, Sven and Michael are just about to embark on some life-size Quake-type experience - properly outfitted with the stylish shutter glasses required for the 'true' 3D experience...

Formal methods and (ubiquitous) UI

Today and yesterday, we hosted a 'mini-conference' on HCI and formal methods. Harold Thimbleby and five (of his) PhD students were visiting, who all gave talks on what they were (planning on) doing. On the Newcastle side, three PhD students who Michael and I are involved in supervising, also presented their work.

Andy Gimblett (Swansea) talked about finite automata models for (discrete) UIs, taking an interesting (generative) approach on discovering sequences of actions that are part of a user's mental model of a device (i.e. it's UI). Jose Luis Cardosa Silva (Newcastle/Portugal) presented his work on Virtual Environments, comparing different modelling approaches for this kind of setting. Jennifer Pearson (Swansea) talked about her work on improving reading tools for electronic devices, i.e. by replicating some of the physical tools we use when reading 'real' books (e.g. bookmarks, annotations). She mentioned being interesting in exploring a more physical ways of interacting with digital content, which reminded me of [cite] and our initial forays into using a camera/projector-based lamp for similar purposes.

Chitra Acharya (Swansea) then presented her initial analysis of a modern hospital bed. These beds are really sophisticated devices, and the particular one she had analysed there were actually three separate physical interfaces (for the nurse, the patient and the attendant). Patrick Oladimeji (Swansea) discussed some ideas regarding the discovery of interaction errors in devices based on a simulation of a device UI he built in ActionScript. Robin Green's talk (Swansea) was about formal modelling for reactive programming. Abdulrahman Alhaziool (Newcastle) presented his first steps towards creating a generic approach to using/running formal models in the context of instrumented environments. Due to the post-grad congregation, I missed Haryani's talk (Newcastle) on Persuasive Technologies and Security.

All in all, it was a mixed but complimentary set of topics, and it will be interesting to see what will come out of them in a year's time. In the discussion, we talked about whether there might be a gap between "low-level" models for interacting with an individual device (such as a drug pump) and "high-level" models that deal with the overall workflow (such as how nurses attend to different emergencies). Some errors/issues might not be uncovered by looking at either one, for example, when two devices are used sequentially (or in parallel) during a particular workflow, there might be some interference that could result in incorrect dosages of certain drugs being administered to a patient.

Harold Thimbleby also gave a talk on understanding designed-in errors in devices (in the context of the School's research seminar). He was particularly focussing on devices used in hospitals (such as drug pumps and monitors), showing how even with the very simple functionality and interfaces they provide, there are still significant numbers of built-in errors that can be fatal. He was presenting a number of examples and some statistics to illustrate that large numbers of people are killed each year (at least partially) due to these relatively simple HCI problems, which can be detected (semi-)automatically.

Rising sun...

I just came across a post on a research project by Adobe, which makes manipulating video very easy and straightforward. I remember having read about a similar research project before, it might have been Zunavision but I'm not sure. This is, of course, exactly what is needed for Immersive Video [1] to make it the perfect playground for evaluating dynamic public displays. It also brings us closer to the scenario, where video cannot be trusted anymore. There is a great movie with Sean Connery and Wesley Snipes, Rising Sun, where a whole lab works full time over a weekend to manipulate video evidence in a murder case.

[1] Singh et al.: Immersive video as a rapid prototyping and evaluation tool for mobile and ambient applications, Mobile HCI 2006, DOI, PDF.