News from Nullspace

Tim Cowlishaw's blog

About me | Twitter | Github

Weeknotes: 29th July - 5th August 2015

I started this week by catching up with Filip on my progress with the logger. This was very useful - he was able to help with a couple of boring technical practicalities which I’d been struggling with related to calling the Accessibility APIs from F#. It appears that a lot of this is very sparsely documented, so it’s really useful to be able to talk through what I’m doing with someone who’s done similar things before!

One particular issue I’d dealt with the previous week is the fact that the accessibility events we’re listening for are very low-level - relating to individual keypresses, mouse gestures and interactions with UI elements. This is a very rich source of information, however, I’m specifically interested in higher level information - when focus is switched between applications, when URLS and files are opened and closed, that sort of thing. Now, this information is clearly captured in the low-level stream of events, however, identifying the ‘signature’ of a set of low level events which make up a higher-level interaction is tricky. For the case of identifying focus switches, however, we came up with the idea of polling for the currently focused application periodically, alongside capturing the event data. That way, we can use the polled ‘current application’ data as training data in order to learn when context switches occur from the event data.

We also briefly discussed one of the more interesting (to my mind) research applications of the technology - providing data on the personal provenance of a piece of information (the path a user took to discover it) for the purposes of information refinding - Filip was very interested in this, which was most encouraging.

Much of the rest of the week was spent doing further work on the tasklogger, In particular, attempting to get more information about the accessible object which was the subject of an accessibility event. So far, I’ve had limited success with this - I can get the accessibility object for the window for which an event occurred, and then traverse its children, aggregating all the text properties I can find. However, it appears that many applications implementation of this is very inconsistent, so I have to guard against all sorts of interesting conditions where an application doesn’t fulfil the contracts of the IAccessible interface. I now need to extend this work to do the same for the specific IAccessible which fired the event, which might give us some more useful contextual information about the underlying activity. I am also going to implement the ‘polling’ idea discussed with Filip above.

This week I also arranged and installed the server side of our logger application on a VPS hosted within the computer science department (it’s important to use an internal machine for ethics and data protection reasons). This was a pretty simple process, however, the UCL information services department have a few additional security requirements due to the sensitive nature of the data we’re gathering. In particular, I need to ensure that our log data (in a MongoDB database) is encrypted at rest - I’ll be looking into how best to do this this week!); I also submitted the (hopefully) final set of revisions to our research ethics application and await a decision!

I’ve also continued my reading on grounded theory, as well as Dumas et al.’s Stuff I’ve seen paper which provides some excellent background on IR systems for refinding, as well as experimental results into effective cues for information refinding. I also read This recent paper by Guha et al. on the user modelling approach taken by the Google Now system, which is useful both for the insights into the user modelling process itself, as well as the practicalities of implementing it in production.

In other news, I started using Spacemacs on a whim, and it appears to be replacing vim as my main text editor. Taking the best ideas from vim (Modal editing, a leader key with memorable and ergonomic shortcuts), and implementing them within Emacs, while providing an excellent set of configuration defaults (and a well-thought-out means of extending them) is a complete winner to me.

Outside work, My friend Tom came to stay for the weekend, and we had a great bike ride out through Epping Forest, along with Ben and Aanand. I also spent Saturday afternoon out leafleting for the Jeremy Corbyn campaign in Walthamstow town centre, which was a thoroughly enjoyable experience - I met so many of our neighbours who were friendly and enthusiastic about the campaign. Finally, on Monday evening, Tom, Vicky and myself all went to Cafe OTO for a gig put on by my friends at The Quietus featuring Hirvikolari, Chrononautz and Sly and the Family Drone which was absolutely incredible.

Interesting links:

Music: Empress of, A load of Blue Tapes stuff, Wolf Alice, Leonard Cohen, Martin Dubka, The Membranes, Jam City, and a lot of disco records.