Ph.D - Video-augmented Environments

My Ph.D. thesis subject was Video-Augmented Environments, in which I looked at ways of using video input for the creation of novel human-computer interfaces. Video cameras cost little more now than keyboards and mice, and there are plenty of interesting things you can do with them, but at present it often isn't worth the programming effort. I built an architecture which allowed you to create simple 'triggering' circuits from basic components (sources, sinks, filters & sensors) using a scripting language.

My chief example of this was BrightBoard, a program which monitors whiteboards and can perform a variety of actions when appropriate marks are made on the board. Here it is in use:

This two-minute video clip is also available as a QuickTime movie if you'd like to download it.

A camera fixed to the ceiling watches the activity on the board. It is connected to a computer which uses low-resolution images to detect when the user has stepped aside and so is not obscuring the board. The system then captures a high  resolution image, thresholds it, and attempts to recognise the marks that are in the image.

Details of the marks found, their position, and their type if recognised, are passed to a Prolog engine which can analyse the relationships between them and so determine if the combination of symbols required to constitute, say, a 'Print' command are present on the board. Appropriate actions can then be taken.

A paper called "BrightBoard: A Video-Augmented Environment" appeared in the Proceedings of CHI'96. You can download it as a 334KB PDF file here.

If you're really keen, my full Ph.D. thesis is available as a 2.3MB PDF here.

BrightBoard in use
thesis.pdf 2.3 MB
chi96.pdf 334.83 KB
Controlling Computers By Video (1st yr report).pdf 8.86 MB