A brief history of Microsoft's Sensecam

Oct 15th 2009, Microsoft finally license Sensecam see here. This is a sensing camera to help people with Alzheimer's.

(Image shows early version of SenseCam)

I invented and built the first sensecam in 1999, my patent filed 2004 as here , first paragraph explains my plans for helping Alzheimer's .

 I had been approached by Roger Needham, then head of Microsoft Research Cambridge in 1998, to work for Microsoft Research after invention of my SmartQuill, a sensor telephone appeared on www.slashdot.org , this technology now used in iPhone. * See end of article.

Why did I invent SenseCam?

I wanted to help a person with memory problems who was always losing keys, also a dentist friend of mine, Dr Jane E. Jacott, Woodford,  Cheshire,  wanted some technology for time and motion studies of how dentists spend their time, sitting, standing, doing surgery, paperwork, etc. This really inspired me and concerned me for some months, as I wanted to help her.  I also have lost a few months memory after being hit by a car . It is torture to lose your memory after a car crash. I think it was so much worse for Mum & Dad as they knew all but I lost my memories for about a year.

I built the first SenseCam using an Analog Devices accelerometer, a PIC microcontroller a digital camera that could take 64 images, a good capacity for 1999! It was hand soldered on Veroboard and programmed in PIC assembler by me in 4KB of memory. I attached it to front of my bike basket to capture images of all the drivers in Cambridge who kept hitting me. I curiously ended up with more pictures of ladies not looking after children in buggies and buggies ran in road and I got pics of wide eyed scared babies when I braked ! (accelerometer captured braking and triggered pic) But a lot of the images were poor and blurred and pic of a car bumper is dull. I searched for months for a suitable tiny fish eye lens and found one , this allowed a very wide angle (104 degrees) image. So wide angle it could even remember the shoes I wore on the day! I put the project on hold for a year or so until I found a camera module with 128Mbyte of memory. Meanwhile I was working with John Krumm, of Microsoft Research Redmond on people location badges using radio and accelerometers. This worked like indoor GPS. I have to say at Microsoft Research, my employer, there were many other supportive folk, but Andrew Blake was one of the the best. (Roger Needham also but sadly died,) Andrew knew I had being doing experiments with accelerometers on the body and prompted me to do a device with lots of raw sensor data output so the machine learning people could analyze it. Perhaps add a camera? The key bit to getting sensecam working was my algorithm for recording a sequence of rooms. I said to my then husband, who kept loosing his keys, which room where you in? Library, car, hall, lounge..retrace your steps. I designed software and hardware with a light and colour detector, accelerometer,temperature sensor and passive infrared detector. When we walk through a door, a step change in light occurs, i.e. a door frame detector, so an image can be triggered. Person in front of you, triggered by heat, captured image. Accelerometer, used to capture pic when you turned around, so pics all blurred! But I turned this bug around to capture image when person *not* moving so steady image. This was all done in an 8 bit PIC microcontroller, I did the hardware and software for this and James Srinivasen did the memory card software and other clever bits.

Trevor Taylor did a lot of the early pcbs and hardware. I designed and built the hardware and software. There were many late solitary nights with soldering iron and oscilloscope. Stuffing all my rat's nest of hardware into a case the size of a match box was hard! It was very embarrassing when early SenseCams burst into flames due to lithium battery recharging issues and the whole of Microsoft Research (then in offices in Petty Cury, Cambridge) was evacuated.

Here is first pic taken with sensecam,

Kings College, Cambridge

Grass is pink as I took out the Infra red lens, so the chlorophyl in grass looks red.

I had about two weeks to get a demo working for a Microsoft trade show, Techfest in Seattle and Bill Gates wanted to see it! I worked late weekends, luckily my husband looked after daughter. My managers did not want to show it at Seattle, so I had to be persuasive. Bill liked it, smiled a lot, and said he would like one for his pet dog to check how it was treated when it visited the neighbours. I hand built about 18 sensecams, then my boss said could I make another 100 including some for training American soldiers in Iraq as mentioned here? I said there are better people than me to do production engineering, suggested a company and that I didn't support the war in Iraq. (My boss was North American, I am British) At my termination meeting in 2007, this was mentioned as a reason for making me redundant. I left and I did next version of sensing camera, which I am testing today. Loopholes in my patent allow new inventions.

I must mention the time I was honoured by Gordon Bell visiting my office around 2000. Gordon said, never throw any emails away, this was a flashbulb memory moment for me, I was sat in my chair and Gordon stood at my office door. Gordon has continued to be of great support to me and my memory projects before and after I left Microsoft, thank you. Some of Gordon's and Jim Gemmell's work with SenseCam covered in this book Total Recall.

Also was very pleased that Narinder Kapur of Addenbrookes approached me to help with memory problems (wanted SmartQuill but I suggested SenseCam) and Emma Berry who did the neuro trials with patients with only a few hours of memory.

Microsoft Seattle helped me with later case designs.

Today, (16th Oct) I am happily testing my Sensebulb, a sensing lightbulb that sends text messages if people have kitchen accidents, fall over, leave the cooker on, really for Alzheimer's . Chris Curry, who set up Acorn Computers after parting with Clive Sinclair, approached me re SenseBulb, and I am happy to be behind an oscilloscope debugging my software.

Here are some of the my SenseCam videos click here

I will update this blog in case I have forgotten anyone, email me at sensecam@gmail.com .

Thanks also to Jack Schoefield of The Guardian for this article about my work.

Anyway, me and my team of trusty hardware/software engineers, many from Microsoft, past and present, are available at Girton Labs if you want help or questions with your latest projects. I am at sensecam@gmail.com, or cell +44 (0) 7970 101578.

* SmartQuill telephone patent with sensors? What happened to this invention of mine from 1997? Apple bought the patent in 2008 as a lot of the technology of this patent in the iPhone.

Lyndsay Williams

www.girtonlabs.com