The Story Behind Sense

When we started Sense a few years ago, Chris, Ryan, and I had this notion that if we knew in detail what all of the electronics, appliances, and other electricity-using devices in a home were doing, then we could help people make their homes work better, by helping them to better understand energy efficiency, know when things are broken, and get awareness of what is going on in their home from day to day. We knew from our own experience that there were not really good solutions to this – we had each tried running around the house with an electric meter to figure out how much energy every device used, or calling the HVAC repair guy over and over again because the heat pump really didn’t seem to be working quite right.

Based on our past work in speech recognition, we thought the electrical signals may have enough information to allow us to learn what various electrical devices in the home are doing just by carefully monitoring the overall power into the house and then doing signal processing and machine learning to figure out usage signatures for each device. If we could do that, we could then feed this information into an application to give people detailed visibility into what all the devices in their homes were doing.

So, we started digging into this, and sure enough, people had been doing research on this for a while (starting with some work at MIT back in the 1990’s). The concept was called “load disaggregation” and it seemed like it shouldn’t be too hard. So, we started building some early prototypes and collecting data to see if the stuff we read about in the research papers would really work.

Below is an early prototype we installed in Ryan’s house:

Based on what we saw, we quickly realized that the technical challenge of making this work was much more difficult than the academic research had made it out to be!

The problems we ran into were actually somewhat familiar – when we started in the speech recognition industry we had also started with technology from MIT (I was a researcher there before starting SpeechWorks in 1994) and the difficulty we had implementing the technology was pretty much the same. We had systems back then that worked “in the lab” – so would work well for a small number of people speaking in quiet environments if they said the right things. But, the real world challenge was MUCH more difficult. In speech recognition, the challenge is the huge variety of accents, speaking styles, noisy environments etc.

For load disaggregation, the problem is similar — the research shows how to make load disaggregation work for small numbers of predictable devices in predictable environments. But in the real world, there is a huge variety of devices (and therefore electrical signatures) and a huge variety of electrical noise in a typical home. And, the signals we are able to measure is the combined effect of all the different devices in the home – kind of like trying to do speech recognition with 30 people all talking at the same time.

So, we’ve spent the past few years working hard on making load disaggregation work for real houses –using a lot of what we learned from the speech recognition world, as well as a ton of new techniques that we developed in the process. The basic answer for how to handle all the variability we see (and this was the same answer for speech recognition as well) is to use more and more complex models, add in lots of data from the real world (so, lots of examples of the things we want to recognize), and drive it all with machine learning algorithms to figure out the right models based on the data.

We’ve been making a lot of progress and now have gotten to the point where we are able to make things work well enough for the initial products, which we started shipping at the end of 2015. The good news is now that more Sense monitors are out there, we have an increasing amount of data coming in, which is what allows us to make the technology work better and better – the positive snowball effect has begun!

We have a great team of people working on the modeling and machine learning techniques and are excited about the progress we can make on increasing the coverage and accuracy of our models. And, for those of you who are early customers, don’t worry – our hardware/software architecture allow us to constantly update models in the field, so your Sense monitors will benefit from all of the progress we are making on the technology front.