Blog of FUSE Labs at Microsoft Research

Sep 12

Fidgebot: Robot Experience

[Written by Jürgen Brandstetter] Before picking back up from the previous 2 posts Working out While Working and Fidgebot, by the Numbers here is a recap of Fidgebot. During our internships, Noah Liebman and I developed a game based system to motivate people to be more physically active at work via standing desks & micro-exercises. The system included: a desktop app for goal setting and logging achievements; a public display of individual & team progress, and a notification system to help players meet their goals.

This post covers the one team (out of 4 teams) that had social humanoid robot called NAO in place of desktop notifications. The goal was to find out how a social robot would subjectively and objectively change individual and group behavior in this context. Given our 2.5 week pilot, objectively there wasn’t any measurable performance difference between the teams’ with desktop notifications vs. the team with the humanoid robot notification. Subjectively, there was a great difference in terms of engagement. In the words of my old Professor Peter Purgathofer: “If we could not change any measurable results, but people are happier now, when using it, we did our job right.”

How it works

The Movers, a NYC based team, were reminded to exercise via “female” NAO robot. Periodically NAO would stand up and approach a player who needed motivation to do their micro-exercises.


Next, NAO invited the player to exercise with her. 


The player could select her right or left hand to indicate yes or no. If the player selected yes, NAO would begin a fairly easy micro-exercise, and would expect the player to do it as well. 


If the player responded no, NAO would do the exercise alone. And finally, NAO walked back to her place.

Why a Robot

Building on my existing research designing humanoid social robot experiments, we decided to use one in Fidgebot as it is designed to trigger natural human behavior. We wanted to test this distinction between the conventional notification system.

There were some big differences between NAO and the screen based notifications. NAO is loud, any nearby player instantly hears her beginning to approach them. Moving objects grasp our attention better than static objects. NAO calls players by their real name. It’s well known that using a persons name is highly persuasive - it triggers the “cocktail party effect”. Finally, it is harder to ignore a real world entity that approaches you at your desk in front of others.

Having a companion, in this case NAO, creates multiple additional benefits.

  1. Making individually socially unacceptable behaviors acceptable by being in a group.
  2. Performing ridiculous movements with a partner in an open space office environment feels much less embarrassing than doing them alone.
  3. A sad robot when you decline a short exercise break provokes guilt in a player.

What We Learned

Looking back at Noah’s last post, and comparing the robot group with the groups only using desktop notifications, we can not see any major difference in the amount of micro-exercises and time spent standing at their desks. However this should not create the impression that the robot had no impact overall. Qualitative feedback and observation told a different story.

We have learned that players were more engaged if they interacted with the robot than the desktop notification system.

"I like the Robot because it makes me excited" - [Player 3]

Some people even projected feelings into NAO:

"When participant 1 rejected NAO I actually felt bad for the robot" - [Player 3].

The interviews revealed some unexpected behaviors. Players began mimicking NAOs gestures and way of sitting. NAO’s presence and interactions “forced” each one in the group to participate in often ridiculous tasks, breaking down hierarchical social barriers which was positively perceived by the players and led to more casual team communication.

Next up: Noah and I are summarizing what we learned developing and deploying this project in a CHI Work In Progress paper. 

Sep 11

Fidgebot by the Numbers

[Written by Noah Liebman] During my internship at FUSE Labs, Jürgen Brandstetter and I deployed Fidgebot for about 2.5 weeks in the MSR NYC and FUSE Labs in Redmond, WA. We collected data on the system use to examine how Fidgebot supported players’ physical-activity -at-work goals. We interviewed participants about their experience. This post focuses on preliminary analysis of the quantitative data.


Did players use Fidgebot? People came and went for conferences and vacation, the orange shows the number of possible players each day. The blue shows how many people actually engaged with Fidgebot for that day.

In general, there’s a tendency for fewer than half the eligible players to be active on any given day. Despite this, there is a clear trend: participation is highest at the beginning of each week, and drops off through the week. This may be partially attributable to reminder emails we sent to each participant, but there were also emails sent during other parts of the week.

How much players met their goals was substantially split between micro-exercise and standing goals. We looked at each by individual, and over time.

The number of micro-exercise breaks each person aimed to do over the course of the 2.5-week period was based on both their daily goals and the number of days they were active. When looking at this graph, the orange bars are the number of micro-exercises each person wanted to do. Each person wanted to do ten per day, so in effect it tells us how many days each person was active. The blue shows how many of those were actually achieved.

imageThere are a few things to notice about this. First, the two teams on the left (both located in FUSE Labs in Redmond, WA) were active way more than the two teams on the right (both located at MSR NYC). FUSE teams outperformed the NYC teams in terms of goal achievement, although user 34 brings NYC’s goal achievement percentage to about on par with Redmond. The other thing to notice is that with the exception of a few outliers people didn’t come close to meeting their micro-exercise goals. This is less than impressive, and may provide insight for system designers about the ways goals and progress are made apparent. The fact that no players changed the number of their micro-exercise breaks per day from the default, also implies that we overshot this goal.

When reviewed over time, we see the opposite of participation: micro-exercise goal achievement increases through each week. This is probably because as people who are less interested or motivated drop off during the week, it pushes up the average among those who are still active.


When looked at by player, standing desk usage paints a similar picture. This time, orange is the number of hours people wanted to stand, which again, increased with the number of days they were using Fidgebot. Blue is the actual time spent standing. If the blue is greater than the orange, a person exceeded their goal. Again we see that the teams at FUSE Labs in Redmond are more active than the teams at MSR NYC; that is, orange is higher, but goal achievement is the same.


The main difference in an absolute sense: goal achievement was much higher. This, unfortunately, is not because people were much better standers. This is because people would frequently forget to log when they came or went from their desks, or when they sat or stood. We were able to remove most of the most egregious cases, like people saying that they were standing and at their desks all night, but we were still left with quite a bit of what is presumably excess standing time.


Standing goal achievement over time looks substantially different than it did for micro-exercises. It declines for the first 1.5 weeks, then heads back up. We’re still working to figure out the reason for this, but this may be one case when the back-to-back emails on the final Monday and Tuesday of the pilot impacted activity. Alternatively, this trend might be overwhelmingly driven by just a few individuals. We still need to tease this apart.

Finally, when we look at overall goal achievement (combined micro-exercises and standing) broken down by team, we can see a few things. One is that it generally falls within a pretty small range (55%-63%), suggesting that, while there’s a lot of variation between people, it tends to average out over the four or five team members. We also see that there’s one relatively high performing team and one relatively low performing team in Redmond and New York. While not conclusive, this suggests that although participants in Redmond are more active in the system, they do not perform any better. That engagement and goal achievement may be independent is interesting, and worth investigating further.


Keep a look out for Jürgen’s post on the team that had a humanoid robot encourage their exercise breaks at the office.

Jürgen and I are looking forward to diving into further analysis for a CHI Work In Progress paper.

Be sure to check out our other posts on why, the robot, and qualitative results (upcoming).

Sep 10

Working Out While Working

[Written by Noah Liebman] Americans spend more than half of their waking hours at work, so during my internship at FUSE Labs along with Jürgen Brandstetter we focused on helping people achieve their personal goals in the workplace. When we surveyed people around the office a consistent (and unsurprising) theme emerged: people wanted to exercise more.

Our office like many offices is full of people sitting in chairs at desks. This is problematic when it comes to health. Medical research shows that sitting all day is very bad for health, and moving even a little bit, referred to by some researchers as fidgeting, can be extremely valuable.

We set out to help people reach their physical activity goals, building on a long tradition of persuasive technology in Human Computer Interaction.

We were interested in using team motivation, inspired by solidarity lending. This is a highly effective financial arrangement used in microfinance in which loan recipients are placed in groups of three to six. Failure of any one member to make a payment negatively impacts the standing of the others, such that there is a powerful social, as well as economic, incentive to pay on time.

Read More

Aug 14

Updates to Socl

Today the team is refreshing Socl with some much asked for capabilities.

Happy Making!

The Socl Team and Microsoft Research Fuse Labs

Faster post creation with new quick postimage

Streamlined navigation with Socl appsimage

Directed postsimage

Collect while creatingimage

Additional inline video playingimage

Aug 11

Data-driven Cartoons for NYC Neighborhoods

Last week we released a new version of HereHere NYC Weekly NYC cartoons & lightweight tools for understanding neighborhood concerns to provoke civic participation. Based on user feedback from our earlier prototype (spring 2014) we’ve made massive changes. HereHere now provides:

We’re hoping to get as many New Yorkers using it as possible as we’re doing a month long study. New Yorkers please check it out!



Aug 10

[Talk] The Visual Language of Comics - 8/14

Drawings and sequential images are an integral part of human expression dating back at least as far as cave paintings, and in contemporary society appear most prominently in comics. Just how is it that our brains understand this deeply rooted expressive system? Neil will present a provocative theory: that the structure and cognition of drawings and sequential images is similar to language.


Read More

Jul 21

Creating Data-driven Comics

“What are you working on this summer?”

“Data-driven comics.”

Usually, at this point, the person I’m talking to puts a huge grin on their face and exclaims how awesomethat sounds, and I get to bask in the satisfaction of having found a really catchy way of summing up my internship project at FUSE Labs.

Then, they think for a moment and ask the inevitable question: “So what does that mean?” That would be the point where I’d admit that I actually had no idea but doesn’t it sound cool?

However, a couple weeks into the internship and several exploratory prototypes later, I’d like to share some of the thoughts we have on how data and comics can work together to help people make sense of what is going on around them. 


Comics are often used to tell stories about how events were perceived.
Maus by Art Spiegelman; Stop Paying Attention by Lucy Knisley; Johnny Wander by Yuko Ota and Ananth Panagariya; Footnotes in Gaza by Joe Sacco.

Read More

Jul 10

[PUBLIC EVENT] 9 Awesome Projects that Aim to Make Sense of a Billion Sensors - July 15

In a world with a billion sensors, how will we make sense of it all?  Come see project presentations from teams at the top design schools from around the world responding to this challenge at the Design Expo part of the Microsoft Faculty Summit.    

Mine is by Omer Shapira, Carl Jamilkowski, Donna Miller Watts and Michael Rothman at NYU’s ITP program, advised by Clay Shirky, Robert Dietz, and Jenny Rodenhouse.

The projects are:

The design critics  this year are Bill Buxton (Microsoft Research), Liz Gerber (Northwestern), and Tom Igoe (New York University).  To give you a sense of what happened last year, check out this PBS report.

WHEN: Tuesday, July 15, 10:15AM - 12:45PM

WHERE: Microsoft Conference Center, Building 33, Kodiak Room

16070 N.E. 36th Way REDMOND, WA 98052

HOW: Contact to be added to the guest list. Microsoft employees, and attendees to the Faculty Summit can attend without prior approval.

Jun 30

Welcoming FUSE East (NYC) Interns 2014

As internship season progresses, we are thrilled to welcome 2 interns to FUSE Labs in NYC, both Juergen Brandstetter and Noah Liebman. We’re already having a blast working together!

Juergen Brandstetter @BrandiATMuhkuh is doing his PhD in Human Robot Interaction at the University of Canterbury in New Zealand. Juergen earned a Masters in Human Computer Interaction at the Institute of Technology in Vienna. His research interest is in persuasive robotics with a focus on linguistic cues. Besides his PhD, he’s also a design thinking tutor and prototyper.

This summer he’ll be focusing on persuasive behavior techniques for social robots to influence the performance of human routines in the workspace. While the amount of industrial and service robots increased dramatically over the last couple of years, the paradigm of social robots is still a largely unexplored field. Social robots, in comparison to the other two types of robots (industrial, service) are created for direct contact with humans and work on a more emotional rather than technical level. Most research with social robots is done in isolated lab environments. Jürgen will apply social robotics in a real environment to test and study persuasive techniques aimed at helping employees improve their work-life balance in a socially acceptable way.

Noah Liebman  @Noleli is a PhD student in both Computer Science and Communication Studies at Northwestern University. He is excited by how the design of technological artifacts affects people’s social behavior when interacting with technology and each other. His work spans areas as diverse as emoticon use in instant messaging and the design and prototyping of a haptic system to help people coordinate.

Whether in their personal lives or at work, people are always striving to improve. This summer he’s focusing on designing persuasive technology to help people stay “on track”. This project will draw on the combined behavior of crowds to help people realize what is feasible and what they are capable of achieving.

Jun 27

Introducing Joy Kim, data-driven comics research intern

I have the pleasure to introduce Joy Kim, a new member of the FUSE intern family. She is a PhD student at Stanford HCI where she works on social computing and creativity. One of her recent projects is Ensemble, a platform for writing collaborative stories.

At FUSE, we will be working on a project focused on data-driven comics. Her first assignment was to introduce herself through a comic, which, as you can see, she’s quite good at! 


If you like data, or comics, or both, come back to read more about her work in a few weeks.