Values of Observational Research

Fundamentally, research is about answering questions.  Want to find out why are plants green? Do some research with a botanist. Want to know how gravity works? Do some research with a physicist. Want to know who won the 1977 World Series? Do some research with a historian.  But if we have a question about what a group of people want or think or do, we turn to opinion research to find the answer. When people think of opinion research, they usually think of surveys –asking a bunch of people what they want or think or do. But surveys are not the only way to find these answers. In many cases, surveys are actually an imperfect tool for the job.

Surveys are reliant on respondents telling us the answers to our questions. This presents a number of challenges. First, surveys are totally dependent on what people tell us. If respondents are dishonest or embarrassed or forgetful, the quality of our answers will suffer. Second, most surveys consist of what we call “closed-ended” questions. That is, we ask a question then give the respondents specific answer options to choose from. “Do you travel outside of your community to receive healthcare?” is a closed-ended question where the respondent has two options for the answer (yes or no). The nature of closed-ended questions means that by definition our answers are going to be limited. If the researcher designing the questions does a poor job of providing answer options, we will miss important details and again, our answers will suffer.

These challenges of survey research are particularly pronounced when we want to understand behavior; understanding what people actually do (as opposed to say they do). Behavior is tricky because not only are we asking people to remember what they have done in the past, but we want them to remember the context and details of the action. Additionally, if the subject is sensitive, we want them to report honestly on a difficult subject matter. While surveys can provide good data in these situations, there is another tool, observational research.

Observational research, and its sibling, ethnography, is a great tool for understanding real behavior in context. Observational research involves embedding a trained qualitative researcher into a group and charging the observer with recording the key details of that group going about their daily activities. Observation is wonderful because it lets us capture the entire context leading to behavior and understand the interaction between behavior and environment.

For instance, we work with a large international development organization that has developed a cell phone enabled tool to promote health in rural central Africa. Their program looked great, the technology worked well, and the early reviewers were enthusiastic about its potential impact. However, when this health tool was deployed to the field and given to its proposed users, it did not seem to work quite right. Ipsos partnered with the organization and went into the field with interviewers and observational researchers.

We found that our (both our partner’s and ours) assumptions about how people were going to interact with the technology was wrong. Our observers noticed that while almost all rural Africans had access to a cell phone and had unique phone numbers, they had limited access to the physical cell phone. Multiple people in a community would share a single device and each of them would have their own SIM card. The group would take turns with the device and when it was one person’s turn, they would put their SIM card in the phone and they would have cell phone access.

Our teams noticed that if a person did not have possession of the cell phone right when they would need to use the mobile health technology tool (which was most of the time), they would not come back to use it once they did have the cell phone. This observation allowed us to work with our partners to refine their tool and take better advantage of the time-shifted way rural Africans interact with their mobile phones.

The take-away from this is for questions involving complicated subjects where we don’t necessarily understand all the factors at play, observational research is a great tool for uncovering answers in a relatively quick and efficient fashion. When we combine observational research with controlled experiments or videography, we have an even more versatile and powerful research tool. But that is a topic for another post…