Data collection methods

Researchers may use elements from both research approaches (quantitative and qualitative) in one study. This is known as mixed methods. Some ways that qualitative and quantitative research can be used together include:

• Quantitative data (such as anxiety level measured on a scale) may be collected alongside qualitative data (such as verbal accounts of an individual’s experience of anxiety).
• Qualitative data can be used to produce quantitative data through the process of quantification. For example, a researcher could count the number of times certain words appear in written accounts of homelessness by different individuals.
• A researcher might start their study by collecting qualitative data, producing research questions that they then try to address through the collection and analysis of quantitative data.

Quantitative and qualitative data can be collected by research studies in many ways and the different methods used have different advantages and disadvantages. In this section, the collection of two types of data will be presented: measured data and reported data.

Measured data

Researchers can collect data by taking measurements. Most people are familiar with collecting measured data in their daily life. For example, people collect data when they weigh themselves on a pair of scales, take their temperature using a thermometer or use a stopwatch to assess their running speed. There are set systems to measure these physical and temporal properties. For example, the international system of units (SI) provides an agreed way to measure the various properties, including time (seconds), length (metre), mass (kilogram) and amount of a substance (mole) [134]. Systems of measurement are not just used in daily life but are also essential for scientific research. Many clinical measures are derived from SI units. For example, blood pressure, lung capacity and cholesterol level are forms of measured data derived from SI units. These measures are always quantitative and are consistent, validated, reliable ways to measure certain properties. However, this form of data is not suitable for all studies and collecting the wrong type of data can invalidate a study’s findings. There are two main pitfalls for measured data: using an invalid measure and inappropriate use of a surrogate endpoint. An example of an invalid measure could be cholesterol level, which would be a valid way to study the effect of a new medicine on cholesterol but would be an invalid way to measure a participant’s lung capacity. Measuring cholesterol could also be a surrogate endpoint if a researcher is interested in whether a new medicine can reduce death from heart and circulatory diseases. Although cholesterol may be related to death from heart and circulatory diseases, it is not the same as looking directly at the mortality rates. Indeed, there is a possibility that the medicine might be highly effective at reducing cholesterol but have no effect on future mortality, or even increase future mortality [135].

The kinds of measurement described above are often used in studies in medicine or psychology. However, there are many other forms of measurement used in these fields and others. For example, quantitative data can be collected by monitoring a person or situation, including:

• Monitoring the number of a particular entity in a certain place at a certain time (for example, the number of people in a shopping centre, the number of newts around a pond, or the number of comments on a social media site)
• Observing the behaviour of individuals in certain situations and quantifying them (for example, the pattern of nocturnal animals’ feeding times, or the number of times children raise their hands in class)
• Testing the knowledge/ability of individuals (for example, assessing reaction time in a computer driving game).

Collecting data in this way can be an accurate way of measuring certain things. However, the methods used can vary in their reliability and validity. A key issue is whether the data observed are an accurate indicator of the thing being studied. For example, a researcher looking at classroom confidence might use the number of times a child raises their hand in class as an indicator of confidence. However, this indicator is also likely to be influenced by other factors (such as whether a child knows the answer, how tired the child is, or how much the child likes the subject being taught or the teacher leading the class).

The collection of qualitative data includes:

• Controlled observation, where the behaviour of individuals is observed as covertly as possible in a managed environment, such as a laboratory (for example, watching the behaviour of children interacting with their parents in an unfamiliar environment [136])
• Naturalistic observation, where the behaviour of individuals is observed as covertly as possible in a natural environment (for example, watching the play behaviour of children in a playground [137])
• Ethnography, where the behaviour of individuals is observed and experienced from within the environment being researched (for example, by documenting the behaviour of teachers in a staff room while in that room). The researcher may be an independent observer (present but not part of the group) or be a participant and observer (being part of the group, such as a teacher observing within the school staff room [138]).

Qualitative observation can provide rich descriptions of a situation. However, like collecting quantitative data by monitoring, there are issues to consider with these data collection methods. In a controlled observation study, the data gathered may be influenced by the unnatural setting, which makes it less generalisable to situations in the real world. For example, the way individuals interact in a laboratory when they know they are being studied may be different from the way those individuals would interact in a familiar home environment. Similarly, in an ethnographic study, individuals’ behaviour may be influenced by knowing that they are being studied. However, qualitative research recognises this dynamic and is usually more concerned with the data being an authentic and trustworthy reflection of the circumstances in which it was collected [139]. Another consideration, especially for ethnographic studies, is the objectivity of the researcher collecting the data. However, some qualitative research approaches argue that there is no objective reality and, therefore, instead of trying to eliminate their subjectivity, they try to recognise it and explore it [140]. This helps them to understand how their subjectivity may influence the study and allows them to limit this influence.

Self-reported data

Self-reported data are the information collected from an individual about their experiences and opinions. As with observed data, they can be qualitative or quantitative. Below are some examples of reported data collection methods:

• Interviews involve an interviewer asking a participant questions (for example, interviewing individuals who have experienced homelessness). Structured interviews have set questions asked of each participant in the same order. Semi-structured interviews are more flexible where questions (and their order) may vary and/or be led by the discussion with each participant. Unstructured interviews tend to have no set questions and the conversation is led by the interviewee, although the interviewer usually has an idea of the sort of topics they want to cover. Interviews produce qualitative data [141].
• Focus groups are discussions with several people at the same time on a specific topic or issue (for example, talking with a small group of people about their experience of using a particular social care service). The interaction between participants in the focus group is used to reveal areas of consensus and disagreement. Focus groups produce qualitative data [142].
• Questionnaires are a set of short questions that can be asked/answered in person, over the phone, online or on paper. Questionnaires usually produce quantitative data through asking closed questions with a set of possible responses (for example, asking a participant how much they agree with a statement on a five-point Likert scale from ‘strongly disagree’ to ‘strongly agree’). However, questionnaires may also produce qualitative data (for example, asking a participant to give a short explanation as to why they agree/disagree with a statement). Questionnaire responses are frequently quantified before being analysed, although they may be qualitatively analysed. Questionnaires can be used as part of survey, which is the process of gathering, aggregating and analysing data from a group [143].
• Diaries are accounts written or recorded by participants about their activities and habits. Diaries usually produce qualitative data. However, diaries may also include quantitative data, such as logs where participants keep track of the number of occasions they take part in an activity and the timings (for example, participants in a study on the link between exercise and well-being might log whenever they exercise over a set period) [144].

  Reported data are essential for understanding the opinions and experiences of individuals. When this type of data collection is carefully managed it can produce reliable and replicable data. Some of the most widely-used official statistics and national statistics (those which have been accredited by the Office for Statistics Regulation) use reported data collection methods [145]. For example, data on the outcomes for patients in the NHS is collected by staff logging it and the experiences of individuals using NHS services is collected by questionnaires. However, there are specific considerations for reported data because it relies on individuals accurately responding to questions or recording information. If the data collection is not carefully managed, results may be affected by biases. For example, participants may not be entirely honest in their responses and may try to respond in the way they think is expected by those around them. This is not a concern for all reported measures, as some types of data collection method are interested in the interaction between individuals and their expectations (such as focus groups).

Additional considerations for reported data collection methods include that response rates can be low and those who respond may not represent the group being studied. For example, a postal questionnaire sent out to everyone in a neighbourhood to ask about noise pollution may only be filled in and returned by residents with the strongest opinions on the topic. There are occasions where researchers prefer to collect measured rather than self-reported data because there are fewer potential biases. For example, it might be preferable to monitor how many times an individual exercises by asking them to wear a fitness device than it would be to ask them to log this information themselves. However, there are also occasions where researchers want to collect self-reported data to reflect the perspectives and experiences of participants. For example, running focus groups to find out how much participants think they should be exercising and how much they believe they do exercise.