{"id":49,"date":"2023-09-14T18:57:23","date_gmt":"2023-09-14T22:57:23","guid":{"rendered":"https:\/\/openbooks.macewan.ca\/researchmethods\/?post_type=chapter&p=49"},"modified":"2024-07-16T15:56:48","modified_gmt":"2024-07-16T19:56:48","slug":"chapter-4-research-design-and-measurement","status":"publish","type":"chapter","link":"https:\/\/openbooks.macewan.ca\/researchmethods\/chapter\/chapter-4-research-design-and-measurement\/","title":{"raw":"Chapter 4: Research Design and Measurement","rendered":"Chapter 4: Research Design and Measurement"},"content":{"raw":"

In any discussion about improving measurement, it is important to begin with basic questions. What exactly are we trying to measure, and why? <\/em><\/p>\r\n

\u2014 Christine Bachrach, 2007, p. 435<\/p>\r\n\r\n

\r\n

Learning Objectives<\/p>\r\n\r\n<\/header>\r\n

\r\n\r\nAfter reading this chapter, students should be able to do the following:\r\n
    \r\n \t
  1. Describe the main components of a research design.<\/li>\r\n \t
  2. Explain what conceptualization and operationalization processes entail.<\/li>\r\n \t
  3. Explain how the purpose of a variable is directly related to how it is measured in research.<\/li>\r\n \t
  4. Outline the main techniques used to assess reliability and validity.<\/li>\r\n \t
  5. Distinguish between random and systematic errors.<\/li>\r\n \t
  6. Explain how rigour is achieved in qualitative research.<\/li>\r\n<\/ol>\r\n<\/div>\r\n<\/div>\r\n

    INTRODUCTION<\/h2>\r\nAfter carefully considering research foundations, the importance of theory, and the ethics involving research with humans, you are almost ready to delve into the techniques used for obtaining answers to social research questions. But before you can start collecting data, you need to develop a research plan that outlines who or what it is you are studying and how you will go about measuring and evaluating the attitudes, behaviours, or processes that you want to learn more about.\r\n

    MAIN COMPONENTS OF A RESEARCH DESIGN<\/h2>\r\nLinked to a specific research question, a [pb_glossary id=\"526\"]research design[\/pb_glossary]<\/strong> \u201cis the plan or blueprint for a study and includes the who, what, where, when, why, and how of an investigation\u201d (Hagan, 2021, Chapter 3, para 1.). Beginning with the \u201cwho,\u201d researchers in the social sciences most often study people, so individuals are usually the focus of investigation, called the [pb_glossary id=\"632\"]unit of analysis[\/pb_glossary]<\/strong>. Individuals are often studied as part of a collective or group. For example, the unit of analysis could be students, employees, single-parent-headed families, low-income earners, or some other group of interest. Researchers also compare groups of individuals along particular variables of interest, such as a sample of individuals with less than a high school education versus those who have a grade 12 diploma, single- versus dual-income families, or patients who completed a treatment program versus ones who dropped out. Social institutions and organizations that guide individuals and groups can also be the units of analysis for research, such as the university you are attending, a not-for-profit agency such as the Canadian Red Cross, or a healthcare organization such as the Canadian Medical Association. Finally, social researchers are sometimes interested in artifacts created by people rather than people themselves. For example, researchers might examine news articles, television shows, motion pictures, profiles on an online dating site, YouTube videos, Facebook postings, or X tweets.\r\n\r\nThe \u201cwhat\u201d component of research design refers to whatever is specifically examined and measured in a study, such as attitudes, beliefs, views, behaviours, outcomes, and\/or processes. The measured component is usually referred to as the \u201cunit of observation\u201d since this is what the data is collected on. For example, a researcher might be interested in factors affecting instructors\u2019 views on published ratings of instruction. Instructors at a university who take part in the study constitute the unit of analysis from whom the views on published ratings are obtained (i.e., the more specific focus of the research that comprises the data). Similarly, a researcher might be interested in dating preferences of individuals who use online matchmaking sites such as eharmony or EliteSingles. The individuals seeking dates online are the units of analysis, while their posted profiles containing the characteristics of interest in the study are the units of observation.\r\n\r\nThe \u201cwhere\u201d pertains to the location for a study. The possibilities are endless, from research conducted first-hand in the field to studies conducted in a public setting such as a coffee shop or a private one such as the home of a participant. The location for a study is closely linked to the unit of analysis since the researcher often needs to go to the individuals, groups, or organizations to collect information from them or about them. For example, a researcher interested in interviewing couples who met online might set up appointments to visit dating couples in restaurants or coffee shops near to where the couples reside, at the discretion of the participants. Alternatively, a researcher interested in online dating relationships might choose to gather information in a virtual environment by posting a survey on the internet.\r\n\r\nThe \u201cwhen\u201d relates to an important time consideration in the research design. Some studies are conducted once at a single point in time, and others are carried out at multiple points over time. Most often, when social researchers carry out a study, it takes place at a single point in time or within a single time frame, such as when a researcher develops a questionnaire about internet dating and administers it to a group of people attending a singles\u2019 social function. This is called [pb_glossary id=\"269\"]cross-sectional research[\/pb_glossary]<\/strong> because the study is made up of a cross-section of the dating population taken at a point in time, like taking a photo that captures a person or group at a single point in time. Alternatively, social researchers sometimes study individuals or groups at multiple points in time using what is called [pb_glossary id=\"412\"]longitudinal research[\/pb_glossary]<\/strong>. For example, a sample of dating couples might be surveyed shortly after they meet and again after dating for one year to capture their initial viewpoints and to see how their perceptions change once they get to know each other better. Four longitudinal designs are discussed below.\r\n\r\nA [pb_glossary id=\"458\"]panel study[\/pb_glossary]<\/strong> is a longitudinal study in which a researcher observes the same people, group, or organization at multiple points in time. A new phase of a large-scale panel study called Alberta\u2019s Tomorrow Project was launched in 2017 to learn more about the causes of cancer. The study, active since 2000, includes 55,000 cancer-free Albertans (Alberta\u2019s Tomorrow Project, 2024b). Researchers from Alberta\u2019s Tomorrow Project collect information on participants\u2019 health and lifestyle through surveys and the occasional collection of other specimens, such as blood samples. Going back to the dating example, by collecting information on the same dating couples at various intervals, a researcher could use a panel study to examine how relations evolve or change over time.\r\n\r\nA [pb_glossary id=\"218\"]cohort study[\/pb_glossary]<\/strong> is like a panel study, except, rather than focusing on the same people, it focuses on comparable people over time. \u201cComparable\u201d refers to a general category of people who are deemed to share a similar life experience in a specified period. The most common example of this is a birth cohort. Think about the classmates you went through school with. Although there were individual differences between you and your classmates, you also shared some common life events, such as the music that topped the charts at that time, the clothing fads and hairstyles, and the political events that occurred during that era. Following the earlier example, a researcher might include people who met online and married their dating partner in 2020 as Covid-19 was unfolding as a unit of analysis. Several couples might be studied in 2020 shortly after they were married, several other couples who also married in 2020 might be studied in 2021, another group in 2022, another group in 2023, and so on over a period of five years.\r\n\r\nA [pb_glossary id=\"622\"]time-series study[\/pb_glossary]<\/strong> is a longitudinal study in which a researcher examines different people at multiple points in time. Every year, Statistics Canada gathers information from thousands of Canadians using what is called the General Social Survey. Although the participants are different each time, similar forms of information are gathered over time to detect patterns and trends. For example, we can readily discern that Canadians today are delaying marriage (i.e., getting married for the first time at a later age) and they are having fewer children relative to 20 years ago. Going back to the online dating example used throughout this section, a researcher might use a time-series study in order gather information from new dating couples at an online site each year for five years. Looking at the data over time, a researcher would be able to determine if there are changes in the overall profiles of online daters at that site. For example, the use of the site might be increasing for groups such as highly educated women or single individuals over the age of 60 years.\r\n\r\nLastly, a [pb_glossary id=\"204\"]case study[\/pb_glossary]<\/strong> is a research method in which a researcher focuses on a small number of individuals or even a single person over an extended period. You\u2019ll learn more about this method in Chapter 11. For now, you can think of a case study as a highly detailed study of a single person, group, or organization over time. For example, a researcher might study an alcoholic to better understand the progression of the disease over time, or a researcher might join a subculture, such as a Magic card club that meets every Friday night, to gain an insider\u2019s perspective of the group. Similarly, a researcher could examine the experiences of a frequent online dater over time to get a sense of how online dating works at various sites. See figure 4.1 for an overview of time considerations in relation to units of analysis.<\/a>\r\n\r\n[caption id=\"attachment_1100\" align=\"aligncenter\" width=\"1022\"]\"Figure<\/a> Figure 4.1. Time Considerations by Units of Analysis [Image description - See Appendix C Figure 4.1<\/a>][\/caption]\r\n
    \r\n

    Research in Action<\/p>\r\n\r\n<\/header>\r\n

    \r\n

    Longitudinal Panel Study on Smokers\u2019 Perceptions of Health Risks<\/strong><\/p>\r\n

    Cho and colleagues (2018) examined changes in smokers\u2019 perceptions about health risks following amendments to warning labels on cigarette packages using a longitudinal panel study. Four thousand six hundred and twenty-one smokers from Canada, Australia, Mexico, and the United States were surveyed every four months for a total of five times to determine if their knowledge of health risks increased after toxic constituents were added to pictorial health warning labels. Results showed that knowledge of toxic constituents such as cyanide and benzene increased over time and was associated with a stronger perceived risk of vulnerability to smoking-related diseases, including bladder cancer and blindness, for participants in Canada, Australia, and Mexico, but not the United States (Cho et al., 2018).<\/p>\r\n\r\n\r\n[caption id=\"attachment_1087\" align=\"aligncenter\" width=\"1024\"]\"Cigarettes Image 4.1 Warning labels are designed to increase smokers\u2019 knowledge of health risks.[\/caption]\r\n\r\n<\/div>\r\n<\/div>\r\nIn addition to the time dimension, a research design also includes the \u201cwhy\u201d and the \u201chow\u201d plan for a study. \u201cWhy\u201d relates to the purpose of a study as discussed in chapter 1. A study on internet dating could be focused on exploring ways in which people represent themselves in posted profiles; describing who uses internet dating sites; explaining the importance of including certain types of information, such as appearance or personality factors, for attracting dates; or evaluating the effectiveness of a given site in matching suitable partners. The nature of the study really depends on the interests of the researcher and the nature of the issue itself.\r\n\r\n\u201cHow\u201d refers to the specific method or methods used to gather the data examined in a study. With an interest in the merit of including certain items in a dating profile, a researcher might opt for an experimental design to explain, for example, whether certain characteristics in posted dating profiles are better than others for eliciting potential dates (see chapter 6 for a discussion of different types of experiments). An experiment is a special type of quantitative research design in which a researcher changes or alters something to see what effect it has. In the dating example, an experimenter might compare the number of responses to two profiles posted at an internet dating site. If the profiles are identical except for one trait\u2014for example, one profile might contain an additional statement saying the person has a great sense of humour\u2014it would be possible to determine the importance of that trait in attracting potential partners.\r\n\r\nRecall from chapter 1 that the specific methods or techniques differ depending on whether a researcher adopts a qualitative or quantitative orientation, and the approach itself links back to the research interest. A researcher interested in learning more about why people make dates online or how people define themselves online is more apt to use a qualitative approach. Qualitative researchers tend to engage in research that seeks to better understand or explain some phenomenon using field research and in-depth interviews, as well as strategies involving discourse analyses and content analyses that can be used to help to uncover meaning. Regardless of the approach taken and specific methods used, all researchers must work through various design considerations and measurement issues in their quest to carry out scientific research. Similarly, both qualitative and quantitative researchers undertake a process of conceptualization and measurement\u2014it just occurs differently and at different stages within the overall research process, as discussed in the next section.\r\n

    \r\n

    Test Yourself<\/p>\r\n\r\n<\/header>\r\n

    \r\n