Quantitative Research with Nonexperimental Designs

by Janet Salmons, PhD Manager, Sage Research Methods Community

---

What is the difference between experimental and non-experimental research designs?

There are two types of quantitative research designs: experimental and nonexperimental. This introduction draws from The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Leung and Shek (2018) explain:

Experimental research design utilizes the principle of manipulation of the independent variables and examines its cause-and-effect relationship on the dependent variables by controlling the effects of other variables. Usually, the experimenter assigns two or more groups with similar characteristics. Different interventions will be given to the groups. In case there are differences in the outcomes among the groups, the experimenter can conclude that the differences result from the interventions that the experimenter performed. [Learn more about experimental design in an earlier Sage Research Methods Community post, Quantitative Research with Experimental Designs. ]

Nonexperimental research designs examine social phenomena without direct manipulation of the conditions that subjects experience. Subjects are not randomly assigned to different groups. As such, evidence that supports the cause-and-effect relationships is largely limited.

There are two main types of nonexperimental research designs: comparative design and correlational design.

In comparative research, the researcher examines the differences between two or more groups on the phenomenon that is being studied. For example, studying gender difference in learning mathematics is a comparative research.

The correlational design is a study of relationships between two or more constructs. A positive correlation means that high values of a variable are associated with high values of another variable. For instance, academic performance of students is positively related to their self-esteem. On the contrary, a negative correlation means that high values of a variable are associated with low values of the other variable. For example, teacher–student conflicts are negatively related to the students’ sense of belonging to the school.

See how researchers use non-experimental research design in this multidisciplinary collection of open-access articles.

Open Access Comparative Studies

Humprecht, E., Hellmueller, L., & Lischka, J. A. (2020). Hostile Emotions in News Comments: A Cross-National Analysis of Facebook Discussions. Social Media + Society. https://doi.org/10.1177/2056305120912481

Abstract. Recent work demonstrates that hostile emotions can contribute to a strong polarization of political discussion on social media. However, little is known regarding the extent to which media organizations and media systems trigger hostile emotions. We content-analyzed comments on Facebook pages from six news organizations (N = 1,800) based in the United States and Germany. Our results indicate that German news organizations’ Facebook comments are more balanced, containing lower levels of hostile emotions. Such emotions are particularly prevalent in the polarized information environment of the United States—in both news posts and comments. Moreover, alternative right-wing media outlets in both countries provoke significantly higher levels of hostile emotions, thus limiting deliberative discussions. Our results demonstrate that the application of technology—such as the use of comment sections—has different implications depending on cultural and social contexts.

Huang, J., Kumar, S., & Hu, C. (2020). Does Culture Matter? A Comparative Study on the Motivations for Online Identity Reconstruction Between China and Malaysia. SAGE Open. https://doi.org/10.1177/2158244020929311

Abstract. On social network platforms, people may reconstruct an identity due to various reasons, such as vanity, disinhibition, bridging social capital, and privacy concerns. This study aims to identify cultural differences in the motivations for online identity reconstruction between China and Malaysia. Data were collected from China and Malaysia using an online survey. A total of 815 respondents (418 Chinese and 397 Malaysians) participated in this study. Differences were found not only between Chinese and Malaysian participants but also among participants from different ethnic groups (e.g., the Malaysian-Malays and the Malaysian-Chinese). This study adds knowledge to the research concerning online identity reconstruction by taking into account national culture. It also extends the cross-cultural research concerning social network platforms and sheds light on the specific differences between Chinese and Malaysian participants. The findings of this study can help service providers to deploy specific strategies to better serve social network platform users from different countries.

Kalogeropoulos, A., Negredo, S., Picone, I., & Nielsen, R. K. (2017). Who Shares and Comments on News?: A Cross-National Comparative Analysis of Online and Social Media Participation. Social Media + Society. https://doi.org/10.1177/2056305117735754

Abstract. In this article, we present a cross-national comparative analysis of which online news users in practice engage with the participatory potential for sharing and commenting on news afforded by interactive features in news websites and social media technologies across a strategic sample of six different countries. Based on data from the 2016 Reuters Institute Digital News Report, and controlling for a range of factors, we find that (1) people who use social media for news and a high number of different social media platforms are more likely to also engage more actively with news outside social media by commenting on news sites and sharing news via email, (2) political partisans on both sides are more likely to engage in sharing and commenting particularly on news stories in social media, and (3) people with high interest in hard news are more likely to comment on news on both news sites and social media and share stores via social media (and people with high interest in any kind of news [hard or soft] are more likely to share stories via email). Our analysis suggests that the online environment reinforces some long-standing inequalities in participation while countering other long-standing inequalities. The findings indicate a self-reinforcing positive spiral where the already motivated are more likely in practice to engage with the potential for participation offered by digital media, and a negative spiral where those who are less engaged participate less.

Phillips, M., & Smith, D. P. (2018). Comparative approaches to gentrification: Lessons from the rural. Dialogues in Human Geography, 8(1), 3–25. https://doi.org/10.1177/2043820617752009

Abstract. The epistemologies and politics of comparative research are prominently debated within urban studies, with ‘comparative urbanism’ emerging as a contemporary lexicon of urban studies. The study of urban gentrification has, after some delay, come to engage with these debates, which can be seen to pose a major challenge to the very concept of gentrification. To date, similar debates or developments have not unfolded within the study of rural gentrification. This article seeks to address some of the challenges posed to gentrification studies through an examination of strategies of comparison and how they might be employed within a comparative study of rural gentrification. Drawing on Tilly (Big structures Large Processes Huge Comparisons. New York: Russell Sage), examples of four ‘strategies of comparison’ are identified within studies of urban and rural gentrification, before the paper explores how ‘geographies of the concept’ and ‘geographies of the phenomenon’ of rural gentrification in the United Kingdom, United States and France may be investigated using Latour’s (Pandora’s Hope. London: Harvard University Press) notion of ‘circulatory sociologies of translation’. The aim of our comparative discussion is to open up dialogues on the challenges of comparative studies that employ conceptions of gentrification and also to promote reflections of the metrocentricity of recent discussions of comparative research.

Yaşar, H., & Sağsan, M. (2020). The Mediating Effect of Organizational Stress on Organizational Culture and Time Management: A Comparative Study With Two Universities.SAGE Open.  https://doi.org/10.1177/2158244020919507

Abstract. This research was designed to investigate whether organizational stress had an intermediary role in the effect of Hofstede cultural dimensions on time management. Near East University from Cyprus, which represents the individual culture, and Hakkari University from Turkey representing the collectivist culture were selected for the research analyses. In all, 638 administrative and academic members from both universities were interviewed face-to-face on a voluntary basis, and data were collected by the simple random sampling method. The research findings suggest that time should be managed after identifying the type of culture—individualistic or collectivist—to decrease the level of stress experienced by university staff. In other words, Hofstede’s cultural dimension has an impact on time management, and organizational stress has a partial mediation effect on this dimension. Although the variables in the study have been studied in the literature together with many different factors, Hofstede is significant in terms of determining the role of organizational stress in the effect of cultural dimensions on time management. The effectiveness of Hofstede’s cultural dimensions through organizational stress in time management allows business and project plans to be carried out in a way that manages individual, team or departmental performances taking into account the organizational stress elements. It is considered that this study will particularly be effective in medicine, project management, and independent auditing.

Open Access Correlational Studies

Adams, R. V., & Blair, E. (2019). Impact of Time Management Behaviors on Undergraduate Engineering Students’ Performance. SAGE Open. https://doi.org/10.1177/2158244018824506

Abstract. Effective time management is associated with greater academic performance and lower levels of anxiety in students; however many students find it hard to find a balance between their studies and their day-to-day lives. This article examines the self-reported time management behaviors of undergraduate engineering students using the Time Management Behavior Scale. Correlation analysis, regression analysis, and model reduction are used to attempt to determine which aspects of time management the students practiced, which time management behaviors were more strongly associated with higher grades within the program, and whether or not those students who self-identified with specific time management behaviors achieved better grades in the program. It was found that students’ perceived control of time was the factor that correlated significantly with cumulative grade point average. On average, it was found that time management behaviors were not significantly different across gender, age, entry qualification, and time already spent in the program.

Cooper, B., & Glaesser, J. (2010). Contrasting Variable-Analytic and Case-Based Approaches to the Analysis of Survey Datasets: Exploring How Achievement Varies by Ability across Configurations of Social Class and Sex. Methodological Innovations Online, 5(1), 3–23. https://doi.org/10.4256/mio.2010.0007

Abstract. The context for this paper is the ongoing debate concerning the relative merits, for the analysis of quantitative data, of, on the one hand, variable-analytic correlational methods, and, on the other, the case-based set theoretic methods developed by Charles Ragin. While correlational approaches, based in linear algebra, typically use regression to establish the net effects of several “independent” variables on an outcome, the set theoretic approach analyses, more holistically, the conjunctions of factors sufficient and/or necessary for an outcome to occur. Here, in order to bring out key differences between the approaches, we focus our attention on the basic building blocks of the two approaches: respectively, the concept of linear correlation and the concept of a sufficient and/or necessary condition. We initially use invented data (for ability, educational achievement, and social class) to simulate what is at stake in this methodological debate and we then employ data taken from the British National Child Development Study to explore the structuring of the relationship between respondents' early measured ability and later educational achievement across various configurations of parental and grandparental class origin and sex. The substantive idea informing the analysis, derived from Boudon's work, is that, for respondents from higher class origins, ability will tend to be sufficient but not necessary for later educational achievement while, for lower class respondents, ability will tend to be necessary but not sufficient. We compare correlational analyses, controlling for class and gender, with fuzzy set analyses to show that set theoretic indices can better capture these varying relationships than correlational measures. In conclusion, we briefly consider how our demonstration of some of the advantages of the set theoretic approach for modelling empirical relationships might be related to the debate concerning the relation between observed regularities and causal mechanisms.

Fang, C., Gai, Q., He, C., & Shi, Q. (2020). The Experience of Poverty Reduction in Rural China. SAGE Open. https://doi.org/10.1177/2158244020982288

Abstract. Since 1978, China has greatly reduced the rural poverty rate. This article provides an overview of the experience of China’s poverty reduction. Using panel data from 1996 to 2013 to calculate farmers’ income dynamics, we found that the pace of poverty reduction was relatively slow from 1996 to 2002 and that the rate of reversion to poverty was high. Since 2003, the pace of poverty reduction has accelerated, whereas the rate of reversion has decreased. Using econometric ordinary least squares and probit models, we explore the factors that drive poverty reduction. We found correlational evidence that the main reasons for poverty reduction in China since 1996 have been the increase in income from household farms and migrant work. In addition, rural public insurance prevented farmers from falling into poverty.

Hayn-Leichsenring, G. U., Lehmann, T., & Redies, C. (2017). Subjective Ratings of Beauty and Aesthetics: Correlations With Statistical Image Properties in Western Oil Paintings. I-Perception. https://doi.org/10.1177/2041669517715474

Abstract. For centuries, oil paintings have been a major segment of the visual arts. The JenAesthetics data set consists of a large number of high-quality images of oil paintings of Western provenance from different art periods. With this database, we studied the relationship between objective image measures and subjective evaluations of the images, especially evaluations on aesthetics (defined as artistic value) and beauty (defined as individual liking). The objective measures represented low-level statistical image properties that have been associated with aesthetic value in previous research. Subjective rating scores on aesthetics and beauty correlated not only with each other but also with different combinations of the objective measures. Furthermore, we found that paintings from different art periods vary with regard to the objective measures, that is, they exhibit specific patterns of statistical image properties. In addition, clusters of participants preferred different combinations of these properties. In conclusion, the results of the present study provide evidence that statistical image properties vary between art periods and subject matters and, in addition, they correlate with the subjective evaluation of paintings by the participants.

Srinivasan P, Rentala S, Kumar P. Impulsivity and Aggression Among Male Delinquent Adolescents Residing in Observation Homes—A Descriptive Correlation Study from East India. Journal of Indian Association for Child and Adolescent Mental Health. 2022;18(4):327-336. doi:10.1177/09731342231171305

Aggression and crime are connected and highly reported among juveniles in recent times as compared to adults, which ends up in delinquency. It is not just aggression that dominates but the associated impulsiveness also plays a vital role. This study was intended to assess impulsivity and aggression, and their relationship among male delinquent adolescents residing in observation homes. A quantitative research approach with the nonexperimental descriptive correlation design was adopted. One hundred and seventy-nine male delinquent adolescents residing in 2 observation homes in the state of Bihar, India, were selected by convenience sampling technique. The standardized Buss & Perry Aggression questionnaire, and Barratt Impulsiveness scale were used for collecting the data regarding impulsivity and aggression among male delinquent adolescents.

Yamak, O. U., & Eyupoglu, S. Z. (2021). Authentic Leadership and Service Innovative Behavior: Mediating Role of Proactive Personality. SAGE Open. https://doi.org/10.1177/2158244021989629

Abstract. The present study aims to examine the effect of authentic leadership (AL) on service innovative behavior (SIB) of employees as well as to identify whether proactive personality (PP) mediates this connection at an individual level. The quantitative cross-sectional study design was utilized to gather information from a study sample which consisted of 428 front-line employees (FLE) working at banks located in North Cyprus. Specifically, the study uses confirmatory factor analysis (CFA), correlation, structural equation modeling (SEM), and bootstrapping techniques to test the hypothesized relationships. The results reveal that both AL and PP have a significant positive effect on SIB; AL has a positive impact on PP of FLE, and PP plays a partial mediating role between AL and SIB of FLE. By relating the study findings, authenticity and proactivity in the banking sector in North Cyprus play a critical role in fostering the innovative behaviors of FLE. The study also discusses the practical and managerial implications, as well as the future scope.

Books about Quantitative Methods from Sage Publishing

Use the code COMMUNITY3 for a 20% discount when you order her book, valid worldwide until March 31, 2024.

The Research Experience: Planning, Conducting, and Reporting Research by Ann Sloan Devlin (2020). (See Sage Research Methods Community posts by Dr. Devlin.)

Understanding Quantitative Data in Educational Research by Nicoleta Gaciu (2021).

Essentials of Social Statistics for a Diverse Society by Anna Leon-Guerrero, Chava Frankfort-Nachmias, and Georgiann Davis.(2020). (See an author interview for more on this book.)

Understanding Correlation Matrices by Alexandria Hadd and Joseph Lee Rodgers (2021)

Know Your Variables: Little Quick Fix and other titles in the LQF series by John MacInnes

Research Methods for the Behavioral Sciences by Gregory J. Privitera (2022)

---

Reference
Frey, B. (2018). The SAGE encyclopedia of educational research, measurement, and evaluation (Vols. 1-4). Thousand Oaks,, CA: SAGE Publications, Inc. doi: 10.4135/9781506326139

---

More posts about quantitative research methods