Regression
Applied Linear Regression Notes set 1
Jamie DeCoster
Department of Psychology
University of Alabama
348 Gordon Palmer Hall
Box 870348
Tuscaloosa, AL 35487-0348
Phone: (205) 348-4431
Fax: (205) 348-8648
September 26, 2006
Textbook references refer to Cohen, Cohen, West, & Aiken’s (2003) Applied Multiple Regression/Correlation
Analysis for the Behavioral Sciences. I would like to thank Angie Maitner and Anne-Marie Leistico for comments made on earlier versions of these notes. If you wish to cite the contents of this document, the
APA reference for them would be:
DeCoster, J. (2006). Applied Linear Regression Notes set 1. Retrieved (month, day, and year you downloaded this file, without the parentheses) from http://www.stat-help.com/notes.html
For future versions of these notes or help with data analysis visit http://www.stat-help.com ALL RIGHTS TO THIS DOCUMENT ARE RESERVED
Contents
1 Introduction and Review
1
2 Bivariate Correlation and Regression
9
3 Multiple Correlation and Regression
21
4 Regression Assumptions and Basic Diagnostics
29
5 Sequential Regression, Stepwise Regression, and Analysis of IV Sets
37
6 Dealing with Nonlinear Relationships
45
7 Interactions Among Continuous IVs
51
8 Regression with Categorical IVs
59
9 Interactions involving Categorical IVs
69
10 Outlier and Multicollinearity Diagnostics
75
i
Chapter 1
Introduction and Review
1.1
Data, Data Sources, and Data Sets
• Most generally, data can be defined as a list of numbers with meaningful relations. We are interested in data because understanding the relations among the numbers can help us understand the relations among the things that the numbers measure.
• The numbers that you collect from an experiment, survey, or archival source is known as a data source.
Before you can learn anything from a data source, however, you must first translate it into a data set.
A data set is a
Jamie DeCoster
Department of Psychology
University of Alabama
348 Gordon Palmer Hall
Box 870348
Tuscaloosa, AL 35487-0348
Phone: (205) 348-4431
Fax: (205) 348-8648
September 26, 2006
Textbook references refer to Cohen, Cohen, West, & Aiken’s (2003) Applied Multiple Regression/Correlation
Analysis for the Behavioral Sciences. I would like to thank Angie Maitner and Anne-Marie Leistico for comments made on earlier versions of these notes. If you wish to cite the contents of this document, the
APA reference for them would be:
DeCoster, J. (2006). Applied Linear Regression Notes set 1. Retrieved (month, day, and year you downloaded this file, without the parentheses) from http://www.stat-help.com/notes.html
For future versions of these notes or help with data analysis visit http://www.stat-help.com ALL RIGHTS TO THIS DOCUMENT ARE RESERVED
Contents
1 Introduction and Review
1
2 Bivariate Correlation and Regression
9
3 Multiple Correlation and Regression
21
4 Regression Assumptions and Basic Diagnostics
29
5 Sequential Regression, Stepwise Regression, and Analysis of IV Sets
37
6 Dealing with Nonlinear Relationships
45
7 Interactions Among Continuous IVs
51
8 Regression with Categorical IVs
59
9 Interactions involving Categorical IVs
69
10 Outlier and Multicollinearity Diagnostics
75
i
Chapter 1
Introduction and Review
1.1
Data, Data Sources, and Data Sets
• Most generally, data can be defined as a list of numbers with meaningful relations. We are interested in data because understanding the relations among the numbers can help us understand the relations among the things that the numbers measure.
• The numbers that you collect from an experiment, survey, or archival source is known as a data source.
Before you can learn anything from a data source, however, you must first translate it into a data set.
A data set is a
References: Aldrich, J. (2005). Fisher and Regression. Retrieved August 24, 2005 from http://www.economics.soton.ac.uk/staff/aldrich/aldrich.htm Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural modeling. Sociological Mathods & Research, 16, 78-117. Carmer, S. G., & Swanson, M. R. (1973). An evaluation of ten pairwise multiple comparison procedures by Monte Carlo methods Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1336. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (3rd edition) Enders, W. (2004). Applied Econometric Time Series. Hoboken, NJ: Wiley. Fisher, R. A. (1928). Statistical Methods for Research Workers (2nd ed.). London: Oliver & Boyd. Freeman, M. F., & Tukey, J. W. (1950). Transformations related to the angular and the square root. Granger, C., & Newbold, P. (1974). Spurious regressions in econometrics. Journal of Econometrics, 2, 111-120. Kenny, D. A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). Applied Linear Statistical Models (4th edition) Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisition and the law of practice. In J. Olkin, I., & Finn, J. D. (1995). Correlations redux. Psychological Bulletin, 118, 155-164. Rencher, A. C., & Scott, D. T. (1990). Assessing the contribution of individual variables following rejection of a multivariate hypothesis Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.