Πολλαπλή Παλινδρόμηση

Transcript

1 Πολλαπλή Παλινδρόμηση Το μοντέλο της απλής παλινδρόμησης χρησιμοποιήθηκε για την ανάλυση μιας συνεχούς μεταβλητής (η εξαρτημένη μεταβλητή y) με τη βοήθεια μίας άλλης συνεχούς μεταβλητής (την ανεξάρτητη μεταβλητή x). Η πολλαπλή παλινδρόμηση επιτρέπει τη χρήση πολλών ανεξάρτητων μεταβλητών. Το μοντέλο που προκύπτει αναμένεται να περιγράφει καλύτερα τα δεδομένα απ ότι το μοντέλο της απλής παλινδρόμησης. Διαχείριση Πληροφοριών 12.1

2 Το Μοντέλο Υποθέτουμε ότι έχουμε k ανεξάρτητες μεταβλητές που εξαρτώνται δυνητικά με την ανεξάρτητη μεταβλητή. Η εξάρτηση αυτή εκφράζεται με τη γραμμική εξίσωση: εξαρτημένη μεταβλητή ανεξάρτητη μεταβλητή συντελεστές μεταβλητή σφάλματος Στην απλή παλινδρόμηση η γραφική παράσταση είναι μία ευθεία. Στην πολλαπλή έχουμε μία επιφάνεια απόκρισης (response surface). Διαχείριση Πληροφοριών 12.2

3 Required Conditions For these regression methods to be valid the following four conditions for the error variable ( ) must be met: The probability distribution of the error variable ( normal. The mean of the error variable is 0. The standard deviation of is, which is a constant. The errors are independent. ) is Διαχείριση Πληροφοριών 12.3

4 Estimating the Coefficients The sample regression equation is expressed as: We will use computer output to: Assess the model How well it fits the data Is it useful Are any required conditions violated? Employ the model Interpreting the coefficients Predictions using the prediction equation Estimating the expected value of the dependent variable Διαχείριση Πληροφοριών 12.4

5 Regression Analysis Steps u Use a computer and software to generate the coefficients and the statistics used to assess the model. v Diagnose violations of required conditions. If there are problems, attempt to remedy them. w Assess the model s fit. standard error of estimate, coefficient of determination, F-test of the analysis of variance. x If u, v, and w are OK, use the model to predict or estimate the expected value of the dependent variable. Διαχείριση Πληροφοριών 12.5

6 Example 17.1 La Quinta Motor Inns is a moderately priced chain of motor inns located across the United States. Its market is the frequent business traveler. The chain recently launched a campaign to increase market share by building new inns. The management of the chain is aware of the difficulty in choosing locations for new motels. Moreover, making decisions without adequate information often results in poor decisions. Consequently the chain management acquired data on 100 randomly selected inns belonging to La Quinta. The objective was to predict which sites are likely to be profitable. Διαχείριση Πληροφοριών 12.6

7 Example 17.1 To measure profitability La Quinta used operating margin, which is the ratio of the sum of profit, depreciation, and interest expenses divided by total revenue. The higher the operating margin, the greater the success of the inn. La Quinta defines profitable inns as those with an operating margin in excess of 50% and unprofitable inns with margins of less than 30%. Διαχείριση Πληροφοριών 12.7

8 Example 17.1 After a discussion with a number of experienced managers La Quinta decided to select one or two independent variables from each of the categories: Competition Market awareness Demand generators Demographics Physical Διαχείριση Πληροφοριών 12.8

9 Example 17.1 To measure the degree of competition they determined the total number of motel and hotel rooms within 3 miles of each La Quinta inn. Market awareness was measured by the number of miles to the closest competing motel. Two variables that represent sources of customers were chosen. The amount of office space and college and university enrollment in the surrounding community are demand generators. Both of these are measures of economic activity. Διαχείριση Πληροφοριών 12.9

10 Example 17.1 A demographic variable that describes the community is the median household income. Finally, as a measure of the physical qualities of the location La Quinta chose the distance to the downtown core. Διαχείριση Πληροφοριών 12.10

11 Example 17.1 La Quinta Inns Where should La Qunita locate a new motel? Factors influencing profitability Profitability factor Competition Market Awareness Demand Generators Community Physical measure # of rooms within 3 mile radius Distance to the nearest La Quinta inn. Offices, Higher Ed. Median household income. Distance to downtown. *these need to be interval data! Διαχείριση Πληροφοριών 12.11

12 Example 17.1 La Quinta Inns Where should La Qunita locate a new motel? Several possible predictors of profitability were identified, and data (Xm17-01) were collected. Its believed that operating margin (y) is dependent upon these factors: x 1 = Total motel and hotel rooms within 3 mile radius x 2 = Number of miles to closest competition x 3 = Volume of office space in surrounding community x 4 = College and university student numbers in community x 5 = Median household income in community x 6 = Distance (in miles) to the downtown core. Διαχείριση Πληροφοριών 12.12

13 Transformation Can we transform this data: into a mathematical model that looks like this: margin competition (i.e. # of rooms) awareness (distance to nearest alt.) physical (distance to downtown) Διαχείριση Πληροφοριών 12.13

14 Example 17.1 COMPUTE In Excel: Data > Data Analysis > Regression Διαχείριση Πληροφοριών 12.14

15 The Model INTERPRET Although we haven t done any assessment of the model yet, at first pass: it suggests that increases in The number of miles to closest competition, office space, student enrollment and household income will positively impact the operating margin. Likewise, increases in the total number of lodging rooms within a short distance and the distance from downtown will negatively impact the operating margin Διαχείριση Πληροφοριών 12.15

16 Model Assessment We will assess the model in three ways: Standard error of estimate, Coefficient of determination, and F-test of the analysis of variance. Διαχείριση Πληροφοριών 12.16

17 Standard Error of Estimate In multiple regression, the standard error of estimate is defined as: n is the sample size and k is the number of independent variables in the model. We compare this value to the mean value of y: calculate It seems the standard error of estimate is not particularly small. What can we conclude? Διαχείριση Πληροφοριών 12.17

18 Coefficient of Determination Again, the coefficient of determination is defined as: This means that 52.51% of the variation in operating margin is explained by the six independent variables, but 47.49% remains unexplained. Διαχείριση Πληροφοριών 12.18

19 Adjusted R 2 value What s this? The adjusted R 2 is: the coefficient of determination adjusted for degrees of freedom. It takes into account the sample size n, and k, the number of independent variables, and is given by: Διαχείριση Πληροφοριών 12.19

20 Testing the Validity of the Model In a multiple regression model (i.e. more than one independent variable), we utilize an analysis of variance technique to test the overall validity of the model. Here s the idea: H 0 : H 1 : At least one is not equal to zero. If the null hypothesis is true, none of the independent variables is linearly related to y, and so the model is invalid. If at least one validity. is not equal to 0, the model does have some Διαχείριση Πληροφοριών 12.20

21 Testing the Validity of the Model ANOVA table for regression analysis Source of Variation degrees of freedom Sums of Squares Mean Squares F-Statistic Regression k SSR MSR = SSR/k F=MSR/MSE Error n k 1 SSE MSE = SSE/(n k-1) Total n 1 A large value of F indicates that most of the variation in y is explained by the regression equation and that the model is valid. A small value of F indicates that most of the variation in y is unexplained. Διαχείριση Πληροφοριών 12.21

22 Testing the Validity of the Model Our rejection region is: Since Excel calculated the F statistic as F = and our F Critical = 2.17, (and the p-value is zero) we reject H 0 in favor of H 1, that is: there is a great deal of evidence to infer that the model is valid Διαχείριση Πληροφοριών 12.22

23 Table 17.2 Summary SSE R 2 F Assessment of Model Perfect small small close to 1 large Good large large close to 0 small Poor 0 0 Useless Once we re satisfied that the model fits the data as well as possible, and that the required conditions are satisfied, we can interpret and test the individual coefficients and use the model to predict and estimate Διαχείριση Πληροφοριών 12.23

24 Interpreting the Coefficients* Intercept (b 0 ) This is the average operating margin when all of the independent variables are zero. Its meaningless to try and interpret this value, particularly if 0 is outside the range of the values of the independent variables (as is the case here). # of motel and hotel rooms (b 1 ).0076 Each additional room within three miles of the La Quinta inn, will decrease the operating margin. (I.e. for each additional 1000 rooms the margin decreases by 7.6%) Distance to nearest competitor (b 2 ) 1.65 For each additional mile that the nearest competitor is to a La Quinta inn, the average operating margin increases by 1.65%. *in each case we assume all other variables are held constant Διαχείριση Πληροφοριών 12.24

25 Interpreting the Coefficients* Office space (b 3 ).020 For each additional thousand square feet of office space, the margin will increase by.020. E.g. an extra 100,000 square feet of office space will increase margin (on average) by 2.0%. Student enrollment (b 4 ).21 For each additional thousand students, the average operating margin increases by.21% Median household income (b 5 ).41 For each additional thousand dollar increase in median household income, the average operating margin increases by.41% Distance to downtown core (b 6 ).23 For each additional mile to the downtown center, the operating margin decreases on average by.23% *in each case we assume all other variables are held constant Διαχείριση Πληροφοριών 12.25

26 Testing the Coefficients For each independent variable, we can test to determine whether there is enough evidence of a linear relationship between it and the dependent variable for the entire population H 0 : = 0 H 1 : 0 (for i = 1, 2,, k) and using: as our test statistic (with n k 1 degrees of freedom). Διαχείριση Πληροφοριών 12.26

27 Testing the Coefficients INTERPRET We can use our Excel output to quickly test each of the six coefficients in our model Thus, the number of hotel and motel rooms, distance to the nearest motel, amount of office space, and median household income are linearly related to the operating margin. There is no evidence to infer that college enrollment and distance to downtown center are linearly related to operating margin. Διαχείριση Πληροφοριών 12.27

28 Using the Regression Equation Much like we did with simple linear regression, we can produce a prediction interval for a particular value of y. As well, we can produce the confidence interval estimate of the expected value of y. Excel s tools will do the work; our role is to set up the problem, understand and interpret the results. Διαχείριση Πληροφοριών 12.28

29 Using the Regression Equation Predict the operating margin if a La Quinta Inn is built at a location where u There are 3815 rooms within 3 miles of the site. v The closest other hotel or motel is.9 miles away. w The amount of office space is 476,000 square feet. x There is one college and one university nearby with a total enrollment of 24,500 students. y Census data indicates the median household income in the area (rounded to the nearest thousand) is $35,000, and, z The distance to the downtown center is11.2 miles. our x i s Διαχείριση Πληροφοριών 12.29

30 Using the Regression Equation We add one row (our given values for the independent variables) to the bottom of our data set: Then we use: Add-Ins > Data Analysis Plus > Prediction Interval to crunch the numbers Διαχείριση Πληροφοριών 12.30

31 Prediction Interval INTERPRET We predict that the operating margin will fall between 25.4 and If management defines a profitable inn as one with an operating margin greater than 50% and an unprofitable inn as one with an operating margin below 30%, they will pass on this site, since the entire prediction interval is below 50%. Διαχείριση Πληροφοριών 12.31

32 Confidence Interval INTERPRET The expected operating margin of all sites that fit this category is estimated to be between 33.0 and We interpret this to mean that if we built inns on an infinite number of sites that fit the category described, the mean operating margin would fall between 33.0 and In other words, the average inn would not be profitable either Διαχείριση Πληροφοριών 12.32

33 Regression Diagnostics Calculate the residuals and check the following: Is the error variable nonnormal? Draw the histogram of the residuals Is the error variance constant? Plot the residuals versus the predicted values of y. Are the errors independent (time-series data)? Plot the residuals versus the time periods. Are there observations that are inaccurate or do not belong to the target population? Double-check the accuracy of outliers and influential observations. Διαχείριση Πληροφοριών 12.33

34 Regression Diagnostics Multiple regression models have a problem that simple regressions do not, namely multicollinearity. It happens when the independent variables are highly correlated. We ll explore this concept through the following example Διαχείριση Πληροφοριών 12.34

35 Example 17.2 A real estate agent wanted to develop a model to predict the selling price of a home. The agent believed that the most important variables in determining the price of a house are its: u size, v number of bedrooms, w and lot size. The proposed model is: Housing market data has been gathered and Excel is the analysis tool of choice Xm17-02 Διαχείριση Πληροφοριών 12.35

36 Example 17.2 Data > Data Analysis > Regression The F-test indicates the mode is valid but these t-stats suggest none of the variables are related to the selling price. Διαχείριση Πληροφοριών 12.36

37 Example 17.2 Unlike the t-tests in the multiple regression model, these three t-tests tell us that the number of bedrooms, the house size, and the lot size are all linearly related to the price Διαχείριση Πληροφοριών 12.37

38 Example 17.2 How to account for this apparent contradiction? The answer is that the three independent variables are correlated with each other! (i.e. this is reasonable: larger houses have more bedrooms and are situated on larger lots, and smaller houses have fewer bedrooms and are located on smaller lots.) multicollinearity affected the t-tests so that they implied that none of the independent variables is linearly related to price when, in fact, all are Διαχείριση Πληροφοριών 12.38

39 Regression Diagnostics Time Series The Durbin-Watson test allows us to determine whether there is evidence of first-order autocorrelation a condition in which a relationship exists between consecutive residuals, i.e. e i-1 and e i (i is the time period). The statistic for this test is defined as: d has a range of values: 0 d 4. Διαχείριση Πληροφοριών 12.39

40 Durbin Watson 0 d 4 Small values of d (d < 2) indicate a positive firstorder autocorrelation. Large values of d (d > 2) imply a negative firstorder autocorrelation. Διαχείριση Πληροφοριών 12.40

41 Durbin Watson (one-tail test) To test for positive first-order autocorrelation: If d < d L, we conclude that there is enough evidence to show that positive first-order autocorrelation exists. If d > d U, we conclude that there is not enough evidence to show that positive first-order autocorrelation exists. And if d L d d U, the test is inconclusive. d L, d U from table 11, appendix B positive first-order autocorrelation exists inconclusive test positive first-order autocorrelation does not exist d L d U Διαχείριση Πληροφοριών 12.41

42 Durbin Watson (one-tail test) To test for negative first-order autocorrelation: If d > 4 d L, we conclude that there is enough evidence to show that negative first-order autocorrelation exists. If d < 4 d U, we conclude that there is not enough evidence to show that negative first-order autocorrelation exists. And if 4 d U d 4 d L, the test is inconclusive. d L, d U from table 11, appendix B negative first-order autocorrelation does not exist inconclusive test negative first-order autocorrelation exists 4-d U 4-d L Διαχείριση Πληροφοριών 12.42

43 Durbin Watson (two-tail test) To test for first-order autocorrelation: If d < d L or d > 4 d L, first-order autocorrelation exists. If d falls between d L and d U or between 4 d U and 4 d U, the test is inconclusive. If d falls between d U and 4 d U there is no evidence of first order autocorrelation. exists inconclusive doesn t exist inconclusive exists 0 d L d U 2 4-d U 4-d L 4 Διαχείριση Πληροφοριών 12.43

44 Example 17.3 Can we create a model that will predict lift ticket sales at a ski hill based on two weather parameters? Variables: y - lift ticket sales during Christmas week, x 1 - total snowfall (inches), and x 2 - average temperature (degrees Fahrenheit) Our ski hill manager collected 20 years of data. Xm17-03 Διαχείριση Πληροφοριών 12.44

45 Example 17.3 Both the coefficient of determination and the p-value of the F-test indicate the model is poor Neither variable is linearly related to ticket sale Διαχείριση Πληροφοριών 12.45

46 Example 17.3 The histogram of residuals reveals the errors may be normally distributed Διαχείριση Πληροφοριών 12.46

47 Example 17.3 In the plot of residuals versus predicted values (testing for heteroscedasticity) the error variance appears to be constant Διαχείριση Πληροφοριών 12.47

48 Example 17.3 Durbin-Watson Apply the Durbin-Watson Statistic from Data Analysis Plus to the entire list of residuals. Διαχείριση Πληροφοριών 12.48

49 Example 17.3 To test for positive first-order autocorrelation with α =.05, we find in Table 8(a) in Appendix B d L = 1.10 and d U = 1.54 The null and alternative hypotheses are H 0 : There is no first-order autocorrelation. H 1 : There is positive first-order autocorrelation. The rejection region is d < d L = Since d =.59, we reject the null hypothesis and conclude that there is enough evidence to infer that positive first-order autocorrelation exists. Διαχείριση Πληροφοριών 12.49

50 Example 17.3 Autocorrelation usually indicates that the model needs to include an independent variable that has a time-ordered effect on the dependent variable. The simplest such independent variable represents the time periods. We included a third independent variable that records the number of years since the year the data were gathered. Thus, x 3 = 1, 2,..., 20. The new model is y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3 + ε Διαχείριση Πληροφοριών 12.50

51 Example 17.3 The fit of the model is high, The model is valid our new variable Snowfall and time are linearly related to ticket sales; temperature is not Διαχείριση Πληροφοριών 12.51

52 Example 17.3 If we re-run the Durbin-Watson statistic against the residuals from our Regression analysis, we can conclude that there is not enough evidence to infer the presence of first-order autocorrelation. (Determining d L and d U is left as an exercise for the reader ) Hence, we have improved out model dramatically! Διαχείριση Πληροφοριών 12.52

53 Example 17.3 Notice that the model is improved dramatically. The F-test tells us that the model is valid. The t-tests tell us that both the amount of snowfall and time are significantly linearly related to the number of lift tickets. This information could prove useful in advertising for the resort. For example, if there has been a recent snowfall, the resort could emphasize that in its advertising. If no new snow has fallen, it may emphasize their snow-making facilities. Διαχείριση Πληροφοριών 12.53

54 Η ψηφιοποίηση του εκπαιδευτικού υλικού έγινε στο πλαίσιο υλοποίησης της πράξης με τίτλο «ΕΝΙΣΧΥΣΗ ΣΠΟΥΔΩΝ ΠΛΗΡΟΦΟΡΙΚΗΣ στο ΤΕΙ ΚΑΒΑΛΑΣ», του Μέτρου 2.2 «Αναμόρφωση Προγραμμάτων Σπουδών - Διεύρυνση Τριτοβάθμιας Εκπαίδευσης» του ΕΠΕΑΕΚ ΙΙ, που συγχρηματοδοτείται από το Ευρωπαϊκό Κοινωνικό Ταμείο (Ε.Κ.Τ.) κατά 80% και Εθνικούς πόρους κατά 20%. Διαχείριση Πληροφοριών