ICPSRSuerProgra Coparinggroupsin regressionodelsfor binaryoutcoes ScottLong IndianaUniversity BlalockLecture August2010 Arethe effects ofproductivityontenurethesaeforenandoen? 1. Acoonsolutionforcoparinggroups: a. Estiateodelforoen. b. Estiatesaeodelforen. c. Coparecoefficientsacrossgroups. d. Concludeeithertheyarethesaeortheyarenot. 2. Toprobles: a. NRCpaneldidnotfindthecoefficientsinforative. b. PaulAllisonsenteaorkingpaperthatsaid: Differencesintheestiatedcoefficientstellusnothingaboutthe differencesintheunderlyingipactof[publications]on[tenurefor]the togroups.
1. RevieethodsforcoparinggroupsintheLRM. 2. Reviebinarylogitandprobit(BRM)intersof: a. Alatentvariableodelfory*. b. AprobabilityodelforPr(y=1 x). 3. WhyisthereaproblecoparingcoefficientsacrossgroupsinBRM? 4. ToapproachestocoparinggroupsinBRM: a. Briefly:Allison'stestsforcoparingcoefficients. b. Indepth:Usingpredictedprobabilitiestocoparegroups. Men: Woen: y art icles prestige prestige y articl es prestige prestige 1. Doenandoenhavethesaereturnfor? A : H 0 2. Wecoputeattest: ˆ ˆ t ˆ ˆ Var Var 3. Wecouldtestthatallcoefficientsareequal: B : ; ; H0 B 4. Critically, H0 thisdoesnotiply: C 2 2 H 0 : R R prestige prestige
1. Testing H 0: isappropriateinlrm. 2. IntheBRMthetestconfounds: a. Groupdifferencesintheeffectofx. b. Groupdifferencesinunexplainedvariation. 3. ToapproachescanbeusedfortheBRM: a. Allison'stestthatassueseffectsofothervariablesareequalacross groups. b. Testscoparingpredictedprobabilities. Pr y 1 x F0 xxzz Logit Probit y x0 xxzz exp 0 xx 1 exp x Pr 1 z y x x z Pr 1 0 0 x x z z 2 0xxzz 1 t dt 2 2 z z
1. Duyonly:Useonlyaduyvariableforgroup. 2. Interactions:Allogroupdifferencesintheeffectsofthepredictors suchas and. a. Testtheequalityofcoefficients. b. Coparepredictionsacrossgroups. y Pr y 1 Pr 1 x x Variable Mean StdDev Miniu Maxiu Label ------------------ tenure 0.12 0.33 0.00 1.00 Is tenured? feale 0.38 0.48 0.00 1.00 Scientist is feale? year 3.86 2.30 1.00 10.00 Years in rank. yearsq 20.17 22.15 1.00 100.00 Years in rank squared. select 5.00 1.41 1.00 7.00 Selectivity of bachelor's 7.05 6.58 0.00 73.00 Total nuber of. prestige 2.65 0.78 0.65 4.80 Prestige of departent. presthi 0.05 0.21 0.00 1.00 Prestige is 4 or higher? N = 2797 Pr tenure 1 x 0 fealefeale yearyear+ yearsqyearsq s electselect presthipresthi logit (N=2797): Factor Change in Odds Odds of: Tenure vs NoTenure tenure b z P> z e^b e^bstdx feale -0.35260-2.677 0.007 0.7029 0.8429 year 1.69865 10.426 0.000 5.4666 49.9816 yearsq -0.12295-8.748 0.000 0.8843 0.0656 select 0.12228 2.699 0.007 1.1301 1.1878 0.04948 5.986 0.000 1.0507 1.3845 presthi -1.05052-2.662 0.008 0.3498 0.8009 _cons -7.59000-15.025 0.000 b = ra coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bstdx = exp(b*sd of X) = change in odds for SD increase in X
1. Odds xz, 2. Odds: Pr y 1 x, z Pr y 0 x, z Oddsratios: Oddsx1, z exp x Odds xz, exp 0.70. 3. Oddsratioforfeale: feale Beingafealescientistdecreasestheoddsoftenurebyafactorof.70, holdingallothervariablesconstant. exp 1.05. Foreachadditionalarticle,theoddsoftenureincreasebyafactorof1.05, holdingallothervariablesconstant. 2. Oddsratiofor: Botharroscorrespondtothesaeoddsratio. 1. Coputethepredictedprobabilityatspecificvaluesoftheindependent variables: 0 feale feale yearyear+ yearsqyearsq Pr tenure 1 x selectselect presthipresthi 2. Forexaple,theprobabilityoftenurefornonpublishingoeninyear7, ithselectivity4andloprestige: 0 feale 1 year 7yearsq 49 0.16 select 4 0 presthi.05 3. Extendingthisidea,plotsofprobabilitiescanbeconstructed...
Pr y 1 x F0 x z x z For average scientists in year seven Probability of Tenure 0.2.4.6.8 1 Woen Men Nuber of Articles #3 groups02_argins.do jsl 2010-07-28 1. Probabilityoftenureisabout.06higherforenatalllevelsofproductivity. 2. Lackofinteractionsithgenderisunrealistic. 1. IntheBRM: W oen: Pry 1 ar ticles prest ige prestige Men: Pry 1 presti ge prestige 2. CaneuseaChotypetest? : H0 3. Allison(1999)shothatbecauseofanidentificationproble,theusual testsofthishypothesistellsusnothingabouttheunderlyingipactof forenandoen.
1. Structuralodelithalatenty*: y x 2. 3. 4. Errorisnoral(0,1)forprobit;islogistic(0, 2 /3)forlogit. Observedyandlatenty*arelinkedby: 1 if y 0 y 0 if y 0 Graphically 5. Theprobabilitydependsontheerrordistributionandthecoefficients: y x y x Pr 1 Pr 0 Pr x x 6. Thereisanidentificationproblethatcanbeillustratedgraphically.
GroupW:=12,=2,=2 GroupM:=6,=1,=1 7. The effect (i.e.,)ofxisticeaslargeforwthanm. 8. Intersof Pr( y 1),theseMandWareepiricallyindistinguishable: ForW: Achangeof1inxhen=2and=2. ForM: Achangeof1inxhen=1and=1. 1. Lety*bethelatentvariableassociatedithreceiptoftenure: Woen: y Men: y 2. Assuethecoefficientsforareequal: 3. Assueoenhaveoreunexplainedvariation(hyouldthey?): 2 2 4. Whenyouestiatetheodel,thesoftareakesiplicitassuptions: Logit:Var 2 /3 Probit:Var 1 5. Whatistheeffectoftheseiplicitassuptions? 6. Withprobit,isrescaledsothat: Var Var 1 7. Foroen,theestiatedodelforprobitis: y, here 1 8. Foren,theestiatedodelforprobitis: y, here 1
9. Weanttotest: H : NTilda o 0 s article 10. But,standardsoftareonlyallosustotest: H Tilda : 0 11. Unless, 2 2 NoTilda Tilda H is not equivalent to H 0 0 Aside: rescaling errors in logit is essier 1. Theodelis: y 2. Werescaletheerrorssothat: Var 2 3 rather than 1 for probit 3. Thisleadstotheequationthatisestiated: y 3 3 3 3 Todistinctapproachesaddresstheidentificationproble. 1. Allison'stestof H 0: x x disentanglesthe 'sandvar. a. Thetestrequiresthestrongassuption: z z z or equivalently 1 z b. Theratioof z sestatesgroupdifferencesinunexplainedvariation: z z / z 1 z z / z c. Thisprovidesleveragetotest: : H 0 x x
2. Alternatively,sinceprobabilitiesareinvarianttoVar,Iproposetesting: x x H0: Pr y 1 Pr y 1 Woen:=12,=2,=2 Men:=6,=1,=1 1. Let 1 foroen,else0and x x; let 1 foren,else0and x x. Pr y 1 x F x x x x 2.Then: x x x F x x Pr y 1 F x if 1, 0 Pr y 1 if 0, 1 3.Thegenderdifferenceintheprobabilityoftenureis: Pr y 1 Pr y 1 x x x Startithasipleodelithonlypublicationspredictingtenure: logit (N=2797): Factor Change in Odds Odds of: Tenure vs NoTenure WOMEN b z P> z e^b e^bstdx constant -2.50116-17.858 0.000 0.0820 0.2974 0.04714 4.490 0.000 1.0483 1.3150 MEN b z P> z e^b e^bstdx constant -2.72101-22.402 0.000 0.0658 0.2673 0.10239 9.756 0.000 1.1078 1.8054 ----------- b = ra coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bstdx = exp(b*sd of X) = change in odds for SD increase in X
#7 groups02_argins.do jsl 2010-07-28 #7 groups02_argins.do jsl 2010-07-28 #7 groups02_argins.do jsl 2010-07-28 #7 groups02_argins.do jsl 2010-07-28 Tocoparegroupsatdifferentlevelsof: 1. Coputediscretechange(aka:firstdifference): Pr y 1 Pr y 1 2. Confidenceintervalsforpredictions:, LoerBound UpperBound Withrepeatedsapling,eouldexpectthepredictedprobabilitytofall ithinthecininetyfivepercentofthetie. 3. ForCI,deltaorendpointethodareeasyandfast; bootstraprequiresatleast1,000replicationstogetreliableresults. 4. WithoneRHSvariable,ecanplotallcoparisons. 5.Movingfropredictionsforeachgrouptodifferencesinpredictions: A:Probablitiesbygroup B:DiscretechangeithCI Probability of Tenure 0.2.4.6.8 1 Woen Men Pr(en) - Pr(oen) -.1 0.1.2.3.4.5.6.7.8 Difference is not significant 95% confidence interval Nuber of Articles Nuber of Articles 6.Orusingdashedlinestoindicatedifferencesthatarenotsignificant. C:Discretechangeithbrokenline B:DiscretechangeithCI Pr(en) - Pr(oen) -.1 0.1.2.3.4.5.6.7.8 Difference is not significant Difference is significant Pr(en) - Pr(oen) -.1 0.1.2.3.4.5.6.7.8 Difference is not significant 95% confidence interval Nuber of Articles Nuber of Articles
1. 2. Addingvariablesintroducessubstantialcoplicationsforinterpretation: Withtoindependentvariables: Pr y 1 xz, F xxzz 3. Setting z Z changestheinterceptinanequationithonly x: Pr y 1 x, Z F xxzz 4. F zz xx F x x Thus,probabilitiesanddiscretechangesdependonthelevelsofeach variable. Controlvariableschangetheintercept 1. Foragiven z Z: 2. Men: Woen: Pr 1 y x Z F x x y x, Z F x x Pr 1, Differencesinprobabilitiesforagiven x dependsonthelevelofother variables: x, Z Pr y 1 x, Z Pr y 1 x, Z logit (N=2797): Factor Change in Odds Odds of: Tenure vs NoTenure WOMEN b z P> z e^b e^bstdx constant -2.60432-17.320 0.000 0.0740 0.2829 0.06761 5.358 0.000 1.0699 1.4811 presthi -1.98396-2.685 0.007 0.1375 0.7484 MEN b z P> z e^b e^bstdx constant -2.71499-22.268 0.000 0.0662 0.2681 0.10554 9.890 0.000 1.1113 1.8385 presthi -0.94529-2.058 0.040 0.3886 0.8627
Probability of Tenure 0.2.4.6.8 1 Woen - not distinguished Woen - distinguished Men - not distinguished Men - distinguished Nuber of Articles #12 groups02_argins.do jsl 2010-07-28 Pr(en) - Pr(oen) -.1 0.1.2.3.4.5.6.7.8 Distinguished if not significant Not distinguished if not significant Nuber of Articles #13 groups02_argins.do jsl 2010-07-28 logit (N= 2797): Factor Change in Odds Odds of: Tenure vs NoTenure WOMEN b z P> z e^b e^bstdx constant -5.84198-6.747 0.000 0.0029 0.0589 year 1.40777 5.472 0.000 4.0868 30.1273 yearsq -0.09559-4.364 0.000 0.9088 0.1857 select 0.05513 0.769 0.442 1.0567 1.1534 0.03395 2.693 0.007 1.0345 1.2181 prestige -0.37079-2.376 0.017 0.6902 0.6013 b = ra coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bstdx = exp(b*sd of X) = change in odds for SD increase in X
#18 icpsr_groups.do jsl 2009-07-15 #18 icpsr_groups_2010.do jsl 2010-07-26 #18 icpsr_groups_2010.do jsl 2010-07-26 #18 icpsr_groups_2010.do jsl 2010-07-26 logit (N= 2797): Factor Change in Odds Odds of: Tenure vs NoTenure MEN b z P> z e^b e^bstdx constant -7.68016-11.271 0.000 0.0005 0.0241 year 1.90885 8.915 0.000 6.7454 130.9789 yearsq -0.14322-7.699 0.000 0.8666 0.0622 select 0.21577 3.513 0.000 1.2408 1.7711 0.07369 6.367 0.000 1.0765 1.5299 prestige -0.43119-3.963 0.000 0.6497 0.5418 b = ra coefficient z = z-score for test of b=0 P> z = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bstdx = exp(b*sd of X) = change in odds for SD increase in X Probabilitiesbygroup DiscretechangeithCI Plotted at prestige = 5 Plotted at prestige = 5 Pr(tenure) 0.2.4.6.8 1 Men Woen Pr(en) - Pr(oen) -.1.1.3.5.7 95% confidence interval Male-Feale difference Nuber of Nuber of DiscretechangeithCI Discretechangeithbrokenline Plotted at prestige = 5 Plotted at prestige = 5 Pr(en) - Pr(oen) -.1.1.3.5.7 95% confidence interval Male-Feale difference Pr(en) - Pr(oen) -.1.1.3.5.7 Nuber of Nuber of
Holdingallothervariablesconstant: Pr(en) - Pr(oen) 0.1.2.3.4.5 Weak (prestige=1) Adequate (prestige=2) Good (prestige=3) Strong (prestige=4) Distinguished (prestige=5) Nuber of #19 icpsr_groups.do jsl 2009-07-15 Holdingallothervariablesconstant: Pr(en) - Pr(oen) 0.1.2.3.4.5 no 10 20 30 40 50 1 2 3 4 5 Job Prestige #21 icpsr_groups.do jsl 2009-07-15
1. IntheLRMecanusestandardteststocoparecoefficientsacrossgroups. 2. IntheBRM,thesetestsareprobleaticduetoidentificationissues. 3. Wecanakestrongerassuptionstotestthecoefficients. 4. Testscoparingpredictedprobabilitiesareunaffectedbytheidentification proble. 1. 2. 3. Youustdealiththeinterpretationofnonlinearodelssinceasingletest isnotpossible.shouldyouadjustforultipletests? Doprobabilitiesshothe effect of(say)? Doyouhavesufficientobservationstosupportyourconclusions.Arethe preditions onthesupport? Allison,PaulD.1999.CoparingLogitandProbitCoefficientsAcrossGroups.Sociological MethodsandResearch28:186208. Cho,G.C.1960.Testsofequalitybeteensetsofcoefficientsintolinearregressions. Econoetrica28:591605. Long,J.S.2009(2005).CoparingGroupsusingPredictedOutcoes. Long,J.S.andFreese,J.2005.RegressionModelsforCategoricalandLiitedDependent VariablesithStata.SecondEdition.CollegeStation,TX:StataPress. Long,J.Scott,PaulD.Allison,andRobertMcGinnis.1993.RankAdvanceentinAcadeic Careers:SexDifferencesandtheEffectsofProductivity.AericanSociologicalRevie 58:703722. Willias,Richard.2009.UsingHeterogeneousChoiceModelstoCopareLogitandProbit CoefficientsacrossGroups.SociologicalMethods&Research37:531559. Xu,JunandLong,J.2005,Confidenceintervalsforpredictedoutcoesinregression odelsforcategoricaloutcoes.thestatajournal5:537559.
.indiana.edu/~jslsoc/index.ht Stata10at.stata.coithauxiliarySPostpackage(seeLongand Freese2005). Stata11usingargins.