5.1 logistic regresssion Chris Parrish July 3, 2016

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "5.1 logistic regresssion Chris Parrish July 3, 2016"

Ἄμμων Αλεβιζόπουλος
6 χρόνια πριν
Προβολές:

1 5.1 logistic regresssion Chris Parrish July 3, 2016 Contents logistic regression model vote 1 data model fit with stan fitted model figures figure 5.1a figure 5.1b fit with glm fit with stan_glm logistic regresssion reference: - ARM chapter 05, github library(rstan) rstan_options(auto_write = TRUE) options(mc.cores = parallel::detectcores()) library(ggplot2) logistic regression model y i Bernoulli(p i ) logit(p i ) = X i β logit <- function(x){ log(x / (1 - x)) logistic <- function(x){ 1 / (1 + exp(-x)) # logistic = invlogit 1992 vote data # Data source("nes1992_vote.data.r", echo = TRUE) 1

2 > N < > vote <- c(1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, + 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, + 1, 1, 0, 1, 0, 1, 0, 0, 1... [TRUNCATED] > income <- c(4, 2, 1, 2, 3, 4, 2, 4, 1, 4, 4, 1, 3, + 2, 3, 3, 2, 3, 4, 3, 3, 2, 4, 3, 4, 4, 2, 3, 2, 3, 3, 3, + 2, 5, 3, 3, 3, 4, 1, 4, 3,... [TRUNCATED] model nes_logit.stan data { int<lower=0> N; vector[n] income; int<lower=0,upper=1> vote[n]; parameters { vector[2] beta; model { vote ~ bernoulli_logit(beta[1] + beta[2] * income); fit with stan # Logistic model: vote ~ income data.list <- c("n", "vote", "income") nes_logit.sf <- stan(file='nes_logit.stan', data=data.list, iter=1000, chains=4) plot(nes_logit.sf) ci_level: 0.8 (80% intervals) outer_level: 0.95 (95% intervals) beta[1] beta[2] pairs(nes_logit.sf)

3 beta[1] beta[2] lp print(nes_logit.sf, pars = c("beta", "lp ")) Inference for Stan model: nes_logit. 4 chains, each with iter=1000; warmup=500; thin=1; post-warmup draws per chain=500, total post-warmup draws=2000. mean se_mean sd 2.5% 25% 50% 75% 97.5% n_eff beta[1] beta[2] lp Rhat beta[1] 1.01 beta[2] 1.00 lp 1.00 Samples were drawn using NUTS(diag_e) at Tue Jul 5 02:10: For each parameter, n_eff is a crude measure of effective sample size, and Rhat is the potential scale reduction factor on split chains (at convergence, Rhat=1). The estimated Bayesian Fraction of Missing Information is a measure of the efficiency of the sampler with values close to 1 being ideal. For each chain, these estimates are summary(nes_logit.sf) $summary mean se_mean sd 2.5% 25% beta[1] beta[2] lp

4 50% 75% 97.5% n_eff Rhat beta[1] beta[2] lp $c_summary,, chains = chain:1 stats parameter mean sd 2.5% 25% 50% beta[1] beta[2] lp stats parameter 75% 97.5% beta[1] beta[2] lp ,, chains = chain:2 stats parameter mean sd 2.5% 25% 50% beta[1] beta[2] lp stats parameter 75% 97.5% beta[1] beta[2] lp ,, chains = chain:3 stats parameter mean sd 2.5% 25% 50% beta[1] beta[2] lp stats parameter 75% 97.5% beta[1] beta[2] lp ,, chains = chain:4 stats parameter mean sd 2.5% 25% 50% beta[1] beta[2] lp stats parameter 75% 97.5% 4

5 beta[1] beta[2] lp fitted model P (y i = 1) = logit 1 ( income) figures # Figures beta.post <- extract(nes_logit.sf, "beta")$beta beta.mean <- colmeans(beta.post) figure 5.1a # Figure 5.1 (a) len <- 20 x <- seq(1, 5, length.out = len) y <- 1 / (1 + exp(- beta.mean[1] - beta.mean[2] * x)) nes_vote.ggdf.1 <- data.frame(x, y) p1 <- ggplot(data.frame(income, vote), aes(x = income, y = vote)) + geom_jitter(position = position_jitter(height = 0.04, width = 0.4), shape = 20, color = "darkred") + geom_line(aes(x, y), data = nes_vote.ggdf.1, size = 2) + stat_function(fun = function(x) 1 / (1 + exp(- beta.mean[1] - beta.mean[2] * x))) + scale_x_continuous("income", limits = c(-2, 8), breaks = seq(1, 5), labels = c("1\n(poor)", "2", "3", "4", "5\n(rich)")) + scale_y_continuous("pr(republican Vote)", limits = c(-0.05, 1.05), breaks = seq(0, 1, 0.2)) print(p1) 5

6 Pr(Republican Vote) (poor) (rich) Income figure 5.1b # Figure 5.1 (b) # dev.new() n <- 20 ndx <- sample(nrow(beta.post), n) min.x <- 0.5 max.x <- 5.5 x <- seq(min.x, max.x, length.out = len) nes_vote.ggdf.2 <- data.frame(c(), c(), c()) # empty data frame for (i in ndx) { y <- 1 / (1 + exp(- beta.post[i, 1] - beta.post[i, 2] * x)) nes_vote.ggdf.2 <- rbind(nes_vote.ggdf.2, data.frame(id = rep(i, len), x, y)) p2 <- ggplot(data.frame(income, vote), aes(x = income, y = vote)) + geom_jitter(position = position_jitter(height =.04, width =.4), shape = 20, color = "darkred") + geom_line(aes(x, y, group = id), data = nes_vote.ggdf.2, alpha = 0.1) + geom_line(aes(x, y = 1 / (1 + exp(- beta.mean[1] - beta.mean[2] * x))), data = nes_vote.ggdf.2) + scale_x_continuous("income", limits = c(min.x, max.x), breaks = seq(1, 5), labels = c("1\n(poor)", "2", "3", "4", "5\n(rich)")) + scale_y_continuous("pr(republican Vote)", limits = c(-0.05, 1.05), 6

7 print(p2) breaks = seq(0, 1, 0.2)) Pr(Republican Vote) (poor) (rich) Income fit with glm Gelman and Hill, p.79 library(arm) glm.fit1 <- glm(vote ~ income, family = binomial(link = "logit")) display(glm.fit1) glm(formula = vote ~ income, family = binomial(link = "logit")) coef.est coef.se (Intercept) income n = 1179, k = 2 residual deviance = , null deviance = (difference = 34.3) options(show.signif.stars = FALSE) summary(glm.fit1) Call: glm(formula = vote ~ income, family = binomial(link = "logit")) Deviance Residuals: 7

8 Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-13 income e-09 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 1178 degrees of freedom Residual deviance: on 1177 degrees of freedom AIC: Number of Fisher Scoring iterations: 4 fit with stan_glm library(rstanarm) data <- data.frame(income, vote) stan_glm.fit1 <- stan_glm(vote ~ income, data = data, family = binomial(link = "logit"), chains = 4) plot(stan_glm.fit1) ci_level: 0.8 (80% intervals) outer_level: 0.95 (95% intervals) (Intercept) income pairs(stan_glm.fit1)

9 (Intercept) income mean_ppd summary(stan_glm.fit1) log posterior stan_glm(formula = vote ~ income, family = binomial(link = "logit"), data = data, chains = 4) Family: binomial (logit) Algorithm: sampling Posterior sample size: 4000 Observations: 1179 Estimates: mean sd 2.5% 25% 50% 75% 97.5% (Intercept) income mean_ppd log-posterior Diagnostics: mcse Rhat n_eff (Intercept) income mean_ppd log-posterior For each parameter, mcse is Monte Carlo standard error, n_eff is a crude measure of effective sample 9

Σχετικά έγγραφα

5.6 evaluating, checking, comparing Chris Parrish July 3, 2016

5.6 evaluating, checking, comparing Chris Parrish July 3, 2016 Contents residuals 1 evaluating, checking, comparing 1 data..................................................... 1 model....................................................