McDaniel1994
Michael Brannick
12/31/2017
McDaniel’s Employment Interview Data
library(metafor) # wake up metafor so it will run## Loading required package: Matrix## Loading 'metafor' package (version 2.0-0). For an overview
## and introduction to the package please type: help(metafor).dat.mcdaniel1994 # print the McDaniel 1994 dataset## study ni ri type struct
## 1 1 123 0.00 j s
## 2 2 95 0.06 p u
## 3 3 69 0.36 j s
## 4 4 1832 0.15 j s
## 5 5 78 0.14 j s
## 6 6 329 0.06 j s
## 7 7 153 0.09 j s
## 8 8 29 0.40 j s
## 9 9 29 0.39 s s
## 10 10 157 0.14 s s
## 11 11 149 0.36 s s
## 12 12 92 0.28 j u
## 13 13 15 0.62 j s
## 14 14 15 0.07 j u
## 15 15 170 0.18 j u
## 16 16 19 0.42 j s
## 17 17 19 0.08 j u
## 18 18 68 0.18 p u
## 19 19 93 0.43 j u
## 20 20 57 0.04 j u
## 21 21 80 -0.04 p <NA>
## 22 22 53 0.05 p <NA>
## 23 23 24 -0.14 p <NA>
## 24 24 57 0.05 j s
## 25 25 275 0.35 j s
## 26 26 45 -0.08 p <NA>
## 27 27 79 0.24 p <NA>
## 28 28 107 0.16 p <NA>
## 29 29 31 0.25 j u
## 30 30 407 0.68 j s
## 31 31 84 0.61 j s
## 32 32 8 0.81 j s
## 33 33 6 0.99 j s
## 34 34 7 0.66 j s
## 35 35 12 0.45 j s
## 36 36 14 0.71 j s
## 37 37 40 0.27 j s
## 38 38 40 -0.02 j s
## 39 39 99 0.29 j u
## 40 40 164 0.13 j u
## 41 41 67 0.03 j u
## 42 42 57 0.00 j u
## 43 43 50 0.09 j s
## 44 44 129 -0.03 j u
## 45 45 49 0.46 s s
## 46 46 63 0.30 s s
## 47 47 56 0.33 s s
## 48 48 238 0.24 p <NA>
## 49 49 20 0.64 j s
## 50 50 122 0.12 j u
## 51 51 51 0.15 j u
## 52 52 40 0.44 j u
## 53 53 210 0.00 j s
## 54 54 334 0.16 j s
## 55 55 310 0.21 p <NA>
## 56 56 180 0.29 j s
## 57 57 93 0.19 j u
## 58 58 472 0.04 j u
## 59 59 44 0.56 j u
## 60 60 75 0.14 j u
## 61 61 68 0.44 j u
## 62 62 38 0.36 j u
## 63 63 42 0.34 j <NA>
## 64 64 39 0.11 j <NA>
## 65 65 49 0.40 j <NA>
## 66 66 41 0.23 j <NA>
## 67 67 200 0.22 j s
## 68 68 850 0.44 j s
## 69 69 41 0.27 j s
## 70 70 32 0.11 j s
## 71 71 65 0.27 j s
## 72 72 125 -0.07 j s
## 73 73 134 0.32 j s
## 74 74 21 0.05 j u
## 75 75 44 0.20 j u
## 76 76 170 0.18 j u
## 77 77 149 0.34 j s
## 78 78 296 0.03 j s
## 79 79 24 0.45 s s
## 80 80 312 0.34 j s
## 81 81 205 0.51 j s
## 82 82 30 0.41 s s
## 83 83 11 0.37 s s
## 84 84 22 0.25 s s
## 85 85 37 -0.17 j s
## 86 86 43 0.47 j s
## 87 87 72 0.32 j s
## 88 88 72 -0.09 s s
## 89 89 108 0.33 j s
## 90 90 73 0.22 j s
## 91 91 73 0.27 s s
## 92 92 117 0.00 j s
## 93 93 80 0.41 j s
## 94 94 95 0.16 j s
## 95 95 182 0.00 j s
## 96 96 93 0.03 j s
## 97 97 64 0.01 j s
## 98 98 370 0.03 j s
## 99 99 131 0.14 j s
## 100 100 87 0.11 j s
## 101 101 80 0.08 j s
## 102 102 41 -0.13 j s
## 103 103 35 0.13 j u
## 104 104 106 0.36 j s
## 105 105 86 0.06 j s
## 106 106 54 0.19 j s
## 107 107 393 0.27 j s
## 108 108 102 0.17 j s
## 109 109 115 0.34 j s
## 110 110 63 0.28 s s
## 111 111 22 0.11 j s
## 112 112 37 0.07 j u
## 113 113 116 -0.13 <NA> <NA>
## 114 114 416 0.12 j u
## 115 115 101 0.12 j u
## 116 116 1359 0.37 j u
## 117 117 82 0.26 p u
## 118 118 32 0.42 j s
## 119 119 42 0.37 j s
## 120 120 196 0.17 j s
## 121 121 44 0.19 j s
## 122 122 47 0.32 s s
## 123 123 37 0.33 <NA> <NA>
## 124 124 12 0.24 j s
## 125 125 1807 0.09 <NA> <NA>
## 126 126 73 0.36 j s
## 127 127 73 0.26 s s
## 128 128 70 0.42 j s
## 129 129 30 0.62 j s
## 130 130 60 0.87 j s
## 131 131 38 -0.07 j s
## 132 132 12 0.65 j s
## 133 133 33 0.17 j u
## 134 134 33 0.30 j u
## 135 135 28 0.45 s s
## 136 136 51 0.24 p u
## 137 137 49 0.02 p u
## 138 138 164 0.23 j s
## 139 139 195 0.17 j s
## 140 140 165 0.32 j s
## 141 141 40 0.36 j s
## 142 142 100 0.09 p s
## 143 143 4195 0.13 j u
## 144 144 179 0.29 j s
## 145 145 74 0.49 j s
## 146 146 110 0.40 j s
## 147 147 31 0.23 j s
## 148 148 70 0.31 j s
## 149 149 21 0.46 j s
## 150 150 29 -0.12 j s
## 151 151 51 0.22 j u
## 152 152 51 0.59 j s
## 153 153 40 0.21 j s
## 154 154 40 0.02 j s
## 155 155 129 -0.03 j s
## 156 156 196 0.28 j s
## 157 157 31 -0.04 j s
## 158 158 494 0.19 j u
## 159 159 101 0.23 j s
## 160 160 175 0.30 j sresM <- rma(ri=ri, ni=ni, measure='ZCOR', data=dat.mcdaniel1994) #run metafor
resM # print results of running metafor##
## Random-Effects Model (k = 160; tau^2 estimator: REML)
##
## tau^2 (estimated amount of total heterogeneity): 0.0293 (SE = 0.0049)
## tau (square root of estimated tau^2 value): 0.1712
## I^2 (total heterogeneity / total variability): 81.29%
## H^2 (total variability / sampling variability): 5.35
##
## Test for Heterogeneity:
## Q(df = 159) = 789.7321, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2374 0.0170 13.9995 <.0001 0.2042 0.2706 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1As you can see, metafor computes a meta-analysis on the 160 studies in the dataset. Because I specificied ‘ZCOR,’ the analysis is done in the Fisher z transformation rather than the original Pearson r metric. The results are in z as well, so you have to translate back.
forest(resM) # create a forest plot
funnel(resM) # create a funnel plot
There are too many studies for the forest plot to work well. More than about 40 studies presents a problem for the traditional forest plot. But there are ways around this. The funnel plot works well with lots of studies.
Now to run a moderator analysis. First, we need to sort the data by the moderator, then run the analysis.
mcdanielStruct <- dat.mcdaniel1994[order(dat.mcdaniel1994$struct, dat.mcdaniel1994$ri), ]
mcdanielStruct## study ni ri type struct
## 85 85 37 -0.17 j s
## 102 102 41 -0.13 j s
## 150 150 29 -0.12 j s
## 88 88 72 -0.09 s s
## 72 72 125 -0.07 j s
## 131 131 38 -0.07 j s
## 157 157 31 -0.04 j s
## 155 155 129 -0.03 j s
## 38 38 40 -0.02 j s
## 1 1 123 0.00 j s
## 53 53 210 0.00 j s
## 92 92 117 0.00 j s
## 95 95 182 0.00 j s
## 97 97 64 0.01 j s
## 154 154 40 0.02 j s
## 78 78 296 0.03 j s
## 96 96 93 0.03 j s
## 98 98 370 0.03 j s
## 24 24 57 0.05 j s
## 6 6 329 0.06 j s
## 105 105 86 0.06 j s
## 101 101 80 0.08 j s
## 7 7 153 0.09 j s
## 43 43 50 0.09 j s
## 142 142 100 0.09 p s
## 70 70 32 0.11 j s
## 100 100 87 0.11 j s
## 111 111 22 0.11 j s
## 5 5 78 0.14 j s
## 10 10 157 0.14 s s
## 99 99 131 0.14 j s
## 4 4 1832 0.15 j s
## 54 54 334 0.16 j s
## 94 94 95 0.16 j s
## 108 108 102 0.17 j s
## 120 120 196 0.17 j s
## 139 139 195 0.17 j s
## 106 106 54 0.19 j s
## 121 121 44 0.19 j s
## 153 153 40 0.21 j s
## 67 67 200 0.22 j s
## 90 90 73 0.22 j s
## 138 138 164 0.23 j s
## 147 147 31 0.23 j s
## 159 159 101 0.23 j s
## 124 124 12 0.24 j s
## 84 84 22 0.25 s s
## 127 127 73 0.26 s s
## 37 37 40 0.27 j s
## 69 69 41 0.27 j s
## 71 71 65 0.27 j s
## 91 91 73 0.27 s s
## 107 107 393 0.27 j s
## 110 110 63 0.28 s s
## 156 156 196 0.28 j s
## 56 56 180 0.29 j s
## 144 144 179 0.29 j s
## 46 46 63 0.30 s s
## 160 160 175 0.30 j s
## 148 148 70 0.31 j s
## 73 73 134 0.32 j s
## 87 87 72 0.32 j s
## 122 122 47 0.32 s s
## 140 140 165 0.32 j s
## 47 47 56 0.33 s s
## 89 89 108 0.33 j s
## 77 77 149 0.34 j s
## 80 80 312 0.34 j s
## 109 109 115 0.34 j s
## 25 25 275 0.35 j s
## 3 3 69 0.36 j s
## 11 11 149 0.36 s s
## 104 104 106 0.36 j s
## 126 126 73 0.36 j s
## 141 141 40 0.36 j s
## 83 83 11 0.37 s s
## 119 119 42 0.37 j s
## 9 9 29 0.39 s s
## 8 8 29 0.40 j s
## 146 146 110 0.40 j s
## 82 82 30 0.41 s s
## 93 93 80 0.41 j s
## 16 16 19 0.42 j s
## 118 118 32 0.42 j s
## 128 128 70 0.42 j s
## 68 68 850 0.44 j s
## 35 35 12 0.45 j s
## 79 79 24 0.45 s s
## 135 135 28 0.45 s s
## 45 45 49 0.46 s s
## 149 149 21 0.46 j s
## 86 86 43 0.47 j s
## 145 145 74 0.49 j s
## 81 81 205 0.51 j s
## 152 152 51 0.59 j s
## 31 31 84 0.61 j s
## 13 13 15 0.62 j s
## 129 129 30 0.62 j s
## 49 49 20 0.64 j s
## 132 132 12 0.65 j s
## 34 34 7 0.66 j s
## 30 30 407 0.68 j s
## 36 36 14 0.71 j s
## 32 32 8 0.81 j s
## 130 130 60 0.87 j s
## 33 33 6 0.99 j s
## 44 44 129 -0.03 j u
## 42 42 57 0.00 j u
## 137 137 49 0.02 p u
## 41 41 67 0.03 j u
## 20 20 57 0.04 j u
## 58 58 472 0.04 j u
## 74 74 21 0.05 j u
## 2 2 95 0.06 p u
## 14 14 15 0.07 j u
## 112 112 37 0.07 j u
## 17 17 19 0.08 j u
## 50 50 122 0.12 j u
## 114 114 416 0.12 j u
## 115 115 101 0.12 j u
## 40 40 164 0.13 j u
## 103 103 35 0.13 j u
## 143 143 4195 0.13 j u
## 60 60 75 0.14 j u
## 51 51 51 0.15 j u
## 133 133 33 0.17 j u
## 15 15 170 0.18 j u
## 18 18 68 0.18 p u
## 76 76 170 0.18 j u
## 57 57 93 0.19 j u
## 158 158 494 0.19 j u
## 75 75 44 0.20 j u
## 151 151 51 0.22 j u
## 136 136 51 0.24 p u
## 29 29 31 0.25 j u
## 117 117 82 0.26 p u
## 12 12 92 0.28 j u
## 39 39 99 0.29 j u
## 134 134 33 0.30 j u
## 62 62 38 0.36 j u
## 116 116 1359 0.37 j u
## 19 19 93 0.43 j u
## 52 52 40 0.44 j u
## 61 61 68 0.44 j u
## 59 59 44 0.56 j u
## 23 23 24 -0.14 p <NA>
## 113 113 116 -0.13 <NA> <NA>
## 26 26 45 -0.08 p <NA>
## 21 21 80 -0.04 p <NA>
## 22 22 53 0.05 p <NA>
## 125 125 1807 0.09 <NA> <NA>
## 64 64 39 0.11 j <NA>
## 28 28 107 0.16 p <NA>
## 55 55 310 0.21 p <NA>
## 66 66 41 0.23 j <NA>
## 27 27 79 0.24 p <NA>
## 48 48 238 0.24 p <NA>
## 123 123 37 0.33 <NA> <NA>
## 63 63 42 0.34 j <NA>
## 65 65 49 0.40 j <NA>Note how the data are sorted (s, u, then NA for missing). Now the analysis.
resStruct <- rma(ri=ri, ni=ni, measure='ZCOR', method='DL', data=mcdanielStruct, mods = ~mcdanielStruct$struct)## Warning in rma(ri = ri, ni = ni, measure = "ZCOR", method = "DL", data =
## mcdanielStruct, : Studies with NAs omitted from model fitting.resStruct # print the results of the moderator analysis##
## Mixed-Effects Model (k = 145; tau^2 estimator: DL)
##
## tau^2 (estimated amount of residual heterogeneity): 0.0293 (SE = 0.0084)
## tau (square root of estimated tau^2 value): 0.1711
## I^2 (residual heterogeneity / unaccounted variability): 79.61%
## H^2 (unaccounted variability / sampling variability): 4.90
## R^2 (amount of heterogeneity accounted for): 0.00%
##
## Test for Residual Heterogeneity:
## QE(df = 143) = 701.2270, p-val < .0001
##
## Test of Moderators (coefficient(s) 2):
## QM(df = 1) = 3.8786, p-val = 0.0489
##
## Model Results:
##
## estimate se zval pval ci.lb
## intrcpt 0.2696 0.0210 12.8574 <.0001 0.2285
## mcdanielStruct$structu -0.0785 0.0398 -1.9694 0.0489 -0.1566
## ci.ub
## intrcpt 0.3106 ***
## mcdanielStruct$structu -0.0004 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Note that the result is (barely) significant. Note the ‘u’ at the end of the term below the intercept. This is the difference between structured and unstructured interviews (unstructured interview scores are slightly less correlated with job performance).