*Stata code for Williams＆Bornmann book chapter on effect sizes.

*Be careful when running this code-make sure it doesn't

*overwrite existing files or graphs that use the same names.

version 13.1

use"http：//www3.nd.edu/～rwilliam/statafiles/rwlbes"，clear

gen inst12＝inst if inst!＝3

gen inst13＝inst if inst!＝2

gen inst23＝inst if inst!＝1

gen top10＝perc＜＝10

*Limit to 2001＆2002；this can be changed

keep if py＜＝2002

}

*Table 12.2

*Single group designs-pages 286-287 of Cumming

*For each institution，test whether percentile mu＝50

*Note that negative differences mean better than average performance forval instnum＝1/3{

Display

display"Institution‘instnum’"

ttest perc 1/4 50 if inst＝＝‘instnum’

display

display"Cohen's d＝"r(t)/sqrt(r(N_1))

*DOUBLE CHECK：Compares above CIs and t-tests with bootstrap

*Results from the test command should be similar to the t-test

*significance level

bootstrap，reps(100)：reg perc if inst1/41/4‘instnum’

test_cons＝50

}}

*Table12.3

*Two group designs-Test whether two institutions

*differ from each other on mean percentile rating.

*Starts around p.155

*Get both the t-tests and the ES stats，e.g.Cohen's d

*Note：you should flip the signs for the 3 vs 2 comparison

}

foreach iv of varlist inst12 inst13 inst23{

display"perc is dependent，‘iv’"

}

ttest perc，by(‘iv’)

scalar n1＝r(N_1)

scalar n2＝r(N_2)

scalar s1＝r(sd_1)

scalar s2＝r(sd_2)

display

display"Pooled sd is"///

sqrt(((n1-1)*s1＾2＋(n2-1)*s2＾2)/(n1＋n2-2))

display

esize two perc，by(‘iv’)all

display

*DOUBLE CHECKS：Compare Mann-Whitney＆bootstrap results with above

*Mann-Whitney test

ranksum perc，by(‘iv’)

*Bootstrap

bootstrap，rep(100)：reg perc i.‘iv’

}}}

*Table 12.4

*Proportions in Top 10，pp.399-402

*Single institution tests

*Numbers in table are multiplied by 100

forval instnum-1/3{

display(www.daowen.com)

display"Institution‘instnum’"

prtest top10 1/4.10 if inst＝＝‘instnum’

display

display

scalar phi1＝2*asin(sqrt(r(P_1)))

scalar phi2＝2*asin(sqrt(.10))

di"h effect size＝"phi1-phi2

display

}

}}

*Table 12.5

*Proportions in Top 10-pairwise comparisons of institutions

*Numbers in table are multiplied by 100

foreach instpair of varlist inst12 inst13 inst23{

display

display"‘instpair’"

prtest top10，by(‘instpair’)

display

scalar phi1 1/4 2*asin(sqrt(r(P_1)))

scalar phi2 1/4 2*asin(sqrt(r(P_2)))

di"h effect size 1/4"phi1-phi2

display

*NOTE：Cohen's d provides very similar results to Cohen's h

esize two top10，by(‘instpair’)all

display

}

*Do graphs with Stata

*NOTE：Additional editing was done with the Stata Graph Editor

*Use ciplot for Univariate graphs

}

*Figure 12.1-Average percentile score by inst with CI

ciplot perc，by(inst)name(fig1，replace)

}

*Figure 12.3

*Was edited to multiply by 100

ciplot top10，bin by(inst)name(fig3，replace)

}

***Save figures before running figure 12.2 code

}

*Figure 12.2-Differences in mean percentile rankings

*Use statsby and serrbar for tests of group differences

*Note：Data in memory is overwritten

gen inst32＝inst23*-1＋4

tab2 inst32 inst23

statsby_b_se，saving(xb12，replace)：reg perc i.inst12

statsby_b_se，saving(xb13，replace)：reg perc i.inst13

statsby_b_se，saving(xb32，replace)：reg perc i.inst32

clear all

append using xb12 xb13 xb32，gen(pairing)

label define pairing 1"1 vs 2"2"1 vs 3"3"3 vs 2"

label values pairing pairing

serrbar_stat_2_stat_5 pairing，scale(1.96)name(fig2，replace)

【注释】

[1]R.Williams，Department of Sociology，University of Notre Dame，810 Flanner Hall，Notre Dame，IN 46556，USA，E-mail：Richard.A.Williams.5＠ND.Edu；L.Bornmann，Division for Science and Innovation Studies，Administrative Headquarters of the Max Planck Society，Hofgartenstr.8，80539，Munich，Germany E-mail：bornmann＠gv.mpg.de.

[2]Cumming将从分析中获得的CI称为“One from the Dance”。他的意思是“均值的真实值落在可信区间的概率为95%”这种说法是错误的。真值可能落在可信区间内，也可能落在可信区间外。我们可以这样认为：如果这一过程重复无限次，那么可信区间有95%的机会包含真值，5%的机会不包含均值。对于特定的结果是否如此，我们不得而知。

[3]对于使用Cohen's d时应注意的事项，Cumming给出了许多注释。例如，虽然正如我们这里一样，使用样本标准差是很常见的现象，其他的一些“Standardizer”也可以使用，比如你可能使用参照人群的标准差，如杰出机构。研究人员应该对如何计算Cohen's d有清楚的认识。

[4]对于独立样本，有两种计算t检验的方式。第一种，正如这里使用的，假设每组方差是相同的。第二种，允许两组间方差不齐。在我们的例子中，无论使用哪种方式，结果差异不大，这是因为，如表12.2所示，三个组之间的标准差很相似。如果方差明显不齐，那么应该使用第二种方式。大多数统计软件包都可以很容易地实现这两种t检验。

[5]然而，正如我们在其他测量方式中发现的，Cohen's d在不满足假设的情况下仍表现得很好。当我们使用二分因变量估计Cohen's d时，我们几乎可以得到与Cohen's h一样的结果。