personal/nunosempere.github.io
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update 3-Power-calculations.md

Browse Source
This commit is contained in:
Nuño Sempere 2018-10-14 19:15:02 +02:00 committed by GitHub
parent 52f7cb24a6
commit 64eccfdc03
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 180 additions and 176 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

356
ESPR-Evaluation/3-Power-calculations.md
View File

@ -1,176 +1,180 @@
# Power calculations # Power calculations
Using R we will do some power calculations Using R we will do some power calculations
Necessary library pwr, loads with library(pwr) Necessary library pwr, loads with library(pwr)
Necessary function: pwr.t2n.test Necessary function: pwr.t2n.test
See: https://www.statmethods.net/stats/power.html See: https://www.statmethods.net/stats/power.html
## Year 1, pessimistic projections Optimistic: We reach everyone
With n-treatment=20, n-control = 20, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect = ? Pessimistic: We reach 66% of treatment and control group.
t test power calculation ## Year 1, pessimistic projections
ith n-treatment=20, n-control = 20, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect in standard deviations (d) = ?
n1 = 20
n2 = 20 t test power calculation
d = 1.051997
sig.level = 0.05 n1 = 20
power = 0.9 n2 = 20
alternative = two.sided d = 1.051997
sig.level = 0.05
## Year 1, optimistic projections power = 0.9
With n_treatment=30, n_control = 60, power = 0.9,sig.level= 0.05, minimal detectable effect = ? alternative = two.sided
t test power calculation ## Year 1, optimistic projections
With n_treatment=30, n_control = 60, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 30
n2 = 60 t test power calculation
d = 0.7328756
sig.level = 0.05 n1 = 30
power = 0.9 n2 = 60
alternative = two.sided d = 0.7328756
sig.level = 0.05
Withn = ?, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect = 0.5 power = 0.9
alternative = two.sided
Two-sample t test power calculation
With n = ?, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect = 0.5
n = 85.03128
d = 0.5 Two-sample t test power calculation
sig.level = 0.05
power = 0.9 n = 85.03128
alternative = two.sided d = 0.5
sig.level = 0.05
NOTE: n is number in *each* group power = 0.9
alternative = two.sided
## Year 2, pessimistic projections NOTE: n is number in *each* group
With n_treatment=40, n_control = 40, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
t test power calculation ## Year 2, pessimistic projections
With n_treatment=40, n_control = 40, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 40
n2 = 40 t test power calculation
d = 0.7339255
sig.level = 0.05 n1 = 40
power = 0.9 n2 = 40
alternative = two.sided d = 0.7339255
sig.level = 0.05
## Year 2, optimistic projections power = 0.9
With n_treatment=60, n_control = 120, power = 0.9,sig.level= 0.05, minimal detectable effect = ? alternative = two.sided
t test power calculation ## Year 2, optimistic projections
With n_treatment=60, n_control = 120, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 60
n2 = 120 t test power calculation
d = 0.5153056
sig.level = 0.05 n1 = 60
power = 0.9 n2 = 120
alternative = two.sided d = 0.5153056
sig.level = 0.05
power = 0.9
## Year 3, pessimistic projections alternative = two.sided
With n_treatment=60, n_control = 60, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
t test power calculation ## Year 3, pessimistic projections
With n_treatment=60, n_control = 60, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 60
n2 = 60 t test power calculation
d = 0.5967207
sig.level = 0.05 n1 = 60
power = 0.9 n2 = 60
alternative = two.sided d = 0.5967207
sig.level = 0.05
## Year 3, optimistic projections power = 0.9
With n_treatment=90, n_control = 180, power = 0.9,sig.level= 0.05, minimal detectable effect = ? alternative = two.sided
t test power calculation ## Year 3, optimistic projections
With n_treatment=90, n_control = 180, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 90
n2 = 180 t test power calculation
d = 0.4200132
sig.level = 0.05 n1 = 90
power = 0.9 n2 = 180
alternative = two.sided d = 0.4200132
sig.level = 0.05
## Year 4, pessimistic projections power = 0.9
With n_treatment=80, n_control = 80, power = 0.9,sig.level= 0.05, minimal detectable effect = ? alternative = two.sided
t test power calculation ## Year 4, pessimistic projections
With n_treatment=80, n_control = 80, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 80
n2 = 80 t test power calculation
d = 0.5156619
sig.level = 0.05 n1 = 80
power = 0.9 n2 = 80
alternative = two.sided d = 0.5156619
sig.level = 0.05
## Year 4, optimistic projections power = 0.9
With n_treatment=120, n_control = 240, power = 0.9,sig.level= 0.05, minimal detectable effect = ? alternative = two.sided
t test power calculation ## Year 4, optimistic projections
With n_treatment=120, n_control = 240, power = 0.9,sig.level= 0.05, minimal detectable effect = ?
n1 = 120
n2 = 240 t test power calculation
d = 0.3633959
sig.level = 0.05 n1 = 120
power = 0.9 n2 = 240
alternative = two.sided d = 0.3633959
sig.level = 0.05
## Population necessary to detect an effect size of 0.2 with significance level = 0.05 and power = 0.9 power = 0.9
alternative = two.sided
here the free variable was d= minimal detectable effect
Withn = ?, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect = 0.2 ## Population necessary to detect an effect size of 0.2 with significance level = 0.05 and power = 0.9
Two-sample t test power calculation Here the free variable was d= minimal detectable effect
With n = ?, power = 0.9,sig.level= 0.05, power = 0.9, minimal detectable effect = 0.2
n = 526.3332
d = 0.2 Two-sample t test power calculation
sig.level = 0.05
power = 0.9 n = 526.3332
alternative = two.sided d = 0.2
sig.level = 0.05
NOTE: n is number in *each* group power = 0.9
alternative = two.sided
here the free variable was n, the population of the treatment group
son = population of the treatmente group = population of the control group NOTE: n is number in *each* group
necessary to detect an effect of 0.2
here the free variable was n, the population of the treatment group
## Population necessary to detect an effect size of 0.5 with significance level = 0.05 and power = 0.9 son = population of the treatmente group = population of the control group
necessary to detect an effect of 0.2
Two-sample t test power calculation
## Population necessary to detect an effect size of 0.5 with significance level = 0.05 and power = 0.9
n = 85.03128
d = 0.5 Two-sample t test power calculation
sig.level = 0.05
power = 0.9 n = 85.03128
alternative = two.sided d = 0.5
sig.level = 0.05
NOTE: n is number in *each* group power = 0.9
alternative = two.sided
## Population necessary to detect an effect size of 0.2 with significance level = 0.10 and power = 0.9
NOTE: n is number in *each* group
Two-sample t test power calculation
## Population necessary to detect an effect size of 0.2 with significance level = 0.10 and power = 0.9
n = 428.8664
d = 0.2 Two-sample t test power calculation
sig.level = 0.1
power = 0.9 n = 428.8664
alternative = two.sided d = 0.2
sig.level = 0.1
NOTE: n is number in *each* group power = 0.9
alternative = two.sided
## Population necessary to detect an effect size of 0.5 with significance level = 0.10 and power = 0.9 NOTE: n is number in *each* group
Two-sample t test power calculation
## Population necessary to detect an effect size of 0.5 with significance level = 0.10 and power = 0.9
n = 69.19719
d = 0.5 Two-sample t test power calculation
sig.level = 0.1
power = 0.9 n = 69.19719
alternative = two.sided d = 0.5
sig.level = 0.1
NOTE: n is number in *each* group power = 0.9
alternative = two.sided
## Conclusions. NOTE: n is number in *each* group
Even after 4 years, under the most optimistic population projections (i.e., every participant answers our surveys every year, and 60 students who didn't get selected also do), we wouldn't have enough power to detect an effect size of 0.2 standard deviations with significance level = 0.05. However, it seems feasible to detect the kinds of effects which would justify the upward of $150.000 / year costs of ESPR within 3 years. The minimum effect which justifies the costs of ESPR should be determined beforehand, as should the axis along which we measure. I would also suggest to expand the RCT to SPARC once its feasibility has been tested at ESPR.
## Conclusions.
Even after 4 years, under the most optimistic population projections (i.e., every participant answers our surveys every year, and 60 students who didn't get selected also do), we wouldn't have enough power to detect an effect size of 0.2 standard deviations with significance level = 0.05. However, it seems feasible to detect the kinds of effects which would justify the upward of $150.000 / year costs of ESPR within 3 years. The minimum effect which justifies the costs of ESPR should be determined beforehand, as should the axis along which we measure. I would also suggest to expand the RCT to SPARC once its feasibility has been tested at ESPR.