Sample Size Calculator for a SMART Design

Description

This web application calculates the sample size necessary to identify the best strategy when using data from a SMART experimental design with two first stage treatments followed by either one or two second stage treatments (see Details). The best adaptive treatment strategy is the strategy that would result in the highest mean for the outcome Y. (Formulae for calculating the strategy means can be found in the first reference listed below.)

Instructions

To use the sample size calculator, enter the following quantities in the appropriate boxes to the right:

δ : the standardized effect size one wants to detect, (0 < δ < 1) (See note on effect size below)
π : the desired probability of correctly choosing the best strategy, (0 < π < 1)
N_max : the maximum sample size that the user can afford/has available

Output

N : the sample size necessary to identify the best strategy.

Sample size calculation

Note: We use Cohen's definition for standardized effect size (Cohen, J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.; 1988.) We define the effect size using the difference in the mean of the strategy with the highest mean outcome and the mean of the strategy with the next highest mean outcome.

Example

Suppose we are interested in detecting an effect size of δ = 0.2 with probability π = 0.9. The maximum sample size we can allow is N_max = 1000.

The sample size N returned will be around 605. The number will change slightly each time because it is based on simulations.

Possible problems in calculating sample size:

If N_max in the above example is 300, then the algorithm will not find an answer for N. This is because the sample size required to detect an effect of δ = 0.2 with probability π = 0.9 is larger than 300. In this case, the user has three options: increase Nmax, increase δ, or decrease π.

Details

This web application calculates the required sample size for sizing a study designed to discover the best adaptive treatment strategy. We assume that the data comes from a SMART experimental design of the following type:

there are two initial treatments at the first stage;
there are two treatments for non-responders (or responders) to the first stage;
there is one treatment only for responders (or non-responders) to the first stage; and
the final outcome, Y, is continuous.

We also assume the patients are randomized equally between the two treatments at each stage. Furthermore, the two treatments for non-responders to the first stage are the same regardless of which treatment they received in the first stage. This means that we are comparing four different treatment strategies, which can be denoted {(1, 1), (1, 0), (0, 1), (0, 0)}.

The sample size calculator makes the following working assumptions:

The sample sizes will be large enough so that the estimators for the strategy means are approximately normally distributed.
The correlation between the estimated mean outcome resulting for strategy (1, 1) and the estimated mean outcome resulting for strategy (1, 0) is the same as the correlation between the estimated mean outcome resulting for strategy (0, 1) and the estimated mean outcome for strategy (0, 0).
For the purpose of this sample size calculation, we assume that three of the strategies have the same mean and the one remaining strategy produces the largest mean; this is an extreme scenario in which it is the most difficult to detect the presence of an effect.

Please see the references below for more details.

If you are interested in the Matlab code that performs this calculation, please click here.

References

Scott AI, Levy JA, Murphy SA. Statistical Methodology for a SMART Design in the Development of Adaptive Treatment Strategies. (Tech. Rep. No. 07-82). University Park, PA: The Pennsylvania State University, The Methodology Center. 2007. Click here to view paper.

Scott AI, Levy JA, Murphy SA. Evaluation of Sample Size Formulae for Developing Adaptive Treatment Strategies Using a SMART Design. (Tech. Rep. No. 07-81). University Park, PA: The Pennsylvania State University, The Methodology Center. 2007. Click here to view paper.