Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions – a calibration approach

Introduction

This vignette reproduces results and figures in

Kopp-Schneider, A., Wiesenfarth, M., Witt, R., Edelmann, D., Witt, D. and Abel, U. (2018). Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions - a calibration approach. Biometrical Journal, to appear.

Note that most figures are also shown in the interactive shiny app using BDP2workflow()

Parameters

First we set the parameters as in the paper. Notation is identical as in the paper: p0, p1, pF, pE, cF, cE as in the paper, shape1F, shape2F are the parameters for the prior for futility, and shape1E, shape2E are the parameters for the prior for efficacy.

library(BDP2)
p0=0.12
p1=0.3
pF=0.3
pE=0.12
shape1F=0.3
shape2F=0.7
shape1E=0.12
shape2E=0.88
cF=0.01
cE=0.9

Figure 1

Figure 1: Posterior distribution of response probability used for futility stop decision: the trial will be stopped for futility if the coloured area is  < cF (left panel). Posterior distribution of response probability used for efficacy calling: the trial will be called efficaceous if the coloured area is  ≥ cE (right panel). The Figure also shows the choice of pF = p1 and of pE = p0 as discussed in the actual design of the trial.

##plot PF
  plot(function(x) x,0,0.8,add=F,type="n",col="black",xlab="",ylab="",xaxt="n",yaxt="n",ylim=c(0,4),
       cex.axis=1.5,cex.lab=1.5,xaxs='i',yaxs='i')
  plot(function(x) dbeta(x,shape1F+2,shape2F+8),add=T,col="black",lwd=2,lty=1)
  xy <- seq(0.3,1,length=1000)
  fxy <- dbeta(xy,shape1F+2,shape2F+8)
  xyx <- c(1,0.3,xy)
  yyx <- c(0.0001,0.0001,fxy)
  polygon(xyx,yyx,col='red')
  lines(c(0.3,0.3),c(0,dbeta(0.3,shape1F+2,shape2F+8)),lwd=3)
  axis(side=1,at=c(0.12,0.3),label=c(expression(p[E]),expression(p[F])),cex.axis=1.5)
  axis(side=1,at=c(0.12,0.3),label=c(expression(paste("=",p[0])),expression(paste("=",p[1]))),cex.axis=1.5,line=1.5,lty=0)
  text(0.65,0.7,labels=expression(paste("P(p > ",p[F],"| Data)")),cex=1.5)
  lines(c(0.4,0.5),c(0.5,0.7),lwd=3)

##plot PE
  plot(function(x) x,0,0.8,add=F,type="n",col="black",xlab="",ylab="",xaxt="n",yaxt="n",ylim=c(0,4),
       cex.axis=1.5,cex.lab=1.5,xaxs='i',yaxs='i')
  plot(function(x) dbeta(x,shape1E+2,shape2E+8),add=T,col="black",lwd=2,lty=1)
  xy <- seq(0.12,1,length=1000)
  fxy <- dbeta(xy,shape1E+2,shape2E+8)
  xyx <- c(1,0.12,xy)
  yyx <- c(0.0001,0.0001,fxy)
  polygon(xyx,yyx,col='green')
  lines(c(0.12,0.12),c(0,dbeta(0.12,shape1E+2,shape2E+8)),lwd=3)
  axis(side=1,at=c(0.12,0.3),label=c(expression(p[E]),expression(p[F])),cex.axis=1.5)
  axis(side=1,at=c(0.12,0.3),label=c(expression(paste("=",p[0])),expression(paste("=",p[1]))),cex.axis=1.5,line=1.5,lty=0)
  text(0.65,0.7,labels=expression(paste("P(p > ",p[E],"| Data)")),cex=1.5)
  lines(c(0.4,0.5),c(0.5,0.7),lwd=3)

Figure 2

Upper tail function for n = 20, for a design with pF = 0.3, cF = 0.01 and prior distribution Be(pF, 1 − pF) for futility (in red), and pE = 0.12, cE = 0.9 and prior distribution Be(pE, 1 − pE) for efficacy (in green). From the graphs, bF(20) = 1 and bE(20) = 5.

n=20

plot(function(x) pbeta(pF,shape1=shape1F+x,shape2=shape2F+n-x,lower.tail = F)-cF,
     0,n,ylim=c(-1,1),lwd=3,
     xlab="x",ylab=expression(paste("P(p>",p[R],"|x,n = 20) - c")),col="red") 

plot(function(x) pbeta(pE,shape1=shape1E+x,shape2=shape2E+n-x,lower.tail = F)-cE,
     0,n,ylim=c(-1,1),lwd=3,col="green",add=TRUE) 
abline(h=0)
legend("bottomright",legend=c(expression(paste("Futility decision: c=",c[F],", ",p[R],"=",p[F])), 
                              expression(paste("Efficacy decision: c=",c[E],", ",p[R],"=",p[E]))),
       lty=1,col = c("red", "green"),cex=1)
abline(v=c(0:20),lty=3,col="grey")

Figure 3

Binomial density with zero responders,(1 − p)n, for varying n, evaluated for p = p0 = 0.12 and for p = p1 = 0.3 (left panel). Posterior probabilities P(p ≥ pF|0, n) and P(p ≥ pF|1, n) for varying n for pF = 0.3 with prior distribution Beta(pF, 1 − pF) (right panel).

nmin=4
nmax=15

plotBDP2(x="n",y="Prob0Successes",n=c(nmin,nmax),p0=p0,p1=p1)

plotBDP2(x="n",y="PostProb0or1Successes",n=c(nmin,nmax),pF=pF,shape1F=shape1F,shape2F=shape2F)

Figure 4

Probability of calling efficacy at final analysis with nfinal = 20 patients as a function of cE. Design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE), cF = 0.01 and one interim at 10 patients. For ptrue = p0 = 0.12 this corresponds to type I error, for ptrue = p1 = 0.3 this corresponds to power.

n=20
interim.at=10
cE=c(7500:9900)/10000

plotBDP2(x = "cE", 
         y = "PEcall", 
         n=n, interim.at=interim.at, p0=p0,p1=p1,
         pF=pF,cF=cF,pE=pE,cE=cE,
         shape1F=shape1F,shape2F=shape2F,shape1E=shape1E,shape2E=shape2E, 
         col = c("green", "red"))

Figure 5

Decision boundaries for futility (in red) and efficacy (in green) for a design with cF = 0.01 and cE = 0.9. Other design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE).

n=40
cE=0.9
cF=0.01
plotBDP2(x = "n", 
         y = "bFbE", 
         n=n,pF=pF,cF=cF,pE=pE,cE=cE,
         shape1F=shape1F,shape2F=shape2F,shape1E=shape1E,shape2E=shape2E,
         col=c("red","green"))

Figure 6

Type I error (CumPcallE) and probability of true stopping (CumPstopF) for the uninteresting response rate p = p0 = 0.12 (left panel). Power (CumPcallE) and probability of false stopping (CumPstopF) for the target response rate p = p1 = 0.3 (right panel). Other design parameters are cF = 0.01 and cE = 0.9, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE), interim analyses every 10 patients.

nvec=c(18:40)
interim.at=c(10,20,30)
ptrue=0.12
plotBDP2(x="n", y="PFstopEcall",
         n =nvec, interim.at = interim.at,
         pF=pF,cF=cF,pE=pE,cE=cE,ptrue=ptrue,shape1F=shape1F,shape2F=shape2F,shape1E=shape1E,shape2E=shape2E)

  
ptrue=0.3
plotBDP2(x="n", y="PFstopEcall",
         n =nvec, interim.at = interim.at,
         pF=pF,cF=cF,pE=pE,cE=cE,ptrue=ptrue,shape1F=shape1F,shape2F=shape2F,shape1E=shape1E,shape2E=shape2E)

Figure 7

Power function (CumPcallE) at nfinal = 20 (in green) and nfinal = 30 (in red) as a function of p. Other design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE), cF = 0.01 and cE = 0.3, interim analyses every 10 patients.

n=20
interim.at=10
ptrue=c(0:40)/100
plotBDP2(x = "ptrue", y = "PEcall", 
         n=n, interim.at=interim.at, ptrue=ptrue, 
         pF=pF, cF=cF, pE=pE, cE=cE, p0=p0, p1=p1, 
         shape1F=shape1F, shape2F=shape2F, shape1E=shape1E , shape2E=shape2E , 
         col = "green")
n=30
interim.at=c(10,20)
plotBDP2(x = "ptrue", y = "PEcall", 
         n=n, interim.at=interim.at, ptrue=ptrue, 
         pF=pF, cF=cF, pE=pE, cE=cE, p0=p0, p1=p1, 
         shape1F=shape1F, shape2F=shape2F, shape1E=shape1E , shape2E=shape2E , 
         col = "red",lty=2,add=TRUE)
abline(v=0.12,col="grey",lty=2)
abline(v=0.3,col="grey",lty=2)

Figure 8

Expected number of patients in the trial for nfinal = 20 (in green) and nfinal = 30 (in red) as a function of p. Other design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE), cF = 0.01 and cE = 0.3, interim analyses every 10 patients. Maximal numbers of patients (n = 20 and n = 30, resp.) are shown as dotted lines.

n=30
interim.at=c(10,20)
pvec=c(0:40)/100
interim.at=interim.at[interim.at<n]
plotBDP2(x="ptrue", y="ExpectedNumber",
         n=n,interim.at=interim.at,pF=pF,cF=cF,pE=pE,cE=cE,ptrue=pvec,
         shape1F=shape1F,shape2F=shape2F,col="red",ylim=c(0,n),cex.lab=1.4)
#> [[1]]
#> [1] 10
n=20
interim.at=interim.at[interim.at<n]
plotBDP2(x="ptrue", y="ExpectedNumber",
         n=n,interim.at=interim.at,pF=pF,cF=cF,pE=pE,cE=cE,ptrue=pvec,
         shape1F=shape1F,shape2F=shape2F,col="green",add=TRUE)
#> [[1]]
#> [1] 10

abline(h=20,col="grey",lty=2)
abline(h=30,col="grey",lty=2)

Figure 9

Predictive power (including futility stop) as function of observed responders, evaluated at interim at 10 patients (left panel) or 20 patients (right panel). Final analysis at nfinal = 30. Other design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE), cF = 0.01 and cE = 0.3

interim.at=c(10,20)
nfinal=30
  
plotBDP2(x="k",y="PredictivePower",n=nfinal,interim.at = interim.at, cE=cE,cF=cF, pE=pE,pF=pF,
                    shape1E=shape1E,shape2E=shape2E,shape1F=shape1F,shape2F=shape2F)

Figure 10

Decision boundaries for futility (in red) and efficacy (in green) for a design with cF = 0.01 and cE = 0.9 for up to 100 enrolled patients. Other design parameters are pF = 0.3, pE = 0.12, prior distributions Beta(pF, 1 − pF) and Beta(pE, 1 − pE).

n=100

plotBDP2(x = "n", 
         y = "bFbE", 
         n=n,pF=pF,cF=cF,pE=pE,cE=cE,
         shape1F=shape1F,shape2F=shape2F,shape1E=shape1E,shape2E=shape2E,
         col=c("red","green"))

Session info

sessionInfo()
#> R version 4.4.1 (2024-06-14)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 24.04.1 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C              
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: Etc/UTC
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] BDP2_0.1.3     shinyBS_0.61.1 shiny_1.9.1    rmarkdown_2.28
#> 
#> loaded via a namespace (and not attached):
#>  [1] cli_3.6.3         knitr_1.48        rlang_1.1.4       xfun_0.48        
#>  [5] highr_0.11        promises_1.3.0    jsonlite_1.8.9    xtable_1.8-4     
#>  [9] buildtools_1.0.0  htmltools_0.5.8.1 maketools_1.3.1   httpuv_1.6.15    
#> [13] sys_3.4.3         sass_0.4.9        evaluate_1.0.1    jquerylib_0.1.4  
#> [17] fastmap_1.2.0     yaml_2.3.10       lifecycle_1.0.4   compiler_4.4.1   
#> [21] Rcpp_1.0.13       later_1.3.2       digest_0.6.37     R6_2.5.1         
#> [25] magrittr_2.0.3    bslib_0.8.0       tools_4.4.1       mime_0.12        
#> [29] cachem_1.1.0