1 Introduction

Researchers assessed the effect of spinal nerve ligation (SNL) on the transcriptome of rats. In this experiment, transcriptome profiling occurred at two weeks and two months after treatment, for both the SNL group and a control group. Two biological replicates are used for every treatment - time combination. The researchers are interested in early and late effects and in genes for which the effect changes over time.

file="http://bowtie-bio.sourceforge.net/recount/ExpressionSets/hammer_eset.RData"
load(url(file))
hammer.eset
## ExpressionSet (storageMode: lockedEnvironment)
## assayData: 29516 features, 8 samples 
##   element names: exprs 
## protocolData: none
## phenoData
##   sampleNames: SRX020102 SRX020103 ... SRX020098-101 (8 total)
##   varLabels: sample.id num.tech.reps ... Time (5 total)
##   varMetadata: labelDescription
## featureData
##   featureNames: ENSRNOG00000000001 ENSRNOG00000000007 ...
##     ENSRNOG00000045521 (29516 total)
##   fvarLabels: gene
##   fvarMetadata: labelDescription
## experimentData: use 'experimentData(object)'
## Annotation:
pData(hammer.eset)
##                   sample.id num.tech.reps protocol         strain     Time
## SRX020102         SRX020102             1  control Sprague Dawley 2 months
## SRX020103         SRX020103             2  control Sprague Dawley 2 months
## SRX020104         SRX020104             1   L5 SNL Sprague Dawley 2 months
## SRX020105         SRX020105             2   L5 SNL Sprague Dawley  2months
## SRX020091-3     SRX020091-3             1  control Sprague Dawley  2 weeks
## SRX020088-90   SRX020088-90             2  control Sprague Dawley  2 weeks
## SRX020094-7     SRX020094-7             1   L5 SNL Sprague Dawley  2 weeks
## SRX020098-101 SRX020098-101             2   L5 SNL Sprague Dawley  2 weeks
library(tidyverse)
pData(hammer.eset)
##                   sample.id num.tech.reps protocol         strain     Time
## SRX020102         SRX020102             1  control Sprague Dawley 2 months
## SRX020103         SRX020103             2  control Sprague Dawley 2 months
## SRX020104         SRX020104             1   L5 SNL Sprague Dawley 2 months
## SRX020105         SRX020105             2   L5 SNL Sprague Dawley  2months
## SRX020091-3     SRX020091-3             1  control Sprague Dawley  2 weeks
## SRX020088-90   SRX020088-90             2  control Sprague Dawley  2 weeks
## SRX020094-7     SRX020094-7             1   L5 SNL Sprague Dawley  2 weeks
## SRX020098-101 SRX020098-101             2   L5 SNL Sprague Dawley  2 weeks
hammer.eset %>% exprs %>% head
##                    SRX020102 SRX020103 SRX020104 SRX020105 SRX020091-3
## ENSRNOG00000000001         2         4        18        24           7
## ENSRNOG00000000007         4         1         3         1           5
## ENSRNOG00000000008         0         1         4         2           0
## ENSRNOG00000000009         0         0         0         0           0
## ENSRNOG00000000010        19        10        19        13          50
## ENSRNOG00000000012         7         5         1         0          31
##                    SRX020088-90 SRX020094-7 SRX020098-101
## ENSRNOG00000000001            4          93            77
## ENSRNOG00000000007            4           9             4
## ENSRNOG00000000008            5           2             6
## ENSRNOG00000000009            0           0             0
## ENSRNOG00000000010           57          45            58
## ENSRNOG00000000012           26          12             9

2 Design

The researchers are interested in an effect of the treatment at the early time point, the late timepoint and the treatment \(\times\) time interaction.

The following model is used at the gene-level to model the read count \(y_{ig}\) for gene \(g\) of mouse \(i\).

For quasi-likelihood we do not specify the full distribution, only the first two moments.

\[ \left\{ \begin{array}{lcl} E[y_{ig}\vert \mathbf{x}_{ig}]&=&\mu_{ig}\\ log(\mu_{ig})&=&\eta_{ig}\\ \eta_{ig}&=&\beta_0 + \beta_{snl} x_{snl,i} + \beta_{t2m}x_{t2m,i} + \beta_\text{snl,t2m} x_{snl,i}x_{t2m,i} + \log N_i\\ \text{Var}[y_{ig}\vert \mathbf{x}_{ig}]&=&\sigma^2_g\left(\mu_{ig}+\phi\mu_{ig}^2\right) \end{array}\right. \]

with \(x_{snl,i}\) a dummy variable that is 1 if a mouse had the spinal nerve ligation and is 0 otherwise, \(x_{t2m,i}\) a dummy variable that equals 1 if the mouse was sacrificed after 2 months and 0 otherwise, and, \(\log{N}_i\) a normalisation offset to correct for sequencing depth. Note, that \(\beta_{snl}\) is the main effect for spinal nerve ligation, and corresponds to the average log fold change between treated and control mice after two weeks. The interaction \(\beta_\text{snl,t2m}\) can be interpreted as the average change in log FC between treated and control mouse at the late and early timepoint. The researchers are also interested in a assessing third contrast: the effect of the treatment at the late time point.

\[ \log \text{FC}^\text{2 months}_\text{snl-c}= \beta_{snl}+\beta_{snl,t2m}\]

The design matrix is constructed for the linear predictor.

pData(hammer.eset)$time<-factor(rep(c("2m","2w"),each=4),levels = c("2w","2m"))
levels(pData(hammer.eset)$protocol)<-c("c","snl")
dge <- DGEList(counts=exprs(hammer.eset))
dge$sample
##               group lib.size norm.factors
## SRX020102         1  5282855            1
## SRX020103         1  4562100            1
## SRX020104         1  4897807            1
## SRX020105         1  5123782            1
## SRX020091-3       1 17705411            1
## SRX020088-90      1 17449646            1
## SRX020094-7       1 23649094            1
## SRX020098-101     1 23537179            1
design <- model.matrix(~time*protocol,pData(hammer.eset))
rownames(design) = colnames(dge)
design
##               (Intercept) time2m protocolsnl time2m:protocolsnl
## SRX020102               1      1           0                  0
## SRX020103               1      1           0                  0
## SRX020104               1      1           1                  1
## SRX020105               1      1           1                  1
## SRX020091-3             1      0           0                  0
## SRX020088-90            1      0           0                  0
## SRX020094-7             1      0           1                  0
## SRX020098-101           1      0           1                  0
## attr(,"assign")
## [1] 0 1 2 3
## attr(,"contrasts")
## attr(,"contrasts")$time
## [1] "contr.treatment"
## 
## attr(,"contrasts")$protocol
## [1] "contr.treatment"

Filtering

keep <- filterByExpr(dge, design)
 dge <- dge[keep, , keep.lib.sizes=FALSE]
dge <- calcNormFactors(dge)
dge$samples
##               group lib.size norm.factors
## SRX020102         1  5279636    0.9980777
## SRX020103         1  4559314    0.9860762
## SRX020104         1  4894684    1.0233202
## SRX020105         1  5120633    1.0194303
## SRX020091-3       1 17694917    0.9642809
## SRX020088-90      1 17438982    0.9784500
## SRX020094-7       1 23631984    1.0185845
## SRX020098-101     1 23521582    1.0134838

3 Data exploration

An MDS plot shows the leading log fold changes between the 8 samples. There is a large effect according to the SNL. Are there issues with the design?

plotMDS(dge,labels=paste(hammer.eset$protocol,hammer.eset$time,sep="-"),col=as.double(hammer.eset$protocol))

4 Analysis

By replacing the glmFit function by the glmQLFit function we can perform inference with quasi likelihood. We will also replace the glmLRT function by the glmQLFTest function. Note, that to estimate the additional dispersion parameter \(\sigma^2_g\) we can correct for the degrees of freedom that are used to estimate the mean model parameters. This will enable us to correct for the degrees of freedom in the inference. Indeed, the glmQLFit test uses an F-distribution instead of the asymptotic \(\chi^2\)-distribution.

dge <- estimateDisp(dge, design, robust=TRUE)  
plotBCV(dge)

fit <- glmQLFit(dge,design)

4.1 Early

DE at the early timepoint can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 weeks}_\text{snl-c}=0 \rightarrow \beta_\text{snl}=0 \]

against the two side alternative hypothesis

\[ H_1: \log \text{FC}^\text{2 weeks}_\text{snl-c}\neq 0 \rightarrow \beta_\text{snl}\neq0 \]

early <- glmQLFTest(fit,coef="protocolsnl")
ttEarly<-topTags(early, n = nrow(dge)) # all genes
hist(ttEarly$table$PValue)

summary(dtEarly <- decideTestsDGE(early))
##        protocolsnl
## Down          3425
## NotSig        6924
## Up            3677
volcano<- ggplot(ttEarly$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(early,de.tags=rownames(dge)[as.logical(dtEarly)],ylab="log FC_late - log FC_early")

pheatmap(cpm(dge,log=TRUE)[rownames(ttEarly$table)[1:30],],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.2 Late

The effect of the treatment after two months can be estimated by the log fold change corresponding to the sum of the main effect and the interaction. This can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 months}_\text{snl-c}=0 \rightarrow \beta_\text{snl}+\beta_\text{snl,t2m}=0 \]

against the two side alternative hypothesis

\[ H_1: \log \text{FC}^\text{2 months}_\text{snl-c}\neq0 \rightarrow \beta_\text{snl}+\beta_\text{snl,t2m}\neq0 \]

L<-array(0,ncol(design))
names(L)<-colnames(design)
L[c(3,4)] <- 1
L
##        (Intercept)             time2m        protocolsnl time2m:protocolsnl 
##                  0                  0                  1                  1
late<-glmQLFTest(fit,contrast=L)
ttLate<-topTags(late, n = nrow(dge)) # all genes
hist(ttLate$table$PValue)

summary(dtLate <- decideTestsDGE(late))
##        1*protocolsnl 1*time2m:protocolsnl
## Down                                 3113
## NotSig                               7465
## Up                                   3448
volcano<- ggplot(ttLate$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(late,de.tags=rownames(dge)[as.logical(dtLate)],ylab="log FC_late - log FC_early")

pheatmap(cpm(dge,log=TRUE)[rownames(ttLate$table)[1:30],],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.3 Interaction

To assess if the treatment effect changes over time, we will test the null hypothesis that the interaction term equals zero vs the alternative that the interaction term is different from zero.

This can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 months}_\text{snl-c}-\log \text{FC}^\text{2 weeks}_\text{snl-c}=0 \rightarrow \beta_\text{snl,t2m}=0 \]

against the alternative hyptothesis

\[ H_1: \log \text{FC}^\text{2 months}_\text{snl-c}-\log \text{FC}^\text{2 weeks}_\text{snl-c}\neq0 \rightarrow \beta_\text{snl,t2m}\neq0 \]

inter <- glmQLFTest(fit,coef="time2m:protocolsnl")
ttInter<-topTags(inter, n = nrow(dge)) # all genes
hist(ttInter$table$PValue)

summary(dtInter <- decideTestsDGE(inter))
##        time2m:protocolsnl
## Down                    0
## NotSig              14025
## Up                      1
volcano<- ggplot(ttInter$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(inter,de.tags=rownames(dge)[as.logical(dtInter)],ylab="log FC_late - log FC_early")

pheatmap(cpm(dge,log=TRUE)[rownames(ttInter$table)[1:30],],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.4 Remarks

  • There are very many DE genes according to the SNL treatment at the early and late timepoint.

  • Issues with the design?

  • There are very few interactions significant. Can you explain this?

5 Assess p-values

ttEarly$table <- ttEarly$table %>%
  mutate(z = sign(logFC) * abs(qnorm(PValue/2)))

ttEarly$table %>%
  ggplot(aes(x=z)) +
  geom_histogram(aes(y = ..density..), color = "black") +
  stat_function(fun = dnorm,
      args = list(
    mean = 0,
    sd=1)
  )
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

LS0tCnRpdGxlOiAiSGFtbWVyIERhdGFzZXQgd2l0aCBRdWFzaSBsaWtlbGlob29kIgphdXRob3I6ICJMaWV2ZW4gQ2xlbWVudCIKZGF0ZTogInN0YXRPbWljcywgR2hlbnQgVW5pdmVyc2l0eSAoaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSIKb3V0cHV0OgogICAgaHRtbF9kb2N1bWVudDoKICAgICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgICB0aGVtZTogY29zbW8KICAgICAgdG9jOiB0cnVlCiAgICAgIHRvY19mbG9hdDogdHJ1ZQogICAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQotLS0KCgpgYGB7cixlY2hvPUZBTFNFfQpsaWJyYXJ5KGVkZ2VSKQpsaWJyYXJ5KGdwbG90cykKbGlicmFyeShwaGVhdG1hcCkKbGlicmFyeShSQ29sb3JCcmV3ZXIpCmBgYAoKIyBJbnRyb2R1Y3Rpb24KClJlc2VhcmNoZXJzIGFzc2Vzc2VkIHRoZSBlZmZlY3Qgb2Ygc3BpbmFsIG5lcnZlIGxpZ2F0aW9uIChTTkwpIG9uIHRoZSB0cmFuc2NyaXB0b21lIG9mIHJhdHMuIEluIHRoaXMgZXhwZXJpbWVudCwgdHJhbnNjcmlwdG9tZSBwcm9maWxpbmcgb2NjdXJyZWQgYXQgdHdvIHdlZWtzIGFuZCB0d28gbW9udGhzIGFmdGVyIHRyZWF0bWVudCwgZm9yIGJvdGggdGhlIFNOTCBncm91cCBhbmQgYSBjb250cm9sIGdyb3VwLiBUd28gYmlvbG9naWNhbCByZXBsaWNhdGVzIGFyZSB1c2VkIGZvciBldmVyeSB0cmVhdG1lbnQgLSB0aW1lIGNvbWJpbmF0aW9uLiBUaGUgcmVzZWFyY2hlcnMgYXJlIGludGVyZXN0ZWQgaW4gZWFybHkgYW5kIGxhdGUgZWZmZWN0cyBhbmQgaW4gZ2VuZXMgZm9yIHdoaWNoIHRoZSBlZmZlY3QgY2hhbmdlcyBvdmVyIHRpbWUuCgpgYGB7cn0KZmlsZT0iaHR0cDovL2Jvd3RpZS1iaW8uc291cmNlZm9yZ2UubmV0L3JlY291bnQvRXhwcmVzc2lvblNldHMvaGFtbWVyX2VzZXQuUkRhdGEiCmxvYWQodXJsKGZpbGUpKQpoYW1tZXIuZXNldApgYGAKCmBgYHtyfQpwRGF0YShoYW1tZXIuZXNldCkKYGBgYAoKYGBge3J9CmxpYnJhcnkodGlkeXZlcnNlKQpwRGF0YShoYW1tZXIuZXNldCkKaGFtbWVyLmVzZXQgJT4lIGV4cHJzICU+JSBoZWFkCmBgYAoKIyBEZXNpZ24KCgoKVGhlIHJlc2VhcmNoZXJzIGFyZSBpbnRlcmVzdGVkIGluIGFuIGVmZmVjdCBvZiB0aGUgdHJlYXRtZW50IGF0IHRoZSBlYXJseSB0aW1lIHBvaW50LCB0aGUgbGF0ZSB0aW1lcG9pbnQgYW5kIHRoZSB0cmVhdG1lbnQgJFx0aW1lcyQgdGltZSBpbnRlcmFjdGlvbi4KClRoZSBmb2xsb3dpbmcgbW9kZWwgaXMgdXNlZCBhdCB0aGUgZ2VuZS1sZXZlbCB0byBtb2RlbCB0aGUgcmVhZCBjb3VudCAkeV97aWd9JCBmb3IgZ2VuZSAkZyQgb2YgbW91c2UgJGkkLgoKRm9yIHF1YXNpLWxpa2VsaWhvb2Qgd2UgZG8gbm90IHNwZWNpZnkgdGhlIGZ1bGwgZGlzdHJpYnV0aW9uLCBvbmx5IHRoZSBmaXJzdCB0d28gbW9tZW50cy4KCiQkClxsZWZ0XHsKXGJlZ2lue2FycmF5fXtsY2x9CkVbeV97aWd9XHZlcnQgXG1hdGhiZnt4fV97aWd9XSY9JlxtdV97aWd9XFwKbG9nKFxtdV97aWd9KSY9JlxldGFfe2lnfVxcClxldGFfe2lnfSY9JlxiZXRhXzAgKyBcYmV0YV97c25sfSB4X3tzbmwsaX0gKyBcYmV0YV97dDJtfXhfe3QybSxpfSArIFxiZXRhX1x0ZXh0e3NubCx0Mm19IHhfe3NubCxpfXhfe3QybSxpfSArIFxsb2cgTl9pXFwKXHRleHR7VmFyfVt5X3tpZ31cdmVydCBcbWF0aGJme3h9X3tpZ31dJj0mXHNpZ21hXjJfZ1xsZWZ0KFxtdV97aWd9K1xwaGlcbXVfe2lnfV4yXHJpZ2h0KQpcZW5ke2FycmF5fVxyaWdodC4KJCQKCndpdGggJHhfe3NubCxpfSQgYSBkdW1teSB2YXJpYWJsZSB0aGF0IGlzIDEgaWYgYSBtb3VzZSBoYWQgdGhlIHNwaW5hbCBuZXJ2ZSBsaWdhdGlvbiBhbmQgaXMgMCBvdGhlcndpc2UsICR4X3t0Mm0saX0kIGEgZHVtbXkgdmFyaWFibGUgdGhhdCBlcXVhbHMgMSBpZiB0aGUgbW91c2Ugd2FzIHNhY3JpZmljZWQgYWZ0ZXIgMiBtb250aHMgYW5kIDAgb3RoZXJ3aXNlLCBhbmQsICRcbG9ne059X2kkIGEgbm9ybWFsaXNhdGlvbiBvZmZzZXQgdG8gY29ycmVjdCBmb3Igc2VxdWVuY2luZyBkZXB0aC4gTm90ZSwgdGhhdCAkXGJldGFfe3NubH0kIGlzIHRoZSBtYWluIGVmZmVjdCBmb3Igc3BpbmFsIG5lcnZlIGxpZ2F0aW9uLCBhbmQgY29ycmVzcG9uZHMgdG8gdGhlIGF2ZXJhZ2UgbG9nIGZvbGQgY2hhbmdlIGJldHdlZW4gdHJlYXRlZCBhbmQgY29udHJvbCBtaWNlIGFmdGVyIHR3byB3ZWVrcy4gVGhlIGludGVyYWN0aW9uICAkXGJldGFfXHRleHR7c25sLHQybX0kIGNhbiBiZSBpbnRlcnByZXRlZCBhcyB0aGUgYXZlcmFnZSBjaGFuZ2UgaW4gbG9nIEZDIGJldHdlZW4gdHJlYXRlZCBhbmQgY29udHJvbCBtb3VzZSBhdCB0aGUgbGF0ZSBhbmQgZWFybHkgdGltZXBvaW50LiBUaGUgcmVzZWFyY2hlcnMgYXJlIGFsc28gaW50ZXJlc3RlZCBpbiBhIGFzc2Vzc2luZyB0aGlyZCBjb250cmFzdDogdGhlIGVmZmVjdCBvZiB0aGUgdHJlYXRtZW50IGF0IHRoZSBsYXRlIHRpbWUgcG9pbnQuCgokJCBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIG1vbnRoc31fXHRleHR7c25sLWN9PSBcYmV0YV97c25sfStcYmV0YV97c25sLHQybX0kJAoKVGhlIGRlc2lnbiBtYXRyaXggaXMgY29uc3RydWN0ZWQgZm9yIHRoZSBsaW5lYXIgcHJlZGljdG9yLgoKYGBge3J9CnBEYXRhKGhhbW1lci5lc2V0KSR0aW1lPC1mYWN0b3IocmVwKGMoIjJtIiwiMnciKSxlYWNoPTQpLGxldmVscyA9IGMoIjJ3IiwiMm0iKSkKbGV2ZWxzKHBEYXRhKGhhbW1lci5lc2V0KSRwcm90b2NvbCk8LWMoImMiLCJzbmwiKQpgYGAKCmBgYHtyfQpkZ2UgPC0gREdFTGlzdChjb3VudHM9ZXhwcnMoaGFtbWVyLmVzZXQpKQpkZ2Ukc2FtcGxlCmBgYAoKYGBge3J9CmRlc2lnbiA8LSBtb2RlbC5tYXRyaXgofnRpbWUqcHJvdG9jb2wscERhdGEoaGFtbWVyLmVzZXQpKQpyb3duYW1lcyhkZXNpZ24pID0gY29sbmFtZXMoZGdlKQpkZXNpZ24KYGBgCgpGaWx0ZXJpbmcKYGBge3J9CmtlZXAgPC0gZmlsdGVyQnlFeHByKGRnZSwgZGVzaWduKQogZGdlIDwtIGRnZVtrZWVwLCAsIGtlZXAubGliLnNpemVzPUZBTFNFXQpgYGAKCmBgYHtyfQpkZ2UgPC0gY2FsY05vcm1GYWN0b3JzKGRnZSkKZGdlJHNhbXBsZXMKYGBgCgojIERhdGEgZXhwbG9yYXRpb24KCkFuIE1EUyBwbG90IHNob3dzIHRoZSBsZWFkaW5nIGxvZyBmb2xkIGNoYW5nZXMgYmV0d2VlbiB0aGUgOCBzYW1wbGVzLiBUaGVyZSBpcyBhIGxhcmdlIGVmZmVjdCBhY2NvcmRpbmcgdG8gdGhlIFNOTC4gQXJlIHRoZXJlIGlzc3VlcyB3aXRoIHRoZSBkZXNpZ24/CgpgYGB7cn0KcGxvdE1EUyhkZ2UsbGFiZWxzPXBhc3RlKGhhbW1lci5lc2V0JHByb3RvY29sLGhhbW1lci5lc2V0JHRpbWUsc2VwPSItIiksY29sPWFzLmRvdWJsZShoYW1tZXIuZXNldCRwcm90b2NvbCkpCmBgYAoKIyBBbmFseXNpcwoKQnkgcmVwbGFjaW5nIHRoZSBnbG1GaXQgZnVuY3Rpb24gYnkgdGhlIGdsbVFMRml0IGZ1bmN0aW9uIHdlIGNhbiBwZXJmb3JtIGluZmVyZW5jZSB3aXRoIHF1YXNpIGxpa2VsaWhvb2QuCldlIHdpbGwgYWxzbyByZXBsYWNlIHRoZSBnbG1MUlQgZnVuY3Rpb24gYnkgdGhlIGdsbVFMRlRlc3QgZnVuY3Rpb24uIE5vdGUsIHRoYXQgdG8gZXN0aW1hdGUgdGhlIGFkZGl0aW9uYWwgZGlzcGVyc2lvbiBwYXJhbWV0ZXIgJFxzaWdtYV4yX2ckIHdlIGNhbiBjb3JyZWN0IGZvciB0aGUgZGVncmVlcyBvZiBmcmVlZG9tIHRoYXQgYXJlIHVzZWQgdG8gZXN0aW1hdGUgdGhlIG1lYW4gbW9kZWwgcGFyYW1ldGVycy4gVGhpcyB3aWxsIGVuYWJsZSB1cyB0byBjb3JyZWN0IGZvciB0aGUgZGVncmVlcyBvZiBmcmVlZG9tIGluIHRoZSBpbmZlcmVuY2UuIEluZGVlZCwgdGhlIGdsbVFMRml0IHRlc3QgdXNlcyBhbiBGLWRpc3RyaWJ1dGlvbiBpbnN0ZWFkIG9mIHRoZSBhc3ltcHRvdGljICRcY2hpXjIkLWRpc3RyaWJ1dGlvbi4KCmBgYHtyfQpkZ2UgPC0gZXN0aW1hdGVEaXNwKGRnZSwgZGVzaWduLCByb2J1c3Q9VFJVRSkgIApwbG90QkNWKGRnZSkKZml0IDwtIGdsbVFMRml0KGRnZSxkZXNpZ24pCmBgYAoKCiMjIEVhcmx5CgpERSBhdCB0aGUgZWFybHkgdGltZXBvaW50IGNhbiBiZSBhc3Nlc3NlZCBieSB0ZXN0aW5nIHRoZSBudWxsIGh5cG90aGVzaXMKCiQkCkhfMDogXGxvZyBcdGV4dHtGQ31eXHRleHR7MiB3ZWVrc31fXHRleHR7c25sLWN9PTAgXHJpZ2h0YXJyb3cgXGJldGFfXHRleHR7c25sfT0wCiQkCgphZ2FpbnN0IHRoZSB0d28gc2lkZSBhbHRlcm5hdGl2ZSBoeXBvdGhlc2lzCgokJApIXzE6IFxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgd2Vla3N9X1x0ZXh0e3NubC1jfVxuZXEgMCBccmlnaHRhcnJvdyBcYmV0YV9cdGV4dHtzbmx9XG5lcTAKJCQKCmBgYHtyfQplYXJseSA8LSBnbG1RTEZUZXN0KGZpdCxjb2VmPSJwcm90b2NvbHNubCIpCnR0RWFybHk8LXRvcFRhZ3MoZWFybHksIG4gPSBucm93KGRnZSkpICMgYWxsIGdlbmVzCmhpc3QodHRFYXJseSR0YWJsZSRQVmFsdWUpCnN1bW1hcnkoZHRFYXJseSA8LSBkZWNpZGVUZXN0c0RHRShlYXJseSkpCnZvbGNhbm88LSBnZ3Bsb3QodHRFYXJseSR0YWJsZSxhZXMoeD1sb2dGQyx5PS1sb2cxMChQVmFsdWUpLGNvbG9yPUZEUjwwLjA1KSkgKyBnZW9tX3BvaW50KCkgKyBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzPWMoImJsYWNrIiwicmVkIikpCnZvbGNhbm8KcGxvdFNtZWFyKGVhcmx5LGRlLnRhZ3M9cm93bmFtZXMoZGdlKVthcy5sb2dpY2FsKGR0RWFybHkpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCnBoZWF0bWFwKGNwbShkZ2UsbG9nPVRSVUUpW3Jvd25hbWVzKHR0RWFybHkkdGFibGUpWzE6MzBdLF0sbGFiZWxzX2NvbCA9IHBhc3RlKGhhbW1lci5lc2V0JHByb3RvY29sLGhhbW1lci5lc2V0JHRpbWUsc2VwPSItIikpCmBgYAoKCiMjIExhdGUKClRoZSBlZmZlY3Qgb2YgdGhlIHRyZWF0bWVudCBhZnRlciB0d28gbW9udGhzIGNhbiBiZSBlc3RpbWF0ZWQgYnkgdGhlIGxvZyBmb2xkIGNoYW5nZSBjb3JyZXNwb25kaW5nIHRvIHRoZSBzdW0gb2YgdGhlIG1haW4gZWZmZWN0IGFuZCB0aGUgaW50ZXJhY3Rpb24uClRoaXMgY2FuIGJlIGFzc2Vzc2VkIGJ5IHRlc3RpbmcgdGhlIG51bGwgaHlwb3RoZXNpcwoKJCQKSF8wOiBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIG1vbnRoc31fXHRleHR7c25sLWN9PTAgXHJpZ2h0YXJyb3cgXGJldGFfXHRleHR7c25sfStcYmV0YV9cdGV4dHtzbmwsdDJtfT0wCiQkCgphZ2FpbnN0IHRoZSB0d28gc2lkZSBhbHRlcm5hdGl2ZSBoeXBvdGhlc2lzCgokJApIXzE6IFxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgbW9udGhzfV9cdGV4dHtzbmwtY31cbmVxMCBccmlnaHRhcnJvdyBcYmV0YV9cdGV4dHtzbmx9K1xiZXRhX1x0ZXh0e3NubCx0Mm19XG5lcTAKJCQKCgpgYGB7cn0KTDwtYXJyYXkoMCxuY29sKGRlc2lnbikpCm5hbWVzKEwpPC1jb2xuYW1lcyhkZXNpZ24pCkxbYygzLDQpXSA8LSAxCkwKbGF0ZTwtZ2xtUUxGVGVzdChmaXQsY29udHJhc3Q9TCkKdHRMYXRlPC10b3BUYWdzKGxhdGUsIG4gPSBucm93KGRnZSkpICMgYWxsIGdlbmVzCmhpc3QodHRMYXRlJHRhYmxlJFBWYWx1ZSkKc3VtbWFyeShkdExhdGUgPC0gZGVjaWRlVGVzdHNER0UobGF0ZSkpCnZvbGNhbm88LSBnZ3Bsb3QodHRMYXRlJHRhYmxlLGFlcyh4PWxvZ0ZDLHk9LWxvZzEwKFBWYWx1ZSksY29sb3I9RkRSPDAuMDUpKSArIGdlb21fcG9pbnQoKSArIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXM9YygiYmxhY2siLCJyZWQiKSkKdm9sY2FubwpwbG90U21lYXIobGF0ZSxkZS50YWdzPXJvd25hbWVzKGRnZSlbYXMubG9naWNhbChkdExhdGUpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCnBoZWF0bWFwKGNwbShkZ2UsbG9nPVRSVUUpW3Jvd25hbWVzKHR0TGF0ZSR0YWJsZSlbMTozMF0sXSxsYWJlbHNfY29sID0gcGFzdGUoaGFtbWVyLmVzZXQkcHJvdG9jb2wsaGFtbWVyLmVzZXQkdGltZSxzZXA9Ii0iKSkKYGBgCgojIyBJbnRlcmFjdGlvbgoKVG8gYXNzZXNzIGlmIHRoZSB0cmVhdG1lbnQgZWZmZWN0IGNoYW5nZXMgb3ZlciB0aW1lLCB3ZSB3aWxsIHRlc3QgdGhlIG51bGwgaHlwb3RoZXNpcyB0aGF0IHRoZSBpbnRlcmFjdGlvbiB0ZXJtIGVxdWFscyB6ZXJvIHZzIHRoZSBhbHRlcm5hdGl2ZSB0aGF0IHRoZSBpbnRlcmFjdGlvbiB0ZXJtIGlzIGRpZmZlcmVudCBmcm9tIHplcm8uICAKClRoaXMgY2FuIGJlIGFzc2Vzc2VkIGJ5IHRlc3RpbmcgdGhlIG51bGwgaHlwb3RoZXNpcwoKJCQKSF8wOiBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIG1vbnRoc31fXHRleHR7c25sLWN9LVxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgd2Vla3N9X1x0ZXh0e3NubC1jfT0wIFxyaWdodGFycm93IFxiZXRhX1x0ZXh0e3NubCx0Mm19PTAKJCQKCmFnYWluc3QgdGhlIGFsdGVybmF0aXZlIGh5cHRvdGhlc2lzCgokJApIXzE6IFxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgbW9udGhzfV9cdGV4dHtzbmwtY30tXGxvZyBcdGV4dHtGQ31eXHRleHR7MiB3ZWVrc31fXHRleHR7c25sLWN9XG5lcTAgXHJpZ2h0YXJyb3cgXGJldGFfXHRleHR7c25sLHQybX1cbmVxMAokJAoKYGBge3J9CmludGVyIDwtIGdsbVFMRlRlc3QoZml0LGNvZWY9InRpbWUybTpwcm90b2NvbHNubCIpCnR0SW50ZXI8LXRvcFRhZ3MoaW50ZXIsIG4gPSBucm93KGRnZSkpICMgYWxsIGdlbmVzCmhpc3QodHRJbnRlciR0YWJsZSRQVmFsdWUpCnN1bW1hcnkoZHRJbnRlciA8LSBkZWNpZGVUZXN0c0RHRShpbnRlcikpCnZvbGNhbm88LSBnZ3Bsb3QodHRJbnRlciR0YWJsZSxhZXMoeD1sb2dGQyx5PS1sb2cxMChQVmFsdWUpLGNvbG9yPUZEUjwwLjA1KSkgKyBnZW9tX3BvaW50KCkgKyBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzPWMoImJsYWNrIiwicmVkIikpCnZvbGNhbm8KcGxvdFNtZWFyKGludGVyLGRlLnRhZ3M9cm93bmFtZXMoZGdlKVthcy5sb2dpY2FsKGR0SW50ZXIpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCnBoZWF0bWFwKGNwbShkZ2UsbG9nPVRSVUUpW3Jvd25hbWVzKHR0SW50ZXIkdGFibGUpWzE6MzBdLF0sbGFiZWxzX2NvbCA9IHBhc3RlKGhhbW1lci5lc2V0JHByb3RvY29sLGhhbW1lci5lc2V0JHRpbWUsc2VwPSItIikpCmBgYAoKIyMgUmVtYXJrcwoKLSBUaGVyZSBhcmUgdmVyeSBtYW55IERFIGdlbmVzIGFjY29yZGluZyB0byB0aGUgU05MIHRyZWF0bWVudCBhdCB0aGUgZWFybHkgYW5kIGxhdGUgdGltZXBvaW50LgoKLSBJc3N1ZXMgd2l0aCB0aGUgZGVzaWduPwoKLSBUaGVyZSBhcmUgdmVyeSBmZXcgaW50ZXJhY3Rpb25zIHNpZ25pZmljYW50LiBDYW4geW91IGV4cGxhaW4gdGhpcz8KCiMgQXNzZXNzIHAtdmFsdWVzCgpgYGB7cn0KdHRFYXJseSR0YWJsZSA8LSB0dEVhcmx5JHRhYmxlICU+JQogIG11dGF0ZSh6ID0gc2lnbihsb2dGQykgKiBhYnMocW5vcm0oUFZhbHVlLzIpKSkKCnR0RWFybHkkdGFibGUgJT4lCiAgZ2dwbG90KGFlcyh4PXopKSArCiAgZ2VvbV9oaXN0b2dyYW0oYWVzKHkgPSAuLmRlbnNpdHkuLiksIGNvbG9yID0gImJsYWNrIikgKwogIHN0YXRfZnVuY3Rpb24oZnVuID0gZG5vcm0sCiAgICAgIGFyZ3MgPSBsaXN0KAogICAgbWVhbiA9IDAsCiAgICBzZD0xKQogICkKYGBgCg==