1 Introduction

Researchers assessed the effect of spinal nerve ligation (SNL) on the transcriptome of rats. In this experiment, transcriptome profiling occurred at two weeks and two months after treatment, for both the SNL group and a control group. Two biological replicates are used for every treatment - time combination. The researchers are interested in early and late effects and in genes for which the effect changes over time.

file="http://bowtie-bio.sourceforge.net/recount/ExpressionSets/hammer_eset.RData"
load(url(file))
hammer.eset
## ExpressionSet (storageMode: lockedEnvironment)
## assayData: 29516 features, 8 samples 
##   element names: exprs 
## protocolData: none
## phenoData
##   sampleNames: SRX020102 SRX020103 ... SRX020098-101 (8 total)
##   varLabels: sample.id num.tech.reps ... Time (5 total)
##   varMetadata: labelDescription
## featureData
##   featureNames: ENSRNOG00000000001 ENSRNOG00000000007 ...
##     ENSRNOG00000045521 (29516 total)
##   fvarLabels: gene
##   fvarMetadata: labelDescription
## experimentData: use 'experimentData(object)'
## Annotation:
pData(hammer.eset)
##                   sample.id num.tech.reps protocol         strain     Time
## SRX020102         SRX020102             1  control Sprague Dawley 2 months
## SRX020103         SRX020103             2  control Sprague Dawley 2 months
## SRX020104         SRX020104             1   L5 SNL Sprague Dawley 2 months
## SRX020105         SRX020105             2   L5 SNL Sprague Dawley  2months
## SRX020091-3     SRX020091-3             1  control Sprague Dawley  2 weeks
## SRX020088-90   SRX020088-90             2  control Sprague Dawley  2 weeks
## SRX020094-7     SRX020094-7             1   L5 SNL Sprague Dawley  2 weeks
## SRX020098-101 SRX020098-101             2   L5 SNL Sprague Dawley  2 weeks
library(tidyverse)
pData(hammer.eset)
##                   sample.id num.tech.reps protocol         strain     Time
## SRX020102         SRX020102             1  control Sprague Dawley 2 months
## SRX020103         SRX020103             2  control Sprague Dawley 2 months
## SRX020104         SRX020104             1   L5 SNL Sprague Dawley 2 months
## SRX020105         SRX020105             2   L5 SNL Sprague Dawley  2months
## SRX020091-3     SRX020091-3             1  control Sprague Dawley  2 weeks
## SRX020088-90   SRX020088-90             2  control Sprague Dawley  2 weeks
## SRX020094-7     SRX020094-7             1   L5 SNL Sprague Dawley  2 weeks
## SRX020098-101 SRX020098-101             2   L5 SNL Sprague Dawley  2 weeks
hammer.eset %>% exprs %>% head
##                    SRX020102 SRX020103 SRX020104 SRX020105 SRX020091-3
## ENSRNOG00000000001         2         4        18        24           7
## ENSRNOG00000000007         4         1         3         1           5
## ENSRNOG00000000008         0         1         4         2           0
## ENSRNOG00000000009         0         0         0         0           0
## ENSRNOG00000000010        19        10        19        13          50
## ENSRNOG00000000012         7         5         1         0          31
##                    SRX020088-90 SRX020094-7 SRX020098-101
## ENSRNOG00000000001            4          93            77
## ENSRNOG00000000007            4           9             4
## ENSRNOG00000000008            5           2             6
## ENSRNOG00000000009            0           0             0
## ENSRNOG00000000010           57          45            58
## ENSRNOG00000000012           26          12             9

2 Design

The researchers are interested in an effect of the treatment at the early time point, the late timepoint and the treatment \(\times\) time interaction.

pData(hammer.eset)$time<-factor(rep(c("2m","2w"),each=4),levels = c("2w","2m"))
levels(pData(hammer.eset)$protocol)<-c("c","snl")

The read count \(y_{ig}\) for gene \(g\) of mouse \(i\) are modelled as follows:

\[ \left\{ \begin{array}{lcl} y_{ig} &\sim& NB(\mu_{ig},\phi_g)\\ E[y_{ig}\vert \mathbf{x}_{ig}]&=&\mu_{ig}\\ log(\mu_{ig})&=&\eta_{ig}\\ \eta_{ig}&=&\beta_0 + \beta_{snl} x_{snl,i} + \beta_{t2m}x_{t2m,i} + \beta_\text{snl,t2m} x_{snl,i}x_{t2m,i} + \log N_i\\ \end{array}\right. \]

with \(x_{snl,i}\) a dummy variable that is 1 if a mouse had the spinal nerve ligation and is 0 otherwise, \(x_{t2m,i}\) a dummy variable that equals 1 if the mouse was sacrificed after 2 months and 0 otherwise, and, \(\log{N}_i\) a normalisation offset to correct for sequencing depth. Note, that \(\beta_{snl}\) is the main effect for spinal nerve ligation, and corresponds to the average log fold change between treated and control mice after two weeks. The interaction \(\beta_\text{snl,t2m}\) can be interpreted as the average change in log FC between treated and control mouse at the late and early timepoint. The researchers are also interested in a assessing third contrast: the effect of the treatment at the late time point.

\[ \log \text{FC}^\text{2 months}_\text{snl-c}= \beta_{snl}+\beta_{snl,t2m}\]

The design matrix is constructed for the linear predictor.

dge <- DGEList(counts=exprs(hammer.eset))
dge$sample
##               group lib.size norm.factors
## SRX020102         1  5282855            1
## SRX020103         1  4562100            1
## SRX020104         1  4897807            1
## SRX020105         1  5123782            1
## SRX020091-3       1 17705411            1
## SRX020088-90      1 17449646            1
## SRX020094-7       1 23649094            1
## SRX020098-101     1 23537179            1
design <- model.matrix(~time*protocol,pData(hammer.eset))
rownames(design) = colnames(dge)
design
##               (Intercept) time2m protocolsnl time2m:protocolsnl
## SRX020102               1      1           0                  0
## SRX020103               1      1           0                  0
## SRX020104               1      1           1                  1
## SRX020105               1      1           1                  1
## SRX020091-3             1      0           0                  0
## SRX020088-90            1      0           0                  0
## SRX020094-7             1      0           1                  0
## SRX020098-101           1      0           1                  0
## attr(,"assign")
## [1] 0 1 2 3
## attr(,"contrasts")
## attr(,"contrasts")$time
## [1] "contr.treatment"
## 
## attr(,"contrasts")$protocol
## [1] "contr.treatment"

Filtering

keep <- filterByExpr(dge, design)
 dge <- dge[keep, , keep.lib.sizes=FALSE]
dge <- calcNormFactors(dge)
dge$samples
##               group lib.size norm.factors
## SRX020102         1  5279636    0.9980777
## SRX020103         1  4559314    0.9860762
## SRX020104         1  4894684    1.0233202
## SRX020105         1  5120633    1.0194303
## SRX020091-3       1 17694917    0.9642809
## SRX020088-90      1 17438982    0.9784500
## SRX020094-7       1 23631984    1.0185845
## SRX020098-101     1 23521582    1.0134838

3 Data exploration

An MDS plot shows the leading log fold changes between the 8 samples. There is a large effect according to the SNL. Are there issues with the design?

plotMDS(dge,labels=paste(hammer.eset$protocol,hammer.eset$time,sep="-"),col=as.double(hammer.eset$protocol))

4 Analysis

dge <- estimateDisp(dge, design, robust=TRUE)  
plotBCV(dge)

fit <- glmFit(dge,design)

4.1 Early

DE at the early timepoint can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 weeks}_\text{snl-c}=0 \rightarrow \beta_\text{snl}=0 \]

against the two side alternative hypothesis

\[ H_1: \log \text{FC}^\text{2 weeks}_\text{snl-c}\neq 0 \rightarrow \beta_\text{snl}\neq0 \]

early <- glmLRT(fit,coef="protocolsnl")
ttEarly<-topTags(early, n = nrow(dge)) # all genes
hist(ttEarly$table$PValue,main="early",xlab="p-values")

summary(dtEarly <- decideTestsDGE(early))
##        protocolsnl
## Down          3445
## NotSig        6859
## Up            3722
volcano<- ggplot(ttEarly$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(early,de.tags=rownames(dge)[as.logical(dtEarly)],ylab="log FC_late - log FC_early")

Because there are 7167 significant genes we restrict the heatmap to the top 30 genes.

pheatmap(cpm(dge,log=TRUE)[rownames(ttEarly$table)[1:30],],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.2 Late

The effect of the treatment after two months can be estimated by the log fold change corresponding to the sum of the main effect and the interaction. This can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 months}_\text{snl-c}=0 \rightarrow \beta_\text{snl}+\beta_\text{snl,t2m}=0 \]

against the two side alternative hypothesis

\[ H_1: \log \text{FC}^\text{2 months}_\text{snl-c}\neq0 \rightarrow \beta_\text{snl}+\beta_\text{snl,t2m}\neq0 \]

L<-array(0,ncol(design))
names(L)<-colnames(design)
L[c(3,4)] <- 1
L
##        (Intercept)             time2m        protocolsnl time2m:protocolsnl 
##                  0                  0                  1                  1
late<-glmLRT(fit,contrast=L)
ttLate<-topTags(late, n = nrow(dge)) # all genes
hist(ttLate$table$PValue,main="late",xlab="p-values")

summary(dtLate <- decideTestsDGE(late))
##        1*protocolsnl 1*time2m:protocolsnl
## Down                                 3103
## NotSig                               7446
## Up                                   3477
volcano<- ggplot(ttLate$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(late,de.tags=rownames(dge)[as.logical(dtLate)],ylab="log FC_late - log FC_early")

Because there are 6580 significant genes we restrict the heatmap to the top 30 genes.

pheatmap(cpm(dge,log=TRUE)[rownames(ttLate$table)[1:30],],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.3 Interaction

To assess if the treatment effect changes over time, we will test the null hypothesis that the interaction term equals zero vs the alternative that the interaction term is different from zero.

This can be assessed by testing the null hypothesis

\[ H_0: \log \text{FC}^\text{2 months}_\text{snl-c}-\log \text{FC}^\text{2 weeks}_\text{snl-c}=0 \rightarrow \beta_\text{snl,t2m}=0 \]

against the alternative hyptothesis

\[ H_1: \log \text{FC}^\text{2 months}_\text{snl-c}-\log \text{FC}^\text{2 weeks}_\text{snl-c}\neq0 \rightarrow \beta_\text{snl,t2m}\neq0 \]

inter <- glmLRT(fit,coef="time2m:protocolsnl")
ttInter<-topTags(inter, n = nrow(dge)) # all genes
hist(ttInter$table$PValue,main="interaction",xlab="p-values")

summary(dtInter <- decideTestsDGE(inter))
##        time2m:protocolsnl
## Down                   17
## NotSig              13992
## Up                     17
volcano<- ggplot(ttInter$table,aes(x=logFC,y=-log10(PValue),color=FDR<0.05)) + geom_point() + scale_color_manual(values=c("black","red"))
volcano

plotSmear(inter,de.tags=rownames(dge)[as.logical(dtInter)],ylab="log FC_late - log FC_early")

Because there are 34 significant genes we plot them all in the heatmap.

pheatmap(cpm(dge,log=TRUE)[as.logical(dtInter),],labels_col = paste(hammer.eset$protocol,hammer.eset$time,sep="-"))

4.4 Remarks

  • There are very many DE genes according to the SNL treatment at the early and late timepoint.

  • Issues with the design?

  • There are very few interactions significant. Can you explain this?

5 Assess p-values

ttEarly$table <- ttEarly$table %>%
  mutate(z = sign(logFC) * abs(qnorm(PValue/2)))

ttEarly$table %>%
  ggplot(aes(x=z)) +
  geom_histogram(aes(y = ..density..), color = "black") +
  stat_function(fun = dnorm,
      args = list(
    mean = 0,
    sd=1)
  )
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

LS0tCnRpdGxlOiAiSGFtbWVyIERhdGFzZXQiCmF1dGhvcjogIkxpZXZlbiBDbGVtZW50IgpkYXRlOiAic3RhdE9taWNzLCBHaGVudCBVbml2ZXJzaXR5IChodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8pIgpvdXRwdXQ6CiAgICBodG1sX2RvY3VtZW50OgogICAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAgIHRoZW1lOiBjb3NtbwogICAgICB0b2M6IHRydWUKICAgICAgdG9jX2Zsb2F0OiB0cnVlCiAgICAgIGhpZ2hsaWdodDogdGFuZ28KICAgICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCi0tLQoKCmBgYHtyLGVjaG89RkFMU0V9CmxpYnJhcnkoZWRnZVIpCmxpYnJhcnkoZ3Bsb3RzKQpsaWJyYXJ5KHBoZWF0bWFwKQpsaWJyYXJ5KFJDb2xvckJyZXdlcikKYGBgCgojIEludHJvZHVjdGlvbgoKUmVzZWFyY2hlcnMgYXNzZXNzZWQgdGhlIGVmZmVjdCBvZiBzcGluYWwgbmVydmUgbGlnYXRpb24gKFNOTCkgb24gdGhlIHRyYW5zY3JpcHRvbWUgb2YgcmF0cy4gSW4gdGhpcyBleHBlcmltZW50LCB0cmFuc2NyaXB0b21lIHByb2ZpbGluZyBvY2N1cnJlZCBhdCB0d28gd2Vla3MgYW5kIHR3byBtb250aHMgYWZ0ZXIgdHJlYXRtZW50LCBmb3IgYm90aCB0aGUgU05MIGdyb3VwIGFuZCBhIGNvbnRyb2wgZ3JvdXAuIFR3byBiaW9sb2dpY2FsIHJlcGxpY2F0ZXMgYXJlIHVzZWQgZm9yIGV2ZXJ5IHRyZWF0bWVudCAtIHRpbWUgY29tYmluYXRpb24uIFRoZSByZXNlYXJjaGVycyBhcmUgaW50ZXJlc3RlZCBpbiBlYXJseSBhbmQgbGF0ZSBlZmZlY3RzIGFuZCBpbiBnZW5lcyBmb3Igd2hpY2ggdGhlIGVmZmVjdCBjaGFuZ2VzIG92ZXIgdGltZS4KCmBgYHtyfQpmaWxlPSJodHRwOi8vYm93dGllLWJpby5zb3VyY2Vmb3JnZS5uZXQvcmVjb3VudC9FeHByZXNzaW9uU2V0cy9oYW1tZXJfZXNldC5SRGF0YSIKbG9hZCh1cmwoZmlsZSkpCmhhbW1lci5lc2V0CmBgYAoKYGBge3J9CnBEYXRhKGhhbW1lci5lc2V0KQpgYGBgCgpgYGB7cn0KbGlicmFyeSh0aWR5dmVyc2UpCnBEYXRhKGhhbW1lci5lc2V0KQpoYW1tZXIuZXNldCAlPiUgZXhwcnMgJT4lIGhlYWQKYGBgCgojIERlc2lnbgoKCgpUaGUgcmVzZWFyY2hlcnMgYXJlIGludGVyZXN0ZWQgaW4gYW4gZWZmZWN0IG9mIHRoZSB0cmVhdG1lbnQgYXQgdGhlIGVhcmx5IHRpbWUgcG9pbnQsIHRoZSBsYXRlIHRpbWVwb2ludCBhbmQgdGhlIHRyZWF0bWVudCAkXHRpbWVzJCB0aW1lIGludGVyYWN0aW9uLgoKCmBgYHtyfQpwRGF0YShoYW1tZXIuZXNldCkkdGltZTwtZmFjdG9yKHJlcChjKCIybSIsIjJ3IiksZWFjaD00KSxsZXZlbHMgPSBjKCIydyIsIjJtIikpCmxldmVscyhwRGF0YShoYW1tZXIuZXNldCkkcHJvdG9jb2wpPC1jKCJjIiwic25sIikKYGBgCgpUaGUgcmVhZCBjb3VudCAkeV97aWd9JCBmb3IgZ2VuZSAkZyQgb2YgbW91c2UgJGkkIGFyZSBtb2RlbGxlZCBhcyBmb2xsb3dzOgoKJCQKXGxlZnRcewpcYmVnaW57YXJyYXl9e2xjbH0KeV97aWd9ICZcc2ltJiBOQihcbXVfe2lnfSxccGhpX2cpXFwKRVt5X3tpZ31cdmVydCBcbWF0aGJme3h9X3tpZ31dJj0mXG11X3tpZ31cXApsb2coXG11X3tpZ30pJj0mXGV0YV97aWd9XFwKXGV0YV97aWd9Jj0mXGJldGFfMCArIFxiZXRhX3tzbmx9IHhfe3NubCxpfSArIFxiZXRhX3t0Mm19eF97dDJtLGl9ICsgXGJldGFfXHRleHR7c25sLHQybX0geF97c25sLGl9eF97dDJtLGl9ICsgXGxvZyBOX2lcXApcZW5ke2FycmF5fVxyaWdodC4KJCQKCndpdGggJHhfe3NubCxpfSQgYSBkdW1teSB2YXJpYWJsZSB0aGF0IGlzIDEgaWYgYSBtb3VzZSBoYWQgdGhlIHNwaW5hbCBuZXJ2ZSBsaWdhdGlvbiBhbmQgaXMgMCBvdGhlcndpc2UsICR4X3t0Mm0saX0kIGEgZHVtbXkgdmFyaWFibGUgdGhhdCBlcXVhbHMgMSBpZiB0aGUgbW91c2Ugd2FzIHNhY3JpZmljZWQgYWZ0ZXIgMiBtb250aHMgYW5kIDAgb3RoZXJ3aXNlLCBhbmQsICRcbG9ne059X2kkIGEgbm9ybWFsaXNhdGlvbiBvZmZzZXQgdG8gY29ycmVjdCBmb3Igc2VxdWVuY2luZyBkZXB0aC4gTm90ZSwgdGhhdCAkXGJldGFfe3NubH0kIGlzIHRoZSBtYWluIGVmZmVjdCBmb3Igc3BpbmFsIG5lcnZlIGxpZ2F0aW9uLCBhbmQgY29ycmVzcG9uZHMgdG8gdGhlIGF2ZXJhZ2UgbG9nIGZvbGQgY2hhbmdlIGJldHdlZW4gdHJlYXRlZCBhbmQgY29udHJvbCBtaWNlIGFmdGVyIHR3byB3ZWVrcy4gVGhlIGludGVyYWN0aW9uICAkXGJldGFfXHRleHR7c25sLHQybX0kIGNhbiBiZSBpbnRlcnByZXRlZCBhcyB0aGUgYXZlcmFnZSBjaGFuZ2UgaW4gbG9nIEZDIGJldHdlZW4gdHJlYXRlZCBhbmQgY29udHJvbCBtb3VzZSBhdCB0aGUgbGF0ZSBhbmQgZWFybHkgdGltZXBvaW50LiBUaGUgcmVzZWFyY2hlcnMgYXJlIGFsc28gaW50ZXJlc3RlZCBpbiBhIGFzc2Vzc2luZyB0aGlyZCBjb250cmFzdDogdGhlIGVmZmVjdCBvZiB0aGUgdHJlYXRtZW50IGF0IHRoZSBsYXRlIHRpbWUgcG9pbnQuCgokJCBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIG1vbnRoc31fXHRleHR7c25sLWN9PSBcYmV0YV97c25sfStcYmV0YV97c25sLHQybX0kJAoKClRoZSBkZXNpZ24gbWF0cml4IGlzIGNvbnN0cnVjdGVkIGZvciB0aGUgbGluZWFyIHByZWRpY3Rvci4KCgpgYGB7cn0KZGdlIDwtIERHRUxpc3QoY291bnRzPWV4cHJzKGhhbW1lci5lc2V0KSkKZGdlJHNhbXBsZQpgYGAKCmBgYHtyfQpkZXNpZ24gPC0gbW9kZWwubWF0cml4KH50aW1lKnByb3RvY29sLHBEYXRhKGhhbW1lci5lc2V0KSkKcm93bmFtZXMoZGVzaWduKSA9IGNvbG5hbWVzKGRnZSkKZGVzaWduCmBgYAoKRmlsdGVyaW5nCmBgYHtyfQprZWVwIDwtIGZpbHRlckJ5RXhwcihkZ2UsIGRlc2lnbikKIGRnZSA8LSBkZ2Vba2VlcCwgLCBrZWVwLmxpYi5zaXplcz1GQUxTRV0KYGBgCgpgYGB7cn0KZGdlIDwtIGNhbGNOb3JtRmFjdG9ycyhkZ2UpCmRnZSRzYW1wbGVzCmBgYAoKIyBEYXRhIGV4cGxvcmF0aW9uCgpBbiBNRFMgcGxvdCBzaG93cyB0aGUgbGVhZGluZyBsb2cgZm9sZCBjaGFuZ2VzIGJldHdlZW4gdGhlIDggc2FtcGxlcy4gVGhlcmUgaXMgYSBsYXJnZSBlZmZlY3QgYWNjb3JkaW5nIHRvIHRoZSBTTkwuIEFyZSB0aGVyZSBpc3N1ZXMgd2l0aCB0aGUgZGVzaWduPwoKYGBge3J9CnBsb3RNRFMoZGdlLGxhYmVscz1wYXN0ZShoYW1tZXIuZXNldCRwcm90b2NvbCxoYW1tZXIuZXNldCR0aW1lLHNlcD0iLSIpLGNvbD1hcy5kb3VibGUoaGFtbWVyLmVzZXQkcHJvdG9jb2wpKQpgYGAKCiMgQW5hbHlzaXMKCgpgYGB7cn0KZGdlIDwtIGVzdGltYXRlRGlzcChkZ2UsIGRlc2lnbiwgcm9idXN0PVRSVUUpICAKcGxvdEJDVihkZ2UpCmZpdCA8LSBnbG1GaXQoZGdlLGRlc2lnbikKYGBgCgoKIyMgRWFybHkKCkRFIGF0IHRoZSBlYXJseSB0aW1lcG9pbnQgY2FuIGJlIGFzc2Vzc2VkIGJ5IHRlc3RpbmcgdGhlIG51bGwgaHlwb3RoZXNpcwoKJCQKSF8wOiBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIHdlZWtzfV9cdGV4dHtzbmwtY309MCBccmlnaHRhcnJvdyBcYmV0YV9cdGV4dHtzbmx9PTAKJCQKCmFnYWluc3QgdGhlIHR3byBzaWRlIGFsdGVybmF0aXZlIGh5cG90aGVzaXMKCiQkCkhfMTogXGxvZyBcdGV4dHtGQ31eXHRleHR7MiB3ZWVrc31fXHRleHR7c25sLWN9XG5lcSAwIFxyaWdodGFycm93IFxiZXRhX1x0ZXh0e3NubH1cbmVxMAokJAoKYGBge3J9CmVhcmx5IDwtIGdsbUxSVChmaXQsY29lZj0icHJvdG9jb2xzbmwiKQp0dEVhcmx5PC10b3BUYWdzKGVhcmx5LCBuID0gbnJvdyhkZ2UpKSAjIGFsbCBnZW5lcwpoaXN0KHR0RWFybHkkdGFibGUkUFZhbHVlLG1haW49ImVhcmx5Iix4bGFiPSJwLXZhbHVlcyIpCnN1bW1hcnkoZHRFYXJseSA8LSBkZWNpZGVUZXN0c0RHRShlYXJseSkpCnZvbGNhbm88LSBnZ3Bsb3QodHRFYXJseSR0YWJsZSxhZXMoeD1sb2dGQyx5PS1sb2cxMChQVmFsdWUpLGNvbG9yPUZEUjwwLjA1KSkgKyBnZW9tX3BvaW50KCkgKyBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzPWMoImJsYWNrIiwicmVkIikpCnZvbGNhbm8KcGxvdFNtZWFyKGVhcmx5LGRlLnRhZ3M9cm93bmFtZXMoZGdlKVthcy5sb2dpY2FsKGR0RWFybHkpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCmBgYAoKQmVjYXVzZSB0aGVyZSBhcmUgYHIgc3VtKGFzLmxvZ2ljYWwoZHRFYXJseSkpYCBzaWduaWZpY2FudCBnZW5lcyB3ZSByZXN0cmljdCB0aGUgaGVhdG1hcCB0byB0aGUgdG9wIDMwIGdlbmVzLgoKYGBge3J9CnBoZWF0bWFwKGNwbShkZ2UsbG9nPVRSVUUpW3Jvd25hbWVzKHR0RWFybHkkdGFibGUpWzE6MzBdLF0sbGFiZWxzX2NvbCA9IHBhc3RlKGhhbW1lci5lc2V0JHByb3RvY29sLGhhbW1lci5lc2V0JHRpbWUsc2VwPSItIikpCmBgYAoKCiMjIExhdGUKClRoZSBlZmZlY3Qgb2YgdGhlIHRyZWF0bWVudCBhZnRlciB0d28gbW9udGhzIGNhbiBiZSBlc3RpbWF0ZWQgYnkgdGhlIGxvZyBmb2xkIGNoYW5nZSBjb3JyZXNwb25kaW5nIHRvIHRoZSBzdW0gb2YgdGhlIG1haW4gZWZmZWN0IGFuZCB0aGUgaW50ZXJhY3Rpb24uClRoaXMgY2FuIGJlIGFzc2Vzc2VkIGJ5IHRlc3RpbmcgdGhlIG51bGwgaHlwb3RoZXNpcwoKJCQKSF8wOiBcbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIG1vbnRoc31fXHRleHR7c25sLWN9PTAgXHJpZ2h0YXJyb3cgXGJldGFfXHRleHR7c25sfStcYmV0YV9cdGV4dHtzbmwsdDJtfT0wCiQkCgphZ2FpbnN0IHRoZSB0d28gc2lkZSBhbHRlcm5hdGl2ZSBoeXBvdGhlc2lzCgokJApIXzE6IFxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgbW9udGhzfV9cdGV4dHtzbmwtY31cbmVxMCBccmlnaHRhcnJvdyBcYmV0YV9cdGV4dHtzbmx9K1xiZXRhX1x0ZXh0e3NubCx0Mm19XG5lcTAKJCQKCgpgYGB7cn0KTDwtYXJyYXkoMCxuY29sKGRlc2lnbikpCm5hbWVzKEwpPC1jb2xuYW1lcyhkZXNpZ24pCkxbYygzLDQpXSA8LSAxCkwKbGF0ZTwtZ2xtTFJUKGZpdCxjb250cmFzdD1MKQp0dExhdGU8LXRvcFRhZ3MobGF0ZSwgbiA9IG5yb3coZGdlKSkgIyBhbGwgZ2VuZXMKaGlzdCh0dExhdGUkdGFibGUkUFZhbHVlLG1haW49ImxhdGUiLHhsYWI9InAtdmFsdWVzIikKc3VtbWFyeShkdExhdGUgPC0gZGVjaWRlVGVzdHNER0UobGF0ZSkpCnZvbGNhbm88LSBnZ3Bsb3QodHRMYXRlJHRhYmxlLGFlcyh4PWxvZ0ZDLHk9LWxvZzEwKFBWYWx1ZSksY29sb3I9RkRSPDAuMDUpKSArIGdlb21fcG9pbnQoKSArIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXM9YygiYmxhY2siLCJyZWQiKSkKdm9sY2FubwpwbG90U21lYXIobGF0ZSxkZS50YWdzPXJvd25hbWVzKGRnZSlbYXMubG9naWNhbChkdExhdGUpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCmBgYAoKQmVjYXVzZSB0aGVyZSBhcmUgYHIgc3VtKGFzLmxvZ2ljYWwoZHRMYXRlKSlgIHNpZ25pZmljYW50IGdlbmVzIHdlIHJlc3RyaWN0IHRoZSBoZWF0bWFwIHRvIHRoZSB0b3AgMzAgZ2VuZXMuCgpgYGB7cn0KcGhlYXRtYXAoY3BtKGRnZSxsb2c9VFJVRSlbcm93bmFtZXModHRMYXRlJHRhYmxlKVsxOjMwXSxdLGxhYmVsc19jb2wgPSBwYXN0ZShoYW1tZXIuZXNldCRwcm90b2NvbCxoYW1tZXIuZXNldCR0aW1lLHNlcD0iLSIpKQpgYGAKCiMjIEludGVyYWN0aW9uCgpUbyBhc3Nlc3MgaWYgdGhlIHRyZWF0bWVudCBlZmZlY3QgY2hhbmdlcyBvdmVyIHRpbWUsIHdlIHdpbGwgdGVzdCB0aGUgbnVsbCBoeXBvdGhlc2lzIHRoYXQgdGhlIGludGVyYWN0aW9uIHRlcm0gZXF1YWxzIHplcm8gdnMgdGhlIGFsdGVybmF0aXZlIHRoYXQgdGhlIGludGVyYWN0aW9uIHRlcm0gaXMgZGlmZmVyZW50IGZyb20gemVyby4gIAoKVGhpcyBjYW4gYmUgYXNzZXNzZWQgYnkgdGVzdGluZyB0aGUgbnVsbCBoeXBvdGhlc2lzCgokJApIXzA6IFxsb2cgXHRleHR7RkN9Xlx0ZXh0ezIgbW9udGhzfV9cdGV4dHtzbmwtY30tXGxvZyBcdGV4dHtGQ31eXHRleHR7MiB3ZWVrc31fXHRleHR7c25sLWN9PTAgXHJpZ2h0YXJyb3cgXGJldGFfXHRleHR7c25sLHQybX09MAokJAoKYWdhaW5zdCB0aGUgYWx0ZXJuYXRpdmUgaHlwdG90aGVzaXMKCiQkCkhfMTogXGxvZyBcdGV4dHtGQ31eXHRleHR7MiBtb250aHN9X1x0ZXh0e3NubC1jfS1cbG9nIFx0ZXh0e0ZDfV5cdGV4dHsyIHdlZWtzfV9cdGV4dHtzbmwtY31cbmVxMCBccmlnaHRhcnJvdyBcYmV0YV9cdGV4dHtzbmwsdDJtfVxuZXEwCiQkCgpgYGB7cn0KaW50ZXIgPC0gZ2xtTFJUKGZpdCxjb2VmPSJ0aW1lMm06cHJvdG9jb2xzbmwiKQp0dEludGVyPC10b3BUYWdzKGludGVyLCBuID0gbnJvdyhkZ2UpKSAjIGFsbCBnZW5lcwpoaXN0KHR0SW50ZXIkdGFibGUkUFZhbHVlLG1haW49ImludGVyYWN0aW9uIix4bGFiPSJwLXZhbHVlcyIpCnN1bW1hcnkoZHRJbnRlciA8LSBkZWNpZGVUZXN0c0RHRShpbnRlcikpCnZvbGNhbm88LSBnZ3Bsb3QodHRJbnRlciR0YWJsZSxhZXMoeD1sb2dGQyx5PS1sb2cxMChQVmFsdWUpLGNvbG9yPUZEUjwwLjA1KSkgKyBnZW9tX3BvaW50KCkgKyBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzPWMoImJsYWNrIiwicmVkIikpCnZvbGNhbm8KcGxvdFNtZWFyKGludGVyLGRlLnRhZ3M9cm93bmFtZXMoZGdlKVthcy5sb2dpY2FsKGR0SW50ZXIpXSx5bGFiPSJsb2cgRkNfbGF0ZSAtIGxvZyBGQ19lYXJseSIpCmBgYAoKQmVjYXVzZSB0aGVyZSBhcmUgYHIgc3VtKGFzLmxvZ2ljYWwoZHRJbnRlcikpYCBzaWduaWZpY2FudCBnZW5lcyB3ZSBwbG90IHRoZW0gYWxsIGluIHRoZSBoZWF0bWFwLgoKYGBge3J9CnBoZWF0bWFwKGNwbShkZ2UsbG9nPVRSVUUpW2FzLmxvZ2ljYWwoZHRJbnRlciksXSxsYWJlbHNfY29sID0gcGFzdGUoaGFtbWVyLmVzZXQkcHJvdG9jb2wsaGFtbWVyLmVzZXQkdGltZSxzZXA9Ii0iKSkKYGBgCgojIyBSZW1hcmtzCgotIFRoZXJlIGFyZSB2ZXJ5IG1hbnkgREUgZ2VuZXMgYWNjb3JkaW5nIHRvIHRoZSBTTkwgdHJlYXRtZW50IGF0IHRoZSBlYXJseSBhbmQgbGF0ZSB0aW1lcG9pbnQuCgotIElzc3VlcyB3aXRoIHRoZSBkZXNpZ24/CgotIFRoZXJlIGFyZSB2ZXJ5IGZldyBpbnRlcmFjdGlvbnMgc2lnbmlmaWNhbnQuIENhbiB5b3UgZXhwbGFpbiB0aGlzPwoKIyBBc3Nlc3MgcC12YWx1ZXMKCmBgYHtyfQp0dEVhcmx5JHRhYmxlIDwtIHR0RWFybHkkdGFibGUgJT4lCiAgbXV0YXRlKHogPSBzaWduKGxvZ0ZDKSAqIGFicyhxbm9ybShQVmFsdWUvMikpKQoKdHRFYXJseSR0YWJsZSAlPiUKICBnZ3Bsb3QoYWVzKHg9eikpICsKICBnZW9tX2hpc3RvZ3JhbShhZXMoeSA9IC4uZGVuc2l0eS4uKSwgY29sb3IgPSAiYmxhY2siKSArCiAgc3RhdF9mdW5jdGlvbihmdW4gPSBkbm9ybSwKICAgICAgYXJncyA9IGxpc3QoCiAgICBtZWFuID0gMCwKICAgIHNkPTEpCiAgKQpgYGAK