Processing math: 100%
  • Outline
  • 1 Francisella tularensis experiment
    • 1.1 Import the data in R
    • 1.2 Preprocessing
    • 1.3 Summarized data structure
      • 1.3.1 Design
      • 1.3.2 Summarized intensity matrix
      • 1.3.3 Hypothesis testing: a single protein
    • 1.4 Multiple hypothesis testing
      • 1.4.1 Multiple testing
    • 1.5 Moderated Statistics
      • 1.5.1 Empirical Bayes
      • 1.5.2 Shrinkage of the variance and moderated t-statistics
    • 1.6 Plots
  • 2 Experimental Design
    • 2.1 Sample size
    • 2.2 Blocking
    • 2.3 Nature methods: Points of significance - Blocking
    • 2.4 Mouse example
      • 2.4.1 Assess the impact of blocking in the tutorial session!
  • 3 Software & code
  • References

Creative Commons License

This is part of the online course Proteomics Data Analysis 2021 (PDA21)

Outline

  • Francisella tularensis Example
  • Hypothesis testing
  • Multiple testing
  • Moderated statistics
  • Experimental design

Note, that the R-code is included for learners who are aiming to develop R/markdown scripts to automate their quantitative proteomics data analyses. According to the target audience of the course we either work with a graphical user interface (GUI) in a R/shiny App msqrob2gui (e.g. Proteomics Bioinformatics course of the EBI and the Proteomics Data Analysis course at the Gulbenkian institute) or with R/markdowns scripts (e.g. Bioinformatics Summer School at UCLouvain or the Statistical Genomics Course at Ghent University).

1 Francisella tularensis experiment

  • Pathogen: causes tularemia
  • Metabolic adaptation key for intracellular life cycle of pathogenic microorganisms.
  • Upon entry into host cells quick phasomal escape and active multiplication in cytosolic compartment.
  • Franciscella is auxotroph for several amino acids, including arginine.
  • Inactivation of arginine transporter delayed bacterial phagosomal escape and intracellular multiplication.
  • Experiment to assess difference in proteome using 3 WT vs 3 ArgP KO mutants

1.1 Import the data in R

Click to see code

  1. Load libraries
library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)
library(ggplot2)
  1. We use a peptides.txt file from MS-data quantified with maxquant that contains MS1 intensities summarized at the peptide level.
peptidesFile <- "https://raw.githubusercontent.com/statOmics/PDA21/data/quantification/francisella/peptides.txt"
  1. Maxquant stores the intensity data for the different samples in columnns that start with Intensity. We can retreive the column names with the intensity data with the code below:
ecols <- grep("Intensity\\.", names(read.delim(peptidesFile)))
  1. Read the data and store it in QFeatures object
pe <- readQFeatures(
  table = peptidesFile,
  fnames = 1,
  ecol = ecols,
  name = "peptideRaw", sep="\t")
  1. Update data with information on design
colData(pe)$genotype <- pe[[1]] %>% 
  colnames %>% 
  substr(12,13) %>%
  as.factor %>% 
  relevel("WT")
pe %>% colData
## DataFrame with 6 rows and 1 column
##                          genotype
##                          <factor>
## Intensity.1WT_20_2h_n3_3       WT
## Intensity.1WT_20_2h_n4_3       WT
## Intensity.1WT_20_2h_n5_3       WT
## Intensity.3D8_20_2h_n3_3       D8
## Intensity.3D8_20_2h_n4_3       D8
## Intensity.3D8_20_2h_n5_3       D8

1.2 Preprocessing

Click to see code to log-transfrom the data

  1. Log transform
  • Calculate number of non zero intensities for each peptide
rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)
  • Peptides with zero intensities are missing peptides and should be represent with a NA value rather than 0.
pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA
  • Logtransform data with base 2
pe <- logTransform(pe, base = 2, i = "peptideRaw", name = "peptideLog")
  1. Filtering
  • Handling overlapping protein groups
pe <- filterFeatures(pe, ~ Proteins %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins))
  • Remove reverse sequences (decoys) and contaminants. Note that this is indicated by the column names Reverse and depending on the version of maxQuant with Potential.contaminants or Contaminants.
pe <- filterFeatures(pe,~Reverse != "+")
pe <- filterFeatures(pe,~ Contaminant != "+")
  • Drop peptides that were only identified in one sample
pe <- filterFeatures(pe,~ nNonZero >=2)
nrow(pe[["peptideLog"]])
## [1] 6525

We keep 6525 peptides upon filtering.

  1. Normalization by median centering
pe <- normalize(pe, 
                i = "peptideLog", 
                name = "peptideNorm", 
                method = "center.median")
  1. Summarization. We use the standard sumarisation in aggregateFeatures, which is a robust summarisation method.
pe <- aggregateFeatures(pe,
    i = "peptideNorm", 
    fcol = "Proteins", 
    na.rm = TRUE,
    name = "protein")
## Your quantitative and row data contain missing values. Please read the
## relevant section(s) in the aggregateFeatures manual page regarding the
## effects of missing values on data aggregation.

Plot of preprocessed data

pe[["peptideNorm"]] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(genotype = colData(pe)[sample,"genotype"]) %>%
  ggplot(aes(x = intensity,group = sample,color = genotype)) + 
    geom_density() +
    ggtitle("Peptide-level")
## Warning: Removed 7561 rows containing non-finite values (stat_density).

pe[["protein"]] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(genotype = colData(pe)[sample,"genotype"]) %>%
  ggplot(aes(x = intensity,group = sample,color = genotype)) + 
    geom_density() +
    ggtitle("Protein-level")
## Warning: Removed 428 rows containing non-finite values (stat_density).

1.3 Summarized data structure

1.3.1 Design

pe %>% 
  colData %>% 
  knitr::kable()
genotype
Intensity.1WT_20_2h_n3_3 WT
Intensity.1WT_20_2h_n4_3 WT
Intensity.1WT_20_2h_n5_3 WT
Intensity.3D8_20_2h_n3_3 D8
Intensity.3D8_20_2h_n4_3 D8
Intensity.3D8_20_2h_n5_3 D8
  • WT vs KO
  • 3 vs 3 repeats

1.3.2 Summarized intensity matrix

pe[["protein"]] %>% assay() %>% head() %>% knitr::kable()
Intensity.1WT_20_2h_n3_3 Intensity.1WT_20_2h_n4_3 Intensity.1WT_20_2h_n5_3 Intensity.3D8_20_2h_n3_3 Intensity.3D8_20_2h_n4_3 Intensity.3D8_20_2h_n5_3
WP_003013731 -0.2748775 -0.0856247 0.1595370 -0.2809009 0.0035526 0.0567110
WP_003013860 NA NA -0.2512039 NA NA -0.4865646
WP_003013909 -0.6851118 -0.8161658 -0.7557906 -0.4591476 -0.5449424 -0.4962482
WP_003014068 0.6495386 0.8522239 1.1344852 0.5459176 0.9187714 0.5974741
WP_003014122 -0.7630863 -1.0430741 -0.8091715 -1.1743951 -1.1924725 -1.2565893
WP_003014123 -0.2051672 -0.3361704 -0.2151930 -0.3855747 -0.2802011 -0.5801771
  • 1115 proteins

1.3.3 Hypothesis testing: a single protein

1.3.3.1 T-test

log2FC=ˉyp1−ˉyp2

Tg=log2FCselog2FC

Tg=^signal^Noise

If we can assume equal variance in both treatment groups:

selog2FC=SD√1n1+1n2

WP_003023392 <- data.frame(
    intensity = assay(pe[["protein"]]["WP_003023392",]) %>% c(), 
    genotype = colData(pe)[,1]) 

WP_003023392 %>% 
  ggplot(aes(x=genotype,y=intensity)) + 
  geom_point() +
  ggtitle("Protein WP_003023392")

t=log2^FCselog2^FC=−1.430.0577=−24.7

  • Is t = -24.7 indicating that there is an effect?

  • How likely is it to observe t = -24.7 when there is no effect of the argP KO on the protein expression?

1.3.3.2 Null hypothesis (H0) and alternative hypothesis (H1)

  • With data we can never prove a hypothesis (falsification principle of Popper)

  • With data we can only reject a hypothesis

  • In general we start from alternative hypothese H1: we want to show an effect of the KO on a protein

H1: On average the protein abundance in WT is different from that in KO
  • But, we will assess this by falsifying the opposite:
    H0: On average the protein abundance in WT is equal to that in KO<- 
t.test(intensity ~ genotype, data = WP_003023392, var.equal=TRUE)
## 
##  Two Sample t-test
## 
## data:  intensity by genotype
## t = 24.747, df = 4, p-value = 1.582e-05
## alternative hypothesis: true difference in means between group WT and group D8 is not equal to 0
## 95 percent confidence interval:
##  1.267666 1.588058
## sample estimates:
## mean in group WT mean in group D8 
##       -0.1821147       -1.6099769

  • How likely is it to observe an equal or more extreme effect than the one observed in the sample when the null hypothesis is true?

  • When we make assumptions about the distribution of our test statistic we can quantify this probability: p-value. The p-value will only be calculated correctly if the underlying assumptions hold!

  • When we repeat the experiment, the probability to observe a fold change for this gene that is more extreme than a 2.69 fold (log2FC=−1.43) down or up regulation by random change (if H0 is true) is 16 out of 1 000 000.

  • If the p-value is below a significance threshold α we reject the null hypothesis. We control the probability on a false positive result at the α-level (type I error)

  • Note, that the p-values are uniform under the null hypothesis, i.e. when H0 is true all p-values are equally likely.

1.4 Multiple hypothesis testing

  • Consider testing DA for all m=1066 proteins simultaneously

  • What if we assess each individual test at level α? → Probability to have a false positive (FP) among all m simultatenous test >>>α=0.05

  • Indeed for each non DA protein we have a probability of 5% to return a FP.

  • In a typical experiment the majority of the proteins are non DA.

  • So an upperbound of the expected FP is m×α or 1066×0.05=53.

→ Hence, we are bound to call many false positive proteins each time we run the experiment.

1.4.1 Multiple testing

1.4.1.1 Family-wise error rate

The family-wise error rate (FWER) addresses the multiple testing issue by no longer controlling the individual type I error for each protein, instead it controls:

FWER=P[FP≥1].

The Bonferroni method is widely used to control the type I error:

  • assess each test at αadj=αm
  • or use adjusted p-values and compare them to α: padj=min(p×m,1)

Problem, the method is very conservative!

1.4.1.2 False discovery rate

  • FDR: Expected proportion of false positives on the total number of positives you return.
  • An FDR of 1% means that on average we expect 1% false positive proteins in the list of proteins that are called significant.
  • Defined by Benjamini and Hochberg in their seminal paper Benjamini, Y. and Hochberg, Y. (1995). “Controlling the false discovery rate: a practical and powerful approach to multiple testing”. Journal of the Royal Statistical Society Series B, 57 (1): 289–300.

The False Discovery Proportion (FDP) is the fraction of false positives that are returned, i.e. 

FDP=FPR

  • However, this quantity cannot be observed because in practice we only know the number of proteins for which we rejected H0, R.
  • But, we do not know the number of false positives, FP.

Therefore, Benjamini and Hochberg, 1995, defined The False Discovery Rate (FDR) as FDR=E[FPR]=E[FDP] the expected FDP.

  • Controlling the FDR allows for more discoveries (i.e. longer lists with significant results), while the fraction of false discoveries among the significant results in well controlled on average. As a consequence, more of the true positive hypotheses will be detected.

1.4.1.3 Intuition of BH-FDR procedure

Consider m=1000 tests

  • Suppose that a researcher rejects all null hypotheses for which p<0.01.

  • If we use p<0.01, we expect 0.01×m0 tests to return false positives.

  • A conservative estimate of the number of false positives that we can expect can be obtained by considering that the null hypotheses are true for all features, m0=m=1000.

  • We then would expect 0.01×1000=10 false positives (FP=10).

  • Suppose that the researcher found 200 genes with p<0.01 (R=200).

  • The proportion of false positive results (FDP = false positive proportion) among the list of R=200 genes can then be estimated as ^FDP=FPR=10200=0.01×1000200=0.05.

1.4.1.4 Benjamini and Hochberg (1995) procedure for controlling the FDR at α

  1. Let p(1)≤…≤p(m) denote the ordered p-values.

  2. Find the largest integer k so that p(k)×mk≤α or p(k)≤k×α/m

  3. If such a k exists, reject the k null hypotheses associated with p(1),…,p(k). Otherwise none of the null hypotheses is rejected.

The adjusted p-value (also known as the q-value in FDR literature): q(i)=˜p(i)=min[minj=i,…,m(mp(j)/j),1]. In the hypothetical example above: k=200, p(k)=0.01, m=1000 and α=0.05.

1.4.1.5 Francisella Example

Click to see code 

ttestMx <- function(y,group) {
    test <- try(t.test(y[group],y[!group],var.equal=TRUE),silent=TRUE)
    if(is(test,"try-error")) {
      return(c(log2FC=NA,se=NA,tstat=NA,p=NA))
      } else {
      return(c(log2FC= (test$estimate%*%c(1,-1)),se=test$stderr,tstat=test$statistic,pval=test$p.value))
      }
 }
 
 res <- apply(
    assay(pe[["protein"]]), 
    1, 
    ttestMx,
    group = colData(pe)$genotype=="D8") %>% 
  t 
 colnames(res) <- c("logFC","se","tstat","pval")
 res <- res %>% as.data.frame %>% na.exclude %>% arrange(pval)
 res$adjPval <- p.adjust(res$pval, "fdr")
 alpha <- 0.05
res$adjAlphaForm <- paste0(1:nrow(res)," x ",alpha,"/",nrow(res))
res$adjAlpha <- alpha * (1:nrow(res))/nrow(res) 
res$"pval < adjAlpha" <- res$pval < res$adjAlpha 
res$"adjPval < alpha" <- res$adjPval < alpha

FWER: Bonferroni method:αadj=α/m=0.05/1066=5×10−5

logFC pval adjPval adjAlphaForm adjAlpha pval < adjAlpha adjPval < alpha
WP_003038940 -0.2876290 0.0000146 0.0084347 1 x 0.05/1066 0.0000469 TRUE TRUE
WP_003023392 -1.4278622 0.0000158 0.0084347 2 x 0.05/1066 0.0000938 TRUE TRUE
WP_003039212 -0.2658247 0.0000820 0.0291520 3 x 0.05/1066 0.0001407 TRUE TRUE
WP_003026016 -1.0800305 0.0001395 0.0346124 4 x 0.05/1066 0.0001876 TRUE TRUE
WP_003039615 -0.3992190 0.0001623 0.0346124 5 x 0.05/1066 0.0002345 TRUE TRUE
WP_011733588 -0.4323262 0.0002291 0.0407034 6 x 0.05/1066 0.0002814 TRUE TRUE
WP_003014552 -0.9843865 0.0003224 0.0440266 7 x 0.05/1066 0.0003283 TRUE TRUE
WP_003040849 -1.2780743 0.0003304 0.0440266 8 x 0.05/1066 0.0003752 TRUE TRUE
WP_003038430 -0.4331987 0.0004505 0.0489078 9 x 0.05/1066 0.0004221 FALSE TRUE
WP_003033975 -0.2949061 0.0005047 0.0489078 10 x 0.05/1066 0.0004690 FALSE TRUE
WP_011733645 0.3531405 0.0005171 0.0489078 11 x 0.05/1066 0.0005159 FALSE TRUE
WP_011733723 -0.3935768 0.0005506 0.0489078 12 x 0.05/1066 0.0005629 TRUE TRUE
WP_003038679 -0.3909725 0.0007083 0.0580821 13 x 0.05/1066 0.0006098 FALSE FALSE
WP_003033719 -1.1865453 0.0008426 0.0603810 14 x 0.05/1066 0.0006567 FALSE FALSE
… … … … … … … …
WP_003040562 0.0039480 0.9976429 0.9985797 1065 x 0.05/1066 0.0499531 FALSE FALSE
WP_003041130 0.0002941 0.9992812 0.9992812 1066 x 0.05/1066 0.05 FALSE FALSE

1.4.1.6 Results

Click to see code 

volcanoT <- res %>% 
  ggplot(aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
    geom_point(cex = 2.5) +
    scale_color_manual(values = alpha(c("black", "red"), 0.5)) +
    theme_minimal() 

volcanoT

1.5 Moderated Statistics

Problems with ordinary t-test

Click to see code 

problemPlots <- list() 
problemPlots[[1]] <- res %>% 
  ggplot(aes(x = logFC, y = se, color = adjPval < 0.05)) +
    geom_point(cex = 2.5) +
    scale_color_manual(values = alpha(c("black", "red"), 0.5)) +
    theme_minimal() 

for (i in 2:3)
{
problemPlots[[i]] <- colData(pe) %>% 
    as.data.frame %>% 
    mutate(intensity = pe[["protein"]][rownames(res)[i],] %>% 
             assay %>% 
             c) %>% 
    ggplot(aes(x=genotype,y=intensity)) +
    geom_point() + 
    ylim(-3,0) +
    ggtitle(rownames(res)[i])
}

problemPlots
## [[1]]

## 
## [[2]]

## 
## [[3]]

A general class of moderated test statistics is given by Tmodg=ˉYg1−ˉYg2C˜Sg, where ˜Sg is a moderated standard deviation estimate.

  • C is a constant depending on the design e.g. √1/n1+1/n2 for a t-test and of another form for linear models.
  • ˜Sg=Sg+S0: add small positive constant to denominator of t-statistic.
  • This can be adopted in Perseus.
Click to see code 

simI<-sapply(res$se/sqrt(1/3+1/3),function(n,mean,sd) rnorm(n,mean,sd),n=6,mean=0) %>% t
resSim <- apply(
    simI, 
    1, 
    ttestMx,
    group = colData(pe)$genotype=="D8") %>% 
  t 
 colnames(resSim) <- c("logFC","se","tstat","pval")
 resSim <- as.data.frame(resSim)
 tstatSimPlot <- resSim %>% 
   ggplot(aes(x=tstat)) +
     geom_histogram(aes(y=..density.., fill=..count..),bins=30) +
     stat_function(fun=dt,
    color="red",
    args=list(df=4)) + 
   ylim(0,.6) +
   ggtitle("t-statistic")

 
 resSim$C <- sqrt(1/3+1/3) 
 resSim$sd <- resSim$se/resSim$C 
 tstatSimPerseus <- resSim %>% 
   ggplot(aes(x=logFC/((sd+.1)*C))) +
     geom_histogram(aes(y=..density.., fill=..count..),bins=30) +
     stat_function(fun=dt,
                   color="red",
                  args=list(df=4)) + 
     ylim(0,.6) +
    ggtitle("Perseus")

gridExtra::grid.arrange(tstatSimPlot,tstatSimPerseus,nrow=1)

  • The choice of S0 in Perseus is ad hoc and the t-statistic is no-longer t-distributed.
  • Permutation test, but is difficult for more complex designs.
  • Allows for Data Dredging because user can choose S0

1.5.1 Empirical Bayes

Figure courtesy to Rafael Irizarry

Tmodg=ˉYg1−ˉYg2C˜Sg,

  • empirical Bayes theory provides formal framework for borrowing strength across proteins,
  • Implemented in popular bioconductor package limma and msqrob2

˜Sg=√dgS2g+d0S20dg+d0,

  • S20: common variance (over all proteins)
  • Moderated t-statistic is t-distributed with d0+dg degrees of freedom.
  • Note that the degrees of freedom increase by borrowing strength across proteins!
Click to see the code

  1. We model the protein level expression values using the msqrob function. By default msqrob2 estimates the model parameters using robust regression.

We will model the data with a different group mean for every genotype. The group is incoded in the variable genotype of the colData. We can specify this model by using a formula with the factor genotype as its predictor: formula = ~genotype.

Note, that a formula always starts with a symbol ‘~’.

pe <- msqrob(object = pe, i = "protein", formula = ~genotype)
  1. Inference

We first explore the design of the model that we specified using the the package ExploreModelMatrix

library(ExploreModelMatrix)
VisualizeDesign(colData(pe),~genotype)$plotlist[[1]]

We have two model parameters, the (Intercept) and genotypeD8. This results in a model with two group means:

  1. For the wild type (WT) the expected value (mean) of the log2 transformed intensity y for a protein will be modelled using

E[Y|genotype=WT]=(Intercept)

  1. For the knockout genotype D8 the expected value (mean) of the log2 transformed intensity y for a protein will be modelled using

E[Y|genotype=D8]=(Intercept)+genotypeD8

The average log2FC between D8 and WT is thus log2FCD8−WT=E[Y|genotype=D8]−E[Y|genotype=WT]=genotypeD8

Hence, assessing the null hypothesis that there is no differential abundance between D8 and WT can be reformulated as

H0:genotypeD8=0 We can implement a hypothesis test for each protein in msqrob2 using the code below:

L <- makeContrast("genotypeD8 = 0", parameterNames = c("genotypeD8"))
pe <- hypothesisTest(object = pe, i = "protein", contrast = L)

We can show the list with all significant DE proteins at the 5% FDR using

rowData(pe[["protein"]])$genotypeD8 %>% 
  arrange(pval) %>%
  filter(adjPval<0.05)
ABCDEFGHIJ0123456789
 
 
logFC
<dbl>
se
<dbl>
df
<dbl>
t
<dbl>
pval
<dbl>
adjPval
<dbl>
WP_003023392-1.42786220.095461046.353267-14.9575383.435824e-060.003456439
WP_003040849-1.28577330.122268206.236222-10.5160073.372825e-050.016965307
WP_003014552-0.98438650.107706826.353267-9.1395016.920701e-050.022593404
WP_003033719-1.19700730.134087756.220667-8.9270448.983461e-050.022593404
WP_003040790-1.35313360.161879476.353267-8.3588961.174333e-040.023627585
WP_003026016-1.05844480.112408925.442252-9.4160221.429981e-040.023976009
WP_003033905-0.88153740.121996896.353267-7.2259012.740575e-040.039385984
WP_003039713-0.76350440.109027446.353267-7.0028653.279672e-040.041241874
WP_003039530-0.91843990.137553636.353267-6.6769594.299645e-040.041477282
WP_003014581-0.97890580.127677825.353267-7.6670004.409838e-040.041477282
Next
12
Previous
1-10 of 12 rows

We can also visualise the results using a volcanoplot

volcano <- ggplot(
    rowData(pe[["protein"]])$genotypeD8,
    aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)
) +
    geom_point(cex = 2.5) +
    scale_color_manual(values = alpha(c("black", "red"), 0.5)) +
    theme_minimal() +
    ggtitle("msqrob2")

gridExtra::grid.arrange(
  volcanoT +    
    xlim(-3,3) +
  ggtitle("ordinary t-test"),
  volcano +     
    xlim(-3,3)
,nrow=2)
## Warning: Removed 109 rows containing missing values (geom_point).

  • The volcano plot opens up when using the EB variance estimator

  • Borrowing strength to estimate the variance using empirical Bayes solves the issue of returning proteins with a low fold change as significant due to a low variance.

1.5.2 Shrinkage of the variance and moderated t-statistics

qplot(
  sapply(rowData(pe[["protein"]])$msqrobModels,getSigma),
  sapply(rowData(pe[["protein"]])$msqrobModels,getSigmaPosterior)) +
  xlab("SD") +
  ylab("moderated SD") +
  geom_abline(intercept = 0,slope = 1) +
  geom_hline(yintercept = ) 
## Warning: Removed 109 rows containing missing values (geom_point).

  • Small variances are shrunken towards the common variance resulting in large EB variance estimates
  • Large variances are shrunken towards the common variance resulting in smaller EB variance estimates
  • Pooled degrees of freedom of the EB variance estimator are larger because information is borrowed across proteins to estimate the variance

1.6 Plots

sigNames <- rowData(pe[["protein"]])$genotypeD8 %>%
    rownames_to_column("protein") %>%
    filter(adjPval < 0.05) %>%
    pull(protein)
heatmap(assay(pe[["protein"]])[sigNames, ])

for (protName in sigNames)
    {
        pePlot <- pe[protName, , c("peptideNorm", "protein")]
        pePlotDf <- data.frame(longFormat(pePlot))
        pePlotDf$assay <- factor(pePlotDf$assay,
            levels = c("peptideNorm", "protein")
        )
        pePlotDf$genotype <- as.factor(colData(pePlot)[pePlotDf$colname, "genotype"])

        # plotting
        p1 <- ggplot(
            data = pePlotDf,
            aes(x = colname, y = value, group = rowname)
        ) +
            geom_line() +
            geom_point() +
            facet_grid(~assay) +
            theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
            ggtitle(protName)
        print(p1)

        # plotting 2
        p2 <- ggplot(pePlotDf, aes(x = colname, y = value, fill = genotype)) +
            geom_boxplot(outlier.shape = NA) +
            geom_point(
                position = position_jitter(width = .1),
                aes(shape = rowname)
            ) +
            scale_shape_manual(values = 1:nrow(pePlotDf)) +
            labs(title = protName, x = "sample", y = "peptide intensity (log2)") +
            theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
            facet_grid(~assay)
        print(p2)
}

2 Experimental Design

2.1 Sample size

log2FC=ˉyp1−ˉyp2

Tg=log2FCselog2FC

Tg=^signal^Noise

If we can assume equal variance in both treatment groups:

selog2FC=SD√1n1+1n2

→ if number of bio-repeats increases we have a higher power!

  • cfr. Study of tamoxifen treated Estrogen Recepter (ER) positive breast cancer patients

2.2 Blocking

σ2=σ2bio+σ2lab+σ2extraction+σ2run+…

  • Biological: fluctuations in protein level between mice, fluctations in protein level between cells, …
  • Technical: cage effect, lab effect, week effect, plasma extraction, MS-run, …

2.3 Nature methods: Points of significance - Blocking

https://www.nature.com/articles/nmeth.3005.pdf

2.4 Mouse example

Duguet et al. (2017) MCP 16(8):1416-1432. doi: 10.1074/mcp.m116.062745

  • All treatments of interest are present within block!
  • We can estimate the effect of the treatment within block!
  • We can isolate the between block variability from the analysis using linear model: y∼type+mouse
  • Not possible with Perseus!

2.4.1 Assess the impact of blocking in the tutorial session!

  • Completely randomized design with only one cell type per mouse (Treg and Tconv)

↕

  • Randomized complete block design assessing Treg and Tconv on each mouse

3 Software & code

  • Our R/Bioconductor package msqrob2 can be used in R markdown scripts or with a GUI/shinyApp in the msqrob2gui package.

  • The GUI is intended as a introduction to the key concepts of proteomics data analysis for users who have no experience in R.

  • However, learning how to code data analyses in R markdown scripts is key for open en reproducible science and for reporting your proteomics data analyses and interpretation in a reproducible way.

  • More information on our tools can be found in our papers (Goeminne, Gevaert, and Clement 2016), (Goeminne et al. 2020) and (Sticker et al. 2020). Please refer to our work when using our tools.

  • Clips on the code on importing the data and preprocessing can be found in Part I Preprocessing

  • A clip on the code for modelling and statistical inference with msqrob2 is included below

References

Goeminne, L. J. E., A. Sticker, L. Martens, K. Gevaert, and L. Clement. 2020. “MSqRob Takes the Missing Hurdle: Uniting Intensity- and Count-Based Proteomics.” Anal Chem 92 (9): 6278–87.

Goeminne, L. J., K. Gevaert, and L. Clement. 2016. “Peptide-level Robust Ridge Regression Improves Estimation, Sensitivity, and Specificity in Data-dependent Quantitative Label-free Shotgun Proteomics.” Mol Cell Proteomics 15 (2): 657–68.

Sticker, A., L. Goeminne, L. Martens, and L. Clement. 2020. “Robust Summarization and Inference in Proteome-wide Label-free Quantification.” Mol Cell Proteomics 19 (7): 1209–19.

LS0tCnRpdGxlOiAiU3RhdGlzdGljYWwgTWV0aG9kcyBmb3IgUXVhbnRpdGF0aXZlIE1TLWJhc2VkIFByb3Rlb21pY3M6IFBhcnQgSUkuIERpZmZlcmVudGlhbCBBYnVuZGFuY2UgQW5hbHlzaXMiCmF1dGhvcjogIkxpZXZlbiBDbGVtZW50IgpkYXRlOiAiW3N0YXRPbWljc10oaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSwgR2hlbnQgVW5pdmVyc2l0eSIKb3V0cHV0OgogICAgaHRtbF9kb2N1bWVudDoKICAgICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgICB0aGVtZTogY29zbW8KICAgICAgdG9jOiB0cnVlCiAgICAgIHRvY19mbG9hdDogdHJ1ZQogICAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgcGRmX2RvY3VtZW50OgogICAgICB0b2M6IHRydWUKICAgICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCmxpbmtjb2xvcjogYmx1ZQp1cmxjb2xvcjogYmx1ZQpjaXRlY29sb3I6IGJsdWUKCmJpYmxpb2dyYXBoeTogbXNxcm9iMi5iaWIKCi0tLQoKPGEgcmVsPSJsaWNlbnNlIiBocmVmPSJodHRwczovL2NyZWF0aXZlY29tbW9ucy5vcmcvbGljZW5zZXMvYnktbmMtc2EvNC4wIj48aW1nIGFsdD0iQ3JlYXRpdmUgQ29tbW9ucyBMaWNlbnNlIiBzdHlsZT0iYm9yZGVyLXdpZHRoOjAiIHNyYz0iaHR0cHM6Ly9pLmNyZWF0aXZlY29tbW9ucy5vcmcvbC9ieS1uYy1zYS80LjAvODh4MzEucG5nIiAvPjwvYT4KClRoaXMgaXMgcGFydCBvZiB0aGUgb25saW5lIGNvdXJzZSBbUHJvdGVvbWljcyBEYXRhIEFuYWx5c2lzIDIwMjEgKFBEQTIxKV0oaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvL1BEQTIxLykKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvMW1oZzNCQ3VFbTgiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgotIFtQbGF5bGlzdCBQREEgUHJlcHJvY2Vzc2luZ10oaHR0cHM6Ly93d3cueW91dHViZS5jb20vd2F0Y2g/dj0xbWhnM0JDdUVtOCZsaXN0PVBMWkgxaFA4X0xiSktxblBTUzRoeFRrbi10UW9sU0NHalApCgojIE91dGxpbmUgey19CgotIEZyYW5jaXNlbGxhIHR1bGFyZW5zaXMgRXhhbXBsZQotIEh5cG90aGVzaXMgdGVzdGluZwotIE11bHRpcGxlIHRlc3RpbmcKLSBNb2RlcmF0ZWQgc3RhdGlzdGljcwotIEV4cGVyaW1lbnRhbCBkZXNpZ24KCgpOb3RlLCB0aGF0IHRoZSBSLWNvZGUgaXMgaW5jbHVkZWQgZm9yIGxlYXJuZXJzIHdobyBhcmUgYWltaW5nIHRvIGRldmVsb3AgUi9tYXJrZG93biBzY3JpcHRzIHRvIGF1dG9tYXRlIHRoZWlyIHF1YW50aXRhdGl2ZSBwcm90ZW9taWNzIGRhdGEgYW5hbHlzZXMuCkFjY29yZGluZyB0byB0aGUgdGFyZ2V0IGF1ZGllbmNlIG9mIHRoZSBjb3Vyc2Ugd2UgZWl0aGVyIHdvcmsgd2l0aCBhIGdyYXBoaWNhbCB1c2VyIGludGVyZmFjZSAoR1VJKSBpbiBhIFIvc2hpbnkgQXBwIG1zcXJvYjJndWkgKGUuZy4gUHJvdGVvbWljcyBCaW9pbmZvcm1hdGljcyBjb3Vyc2Ugb2YgdGhlIEVCSSBhbmQgdGhlIFByb3Rlb21pY3MgRGF0YSBBbmFseXNpcyBjb3Vyc2UgYXQgdGhlIEd1bGJlbmtpYW4gaW5zdGl0dXRlKSBvciB3aXRoIFIvbWFya2Rvd25zIHNjcmlwdHMgKGUuZy4gQmlvaW5mb3JtYXRpY3MgU3VtbWVyIFNjaG9vbCBhdCBVQ0xvdXZhaW4gb3IgdGhlIFN0YXRpc3RpY2FsIEdlbm9taWNzIENvdXJzZSBhdCBHaGVudCBVbml2ZXJzaXR5KS4gCgotLS0KCiMgRnJhbmNpc2VsbGEgdHVsYXJlbnNpcyBleHBlcmltZW50Cgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL1RXNXJrMXk3YU9jIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKYGBge3IgZWNobz1GQUxTRSxvdXQud2lkdGg9IjUwJSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvZnJhbmNpc2VsbGEuanBnIikKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL3R1bGFyZW1pYV9sZXNpb24uanBnIikKYGBgCgotIFBhdGhvZ2VuOiBjYXVzZXMgdHVsYXJlbWlhCi0gTWV0YWJvbGljIGFkYXB0YXRpb24ga2V5IGZvciBpbnRyYWNlbGx1bGFyIGxpZmUgY3ljbGUgb2YgcGF0aG9nZW5pYyBtaWNyb29yZ2FuaXNtcy4gCi0gVXBvbiBlbnRyeSBpbnRvIGhvc3QgY2VsbHMgcXVpY2sgcGhhc29tYWwgZXNjYXBlIGFuZCBhY3RpdmUgbXVsdGlwbGljYXRpb24gaW4gY3l0b3NvbGljIGNvbXBhcnRtZW50LgotIEZyYW5jaXNjZWxsYSBpcyBhdXhvdHJvcGggZm9yIHNldmVyYWwgYW1pbm8gYWNpZHMsIGluY2x1ZGluZyBhcmdpbmluZS4gCi0gSW5hY3RpdmF0aW9uIG9mIGFyZ2luaW5lIHRyYW5zcG9ydGVyIGRlbGF5ZWQgYmFjdGVyaWFsIHBoYWdvc29tYWwgZXNjYXBlIGFuZCBpbnRyYWNlbGx1bGFyIG11bHRpcGxpY2F0aW9uLiAKLSBFeHBlcmltZW50IHRvIGFzc2VzcyBkaWZmZXJlbmNlIGluIHByb3Rlb21lIHVzaW5nIDMgV1QgdnMgMyBBcmdQIEtPIG11dGFudHMKCgojIyBJbXBvcnQgdGhlIGRhdGEgaW4gUiAKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvMkxqZDk1SUVfQVkiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIDwvc3VtbWFyeT48cD4KMS4gTG9hZCBsaWJyYXJpZXMgCgpgYGB7ciwgd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KbGlicmFyeSh0aWR5dmVyc2UpCmxpYnJhcnkobGltbWEpCmxpYnJhcnkoUUZlYXR1cmVzKQpsaWJyYXJ5KG1zcXJvYjIpCmxpYnJhcnkocGxvdGx5KQpsaWJyYXJ5KGdncGxvdDIpCmBgYAoKMi4gV2UgdXNlIGEgcGVwdGlkZXMudHh0IGZpbGUgZnJvbSBNUy1kYXRhIHF1YW50aWZpZWQgd2l0aCBtYXhxdWFudCB0aGF0IApjb250YWlucyBNUzEgaW50ZW5zaXRpZXMgc3VtbWFyaXplZCBhdCB0aGUgcGVwdGlkZSBsZXZlbC4gCmBgYHtyfQpwZXB0aWRlc0ZpbGUgPC0gImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9zdGF0T21pY3MvUERBMjEvZGF0YS9xdWFudGlmaWNhdGlvbi9mcmFuY2lzZWxsYS9wZXB0aWRlcy50eHQiCmBgYAoKMy4gTWF4cXVhbnQgc3RvcmVzIHRoZSBpbnRlbnNpdHkgZGF0YSBmb3IgdGhlIGRpZmZlcmVudCBzYW1wbGVzIGluIGNvbHVtbm5zIHRoYXQgc3RhcnQgd2l0aCBJbnRlbnNpdHkuIFdlIGNhbiByZXRyZWl2ZSB0aGUgY29sdW1uIG5hbWVzIHdpdGggdGhlIGludGVuc2l0eSBkYXRhIHdpdGggdGhlIGNvZGUgYmVsb3c6IAoKYGBge3J9CmVjb2xzIDwtIGdyZXAoIkludGVuc2l0eVxcLiIsIG5hbWVzKHJlYWQuZGVsaW0ocGVwdGlkZXNGaWxlKSkpCmBgYAoKNC4gUmVhZCB0aGUgZGF0YSBhbmQgc3RvcmUgaXQgaW4gIFFGZWF0dXJlcyBvYmplY3QgCgpgYGB7cn0KcGUgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHBlcHRpZGVzRmlsZSwKICBmbmFtZXMgPSAxLAogIGVjb2wgPSBlY29scywKICBuYW1lID0gInBlcHRpZGVSYXciLCBzZXA9Ilx0IikKYGBgCgo1LiBVcGRhdGUgZGF0YSB3aXRoIGluZm9ybWF0aW9uIG9uIGRlc2lnbgoKYGBge3J9CmNvbERhdGEocGUpJGdlbm90eXBlIDwtIHBlW1sxXV0gJT4lIAogIGNvbG5hbWVzICU+JSAKICBzdWJzdHIoMTIsMTMpICU+JQogIGFzLmZhY3RvciAlPiUgCiAgcmVsZXZlbCgiV1QiKQpwZSAlPiUgY29sRGF0YQpgYGAKCjwvcD48L2RldGFpbHM+CgojIyBQcmVwcm9jZXNzaW5nCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbG9nLXRyYW5zZnJvbSB0aGUgZGF0YSA8L3N1bW1hcnk+PHA+CgoxLiBMb2cgdHJhbnNmb3JtCgogIC0gQ2FsY3VsYXRlIG51bWJlciBvZiBub24gemVybyBpbnRlbnNpdGllcyBmb3IgZWFjaCBwZXB0aWRlCmBgYHtyfQpyb3dEYXRhKHBlW1sicGVwdGlkZVJhdyJdXSkkbk5vblplcm8gPC0gcm93U3Vtcyhhc3NheShwZVtbInBlcHRpZGVSYXciXV0pID4gMCkKYGBgCgogIC0gUGVwdGlkZXMgd2l0aCB6ZXJvIGludGVuc2l0aWVzIGFyZSBtaXNzaW5nIHBlcHRpZGVzIGFuZCBzaG91bGQgYmUgcmVwcmVzZW50CndpdGggYSBgTkFgIHZhbHVlIHJhdGhlciB0aGFuIGAwYC4KCmBgYHtyfQpwZSA8LSB6ZXJvSXNOQShwZSwgInBlcHRpZGVSYXciKSAjIGNvbnZlcnQgMCB0byBOQQpgYGAKCiAgLSBMb2d0cmFuc2Zvcm0gZGF0YSB3aXRoIGJhc2UgMgoKYGBge3J9CnBlIDwtIGxvZ1RyYW5zZm9ybShwZSwgYmFzZSA9IDIsIGkgPSAicGVwdGlkZVJhdyIsIG5hbWUgPSAicGVwdGlkZUxvZyIpCmBgYAoKMi4gRmlsdGVyaW5nCgogIC0gSGFuZGxpbmcgb3ZlcmxhcHBpbmcgcHJvdGVpbiBncm91cHMKCmBgYHtyfQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSwgfiBQcm90ZWlucyAlaW4lIHNtYWxsZXN0VW5pcXVlR3JvdXBzKHJvd0RhdGEocGVbWyJwZXB0aWRlTG9nIl1dKSRQcm90ZWlucykpCmBgYAoKICAtIFJlbW92ZSByZXZlcnNlIHNlcXVlbmNlcyAoZGVjb3lzKSBhbmQgY29udGFtaW5hbnRzLiBOb3RlIHRoYXQgdGhpcyBpcyBpbmRpY2F0ZWQgYnkgdGhlIGNvbHVtbiBuYW1lcyBSZXZlcnNlIGFuZCBkZXBlbmRpbmcgb24gdGhlIHZlcnNpb24gb2YgbWF4UXVhbnQgd2l0aCBQb3RlbnRpYWwuY29udGFtaW5hbnRzIG9yIENvbnRhbWluYW50cy4KCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsflJldmVyc2UgIT0gIisiKQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IENvbnRhbWluYW50ICE9ICIrIikKYGBgCgogIC0gRHJvcCBwZXB0aWRlcyB0aGF0IHdlcmUgb25seSBpZGVudGlmaWVkIGluIG9uZSBzYW1wbGUKCmBgYHtyfQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IG5Ob25aZXJvID49MikKbnJvdyhwZVtbInBlcHRpZGVMb2ciXV0pCmBgYAoKV2Uga2VlcCBgciBucm93KHBlW1sicGVwdGlkZUxvZyJdXSlgIHBlcHRpZGVzIHVwb24gZmlsdGVyaW5nLgoKMy4gTm9ybWFsaXphdGlvbiBieSBtZWRpYW4gY2VudGVyaW5nCgpgYGB7cn0KcGUgPC0gbm9ybWFsaXplKHBlLCAKICAgICAgICAgICAgICAgIGkgPSAicGVwdGlkZUxvZyIsIAogICAgICAgICAgICAgICAgbmFtZSA9ICJwZXB0aWRlTm9ybSIsIAogICAgICAgICAgICAgICAgbWV0aG9kID0gImNlbnRlci5tZWRpYW4iKQpgYGAKCgo0LiBTdW1tYXJpemF0aW9uLiBXZSB1c2UgdGhlIHN0YW5kYXJkIHN1bWFyaXNhdGlvbiBpbiBhZ2dyZWdhdGVGZWF0dXJlcywgd2hpY2ggaXMgYQpyb2J1c3Qgc3VtbWFyaXNhdGlvbiBtZXRob2QuCgpgYGB7cix3YXJuaW5nPUZBTFNFfQpwZSA8LSBhZ2dyZWdhdGVGZWF0dXJlcyhwZSwKICAgIGkgPSAicGVwdGlkZU5vcm0iLCAKICAgIGZjb2wgPSAiUHJvdGVpbnMiLCAKICAgIG5hLnJtID0gVFJVRSwKICAgIG5hbWUgPSAicHJvdGVpbiIpCmBgYAoKClBsb3Qgb2YgcHJlcHJvY2Vzc2VkIGRhdGEgCgpgYGB7cn0KcGVbWyJwZXB0aWRlTm9ybSJdXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShnZW5vdHlwZSA9IGNvbERhdGEocGUpW3NhbXBsZSwiZ2Vub3R5cGUiXSkgJT4lCiAgZ2dwbG90KGFlcyh4ID0gaW50ZW5zaXR5LGdyb3VwID0gc2FtcGxlLGNvbG9yID0gZ2Vub3R5cGUpKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgiUGVwdGlkZS1sZXZlbCIpCgpwZVtbInByb3RlaW4iXV0gJT4lIAogIGFzc2F5ICU+JQogIGFzLmRhdGEuZnJhbWUoKSAlPiUKICBnYXRoZXIoc2FtcGxlLCBpbnRlbnNpdHkpICU+JSAKICBtdXRhdGUoZ2Vub3R5cGUgPSBjb2xEYXRhKHBlKVtzYW1wbGUsImdlbm90eXBlIl0pICU+JQogIGdncGxvdChhZXMoeCA9IGludGVuc2l0eSxncm91cCA9IHNhbXBsZSxjb2xvciA9IGdlbm90eXBlKSkgKyAKICAgIGdlb21fZGVuc2l0eSgpICsKICAgIGdndGl0bGUoIlByb3RlaW4tbGV2ZWwiKQpgYGAKPC9wPjwvZGV0YWlscz4KCiMjIFN1bW1hcml6ZWQgZGF0YSBzdHJ1Y3R1cmUKCiMjIyBEZXNpZ24KCmBgYHtyfQpwZSAlPiUgCiAgY29sRGF0YSAlPiUgCiAga25pdHI6OmthYmxlKCkKYGBgCgotIFdUIHZzIEtPIAotIDMgdnMgMyByZXBlYXRzIAoKIyMjIFN1bW1hcml6ZWQgaW50ZW5zaXR5IG1hdHJpeAoKYGBge3J9CnBlW1sicHJvdGVpbiJdXSAlPiUgYXNzYXkoKSAlPiUgaGVhZCgpICU+JSBrbml0cjo6a2FibGUoKQpgYGAKCi0gYHIgbnJvdyhwZVtbInByb3RlaW4iXV0pYCBwcm90ZWlucyAKCiMjIyBIeXBvdGhlc2lzIHRlc3Rpbmc6IGEgc2luZ2xlIHByb3RlaW4gCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL215UDZTVWxTd3NNIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKYGBge3IgZWNobz1GQUxTRX0KaWYgKCJwaSIlaW4lbHMoKSkgcm0oInBpIikKa29wdm9ldGVyPC1mdW5jdGlvbih4LHksYW5nbGU9MCxsPS4yLGNleC5kb3Q9LjUscGNoPTE5LGNvbD0iYmxhY2siKQp7CmFuZ2xlPWFuZ2xlLzE4MCpwaQpwb2ludHMoeCx5LGNleD1jZXguZG90LHBjaD1wY2gsY29sPWNvbCkKbGluZXMoYyh4LHgrbCpjb3MoLXBpLzIrYW5nbGUpKSxjKHkseStsKnNpbigtcGkvMithbmdsZSkpLGNvbD1jb2wpCmxpbmVzKGMoeCtsLzIqY29zKC1waS8yK2FuZ2xlKSx4K2wvMipjb3MoLXBpLzIrYW5nbGUpK2wvNCpjb3MoYW5nbGUpKSxjKHkrbC8yKnNpbigtcGkvMithbmdsZSkseStsLzIqc2luKC1waS8yK2FuZ2xlKStsLzQqc2luKGFuZ2xlKSksY29sPWNvbCkKbGluZXMoYyh4K2wvMipjb3MoLXBpLzIrYW5nbGUpLHgrbC8yKmNvcygtcGkvMithbmdsZSkrbC80KmNvcyhwaSthbmdsZSkpLGMoeStsLzIqc2luKC1waS8yK2FuZ2xlKSx5K2wvMipzaW4oLXBpLzIrYW5nbGUpK2wvNCpzaW4ocGkrYW5nbGUpKSxjb2w9Y29sKQpsaW5lcyhjKHgrbCpjb3MoLXBpLzIrYW5nbGUpLHgrbCpjb3MoLXBpLzIrYW5nbGUpK2wvMipjb3MoLXBpLzIrcGkvNCthbmdsZSkpLGMoeStsKnNpbigtcGkvMithbmdsZSkseStsKnNpbigtcGkvMithbmdsZSkrbC8yKnNpbigtcGkvMitwaS80K2FuZ2xlKSksY29sPWNvbCkKbGluZXMoYyh4K2wqY29zKC1waS8yK2FuZ2xlKSx4K2wqY29zKC1waS8yK2FuZ2xlKStsLzIqY29zKC1waS8yLXBpLzQrYW5nbGUpKSxjKHkrbCpzaW4oLXBpLzIrYW5nbGUpLHkrbCpzaW4oLXBpLzIrYW5nbGUpK2wvMipzaW4oLXBpLzItcGkvNCthbmdsZSkpLGNvbD1jb2wpCn0KCnBhcihtYXI9YygwLDAsMCwwKSxtYWk9YygwLDAsMCwwKSkKcGxvdCgwLDAseGxhYj0iIix5bGFiPSIiLHhsaW09YygwLDEwKSx5bGltPWMoMCwxMCksY29sPTAseGF4dD0ibm9uZSIseWF4dD0ibm9uZSIsYXhlcz1GQUxTRSkKcmVjdCgwLDYsMTAsMTAsYm9yZGVyPSJyZWQiLGx3ZD0yKQp0ZXh0KC41LDgsInBvcHVsYXRpb24iLHNydD05MCxjb2w9InJlZCIsY2V4PTIpCnN5bWJvbHMgKDMsIDgsIGNpcmNsZXM9MS4yLCBjb2w9InJlZCIsYWRkPVRSVUUsZmc9InJlZCIsaW5jaGVzPUZBTFNFLGx3ZD0yKQpzZXQuc2VlZCgzMzApCmdyaWQ9c2VxKDAsMSwuMDEpCgpmb3IgKGkgaW4gMTo1MCkKewoJYW5nbGUxPXJ1bmlmKG49MSxtaW49MCxtYXg9MzYwKQoJYW5nbGUyPXJ1bmlmKG49MSxtaW49MCxtYXg9MzYwKQoJcmFkaXVzPXNhbXBsZShncmlkLHByb2I9Z3JpZF4yKnBpL3N1bShncmlkXjIqcGkpLHNpemU9MSkKCWtvcHZvZXRlcigzK3JhZGl1cypjb3MoYW5nbGUxLzE4MCpwaSksOCtyYWRpdXMqc2luKGFuZ2xlMS8xODAqcGkpLGFuZ2xlPWFuZ2xlMikKfQp0ZXh0KDcuNSw4LCJFZmZlY3Qgb2YgYXJnaW5pbmUgZGVmLiBpbiBwb3B1bGF0aW9uIixjb2w9InJlZCIsY2V4PTEuMikKCnJlY3QoMCwwLDEwLDQsYm9yZGVyPSJibHVlIixsd2Q9MikKdGV4dCguNSwyLCJzYW1wbGUiLHNydD05MCxjb2w9ImJsdWUiLGNleD0yKQpzeW1ib2xzICgzLCAyLCBjaXJjbGVzPTEuMiwgY29sPSJyZWQiLGFkZD1UUlVFLGZnPSJibHVlIixpbmNoZXM9RkFMU0UsbHdkPTIpCmZvciAoaSBpbiAwOjEpCglmb3IgKGogaW4gMDoyKQp7CgoJa29wdm9ldGVyKDIuNStqKigzLjktMi4xKS80LDEuNStpKQp9CnRleHQoNy41LDIsIkVmZmVjdCBvZiBhcmdpbmluZSBkZWYuIGluIHNhbXBsZSIsY29sPSJibHVlIixjZXg9MS4yKQoKYXJyb3dzKDMsNS45LDMsNC4xLGNvbD0iYmxhY2siLGx3ZD0zKQphcnJvd3MoNyw0LjEsNyw1LjksY29sPSJibGFjayIsbHdkPTMpCnRleHQoMS41LDUsIkV4cC4gRGVzaWduIixjb2w9ImJsYWNrIixjZXg9MS4yKQp0ZXh0KDguNSw1LCJFc3RpbWF0aW9uIFxuIEluZmVyZW5jZSAiLGNvbD0iYmxhY2siLGNleD0xLjIpCnRleHQoNy41LC41LCJEYXRhIGV4cGxvcmF0aW9uIixjb2w9ImJsYWNrIixjZXg9MS4yKQpgYGAKCiMjIyMgVC10ZXN0Cgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL2NWdzVrZFNSWkNFIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKCiQkCiBcbG9nXzIgXHRleHR7RkN9ID0gXGJhcnt5fV97cDF9LVxiYXJ7eX1fe3AyfQokJAoKJCQKVF9nPVxmcmFje1xsb2dfMiBcdGV4dHtGQ319e1x0ZXh0e3NlfV97XGxvZ18yIFx0ZXh0e0ZDfX19CiQkCgokJApUX2c9XGZyYWN7XHdpZGVoYXR7XHRleHR7c2lnbmFsfX19e1x3aWRlaGF0e1x0ZXh0e05vaXNlfX19CiQkCgpJZiB3ZSBjYW4gYXNzdW1lIGVxdWFsIHZhcmlhbmNlIGluIGJvdGggdHJlYXRtZW50IGdyb3VwczoKCiQkClx0ZXh0e3NlfV97XGxvZ18yIFx0ZXh0e0ZDfX09XHRleHR7U0R9XHNxcnR7XGZyYWN7MX17bl8xfStcZnJhY3sxfXtuXzJ9fQokJAoKYGBge3J9CldQXzAwMzAyMzM5MiA8LSBkYXRhLmZyYW1lKAogICAgaW50ZW5zaXR5ID0gYXNzYXkocGVbWyJwcm90ZWluIl1dWyJXUF8wMDMwMjMzOTIiLF0pICU+JSBjKCksIAogICAgZ2Vub3R5cGUgPSBjb2xEYXRhKHBlKVssMV0pIAoKV1BfMDAzMDIzMzkyICU+JSAKICBnZ3Bsb3QoYWVzKHg9Z2Vub3R5cGUseT1pbnRlbnNpdHkpKSArIAogIGdlb21fcG9pbnQoKSArCiAgZ2d0aXRsZSgiUHJvdGVpbiBXUF8wMDMwMjMzOTIiKQpgYGAKCmBgYHtyIGVjaG89RkFMU0V9CmxtSGxwIDwtIGxtKGludGVuc2l0eSB+IGdlbm90eXBlLCBkYXRhID0gV1BfMDAzMDIzMzkyKQpgYGAKCiQkCnQ9XGZyYWN7XGxvZ18yXHdpZGVoYXR7XHRleHR7RkN9fX17XHRleHR7c2V9X3tcbG9nXzJcd2lkZWhhdHtcdGV4dHtGQ319fX09XGZyYWN7YHIgZm9ybWF0KHN1bW1hcnkobG1IbHApJGNvZWZbMiwxXSwgZGlnaXQgPSAzKWB9e2ByIGZvcm1hdChzdW1tYXJ5KGxtSGxwKSRjb2VmWzIsMl0sIGRpZ2l0ID0gMylgfT1gciBmb3JtYXQoc3VtbWFyeShsbUhscCkkY29lZlsyLDNdLCBkaWdpdCA9IDMpYAokJAoKLSBJcyB0ID0gYHIgZm9ybWF0KHN1bW1hcnkobG1IbHApJGNvZWZbMiwzXSwgZGlnaXQgPSAzKWAgaW5kaWNhdGluZyB0aGF0CnRoZXJlIGlzIGFuIGVmZmVjdD8KCi0gSG93IGxpa2VseSBpcyBpdCB0byBvYnNlcnZlCnQgPSBgciBmb3JtYXQoc3VtbWFyeShsbUhscCkkY29lZlsyLDNdLCBkaWdpdCA9IDMpYCB3aGVuIHRoZXJlIGlzIG5vIGVmZmVjdCBvZiB0aGUgYXJnUCBLTyBvbiB0aGUgcHJvdGVpbiBleHByZXNzaW9uPwoKIyMjIyBOdWxsIGh5cG90aGVzaXMgKCRIXzAkKSBhbmQgYWx0ZXJuYXRpdmUgaHlwb3RoZXNpcyAoJEhfMSQpCgotIFdpdGggZGF0YSB3ZSBjYW4gbmV2ZXIgcHJvdmUgYSBoeXBvdGhlc2lzIChmYWxzaWZpY2F0aW9uIHByaW5jaXBsZSBvZiBQb3BwZXIpCi0gV2l0aCBkYXRhIHdlIGNhbiBvbmx5IHJlamVjdCBhIGh5cG90aGVzaXMgCgotIEluIGdlbmVyYWwgd2Ugc3RhcnQgZnJvbSAqYWx0ZXJuYXRpdmUgaHlwb3RoZXNlKiAkSF8xJDogd2Ugd2FudCB0byBzaG93IGFuIGVmZmVjdCBvZiB0aGUgS08gb24gYSBwcm90ZWluCgo8Y2VudGVyPgokSF8xJDogT24gYXZlcmFnZSB0aGUgcHJvdGVpbiBhYnVuZGFuY2UgaW4gV1QgaXMgZGlmZmVyZW50IGZyb20gdGhhdCBpbiBLTwo8L2NlbnRlcj4KCi0gQnV0LCB3ZSB3aWxsIGFzc2VzcyB0aGlzIGJ5IGZhbHNpZnlpbmcgdGhlIG9wcG9zaXRlOiAKPGNlbnRlcj4KJEhfMCQ6IE9uIGF2ZXJhZ2UgdGhlIHByb3RlaW4gYWJ1bmRhbmNlIGluIFdUIGlzIGVxdWFsIHRvIHRoYXQgaW4gS088LQo8L2NlbnRlcj4KCgpgYGB7cn0KdC50ZXN0KGludGVuc2l0eSB+IGdlbm90eXBlLCBkYXRhID0gV1BfMDAzMDIzMzkyLCB2YXIuZXF1YWw9VFJVRSkKYGBgCgotIEhvdyBsaWtlbHkgaXMgaXQgdG8gb2JzZXJ2ZSBhbiBlcXVhbCBvciBtb3JlIGV4dHJlbWUgZWZmZWN0IHRoYW4gdGhlIG9uZSBvYnNlcnZlZCBpbiB0aGUgc2FtcGxlIHdoZW4gdGhlIG51bGwgaHlwb3RoZXNpcyBpcyB0cnVlPwotIFdoZW4gd2UgbWFrZSBhc3N1bXB0aW9ucyBhYm91dCB0aGUgZGlzdHJpYnV0aW9uIG9mIG91ciB0ZXN0IHN0YXRpc3RpYyB3ZSBjYW4gcXVhbnRpZnkgdGhpcyBwcm9iYWJpbGl0eTogKnAtdmFsdWUqLiAKVGhlIHAtdmFsdWUgd2lsbCBvbmx5IGJlIGNhbGN1bGF0ZWQgY29ycmVjdGx5IGlmIHRoZSB1bmRlcmx5aW5nIGFzc3VtcHRpb25zIGhvbGQhCi0gV2hlbiB3ZSByZXBlYXQgdGhlIGV4cGVyaW1lbnQsIHRoZSBwcm9iYWJpbGl0eSB0byBvYnNlcnZlIGEgZm9sZCBjaGFuZ2UgZm9yIHRoaXMgZ2VuZSB0aGF0IGlzIG1vcmUgZXh0cmVtZSB0aGFuIGEgYHIgZm9ybWF0KDJeYWJzKGxtSGxwJGNvZWZbMl0pLGRpZ2l0cz0zKWAgZm9sZCAoJFxsb2dfMiBGQz1gciBmb3JtYXQobG1IbHAkY29lZlsyXSxkaWdpdHM9MylgJCkgZG93biBvciB1cCByZWd1bGF0aW9uIGJ5IHJhbmRvbSBjaGFuZ2UgKGlmICRIXzAkIGlzIHRydWUpIGlzIGByIHJvdW5kKHN1bW1hcnkobG1IbHApJGNvZWZbMiw0XSoxZTYsMClgIG91dCBvZiAxIDAwMCAwMDAuICAKLSBJZiB0aGUgcC12YWx1ZSBpcyBiZWxvdyBhIHNpZ25pZmljYW5jZSB0aHJlc2hvbGQgJFxhbHBoYSQgd2UgcmVqZWN0IHRoZSBudWxsIGh5cG90aGVzaXMuICpXZSBjb250cm9sIHRoZSBwcm9iYWJpbGl0eSBvbiBhIGZhbHNlIHBvc2l0aXZlIHJlc3VsdCBhdCB0aGUgJFxhbHBoYSQtbGV2ZWwgKHR5cGUgSSBlcnJvcikqCgotIE5vdGUsIHRoYXQgdGhlIHAtdmFsdWVzIGFyZSB1bmlmb3JtIHVuZGVyIHRoZSBudWxsIGh5cG90aGVzaXMsIGkuZS4gd2hlbiAkSF8wJCBpcyB0cnVlIGFsbCBwLXZhbHVlcyBhcmUgZXF1YWxseSBsaWtlbHkuIAoKIyMgTXVsdGlwbGUgaHlwb3RoZXNpcyB0ZXN0aW5nCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL2NMbi1DRnlBNnBzIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKLSBDb25zaWRlciB0ZXN0aW5nIERBIGZvciBhbGwgJG09MTA2NiQgcHJvdGVpbnMgc2ltdWx0YW5lb3VzbHkKLSBXaGF0IGlmIHdlIGFzc2VzcyBlYWNoIGluZGl2aWR1YWwgdGVzdCBhdCBsZXZlbCAkXGFscGhhJD8KJFxyaWdodGFycm93JCBQcm9iYWJpbGl0eSB0byBoYXZlIGEgZmFsc2UgcG9zaXRpdmUgKEZQKSBhbW9uZyBhbGwgbSBzaW11bHRhdGVub3VzCnRlc3QgJD4+PiAgXGFscGhhPSAwLjA1JAoKLSBJbmRlZWQgZm9yIGVhY2ggbm9uIERBIHByb3RlaW4gd2UgaGF2ZSBhIHByb2JhYmlsaXR5IG9mIDUlIHRvIHJldHVybiBhIEZQLgotIEluIGEgdHlwaWNhbCBleHBlcmltZW50IHRoZSBtYWpvcml0eSBvZiB0aGUgcHJvdGVpbnMgYXJlIG5vbiBEQS4gCi0gU28gYW4gdXBwZXJib3VuZCBvZiB0aGUgZXhwZWN0ZWQgRlAgaXMgJG0gXHRpbWVzIFxhbHBoYSQgb3IgJDEwNjYgXHRpbWVzIDAuMDU9YHIgcm91bmQoMTA2NiowLjA1LDApYCQuIAoKJFxyaWdodGFycm93JCBIZW5jZSwgd2UgYXJlIGJvdW5kIHRvIGNhbGwgbWFueSBmYWxzZSBwb3NpdGl2ZSBwcm90ZWlucyBlYWNoIHRpbWUgd2UgcnVuIHRoZSBleHBlcmltZW50LgoKIyMjIE11bHRpcGxlIHRlc3RpbmcKCiMjIyMgRmFtaWx5LXdpc2UgZXJyb3IgcmF0ZQoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC9JTF9lVVN5UkRSQSIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4KClRoZSBmYW1pbHktd2lzZSBlcnJvciByYXRlIChGV0VSKSBhZGRyZXNzZXMgdGhlIG11bHRpcGxlIHRlc3RpbmcgaXNzdWUgYnkgbm8gbG9uZ2VyIGNvbnRyb2xsaW5nIHRoZSBpbmRpdmlkdWFsIHR5cGUgSSBlcnJvciBmb3IgZWFjaCBwcm90ZWluLCBpbnN0ZWFkIGl0IGNvbnRyb2xzOiAgCgpcWwogICBcdGV4dHtGV0VSfSA9IAogICBcdGV4dHtQfVxsZWZ0W0ZQIFxnZXEgMSBccmlnaHRdLgpcXQoKVGhlIEJvbmZlcnJvbmkgbWV0aG9kIGlzIHdpZGVseSB1c2VkIHRvIGNvbnRyb2wgdGhlIHR5cGUgSSBlcnJvcjogCgotIGFzc2VzcyBlYWNoIHRlc3QgYXQgClxbXGFscGhhX1x0ZXh0e2Fkan09XGZyYWN7XGFscGhhfXttfVxdCi0gb3IgdXNlIGFkanVzdGVkIHAtdmFsdWVzIGFuZCBjb21wYXJlIHRoZW0gdG8gJFxhbHBoYSQ6IApcW3BfXHRleHR7YWRqfT1cdGV4dHttaW59XGxlZnQocCBcdGltZXMgbSwxXHJpZ2h0KVxdCgpQcm9ibGVtLCB0aGUgbWV0aG9kIGlzIHZlcnkgY29uc2VydmF0aXZlISAKCiMjIyMgRmFsc2UgZGlzY292ZXJ5IHJhdGUKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvZW52RHF2RXdSY2MiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgotIEZEUjogRXhwZWN0ZWQgcHJvcG9ydGlvbiBvZiBmYWxzZSBwb3NpdGl2ZXMgb24gdGhlIHRvdGFsIG51bWJlciBvZiBwb3NpdGl2ZXMgeW91IHJldHVybi4KLSBBbiBGRFIgb2YgMSUgbWVhbnMgdGhhdCBvbiBhdmVyYWdlIHdlIGV4cGVjdCAxJSBmYWxzZSBwb3NpdGl2ZSBwcm90ZWlucyBpbiB0aGUgbGlzdCBvZiBwcm90ZWlucyB0aGF0IGFyZSBjYWxsZWQgc2lnbmlmaWNhbnQuCi0gRGVmaW5lZCBieSBCZW5qYW1pbmkgYW5kIEhvY2hiZXJnIGluIHRoZWlyIHNlbWluYWwgcGFwZXIgQmVuamFtaW5pLCBZLiBhbmQgSG9jaGJlcmcsIFkuICgxOTk1KS4gIkNvbnRyb2xsaW5nIHRoZSBmYWxzZSBkaXNjb3ZlcnkgcmF0ZTogYSBwcmFjdGljYWwgYW5kIHBvd2VyZnVsIGFwcHJvYWNoIHRvIG11bHRpcGxlIHRlc3RpbmciLiBKb3VybmFsIG9mIHRoZSBSb3lhbCBTdGF0aXN0aWNhbCBTb2NpZXR5IFNlcmllcyBCLCA1NyAoMSk6IDI4OeKAkzMwMC4gCgpUaGUgKipGYWxzZSBEaXNjb3ZlcnkgUHJvcG9ydGlvbiAoRkRQKSoqIGlzIHRoZSBmcmFjdGlvbiBvZiBmYWxzZSBwb3NpdGl2ZXMgdGhhdCBhcmUgcmV0dXJuZWQsIGkuZS4gCgpcWwpGRFAgPSBcZnJhY3tGUH17Un0KXF0KCi0gSG93ZXZlciwgdGhpcyBxdWFudGl0eSBjYW5ub3QgYmUgb2JzZXJ2ZWQgYmVjYXVzZSBpbiBwcmFjdGljZSB3ZSBvbmx5IGtub3cgdGhlIG51bWJlciBvZiBwcm90ZWlucyBmb3Igd2hpY2ggd2UgcmVqZWN0ZWQgJEhfMCQsICRSJC4gCi0gQnV0LCB3ZSBkbyBub3Qga25vdyB0aGUgbnVtYmVyIG9mIGZhbHNlIHBvc2l0aXZlcywgJEZQJC4KClRoZXJlZm9yZSwgQmVuamFtaW5pIGFuZCBIb2NoYmVyZywgMTk5NSwgZGVmaW5lZCBUaGUgKipGYWxzZSBEaXNjb3ZlcnkgUmF0ZSAoRkRSKSoqIGFzClxbCiAgIFx0ZXh0e0ZEUn0gPSBcdGV4dHtFfVxsZWZ0W1xmcmFje0ZQfXtSfVxyaWdodF0gPVx0ZXh0e0V9XGxlZnRbXHRleHR7RkRQfVxyaWdodF0KXF0KdGhlIGV4cGVjdGVkIEZEUC4gCgotIENvbnRyb2xsaW5nIHRoZSBGRFIgYWxsb3dzIGZvciBtb3JlIGRpc2NvdmVyaWVzIChpLmUuIGxvbmdlciBsaXN0cyB3aXRoIHNpZ25pZmljYW50IHJlc3VsdHMpLCB3aGlsZSB0aGUgZnJhY3Rpb24gb2YgZmFsc2UgZGlzY292ZXJpZXMgYW1vbmcgdGhlIHNpZ25pZmljYW50IHJlc3VsdHMgaW4gd2VsbCBjb250cm9sbGVkIG9uIGF2ZXJhZ2UuIEFzIGEgY29uc2VxdWVuY2UsIG1vcmUgb2YgdGhlIHRydWUgcG9zaXRpdmUgaHlwb3RoZXNlcyB3aWxsIGJlIGRldGVjdGVkLgoKCiMjIyMgSW50dWl0aW9uIG9mIEJILUZEUiBwcm9jZWR1cmUKCkNvbnNpZGVyICRtID0gMTAwMCQgdGVzdHMKCi0gU3VwcG9zZSB0aGF0IGEgcmVzZWFyY2hlciByZWplY3RzIGFsbCBudWxsIGh5cG90aGVzZXMgZm9yIHdoaWNoICRwIDwgMC4wMSQuIAoKLSBJZiB3ZSB1c2UgJHAgPCAwLjAxJCwgd2UgZXhwZWN0ICQwLjAxIFx0aW1lcyBtXzAkIHRlc3RzIHRvIHJldHVybiBmYWxzZSBwb3NpdGl2ZXMuIAotIEEgY29uc2VydmF0aXZlIGVzdGltYXRlIG9mIHRoZSBudW1iZXIgb2YgZmFsc2UgcG9zaXRpdmVzIHRoYXQgd2UgY2FuIGV4cGVjdCBjYW4gYmUgb2J0YWluZWQgYnkgY29uc2lkZXJpbmcgdGhhdCB0aGUgbnVsbCBoeXBvdGhlc2VzIGFyZSB0cnVlIGZvciBhbGwgZmVhdHVyZXMsICRtXzAgPSBtID0gIDEwMDAkLiAKLSBXZSB0aGVuIHdvdWxkIGV4cGVjdCAkMC4wMSBcdGltZXMgMTAwMCA9IDEwJCBmYWxzZSBwb3NpdGl2ZXMgKCRGUD0xMCQpLgoKLSBTdXBwb3NlIHRoYXQgdGhlIHJlc2VhcmNoZXIgZm91bmQgMjAwIGdlbmVzIHdpdGggJHA8MC4wMSQgKCRSPTIwMCQpLgoKLSBUaGUgcHJvcG9ydGlvbiBvZiBmYWxzZSBwb3NpdGl2ZSByZXN1bHRzIChGRFAgPSBmYWxzZSBwb3NpdGl2ZSBwcm9wb3J0aW9uKSBhbW9uZyB0aGUgbGlzdCBvZiAkUj0yMDAkIGdlbmVzIGNhbiB0aGVuIGJlIGVzdGltYXRlZCBhcwogXFsKICAgXHdpZGVoYXR7XHRleHR7RkRQfX09XGZyYWN7RlB9e1J9PVxmcmFjezEwfXsyMDB9PVxmcmFjezAuMDEgXHRpbWVzIDEwMDB9ezIwMH0gPSAwLjA1LgogXF0KCgojIyMjIEJlbmphbWluaSBhbmQgSG9jaGJlcmcgKDE5OTUpIHByb2NlZHVyZSBmb3IgY29udHJvbGxpbmcgdGhlIEZEUiBhdCAkXGFscGhhJAoKMS4gTGV0ICRwX3soMSl9XGxlcSBcbGRvdHMgXGxlcSBwX3sobSl9JCBkZW5vdGUgdGhlIG9yZGVyZWQgJHAkLXZhbHVlcy4KCjIuIEZpbmQgdGhlIGxhcmdlc3QgaW50ZWdlciAkayQgc28gdGhhdCAKJCQKXGZyYWN7cF97KGspfSBcdGltZXMgbX17a30gXGxlcSBcYWxwaGEKJCQKJCRcdGV4dHtvcn0kJAokJApwX3soayl9IFxsZXEgayBcdGltZXMgXGFscGhhL20KJCQKCjMuIElmIHN1Y2ggYSAkayQgZXhpc3RzLCByZWplY3QgdGhlICRrJCBudWxsIGh5cG90aGVzZXMgYXNzb2NpYXRlZCB3aXRoICRwX3soMSl9LCBcbGRvdHMsIHBfeyhrKX0kLgpPdGhlcndpc2Ugbm9uZSBvZiB0aGUgbnVsbCBoeXBvdGhlc2VzIGlzIHJlamVjdGVkLgoKVGhlIGFkanVzdGVkICRwJC12YWx1ZSAoYWxzbyBrbm93biBhcyB0aGUgJHEkLXZhbHVlIGluIEZEUiBsaXRlcmF0dXJlKToKICQkCiAgIHFfeyhpKX09XHRpbGRle3B9X3soaSl9ID0gXG1pblxsZWZ0W1xtaW5fe2o9aSxcbGRvdHMsIG19XGxlZnQobSBwX3soail9L2pccmlnaHQpLCAxIFxyaWdodF0uCiAkJAogSW4gdGhlIGh5cG90aGV0aWNhbCBleGFtcGxlIGFib3ZlOiAkaz0yMDAkLCAkcF97KGspfT0wLjAxJCwgJG09MTAwMCQgYW5kICRcYWxwaGE9MC4wNSQuCgojIyMjIEZyYW5jaXNlbGxhIEV4YW1wbGUKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvQjNCbW5PTGtZZzQiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+Cgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgPC9zdW1tYXJ5PjxwPgpgYGB7cn0KdHRlc3RNeCA8LSBmdW5jdGlvbih5LGdyb3VwKSB7CiAgICB0ZXN0IDwtIHRyeSh0LnRlc3QoeVtncm91cF0seVshZ3JvdXBdLHZhci5lcXVhbD1UUlVFKSxzaWxlbnQ9VFJVRSkKICAgIGlmKGlzKHRlc3QsInRyeS1lcnJvciIpKSB7CiAgICAgIHJldHVybihjKGxvZzJGQz1OQSxzZT1OQSx0c3RhdD1OQSxwPU5BKSkKICAgICAgfSBlbHNlIHsKICAgICAgcmV0dXJuKGMobG9nMkZDPSAodGVzdCRlc3RpbWF0ZSUqJWMoMSwtMSkpLHNlPXRlc3Qkc3RkZXJyLHRzdGF0PXRlc3Qkc3RhdGlzdGljLHB2YWw9dGVzdCRwLnZhbHVlKSkKICAgICAgfQogfQogCiByZXMgPC0gYXBwbHkoCiAgICBhc3NheShwZVtbInByb3RlaW4iXV0pLCAKICAgIDEsIAogICAgdHRlc3RNeCwKICAgIGdyb3VwID0gY29sRGF0YShwZSkkZ2Vub3R5cGU9PSJEOCIpICU+JSAKICB0IAogY29sbmFtZXMocmVzKSA8LSBjKCJsb2dGQyIsInNlIiwidHN0YXQiLCJwdmFsIikKIHJlcyA8LSByZXMgJT4lIGFzLmRhdGEuZnJhbWUgJT4lIG5hLmV4Y2x1ZGUgJT4lIGFycmFuZ2UocHZhbCkKIHJlcyRhZGpQdmFsIDwtIHAuYWRqdXN0KHJlcyRwdmFsLCAiZmRyIikKIGFscGhhIDwtIDAuMDUKcmVzJGFkakFscGhhRm9ybSA8LSBwYXN0ZTAoMTpucm93KHJlcyksIiB4ICIsYWxwaGEsIi8iLG5yb3cocmVzKSkKcmVzJGFkakFscGhhIDwtIGFscGhhICogKDE6bnJvdyhyZXMpKS9ucm93KHJlcykgCnJlcyQicHZhbCA8IGFkakFscGhhIiA8LSByZXMkcHZhbCA8IHJlcyRhZGpBbHBoYSAKcmVzJCJhZGpQdmFsIDwgYWxwaGEiIDwtIHJlcyRhZGpQdmFsIDwgYWxwaGEgCmBgYAo8L3A+PC9kZXRhaWxzPgoKRldFUjogQm9uZmVycm9uaSBtZXRob2Q6JFxhbHBoYV9cdGV4dHthZGp9ID0gXGFscGhhL20gPSAwLjA1IC8gYHIgbnJvdyhyZXMpYD0gYHIgcm91bmQoYWxwaGEvbnJvdyhyZXMpLDUpYCQKCmBgYHtyIGVjaG89RkFMU0V9IApoZWFkKHJlc1ssLWMoMjozKV0sc3VtKHJlcyRhZGpQdmFsIDwgYWxwaGEpKzIpICU+JSBrbml0cjo6a2FibGUoKQpgYGAKfCAuLi4gfCAuLi4gfCAuLi4gfCAuLi4gfCAuLi4gfCAuLi4gfCAuLi4gfCAuLi4gfAp8V1BfMDAzMDQwNTYyIHwgMC4wMDM5NDgwfCAwLjk5NzY0Mjl8IDAuOTk4NTc5N3wxMDY1IHggMC4wNS8xMDY2IHwgMC4wNDk5NTMxfEZBTFNFICAgICAgICAgICB8RkFMU0UgICAgCnxXUF8wMDMwNDExMzAgfCAwLjAwMDI5NDF8IDAuOTk5MjgxMnwgMC45OTkyODEyfDEwNjYgeCAwLjA1LzEwNjYgfCAgICAgMC4wNXxGQUxTRSAgICAgICAgICAgfEZBTFNFICAgICAgICAgICB8CgoKCgojIyMjIFJlc3VsdHMKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIDwvc3VtbWFyeT48cD4KYGBge3J9CnZvbGNhbm9UIDwtIHJlcyAlPiUgCiAgZ2dwbG90KGFlcyh4ID0gbG9nRkMsIHkgPSAtbG9nMTAocHZhbCksIGNvbG9yID0gYWRqUHZhbCA8IDAuMDUpKSArCiAgICBnZW9tX3BvaW50KGNleCA9IDIuNSkgKwogICAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGFscGhhKGMoImJsYWNrIiwgInJlZCIpLCAwLjUpKSArCiAgICB0aGVtZV9taW5pbWFsKCkgCmBgYAo8L3A+PC9kZXRhaWxzPgoKCmBgYHtyfQp2b2xjYW5vVApgYGAKCiMjIE1vZGVyYXRlZCBTdGF0aXN0aWNzCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL19RMTFMWER5MHhVIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKUHJvYmxlbXMgd2l0aCBvcmRpbmFyeSB0LXRlc3QKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSA8L3N1bW1hcnk+PHA+CmBgYHtyfQpwcm9ibGVtUGxvdHMgPC0gbGlzdCgpIApwcm9ibGVtUGxvdHNbWzFdXSA8LSByZXMgJT4lIAogIGdncGxvdChhZXMoeCA9IGxvZ0ZDLCB5ID0gc2UsIGNvbG9yID0gYWRqUHZhbCA8IDAuMDUpKSArCiAgICBnZW9tX3BvaW50KGNleCA9IDIuNSkgKwogICAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGFscGhhKGMoImJsYWNrIiwgInJlZCIpLCAwLjUpKSArCiAgICB0aGVtZV9taW5pbWFsKCkgCgpmb3IgKGkgaW4gMjozKQp7CnByb2JsZW1QbG90c1tbaV1dIDwtIGNvbERhdGEocGUpICU+JSAKICAgIGFzLmRhdGEuZnJhbWUgJT4lIAogICAgbXV0YXRlKGludGVuc2l0eSA9IHBlW1sicHJvdGVpbiJdXVtyb3duYW1lcyhyZXMpW2ldLF0gJT4lIAogICAgICAgICAgICAgYXNzYXkgJT4lIAogICAgICAgICAgICAgYykgJT4lIAogICAgZ2dwbG90KGFlcyh4PWdlbm90eXBlLHk9aW50ZW5zaXR5KSkgKwogICAgZ2VvbV9wb2ludCgpICsgCiAgICB5bGltKC0zLDApICsKICAgIGdndGl0bGUocm93bmFtZXMocmVzKVtpXSkKfQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpwcm9ibGVtUGxvdHMKYGBgCgpBIGdlbmVyYWwgY2xhc3Mgb2YgbW9kZXJhdGVkIHRlc3Qgc3RhdGlzdGljcyBpcyBnaXZlbiBieQogXFsKICAgVF9nXnttb2R9ID0gXGZyYWN7XGJhcntZfV97ZzF9IC0gXGJhcntZfV97ZzJ9fXtDIFxxdWFkIFx0aWxkZXtTfV9nfSAsCiBcXQogd2hlcmUgJFx0aWxkZXtTfV9nJCBpcyBhIG1vZGVyYXRlZCBzdGFuZGFyZCBkZXZpYXRpb24gZXN0aW1hdGUuIAoKLSAkQyQgaXMgYSBjb25zdGFudCBkZXBlbmRpbmcgb24gdGhlIGRlc2lnbiBlLmcuICRcc3FydHsxL3tuXzF9KzEvbl8yfSQgZm9yIGEgdC10ZXN0IGFuZCBvZiBhbm90aGVyIGZvcm0gZm9yIGxpbmVhciBtb2RlbHMuCi0gJFx0aWxkZXtTfV9nPVNfZytTXzAkOiBhZGQgc21hbGwgcG9zaXRpdmUgY29uc3RhbnQgdG8gZGVub21pbmF0b3Igb2YgdC1zdGF0aXN0aWMuIAotIFRoaXMgY2FuIGJlIGFkb3B0ZWQgaW4gUGVyc2V1cy4gCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgPC9zdW1tYXJ5PjxwPgpgYGB7cn0Kc2ltSTwtc2FwcGx5KHJlcyRzZS9zcXJ0KDEvMysxLzMpLGZ1bmN0aW9uKG4sbWVhbixzZCkgcm5vcm0obixtZWFuLHNkKSxuPTYsbWVhbj0wKSAlPiUgdApyZXNTaW0gPC0gYXBwbHkoCiAgICBzaW1JLCAKICAgIDEsIAogICAgdHRlc3RNeCwKICAgIGdyb3VwID0gY29sRGF0YShwZSkkZ2Vub3R5cGU9PSJEOCIpICU+JSAKICB0IAogY29sbmFtZXMocmVzU2ltKSA8LSBjKCJsb2dGQyIsInNlIiwidHN0YXQiLCJwdmFsIikKIHJlc1NpbSA8LSBhcy5kYXRhLmZyYW1lKHJlc1NpbSkKIHRzdGF0U2ltUGxvdCA8LSByZXNTaW0gJT4lIAogICBnZ3Bsb3QoYWVzKHg9dHN0YXQpKSArCiAgICAgZ2VvbV9oaXN0b2dyYW0oYWVzKHk9Li5kZW5zaXR5Li4sIGZpbGw9Li5jb3VudC4uKSxiaW5zPTMwKSArCiAgICAgc3RhdF9mdW5jdGlvbihmdW49ZHQsCiAgICBjb2xvcj0icmVkIiwKICAgIGFyZ3M9bGlzdChkZj00KSkgKyAKICAgeWxpbSgwLC42KSArCiAgIGdndGl0bGUoInQtc3RhdGlzdGljIikKCiAKIHJlc1NpbSRDIDwtIHNxcnQoMS8zKzEvMykgCiByZXNTaW0kc2QgPC0gcmVzU2ltJHNlL3Jlc1NpbSRDIAogdHN0YXRTaW1QZXJzZXVzIDwtIHJlc1NpbSAlPiUgCiAgIGdncGxvdChhZXMoeD1sb2dGQy8oKHNkKy4xKSpDKSkpICsKICAgICBnZW9tX2hpc3RvZ3JhbShhZXMoeT0uLmRlbnNpdHkuLiwgZmlsbD0uLmNvdW50Li4pLGJpbnM9MzApICsKICAgICBzdGF0X2Z1bmN0aW9uKGZ1bj1kdCwKICAgICAgICAgICAgICAgICAgIGNvbG9yPSJyZWQiLAogICAgICAgICAgICAgICAgICBhcmdzPWxpc3QoZGY9NCkpICsgCiAgICAgeWxpbSgwLC42KSArCiAgICBnZ3RpdGxlKCJQZXJzZXVzIikKYGBgCgo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CmdyaWRFeHRyYTo6Z3JpZC5hcnJhbmdlKHRzdGF0U2ltUGxvdCx0c3RhdFNpbVBlcnNldXMsbnJvdz0xKQpgYGAKCi0gVGhlIGNob2ljZSBvZiAkU18wJCBpbiBQZXJzZXVzIGlzIGFkIGhvYyBhbmQgdGhlIHQtc3RhdGlzdGljIGlzIG5vLWxvbmdlciB0LWRpc3RyaWJ1dGVkLiAKLSBQZXJtdXRhdGlvbiB0ZXN0LCBidXQgaXMgZGlmZmljdWx0IGZvciBtb3JlIGNvbXBsZXggZGVzaWducy4KLSBBbGxvd3MgZm9yIERhdGEgRHJlZGdpbmcgYmVjYXVzZSB1c2VyIGNhbiBjaG9vc2UgJFNfMCQgCgoKIyMjIEVtcGlyaWNhbCBCYXllcyAKCgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjUwJSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvbGltbWFTaHJpbmthZ2UucG5nIikKYGBgCgpGaWd1cmUgY291cnRlc3kgdG8gUmFmYWVsIElyaXphcnJ5CgokJAogICBUX2dee21vZH0gPSBcZnJhY3tcYmFye1l9X3tnMX0gLSBcYmFye1l9X3tnMn19e0MgXHF1YWQgXHRpbGRle1N9X2d9ICwKICQkCgotICoqZW1waXJpY2FsIEJheWVzKiogdGhlb3J5IHByb3ZpZGVzIGZvcm1hbCBmcmFtZXdvcmsgZm9yIGJvcnJvd2luZyBzdHJlbmd0aCBhY3Jvc3MgcHJvdGVpbnMsCi0gSW1wbGVtZW50ZWQgaW4gcG9wdWxhciBiaW9jb25kdWN0b3IgcGFja2FnZSAqKmxpbW1hKiogYW5kICoqbXNxcm9iMioqCgokJAogIFx0aWxkZXtTfV9nPVxzcXJ0e1xmcmFje2RfZ1NfZ14yK2RfMFNfMF4yfXtkX2crZF8wfX0sCiQkCgotICRTXzBeMiQ6IGNvbW1vbiB2YXJpYW5jZSAob3ZlciBhbGwgcHJvdGVpbnMpIAotIE1vZGVyYXRlZCB0LXN0YXRpc3RpYyBpcyB0LWRpc3RyaWJ1dGVkIHdpdGggJGRfMCtkX2ckIGRlZ3JlZXMgb2YgZnJlZWRvbS4gCi0gTm90ZSB0aGF0IHRoZSBkZWdyZWVzIG9mIGZyZWVkb20gaW5jcmVhc2UgYnkgYm9ycm93aW5nIHN0cmVuZ3RoIGFjcm9zcyBwcm90ZWlucyEKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgdGhlIGNvZGUgPC9zdW1tYXJ5PjxwPiAgIAoKMS4gV2UgbW9kZWwgdGhlIHByb3RlaW4gbGV2ZWwgZXhwcmVzc2lvbiB2YWx1ZXMgdXNpbmcgdGhlIGBtc3Fyb2JgIGZ1bmN0aW9uLgpCeSBkZWZhdWx0IGBtc3Fyb2IyYCBlc3RpbWF0ZXMgdGhlIG1vZGVsIHBhcmFtZXRlcnMgdXNpbmcgcm9idXN0IHJlZ3Jlc3Npb24uCgpXZSB3aWxsIG1vZGVsIHRoZSBkYXRhIHdpdGggYSBkaWZmZXJlbnQgZ3JvdXAgbWVhbiBmb3IgZXZlcnkgZ2Vub3R5cGUuIApUaGUgZ3JvdXAgaXMgaW5jb2RlZCBpbiB0aGUgdmFyaWFibGUgYGdlbm90eXBlYCBvZiB0aGUgY29sRGF0YS4gCldlIGNhbiBzcGVjaWZ5IHRoaXMgbW9kZWwgYnkgdXNpbmcgYSBmb3JtdWxhIHdpdGggdGhlIGZhY3RvciBgZ2Vub3R5cGVgIGFzIGl0cyBwcmVkaWN0b3I6IApgZm9ybXVsYSA9IH5nZW5vdHlwZWAuCgpOb3RlLCB0aGF0IGEgZm9ybXVsYSBhbHdheXMgc3RhcnRzIHdpdGggYSBzeW1ib2wgJ34nLgpgYGB7ciB3YXJuaW5nPUZBTFNFfQpwZSA8LSBtc3Fyb2Iob2JqZWN0ID0gcGUsIGkgPSAicHJvdGVpbiIsIGZvcm11bGEgPSB+Z2Vub3R5cGUpCmBgYAoKMi4gSW5mZXJlbmNlIAoKV2UgZmlyc3QgZXhwbG9yZSB0aGUgZGVzaWduIG9mIHRoZSBtb2RlbCB0aGF0IHdlIHNwZWNpZmllZCB1c2luZyB0aGUgdGhlIHBhY2thZ2UgYEV4cGxvcmVNb2RlbE1hdHJpeGAgCgpgYGB7cn0KbGlicmFyeShFeHBsb3JlTW9kZWxNYXRyaXgpClZpc3VhbGl6ZURlc2lnbihjb2xEYXRhKHBlKSx+Z2Vub3R5cGUpJHBsb3RsaXN0W1sxXV0KYGBgCgpXZSBoYXZlIHR3byBtb2RlbCBwYXJhbWV0ZXJzLCB0aGUgKEludGVyY2VwdCkgYW5kIGdlbm90eXBlRDguIApUaGlzIHJlc3VsdHMgaW4gYSBtb2RlbCB3aXRoIHR3byBncm91cCBtZWFuczogCgoxLiBGb3IgdGhlIHdpbGQgdHlwZSAoV1QpIHRoZSBleHBlY3RlZCB2YWx1ZSAobWVhbikgb2YgdGhlIGxvZzIgdHJhbnNmb3JtZWQgaW50ZW5zaXR5IHkgZm9yIGEgcHJvdGVpbiB3aWxsIGJlIG1vZGVsbGVkIHVzaW5nIAoKJCRcdGV4dHtFfVtZXHZlcnQgXHRleHR7Z2Vub3R5cGV9PVx0ZXh0e1dUfV0gPSBcdGV4dHsoSW50ZXJjZXB0KX0kJCAKCjIuIEZvciB0aGUga25vY2tvdXQgZ2Vub3R5cGUgRDggdGhlIGV4cGVjdGVkIHZhbHVlIChtZWFuKSBvZiB0aGUgbG9nMiB0cmFuc2Zvcm1lZCBpbnRlbnNpdHkgeSBmb3IgYSBwcm90ZWluIHdpbGwgYmUgbW9kZWxsZWQgdXNpbmcgCgokJFx0ZXh0e0V9W1lcdmVydCBcdGV4dHtnZW5vdHlwZX09XHRleHR7RDh9XSA9IFx0ZXh0eyhJbnRlcmNlcHQpfSArIFx0ZXh0e2dlbm90eXBlRDh9JCQgCgpUaGUgYXZlcmFnZSBsb2cyRkMgYmV0d2VlbiBEOCBhbmQgV1QgaXMgdGh1cwokJFxsb2dfMlx0ZXh0e0ZDfV97RDgtV1R9PSBcdGV4dHtFfVtZXHZlcnQgXHRleHR7Z2Vub3R5cGV9PVx0ZXh0e0Q4fV0gLSBcdGV4dHtFfVtZXHZlcnQgXHRleHR7Z2Vub3R5cGV9PVx0ZXh0e1dUfV0gPSBcdGV4dHtnZW5vdHlwZUQ4fQokJAoKSGVuY2UsIGFzc2Vzc2luZyB0aGUgbnVsbCBoeXBvdGhlc2lzIHRoYXQgdGhlcmUgaXMgbm8gZGlmZmVyZW50aWFsIGFidW5kYW5jZSBiZXR3ZWVuIEQ4IGFuZCBXVCBjYW4gYmUgcmVmb3JtdWxhdGVkIGFzCgokJEhfMDogIFx0ZXh0e2dlbm90eXBlRDh9PTAkJApXZSBjYW4gaW1wbGVtZW50IGEgaHlwb3RoZXNpcyB0ZXN0IGZvciBlYWNoIHByb3RlaW4gaW4gbXNxcm9iMiB1c2luZyB0aGUgY29kZSBiZWxvdzogCgpgYGB7cn0KTCA8LSBtYWtlQ29udHJhc3QoImdlbm90eXBlRDggPSAwIiwgcGFyYW1ldGVyTmFtZXMgPSBjKCJnZW5vdHlwZUQ4IikpCnBlIDwtIGh5cG90aGVzaXNUZXN0KG9iamVjdCA9IHBlLCBpID0gInByb3RlaW4iLCBjb250cmFzdCA9IEwpCmBgYAoKV2UgY2FuIHNob3cgdGhlIGxpc3Qgd2l0aCBhbGwgc2lnbmlmaWNhbnQgREUgcHJvdGVpbnMgYXQgdGhlIDUlIEZEUiB1c2luZyAKYGBge3J9CnJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRnZW5vdHlwZUQ4ICU+JSAKICBhcnJhbmdlKHB2YWwpICU+JQogIGZpbHRlcihhZGpQdmFsPDAuMDUpCmBgYAoKV2UgY2FuIGFsc28gdmlzdWFsaXNlIHRoZSByZXN1bHRzIHVzaW5nIGEgdm9sY2Fub3Bsb3QKCmBgYHtyfQp2b2xjYW5vIDwtIGdncGxvdCgKICAgIHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRnZW5vdHlwZUQ4LAogICAgYWVzKHggPSBsb2dGQywgeSA9IC1sb2cxMChwdmFsKSwgY29sb3IgPSBhZGpQdmFsIDwgMC4wNSkKKSArCiAgICBnZW9tX3BvaW50KGNleCA9IDIuNSkgKwogICAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGFscGhhKGMoImJsYWNrIiwgInJlZCIpLCAwLjUpKSArCiAgICB0aGVtZV9taW5pbWFsKCkgKwogICAgZ2d0aXRsZSgibXNxcm9iMiIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CmdyaWRFeHRyYTo6Z3JpZC5hcnJhbmdlKAogIHZvbGNhbm9UICsgICAgCiAgICB4bGltKC0zLDMpICsKICBnZ3RpdGxlKCJvcmRpbmFyeSB0LXRlc3QiKSwKICB2b2xjYW5vICsgICAgIAogICAgeGxpbSgtMywzKQosbnJvdz0yKQpgYGAKCi0gVGhlIHZvbGNhbm8gcGxvdCBvcGVucyB1cCB3aGVuIHVzaW5nIHRoZSBFQiB2YXJpYW5jZSBlc3RpbWF0b3IKCi0gIEJvcnJvd2luZyBzdHJlbmd0aCB0byBlc3RpbWF0ZSB0aGUgdmFyaWFuY2UgdXNpbmcgZW1waXJpY2FsIEJheWVzIHNvbHZlcyB0aGUgaXNzdWUgb2YgcmV0dXJuaW5nIHByb3RlaW5zIHdpdGggYSBsb3cgZm9sZCBjaGFuZ2UgYXMgc2lnbmlmaWNhbnQgZHVlIHRvIGEgbG93IHZhcmlhbmNlLiAKCgojIyMgU2hyaW5rYWdlIG9mIHRoZSAgdmFyaWFuY2UgYW5kIG1vZGVyYXRlZCB0LXN0YXRpc3RpY3MKCmBgYHtyfQpxcGxvdCgKICBzYXBwbHkocm93RGF0YShwZVtbInByb3RlaW4iXV0pJG1zcXJvYk1vZGVscyxnZXRTaWdtYSksCiAgc2FwcGx5KHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRtc3Fyb2JNb2RlbHMsZ2V0U2lnbWFQb3N0ZXJpb3IpKSArCiAgeGxhYigiU0QiKSArCiAgeWxhYigibW9kZXJhdGVkIFNEIikgKwogIGdlb21fYWJsaW5lKGludGVyY2VwdCA9IDAsc2xvcGUgPSAxKSArCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gKSAKYGBgCgotIFNtYWxsIHZhcmlhbmNlcyBhcmUgc2hydW5rZW4gdG93YXJkcyB0aGUgY29tbW9uIHZhcmlhbmNlIHJlc3VsdGluZyBpbiBsYXJnZSBFQiB2YXJpYW5jZSBlc3RpbWF0ZXMKLSBMYXJnZSB2YXJpYW5jZXMgYXJlIHNocnVua2VuIHRvd2FyZHMgdGhlIGNvbW1vbiB2YXJpYW5jZSByZXN1bHRpbmcgaW4gc21hbGxlciBFQiB2YXJpYW5jZSBlc3RpbWF0ZXMgCi0gUG9vbGVkIGRlZ3JlZXMgb2YgZnJlZWRvbSBvZiB0aGUgRUIgdmFyaWFuY2UgZXN0aW1hdG9yIGFyZSBsYXJnZXIgYmVjYXVzZSBpbmZvcm1hdGlvbiBpcyBib3Jyb3dlZCBhY3Jvc3MgcHJvdGVpbnMgdG8gZXN0aW1hdGUgdGhlIHZhcmlhbmNlCgojIyBQbG90cyAKCmBgYHtyfQpzaWdOYW1lcyA8LSByb3dEYXRhKHBlW1sicHJvdGVpbiJdXSkkZ2Vub3R5cGVEOCAlPiUKICAgIHJvd25hbWVzX3RvX2NvbHVtbigicHJvdGVpbiIpICU+JQogICAgZmlsdGVyKGFkalB2YWwgPCAwLjA1KSAlPiUKICAgIHB1bGwocHJvdGVpbikKaGVhdG1hcChhc3NheShwZVtbInByb3RlaW4iXV0pW3NpZ05hbWVzLCBdKQpgYGAKCgpgYGB7ciwgd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KZm9yIChwcm90TmFtZSBpbiBzaWdOYW1lcykKICAgIHsKICAgICAgICBwZVBsb3QgPC0gcGVbcHJvdE5hbWUsICwgYygicGVwdGlkZU5vcm0iLCAicHJvdGVpbiIpXQogICAgICAgIHBlUGxvdERmIDwtIGRhdGEuZnJhbWUobG9uZ0Zvcm1hdChwZVBsb3QpKQogICAgICAgIHBlUGxvdERmJGFzc2F5IDwtIGZhY3RvcihwZVBsb3REZiRhc3NheSwKICAgICAgICAgICAgbGV2ZWxzID0gYygicGVwdGlkZU5vcm0iLCAicHJvdGVpbiIpCiAgICAgICAgKQogICAgICAgIHBlUGxvdERmJGdlbm90eXBlIDwtIGFzLmZhY3Rvcihjb2xEYXRhKHBlUGxvdClbcGVQbG90RGYkY29sbmFtZSwgImdlbm90eXBlIl0pCgogICAgICAgICMgcGxvdHRpbmcKICAgICAgICBwMSA8LSBnZ3Bsb3QoCiAgICAgICAgICAgIGRhdGEgPSBwZVBsb3REZiwKICAgICAgICAgICAgYWVzKHggPSBjb2xuYW1lLCB5ID0gdmFsdWUsIGdyb3VwID0gcm93bmFtZSkKICAgICAgICApICsKICAgICAgICAgICAgZ2VvbV9saW5lKCkgKwogICAgICAgICAgICBnZW9tX3BvaW50KCkgKwogICAgICAgICAgICBmYWNldF9ncmlkKH5hc3NheSkgKwogICAgICAgICAgICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDcwLCBoanVzdCA9IDEsIHZqdXN0ID0gMC41KSkgKwogICAgICAgICAgICBnZ3RpdGxlKHByb3ROYW1lKQogICAgICAgIHByaW50KHAxKQoKICAgICAgICAjIHBsb3R0aW5nIDIKICAgICAgICBwMiA8LSBnZ3Bsb3QocGVQbG90RGYsIGFlcyh4ID0gY29sbmFtZSwgeSA9IHZhbHVlLCBmaWxsID0gZ2Vub3R5cGUpKSArCiAgICAgICAgICAgIGdlb21fYm94cGxvdChvdXRsaWVyLnNoYXBlID0gTkEpICsKICAgICAgICAgICAgZ2VvbV9wb2ludCgKICAgICAgICAgICAgICAgIHBvc2l0aW9uID0gcG9zaXRpb25faml0dGVyKHdpZHRoID0gLjEpLAogICAgICAgICAgICAgICAgYWVzKHNoYXBlID0gcm93bmFtZSkKICAgICAgICAgICAgKSArCiAgICAgICAgICAgIHNjYWxlX3NoYXBlX21hbnVhbCh2YWx1ZXMgPSAxOm5yb3cocGVQbG90RGYpKSArCiAgICAgICAgICAgIGxhYnModGl0bGUgPSBwcm90TmFtZSwgeCA9ICJzYW1wbGUiLCB5ID0gInBlcHRpZGUgaW50ZW5zaXR5IChsb2cyKSIpICsKICAgICAgICAgICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA3MCwgaGp1c3QgPSAxLCB2anVzdCA9IDAuNSkpICsKICAgICAgICAgICAgZmFjZXRfZ3JpZCh+YXNzYXkpCiAgICAgICAgcHJpbnQocDIpCn0KYGBgCgojIEV4cGVyaW1lbnRhbCBEZXNpZ24KCiMjIFNhbXBsZSBzaXplIAoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC91RUxna3pEalZSWSIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4KCiQkCiBcbG9nXzIgXHRleHR7RkN9ID0gXGJhcnt5fV97cDF9LVxiYXJ7eX1fe3AyfQokJAoKJCQKVF9nPVxmcmFje1xsb2dfMiBcdGV4dHtGQ319e1x0ZXh0e3NlfV97XGxvZ18yIFx0ZXh0e0ZDfX19CiQkCgokJApUX2c9XGZyYWN7XHdpZGVoYXR7XHRleHR7c2lnbmFsfX19e1x3aWRlaGF0e1x0ZXh0e05vaXNlfX19CiQkCgpJZiB3ZSBjYW4gYXNzdW1lIGVxdWFsIHZhcmlhbmNlIGluIGJvdGggdHJlYXRtZW50IGdyb3VwczoKCiQkClx0ZXh0e3NlfV97XGxvZ18yIFx0ZXh0e0ZDfX09XHRleHR7U0R9XHNxcnR7XGZyYWN7MX17bl8xfStcZnJhY3sxfXtuXzJ9fQokJAoKJFxyaWdodGFycm93JCBpZiBudW1iZXIgb2YgYmlvLXJlcGVhdHMgaW5jcmVhc2VzIHdlIGhhdmUgYSBoaWdoZXIgcG93ZXIhCgotIGNmci4gU3R1ZHkgb2YgdGFtb3hpZmVuIHRyZWF0ZWQgRXN0cm9nZW4gUmVjZXB0ZXIgKEVSKSBwb3NpdGl2ZSBicmVhc3QgY2FuY2VyIHBhdGllbnRzCgojIyBCbG9ja2luZyAKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvRHNSdWljT05yLVEiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgpcW1xzaWdtYV4yPSBcc2lnbWFeMl97YmlvfStcc2lnbWFeMl9cdGV4dHtsYWJ9ICtcc2lnbWFeMl9cdGV4dHtleHRyYWN0aW9ufSArIFxzaWdtYV4yX1x0ZXh0e3J1bn0gKyBcbGRvdHNcXQoKLSBCaW9sb2dpY2FsOiBmbHVjdHVhdGlvbnMgaW4gcHJvdGVpbiBsZXZlbCBiZXR3ZWVuIG1pY2UsIGZsdWN0YXRpb25zIGluIHByb3RlaW4gbGV2ZWwgYmV0d2VlbiBjZWxscywgLi4uCi0gVGVjaG5pY2FsOiBjYWdlIGVmZmVjdCwgbGFiIGVmZmVjdCwgd2VlayBlZmZlY3QsIHBsYXNtYSBleHRyYWN0aW9uLCBNUy1ydW4sIC4uLgoKIyMgTmF0dXJlIG1ldGhvZHM6IFBvaW50cyBvZiBzaWduaWZpY2FuY2UgLSBCbG9ja2luZyAKCltodHRwczovL3d3dy5uYXR1cmUuY29tL2FydGljbGVzL25tZXRoLjMwMDUucGRmXShodHRwczovL3d3dy5uYXR1cmUuY29tL2FydGljbGVzL25tZXRoLjMwMDUucGRmKQoKCiMjIE1vdXNlIGV4YW1wbGUgCgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9IjUwJSJ9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvbW91c2VUY2VsbF9SQ0JfZGVzaWduLnBuZyIpCmBgYApEdWd1ZXQgZXQgYWwuICgyMDE3KSBNQ1AgMTYoOCk6MTQxNi0xNDMyLiBkb2k6IDEwLjEwNzQvbWNwLm0xMTYuMDYyNzQ1CgotIEFsbCB0cmVhdG1lbnRzIG9mIGludGVyZXN0IGFyZSBwcmVzZW50IHdpdGhpbiBibG9jayEKLSBXZSBjYW4gZXN0aW1hdGUgdGhlIGVmZmVjdCBvZiB0aGUgdHJlYXRtZW50IHdpdGhpbiBibG9jayEKLSBXZSBjYW4gaXNvbGF0ZSB0aGUgYmV0d2VlbiBibG9jayB2YXJpYWJpbGl0eSBmcm9tIHRoZSBhbmFseXNpcyB1c2luZyBsaW5lYXIgbW9kZWw6CiQkIAp5IFxzaW0gXHRleHR7dHlwZX0gKyBcdGV4dHttb3VzZX0KJCQKLSBOb3QgcG9zc2libGUgd2l0aCBQZXJzZXVzIQoKIyMjIEFzc2VzcyB0aGUgaW1wYWN0IG9mIGJsb2NraW5nIGluIHRoZSB0dXRvcmlhbCBzZXNzaW9uIQoKLSBDb21wbGV0ZWx5IHJhbmRvbWl6ZWQgZGVzaWduIHdpdGggb25seSBvbmUgY2VsbCB0eXBlIHBlciBtb3VzZSAoVHJlZyBhbmQgVGNvbnYpCgokJFx1cGRvd25hcnJvdyQkCgotIFJhbmRvbWl6ZWQgY29tcGxldGUgYmxvY2sgZGVzaWduIGFzc2Vzc2luZyBUcmVnIGFuZCBUY29udiBvbiBlYWNoIG1vdXNlCgojIFNvZnR3YXJlICYgY29kZQoKLSBPdXIgUi9CaW9jb25kdWN0b3IgcGFja2FnZSBbbXNxcm9iMl0oaHR0cHM6Ly93d3cuYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL2Jpb2MvaHRtbC9tc3Fyb2IyLmh0bWwpIGNhbiBiZSB1c2VkIGluIFIgbWFya2Rvd24gc2NyaXB0cyBvciB3aXRoIGEgR1VJL3NoaW55QXBwIGluIHRoZSBbbXNxcm9iMmd1aV0oaHR0cHM6Ly9naXRodWIuY29tL3N0YXRPbWljcy9tc3Fyb2IyZ3VpKSBwYWNrYWdlLgoKLSBUaGUgR1VJIGlzIGludGVuZGVkIGFzIGEgaW50cm9kdWN0aW9uIHRvIHRoZSBrZXkgY29uY2VwdHMgb2YgcHJvdGVvbWljcyBkYXRhIGFuYWx5c2lzIGZvciB1c2VycyB3aG8gaGF2ZSBubyBleHBlcmllbmNlIGluIFIuIAoKLSBIb3dldmVyLCBsZWFybmluZyBob3cgdG8gY29kZSBkYXRhIGFuYWx5c2VzIGluIFIgbWFya2Rvd24gc2NyaXB0cyBpcyBrZXkgZm9yIG9wZW4gZW4gcmVwcm9kdWNpYmxlIHNjaWVuY2UgYW5kIGZvciByZXBvcnRpbmcgeW91ciBwcm90ZW9taWNzIGRhdGEgYW5hbHlzZXMgYW5kIGludGVycHJldGF0aW9uIGluIGEgcmVwcm9kdWNpYmxlIHdheS4gCgoKLSBNb3JlIGluZm9ybWF0aW9uIG9uIG91ciB0b29scyBjYW4gYmUgZm91bmQgaW4gb3VyIHBhcGVycyBbQGdvZW1pbm5lMjAxNl0sIFtAZ29lbWlubmUyMDIwXSBhbmQgW0BzdGlja2VyMjAyMF0uIFBsZWFzZSByZWZlciB0byBvdXIgd29yayB3aGVuIHVzaW5nIG91ciB0b29scy4gCgotIENsaXBzIG9uIHRoZSBjb2RlIG9uIGltcG9ydGluZyB0aGUgZGF0YSBhbmQgcHJlcHJvY2Vzc2luZyBjYW4gYmUgZm91bmQgaW4gW1BhcnQgSSBQcmVwcm9jZXNzaW5nXSguL3BkYV9xdWFudGlmaWNhdGlvbl9wcmVwcm9jZXNzaW5nLmh0bWwpCgotIEEgY2xpcCBvbiB0aGUgY29kZSBmb3IgbW9kZWxsaW5nIGFuZCBzdGF0aXN0aWNhbCBpbmZlcmVuY2Ugd2l0aCBtc3Fyb2IyIGlzIGluY2x1ZGVkIGJlbG93Cgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL2VYeElkekdPUGdZIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKIyBSZWZlcmVuY2Vz