Creative Commons License

This is part of the online course Experimental Design and Data-Analysis in Label-Free Quantitative LC/MS Proteomics - A Tutorial with msqrob2 (hupo21)

Click to see libraries that are loaded

library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)
library(gridExtra)

1 Intro

  1. Background + cptac study
  2. Sources of variability
  3. Summarization

1.1 MS-based workflow

  • Peptide Characteristics

    • Modifications
    • Ionisation Efficiency: huge variability
    • Identification
      • Misidentification \(\rightarrow\) outliers
      • MS\(^2\) selection on peptide abundance
      • Context depending missingness
      • Non-random missingness

\(\rightarrow\) Unbalanced pepide identifications across samples and messy data

1.2 CPTAC Spike-in Study

  • Same trypsin-digested yeast proteome background in each sample

  • Trypsin-digested Sigma UPS1 standard: 48 different human proteins spiked in at 5 different concentrations (treatment A-E)

  • Samples repeatedly run on different instruments in different labs

  • After MaxQuant search with match between runs option

    • 41% of all proteins are quantified in all samples
    • 6.6% of all peptides are quantified in all samples

\(\rightarrow\) vast amount of missingness

1.2.1 Maxquant output

1.2.2 Read data

Click to see background and code

  1. We use a peptides.txt file from MS-data quantified with maxquant that contains MS1 intensities summarized at the peptide level.
peptidesFile <- "https://raw.githubusercontent.com/statOmics/PDA21/data/quantification/fullCptacDatasSetNotForTutorial/peptides.txt"
  1. Maxquant stores the intensity data for the different samples in columnns that start with Intensity. We can retreive the column names with the intensity data with the code below:
ecols <- grep("Intensity\\.", names(read.delim(peptidesFile)))
  1. Read the data and store it in QFeatures object
pe <- readQFeatures(
  table = peptidesFile,
  fnames = 1,
  ecol = ecols,
  name = "peptideRaw", sep="\t")

1.2.3 Design

Click to see background and code

pe %>% colnames
## CharacterList of length 1
## [["peptideRaw"]] Intensity.6A_1 Intensity.6A_2 ... Intensity.6E_9
  • Note, that the sample names include the spike-in condition.

  • They also end on a number.

    • 1-3 is from lab 1,
    • 4-6 from lab 2 and
    • 7-9 from lab 3.
  • We update the colData with information on the design

colData(pe)$lab <- rep(rep(paste0("lab",1:3),each=3),5) %>% as.factor
colData(pe)$condition <- pe[["peptideRaw"]] %>% colnames %>% substr(12,12) %>% as.factor
colData(pe)$spikeConcentration <- rep(c(A = 0.25, B = 0.74, C = 2.22, D = 6.67, E = 20),each = 9)
  • We explore the colData
colData(pe)
## DataFrame with 45 rows and 3 columns
##                     lab condition spikeConcentration
##                <factor>  <factor>          <numeric>
## Intensity.6A_1     lab1         A               0.25
## Intensity.6A_2     lab1         A               0.25
## Intensity.6A_3     lab1         A               0.25
## Intensity.6A_4     lab2         A               0.25
## Intensity.6A_5     lab2         A               0.25
## ...                 ...       ...                ...
## Intensity.6E_5     lab2         E                 20
## Intensity.6E_6     lab2         E                 20
## Intensity.6E_7     lab3         E                 20
## Intensity.6E_8     lab3         E                 20
## Intensity.6E_9     lab3         E                 20

2 Sources of variation

2.1 Intensities of one peptide

Peptide AALEELVK from spiked-in UPS protein P12081. We only show data from lab1.

Click to see code to make plot

subset <- pe["AALEELVK",colData(pe)$lab=="lab1"]
plotWhyLog <- data.frame(concentration = colData(subset)$spikeConcentration,
           y = assay(subset[["peptideRaw"]]) %>% c
           ) %>% 
  ggplot(aes(concentration, y)) +
  geom_point() +
  xlab("concentration (fmol/l)") +
  ggtitle("peptide AALEELVK in lab1")

plotWhyLog

  • Variance increases with the mean \(\rightarrow\) Multiplicative error structure
Click to see code to make plot

plotLog <- data.frame(concentration = colData(subset)$spikeConcentration,
           y = assay(subset[["peptideRaw"]]) %>% c
           ) %>% 
  ggplot(aes(concentration, y)) +
  geom_point() + 
  scale_x_continuous(trans='log2') + 
  scale_y_continuous(trans='log2') +
  xlab("concentration (fmol/l)") +
  ggtitle("peptide AALEELVK in lab1 with axes on log scale")

plotLog

  • Data seems to be homoscedastic on log-scale \(\rightarrow\) log transformation of the intensity data
  • In quantitative proteomics analysis on \(\log_2\)

\(\rightarrow\) Differences on a \(\log_2\) scale: \(\log_2\) fold changes

\[ \log_2 B - \log_2 A = \log_2 \frac{B}{A} = \log FC_\text{B - A} \] \[ \begin{array} {l} log_2 FC = 1 \rightarrow FC = 2^1 =2\\ log_2 FC = 2 \rightarrow FC = 2^2 = 4\\ \end{array} \]

2.2 Log-transform

Click to see code to log-transfrom the data

  • We calculate how many non zero intensities we have for each peptide and this can be useful for filtering.
rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)
  • Peptides with zero intensities are missing peptides and should be represent with a NA value rather than 0.
pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA
  • Logtransform data with base 2
pe <- logTransform(pe, base = 2, i = "peptideRaw", name = "peptideLog")

2.3 Filtering

Click to see code to filter the data

  1. Handling overlapping protein groups

In our approach a peptide can map to multiple proteins, as long as there is none of these proteins present in a smaller subgroup.

pe <- filterFeatures(pe, ~ Proteins %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins))
  1. Remove reverse sequences (decoys) and contaminants

We now remove the contaminants, peptides that map to decoy sequences, and proteins which were only identified by peptides with modifications.

pe <- filterFeatures(pe,~Reverse != "+")
pe <- filterFeatures(pe,~ Potential.contaminant != "+")
  1. Drop peptides that were only identified in one sample

We keep peptides that were observed at last twice.

pe <- filterFeatures(pe,~ nNonZero >=2)
nrow(pe[["peptideLog"]])
## [1] 10478
We keep 10478 peptides upon filtering.

2.4 Technical Variability

Click to see code for plot

densityConditionD <- pe[["peptideLog"]][,colData(pe)$condition=="D"] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(lab = colData(pe)[sample,"lab"]) %>%
  ggplot(aes(x=intensity,group=sample,color=lab)) + 
    geom_density() +
    ggtitle("condition D")

densityLab2 <- pe[["peptideLog"]][,colData(pe)$lab=="lab2"] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(condition = colData(pe)[sample,"condition"]) %>%
  ggplot(aes(x=intensity,group=sample,color=condition)) + 
    geom_density() +
    ggtitle("lab2")

densityConditionD
## Warning: Removed 39179 rows containing non-finite values (stat_density).

densityLab2
## Warning: Removed 44480 rows containing non-finite values (stat_density).

  • Even in very clean synthetic dataset (same background, only 48 UPS proteins can be different) the marginal peptide intensity distribution across samples can be quite distinct

  • Considerable effects between and within labs for replicate samples

  • Considerable effects between samples with different spike-in concentration

\(\rightarrow\) Normalization is needed


2.5 Normalization

Normalization of the data by median centering

\[y_{ip}^\text{norm} = y_{ip} - \hat\mu_i\] with \(\hat\mu_i\) the median intensity over all observed peptides in sample \(i\).

Click to see R-code to normalize the data

pe <- normalize(pe, 
                i = "peptideLog", 
                name = "peptideNorm", 
                method = "center.median")

Click to see code to make plot

densityConditionDNorm <- pe[["peptideNorm"]][,colData(pe)$condition=="D"] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(lab = colData(pe)[sample,"lab"]) %>%
  ggplot(aes(x=intensity,group=sample,color=lab)) + 
    geom_density() +
    ggtitle("condition D")

densityLab2Norm <- pe[["peptideNorm"]][,colData(pe)$lab=="lab2"] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(condition = colData(pe)[sample,"condition"]) %>%
  ggplot(aes(x=intensity,group=sample,color=condition)) + 
    geom_density() +
    ggtitle("lab2")

densityConditionDNorm
## Warning: Removed 39179 rows containing non-finite values (stat_density).

densityLab2Norm
## Warning: Removed 44480 rows containing non-finite values (stat_density).

2.6 Pseudo replication

Click to see code to make plot

prot <- "P01031ups|CO5_HUMAN_UPS"
data <- pe[["peptideNorm"]][
  rowData(pe[["peptideNorm"]])$Proteins == prot,
  colData(pe)$lab=="lab3"] %>%
  assay %>%
  as.data.frame %>%
  rownames_to_column(var = "peptide") %>%
  gather(sample, intensity, -peptide) %>% 
  mutate(condition = colData(pe)[sample,"condition"]) %>%
  na.exclude
sumPlot <- data %>%
  ggplot(aes(x = peptide, y = intensity, color = condition, group = sample, label = condition), show.legend = FALSE) +
  geom_text(show.legend = FALSE) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1)) +
  xlab("Peptide") + 
  ylab("Intensity (log2)") +
  ggtitle(paste0("protein: ",prot))

sumPlot +
  geom_line(linetype="dashed",alpha=.4)

  • Sources of variability in plot:

    • Between treatment variability
    • Between sample variability
    • Between peptide variability
    • within sample variability
  • Multiple peptides from same protein in a sample

  • Peptide intensities in the same sample are correlated: Pseudo replication

\(\rightarrow\) Summarization!

  • Strong peptide effect
  • Unbalanced peptide identification

2.6.1 Illustration on subset of CPTAC study: A vs B comparison in lab 3

2.6.1.1 LFQ

Click to see background and code

  1. Import data
proteinsFile <- "https://raw.githubusercontent.com/statOmics/PDA21/data/quantification/cptacAvsB_lab3/proteinGroups.txt"

ecols <- grep("LFQ\\.intensity\\.", names(read.delim(proteinsFile)))

peLFQ <- readQFeatures(
  table = proteinsFile, fnames = 1, ecol = ecols,
  name = "proteinRaw", sep = "\t"
)

cond <- which(
  strsplit(colnames(peLFQ)[[1]][1], split = "")[[1]] == "A") # find where condition is stored

colData(peLFQ)$condition <- substr(colnames(peLFQ), cond, cond) %>%
  unlist %>%  
  as.factor
  1. Preprocessing
rowData(peLFQ[["proteinRaw"]])$nNonZero <- rowSums(assay(peLFQ[["proteinRaw"]]) > 0)

peLFQ <- zeroIsNA(peLFQ, "proteinRaw") # convert 0 to NA

peLFQ <- logTransform(peLFQ, base = 2, i = "proteinRaw", name = "proteinLog")

peLFQ <- filterFeatures(peLFQ,~ Reverse != "+")
peLFQ <- filterFeatures(peLFQ,~ Potential.contaminant != "+")

peLFQ <- normalize(peLFQ, 
                i = "proteinLog", 
                name = "protein", 
                method = "center.median")
  1. Modeling and Inference
peLFQ <- msqrob(object = peLFQ, i = "protein", formula = ~condition)

L <- makeContrast("conditionB=0", parameterNames = c("conditionB"))
peLFQ <- hypothesisTest(object = peLFQ, i = "protein", contrast = L)

volcanoLFQ <- ggplot(rowData(peLFQ[["protein"]])$conditionB,
                  aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
  geom_point(cex = 2.5) +
  scale_color_manual(values = alpha(c("black", "red"), 0.5)) + 
  theme_minimal() +
  ggtitle(paste0("maxLFQ: TP = ",sum(rowData(peLFQ[["protein"]])$conditionB$adjPval<0.05&grepl(rownames(rowData(peLFQ[["protein"]])$conditionB),pattern ="UPS"),na.rm=TRUE), " FP = ", sum(rowData(peLFQ[["protein"]])$conditionB$adjPval<0.05&!grepl(rownames(rowData(peLFQ[["protein"]])$conditionB),pattern ="UPS"),na.rm=TRUE)))

2.6.1.2 Median & robust summarization

Click to see background and code

  1. Import Data
peptidesFile <- "https://raw.githubusercontent.com/statOmics/SGA2020/data/quantification/cptacAvsB_lab3/peptides.txt"

ecols <- grep(
  "Intensity\\.", 
  names(read.delim(peptidesFile))
  )

peAB <- readQFeatures(
  table = peptidesFile,
  fnames = 1,
  ecol = ecols,
  name = "peptideRaw", sep="\t")

cond <- which(
  strsplit(colnames(peAB)[[1]][1], split = "")[[1]] == "A") # find where condition is stored

colData(peAB)$condition <- substr(colnames(peAB), cond, cond) %>%
  unlist %>%  
  as.factor
  1. Preprocessing
rowData(peAB[["peptideRaw"]])$nNonZero <- rowSums(assay(peAB[["peptideRaw"]]) > 0)

peAB <- zeroIsNA(peAB, "peptideRaw") # convert 0 to NA

peAB <- logTransform(peAB, base = 2, i = "peptideRaw", name = "peptideLog")

peAB <- filterFeatures(peAB, ~ Proteins %in% smallestUniqueGroups(rowData(peAB[["peptideLog"]])$Proteins))

peAB <- filterFeatures(peAB,~Reverse != "+")
peAB <- filterFeatures(peAB,~ Potential.contaminant != "+")


peAB <- filterFeatures(peAB,~ nNonZero >=2)
nrow(peAB[["peptideLog"]])
## [1] 7011
peAB <- normalize(peAB, 
                i = "peptideLog", 
                name = "peptideNorm", 
                method = "center.median")

peAB <- aggregateFeatures(peAB,
  i = "peptideNorm",
  fcol = "Proteins",
  na.rm = TRUE,
  name = "proteinMedian",
  fun = matrixStats::colMedians)

peAB <- aggregateFeatures(peAB,
  i = "peptideNorm",
  fcol = "Proteins",
  na.rm = TRUE,
  name = "proteinRobust")
  1. Modeling and inference
peAB <- msqrob(object = peAB, i = "proteinMedian", formula = ~condition)
L <- makeContrast("conditionB=0", parameterNames = c("conditionB"))
peAB <- hypothesisTest(object = peAB, i = "proteinMedian", contrast = L)

peAB <- msqrob(object = peAB, i = "proteinRobust", formula = ~condition)
peAB <- hypothesisTest(object = peAB, i = "proteinRobust", contrast = L)

volcanoMedian <- ggplot(rowData(peAB[["proteinMedian"]])$conditionB,
                  aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
  geom_point(cex = 2.5) +
  scale_color_manual(values = alpha(c("black", "red"), 0.5)) + 
  theme_minimal() +
  ggtitle(paste0("Median: TP = ",sum(rowData(peAB[["proteinMedian"]])$conditionB$adjPval<0.05&grepl(rownames(rowData(peAB[["proteinMedian"]])$conditionB),pattern ="UPS"),na.rm=TRUE), " FP = ", sum(rowData(peAB[["proteinMedian"]])$conditionB$adjPval<0.05&!grepl(rownames(rowData(peAB[["proteinMedian"]])$conditionB),pattern ="UPS"),na.rm=TRUE)))

volcanoRobust<- ggplot(rowData(peAB[["proteinRobust"]])$conditionB,
                  aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
  geom_point(cex = 2.5) +
  scale_color_manual(values = alpha(c("black", "red"), 0.5)) + 
  theme_minimal() +
  ggtitle(paste0("Robust: TP = ",sum(rowData(peAB[["proteinRobust"]])$conditionB$adjPval<0.05&grepl(rownames(rowData(peAB[["proteinRobust"]])$conditionB),pattern ="UPS"),na.rm=TRUE), " FP = ", sum(rowData(peAB[["proteinRobust"]])$conditionB$adjPval<0.05&!grepl(rownames(rowData(peAB[["proteinRobust"]])$conditionB),pattern ="UPS"),na.rm=TRUE)))
ylims <- c(0, 
           ceiling(max(c(-log10(rowData(peLFQ[["protein"]])$conditionB$pval),
               -log10(rowData(peAB[["proteinMedian"]])$conditionB$pval),
               -log10(rowData(peAB[["proteinRobust"]])$conditionB$pval)),
               na.rm=TRUE))
)

xlims <- max(abs(c(rowData(peLFQ[["protein"]])$conditionB$logFC,
               rowData(peAB[["proteinMedian"]])$conditionB$logFC,
               rowData(peAB[["proteinRobust"]])$conditionB$logFC)),
               na.rm=TRUE) * c(-1,1)
compBoxPlot <- rbind(rowData(peLFQ[["protein"]])$conditionB %>% mutate(method="maxLFQ") %>% rownames_to_column(var="protein"),
      rowData(peAB[["proteinMedian"]])$conditionB %>% mutate(method="median")%>% rownames_to_column(var="protein"),
      rowData(peAB[["proteinRobust"]])$conditionB%>% mutate(method="robust")%>% rownames_to_column(var="protein")) %>%
      mutate(ups= grepl(protein,pattern="UPS")) %>%
    ggplot(aes(x = method, y = logFC, fill = ups)) +
    geom_boxplot() +
    geom_hline(yintercept = log2(0.74 / .25), color = "#00BFC4") +
    geom_hline(yintercept = 0, color = "#F8766D")

2.6.1.3 Comparison summarization methods

grid.arrange(volcanoLFQ + xlim(xlims) + ylim(ylims), 
             volcanoMedian + xlim(xlims) + ylim(ylims), 
             volcanoRobust + xlim(xlims) + ylim(ylims),
             ncol=1)
## Warning: Removed 746 rows containing missing values (geom_point).
## Warning: Removed 166 rows containing missing values (geom_point).
## Warning: Removed 167 rows containing missing values (geom_point).

  • Robust summarization: highest power and still good FDR control: \(FDP = \frac{1}{20} = 0.05\).
compBoxPlot
## Warning: Removed 1079 rows containing non-finite values (stat_boxplot).

  • Median: biased logFC estimates for spike-in proteins
  • maxLFQ: more variable logFC estiamtes for spike-in proteins

2.6.2 Median summarization

We first evaluate median summarization for protein P01031ups|CO5_HUMAN_UPS.

Click to see code to make plot

dataHlp <- pe[["peptideNorm"]][
    rowData(pe[["peptideNorm"]])$Proteins == prot,
    colData(pe)$lab=="lab3"] %>% assay 

sumMedian <- data.frame(
  intensity= dataHlp
    %>% colMedians(na.rm=TRUE)
  ,
  condition= colnames(dataHlp) %>% substr(12,12) %>% as.factor )

sumMedianPlot <- sumPlot + 
  geom_hline(
    data = sumMedian,
    mapping = aes(yintercept=intensity,color=condition)) + 
  ggtitle("Median summarization")

sumMedianPlot
## Warning: Removed 1 rows containing missing values (geom_hline).

  • The sample medians are not a good estimate for the protein expression value.
  • Indeed, they do not account for differences in peptide effects
  • Peptides that ionize poorly are also picked up in samples with high spike-in concencentration and not in samples with low spike-in concentration
  • This introduces a bias.

2.6.3 Mean summarization

\[ y_{ip} = \beta_i^\text{sample} + \epsilon_{ip} \]
Click to see code to make plot

sumMeanMod <- lm(intensity ~ -1 + sample,data)

sumMean <- data.frame(
  intensity=sumMeanMod$coef[grep("sample",names(sumMeanMod$coef))],
  condition= names(sumMeanMod$coef)[grep("sample",names(sumMeanMod$coef))] %>% substr(18,18) %>% as.factor )



sumMeanPlot <- sumPlot + geom_hline(
    data = sumMean,
    mapping = aes(yintercept=intensity,color=condition)) +
    ggtitle("Mean summarization")

grid.arrange(sumMedianPlot, sumMeanPlot, ncol=2)
## Warning: Removed 1 rows containing missing values (geom_hline).

2.6.4 Model based summarization

We can use a linear peptide-level model to estimate the protein expression value while correcting for the peptide effect, i.e. 

\[ y_{ip} = \beta_i^\text{sample}+\beta^{peptide}_{p} + \epsilon_{ip} \]

Click to see code to make plot

sumMeanPepMod <- lm(intensity ~ -1 + sample + peptide,data)

sumMeanPep <- data.frame(
  intensity=sumMeanPepMod$coef[grep("sample",names(sumMeanPepMod$coef))] + mean(data$intensity) - mean(sumMeanPepMod$coef[grep("sample",names(sumMeanPepMod$coef))]),
  condition= names(sumMeanPepMod$coef)[grep("sample",names(sumMeanPepMod$coef))] %>% substr(18,18) %>% as.factor )


fitLmPlot <-  sumPlot + geom_line(
    data = data %>% mutate(fit=sumMeanPepMod$fitted.values),
    mapping = aes(x=peptide, y=fit,color=condition, group=sample)) +
    ggtitle("fit: ~ sample + peptide")
sumLmPlot <- sumPlot + geom_hline(
    data = sumMeanPep,
    mapping = aes(yintercept=intensity,color=condition)) +
    ggtitle("Summarization: sample effect")

grid.arrange(sumMedianPlot, sumMeanPlot, sumLmPlot, nrow=1)
## Warning: Removed 1 rows containing missing values (geom_hline).

  • By correcting for the peptide species the protein expression values are much better separated an better reflect differences in abundance induced by the spike-in condition.

  • Indeed, it shows that median and mean summarization that do not account for the peptide effect indeed overestimate the protein expression value in the small spike-in conditions and underestimate that in the large spike-in conditions.

  • Still there seem to be some issues with samples that for which the expression values are not well separated according to the spike-in condition.

A residual analysis clearly indicates potential issues:

Click to see code to make plot

resPlot <- data %>% 
  mutate(res=sumMeanPepMod$residuals) %>%
  ggplot(aes(x = peptide, y = res, color = condition, label = condition), show.legend = FALSE) +
  geom_point(shape=21) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1)) +
  xlab("Peptide") + 
  ylab("residual") +
  ggtitle("residuals: ~ sample + peptide")

grid.arrange(fitLmPlot, resPlot, nrow = 1)

grid.arrange(fitLmPlot, sumLmPlot, nrow = 1)

  • The residual plot shows some large outliers for peptide KIEEIAAK.
  • Indeed, in the original plot the intensities for this peptide do not seem to line up very well with the concentration.
  • This induces a bias in the summarization for some of the samples (e.g. for D and E)

2.6.5 Robust summarization using a peptide-level linear model

\[ y_{ip} = \beta_i^\text{sample}+\beta^{peptide}_{p} + \epsilon_{ip} \]

  • Ordinary least squares: estimate \(\beta\) that minimizes \[ \text{OLS}: \sum\limits_{i,p} \epsilon_{ip}^2 = \sum\limits_{i,p}(y_{ip}-\beta_i^\text{sample}-\beta_p^\text{peptide})^2 \]

We replace OLS by M-estimation with loss function \[ \sum\limits_{i,p} w_{ip}\epsilon_{ip}^2 = \sum\limits_{i,p}w_{ip}(y_{ip}-\beta_i^\text{sample}-\beta_p^\text{peptide})^2 \]

  • Iteratively fit model with observation weights \(w_{ip}\) until convergence
  • The weights are calculated based on standardized residuals
Click to see code to make plot

sumMeanPepRobMod <- MASS::rlm(intensity ~ -1 + sample + peptide,data)
resRobPlot <- data %>%
  mutate(res = sumMeanPepRobMod$residuals,
         w = sumMeanPepRobMod$w) %>%
  ggplot(aes(x = peptide, y = res, color = condition, label = condition,size=w), show.legend = FALSE) +
  geom_point(shape=21,size=.2) +
  geom_point(shape=21) +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1)) +
  xlab("Peptide") + 
  ylab("residual") + 
  ylim(c(-1,1)*max(abs(sumMeanPepRobMod$residuals)))
weightPlot <- qplot(
  seq(-5,5,.01), 
  MASS::psi.huber(seq(-5,5,.01)),
  geom="path") +
  xlab("standardized residual") +
  ylab("weight")

grid.arrange(weightPlot,resRobPlot,nrow=1)

  • We clearly see that the weights in the M-estimation procedure will down-weight errors associated with outliers for peptide KIEEIAAK.
Click to see code to make plot

sumMeanPepRob <- data.frame(
  intensity=sumMeanPepRobMod$coef[grep("sample",names(sumMeanPepRobMod$coef))] + mean(data$intensity) - mean(sumMeanPepRobMod$coef[grep("sample",names(sumMeanPepRobMod$coef))]),
  condition= names(sumMeanPepRobMod$coef)[grep("sample",names(sumMeanPepRobMod$coef))] %>% substr(18,18) %>% as.factor )

sumRlmPlot <- sumPlot + geom_hline(
    data=sumMeanPepRob,
    mapping=aes(yintercept=intensity,color=condition)) + 
    ggtitle("Robust")

 grid.arrange(sumLmPlot + ggtitle("OLS"), sumRlmPlot, nrow = 1)

  • Robust regresion results in a better separation between the protein expression values for the different samples according to their spike-in concentration.

2.6.6 Comparison summarization methods

  • maxLFQ

  • MS-stats also uses a robust peptide level model to perform the summarization, however, they typically first impute missing values

  • Proteus high-flyer method: mean of three peptides with highest intensity

References

Sticker, A., L. Goeminne, L. Martens, and L. Clement. 2020. “Robust Summarization and Inference in Proteome-wide Label-free Quantification.” Mol Cell Proteomics 19 (7): 1209–19.

LS0tCnRpdGxlOiAiIFNvdXJjZXMgb2YgdmFyaWFiaWxpdHkgaW4gbGFiZWwtZnJlZSBwcm90ZW9taWNzIGV4cGVyaW1lbnRzIgphdXRob3I6ICJMaWV2ZW4gQ2xlbWVudCIKZGF0ZTogIltzdGF0T21pY3NdKGh0dHBzOi8vc3RhdG9taWNzLmdpdGh1Yi5pbyksIEdoZW50IFVuaXZlcnNpdHkiCm91dHB1dDoKICAgIGh0bWxfZG9jdW1lbnQ6CiAgICAgIGNvZGVfZG93bmxvYWQ6IHRydWUKICAgICAgdGhlbWU6IGNvc21vCiAgICAgIHRvYzogdHJ1ZQogICAgICB0b2NfZmxvYXQ6IHRydWUKICAgICAgaGlnaGxpZ2h0OiB0YW5nbwogICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIHBkZl9kb2N1bWVudDoKICAgICAgdG9jOiB0cnVlCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQpsaW5rY29sb3I6IGJsdWUKdXJsY29sb3I6IGJsdWUKY2l0ZWNvbG9yOiBibHVlCgpiaWJsaW9ncmFwaHk6IG1zcXJvYjIuYmliCiAgICAgIAotLS0KCjxhIHJlbD0ibGljZW5zZSIgaHJlZj0iaHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LW5jLXNhLzQuMCI+PGltZyBhbHQ9IkNyZWF0aXZlIENvbW1vbnMgTGljZW5zZSIgc3R5bGU9ImJvcmRlci13aWR0aDowIiBzcmM9Imh0dHBzOi8vaS5jcmVhdGl2ZWNvbW1vbnMub3JnL2wvYnktbmMtc2EvNC4wLzg4eDMxLnBuZyIgLz48L2E+CgpUaGlzIGlzIHBhcnQgb2YgdGhlIG9ubGluZSBjb3Vyc2UgW0V4cGVyaW1lbnRhbCBEZXNpZ24gYW5kIERhdGEtQW5hbHlzaXMgaW4gTGFiZWwtRnJlZSBRdWFudGl0YXRpdmUgTEMvTVMgUHJvdGVvbWljcyAtIEEgVHV0b3JpYWwgd2l0aCBtc3Fyb2IyIChodXBvMjEpXShodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8vaHVwbzIxLykKCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL2dHN2ZNZ0ZPeHNjIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBsaWJyYXJpZXMgdGhhdCBhcmUgbG9hZGVkIDwvc3VtbWFyeT48cD4KYGBge3IsIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KGxpbW1hKQpsaWJyYXJ5KFFGZWF0dXJlcykKbGlicmFyeShtc3Fyb2IyKQpsaWJyYXJ5KHBsb3RseSkKbGlicmFyeShncmlkRXh0cmEpCmBgYAo8L3A+PC9kZXRhaWxzPgoKCiMgSW50cm8KCjEuIEJhY2tncm91bmQgKyBjcHRhYyBzdHVkeQoyLiBTb3VyY2VzIG9mIHZhcmlhYmlsaXR5CjMuIFN1bW1hcml6YXRpb24gCgoKIyMgTVMtYmFzZWQgd29ya2Zsb3cKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvMTNncGVsM0cyMnciCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL1Byb3Rlb21pY3NXb3JrZmxvdy5wbmciKQpgYGAKCi0gUGVwdGlkZSBDaGFyYWN0ZXJpc3RpY3MKICAKICAtIE1vZGlmaWNhdGlvbnMKICAtIElvbmlzYXRpb24gRWZmaWNpZW5jeTogaHVnZSB2YXJpYWJpbGl0eQogIC0gSWRlbnRpZmljYXRpb24KICAgIC0gTWlzaWRlbnRpZmljYXRpb24gJFxyaWdodGFycm93JCBvdXRsaWVycwogICAgLSBNUyReMiQgc2VsZWN0aW9uIG9uIHBlcHRpZGUgYWJ1bmRhbmNlCiAgICAtIENvbnRleHQgZGVwZW5kaW5nIG1pc3NpbmduZXNzCiAgICAtIE5vbi1yYW5kb20gbWlzc2luZ25lc3MKCiRccmlnaHRhcnJvdyQgVW5iYWxhbmNlZCBwZXBpZGUgaWRlbnRpZmljYXRpb25zIGFjcm9zcyBzYW1wbGVzIGFuZCBtZXNzeSBkYXRhCgojIyBDUFRBQyBTcGlrZS1pbiBTdHVkeQoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC9iT3F5SHlOd1FFNCIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4KCmBgYHtyIGVjaG89RkFMU0UsIG91dC53aWR0aD0iNTAlIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoIi4vZmlndXJlcy9jcHRhY0xheW91dEx1ZGdlci5wbmciKQpgYGAKCi0gU2FtZSB0cnlwc2luLWRpZ2VzdGVkIHllYXN0IHByb3Rlb21lIGJhY2tncm91bmQgaW4gZWFjaCBzYW1wbGUKLSBUcnlwc2luLWRpZ2VzdGVkIFNpZ21hIFVQUzEgc3RhbmRhcmQ6IDQ4IGRpZmZlcmVudCBodW1hbiBwcm90ZWlucyBzcGlrZWQgaW4gYXQgNSBkaWZmZXJlbnQgY29uY2VudHJhdGlvbnMgKHRyZWF0bWVudCBBLUUpIAotIFNhbXBsZXMgcmVwZWF0ZWRseSBydW4gb24gZGlmZmVyZW50IGluc3RydW1lbnRzIGluIGRpZmZlcmVudCBsYWJzCi0gQWZ0ZXIgTWF4UXVhbnQgc2VhcmNoIHdpdGggbWF0Y2ggYmV0d2VlbiBydW5zIG9wdGlvbgoKICAtIDQxXCUgb2YgYWxsIHByb3RlaW5zIGFyZSBxdWFudGlmaWVkIGluIGFsbCBzYW1wbGVzCiAgLSA2LjZcJSBvZiBhbGwgcGVwdGlkZXMgYXJlIHF1YW50aWZpZWQgaW4gYWxsIHNhbXBsZXMKCiRccmlnaHRhcnJvdyQgdmFzdCBhbW91bnQgb2YgbWlzc2luZ25lc3MKCgojIyMgTWF4cXVhbnQgb3V0cHV0CgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL21heHF1YW50T3V0cHV0RGlyLnBuZyIpCmBgYAoKIyMjIFJlYWQgZGF0YSAKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgYmFja2dyb3VuZCBhbmQgY29kZSA8L3N1bW1hcnk+PHA+CjEuIFdlIHVzZSBhIHBlcHRpZGVzLnR4dCBmaWxlIGZyb20gTVMtZGF0YSBxdWFudGlmaWVkIHdpdGggbWF4cXVhbnQgdGhhdCAKY29udGFpbnMgTVMxIGludGVuc2l0aWVzIHN1bW1hcml6ZWQgYXQgdGhlIHBlcHRpZGUgbGV2ZWwuIApgYGB7cn0KcGVwdGlkZXNGaWxlIDwtICJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1BEQTIxL2RhdGEvcXVhbnRpZmljYXRpb24vZnVsbENwdGFjRGF0YXNTZXROb3RGb3JUdXRvcmlhbC9wZXB0aWRlcy50eHQiCmBgYAoKMi4gTWF4cXVhbnQgc3RvcmVzIHRoZSBpbnRlbnNpdHkgZGF0YSBmb3IgdGhlIGRpZmZlcmVudCBzYW1wbGVzIGluIGNvbHVtbm5zIHRoYXQgc3RhcnQgd2l0aCBJbnRlbnNpdHkuIFdlIGNhbiByZXRyZWl2ZSB0aGUgY29sdW1uIG5hbWVzIHdpdGggdGhlIGludGVuc2l0eSBkYXRhIHdpdGggdGhlIGNvZGUgYmVsb3c6IAoKYGBge3J9CmVjb2xzIDwtIGdyZXAoIkludGVuc2l0eVxcLiIsIG5hbWVzKHJlYWQuZGVsaW0ocGVwdGlkZXNGaWxlKSkpCmBgYAoKMy4gUmVhZCB0aGUgZGF0YSBhbmQgc3RvcmUgaXQgaW4gIFFGZWF0dXJlcyBvYmplY3QgCgpgYGB7cn0KcGUgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHBlcHRpZGVzRmlsZSwKICBmbmFtZXMgPSAxLAogIGVjb2wgPSBlY29scywKICBuYW1lID0gInBlcHRpZGVSYXciLCBzZXA9Ilx0IikKYGBgCjwvcD48L2RldGFpbHM+CgojIyMgRGVzaWduCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGJhY2tncm91bmQgYW5kIGNvZGUgPC9zdW1tYXJ5PjxwPgoKYGBge3J9IApwZSAlPiUgY29sbmFtZXMKYGBgCgotIE5vdGUsIHRoYXQgdGhlIHNhbXBsZSBuYW1lcyBpbmNsdWRlIHRoZSBzcGlrZS1pbiBjb25kaXRpb24uIAotIFRoZXkgYWxzbyBlbmQgb24gYSBudW1iZXIuIAogIAogIC0gMS0zIGlzIGZyb20gbGFiIDEsIAogIC0gNC02IGZyb20gbGFiIDIgYW5kIAogIC0gNy05IGZyb20gbGFiIDMuIAoKLSBXZSB1cGRhdGUgdGhlIGNvbERhdGEgd2l0aCBpbmZvcm1hdGlvbiBvbiB0aGUgZGVzaWduCgpgYGB7cn0KY29sRGF0YShwZSkkbGFiIDwtIHJlcChyZXAocGFzdGUwKCJsYWIiLDE6MyksZWFjaD0zKSw1KSAlPiUgYXMuZmFjdG9yCmNvbERhdGEocGUpJGNvbmRpdGlvbiA8LSBwZVtbInBlcHRpZGVSYXciXV0gJT4lIGNvbG5hbWVzICU+JSBzdWJzdHIoMTIsMTIpICU+JSBhcy5mYWN0b3IKY29sRGF0YShwZSkkc3Bpa2VDb25jZW50cmF0aW9uIDwtIHJlcChjKEEgPSAwLjI1LCBCID0gMC43NCwgQyA9IDIuMjIsIEQgPSA2LjY3LCBFID0gMjApLGVhY2ggPSA5KQpgYGAKCi0gV2UgZXhwbG9yZSB0aGUgY29sRGF0YQoKYGBge3J9CmNvbERhdGEocGUpCmBgYAoKPC9wPjwvZGV0YWlscz4KCiMgU291cmNlcyBvZiB2YXJpYXRpb24gCgojIyBJbnRlbnNpdGllcyBvZiBvbmUgcGVwdGlkZSAKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvZEgyOFFVMzVCekUiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgpQZXB0aWRlIEFBTEVFTFZLIGZyb20gc3Bpa2VkLWluIFVQUyBwcm90ZWluIFAxMjA4MS4gCldlIG9ubHkgc2hvdyBkYXRhIGZyb20gbGFiMS4KCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBtYWtlIHBsb3QgPC9zdW1tYXJ5PjxwPgpgYGB7cn0Kc3Vic2V0IDwtIHBlWyJBQUxFRUxWSyIsY29sRGF0YShwZSkkbGFiPT0ibGFiMSJdCnBsb3RXaHlMb2cgPC0gZGF0YS5mcmFtZShjb25jZW50cmF0aW9uID0gY29sRGF0YShzdWJzZXQpJHNwaWtlQ29uY2VudHJhdGlvbiwKICAgICAgICAgICB5ID0gYXNzYXkoc3Vic2V0W1sicGVwdGlkZVJhdyJdXSkgJT4lIGMKICAgICAgICAgICApICU+JSAKICBnZ3Bsb3QoYWVzKGNvbmNlbnRyYXRpb24sIHkpKSArCiAgZ2VvbV9wb2ludCgpICsKICB4bGFiKCJjb25jZW50cmF0aW9uIChmbW9sL2wpIikgKwogIGdndGl0bGUoInBlcHRpZGUgQUFMRUVMVksgaW4gbGFiMSIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CnBsb3RXaHlMb2cKYGBgCgotIFZhcmlhbmNlIGluY3JlYXNlcyB3aXRoIHRoZSBtZWFuCiRccmlnaHRhcnJvdyQgTXVsdGlwbGljYXRpdmUgZXJyb3Igc3RydWN0dXJlIAoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIHRvIG1ha2UgcGxvdCA8L3N1bW1hcnk+PHA+CmBgYHtyfQpwbG90TG9nIDwtIGRhdGEuZnJhbWUoY29uY2VudHJhdGlvbiA9IGNvbERhdGEoc3Vic2V0KSRzcGlrZUNvbmNlbnRyYXRpb24sCiAgICAgICAgICAgeSA9IGFzc2F5KHN1YnNldFtbInBlcHRpZGVSYXciXV0pICU+JSBjCiAgICAgICAgICAgKSAlPiUgCiAgZ2dwbG90KGFlcyhjb25jZW50cmF0aW9uLCB5KSkgKwogIGdlb21fcG9pbnQoKSArIAogIHNjYWxlX3hfY29udGludW91cyh0cmFucz0nbG9nMicpICsgCiAgc2NhbGVfeV9jb250aW51b3VzKHRyYW5zPSdsb2cyJykgKwogIHhsYWIoImNvbmNlbnRyYXRpb24gKGZtb2wvbCkiKSArCiAgZ2d0aXRsZSgicGVwdGlkZSBBQUxFRUxWSyBpbiBsYWIxIHdpdGggYXhlcyBvbiBsb2cgc2NhbGUiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpwbG90TG9nCmBgYAoKLSBEYXRhIHNlZW1zIHRvIGJlIGhvbW9zY2VkYXN0aWMgb24gbG9nLXNjYWxlICRccmlnaHRhcnJvdyQgbG9nIHRyYW5zZm9ybWF0aW9uIG9mIHRoZSBpbnRlbnNpdHkgZGF0YQotIEluIHF1YW50aXRhdGl2ZSBwcm90ZW9taWNzIGFuYWx5c2lzIG9uICRcbG9nXzIkIAoKJFxyaWdodGFycm93JCBEaWZmZXJlbmNlcyBvbiBhICRcbG9nXzIkIHNjYWxlOiAkXGxvZ18yJCBmb2xkIGNoYW5nZXMKCiQkClxsb2dfMiBCIC0gXGxvZ18yIEEgPSBcbG9nXzIgXGZyYWN7Qn17QX0gPSBcbG9nIEZDX1x0ZXh0e0IgLSBBfQokJAokJCAKXGJlZ2lue2FycmF5fSB7bH0KbG9nXzIgRkMgPSAxIFxyaWdodGFycm93IEZDID0gMl4xID0yXFwKbG9nXzIgRkMgPSAyIFxyaWdodGFycm93IEZDID0gMl4yID0gNFxcClxlbmR7YXJyYXl9CiQkCgojIyBMb2ctdHJhbnNmb3JtCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbG9nLXRyYW5zZnJvbSB0aGUgZGF0YSA8L3N1bW1hcnk+PHA+Ci0gV2UgY2FsY3VsYXRlIGhvdyBtYW55IG5vbiB6ZXJvIGludGVuc2l0aWVzIHdlIGhhdmUgZm9yIGVhY2ggcGVwdGlkZSBhbmQgdGhpcyBjYW4gYmUgdXNlZnVsIGZvciBmaWx0ZXJpbmcuCgpgYGB7cn0Kcm93RGF0YShwZVtbInBlcHRpZGVSYXciXV0pJG5Ob25aZXJvIDwtIHJvd1N1bXMoYXNzYXkocGVbWyJwZXB0aWRlUmF3Il1dKSA+IDApCmBgYAoKCi0gUGVwdGlkZXMgd2l0aCB6ZXJvIGludGVuc2l0aWVzIGFyZSBtaXNzaW5nIHBlcHRpZGVzIGFuZCBzaG91bGQgYmUgcmVwcmVzZW50CndpdGggYSBgTkFgIHZhbHVlIHJhdGhlciB0aGFuIGAwYC4KCmBgYHtyfQpwZSA8LSB6ZXJvSXNOQShwZSwgInBlcHRpZGVSYXciKSAjIGNvbnZlcnQgMCB0byBOQQpgYGAKCi0gTG9ndHJhbnNmb3JtIGRhdGEgd2l0aCBiYXNlIDIKCmBgYHtyfQpwZSA8LSBsb2dUcmFuc2Zvcm0ocGUsIGJhc2UgPSAyLCBpID0gInBlcHRpZGVSYXciLCBuYW1lID0gInBlcHRpZGVMb2ciKQpgYGAKPC9wPjwvZGV0YWlscz4KCgojIyBGaWx0ZXJpbmcKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBmaWx0ZXIgdGhlIGRhdGEgPC9zdW1tYXJ5PjxwPgoKMS4gSGFuZGxpbmcgb3ZlcmxhcHBpbmcgcHJvdGVpbiBncm91cHMKCkluIG91ciBhcHByb2FjaCBhIHBlcHRpZGUgY2FuIG1hcCB0byBtdWx0aXBsZSBwcm90ZWlucywgYXMgbG9uZyBhcyB0aGVyZSBpcwpub25lIG9mIHRoZXNlIHByb3RlaW5zIHByZXNlbnQgaW4gYSBzbWFsbGVyIHN1Ymdyb3VwLgoKYGBge3J9CnBlIDwtIGZpbHRlckZlYXR1cmVzKHBlLCB+IFByb3RlaW5zICVpbiUgc21hbGxlc3RVbmlxdWVHcm91cHMocm93RGF0YShwZVtbInBlcHRpZGVMb2ciXV0pJFByb3RlaW5zKSkKYGBgCgoyLiBSZW1vdmUgcmV2ZXJzZSBzZXF1ZW5jZXMgKGRlY295cykgYW5kIGNvbnRhbWluYW50cwoKV2Ugbm93IHJlbW92ZSB0aGUgY29udGFtaW5hbnRzLCBwZXB0aWRlcyB0aGF0IG1hcCB0byBkZWNveSBzZXF1ZW5jZXMsIGFuZCBwcm90ZWlucwp3aGljaCB3ZXJlIG9ubHkgaWRlbnRpZmllZCBieSBwZXB0aWRlcyB3aXRoIG1vZGlmaWNhdGlvbnMuCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsflJldmVyc2UgIT0gIisiKQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IFBvdGVudGlhbC5jb250YW1pbmFudCAhPSAiKyIpCmBgYAoKMy4gRHJvcCBwZXB0aWRlcyB0aGF0IHdlcmUgb25seSBpZGVudGlmaWVkIGluIG9uZSBzYW1wbGUKCldlIGtlZXAgcGVwdGlkZXMgdGhhdCB3ZXJlIG9ic2VydmVkIGF0IGxhc3QgdHdpY2UuCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsfiBuTm9uWmVybyA+PTIpCm5yb3cocGVbWyJwZXB0aWRlTG9nIl1dKQpgYGAKCldlIGtlZXAgYHIgbnJvdyhwZVtbInBlcHRpZGVMb2ciXV0pYCBwZXB0aWRlcyB1cG9uIGZpbHRlcmluZy4KPC9wPjwvZGV0YWlscz4KCiMjIFRlY2huaWNhbCBWYXJpYWJpbGl0eQoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC95U2ZtOF85TEVMZyIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4KCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSBmb3IgcGxvdCA8L3N1bW1hcnk+PHA+CmBgYHtyfQpkZW5zaXR5Q29uZGl0aW9uRCA8LSBwZVtbInBlcHRpZGVMb2ciXV1bLGNvbERhdGEocGUpJGNvbmRpdGlvbj09IkQiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShsYWIgPSBjb2xEYXRhKHBlKVtzYW1wbGUsImxhYiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1sYWIpKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgiY29uZGl0aW9uIEQiKQoKZGVuc2l0eUxhYjIgPC0gcGVbWyJwZXB0aWRlTG9nIl1dWyxjb2xEYXRhKHBlKSRsYWI9PSJsYWIyIl0gJT4lIAogIGFzc2F5ICU+JQogIGFzLmRhdGEuZnJhbWUoKSAlPiUKICBnYXRoZXIoc2FtcGxlLCBpbnRlbnNpdHkpICU+JSAKICBtdXRhdGUoY29uZGl0aW9uID0gY29sRGF0YShwZSlbc2FtcGxlLCJjb25kaXRpb24iXSkgJT4lCiAgZ2dwbG90KGFlcyh4PWludGVuc2l0eSxncm91cD1zYW1wbGUsY29sb3I9Y29uZGl0aW9uKSkgKyAKICAgIGdlb21fZGVuc2l0eSgpICsKICAgIGdndGl0bGUoImxhYjIiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpkZW5zaXR5Q29uZGl0aW9uRApgYGAKCmBgYHtyfQpkZW5zaXR5TGFiMgpgYGAKCi0gRXZlbiBpbiB2ZXJ5IGNsZWFuIHN5bnRoZXRpYyBkYXRhc2V0IChzYW1lIGJhY2tncm91bmQsIG9ubHkgNDggVVBTCnByb3RlaW5zIGNhbiBiZSBkaWZmZXJlbnQpIHRoZSBtYXJnaW5hbCBwZXB0aWRlIGludGVuc2l0eSBkaXN0cmlidXRpb24KYWNyb3NzIHNhbXBsZXMgY2FuIGJlIHF1aXRlIGRpc3RpbmN0CgotIENvbnNpZGVyYWJsZSBlZmZlY3RzIGJldHdlZW4gYW5kIHdpdGhpbiBsYWJzIGZvciByZXBsaWNhdGUgc2FtcGxlcwotIENvbnNpZGVyYWJsZSBlZmZlY3RzIGJldHdlZW4gc2FtcGxlcyB3aXRoIGRpZmZlcmVudCBzcGlrZS1pbgpjb25jZW50cmF0aW9uCgokXHJpZ2h0YXJyb3ckIE5vcm1hbGl6YXRpb24gaXMgbmVlZGVkCgotLS0KCiMjIE5vcm1hbGl6YXRpb24gCgpOb3JtYWxpemF0aW9uIG9mIHRoZSBkYXRhIGJ5IG1lZGlhbiBjZW50ZXJpbmcKCiQkeV97aXB9Xlx0ZXh0e25vcm19ID0geV97aXB9IC0gXGhhdFxtdV9pJCQgCndpdGggJFxoYXRcbXVfaSQgdGhlIG1lZGlhbiBpbnRlbnNpdHkgb3ZlciBhbGwgb2JzZXJ2ZWQgcGVwdGlkZXMgaW4gc2FtcGxlICRpJC4KCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgUi1jb2RlIHRvIG5vcm1hbGl6ZSB0aGUgZGF0YSA8L3N1bW1hcnk+PHA+CmBgYHtyfQpwZSA8LSBub3JtYWxpemUocGUsIAogICAgICAgICAgICAgICAgaSA9ICJwZXB0aWRlTG9nIiwgCiAgICAgICAgICAgICAgICBuYW1lID0gInBlcHRpZGVOb3JtIiwgCiAgICAgICAgICAgICAgICBtZXRob2QgPSAiY2VudGVyLm1lZGlhbiIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBtYWtlIHBsb3QgPC9zdW1tYXJ5PjxwPgpgYGB7cn0KZGVuc2l0eUNvbmRpdGlvbkROb3JtIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bLGNvbERhdGEocGUpJGNvbmRpdGlvbj09IkQiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShsYWIgPSBjb2xEYXRhKHBlKVtzYW1wbGUsImxhYiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1sYWIpKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgiY29uZGl0aW9uIEQiKQoKZGVuc2l0eUxhYjJOb3JtIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bLGNvbERhdGEocGUpJGxhYj09ImxhYjIiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShjb25kaXRpb24gPSBjb2xEYXRhKHBlKVtzYW1wbGUsImNvbmRpdGlvbiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1jb25kaXRpb24pKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgibGFiMiIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CmRlbnNpdHlDb25kaXRpb25ETm9ybQpgYGAKCmBgYHtyfQpkZW5zaXR5TGFiMk5vcm0KYGBgCgoKIyMgUHNldWRvIHJlcGxpY2F0aW9uCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkLy12cDdFQmF1cjdzIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIHRvIG1ha2UgcGxvdCA8L3N1bW1hcnk+PHA+CmBgYHtyIHBsb3QgPSBGQUxTRX0KCnByb3QgPC0gIlAwMTAzMXVwc3xDTzVfSFVNQU5fVVBTIgpkYXRhIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bCiAgcm93RGF0YShwZVtbInBlcHRpZGVOb3JtIl1dKSRQcm90ZWlucyA9PSBwcm90LAogIGNvbERhdGEocGUpJGxhYj09ImxhYjMiXSAlPiUKICBhc3NheSAlPiUKICBhcy5kYXRhLmZyYW1lICU+JQogIHJvd25hbWVzX3RvX2NvbHVtbih2YXIgPSAicGVwdGlkZSIpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSwgLXBlcHRpZGUpICU+JSAKICBtdXRhdGUoY29uZGl0aW9uID0gY29sRGF0YShwZSlbc2FtcGxlLCJjb25kaXRpb24iXSkgJT4lCiAgbmEuZXhjbHVkZQpzdW1QbG90IDwtIGRhdGEgJT4lCiAgZ2dwbG90KGFlcyh4ID0gcGVwdGlkZSwgeSA9IGludGVuc2l0eSwgY29sb3IgPSBjb25kaXRpb24sIGdyb3VwID0gc2FtcGxlLCBsYWJlbCA9IGNvbmRpdGlvbiksIHNob3cubGVnZW5kID0gRkFMU0UpICsKICBnZW9tX3RleHQoc2hvdy5sZWdlbmQgPSBGQUxTRSkgKwogIHRoZW1lX21pbmltYWwoKSArCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA5MCwgdmp1c3QgPSAwLjUsIGhqdXN0ID0gMSkpICsKICB4bGFiKCJQZXB0aWRlIikgKyAKICB5bGFiKCJJbnRlbnNpdHkgKGxvZzIpIikgKwogIGdndGl0bGUocGFzdGUwKCJwcm90ZWluOiAiLHByb3QpKQpgYGAKPC9wPjwvZGV0YWlscz4KCgpgYGB7cn0Kc3VtUGxvdCArCiAgZ2VvbV9saW5lKGxpbmV0eXBlPSJkYXNoZWQiLGFscGhhPS40KQpgYGAKCi0gU291cmNlcyBvZiB2YXJpYWJpbGl0eSBpbiBwbG90OgoKICAtIEJldHdlZW4gdHJlYXRtZW50IHZhcmlhYmlsaXR5CiAgLSBCZXR3ZWVuIHNhbXBsZSB2YXJpYWJpbGl0eQogIC0gQmV0d2VlbiBwZXB0aWRlIHZhcmlhYmlsaXR5IAogIC0gd2l0aGluIHNhbXBsZSB2YXJpYWJpbGl0eQoKLSBNdWx0aXBsZSBwZXB0aWRlcyBmcm9tIHNhbWUgcHJvdGVpbiBpbiBhIHNhbXBsZQotIFBlcHRpZGUgaW50ZW5zaXRpZXMgaW4gdGhlIHNhbWUgc2FtcGxlIGFyZSBjb3JyZWxhdGVkOiBQc2V1ZG8gcmVwbGljYXRpb24KCiRccmlnaHRhcnJvdyQgU3VtbWFyaXphdGlvbiEgCgotIFN0cm9uZyBwZXB0aWRlIGVmZmVjdAotIFVuYmFsYW5jZWQgcGVwdGlkZSBpZGVudGlmaWNhdGlvbgoKIyMjIElsbHVzdHJhdGlvbiBvbiBzdWJzZXQgb2YgQ1BUQUMgc3R1ZHk6IEEgdnMgQiBjb21wYXJpc29uIGluIGxhYiAzIAoKIyMjIyBMRlEgCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGJhY2tncm91bmQgYW5kIGNvZGUgPC9zdW1tYXJ5PjxwPgoxLiBJbXBvcnQgZGF0YQpgYGB7cn0KcHJvdGVpbnNGaWxlIDwtICJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1BEQTIxL2RhdGEvcXVhbnRpZmljYXRpb24vY3B0YWNBdnNCX2xhYjMvcHJvdGVpbkdyb3Vwcy50eHQiCgplY29scyA8LSBncmVwKCJMRlFcXC5pbnRlbnNpdHlcXC4iLCBuYW1lcyhyZWFkLmRlbGltKHByb3RlaW5zRmlsZSkpKQoKcGVMRlEgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHByb3RlaW5zRmlsZSwgZm5hbWVzID0gMSwgZWNvbCA9IGVjb2xzLAogIG5hbWUgPSAicHJvdGVpblJhdyIsIHNlcCA9ICJcdCIKKQoKY29uZCA8LSB3aGljaCgKICBzdHJzcGxpdChjb2xuYW1lcyhwZUxGUSlbWzFdXVsxXSwgc3BsaXQgPSAiIilbWzFdXSA9PSAiQSIpICMgZmluZCB3aGVyZSBjb25kaXRpb24gaXMgc3RvcmVkCgpjb2xEYXRhKHBlTEZRKSRjb25kaXRpb24gPC0gc3Vic3RyKGNvbG5hbWVzKHBlTEZRKSwgY29uZCwgY29uZCkgJT4lCiAgdW5saXN0ICU+JSAgCiAgYXMuZmFjdG9yCmBgYAoKMi4gUHJlcHJvY2Vzc2luZwoKYGBge3J9CnJvd0RhdGEocGVMRlFbWyJwcm90ZWluUmF3Il1dKSRuTm9uWmVybyA8LSByb3dTdW1zKGFzc2F5KHBlTEZRW1sicHJvdGVpblJhdyJdXSkgPiAwKQoKcGVMRlEgPC0gemVyb0lzTkEocGVMRlEsICJwcm90ZWluUmF3IikgIyBjb252ZXJ0IDAgdG8gTkEKCnBlTEZRIDwtIGxvZ1RyYW5zZm9ybShwZUxGUSwgYmFzZSA9IDIsIGkgPSAicHJvdGVpblJhdyIsIG5hbWUgPSAicHJvdGVpbkxvZyIpCgpwZUxGUSA8LSBmaWx0ZXJGZWF0dXJlcyhwZUxGUSx+IFJldmVyc2UgIT0gIisiKQpwZUxGUSA8LSBmaWx0ZXJGZWF0dXJlcyhwZUxGUSx+IFBvdGVudGlhbC5jb250YW1pbmFudCAhPSAiKyIpCgpwZUxGUSA8LSBub3JtYWxpemUocGVMRlEsIAogICAgICAgICAgICAgICAgaSA9ICJwcm90ZWluTG9nIiwgCiAgICAgICAgICAgICAgICBuYW1lID0gInByb3RlaW4iLCAKICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJjZW50ZXIubWVkaWFuIikKYGBgCgozLiBNb2RlbGluZyBhbmQgSW5mZXJlbmNlCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQpwZUxGUSA8LSBtc3Fyb2Iob2JqZWN0ID0gcGVMRlEsIGkgPSAicHJvdGVpbiIsIGZvcm11bGEgPSB+Y29uZGl0aW9uKQoKTCA8LSBtYWtlQ29udHJhc3QoImNvbmRpdGlvbkI9MCIsIHBhcmFtZXRlck5hbWVzID0gYygiY29uZGl0aW9uQiIpKQpwZUxGUSA8LSBoeXBvdGhlc2lzVGVzdChvYmplY3QgPSBwZUxGUSwgaSA9ICJwcm90ZWluIiwgY29udHJhc3QgPSBMKQoKdm9sY2Fub0xGUSA8LSBnZ3Bsb3Qocm93RGF0YShwZUxGUVtbInByb3RlaW4iXV0pJGNvbmRpdGlvbkIsCiAgICAgICAgICAgICAgICAgIGFlcyh4ID0gbG9nRkMsIHkgPSAtbG9nMTAocHZhbCksIGNvbG9yID0gYWRqUHZhbCA8IDAuMDUpKSArCiAgZ2VvbV9wb2ludChjZXggPSAyLjUpICsKICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYWxwaGEoYygiYmxhY2siLCAicmVkIiksIDAuNSkpICsgCiAgdGhlbWVfbWluaW1hbCgpICsKICBnZ3RpdGxlKHBhc3RlMCgibWF4TEZROiBUUCA9ICIsc3VtKHJvd0RhdGEocGVMRlFbWyJwcm90ZWluIl1dKSRjb25kaXRpb25CJGFkalB2YWw8MC4wNSZncmVwbChyb3duYW1lcyhyb3dEYXRhKHBlTEZRW1sicHJvdGVpbiJdXSkkY29uZGl0aW9uQikscGF0dGVybiA9IlVQUyIpLG5hLnJtPVRSVUUpLCAiIEZQID0gIiwgc3VtKHJvd0RhdGEocGVMRlFbWyJwcm90ZWluIl1dKSRjb25kaXRpb25CJGFkalB2YWw8MC4wNSYhZ3JlcGwocm93bmFtZXMocm93RGF0YShwZUxGUVtbInByb3RlaW4iXV0pJGNvbmRpdGlvbkIpLHBhdHRlcm4gPSJVUFMiKSxuYS5ybT1UUlVFKSkpCmBgYAoKCjwvcD48L2RldGFpbHM+CgojIyMjIE1lZGlhbiAmIHJvYnVzdCBzdW1tYXJpemF0aW9uCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGJhY2tncm91bmQgYW5kIGNvZGUgPC9zdW1tYXJ5PjxwPgoKMS4gSW1wb3J0IERhdGEgCgpgYGB7cn0KcGVwdGlkZXNGaWxlIDwtICJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1NHQTIwMjAvZGF0YS9xdWFudGlmaWNhdGlvbi9jcHRhY0F2c0JfbGFiMy9wZXB0aWRlcy50eHQiCgplY29scyA8LSBncmVwKAogICJJbnRlbnNpdHlcXC4iLCAKICBuYW1lcyhyZWFkLmRlbGltKHBlcHRpZGVzRmlsZSkpCiAgKQoKcGVBQiA8LSByZWFkUUZlYXR1cmVzKAogIHRhYmxlID0gcGVwdGlkZXNGaWxlLAogIGZuYW1lcyA9IDEsCiAgZWNvbCA9IGVjb2xzLAogIG5hbWUgPSAicGVwdGlkZVJhdyIsIHNlcD0iXHQiKQoKY29uZCA8LSB3aGljaCgKICBzdHJzcGxpdChjb2xuYW1lcyhwZUFCKVtbMV1dWzFdLCBzcGxpdCA9ICIiKVtbMV1dID09ICJBIikgIyBmaW5kIHdoZXJlIGNvbmRpdGlvbiBpcyBzdG9yZWQKCmNvbERhdGEocGVBQikkY29uZGl0aW9uIDwtIHN1YnN0cihjb2xuYW1lcyhwZUFCKSwgY29uZCwgY29uZCkgJT4lCiAgdW5saXN0ICU+JSAgCiAgYXMuZmFjdG9yCmBgYAoKMi4gUHJlcHJvY2Vzc2luZwoKYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0Kcm93RGF0YShwZUFCW1sicGVwdGlkZVJhdyJdXSkkbk5vblplcm8gPC0gcm93U3Vtcyhhc3NheShwZUFCW1sicGVwdGlkZVJhdyJdXSkgPiAwKQoKcGVBQiA8LSB6ZXJvSXNOQShwZUFCLCAicGVwdGlkZVJhdyIpICMgY29udmVydCAwIHRvIE5BCgpwZUFCIDwtIGxvZ1RyYW5zZm9ybShwZUFCLCBiYXNlID0gMiwgaSA9ICJwZXB0aWRlUmF3IiwgbmFtZSA9ICJwZXB0aWRlTG9nIikKCnBlQUIgPC0gZmlsdGVyRmVhdHVyZXMocGVBQiwgfiBQcm90ZWlucyAlaW4lIHNtYWxsZXN0VW5pcXVlR3JvdXBzKHJvd0RhdGEocGVBQltbInBlcHRpZGVMb2ciXV0pJFByb3RlaW5zKSkKCnBlQUIgPC0gZmlsdGVyRmVhdHVyZXMocGVBQix+UmV2ZXJzZSAhPSAiKyIpCnBlQUIgPC0gZmlsdGVyRmVhdHVyZXMocGVBQix+IFBvdGVudGlhbC5jb250YW1pbmFudCAhPSAiKyIpCgoKcGVBQiA8LSBmaWx0ZXJGZWF0dXJlcyhwZUFCLH4gbk5vblplcm8gPj0yKQpucm93KHBlQUJbWyJwZXB0aWRlTG9nIl1dKQoKcGVBQiA8LSBub3JtYWxpemUocGVBQiwgCiAgICAgICAgICAgICAgICBpID0gInBlcHRpZGVMb2ciLCAKICAgICAgICAgICAgICAgIG5hbWUgPSAicGVwdGlkZU5vcm0iLCAKICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJjZW50ZXIubWVkaWFuIikKCnBlQUIgPC0gYWdncmVnYXRlRmVhdHVyZXMocGVBQiwKICBpID0gInBlcHRpZGVOb3JtIiwKICBmY29sID0gIlByb3RlaW5zIiwKICBuYS5ybSA9IFRSVUUsCiAgbmFtZSA9ICJwcm90ZWluTWVkaWFuIiwKICBmdW4gPSBtYXRyaXhTdGF0czo6Y29sTWVkaWFucykKCnBlQUIgPC0gYWdncmVnYXRlRmVhdHVyZXMocGVBQiwKICBpID0gInBlcHRpZGVOb3JtIiwKICBmY29sID0gIlByb3RlaW5zIiwKICBuYS5ybSA9IFRSVUUsCiAgbmFtZSA9ICJwcm90ZWluUm9idXN0IikKYGBgCgozLiBNb2RlbGluZyBhbmQgaW5mZXJlbmNlCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQpwZUFCIDwtIG1zcXJvYihvYmplY3QgPSBwZUFCLCBpID0gInByb3RlaW5NZWRpYW4iLCBmb3JtdWxhID0gfmNvbmRpdGlvbikKTCA8LSBtYWtlQ29udHJhc3QoImNvbmRpdGlvbkI9MCIsIHBhcmFtZXRlck5hbWVzID0gYygiY29uZGl0aW9uQiIpKQpwZUFCIDwtIGh5cG90aGVzaXNUZXN0KG9iamVjdCA9IHBlQUIsIGkgPSAicHJvdGVpbk1lZGlhbiIsIGNvbnRyYXN0ID0gTCkKCnBlQUIgPC0gbXNxcm9iKG9iamVjdCA9IHBlQUIsIGkgPSAicHJvdGVpblJvYnVzdCIsIGZvcm11bGEgPSB+Y29uZGl0aW9uKQpwZUFCIDwtIGh5cG90aGVzaXNUZXN0KG9iamVjdCA9IHBlQUIsIGkgPSAicHJvdGVpblJvYnVzdCIsIGNvbnRyYXN0ID0gTCkKCnZvbGNhbm9NZWRpYW4gPC0gZ2dwbG90KHJvd0RhdGEocGVBQltbInByb3RlaW5NZWRpYW4iXV0pJGNvbmRpdGlvbkIsCiAgICAgICAgICAgICAgICAgIGFlcyh4ID0gbG9nRkMsIHkgPSAtbG9nMTAocHZhbCksIGNvbG9yID0gYWRqUHZhbCA8IDAuMDUpKSArCiAgZ2VvbV9wb2ludChjZXggPSAyLjUpICsKICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYWxwaGEoYygiYmxhY2siLCAicmVkIiksIDAuNSkpICsgCiAgdGhlbWVfbWluaW1hbCgpICsKICBnZ3RpdGxlKHBhc3RlMCgiTWVkaWFuOiBUUCA9ICIsc3VtKHJvd0RhdGEocGVBQltbInByb3RlaW5NZWRpYW4iXV0pJGNvbmRpdGlvbkIkYWRqUHZhbDwwLjA1JmdyZXBsKHJvd25hbWVzKHJvd0RhdGEocGVBQltbInByb3RlaW5NZWRpYW4iXV0pJGNvbmRpdGlvbkIpLHBhdHRlcm4gPSJVUFMiKSxuYS5ybT1UUlVFKSwgIiBGUCA9ICIsIHN1bShyb3dEYXRhKHBlQUJbWyJwcm90ZWluTWVkaWFuIl1dKSRjb25kaXRpb25CJGFkalB2YWw8MC4wNSYhZ3JlcGwocm93bmFtZXMocm93RGF0YShwZUFCW1sicHJvdGVpbk1lZGlhbiJdXSkkY29uZGl0aW9uQikscGF0dGVybiA9IlVQUyIpLG5hLnJtPVRSVUUpKSkKCnZvbGNhbm9Sb2J1c3Q8LSBnZ3Bsb3Qocm93RGF0YShwZUFCW1sicHJvdGVpblJvYnVzdCJdXSkkY29uZGl0aW9uQiwKICAgICAgICAgICAgICAgICAgYWVzKHggPSBsb2dGQywgeSA9IC1sb2cxMChwdmFsKSwgY29sb3IgPSBhZGpQdmFsIDwgMC4wNSkpICsKICBnZW9tX3BvaW50KGNleCA9IDIuNSkgKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBhbHBoYShjKCJibGFjayIsICJyZWQiKSwgMC41KSkgKyAKICB0aGVtZV9taW5pbWFsKCkgKwogIGdndGl0bGUocGFzdGUwKCJSb2J1c3Q6IFRQID0gIixzdW0ocm93RGF0YShwZUFCW1sicHJvdGVpblJvYnVzdCJdXSkkY29uZGl0aW9uQiRhZGpQdmFsPDAuMDUmZ3JlcGwocm93bmFtZXMocm93RGF0YShwZUFCW1sicHJvdGVpblJvYnVzdCJdXSkkY29uZGl0aW9uQikscGF0dGVybiA9IlVQUyIpLG5hLnJtPVRSVUUpLCAiIEZQID0gIiwgc3VtKHJvd0RhdGEocGVBQltbInByb3RlaW5Sb2J1c3QiXV0pJGNvbmRpdGlvbkIkYWRqUHZhbDwwLjA1JiFncmVwbChyb3duYW1lcyhyb3dEYXRhKHBlQUJbWyJwcm90ZWluUm9idXN0Il1dKSRjb25kaXRpb25CKSxwYXR0ZXJuID0iVVBTIiksbmEucm09VFJVRSkpKQpgYGAKCmBgYHtyfQp5bGltcyA8LSBjKDAsIAogICAgICAgICAgIGNlaWxpbmcobWF4KGMoLWxvZzEwKHJvd0RhdGEocGVMRlFbWyJwcm90ZWluIl1dKSRjb25kaXRpb25CJHB2YWwpLAogICAgICAgICAgICAgICAtbG9nMTAocm93RGF0YShwZUFCW1sicHJvdGVpbk1lZGlhbiJdXSkkY29uZGl0aW9uQiRwdmFsKSwKICAgICAgICAgICAgICAgLWxvZzEwKHJvd0RhdGEocGVBQltbInByb3RlaW5Sb2J1c3QiXV0pJGNvbmRpdGlvbkIkcHZhbCkpLAogICAgICAgICAgICAgICBuYS5ybT1UUlVFKSkKKQoKeGxpbXMgPC0gbWF4KGFicyhjKHJvd0RhdGEocGVMRlFbWyJwcm90ZWluIl1dKSRjb25kaXRpb25CJGxvZ0ZDLAogICAgICAgICAgICAgICByb3dEYXRhKHBlQUJbWyJwcm90ZWluTWVkaWFuIl1dKSRjb25kaXRpb25CJGxvZ0ZDLAogICAgICAgICAgICAgICByb3dEYXRhKHBlQUJbWyJwcm90ZWluUm9idXN0Il1dKSRjb25kaXRpb25CJGxvZ0ZDKSksCiAgICAgICAgICAgICAgIG5hLnJtPVRSVUUpICogYygtMSwxKQpgYGAKCmBgYHtyfQpjb21wQm94UGxvdCA8LSByYmluZChyb3dEYXRhKHBlTEZRW1sicHJvdGVpbiJdXSkkY29uZGl0aW9uQiAlPiUgbXV0YXRlKG1ldGhvZD0ibWF4TEZRIikgJT4lIHJvd25hbWVzX3RvX2NvbHVtbih2YXI9InByb3RlaW4iKSwKICAgICAgcm93RGF0YShwZUFCW1sicHJvdGVpbk1lZGlhbiJdXSkkY29uZGl0aW9uQiAlPiUgbXV0YXRlKG1ldGhvZD0ibWVkaWFuIiklPiUgcm93bmFtZXNfdG9fY29sdW1uKHZhcj0icHJvdGVpbiIpLAogICAgICByb3dEYXRhKHBlQUJbWyJwcm90ZWluUm9idXN0Il1dKSRjb25kaXRpb25CJT4lIG11dGF0ZShtZXRob2Q9InJvYnVzdCIpJT4lIHJvd25hbWVzX3RvX2NvbHVtbih2YXI9InByb3RlaW4iKSkgJT4lCiAgICAgIG11dGF0ZSh1cHM9IGdyZXBsKHByb3RlaW4scGF0dGVybj0iVVBTIikpICU+JQogICAgZ2dwbG90KGFlcyh4ID0gbWV0aG9kLCB5ID0gbG9nRkMsIGZpbGwgPSB1cHMpKSArCiAgICBnZW9tX2JveHBsb3QoKSArCiAgICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSBsb2cyKDAuNzQgLyAuMjUpLCBjb2xvciA9ICIjMDBCRkM0IikgKwogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAiI0Y4NzY2RCIpCgpgYGAgIAoKPC9wPjwvZGV0YWlscz4KCiMjIyMgQ29tcGFyaXNvbiBzdW1tYXJpemF0aW9uIG1ldGhvZHMgCgpgYGB7cn0KZ3JpZC5hcnJhbmdlKHZvbGNhbm9MRlEgKyB4bGltKHhsaW1zKSArIHlsaW0oeWxpbXMpLCAKICAgICAgICAgICAgIHZvbGNhbm9NZWRpYW4gKyB4bGltKHhsaW1zKSArIHlsaW0oeWxpbXMpLCAKICAgICAgICAgICAgIHZvbGNhbm9Sb2J1c3QgKyB4bGltKHhsaW1zKSArIHlsaW0oeWxpbXMpLAogICAgICAgICAgICAgbmNvbD0xKQpgYGAKCi0gUm9idXN0IHN1bW1hcml6YXRpb246IGhpZ2hlc3QgcG93ZXIgYW5kIHN0aWxsIGdvb2QgRkRSIGNvbnRyb2w6ICRGRFAgPSBcZnJhY3sxfXsyMH0gPSAwLjA1JC4KCgpgYGB7cn0KY29tcEJveFBsb3QKYGBgCgotIE1lZGlhbjogYmlhc2VkIGxvZ0ZDIGVzdGltYXRlcyBmb3Igc3Bpa2UtaW4gcHJvdGVpbnMKLSBtYXhMRlE6IG1vcmUgdmFyaWFibGUgbG9nRkMgZXN0aWFtdGVzIGZvciBzcGlrZS1pbiBwcm90ZWlucyAKCgojIyMgTWVkaWFuIHN1bW1hcml6YXRpb24KCldlIGZpcnN0IGV2YWx1YXRlIG1lZGlhbiBzdW1tYXJpemF0aW9uIGZvciBwcm90ZWluIGByIHByb3RgLgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIHRvIG1ha2UgcGxvdCA8L3N1bW1hcnk+PHA+CmBgYHtyfQpkYXRhSGxwIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bCiAgICByb3dEYXRhKHBlW1sicGVwdGlkZU5vcm0iXV0pJFByb3RlaW5zID09IHByb3QsCiAgICBjb2xEYXRhKHBlKSRsYWI9PSJsYWIzIl0gJT4lIGFzc2F5IAoKc3VtTWVkaWFuIDwtIGRhdGEuZnJhbWUoCiAgaW50ZW5zaXR5PSBkYXRhSGxwCiAgICAlPiUgY29sTWVkaWFucyhuYS5ybT1UUlVFKQogICwKICBjb25kaXRpb249IGNvbG5hbWVzKGRhdGFIbHApICU+JSBzdWJzdHIoMTIsMTIpICU+JSBhcy5mYWN0b3IgKQoKc3VtTWVkaWFuUGxvdCA8LSBzdW1QbG90ICsgCiAgZ2VvbV9obGluZSgKICAgIGRhdGEgPSBzdW1NZWRpYW4sCiAgICBtYXBwaW5nID0gYWVzKHlpbnRlcmNlcHQ9aW50ZW5zaXR5LGNvbG9yPWNvbmRpdGlvbikpICsgCiAgZ2d0aXRsZSgiTWVkaWFuIHN1bW1hcml6YXRpb24iKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpzdW1NZWRpYW5QbG90CmBgYAoKCi0gVGhlIHNhbXBsZSBtZWRpYW5zIGFyZSBub3QgYSBnb29kIGVzdGltYXRlIGZvciB0aGUgcHJvdGVpbiBleHByZXNzaW9uIHZhbHVlLiAKLSBJbmRlZWQsIHRoZXkgZG8gbm90IGFjY291bnQgZm9yIGRpZmZlcmVuY2VzIGluIHBlcHRpZGUgZWZmZWN0cwotIFBlcHRpZGVzIHRoYXQgaW9uaXplIHBvb3JseSBhcmUgYWxzbyBwaWNrZWQgdXAgaW4gc2FtcGxlcyB3aXRoIGhpZ2ggc3Bpa2UtaW4gY29uY2VuY2VudHJhdGlvbiBhbmQgbm90IGluIHNhbXBsZXMgd2l0aCBsb3cgc3Bpa2UtaW4gY29uY2VudHJhdGlvbgotIFRoaXMgaW50cm9kdWNlcyBhIGJpYXMuIAoKIyMjIE1lYW4gc3VtbWFyaXphdGlvbiAKCgokJCAKeV97aXB9ID0gXGJldGFfaV5cdGV4dHtzYW1wbGV9ICsgXGVwc2lsb25fe2lwfQokJAo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3J9CnN1bU1lYW5Nb2QgPC0gbG0oaW50ZW5zaXR5IH4gLTEgKyBzYW1wbGUsZGF0YSkKCnN1bU1lYW4gPC0gZGF0YS5mcmFtZSgKICBpbnRlbnNpdHk9c3VtTWVhbk1vZCRjb2VmW2dyZXAoInNhbXBsZSIsbmFtZXMoc3VtTWVhbk1vZCRjb2VmKSldLAogIGNvbmRpdGlvbj0gbmFtZXMoc3VtTWVhbk1vZCRjb2VmKVtncmVwKCJzYW1wbGUiLG5hbWVzKHN1bU1lYW5Nb2QkY29lZikpXSAlPiUgc3Vic3RyKDE4LDE4KSAlPiUgYXMuZmFjdG9yICkKCgoKc3VtTWVhblBsb3QgPC0gc3VtUGxvdCArIGdlb21faGxpbmUoCiAgICBkYXRhID0gc3VtTWVhbiwKICAgIG1hcHBpbmcgPSBhZXMoeWludGVyY2VwdD1pbnRlbnNpdHksY29sb3I9Y29uZGl0aW9uKSkgKwogICAgZ2d0aXRsZSgiTWVhbiBzdW1tYXJpemF0aW9uIikKYGBgCjwvcD48L2RldGFpbHM+CgpgYGB7cn0KZ3JpZC5hcnJhbmdlKHN1bU1lZGlhblBsb3QsIHN1bU1lYW5QbG90LCBuY29sPTIpCmBgYAoKCiMjIyBNb2RlbCBiYXNlZCBzdW1tYXJpemF0aW9uCgpXZSBjYW4gdXNlIGEgbGluZWFyIHBlcHRpZGUtbGV2ZWwgbW9kZWwgdG8gZXN0aW1hdGUgdGhlIHByb3RlaW4gZXhwcmVzc2lvbiB2YWx1ZSB3aGlsZSBjb3JyZWN0aW5nIGZvciB0aGUgcGVwdGlkZSBlZmZlY3QsIGkuZS4gCgokJCAKeV97aXB9ID0gXGJldGFfaV5cdGV4dHtzYW1wbGV9K1xiZXRhXntwZXB0aWRlfV97cH0gKyBcZXBzaWxvbl97aXB9CiQkCgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBjb2RlIHRvIG1ha2UgcGxvdCA8L3N1bW1hcnk+PHA+CmBgYHtyfQpzdW1NZWFuUGVwTW9kIDwtIGxtKGludGVuc2l0eSB+IC0xICsgc2FtcGxlICsgcGVwdGlkZSxkYXRhKQoKc3VtTWVhblBlcCA8LSBkYXRhLmZyYW1lKAogIGludGVuc2l0eT1zdW1NZWFuUGVwTW9kJGNvZWZbZ3JlcCgic2FtcGxlIixuYW1lcyhzdW1NZWFuUGVwTW9kJGNvZWYpKV0gKyBtZWFuKGRhdGEkaW50ZW5zaXR5KSAtIG1lYW4oc3VtTWVhblBlcE1vZCRjb2VmW2dyZXAoInNhbXBsZSIsbmFtZXMoc3VtTWVhblBlcE1vZCRjb2VmKSldKSwKICBjb25kaXRpb249IG5hbWVzKHN1bU1lYW5QZXBNb2QkY29lZilbZ3JlcCgic2FtcGxlIixuYW1lcyhzdW1NZWFuUGVwTW9kJGNvZWYpKV0gJT4lIHN1YnN0cigxOCwxOCkgJT4lIGFzLmZhY3RvciApCgoKZml0TG1QbG90IDwtICBzdW1QbG90ICsgZ2VvbV9saW5lKAogICAgZGF0YSA9IGRhdGEgJT4lIG11dGF0ZShmaXQ9c3VtTWVhblBlcE1vZCRmaXR0ZWQudmFsdWVzKSwKICAgIG1hcHBpbmcgPSBhZXMoeD1wZXB0aWRlLCB5PWZpdCxjb2xvcj1jb25kaXRpb24sIGdyb3VwPXNhbXBsZSkpICsKICAgIGdndGl0bGUoImZpdDogfiBzYW1wbGUgKyBwZXB0aWRlIikKc3VtTG1QbG90IDwtIHN1bVBsb3QgKyBnZW9tX2hsaW5lKAogICAgZGF0YSA9IHN1bU1lYW5QZXAsCiAgICBtYXBwaW5nID0gYWVzKHlpbnRlcmNlcHQ9aW50ZW5zaXR5LGNvbG9yPWNvbmRpdGlvbikpICsKICAgIGdndGl0bGUoIlN1bW1hcml6YXRpb246IHNhbXBsZSBlZmZlY3QiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpncmlkLmFycmFuZ2Uoc3VtTWVkaWFuUGxvdCwgc3VtTWVhblBsb3QsIHN1bUxtUGxvdCwgbnJvdz0xKQpgYGAKCi0gQnkgY29ycmVjdGluZyBmb3IgdGhlIHBlcHRpZGUgc3BlY2llcyB0aGUgcHJvdGVpbiBleHByZXNzaW9uIHZhbHVlcyBhcmUgbXVjaCBiZXR0ZXIgc2VwYXJhdGVkIGFuIGJldHRlciByZWZsZWN0IGRpZmZlcmVuY2VzIGluIGFidW5kYW5jZSBpbmR1Y2VkIGJ5IHRoZSBzcGlrZS1pbiBjb25kaXRpb24uIAoKLSBJbmRlZWQsIGl0IHNob3dzIHRoYXQgbWVkaWFuIGFuZCBtZWFuIHN1bW1hcml6YXRpb24gdGhhdCBkbyBub3QgYWNjb3VudCBmb3IgdGhlIHBlcHRpZGUgZWZmZWN0IGluZGVlZCBvdmVyZXN0aW1hdGUgdGhlIHByb3RlaW4gZXhwcmVzc2lvbiB2YWx1ZSBpbiB0aGUgc21hbGwgc3Bpa2UtaW4gY29uZGl0aW9ucyBhbmQgdW5kZXJlc3RpbWF0ZSB0aGF0IGluIHRoZSBsYXJnZSBzcGlrZS1pbiBjb25kaXRpb25zLgoKLSBTdGlsbCB0aGVyZSBzZWVtIHRvIGJlIHNvbWUgaXNzdWVzIHdpdGggc2FtcGxlcyB0aGF0IGZvciB3aGljaCB0aGUgZXhwcmVzc2lvbiB2YWx1ZXMgYXJlIG5vdCB3ZWxsIHNlcGFyYXRlZCBhY2NvcmRpbmcgdG8gdGhlIHNwaWtlLWluIGNvbmRpdGlvbi4gCgpBIHJlc2lkdWFsIGFuYWx5c2lzIGNsZWFybHkgaW5kaWNhdGVzIHBvdGVudGlhbCBpc3N1ZXM6Cgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3J9CnJlc1Bsb3QgPC0gZGF0YSAlPiUgCiAgbXV0YXRlKHJlcz1zdW1NZWFuUGVwTW9kJHJlc2lkdWFscykgJT4lCiAgZ2dwbG90KGFlcyh4ID0gcGVwdGlkZSwgeSA9IHJlcywgY29sb3IgPSBjb25kaXRpb24sIGxhYmVsID0gY29uZGl0aW9uKSwgc2hvdy5sZWdlbmQgPSBGQUxTRSkgKwogIGdlb21fcG9pbnQoc2hhcGU9MjEpICsKICB0aGVtZV9taW5pbWFsKCkgKwogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTAsIHZqdXN0ID0gMC41LCBoanVzdCA9IDEpKSArCiAgeGxhYigiUGVwdGlkZSIpICsgCiAgeWxhYigicmVzaWR1YWwiKSArCiAgZ2d0aXRsZSgicmVzaWR1YWxzOiB+IHNhbXBsZSArIHBlcHRpZGUiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpncmlkLmFycmFuZ2UoZml0TG1QbG90LCByZXNQbG90LCBucm93ID0gMSkKZ3JpZC5hcnJhbmdlKGZpdExtUGxvdCwgc3VtTG1QbG90LCBucm93ID0gMSkKYGBgCgotIFRoZSByZXNpZHVhbCBwbG90IHNob3dzIHNvbWUgbGFyZ2Ugb3V0bGllcnMgZm9yIHBlcHRpZGUgS0lFRUlBQUsuIAotIEluZGVlZCwgaW4gdGhlIG9yaWdpbmFsIHBsb3QgdGhlIGludGVuc2l0aWVzIGZvciB0aGlzIHBlcHRpZGUgZG8gbm90IHNlZW0gdG8gbGluZSB1cCB2ZXJ5IHdlbGwgd2l0aCB0aGUgY29uY2VudHJhdGlvbi4gCi0gVGhpcyBpbmR1Y2VzIGEgYmlhcyBpbiB0aGUgc3VtbWFyaXphdGlvbiBmb3Igc29tZSBvZiB0aGUgc2FtcGxlcyAoZS5nLiBmb3IgRCBhbmQgRSkKCiMjIyBSb2J1c3Qgc3VtbWFyaXphdGlvbiB1c2luZyBhIHBlcHRpZGUtbGV2ZWwgbGluZWFyIG1vZGVsIAoKJCQgCnlfe2lwfSA9IFxiZXRhX2leXHRleHR7c2FtcGxlfStcYmV0YV57cGVwdGlkZX1fe3B9ICsgXGVwc2lsb25fe2lwfQokJAoKCi0gT3JkaW5hcnkgbGVhc3Qgc3F1YXJlczogZXN0aW1hdGUgJFxiZXRhJCB0aGF0IG1pbmltaXplcwokJApcdGV4dHtPTFN9OiBcc3VtXGxpbWl0c197aSxwfSBcZXBzaWxvbl97aXB9XjIgPSBcc3VtXGxpbWl0c197aSxwfSh5X3tpcH0tXGJldGFfaV5cdGV4dHtzYW1wbGV9LVxiZXRhX3BeXHRleHR7cGVwdGlkZX0pXjIKJCQKCldlIHJlcGxhY2UgT0xTIGJ5IE0tZXN0aW1hdGlvbiB3aXRoIGxvc3MgZnVuY3Rpb24KJCQKXHN1bVxsaW1pdHNfe2kscH0gd197aXB9XGVwc2lsb25fe2lwfV4yID0gXHN1bVxsaW1pdHNfe2kscH13X3tpcH0oeV97aXB9LVxiZXRhX2leXHRleHR7c2FtcGxlfS1cYmV0YV9wXlx0ZXh0e3BlcHRpZGV9KV4yCiQkCgotIEl0ZXJhdGl2ZWx5IGZpdCBtb2RlbCB3aXRoIG9ic2VydmF0aW9uIHdlaWdodHMgJHdfe2lwfSQgdW50aWwgY29udmVyZ2VuY2UKLSBUaGUgd2VpZ2h0cyBhcmUgY2FsY3VsYXRlZCBiYXNlZCBvbiBzdGFuZGFyZGl6ZWQgcmVzaWR1YWxzCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3J9CnN1bU1lYW5QZXBSb2JNb2QgPC0gTUFTUzo6cmxtKGludGVuc2l0eSB+IC0xICsgc2FtcGxlICsgcGVwdGlkZSxkYXRhKQpyZXNSb2JQbG90IDwtIGRhdGEgJT4lCiAgbXV0YXRlKHJlcyA9IHN1bU1lYW5QZXBSb2JNb2QkcmVzaWR1YWxzLAogICAgICAgICB3ID0gc3VtTWVhblBlcFJvYk1vZCR3KSAlPiUKICBnZ3Bsb3QoYWVzKHggPSBwZXB0aWRlLCB5ID0gcmVzLCBjb2xvciA9IGNvbmRpdGlvbiwgbGFiZWwgPSBjb25kaXRpb24sc2l6ZT13KSwgc2hvdy5sZWdlbmQgPSBGQUxTRSkgKwogIGdlb21fcG9pbnQoc2hhcGU9MjEsc2l6ZT0uMikgKwogIGdlb21fcG9pbnQoc2hhcGU9MjEpICsKICB0aGVtZV9taW5pbWFsKCkgKwogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTAsIHZqdXN0ID0gMC41LCBoanVzdCA9IDEpKSArCiAgeGxhYigiUGVwdGlkZSIpICsgCiAgeWxhYigicmVzaWR1YWwiKSArIAogIHlsaW0oYygtMSwxKSptYXgoYWJzKHN1bU1lYW5QZXBSb2JNb2QkcmVzaWR1YWxzKSkpCndlaWdodFBsb3QgPC0gcXBsb3QoCiAgc2VxKC01LDUsLjAxKSwgCiAgTUFTUzo6cHNpLmh1YmVyKHNlcSgtNSw1LC4wMSkpLAogIGdlb209InBhdGgiKSArCiAgeGxhYigic3RhbmRhcmRpemVkIHJlc2lkdWFsIikgKwogIHlsYWIoIndlaWdodCIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CmdyaWQuYXJyYW5nZSh3ZWlnaHRQbG90LHJlc1JvYlBsb3QsbnJvdz0xKQpgYGAKCi0gV2UgY2xlYXJseSBzZWUgdGhhdCB0aGUgd2VpZ2h0cyBpbiB0aGUgTS1lc3RpbWF0aW9uIHByb2NlZHVyZSB3aWxsIGRvd24td2VpZ2h0IGVycm9ycyBhc3NvY2lhdGVkIHdpdGggb3V0bGllcnMgZm9yIHBlcHRpZGUgS0lFRUlBQUsuCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3J9CnN1bU1lYW5QZXBSb2IgPC0gZGF0YS5mcmFtZSgKICBpbnRlbnNpdHk9c3VtTWVhblBlcFJvYk1vZCRjb2VmW2dyZXAoInNhbXBsZSIsbmFtZXMoc3VtTWVhblBlcFJvYk1vZCRjb2VmKSldICsgbWVhbihkYXRhJGludGVuc2l0eSkgLSBtZWFuKHN1bU1lYW5QZXBSb2JNb2QkY29lZltncmVwKCJzYW1wbGUiLG5hbWVzKHN1bU1lYW5QZXBSb2JNb2QkY29lZikpXSksCiAgY29uZGl0aW9uPSBuYW1lcyhzdW1NZWFuUGVwUm9iTW9kJGNvZWYpW2dyZXAoInNhbXBsZSIsbmFtZXMoc3VtTWVhblBlcFJvYk1vZCRjb2VmKSldICU+JSBzdWJzdHIoMTgsMTgpICU+JSBhcy5mYWN0b3IgKQoKc3VtUmxtUGxvdCA8LSBzdW1QbG90ICsgZ2VvbV9obGluZSgKICAgIGRhdGE9c3VtTWVhblBlcFJvYiwKICAgIG1hcHBpbmc9YWVzKHlpbnRlcmNlcHQ9aW50ZW5zaXR5LGNvbG9yPWNvbmRpdGlvbikpICsgCiAgICBnZ3RpdGxlKCJSb2J1c3QiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQogZ3JpZC5hcnJhbmdlKHN1bUxtUGxvdCArIGdndGl0bGUoIk9MUyIpLCBzdW1SbG1QbG90LCBucm93ID0gMSkKYGBgCgotIFJvYnVzdCByZWdyZXNpb24gcmVzdWx0cyBpbiBhIGJldHRlciBzZXBhcmF0aW9uIGJldHdlZW4gdGhlIHByb3RlaW4gZXhwcmVzc2lvbiB2YWx1ZXMgZm9yIHRoZSBkaWZmZXJlbnQgc2FtcGxlcyBhY2NvcmRpbmcgdG8gdGhlaXIgc3Bpa2UtaW4gY29uY2VudHJhdGlvbi4gCgoKCiMjIyBDb21wYXJpc29uIHN1bW1hcml6YXRpb24gbWV0aG9kcyAKCi0gbWF4TEZRCgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL21heExGUV9wcmluY2lwbGUucG5nIikKYGBgCgotIE1TLXN0YXRzIGFsc28gdXNlcyBhIHJvYnVzdCBwZXB0aWRlIGxldmVsIG1vZGVsIHRvIHBlcmZvcm0gdGhlIHN1bW1hcml6YXRpb24sIGhvd2V2ZXIsIHRoZXkgdHlwaWNhbGx5IGZpcnN0IGltcHV0ZSBtaXNzaW5nIHZhbHVlcwoKLSBQcm90ZXVzIGhpZ2gtZmx5ZXIgbWV0aG9kOiBtZWFuIG9mIHRocmVlIHBlcHRpZGVzIHdpdGggaGlnaGVzdCBpbnRlbnNpdHkKCgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL21zcXJvYnN1bV9zdW1fbm92ZWwucG5nIikKYGBgCgotIFtAc3RpY2tlcjIwMjBdCi0gZG9pOiBodHRwczovL2RvaS5vcmcvMTAuMTA3NC9tY3AuUkExMTkuMDAxNjI0ICAKLSBbcGRmXShodHRwczovL3d3dy5tY3BvbmxpbmUub3JnL2FjdGlvbi9zaG93UGRmP3BpaT1TMTUzNS05NDc2JTI4MjAlMjkzNDk4Mi0zKQoKIyBSZWZlcmVuY2Vz