1 Background

This case-study is a subset of the data of the 6th study of the Clinical Proteomic Technology Assessment for Cancer (CPTAC). In this experiment, the authors spiked the Sigma Universal Protein Standard mixture 1 (UPS1) containing 48 different human proteins in a protein background of 60 ng/\(\mu\)L Saccharomyces cerevisiae strain BY4741. Two different spike-in concentrations were used: 6A (0.25 fmol UPS1 proteins/\(\mu\)L) and 6B (0.74 fmol UPS1 proteins/\(\mu\)L) [5]. We limited ourselves to the data of LTQ-Orbitrap W at site 56. The data were searched with MaxQuant version 1.5.2.8, and detailed search settings were described in Goeminne et al. (2016) [1]. Three replicates are available for each concentration.

2 Data

We first import the data from peptideRaws.txt file. This is the file containing your peptideRaw-level intensities. For a MaxQuant search [6], this peptideRaws.txt file can be found by default in the “path_to_raw_files/combined/txt/” folder from the MaxQuant output, with “path_to_raw_files” the folder where the raw files were saved. In this vignette, we use a MaxQuant peptideRaws file which is a subset of the cptac study. This data is available in the msdata package. To import the data we use the QFeatures package.

We generate the object peptideRawFile with the path to the peptideRaws.txt file. Using the grepEcols function, we find the columns that contain the expression data of the peptideRaws in the peptideRaws.txt file.

library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)

peptidesFile <- "https://raw.githubusercontent.com/statOmics/SGA2020/data/quantification/cptacAvsB_lab3/peptides.txt"

ecols <- MSnbase::grepEcols(
  peptidesFile,
  "Intensity ",
  split = "\t")

pe <- readQFeatures(
  table = peptidesFile,
  fnames = 1,
  ecol = ecols,
  name = "peptideRaw", sep="\t")

In the following code chunk, we can extract the spikein condition from the raw file name.

cond <- which(
  strsplit(colnames(pe)[[1]][1], split = "")[[1]] == "A") # find where condition is stored

colData(pe)$condition <- substr(colnames(pe), cond, cond) %>%
  unlist %>%  
  as.factor

We calculate how many non zero intensities we have per peptide and this will be useful for filtering.

rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)

Peptides with zero intensities are missing peptides and should be represent with a NA value rather than 0.

pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA

2.1 Data exploration

We can inspect the missingness in our data with the plotNA() function provided with MSnbase. 45% of all peptide intensities are missing and for some peptides we do not even measure a signal in any sample. The missingness is similar across samples.

MSnbase::plotNA(assay(pe[["peptideRaw"]])) +
  xlab("Peptide index (ordered by data completeness)")

3 Preprocessing

This section preforms standard preprocessing for the peptide data. This include log transformation, filtering and summarisation of the data.

3.1 Log transform the data

pe <- logTransform(pe, base = 2, i = "peptideRaw", name = "peptideLog")
limma::plotDensities(assay(pe[["peptideLog"]]))

3.2 Filtering

3.2.1 Handling overlapping protein groups

In our approach a peptide can map to multiple proteins, as long as there is none of these proteins present in a smaller subgroup.

pe[["peptideLog"]] <-
  pe[["peptideLog"]][rowData(pe[["peptideLog"]])$Proteins
  %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins),]

3.2.2 Remove reverse sequences (decoys) and contaminants

We now remove the contaminants, peptides that map to decoy sequences, and proteins which were only identified by peptides with modifications.

pe[["peptideLog"]] <- pe[["peptideLog"]][rowData(pe[["peptideLog"]])$Reverse != "+", ]
pe[["peptideLog"]] <- pe[["peptideLog"]][rowData(pe[["peptideLog"]])$
Potential.contaminant != "+", ]

3.2.3 Remove peptides of proteins that were only identified with modified peptides

I will skip this step for the moment. Large protein groups file needed for this.

3.2.4 Drop peptides that were only identified in one sample

We keep peptides that were observed at last twice.

pe[["peptideLog"]] <- pe[["peptideLog"]][rowData(pe[["peptideLog"]])$nNonZero >= 2, ]
nrow(pe[["peptideLog"]])
## [1] 7011

We keep 7011 peptides after filtering.

3.3 Quantile normalize the data

pe <- normalize(pe, i = "peptideLog", method = "quantiles", name = "peptideNorm")

3.4 Explore quantile normalized data

After quantile normalisation the density curves for all samples coincide.

limma::plotDensities(assay(pe[["peptideNorm"]]))

This is more clearly seen is a boxplot.

boxplot(assay(pe[["peptideNorm"]]), col = palette()[-1],
        main = "Peptide distribtutions after normalisation", ylab = "intensity")

We can visualize our data using a Multi Dimensional Scaling plot, eg. as provided by the limma package.

limma::plotMDS(assay(pe[["peptideNorm"]]), col = as.numeric(colData(pe)$condition))

The first axis in the plot is showing the leading log fold changes (differences on the log scale) between the samples. We notice that the leading differences (log FC) in the peptideRaw data seems to be driven by technical variability. Indeed, the samples do not seem to be clearly separated according to the spike-in condition.

3.5 Summarization to protein level

We use median summarization in aggregateFeatures.

pe <- aggregateFeatures(pe,
  i = "peptideNorm",
  fcol = "Proteins",
  na.rm = TRUE,
  name = "proteinMedian",
  fun = matrixStats::colMedians)
## Your quantitative and row data contain missing values. Please read the
## relevant section(s) in the aggregateFeatures manual page regarding the
## effects of missing values on data aggregation.

We notice that the leading differences (log FC) in the protein data are still according to technical variation.

plotMDS(assay(pe[["proteinMedian"]]), col = as.numeric(colData(pe)$condition))

4 Data Analysis

4.1 Estimation

We model the protein level expression values using msqrob. By default msqrob2 estimates the model parameters using robust regression.

pe <- msqrob(object = pe, i = "proteinMedian", formula = ~condition)

4.2 Inference

First, we extract the parameter names of the model.

getCoef(rowData(pe[["proteinMedian"]])$msqrobModels[[1]])
## (Intercept)  conditionB 
##   14.942190    1.604333

Spike-in condition a is the reference class. So the mean log2 expression for samples from condition a is ‘(Intercept). The mean log2 expression for samples from condition B is’(Intercept)+conditionB’. Hence, the average log2 fold change between condition b and condition a is modelled using the parameter ‘conditionB’. Thus, we assess the contrast ‘conditionB=0’ with our statistical test.

L <- makeContrast("conditionB=0", parameterNames = c("conditionB"))
pe <- hypothesisTest(object = pe, i = "proteinMedian", contrast = L)

4.3 Plots

4.3.1 Volcano-plot

volcano <- ggplot(rowData(pe[["proteinMedian"]])$conditionB,
                  aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
  geom_point(cex = 2.5) +
  scale_color_manual(values = alpha(c("black", "red"), 0.5)) + theme_minimal()
volcano

4.3.2 Heatmap

We first select the names of the proteins that were declared signficant.

sigNames <- rowData(pe[["proteinMedian"]])$conditionB %>%
  rownames_to_column("proteinMedian") %>%
  filter(adjPval<0.05) %>%
  pull(proteinMedian)
heatmap(assay(pe[["proteinMedian"]])[sigNames, ])

4.3.3 Boxplots

We make boxplot of the log2 FC and stratify according to the whether a protein is spiked or not.

rowData(pe[["proteinMedian"]])$conditionB %>%
  rownames_to_column(var = "protein") %>%
  ggplot(aes(x=grepl("UPS",protein),y=logFC)) +
  geom_boxplot() +
  xlab("UPS") +
  geom_segment(
    x = 1.5,
    xend = 2.5,
    y = log2(0.74/0.25),
    yend = log2(0.74/0.25),
    colour="red") +
  geom_segment(
    x = 0.5,
    xend = 1.5,
    y = 0,
    yend = 0,
    colour="red") +
  annotate(
    "text",
    x = c(1,2),
    y = c(0,log2(0.74/0.25))+.1,
    label = c(
      "log2 FC Ecoli = 0",
      paste0("log2 FC UPS = ",round(log2(0.74/0.25),2))
      ),
    colour = "red")
## Warning: Removed 166 rows containing non-finite values (stat_boxplot).

What do you observe?

4.3.4 Detail plots

We first extract the normalized peptideRaw expression values for a particular protein.

for (protName in sigNames)
{
pePlot <- pe[protName, , c("peptideNorm","proteinMedian")]
pePlotDf <- data.frame(longFormat(pePlot))
pePlotDf$assay <- factor(pePlotDf$assay,
                        levels = c("peptideNorm", "proteinMedian"))
pePlotDf$condition <- as.factor(colData(pePlot)[pePlotDf$colname, "condition"])

# plotting
p1 <- ggplot(data = pePlotDf,
       aes(x = colname, y = value, group = rowname)) +
    geom_line() + geom_point() +  theme_minimal() +
    facet_grid(~assay) + ggtitle(protName)
print(p1)

# plotting 2
p2 <- ggplot(pePlotDf, aes(x = colname, y = value, fill = condition)) +
  geom_boxplot(outlier.shape = NA) + geom_point(position = position_jitter(width = .1),
                                                aes(shape = rowname)) +
  scale_shape_manual(values = 1:nrow(pePlotDf)) +
  labs(title = protName, x = "sample", y = "peptide intensity (log2)") + theme_minimal()
  facet_grid(~assay)
print(p2)
}

LS0tCnRpdGxlOiAiSW50cm9kdWN0aW9uIHRvIHByb3Rlb21pY3MgZGF0YSBhbmFseXNpczogbWVkaWFuIHN1bW1hcml6YXRpb24iCmF1dGhvcjogIkxpZXZlbiBDbGVtZW50IgpkYXRlOiAic3RhdE9taWNzLCBHaGVudCBVbml2ZXJzaXR5IChodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8pIgpvdXRwdXQ6CiAgICBodG1sX2RvY3VtZW50OgogICAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAgIHRoZW1lOiBjb3NtbwogICAgICB0b2M6IHRydWUKICAgICAgdG9jX2Zsb2F0OiB0cnVlCiAgICAgIGhpZ2hsaWdodDogdGFuZ28KICAgICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCi0tLQoKIyBCYWNrZ3JvdW5kClRoaXMgY2FzZS1zdHVkeSBpcyBhIHN1YnNldCBvZiB0aGUgZGF0YSBvZiB0aGUgNnRoIHN0dWR5IG9mIHRoZSBDbGluaWNhbApQcm90ZW9taWMgVGVjaG5vbG9neSBBc3Nlc3NtZW50IGZvciBDYW5jZXIgKENQVEFDKS4KSW4gdGhpcyBleHBlcmltZW50LCB0aGUgYXV0aG9ycyBzcGlrZWQgdGhlIFNpZ21hIFVuaXZlcnNhbCBQcm90ZWluIFN0YW5kYXJkCm1peHR1cmUgMSAoVVBTMSkgY29udGFpbmluZyA0OCBkaWZmZXJlbnQgaHVtYW4gcHJvdGVpbnMgaW4gYSBwcm90ZWluIGJhY2tncm91bmQKb2YgNjAgbmcvJFxtdSRMIFNhY2NoYXJvbXljZXMgY2VyZXZpc2lhZSBzdHJhaW4gQlk0NzQxLgpUd28gZGlmZmVyZW50IHNwaWtlLWluIGNvbmNlbnRyYXRpb25zIHdlcmUgdXNlZDoKNkEgKDAuMjUgZm1vbCBVUFMxIHByb3RlaW5zLyRcbXUkTCkgYW5kIDZCICgwLjc0IGZtb2wgVVBTMSBwcm90ZWlucy8kXG11JEwpIFs1XS4KV2UgbGltaXRlZCBvdXJzZWx2ZXMgdG8gdGhlIGRhdGEgb2YgTFRRLU9yYml0cmFwIFcgYXQgc2l0ZSA1Ni4KVGhlIGRhdGEgd2VyZSBzZWFyY2hlZCB3aXRoIE1heFF1YW50IHZlcnNpb24gMS41LjIuOCwgYW5kCmRldGFpbGVkIHNlYXJjaCBzZXR0aW5ncyB3ZXJlIGRlc2NyaWJlZCBpbiBHb2VtaW5uZSBldCBhbC4gKDIwMTYpIFsxXS4KVGhyZWUgcmVwbGljYXRlcyBhcmUgYXZhaWxhYmxlIGZvciBlYWNoIGNvbmNlbnRyYXRpb24uCgojIERhdGEKCldlIGZpcnN0IGltcG9ydCB0aGUgZGF0YSBmcm9tIHBlcHRpZGVSYXdzLnR4dCBmaWxlLiBUaGlzIGlzIHRoZSBmaWxlIGNvbnRhaW5pbmcKeW91ciBwZXB0aWRlUmF3LWxldmVsIGludGVuc2l0aWVzLiBGb3IgYSBNYXhRdWFudCBzZWFyY2ggWzZdLAp0aGlzIHBlcHRpZGVSYXdzLnR4dCBmaWxlIGNhbiBiZSBmb3VuZCBieSBkZWZhdWx0IGluIHRoZQoicGF0aF90b19yYXdfZmlsZXMvY29tYmluZWQvdHh0LyIgZm9sZGVyIGZyb20gdGhlIE1heFF1YW50IG91dHB1dCwKd2l0aCAicGF0aF90b19yYXdfZmlsZXMiIHRoZSBmb2xkZXIgd2hlcmUgdGhlIHJhdyBmaWxlcyB3ZXJlIHNhdmVkLgpJbiB0aGlzIHZpZ25ldHRlLCB3ZSB1c2UgYSBNYXhRdWFudCBwZXB0aWRlUmF3cyBmaWxlIHdoaWNoIGlzIGEgc3Vic2V0Cm9mIHRoZSBjcHRhYyBzdHVkeS4gVGhpcyBkYXRhIGlzIGF2YWlsYWJsZSBpbiB0aGUgYG1zZGF0YWAgcGFja2FnZS4KVG8gaW1wb3J0IHRoZSBkYXRhIHdlIHVzZSB0aGUgYFFGZWF0dXJlc2AgcGFja2FnZS4KCldlIGdlbmVyYXRlIHRoZSBvYmplY3QgcGVwdGlkZVJhd0ZpbGUgd2l0aCB0aGUgcGF0aCB0byB0aGUgcGVwdGlkZVJhd3MudHh0IGZpbGUuClVzaW5nIHRoZSBgZ3JlcEVjb2xzYCBmdW5jdGlvbiwgd2UgZmluZCB0aGUgY29sdW1ucyB0aGF0IGNvbnRhaW4gdGhlIGV4cHJlc3Npb24KZGF0YSBvZiB0aGUgcGVwdGlkZVJhd3MgaW4gdGhlIHBlcHRpZGVSYXdzLnR4dCBmaWxlLgoKYGBge3IsIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KGxpbW1hKQpsaWJyYXJ5KFFGZWF0dXJlcykKbGlicmFyeShtc3Fyb2IyKQpsaWJyYXJ5KHBsb3RseSkKCnBlcHRpZGVzRmlsZSA8LSAiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL3N0YXRPbWljcy9TR0EyMDIwL2RhdGEvcXVhbnRpZmljYXRpb24vY3B0YWNBdnNCX2xhYjMvcGVwdGlkZXMudHh0IgoKZWNvbHMgPC0gTVNuYmFzZTo6Z3JlcEVjb2xzKAogIHBlcHRpZGVzRmlsZSwKICAiSW50ZW5zaXR5ICIsCiAgc3BsaXQgPSAiXHQiKQoKcGUgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHBlcHRpZGVzRmlsZSwKICBmbmFtZXMgPSAxLAogIGVjb2wgPSBlY29scywKICBuYW1lID0gInBlcHRpZGVSYXciLCBzZXA9Ilx0IikKYGBgCgpJbiB0aGUgZm9sbG93aW5nIGNvZGUgY2h1bmssIHdlIGNhbiBleHRyYWN0IHRoZSBzcGlrZWluIGNvbmRpdGlvbiBmcm9tIHRoZSByYXcgZmlsZSBuYW1lLgoKYGBge3J9CmNvbmQgPC0gd2hpY2goCiAgc3Ryc3BsaXQoY29sbmFtZXMocGUpW1sxXV1bMV0sIHNwbGl0ID0gIiIpW1sxXV0gPT0gIkEiKSAjIGZpbmQgd2hlcmUgY29uZGl0aW9uIGlzIHN0b3JlZAoKY29sRGF0YShwZSkkY29uZGl0aW9uIDwtIHN1YnN0cihjb2xuYW1lcyhwZSksIGNvbmQsIGNvbmQpICU+JQogIHVubGlzdCAlPiUgIAogIGFzLmZhY3RvcgpgYGAKCgpXZSBjYWxjdWxhdGUgaG93IG1hbnkgbm9uIHplcm8gaW50ZW5zaXRpZXMgd2UgaGF2ZSBwZXIgcGVwdGlkZSBhbmQgdGhpcwp3aWxsIGJlIHVzZWZ1bCBmb3IgZmlsdGVyaW5nLgoKYGBge3J9CnJvd0RhdGEocGVbWyJwZXB0aWRlUmF3Il1dKSRuTm9uWmVybyA8LSByb3dTdW1zKGFzc2F5KHBlW1sicGVwdGlkZVJhdyJdXSkgPiAwKQpgYGAKCgpQZXB0aWRlcyB3aXRoIHplcm8gaW50ZW5zaXRpZXMgYXJlIG1pc3NpbmcgcGVwdGlkZXMgYW5kIHNob3VsZCBiZSByZXByZXNlbnQKd2l0aCBhIGBOQWAgdmFsdWUgcmF0aGVyIHRoYW4gYDBgLgpgYGB7cn0KcGUgPC0gemVyb0lzTkEocGUsICJwZXB0aWRlUmF3IikgIyBjb252ZXJ0IDAgdG8gTkEKYGBgCgoKIyMgRGF0YSBleHBsb3JhdGlvbgoKV2UgY2FuIGluc3BlY3QgdGhlIG1pc3NpbmduZXNzIGluIG91ciBkYXRhIHdpdGggdGhlIGBwbG90TkEoKWAgZnVuY3Rpb24KcHJvdmlkZWQgd2l0aCBgTVNuYmFzZWAuCmByIGZvcm1hdChtZWFuKGlzLm5hKGFzc2F5KHBlW1sicGVwdGlkZVJhdyJdXSkpKSoxMDAsZGlnaXRzPTIpYCUgb2YgYWxsIHBlcHRpZGUKaW50ZW5zaXRpZXMgYXJlIG1pc3NpbmcgYW5kIGZvciBzb21lIHBlcHRpZGVzIHdlIGRvIG5vdCBldmVuIG1lYXN1cmUgYSBzaWduYWwKaW4gYW55IHNhbXBsZS4gVGhlIG1pc3NpbmduZXNzIGlzIHNpbWlsYXIgYWNyb3NzIHNhbXBsZXMuCgoKYGBge3IsIGNhY2hlPSBUUlVFLCBldmFsPUZBTFNFfQpNU25iYXNlOjpwbG90TkEoYXNzYXkocGVbWyJwZXB0aWRlUmF3Il1dKSkgKwogIHhsYWIoIlBlcHRpZGUgaW5kZXggKG9yZGVyZWQgYnkgZGF0YSBjb21wbGV0ZW5lc3MpIikKYGBgCgoKIyBQcmVwcm9jZXNzaW5nCgpUaGlzIHNlY3Rpb24gcHJlZm9ybXMgc3RhbmRhcmQgcHJlcHJvY2Vzc2luZyBmb3IgdGhlIHBlcHRpZGUgZGF0YS4gVGhpcwppbmNsdWRlIGxvZyB0cmFuc2Zvcm1hdGlvbiwgZmlsdGVyaW5nIGFuZCBzdW1tYXJpc2F0aW9uIG9mIHRoZSBkYXRhLgoKIyMgTG9nIHRyYW5zZm9ybSB0aGUgZGF0YQoKYGBge3J9CnBlIDwtIGxvZ1RyYW5zZm9ybShwZSwgYmFzZSA9IDIsIGkgPSAicGVwdGlkZVJhdyIsIG5hbWUgPSAicGVwdGlkZUxvZyIpCmxpbW1hOjpwbG90RGVuc2l0aWVzKGFzc2F5KHBlW1sicGVwdGlkZUxvZyJdXSkpCmBgYAoKCiMjIEZpbHRlcmluZwoKIyMjIEhhbmRsaW5nIG92ZXJsYXBwaW5nIHByb3RlaW4gZ3JvdXBzCkluIG91ciBhcHByb2FjaCBhIHBlcHRpZGUgY2FuIG1hcCB0byBtdWx0aXBsZSBwcm90ZWlucywgYXMgbG9uZyBhcyB0aGVyZSBpcwpub25lIG9mIHRoZXNlIHByb3RlaW5zIHByZXNlbnQgaW4gYSBzbWFsbGVyIHN1Ymdyb3VwLgoKYGBge3J9CnBlW1sicGVwdGlkZUxvZyJdXSA8LQogIHBlW1sicGVwdGlkZUxvZyJdXVtyb3dEYXRhKHBlW1sicGVwdGlkZUxvZyJdXSkkUHJvdGVpbnMKICAlaW4lIHNtYWxsZXN0VW5pcXVlR3JvdXBzKHJvd0RhdGEocGVbWyJwZXB0aWRlTG9nIl1dKSRQcm90ZWlucyksXQpgYGAKCiMjIyBSZW1vdmUgcmV2ZXJzZSBzZXF1ZW5jZXMgKGRlY295cykgYW5kIGNvbnRhbWluYW50cwoKV2Ugbm93IHJlbW92ZSB0aGUgY29udGFtaW5hbnRzLCBwZXB0aWRlcyB0aGF0IG1hcCB0byBkZWNveSBzZXF1ZW5jZXMsIGFuZCBwcm90ZWlucwp3aGljaCB3ZXJlIG9ubHkgaWRlbnRpZmllZCBieSBwZXB0aWRlcyB3aXRoIG1vZGlmaWNhdGlvbnMuCgpgYGB7cn0KcGVbWyJwZXB0aWRlTG9nIl1dIDwtIHBlW1sicGVwdGlkZUxvZyJdXVtyb3dEYXRhKHBlW1sicGVwdGlkZUxvZyJdXSkkUmV2ZXJzZSAhPSAiKyIsIF0KcGVbWyJwZXB0aWRlTG9nIl1dIDwtIHBlW1sicGVwdGlkZUxvZyJdXVtyb3dEYXRhKHBlW1sicGVwdGlkZUxvZyJdXSkkClBvdGVudGlhbC5jb250YW1pbmFudCAhPSAiKyIsIF0KYGBgCgojIyMgUmVtb3ZlIHBlcHRpZGVzIG9mIHByb3RlaW5zIHRoYXQgd2VyZSBvbmx5IGlkZW50aWZpZWQgd2l0aCBtb2RpZmllZCBwZXB0aWRlcwoKSSB3aWxsIHNraXAgdGhpcyBzdGVwIGZvciB0aGUgbW9tZW50LiBMYXJnZSBwcm90ZWluIGdyb3VwcyBmaWxlIG5lZWRlZCBmb3IgdGhpcy4KCiMjIyBEcm9wIHBlcHRpZGVzIHRoYXQgd2VyZSBvbmx5IGlkZW50aWZpZWQgaW4gb25lIHNhbXBsZQoKV2Uga2VlcCBwZXB0aWRlcyB0aGF0IHdlcmUgb2JzZXJ2ZWQgYXQgbGFzdCB0d2ljZS4KCmBgYHtyfQpwZVtbInBlcHRpZGVMb2ciXV0gPC0gcGVbWyJwZXB0aWRlTG9nIl1dW3Jvd0RhdGEocGVbWyJwZXB0aWRlTG9nIl1dKSRuTm9uWmVybyA+PSAyLCBdCm5yb3cocGVbWyJwZXB0aWRlTG9nIl1dKQpgYGAKCldlIGtlZXAgYHIgbnJvdyhwZVtbInBlcHRpZGVMb2ciXV0pYCBwZXB0aWRlcyBhZnRlciBmaWx0ZXJpbmcuCgojIyBRdWFudGlsZSBub3JtYWxpemUgdGhlIGRhdGEKYGBge3J9CnBlIDwtIG5vcm1hbGl6ZShwZSwgaSA9ICJwZXB0aWRlTG9nIiwgbWV0aG9kID0gInF1YW50aWxlcyIsIG5hbWUgPSAicGVwdGlkZU5vcm0iKQpgYGAKCgojIyBFeHBsb3JlIHF1YW50aWxlIG5vcm1hbGl6ZWQgZGF0YQoKQWZ0ZXIgcXVhbnRpbGUgbm9ybWFsaXNhdGlvbiB0aGUgZGVuc2l0eSBjdXJ2ZXMgZm9yIGFsbCBzYW1wbGVzIGNvaW5jaWRlLgoKYGBge3J9CmxpbW1hOjpwbG90RGVuc2l0aWVzKGFzc2F5KHBlW1sicGVwdGlkZU5vcm0iXV0pKQpgYGAKClRoaXMgaXMgbW9yZSBjbGVhcmx5IHNlZW4gaXMgYSBib3hwbG90LgoKYGBge3IsfQpib3hwbG90KGFzc2F5KHBlW1sicGVwdGlkZU5vcm0iXV0pLCBjb2wgPSBwYWxldHRlKClbLTFdLAogICAgICAgIG1haW4gPSAiUGVwdGlkZSBkaXN0cmlidHV0aW9ucyBhZnRlciBub3JtYWxpc2F0aW9uIiwgeWxhYiA9ICJpbnRlbnNpdHkiKQoKYGBgCgoKV2UgY2FuIHZpc3VhbGl6ZSBvdXIgZGF0YSB1c2luZyBhIE11bHRpIERpbWVuc2lvbmFsIFNjYWxpbmcgcGxvdCwKZWcuIGFzIHByb3ZpZGVkIGJ5IHRoZSBgbGltbWFgIHBhY2thZ2UuCgpgYGB7cn0KbGltbWE6OnBsb3RNRFMoYXNzYXkocGVbWyJwZXB0aWRlTm9ybSJdXSksIGNvbCA9IGFzLm51bWVyaWMoY29sRGF0YShwZSkkY29uZGl0aW9uKSkKYGBgCgpUaGUgZmlyc3QgYXhpcyBpbiB0aGUgcGxvdCBpcyBzaG93aW5nIHRoZSBsZWFkaW5nIGxvZyBmb2xkIGNoYW5nZXMKKGRpZmZlcmVuY2VzIG9uIHRoZSBsb2cgc2NhbGUpIGJldHdlZW4gdGhlIHNhbXBsZXMuCldlIG5vdGljZSB0aGF0IHRoZSBsZWFkaW5nIGRpZmZlcmVuY2VzIChsb2cgRkMpCmluIHRoZSBwZXB0aWRlUmF3IGRhdGEgc2VlbXMgdG8gYmUgZHJpdmVuIGJ5IHRlY2huaWNhbCB2YXJpYWJpbGl0eS4KSW5kZWVkLCB0aGUgc2FtcGxlcyBkbyBub3Qgc2VlbSB0byBiZSBjbGVhcmx5IHNlcGFyYXRlZCBhY2NvcmRpbmcKdG8gdGhlIHNwaWtlLWluIGNvbmRpdGlvbi4KCgojIyBTdW1tYXJpemF0aW9uIHRvIHByb3RlaW4gbGV2ZWwKCldlIHVzZSBtZWRpYW4gc3VtbWFyaXphdGlvbiBpbiBhZ2dyZWdhdGVGZWF0dXJlcy4KCmBgYHtyLHdhcm5pbmc9RkFMU0V9CnBlIDwtIGFnZ3JlZ2F0ZUZlYXR1cmVzKHBlLAogIGkgPSAicGVwdGlkZU5vcm0iLAogIGZjb2wgPSAiUHJvdGVpbnMiLAogIG5hLnJtID0gVFJVRSwKICBuYW1lID0gInByb3RlaW5NZWRpYW4iLAogIGZ1biA9IG1hdHJpeFN0YXRzOjpjb2xNZWRpYW5zKQpgYGAKCgpXZSBub3RpY2UgdGhhdCB0aGUgbGVhZGluZyBkaWZmZXJlbmNlcyAobG9nIEZDKSBpbiB0aGUgcHJvdGVpbiBkYXRhIGFyZSBzdGlsbAphY2NvcmRpbmcgdG8gdGVjaG5pY2FsIHZhcmlhdGlvbi4KCmBgYHtyfQpwbG90TURTKGFzc2F5KHBlW1sicHJvdGVpbk1lZGlhbiJdXSksIGNvbCA9IGFzLm51bWVyaWMoY29sRGF0YShwZSkkY29uZGl0aW9uKSkKYGBgCgojIERhdGEgQW5hbHlzaXMKCiMjIEVzdGltYXRpb24KCldlIG1vZGVsIHRoZSBwcm90ZWluIGxldmVsIGV4cHJlc3Npb24gdmFsdWVzIHVzaW5nIGBtc3Fyb2JgLgpCeSBkZWZhdWx0IGBtc3Fyb2IyYCBlc3RpbWF0ZXMgdGhlIG1vZGVsIHBhcmFtZXRlcnMgdXNpbmcgcm9idXN0IHJlZ3Jlc3Npb24uICAKCmBgYHtyLCB3YXJuaW5nPUZBTFNFfQpwZSA8LSBtc3Fyb2Iob2JqZWN0ID0gcGUsIGkgPSAicHJvdGVpbk1lZGlhbiIsIGZvcm11bGEgPSB+Y29uZGl0aW9uKQpgYGAKCiMjIEluZmVyZW5jZQoKRmlyc3QsIHdlIGV4dHJhY3QgdGhlIHBhcmFtZXRlciBuYW1lcyBvZiB0aGUgbW9kZWwuCmBgYHtyfQpnZXRDb2VmKHJvd0RhdGEocGVbWyJwcm90ZWluTWVkaWFuIl1dKSRtc3Fyb2JNb2RlbHNbWzFdXSkKYGBgCgpTcGlrZS1pbiBjb25kaXRpb24gYSBpcyB0aGUgcmVmZXJlbmNlIGNsYXNzLiBTbyB0aGUgbWVhbiBsb2cyIGV4cHJlc3Npb24KZm9yIHNhbXBsZXMgZnJvbSBjb25kaXRpb24gYSBpcyAnKEludGVyY2VwdCkuClRoZSBtZWFuIGxvZzIgZXhwcmVzc2lvbiBmb3Igc2FtcGxlcyBmcm9tIGNvbmRpdGlvbiBCIGlzICcoSW50ZXJjZXB0KStjb25kaXRpb25CJy4KSGVuY2UsIHRoZSBhdmVyYWdlIGxvZzIgZm9sZCBjaGFuZ2UgYmV0d2VlbiBjb25kaXRpb24gYiBhbmQKY29uZGl0aW9uIGEgaXMgbW9kZWxsZWQgdXNpbmcgdGhlIHBhcmFtZXRlciAnY29uZGl0aW9uQicuClRodXMsIHdlIGFzc2VzcyB0aGUgY29udHJhc3QgJ2NvbmRpdGlvbkI9MCcgd2l0aCBvdXIgc3RhdGlzdGljYWwgdGVzdC4KCmBgYHtyfQpMIDwtIG1ha2VDb250cmFzdCgiY29uZGl0aW9uQj0wIiwgcGFyYW1ldGVyTmFtZXMgPSBjKCJjb25kaXRpb25CIikpCnBlIDwtIGh5cG90aGVzaXNUZXN0KG9iamVjdCA9IHBlLCBpID0gInByb3RlaW5NZWRpYW4iLCBjb250cmFzdCA9IEwpCmBgYAoKCiMjIFBsb3RzCgojIyMgVm9sY2Fuby1wbG90CgoKYGBge3Isd2FybmluZz1GQUxTRX0Kdm9sY2FubyA8LSBnZ3Bsb3Qocm93RGF0YShwZVtbInByb3RlaW5NZWRpYW4iXV0pJGNvbmRpdGlvbkIsCiAgICAgICAgICAgICAgICAgIGFlcyh4ID0gbG9nRkMsIHkgPSAtbG9nMTAocHZhbCksIGNvbG9yID0gYWRqUHZhbCA8IDAuMDUpKSArCiAgZ2VvbV9wb2ludChjZXggPSAyLjUpICsKICBzY2FsZV9jb2xvcl9tYW51YWwodmFsdWVzID0gYWxwaGEoYygiYmxhY2siLCAicmVkIiksIDAuNSkpICsgdGhlbWVfbWluaW1hbCgpCnZvbGNhbm8KYGBgCgojIyMgSGVhdG1hcAoKV2UgZmlyc3Qgc2VsZWN0IHRoZSBuYW1lcyBvZiB0aGUgcHJvdGVpbnMgdGhhdCB3ZXJlIGRlY2xhcmVkIHNpZ25maWNhbnQuCgpgYGB7cn0Kc2lnTmFtZXMgPC0gcm93RGF0YShwZVtbInByb3RlaW5NZWRpYW4iXV0pJGNvbmRpdGlvbkIgJT4lCiAgcm93bmFtZXNfdG9fY29sdW1uKCJwcm90ZWluTWVkaWFuIikgJT4lCiAgZmlsdGVyKGFkalB2YWw8MC4wNSkgJT4lCiAgcHVsbChwcm90ZWluTWVkaWFuKQpoZWF0bWFwKGFzc2F5KHBlW1sicHJvdGVpbk1lZGlhbiJdXSlbc2lnTmFtZXMsIF0pCmBgYAoKIyMjIEJveHBsb3RzCgpXZSBtYWtlIGJveHBsb3Qgb2YgdGhlIGxvZzIgRkMgYW5kIHN0cmF0aWZ5IGFjY29yZGluZyB0byB0aGUgd2hldGhlciBhIHByb3RlaW4gaXMgc3Bpa2VkIG9yIG5vdC4KCmBgYHtyfQpyb3dEYXRhKHBlW1sicHJvdGVpbk1lZGlhbiJdXSkkY29uZGl0aW9uQiAlPiUKICByb3duYW1lc190b19jb2x1bW4odmFyID0gInByb3RlaW4iKSAlPiUKICBnZ3Bsb3QoYWVzKHg9Z3JlcGwoIlVQUyIscHJvdGVpbikseT1sb2dGQykpICsKICBnZW9tX2JveHBsb3QoKSArCiAgeGxhYigiVVBTIikgKwogIGdlb21fc2VnbWVudCgKICAgIHggPSAxLjUsCiAgICB4ZW5kID0gMi41LAogICAgeSA9IGxvZzIoMC43NC8wLjI1KSwKICAgIHllbmQgPSBsb2cyKDAuNzQvMC4yNSksCiAgICBjb2xvdXI9InJlZCIpICsKICBnZW9tX3NlZ21lbnQoCiAgICB4ID0gMC41LAogICAgeGVuZCA9IDEuNSwKICAgIHkgPSAwLAogICAgeWVuZCA9IDAsCiAgICBjb2xvdXI9InJlZCIpICsKICBhbm5vdGF0ZSgKICAgICJ0ZXh0IiwKICAgIHggPSBjKDEsMiksCiAgICB5ID0gYygwLGxvZzIoMC43NC8wLjI1KSkrLjEsCiAgICBsYWJlbCA9IGMoCiAgICAgICJsb2cyIEZDIEVjb2xpID0gMCIsCiAgICAgIHBhc3RlMCgibG9nMiBGQyBVUFMgPSAiLHJvdW5kKGxvZzIoMC43NC8wLjI1KSwyKSkKICAgICAgKSwKICAgIGNvbG91ciA9ICJyZWQiKQpgYGAKCldoYXQgZG8geW91IG9ic2VydmU/CgojIyMgRGV0YWlsIHBsb3RzCgpXZSBmaXJzdCBleHRyYWN0IHRoZSBub3JtYWxpemVkIHBlcHRpZGVSYXcgZXhwcmVzc2lvbiB2YWx1ZXMgZm9yIGEgcGFydGljdWxhciBwcm90ZWluLiAgCgoKYGBge3IsIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CmZvciAocHJvdE5hbWUgaW4gc2lnTmFtZXMpCnsKcGVQbG90IDwtIHBlW3Byb3ROYW1lLCAsIGMoInBlcHRpZGVOb3JtIiwicHJvdGVpbk1lZGlhbiIpXQpwZVBsb3REZiA8LSBkYXRhLmZyYW1lKGxvbmdGb3JtYXQocGVQbG90KSkKcGVQbG90RGYkYXNzYXkgPC0gZmFjdG9yKHBlUGxvdERmJGFzc2F5LAogICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKCJwZXB0aWRlTm9ybSIsICJwcm90ZWluTWVkaWFuIikpCnBlUGxvdERmJGNvbmRpdGlvbiA8LSBhcy5mYWN0b3IoY29sRGF0YShwZVBsb3QpW3BlUGxvdERmJGNvbG5hbWUsICJjb25kaXRpb24iXSkKCiMgcGxvdHRpbmcKcDEgPC0gZ2dwbG90KGRhdGEgPSBwZVBsb3REZiwKICAgICAgIGFlcyh4ID0gY29sbmFtZSwgeSA9IHZhbHVlLCBncm91cCA9IHJvd25hbWUpKSArCiAgICBnZW9tX2xpbmUoKSArIGdlb21fcG9pbnQoKSArICB0aGVtZV9taW5pbWFsKCkgKwogICAgZmFjZXRfZ3JpZCh+YXNzYXkpICsgZ2d0aXRsZShwcm90TmFtZSkKcHJpbnQocDEpCgojIHBsb3R0aW5nIDIKcDIgPC0gZ2dwbG90KHBlUGxvdERmLCBhZXMoeCA9IGNvbG5hbWUsIHkgPSB2YWx1ZSwgZmlsbCA9IGNvbmRpdGlvbikpICsKICBnZW9tX2JveHBsb3Qob3V0bGllci5zaGFwZSA9IE5BKSArIGdlb21fcG9pbnQocG9zaXRpb24gPSBwb3NpdGlvbl9qaXR0ZXIod2lkdGggPSAuMSksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGFlcyhzaGFwZSA9IHJvd25hbWUpKSArCiAgc2NhbGVfc2hhcGVfbWFudWFsKHZhbHVlcyA9IDE6bnJvdyhwZVBsb3REZikpICsKICBsYWJzKHRpdGxlID0gcHJvdE5hbWUsIHggPSAic2FtcGxlIiwgeSA9ICJwZXB0aWRlIGludGVuc2l0eSAobG9nMikiKSArIHRoZW1lX21pbmltYWwoKQogIGZhY2V0X2dyaWQofmFzc2F5KQpwcmludChwMikKfQpgYGAK