This is part of the online course Proteomics Data Analysis 2021 (PDA21)
Background
This case-study is a subset of the data of the 6th study of the Clinical Proteomic Technology Assessment for Cancer (CPTAC). In this experiment, the authors spiked the Sigma Universal Protein Standard mixture 1 (UPS1) containing 48 different human proteins in a protein background of 60 ng/\(\mu\)L Saccharomyces cerevisiae strain BY4741. Two different spike-in concentrations were used: 6A (0.25 fmol UPS1 proteins/\(\mu\)L) and 6B (0.74 fmol UPS1 proteins/\(\mu\)L) [5]. We limited ourselves to the data of LTQ-Orbitrap W at site 56. The data were searched with MaxQuant version 1.5.2.8, and detailed search settings were described in Goeminne et al. (2016) [1]. Three replicates are available for each concentration.
Data
We first import the data from peptideRaws.txt file. This is the file containing your peptideRaw-level intensities. For a MaxQuant search [6], this peptideRaws.txt file can be found by default in the “path_to_raw_files/combined/txt/” folder from the MaxQuant output, with “path_to_raw_files” the folder where the raw files were saved. In this vignette, we use a MaxQuant peptideRaws file which is a subset of the cptac study. This data is available in the msdata
package. To import the data we use the QFeatures
package.
We generate the object peptideRawFile with the path to the peptideRaws.txt file. Using the grepEcols
function, we find the columns that contain the expression data of the peptideRaws in the peptideRaws.txt file.
library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)
peptidesFile <- "https://raw.githubusercontent.com/statOmics/SGA2020/data/quantification/cptacAvsB_lab3/peptides.txt"
ecols <- grep(
"Intensity\\.",
names(read.delim(peptidesFile))
)
pe <- readQFeatures(
table = peptidesFile,
fnames = 1,
ecol = ecols,
name = "peptideRaw", sep="\t")
colnames(pe)
## CharacterList of length 1
## [["peptideRaw"]] Intensity.6A_7 Intensity.6A_8 ... Intensity.6B_9
In the following code chunk, we can extract the spikein condition from the raw file name.
cond <- which(
strsplit(colnames(pe)[[1]][1], split = "")[[1]] == "A") # find where condition is stored
colData(pe)$condition <- substr(colnames(pe), cond, cond) %>%
unlist %>%
as.factor
We calculate how many non zero intensities we have per peptide and this will be useful for filtering.
rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)
Peptides with zero intensities are missing peptides and should be represent with a NA
value rather than 0
.
pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA
Data exploration
45% of all peptide intensities are missing and for some peptides we do not even measure a signal in any sample.
Preprocessing
This section preforms preprocessing for the peptide data. This include
- log transformation,
- filtering and
- summarisation of the data.
Filtering
- Handling overlapping protein groups
In our approach a peptide can map to multiple proteins, as long as there is none of these proteins present in a smaller subgroup.
pe <- filterFeatures(pe, ~ Proteins %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins))
- Remove reverse sequences (decoys) and contaminants
We now remove the contaminants and peptides that map to decoy sequences.
pe <- filterFeatures(pe,~Reverse != "+")
pe <- filterFeatures(pe,~ Potential.contaminant != "+")
- Drop peptides that were only identified in one sample
We keep peptides that were observed at last twice.
pe <- filterFeatures(pe,~ nNonZero >=2)
nrow(pe[["peptideLog"]])
## [1] 7011
We keep 7011 peptides upon filtering.
Explore normalized data
Upon the normalisation the density curves are nicely registered
pe[["peptideNorm"]] %>%
assay %>%
as.data.frame() %>%
gather(sample, intensity) %>%
mutate(condition = colData(pe)[sample,"condition"]) %>%
ggplot(aes(x = intensity,group = sample,color = condition)) +
geom_density()
## Warning: Removed 8167 rows containing non-finite values (stat_density).
We can visualize our data using a Multi Dimensional Scaling plot, eg. as provided by the limma
package.
pe[["peptideNorm"]] %>%
assay %>%
limma::plotMDS(col = as.numeric(colData(pe)$condition))
The first axis in the plot is showing the leading log fold changes (differences on the log scale) between the samples.
We notice that the leading differences (log FC) in the peptide data seems to be driven by technical variability. Indeed, the samples do not seem to be clearly separated according to the spike-in condition.
Summarization to protein level
- By default robust summarization is used:
fun = MsCoreUtils::robustSummary()
pe <- aggregateFeatures(pe,
i = "peptideNorm",
fcol = "Proteins",
na.rm = TRUE,
name = "protein")
## Your quantitative and row data contain missing values. Please read the
## relevant section(s) in the aggregateFeatures manual page regarding the
## effects of missing values on data aggregation.
plotMDS(assay(pe[["protein"]]), col = as.numeric(colData(pe)$condition))
Note that the samples upon robust summarisation show a clear separation according to the spike-in condition in the second dimension of the MDS plot.
Data Analysis
Estimation
We model the protein level expression values using msqrob
. By default msqrob2
estimates the model parameters using robust regression.
We will model the data with a different group mean. The group is incoded in the variable condition
of the colData. We can specify this model by using a formula with the factor condition as its predictor: formula = ~condition
.
Note, that a formula always starts with a symbol ‘~’.
pe <- msqrob(object = pe, i = "protein", formula = ~condition)
Inference
First, we extract the parameter names of the model by looking at the first model. The models are stored in the row data of the assay under the default name msqrobModels.
getCoef(rowData(pe[["protein"]])$msqrobModels[[1]])
## (Intercept) conditionB
## -2.672396 1.513682
We can also explore the design of the model that we specified using the the package ExploreModelMatrix
library(ExploreModelMatrix)
VisualizeDesign(colData(pe),~condition)$plotlist[[1]]
Spike-in condition A
is the reference class. So the mean log2 expression for samples from condition A is ‘(Intercept). The mean log2 expression for samples from condition B is’(Intercept)+conditionB’. Hence, the average log2 fold change between condition b and condition a is modelled using the parameter ‘conditionB’. Thus, we assess the contrast ‘conditionB = 0’ with our statistical test.
L <- makeContrast("conditionB=0", parameterNames = c("conditionB"))
pe <- hypothesisTest(object = pe, i = "protein", contrast = L)
Plots
Volcano-plot
volcano <- ggplot(rowData(pe[["protein"]])$conditionB,
aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
geom_point(cex = 2.5) +
scale_color_manual(values = alpha(c("black", "red"), 0.5)) + theme_minimal()
volcano
Note, that 20 proteins are found to be differentially abundant.
Heatmap
We first select the names of the proteins that were declared signficant.
sigNames <- rowData(pe[["protein"]])$conditionB %>%
rownames_to_column("protein") %>%
filter(adjPval<0.05) %>%
pull(protein)
heatmap(assay(pe[["protein"]])[sigNames, ])
The majority of the proteins are indeed UPS proteins. 1 yeast protein is returned. Note, that the yeast protein indeed shows evidence for differential abundance.
Boxplots
We make boxplot of the log2 FC and stratify according to the whether a protein is spiked or not.
rowData(pe[["protein"]])$conditionB %>%
rownames_to_column(var = "protein") %>%
ggplot(aes(x=grepl("UPS",protein),y=logFC)) +
geom_boxplot() +
xlab("UPS") +
geom_segment(
x = 1.5,
xend = 2.5,
y = log2(0.74/0.25),
yend = log2(0.74/0.25),
colour="red") +
geom_segment(
x = 0.5,
xend = 1.5,
y = 0,
yend = 0,
colour="red") +
annotate(
"text",
x = c(1,2),
y = c(0,log2(0.74/0.25))+.1,
label = c(
"log2 FC Ecoli = 0",
paste0("log2 FC UPS = ",round(log2(0.74/0.25),2))
),
colour = "red")
## Warning: Removed 167 rows containing non-finite values (stat_boxplot).
What do you observe?
Detail plots
We first extract the normalized peptideRaw expression values for a particular protein.
for (protName in sigNames)
{
pePlot <- pe[protName, , c("peptideNorm","protein")]
pePlotDf <- data.frame(longFormat(pePlot))
pePlotDf$assay <- factor(pePlotDf$assay,
levels = c("peptideNorm", "protein"))
pePlotDf$condition <- as.factor(colData(pePlot)[pePlotDf$colname, "condition"])
# plotting
p1 <- ggplot(data = pePlotDf,
aes(x = colname, y = value, group = rowname)) +
geom_line() +
geom_point() +
theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
facet_grid(~assay) +
ggtitle(protName)
print(p1)
# plotting 2
p2 <- ggplot(pePlotDf, aes(x = colname, y = value, fill = condition)) +
geom_boxplot(outlier.shape = NA) +
geom_point(
position = position_jitter(width = .1),
aes(shape = rowname)) +
scale_shape_manual(values = 1:nrow(pePlotDf)) +
labs(title = protName, x = "sample", y = "peptide intensity (log2)") +
theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
facet_grid(~assay)
print(p2)
}
Note, that the yeast protein is only covered by 3 peptides. Only one peptide is picked up in condition A. This peptide is also only once observed in spike-in condition B. This puts a considerable burden upon the inference and could be avoided by more stringent filtering.
Session Info
With respect to reproducibility, it is highly recommended to include a session info in your script so that readers of your output can see your particular setup of R.
## R version 4.1.2 (2021-11-01)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur 10.16
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
## [1] stats4 stats graphics grDevices utils datasets methods
## [8] base
##
## other attached packages:
## [1] ExploreModelMatrix_1.6.0 plotly_4.10.0
## [3] msqrob2_1.2.0 QFeatures_1.4.0
## [5] MultiAssayExperiment_1.20.0 SummarizedExperiment_1.24.0
## [7] Biobase_2.54.0 GenomicRanges_1.46.1
## [9] GenomeInfoDb_1.30.0 IRanges_2.28.0
## [11] S4Vectors_0.32.3 BiocGenerics_0.40.0
## [13] MatrixGenerics_1.6.0 matrixStats_0.61.0
## [15] limma_3.50.0 forcats_0.5.1
## [17] stringr_1.4.0 dplyr_1.0.7
## [19] purrr_0.3.4 readr_2.1.1
## [21] tidyr_1.1.4 tibble_3.1.6
## [23] ggplot2_3.3.5 tidyverse_1.3.1
##
## loaded via a namespace (and not attached):
## [1] minqa_1.2.4 colorspace_2.0-2 ellipsis_0.3.2
## [4] XVector_0.34.0 fs_1.5.1 clue_0.3-60
## [7] rstudioapi_0.13 farver_2.1.0 DT_0.20
## [10] fansi_0.5.0 lubridate_1.8.0 xml2_1.3.3
## [13] codetools_0.2-18 splines_4.1.2 knitr_1.36
## [16] jsonlite_1.7.2 nloptr_1.2.2.3 broom_0.7.10
## [19] cluster_2.1.2 dbplyr_2.1.1 shinydashboard_0.7.2
## [22] shiny_1.7.1 compiler_4.1.2 httr_1.4.2
## [25] backports_1.4.0 assertthat_0.2.1 Matrix_1.3-4
## [28] fastmap_1.1.0 lazyeval_0.2.2 cli_3.1.0
## [31] later_1.3.0 htmltools_0.5.2 tools_4.1.2
## [34] igraph_1.2.9 gtable_0.3.0 glue_1.5.1
## [37] GenomeInfoDbData_1.2.7 Rcpp_1.0.7 cellranger_1.1.0
## [40] jquerylib_0.1.4 vctrs_0.3.8 nlme_3.1-153
## [43] rintrojs_0.3.0 xfun_0.28 lme4_1.1-27.1
## [46] rvest_1.0.2 mime_0.12 lifecycle_1.0.1
## [49] zlibbioc_1.40.0 MASS_7.3-54 scales_1.1.1
## [52] promises_1.2.0.1 hms_1.1.1 ProtGenerics_1.26.0
## [55] parallel_4.1.2 AnnotationFilter_1.18.0 yaml_2.2.1
## [58] sass_0.4.0 stringi_1.7.6 highr_0.9
## [61] boot_1.3-28 BiocParallel_1.28.2 rlang_0.4.12
## [64] pkgconfig_2.0.3 bitops_1.0-7 evaluate_0.14
## [67] lattice_0.20-45 htmlwidgets_1.5.4 labeling_0.4.2
## [70] cowplot_1.1.1 tidyselect_1.1.1 magrittr_2.0.1
## [73] R6_2.5.1 generics_0.1.1 DelayedArray_0.20.0
## [76] DBI_1.1.1 pillar_1.6.4 haven_2.4.3
## [79] withr_2.4.3 MsCoreUtils_1.6.0 RCurl_1.98-1.5
## [82] modelr_0.1.8 crayon_1.4.2 utf8_1.2.2
## [85] tzdb_0.2.0 rmarkdown_2.11 grid_4.1.2
## [88] readxl_1.3.1 data.table_1.14.2 reprex_2.0.1
## [91] digest_0.6.29 xtable_1.8-4 httpuv_1.6.3
## [94] munsell_0.5.0 viridisLite_0.4.0 bslib_0.3.1
## [97] shinyjs_2.0.0
LS0tCnRpdGxlOiAiSW50cm9kdWN0aW9uIHRvIHByb3Rlb21pY3MgZGF0YSBhbmFseXNpczogcm9idXN0IHN1bW1hcml6YXRpb24iCmF1dGhvcjogIkxpZXZlbiBDbGVtZW50IgpkYXRlOiAic3RhdE9taWNzLCBHaGVudCBVbml2ZXJzaXR5IChodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8pIgpvdXRwdXQ6CiAgICBodG1sX2RvY3VtZW50OgogICAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAgIHRoZW1lOiBjb3NtbwogICAgICB0b2M6IHRydWUKICAgICAgdG9jX2Zsb2F0OiB0cnVlCiAgICAgIGhpZ2hsaWdodDogdGFuZ28KICAgICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCiAgICBwZGZfZG9jdW1lbnQ6CiAgICAgIHRvYzogdHJ1ZQogICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKbGlua2NvbG9yOiBibHVlCnVybGNvbG9yOiBibHVlCmNpdGVjb2xvcjogYmx1ZQoKYmlibGlvZ3JhcGh5OiBtc3Fyb2IyLmJpYgoKLS0tCgo8YSByZWw9ImxpY2Vuc2UiIGhyZWY9Imh0dHBzOi8vY3JlYXRpdmVjb21tb25zLm9yZy9saWNlbnNlcy9ieS1uYy1zYS80LjAiPjxpbWcgYWx0PSJDcmVhdGl2ZSBDb21tb25zIExpY2Vuc2UiIHN0eWxlPSJib3JkZXItd2lkdGg6MCIgc3JjPSJodHRwczovL2kuY3JlYXRpdmVjb21tb25zLm9yZy9sL2J5LW5jLXNhLzQuMC84OHgzMS5wbmciIC8+PC9hPgoKVGhpcyBpcyBwYXJ0IG9mIHRoZSBvbmxpbmUgY291cnNlIFtQcm90ZW9taWNzIERhdGEgQW5hbHlzaXMgMjAyMSAoUERBMjEpXShodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8vUERBMjEvKQoKIyBCYWNrZ3JvdW5kClRoaXMgY2FzZS1zdHVkeSBpcyBhIHN1YnNldCBvZiB0aGUgZGF0YSBvZiB0aGUgNnRoIHN0dWR5IG9mIHRoZSBDbGluaWNhbApQcm90ZW9taWMgVGVjaG5vbG9neSBBc3Nlc3NtZW50IGZvciBDYW5jZXIgKENQVEFDKS4KSW4gdGhpcyBleHBlcmltZW50LCB0aGUgYXV0aG9ycyBzcGlrZWQgdGhlIFNpZ21hIFVuaXZlcnNhbCBQcm90ZWluIFN0YW5kYXJkCm1peHR1cmUgMSAoVVBTMSkgY29udGFpbmluZyA0OCBkaWZmZXJlbnQgaHVtYW4gcHJvdGVpbnMgaW4gYSBwcm90ZWluIGJhY2tncm91bmQKb2YgNjAgbmcvJFxtdSRMIFNhY2NoYXJvbXljZXMgY2VyZXZpc2lhZSBzdHJhaW4gQlk0NzQxLgpUd28gZGlmZmVyZW50IHNwaWtlLWluIGNvbmNlbnRyYXRpb25zIHdlcmUgdXNlZDoKNkEgKDAuMjUgZm1vbCBVUFMxIHByb3RlaW5zLyRcbXUkTCkgYW5kIDZCICgwLjc0IGZtb2wgVVBTMSBwcm90ZWlucy8kXG11JEwpIFs1XS4KV2UgbGltaXRlZCBvdXJzZWx2ZXMgdG8gdGhlIGRhdGEgb2YgTFRRLU9yYml0cmFwIFcgYXQgc2l0ZSA1Ni4KVGhlIGRhdGEgd2VyZSBzZWFyY2hlZCB3aXRoIE1heFF1YW50IHZlcnNpb24gMS41LjIuOCwgYW5kCmRldGFpbGVkIHNlYXJjaCBzZXR0aW5ncyB3ZXJlIGRlc2NyaWJlZCBpbiBHb2VtaW5uZSBldCBhbC4gKDIwMTYpIFsxXS4KVGhyZWUgcmVwbGljYXRlcyBhcmUgYXZhaWxhYmxlIGZvciBlYWNoIGNvbmNlbnRyYXRpb24uCgoKIyBEYXRhCgpXZSBmaXJzdCBpbXBvcnQgdGhlIGRhdGEgZnJvbSBwZXB0aWRlUmF3cy50eHQgZmlsZS4gVGhpcyBpcyB0aGUgZmlsZSBjb250YWluaW5nCnlvdXIgcGVwdGlkZVJhdy1sZXZlbCBpbnRlbnNpdGllcy4gRm9yIGEgTWF4UXVhbnQgc2VhcmNoIFs2XSwKdGhpcyBwZXB0aWRlUmF3cy50eHQgZmlsZSBjYW4gYmUgZm91bmQgYnkgZGVmYXVsdCBpbiB0aGUKInBhdGhfdG9fcmF3X2ZpbGVzL2NvbWJpbmVkL3R4dC8iIGZvbGRlciBmcm9tIHRoZSBNYXhRdWFudCBvdXRwdXQsCndpdGggInBhdGhfdG9fcmF3X2ZpbGVzIiB0aGUgZm9sZGVyIHdoZXJlIHRoZSByYXcgZmlsZXMgd2VyZSBzYXZlZC4KSW4gdGhpcyB2aWduZXR0ZSwgd2UgdXNlIGEgTWF4UXVhbnQgcGVwdGlkZVJhd3MgZmlsZSB3aGljaCBpcyBhIHN1YnNldApvZiB0aGUgY3B0YWMgc3R1ZHkuIFRoaXMgZGF0YSBpcyBhdmFpbGFibGUgaW4gdGhlIGBtc2RhdGFgIHBhY2thZ2UuClRvIGltcG9ydCB0aGUgZGF0YSB3ZSB1c2UgdGhlIGBRRmVhdHVyZXNgIHBhY2thZ2UuCgpXZSBnZW5lcmF0ZSB0aGUgb2JqZWN0IHBlcHRpZGVSYXdGaWxlIHdpdGggdGhlIHBhdGggdG8gdGhlIHBlcHRpZGVSYXdzLnR4dCBmaWxlLgpVc2luZyB0aGUgYGdyZXBFY29sc2AgZnVuY3Rpb24sIHdlIGZpbmQgdGhlIGNvbHVtbnMgdGhhdCBjb250YWluIHRoZSBleHByZXNzaW9uCmRhdGEgb2YgdGhlIHBlcHRpZGVSYXdzIGluIHRoZSBwZXB0aWRlUmF3cy50eHQgZmlsZS4KCgpgYGB7ciwgd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KbGlicmFyeSh0aWR5dmVyc2UpCmxpYnJhcnkobGltbWEpCmxpYnJhcnkoUUZlYXR1cmVzKQpsaWJyYXJ5KG1zcXJvYjIpCmxpYnJhcnkocGxvdGx5KQoKcGVwdGlkZXNGaWxlIDwtICJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1NHQTIwMjAvZGF0YS9xdWFudGlmaWNhdGlvbi9jcHRhY0F2c0JfbGFiMy9wZXB0aWRlcy50eHQiCgplY29scyA8LSBncmVwKAogICJJbnRlbnNpdHlcXC4iLCAKICBuYW1lcyhyZWFkLmRlbGltKHBlcHRpZGVzRmlsZSkpCiAgKQoKcGUgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHBlcHRpZGVzRmlsZSwKICBmbmFtZXMgPSAxLAogIGVjb2wgPSBlY29scywKICBuYW1lID0gInBlcHRpZGVSYXciLCBzZXA9Ilx0IikKCmNvbG5hbWVzKHBlKQpgYGAKCkluIHRoZSBmb2xsb3dpbmcgY29kZSBjaHVuaywgd2UgY2FuIGV4dHJhY3QgdGhlIHNwaWtlaW4gY29uZGl0aW9uIGZyb20gdGhlIHJhdyBmaWxlIG5hbWUuCgpgYGB7cn0KY29uZCA8LSB3aGljaCgKICBzdHJzcGxpdChjb2xuYW1lcyhwZSlbWzFdXVsxXSwgc3BsaXQgPSAiIilbWzFdXSA9PSAiQSIpICMgZmluZCB3aGVyZSBjb25kaXRpb24gaXMgc3RvcmVkCgpjb2xEYXRhKHBlKSRjb25kaXRpb24gPC0gc3Vic3RyKGNvbG5hbWVzKHBlKSwgY29uZCwgY29uZCkgJT4lCiAgdW5saXN0ICU+JSAgCiAgYXMuZmFjdG9yCmBgYAoKCldlIGNhbGN1bGF0ZSBob3cgbWFueSBub24gemVybyBpbnRlbnNpdGllcyB3ZSBoYXZlIHBlciBwZXB0aWRlIGFuZCB0aGlzCndpbGwgYmUgdXNlZnVsIGZvciBmaWx0ZXJpbmcuCgpgYGB7cn0Kcm93RGF0YShwZVtbInBlcHRpZGVSYXciXV0pJG5Ob25aZXJvIDwtIHJvd1N1bXMoYXNzYXkocGVbWyJwZXB0aWRlUmF3Il1dKSA+IDApCmBgYAoKClBlcHRpZGVzIHdpdGggemVybyBpbnRlbnNpdGllcyBhcmUgbWlzc2luZyBwZXB0aWRlcyBhbmQgc2hvdWxkIGJlIHJlcHJlc2VudAp3aXRoIGEgYE5BYCB2YWx1ZSByYXRoZXIgdGhhbiBgMGAuCmBgYHtyfQpwZSA8LSB6ZXJvSXNOQShwZSwgInBlcHRpZGVSYXciKSAjIGNvbnZlcnQgMCB0byBOQQpgYGAKCgojIyBEYXRhIGV4cGxvcmF0aW9uCgpgciBmb3JtYXQobWVhbihpcy5uYShhc3NheShwZVtbInBlcHRpZGVSYXciXV0pKSkqMTAwLGRpZ2l0cz0yKWAlIG9mIGFsbCBwZXB0aWRlCmludGVuc2l0aWVzIGFyZSBtaXNzaW5nIGFuZCBmb3Igc29tZSBwZXB0aWRlcyB3ZSBkbyBub3QgZXZlbiBtZWFzdXJlIGEgc2lnbmFsCmluIGFueSBzYW1wbGUuCgoKIyBQcmVwcm9jZXNzaW5nCgpUaGlzIHNlY3Rpb24gcHJlZm9ybXMgcHJlcHJvY2Vzc2luZyBmb3IgdGhlIHBlcHRpZGUgZGF0YS4gClRoaXMgaW5jbHVkZSAKCi0gbG9nIHRyYW5zZm9ybWF0aW9uLCAKLSBmaWx0ZXJpbmcgYW5kIAotIHN1bW1hcmlzYXRpb24gb2YgdGhlIGRhdGEuCgojIyBMb2cgdHJhbnNmb3JtIHRoZSBkYXRhCgpgYGB7cn0KcGUgPC0gbG9nVHJhbnNmb3JtKHBlLCBiYXNlID0gMiwgaSA9ICJwZXB0aWRlUmF3IiwgbmFtZSA9ICJwZXB0aWRlTG9nIikKYGBgCgojIyBGaWx0ZXJpbmcKCjEuIEhhbmRsaW5nIG92ZXJsYXBwaW5nIHByb3RlaW4gZ3JvdXBzCgpJbiBvdXIgYXBwcm9hY2ggYSBwZXB0aWRlIGNhbiBtYXAgdG8gbXVsdGlwbGUgcHJvdGVpbnMsIGFzIGxvbmcgYXMgdGhlcmUgaXMKbm9uZSBvZiB0aGVzZSBwcm90ZWlucyBwcmVzZW50IGluIGEgc21hbGxlciBzdWJncm91cC4KCmBgYHtyfQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSwgfiBQcm90ZWlucyAlaW4lIHNtYWxsZXN0VW5pcXVlR3JvdXBzKHJvd0RhdGEocGVbWyJwZXB0aWRlTG9nIl1dKSRQcm90ZWlucykpCmBgYAoKMi4gUmVtb3ZlIHJldmVyc2Ugc2VxdWVuY2VzIChkZWNveXMpIGFuZCBjb250YW1pbmFudHMKCldlIG5vdyByZW1vdmUgdGhlIGNvbnRhbWluYW50cyBhbmQgcGVwdGlkZXMgdGhhdCBtYXAgdG8gZGVjb3kgc2VxdWVuY2VzLgoKYGBge3J9CnBlIDwtIGZpbHRlckZlYXR1cmVzKHBlLH5SZXZlcnNlICE9ICIrIikKcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsfiBQb3RlbnRpYWwuY29udGFtaW5hbnQgIT0gIisiKQpgYGAKCjMuIERyb3AgcGVwdGlkZXMgdGhhdCB3ZXJlIG9ubHkgaWRlbnRpZmllZCBpbiBvbmUgc2FtcGxlCgpXZSBrZWVwIHBlcHRpZGVzIHRoYXQgd2VyZSBvYnNlcnZlZCBhdCBsYXN0IHR3aWNlLgoKYGBge3J9CnBlIDwtIGZpbHRlckZlYXR1cmVzKHBlLH4gbk5vblplcm8gPj0yKQpucm93KHBlW1sicGVwdGlkZUxvZyJdXSkKYGBgCgpXZSBrZWVwIGByIG5yb3cocGVbWyJwZXB0aWRlTG9nIl1dKWAgcGVwdGlkZXMgdXBvbiBmaWx0ZXJpbmcuCgoKIyMgTm9ybWFsaXplIHRoZSBkYXRhIHVzaW5nIG1lZGlhbiBjZW50ZXJpbmcgCgpXZSBub3JtYWxpemUgdGhlIGRhdGEgYnkgc3Vic3RyYWN0aW5nIHRoZSBzYW1wbGUgbWVkaWFuIGZyb20gZXZlcnkgaW50ZW5zaXR5IGZvciBwZXB0aWRlICRwJCAgaW4gYSBzYW1wbGUgJGkkOiAKCiQkeV97aXB9Xlx0ZXh0e25vcm19ID0geV97aXB9IC0gXGhhdFxtdV9pJCQgCgp3aXRoICRcaGF0XG11X2kkIHRoZSBtZWRpYW4gaW50ZW5zaXR5IG92ZXIgYWxsIG9ic2VydmVkIHBlcHRpZGVzIGluIHNhbXBsZSAkaSQuCgpgYGB7cn0KcGUgPC0gbm9ybWFsaXplKHBlLCAKICAgICAgICAgICAgICAgIGkgPSAicGVwdGlkZUxvZyIsIAogICAgICAgICAgICAgICAgbmFtZSA9ICJwZXB0aWRlTm9ybSIsIAogICAgICAgICAgICAgICAgbWV0aG9kID0gImNlbnRlci5tZWRpYW4iKQpgYGAKCgojIyBFeHBsb3JlICBub3JtYWxpemVkIGRhdGEKClVwb24gdGhlIG5vcm1hbGlzYXRpb24gdGhlIGRlbnNpdHkgY3VydmVzIGFyZSBuaWNlbHkgcmVnaXN0ZXJlZAoKYGBge3J9CnBlW1sicGVwdGlkZU5vcm0iXV0gJT4lIAogIGFzc2F5ICU+JQogIGFzLmRhdGEuZnJhbWUoKSAlPiUKICBnYXRoZXIoc2FtcGxlLCBpbnRlbnNpdHkpICU+JSAKICBtdXRhdGUoY29uZGl0aW9uID0gY29sRGF0YShwZSlbc2FtcGxlLCJjb25kaXRpb24iXSkgJT4lCiAgZ2dwbG90KGFlcyh4ID0gaW50ZW5zaXR5LGdyb3VwID0gc2FtcGxlLGNvbG9yID0gY29uZGl0aW9uKSkgKyAKICAgIGdlb21fZGVuc2l0eSgpCmBgYAoKV2UgY2FuIHZpc3VhbGl6ZSBvdXIgZGF0YSB1c2luZyBhIE11bHRpIERpbWVuc2lvbmFsIFNjYWxpbmcgcGxvdCwKZWcuIGFzIHByb3ZpZGVkIGJ5IHRoZSBgbGltbWFgIHBhY2thZ2UuCgpgYGB7cn0KcGVbWyJwZXB0aWRlTm9ybSJdXSAlPiUgCiAgYXNzYXkgJT4lCiAgbGltbWE6OnBsb3RNRFMoY29sID0gYXMubnVtZXJpYyhjb2xEYXRhKHBlKSRjb25kaXRpb24pKQpgYGAKClRoZSBmaXJzdCBheGlzIGluIHRoZSBwbG90IGlzIHNob3dpbmcgdGhlIGxlYWRpbmcgbG9nIGZvbGQgY2hhbmdlcwooZGlmZmVyZW5jZXMgb24gdGhlIGxvZyBzY2FsZSkgYmV0d2VlbiB0aGUgc2FtcGxlcy4KCldlIG5vdGljZSB0aGF0IHRoZSBsZWFkaW5nIGRpZmZlcmVuY2VzIChsb2cgRkMpCmluIHRoZSBwZXB0aWRlIGRhdGEgc2VlbXMgdG8gYmUgZHJpdmVuIGJ5IHRlY2huaWNhbCB2YXJpYWJpbGl0eS4KSW5kZWVkLCB0aGUgc2FtcGxlcyBkbyBub3Qgc2VlbSB0byBiZSBjbGVhcmx5IHNlcGFyYXRlZCBhY2NvcmRpbmcKdG8gdGhlIHNwaWtlLWluIGNvbmRpdGlvbi4KCgojIyBTdW1tYXJpemF0aW9uIHRvIHByb3RlaW4gbGV2ZWwKCi0gQnkgZGVmYXVsdCByb2J1c3Qgc3VtbWFyaXphdGlvbiBpcyB1c2VkOiAgYGZ1biA9IE1zQ29yZVV0aWxzOjpyb2J1c3RTdW1tYXJ5KClgCgpgYGB7cix3YXJuaW5nPUZBTFNFfQpwZSA8LSBhZ2dyZWdhdGVGZWF0dXJlcyhwZSwKICBpID0gInBlcHRpZGVOb3JtIiwKICBmY29sID0gIlByb3RlaW5zIiwKICBuYS5ybSA9IFRSVUUsCiAgbmFtZSA9ICJwcm90ZWluIikKYGBgCgoKCmBgYHtyfQpwbG90TURTKGFzc2F5KHBlW1sicHJvdGVpbiJdXSksIGNvbCA9IGFzLm51bWVyaWMoY29sRGF0YShwZSkkY29uZGl0aW9uKSkKYGBgCgpOb3RlIHRoYXQgdGhlIHNhbXBsZXMgdXBvbiByb2J1c3Qgc3VtbWFyaXNhdGlvbiBzaG93IGEgY2xlYXIgc2VwYXJhdGlvbiBhY2NvcmRpbmcgdG8gdGhlIHNwaWtlLWluIGNvbmRpdGlvbiBpbiB0aGUgc2Vjb25kIGRpbWVuc2lvbiBvZiB0aGUgTURTIHBsb3QuCgojIERhdGEgQW5hbHlzaXMKCiMjIEVzdGltYXRpb24KCldlIG1vZGVsIHRoZSBwcm90ZWluIGxldmVsIGV4cHJlc3Npb24gdmFsdWVzIHVzaW5nIGBtc3Fyb2JgLgpCeSBkZWZhdWx0IGBtc3Fyb2IyYCBlc3RpbWF0ZXMgdGhlIG1vZGVsIHBhcmFtZXRlcnMgdXNpbmcgcm9idXN0IHJlZ3Jlc3Npb24uCgpXZSB3aWxsIG1vZGVsIHRoZSBkYXRhIHdpdGggYSBkaWZmZXJlbnQgZ3JvdXAgbWVhbi4gClRoZSBncm91cCBpcyBpbmNvZGVkIGluIHRoZSB2YXJpYWJsZSBgY29uZGl0aW9uYCBvZiB0aGUgY29sRGF0YS4gCldlIGNhbiBzcGVjaWZ5IHRoaXMgbW9kZWwgYnkgdXNpbmcgYSBmb3JtdWxhIHdpdGggdGhlIGZhY3RvciBjb25kaXRpb24gYXMgaXRzIHByZWRpY3RvcjogCmBmb3JtdWxhID0gfmNvbmRpdGlvbmAuCgpOb3RlLCB0aGF0IGEgZm9ybXVsYSBhbHdheXMgc3RhcnRzIHdpdGggYSBzeW1ib2wgJ34nLgoKYGBge3IsIHdhcm5pbmc9RkFMU0V9CnBlIDwtIG1zcXJvYihvYmplY3QgPSBwZSwgaSA9ICJwcm90ZWluIiwgZm9ybXVsYSA9IH5jb25kaXRpb24pCmBgYAoKIyMgSW5mZXJlbmNlCgpGaXJzdCwgd2UgZXh0cmFjdCB0aGUgcGFyYW1ldGVyIG5hbWVzIG9mIHRoZSBtb2RlbCBieSBsb29raW5nIGF0IHRoZSBmaXJzdCBtb2RlbC4gClRoZSBtb2RlbHMgYXJlIHN0b3JlZCBpbiB0aGUgcm93IGRhdGEgb2YgdGhlIGFzc2F5IHVuZGVyIHRoZSBkZWZhdWx0IG5hbWUgbXNxcm9iTW9kZWxzLiAKCmBgYHtyfQpnZXRDb2VmKHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRtc3Fyb2JNb2RlbHNbWzFdXSkKYGBgCgpXZSBjYW4gYWxzbyBleHBsb3JlIHRoZSBkZXNpZ24gb2YgdGhlIG1vZGVsIHRoYXQgd2Ugc3BlY2lmaWVkIHVzaW5nIHRoZSB0aGUgcGFja2FnZSBgRXhwbG9yZU1vZGVsTWF0cml4YCAKCmBgYHtyfQpsaWJyYXJ5KEV4cGxvcmVNb2RlbE1hdHJpeCkKVmlzdWFsaXplRGVzaWduKGNvbERhdGEocGUpLH5jb25kaXRpb24pJHBsb3RsaXN0W1sxXV0KYGBgCgpTcGlrZS1pbiBjb25kaXRpb24gYEFgIGlzIHRoZSByZWZlcmVuY2UgY2xhc3MuIFNvIHRoZSBtZWFuIGxvZzIgZXhwcmVzc2lvbgpmb3Igc2FtcGxlcyBmcm9tIGNvbmRpdGlvbiBBIGlzICcoSW50ZXJjZXB0KS4KVGhlIG1lYW4gbG9nMiBleHByZXNzaW9uIGZvciBzYW1wbGVzIGZyb20gY29uZGl0aW9uIEIgaXMgJyhJbnRlcmNlcHQpK2NvbmRpdGlvbkInLgpIZW5jZSwgdGhlIGF2ZXJhZ2UgbG9nMiBmb2xkIGNoYW5nZSBiZXR3ZWVuIGNvbmRpdGlvbiBiIGFuZApjb25kaXRpb24gYSBpcyBtb2RlbGxlZCB1c2luZyB0aGUgcGFyYW1ldGVyICdjb25kaXRpb25CJy4KVGh1cywgd2UgYXNzZXNzIHRoZSBjb250cmFzdCAnY29uZGl0aW9uQiA9IDAnIHdpdGggb3VyIHN0YXRpc3RpY2FsIHRlc3QuCgpgYGB7cn0KTCA8LSBtYWtlQ29udHJhc3QoImNvbmRpdGlvbkI9MCIsIHBhcmFtZXRlck5hbWVzID0gYygiY29uZGl0aW9uQiIpKQpwZSA8LSBoeXBvdGhlc2lzVGVzdChvYmplY3QgPSBwZSwgaSA9ICJwcm90ZWluIiwgY29udHJhc3QgPSBMKQpgYGAKCgojIyBQbG90cwoKIyMjIFZvbGNhbm8tcGxvdAoKCmBgYHtyLHdhcm5pbmc9RkFMU0V9CnZvbGNhbm8gPC0gZ2dwbG90KHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRjb25kaXRpb25CLAogICAgICAgICAgICAgICAgICBhZXMoeCA9IGxvZ0ZDLCB5ID0gLWxvZzEwKHB2YWwpLCBjb2xvciA9IGFkalB2YWwgPCAwLjA1KSkgKwogIGdlb21fcG9pbnQoY2V4ID0gMi41KSArCiAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGFscGhhKGMoImJsYWNrIiwgInJlZCIpLCAwLjUpKSArIHRoZW1lX21pbmltYWwoKQp2b2xjYW5vCmBgYAoKTm90ZSwgdGhhdCBgciBzdW0ocm93RGF0YShwZVtbInByb3RlaW4iXV0pJGNvbmRpdGlvbkIkYWRqUHZhbCA8IDAuMDUsIG5hLnJtID0gVFJVRSlgIHByb3RlaW5zIGFyZSBmb3VuZCB0byBiZSBkaWZmZXJlbnRpYWxseSBhYnVuZGFudC4KCiMjIyBIZWF0bWFwCgpXZSBmaXJzdCBzZWxlY3QgdGhlIG5hbWVzIG9mIHRoZSBwcm90ZWlucyB0aGF0IHdlcmUgZGVjbGFyZWQgc2lnbmZpY2FudC4KCmBgYHtyfQpzaWdOYW1lcyA8LSByb3dEYXRhKHBlW1sicHJvdGVpbiJdXSkkY29uZGl0aW9uQiAlPiUKICByb3duYW1lc190b19jb2x1bW4oInByb3RlaW4iKSAlPiUKICBmaWx0ZXIoYWRqUHZhbDwwLjA1KSAlPiUKICBwdWxsKHByb3RlaW4pCmhlYXRtYXAoYXNzYXkocGVbWyJwcm90ZWluIl1dKVtzaWdOYW1lcywgXSkKYGBgCgpUaGUgbWFqb3JpdHkgb2YgdGhlIHByb3RlaW5zIGFyZSBpbmRlZWQgVVBTIHByb3RlaW5zLiAKMSB5ZWFzdCBwcm90ZWluIGlzIHJldHVybmVkLiAKTm90ZSwgdGhhdCB0aGUgeWVhc3QgcHJvdGVpbiBpbmRlZWQgc2hvd3MgZXZpZGVuY2UgZm9yIGRpZmZlcmVudGlhbCBhYnVuZGFuY2UuIAoKIyMjIEJveHBsb3RzCgpXZSBtYWtlIGJveHBsb3Qgb2YgdGhlIGxvZzIgRkMgYW5kIHN0cmF0aWZ5IGFjY29yZGluZyB0byB0aGUgd2hldGhlciBhIHByb3RlaW4gaXMgc3Bpa2VkIG9yIG5vdC4KCmBgYHtyfQpyb3dEYXRhKHBlW1sicHJvdGVpbiJdXSkkY29uZGl0aW9uQiAlPiUKICByb3duYW1lc190b19jb2x1bW4odmFyID0gInByb3RlaW4iKSAlPiUKICBnZ3Bsb3QoYWVzKHg9Z3JlcGwoIlVQUyIscHJvdGVpbikseT1sb2dGQykpICsKICBnZW9tX2JveHBsb3QoKSArCiAgeGxhYigiVVBTIikgKwogIGdlb21fc2VnbWVudCgKICAgIHggPSAxLjUsCiAgICB4ZW5kID0gMi41LAogICAgeSA9IGxvZzIoMC43NC8wLjI1KSwKICAgIHllbmQgPSBsb2cyKDAuNzQvMC4yNSksCiAgICBjb2xvdXI9InJlZCIpICsKICBnZW9tX3NlZ21lbnQoCiAgICB4ID0gMC41LAogICAgeGVuZCA9IDEuNSwKICAgIHkgPSAwLAogICAgeWVuZCA9IDAsCiAgICBjb2xvdXI9InJlZCIpICsKICBhbm5vdGF0ZSgKICAgICJ0ZXh0IiwKICAgIHggPSBjKDEsMiksCiAgICB5ID0gYygwLGxvZzIoMC43NC8wLjI1KSkrLjEsCiAgICBsYWJlbCA9IGMoCiAgICAgICJsb2cyIEZDIEVjb2xpID0gMCIsCiAgICAgIHBhc3RlMCgibG9nMiBGQyBVUFMgPSAiLHJvdW5kKGxvZzIoMC43NC8wLjI1KSwyKSkKICAgICAgKSwKICAgIGNvbG91ciA9ICJyZWQiKQpgYGAKCldoYXQgZG8geW91IG9ic2VydmU/CgojIyMgRGV0YWlsIHBsb3RzCgpXZSBmaXJzdCBleHRyYWN0IHRoZSBub3JtYWxpemVkIHBlcHRpZGVSYXcgZXhwcmVzc2lvbiB2YWx1ZXMgZm9yIGEgcGFydGljdWxhciBwcm90ZWluLiAgCgoKYGBge3IsIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CmZvciAocHJvdE5hbWUgaW4gc2lnTmFtZXMpCnsKcGVQbG90IDwtIHBlW3Byb3ROYW1lLCAsIGMoInBlcHRpZGVOb3JtIiwicHJvdGVpbiIpXQpwZVBsb3REZiA8LSBkYXRhLmZyYW1lKGxvbmdGb3JtYXQocGVQbG90KSkKcGVQbG90RGYkYXNzYXkgPC0gZmFjdG9yKHBlUGxvdERmJGFzc2F5LAogICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKCJwZXB0aWRlTm9ybSIsICJwcm90ZWluIikpCnBlUGxvdERmJGNvbmRpdGlvbiA8LSBhcy5mYWN0b3IoY29sRGF0YShwZVBsb3QpW3BlUGxvdERmJGNvbG5hbWUsICJjb25kaXRpb24iXSkKCiMgcGxvdHRpbmcKcDEgPC0gZ2dwbG90KGRhdGEgPSBwZVBsb3REZiwKICAgICAgIGFlcyh4ID0gY29sbmFtZSwgeSA9IHZhbHVlLCBncm91cCA9IHJvd25hbWUpKSArCiAgICBnZW9tX2xpbmUoKSArIAogICAgZ2VvbV9wb2ludCgpICsgIAogICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA3MCwgaGp1c3QgPSAxLCB2anVzdCA9IDAuNSkpICsKICAgIGZhY2V0X2dyaWQofmFzc2F5KSArIAogICAgZ2d0aXRsZShwcm90TmFtZSkKcHJpbnQocDEpCgojIHBsb3R0aW5nIDIKcDIgPC0gZ2dwbG90KHBlUGxvdERmLCBhZXMoeCA9IGNvbG5hbWUsIHkgPSB2YWx1ZSwgZmlsbCA9IGNvbmRpdGlvbikpICsKICBnZW9tX2JveHBsb3Qob3V0bGllci5zaGFwZSA9IE5BKSArIAogIGdlb21fcG9pbnQoCiAgICBwb3NpdGlvbiA9IHBvc2l0aW9uX2ppdHRlcih3aWR0aCA9IC4xKSwKICAgIGFlcyhzaGFwZSA9IHJvd25hbWUpKSArCiAgc2NhbGVfc2hhcGVfbWFudWFsKHZhbHVlcyA9IDE6bnJvdyhwZVBsb3REZikpICsKICBsYWJzKHRpdGxlID0gcHJvdE5hbWUsIHggPSAic2FtcGxlIiwgeSA9ICJwZXB0aWRlIGludGVuc2l0eSAobG9nMikiKSArIAogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gNzAsIGhqdXN0ID0gMSwgdmp1c3QgPSAwLjUpKSArCiAgZmFjZXRfZ3JpZCh+YXNzYXkpCnByaW50KHAyKQp9CmBgYAoKTm90ZSwgdGhhdCB0aGUgeWVhc3QgcHJvdGVpbiBpcyBvbmx5IGNvdmVyZWQgYnkgMyBwZXB0aWRlcy4gCk9ubHkgb25lIHBlcHRpZGUgaXMgcGlja2VkIHVwIGluIGNvbmRpdGlvbiBBLiAKVGhpcyBwZXB0aWRlIGlzIGFsc28gb25seSBvbmNlIG9ic2VydmVkIGluIHNwaWtlLWluIGNvbmRpdGlvbiBCLiAKVGhpcyBwdXRzIGEgY29uc2lkZXJhYmxlIGJ1cmRlbiB1cG9uIHRoZSBpbmZlcmVuY2UgYW5kIGNvdWxkIGJlIGF2b2lkZWQgYnkgbW9yZSBzdHJpbmdlbnQgZmlsdGVyaW5nLiAKCiMgU2Vzc2lvbiBJbmZvCgpXaXRoIHJlc3BlY3QgdG8gcmVwcm9kdWNpYmlsaXR5LCBpdCBpcyBoaWdobHkgcmVjb21tZW5kZWQgdG8gaW5jbHVkZSBhIHNlc3Npb24gaW5mbyBpbiB5b3VyIHNjcmlwdCBzbyB0aGF0IHJlYWRlcnMgb2YgeW91ciBvdXRwdXQgY2FuIHNlZSB5b3VyIHBhcnRpY3VsYXIgc2V0dXAgb2YgUi4gCgpgYGB7cn0Kc2Vzc2lvbkluZm8oKQpgYGAK