Creative Commons License

This is part of the online course Proteomics Data Analysis (PDA)

1 Background

Twelve Estrogen Receptor Positive Breast cancer tissues from from patients treated with tamoxifen upon recurrence have been assessed in a proteomics study. Six patients had a good outcome (OR) and the other Six had a poor outcome (PD). The proteomes have been assessed using an LTQ-Orbitrap and the thermo output .RAW files were searched with MaxQuant (version 1.4.1.2) against the human proteome database (FASTA version 2012-09, human canonical proteome).

2 Data

We first import the data from peptide.txt file. This is the file containing your peptide-level intensities. For a MaxQuant search [6], this peptide.txt file can be found by default in the “path_to_raw_files/combined/txt/” folder from the MaxQuant output, with “path_to_raw_files” the folder where the raw files were saved.

We generate the object peptideFile with the path to the peptide.txt file. Using the grepEcols function, we find the columns that contain the expression data of the peptide in the peptide.txt file.

library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)

peptidesFile <- "https://raw.githubusercontent.com/statOmics/PDA22GTPB/data/quantification/cancer/peptides3vs3.txt"

ecols <- grep(
  "Intensity\\.", 
  names(read.delim(peptidesFile))
  )

Next, we read the data and store it in QFeatures object

pe <- readQFeatures(
  table = peptidesFile,
  fnames = 1,
  ecol = ecols,
  name = "peptideRaw", sep="\t")

The QFeatures object pe currently contains a single assay, named peptideRaw.

We extract the column names from the peptideRaw assay and see that this contains information about the prognosis.

colnames(pe[["peptideRaw"]])
## [1] "Intensity.OR.01" "Intensity.OR.04" "Intensity.OR.07" "Intensity.PD.02"
## [5] "Intensity.PD.03" "Intensity.PD.04"

We rename the colnames by dropping the “Intensity.” from the name.

(newNames <-  sub(
  pattern = "Intensity\\.", 
  replacement = "",
  colnames(pe[["peptideRaw"]]))
 )
## [1] "OR.01" "OR.04" "OR.07" "PD.02" "PD.03" "PD.04"
pe <- renameColname(pe, 
                    i = "peptideRaw",
                    newNames)
pe <- renamePrimary(pe, newNames)
colnames(pe[["peptideRaw"]])
## [1] "OR.01" "OR.04" "OR.07" "PD.02" "PD.03" "PD.04"

In the following code chunk, we add the prognosis of the patients that we can read in the raw file name to the colData.

colData(pe)$prognosis <-  
  colnames(pe[["peptideRaw"]]) %>%
  substr(start = 1, stop = 2) %>%
  as.factor
colData(pe)$prognosis
## [1] OR OR OR PD PD PD
## Levels: OR PD

We calculate how many non zero intensities we have per peptide and this will be useful for filtering.

rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)

Peptides with zero intensities are missing peptides and should be represent with a NA value rather than 0.

pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA

Look at the column names of the data to know the variables that you can use for filtering.

pe[["peptideRaw"]] %>% rowData %>% names
##  [1] "Sequence"               "Proteins"               "Leading.razor.protein" 
##  [4] "Gene.names"             "Protein.names"          "Unique..Groups."       
##  [7] "Unique..Proteins."      "Charges"                "PEP"                   
## [10] "Score"                  "Slice.Average"          "Slice.Std..Dev."       
## [13] "Slice.1"                "Unique.Slice.Average"   "Unique.Slice.Std..Dev."
## [16] "Unique.Slice.1"         "Experiment.OR.01"       "Experiment.OR.04"      
## [19] "Experiment.OR.07"       "Experiment.OR.09"       "Experiment.OR.10"      
## [22] "Experiment.OR.13"       "Experiment.PD.02"       "Experiment.PD.03"      
## [25] "Experiment.PD.04"       "Experiment.PD.06"       "Experiment.PD.07"      
## [28] "Experiment.PD.08"       "Intensity"              "Reverse"               
## [31] "Contaminant"            "id"                     "Protein.group.IDs"     
## [34] "Mod..peptide.IDs"       "Evidence.IDs"           "MS.MS.IDs"             
## [37] "Best.MS.MS"             "Oxidation..M..site.IDs" "nNonZero"

So we will filter on the “Reverse”, “Contaminant” and “nNonZero” column.

2.1 Data exploration

44% of all peptide intensities are missing and for some peptides we do not even measure a signal in any sample.

3 Preprocessing

This section preforms preprocessing for the peptide data. This include

  • log transformation,
  • filtering and
  • summarisation of the data.

3.1 Log transform the data

pe <- logTransform(pe, base = 2, i = "peptideRaw", name = "peptideLog")

3.2 Filtering

  1. Handling overlapping protein groups

In our approach a peptide can map to multiple proteins, as long as there is none of these proteins present in a smaller subgroup.

pe <- filterFeatures(pe, ~ Proteins %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins))
  1. Remove reverse sequences (decoys) and contaminants

We now remove the contaminants and peptides that map to decoy sequences.

pe <- filterFeatures(pe,~Reverse != "+")
pe <- filterFeatures(pe,~ Contaminant != "+")
  1. Drop peptides that were only identified in one sample

We keep peptides that were observed at last twice.

pe <- filterFeatures(pe,~ nNonZero >=2)
nrow(pe[["peptideLog"]])
## [1] 22413

We keep 22413 peptides upon filtering.

3.3 Normalize the data using median centering

We normalize the data by substracting the sample median from every intensity for peptide \(p\) in a sample \(i\):

\[y_{ip}^\text{norm} = y_{ip} - \hat\mu_i\]

with \(\hat\mu_i\) the median intensity over all observed peptides in sample \(i\).

pe <- normalize(pe, 
                i = "peptideLog", 
                name = "peptideNorm", 
                method = "center.median")

3.4 Explore normalized data

Upon the normalisation the density curves are nicely registered

pe[["peptideNorm"]] %>% 
  assay %>%
  as.data.frame() %>%
  gather(sample, intensity) %>% 
  mutate(prognosis = colData(pe)[sample,"prognosis"]) %>%
  ggplot(aes(x = intensity,group = sample,color = prognosis)) + 
    geom_density()
## Warning: Removed 34599 rows containing non-finite values (stat_density).

We can visualize our data using a Multi Dimensional Scaling plot, eg. as provided by the limma package.

pe[["peptideNorm"]] %>% 
  assay %>%
  limma::plotMDS(col = as.numeric(colData(pe)$prognosis))

The first axis in the plot is showing the leading log fold changes (differences on the log scale) between the samples.

3.5 Summarization to protein level

  • By default robust summarization is used: fun = MsCoreUtils::robustSummary()
pe <- aggregateFeatures(pe,
  i = "peptideNorm",
  fcol = "Proteins",
  na.rm = TRUE,
  name = "protein")
## Your quantitative and row data contain missing values. Please read the
## relevant section(s) in the aggregateFeatures manual page regarding the
## effects of missing values on data aggregation.
plotMDS(assay(pe[["protein"]]), col = as.numeric(colData(pe)$prognosis))

Note that the samples upon robust summarisation show a separation according to the prognosis.

4 Data Analysis

4.1 Estimation

We model the protein level expression values using msqrob. By default msqrob2 estimates the model parameters using robust regression.

We will model the data with a different group mean. The group is incoded in the variable prognosis of the colData. We can specify this model by using a formula with the factor condition as its predictor: formula = ~prognosis.

Note, that a formula always starts with a symbol ‘~’.

pe <- msqrob(object = pe, i = "protein", formula = ~prognosis)

4.2 Inference

First, we extract the parameter names of the model by looking at the first model. The models are stored in the row data of the assay under the default name msqrobModels.

getCoef(rowData(pe[["protein"]])$msqrobModels[[1]])
## (Intercept) prognosisPD 
##  -1.5083003   0.8795213

We can also explore the design of the model that we specified using the the package ExploreModelMatrix

library(ExploreModelMatrix)
VisualizeDesign(colData(pe),~prognosis)$plotlist
## [[1]]

Spike-in condition A is the reference class. So the mean log2 expression for samples from good prognosis (OR) is ‘(Intercept). The mean log2 expression for samples from poor prognosis (PD) is’(Intercept)+prognosisPD’. Hence, the average log2 fold change between prognosis PD and prognosis OR is modelled using the parameter ‘conditionPD’. Thus, we assess the contrast ‘conditionPD = 0’ with our statistical test.

L <- makeContrast("prognosisPD=0", parameterNames = c("prognosisPD"))
pe <- hypothesisTest(object = pe, i = "protein", contrast = L)

4.3 Plots

4.3.1 Volcano-plot

volcano <- ggplot(rowData(pe[["protein"]])$prognosisPD,
                  aes(x = logFC, y = -log10(pval), color = adjPval < 0.05)) +
  geom_point(cex = 2.5) +
  scale_color_manual(values = alpha(c("black", "red"), 0.5)) + theme_minimal()
volcano

Note, that 0 proteins are found to be differentially abundant.

4.3.2 Heatmap

There were no proteins significantly differentially abundant at the 5% FDR level. We will therefore make an heatmap for the top 10 DE proteins. Note, however that they are not statistically significant!

#sigNames <- rowData(pe[["protein"]])$prognosisPD %>%
#  rownames_to_column("protein") %>%
#  filter(adjPval<0.05) %>%
#  pull(protein)
#heatmap(assay(pe[["protein"]])[sigNames, ])
orderProt <- rowData(pe[["protein"]])$prognosisPD %>% 
    rownames_to_column("protein") %>%
    arrange(pval) %>% 
    pull(protein)

heatmap(assay(pe[["protein"]])[orderProt[1:10],])

4.3.3 Detail plots

We make detail plots for the top 10 proteins. Note, however that their fold changes are not statistically significant at the 5% FDR level.

#for (protName in sigNames)
for (protName in orderProt[1:10])
{
pePlot <- pe[protName, , c("peptideNorm","protein")]
pePlotDf <- data.frame(longFormat(pePlot))
pePlotDf$assay <- factor(pePlotDf$assay,
                        levels = c("peptideNorm", "protein"))
pePlotDf$prognosis <- as.factor(colData(pePlot)[pePlotDf$colname, "prognosis"])

# plotting
p1 <- ggplot(data = pePlotDf,
       aes(x = colname, y = value, group = rowname)) +
    geom_line() + 
    geom_point() +  
    theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
    facet_grid(~assay) + 
    ggtitle(protName)
print(p1)

# plotting 2
p2 <- ggplot(pePlotDf, aes(x = colname, y = value, fill = prognosis)) +
  geom_boxplot(outlier.shape = NA) + 
  geom_point(
    position = position_jitter(width = .1),
    aes(shape = rowname)) +
  scale_shape_manual(values = 1:nrow(pePlotDf)) +
  labs(title = protName, x = "sample", y = "peptide intensity (log2)") + 
  theme(axis.text.x = element_text(angle = 70, hjust = 1, vjust = 0.5)) +
  facet_grid(~assay)
print(p2)
}

5 Session Info

With respect to reproducibility, it is highly recommended to include a session info in your script so that readers of your output can see your particular setup of R.

sessionInfo()
## R version 4.2.3 (2023-03-15)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 22.04.3 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
## 
## locale:
##  [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
##  [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
##  [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
## [10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   
## 
## attached base packages:
## [1] stats4    stats     graphics  grDevices datasets  utils     methods  
## [8] base     
## 
## other attached packages:
##  [1] ExploreModelMatrix_1.8.0    plotly_4.10.0              
##  [3] msqrob2_1.4.0               QFeatures_1.6.0            
##  [5] MultiAssayExperiment_1.22.0 SummarizedExperiment_1.26.1
##  [7] Biobase_2.56.0              GenomicRanges_1.48.0       
##  [9] GenomeInfoDb_1.32.2         IRanges_2.30.0             
## [11] S4Vectors_0.34.0            BiocGenerics_0.42.0        
## [13] MatrixGenerics_1.8.0        matrixStats_0.62.0         
## [15] limma_3.52.1                forcats_0.5.1              
## [17] stringr_1.4.1               dplyr_1.0.9                
## [19] purrr_0.3.4                 readr_2.1.2                
## [21] tidyr_1.2.0                 tibble_3.1.7               
## [23] ggplot2_3.3.6               tidyverse_1.3.2            
## 
## loaded via a namespace (and not attached):
##   [1] googledrive_2.0.0       minqa_1.2.4             colorspace_2.0-3       
##   [4] ellipsis_0.3.2          XVector_0.36.0          fs_1.5.2               
##   [7] clue_0.3-61             farver_2.1.0            DT_0.23                
##  [10] fansi_1.0.3             lubridate_1.8.0         xml2_1.3.3             
##  [13] codetools_0.2-18        splines_4.2.3           knitr_1.40.1           
##  [16] jsonlite_1.8.0          nloptr_2.0.3            broom_0.8.0            
##  [19] cluster_2.1.3           dbplyr_2.1.1            shinydashboard_0.7.2   
##  [22] shiny_1.7.1             BiocManager_1.30.18     compiler_4.2.3         
##  [25] httr_1.4.3              backports_1.4.1         assertthat_0.2.1       
##  [28] Matrix_1.4-1            fastmap_1.1.0           lazyeval_0.2.2         
##  [31] gargle_1.2.0            cli_3.3.0               later_1.3.0            
##  [34] htmltools_0.5.2         tools_4.2.3             igraph_1.3.2           
##  [37] gtable_0.3.0            glue_1.6.2              GenomeInfoDbData_1.2.8 
##  [40] Rcpp_1.0.8.3            cellranger_1.1.0        jquerylib_0.1.4        
##  [43] vctrs_0.4.1             nlme_3.1-157            rintrojs_0.3.0         
##  [46] xfun_0.33               lme4_1.1-29             rvest_1.0.2            
##  [49] mime_0.12               lifecycle_1.0.1         renv_0.15.4            
##  [52] googlesheets4_1.0.0     zlibbioc_1.42.0         MASS_7.3-57            
##  [55] scales_1.2.0            promises_1.2.0.1        hms_1.1.1              
##  [58] ProtGenerics_1.28.0     parallel_4.2.3          AnnotationFilter_1.20.0
##  [61] yaml_2.3.5              sass_0.4.1              stringi_1.7.8          
##  [64] highr_0.9               boot_1.3-28             BiocParallel_1.30.2    
##  [67] rlang_1.0.2             pkgconfig_2.0.3         bitops_1.0-7           
##  [70] evaluate_0.16           lattice_0.20-45         htmlwidgets_1.5.4      
##  [73] labeling_0.4.2          cowplot_1.1.1           tidyselect_1.1.2       
##  [76] magrittr_2.0.3          R6_2.5.1                generics_0.1.2         
##  [79] DelayedArray_0.22.0     DBI_1.1.2               pillar_1.7.0           
##  [82] haven_2.5.0             withr_2.5.0             MsCoreUtils_1.8.0      
##  [85] RCurl_1.98-1.6          modelr_0.1.8            crayon_1.5.1           
##  [88] utf8_1.2.2              tzdb_0.3.0              rmarkdown_2.14         
##  [91] grid_4.2.3              readxl_1.4.0            data.table_1.14.2      
##  [94] reprex_2.0.1            digest_0.6.29           xtable_1.8-4           
##  [97] httpuv_1.6.5            munsell_0.5.0           viridisLite_0.4.0      
## [100] bslib_0.3.1             shinyjs_2.1.0
LS0tCnRpdGxlOiAiQ2FuY2VyIgphdXRob3I6ICJMaWV2ZW4gQ2xlbWVudCIKZGF0ZTogInN0YXRPbWljcywgR2hlbnQgVW5pdmVyc2l0eSAoaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSIKb3V0cHV0OgogICAgaHRtbF9kb2N1bWVudDoKICAgICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgICB0aGVtZTogZmxhdGx5CiAgICAgIHRvYzogdHJ1ZQogICAgICB0b2NfZmxvYXQ6IHRydWUKICAgICAgaGlnaGxpZ2h0OiB0YW5nbwogICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIHBkZl9kb2N1bWVudDoKICAgICAgdG9jOiB0cnVlCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQpsaW5rY29sb3I6IGJsdWUKdXJsY29sb3I6IGJsdWUKY2l0ZWNvbG9yOiBibHVlCgpiaWJsaW9ncmFwaHk6IG1zcXJvYjIuYmliCgotLS0KCjxhIHJlbD0ibGljZW5zZSIgaHJlZj0iaHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LW5jLXNhLzQuMCI+PGltZyBhbHQ9IkNyZWF0aXZlIENvbW1vbnMgTGljZW5zZSIgc3R5bGU9ImJvcmRlci13aWR0aDowIiBzcmM9Imh0dHBzOi8vaS5jcmVhdGl2ZWNvbW1vbnMub3JnL2wvYnktbmMtc2EvNC4wLzg4eDMxLnBuZyIgLz48L2E+CgpUaGlzIGlzIHBhcnQgb2YgdGhlIG9ubGluZSBjb3Vyc2UgW1Byb3Rlb21pY3MgRGF0YSBBbmFseXNpcyAoUERBKV0oaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvL1BEQS8pCgojIEJhY2tncm91bmQKClR3ZWx2ZSBFc3Ryb2dlbiBSZWNlcHRvciBQb3NpdGl2ZSBCcmVhc3QgY2FuY2VyIHRpc3N1ZXMgZnJvbSBmcm9tIHBhdGllbnRzIHRyZWF0ZWQgd2l0aCB0YW1veGlmZW4gdXBvbiByZWN1cnJlbmNlIGhhdmUgYmVlbiBhc3Nlc3NlZCBpbiBhIHByb3Rlb21pY3Mgc3R1ZHkuIFNpeCBwYXRpZW50cyBoYWQgYSBnb29kIG91dGNvbWUgKE9SKSBhbmQgdGhlIG90aGVyIFNpeCBoYWQgYSBwb29yIG91dGNvbWUgKFBEKS4gVGhlIHByb3Rlb21lcyBoYXZlIGJlZW4gYXNzZXNzZWQgdXNpbmcgYW4gTFRRLU9yYml0cmFwIGFuZCB0aGUgdGhlcm1vIG91dHB1dCAuUkFXIGZpbGVzIHdlcmUgc2VhcmNoZWQgd2l0aCBNYXhRdWFudCAodmVyc2lvbiAxLjQuMS4yKSBhZ2FpbnN0IHRoZSBodW1hbiBwcm90ZW9tZSBkYXRhYmFzZSAoRkFTVEEgdmVyc2lvbiAyMDEyLTA5LCBodW1hbiBjYW5vbmljYWwgcHJvdGVvbWUpLgoKCgojIERhdGEKCldlIGZpcnN0IGltcG9ydCB0aGUgZGF0YSBmcm9tIHBlcHRpZGUudHh0IGZpbGUuIFRoaXMgaXMgdGhlIGZpbGUgY29udGFpbmluZwp5b3VyIHBlcHRpZGUtbGV2ZWwgaW50ZW5zaXRpZXMuIEZvciBhIE1heFF1YW50IHNlYXJjaCBbNl0sCnRoaXMgcGVwdGlkZS50eHQgZmlsZSBjYW4gYmUgZm91bmQgYnkgZGVmYXVsdCBpbiB0aGUKInBhdGhfdG9fcmF3X2ZpbGVzL2NvbWJpbmVkL3R4dC8iIGZvbGRlciBmcm9tIHRoZSBNYXhRdWFudCBvdXRwdXQsCndpdGggInBhdGhfdG9fcmF3X2ZpbGVzIiB0aGUgZm9sZGVyIHdoZXJlIHRoZSByYXcgZmlsZXMgd2VyZSBzYXZlZC4KCldlIGdlbmVyYXRlIHRoZSBvYmplY3QgcGVwdGlkZUZpbGUgd2l0aCB0aGUgcGF0aCB0byB0aGUgcGVwdGlkZS50eHQgZmlsZS4KVXNpbmcgdGhlIGBncmVwRWNvbHNgIGZ1bmN0aW9uLCB3ZSBmaW5kIHRoZSBjb2x1bW5zIHRoYXQgY29udGFpbiB0aGUgZXhwcmVzc2lvbgpkYXRhIG9mIHRoZSBwZXB0aWRlIGluIHRoZSBwZXB0aWRlLnR4dCBmaWxlLgoKCmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShsaW1tYSkKbGlicmFyeShRRmVhdHVyZXMpCmxpYnJhcnkobXNxcm9iMikKbGlicmFyeShwbG90bHkpCgpwZXB0aWRlc0ZpbGUgPC0gImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9zdGF0T21pY3MvUERBMjJHVFBCL2RhdGEvcXVhbnRpZmljYXRpb24vY2FuY2VyL3BlcHRpZGVzM3ZzMy50eHQiCgplY29scyA8LSBncmVwKAogICJJbnRlbnNpdHlcXC4iLCAKICBuYW1lcyhyZWFkLmRlbGltKHBlcHRpZGVzRmlsZSkpCiAgKQpgYGAKCk5leHQsIHdlIHJlYWQgdGhlIGRhdGEgYW5kIHN0b3JlIGl0IGluIFFGZWF0dXJlcyBvYmplY3QKCmBgYHtyfQpwZSA8LSByZWFkUUZlYXR1cmVzKAogIHRhYmxlID0gcGVwdGlkZXNGaWxlLAogIGZuYW1lcyA9IDEsCiAgZWNvbCA9IGVjb2xzLAogIG5hbWUgPSAicGVwdGlkZVJhdyIsIHNlcD0iXHQiKQpgYGAKClRoZSBRRmVhdHVyZXMgb2JqZWN0IHBlIGN1cnJlbnRseSBjb250YWlucyBhIHNpbmdsZSBhc3NheSwgbmFtZWQgcGVwdGlkZVJhdy4KCldlIGV4dHJhY3QgdGhlIGNvbHVtbiBuYW1lcyBmcm9tIHRoZSBwZXB0aWRlUmF3IGFzc2F5IGFuZCBzZWUgdGhhdCB0aGlzIGNvbnRhaW5zIGluZm9ybWF0aW9uIGFib3V0IHRoZSBwcm9nbm9zaXMuCgpgYGB7cn0KY29sbmFtZXMocGVbWyJwZXB0aWRlUmF3Il1dKQpgYGAKCldlIHJlbmFtZSB0aGUgY29sbmFtZXMgYnkgZHJvcHBpbmcgdGhlICJJbnRlbnNpdHkuIiBmcm9tIHRoZSBuYW1lLgoKYGBge3J9CihuZXdOYW1lcyA8LSAgc3ViKAogIHBhdHRlcm4gPSAiSW50ZW5zaXR5XFwuIiwgCiAgcmVwbGFjZW1lbnQgPSAiIiwKICBjb2xuYW1lcyhwZVtbInBlcHRpZGVSYXciXV0pKQogKQpgYGAKCmBgYHtyfQpwZSA8LSByZW5hbWVDb2xuYW1lKHBlLCAKICAgICAgICAgICAgICAgICAgICBpID0gInBlcHRpZGVSYXciLAogICAgICAgICAgICAgICAgICAgIG5ld05hbWVzKQpwZSA8LSByZW5hbWVQcmltYXJ5KHBlLCBuZXdOYW1lcykKY29sbmFtZXMocGVbWyJwZXB0aWRlUmF3Il1dKQpgYGAKCkluIHRoZSBmb2xsb3dpbmcgY29kZSBjaHVuaywgd2UgYWRkIHRoZSBwcm9nbm9zaXMgb2YgdGhlIHBhdGllbnRzIHRoYXQgd2UgY2FuIHJlYWQgaW4gdGhlIHJhdyBmaWxlIG5hbWUgdG8gdGhlIGNvbERhdGEuCgpgYGB7cn0KY29sRGF0YShwZSkkcHJvZ25vc2lzIDwtICAKICBjb2xuYW1lcyhwZVtbInBlcHRpZGVSYXciXV0pICU+JQogIHN1YnN0cihzdGFydCA9IDEsIHN0b3AgPSAyKSAlPiUKICBhcy5mYWN0b3IKY29sRGF0YShwZSkkcHJvZ25vc2lzCmBgYAoKCldlIGNhbGN1bGF0ZSBob3cgbWFueSBub24gemVybyBpbnRlbnNpdGllcyB3ZSBoYXZlIHBlciBwZXB0aWRlIGFuZCB0aGlzCndpbGwgYmUgdXNlZnVsIGZvciBmaWx0ZXJpbmcuCgpgYGB7cn0Kcm93RGF0YShwZVtbInBlcHRpZGVSYXciXV0pJG5Ob25aZXJvIDwtIHJvd1N1bXMoYXNzYXkocGVbWyJwZXB0aWRlUmF3Il1dKSA+IDApCmBgYAoKClBlcHRpZGVzIHdpdGggemVybyBpbnRlbnNpdGllcyBhcmUgbWlzc2luZyBwZXB0aWRlcyBhbmQgc2hvdWxkIGJlIHJlcHJlc2VudAp3aXRoIGEgYE5BYCB2YWx1ZSByYXRoZXIgdGhhbiBgMGAuCmBgYHtyfQpwZSA8LSB6ZXJvSXNOQShwZSwgInBlcHRpZGVSYXciKSAjIGNvbnZlcnQgMCB0byBOQQpgYGAKCkxvb2sgYXQgdGhlIGNvbHVtbiBuYW1lcyBvZiB0aGUgZGF0YSB0byBrbm93IHRoZSB2YXJpYWJsZXMgdGhhdCB5b3UgY2FuIHVzZSBmb3IgZmlsdGVyaW5nLiAKCmBgYHtyfQpwZVtbInBlcHRpZGVSYXciXV0gJT4lIHJvd0RhdGEgJT4lIG5hbWVzCmBgYApTbyB3ZSB3aWxsIGZpbHRlciBvbiB0aGUgIlJldmVyc2UiLCAiQ29udGFtaW5hbnQiIGFuZCAibk5vblplcm8iIGNvbHVtbi4KCiMjIERhdGEgZXhwbG9yYXRpb24KCmByIGZvcm1hdChtZWFuKGlzLm5hKGFzc2F5KHBlW1sicGVwdGlkZVJhdyJdXSkpKSoxMDAsZGlnaXRzPTIpYCUgb2YgYWxsIHBlcHRpZGUKaW50ZW5zaXRpZXMgYXJlIG1pc3NpbmcgYW5kIGZvciBzb21lIHBlcHRpZGVzIHdlIGRvIG5vdCBldmVuIG1lYXN1cmUgYSBzaWduYWwKaW4gYW55IHNhbXBsZS4KCgojIFByZXByb2Nlc3NpbmcKClRoaXMgc2VjdGlvbiBwcmVmb3JtcyBwcmVwcm9jZXNzaW5nIGZvciB0aGUgcGVwdGlkZSBkYXRhLiAKVGhpcyBpbmNsdWRlIAoKLSBsb2cgdHJhbnNmb3JtYXRpb24sIAotIGZpbHRlcmluZyBhbmQgCi0gc3VtbWFyaXNhdGlvbiBvZiB0aGUgZGF0YS4KCiMjIExvZyB0cmFuc2Zvcm0gdGhlIGRhdGEKCmBgYHtyfQpwZSA8LSBsb2dUcmFuc2Zvcm0ocGUsIGJhc2UgPSAyLCBpID0gInBlcHRpZGVSYXciLCBuYW1lID0gInBlcHRpZGVMb2ciKQpgYGAKCiMjIEZpbHRlcmluZwoKMS4gSGFuZGxpbmcgb3ZlcmxhcHBpbmcgcHJvdGVpbiBncm91cHMKCkluIG91ciBhcHByb2FjaCBhIHBlcHRpZGUgY2FuIG1hcCB0byBtdWx0aXBsZSBwcm90ZWlucywgYXMgbG9uZyBhcyB0aGVyZSBpcwpub25lIG9mIHRoZXNlIHByb3RlaW5zIHByZXNlbnQgaW4gYSBzbWFsbGVyIHN1Ymdyb3VwLgoKYGBge3J9CnBlIDwtIGZpbHRlckZlYXR1cmVzKHBlLCB+IFByb3RlaW5zICVpbiUgc21hbGxlc3RVbmlxdWVHcm91cHMocm93RGF0YShwZVtbInBlcHRpZGVMb2ciXV0pJFByb3RlaW5zKSkKYGBgCgoyLiBSZW1vdmUgcmV2ZXJzZSBzZXF1ZW5jZXMgKGRlY295cykgYW5kIGNvbnRhbWluYW50cwoKV2Ugbm93IHJlbW92ZSB0aGUgY29udGFtaW5hbnRzIGFuZCBwZXB0aWRlcyB0aGF0IG1hcCB0byBkZWNveSBzZXF1ZW5jZXMuCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsflJldmVyc2UgIT0gIisiKQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IENvbnRhbWluYW50ICE9ICIrIikKYGBgCgozLiBEcm9wIHBlcHRpZGVzIHRoYXQgd2VyZSBvbmx5IGlkZW50aWZpZWQgaW4gb25lIHNhbXBsZQoKV2Uga2VlcCBwZXB0aWRlcyB0aGF0IHdlcmUgb2JzZXJ2ZWQgYXQgbGFzdCB0d2ljZS4KCmBgYHtyfQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IG5Ob25aZXJvID49MikKbnJvdyhwZVtbInBlcHRpZGVMb2ciXV0pCmBgYAoKV2Uga2VlcCBgciBucm93KHBlW1sicGVwdGlkZUxvZyJdXSlgIHBlcHRpZGVzIHVwb24gZmlsdGVyaW5nLgoKCiMjIE5vcm1hbGl6ZSB0aGUgZGF0YSB1c2luZyBtZWRpYW4gY2VudGVyaW5nIAoKV2Ugbm9ybWFsaXplIHRoZSBkYXRhIGJ5IHN1YnN0cmFjdGluZyB0aGUgc2FtcGxlIG1lZGlhbiBmcm9tIGV2ZXJ5IGludGVuc2l0eSBmb3IgcGVwdGlkZSAkcCQgIGluIGEgc2FtcGxlICRpJDogCgokJHlfe2lwfV5cdGV4dHtub3JtfSA9IHlfe2lwfSAtIFxoYXRcbXVfaSQkIAoKd2l0aCAkXGhhdFxtdV9pJCB0aGUgbWVkaWFuIGludGVuc2l0eSBvdmVyIGFsbCBvYnNlcnZlZCBwZXB0aWRlcyBpbiBzYW1wbGUgJGkkLgoKYGBge3J9CnBlIDwtIG5vcm1hbGl6ZShwZSwgCiAgICAgICAgICAgICAgICBpID0gInBlcHRpZGVMb2ciLCAKICAgICAgICAgICAgICAgIG5hbWUgPSAicGVwdGlkZU5vcm0iLCAKICAgICAgICAgICAgICAgIG1ldGhvZCA9ICJjZW50ZXIubWVkaWFuIikKYGBgCgoKIyMgRXhwbG9yZSAgbm9ybWFsaXplZCBkYXRhCgpVcG9uIHRoZSBub3JtYWxpc2F0aW9uIHRoZSBkZW5zaXR5IGN1cnZlcyBhcmUgbmljZWx5IHJlZ2lzdGVyZWQKCmBgYHtyfQpwZVtbInBlcHRpZGVOb3JtIl1dICU+JSAKICBhc3NheSAlPiUKICBhcy5kYXRhLmZyYW1lKCkgJT4lCiAgZ2F0aGVyKHNhbXBsZSwgaW50ZW5zaXR5KSAlPiUgCiAgbXV0YXRlKHByb2dub3NpcyA9IGNvbERhdGEocGUpW3NhbXBsZSwicHJvZ25vc2lzIl0pICU+JQogIGdncGxvdChhZXMoeCA9IGludGVuc2l0eSxncm91cCA9IHNhbXBsZSxjb2xvciA9IHByb2dub3NpcykpICsgCiAgICBnZW9tX2RlbnNpdHkoKQpgYGAKCldlIGNhbiB2aXN1YWxpemUgb3VyIGRhdGEgdXNpbmcgYSBNdWx0aSBEaW1lbnNpb25hbCBTY2FsaW5nIHBsb3QsCmVnLiBhcyBwcm92aWRlZCBieSB0aGUgYGxpbW1hYCBwYWNrYWdlLgoKYGBge3J9CnBlW1sicGVwdGlkZU5vcm0iXV0gJT4lIAogIGFzc2F5ICU+JQogIGxpbW1hOjpwbG90TURTKGNvbCA9IGFzLm51bWVyaWMoY29sRGF0YShwZSkkcHJvZ25vc2lzKSkKYGBgCgpUaGUgZmlyc3QgYXhpcyBpbiB0aGUgcGxvdCBpcyBzaG93aW5nIHRoZSBsZWFkaW5nIGxvZyBmb2xkIGNoYW5nZXMKKGRpZmZlcmVuY2VzIG9uIHRoZSBsb2cgc2NhbGUpIGJldHdlZW4gdGhlIHNhbXBsZXMuCgoKIyMgU3VtbWFyaXphdGlvbiB0byBwcm90ZWluIGxldmVsCgotIEJ5IGRlZmF1bHQgcm9idXN0IHN1bW1hcml6YXRpb24gaXMgdXNlZDogIGBmdW4gPSBNc0NvcmVVdGlsczo6cm9idXN0U3VtbWFyeSgpYAoKYGBge3Isd2FybmluZz1GQUxTRX0KcGUgPC0gYWdncmVnYXRlRmVhdHVyZXMocGUsCiAgaSA9ICJwZXB0aWRlTm9ybSIsCiAgZmNvbCA9ICJQcm90ZWlucyIsCiAgbmEucm0gPSBUUlVFLAogIG5hbWUgPSAicHJvdGVpbiIpCmBgYAoKCgpgYGB7cn0KcGxvdE1EUyhhc3NheShwZVtbInByb3RlaW4iXV0pLCBjb2wgPSBhcy5udW1lcmljKGNvbERhdGEocGUpJHByb2dub3NpcykpCmBgYAoKTm90ZSB0aGF0IHRoZSBzYW1wbGVzIHVwb24gcm9idXN0IHN1bW1hcmlzYXRpb24gc2hvdyBhIHNlcGFyYXRpb24gYWNjb3JkaW5nIHRvIHRoZSBwcm9nbm9zaXMuCgojIERhdGEgQW5hbHlzaXMKCiMjIEVzdGltYXRpb24KCldlIG1vZGVsIHRoZSBwcm90ZWluIGxldmVsIGV4cHJlc3Npb24gdmFsdWVzIHVzaW5nIGBtc3Fyb2JgLgpCeSBkZWZhdWx0IGBtc3Fyb2IyYCBlc3RpbWF0ZXMgdGhlIG1vZGVsIHBhcmFtZXRlcnMgdXNpbmcgcm9idXN0IHJlZ3Jlc3Npb24uCgpXZSB3aWxsIG1vZGVsIHRoZSBkYXRhIHdpdGggYSBkaWZmZXJlbnQgZ3JvdXAgbWVhbi4gClRoZSBncm91cCBpcyBpbmNvZGVkIGluIHRoZSB2YXJpYWJsZSBgcHJvZ25vc2lzYCBvZiB0aGUgY29sRGF0YS4gCldlIGNhbiBzcGVjaWZ5IHRoaXMgbW9kZWwgYnkgdXNpbmcgYSBmb3JtdWxhIHdpdGggdGhlIGZhY3RvciBjb25kaXRpb24gYXMgaXRzIHByZWRpY3RvcjogCmBmb3JtdWxhID0gfnByb2dub3Npc2AuCgpOb3RlLCB0aGF0IGEgZm9ybXVsYSBhbHdheXMgc3RhcnRzIHdpdGggYSBzeW1ib2wgJ34nLgoKYGBge3IsIHdhcm5pbmc9RkFMU0V9CnBlIDwtIG1zcXJvYihvYmplY3QgPSBwZSwgaSA9ICJwcm90ZWluIiwgZm9ybXVsYSA9IH5wcm9nbm9zaXMpCmBgYAoKIyMgSW5mZXJlbmNlCgpGaXJzdCwgd2UgZXh0cmFjdCB0aGUgcGFyYW1ldGVyIG5hbWVzIG9mIHRoZSBtb2RlbCBieSBsb29raW5nIGF0IHRoZSBmaXJzdCBtb2RlbC4gClRoZSBtb2RlbHMgYXJlIHN0b3JlZCBpbiB0aGUgcm93IGRhdGEgb2YgdGhlIGFzc2F5IHVuZGVyIHRoZSBkZWZhdWx0IG5hbWUgbXNxcm9iTW9kZWxzLiAKCmBgYHtyfQpnZXRDb2VmKHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRtc3Fyb2JNb2RlbHNbWzFdXSkKYGBgCgpXZSBjYW4gYWxzbyBleHBsb3JlIHRoZSBkZXNpZ24gb2YgdGhlIG1vZGVsIHRoYXQgd2Ugc3BlY2lmaWVkIHVzaW5nIHRoZSB0aGUgcGFja2FnZSBgRXhwbG9yZU1vZGVsTWF0cml4YCAKCmBgYHtyfQpsaWJyYXJ5KEV4cGxvcmVNb2RlbE1hdHJpeCkKVmlzdWFsaXplRGVzaWduKGNvbERhdGEocGUpLH5wcm9nbm9zaXMpJHBsb3RsaXN0CmBgYAoKU3Bpa2UtaW4gY29uZGl0aW9uIGBBYCBpcyB0aGUgcmVmZXJlbmNlIGNsYXNzLiBTbyB0aGUgbWVhbiBsb2cyIGV4cHJlc3Npb24KZm9yIHNhbXBsZXMgZnJvbSBnb29kIHByb2dub3NpcyAoT1IpIGlzICcoSW50ZXJjZXB0KS4KVGhlIG1lYW4gbG9nMiBleHByZXNzaW9uIGZvciBzYW1wbGVzIGZyb20gcG9vciBwcm9nbm9zaXMgKFBEKSBpcyAnKEludGVyY2VwdCkrcHJvZ25vc2lzUEQnLgpIZW5jZSwgdGhlIGF2ZXJhZ2UgbG9nMiBmb2xkIGNoYW5nZSBiZXR3ZWVuIHByb2dub3NpcyBQRCBhbmQgcHJvZ25vc2lzIE9SIGlzIG1vZGVsbGVkIHVzaW5nIHRoZSBwYXJhbWV0ZXIgJ2NvbmRpdGlvblBEJy4KVGh1cywgd2UgYXNzZXNzIHRoZSBjb250cmFzdCAnY29uZGl0aW9uUEQgPSAwJyB3aXRoIG91ciBzdGF0aXN0aWNhbCB0ZXN0LgoKYGBge3J9CkwgPC0gbWFrZUNvbnRyYXN0KCJwcm9nbm9zaXNQRD0wIiwgcGFyYW1ldGVyTmFtZXMgPSBjKCJwcm9nbm9zaXNQRCIpKQpwZSA8LSBoeXBvdGhlc2lzVGVzdChvYmplY3QgPSBwZSwgaSA9ICJwcm90ZWluIiwgY29udHJhc3QgPSBMKQpgYGAKCgojIyBQbG90cwoKIyMjIFZvbGNhbm8tcGxvdAoKCmBgYHtyLHdhcm5pbmc9RkFMU0V9CnZvbGNhbm8gPC0gZ2dwbG90KHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRwcm9nbm9zaXNQRCwKICAgICAgICAgICAgICAgICAgYWVzKHggPSBsb2dGQywgeSA9IC1sb2cxMChwdmFsKSwgY29sb3IgPSBhZGpQdmFsIDwgMC4wNSkpICsKICBnZW9tX3BvaW50KGNleCA9IDIuNSkgKwogIHNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBhbHBoYShjKCJibGFjayIsICJyZWQiKSwgMC41KSkgKyB0aGVtZV9taW5pbWFsKCkKdm9sY2FubwpgYGAKCk5vdGUsIHRoYXQgYHIgc3VtKHJvd0RhdGEocGVbWyJwcm90ZWluIl1dKSRwcm9nbm9zaXNQRCRhZGpQdmFsIDwgMC4wNSwgbmEucm0gPSBUUlVFKWAgcHJvdGVpbnMgYXJlIGZvdW5kIHRvIGJlIGRpZmZlcmVudGlhbGx5IGFidW5kYW50LgoKIyMjIEhlYXRtYXAKClRoZXJlIHdlcmUgbm8gcHJvdGVpbnMgc2lnbmlmaWNhbnRseSBkaWZmZXJlbnRpYWxseSBhYnVuZGFudCBhdCB0aGUgNSUgRkRSIGxldmVsLiAKV2Ugd2lsbCB0aGVyZWZvcmUgbWFrZSBhbiBoZWF0bWFwIGZvciB0aGUgdG9wIDEwIERFIHByb3RlaW5zLiBOb3RlLCBob3dldmVyIHRoYXQgdGhleSBhcmUgbm90IHN0YXRpc3RpY2FsbHkgc2lnbmlmaWNhbnQhIAoKYGBge3J9CiNzaWdOYW1lcyA8LSByb3dEYXRhKHBlW1sicHJvdGVpbiJdXSkkcHJvZ25vc2lzUEQgJT4lCiMgIHJvd25hbWVzX3RvX2NvbHVtbigicHJvdGVpbiIpICU+JQojICBmaWx0ZXIoYWRqUHZhbDwwLjA1KSAlPiUKIyAgcHVsbChwcm90ZWluKQojaGVhdG1hcChhc3NheShwZVtbInByb3RlaW4iXV0pW3NpZ05hbWVzLCBdKQpvcmRlclByb3QgPC0gcm93RGF0YShwZVtbInByb3RlaW4iXV0pJHByb2dub3Npc1BEICU+JSAKICAgIHJvd25hbWVzX3RvX2NvbHVtbigicHJvdGVpbiIpICU+JQogICAgYXJyYW5nZShwdmFsKSAlPiUgCiAgICBwdWxsKHByb3RlaW4pCgpoZWF0bWFwKGFzc2F5KHBlW1sicHJvdGVpbiJdXSlbb3JkZXJQcm90WzE6MTBdLF0pCmBgYAoKCiMjIyBEZXRhaWwgcGxvdHMKCldlIG1ha2UgZGV0YWlsIHBsb3RzIGZvciB0aGUgdG9wIDEwIHByb3RlaW5zLiBOb3RlLCBob3dldmVyIHRoYXQgdGhlaXIgZm9sZCBjaGFuZ2VzIGFyZSBub3Qgc3RhdGlzdGljYWxseSBzaWduaWZpY2FudCBhdCB0aGUgNSUgRkRSIGxldmVsLiAKCmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQojZm9yIChwcm90TmFtZSBpbiBzaWdOYW1lcykKZm9yIChwcm90TmFtZSBpbiBvcmRlclByb3RbMToxMF0pCnsKcGVQbG90IDwtIHBlW3Byb3ROYW1lLCAsIGMoInBlcHRpZGVOb3JtIiwicHJvdGVpbiIpXQpwZVBsb3REZiA8LSBkYXRhLmZyYW1lKGxvbmdGb3JtYXQocGVQbG90KSkKcGVQbG90RGYkYXNzYXkgPC0gZmFjdG9yKHBlUGxvdERmJGFzc2F5LAogICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKCJwZXB0aWRlTm9ybSIsICJwcm90ZWluIikpCnBlUGxvdERmJHByb2dub3NpcyA8LSBhcy5mYWN0b3IoY29sRGF0YShwZVBsb3QpW3BlUGxvdERmJGNvbG5hbWUsICJwcm9nbm9zaXMiXSkKCiMgcGxvdHRpbmcKcDEgPC0gZ2dwbG90KGRhdGEgPSBwZVBsb3REZiwKICAgICAgIGFlcyh4ID0gY29sbmFtZSwgeSA9IHZhbHVlLCBncm91cCA9IHJvd25hbWUpKSArCiAgICBnZW9tX2xpbmUoKSArIAogICAgZ2VvbV9wb2ludCgpICsgIAogICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA3MCwgaGp1c3QgPSAxLCB2anVzdCA9IDAuNSkpICsKICAgIGZhY2V0X2dyaWQofmFzc2F5KSArIAogICAgZ2d0aXRsZShwcm90TmFtZSkKcHJpbnQocDEpCgojIHBsb3R0aW5nIDIKcDIgPC0gZ2dwbG90KHBlUGxvdERmLCBhZXMoeCA9IGNvbG5hbWUsIHkgPSB2YWx1ZSwgZmlsbCA9IHByb2dub3NpcykpICsKICBnZW9tX2JveHBsb3Qob3V0bGllci5zaGFwZSA9IE5BKSArIAogIGdlb21fcG9pbnQoCiAgICBwb3NpdGlvbiA9IHBvc2l0aW9uX2ppdHRlcih3aWR0aCA9IC4xKSwKICAgIGFlcyhzaGFwZSA9IHJvd25hbWUpKSArCiAgc2NhbGVfc2hhcGVfbWFudWFsKHZhbHVlcyA9IDE6bnJvdyhwZVBsb3REZikpICsKICBsYWJzKHRpdGxlID0gcHJvdE5hbWUsIHggPSAic2FtcGxlIiwgeSA9ICJwZXB0aWRlIGludGVuc2l0eSAobG9nMikiKSArIAogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gNzAsIGhqdXN0ID0gMSwgdmp1c3QgPSAwLjUpKSArCiAgZmFjZXRfZ3JpZCh+YXNzYXkpCnByaW50KHAyKQp9CmBgYAoKCiMgU2Vzc2lvbiBJbmZvCgpXaXRoIHJlc3BlY3QgdG8gcmVwcm9kdWNpYmlsaXR5LCBpdCBpcyBoaWdobHkgcmVjb21tZW5kZWQgdG8gaW5jbHVkZSBhIHNlc3Npb24gaW5mbyBpbiB5b3VyIHNjcmlwdCBzbyB0aGF0IHJlYWRlcnMgb2YgeW91ciBvdXRwdXQgY2FuIHNlZSB5b3VyIHBhcnRpY3VsYXIgc2V0dXAgb2YgUi4gCgpgYGB7cn0Kc2Vzc2lvbkluZm8oKQpgYGAK