Statistical Methods for Quantitative MS-based Proteomics: Part I. Preprocessing
Lieven Clement
statOmics, Ghent University
This is part of the online course Proteomics Data Analysis (PDA)
Outline
Introduction
Preprocessing
- Log-transformation
- Filtering
- Normalization
- Summarization
Note, that the R-code is included for learners who are aiming to develop R/markdown scripts to automate their quantitative proteomics data analyses. According to the target audience of the course we either work with a graphical user interface (GUI) in a R/shiny App msqrob2gui (e.g. Proteomics Bioinformatics course of the EBI and the Proteomics Data Analysis course at the Gulbenkian institute) or with R/markdowns scripts (e.g. Bioinformatics Summer School at UCLouvain or the Statistical Genomics Course at Ghent University).
1 Intro: Challenges in Label-Free Quantitative Proteomics
1.1 MS-based workflow
Peptide Characteristics
- Modifications
- Ionisation Efficiency: huge variability
- Identification
- Misidentification \(\rightarrow\) outliers
- MS\(^2\) selection on peptide abundance
- Context depending missingness
- Non-random missingness
\(\rightarrow\) Unbalanced pepide identifications across samples and messy data
1.2 Level of quantification
- MS-based proteomics returns peptides: pieces of proteins
- Quantification commonly required on the protein level
1.3 Label-free Quantitative Proteomics Data Analysis Workflows
1.4 CPTAC Spike-in Study
Same trypsin-digested yeast proteome background in each sample
Trypsin-digested Sigma UPS1 standard: 48 different human proteins spiked in at 5 different concentrations (treatment A-E)
Samples repeatedly run on different instruments in different labs
After MaxQuant search with match between runs option
- 41% of all proteins are quantified in all samples
- 6.6% of all peptides are quantified in all samples
\(\rightarrow\) vast amount of missingness
1.5 Maxquant output
2 Import the data in R
2.1 Data infrastructure
Click to see background on data infrastructure used in R to store proteomics data
- We use the
QFeaturespackage that provides the infrastructure to- store,
- process,
- manipulate and
- analyse quantitative data/features from mass spectrometry experiments.
- store,
- It is based on the
SummarizedExperimentandMultiAssayExperimentclasses.
Conceptual representation of a SummarizedExperiment object.
Assays contain information on the measured omics features (rows) for
different samples (columns). The rowData contains
information on the omics features, the colData contains
information on the samples, i.e. experimental design etc.
Assays in a QFeatures object have a hierarchical relation:
- proteins are composed of peptides,
- themselves produced by spectra
- relations between assays are tracked and recorded throughout data processing
Conceptual representation of a QFeatures object and the
aggregative relation between different assays.
2.2 Import data in R
2.2.1 Load libraries
Click to see code
library(tidyverse)
library(limma)
library(QFeatures)
library(msqrob2)
library(plotly)
library(ggplot2)2.2.2 Read data
Click to see background and code
- We use a peptides.txt file from MS-data quantified with maxquant that contains MS1 intensities summarized at the peptide level.
peptidesFile <- "https://raw.githubusercontent.com/statOmics/PDA/data/quantification/fullCptacDatasSetNotForTutorial/peptides.txt"- Maxquant stores the intensity data for the different samples in columnns that start with Intensity. We can retreive the column names with the intensity data with the code below:
ecols <- grep("Intensity\\.", names(read.delim(peptidesFile)))- Read the data and store it in QFeatures object
pe <- readQFeatures(
table = peptidesFile,
fnames = 1,
ecol = ecols,
name = "peptideRaw", sep="\t")2.2.3 Explore object
Click to see background and code
- The rowData contains information on the features (peptides) in the assay. E.g. Sequence, protein, …
rowData(pe[["peptideRaw"]])## DataFrame with 11466 rows and 143 columns
## Sequence N.term.cleavage.window C.term.cleavage.window
## <character> <character> <character>
## AAAAGAGGAGDSGDAVTK AAAAGAGGAG... EHQHDEQKAA... DSGDAVTKIG...
## AAAALAGGK AAAALAGGK QQLSKAAKAA... AAALAGGKKS...
## AAAALAGGKK AAAALAGGKK QQLSKAAKAA... AALAGGKKSK...
## AAADALSDLEIK AAADALSDLE... MPKETPSKAA... ALSDLEIKDS...
## AAADALSDLEIKDSK AAADALSDLE... MPKETPSKAA... DLEIKDSKSN...
## ... ... ... ...
## YYSIYDLGNNAVGLAK YYSIYDLGNN... VGDAFLRKYY... NNAVGLAKAI...
## YYTFNGPNYNENETIR YYTFNGPNYN... FKDGSYPKYY... YNENETIRHI...
## YYTITEVATR YYTITEVATR QEWDINERYY... TITEVATRAK...
## YYTVFDRDNNR YYTVFDRDNN... LGDVFIGRYY... VFDRDNNRVG...
## YYTVFDRDNNRVGFAEAAR YYTVFDRDNN... LGDVFIGRYY... VGFAEAARL_...
## Amino.acid.before First.amino.acid Second.amino.acid
## <character> <character> <character>
## AAAAGAGGAGDSGDAVTK K A A
## AAAALAGGK K A A
## AAAALAGGKK K A A
## AAADALSDLEIK K A A
## AAADALSDLEIKDSK K A A
## ... ... ... ...
## YYSIYDLGNNAVGLAK K Y Y
## YYTFNGPNYNENETIR K Y Y
## YYTITEVATR R Y Y
## YYTVFDRDNNR R Y Y
## YYTVFDRDNNRVGFAEAAR R Y Y
## Second.last.amino.acid Last.amino.acid Amino.acid.after
## <character> <character> <character>
## AAAAGAGGAGDSGDAVTK T K I
## AAAALAGGK G K K
## AAAALAGGKK K K S
## AAADALSDLEIK I K D
## AAADALSDLEIKDSK S K S
## ... ... ... ...
## YYSIYDLGNNAVGLAK A K A
## YYTFNGPNYNENETIR I R H
## YYTITEVATR T R A
## YYTVFDRDNNR N R V
## YYTVFDRDNNRVGFAEAAR A R L
## A.Count R.Count N.Count D.Count C.Count Q.Count
## <integer> <integer> <integer> <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 7 0 0 2 0 0
## AAAALAGGK 5 0 0 0 0 0
## AAAALAGGKK 5 0 0 0 0 0
## AAADALSDLEIK 4 0 0 2 0 0
## AAADALSDLEIKDSK 4 0 0 3 0 0
## ... ... ... ... ... ... ...
## YYSIYDLGNNAVGLAK 2 0 2 1 0 0
## YYTFNGPNYNENETIR 0 1 4 0 0 0
## YYTITEVATR 1 1 0 0 0 0
## YYTVFDRDNNR 0 2 2 2 0 0
## YYTVFDRDNNRVGFAEAAR 3 3 2 2 0 0
## E.Count G.Count H.Count I.Count L.Count K.Count
## <integer> <integer> <integer> <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 0 5 0 0 0 1
## AAAALAGGK 0 2 0 0 1 1
## AAAALAGGKK 0 2 0 0 1 2
## AAADALSDLEIK 1 0 0 1 2 1
## AAADALSDLEIKDSK 1 0 0 1 2 2
## ... ... ... ... ... ... ...
## YYSIYDLGNNAVGLAK 0 2 0 1 2 1
## YYTFNGPNYNENETIR 2 1 0 1 0 0
## YYTITEVATR 1 0 0 1 0 0
## YYTVFDRDNNR 0 0 0 0 0 0
## YYTVFDRDNNRVGFAEAAR 1 1 0 0 0 0
## M.Count F.Count P.Count S.Count T.Count W.Count
## <integer> <integer> <integer> <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 0 0 0 1 1 0
## AAAALAGGK 0 0 0 0 0 0
## AAAALAGGKK 0 0 0 0 0 0
## AAADALSDLEIK 0 0 0 1 0 0
## AAADALSDLEIKDSK 0 0 0 2 0 0
## ... ... ... ... ... ... ...
## YYSIYDLGNNAVGLAK 0 0 0 1 0 0
## YYTFNGPNYNENETIR 0 1 1 0 2 0
## YYTITEVATR 0 0 0 0 3 0
## YYTVFDRDNNR 0 1 0 0 1 0
## YYTVFDRDNNRVGFAEAAR 0 2 0 0 1 0
## Y.Count V.Count U.Count Length Missed.cleavages
## <integer> <integer> <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 0 1 0 18 0
## AAAALAGGK 0 0 0 9 0
## AAAALAGGKK 0 0 0 10 1
## AAADALSDLEIK 0 0 0 12 0
## AAADALSDLEIKDSK 0 0 0 15 1
## ... ... ... ... ... ...
## YYSIYDLGNNAVGLAK 3 1 0 16 0
## YYTFNGPNYNENETIR 3 0 0 16 0
## YYTITEVATR 2 1 0 10 0
## YYTVFDRDNNR 2 1 0 11 1
## YYTVFDRDNNRVGFAEAAR 2 2 0 19 2
## Mass Proteins Leading.razor.protein
## <numeric> <character> <character>
## AAAAGAGGAGDSGDAVTK 1445.675 sp|P38915|... sp|P38915|...
## AAAALAGGK 728.418 sp|Q3E792|... sp|Q3E792|...
## AAAALAGGKK 856.513 sp|Q3E792|... sp|Q3E792|...
## AAADALSDLEIK 1215.635 sp|P09938|... sp|P09938|...
## AAADALSDLEIKDSK 1545.789 sp|P09938|... sp|P09938|...
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1759.88 sp|P07267|... sp|P07267|...
## YYTFNGPNYNENETIR 1993.88 sp|Q00955|... sp|Q00955|...
## YYTITEVATR 1215.61 sp|P38891|... sp|P38891|...
## YYTVFDRDNNR 1461.66 P07339ups|... P07339ups|...
## YYTVFDRDNNRVGFAEAAR 2263.08 P07339ups|... P07339ups|...
## Start.position End.position Unique..Groups.
## <integer> <integer> <character>
## AAAAGAGGAGDSGDAVTK 97 114 yes
## AAAALAGGK 13 21 yes
## AAAALAGGKK 13 22 yes
## AAADALSDLEIK 9 20 yes
## AAADALSDLEIKDSK 9 23 yes
## ... ... ... ...
## YYSIYDLGNNAVGLAK 388 403 yes
## YYTFNGPNYNENETIR 1275 1290 yes
## YYTITEVATR 311 320 yes
## YYTVFDRDNNR 225 235 yes
## YYTVFDRDNNRVGFAEAAR 225 243 yes
## Unique..Proteins. Charges PEP Score
## <character> <character> <numeric> <numeric>
## AAAAGAGGAGDSGDAVTK yes 2 1.1843e-05 82.942
## AAAALAGGK no 2 7.4562e-06 134.810
## AAAALAGGKK no 2 3.3094e-09 143.730
## AAADALSDLEIK yes 2 9.1593e-23 182.230
## AAADALSDLEIKDSK yes 3 1.5319e-04 73.927
## ... ... ... ... ...
## YYSIYDLGNNAVGLAK yes 2 7.7415e-37 174.240
## YYTFNGPNYNENETIR yes 2 4.2208e-21 147.750
## YYTITEVATR yes 2 1.3566e-04 109.160
## YYTVFDRDNNR yes 2 6.1425e-04 110.930
## YYTVFDRDNNRVGFAEAAR yes 3 8.9859e-04 59.728
## Identification.type.6A_1 Identification.type.6A_2
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By MS/MS
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By matchin... By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6A_3 Identification.type.6A_4
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By MS/MS
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By MS/MS
## AAADALSDLEIK By matchin... By MS/MS
## AAADALSDLEIKDSK By matchin... By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By MS/MS
## YYTFNGPNYNENETIR By matchin... By MS/MS
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6A_5 Identification.type.6A_6
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By MS/MS
## YYTFNGPNYNENETIR By MS/MS By MS/MS
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6A_7 Identification.type.6A_8
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By MS/MS
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By MS/MS By matchin...
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6A_9 Identification.type.6B_1
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By matchin...
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By MS/MS
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6B_2 Identification.type.6B_3
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By MS/MS By matchin...
## AAADALSDLEIKDSK By matchin... By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6B_4 Identification.type.6B_5
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By matchin...
## YYTFNGPNYNENETIR By MS/MS By MS/MS
## YYTITEVATR By MS/MS By MS/MS
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6B_6 Identification.type.6B_7
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By MS/MS
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By MS/MS By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6B_8 Identification.type.6B_9
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By MS/MS
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By matchin... By matchin...
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6C_1 Identification.type.6C_2
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By MS/MS
## AAADALSDLEIK By MS/MS By matchin...
## AAADALSDLEIKDSK By matchin... By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6C_3 Identification.type.6C_4
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By MS/MS
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By matchin... By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By MS/MS
## YYTFNGPNYNENETIR By matchin... By MS/MS
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6C_5 Identification.type.6C_6
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By MS/MS
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6C_7 Identification.type.6C_8
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By matchin...
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By matchin... By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By MS/MS
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6C_9 Identification.type.6D_1
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By MS/MS By matchin...
## AAAALAGGKK By MS/MS By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6D_2 Identification.type.6D_3
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By matchin... By matchin...
## AAADALSDLEIKDSK By MS/MS By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By MS/MS
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6D_4 Identification.type.6D_5
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By MS/MS By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By MS/MS
## YYTFNGPNYNENETIR By MS/MS By MS/MS
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6D_6 Identification.type.6D_7
## <character> <character>
## AAAAGAGGAGDSGDAVTK By MS/MS By matchin...
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By MS/MS
## AAADALSDLEIK By MS/MS By matchin...
## AAADALSDLEIKDSK By matchin... By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By matchin...
## YYTFNGPNYNENETIR By MS/MS By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By MS/MS
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6D_8 Identification.type.6D_9
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By MS/MS By matchin...
## YYTVFDRDNNR By MS/MS By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6E_1 Identification.type.6E_2
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6E_3 Identification.type.6E_4
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By MS/MS
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By matchin... By MS/MS
## AAADALSDLEIKDSK By MS/MS By matchin...
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By MS/MS
## YYTFNGPNYNENETIR By matchin... By MS/MS
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By matchin... By MS/MS
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6E_5 Identification.type.6E_6
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By matchin... By matchin...
## AAAALAGGKK By matchin... By matchin...
## AAADALSDLEIK By MS/MS By matchin...
## AAADALSDLEIKDSK By MS/MS By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By MS/MS By MS/MS
## YYTFNGPNYNENETIR By MS/MS By MS/MS
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By MS/MS By matchin...
## YYTVFDRDNNRVGFAEAAR By matchin... By MS/MS
## Identification.type.6E_7 Identification.type.6E_8
## <character> <character>
## AAAAGAGGAGDSGDAVTK By matchin... By matchin...
## AAAALAGGK By MS/MS By MS/MS
## AAAALAGGKK By MS/MS By MS/MS
## AAADALSDLEIK By MS/MS By MS/MS
## AAADALSDLEIKDSK By matchin... By MS/MS
## ... ... ...
## YYSIYDLGNNAVGLAK By matchin... By matchin...
## YYTFNGPNYNENETIR By matchin... By matchin...
## YYTITEVATR By matchin... By matchin...
## YYTVFDRDNNR By MS/MS By MS/MS
## YYTVFDRDNNRVGFAEAAR By matchin... By matchin...
## Identification.type.6E_9 Experiment.6A_1 Experiment.6A_2
## <character> <integer> <integer>
## AAAAGAGGAGDSGDAVTK By matchin... NA 1
## AAAALAGGK By MS/MS NA 1
## AAAALAGGKK By MS/MS NA 1
## AAADALSDLEIK By MS/MS 1 1
## AAADALSDLEIKDSK By MS/MS 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK By matchin... NA NA
## YYTFNGPNYNENETIR By MS/MS NA NA
## YYTITEVATR By matchin... 1 NA
## YYTVFDRDNNR By MS/MS NA NA
## YYTVFDRDNNRVGFAEAAR By matchin... NA NA
## Experiment.6A_3 Experiment.6A_4 Experiment.6A_5
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA 1 1
## AAAALAGGK 2 1 1
## AAAALAGGKK NA 1 NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK NA 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA 1 1
## YYTFNGPNYNENETIR NA 1 1
## YYTITEVATR 1 NA NA
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6A_6 Experiment.6A_7 Experiment.6A_8
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 1 1
## AAAALAGGK 1 2 1
## AAAALAGGKK 1 1 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 NA NA
## YYTFNGPNYNENETIR 1 1 NA
## YYTITEVATR 1 1 NA
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6A_9 Experiment.6B_1 Experiment.6B_2
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 NA NA
## AAAALAGGK 1 1 1
## AAAALAGGKK 1 NA 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 NA 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA NA NA
## YYTFNGPNYNENETIR 1 NA NA
## YYTITEVATR NA 1 1
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6B_3 Experiment.6B_4 Experiment.6B_5
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA NA 1
## AAAALAGGK 1 2 1
## AAAALAGGKK 1 1 NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK NA 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA 1 1
## YYTFNGPNYNENETIR NA 1 1
## YYTITEVATR 1 1 1
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6B_6 Experiment.6B_7 Experiment.6B_8
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 NA 1
## AAAALAGGK NA 2 1
## AAAALAGGKK NA 1 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 NA NA
## YYTFNGPNYNENETIR 1 1 NA
## YYTITEVATR 1 NA 1
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6B_9 Experiment.6C_1 Experiment.6C_2
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 NA NA
## AAAALAGGK 2 NA 1
## AAAALAGGKK 1 NA 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA NA NA
## YYTFNGPNYNENETIR NA NA NA
## YYTITEVATR NA 1 1
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6C_3 Experiment.6C_4 Experiment.6C_5
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA 1 1
## AAAALAGGK 2 2 NA
## AAAALAGGKK NA 1 NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA 1 1
## YYTFNGPNYNENETIR NA 1 1
## YYTITEVATR 1 1 NA
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6C_6 Experiment.6C_7 Experiment.6C_8
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 1 1
## AAAALAGGK NA 2 1
## AAAALAGGKK NA 1 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 NA NA
## YYTFNGPNYNENETIR 1 1 1
## YYTITEVATR 1 NA 1
## YYTVFDRDNNR 1 NA 1
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6C_9 Experiment.6D_1 Experiment.6D_2
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 NA NA
## AAAALAGGK 1 NA 1
## AAAALAGGKK 1 NA NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA NA NA
## YYTFNGPNYNENETIR 1 NA NA
## YYTITEVATR 1 NA 1
## YYTVFDRDNNR NA NA NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6D_3 Experiment.6D_4 Experiment.6D_5
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA 1 1
## AAAALAGGK 1 1 1
## AAAALAGGKK NA 1 NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA 1 1
## YYTFNGPNYNENETIR NA 1 1
## YYTITEVATR 1 1 1
## YYTVFDRDNNR NA 1 1
## YYTVFDRDNNRVGFAEAAR NA 1 NA
## Experiment.6D_6 Experiment.6D_7 Experiment.6D_8
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 1 NA
## AAAALAGGK NA 2 1
## AAAALAGGKK NA 1 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 1 NA
## YYTFNGPNYNENETIR 1 1 1
## YYTITEVATR 1 NA 1
## YYTVFDRDNNR 1 1 1
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6D_9 Experiment.6E_1 Experiment.6E_2
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA NA 1
## AAAALAGGK 2 NA 1
## AAAALAGGKK 1 NA NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK NA NA NA
## YYTFNGPNYNENETIR 1 NA NA
## YYTITEVATR NA NA 1
## YYTVFDRDNNR 1 1 NA
## YYTVFDRDNNRVGFAEAAR NA NA NA
## Experiment.6E_3 Experiment.6E_4 Experiment.6E_5
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK NA NA 1
## AAAALAGGK 2 2 1
## AAAALAGGKK NA 1 NA
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 1 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 1 1
## YYTFNGPNYNENETIR NA 1 1
## YYTITEVATR 1 1 1
## YYTVFDRDNNR 1 1 1
## YYTVFDRDNNRVGFAEAAR NA 1 1
## Experiment.6E_6 Experiment.6E_7 Experiment.6E_8
## <integer> <integer> <integer>
## AAAAGAGGAGDSGDAVTK 1 NA NA
## AAAALAGGK NA 2 2
## AAAALAGGKK NA 1 1
## AAADALSDLEIK 1 1 1
## AAADALSDLEIKDSK 1 NA 1
## ... ... ... ...
## YYSIYDLGNNAVGLAK 1 NA NA
## YYTFNGPNYNENETIR 1 1 1
## YYTITEVATR NA NA NA
## YYTVFDRDNNR 1 1 1
## YYTVFDRDNNRVGFAEAAR 1 1 1
## Experiment.6E_9 Intensity Reverse Potential.contaminant
## <integer> <numeric> <character> <character>
## AAAAGAGGAGDSGDAVTK NA 1190800
## AAAALAGGK 1 280990000
## AAAALAGGKK 1 33360000
## AAADALSDLEIK 1 54622000
## AAADALSDLEIKDSK 1 18910000
## ... ... ... ... ...
## YYSIYDLGNNAVGLAK NA 2145900
## YYTFNGPNYNENETIR 1 5608800
## YYTITEVATR NA 13034000
## YYTVFDRDNNR 1 8702500
## YYTVFDRDNNRVGFAEAAR 1 2391100
## id Protein.group.IDs Mod..peptide.IDs Evidence.IDs
## <integer> <character> <character> <character>
## AAAAGAGGAGDSGDAVTK 0 859 0 0;1;2;3;4;...
## AAAALAGGK 1 230 1 24;25;26;2...
## AAAALAGGKK 2 230 2 74;75;76;7...
## AAADALSDLEIK 3 229 3 99;100;101...
## AAADALSDLEIKDSK 4 229 4 144;145;14...
## ... ... ... ... ...
## YYSIYDLGNNAVGLAK 11461 196 12240 331367;331...
## YYTFNGPNYNENETIR 11462 1254 12241 331384;331...
## YYTITEVATR 11463 854 12242 331411;331...
## YYTVFDRDNNR 11464 34 12243 331439;331...
## YYTVFDRDNNRVGFAEAAR 11465 34 12244 331455;331...
## MS.MS.IDs Best.MS.MS Oxidation..M..site.IDs MS.MS.Count
## <character> <integer> <character> <integer>
## AAAAGAGGAGDSGDAVTK 0;1;2;3;4;... 0 10
## AAAALAGGK 10;11;12;1... 21 18
## AAAALAGGKK 30;31;32;3... 31 21
## AAADALSDLEIK 51;52;53;5... 72 29
## AAADALSDLEIKDSK 85;86;87;8... 94 32
## ... ... ... ... ...
## YYSIYDLGNNAVGLAK 169138;169... 169147 13
## YYTFNGPNYNENETIR 169151;169... 169159 14
## YYTITEVATR 169165;169... 169173 12
## YYTVFDRDNNR 169177;169... 169180 7
## YYTVFDRDNNRVGFAEAAR 169184 169184 1- The colData contains information on the samples
colData(pe)## DataFrame with 45 rows and 0 columns- No information is stored yet on the design.
pe %>% colnames## CharacterList of length 1
## [["peptideRaw"]] Intensity.6A_1 Intensity.6A_2 ... Intensity.6E_9Note, that the sample names include the spike-in condition.
They also end on a number.
- 1-3 is from lab 1,
- 4-6 from lab 2 and
- 7-9 from lab 3.
We update the colData with information on the design
colData(pe)$lab <- rep(rep(paste0("lab",1:3),each=3),5) %>% as.factor
colData(pe)$condition <- pe[["peptideRaw"]] %>% colnames %>% substr(12,12) %>% as.factor
colData(pe)$spikeConcentration <- rep(c(A = 0.25, B = 0.74, C = 2.22, D = 6.67, E = 20),each = 9)- We explore the colData again
colData(pe)## DataFrame with 45 rows and 3 columns
## lab condition spikeConcentration
## <factor> <factor> <numeric>
## Intensity.6A_1 lab1 A 0.25
## Intensity.6A_2 lab1 A 0.25
## Intensity.6A_3 lab1 A 0.25
## Intensity.6A_4 lab2 A 0.25
## Intensity.6A_5 lab2 A 0.25
## ... ... ... ...
## Intensity.6E_5 lab2 E 20
## Intensity.6E_6 lab2 E 20
## Intensity.6E_7 lab3 E 20
## Intensity.6E_8 lab3 E 20
## Intensity.6E_9 lab3 E 203 Preprocessing
3.1 Log-transformation
3.1.1 Explore the data with plots
Peptide AALEELVK from spiked-in UPS protein P12081. We only show data from lab1.
Click to see code to make plot
subset <- pe["AALEELVK",colData(pe)$lab=="lab1"]
plotWhyLog <- data.frame(concentration = colData(subset)$spikeConcentration,
y = assay(subset[["peptideRaw"]]) %>% c
) %>%
ggplot(aes(concentration, y)) +
geom_point() +
xlab("concentration (fmol/l)") +
ggtitle("peptide AALEELVK in lab1")plotWhyLog
- Variance increases with the mean \(\rightarrow\) Multiplicative error structure
Click to see code to make plot
plotLog <- data.frame(concentration = colData(subset)$spikeConcentration,
y = assay(subset[["peptideRaw"]]) %>% c
) %>%
ggplot(aes(concentration, y)) +
geom_point() +
scale_x_continuous(trans='log2') +
scale_y_continuous(trans='log2') +
xlab("concentration (fmol/l)") +
ggtitle("peptide AALEELVK in lab1 with axes on log scale")plotLog
- Data seems to be homoscedastic on log-scale \(\rightarrow\) log transformation of the intensity data
- In quantitative proteomics analysis on \(\log_2\)
\(\rightarrow\) Differences on a \(\log_2\) scale: \(\log_2\) fold changes
\[ \log_2 B - \log_2 A = \log_2 \frac{B}{A} = \log FC_\text{B - A} \] \[ \begin{array} {l} log_2 FC = 1 \rightarrow FC = 2^1 =2\\ log_2 FC = 2 \rightarrow FC = 2^2 = 4\\ \end{array} \]
3.1.2 log-transformation of the data
Click to see code to log-transfrom the data
- We calculate how many non zero intensities we have for each peptide and this can be useful for filtering.
rowData(pe[["peptideRaw"]])$nNonZero <- rowSums(assay(pe[["peptideRaw"]]) > 0)- Peptides with zero intensities are missing peptides and should be
represent with a
NAvalue rather than0.
pe <- zeroIsNA(pe, "peptideRaw") # convert 0 to NA- Logtransform data with base 2
pe <- logTransform(pe, base = 2, i = "peptideRaw", name = "peptideLog")3.2 Filtering
- Reverse sequences
- Only identified by modification site (only modified peptides detected)
- Razor peptides: non-unique peptides assigned to the protein group with the most other peptides
- Contaminants
- Peptides few identifications
- Proteins that are only identified with one or a few peptides
Filtering does not induce bias if the criterion is independent from the downstream data analysis!
Click to see code to filter the data
- Handling overlapping protein groups
In our approach a peptide can map to multiple proteins, as long as there is none of these proteins present in a smaller subgroup.
pe <- filterFeatures(pe, ~ Proteins %in% smallestUniqueGroups(rowData(pe[["peptideLog"]])$Proteins))- Remove reverse sequences (decoys) and contaminants
We now remove the contaminants, peptides that map to decoy sequences, and proteins which were only identified by peptides with modifications.
pe <- filterFeatures(pe,~Reverse != "+")
pe <- filterFeatures(pe,~ Potential.contaminant != "+")- Drop peptides that were only identified in one sample
We keep peptides that were observed at last twice.
pe <- filterFeatures(pe,~ nNonZero >=2)
nrow(pe[["peptideLog"]])## [1] 104783.3 Normalization
Click to see code to make plot
densityConditionD <- pe[["peptideLog"]][,colData(pe)$condition=="D"] %>%
assay %>%
as.data.frame() %>%
gather(sample, intensity) %>%
mutate(lab = colData(pe)[sample,"lab"]) %>%
ggplot(aes(x=intensity,group=sample,color=lab)) +
geom_density() +
ggtitle("condition D")
densityLab2 <- pe[["peptideLog"]][,colData(pe)$lab=="lab2"] %>%
assay %>%
as.data.frame() %>%
gather(sample, intensity) %>%
mutate(condition = colData(pe)[sample,"condition"]) %>%
ggplot(aes(x=intensity,group=sample,color=condition)) +
geom_density() +
ggtitle("lab2")densityConditionD## Warning: Removed 39179 rows containing non-finite values (stat_density).
densityLab2## Warning: Removed 44480 rows containing non-finite values (stat_density).
- Even in very clean synthetic dataset (same background, only 48 UPS
proteins can be different) the marginal peptide intensity distribution
across samples can be quite distinct
- Considerable effects between and within labs for replicate samples
- Considerable effects between samples with different spike-in concentration
\(\rightarrow\) Normalization is needed
3.3.1 Mean or median?
- Miller and Fishkin (1997) reported that over a period of 30 years males would like to have on average 64.3 partners and females 2.8.
- Miller and Fishkin (1997) reported that the median number of partners someone would like to have over a period of 30 years males is 1 for both males and females.
Mean is very sensitive to outliers!
3.3.2 Normalization of the data by median centering
\[y_{ip}^\text{norm} = y_{ip} - \hat\mu_i\] with \(\hat\mu_i\) the median intensity over all observed peptides in sample \(i\).
Click to see R-code to normalize the data
pe <- normalize(pe,
i = "peptideLog",
name = "peptideNorm",
method = "center.median")3.3.3 Plots of normalized data
Click to see code to make plot
densityConditionDNorm <- pe[["peptideNorm"]][,colData(pe)$condition=="D"] %>%
assay %>%
as.data.frame() %>%
gather(sample, intensity) %>%
mutate(lab = colData(pe)[sample,"lab"]) %>%
ggplot(aes(x=intensity,group=sample,color=lab)) +
geom_density() +
ggtitle("condition D")
densityLab2Norm <- pe[["peptideNorm"]][,colData(pe)$lab=="lab2"] %>%
assay %>%
as.data.frame() %>%
gather(sample, intensity) %>%
mutate(condition = colData(pe)[sample,"condition"]) %>%
ggplot(aes(x=intensity,group=sample,color=condition)) +
geom_density() +
ggtitle("lab2")densityConditionDNorm## Warning: Removed 39179 rows containing non-finite values (stat_density).
densityLab2Norm## Warning: Removed 44480 rows containing non-finite values (stat_density).
- Upon normalization the marginal distributions of the peptide intensities across samples are much more comparable
- We still see deviations
- This can be due to technical variability
- In micro-array literature, quantile normalisation is used to force the median and all other quantiles to be equal across samples
- In proteomics quantile normalisation often introduces artifacts due to a difference in missing peptides across samples
- More advanced methods should be developed for normalizing proteomics data
- If there are differences in the width of the marginal distributions of the data across samples. They can also be standardized by using a robust estimator for location and scale, i.e. \[y_{ip}^\text{norm} = \frac{y_{ip} - \mu_i}{s_i}\]
3.4 Summarization
- We illustrate summarization issues using a subset of the cptac study (Lab 2, condition A and E) for a spiked protein (UPS P12081).
Click to see code to make plot
summaryPlot <- pe[["peptideNorm"]][
rowData(pe[["peptideNorm"]])$Proteins == "P12081ups|SYHC_HUMAN_UPS",
colData(pe)$lab=="lab2"&colData(pe)$condition %in% c("A","E")] %>%
assay %>%
as.data.frame %>%
rownames_to_column(var = "peptide") %>%
gather(sample, intensity, -peptide) %>%
mutate(condition = colData(pe)[sample,"condition"]) %>%
ggplot(aes(x = peptide, y = intensity, color = sample, group = sample, label = condition), show.legend = FALSE) +
geom_line(show.legend = FALSE) +
geom_text(show.legend = FALSE) +
theme_minimal() +
theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1)) +
xlab("Peptide") +
ylab("Intensity (log2)")summaryPlot## Warning: Removed 10 row(s) containing missing values (geom_path).## Warning: Removed 90 rows containing missing values (geom_text).
We observe:
- intensities from multiple peptides for each protein in a sample
- Strong peptide effect -Unbalanced peptide identification
- Pseudo-replication: peptide intensities from a particular protein in the same sample are correlated, i.e. they more alike than peptide intensities from a particular protein between samples.
\(\rightarrow\) Summarize all peptide intensities from the same protein in a sample into a single protein expression value
Commonly used methods are
Mean summarization \[ y_{ip}=\beta_i^\text{samp} + \epsilon_{ip} \]
Median summarization
Maxquant’s maxLFQ summarization (in protein groups file)
Model based summarization: \[ y_{ip}=\beta_i^\text{samp} + \beta_p^\text{pep} + \epsilon_{ip} \]
Click to see R-code to normalize the data
We use the standard sumarization in aggregateFeatures, which is robust model based summarization.
pe <- aggregateFeatures(pe,
i = "peptideNorm",
fcol = "Proteins",
na.rm = TRUE,
name = "protein")## Your quantitative and row data contain missing values. Please read the
## relevant section(s) in the aggregateFeatures manual page regarding the
## effects of missing values on data aggregation.Other summarization methods can be implemented by using the
fun argument in the aggregateFeatures
function.
fun = MsCoreUtils::medianPolish()to fits an additive model (two way decomposition) using Tukey’s median polish_ procedure using stats::medpolish()fun = MsCoreUtils::robustSummary()to calculate a robust aggregation using MASS::rlm() (default)fun = base::colMeans()to use the mean of each columnfun = matrixStats::colMedians()to use the median of each columnfun = base::colSums()to use the sum of each column
4 Exercise
- We will evaluate different summarization methods (Maxquant maxLFQ, median and robust model based) in the tutorial session before discussing on their advantages/disadvantages.
- Can you anticipate on potential problems related to the summarization?
5 Software & code
Our R/Bioconductor package msqrob2 can be used in R markdown scripts or with a GUI/shinyApp in the msqrob2gui package.
The GUI is intended as a introduction to the key concepts of proteomics data analysis for users who have no experience in R.
However, learning how to code data analyses in R markdown scripts is key for open en reproducible science and for reporting your proteomics data analyses and interpretation in a reproducible way.
More information on our tools can be found in our papers (L. J. Goeminne, Gevaert, and Clement 2016), (L. J. E. Goeminne et al. 2020) and (Sticker et al. 2020). Please refer to our work when using our tools.
5.1 Code
Data infrastructure
Import proteomics data
Preprocessing
Log-transformation
Filtering
Normalisation
Summarization
5.2 Data analysis with the GUI/shinyApp msqrob2gui
References
Goeminne, L. J. E., A. Sticker, L. Martens, K. Gevaert, and L. Clement.
2020. “MSqRob
Takes the Missing Hurdle:
Uniting Intensity- and
Count-Based
Proteomics.” Anal Chem 92 (9):
6278–87.
Goeminne, L. J., K. Gevaert, and L. Clement. 2016. “Peptide-level Robust
Ridge Regression Improves
Estimation, Sensitivity, and
Specificity in Data-dependent
Quantitative Label-free Shotgun
Proteomics.” Mol Cell Proteomics 15
(2): 657–68.
Sticker, A., L. Goeminne, L. Martens, and L. Clement. 2020. “Robust Summarization and
Inference in Proteome-wide
Label-free Quantification.”
Mol Cell Proteomics 19 (7): 1209–19.
LS0tCnRpdGxlOiAiU3RhdGlzdGljYWwgTWV0aG9kcyBmb3IgUXVhbnRpdGF0aXZlIE1TLWJhc2VkIFByb3Rlb21pY3M6IFBhcnQgSS4gUHJlcHJvY2Vzc2luZyIKYXV0aG9yOiAiTGlldmVuIENsZW1lbnQiCmRhdGU6ICJbc3RhdE9taWNzXShodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8pLCBHaGVudCBVbml2ZXJzaXR5IgpvdXRwdXQ6CiAgICBodG1sX2RvY3VtZW50OgogICAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCiAgICAgIHRoZW1lOiBmbGF0bHkKICAgICAgdG9jOiB0cnVlCiAgICAgIHRvY19mbG9hdDogdHJ1ZQogICAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgcGRmX2RvY3VtZW50OgogICAgICB0b2M6IHRydWUKICAgICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCmxpbmtjb2xvcjogYmx1ZQp1cmxjb2xvcjogYmx1ZQpjaXRlY29sb3I6IGJsdWUKCmJpYmxpb2dyYXBoeTogbXNxcm9iMi5iaWIKICAgICAgCi0tLQoKPGEgcmVsPSJsaWNlbnNlIiBocmVmPSJodHRwczovL2NyZWF0aXZlY29tbW9ucy5vcmcvbGljZW5zZXMvYnktbmMtc2EvNC4wIj48aW1nIGFsdD0iQ3JlYXRpdmUgQ29tbW9ucyBMaWNlbnNlIiBzdHlsZT0iYm9yZGVyLXdpZHRoOjAiIHNyYz0iaHR0cHM6Ly9pLmNyZWF0aXZlY29tbW9ucy5vcmcvbC9ieS1uYy1zYS80LjAvODh4MzEucG5nIiAvPjwvYT4KClRoaXMgaXMgcGFydCBvZiB0aGUgb25saW5lIGNvdXJzZSBbUHJvdGVvbWljcyBEYXRhIEFuYWx5c2lzIChQREEpXShodHRwczovL3N0YXRvbWljcy5naXRodWIuaW8vUERBLykKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvVjZJay1TT1JvS28iCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgotIFtQbGF5bGlzdCBQREEgUHJlcHJvY2Vzc2luZ10oaHR0cHM6Ly93d3cueW91dHViZS5jb20vcGxheWxpc3Q/bGlzdD1QTFpIMWhQOF9MYkpKWFFlUV9LWUROb3EtQVh5RkJHNm5YKQoKIyBPdXRsaW5lIHstfQoKMS4gSW50cm9kdWN0aW9uCgoyLiBQcmVwcm9jZXNzaW5nCgogICAgLSBMb2ctdHJhbnNmb3JtYXRpb24KICAgIC0gRmlsdGVyaW5nCiAgICAtIE5vcm1hbGl6YXRpb24KICAgIC0gU3VtbWFyaXphdGlvbgoKTm90ZSwgdGhhdCB0aGUgUi1jb2RlIGlzIGluY2x1ZGVkIGZvciBsZWFybmVycyB3aG8gYXJlIGFpbWluZyB0byBkZXZlbG9wIFIvbWFya2Rvd24gc2NyaXB0cyB0byBhdXRvbWF0ZSB0aGVpciBxdWFudGl0YXRpdmUgcHJvdGVvbWljcyBkYXRhIGFuYWx5c2VzLgpBY2NvcmRpbmcgdG8gdGhlIHRhcmdldCBhdWRpZW5jZSBvZiB0aGUgY291cnNlIHdlIGVpdGhlciB3b3JrIHdpdGggYSBncmFwaGljYWwgdXNlciBpbnRlcmZhY2UgKEdVSSkgaW4gYSBSL3NoaW55IEFwcCBtc3Fyb2IyZ3VpIChlLmcuIFByb3Rlb21pY3MgQmlvaW5mb3JtYXRpY3MgY291cnNlIG9mIHRoZSBFQkkgYW5kIHRoZSBQcm90ZW9taWNzIERhdGEgQW5hbHlzaXMgY291cnNlIGF0IHRoZSBHdWxiZW5raWFuIGluc3RpdHV0ZSkgb3Igd2l0aCBSL21hcmtkb3ducyBzY3JpcHRzIChlLmcuIEJpb2luZm9ybWF0aWNzIFN1bW1lciBTY2hvb2wgYXQgVUNMb3V2YWluIG9yIHRoZSBTdGF0aXN0aWNhbCBHZW5vbWljcyBDb3Vyc2UgYXQgR2hlbnQgVW5pdmVyc2l0eSkuIAoKLS0tCgojIEludHJvOiBDaGFsbGVuZ2VzIGluIExhYmVsLUZyZWUgUXVhbnRpdGF0aXZlIFByb3Rlb21pY3MKCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkLzN3X0s0U19HVXpFIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKCiMjIE1TLWJhc2VkIHdvcmtmbG93CgpgYGB7ciBlY2hvPUZBTFNFfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL1Byb3Rlb21pY3NXb3JrZmxvdy5wbmciKQpgYGAKICAKLSBQZXB0aWRlIENoYXJhY3RlcmlzdGljcwogIAogIC0gTW9kaWZpY2F0aW9ucwogIC0gSW9uaXNhdGlvbiBFZmZpY2llbmN5OiBodWdlIHZhcmlhYmlsaXR5CiAgLSBJZGVudGlmaWNhdGlvbgogICAgLSBNaXNpZGVudGlmaWNhdGlvbiAkXHJpZ2h0YXJyb3ckIG91dGxpZXJzCiAgICAtIE1TJF4yJCBzZWxlY3Rpb24gb24gcGVwdGlkZSBhYnVuZGFuY2UKICAgIC0gQ29udGV4dCBkZXBlbmRpbmcgbWlzc2luZ25lc3MKICAgIC0gTm9uLXJhbmRvbSBtaXNzaW5nbmVzcwoKJFxyaWdodGFycm93JCBVbmJhbGFuY2VkIHBlcGlkZSBpZGVudGlmaWNhdGlvbnMgYWNyb3NzIHNhbXBsZXMgYW5kIG1lc3N5IGRhdGEKCi0tLQoKIyMgTGV2ZWwgb2YgcXVhbnRpZmljYXRpb24KCi0gTVMtYmFzZWQgcHJvdGVvbWljcyByZXR1cm5zIHBlcHRpZGVzOiBwaWVjZXMgb2YgcHJvdGVpbnMKCmBgYHtyIGVjaG89RkFMU0V9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvY2hhbGxlbmdlc19wZXB0aWRlcy5wbmciKQpgYGAKCi0gUXVhbnRpZmljYXRpb24gY29tbW9ubHkgcmVxdWlyZWQgb24gdGhlIHByb3RlaW4gbGV2ZWwKCmBgYHtyIGVjaG89RkFMU0V9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvY2hhbGxlbmdlc19wcm90ZWlucy5wbmciKQpgYGAKCi0tLQoKIyMgTGFiZWwtZnJlZSBRdWFudGl0YXRpdmUgUHJvdGVvbWljcyBEYXRhIEFuYWx5c2lzIFdvcmtmbG93cwoKYGBge3IgZWNobz1GQUxTRX0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoIi4vZmlndXJlcy9wcm90ZW9taWNzRGF0YUFuYWx5c2lzLnBuZyIpCmBgYAoKLS0tCgojIyBDUFRBQyBTcGlrZS1pbiBTdHVkeQoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC82VjVHaUE0Q2cyVSIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4KCmBgYHtyIGVjaG89RkFMU0UsIG91dC53aWR0aD0iNTAlIn0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoIi4vZmlndXJlcy9jcHRhY0xheW91dEx1ZGdlci5wbmciKQpgYGAKCi0gU2FtZSB0cnlwc2luLWRpZ2VzdGVkIHllYXN0IHByb3Rlb21lIGJhY2tncm91bmQgaW4gZWFjaCBzYW1wbGUKLSBUcnlwc2luLWRpZ2VzdGVkIFNpZ21hIFVQUzEgc3RhbmRhcmQ6IDQ4IGRpZmZlcmVudCBodW1hbiBwcm90ZWlucyBzcGlrZWQgaW4gYXQgNSBkaWZmZXJlbnQgY29uY2VudHJhdGlvbnMgKHRyZWF0bWVudCBBLUUpIAotIFNhbXBsZXMgcmVwZWF0ZWRseSBydW4gb24gZGlmZmVyZW50IGluc3RydW1lbnRzIGluIGRpZmZlcmVudCBsYWJzCi0gQWZ0ZXIgTWF4UXVhbnQgc2VhcmNoIHdpdGggbWF0Y2ggYmV0d2VlbiBydW5zIG9wdGlvbgoKICAtIDQxXCUgb2YgYWxsIHByb3RlaW5zIGFyZSBxdWFudGlmaWVkIGluIGFsbCBzYW1wbGVzCiAgLSA2LjZcJSBvZiBhbGwgcGVwdGlkZXMgYXJlIHF1YW50aWZpZWQgaW4gYWxsIHNhbXBsZXMKCiRccmlnaHRhcnJvdyQgdmFzdCBhbW91bnQgb2YgbWlzc2luZ25lc3MKCgojIyBNYXhxdWFudCBvdXRwdXQKCmBgYHtyIGVjaG89RkFMU0V9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCIuL2ZpZ3VyZXMvbWF4cXVhbnRPdXRwdXREaXIucG5nIikKYGBgCgotLS0KCgojIEltcG9ydCB0aGUgZGF0YSBpbiBSIAoKIyMgRGF0YSBpbmZyYXN0cnVjdHVyZQoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBiYWNrZ3JvdW5kIG9uIGRhdGEgaW5mcmFzdHJ1Y3R1cmUgdXNlZCBpbiBSIHRvIHN0b3JlIHByb3Rlb21pY3MgZGF0YSA8L3N1bW1hcnk+PHA+Ci0gV2UgdXNlIHRoZSBgUUZlYXR1cmVzYCBwYWNrYWdlIHRoYXQgcHJvdmlkZXMgdGhlIGluZnJhc3RydWN0dXJlIHRvCiAgLSBzdG9yZSwgIAogIC0gcHJvY2VzcywgCiAgLSBtYW5pcHVsYXRlIGFuZCAKICAtIGFuYWx5c2UgcXVhbnRpdGF0aXZlIGRhdGEvZmVhdHVyZXMgZnJvbSBtYXNzIHNwZWN0cm9tZXRyeQpleHBlcmltZW50cy4gCgotIEl0IGlzIGJhc2VkIG9uIHRoZSBgU3VtbWFyaXplZEV4cGVyaW1lbnRgIGFuZApgTXVsdGlBc3NheUV4cGVyaW1lbnRgIGNsYXNzZXMuIAoKCgpgYGB7ciBmaWcuY2FwID0gIkNvbmNlcHR1YWwgcmVwcmVzZW50YXRpb24gb2YgYSBgU3VtbWFyaXplZEV4cGVyaW1lbnRgIG9iamVjdC4gIEFzc2F5cyBjb250YWluIGluZm9ybWF0aW9uIG9uIHRoZSBtZWFzdXJlZCBvbWljcyBmZWF0dXJlcyAocm93cykgZm9yIGRpZmZlcmVudCBzYW1wbGVzIChjb2x1bW5zKS4gVGhlIGByb3dEYXRhYCBjb250YWlucyBpbmZvcm1hdGlvbiBvbiB0aGUgb21pY3MgZmVhdHVyZXMsIHRoZSBgY29sRGF0YWAgY29udGFpbnMgaW5mb3JtYXRpb24gb24gdGhlIHNhbXBsZXMsIGkuZS4gZXhwZXJpbWVudGFsIGRlc2lnbiBldGMuIiwgZWNobz1GQUxTRSwgb3V0LndpZHRoPSI4MCUifQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygiLi9maWd1cmVzL1NFLnBuZyIpCmBgYAoKLSBBc3NheXMgaW4gYSBRRmVhdHVyZXMgb2JqZWN0IGhhdmUgYQpoaWVyYXJjaGljYWwgcmVsYXRpb246IAogIAogIC0gcHJvdGVpbnMgYXJlIGNvbXBvc2VkIG9mIHBlcHRpZGVzLCAKICAtIHRoZW1zZWx2ZXMgcHJvZHVjZWQgYnkgc3BlY3RyYQogIC0gcmVsYXRpb25zIGJldHdlZW4gYXNzYXlzIGFyZSB0cmFja2VkIGFuZCByZWNvcmRlZCB0aHJvdWdob3V0IGRhdGEgcHJvY2Vzc2luZwoKYGBge3IgZmVhdHVyZXNwbG90LCBmaWcuY2FwID0gIkNvbmNlcHR1YWwgcmVwcmVzZW50YXRpb24gb2YgYSBgUUZlYXR1cmVzYCBvYmplY3QgYW5kIHRoZSBhZ2dyZWdhdGl2ZSByZWxhdGlvbiBiZXR3ZWVuIGRpZmZlcmVudCBhc3NheXMuIiwgZWNobyA9IEZBTFNFfQpwYXIobWFyID0gYygwLCAwLCAwLCAwKSkKcGxvdChOQSwgeGxpbSA9IGMoMCwgMTIpLCB5bGltID0gYygwLCAyMCksCiAgICAgeGF4dCA9ICJuIiwgeWF4dCA9ICJuIiwKICAgICB4bGFiID0gIiIsIHlsYWIgPSAiIiwgYnR5ID0gIm4iKQpmb3IgKGkgaW4gMDo3KQogICAgcmVjdCgwLCBpLCAzLCBpKzEsIGNvbCA9ICJsaWdodGdyZXkiLCBib3JkZXIgPSAid2hpdGUiKQpmb3IgKGkgaW4gODoxMikKICAgIHJlY3QoMCwgaSwgMywgaSsxLCBjb2wgPSAic3RlZWxibHVlIiwgYm9yZGVyID0gIndoaXRlIikKZm9yIChpIGluIDEzOjE4KQogICAgcmVjdCgwLCBpLCAzLCBpKzEsIGNvbCA9ICJvcmFuZ2UiLCBib3JkZXIgPSAid2hpdGUiKQpmb3IgKGkgaW4gMTkpCiAgICByZWN0KDAsIGksIDMsIGkrMSwgY29sID0gImRhcmtncmV5IiwgYm9yZGVyID0gIndoaXRlIikKZm9yIChpIGluIDU6NykKICAgIHJlY3QoNSwgaSwgOCwgaSsxLCBjb2wgPSAibGlnaHRncmV5IiwgYm9yZGVyID0gIndoaXRlIikKZm9yIChpIGluIDg6MTApCiAgICByZWN0KDUsIGksIDgsIGkrMSwgY29sID0gInN0ZWVsYmx1ZSIsIGJvcmRlciA9ICJ3aGl0ZSIpCmZvciAoaSBpbiAxMToxMykKICAgIHJlY3QoNSwgaSwgOCwgaSsxLCBjb2wgPSAib3JhbmdlIiwgYm9yZGVyID0gIndoaXRlIikKZm9yIChpIGluIDE0KQogICAgcmVjdCg1LCBpLCA4LCBpKzEsIGNvbCA9ICJkYXJrZ3JleSIsIGJvcmRlciA9ICJ3aGl0ZSIpCnJlY3QoOSwgOCwgMTIsIDgrMSwgY29sID0gImxpZ2h0Z3JleSIsIGJvcmRlciA9ICJ3aGl0ZSIpCnJlY3QoOSwgOSwgMTIsIDkrMSwgY29sID0gInN0ZWVsYmx1ZSIsIGJvcmRlciA9ICJ3aGl0ZSIpCnJlY3QoOSwgMTAsIDEyLCAxMCsxLCBjb2wgPSAib3JhbmdlIiwgYm9yZGVyID0gIndoaXRlIikKcmVjdCg5LCAxMSwgMTIsIDExKzEsIGNvbCA9ICJkYXJrZ3JleSIsIGJvcmRlciA9ICJ3aGl0ZSIpCnNlZ21lbnRzKDMsIDgsIDUsIDgsIGx0eSA9ICJkYXNoZWQiKQpzZWdtZW50cygzLCA2LCA1LCA3LCBsdHkgPSAiZGFzaGVkIikKc2VnbWVudHMoMywgNCwgNSwgNiwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDAsIDUsIDUsIGx0eSA9ICJkYXNoZWQiKQpzZWdtZW50cygzLCAxMCwgNSwgOSwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDExLCA1LCAxMCwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDEzLCA1LCAxMSwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDE0LCA1LCAxMiwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDE2LCA1LCAxMywgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDE5LCA1LCAxNCwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDMsIDIwLCA1LCAxNSwgbHR5ID0gImRhc2hlZCIpCnNlZ21lbnRzKDgsIDUsIDksIDgsIGx0eSA9ICJkYXNoZWQiKQpzZWdtZW50cyg4LCA4LCA5LCA5LCBsdHkgPSAiZGFzaGVkIikKc2VnbWVudHMoOCwgMTEsIDksIDEwLCBsdHkgPSAiZGFzaGVkIikKc2VnbWVudHMoOCwgMTQsIDksIDExLCBsdHkgPSAiZGFzaGVkIikKc2VnbWVudHMoOCwgMTUsIDksIDEyLCBsdHkgPSAiZGFzaGVkIikKYGBgCgo8L3A+PC9kZXRhaWxzPgoKIyMgSW1wb3J0IGRhdGEgaW4gUgoKCiMjIyBMb2FkIGxpYnJhcmllcyAKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSA8L3N1bW1hcnk+PHA+CmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShsaW1tYSkKbGlicmFyeShRRmVhdHVyZXMpCmxpYnJhcnkobXNxcm9iMikKbGlicmFyeShwbG90bHkpCmxpYnJhcnkoZ2dwbG90MikKYGBgCjwvcD48L2RldGFpbHM+CgojIyMgUmVhZCBkYXRhIAoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBiYWNrZ3JvdW5kIGFuZCBjb2RlIDwvc3VtbWFyeT48cD4KMS4gV2UgdXNlIGEgcGVwdGlkZXMudHh0IGZpbGUgZnJvbSBNUy1kYXRhIHF1YW50aWZpZWQgd2l0aCBtYXhxdWFudCB0aGF0IApjb250YWlucyBNUzEgaW50ZW5zaXRpZXMgc3VtbWFyaXplZCBhdCB0aGUgcGVwdGlkZSBsZXZlbC4gCmBgYHtyfQpwZXB0aWRlc0ZpbGUgPC0gImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9zdGF0T21pY3MvUERBL2RhdGEvcXVhbnRpZmljYXRpb24vZnVsbENwdGFjRGF0YXNTZXROb3RGb3JUdXRvcmlhbC9wZXB0aWRlcy50eHQiCmBgYAoKMi4gTWF4cXVhbnQgc3RvcmVzIHRoZSBpbnRlbnNpdHkgZGF0YSBmb3IgdGhlIGRpZmZlcmVudCBzYW1wbGVzIGluIGNvbHVtbm5zIHRoYXQgc3RhcnQgd2l0aCBJbnRlbnNpdHkuIFdlIGNhbiByZXRyZWl2ZSB0aGUgY29sdW1uIG5hbWVzIHdpdGggdGhlIGludGVuc2l0eSBkYXRhIHdpdGggdGhlIGNvZGUgYmVsb3c6IAoKYGBge3J9CmVjb2xzIDwtIGdyZXAoIkludGVuc2l0eVxcLiIsIG5hbWVzKHJlYWQuZGVsaW0ocGVwdGlkZXNGaWxlKSkpCmBgYAoKMy4gUmVhZCB0aGUgZGF0YSBhbmQgc3RvcmUgaXQgaW4gIFFGZWF0dXJlcyBvYmplY3QgCgpgYGB7cn0KcGUgPC0gcmVhZFFGZWF0dXJlcygKICB0YWJsZSA9IHBlcHRpZGVzRmlsZSwKICBmbmFtZXMgPSAxLAogIGVjb2wgPSBlY29scywKICBuYW1lID0gInBlcHRpZGVSYXciLCBzZXA9Ilx0IikKYGBgCjwvcD48L2RldGFpbHM+CgojIyMgRXhwbG9yZSBvYmplY3QKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgYmFja2dyb3VuZCBhbmQgY29kZSA8L3N1bW1hcnk+PHA+Ci0gVGhlIHJvd0RhdGEgY29udGFpbnMgaW5mb3JtYXRpb24gb24gdGhlIGZlYXR1cmVzIChwZXB0aWRlcykgaW4gdGhlIGFzc2F5LiBFLmcuIFNlcXVlbmNlLCBwcm90ZWluLCAuLi4KCmBgYHtyfQpyb3dEYXRhKHBlW1sicGVwdGlkZVJhdyJdXSkKYGBgCgotIFRoZSBjb2xEYXRhIGNvbnRhaW5zIGluZm9ybWF0aW9uIG9uIHRoZSBzYW1wbGVzCgpgYGB7cn0gCmNvbERhdGEocGUpCmBgYAoKLSBObyBpbmZvcm1hdGlvbiBpcyBzdG9yZWQgeWV0IG9uIHRoZSBkZXNpZ24uIAoKCmBgYHtyfSAKcGUgJT4lIGNvbG5hbWVzCmBgYAoKLSBOb3RlLCB0aGF0IHRoZSBzYW1wbGUgbmFtZXMgaW5jbHVkZSB0aGUgc3Bpa2UtaW4gY29uZGl0aW9uLiAKLSBUaGV5IGFsc28gZW5kIG9uIGEgbnVtYmVyLiAKICAKICAtIDEtMyBpcyBmcm9tIGxhYiAxLCAKICAtIDQtNiBmcm9tIGxhYiAyIGFuZCAKICAtIDctOSBmcm9tIGxhYiAzLiAKCi0gV2UgdXBkYXRlIHRoZSBjb2xEYXRhIHdpdGggaW5mb3JtYXRpb24gb24gdGhlIGRlc2lnbgoKYGBge3J9CmNvbERhdGEocGUpJGxhYiA8LSByZXAocmVwKHBhc3RlMCgibGFiIiwxOjMpLGVhY2g9MyksNSkgJT4lIGFzLmZhY3Rvcgpjb2xEYXRhKHBlKSRjb25kaXRpb24gPC0gcGVbWyJwZXB0aWRlUmF3Il1dICU+JSBjb2xuYW1lcyAlPiUgc3Vic3RyKDEyLDEyKSAlPiUgYXMuZmFjdG9yCmNvbERhdGEocGUpJHNwaWtlQ29uY2VudHJhdGlvbiA8LSByZXAoYyhBID0gMC4yNSwgQiA9IDAuNzQsIEMgPSAyLjIyLCBEID0gNi42NywgRSA9IDIwKSxlYWNoID0gOSkKYGBgCgotIFdlIGV4cGxvcmUgdGhlIGNvbERhdGEgYWdhaW4KCmBgYHtyfQpjb2xEYXRhKHBlKQpgYGAKCjwvcD48L2RldGFpbHM+CgotLS0KCiMgUHJlcHJvY2Vzc2luZwoKIyMgTG9nLXRyYW5zZm9ybWF0aW9uCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL0FQc2RTYnUxUkJNIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKIyMjIEV4cGxvcmUgdGhlIGRhdGEgd2l0aCBwbG90cwoKUGVwdGlkZSBBQUxFRUxWSyBmcm9tIHNwaWtlZC1pbiBVUFMgcHJvdGVpbiBQMTIwODEuIApXZSBvbmx5IHNob3cgZGF0YSBmcm9tIGxhYjEuCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3J9CnN1YnNldCA8LSBwZVsiQUFMRUVMVksiLGNvbERhdGEocGUpJGxhYj09ImxhYjEiXQpwbG90V2h5TG9nIDwtIGRhdGEuZnJhbWUoY29uY2VudHJhdGlvbiA9IGNvbERhdGEoc3Vic2V0KSRzcGlrZUNvbmNlbnRyYXRpb24sCiAgICAgICAgICAgeSA9IGFzc2F5KHN1YnNldFtbInBlcHRpZGVSYXciXV0pICU+JSBjCiAgICAgICAgICAgKSAlPiUgCiAgZ2dwbG90KGFlcyhjb25jZW50cmF0aW9uLCB5KSkgKwogIGdlb21fcG9pbnQoKSArCiAgeGxhYigiY29uY2VudHJhdGlvbiAoZm1vbC9sKSIpICsKICBnZ3RpdGxlKCJwZXB0aWRlIEFBTEVFTFZLIGluIGxhYjEiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpwbG90V2h5TG9nCmBgYAoKLSBWYXJpYW5jZSBpbmNyZWFzZXMgd2l0aCB0aGUgbWVhbgokXHJpZ2h0YXJyb3ckIE11bHRpcGxpY2F0aXZlIGVycm9yIHN0cnVjdHVyZSAKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBtYWtlIHBsb3QgPC9zdW1tYXJ5PjxwPgpgYGB7cn0KcGxvdExvZyA8LSBkYXRhLmZyYW1lKGNvbmNlbnRyYXRpb24gPSBjb2xEYXRhKHN1YnNldCkkc3Bpa2VDb25jZW50cmF0aW9uLAogICAgICAgICAgIHkgPSBhc3NheShzdWJzZXRbWyJwZXB0aWRlUmF3Il1dKSAlPiUgYwogICAgICAgICAgICkgJT4lIAogIGdncGxvdChhZXMoY29uY2VudHJhdGlvbiwgeSkpICsKICBnZW9tX3BvaW50KCkgKyAKICBzY2FsZV94X2NvbnRpbnVvdXModHJhbnM9J2xvZzInKSArIAogIHNjYWxlX3lfY29udGludW91cyh0cmFucz0nbG9nMicpICsKICB4bGFiKCJjb25jZW50cmF0aW9uIChmbW9sL2wpIikgKwogIGdndGl0bGUoInBlcHRpZGUgQUFMRUVMVksgaW4gbGFiMSB3aXRoIGF4ZXMgb24gbG9nIHNjYWxlIikKYGBgCjwvcD48L2RldGFpbHM+CgpgYGB7cn0KcGxvdExvZwpgYGAKCi0gRGF0YSBzZWVtcyB0byBiZSBob21vc2NlZGFzdGljIG9uIGxvZy1zY2FsZSAkXHJpZ2h0YXJyb3ckIGxvZyB0cmFuc2Zvcm1hdGlvbiBvZiB0aGUgaW50ZW5zaXR5IGRhdGEKLSBJbiBxdWFudGl0YXRpdmUgcHJvdGVvbWljcyBhbmFseXNpcyBvbiAkXGxvZ18yJCAKCiRccmlnaHRhcnJvdyQgRGlmZmVyZW5jZXMgb24gYSAkXGxvZ18yJCBzY2FsZTogJFxsb2dfMiQgZm9sZCBjaGFuZ2VzCgokJApcbG9nXzIgQiAtIFxsb2dfMiBBID0gXGxvZ18yIFxmcmFje0J9e0F9ID0gXGxvZyBGQ19cdGV4dHtCIC0gQX0KJCQKJCQgClxiZWdpbnthcnJheX0ge2x9CmxvZ18yIEZDID0gMSBccmlnaHRhcnJvdyBGQyA9IDJeMSA9MlxcCmxvZ18yIEZDID0gMiBccmlnaHRhcnJvdyBGQyA9IDJeMiA9IDRcXApcZW5ke2FycmF5fQokJAoKCiMjIyBsb2ctdHJhbnNmb3JtYXRpb24gb2YgdGhlIGRhdGEgCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbG9nLXRyYW5zZnJvbSB0aGUgZGF0YSA8L3N1bW1hcnk+PHA+Ci0gV2UgY2FsY3VsYXRlIGhvdyBtYW55IG5vbiB6ZXJvIGludGVuc2l0aWVzIHdlIGhhdmUgZm9yIGVhY2ggcGVwdGlkZSBhbmQgdGhpcyBjYW4gYmUgdXNlZnVsIGZvciBmaWx0ZXJpbmcuCgpgYGB7cn0Kcm93RGF0YShwZVtbInBlcHRpZGVSYXciXV0pJG5Ob25aZXJvIDwtIHJvd1N1bXMoYXNzYXkocGVbWyJwZXB0aWRlUmF3Il1dKSA+IDApCmBgYAoKCi0gUGVwdGlkZXMgd2l0aCB6ZXJvIGludGVuc2l0aWVzIGFyZSBtaXNzaW5nIHBlcHRpZGVzIGFuZCBzaG91bGQgYmUgcmVwcmVzZW50CndpdGggYSBgTkFgIHZhbHVlIHJhdGhlciB0aGFuIGAwYC4KCmBgYHtyfQpwZSA8LSB6ZXJvSXNOQShwZSwgInBlcHRpZGVSYXciKSAjIGNvbnZlcnQgMCB0byBOQQpgYGAKCi0gTG9ndHJhbnNmb3JtIGRhdGEgd2l0aCBiYXNlIDIKCmBgYHtyfQpwZSA8LSBsb2dUcmFuc2Zvcm0ocGUsIGJhc2UgPSAyLCBpID0gInBlcHRpZGVSYXciLCBuYW1lID0gInBlcHRpZGVMb2ciKQpgYGAKPC9wPjwvZGV0YWlscz4KCi0tLQoKIyMgRmlsdGVyaW5nCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkLzN6VkZxZFRsQl9jIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKLSBSZXZlcnNlIHNlcXVlbmNlcwotIE9ubHkgaWRlbnRpZmllZCBieSBtb2RpZmljYXRpb24gc2l0ZSAob25seSBtb2RpZmllZCBwZXB0aWRlcyBkZXRlY3RlZCkKLSBSYXpvciBwZXB0aWRlczogbm9uLXVuaXF1ZSBwZXB0aWRlcyBhc3NpZ25lZCB0byB0aGUgcHJvdGVpbiBncm91cCB3aXRoIHRoZSBtb3N0IG90aGVyIHBlcHRpZGVzIAotIENvbnRhbWluYW50cwotIFBlcHRpZGVzIGZldyBpZGVudGlmaWNhdGlvbnMKLSBQcm90ZWlucyB0aGF0IGFyZSBvbmx5IGlkZW50aWZpZWQgd2l0aCBvbmUgb3IgYSBmZXcgcGVwdGlkZXMKCkZpbHRlcmluZyBkb2VzIG5vdCBpbmR1Y2UgYmlhcyBpZiB0aGUgY3JpdGVyaW9uIGlzIGluZGVwZW5kZW50IGZyb20gdGhlIGRvd25zdHJlYW0gZGF0YSBhbmFseXNpcyEKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBmaWx0ZXIgdGhlIGRhdGEgPC9zdW1tYXJ5PjxwPgoKMS4gSGFuZGxpbmcgb3ZlcmxhcHBpbmcgcHJvdGVpbiBncm91cHMKCkluIG91ciBhcHByb2FjaCBhIHBlcHRpZGUgY2FuIG1hcCB0byBtdWx0aXBsZSBwcm90ZWlucywgYXMgbG9uZyBhcyB0aGVyZSBpcwpub25lIG9mIHRoZXNlIHByb3RlaW5zIHByZXNlbnQgaW4gYSBzbWFsbGVyIHN1Ymdyb3VwLgoKYGBge3J9CnBlIDwtIGZpbHRlckZlYXR1cmVzKHBlLCB+IFByb3RlaW5zICVpbiUgc21hbGxlc3RVbmlxdWVHcm91cHMocm93RGF0YShwZVtbInBlcHRpZGVMb2ciXV0pJFByb3RlaW5zKSkKYGBgCgoyLiBSZW1vdmUgcmV2ZXJzZSBzZXF1ZW5jZXMgKGRlY295cykgYW5kIGNvbnRhbWluYW50cwoKV2Ugbm93IHJlbW92ZSB0aGUgY29udGFtaW5hbnRzLCBwZXB0aWRlcyB0aGF0IG1hcCB0byBkZWNveSBzZXF1ZW5jZXMsIGFuZCBwcm90ZWlucwp3aGljaCB3ZXJlIG9ubHkgaWRlbnRpZmllZCBieSBwZXB0aWRlcyB3aXRoIG1vZGlmaWNhdGlvbnMuCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsflJldmVyc2UgIT0gIisiKQpwZSA8LSBmaWx0ZXJGZWF0dXJlcyhwZSx+IFBvdGVudGlhbC5jb250YW1pbmFudCAhPSAiKyIpCmBgYAoKMy4gRHJvcCBwZXB0aWRlcyB0aGF0IHdlcmUgb25seSBpZGVudGlmaWVkIGluIG9uZSBzYW1wbGUKCldlIGtlZXAgcGVwdGlkZXMgdGhhdCB3ZXJlIG9ic2VydmVkIGF0IGxhc3QgdHdpY2UuCgpgYGB7cn0KcGUgPC0gZmlsdGVyRmVhdHVyZXMocGUsfiBuTm9uWmVybyA+PTIpCm5yb3cocGVbWyJwZXB0aWRlTG9nIl1dKQpgYGAKCldlIGtlZXAgYHIgbnJvdyhwZVtbInBlcHRpZGVMb2ciXV0pYCBwZXB0aWRlcyB1cG9uIGZpbHRlcmluZy4KPC9wPjwvZGV0YWlscz4KCi0tLQoKIyMgTm9ybWFsaXphdGlvbiAKCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvOXFDSkVrZFNTaEkiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+Cgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KCmBgYHtyfQpkZW5zaXR5Q29uZGl0aW9uRCA8LSBwZVtbInBlcHRpZGVMb2ciXV1bLGNvbERhdGEocGUpJGNvbmRpdGlvbj09IkQiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShsYWIgPSBjb2xEYXRhKHBlKVtzYW1wbGUsImxhYiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1sYWIpKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgiY29uZGl0aW9uIEQiKQoKZGVuc2l0eUxhYjIgPC0gcGVbWyJwZXB0aWRlTG9nIl1dWyxjb2xEYXRhKHBlKSRsYWI9PSJsYWIyIl0gJT4lIAogIGFzc2F5ICU+JQogIGFzLmRhdGEuZnJhbWUoKSAlPiUKICBnYXRoZXIoc2FtcGxlLCBpbnRlbnNpdHkpICU+JSAKICBtdXRhdGUoY29uZGl0aW9uID0gY29sRGF0YShwZSlbc2FtcGxlLCJjb25kaXRpb24iXSkgJT4lCiAgZ2dwbG90KGFlcyh4PWludGVuc2l0eSxncm91cD1zYW1wbGUsY29sb3I9Y29uZGl0aW9uKSkgKyAKICAgIGdlb21fZGVuc2l0eSgpICsKICAgIGdndGl0bGUoImxhYjIiKQpgYGAKPC9wPjwvZGV0YWlscz4KCmBgYHtyfQpkZW5zaXR5Q29uZGl0aW9uRApgYGAKCmBgYHtyfQpkZW5zaXR5TGFiMgpgYGAKLSBFdmVuIGluIHZlcnkgY2xlYW4gc3ludGhldGljIGRhdGFzZXQgKHNhbWUgYmFja2dyb3VuZCwgb25seSA0OCBVUFMKcHJvdGVpbnMgY2FuIGJlIGRpZmZlcmVudCkgdGhlIG1hcmdpbmFsIHBlcHRpZGUgaW50ZW5zaXR5IGRpc3RyaWJ1dGlvbgphY3Jvc3Mgc2FtcGxlcyBjYW4gYmUgcXVpdGUgZGlzdGluY3QKCi0gQ29uc2lkZXJhYmxlIGVmZmVjdHMgYmV0d2VlbiBhbmQgd2l0aGluIGxhYnMgZm9yIHJlcGxpY2F0ZSBzYW1wbGVzCi0gQ29uc2lkZXJhYmxlIGVmZmVjdHMgYmV0d2VlbiBzYW1wbGVzIHdpdGggZGlmZmVyZW50IHNwaWtlLWluCmNvbmNlbnRyYXRpb24KCiRccmlnaHRhcnJvdyQgTm9ybWFsaXphdGlvbiBpcyBuZWVkZWQKCi0tLQoKCiMjIyBNZWFuIG9yIG1lZGlhbj8KCi0gTWlsbGVyIGFuZCBGaXNoa2luICgxOTk3KSByZXBvcnRlZCB0aGF0IG92ZXIgYSBwZXJpb2Qgb2YgMzAgeWVhcnMgbWFsZXMgd291bGQgbGlrZSB0byBoYXZlIG9uIGF2ZXJhZ2UgNjQuMyBwYXJ0bmVycyBhbmQgZmVtYWxlcyAyLjguIAoKCjxkZXRhaWxzPjxzdW1tYXJ5PiA8L3N1bW1hcnk+PHA+Ci0gTWlsbGVyIGFuZCBGaXNoa2luICgxOTk3KSByZXBvcnRlZCB0aGF0IHRoZSBtZWRpYW4gbnVtYmVyIG9mIHBhcnRuZXJzIHNvbWVvbmUgd291bGQgbGlrZSB0byBoYXZlIG92ZXIgYSBwZXJpb2Qgb2YgMzAgeWVhcnMgbWFsZXMgaXMgMSBmb3IgYm90aCBtYWxlcyBhbmQgZmVtYWxlcy4gCjwvcD48L2RldGFpbHM+Cgo8ZGV0YWlscz48c3VtbWFyeT4gPC9zdW1tYXJ5PjxwPgpNZWFuIGlzIHZlcnkgc2Vuc2l0aXZlIHRvIG91dGxpZXJzIQoKYGBge3IgZWNobz1GQUxTRX0Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoIi4vZmlndXJlcy9wYXJ0bmVycy5wbmciKQpgYGAKPC9wPjwvZGV0YWlscz4KCgotLS0KCiMjIyBOb3JtYWxpemF0aW9uIG9mIHRoZSBkYXRhIGJ5IG1lZGlhbiBjZW50ZXJpbmcKCiQkeV97aXB9Xlx0ZXh0e25vcm19ID0geV97aXB9IC0gXGhhdFxtdV9pJCQgCndpdGggJFxoYXRcbXVfaSQgdGhlIG1lZGlhbiBpbnRlbnNpdHkgb3ZlciBhbGwgb2JzZXJ2ZWQgcGVwdGlkZXMgaW4gc2FtcGxlICRpJC4KCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgUi1jb2RlIHRvIG5vcm1hbGl6ZSB0aGUgZGF0YSA8L3N1bW1hcnk+PHA+CmBgYHtyfQpwZSA8LSBub3JtYWxpemUocGUsIAogICAgICAgICAgICAgICAgaSA9ICJwZXB0aWRlTG9nIiwgCiAgICAgICAgICAgICAgICBuYW1lID0gInBlcHRpZGVOb3JtIiwgCiAgICAgICAgICAgICAgICBtZXRob2QgPSAiY2VudGVyLm1lZGlhbiIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKIyMjIFBsb3RzIG9mIG5vcm1hbGl6ZWQgZGF0YQoKCjxkZXRhaWxzPjxzdW1tYXJ5PiBDbGljayB0byBzZWUgY29kZSB0byBtYWtlIHBsb3QgPC9zdW1tYXJ5PjxwPgpgYGB7cn0KZGVuc2l0eUNvbmRpdGlvbkROb3JtIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bLGNvbERhdGEocGUpJGNvbmRpdGlvbj09IkQiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShsYWIgPSBjb2xEYXRhKHBlKVtzYW1wbGUsImxhYiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1sYWIpKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgiY29uZGl0aW9uIEQiKQoKZGVuc2l0eUxhYjJOb3JtIDwtIHBlW1sicGVwdGlkZU5vcm0iXV1bLGNvbERhdGEocGUpJGxhYj09ImxhYjIiXSAlPiUgCiAgYXNzYXkgJT4lCiAgYXMuZGF0YS5mcmFtZSgpICU+JQogIGdhdGhlcihzYW1wbGUsIGludGVuc2l0eSkgJT4lIAogIG11dGF0ZShjb25kaXRpb24gPSBjb2xEYXRhKHBlKVtzYW1wbGUsImNvbmRpdGlvbiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHg9aW50ZW5zaXR5LGdyb3VwPXNhbXBsZSxjb2xvcj1jb25kaXRpb24pKSArIAogICAgZ2VvbV9kZW5zaXR5KCkgKwogICAgZ2d0aXRsZSgibGFiMiIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CmRlbnNpdHlDb25kaXRpb25ETm9ybQpgYGAKCmBgYHtyfQpkZW5zaXR5TGFiMk5vcm0KYGBgCgotIFVwb24gbm9ybWFsaXphdGlvbiB0aGUgbWFyZ2luYWwgZGlzdHJpYnV0aW9ucyBvZiB0aGUgcGVwdGlkZSBpbnRlbnNpdGllcyBhY3Jvc3Mgc2FtcGxlcyBhcmUgbXVjaCBtb3JlIGNvbXBhcmFibGUKLSBXZSBzdGlsbCBzZWUgZGV2aWF0aW9ucwotIFRoaXMgY2FuIGJlIGR1ZSB0byB0ZWNobmljYWwgdmFyaWFiaWxpdHkKLSBJbiBtaWNyby1hcnJheSBsaXRlcmF0dXJlLCBxdWFudGlsZSBub3JtYWxpc2F0aW9uIGlzIHVzZWQgdG8gZm9yY2UgdGhlIG1lZGlhbiBhbmQgYWxsIG90aGVyIHF1YW50aWxlcyB0byBiZSBlcXVhbCBhY3Jvc3Mgc2FtcGxlcwotIEluIHByb3Rlb21pY3MgcXVhbnRpbGUgbm9ybWFsaXNhdGlvbiBvZnRlbiBpbnRyb2R1Y2VzIGFydGlmYWN0cyBkdWUgdG8gYSBkaWZmZXJlbmNlIGluIG1pc3NpbmcgcGVwdGlkZXMgYWNyb3NzIHNhbXBsZXMgCi0gTW9yZSBhZHZhbmNlZCBtZXRob2RzIHNob3VsZCBiZSBkZXZlbG9wZWQgZm9yIG5vcm1hbGl6aW5nIHByb3Rlb21pY3MgZGF0YQotIElmIHRoZXJlIGFyZSBkaWZmZXJlbmNlcyBpbiB0aGUgd2lkdGggb2YgdGhlIG1hcmdpbmFsIGRpc3RyaWJ1dGlvbnMgb2YgdGhlIGRhdGEgYWNyb3NzIHNhbXBsZXMuIFRoZXkgY2FuIGFsc28gYmUgc3RhbmRhcmRpemVkIGJ5IHVzaW5nIGEgcm9idXN0IGVzdGltYXRvciBmb3IgbG9jYXRpb24gYW5kIHNjYWxlLCBpLmUuIAokJHlfe2lwfV5cdGV4dHtub3JtfSA9IFxmcmFje3lfe2lwfSAtIFxtdV9pfXtzX2l9JCQgCgotLS0KCiMjIFN1bW1hcml6YXRpb24gCgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL05VUEpXUEpnUVVjIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKLSBXZSBpbGx1c3RyYXRlIHN1bW1hcml6YXRpb24gaXNzdWVzIHVzaW5nIGEgc3Vic2V0IG9mIHRoZSBjcHRhYyBzdHVkeSAoTGFiIDIsIGNvbmRpdGlvbiBBIGFuZCBFKSBmb3IgYSBzcGlrZWQgcHJvdGVpbiAoVVBTIFAxMjA4MSkuCgo8ZGV0YWlscz48c3VtbWFyeT4gQ2xpY2sgdG8gc2VlIGNvZGUgdG8gbWFrZSBwbG90IDwvc3VtbWFyeT48cD4KYGBge3IgcGxvdCA9IEZBTFNFfQpzdW1tYXJ5UGxvdCA8LSBwZVtbInBlcHRpZGVOb3JtIl1dWwogICAgcm93RGF0YShwZVtbInBlcHRpZGVOb3JtIl1dKSRQcm90ZWlucyA9PSAiUDEyMDgxdXBzfFNZSENfSFVNQU5fVVBTIiwKICAgIGNvbERhdGEocGUpJGxhYj09ImxhYjIiJmNvbERhdGEocGUpJGNvbmRpdGlvbiAlaW4lIGMoIkEiLCJFIildICU+JQogIGFzc2F5ICU+JQogIGFzLmRhdGEuZnJhbWUgJT4lCiAgcm93bmFtZXNfdG9fY29sdW1uKHZhciA9ICJwZXB0aWRlIikgJT4lCiAgZ2F0aGVyKHNhbXBsZSwgaW50ZW5zaXR5LCAtcGVwdGlkZSkgJT4lIAogIG11dGF0ZShjb25kaXRpb24gPSBjb2xEYXRhKHBlKVtzYW1wbGUsImNvbmRpdGlvbiJdKSAlPiUKICBnZ3Bsb3QoYWVzKHggPSBwZXB0aWRlLCB5ID0gaW50ZW5zaXR5LCBjb2xvciA9IHNhbXBsZSwgZ3JvdXAgPSBzYW1wbGUsIGxhYmVsID0gY29uZGl0aW9uKSwgc2hvdy5sZWdlbmQgPSBGQUxTRSkgKwogIGdlb21fbGluZShzaG93LmxlZ2VuZCA9IEZBTFNFKSArCiAgZ2VvbV90ZXh0KHNob3cubGVnZW5kID0gRkFMU0UpICsKICB0aGVtZV9taW5pbWFsKCkgKwogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTAsIHZqdXN0ID0gMC41LCBoanVzdCA9IDEpKSArCiAgeGxhYigiUGVwdGlkZSIpICsgCiAgeWxhYigiSW50ZW5zaXR5IChsb2cyKSIpCmBgYAo8L3A+PC9kZXRhaWxzPgoKYGBge3J9CnN1bW1hcnlQbG90CmBgYAoKV2Ugb2JzZXJ2ZTogIAoKLSBpbnRlbnNpdGllcyBmcm9tIG11bHRpcGxlIHBlcHRpZGVzIGZvciBlYWNoIHByb3RlaW4gaW4gYSBzYW1wbGUKLSBTdHJvbmcgcGVwdGlkZSBlZmZlY3QgCi1VbmJhbGFuY2VkIHBlcHRpZGUgaWRlbnRpZmljYXRpb24KLSBQc2V1ZG8tcmVwbGljYXRpb246IHBlcHRpZGUgaW50ZW5zaXRpZXMgZnJvbSBhIHBhcnRpY3VsYXIgcHJvdGVpbiBpbiB0aGUgc2FtZSBzYW1wbGUgYXJlIGNvcnJlbGF0ZWQsIGkuZS4gdGhleSBtb3JlIGFsaWtlIHRoYW4gcGVwdGlkZSBpbnRlbnNpdGllcyBmcm9tIGEgcGFydGljdWxhciBwcm90ZWluIGJldHdlZW4gc2FtcGxlcy4gCgoKJFxyaWdodGFycm93JCBTdW1tYXJpemUgYWxsIHBlcHRpZGUgaW50ZW5zaXRpZXMgZnJvbSB0aGUgc2FtZSBwcm90ZWluIGluIGEgc2FtcGxlIGludG8gYSBzaW5nbGUgcHJvdGVpbiBleHByZXNzaW9uIHZhbHVlCgpDb21tb25seSB1c2VkIG1ldGhvZHMgYXJlIAoKLSBNZWFuIHN1bW1hcml6YXRpb24KJCQKeV97aXB9PVxiZXRhX2leXHRleHR7c2FtcH0gKyBcZXBzaWxvbl97aXB9CiQkCgotIE1lZGlhbiBzdW1tYXJpemF0aW9uCi0gTWF4cXVhbnQncyBtYXhMRlEgc3VtbWFyaXphdGlvbiAoaW4gcHJvdGVpbiBncm91cHMgZmlsZSkKLSBNb2RlbCBiYXNlZCBzdW1tYXJpemF0aW9uOiAKJCQKeV97aXB9PVxiZXRhX2leXHRleHR7c2FtcH0gKyBcYmV0YV9wXlx0ZXh0e3BlcH0gKyBcZXBzaWxvbl97aXB9CiQkCgoKPGRldGFpbHM+PHN1bW1hcnk+IENsaWNrIHRvIHNlZSBSLWNvZGUgdG8gbm9ybWFsaXplIHRoZSBkYXRhIDwvc3VtbWFyeT48cD4KV2UgdXNlIHRoZSBzdGFuZGFyZCBzdW1hcml6YXRpb24gaW4gYWdncmVnYXRlRmVhdHVyZXMsIHdoaWNoIGlzCnJvYnVzdCBtb2RlbCBiYXNlZCBzdW1tYXJpemF0aW9uLiAKCmBgYHtyLHdhcm5pbmc9RkFMU0V9CnBlIDwtIGFnZ3JlZ2F0ZUZlYXR1cmVzKHBlLAogICAgaSA9ICJwZXB0aWRlTm9ybSIsIAogICAgZmNvbCA9ICJQcm90ZWlucyIsIAogICAgbmEucm0gPSBUUlVFLAogICAgbmFtZSA9ICJwcm90ZWluIikKYGBgCgpPdGhlciBzdW1tYXJpemF0aW9uIG1ldGhvZHMgY2FuIGJlIGltcGxlbWVudGVkIGJ5IHVzaW5nIHRoZSBgZnVuYCBhcmd1bWVudCBpbiB0aGUgYGFnZ3JlZ2F0ZUZlYXR1cmVzYCBmdW5jdGlvbi4gCgotIGBmdW4gPSBNc0NvcmVVdGlsczo6bWVkaWFuUG9saXNoKClgIHRvIGZpdHMgYW4gYWRkaXRpdmUgbW9kZWwgKHR3byB3YXkgZGVjb21wb3NpdGlvbikgdXNpbmcgVHVrZXkncyBtZWRpYW4gcG9saXNoXyBwcm9jZWR1cmUgdXNpbmcgc3RhdHM6Om1lZHBvbGlzaCgpCgotIGBmdW4gPSBNc0NvcmVVdGlsczo6cm9idXN0U3VtbWFyeSgpYCB0byBjYWxjdWxhdGUgYSByb2J1c3QgYWdncmVnYXRpb24gdXNpbmcgTUFTUzo6cmxtKCkgKGRlZmF1bHQpCgotIGBmdW4gPSBiYXNlOjpjb2xNZWFucygpYCB0byB1c2UgdGhlIG1lYW4gb2YgZWFjaCBjb2x1bW4KCi0gYGZ1biA9IG1hdHJpeFN0YXRzOjpjb2xNZWRpYW5zKClgIHRvIHVzZSB0aGUgbWVkaWFuIG9mIGVhY2ggY29sdW1uCgotIGBmdW4gPSBiYXNlOjpjb2xTdW1zKClgIHRvIHVzZSB0aGUgc3VtIG9mIGVhY2ggY29sdW1uCgo8L3A+PC9kZXRhaWxzPgoKLS0tCgojIEV4ZXJjaXNlCgoxLiBXZSB3aWxsIGV2YWx1YXRlIGRpZmZlcmVudCBzdW1tYXJpemF0aW9uIG1ldGhvZHMgKE1heHF1YW50IG1heExGUSwgbWVkaWFuIGFuZCByb2J1c3QgbW9kZWwgYmFzZWQpIGluIHRoZSB0dXRvcmlhbCBzZXNzaW9uIGJlZm9yZSBkaXNjdXNzaW5nIG9uIHRoZWlyIGFkdmFudGFnZXMvZGlzYWR2YW50YWdlcy4KMi4gQ2FuIHlvdSBhbnRpY2lwYXRlIG9uIHBvdGVudGlhbCBwcm9ibGVtcyByZWxhdGVkIHRvIHRoZSBzdW1tYXJpemF0aW9uPwoKLS0tCgojIFNvZnR3YXJlICYgY29kZQoKLSBPdXIgUi9CaW9jb25kdWN0b3IgcGFja2FnZSBbbXNxcm9iMl0oaHR0cHM6Ly93d3cuYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL2Jpb2MvaHRtbC9tc3Fyb2IyLmh0bWwpIGNhbiBiZSB1c2VkIGluIFIgbWFya2Rvd24gc2NyaXB0cyBvciB3aXRoIGEgR1VJL3NoaW55QXBwIGluIHRoZSBbbXNxcm9iMmd1aV0oaHR0cHM6Ly9naXRodWIuY29tL3N0YXRPbWljcy9tc3Fyb2IyZ3VpKSBwYWNrYWdlLgoKLSBUaGUgR1VJIGlzIGludGVuZGVkIGFzIGEgaW50cm9kdWN0aW9uIHRvIHRoZSBrZXkgY29uY2VwdHMgb2YgcHJvdGVvbWljcyBkYXRhIGFuYWx5c2lzIGZvciB1c2VycyB3aG8gaGF2ZSBubyBleHBlcmllbmNlIGluIFIuIAoKLSBIb3dldmVyLCBsZWFybmluZyBob3cgdG8gY29kZSBkYXRhIGFuYWx5c2VzIGluIFIgbWFya2Rvd24gc2NyaXB0cyBpcyBrZXkgZm9yIG9wZW4gZW4gcmVwcm9kdWNpYmxlIHNjaWVuY2UgYW5kIGZvciByZXBvcnRpbmcgeW91ciBwcm90ZW9taWNzIGRhdGEgYW5hbHlzZXMgYW5kIGludGVycHJldGF0aW9uIGluIGEgcmVwcm9kdWNpYmxlIHdheS4gCgoKLSBNb3JlIGluZm9ybWF0aW9uIG9uIG91ciB0b29scyBjYW4gYmUgZm91bmQgaW4gb3VyIHBhcGVycyBbQGdvZW1pbm5lMjAxNl0sIFtAZ29lbWlubmUyMDIwXSBhbmQgW0BzdGlja2VyMjAyMF0uIFBsZWFzZSByZWZlciB0byBvdXIgd29yayB3aGVuIHVzaW5nIG91ciB0b29scy4gCgoKCiMjIENvZGUgCgoxLiBEYXRhIGluZnJhc3RydWN0dXJlCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvTjB5SVoxWUIyYVEiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgoyLiBJbXBvcnQgcHJvdGVvbWljcyBkYXRhCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvRFBIdHJHajhza2siCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgozLiBQcmVwcm9jZXNzaW5nCgogICAgLSAgIExvZy10cmFuc2Zvcm1hdGlvbgo8aWZyYW1lIHdpZHRoPSI1NjAiIGhlaWdodD0iMzE1IgpzcmM9Imh0dHBzOi8vd3d3LnlvdXR1YmUuY29tL2VtYmVkL2F4RFQydEpOTHdNIgpmcmFtZWJvcmRlcj0iMCIKc3R5bGU9ImRpc3BsYXk6IGJsb2NrOyBtYXJnaW46IGF1dG87IgphbGxvdz0iYXV0b3BsYXk7IGVuY3J5cHRlZC1tZWRpYSIgYWxsb3dmdWxsc2NyZWVuPjwvaWZyYW1lPgoKICAgIC0gRmlsdGVyaW5nCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvY0l3QXhmSjVDR28iCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgogICAgLSBOb3JtYWxpc2F0aW9uCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvOXFDSkVrZFNTaEkiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+CgogICAgLSBTdW1tYXJpemF0aW9uCjxpZnJhbWUgd2lkdGg9IjU2MCIgaGVpZ2h0PSIzMTUiCnNyYz0iaHR0cHM6Ly93d3cueW91dHViZS5jb20vZW1iZWQvZldxNHJNeG1KVFUiCmZyYW1lYm9yZGVyPSIwIgpzdHlsZT0iZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiCmFsbG93PSJhdXRvcGxheTsgZW5jcnlwdGVkLW1lZGlhIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+ICAgIAoKIyMgRGF0YSBhbmFseXNpcyB3aXRoIHRoZSBHVUkvc2hpbnlBcHAgbXNxcm9iMmd1aQoKPGlmcmFtZSB3aWR0aD0iNTYwIiBoZWlnaHQ9IjMxNSIKc3JjPSJodHRwczovL3d3dy55b3V0dWJlLmNvbS9lbWJlZC9GNWQwRS1VNHJPTSIKZnJhbWVib3JkZXI9IjAiCnN0eWxlPSJkaXNwbGF5OiBibG9jazsgbWFyZ2luOiBhdXRvOyIKYWxsb3c9ImF1dG9wbGF5OyBlbmNyeXB0ZWQtbWVkaWEiIGFsbG93ZnVsbHNjcmVlbj48L2lmcmFtZT4gICAKCiMgUmVmZXJlbmNlcw==