目录
- DBAdd files via upload3年前
- ExampleDatasetsAdd files via upload3年前
- ipaPy2Fixing small bug on reporting ppmunk9个月前
- testsAdd files via upload3年前
- Beer_example.ipynbAdd files via upload3年前
- Ecoli_example.ipynbAdd files via upload3年前
- Integration_with_mzMatch.ipynbAdd files via upload3年前
- LICENSEInitial commit3年前
- PeakMLViewerpy_example.pngAdd files via upload3年前
- QTOF_DDA_example.ipynbAdd files via upload3年前
- README.ipynbfixing typos2年前
- README.mdUpdate README.md2年前
- Synthetic_example.ipynbAdd files via upload3年前
- figure_paper.pngAdd files via upload3年前
- ipaPy2.tomlupdate of setup and toml file9个月前
- setup.cfgCreate setup.cfg3年前
- setup.pyupdate of setup and toml file9个月前
邀请码
ipaPy2
Python implementation of the Integrated Probabilistic Annotation (IPA) - A Bayesian annotation method for LC/MS data integrating biochemical relations, isotope patterns and adduct formation.
Installation
ipaPy2 requires Python 3.9 or higher
Install via pip (recommended)
Install via bioconda
Compiling from source (macOS)
Compiling from source (Linux)
Compiling from source (Windows)
Databases
One of the most powerful features of the IPA method is that it is able to integrate the knowledge gained from previous experiments in the annotation process. There are three files that are used as the IPA database:
1. Adducts file (required)
The ipaPy2 library requires a file contains all the information required for the computation of the adducts. An adducts.csv file is provided with the package here. The file contains the most common adducts. If any exotic adduct (or in-source fragment) needs to be considered, the user must modify the file accordingly. The format required for the adducts file is shown below.
2. MS1 database file (required)
The IPA method requires a pandas dataframe containing the database against which the annotation is performed. This dataframe must contain the following columns in this exact order (optional columns can have empty fields):
The column names must be the ones reported here. While users are strongly advised to build their own ad-hoc database to match their specific instrument setup and sample types, here you can find a relatively big example database.
This example databases was obtained considering the KEGG database, the Natural Products Atlas database and the MoNa database (only compounds having at least one fragmentation spectrum obtained with a QExactive). For each entry, only a handful of the most common adducts are considered. To fully exploit the IPA method, it is strongly recommended to constantly update the database when new knowledge is gained from previous experience. Providing a retention time window for compounds previously detected with the analytical system at hand it is particularly useful. For the sake of the example in this tutorial, a reduced example database is also provided.
3. MS2 database file (only required if MS2 data is available)
This new implementation of the IPA method also allows the user to include MS2 data in the annotation pipeline. In order to exploit this functionality an MS2 spectra database must be provided. The MS2 database must be provided as a pandas dataframe including the following columns in this exact order:
It is necessary that the user uses an MS2 database specific to the instrument used to acquire the data. The MS2 database found here, contains all the MS2 spectra found in the MoNa database acquired with a QExactive. This is a relatively big file, and for the sake of this tutorial a drastically reduced version of it has been included within this repository, and can be found here.
Data preparation
Before using the ipaPy2 package, the processed data coming from an untargeted metabolomics experiment must be properly prepared.
1. MS1 data
The data must be organized in a pandas dataframe containing the following columns:
Below is reported an example:
The clustering of the features is a necessary and must be performed before running the IPA method. For this step, the use of widely used data processing software such as mzMatch and CAMERA is recommended. Nevertheless, the ipaPy2 library provides a function (clusterFeatures()) able to perform such step, starting from a dataframe containing the measured intensities across several samples (at least 3 samples, the more samples the better). Such dataframe should be organized as follows:
All information about the function can be found in the help of the function
After running, this function returns a pandas dataframe in the correct format for the ipaPy2 package
2. MS2 data
If fragmentation data was acquired during the experiment, it can be included in the IPA annotation process. To do so, the data must be organized in a pandas dataframe containing the following columns, in this exact order:
Below is reported an example:
Usage
The Integrated Probabilistic Annotation (IPA) method can be applied in different situations, and the ipaPy2 package allows the users to tailor the IPA pipeline around their specific needs.
This brief tutorial describes the most common scenarios the IPA method can be applied to.
1. Mapping isotope patterns
The first step of the IPA pipeline consists in the mapping of the isotope patterns within the dataset considered. This is achieved through the map_isotope_patterns(). The help of this function provides a detailed description of it.
For the sake of this tutorial, the small dataset example introduced above is considered.
Once finished, this function modifies the pandas dataframe provided as input annotating all isotope patterns.
Some data processing pipelines already have an isotope mapping function and the user can use them as long as they organise the data in the correct format.
2. Compute all adducts
The second step of the pipeline consists in the calculation of all possible adducts that could be formed by the compounds included in the database. This is done by the function compute_all_adducts(). This function comes with a very detailed help.
Depending on the size of the dataset used (i.e., number of compounds included), this step can become rather time-consuming, and the use of multiple cores should be considered. In the context of this tutorial, the heavily reduced example dataset introduced before is considered.
If the same database is used for subsequent experiments without introducing new information, it is recommended to save the results of this function into a .csv file. In this case, the user would need to repeat this step in the future only if the DB changed.
3. Annotation based on MS1 information
At this point, the actual annotation process can start. If no fragmentation data is available, the MS1annotation() function should be used. This function annotates the dataset using the MS1 data and the information stored in the dataset. A detailed description of the function can be accessed through the help:
This function returns all the possible annotations for all the mass spectrometry features (excluding the ones previously identified as isotopes). The annotations are provided in the form of a dictionary. The keys of the dictionary are the unique ids for the features present in df. For each feature, all possible annotations are summarised in a dataframe including the following information:
As an example, possible annotations for the feature associated with id=1 (m/z=116.0705438, RT=45.77) is shown below:
It should be noticed that in this example, the prior probabilities associated with L-Proline M+H, D-Proline M+H and 3-Acetamidopropanal are exactly the same. This is because all three ions have exactly the same theoretical mass. However, the post probabilities are different. This is because the retention time associated with this feature is within the retention range reported in the database for L-Proline and outside the one reported for 3-Acetamidopropanal.
An expert in LC/MS-based mass spectrometry would argue that with most chromatographic columns stereoisomers such as L- and D-Proline would share the same RT range. While this is likely to be correct, it must be noted that the IPA method can only use the information present in the database. When populating it, we opted for a more agnostic approach and only included RT ranges for compounds that where actually detected as standards with our experimental setting. If the user wants to include the notion that ‘stereoisomers share the same RT ranges’, they should simply add this information in the database.
Here another example:
Also in this case, all the prior probabilities associated with the four ions are exactly the same since all the ions have the same theoretical mass-to-charge ratio. However, the posterior probabilities are significantly different. Two of these ions (Coproporphyrinogen III M+2H and Coproporphyrinogen I M+2H) have charge +2, while the other two possible annotations have charge +1. The observed isotope pattern is consistent with an ion with charge +1 (i.e., difference between isotopes = 1), and this is reflected in the isotope score pattern and consequently on the posterior probabilities. Moreover, the retention time associated with this feature is within the range reported for L-Phenylalanine in the database. Therefore, the posterior probability associated with L-Phenylalanine 2M+H is the most highest.
4. Annotation based on MS1 and MS2 information
As already mentioned above, fragmentation data can be included in the annotation process by using the MSMSannotation() function. A detailed description of the function can be accessed through the help:
The line below integrates the fragmentation data and the fragmentation database introduced above in the annotation process. The role of the CSunk (“cosine unknown”) parameter should be briefly discussed here. In most cases, the fragmentation database contains fragmentation spectra only for a subset of the compounds in the database. Therefore, when considering a feature for which the fragmentation spectra was acquired, it is often the case that the cosine similarity can only be computed for a subset of the possible annotations. The CSunk value is then assigned to the other possible annotations for comparison.
The output of this function has the same structure as the one from the MSannotation() function, but it also includes the fragmentation pattern scores when the fragmentation data is available. As an example, possible annotations for the feature associated with id=1 is shown below:
In this case, the cosine similarity score for the annotation L-Proline M+H is very high, therefore the posterior probability associated with it is higher than the one obtained without considering the MS2 data.
Here another example for a feature having a very similar mass-to-charge ratio (id=90, m/z=117.0705223, RT=63.45).
In this case, the cosine similarity score for the annotation L-Proline M+H is not very high. Moreover, the retention time assigned to this feature is outside both retention time ranges reported in the database for L-Proline and 3-Acetamidopropanal. Therefore, the most likely annotation for this feature is D-Proline M+H, the one annotation not rejected directly by the available evidence. It should be noted that the fragmentation pattern score has a rather weak effect on the posterior probability associated with L-Proline, given how close it is to the fragmentation pattern score associated features that do not have MS2 info in the database (CSunk=0.7). The main reason why the D-Proline annotation appears to be the most likely is due to the fact that the retention time associated to this feature (63.45 s) is outside the retention time ranges associated with L-Proline and 3-Acetamidopropanal.
5. Computing posterior probabilities integrating adducts connections
Until this point, the putative annotations and the associated probabilities computed for each feature are independent from each other. However, the IPA method can be used to update the probabilities by considering the possible relationship between annotations. For example, the Gibbs_sampler_add() function uses a Gibbs sampler to estimate the posterior probabilities obtained by considering all possible adduct connections.
The help() provides a detailed description of this function:
The function modifies the annotations dictionary by adding two additional columns to each dataframe:
If all_out=True, the function also returns the full list of assignments computed. If provided as an input to the Gibbs sampler, it allows to restart it from where you finished.
As an example, the possible annotations for the feature associated with the id 501 is shown below.
This feature is clustered with feature id=1, the most likely annotation of which is L-Proline M+H. As expected, considering the adducts connections the ‘post Gibbs’ probability associated with L-Proline 2M+H is significantly higher than the alternative.
6. Computing posterior probabilities integrating biochemical connections
The IPA method can also update the probabilities associated to each possible annotations by considering all possible biochemical connections.
Before doing so, it is necessary to provide a pandas dataframe reporting which compounds can be considered biochemically related. The function Compute_Bio() can be used to compute such a dataframe. The help() provides a detailed description of this function:
According to the value assigned to the ‘mode’ parameter, the function can compute all possible biochemical connections in two ways. If mode=’reactions’, the function connects the compounds that share the same reaction id(s) according to what is reported in the database.
If mode=’connections’, the function computes the ‘chemical formula difference’ for each pair of compounds considered. If the difference is included in the list of connections, the two compounds are considered connected. A list of connections is provided as default, but it can be modified.
Depending on the size of the database and the dataset, computing all possible biochemical connections can be extremely computationally demanding and can drastically increase the computation time needed for the annotation. For this reason, a precomputed list of biochemical connections based on the database provided (computed based on ‘reaction’ or ‘connections’ mode) is included in the library and can be used directly without the need of computing the biochemical connections.
The list of connections computed with mode=’connections’ needs to be unzipped first.
Alternatively, the user can define their own biochemical connections. For example: L-Proline C00148 L-Valine C00183 L-Phenylalanine C00079 L-Leucine C00123 5-Oxoproline C01879 Betaine C00719 Hordatine A C08307 L-Tyrosine C00082 D-Proline C00763 D-Phenylalanine C02265
As an example, the possible annotations for the feature associated with the id 992 is shown below.
The probability associated with the D-Proline M+H is significantly higher after considering the biochemical connections. This is because D-Proline is biochemically connected to L-Proline (by proline racemase), and the most likely annotation for the feature id=1 is L-Proline M+H (>50%).
7. Computing posterior probabilities integrating both adducts and biochemical connections
It is also possible to run the Gibbs sampler considering biochemical and adduct connections at the same time. To do so, one can use the function Gibbs_sampler_bio_add(). The help() provides a detailed explanation of the function.
8. Running the whole pipeline with a single function
Finally, the ipaPy2 library also include a wrapper function that allows running the whole IPA pipeline in one step. A detailed description of the function can be accessed with the help.
Based on the parameters passed on to the function, the end-result of this function will be different.
For example, if one wants to use both the MS1 and MS2 data and not use the Gibbs sampler, the following should be used:
If instead one wants to use only the MS1 data and only consider the adducts connections in the Gibbs sampler, one should use the following:
Or, if one wants to use both the MS1 and MS2 data and consider both adducts and biochemical connections in the Gibbs sampler, the following should be used.