Difference between revisions of "Description of GenGIS plugins"

From The GenGIS wiki
Jump to navigationJump to search
Line 9: Line 9:
 
The ''GBIF Query'' plugin creates location and sequence data for use in GenGIS from the Global Biodiversity Information Facility (GBIF) [http://data.gbif.org/]. It queries the GBIF database with one or more user-provided taxon names and a geographic range, and returns all instances with geographic location data that match the query. When using the GBIF plugin to create datasets, please read and adhere to GBIF's [http://data.gbif.org/tutorial/datauseagreement '''Data Use Agreement'''] and [http://data.gbif.org/tutorial/datasharingagreement '''Data Sharing Agreements'''].
 
The ''GBIF Query'' plugin creates location and sequence data for use in GenGIS from the Global Biodiversity Information Facility (GBIF) [http://data.gbif.org/]. It queries the GBIF database with one or more user-provided taxon names and a geographic range, and returns all instances with geographic location data that match the query. When using the GBIF plugin to create datasets, please read and adhere to GBIF's [http://data.gbif.org/tutorial/datauseagreement '''Data Use Agreement'''] and [http://data.gbif.org/tutorial/datasharingagreement '''Data Sharing Agreements'''].
  
For the purposes of the next example, it is assumed that the user has loaded a ''Raster Map'' file. This allows for the full functionality to be displayed. If a user has not loaded a ''Raster Map'' then ''Add Data'' will not be available, but the retrieved data can still be saved to a pair of files.
+
For the purposes of the next example, it is assumed that the user has loaded a ''Raster Map'' file. If a user has not loaded a ''Raster Map'' then ''Add Data'' will not be available, but the retrieved data can still be saved to disk.
  
 
Please be aware that large queries may result in the plugin entering a ''Not Responding'' state. This is controlled by the operating system, and while the plugin will not respond to user input, it is still performing its query.
 
Please be aware that large queries may result in the plugin entering a ''Not Responding'' state. This is controlled by the operating system, and while the plugin will not respond to user input, it is still performing its query.
Line 39: Line 39:
 
===Step 4: Add/Export Data===
 
===Step 4: Add/Export Data===
  
Finally the user can choose to export their data to a location on their disk drive, or add it directly to GenGIS. The ''Export'' button writes three separate files to a user specified location on disk. These files are the location file, sequence file and a source file containing collection metadata for the data set, any specialized rights associated with that data, and how to cite them for published works. Saving data in files eliminates the need to redo lengthy queries at a later date. If ''Add Data'' is selected then the location and sequence files are added directly to GenGIS without saving. The source information is imported into the description of the location layer.
+
Finally the user can choose to export their data to a location on their disk drive, or add it directly to GenGIS. The ''Export'' button writes three separate files to a user-specified location on disk. These files are the location file, sequence file and a source file containing collection metadata for the data set, any specialized rights associated with that data, and how to cite them for published works. Saving data in files eliminates the need to redo lengthy queries at a later date. If ''Add Data'' is selected then the location and sequence files are added directly to GenGIS without saving. The source information is imported into the description of the location layer.
  
 
[[Image:Done.png|thumb|center|401px|Data added to GenGIS.]]
 
[[Image:Done.png|thumb|center|401px|Data added to GenGIS.]]

Revision as of 13:29, 7 August 2013

GenGIS provides the following Python plugins which can be accessed through the Data and Plugins menus. Please contact us if you have questions about using the plugins, or if you have suggestions for new plugins.

Data Retrieval

We are currently developing plugins to retrieve data from several online sources. In all cases, we ask that you familiarize yourself with the relevant Terms of Use and any Disclaimers regarding use of the data; we link to these below wherever possible. The developers of GenGIS are in no way responsible for data provided by third-party sources, and are not liable for any consequences arising from the use of our software and plugins.

GBIF Query

The GBIF Query plugin creates location and sequence data for use in GenGIS from the Global Biodiversity Information Facility (GBIF) [1]. It queries the GBIF database with one or more user-provided taxon names and a geographic range, and returns all instances with geographic location data that match the query. When using the GBIF plugin to create datasets, please read and adhere to GBIF's Data Use Agreement and Data Sharing Agreements.

For the purposes of the next example, it is assumed that the user has loaded a Raster Map file. If a user has not loaded a Raster Map then Add Data will not be available, but the retrieved data can still be saved to disk.

Please be aware that large queries may result in the plugin entering a Not Responding state. This is controlled by the operating system, and while the plugin will not respond to user input, it is still performing its query.

GBIF Query plugin.

Step 1: The Query

In order to query GBIF two things must be entered: a taxon name and a geographic range. If a map is loaded prior to running this plugin the default range borders will be the extents of the map, if not they will be the entire world. The geographic range can be fine tuned using either text input or the scroll wheels. After the appropriate information has been entered hitting Search will query GBIF for all possible taxonomic matches.

Look for taxon instances.

Step 2: Add/Remove Items

Hitting Search populates the Results Table. This is where all matches are returned by GBIF:

Unique ID Number | Full Name | Biological Classification | Data Source

Highlighting entries in this list and clicking Add or double-clicking entries adds them to the ID List. This list is what will be used to query GBIF to create the location and sequence file. Highlighting an entry in this list and clicking Remove or double-clicking an entry removes it from consideration. A user can perform multiple queries and add multiple taxa to this list, but only one geographic range can be defined.

Prepare data to be queried.

Step 3: Retrieve Data/Query Records

Once the user is satisfied with the contents of the ID List they can choose either Retrieve Data or Query Records. Query Records quickly retrieves the number of results without retrieving the results themselves; this can be used to quickly determine whether the size of the data set will be suitable for use in GenGIS. This information is displayed in the Summary dialog box. Large data sets (e.g., >1000 locations) will take more time to retrieve and process, as well as slow down GenGIS. If the user is satisfied with the amount of records they are about to retrieve they can move on to the Retrieve Data option. Here GBIF is queried and the progress of that query is displayed in the Progress box.

Output from 'Calculate'.

Step 4: Add/Export Data

Finally the user can choose to export their data to a location on their disk drive, or add it directly to GenGIS. The Export button writes three separate files to a user-specified location on disk. These files are the location file, sequence file and a source file containing collection metadata for the data set, any specialized rights associated with that data, and how to cite them for published works. Saving data in files eliminates the need to redo lengthy queries at a later date. If Add Data is selected then the location and sequence files are added directly to GenGIS without saving. The source information is imported into the description of the location layer.

Data added to GenGIS.

Analysis Plugins

Alpha Diversity

The Alpha Diversity plugin calculates alpha diversity for active locations. It currently calculate richness, Shannon, and Simpson alpha diversity. To calculate alpha diversity, you must select the Measure you wish to calculate and the Category field in your sequence file over which diversity will be calculated. You may optionally select a Count field which indicates the number of times a given sequence is observed at a location. Pressing Calculate causes alpha diversity to be calculated. Results are reported within the plugin and added to the location table for use within GenGIS and other plugins.

Alpha Diversity plugin.

Alpha Diversity Visualizer

The Alpha Diversity Visualizer plugin can calculate alpha diversity for active locations, regress alpha diversity against location specific metadata, and produce visualizations of the resulting linear regression analysis. It currently calculate richness, Shannon, and Simpson alpha diversity. To calculate alpha diversity, you must select the Measure you wish to calculate and the Category field in your sequence file over which diversity will be calculated. You may optionally select a Count field which indicates the number of times a given sequence is observed at a location. Pressing Calculate causes alpha diversity to be calculated. Linear regression results of alpha diversity versus all numeric fields associated with locations are reported within the Linear Regression Results table. Selecting a row within this table causes a linear regression scatter plot of alpha diversity versus the selected Field to be generated. The Viewport Display section allows different Viewport visualization to be produced.

Alpha Diversity Visualizer plugin.

Bar Graph

The Bar Graph plugin provides bar graphs showing the relative abundance of sequence data from two groups. Groups can be defined be any field in your Location file and bar plots created for any numeric field in your Sequence file. You may optionally specify a Count field from the Sequence file indicates the number of times a given sequence is observed. This allows both qualitative and quantitative bar plots to be generated.

Bar Graph plugin.

Beta Diversity Calculator

The Beta Diversity plugin calculates beta diversity between active locations. The resulting biotic dissimilarity matrix can be saved to file and visualized in GenGIS using the Dissimilarity Matrix Viewer plugin. It currently calculate 9 measures of beta diversity (e.g., Bray-Curtis, Jaccard) across any field defined in your Sequence File. Sequences classified as Other or Unclassified can be optionally ignored during the calculation of beta diversity. In order to account for unequal sampling depth, subsampling with replacement (i.e., jackknifing) can be performed and the mean beta-diversity between jackknifed samples reported. Hierarchical cluster trees indicating the relative similarity of locations can be produced and used as an input Tree File to GenGIS.

Beta Diversity Plugin‎

Canonical Correlation Analysis

  • Requirements: R with the cca library must be installed on your system (see the GenGIS manual).

The Canonical Correlation Analysis or CCA plugin implements the widely used statistical technique for joint analysis of biodiversity and environmental data across a number of sites. The plugin also generates Phenotype-Environment Network (PEN) graphs as described in Patel et al. (2010) Analysis of membrane proteins in metagenomics: Networks of correlated environmental features and protein families once a CCA has been carried out. The reference for the required R CCA package is Gonzalez et al (2008). The following example uses data from the Global Ocean Sampling dataset.

Step 1: Matrix Correlation

Before carrying out CCA, run the 'Matrix Correlation' function to ensure there is some level of correlation in the dataset. The figure below shows some evidence of strong and negative correlations, so we can proceed to the next step.

CMatrix Correlation.

Step 2: Grid Search

The cca library implements a grid search function to determine the optimum value of two key parameters, λ1 and λ2. To perform the grid search in reasonable time, we recommend starting with a coarse search (e.g., the default ranges as specified by the plugin) and iteratively seeking the best values by refining the parameters.

Grid Search.

Step 3: Run CCA

After choosing the most appropriate values of λ1 and λ2, we run the CCA to generate biplots that show the relationships between our input habitat and sequence count variables. The abundance of certain taxonomic classes seems to correlate with the three environmental variables considered.

CCA output.

Step 4: Generate PEN and view in Cytoscape

To gain a better perspective on the relationships between variables, we can generate a phenotype-environment network that displays each variable as a node, and connects nodes for which the products of canonical correlates for the chosen number of dimensions sum to an absolute value greater than the chosen threshold. The network below, exported as a .xgmml file and imported into Cytoscape, shows relationships based on the first two dimensions, with positive correlations in green and negative ones in red.

Phenotype-environment network viewed in Cytoscape.

Dissimilarity Matrix Viewer

The Dissimilarity Matrix Viewer plugin provides functionality for visualizing a matrix which indicates the dissimilarity between all pairs of locations. The dissimilarity matrix must be in the following format, where a \t indicates a tab:

 3
A\t0\t2\t3
B\t1\t0\t4
C\t3\t5\t0

The first line indicates the number of locations and each of the following rows gives the dissimilarity values for the specified location. The location names (first column) must match those in your location file. The upper and lower triangles of the matrix can be different. For example, in this HIV-1 data set, the two triangle indicate import and export rates.

Elements in the matrix are selected by setting the Selection criteria.

Dissimilarity Matrix Viewer plugin.

Lines between the selected pairs are displayed in the Viewport using the specified Visual properties. To update the Viewport display click Apply.

Display of all matrix elements between 5 and 10.

Environmental Data Visualizer

Under construction - for 2.11 release

The Environmental Data Visualizer plugin displays environmental data as bar graphs and colored points on a map, in a manner similar to the Alpha Diversity Visualizer plugin but without the need for sequence data to be loaded or defined.

Environmental Data plugin.
Map showing bar graph and color gradient representation of site pH.

Linear Regression

The Linear Regression plugin can be used to perform a linear regress between any two variables in the Location Table (see Location Table Viewer below). To perform the regression, the independent and dependent variables must be specified in the Regression analysis section of the plugin.

Linear Regression plugin.

The results of the regression are reported within the plugin and shown as a scatter plot. A visualization within the GenGIS Viewport is also generated based on the properties set in the Viewport display section of the plugin.

Residuals of linear regression shown within the GenGIS Viewport.

Location Table Viewer

The Location Table Viewer plugin display a table indicating the metadata associated with each location. Other plugins and custom Python scripts can be used to add data to the Location Table. By default, only data for active locations is shown. To show data for all locations check the Show data for all locations checkbox.

Location Table plugin.

Mantel Test

  • Requirements: R with the ade4 library must be installed on your system (see the GenGIS manual).

The Mantel plugin can be used to perform a Mantel test between any two variables in the Location Table or Sequence Table.

Mantel plugin.

Multi-Tree Optimal-Crossing Test

This plugin will calculate the optimal angle for a set of loaded trees, and show the distribution of crossings for any azimuthal angle. A bar graph shows, for each tree, how close the number of crossings is to the number of crossings observed in the optimal layout for that tree.

Multi-Tree Optimal-Crossing Test plugin.

Sequence Table Viewer

The Sequence Table Viewer plugin display a table indicating the metadata associated with each sequence. Other plugins and custom Python scripts can be used to add data to the Sequence Table. By default, only data for active locations and active sequences is shown. To show data for all locations check the Show data for all locations checkbox. To show data for all sequences check the Show data for all sequences checkbox.

Sequence Table plugin.

Reference Condition Analysis

  • Overview:
    • The Reference Condition Analysis plugin is used to evaluate impacts on biodiversity by computing the expected diversity based on several types of habitat metadata and compares these to the observed diversity.
    • Also, see the RCA_Tutorial.
  • Requirements:
    • R with the Vegan library must be installed on your system (see the GenGIS manual).
  • Running RCA:
    • Choose the appropriate RCA Model (currently only 'atlantic_rca_model' available). Select the appropriate data labels for Taxon Names and Taxon Counts.
  • Browsing Results:
    • The O/E (Observed over Expected diversity ratios) are displayed in the table for various alpha diversity measures including Richness, Shannon, Simpson, Pielou, and Berker-Parker.
    • Each of these results can be plotted on the main GenGIS map by selecting a column in the table, optionally adjusting the "Bar plot scale factor", and clicking "Plot Selected Data".
    • The data can be exported from the plugin table into GenGIS as another metadata habitat field allowing the use of other plugins (e.g. Linear Regression) by selecting a column and clicking "Add Selected To GenGIS".
    • Lastly, the entire table of results can be saved to a tab-delimited file by using the "Browse" button.
Reference Condition Analysis plugin.