# What is iMaps? iMaps is a unique analysis platform and database for high-throughput sequencing data. Currently analysis of iCLIP data is supported, but this will be extended to other sequencing types in the very near future. On the front-end we provide a user friendly interface to enable code-free analysis of your data. On the back-end the pipelines are open source and written in Nextflow DSL2, so you can know exactly what is happening to your data. If you have any questions that aren't addressed in this documentation please let us know in our [iMaps Slack Workspace](https://join.slack.com/t/imapsgroup/shared_invite/zt-r24y3591-Xbhnym2t38u_urU~I0K0lQ). Additionally, we appreciate all comments and feedback. ## Getting Started Follow our handy guides to get started on the basics as quickly as possible: {% content-ref url="/pages/K3gRugzA9gjpulHUlXrK" %} [Getting Started](/guides/getting-started) {% endcontent-ref %} ### Workflows: Dive a little deeper Learn about the workflows available on iMaps: {% content-ref url="/pages/QCSAT7CxvF7muE5cuoCm" %} [Demultiplex and Analyse](/workflows/iclip/demultiplex-and-analyse) {% endcontent-ref %} # Getting Started Information to get started on iMaps. Last updated 25th May 2022. ### Running your own analysis #### Making an account To run your own analysis on iMaps you will need to make a free account. (Alternatively if you just want to browse publically available data you can "Explore as a guest"). To create an account, fill out the information on the home page: ![](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2FvwzF7rnZszVPgXFijWby%2FScreen%20Shot%202022-02-04%20at%2010.18.24.png?alt=media\&token=aefa5d04-f1d3-4932-9020-86f7f1975edb) You will immediately find yourself in the iMaps interface: ![The iMaps interface.](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2FwyhlrwyiYpSKgwYxAz2l%2FScreen%20Shot%202022-05-25%20at%2011.58.15.png?alt=media\&token=e96dced3-da6c-4ffd-9719-8025c838d0e0) Click on the "Upload Data" link in the left-most menu. You can upload any data here, but for a typical Demultiplex and Analyse run, you will need a Multiplexed FASTQ file and an Annotation spreadsheet (XLSX or CSV). {% hint style="warning" %} Spaces in file names are not permitted. {% endhint %} ![The upload portal.](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2F9FNh1KDSkchpk5CZ9XrL%2FScreen%20Shot%202022-05-25%20at%2012.00.46.png?alt=media\&token=09cb23ce-dfbe-4964-88f6-9e601fd6bfbd) Then, I will upload the annotation for my lane, currently you must save and upload this as csv or xls. You can download the annotation template [here.](https://files.imaps.goodwright.com/media/annotation_template.xlsx) \ Some notes on the annotation - scientist is who did the experiment, both scientist and PI columns need to be the iMaps usernames of these people! These columns are optional in the case that you don't know. The protein/gene name needs to be the official name as found in Ensembl, or one of the other terms allowed in the second spreadsheet of the Excel template. The cell line must also be an official term - you can check the allowed terms in the second spreadsheet in the Excel template. {% hint style="info" %} To request a new term, please post on our Slack workspace in [#request-new-terms](https://app.slack.com/client/T023JE90B3P/C03GX8MSSRH) {% endhint %} \ You might have some errors that you need to correct - once the sheet has been validated it will be uploaded. \ Now to run the main analysis pipeline you need to click on "Run Analysis" in the left-hand menu, followed by the "Demultiplex and Analyse" pipeline. Now you are prompted to add your annotation csv/xls and multiplexed fastq file. \ For every species you are mapping to you will need to select the genome version - most of the time you will choose the Defaults. ![](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2FHYuyVoay7WsurUkOvtYg%2FScreen%20Shot%202022-02-04%20at%2011.07.23.png?alt=media\&token=3c14813f-cd8c-4277-8cbd-56aa1e2a300a) After clicking "Run Command" you will be taken to a page where you can follow the progress of your analysis. ![](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2FGhWFJntxxEqyAq079x80%2FScreen%20Shot%202022-02-04%20at%2011.09.01.png?alt=media\&token=2a074828-ba16-4127-9dd5-7a519ecda28e) # Working with collections # iCLIP For a more in-depth look at CLIP experimental and computational techniques, please visit the [CLIP Forum](https://clipforum.imaps.goodwright.com/) . # Demultiplex and Analyse When you have a multiplexed fastq file your first port of call for analysing iCLIP data will be the "Demultiplex and Analyse" workflow. # Primary Analysis The "primary analysis" workflow is the standard iCLIP analysis pipeline. This document describes each step in the pipeline (a step is called a module in Nextflow), the inputs and the outputs you can expect. ### **1. FASTQC** *Input: demultiplexed reads, Output: FastQC report.* This step runs the program FastQC on your demultiplexed reads file. It's useful to check the html report to see how many reads you have in your file and their general sequencing quality. Read more about FastQC and how to interpret the report [here](https://rtsf.natsci.msu.edu/genomics/tech-notes/fastqc-tutorial-and-faq/) in this helpful article from Michegan State University. ### **2. TRIMGALORE** *Input: demultiplexed reads, Output: trimmed reads, Trim Galore! report* This step runs the program Trim Galore! The purpose is to remove any remaining Illumina sequencing adapter at the 3' end of reads or low quality bases at the end of the reads. The default settings are mostly used, meaning that bases at the 3' end of reads are trimmed if they have Phred score < 20. The first 13 bp of the Illumina adapter 'AGATCGGAAGAGC' are searched for at the end of reads and will be trimmed if even 1 base overlaps - an error rate of 0.1 is allowed and the matching sequence will still be trimmed. Reads that are shorter than 10 bases long after trimming are removed. Read more about Trim Galore! [here](https://www.bioinformatics.babraham.ac.uk/projects/trim_galore/trim_galore_User_Guide_v0.3.7.pdf). ### **3. BOWTIE\_ALIGN** Your demultiplexed reads are aligned against an index of rRNA and mature tRNA using the Bowtie aligner. The purpose of this mapping is twofold: 1) It prevents the contamination of rRNA and tRNA derived reads in your genomic alignment. tRNA genes for example can reside within protein-coding genes, and they are not annotated in most primary genomic annotations meaning if we do not filter them out, we may mistakenly attribute these reads to mRNA. 2) You might discover you have a large amount of tRNA and/or rRNA mapping, which could lead you to reanalyse your data with a more specialised pipeline to quantify this in more detail. We use the parameters "-v 2 -m 100 --norc --best --strata". Let's break down these parameters:
ParameterMeaning
-v 2Allow a maximum of 2 mismatches in valid alignments.
-m 100Only report a read alignment if it has < 100 possible valid alignments.
--norcDo not attempt to map to the reverse of the index sequences (this is because we are providing transcripts).
--best --strataIn the alignment file (sam/bam) only report the best possible alignments. eg. if there is an alignment with 2 mismatches and 3 alignments with 1 mismatch, the alignment with 2 mismatches will not be reported.
--un {sample}_unaligned.fqStore the unaligned reads - we want to use these to map to the genome.
Read more about Bowtie [here](http://bowtie-bio.sourceforge.net/manual.shtml). ### **4. STAR\_ALIGN** | Parameter | Meaning | | --------- | ------- | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # Graphical Overview ![](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2F7t3p8S67BSBcJvXb1lLL%2FFrame%2014.png?alt=media\&token=175902a9-a799-419e-8212-178a01db68c7) ![](https://2121126484-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FYIWy45UkwCiRxhTAx6VW%2Fuploads%2Fb7LbBG4R2hT7ZhY22E8U%2FFrame%2013.png?alt=media\&token=37b4b544-6670-4ac1-abcb-c9f10c8ed22e) # Bioinformatics file types In the world of bioinformatics there are a huge number of file types. This guide aims to help you to understand what these filetypes are and when they are commonly used. Here we provide a short primer so that you can understand the file types you will commonly encounter using iMaps, but also in bioinformatics at large. **fastq** **sam/bam** **bed** **bedgraph** **bigWig** **gtf** **"genome index"** **csv/tsv** # Data In iMaps data refers to a file of some sort (or, occasionally, a folder). These can be reads files, FASTA files, log files - literally anything. There are two ways to create data in iMaps: 1. You can upload data from your computer using the upload page. Some data has special labels associated with it, such as labels indicating that a file represents multiplexed reads, or is an annotation spreadsheet. The particular upload portal you use will handle this for you. 2. You can run a pipeline and generate data outputs. Any data you generate is owned by you (whether by uploading it or by running the pipeline that generates it), with one crucial exception. If you run a pipeline which demultiplexes a multiplexed reads file into multiple demultiplexed reads files, each of those individual reads files (and data downstream of them) may have a different owner, specified in an associated annotation sheet. # Pipelines Pipelines are data analysis programs, typically comprised of multiple bioinformatics modules chained together. iMaps runs on Nextflow, and each pipeline in iMaps is powered by a single DSL2 Nextflow pipeline, which is open source on examinable on GitHub. To run a pipeline, you just need to select it from the list on the pipelines page, and provide it with inputs. The inputs to the pipeline are iMaps Data objects, which can either be newly uploaded Data, or Data which was itself produced by previous pipelines. iMaps will filter the Data available to you by those the pipeline can accept for a given input. In some cases, a pipeline will take as its input not a single Data object, but an Execution object (see next page). In this case it will get access to all the output files of that Execution for that input. This is used when a pipeline needs access to an entire genome description, for example, whose files are generated by the 'Prepare Genome' pipeline. Once everything is ready, you can run the pipeline. The job may sometimes go to a queue first, but in any case every time you run a pipeline you create an Execution, and you will be immediately taken to the page for that Execution. # Executions An Execution is the record of a single run of a Pipeline. It contains all the information about that run - when it was run, who ran it, what data was produced etc. Nextflow pipeline runs consist of a series of invidual processes, which run in their own self contained environment and have their own inputs and outputs. These are chained together to form the entire pipeline execution. You can see all the processes that executed within an Execution (or watch them running live if the run is ongoing). You can also see all the Data that is produced. Executions and Data together form a network, with Data nodes feeding into Execution nodes, which themselves produce more Data nodes. # Samples In Bioinformatics, typically analysis looks like a tree, where if you trace the analysis back it all originates from one original raw data file that came from the actual wet lab experiment. In iMaps these initial raw data files, along with their associated metadata, are used to initialise Samples. Samples can be thought of as a way of keeping all the data from a single sequencing experiment together, and easily traced back to its source. Any Data generated from Sample data will itself belong to that Sample, so you can easily see at a glance the data belonging to a given experiment. iMaps Samples contain metadata describing the scientist who performed the experiment, some experimental conditions, and quality checks of the original file. # Collections Collections are a way of organising Samples by overarching research question, or in whatever way makes sense for you and your lab. Most papers or projects will have multiple wet lab experiments associated with them, and so multiple Samples. Collections are a way of organising these together in a single, shareable resource, with associated papers as a showcase for all the relevant Data. # Genomes Many Bioinformatics workflows require aligning things to genomes, or some kind of genome 'context' in which to operate. iMaps therefore provides reference genomes for common model organisms, which can be easily selected when running these pipelines. The files for a genome are generated by the Prepare Genome pipeline. Each species will have one 'reference genome' - the public, iMaps-provided default version for that species - and there may be others that can be used as well. Anyone can run the Prepare Genome pipeline to generate versions of those genomes that are best suited to your research if the defaults do not suffice. # Permissions Every Data, Execution, Sample and Collection has a single owner - typically the user responsible for creating them. Owners have full access to the object, and have the ability to remove them entirely. In addition, each of these objects can have other users associated, with the ability to access the object, access and edit the object, or access *and* edit *and* share the object with others. These can be controlled by going to the object's page and clicking 'Edit' (which will only appear if you have the relevant permissions). All these objects can also be either private (the default) or public. Public objects can always be viewed by anyone, regarless of whether they have been given access or even if they are signed in or not. In general, permissions cascade downwards - permissions given to an Execution will apply to its produced Data, permissions given to Samples will apply to any Executions and Data run within them, etc. There are some important exceptions though: * If the Execution owner does not own some of the produced Data, that Data will not automatically inhertit permissions from the Execution. * If an Execution is part of a Sample and the owner of the Execution is not the owner of the Sample, then by default it will not inherit permissions from the Execution (though this can easily be enabled). This is to avoid situations where a user may use data from a private Sample that they have been given access to, possibly without realising that everything they in turn produce is part of the Sample, and then get surprised when the original Sample is made public.