FOR RESEARCH USE ONLY.

Transcription

1 Gene Expressin Data Analysis Guidelines Gene Expressin Data Analysis Guidelines Purpse This dcument describes recmmended prcedures fr the analysis f ncunter gene expressin assays. It can serve either as a stand-alne guide fr analyzing ncunter datasets using a third-party sftware package, r it may be utilized as a cmpanin guide t ur wn nslver Analysis Sftware ( Getting Started tab). The fllwing cntent mirrrs the available tls and ptins within nslver, and cntains brief break-ut bxes illustrating hw t implement a subset f these ptins in this Analysis Sftware. Fr a shrter verview f wrkflw withut any accmpanying recmmendatins, refer t the Quick Start Guide t nslver, r, fr a mre cmprehensive guide n the executin f all nslver features, see the full nslver Analysis Sftware User Manual ( Supprt Dcuments tab). Intrductin Gene expressin assays n the ncunter system prvide reliable, sensitive, and highly-multiplexed detectin f mrna targets. These assays are usually perfrmed withut the use f enzymes r amplificatin prtcls, and d nt rely n degree f flurescence intensity t determine target abundance. These characteristics, cmbined with the highly-autmated nature f barcded sample prcessing, result in a platfrm that yields data which is bth precise and reprducible. Nevertheless, it is prudent t fllw prcedures utlined herein t ensure that yu are generating the highest data quality pssible. Figure 1 shws a general wrkflw fr analyzing an ncunter gene expressin experiment. In an iterative prcess, data quality is assessed after every stage f analysis. We start by evaluating the general assay perfrmance using Quality Cntrl (QC) metrics recrded in the ncunter data files (RCC files). Afterwards, we perfrm Backgrund crrectin and data Nrmalizatin fllwed by anther rund f data QC. Finally, we assess the resulting Ratis, Fld-Changes and Differential Expressin, and these results, t, shuld be carefully checked fr quality cntrl characteristics. FOR RESEARCH USE ONLY. Nt fr use in diagnstic prcedures 2017 NanString Technlgies, Inc. All rights reserved. NanString, NanString Technlgies, the NanString lg, ncunter, and nslver are trademarks r registered trademarks f NanString Technlgies, Inc., in the United States and/r ther cuntries

2 Gene Expressin Data Analysis Guidelines MAN-C Figure 1: Quality Cntrl wrkflw fr NanString ncunter Gene Expressin experiments 2

3 Gene Expressin Data Analysis Guidelines Quality Cntrl T ensure that data resulting frm an ncunter Gene Expressin experiment n the ncunter analysis system is f substantial quality t be used in a subsequent in-depth statistical analysis, it is necessary t perfrm a thrugh quality cntrl check n that data. In this sectin, we address Quality Cntrl (QC) fr General Assay Perfrmance. We start checking the verall perfrmance f the ncunter assay by evaluating the Imaging and Binding Density QC metrics. Next, we assess the perfrmance f the psitive cntrls using the Psitive Cntrl Linearity and Limit f Detectin parameters. Finally, we d an verall visual inspectin f the data and assess the severity f any QC flags. Imaging QC Each individual lane scanned n an ncunter system is divided int a few hundred imaging sectins, called Fields f View (FOV), the exact number f which will depend n the system being used (i.e., MAX/FLEX r SPRINT), and the scanner settings selected by the user. The system images these FOVs separately, and sums the barcde cunts f all FOVs frm a single lane t frm the final raw data cunt fr each unique barcde target. Finally, the system reprts the number f FOVs successfully imaged as FOV Cunted. Significant discrepancy between the number f FOV fr which imaging was attempted (FOV Cunt) and fr which imaging was successful (FOV Cunted) may indicate an issue with imaging perfrmance. Recmmended percentage f registered FOVs (i.e. FOV Cunted ver FOV Cunt) is 75%. Lanes will be flagged if this percentage is lwer. Factrs Affecting Imaging Cartridge surface: fingerprints r dust particles n the cartridge shuld be wiped ff. Cartridge placement: fr MAX/FLEX instruments, a slightly tilted cartridge can lead t lw FOV registratin. Rescan the cartridge after ensuring that it is firmly seated in the imaging slt. FOV Cunted (Fields f View successfully cunted) FOV Cunt (Fields f View attempted) = at least 75% 3

4 Gene Expressin Data Analysis Guidelines MAN-C Binding Density QC The imaging unit nly cunts the cdes that are unambiguusly distinguishable. It simply will nt cunt cdes that verlap within an image. This prvides increased cnfidence that the mlecular cunts yu receive are frm truly recgnizable cdes. Under mst cnditins, frging the few barcdes that d verlap will nt impact yur data. T many verlapping cdes in the image, hwever, will create a cnditin called image saturatin in which significant data lss culd ccur (critical data lss frm saturatin is uncmmn; see Assessing QC Flags, belw). T determine the level f image saturatin, the ncunter instrument calculates the number f ptical features per square micrn fr each lane as it prcesses the images. This is called the Binding Density. The Binding Density is useful fr determining whether data cllectin has been cmprmised due t image saturatin. With nslver 4.0 (if yu use a versin ther than 4.0, cnsult the user manual fr that versin), the acceptable range fr Binding Density is: fr MAX/FLEX instruments fr SPRINT instruments Within these ranges, relatively few reprters n the slide surface will verlap, enabling the instrument t accurately tabulate cunts fr each reprter species. A Binding Density significantly greater than the upper limit in either range is indicative f verlapping reprters n the slide surface. The cunts bserved in lanes with a Binding Density at this level may have had significant numbers f cdes ignred, which culd ptentially affect quantificatin and linearity f the assay. Sme f the factrs that may cntribute t increased Binding Density are listed in the Factrs affecting Binding Density bx. Factrs Affecting Binding Density Assay input quantity: the higher the amunt f input used fr the assay, the higher the Binding Density will be. The relatinship between input amunt and Binding Density is linear until the pint f assay saturatin. Expressin level f genes: if the target genes have high expressin levels, there will be mre mlecules n the lane surface which will increase the Binding Density value. Size f the CdeSet: a large CdeSet with prbes fr many targets is mre likely t have high Binding Density values than a CdeSet with prbes fr significantly fewer targets. 4

5 Gene Expressin Data Analysis Guidelines Psitive Cntrl Linearity QC Six synthetic DNA cntrl targets are included with every ncunter Gene Expressin assay. Their cncentratins range linearly frm 128 fm t fm, and they are referred t as POS_A t POS_F, respectively. These psitive cntrls are typically used t measure the efficiency f the hybridizatin reactin, and their step-wise cncentratins als make them useful in checking the linearity perfrmance f the assay. An R 2 value is calculated frm the regressin between the knwn cncentratin f each f the psitive cntrls and the resulting cunts frm them (this calculatin is perfrmed using lg 2-transfrmed values). R 2 = linear regressin f Lg 2 Knwn [psitive cntrls] Lg 2 Measured [psitive cntrls] > 0.95 Since the knwn cncentratins f the psitive cntrls increase in a linear fashin, the resulting cunts shuld, as well. Therefre, R 2 values shuld be higher than Nte that because POS_F has a knwn cncentratin f fm, which is cnsidered belw the limit f detectin f the system, it shuld be excluded frm this calculatin (althugh yu will see that POS_F cunts are significantly higher than the negative cntrl cunts in mst cases). Factrs Affecting POS Cntrl Linearity Prbe vs sample interactins: Uncmmnly, a single psitive cntrl prbe will shw unusually high r lw cunts, usually because f an interactin between this prbe and a sample, which will nt manifest during NanString manufacturing QC. This will usually have little t n effect n data quality. Very lw assay efficiency: If the cunts f all the psitive cntrls are very lw (less than ~500 even fr POS_A), linearity will be cmprmised. This is usually the result f a critical assay failure and data will likely be cmprmised. 5

6 Gene Expressin Data Analysis Guidelines MAN-C Limit f Detectin QC The limit f detectin is determined by measuring the ability t detect POS_E, the 0.5 fm psitive cntrl prbe, which crrespnds t abut 10,000 cpies f this target within each sample tube. On a FLEX/MAX system, the standard input f 100 ng f ttal RNA will rughly crrespnd t abut 10,000 cell equivalents (assuming ne cell cntains 10 pg ttal RNA n average). An ncunter assay run n the FLEX/MAX system shuld thus cnservatively be able t detect rughly 1 transcript cpy per cell fr each target (r 10,000 ttal transcript cpies). In mst assays, yu will bserve that even the POS_F prbe (equivalent t 0.25 cpies per cell) is detectable abve backgrund. T determine whether POS_E is detectable, it can be cmpared t the cunts fr the negative cntrl prbes. Every ncunter Gene Expressin assay is manufactured with eight negative cntrl prbes that shuld nt hybridize t any targets within the sample. Cunts frm these will apprximate general nn-specific binding f prbes within the samples being run. The cunts f POS_E shuld be higher than tw times the standard deviatin abve the mean f the negative cntrl. Factrs Affecting Limit f Detectin T high backgrund may be due t: Premature mixing f reprter and capture prbes in hybridizatin buffer master mix. T much time elapsed between adding capture prbe and lading in thermal cycler. High cunts in ne f the negative cntrls NEG cntrls culd be elevated due t crss hybridizatin with targets in the sample. Very lw POS cunts may be due t: Sub-ptimal hybridizatin check thermal cycler temperature and cnsider whether sample impurities (chatrpic salts, fr example) may have been intrduced. Hw t D It In nslver: General Assay Perfrmance QC Imprt unzipped data files using the Imprt RCC Files buttn. Fllw the prmpts f the RCC Imprt Wizard t the Run QC windw. Use the duble arrw t view the System QC settings. The RCC Imprt Wizard will assign the default settings fr System QC as well as mrna QC as shwn in Figure 2. Figure 2: nslver GX QC windw 6

7 Gene Expressin Data Analysis Guidelines Assessing QC Flags The recmmended quality cntrl specificatins fr NanString data are designed t highlight samples that have mderate-t-severe reductins in assay efficiency r data quality. Sme samples that d nt pass quality cntrl may still prvide rbust cunt data, but they are singled ut t highlight a ptential cause f pr assay efficiency that culd result in data lss in future assays. Even if the data frm these samples is usable, it is a gd practice t determine the rt cause f all such detectable changes in assay efficiency. Fr tips n hw t assess these rt causes, see the Factrs Affecting bxes fr each QC parameter sectin, abve. NanString Technical Services (supprt@nanstring.cm) and/r yur Field Applicatin Scientist can als assist yu with trubleshting, if necessary. Samples with lwered assay efficiency (r with reduced quality r quantity f RNA) will als exhibit lwered raw cunts fr prbes in the NanString CdeSet. Depending n hw backgrund crrectin and nrmalizatin is perfrmed, lw expressing targets in these samples will either fail t be detected (false negatives) r will have higher-than-expected cunts (false psitives). T determine whether it wuld be necessary t discard the data frm flagged samples, see the sectins belw fr sme rugh guidelines t use. Examine the Psitive and Negative Cntrls If samples were flagged because a single psitive r negative cntrl prbe was much different than expected (leading t psitive cntrl linearity flag r Limit f Detectin flag), it is very likely the data frm that sample is still usable. If all the psitive cntrls are extremely lw, such that mst (if nt all) are clse t the backgrund (negative cntrl prbe cunts), assay linearity and/r sensitivity are likely cmprmised and such samples may nt prvide rbust data. Examine the Calculated QC Metrics If the QC metrics fail the flag threshld by a very small margin (i.e., FOV registratin was 72%, just belw the 75% recmmended cut ff), the data is likely rbust and usable. Cnversely, samples with QC metrics which miss the cutff by a wide margin (i.e., FOV registratin was 11% instead f >75%), will usually nt prvide rbust data. NOTE: Binding density flags seldm result in critical data lss; use the Visually Inspect the Data and the Identify Outlier Samples sectins belw, especially when the Binding Density threshld has been exceeded by a cnsiderable margin. 7

8 Gene Expressin Data Analysis Guidelines MAN-C Visually Inspect the Data A simple inspectin f the data may either identify prblematic samples r give greater cnfidence that data fr the sample in questin is sufficiently rbust fr dwnstream inclusin. Samples may need t be excluded if an inspectin f raw cunts indicates that substantially fewer genes are being detected abve the backgrund (negative cntrl) prbes cmpared t ther samples frm a similar treatment grup. Samples may need t be excluded if an inspectin f nrmalized cunts indicates that substantially mre genes are being detected abve the backgrund prbes cmpared t ther samples frm the same treatment grup. NOTE: The Backgrund Threshlding ptin in nslver will prevent these false psitives frm shwing up; this apprach is nt recmmended if Backgrund Threshlding has been perfrmed n the data. Identify Outlier Samples Instead f a visual inspectin, varius methds may be used t identify utlier samples, which may be indicative f assay- r sample-level prblems which necessitate remval f such data frm dwnstream analyses. One way t identify utlier samples is t generate a heat map f nrmalized data frm all samples. If the flagged samples in questin are strngly divergent frm ther samples with similar pathlgy, they may need t be remved frm further analyses. In nslver, yu will need t prceed thrugh the steps t create an experiment befre ding a heat map Analysis, at which pint yu can refer t the Agglmerative Cluster (Heat Map) sectin in the nslver User Manual. Hw t D It In nslver: Assessing QC flags Highlight yur CdeSet n the Raw Data tab and scrll acrss the central table t view varius QC flag clumns. Right-click n the clumn headers r select the Clumn Optins icn t reveal hidden clumns, as the QC metric clumns may be initially hidden. Alternatively, r in additin, nce yu have created an experiment, yu can select it n the Experiments tab and select the Experiment Reprt buttn t generate a reprt. 8

9 Gene Expressin Data Analysis Guidelines Backgrund Crrectin Despite the high specificity f ncunter prbes, very lw levels f nn-specific cunting are still inherent t any NanString assay. Thus, a small number f cunts fr each f the targets in a CdeSet will represent false psitives. Typically, we d nt recmmend crrecting fr nn-specific cunting since true cunts fr mst mrna targets will far utnumber false psitives, and thus the effects f the latter n fld-change estimates will be negligible. Hwever, fr thse prjects where lw expressing targets are cmmn, r when the presence r absence f a transcript has an imprtant research implicatin, it may be useful t mre precisely delineate which cunts are false psitives. Fr these reasns, we intrduce tw methds f backgrund crrectin: Backgrund Threshlding and Backgrund Subtractin. We als describe three different appraches in establishing a value t use in cnjunctin with threshlding r subtractin: using the Negative Cntrl Prbes, a Fixed Backgrund, r a Negative Cntrl Sample. Backgrund Threshld Backgrund threshlding is a prcess whereby a threshld value is set and all cunts which fall belw that value are adjusted t match it. When data is backgrund subtracted (see next sectin), in cntrast, nnspecific cunts are subtracted frm the raw cunts t btain a new estimate f cunts abve backgrund. Fr mst studies f gene expressin where fld-change estimatins are an imprtant result, backgrund threshlding is preferred ver backgrund subtractin. This is because the latter tends t substantially verestimate fld-changes in lw-expressing targets; backgrund subtracted cunts fr lw-expressing targets can be clse t zer, ptentially setting either the numeratr r denminatr f the fld-change calculatin t a very small number. The bilgical imprtance f large abslute fld-changes in backgrund subtracted data may be verestimated when the expressin level f the gene is clse t the backgrund. Backgrund Subtractin If, instead f fld-changes, estimates f cunted transcripts abve backgrund nise are an imprtant result, then yu may cnsider using backgrund subtractin. Cnsider, hwever, the fllwing cautinary ntes abut backgrund subtractin values in nslver: Because the nslver math engine perates in lg 2 space, any cunts belw 1 (including negative cunts) will be reset t 1.0. Thus, cunts which appear as 1.0 in nslver utput will frequently represent cunts less than 1. Backgrund subtracted values may subsequently be multiplied by nrmalizatin scaling factrs. Particularly fr samples with nrmalizatin flags, sme targets belw backgrund will be nrmalized t ptentially large values depending n the size f the scaling factrs, and thus may appear t be much higher than backgrund. Cunts in nslver which have been set t the backgrund threshld will nt be altered further during nrmalizatin. 9

10 Gene Expressin Data Analysis Guidelines MAN-C Negative Cntrl Prbes Every ncunter CdeSet r TagSet is manufactured t cntain 6 r 8 negative cntrl prbes, designated by the prbe name, Negative A (NEG_A), Negative B (NEG_B), etc. These cntrl prbes are designed against engineered RNA sequences (ERCC RNA cntrls) which are nt present in bilgical samples. Negative cntrl prbes are cmmnly used t set backgrund threshlds. Every prbe in an ncunter assay will exhibit sme variability in nn-specific cunting. By bserving this variability amng the negative cntrl prbe targets, we can predict the level f variability in baseline nnspecific cunts that we expect t see fr the endgenus and husekeeping targets. We can use the mean, mean plus standard deviatin, median, gemetric mean, r maximum f the negative cntrl cunts. The level f stringency used in setting this threshld will affect the balance between false psitive and false negative target ccurrences. Examples f metrics yu can use and their ptential implicatins include: Yu can minimize the number f false psitives by setting the threshld t the mean plus tw standard deviatins r t the maximum f the negative cntrl cunts; nte, hwever, that althugh false psitives will be rare, false negatives may be relatively abundant. Cnversely, yu can set a mre liberal threshld, such as the gemetric mean f the negative cntrls. This will increase the number f false psitives, but simultaneusly decrease the number f false negatives. Rare Negative Cntrl Deviatins Rarely, unique and difficult-t-predict prbe-t-prbe interactins may ccur, resulting in ne r tw negative cntrl prbes being mre than 3-fld higher than all the ther negative cntrl prbes. It is acceptable t remve these utliers fr purpses f backgrund threshld r backgrund subtractin. Fixed Value Backgrund Anther apprach t setting the backgrund is t simply define a fixed value. This wuld nly be apprpriate if previus experiments had identified a rbust measure f prbe nn-specific binding that met the prject qualificatins fr target exclusin. Used in cnjunctin with backgrund threshlding, all targets with raw cunts equal t r less than this defined value wuld be set t this value. Used with backgrund subtractin, this defined value wuld be subtracted frm the raw cunts f all targets. 10

11 Gene Expressin Data Analysis Guidelines Negative Cntrl Sample One limitatin f the previusly-described appraches (negative cntrls r a fixed value) is that sme prprtin f false psitives r false negatives are expected, given the variance in nn-specific cunting acrss prbes. One apprach t defining backgrund mre precisely wuld be t run samples which cntain n target transcripts, such as an RNA-free water sample. This wuld require using ne r mre sample lanes per prject (nt per cartridge) fr these negative cntrl samples in lieu f experimental samples. Ideally, three negative cntrl samples wuld be run fr each build f a CdeSet, and a t-test perfrmed fr each target transcript. These three negative cntrl replicates wuld then be cmpared t the bilgical replicates run within each f the ther treatment grups. Hw t D It In nslver: Backgrund Threshlding Create an experiment and fllw the Experiment Wizard prmpts t the Backgrund Subtractin/Threshlding screen. Check the bx fr Backgrund Subtractin/Threshlding. Check the bx fr Negative cntrl cunt. Set the Threshld t mean +2 standard deviatins abve the mean. Figure 3: nslver backgrund settings windw 11

12 Gene Expressin Data Analysis Guidelines MAN-C Nrmalizatin Data nrmalizatin is designed t remve surces f technical variability frm an experiment, s that the remaining variance can be attributed t the underlying bilgy f the system under study. The precisin and accuracy f ncunter Gene Expressin assays are dependent upn rbust methds f nrmalizatin t allw direct cmparisn between samples. There are many surces f technical variability that can ptentially be intrduced int ncunter assays. The largest and mst cmmn categries f variability riginate frm either the platfrm r the sample. We describe hw bth types f variability can be nrmalized using standard nrmalizatin prcedures fr Gene Expressin assays. Standard nrmalizatin uses a cmbinatin f Psitive Cntrl Nrmalizatin, which uses synthetic psitive cntrl targets, and CdeSet Cntent Nrmalizatin, which uses husekeeping genes, t apply a sample-specific crrectin factr t all the target prbes within that sample lane. These crrectin factrs will cntrl fr all thse technical surces f variability which shuld affect all the prbes equally, cmpensating fr surces f errr including pipetting errrs, instrument scan reslutin, lt-t-lt variatin in Prep Plates and Cartridges, and sample input variability. These prcedures are effective as lng as the CdeSets r prbes themselves are derived frm the same manufacturing lt (if nt, see the Lt t Lt Variatin in Prbes bx). Nte that psitive cntrl nrmalizatin will nt crrect fr sample input variability, and thus shuld usually be used in cmbinatin with CdeSet Cntent (husekeeping gene) Nrmalizatin. Perfrming such a tw-step nrmalizatin will usually nt differ mathematically frm Cntent Nrmalizatin alne, and thus is mathematically smewhat redundant. Nevertheless, nrmalizing t bth target classes will prvide a gd indicatr f hw technical variability is partitined between the tw majr surces f assay nise (platfrm and sample), and thus may prvide a gd tl fr trubleshting lw assay perfrmance. Belw, we describe bth types f nrmalizatin, as well as Nrmalizatin QC. Lt t Lt Variatin in Prbes Due t the level f precisin and sensitivity f the ncunter platfrm, it will likely detect any differences in the cunting efficiency between lts f ncunter prbes manufactured at different times. Hwever, unlike batch effects fr ther reagents, the prbe efficiencies frm new lts f prbes will ptentially differ in magnitude and directin fr each prbe. Althugh the size f this change is generally small (<30% fr each prbe), a sample-wide nrmalizatin will nt crrect fr this extra nise. The ptimal slutin fr crrecting lt t lt variatin in CdeSets is t run reference samples within each lt f CdeSet, and use the cunts frm these t mathematically crrect the prbe efficiency changes. Please cnsult with yur Applicatins Scientist r supprt@nanstring.cm fr help in ptimizing this and ther appraches fr cntrlling fr this variance. 12

13 Gene Expressin Data Analysis Guidelines Psitive Cntrl Nrmalizatin ncunter Reprter prbe (r TagSet) tubes are manufactured t cntain six synthetic ssdna cntrl targets. The cunts frm these targets may be used t nrmalize all platfrm-assciated surces f variatin (e.g., autmated purificatin, hybridizatin cnditins, etc.). Hw t D It In nslver: Psitive Cntrl Nrmalizatin Figure 1 Default Figure recmmendatins 1 Default fr psitive recmmendatins fr psitive Check the Psitive Cntrl Nrmalizatin bx within the Experiment Wizard. (ptinal) If the cunts fr the POS_F cntrl are clse t backgrund, then exclude this prbe frm nrmalizatin (Uncheck the bx as shwn n the right). Select the gemetric mean t cmpute nrmalizatin factrs. This will weigh the targets apprximately equally in the ensuing calculatins regardless f their abslute cunts. Figure 4: nslver Psitive Cntrl Nrmalizatin settings Althugh the calculatins are perfrmed autmatically in nslver, the prcedure is as fllws (see Figure 5): 1. Calculate the gemetric mean (Gemean) f the psitive cntrls fr each lane. 2. Calculate the arithmetic mean f these gemetric means fr all sample lanes. 3. Divide this arithmetic mean by the gemetric mean f each lane t generate a lane-specific nrmalizatin factr. 4. Multiply the cunts fr every gene by its lanespecific nrmalizatin factr. Raw Data Sample 1 Sample 2 Sample 3 Psitive POS_A ERCC_ Psitive POS_B ERCC_ Psitive POS_C ERCC_ Psitive POS_D ERCC_ Psitive POS_E ERCC_ Psitive POS_F ERCC_ Gemean f POS: Arithmetic mean f gemeans: POS cntrl nrmalizatin factrs: Figure 5: spreadsheet calculatins fr psitive cntrl nrmalizatin 13

14 Gene Expressin Data Analysis Guidelines MAN-C CdeSet Cntent r Husekeeping Gene Nrmalizatin It is expected that sme nise will invariably be intrduced int the ncunter assay due t variability in sample input. Since it is nt always practical t measure real wrld samples with the precisin and accuracy necessary t ensure that each lane cntains the same amunt and quality f RNA, it is desirable t crrect fr this variability. We can d this by nrmalizing t a surrgate measure f ttal RNA input int the assay. Fr mst experiments run n the ncunter platfrm, this is mst effectively dne using s-called husekeeping genes; these are mrna targets included in a CdeSet which are knwn t r suspected t shw little-t-n variability in expressin acrss all treatment cnditins in the experiment. Because f this, these targets will ideally vary nly accrding t hw much sample RNA was laded. Hw t D It In nslver: CdeSet Cntent Nrmalizatin Check the CdeSet Cntent Nrmalizatin bx within the Experiment Wizard. Check the Standard ptin. If husekeeping genes have been identified fr the CdeSet being analyzed, these will autmatically be mved int the table Nrmalizatin Cdes. If husekeeping genes were nt identified during CdeSet design, they will be classified as Endgenus genes and will need t be manually added t the Nrmalizatin Cdes table via the green arrw buttns. Using the gemetric mean f at least three husekeeping genes t calculate nrmalizatin factrs is recmmended t minimize the nise frm individual genes as well as t ensure that the calculatins are nt weighted twards the highest expressin husekeeping targets. It is imprtant t nte that sme previusly-identified husekeeping genes may, in fact, behave prly as nrmalizing targets in the current experiment, and may therefre need t be excluded frm nrmalizatin (see sectin belw, Selecting Husekeeping Genes). Figure 6: nslver CdeSet Cntent Nrmalizatin settings 14

15 Gene Expressin Data Analysis Guidelines The calculatins are perfrmed autmatically in nslver, which fllws this prcedure (fr an example f this prcedure, see the Psitive Cntrl Nrmalizatin sectin and Figure 5, abve): 1. Calculate the gemetric mean f the selected husekeeping genes fr each lane. 2. Calculate the arithmetic mean f these gemetric means fr all sample lanes. 3. Divide this arithmetic mean by the gemetric mean f each lane t generate a lane-specific nrmalizatin factr. 4. Multiply the cunts fr every gene by its lane-specific nrmalizatin factr. 15

16 Gene Expressin Data Analysis Guidelines MAN-C Nrmalizatin QC It is imprtant t perfrm sme Quality Cntrl n nrmalized ncunter data t ensure that nrmalizatin has nt inadvertently intrduced variability int the data set. The scaling factrs generated by Psitive Cntrl and Husekeeping (CdeSet Cntent) Nrmalizatin can be used as surrgate measures fr ptential bias which may be intrduced by these prcedures. A Psitive Cntrl scaling factr fr a sample that exceeds 3-fld (that is, it is less than 0.3 r greater than 3.0), may skew results frm the samples, particularly fr thse lw expressin targets which are clse t the backgrund nise. Scaling factrs in these extreme ranges mst cmmnly result frm samples with psitive cntrl cunts that are much lwer than nrmal. Even if these samples appear therwise nrmal, psitive cntrls that are lwer than expected may indicate a reductin in assay efficiency. A CdeSet Cntent nrmalizatin scaling factr that exceeds 10-fld (less than 0.1 r greater than 10.0) indicates that husekeeping genes have cunts that are much lwer than average fr that sample. The safest curse wuld be t exclude samples with Nrmalizatin factrs utside the expected range frm further analysis, althugh samples that yielded marginal QC specs (i.e., a 12.0 fr CdeSet Cntent Nrmalizatin) are ften still usable. Even fr samples with nrmalizatin flags, data frm mderate- t highly-expressed targets will ften shw little bias. Mre cncerning are the data frm lw cunt targets; these can becme biased upwards fr samples with nrmalizatin flags, and their inclusin may ptentially skew the results f a study. The use f backgrund threshlding in nslver will minimize this upward bias in lw cunt genes, but samples with nrmalizatin flags will still have a reduced ability t detect lw expressin genes. See the Assessing QC Flags sectin t determine if these samples are utliers in the nrmalized dataset. Selecting Husekeeping Genes Nt every pre-selected husekeeping gene will necessarily crrelate well with sample input. Even wellvalidated husekeepers may shw increased variability when tested under nvel experimental cnditins, new pathlgies, r new tissue types. Therefre, it is gd practice t critically examine the data frm putative husekeeping genes and exclude any targets with ntable instability in expressin. Indicatins that yu may need t discard a husekeeper include: A husekeeping target that is expressed at r near the backgrund; this will likely intrduce nise int the nrmalized data. A husekeeping target with a higher variability than ther targets (as measured by %CV; this clumn can be seen in Figure 6 abve, Hw t D It In nslver: CdeSet Cntent Nrmalizatin). A husekeeping target whse expressin des nt crrelate with ther husekeepers when cmpared acrss samples. The nslver Advanced Analysis mdule includes a published methdlgy and algrithm (genrm) t frmally assess these characteristics and identify the ptimal husekeepers fr the current data set. See the Advanced Analysis User Manual fr mre infrmatin n ptimizing nrmalizatin. 16

17 Gene Expressin Data Analysis Guidelines Assessing a Nrmalizatin QC Flag POS Cntrl Flag: Samples with a POS Cntrl Nrmalizatin flag have lwer than typical POS cntrl cunts, and may be experiencing a reduced level f assay level efficiency. These samples will als smetimes receive an Assay Perfrmance QC flag, due t factrs such as: failed imaging QC, high binding density, a failed pipetting step, r lw hybridizatin efficiency, which shuld aid in determining the rt cause f the lwered efficiency. See the Assessing QC Flags sectin, abve, t determine hw t treat these samples. Cntent (Husekeeping) Nrmalizatin Flag: Samples with a Cntent Nrmalizatin flag typically have very lw husekeeping gene cunts, and thus likely have very lw amunts f intact RNA. This may result frm pipetting errrs, inaccurate quantificatin, r sample degradatin. Fr help in determining the rt cause f lw RNA cunts, cnsult with NanString supprt (supprt@nanstring.cm), but als refer t the Assessing QC Flags sectin, abve, t determine if the data frm these samples shuld still be included in further analyses. Hw t D It In nslver: Nrmalizatin QC nslver will autmatically cmpare nrmalizatin factrs fr POS cntrls and Husekeeping genes t the default cutffs mentined in the text (3-fld and 10-fld, respectively). T determine whether samples received nrmalizatin flags: Select the Nrmalized Data table in the finished experiment (see Figure 7). Scrll right in the central table viewer: there are tw clumns reserved fr ptential nrmalizatin flags, then tw mre fr the nrmalizatin crrectin factrs. Figure 7: Nrmalizatin QC in nslver 17

18 Gene Expressin Data Analysis Guidelines MAN-C Ratis, Fld-Changes, and Differential Expressin Fr many experiments that quantify mrna expressin levels, a primary end gal is t characterize changes in gene expressin acrss samples that differ in sme key characteristic (i.e., different pathlgy r treatment). Here, we describe the prcess f differential expressin analysis in a NanString gene expressin dataset. We will als highlight imprtant cnsideratins in experimental design and hw they may impact the ability t determine differential expressin. An experiment that is designed t include bilgical replicates is amenable t frmal statistical testing, as it maximizes bilgical inference f expressin differences between sample grups. In cntrast, experiments with ne replicate per cnditin are limited in terms f bth statistical testing ptins and bilgical interpretatin. Technical replicates are nt required, since the enzyme-free, digital nature f the assay generates exceptinally precise and reprducible measurements. Bilgical replicates, therefre, are strngly recmmended. In the fllwing sectins, we discuss creating ratis and fld-change estimates, and fr thse experiments with sufficient bilgical replicates, we discuss apprpriate statistical tests and multiple testing crrectin. Rati and Fld-Change Calculatins Nrmalized NanString gene expressin data is mst cmmnly analyzed in terms f ratis r fld-changes. These calculatins are used t determine relative ver- r under-expressin f each gene acrss different sample grups in an experiment. These expressins can be calculated in either linear r lg 2 scale. Here, we illustrate hw t calculate ratis using a hypthetical experiment with three grups f samples: thse in a cntrl grup, thse in grup 1, and thse in grup 2. Calculating linear ratis fr a single gene in this data set is shwn in Figure 8. Overexpressed genes will have ratis >1, but under-expressed genes will have a rati which ranges in value frm 0 1 and are therefre cmpressed in the linear scale. Cmparisn Linear Rati Calculatin Interpretatin Grup #1 vs. Cntrl Grup #2 vs. Cntrl Grup #1 vs. Grup #2 Rati = Rati = Rati = gemetric mean f gene A frm grup #1 gemetric mean f gene A frm cntrl grup gemetric mean f gene A frm grup #2 gemetric mean f gene A frm cntrl grup gemetric mean f gene A frm grup #1 gemetric mean f gene A frm grup #2 Relative ver- r under-expressin f gene A in grup #1 vs. cntrl Relative ver- r under-expressin f gene A in grup #2 vs. cntrl Relative ver- r under-expressin f gene A in grup #1 vs. grup #2 Figure 8: Linear rati calculatins 18

19 Gene Expressin Data Analysis Guidelines Althugh ratis are relatively easy t calculate, these are cmmnly cnverted int lg 2 ratis, fld-changes, r lg 2 fld-changes. These transfrmed metrics are generally mre intuitive t interpret since their distributins are symmetrical. Calculatins fr ratis in lg 2 scale are shwn in the table belw, and are mathematically identical t lg 2 fld-changes. Cmparisn Lg2 Rati (= lg2 fld-change) Interpretatin Grup #1 vs. Cntrl Grup #2 vs. Cntrl Grup #1 vs. Grup #2 (arithmetic mean f lg2 cunts f gene A in grup #1) (arithmetic mean f lg2 cunts f gene A in cntrl grup) (arithmetic mean f lg2 cunts f gene A in grup #2) (arithmetic mean f lg2 cunts f gene A in cntrl grup) (arithmetic mean f lg2 cunts f gene A in grup #1) (arithmetic mean f lg2 cunts f gene A in grup #2) Relative ver- r under-expressin f gene A in grup #1 vs. cntrl Relative ver- r under-expressin f gene A in grup #2 vs. cntrl Relative ver- r under-expressin f gene A in grup #1 vs. grup #2 Figure 9: Lg 2 rati calculatins Unlike in linear data, these lg 2 metrics are determined by subtracting, rather than dividing, grup means. They are mrever bth symmetrical arund zer. Thus, fr example, a lg 2 rati f 0.8 has the same magnitude as a lg 2 rati f It is imprtant t nte that ratis calculated using the gemetric mean f linear data crrespnd t ratis calculated using the arithmetic mean f lg 2-transfrmed data. A simple mathematical frmula can be applied t cnvert between the tw types f ratis. T cnvert lg 2 rati y t linear rati x, use the antilg functin such that: x = antilg 2(y), which is equivalent t x = 2 y. Cnversely, linear rati x can be cnverted t lg 2 rati y with the fllwing frmula: y = lg 2(x). Fld-changes are prbably the mst cmmnly displayed metric fr measures f differential expressin, and are readily calculated frm linear rati data. If the rati numeratr is greater than the rati denminatr (i.e., rati > 1), then fld-change is equal t the rati. If the rati numeratr is less than the rati denminatr (i.e., rati < 1), then fld-change equals the negative inverse f the rati: If rati > 1, then: Fld change = Rati If rati < 1, then: 1 Fld change = 1 Rati The distributin f fld-change is symmetrical arund zer, with all verexpressed genes being represented as values > 1 and all under-expressed genes being represented as values <-1 (thus there are n fld-changes pssible between -1 and 1). 19

20 Gene Expressin Data Analysis Guidelines MAN-C Statistical Tests In this sectin, we will cver a few basic recmmendatins fr statistical testing f differential expressin in ncunter data sets. It is beynd the scpe f this dcument t exhaustively cver the varius appraches that are apprpriate fr making statistical inferences frm gene expressin studies, s we will fcus primarily n the ptins fr testing which can be fund in the nslver analysis sftware. As stated previusly, a minimum f three bilgical replicates are required frm each experimental grup t btain reasnably reliable statistical infrmatin abut the underlying ppulatins frm which thse samples were drawn. If the magnitude f between-grup differences is small, r if variability within grups is high, then mre replicates per grup may still be required t btain statistical significance. T-test A t-test can be perfrmed n lg 2-transfrmed cunt data if there are nly tw grups f samples t cmpare. An example f this wuld be a simple experiment cnsisting f cntrl grup samples that yu wuld like t cmpare t samples in a single treatment grup. The lg 2 transfrmatin will generally satisfy the t-test s requirement f nrmally distributed data, and the use f a heterscedastic t-test (Welch s) als relaxes the assumptin that variance is equally distributed amng grups. The utput f a t-test is a p- value, and standard cnventin is t assign significance t all genes with a nminal p-value f less than The lwer the p-value, the strnger the evidence that a gene has different expressin levels in tw different grups. It is imprtant t nte that a p-value is the prbability f btaining an effect at least as extreme as the ne bserved, assuming the null hypthesis f n difference between sample grups is true. The p-value addresses the questin, hw likely is the data?, assuming the null hypthesis is true. It is a cmmn mistake fr a specific p-value t be incrrectly interpreted as the prbability f a false psitive; this is called a Type I errr. If desired, pst-hc crrectins can be applied t adjust p-values t accunt fr multiple testing (see belw). Multivariate Linear Regressin Nrmalized and lg 2-transfrmed ncunter data may als be analyzed using multivariate linear regressin. This is a pwerful apprach fr dealing with mre cmplex experimental designs than a tw-grup experiment. It allws ne t islate the independent effect f each cvariate n gene expressin levels, permitting cmparisns between multiple experimental grups and variables, including ptentially cnfunding variables. Thugh the details are beynd the scpe f this dcument, guidance n hw t perfrm these tests in the nslver Advanced Analysis mdule may be fund in the Advanced Analysis User Manual. 20

21 Gene Expressin Data Analysis Guidelines False Discvery Rate and Crrecting fr Multiple Cmparisns In large, multiplexed gene expressin datasets, the large number f statistical tests perfrmed may inadvertently increase the chances f btaining false-psitive results. This is because, all else being equal, 1 ut f every 20 statistical tests is expected by chance alne t result in a p-value f apprximately 0.05, the traditinal threshld fr btaining statistical significance. The frequency f these ptential false psitives is expected t g up with increasing numbers f tested genes, s fr larger NanString CdeSets, this may becme a ptential cncern. Despite this pssibility, we d nt necessarily recmmend a universal applicatin f multiple cmparisn crrectins fr NanString datasets. First, CdeSets generally cmprise cllectins f genes knwn r suspected t be affected by the bilgy under study, and the resulting crrelatins in expressin levels vilates the assumptin f independence f significance levels between tests, which frms the basis f the mre cnservative Bnferrni multiple cmparisns tests. As a crllary, if the purpse f the NanString study is t validate the results f a transcriptme-wide survey, the CdeSet will, just as abve, cmprise a highly nn-randm set f genes. Fr such validatin wrk, it may nt be desirable at this stage t limit the number f false psitives. Arguably, instead f statistical adjustments t significance levels, ne might cnsider crss-platfrm validatin as a better standard twards determining the statistical rbustness f any expressin level differences. Lastly, the use f multiple cmparisns crrectins may simply be t cnservative fr experiments where the tlerance fr false negatives is lw. Statistical methds designed t reduce the number f false psitives will invariably result in an increase in the number f false negatives, and fr sme experimental gals, this extra level f stringency may nt be an acceptable tradeff. In general, when multiple cmparisn crrectins are perfrmed n NanString data, using a less cnservative methd which allws gene significance levels t be either psitively r negatively crrelated is preferred (i.e., when prbing fr crrelated and interdependent expressin pathways). The Benjamini- Yekutieli False Discvery Rate methd 1 accunts fr this expectatin that significant changes in genes may be crrelated with r dependent n each ther, and the resulting FDR adjusted p-value prvides a middle grund between the mre cnservative tests which cntrl fr family-wise errr rates (epitmized by tests like the Bnferrni crrectin) and mre permissive uncrrected p-values. 1 The Benjamini-Yekutieli adjustment was riginally described in the fllwing publicatin: Benjamini, Y, and Yekutieli, D. (2001) "The cntrl f the false discvery rate in multiple testing under dependency." Annals f Statistics. 29(4):

22 Gene Expressin Data Analysis Guidelines MAN-C Hw t d it in nslver: Defining Experimental Grups Differential expressin testing in nslver can be cnfigured during the Experiment wizard setup. In the Add Sample Anntatin windw (see Figure 10), replicates within an experimental grup shuld be given an identical anntatin. In this example, a cntrl and treatment grup have bth been created with 6 samples in each. Fld-changes will be calculated between grups f samples with identical anntatins, and t- tests perfrmed if there are tw r mre replicates within each grup. Nte that three replicates per grup is the recmmended minimum sample size, and having mre than tw experimental grups will require ther statistical testing ptins (i.e., ANOVA r Multivariate Linear Regressin) t prvide unbiased statistical results. Advanced Analysis Mdule Tip: Anntatins added in the basic nslver package can be used when setting up an experiment in the Advanced Analysis mdule. Thus, even if an experimental design utstrips the pwer f a t- test in basic nslver, it can simplify wrkflw t include these anntatins in the Advanced Analysis mdule when setting up Differential Expressin testing, which utilizes multivariate linear regressin fr statistical testing. Figure 10: Creating sample anntatins in nslver 22

23 Gene Expressin Data Analysis Guidelines Hw t d it in nslver: T-test and FDR In the Fld-Change Estimatin windw (see Figure 11), select the apprpriate ptin fr building ratis. Fr differential expressin using a t-test, this ptin will usually be Partitining by, defining the reference grup (usually a cntrl grup) in the drp-dwn menu. A t-test will be perfrmed if there are tw r mre replicates in each grup and fld-changes will be calculated regardless f sample sizes. A check-bx is als prvided t have the sftware generate False Discvery Rate threshlds (Benjamini-Yekutieli FDR). Figure 11: Fld change ptins in nslver Nte: If the experiment des nt cntain sufficient replicates t perfrm a t-test, the sftware will instead autmatically generate a Differential Expressin (DE) call which can be fund in the Experiment Rati Data table. The DE call is a test develped fr ncunter data which determines whether bserved cunt differences between individual samples are larger than can be explained by platfrm technical nise, alne. Fr explratry studies with nly a single bilgical replicate per treatment, the DE call can prvide a mre rbust measure f expressin differences than merely calculating fld-changes between individual samples. See the nslver User Manual fr mre infrmatin n the use f this metric. Once yu have created an experiment (including anntatins and fld-change settings), select the Rati Data level n the Experiments tab. Highlight the rati yu d like t examine in the central table and select the Table buttn (see Figure 12). Figure 12: Creating a Rati Table in nslver By default, nly fld-changes are shwn in the Rati Data Table (see Figure 13), but clicking n the Clumn Optins icn (highlighted belw) will allw viewing f additinal hidden data fields. Nte that the clumn titled FDR cntains FDR adjusted p-values rather than FDR threshlds. Figure 13: Rati Data Table in nslver 23