Rigor and Reproducibility

Rigor and Reproducibility Resources

Learn about the NIH Initiative to Enhance Reproducibility through Rigor and Transparency. (NIH Video)

NIH Rigor and Reproducibility FAQs

Grant Resource Authentication Plan Resources:

• Check out instructions on NIH Nexus Blog.
• Key Biological and/or Chemical Resources

FASEB report on enhancing research reproducibility identifies three main gaps to research reproducibility:

• Lack of uniform definitions to describe the problem
• Insufficient reporting of key experimental details
• Gaps in scientific training

Guide to Rigor, Reproducibility, and Transparency for Flow Cytometry Experiments in the UNC Flow Cytometry Core Facility

A.  Eight steps to Rigorous and Reproducible Experiments in Biomolecular Research:

1. If using a core facility, consult with the core staff in the planning stage. Consult with a statistician if you need help developing a Power Analysis to assure that your results will be adequately powered.
2. Design your experiment with sufficient controls (rigor) and replicates (reproducibility).
3. Assure that ALL of your reagents (antibodies, cell lines, mice) are fully validated (see below).
4. Have a clear and detailed protocol (SOP) and data analysis plan. Assure that the protocol is strictly followed or that any deviation is well documented.
5. Assure that the staff or students performing the experiment are well trained and understand each step and the importance of performing them precisely.
6. Use only well-maintained instrumentation, preferably maintained and operated in a core facility with expert staff (see #1 above).
7. Document all steps, reagents, equipment and data analysis methods used in the experiment. Assure that the both the documentation and the data itself are properly stored in a safe data management repository.
8. Acknowledge all grants that support the core, the core (by name), and core staff in publications. (See Publication Acknowledgments)

B.  Guide to Rigor and Reproducibility for the UNC Flow Cytometry Core

1. Consult with the core staff in the planning stage. Consultations are free and can be scheduled through iLabs.
2. Plan for Rigor: Multicolor Flow Cytometry experiments should be run using replicates [be mindful of biological / technical replicates] based on power calculations and include appropriate controls.
   • Include unstained controls and single-color controls with each experiment for sensitivity and compensation calculations. Fluorophore compensation and analysis can be completed on the cytometers or using FlowJo, FCS Express, or similar 3rd party analysis software. Removal of debris, aggregates, and where possible non-relevant cell lineages from single-color control data files may improve the compensation calculation.
   • Include a viability dye to exclude dead cells from the analysis
   • Incorporate Biological controls and Fluorescence Minus One (FMO) controls to validate the expression of rare or low-expressing markers.
   • Run doublet discrimination in your analysis to exclude aggregates
   • Run time as a parameter to assure that the fluidics were running smoothly during acquisition
3. Validate Reagents, in particular, all antibodies should be validated prior to reporting of results. Recommendations for research using antibodies (page 7-8) ‘Although vendor-supplied technical information may help investigators select reagents such as antibodies, this information is insufficient for validation’. See the section below to assure the use of validated antibodies.
4. Optimize your sample preparation and staining protocols. For help you can reach out to the core for a consultation, we also provide some starting points on our Flow Cytometry Associated Protocols page. 
5. Get trained on the proper use of core instrumentation. 
6. Confirm instruments are fit for service. Daily quality control of all instrumentation is performed by staff based on manufacturers’ recommendations to ascertain that all of the UNC Flow Cytometry Core Facility flow cytometers maintain peak performance. In addition to quality control, drop deflection voltages are set, and a drop delay calculation is carried out on all cell sorters prior to sorting.
   • For both sorting and analysis the core recommend Single-color controls are included when appropriate for compensation calculations.
   • For sorting the core recommends that Post-sort analyses are run on each population to verify purity when possible.
7. Properly document and store your flow data. Guidelines for reporting have been adopted by the International Society for Advancement of Cytometry (ISAC). For details see MIFlowCyt – ‘Minimum Information for Flow Cytometry experiments’ for recommendations on reporting Flow Cytometry Data. Additionally, assure that your data is properly stored in a safe data management repository e.g. Flow Repository or UNC Dataverse. 
8. Acknowledge all grants that support the core, the core (by name), and core staff in publications. (See Publication Acknowledgments)