From the Design stage, you should know how you expect, or would like, your system to function, and also have some ideas about how you will test this. You will also have your system implemented from the Build stage. In this stage, you will define your experimental design, and test your system.

This stage is important as it is how you will determine whether your system is working as intended, and also generate data to feed back into the next iteration of the Design-Build-Test-Learn cycle. This data will also be used to show others how your system functions and will guide others in reproducing your work, which is crucial in any engineering or science discipline.

Good Measurement

What is Measurement?

In synthetic biology, measurement refers to the process of testing and characterising a system or device. Good measurement ensures that the data you produce is relevant, comparable, accurate, reliable, and reproducible.

Ensuring that generated data has these properties is also important in other engineering fields. Thinking back to the example of building an aircraft given in the introduction section, it is important that engineers and customers can trust data showing that the aircraft is fit for purpose. It is also important that the correct things are tested. For example, it is not enough to simply show that an aircraft can fly in mild weather. It is important to know how the aircraft will cope with stormy weather, or in cold conditions.

Why is Measurement Important in Synthetic Biology?

At iGEM, we believe that good measurements are the foundation on which scientific progress and societal impact are built. It is not enough to simply perform experiments or build DNA constructs, instead iGEM teams (and researchers in general) should carefully consider what aspects of a device/system need to be characterised, and the purpose of any experiments performed. It is also essential to realise that good measurement doesn't stop once an experiment has been performed; the way in which data is presented is also very important. Ensuring that appropriate data visualisation, statistical analyses, and units are used, along with careful and complete documentation of how the experiment was performed, will help ensure that future iGEM teams and researchers can make the most of your work. Teams can also engage in a wide variety of other measurement activities, including collaborating to test reproducibility, design new and better assays, and create low-cost instruments to make measurement more accessible.

The measurement aspect of a project is also an opportunity for team members to explore the twin questions of knowledge at the heart of science and engineering. How do we know what we know, and how can we predict what will happen when we create something new? We must grapple with these questions if we wish to make responsible choices both as individuals and as a society, and the work done by iGEM teams is one of the many components of that effort.

How can Standards help with Good Measurement?

Every lab has different equipment with different settings, and measurements of fluorescence or absorbance from this equipment are often reported using arbitrary units (AU). These AU values from different labs cannot be directly compared, which hinders reproducibility and can discourage others from building on your work and/or using your systems. Measurement standards can be used to calibrate equipment and convert arbitrary units to absolute units, which are comparable.

Below are methods for obtaining absolute units for three measurements commonly used in synthetic biology.

Converting to Absolute Units

Converting OD600 to Absolute Units

Cell density (i.e. how many cells are in a culture) is normally measured and reported using optical density (OD) readings. Whilst this approach can be used to provide a good estimate of cell density, the units reported are arbitrary and can vary between equipment and labs. Silica microspheres/beads are similar in size and optical properties to bacterial cells, and therefore one microsphere is approximately equal to one bacterial cell. By creating a serial dilution with known numbers of silica microspheres, a standard curve of number of microspheres against OD readings can be produced. Using this standard curve, OD readings for bacterial cell cultures can be converted to an approximate number of cells, which is an absolute unit and comparable between equipment and labs.

The protocol for converting OD readings to absolute cell count measurements can be found here, along with spreadsheets to help with the required calculations.

Converting Fluorescent Intensity for Green Fluorescent Proteins to Absolute Units

Fluorescent intensity (FI) is a measure for the amount of fluorescent protein present in a sample. FI is an arbitrary reading, which means that it cannot be accurately compared between different equipment and labs. Fluorescein is a fluorescent compound which has a similar fluorescent profile to GFPs and YFPs. A serial dilution with known amounts of fluorescein can be prepared to calibrate a piece of equipment's arbitrary FI readings and convert them to concentration, which results in data which can be compared between labs and equipment. The method described above to measure the number of cells should also be used so that concentration of fluorescent protein per cell can be reported.

The protocol for converting FI readings to absolute concentration measurements can be found here, along with spreadsheets to help with the required calculations.

Converting Fluorescent Intensity for Red Fluorescent Proteins to Absolute Units

Fluorescent intensity (FI) is a measure for the amount of fluorescent protein present in a sample. FI is an arbitrary reading, which means that it cannot be accurately compared between different equipment and labs. Texas Red is a fluorescent dye which has a similar fluorescent profile to RFPs. A serial dilution with known amounts of Texas Red can be prepared to calibrate a piece of equipment's arbitrary FI readings and convert them to concentration, which results in data which can be compared between labs and equipment. The method described above to measure the number of cells should also be used so that concentration of fluorescent protein per cell can be reported.

The protocol for converting FI readings to absolute concentration measurements can be found here, along with spreadsheets to help with the required calculations.

Questions to Consider for wet lab projects

The questions below will help guide you in designing your experiments and determining whether your experiment was successful or not. Note that a successful experiment is different from determining whether your system is working as expected: negative data is fine so long as your experiment was performed correctly! Your design and build specifications will both influence and be influenced by these questions.

What are you testing for?
- What aspects of your system do you need to test to determine whether it is functioning as intended?
- Are you looking at how a specific part of your system responds to an inducer? Maybe you want to characterise how well your host chassis grows when expressing your system, or see how your cells look down a microscope?
- Will you conduct these tests as a single experiment, or as a series of experiments?
Which types of measurements are needed?
- What information do you need to determine if your test was successful?
- What will you need to measure? Fluorescence intensity? Absorbance values? Cell size?
What instruments are required to perform these measurements?
- If the required instrumentation is not available, how will you modify your experimental design?
What is the effective range of measurement on your instruments, and how will you distinguish true signals from noise, artifacts, and confirmation bias?
What form will your data take?
- Is it quantitative or qualitative? Will there be images?
- Will you collect time course or end-point data?
What are the physical units of your measurements?
Are your units arbitrary, relative, or absolute?
- How can you convert your measurements to absolute units? Will you need additional samples or standards for this?
What sort of controls can you use to determine whether your experimental protocols are working correctly?
- What data will you generate to determine whether your experiment ran correctly?
How will you analyse your data?
- What analysis is required to determine if your system is functioning as intended? For example, will you compare test samples to controls?
- How many repeats will you need to perform this analysis to an acceptable degree of confidence?
How will you report your data?
- What are the crucial results from your experiments?
- How can you present your data in an easily understandable format?
- How can you make your full data set available for transparency and reproducibility?
How can your measurements be used by others?
- Is there any metadata required to ensure others can repeat your experiments?
- Are you able to test your system modularly so others can easily re-use parts of your design and understand how that aspect will function?

Tips to Get Started

When starting to consider what experiments you will need to perform for your project, it is a good idea to refer back to your design specification. What are the crucial features or functions that your system or device must have? You can then start to design experiments which will specifically test whether your system meets these specifications.

You can also consider which parts of your design might need optimisation. How can you generate data which can be fed back into your design? Maybe you have a model which is informing your design? If so, what data do you need to feed your model, and what format will it need to be in?

Finally, consider how all of your experiments could be reproduced by other researchers, and make sure that your data can be directly compared to others by converting your measurements into absolute units where possible.

In silico Test Stage

For in silico projects, the test stage is where you begin to run your model or software tool. For other dry-lab projects, it might be where you test some hardware you’ve designed and built.

When developing a software tool or model, it is usual for the Build and Test stages to be tightly coupled and you will likely swap between the two. This is because it is usually feasible to write a section of code, and then immediately test whether it works as expected before continuing. Once the entire model or tool has been built, you will likely need to perform more extensive tests. The questions below can help you with designing these tests.

What are you testing for?
- What aspects of your model or tool do you need to test to determine whether it is functioning as intended?
- Do you need to check if changing a parameter in your model has the desired effect? Maybe you want to test how well your tool performs under heavy load?
- Are you trying to predict how your system will function? Maybe you’re trying to use a model to help inform your design?
Which types of measurements are needed?
- What information do you need to determine if your test was successful?
- What will you need to measure? For example, are there performance metrics for your model/tool? Are you generating a specific output?
How will you perform your measurements?
- Are you able to test your model using a simulation tool? Are you generating log files?
What form will your data take?
- Is it quantitative or qualitative?
What are the physical units of your measurements?
What sort of controls can you use?
How will you analyse your data?
- What analysis is required to determine if your model/tool is functioning as intended?
- If you have a stochastic model, how many repeats will you need to perform this analysis to an acceptable degree of confidence?
How will you report your data?
- How can you present your data in an easily understandable format?
How can your measurements be used by others?
- Is there any metadata required to ensure others can repeat your experiments?
How will you use your data?
- If you intend to use your model to inform your design, how do you plan to do this?

Tips to Get Started

If you are building a model, think back to your design specification and what the purpose of the model is. What data will you need to achieve this purpose, and what format will it need to be in? Also consider whether you will need to run multiple iterations of your model with different parameters, and whether you can automate this process?

For software tools, you can also use your design specification to determine the most important use cases and test whether your tool can handle those. There are also plenty of resources available online related to software development which can help you.

For hardware based projects, your experiments might look like a mixture of dry and wet lab experiments. These projects are also likely to fit more neatly into traditional engineering fields, and so you can take inspiration from other engineering processes.

Test Stage Resources

Measurement Validation

The iGEM Measurement Committee is now offering measurement validation to all teams!

We are offering measurement validation for three iGEM protocols at this time (details for these are shown below at iGEM's Standard Protocols):

iGEM 2020 Plate Reader Fluorescence Calibration
iGEM 2020 Plate Reader OD₆₀₀ nm (Optical Density) Calibration
iGEM 2020 Flow Cytometry Fluorescence Calibration

If you make measurements using any of these three protocols, you can submit the resulting Excel calculation sheet to the Measurement Committee for validation. To submit your Excel sheets, please email the Excel sheets to us at measurement@igem.org.

We will use the values in the sheet to generate a "diagnostic report" for you, either validating that your protocol appears to have worked as expected or else indicating potential problems that you may need to address.

We strongly encourage all teams to use these protocols in their project, if possible. Validating measurements can help find problems in protocols or instruments, make it easier to debug your system, make your data easier to use, and increase the future impact of your project!

Protocols.io

iGEM is allowing and encouraging teams to use Protocols.io to create and share their methods instead of putting them on their wikis. Protocols.io is an online, open-access tool for collaboratively creating, sharing and discussing protocols. It supports both wet and dry work.

Teams using protocols.io must still adhere to the wiki freeze deadline. Protocols.io allows authors to “publish” their protocols, making them publicly accessible and generating a digital object identifier (DOI), which also functions as an online link to the protocol. Once published, a protocol cannot be edited.

Teams using protocols.io must publish their protocols and place the DOI link generated on the methods section of their wiki, making sure judges can see what links will lead to which protocols.
Teams can also use these DOI links in their part documentation on the Registry to describe what protocols the parts were used in.
We recommend teams use “iGEM” as a keyword in the protocols to facilitate other teams finding their work in the future.
To make the most of the collaborative features provided by protocols.io, we recommend students all sign up individually and create a group on protocols.io for their team.

Note:

Protocols cannot be edited after publication, but new versions can be generated. Judges will only consider the version you publish before the wiki freeze, but you are free to generate new versions after the freeze for future teams to use.

iGEM's Standard Measurement Protocols

Each of these can be accessed on Protocols.io by pressing the Protocol button or in PDF format using the PDF button. Some protocols also have an associated Excel data analysis template, which you can find by pressing the Excel button.

Every lab has different equipment with different settings, and measurements of fluorescence or absorbance are often reported using arbitrary units (AU). These AU values from different labs cannot be directly compared. These protocols are for use with the iGEM Measurement Kit, which each team can create themselves with the purchase of a few components. The iGEM Measurement Kit refers to resources that allow calibration of plate readers for fluorescent intensity and cell density measurements. Once these calibration protocols have been performed, you’ll be able to convert the arbitrary units you produce during your project into standard units. This will make your results much more powerful by making them directly comparable with those of other teams who have also calibrated their equipment.

Measurement Kit Components:

Item	Supplier	Order Details	Website
Microsphere Particles	NanoCym	950uM size particles	https://nanocym.com/product/silocym/
Fluorescein sodium salt	Sigma-Aldrich	Product number: 46970	https://www.sigmaaldrich.com/catalog/product/sigma/46970
Texas Red (Sulforhodamine 101 acid chloride)	Sigma-Aldrich	Product number: S3388	https://www.sigmaaldrich.com/catalog/product/sigma/s3388

iGEM 2020 Plate Reader Abs600 (OD) Calibration

Use this protocol to be able to convert 600 nm absorbance measurements into an estimated equivalent number of cells. Example data

PROTOCOL

PDF

EXCEL

iGEM 2020 Plate Reader Green Fluorescence Calibration

Use this protocol to be able to convert green fluorescence measurements into an estimated equivalent fluorescent molecules per cell. Example data

PROTOCOL

PDF

EXCEL

iGEM 2020 Flow Cytometry Fluorescence Calibration

Use this protocol to convert measurements of flourescence in your flow cytometer to MEFL units. Example data

PROTOCOL

PDF

EXCEL

iGEM 2020 Plate Reader Red Fluorescence Calibration

Use this protocol to be able to convert red fluorescence measurements into an estimated equivalent fluorescent molecules per cell.

PROTOCOL

PDF

EXCEL

Other iGEM Measurement Protocols

Flow Cytometry Cell Size Calibration

Use this protocol to convert flow cytometry forward scatter to Eμm.

PROTOCOL

PDF

Abs600 Inter-equipment Conversion with LUDOX

Use this protocol to be able to convert absorbance (OD₆₀₀) data from your plate reader into a comparable OD₆₀₀ measurement which would be obtained in a spectrophotometer.

PROTOCOL

PDF

EXCEL

Conversion of OD₆₀₀ to Colony Forming Units (CFUs)

Use this protocol to be able to convert your OD₆₀₀ measurements into CFUs.

PROTOCOL

PDF

EXCEL

Cell Measurement Protocol

A standard testing protocol for various parts in the registry, using a plate reader.

PROTOCOL

PDF

Have a resource to contribute?

Please email the Engineering Committee at engineering [AT] igem [DOT] org and provide links to material with a short description. We’ll check it out and if we believe it will be helpful, we’ll add it to this page!

Engineering/Test

Test

Introduction