Data Harmonization and Why Automate It
This article about data harmonization will define what it is and where it takes place in data logging.
Here is a diagram summarizing the concepts discussed in this article to refer to during your reading. It is tailored with an emphasis on data harmonization in using measurement instruments.
Data logging
Data logging is defined as any of various chronological records made concerning the use of a computer system, the changes made to data, etc. (1)
The chronology does not have to be established with a timestamp; any numerical order can provide it. Data logging is the core activity of a data logger and data acquisition system (DAQ). After collecting the data, it is necessary to extract it from the sensors. With a data logger, one file per sensor is produced. With a DAQ system, there will be a single file containing all the sensors' data.
Data harmonization: a definition
In a broad sense, especially in the biomedical sector, data harmonization means combining studies, experiments, file formats, or databases. To combine them, standard parameters must be established. (2)
There are companies or software to do such data harmonization. Otherwise, it is an arduous, tedious and error-prone manual task.
Data harmonization applied to metrology
Data harmonization “seeks to bring together various types, levels and sources of data, which represent measurement of the same latent construct(s), in such a way that they can be compatible and comparable”. (3)
In short, it is a matter of combining measures according to standard parameters. The collection method and pre-existing data must be compatible to harmonize the data. In the case of metrology, this compatibility can be established by timestamp and unit of measurement.
Why automate data harmonization?
Here is a situation where we have three different types of measurements, with three measuring instruments being part of a system supporting data harmonization.
In our CO2 incubator, we want to ensure that the temperature, humidity and CO2 concentration are optimal for the growth of our cells.
If we have data loggers, each instrument must be set up individually. Then, the collected data will be transferred to a computer and must be aligned manually. To do the alignment, the time stamp is an important parameter. Ensure the logging starts simultaneously and the sampling rate is the same for all three instruments. This will make it easier to align the data. Also, the files produced by each sensor may not be the same type. Some programming skills will be needed here to align the data.
With a DAQ system, the logging of the sensor data starts at the same time, with a defined sampling rate for the duration of the experiment. Only one action for all three instruments. This makes the measurement task much faster than with several different instruments.
Small caveat. Not all DAQ systems allow the harmonization of different types of data (for example, temperature, relative humidity and carbon dioxide level). This is something to check when purchasing the instruments.
A DAQ system that allows for data harmonization will create a single file containing all the aligned data. With the software, logging and harmonization are done simultaneously. No programming skills are required, and the hours invested in producing an aligned data file are reduced to zero.
Dracal and data harmonization
With DracalView data acquisition software, each Dracal measurement instrument is part of a perfectly harmonized Mix & Match ecosystem. Each USB sensor logs data with a synchronized timestamp, allowing for automated data harmonization. In addition, the units of measurement are standardized for a given quantity. Thus, taking the example of our CO2 incubator and the three required measures, a single log file will be generated with DracalView, containing a single line per timestamp for all measurements taken simultaneously.
Moreover, the use of SensGate allows for harmonized data not only from the sensors connected to it but also from all sensors connected to the same network on different SensGate remote data logging stations. If you are interested in setting up your own sub-network of sensors, we invite you to read the simple procedure to do so.
In short
Harmonization of metrological data is their combination and alignment according to standard parameters in a single file. These parameters are often time and measurement units. Harmonization can be complex or straightforward, depending on the automation tools used.
We have seen that a complete data acquisition system like the one offered by Dracal Technologies, including measurement instruments, a remote data logging station and a software solution, automates data harmonization and saves a tremendous amount of time aligning every file from every datalogger, even for different measurement types.
If you want to discuss your project with us, please send us an email!
References:
- Collins Dictionary (2022). Definition of ‘log’. [online] collinsdictionary.com. Available at: https://www.collinsdictionary.com/dictionary/english/log [Accessed 17 Oct. 2022].
- Tetrascience.com. (2022). Scientific Data Harmonization | TetraScience. [online] Available at: https://www.tetrascience.com/platform/data-harmonization [Accessed 28 Sep. 2022].
- NIHR | Cambridge Biomedical Research Centre (2012). Harmonization. [online] DAPA Measurement Toolkit. Available at: https://dapa-toolkit.mrc.ac.uk/concepts/harmonisation [Accessed 28 Sep. 2022].