REDCap: Data Quality Rules
 Overview |
 Pre-Defined Rules |
 Custom Rules |
 Selecting Scope of Run |
|---|
Overview
The Data Quality application within REDCap allows you to execute data quality rules upon your project data to check for discrepancies. The module has 9 pre-defined data rules labeled A-I that you may utilize and run as needed. You may also create your own rules to check specific conditions found within your data or edit, delete, or reorder the rules you have already created.
To find any data discrepancies for a given rule, you may click the Execute button next to it, or click the Execute All Rules button to fire all the rules at once. It will provide you with a total number of discrepancies found for each rule and will allow you to view the details of those discrepancies by clicking the View link next to each. For some pre-defined rules such as Data Quality - Rule H you may also opt to fix all issues immediately. This is useful in cases where calculations have changed and all records need their values updated to reflect the new calculation.
Pre-Defined Rules
Under the initial list of available Data Quality Rules, there are 9 rules that REDCap has pre-defined:
- Blank values - Checks if any field is left blank or missing a value (not including fields hidden by branching logic).
- Blank values (required fields only) - Checks if any field marked as required is blank or missing a value (not including fields hidden by branching logic).
- Field validation errors (incorrect data type) - Checks if any data entered does not match a field's validation type.
- Field validation errors (out of range) - Checks if any data entered does is out of range of a fields set min-max values.
- Outliers for numerical fields (numbers, integers, sliders, calc fields) - Checks if any values for numerical fields would be considered statistical outliers (>2 two standard deviations from the mean).
- Hidden fields that contain values - Checks if there are any fields hidden by branching logic that contain data.
- Multiple choice fields with invalid values - Checks if any multiple choice field has an invalid value (not one of the current available option choices).
- Incorrect values for calculated fields - Checks if there are any incorrectly calculated values saved for any calculated fields or text fields with Action Tag calculations.
- Fields containing "missing data codes" - Checks if there are any fields marked with missing data codes.
Each Data Quality Rule can be executed individually. This will generate a report that is viewable and downloadable with a list of all data flagged as an issue by the currently executed rule. From there you may choose to verify your data and update values as needed individually.
Data Quality - Rule H
One Data Quality Rule that allows REDCap to update data for you is Data Quality Rule H.
After executing this rule, if there are any issues REDCap provides the option to fix all calculations immediately. Because Rule H is applied to fields that REDCap automatically calculates and that you cannot change on your own, REDCap will update and save all of the new values automatically once the "Fix calcs now" button is selected. This is particularly useful if you have made modifications to your project in Production and need to update calculated values to reflect the new output across all records.
If you make changes to your calculations in a Production project with real data, the alternative to using Data Quality - Rule H when updating calculations is to enter each record's form and save the form again manually to update the saved data in the database with a new value. Using Data Quality - Rule H saves time and effort by allowing REDCap to verify values and then update them for you.
Custom Rules
Custom Data Quality Rules may be added to the table of Data Quality rules by entering in a Rule Name, Rule Logic, and selecting the green "Add" button under the "Rule #" column. Once a new rule is added, it will be added to the end of the list. Custom rules will be listed by number in the order in which they were added.
Once a rule is added a new "Execute" button will allow you to run the current logic for the rule across the project's data. If any discrepancies occur, they will be viewable and exportable similarly to the pre-defined rules.
Enabling the 'real-time execution' functionality for a custom rule is a great way to add more data validation on a data entry form itself to ensure that data are getting entered correctly *at the moment* they are entered, as opposed to checking the quality of the data retroactively by executing the rules here on this page.
Checking the 'real-time execution' checkbox for any custom data quality rule will enable the rule to be executed invisibly on data entry forms. Whenever a user clicks the Save button to create or modify a record, iREDCap will execute all relevant data quality rules and will display a warning pop-up message if any of the rules have been violated.
NOTE: The pre-defined rules cannot have 'real-time execution' enabled, but only the custom rules can. Also, the 'real-time execution' functionality does not work on survey pages, nor does it get executed when performing data imports (either via the Data Import Tool or via the API).
Selecting Scope of Run
At the top of the Data Quality Rules table, there are 3 options for filtering the rules before executing them.
- Limit by specific records
- Limit by specific Data Access Group (DAG)
- Limit by specific fields
Selecting parameters for each of the 3 options allows you to execute rules on a smaller subset of data. This may be useful if you do not want to check or update all data in your project, but instead want to select a specific grouping of data to target. The parameters must be added before executing the rule in order for the filter to apply.
