The data loading process loads the data to three main sections of data mart.
- Dimension tables
- Mapping tables
- Observation fact table
|
Except for the OBSERVATION_FACT table, the data load will perform the insert or update operation based on two criteria: (1) whether the data exists in the respective tables and (2) whether the PDO's update date is greater than the existing data's update date.
For example the following table shows the conditions for both the update and insert operations for the VISIT_DIMENSION table.
Loading VISIT_DIMENSION |
Condition |
|
|
|
UPDATE |
INPUT.ENCOUNTER_NUM |
= |
OBSFACT.ENCOUNTER_NUM |
AND |
|
INPUT.PATIENT_NUM |
= |
OBSFACT.PATIENT_NUM |
AND |
|
INPUT.UPDATE_DATE |
> |
OBSFACT.UPDATE_DATE |
|
INSERT |
INPUT.ENCOUNTER_NUM |
!= |
OBSFACT.ENCOUNTER_NUM |
OR |
|
INPUT.PATIENT_NUM |
!= |
OBSFACT.PATIENT_NUM |
|
The data load on the OBSERVATION_FACT table is slightly different compared to the other tables and it can be divided into two cases.
Case 1: Refresh the Observation Fact
This case assumes the user is trying to load a fresh set of observation facts and expects to delete any old observation facts that have matching ENCOUNTER_NUM and PATIENT_NUM.
Set append_flag="false" to support this case.
Case 2: Incremental Observation Fact
In this case the user has the option to just add new observation facts that may match the ENCOUNTER_NUM and PATIENT_NUM of an existing OBSERVATION_FACT entry.
Set append_flag="true" to support this case.
The next section, i2b2 Import: Append Flag, explains the rules for importing observations facts into the OBSERVATION_FACT table.
