Page History
| Table of Contents | ||
|---|---|---|
|
| Children Display |
|---|
Overview
The Data Exporter functionality enables an i2b2 user to request patient data for the patients found by a desired query. In the configured workflow, data is requested and generated through new breakdown types that can be selected in the Run Query dialog. Users will see "data request" breakdowns, which send an e-mail to a predefined data manager requesting a type of data. The data manager (Manager user role) processes the request by rerunning the query with a special "data export" breakdown. The patient data is exported as a file and stored in a specified location for retrieval.
...
The manager logs into the webclient and finds the query from their e-mail by locating the query master id, which is now part of the title of the previous query.
The manager user then re-runs the previous query. In the Run Query dialog, the manager selects the requested export option in the Data export breakdown list.
Info It is very important to put the original query master id (or some other identifier that links the export to the request) into the query name. The export letter will contain the query name, and this is the only way to trace the export back to the original request. (Currently there is no way to tie the export directory to the data requester query_master_id.) If the data manager runs the query as a query-in-query, the query name will automatically be populated with this identifier.
The data export runs in the background and the file is generated in a specified folder
- An e-mail is sent to the manager that can be forwarded to the requesting user, to inform them that their export is available and to provide retrieval info.
...
The i2b2 breakdown architecture has been modified to support new breakdown types for data request and export. HIVE_CELL_PARAMS has new parameters for global configuration, he QT_RESULT_TYPE table has new entries for query export types, and the QT_BREAKDOWN_PATH defines the request and export. These are XML documents containing e-mails to send and SQL to execute. These breakdown types display in the run query dialog corresponding to the entries in the QT_RESULT_TYPE table. When a user requests a breakdown, an e-mail is sent to a data manager. When the data manager user exports user request optionthe requested data, an export file is generated based on the entries in QT_BREAKDOWN_PATH table. Email and email notification sent to the manager is also based on the entries in the QT_BREAKDOWN_PATH table. The generated export file is formatted as per the entries in HIVE_CELL_PARAMS. generation location is on the local drive hosting the i2b2 application. The data requester email in the export definition is not directly sent to the user - it is generated as a README file in the data export directory. The data Manager will be required to manually copy the file content into an email and send it to the data request User
Database changes
Currently QT_BREAKDOWN_PATH .value column has datatype of varchar(2000). In order to support the XML structure needed for the Data Exporter, the value column has been modified to CLOB datatype.
...
| NAME | DESCRIPTION | DISPLAYTYPE_ID | VISUAL_ATTRIBUTE_TYPE_ID | USER_ROLE_CD | CLASSNAME |
| PATIENT_DEMOGRAPHIC_REQUEST | Request Demographics Data | CATNUM | LR | DATA_LDS | edu.harvard.i2b2.crc.dao.setfinder.QueryResultPatientRequest |
| PATIENT_DEMOGRAPHIC_CSV | Export Demographics Data | CATNUM | LX | MANAGER | edu.harvard.i2b2.crc.dao.setfinder.QueryResultPatientDownload |
| Info | |||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Data request ability should have project level user data path set We recommended data requests be limited to DATA_LDS users (irrespective of Admin path role (- admin, user, manager) Data export ability should have project level admin path set to MANAGERand data exports be limited to MANAGER users (irrespective of data path role (- data_lds, data_agg,data_obf, data_deid) QT_QUERY_RESULT_TYPE.USER_ROLE_CD value applies to the visibility of data request/data export UI option. Make sure that QT. The user role determines if exports/requests are visible and runnable.
|
Request/Export file definition details: QT_BREAKDOWN_PATH
QT_BREAKDOWN_PATH defines e-mail content (for request and export) and SQL code (for export) in the following XML format:. Any parameters defined in an export XML override the default in HIVE_CELL_PARAMS (e.g., field separator character.)
| Name | Value |
|---|---|
| <?xml version="1.0"?> |
| <?xml version="1.0"?> <ValueExporter> <Version>[...version number...]</Version> <CreationDateTime><date value></CreationDateTime> <DataManagerEmailMessage>details sent to data Manager on data request</DataManagerEmailMessage> <File> <Filename>[... filename specification..]</Filename> <Query>[... SQL query to generate the data export content...]</Query> <SeparatorCharacter>[...file format seperator...]</SeparatorCharacter> </File> </ValueExporter> |
...
- Data: New entries in HIVE_CELL_PARAMS, QT_QUERY_RESULT_TYPE, and QT_BREAKDOWN_PATH define the exporter configuration.
- Java code: New breakdown classes and updates to existing java classes to support the data exporter functionality.
Important Design Notes
Database Configuration
- QT_BREAKDOWN_PATH table configuration for Request/Export file XML definition support is limited to SQL statements only.
- HIVE_CELL_PARAMS /QT_BREAKDOWN_PATH defaultseperator value parameter overrides the file type parameter set in exportcsv.filename value column parameter
Example default separator value : '|'
file type value: demograhics.csv
export file generated: demographics.csv with '|' separator
Export File generation
- Currently, the export file generation location is on the local drive hosting the i2b2 application
- Data export folder is generated with a new Query_master_id generated when the data Manager reruns the data request query. Therefore, currently there is no way to tie the export directory to the data requester query_master_id. Two Options that can be used as a workaround to identify the data requester query
- Use the data requester query_master_id in the query name when rerunning the data requester query. This will generate the original data requester id as part of the data export queryname and also will be displayed in the generated data requester letter
- Run the data requester query as a query- in-query to obtain the same result as option 1.
- Data requester email is generated as a README file in the data export directory. The data Manager will be required to manually copy the file content into an email and send it to the data request User
- The default export file location and format should be set in the HIVE_CELL_PARAMS without the QT_BREAKDOWN_PATH updates , but requires a restart of the wildfly for the changes to take effect.
