Page History
...
The Data Exporter functionality enables an i2b2 user to create a data table definition and request patient data for a desired cohort. First, a table of variables is designed using create data table for export tool. The defined data tables are then displayed under the Data Request(s) breakdown types that can be selected in the Run Query dialog, After the query run, the Manager user can view and manage the data requests under the Data Request Manager tool. i2b2 Users will be able to view their data requests status and details . Email are generated both An e-mail is generated for the Data Manager and the i2b2 User when the request is submitted. The Data Manager processes the request by generating the data file under the Data Request Manager tool. The patient data is exported as a file and stored in a specified location for retrieval, and the user is notified by e-mail.
| Info |
|---|
The patient data file is generated as per the data table definition design specification |
...
Data Table Creation - User process
The Create Data Table for export feature allows User to design a table of variables of interest using the Design table feature. The table design can then be saved as a template that can be loaded and used to request data export.
...
- User logs into web client and creates a query.
- In the Run Query dialog box, User created table definitions are displayed under Data Request(s) section
- Select one or more data request checkboxes.
- 1.8.2 requests (custom tables) appear as User Created:<table definition Request> checkboxes
- 1.8.1 data requests (long fact-oriented tables) appear as Request <xx> Data checkboxes.
4. Click run query
5. Emails are automatically sent - one to the User, to inform them the request has been made; and one An email is automatically sent to the data manager to inform them on the User of a new data request.
6. The table definition name with the FINISHED status is displayed as one of the breakdown items in the previous query results.
...
- Query Name (Click on Query ID number to display the previous query in the Find Patients window)
- Data Table Definition (Click on View under Data Request Type drop down Request item)
- Option to Withdraw Request
- Option to enter Comments
- Log info box displays the User's actions. and log of the Request status
...
The data export runs in the background and the file is generated in a specified folder .as defined in the HIVE_CELL_PARAMS (see Configuration below). Data file generation status is displayed under the Status column under Data Request Manager
Status is updated as the Data File is processed. , from Submitted to File in Progress to File Available. When the file is available, the requesting user is informed via e-mail. The process for giving the user access to the file will be site-specific.
Example Export files (all data Example Export files (all data are fake)
Two files are generated for each data table exported.
...
| Info |
|---|
Wildfly must be restarted for changes to the HIVE_CELL_PARAMs to take effect. |
| Cell ID | Parameter Name | Example Value | Notes |
| CRC | edu.harvard.i2b2.crc.smtp.host | smtp.partners.org | SMTP host |
| CRC | edu.harvard.i2b2.crc.smtp.port | 25 | SMTP port |
| CRC | edu.harvard.i2b2.crc.smtp.ssl.enabled | FALSE | TRUE will enable SSL |
| CRC | edu.harvard.i2b2.crc.smtp.auth | FALSE | TRUE will enable SMTP authentication |
| CRC | edu.harvard.i2b2.crc.smtp.username | none | SMTP username (required for SMTP authentication) |
| CRC | edu.harvard.i2b2.crc.smtp.password | none | SMTP password (required for SMTP authentication) |
| CRC | edu.harvard.i2b2.crc.smtp.enabled | FALSE | TRUE will enable e-mails |
| CRC | edu.harvard.i2b2.crc.exportcsv.defaultescapecharacter | " | Escape character for export files |
| CRC | edu.harvard.i2b2.crc.exportcsv.maxfetchrows | -1 | Maximum number of rows to export, or -1 for no limit |
| CRC | edu.harvard.i2b2.crc.exportcsv.defaultlineend | \n | Line ending for export files |
| CRC | edu.harvard.i2b2.crc.exportcsv.defaultseperator | \t | Field separator for export files |
| CRC | edu.harvard.i2b2.crc.exportcsv.resultfetchsize | 50000 | Number of records retrieved during each database fetch. |
| CRC | edu.harvard.i2b2.crc.exportcsv.filename | {{{PROJECT_ID}}}/{{{DATE_yyyyMMdd}}}_{{{FULL_NAME}}}.tsv | Parameterized template for export file names. If the extension is .zip, the file is zipped. |
| CRC | edu.harvard.i2b2.crc.exportcsv.defaultquotechar | " | Quote character for export files |
| CRC | edu.harvard.i2b2.crc.exportcsv.workfolder | /tmp/i2b2 | Folder on the i2b2 server for data exports |
| CRC | edu.harvard.i2b2.crc.exportcsv.zipencryptmethod | none | Encryption method for the exported ZIP file. One of STANDARD, NONE, or AES. |
PM_PROJECT_PARAMS
Stores the parameters for the
...
The parameters for the
...
emails sent to the project's Data Manager about a data request and to the requesting user when export is complete are set on a per-project basis.
| Parameter Name | Example Value | Notes |
|---|---|---|
| Data Request Template | This user {{{USER_NAME}}} in project {{{PROJECT_ID}}} requested ... | Template text for the body of the data request message; supports placeholders like {{{USER_NAME}}} and {{{PROJECT_ID}}}. |
| Data Request Email Address | someone@somewhere.org | E-mail address of the Data Manager, where the data request e-mail will be sent. |
| Data Request Letter | "Results of the i2b2 request entitled - '{{{QUERY_NAME}}}', ..." | Template text for the e-mail to the user announcing the successful completion of the data export; supports placeholders like {{{QUERY_NAME}}}. |
| Data Request Subject | i2b2 Data Request | Subject line for the data request e-mail. |
| Info |
|---|
The e-mail address for the data request letter (sent when export is complete) is entered in the Query Options Email box at the time of data request submission |
Template substitution variables (parameters/placeholders inside VALUE)
| Placeholder | Meaning (inferred) | Example |
|---|---|---|
{{{USER_NAME}}} | Display name / username of the requester | Jon Smith |
{{{PROJECT_ID}}} | i2b2 project ID in which the request was run | ACT |
{{{QUERY_NAME}}} | Name/title of the query the user executed | Hypertension Adults |
{{{QUERY_STARTDATE}}} | Timestamp when the query was submitted | 2025-09-19 09:42 |
{{{QUERY_MASTER_ID}}} | Internal i2b2 Query Master ID of the request | 5412 |
{{{PATIENT_COUNT}}} | Number of patients returned by the request | 1999 |
...
Design and Architecture
The i2b2 breakdown architecture is modified to support the data table definition and new breakdown types for User created data requests. The database tables have been modified to support the data table definitions.
- RPDO_TABLE_REQUEST table stores the data table definitions
- HIVE_CELL_PARAMS has new parameters for global and email configurations and the data file generation location (defaults to the server's drive)
- PM_PROJECT_PARAMS has new parameters for e-mail notifications of data export
- QT_RESULT_TYPE table will contain a new entry for each data table definition
- QT_BREAKDOWN_PATH will contain a new entry for each data table definition which has the data export execution details.
- QT_XML_RESULT contains metadata about each data request, such as status and e-mail details.
...
- The select statement in the VALUE column is executed for the default table data file creation. The result_instance_id value of the query is inserted dynamically each time of data file is created for the same table_instance_id. and does not store in the QT_BREAKDOWN_PATH
- The select statement in the VALUE column of the <data request name>.CSV for the data request is executed to create the data file.
| Info |
|---|
| Currently,1.8.1 request submission send out one email per request selected to the User. |
Software Changes:
- Data: Entries in RPDO_TABLE_REQUEST, HIVE_CELL_PARAMS, PM_PROJECT_PARAMS, QT_QUERY_RESULT_TYPE, and QT_BREAKDOWN_PATH, QT_XML_RESULT define the data exporter functionality.
- Java code: New breakdown classes and updates to existing java classes to support the data exporter functionality.
...
