Uploaded image for project: 'i2b2 Core Software'
  1. i2b2 Core Software
  2. CORE-193

Some (Temporal) Queries cause Timeline to hang and Middle Tier OOM

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Cannot Reproduce
    • 1.7.04
    • None
    • CRC Cell
    • None
    • Timeline View
    • Queries - using patient sets
    • i2b2 Core
    • All databases
    • All Web Browsers

    Description

      We have run into an issue where some temporal queries cause the timeline to hang and Middle Tier has this error in the logs:

      {noformat}
      15:08:02,375 ERROR [edu.harvard.i2b2.crc.delegate.pdo.PdoQueryRequestDelegate] (http--0.0.0.0-9090-6) Throwable: java.lang.OutOfMemoryError: Java heap space
      {noformat

      Attachments

        Activity

          matthoag Matthew Hoag added a comment -
          Query that causes the problem (I know this is KU specific, but note the temporal constraints are there)
          matthoag Matthew Hoag added a comment - Query that causes the problem (I know this is KU specific, but note the temporal constraints are there)
          matthoag Matthew Hoag added a comment -
          JConsole paired with the Log of the error
          matthoag Matthew Hoag added a comment - JConsole paired with the Log of the error
          matthoag Matthew Hoag added a comment - - edited
          Log associated with the JConsole Picture of the OOM

          {noformat}
          15:07:51,749 DEBUG [edu.harvard.i2b2.crc.dao.CRCDAO] (http--0.0.0.0-9090-6) ********* Total time for patient sql ****11
          15:07:51,752 DEBUG [edu.harvard.i2b2.crc.dao.CRCDAO] (http--0.0.0.0-9090-6) ********* Total time for patient objects ****11
          15:07:51,753 DEBUG [edu.harvard.i2b2.common.util.jaxb.JAXBUtil] (http--0.0.0.0-9090-6) object.toString()edu.harvard.i2b2.crc.datavo.i2b2message.RequestMessageType
          15:08:02,375 ERROR [edu.harvard.i2b2.crc.delegate.pdo.PdoQueryRequestDelegate] (http--0.0.0.0-9090-6) Throwable: java.lang.OutOfMemoryError: Java heap space

          15:08:02,377 DEBUG [edu.harvard.i2b2.common.util.jaxb.JAXBUtil] (http--0.0.0.0-9090-6) object.toString()edu.harvard.i2b2.crc.datavo.i2b2message.RequestMessageType
          15:08:02,377 DEBUG [edu.harvard.i2b2.crc.axis2.QueryService] (http--0.0.0.0-9090-6) Response in service<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
          <ns5:response xmlns:ns2="http://www.i2b2.org/xsd/hive/pdo/1.1/" xmlns:ns4="http://www.i2b2.org/xsd/cell/crc/psm/1.1/" xmlns:ns3="http://www.i2b2.org/xsd/cell/crc/pdo/1.1/" xmlns:ns9="http://www.i2b2.org/xsd/cell/ont/1.1/" xmlns:ns5="http://www.i2b2.org/xsd/hive/msg/1.1/" xmlns:ns6="http://www.i2b2.org/xsd/cell/crc/psm/querydefinition/1.1/" xmlns:ns10="http://www.i2b2.org/xsd/hive/msg/result/1.1/" xmlns:ns7="http://www.i2b2.org/xsd/cell/crc/psm/analysisdefinition/1.1/" xmlns:ns8="http://www.i2b2.org/xsd/cell/pm/1.1/">
              <message_header>
                  <i2b2_version_compatible>1.1</i2b2_version_compatible>
                  <hl7_version_compatible>2.4</hl7_version_compatible>
                  <sending_application>
                      <application_name>CRC Cell</application_name>
                      <application_version>1.7</application_version>
                  </sending_application>
                  <sending_facility>
                      <facility_name>i2b2 Hive</facility_name>
                  </sending_facility>
                  <receiving_application>
                      <application_name>i2b2_QueryTool</application_name>
                      <application_version>1.6</application_version>
                  </receiving_application>
                  <receiving_facility>
                      <facility_name>i2b2 Hive</facility_name>
                  </receiving_facility>
                  <message_control_id>
                      <instance_num>1</instance_num>
                  </message_control_id>
                  <project_id>BlueHeron</project_id>
              </message_header>
              <response_header>
                  <info>Log information</info>
                  <result_status>
                      <status type="ERROR">java.lang.OutOfMemoryError: Java heap space
          </status>
                      <polling_url interval_ms="100"/>
                  </result_status>
              </response_header>
          </ns5:response>
          {noformat}
          matthoag Matthew Hoag added a comment - - edited Log associated with the JConsole Picture of the OOM {noformat} 15:07:51,749 DEBUG [edu.harvard.i2b2.crc.dao.CRCDAO] (http--0.0.0.0-9090-6) ********* Total time for patient sql ****11 15:07:51,752 DEBUG [edu.harvard.i2b2.crc.dao.CRCDAO] (http--0.0.0.0-9090-6) ********* Total time for patient objects ****11 15:07:51,753 DEBUG [edu.harvard.i2b2.common.util.jaxb.JAXBUtil] (http--0.0.0.0-9090-6) object.toString()edu.harvard.i2b2.crc.datavo.i2b2message.RequestMessageType 15:08:02,375 ERROR [edu.harvard.i2b2.crc.delegate.pdo.PdoQueryRequestDelegate] (http--0.0.0.0-9090-6) Throwable: java.lang.OutOfMemoryError: Java heap space 15:08:02,377 DEBUG [edu.harvard.i2b2.common.util.jaxb.JAXBUtil] (http--0.0.0.0-9090-6) object.toString()edu.harvard.i2b2.crc.datavo.i2b2message.RequestMessageType 15:08:02,377 DEBUG [edu.harvard.i2b2.crc.axis2.QueryService] (http--0.0.0.0-9090-6) Response in service<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <ns5:response xmlns:ns2=" http://www.i2b2.org/xsd/hive/pdo/1.1/ " xmlns:ns4=" http://www.i2b2.org/xsd/cell/crc/psm/1.1/ " xmlns:ns3=" http://www.i2b2.org/xsd/cell/crc/pdo/1.1/ " xmlns:ns9=" http://www.i2b2.org/xsd/cell/ont/1.1/ " xmlns:ns5=" http://www.i2b2.org/xsd/hive/msg/1.1/ " xmlns:ns6=" http://www.i2b2.org/xsd/cell/crc/psm/querydefinition/1.1/ " xmlns:ns10=" http://www.i2b2.org/xsd/hive/msg/result/1.1/ " xmlns:ns7=" http://www.i2b2.org/xsd/cell/crc/psm/analysisdefinition/1.1/ " xmlns:ns8=" http://www.i2b2.org/xsd/cell/pm/1.1/ ">     <message_header>         <i2b2_version_compatible>1.1</i2b2_version_compatible>         <hl7_version_compatible>2.4</hl7_version_compatible>         <sending_application>             <application_name>CRC Cell</application_name>             <application_version>1.7</application_version>         </sending_application>         <sending_facility>             <facility_name>i2b2 Hive</facility_name>         </sending_facility>         <receiving_application>             <application_name>i2b2_QueryTool</application_name>             <application_version>1.6</application_version>         </receiving_application>         <receiving_facility>             <facility_name>i2b2 Hive</facility_name>         </receiving_facility>         <message_control_id>             <instance_num>1</instance_num>         </message_control_id>         <project_id>BlueHeron</project_id>     </message_header>     <response_header>         <info>Log information</info>         <result_status>             <status type="ERROR">java.lang.OutOfMemoryError: Java heap space </status>             <polling_url interval_ms="100"/>         </result_status>     </response_header> </ns5:response> {noformat}
          matthoag Matthew Hoag added a comment - - edited
          After looking at the log and then the source code, I am guessing that the error has a stack trace like:
          JAXBUtil:128
          I2B2MessageResponseFactory:115
          PdoQueryRequestDelegate:220

          Specifically, I believe `marshaller.marshal(element, strWriter)` is causing the OOM, when middle tier app is dumping the XML object into a StringWriter.

          I can't know for sure because the stack trace of error is getting eaten by the XML response and doesn't make it into the debug log.
          matthoag Matthew Hoag added a comment - - edited After looking at the log and then the source code, I am guessing that the error has a stack trace like: JAXBUtil:128 I2B2MessageResponseFactory:115 PdoQueryRequestDelegate:220 Specifically, I believe `marshaller.marshal(element, strWriter)` is causing the OOM, when middle tier app is dumping the XML object into a StringWriter. I can't know for sure because the stack trace of error is getting eaten by the XML response and doesn't make it into the debug log.
          Hi Matthew, are you still having this problem? If so, which version of i2b2 are you using?
          jmd86 Janice Donahoe added a comment - Hi Matthew, are you still having this problem? If so, which version of i2b2 are you using?
          To fix this issue you need to increase the memory of your JAVA RUNTIME.
          jmd86 Janice Donahoe added a comment - To fix this issue you need to increase the memory of your JAVA RUNTIME.

          People

            mem61 Mike Mendis
            matthoag Matthew Hoag
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: