screenshot-1.png
[WEBCLIENT-294] Webclient Reports "QUERY CANCELLED" While Query Is Still Running Created: 12/Aug/19 Updated: 31/May/22 Resolved: 04/Sep/19 |
|
| Status: | Closed |
| Project: | i2b2 Web Client |
| Component/s: | Documentation |
| Affects Version/s: | None |
| Fix Version/s: | 1.7.12, 1.7.13 |
| Type: | Bug | Priority: | Major |
| Reporter: | Mark Abajian | Assignee: | Mike Mendis |
| Resolution: | Working As Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
i2b2 server v1.7.09c on CentOS 7.5, Java 1.8, Wildfly 10.0.0.
Database is SQL Server 2016 on Windows 2012. Webclient v1.7.09c is hosted in Apache on same CentOS machine as the i2b2 server. Browser is Firefox on macOS 10.12. |
||
| Attachments: |
screenshot-1.png
|
| Developer Notes: | Added comments to https://community.i2b2.org/wiki/display/getstarted/2.5+Web+Server+Requirements |
| Participant/s: |
| Description |
|
Hello, i2b2 Engineers,
We ran into an issue back in January and submitted a ticket to the ACT HelpDesk (ACT-177). This is essentially a copy of that ticket. Here is our problem... Any query that takes longer than 60 seconds is "cancelled" by the webclient with the message "QUERY CANCELLED". But the query continues to run on the i2b2 server, and I do not believe that it is moved out of the default queue. It is simply running longer than one minute, but eventually completes. Risk: The risk is that most users will believe that their query is cancelled. And some will re-try their queries, overtaxing the system. Others will forget about their queries, and this will give our service a black eye. Analysis: I can't tell if the server is returning some flag to the weblicent, and that is why the webclient is presenting the Query Cancelled message in the Query Status panel, or if the webclient is just "timing out" because it has not received a reply within 60 seconds. But my hunch is that this is a webclient issue; I'm not an expert. We also find that if we wait long enough, the query results will show up in the Previous Queries panel. We are unable to locate any remedy for this in the i2b2 documentation or configuration files. Two questions: 1. Is there a setting we can change in our i2b2 web client or CRC cell so that queries do not time out like this in 60 seconds? 2. Is there a bug in our web client or in the CRC cell, where it reports the query was cancelled, even though it finally goes through to completion? Please note: This issue was also raised on the i2b2 Install Help Google group with this topic "Problem with CANCELLED queries with web client v 1.7.11" on August 2nd by a 3rd party. Kind regards, Mark Abajian |
| Comments |
| Comment by Mark Abajian [ 26/Aug/19 ] |
|
Hello, Mike, Before you spend too much time on this... I came across something online that seemed to suggest that it could be a FastCGI (PHP) setting that may be causing the early timeout in our webclient. Does this seem plausible to you?
--Mark A for Keck/USC, (818) 726-0372 mobile |
| Comment by Mike Mendis [ 26/Aug/19 ] |
|
Mark, I am trying to reproduce this, is the webclient the standard i2b2 one or the SHRINE webclient? Also in the crc.propeties file what is the edu.harvard.i2b2.crc.jms.small.timeoutsec set to? The default is 180. This is the time that the submitted query should take. The other setting is on the xml message, that executes the query which is part of the runQueryInstance_fromQueryDefinition, in this message what is the request_header set to, here is the default:
<request_header> <result_waittime_ms>180000</result_waittime_ms> </request_header> Normally they are both set to 180sec, but if you are using shrine, then it might be modified. What is happening under the covers is whichever number is lower is the one that is used. So the server (crc.properies) has it own default value, but the user (via xml) can override it make query run with less time. |
| Comment by Mark Abajian [ 26/Aug/19 ] |
|
Hello, Mike, Do you believe this could be a setting in FastCGI on our Apache/PHP configuration?
To answer your questions... we were using i2b2, not SHRINE. All the timeout settings were unchanged from their 1.7.09c default settings. In fact, when I ran the same query on ACT SHRINE 1.25.4, it was *not* timing out. The query ran in ACT, we got results back from other sites, and from our site, too, after about 90 seconds. Again, do you believe this could be a setting we need to update in FastCGI for Apache/PHP? --Mark A for Keck/USC |
| Comment by Mark Abajian [ 28/Aug/19 ] |
|
Hello, Mike and i2b2 Team,
We located a remedy. It is the TimeOut directive in Apache. Question: Is there a preferred timeout setting for Apache? If so, how many seconds? We are now using 240. (It turns out that the default timeout is 60.) I was ignoring the TimeOut directive in Apache, because from what I was seeing online, the default setting in Apache is 300 seconds. But in Apache 2.4, the default TimeOut is 60 seconds. Putting "TimeOut 240" into our VirtualHost directive for i2b2 now allows the i2b2 webclient to behave properly, displaying the "run in the background" notification to the user after 180 seconds, instead of showing "QUERY CANCELLED" after 60 seconds. I regret any inconvenience this ticket may have caused. I have shared this information in the Google group, and hopefully the original poster over there will benefit from this information, too. Thank you. Kind regards, Mark A for Keck/USC |
| Comment by Mark Abajian [ 04/Sep/19 ] |
|
Hello, Mike and Reeta,
Before you close this ticket, can you please advise... Is there a preferred TimeOut setting for Apache? If so, how many seconds? We are now using 240 seconds, and that resolved the "QUERY CANCELLED" issue in our browser. (The default TimeOut in Apache 2.4 is 60 seconds, and that's what was causing our the browser to report "QUERY CANCELLED" after 60 seconds.) Mark |
| Comment by Reeta Metta [ 27/Sep/19 ] |
|
Hi Mark,
where in Apache is the 60 second time out set? I did not see it in httpd.conf file |
| Comment by Mark Abajian [ 27/Sep/19 ] |
|
Hello, Reeta, The 60-second timeout is the default in Apache 2.4 and up.
We counteract that for our webclient by setting it specifically for our VirtualHost. We put our VirtualHost directives into the file /etc/httpd/conf.d/ourhost.ourdomain.com.conf. And we put the "TimeOut 240" inside the VirtualHost directive. I hope this information helps. Let me know if you have any other questions. Mark Abajian |
| Comment by Reeta Metta [ 02/Oct/19 ] |
| Scope for 1.7.12 will be documentation update to reflect Apache 2.4 and higher Server timeout change |
| Comment by Reeta Metta [ 31/May/22 ] |
| the msg will now display the query has timed out instead of query is cancelled, but will still run in the queue and complete. |