Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
-
Windows
-
Rank:0|i00107:
Description
We currently have a proof-of-concept i2b2 instance running that I am using as a jump-off point for our next project. There are a few problems (with the Observation Fact table) that I have noticed so far:
1) Observation Fact looks like it is storing 4-5 times as much data as the incoming feed. This is in part due to the design of the table itself (much more like a property bag or tripple). Also, live values appear to be stored instead of dimensional references.
2) There look to be many fields (for our cases) we do not populate. This isn't a big deal at first, but when this design of table gets large the memory footprint becomes very large.
3) It looks like the Index/Data space ratio is about 3 - 1. Our feed data source is about 5 gigs in size. Loaded into Observation Fact it is about 27 gigs in size, 20 gigs of which is index.
I don't know if there is anything that can be done about this (since it appears by design) but I was wondering if there are any plans to review the efficiency of this storage model?
1) Observation Fact looks like it is storing 4-5 times as much data as the incoming feed. This is in part due to the design of the table itself (much more like a property bag or tripple). Also, live values appear to be stored instead of dimensional references.
2) There look to be many fields (for our cases) we do not populate. This isn't a big deal at first, but when this design of table gets large the memory footprint becomes very large.
3) It looks like the Index/Data space ratio is about 3 - 1. Our feed data source is about 5 gigs in size. Loaded into Observation Fact it is about 27 gigs in size, 20 gigs of which is index.
I don't know if there is anything that can be done about this (since it appears by design) but I was wondering if there are any plans to review the efficiency of this storage model?