Difference between revisions of "MIDX examples"

Revision as of 00:15, 30 May 2018

MIDX is an XML metadata file which defines how multiple IDX file are combined in on single dataset. This, for example, allows dynamic blending of the content of different volumes or also to perform operations (e.g., scripting) using different fields from different data sources.

<dataset typename='IdxMultipleDataset'>

  <dataset url='file://$(CurrentFileDirectory)/../2kbit1/zip/rowmajor/visus.idx' name='first'>
    <M><translate x='0' y='0' z='0'/></M>
  </dataset>

  <dataset url='file://$(CurrentFileDirectory)/../2kbit1/zip/rowmajor/visus.idx' name='second'>
    <M><translate x='0' y='0' z='0'/></M>
  </dataset>
  
</dataset>

Two visible males side by side:

<dataset typename='IdxMultipleDataset'>

  <dataset url='file://$(CurrentFileDirectory)/../male/visus.idx' name='first'>
    <M>
      <translate x='0' y='0' z='0'/>
      <scale     x='1' y='1' z='3'/>
    </M>
  </dataset>

  <dataset url='file://$(CurrentFileDirectory)/../male/visus.idx' name='second'>
    <M>
      <translate x='2048' y='0' z='0'/>
      <scale     x='1'    y='1' z='3'/>
    </M>
  </dataset>
  
</dataset>

2kbit1 overlapping Borg?time=13

<dataset typename='IdxMultipleDataset'>

  <dataset url='file://$(CurrentFileDirectory)/../2kbit1/zip/rowmajor/visus.idx' name='first'>
    <M><translate x='0' y='0' z='0'/></M>
  </dataset>

  <dataset url='file://$(CurrentFileDirectory)/../borg/visus.idx?time=13' name='second'>
    <M><translate x='0' y='0' z='0'/></M>
  </dataset>
  
</dataset>

Blend David timesteps

<dataset typename='IdxMultipleDataset'>
  <dataset url='file://$(CurrentFileDirectory)/../david_subsampled/visus.idx?time=0' name='0' />
  <dataset url='file://$(CurrentFileDirectory)/../david_subsampled/visus.idx?time=1' name='1' />
  <dataset url='file://$(CurrentFileDirectory)/../david_subsampled/visus.idx?time=2' name='2' />
  <dataset url='file://$(CurrentFileDirectory)/../david_subsampled/visus.idx?time=3' name='3' />
</dataset>

Local MIDX. No caching

<dataset name="Example" url="file://./docs/examples/midx/visus.midx">
  <access name="MultipleAccess" type="MultipleAccess">
    <A type='disk' chmod='r' url='file://./docs/examples/midx/A/visus.idx' />
    <B type='disk' chmod='r' url='file://./docs/examples/midx/B/visus.idx' />
    <C type='disk' chmod='r' url='file://./docs/examples/midx/C/visus.idx' />
    <D type='disk' chmod='r' url='file://./docs/examples/midx/D/visus.idx' />
  </access>  
</dataset>

Local MIDX. Local IDX block caching

Note that is useless to create two local copies of child datasets. It would make more sense if you had remote datasets.

<dataset name="Example" url="file://./docs/examples/midx/visus.midx">
  <access name="MultipleAccess" type="MultipleAccess">
    <A type='multiplex'>
      <access type='disk' chmod='rw' url="file://$(VisusCacheDirectory)/A/visus.idx" />
      <access type='disk' chmod='r'  url='file://./docs/examples/midx/A/visus.idx' />
    </A>
    <B type='multiplex'>
      <access type='disk' chmod='rw' url="file://$(VisusCacheDirectory)/B/visus.idx" />
      <access type='disk' chmod='r'  url='file://./docs/examples/midx/B/visus.idx' />
    </B>
    <C type='multiplex'>
      <access type='disk' chmod='rw' url="file://$(VisusCacheDirectory)/C/visus.idx" />
      <access type='disk' chmod='r'  url='file://./docs/examples/midx/C/visus.idx' />
    </C>
    <D type='multiplex'>
      <access type='disk' chmod='rw' url="file://$(VisusCacheDirectory)/D/visus.idx" />
      <access type='disk' chmod='r'  url='file://./docs/examples/midx/D/visus.idx' />
    </D>
  </access> 
</dataset>

Local MIDX. Local MIDX block caching

All datasets must have the same IDX bitmask. Otherwise it won't work.

<dataset name="Example" url="file://./docs/examples/midx/visus.midx">
    
  <access name="DefaultAccess" type="multiplex" >
        
    <!-- here I do the up-midx caching -->
      <access type="DiskAccess" chmod='rw'  dir="./cached" compression="zip" />
        
    <!-- child access -->
    <access name="MultipleAccess" type="MultipleAccess">
      <A type='disk' chmod='r' url='file://./docs/examples/midx/A/visus.idx' />
      <B type='disk' chmod='r' url='file://./docs/examples/midx/B/visus.idx' />
      <C type='disk' chmod='r' url='file://./docs/examples/midx/C/visus.idx' />
      <D type='disk' chmod='r' url='file://./docs/examples/midx/D/visus.idx' />
    </access> 
  </access> 
</dataset>

Remote MIDX. Local MIDX block caching

Example for remote MIDX pin2d on atlantis:

<dataset typename="IdxMultipleDataset" url="file:///usr/sci/cedmav/LDAV/midx/pin2d.midx">
  <dataset name="A" url="$(protocol)://$(hostname):$(port)/mod_visus?action=readdataset&amp;dataset=pin2d/A" />
  <dataset name="B" url="$(protocol)://$(hostname):$(port)/mod_visus?action=readdataset&amp;dataset=pin2d/B" />
  <dataset name="C" url="$(protocol)://$(hostname):$(port)/mod_visus?action=readdataset&amp;dataset=pin2d/C" />
  <dataset name="D" url="$(protocol)://$(hostname):$(port)/mod_visus?action=readdataset&amp;dataset=pin2d/D" />
  <dataset name="E" url="$(protocol)://$(hostname):$(port)/mod_visus?action=readdataset&amp;dataset=pin2d/E" />
</dataset>

The MIDX processing is done server side (example: fieldname is 'avg'). The client caches the MIDX blocks got from server.

<dataset name="Example" url="http://atlantis.sci.utah.edu/mod_visus?dataset=pin2d">
  <access name="DiskAccess"  type='multiplex'>
    <access name="Example" type="DiskAccess" dir="./cached" compression="zip" />
    <access type='network' chmod='r' compression='zip' url="http://atlantis.sci.utah.edu/mod_visus?dataset=pin2d" />
  </access> 
</dataset>

Remote MIDX. Local IDX block caching