Difference between revisions of "ViSUSpy"

From
Jump to: navigation, search
(Write data to an IDX file)
(NumPy utils)
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
ViSUSpy is a library that allows the user to use the ViSUS framework from python.
+
The OpenVisus library is also available in python through a series of packages.
 +
 
 +
You can install the library using python pip:
 +
<syntaxhighlight lang="python">
 +
python -m pip install OpenVisus
 +
python -m OpenVisus configure
 +
</syntaxhighlight>
  
 
In the following sections we provide some examples on how to use it.
 
In the following sections we provide some examples on how to use it.
Line 8: Line 14:
  
 
<syntaxhighlight lang="python">
 
<syntaxhighlight lang="python">
from visuspy import *
+
from OpenVisus import *
from VisusKernelPy import *
 
from VisusIdxPy    import *
 
from VisusDbPy    import *
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
These modules allow the basic interactions with IDX datasets, which can be either local or remote (served by a [[ViSUS Server]]).
 
These modules allow the basic interactions with IDX datasets, which can be either local or remote (served by a [[ViSUS Server]]).
  
'''Important''': your code that interacts with IDX datasets has always to attach the ''IdxModule'' first, as following:
+
'''Important''': your code that interacts with IDX datasets has always to attach the ''DbModule'' first, as following:
  
 
<syntaxhighlight lang="python">
 
<syntaxhighlight lang="python">
IdxModule.attach()
+
DbModule.attach()
 
# your program code
 
# your program code
IdxModule.detach()
+
DbModule.detach()
 
</syntaxhighlight>
 
</syntaxhighlight>
  
=== NumPy utils ===
+
=== Read data into a NumPy array ===
  
 
In order to facilitate interoperation with existing python scientific libraries as NumPy we provide two function to convert data arrays from/to ViSUS Array an NumPy array.
 
In order to facilitate interoperation with existing python scientific libraries as NumPy we provide two function to convert data arrays from/to ViSUS Array an NumPy array.
Line 31: Line 34:
 
import numpy
 
import numpy
  
# convert our buffer to a numpy array.
+
# load a remote dataset
# This is useful when we want to read data from
+
dataset=LoadDataset("http://atlantis.sci.utah.edu/mod_visus?dataset=BlueMarble")
# and IDX dataset and use this array with other libraries
 
numpy_array=convertToNumPyArray(in_visus_array)
 
  
# convert a numpy array to a ViSUS array
+
# get the bounding box of the dataset
visus_array = convertToVisusArray(numpy_array)
+
# the box is defined by a set of 2 points (p1 and p2)
 +
# which define the two corners of a bounding box
 +
box=dataset.getLogicBox()
  
</syntaxhighlight>
+
# get the default field of a dataset
 +
field=dataset.getDefaultField()
  
== Write data in IDX format ==
+
access=dataset.createAccess()
  
<syntaxhighlight lang="python" line='line'>
+
# select the resolution we need
IdxModule.attach()
+
resolution = 21
  
# define a box which defines the bounding box
+
# initialize query
# of the grid where our data will be stored
+
query=BoxQuery(dataset, field, time,ord('r'))
# a box is defined by two points (p1 and p2)
 
# which define the two opposite corners of the box
 
# in this case the box has size 16x16x16
 
dataset_box=NdBox(NdPoint(0,0,0),NdPoint.one(16,16,16))
 
   
 
# create and IDX file where to to store the data
 
idxfile=IdxFile();
 
  
# set dataset box
+
query.logic_box=dataset.getLogicBox()
idxfile.box=NdBox(dataset_box)
+
query.end_resolutions.push_back(resolution)
  
# add integer field to the dataset
+
# execute the query
idxfile.fields.push_back(Field("myfield",DType.fromString("uint32")))
+
dataset.beginQuery(query)
 +
dataset.executeQuery(access,query)
  
# save the metadata on disk (no data are saved yet)
+
# convert our buffer to a numpy array.
bSaved=idxfile.save(self.filename)
+
# This is useful when we want to read data from  
self.assertTrue(bSaved)
+
# and IDX dataset and use this array with other libraries
 
+
data=Array.toNumPy(query.buffer,bSqueeze=True,bShareMem=False)
# load the dataset we just created   
 
dataset=Dataset.loadDataset(self.filename)
 
self.assertIsNotNone(dataset)
 
 
 
# create access to write the data
 
access=dataset.get().createAccess()
 
 
 
# write data slice by slice   
 
sampleid=0   
 
for Z in range(0,16):
 
  # create box to hold the Z slice of the data
 
  slice_box=dataset.get().getBox().getZSlab(Z,Z+1)
 
 
 
  # create write query
 
  query=QueryPtr(Query(dataset.get(),ord('w')))
 
  # set query box
 
  query.get().position=Position(slice_box)
 
     
 
  # initialize query
 
  self.assertTrue(dataset.get().beginQuery(query))
 
  # check if size of the query is equal to the dimension of our dataset
 
  self.assertEqual(query.get().nsamples.innerProduct(),16*16)
 
 
 
  # create a Visus Array to store the data
 
  buffer=Array(query.get().nsamples,query.get().field.dtype)
 
  # set the output buffer of the query to the array we just created
 
  query.get().buffer=buffer
 
     
 
  # convert our buffer to a numpy array  
 
  fill=convertToNumPyArray(buffer)
 
 
 
  # fill up the array with some values
 
  for Y in range(16):
 
    for X in range(16):
 
      fill[Y,X]=sampleid
 
      sampleid+=1
 
 
 
# execute query
 
success = dataset.get().executeQuery(access,query)
 
 
 
IdxModule.detach()
 
</syntaxhighlight>
 
 
 
== Read from IDX file ==
 
 
 
<syntaxhighlight lang="python" line='line'>
 
IdxModule.attach()
 
 
 
# open an IDX dataset
 
dataset=Dataset_loadDataset(self.filename)
 
# check it is valid
 
self.assertIsNotNone(dataset)
 
 
 
# get the bounding box of the dataset
 
# the box is defined by a set of 2 points (p1 and p2)
 
# which define the two corners of a bounding box
 
box=dataset.get().getBox()
 
 
 
# get the default field of a dataset
 
field=dataset.get().getDefaultField()
 
 
 
#create access
 
access=dataset.get().createAccess()
 
 
 
# here we read the datasets slice by slice
 
# making box queries
 
sampleid=0
 
for Z in range(0,16):
 
  slice_box=box.getZSlab(Z,Z+1)
 
     
 
  # define a read query
 
  query=QueryPtr(Query(dataset.get(),ord('r')))
 
  # set the box for the query
 
  query.get().position=Position(slice_box)
 
     
 
  self.assertTrue(dataset.get().beginQuery(query))
 
  # check the size of the output of the query
 
  print("query size", query.get().nsamples.innerProduct())
 
  # execute query
 
  self.assertTrue(dataset.get().executeQuery(access,query))
 
 
 
  # convert output buffer to a numpy array
 
  my_numpy_array=convertToNumPyArray(query.get().buffer)
 
  
IdxModule.detach()
 
 
</syntaxhighlight>
 
</syntaxhighlight>

Latest revision as of 21:06, 28 January 2020

The OpenVisus library is also available in python through a series of packages.

You can install the library using python pip:

python -m pip install OpenVisus
python -m OpenVisus configure

In the following sections we provide some examples on how to use it.

Library imports

To import and use the library in your python application you need to use the following imports:

from OpenVisus import *

These modules allow the basic interactions with IDX datasets, which can be either local or remote (served by a ViSUS Server).

Important: your code that interacts with IDX datasets has always to attach the DbModule first, as following:

DbModule.attach()
# your program code
DbModule.detach()

Read data into a NumPy array

In order to facilitate interoperation with existing python scientific libraries as NumPy we provide two function to convert data arrays from/to ViSUS Array an NumPy array.

import numpy

# load a remote dataset
dataset=LoadDataset("http://atlantis.sci.utah.edu/mod_visus?dataset=BlueMarble")

# get the bounding box of the dataset 
# the box is defined by a set of 2 points (p1 and p2)
# which define the two corners of a bounding box
box=dataset.getLogicBox()

# get the default field of a dataset
field=dataset.getDefaultField()

access=dataset.createAccess()

# select the resolution we need
resolution = 21

# initialize query
query=BoxQuery(dataset, field, time,ord('r'))

query.logic_box=dataset.getLogicBox()
query.end_resolutions.push_back(resolution)

# execute the query
dataset.beginQuery(query)
dataset.executeQuery(access,query)

# convert our buffer to a numpy array.
# This is useful when we want to read data from 
# and IDX dataset and use this array with other libraries
data=Array.toNumPy(query.buffer,bSqueeze=True,bShareMem=False)