Difference between revisions of "ViSUS Data Portal"

From
Jump to: navigation, search
(DataPortal internal settings)
 
(17 intermediate revisions by the same user not shown)
Line 29: Line 29:
  
 
From the main navigation bar you can reach three different areas that will be presented in the next sections.
 
From the main navigation bar you can reach three different areas that will be presented in the next sections.
 
You can change those and other settings from the ''local.php'' file, that you can mount when you run your docker as following:
 
 
<syntaxhighlight lang="bash">
 
docker run -it --rm -p 8080:80 -v my_local.php:/home/visus/dataportal/local.php visus/dataportal
 
</syntaxhighlight>
 
  
 
'''Note''': the default credentials are: username=''admin'', password=''password''.
 
'''Note''': the default credentials are: username=''admin'', password=''password''.
Line 40: Line 34:
 
[[File:Login.png|center|400px]]
 
[[File:Login.png|center|400px]]
  
=== Configure Server ===
+
=== Sample datasets ===
From here you can visualize and edit the list of the datasets that you are serving through this server. Those can be both remote datasets (i.e., with a web URL as in the case of the default dataset BlueMarble) or local (i.e., pointing to an actual file hosted on the server).
+
 
 +
You can use those sample datasets to test the upload, conversion and publishing through the portal.
  
The name of the dataset is colored in red or green to show you if that dataset is ready for streaming or not.
+
Asteroid (single file, raw format):
To update the server you can click the "Update Server" button to trigger a reload of the datasets (metadata) on the server.
+
* size: 500x500x500
 +
* data type: float32
 +
* download: [https://drive.google.com/open?id=1wHdWynj9jnIf5NrRnzSZyOHbOMwKmltQ Asteroid data]
  
[[File:Configure.png|center|600px]]
+
Heart CT scan (stack of images):
 +
* size and datatype will be automatically detected
 +
* download: [https://drive.google.com/open?id=1NLWe2FVhUUoEaaA8JZ2w61H_dT77z1La Heart data]
  
 
=== Manage Data ===
 
=== Manage Data ===
  
From here you can explore the data hosted on the server using the "File Manager" and execute conversions.
 
 
Currently the data portal provides interface to convert single or stack of image (or raw) data.  
 
Currently the data portal provides interface to convert single or stack of image (or raw) data.  
 +
 +
[[File:Convert.png|center|600px]]
 +
 +
From the Browse button the user can open the file manager to select a file (or a folder) to convert. The user can also drop new file in the window, and those will be uploaded on the server.
 +
'''Note''': if you are converting a stack of images put all (and only) the images you want to stack into a volume in the same folder.
  
 
[[File:File_manager.png|center|600px]]
 
[[File:File_manager.png|center|600px]]
  
The conversion process will look into the selected folder for images in order to estimate the size and datatype and automatically fill up the conversion parameters for you. The user can eventually customize those and run the conversion with the "Convert" button.
+
The conversion process will try to estimate the size and datatype and automatically fill up the conversion parameters for you (i.e., size and datatype). The user can eventually customize those and run the conversion with the "Convert" button.
 
 
[[File:Convert.png|center|600px]]
 
  
 
When a conversion process is launched a new entry in the conversion history will show up. This history allows the user to monitor the conversion processes taking place on the server and visualize the corresponding logs.
 
When a conversion process is launched a new entry in the conversion history will show up. This history allows the user to monitor the conversion processes taking place on the server and visualize the corresponding logs.
Line 63: Line 64:
  
 
[[File:Chistory.png|center|600px]]
 
[[File:Chistory.png|center|600px]]
 +
 +
=== Configure Server ===
 +
From here you can visualize and edit the list of the datasets that you are serving through this server. Those can be both remote datasets (i.e., with a web URL as in the case of the default dataset BlueMarble) or local (i.e., pointing to an actual file hosted on the server).
 +
 +
The name of the dataset is colored in red or green to show you if that dataset is ready for streaming or not.
 +
To update the server you can click the "Update Server" button to trigger a reload of the datasets (metadata) on the server.
 +
 +
[[File:Configure.png|center|600px]]
 +
 +
  
 
=== Explore Data ===
 
=== Explore Data ===
Line 68: Line 79:
 
From here you can visualize the data using the [[ViSUS_WebViewer | Web Viewer]].
 
From here you can visualize the data using the [[ViSUS_WebViewer | Web Viewer]].
  
[[File:Hearth.png|center|600px]]
+
[[File:Hearth.png|center|500px]]
 +
 
 +
== Advanced ==
 +
=== DataPortal internal settings ===
 +
 
 +
Some internal settings can be customized using a configuration file (i.e., ''local.php''), that can be mounted when you run your docker as following:
 +
 
 +
<syntaxhighlight lang="bash">
 +
docker run -it --rm -p 8080:80 -v my_local.php:/home/OpenVisus/dataportal/local.php visus/dataportal
 +
</syntaxhighlight>
 +
 
 +
The ''local.php'' file looks like the following:
 +
<syntaxhighlight lang="php">
 +
<?php
 +
  // Admin credentials for this server
 +
  $admin_user="admin";
 +
  $admin_password="password";
 +
 
 +
  // mod_visus_url
 +
  $mod_visus_url="http://localhost:8080/mod_visus?";
 +
 
 +
  /**** DO NOT CHANGE THE REST ****/
 +
  // Default server config file location (on the server's filesystem)
 +
  $default_config_file="/home/OpenVisus/visus.config";
 +
 
 +
  // Data directory
 +
  $data_dir="/data";
 +
 
 +
  // Data output directory
 +
  $out_data_dir="/converted";
 +
   
 +
  // visus executable
 +
  $visus_exe="/usr/local/lib/python3.5/dist-packages/OpenVisus/bin/visus";
 +
?>
 +
</syntaxhighlight>
 +
 
 +
Similarly the web viewer URL configuration can be modified mounting your custom ''config.js'' file as following
 +
 
 +
<syntaxhighlight lang="bash">
 +
docker run -it --rm -p 8080:80 -v /fullpath/to/your/config.js:/home/OpenVisus/dataportal/viewer/config.js visus/dataportal
 +
</syntaxhighlight>
 +
 
 +
This file will have to contain the definition of the server URL that will be accessed remotely (i.e., not anymore localhost), for example as following:
 +
<syntaxhighlight lang="javascript">
 +
DEFAULT_SERVER='http://myserver.mydomain:8080/mod_visus?';
 +
</syntaxhighlight>
 +
 
 +
=== Use persistant storage for your data ===
 +
 
 +
If you want to use your local data directly on the data portal and keep the converted multiresolution data available anytime you run your container you might want to mount this folder to the data portal data folder (''/data'') as following:
 +
 
 +
<syntaxhighlight lang="bash">
 +
docker run -it --rm -p 8080:80 -v /my/data/folder:/data visus/dataportal
 +
</syntaxhighlight>
 +
 
 +
If you want also the converted data to be persistent you can mount another folder for them (i.e., that will be mapped to the ''/converted'' data folder inside the container) as following:
 +
 
 +
<syntaxhighlight lang="bash">
 +
docker run -it --rm -p 8080:80 -v /my/data/folder:/data -v /my/converted_data/folder:/converted visus/dataportal
 +
</syntaxhighlight>

Latest revision as of 01:01, 20 March 2019

The ViSUS Data Portal allows to easily configure your data on the server, convert data to a data-streaming friendly format and visualize them directly from your browser. The ViSUS Data Portal is deployed as a Docker container. Here is how to get started.

Quick start

Start your Docker and login into your account. If you don't have installed follow these instructions: Docker Documentation

Now pull the ViSUS image:

docker pull visus/dataportal

This image contains:

The goal of the data portal is to allow to use all of the above tools from a web interface, so all you have to do is just run the container with

docker run -it --rm -p 8080:80 visus/dataportal

and point your browser to that same port (8080) on your host, as following: URL: http://localhost:8080

Home.png

From the main navigation bar you can reach three different areas that will be presented in the next sections.

Note: the default credentials are: username=admin, password=password.

Login.png

Sample datasets

You can use those sample datasets to test the upload, conversion and publishing through the portal.

Asteroid (single file, raw format):

Heart CT scan (stack of images):

  • size and datatype will be automatically detected
  • download: Heart data

Manage Data

Currently the data portal provides interface to convert single or stack of image (or raw) data.

Convert.png

From the Browse button the user can open the file manager to select a file (or a folder) to convert. The user can also drop new file in the window, and those will be uploaded on the server. Note: if you are converting a stack of images put all (and only) the images you want to stack into a volume in the same folder.

File manager.png

The conversion process will try to estimate the size and datatype and automatically fill up the conversion parameters for you (i.e., size and datatype). The user can eventually customize those and run the conversion with the "Convert" button.

When a conversion process is launched a new entry in the conversion history will show up. This history allows the user to monitor the conversion processes taking place on the server and visualize the corresponding logs. From this list is possible to add the converted (or still converting) dataset to the server using the "Add to server" button.

Chistory.png

Configure Server

From here you can visualize and edit the list of the datasets that you are serving through this server. Those can be both remote datasets (i.e., with a web URL as in the case of the default dataset BlueMarble) or local (i.e., pointing to an actual file hosted on the server).

The name of the dataset is colored in red or green to show you if that dataset is ready for streaming or not. To update the server you can click the "Update Server" button to trigger a reload of the datasets (metadata) on the server.

Configure.png


Explore Data

From here you can visualize the data using the Web Viewer.

Hearth.png

Advanced

DataPortal internal settings

Some internal settings can be customized using a configuration file (i.e., local.php), that can be mounted when you run your docker as following:

docker run -it --rm -p 8080:80 -v my_local.php:/home/OpenVisus/dataportal/local.php visus/dataportal

The local.php file looks like the following:

<?php 
  // Admin credentials for this server
  $admin_user="admin"; 
  $admin_password="password";

  // mod_visus_url
  $mod_visus_url="http://localhost:8080/mod_visus?";
   
  /**** DO NOT CHANGE THE REST ****/
  // Default server config file location (on the server's filesystem)	
  $default_config_file="/home/OpenVisus/visus.config";
  
  // Data directory
  $data_dir="/data";

  // Data output directory
  $out_data_dir="/converted";
    
  // visus executable
  $visus_exe="/usr/local/lib/python3.5/dist-packages/OpenVisus/bin/visus";
?>

Similarly the web viewer URL configuration can be modified mounting your custom config.js file as following

docker run -it --rm -p 8080:80 -v /fullpath/to/your/config.js:/home/OpenVisus/dataportal/viewer/config.js visus/dataportal

This file will have to contain the definition of the server URL that will be accessed remotely (i.e., not anymore localhost), for example as following:

DEFAULT_SERVER='http://myserver.mydomain:8080/mod_visus?';

Use persistant storage for your data

If you want to use your local data directly on the data portal and keep the converted multiresolution data available anytime you run your container you might want to mount this folder to the data portal data folder (/data) as following:

docker run -it --rm -p 8080:80 -v /my/data/folder:/data visus/dataportal

If you want also the converted data to be persistent you can mount another folder for them (i.e., that will be mapped to the /converted data folder inside the container) as following:

docker run -it --rm -p 8080:80 -v /my/data/folder:/data -v /my/converted_data/folder:/converted visus/dataportal