VISAN Developer Reference

This document provides information for programmers who are interested in extending the functionality of VISAN by implementing their own modules. We also provide instructions on how to add third party Python Modules/Packages to VISAN.

Contents

Extending VISAN

VISAN already comes with a broad set of functions, but you may want to create your own special functions that you want to use within VISAN. It is actually very easy to wrap your functions into one or more modules and add these to VISAN. The basic approach is as follows:

  1. Create your function
  2. Save your code into a Python (.py) file
  3. Let VISAN know where your module is located
  4. Import your module in VISAN

Create your function

Let us take the example of creating a median() function. This function will return the median of the elements in an array. A very simple implementation would be:

>>> def median(a):
...     a = asarray(a)
...     return sort(a)[a.size / 2]

On the first line we convert the array argument a into a Numpy array. The next line sorts the array and from this sorted array the element that is located exactly halfway is returned.

If you enter this function definition in VISAN you can immediately start using it:

>>> median((4,2,5,7,2))
4

Save your code into a file

If you had already typed your function in VISAN you can save this function to a file by using the 'Save Log to script...' menu option from VISAN and stripping everything from the logfile but your function definition. Otherwise you could just create a new text file (give it the .py extension) and enter your function definition there. For our example we will put our median() function in a file called myfuncs.py.

When it comes to using .py files it is important to realize the difference between a Python module (which is what we are creating here) and a VISAN script. A VISAN script is something you execute using the 'Load and Execute Script...' menu option in VISAN (or by passing the file as an argument when you start VISAN from the command line). A module, however, is something you 'import' into VISAN (see also Import your module). Although you could use a single Python file both as a script and as a module, depending on how you use it there are some restrictions on what should be in the file. If you run your Python file as a script from VISAN it is run inside the 'global namespace'. When VISAN starts it will automatically add a lot of functions and other definitions to this 'global namespace'. If you use your file as a module then your file won't have access to all these functions and you thus need to add these definitions yourself. In general, if your code works as a module it will also work as a VISAN script, but the opposite is not always true.

For instance, in our median() example we use several items from the Numpy package: asarray(), sort(), and .size. In VISAN all numpy definitions are already available to you, but if you write a module you have to import these Numpy definitions yourself. So, in our example, we would have to put the following code in our myfuncs.py file (note the import on the first line):

from numpy import *

def median(a):
    a = asarray(a)
    return sort(a)[a.size / 2]

The file myfuncs.py is what is known in Python terminology as a pure Python module, which is a .py file containing Python code. There are also two other types of modules in Python. There is the extension module which is a module written in a lower level language such as C/C++ and there is the package which is a directory containing several other modules together with a __init__.py package initialization file. More information about these three kinds of modules can be found in the Python Documentation. Most of what we discuss in the sections below holds for all three types of modules, but we will put our focus on the pure Python modules.

Let VISAN know where your file is located

Now that we have our module file we need to put it in a proper location and let VISAN know about it. There are several ways you can do this:

  1. Put your module in one of the locations that is already in the module searchpath.
    The module searchpath is a list of directories where the Python engine of VISAN will look when you try to import a Python module. You can inspect the module searchpath by printing the sys.path variable in VISAN. The preferred location to store your own modules is in the directory that ends with /site-packages. If <visandir> is your VISAN installation directory, then on Windows this directory will be <visandir>\Lib\site-packages, on Linux it will be <visandir>/lib/python-2.7/site-packages, and on macOS this folder is inside the VISAN.app bundle: VISAN.app/Contents/lib/python-2.7/site-packages
  2. Open VISAN from the directory where your module is located (not supported on macOS).
    The first entry in the module searchpath is the directory from which the VISAN application was started. This means that you can use the command prompt to just go to the directory where your module is located and start VISAN from there. Using this approach can be very convenient if you are still in the stage of developing your module.
  3. Create a path (.pth) file that specifies the location of your module(s).
    When VISAN/Python starts it will automatically look for .pth files in the .../site-packages directory (see above) and add all directories that are mentioned in these text files to the module searchpath. So if, for instance, your modules are located in /home/username/mymodules then you can create a file mymodules.pth in the .../site-packages directory containing just the line:
    /home/username/mymodules
    
    You can give your .pth file any name you want; it does not have to resemble any module or directory it points to.
  4. Set the PYTHONPATH environment variable.
    All paths that you include in the PYTHONPATH environment variable will be automatically added to the module searchpath when VISAN/Python is started (and will thus appear in you sys.path). You set the PYTHONPATH similar to the way you would set your system PATH variable. Using this approach is probably the best way to go if you choose not to install your modules inside the VISAN installation directory.
  5. Modify the module searchpath (sys.path) by hand.
    When VISAN is started you can always manually change your sys.path. For example:
    >>> sys.path.append('/home/username/mymodules')
    
    Manually changing the module searchpath can be convenient if you want to have tight control over the ordering of the directories in your searchpath. You should however be aware that changes made to your sys.path during a VISAN session will be gone once you quit the application.
  6. Install your module via a setup.py file.
    If you also want to share your module(s) with other people you might want to create a setup.py installation script that can be run by users who want to install your module(s). A lot of freely available Python modules come with such an installation script. You can find information about creating and using setup.py scripts in the Python Documentation. In the section Installing Third Party Python Modules below we will provide more information on how to install Python modules in VISAN using a setup.py script.

It is important that you are careful when choosing a name for your module. If there is already another module with a similar name and if that module exists earlier in the module searchpath then Python will use that module instead of yours.

Import your module

The final step is to import your module in VISAN. For this we use the 'import' statement from Python. The module name you use with 'import' should be the name of your Python file without the .py extension. There are three ways to import items from a module:

  1. You can use the default way of importing a module: import <module>. This way of importing requires you to include the module name when you want to call items from the module:
    >>> import myfuncs
    >>> myfuncs.median((4,2,5,7,2))
    4
    
  2. You can import everything from a module into your namespace (this is what we did with numpy in our myfuncs module). You do this by using the 'from <module> import *' construct:
    >>> from myfuncs import *
    >>> median((4,2,5,7,2))
    4
    
  3. You can import only the items you need from a module. These items will be included in your local namespace. You do this by replacing the * from the previous example by a comma separated list of the items you need: from <module> import <item1>, <item2>, ....
    >>> from myfuncs import median
    >>> median((4,2,5,7,2))
    4
    

For more information regarding Python modules (including information about extension modules and packages) please consult the Python Documentation.

Installing Third Party Python Modules

Installing Python modules in VISAN can be done by running the setup.py script of the module using the executescript function. For instance, if the setup.py file is in the directory /home/user/package you would use the following VISAN command to install the module:

>>> executescript('/home/user/package/setup.py', mainargs=['install'])

You should also read the installation instructions that came with the module. You might be required to change the contents of the setup.py file first or pass some additional options to mainargs in order for the installation to work.

As always, more information about installing modules can be found in the Python Documentation. There is even a whole section in the documentation dedicated to installing Python modules.

The Design of VISAN

In this section we try to explain a bit more about the underlying structure of the VISAN application. From the outside VISAN may look like just one application, but underneath VISAN is a combination of various freely available open source toolboxes together with code that was written specifically for VISAN. Below we will provide more information about each of the components that make up VISAN and explain something about the directory structure of the installed VISAN application.

Directory Structure Overview

The directory structure differs a bit between the installation of VISAN on Windows, Linux, and macOS. Below we will provide on overview of the most important directories and what these directories contain. The term <visandir> is used to denote your VISAN installation directory.

Windows
directorycontents
<visandir>\binThe VISAN executables and all DLLs (the shared libraries for each of the included packages).
<visandir>\dataAll data files, such as images and GSHHS coastline data (as used in the Worldplots).
<visandir>\definitionsThe location of the CODA Product Format Definition (.codadef) files.
<visandir>\Doc\htmlThe documentation for VISAN, HARP, and CODA.
<visandir>\examplesSeveral VISAN sample scripts.
<visandir>\LibThis is where all Python modules are stored.
<visandir>\Lib\site-packagesThe location of all third party Python modules (for e.g HARP, CODA, numpy, wxPython, etc.).
<visandir>\unitsThe location of the udunits2 xml definition files as used by HARP.
Linux
directorycontents
<visandir>/binThe VISAN executables.
<visandir>/dataAll data files, such as images and GSHHS coastline data (as used in the Worldplots).
<visandir>/definitionsThe location of the CODA Product Format Definition (.codadef) files.
<visandir>/doc/htmlThe documentation for VISAN, HARP and CODA.
<visandir>/examplesSeveral VISAN sample scripts.
<visandir>/libAll shared libraries for each of the included packages.
<visandir>/lib/python2.7This is where all Python modules are stored.
<visandir>/lib/python2.7/site-packagesThe location of all third party Python modules (for e.g HARP, CODA, numpy, wxPython, etc.).
<visandir>/unitsThe location of the udunits2 xml definition files as used by HARP.
macOS
directorycontents
examplesSeveral VISAN sample scripts.
VISAN.appThe VISAN application.
VISAN.app/Contents/dataAll data files, such as images and GSHHS coastline data (as used in the Worldplots).
VISAN.app/Contents/definitionsThe location of the CODA Product Format Definition (.codadef) files.
VISAN.app/Contents/doc/htmlThe documentation for VISAN, HARP, and CODA.
VISAN.app/Contents/libAll shared libraries for each of the included packages.
VISAN.app/Contents/lib/python2.7This is where all Python modules are stored.
VISAN.app/Contents/lib/python2.7/site-packagesThe location of all third party Python modules (for e.g HARP, CODA, numpy, wxPython, etc.).
VISAN.app/Contents/unitsThe location of the udunits2 xml definition files as used by HARP.

Python

Python is a high level interpreted programming language and the Python package provides an interpreter and a broad set of default modules for this language. The VISAN application is at its core based on a default Python installation. Almost everything from the Python installation is included in VISAN. VISAN however doesn't include the python executable. This program is replaced by the visan executable which is actually nothing more than a small replacement program for the python executable containing some small changes to the startup procedure. In fact, the majority of VISAN is either implemented in Python source code or through Python extension modules.

VISAN subpackage

As already stated, the VISAN application consists of many components, both third party toolboxes and VISAN specific components. The term VISAN subpackage is what is used to describe all VISAN specific components. The subpackage contains three main components:

Numpy

Numpy is a Python package for high performance mathematical operations on arrays of numerical data. The package is used by both HARP/CODA and the VISAN subpackage and is also needed to interact with product data retrieved by HARP from the VISAN command line.

CODA

CODA is the direct product interface of the Atmospheric Toolbox. VISAN includes the Python interfaces of CODA. The other components of CODA, such as the command line tools or the MATLAB and IDL interfaces are not included with VISAN.

HARP

HARP is the high-level harmonized data interface of the Atmospheric Toolbox. VISAN includes the Python interfaces of HARP. The other components of HARP, such as the command line tools are not included with VISAN.

VTK

VTK, the Visualization ToolKit, is a software system for 3D computer graphics, image processing, and visualization. VISAN uses the VTK libraries in combination with some special VTK classes written for VISAN to display the 2D Plots and Worldplots.

wxPython

wxPython is the Python interface for wxWidgets, a cross-platform GUI framework written in C++. All Windows, menus, buttons, etc. that you see in VISAN are created using the wxPython framework. The Python package for wxPython is called wx.

wxvtk

wxvtk is a bridge class that allows embedding of VTK visualizations in the wxWidgets/wxPython GUI. The wxvtk (wxVTKRenderWindowInteractor) C++ files are included in the VISAN subpackage.

PROJ

PROJ is a projection library and is used by one of the VISAN specific VTK classes for the Worldplot. The projection library is used to map latitude/longitude coordinates into one of the several 2D projections that is supported by the VISAN Worldplot.

HDF4

The HDF4 library is used by the HARP/CODA packages to read and write data in HDF4 format.

HDF5

The HDF5 library is used by the HARP/CODA packages to read and write data in HDF5 format.

Common Libraries

Many of the aforementioned packages depend on other libraries for handling e.g. pictures or performing compression. Some packages (such as wxPython and VTK) provide versions of these libraries themselves, others (such as HDF4) require you to have such libraries installed on your system. To make sure all the packages use the same version of these common libraries we included one single version of the following libraries with VISAN: libjpeg, libpng, libtiff, zlib, and szlib. For wxPython and VTK we thus also disabled the internal versions of the common libraries and we built these two packages using our own provided common libraries.

Other tools

If you download the source package of VISAN and look at the packages that come with it you will see that there are more packages included than we have mentioned so far. These other packages are packages that are only needed to build VISAN. We will briefly explain the role of these packages in this section.

SWIG

SWIG is an application that can create wrapper code to make code written in C/C++ available into several languages, among which Python, which is what is used for VISAN. The CODA and VISAN sources are wrapped to Python using this tool.

CFFI

CFFI is the C Foreign Function Interface and has a similar purpose as SWIG. The HARP sources are wrapped to Python using this tool.

The CFFI toolset as included with VISAN consists of the cffi package itself and its dependencies libffi, pycparser, and setuptools.

CMake

CMake is a cross platform 'make' tool. It is only included to be able to build VTK which depends on CMake for its build process.