Up: Massive Remote Batch Visualizer
The main goal of this dCSE project is to port AVS/Express DDR to the Cray XT4
hardware provided by HECToR (Phase2a). This will allow the visualization of
large datasets that currently exceed the capabilities of our GPU-based
visualization systems. Working with Materials Science we have access to datasets
that are typically 50-500GB in size. The datasets provide material density
data acquired by CT scanning equipment, some of which is provided by the Henry
Moseley X-Ray Imaging Facility at Manchester . We also expect to
acquire datasets from the I12 JEEP beamline at the Diamond Light Source, RAL. It
should be noted that AVS/Express DDR is not limited to this type of data and so
will be of use to researchers working with other forms of data. In particular
AVS has NetCDF  and HDF5  readers in addition to the
generalised AVS Field reader which allows many forms of data to be described and
read in to the application.
The dCSE Project
10 months of development time were allocated to the project, beginning May 2009,
with the possibility of 2 months of NAG CSE support to assist with optimisation
during the 10 month period. This option was not taken up as it was deemed
unnecessary to conduct very low level optimisation due to the interactive nature
of the application.
The main development task was to modify the AVS/Express code such that it could
operate within the HECToR runtime environment. This was broken down in to the
- Provide a mechanism to allow the AVS/Express main application (network
editor, module user interfaces, visualization window) to operate on the login
nodes, where X11 functionality is available, yet communicate with parallel
module and rendering processes executing on the backend nodes. Modifications
to the AVS source tree should be kept to a minimum, avoiding significant
architectural changes that would impact other platforms. This will make
AVS/Express appear more like the open source
ParaView [9,10,11] application in structure.
- Modify the image compositing architecture within AVS/Express so that it can
use MPI on the backend nodes. At the time of development an open source image
compositing library is used by AVS/Express. This provides no MPI functionality
and only supports dynamically linked applications.
- Optimise the existing MPI communication within the AVS/Express. It is
known that the parallel renderer uses point-to-point communication
which is expected to reduce scalability to large processor counts.
- Provide AVS networks that demonstrate the common visualization tasks that
users will perform.
AVS/Express must be run from a login node with X11 forwarded over the ssh
connection to the user's X server. The X server must support the GLX
protocol. This is not a particularly efficient method of running an X11
application and results in the overall rendering frame rate having an upper bound
that cannot be improved by parallel rendering because the limiting factor
becomes the time needed to transfer the image from the login node to the user's
X server. In the case of the AVS renderer the upper bound is approximately 5.0
frames per second (fps) for a 512x512 window and 1.1 fps for a 1024x1024 window
when rendering remotely to a test desktop system running linux. While this
appears to be low the visualization does in fact remain sufficiently responsive
that the user can interact with the visualization. We are more concerned with
the ability to scale out the memory usage to increase the size of dataset that
can be visualized.
The AVS/Express source tree has been compiled with the default GNU compiler
(currently 4.4.2). This is one of the compilers supported by the AVS build
process for 64bit linux platforms (the other being the Intel compiler). The
group in Manchester have a source code agreement with AVS allowing access to the
AVS/Express source tree although certain components are only available as
libraries. The build process is to compile the AVS base executable which
then reads the V module description files (AVS/Express uses its own module
description language named V). Processing of the V files generates
C/C++/Fortran code for those modules which is then compiled as part of the build
process. This produces the final express executable.
Up: Massive Remote Batch Visualizer