This project has ported the AVS/Express DDR commercial visualization code to the Cray XT4 architecture provided by the HECToR service. Using this platform we have visualized data acquired by CT X-Ray scanning that is not able to be rendered on available GPU hardware. The system allows the user to interact with the visualization, often at an approximate rate of 5 frames per second. This is just about sufficient for interactive manipulation of the visualization. The inclusion of a new compositor layer and improvements to the parallel rendering code have allowed AVS/Express to be used with processor counts much higher than any previous installations of the product.
The use of the MPI forwarding layer and proxy has eliminated the need for a significant redevelopment of the communication code within AVS/Express, allowing the communication between the parallel module and rendering processes to remain within the vendor MPI domain. This allows users to develop their own AVS parallel modules using the existing AVS framework and move them without modification to the HECToR installation.
The ability to create custom applications with AVS/Express is one of its key strengths and will allow domain-specific applications to take advantage of its parallel renderer on HECToR. There are possible uses of AVS/Express in computational steering applications, where in-place visualization of large datasets is needed. Additionally simulation code may be developed as AVS parallel modules, allowing direct visualization of the simulation data.
An example of a user-developed application is the ParaFEM Viewer [17] used to visualize finite element analysis data generated by the ParaFEM library [18] available on HECToR. The viewer is an AVS/Express application with a custom user interface. It currently uses the standard AVS OpenGL renderer, which does not offer any parallel rendering facilities. Given the size of dataset that can be generated by FEM code it would be useful to build the ParaFEM Viewer application against the AVS/Express DDR product on HECToR. This would give the viewer access to the parallel renderer within DDR. Work would be needed to modify the application so that it used the AVS parallel modules rather than the standard serial modules. There are several custom modules used within the ParaFEM Viewer application and so these would need to be modified to use the parallel module framework.
The XPMT library and the 2-3 Swap compositing implementations can be used in other projects (most usefully in visualization applications) where the user interface is run as part of an MPI job. Non-AVS custom viewer applications (possibly small, lightweight viewers for specific data formats) can be developed where the user interface process runs on the login node and communicates with backend rendering processes. The developer can use MPI for all communication, avoiding the need to develop a non-MPI communication layer for user interface communication to the backend nodes. The 2-3 Swap code allows backend rendering processes to composite their images to form a final complete image.
We are hoping to acquire more datasets once the Diamond Light Source I12 beam line is operational. As imaging hardware improves we expect to see larger datasets becoming available.
Aspects of the work detailed in this report have been accepted for publication in Proceedings of the Theory and Practice of Computer Graphics Conference 2010 (EGUK) and have been submitted for publication to the Philosophical Transactions of the Royal Society A as part of the All Hands Meeting 2010. The work has also been presented at the JISC vizNET 2009 Conference [19].