Coupler infrastructure

The coupler has the following main functions: (a) initialisation of all required communicators, (b) definition of the logical relations and communication maps between the MPI topologies of the DNS and MD applications and (c) data transfer operations. Below we provide a brief description of the algorithms implemented in each sector.

(a) The continuum solver and the molecular dynamics code run concurrently using the multiple program multiple data paradigm. The initialisation of the global communicator (MPI_COMM_WORLD) is followed by a split into continuum (CFD_COMM) and molecular (MD_COMM) communicators corresponding to the DNS and MD applications. In addition, an inter-communicator is also defined, which includes only the processes from the continuum and molecular dynamics which are physically connected by the coupler.

(b) Using the problem parameters (e.g. DNS grid specifications, physical domain extents) the algorithm identifies the required coupling between the DNS and MD simulations and stores this as a mapping between adjacent MPI ranks in both codes. This ensures localised MPI communication - preserving the individual scaling of the coupled codes.

(c) Using the logical layout of processes and the internal maps, the coupler implements all data transfers between the applications during the computation. As the spatial relationship between MPI ranks is established during the initialisation, coupled data transfer subroutines require only simple MPI type send-receive calls. This guarantees the generality of the developed modules and easy integration of new continuum or MD solvers, or new coupling schemes.

The names of the main interface subroutines of the coupler with a brief description of their functionality are listed below. The coupler flowchart is presented in Figure 1 .

! coupler_create_comm (cfd+md) splits MPI_COMM_WORLD, creates 
!                              intercommunicator between CFD and MD
! coupler_create_map  (cfd+md) creates correspondence maps between 
!                              the CFD grid and MD domains
! coupler_cfd_init    (cfd)    initialises coupler and CFD parameters
!                              using data from MD or
! coupler_md_init     (cfd)    initialises coupler and MD parameters 
!                              using data from CFD or
! coupler_send_data   (cfd+md) sends grid data exchanged between 
!                              realms ( generic interface)
! coupler_recv_data   (cfd+md) receives data exchanged between 
!                              realms ( generic interface)
! coupler_cfd_get     (cfd)    returns coupler internal parameters 
!                              for CFD realm
! coupler_md_get      (md)     returns coupler internal parameters 
!                              for MD realm

Figure 1: Flowchart representation of the DNS-MD coupler,
the coloured boxes represent coupler sectors.
Image coupler_flow_chart_revb

Lucian Anton 2012-05-31