Debugger Message Queue Access
The model that the debugger interface uses is organized around (virtual) separate message queues for each communicator. The debugger interface requires a list of active communicators. Since communicator construction is a relatively heavyweight operation, this list is maintained whether or not the debugger support is enabled.
MPIR_Comm_list contains a pointer to the head of the list, along with a sequence number. The sequence number is incremented any time a change is made to the list; this allows the debugger to detect when it needs to rebuild its internal representation of the active communicators.
The list of all communicators is maintained by two routines in
MPIR_Comm_create (creates a new communicator) and
MPIR_Comm_release (frees a communicator if the reference count is zero). The structure that contains this list is defined in this file and the variable name is
MPIR_All_communicators. This is a global variable to allow the debugger to easily find it.
These routines in turn call two macros,
that are defined in
mpiimpl.h to call routines whose implementations are in
src/mpi/debugger/dbg_init.c. These routines are
void MPIR_CommL_remember( MPID_Comm * ); void MPIR_CommL_forget( MPID_Comm * );
The receive queues are part of the device implementation; in the CH3 device, they are implemented in the file
src/mpid/ch3/src/ch3u_recvq.c. Normally, this file does not export the queues directly (the head and tail pointers are
static). To support message queue debugging, the variables
MPID_Recvq_unexpected_head_ptr are exported; these are pointers to the variables that hold the heads of those two lists.
The definition of the "send queue" is somewhat vague. One definition is that it is the queue of user-created
MPI_Requests that have not been completed with an MPI completion call such as
MPI_Wait. Other definitions could include any pending send operations in the communication layer; this could include pending blocking sends in a multi-threaded application.
This is done by adding two functions (really macros, so that they can be easily made into no-ops when debugger support isn't included) that add and remove user send requests from a separate list of requests.
The strategy is to add send requests to the special send queue when they are created within the nonblocking MPI send routines (e.g.,
MPI_Isend). Requests are removed when the
MPIR_Request_complete routine is called (this routine is in
src/mpi/pt2pt/mpir_request.c). The list of send requests is maintained by the routines
MPIR_Sendq_forget which are defined in
To both test the interface and to provide a example of the sorts of operations performed by a debugger that makes use of this interface, the file
src/mpi/debugger/tvtest.c (along with the support file
src/mpi/debugger/dbgstub.c) provides a simple example of using these routines to access and display the message queues.