CH3 VC protocol

From Mpich
Jump to: navigation, search

This document describes the protocol CH3 uses when closing a VC. The close protocol ensures that both sides of the connection are ready to close and all communication has finished.


When a VC is released, and its ref count drops to 0, the VC is closed. Similarly in MPID_Finalize(), MPIDI_PG_Close_VCs() is called which goes through all VCs in all process groups and closes them. A VC is closed by calling MPIDI_CH3U_VC_SendClose().

In MPID_Finalize(), MPIDI_CH3U_VC_WaitForClose() is called to wait for all VCs to finish closing. In MPIDI_CH3U_VC_SendClose(), outstanding_close_ops is incremented to keep track of the number of VCs that are in the process of closing. When the close protocol completes for a VC, outstanding_close_ops is decremented. In this way, MPIDI_CH3U_VC_WaitForClose() can simply keep calling progress while outstanding_close_ops is non-zero.

All VCs start in the INACTIVE state at init time. When a message is sent over a VC the VC is connected by the channel, if necessary, and its state is changed to ACTIVE.

State Diagram

Below is the state diagram for the close protocol. There are eight states in the protocol

  • ACTIVE -- Communication has been initiated and has not yet been completed
  • LOCAL_CLOSE -- Local process has initiated the close protocol
  • REMOTE_CLOSE -- Remote process has initiated the close protocol
  • CLOSE_ACKED -- Local process has initiated the close protocol and has acknowledged the remote process's CLOSE packet
  • CLOSED -- Local process has received and ack from the remote process
  • INACTIVE -- Either no communication has been initiated, or the VC has been succesfully closed and resources have been released
  • INACTIVE_CLOSED -- The process is in finalize and this VC should not be opened
  • MORIBUND -- This connection has terminated prematurely. There may be outstanding sends or receives on this VC. Currently, MORIBUND is a terminal state, but in the future it may be possible to activate the VC again.

In the labels for the transitions, the part above the line indicate the event triggering the transition. The part below the line describes the actions taken in the transition (In the diagram a label for a transition generally appears to the right of the edge). Note that the error transitions are shown with red and green edges to simplify the diagram. A key describing the transitions is shown at the bottom of the diagram.

The ImageMap extension is not installed.

Old version

The ImageMap extension is not installed.