Graceful Termination of Connections

Multicloud Defense Gateway can choose to terminate an established flow for multiple reasons such as:

  • Termination based on the policy. For example, FQDN filtering can only be applied after the flow is established.

  • IDS/IPS can deem any packet in the flow that is sent by either the client or the server to be unsafe and can choose to terminate an established flow.

  • Proxy service on the Multicloud Defense Gateway decides to terminate the flow after the flow is established.

  • When one of the timers in the Multicloud Defense Gateway TCP stack decides that the flow is no longer active or alive.

  • Flow termination during certain configuration changes such as PRS updates, gateway setting changes and so on.

  • Flow termination when the gateway is decommissioned (controller initiated - disable/upgrade/scale-in).

Currently, when a Multicloud Defense Gateway chooses to terminate an established flow for any of the above reasons, it does so without informing the client and the server about the termination (except if there is FQDN Filtering with Reset on Deny turned on). This causes the client and server to rely on TCP or application timeouts to detect the loss of connection, causing application outages.

For TCP flows, Multicloud Defense Gateway introduces a graceful termination mechanism which causes the gateway to send a TCP Reset to the client (initiator) when the gateway stops the flow. This should enable the client TCP stack to terminate the connection quickly, enabling the applications to attempt to re-establish the interrupted flow, thereby minimizing traffic disruption. This applies to all kinds of flows - forwarded, forward proxied, and reverse proxied, that are handled by the Multicloud Defense Gateway.

Also, when a Multicloud Defense Gateway data plane goes down unexpectedly (due to a software issue), this reset mechanism does not apply. Clients will continue to rely on application timeouts to recover.

Troubleshooting

To find flows that are terminated with a TCP Reset by the Multicloud Defense Gateway, download the traffic summary (from the controller) as a CSV and search for RESET. It will be the last connection state for the ingress flow. Connections that are terminated naturally will not have this state as the last state. For non-TCP flows, the last connection state is always AGED OUT.