Summary
TCP keepalive helps detect broken network paths between two machines when the endpoints cannot immediately observe the failure.
libuv and TCP Keepalive
About Keepalive
The keepalive discussed here is different from HTTP keepalive. Here we are talking about TCP-level keepalive. Its purpose is to detect network failures between two communicating machines when neither endpoint can immediately observe that the path has broken.
HTTP keepalive means adding a keep-alive header so the server does not close the TCP connection after a single request, allowing the same connection to be reused for later requests.
The Linux kernel documentation for TCP keepalive is here:
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html
It defines:
1
2
3
4
5
6
| tcp_keepalive_time
the interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further
tcp_keepalive_intvl
the interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime
tcp_keepalive_probes
the number of unacknowledged probes to send before considering the connection dead and notifying the application layer
|
In simpler words:
1
2
3
4
5
6
| tcp_keepalive_time
If no data is received from the peer for N seconds, start probing the peer with keepalive packets.
tcp_keepalive_intvl
The interval, in seconds, between keepalive probes.
tcp_keepalive_probes
The number of probes to send before the connection is considered dead.
|
So if no data arrives from the peer within tcp_keepalive_time, keepalive probing begins. A probe is then sent every tcp_keepalive_intvl seconds. If the peer still does not respond after tcp_keepalive_probes attempts, the connection is closed.
On Linux, you can inspect the system keepalive settings like this:
1
2
3
4
5
6
| [root@localhost ~]# cat /proc/sys/net/ipv4/tcp_keepalive_time
1800
[root@localhost ~]# cat /proc/sys/net/ipv4/tcp_keepalive_probes
9
[root@localhost ~]# cat /proc/sys/net/ipv4/tcp_keepalive_intvl
75
|
Keepalive Socket APIs
You can enable and configure keepalive for a socket like this:
1. Enable keepalive
1
2
3
4
5
| int on = 1;
if (setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &on, sizeof(on))) {
// log
return -1;
}
|
2. Set tcp_keepalive_time
1
2
3
4
5
| int idle = 10;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &idle, sizeof(int)) < 0) {
// log
return -1;
}
|
3. Set tcp_keepalive_probes
1
2
3
4
5
| int probes = 4;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT, &probes, sizeof(int)) < 0) {
// log
return -1;
}
|
4. Set tcp_keepalive_intvl
1
2
3
4
5
| int intvl = 1;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL, &intvl, sizeof(int)) < 0) {
// log
return -1;
}
|
Keepalive in libuv
The API that libuv exposes only lets you configure two things:
- Enable keepalive
- Set
tcp_keepalive_time
The API is uv_tcp_keepalive, with the following signature:
1
2
| int uv_tcp_keepalive(uv_tcp_t* handle, int enable, unsigned int delay)
Enable / disable TCP keep-alive. delay is the initial delay in seconds, ignored when enable is zero.
|
Its implementation looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
| int uv__tcp_keepalive(int fd, int on, unsigned int delay) {
if (setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &on, sizeof(on)))
return -errno;
#ifdef TCP_KEEPIDLE
if (on && setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &delay, sizeof(delay)))
return -errno;
#endif
/* Solaris/SmartOS, if you don't support keep-alive,
* then don't advertise it in your system headers...
*/
/* FIXME(bnoordhuis) That's possibly because sizeof(delay) should be 1. */
#if defined(TCP_KEEPALIVE) && !defined(__sun)
if (on && setsockopt(fd, IPPROTO_TCP, TCP_KEEPALIVE, &delay, sizeof(delay)))
return -errno;
#endif
return 0;
}
int uv_tcp_keepalive(uv_tcp_t* handle, int on, unsigned int delay) {
int err;
if (uv__stream_fd(handle) != -1) {
err = uv__tcp_keepalive(uv__stream_fd(handle), on, delay);
if (err)
return err;
}
if (on)
handle->flags |= UV_TCP_KEEPALIVE;
else
handle->flags &= ~UV_TCP_KEEPALIVE;
/* TODO Store delay if uv__stream_fd(handle) == -1 but don't want to enlarge
* uv_tcp_t with an int that's almost never used...
*/
return 0;
}
|
From this code you can see that if uv__stream_fd(handle) is not available yet, libuv only sets a flag on the handle and does not actually apply delay. In other words, tcp_keepalive_time is only set after the connection has been fully established. If the connection is not established yet, the delay value is not applied.
If you also want to configure the other two values yourself, you need custom code. For example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| //
// Configure the remaining keepalive parameters.
// probes: maps to kernel tcp_keepalive_probes, i.e. how many probes to send
// before closing the connection if there is still no response.
// intvl: maps to kernel tcp_keepalive_intvl, i.e. interval between probes.
// idle: maps to kernel tcp_keepalive_time.
int set_keep_alive(const uv_handle_t* handle, int probes, int intvl, int idle) {
int ret;
uv_os_fd_t fd;
ret = uv_fileno(handle, &fd);
if (ret < 0) {
return ret;
}
if (idle > 0) {
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &idle, sizeof(idle)))
return -1;
}
// Set tcp_keepalive_intvl
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL,
(const void *) &intvl, sizeof(int)) < 0 ) {
return -1;
}
// Set tcp_keepalive_probes
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT,
(const void *) &probes, sizeof(int)) < 0) {
return -1;
}
return 0;
}
|
This function can only be called after the connection has been established, because only then does the handle have an underlying file descriptor and uv_fileno can succeed.