TCP Tracepoints

http://www.brendangregg.com/blog/2018-03-22/tcp-tracepoints.html

• TCP Tracepoints have arrived in Linux. Linux-4.15 added 5 of them, 4.16 will add 2 more
• Versatile sock:inet_sock_set_state. It can be used to track when kernel changes the state of a TCP session (e.g. TCP_SYN_SENT -> TCP_ESTABLISHED)
• tcplife tool in the open-source “bcc” collection.
  • Used “kprobe” (kernel dynamic tracing) of the “tcp_set_state()” kernel function. Relies on kernel implementation, need to change from version to version
• Tracepoints are considered as “stable” and “shouldn’t change” (not “set in stone”)
• “trace” tool in the bcc:


# trace -t 't:sock:inet_sock_set_state "%llx: %d -> %d", args->skaddr, args->oldstate, args->newstate'
       TIME     PID     TID     COMM            FUNC             -
       1.690191 17308   17308   curl            inet_sock_set_state ffff9fd7db589800: 7 -> 2
       1.690798 0       0       swapper/24      inet_sock_set_state ffff9fd7db589800: 2 -> 1

• 7->2, 2->1 - state changes (ENUM)
• tcpstate does the same + translate to the well-known names:


# tcpstates
SKADDR           C-PID C-COMM     LADDR           LPORT RADDR         RPORT OLDSTATE    -> NEWSTATE    MS
ffff9fd7e8192000 22384 curl       100.66.100.185  0     52.33.159.26  80    CLOSE       -> SYN_SENT    0.000
ffff9fd7e8192000 0     swapper/5  100.66.100.185  63446 52.33.159.26  80    SYN_SENT    -> ESTABLISHED 1.373

• tcp:tcp_retransmit_skb traces retransmits, useful to understand network issues, including congestioon
• tcp:tcp_retransmit_synack traces syn and syn/ack restransmits
• tcp:tcp_destroy_sock traces: session has ended, sock address is about to be reused
• tcp:tcp_send_reset traces RST send during a valid socket
• tcp:tcp_receive_reset traces RST receive
• tcp:tcp_probe : for TCP congestion window tracing (TCP окно перегрузки)
• sock:inet_sock_set_state.
• Tracepoints are preferred over libpcap: lower overhead + show kernel internal state