It’s a pretty common occurrence when you have 2 hosts pinging each other and some packets are being lost. And sometimes you need to make sure, that the routing device in between of these host really receives a request, forwards this request to the target, receives the reply and forwards it back to the request’s initiator. If you’re lucky enough to run tcpdump on this intermediate device (if it runs some Unix-like OS, e.g. Linux or FreeBSD), you can wrap it to the script that will analyze each transit packet to find out what exactly is going wrong.
If you watch the tcpdump output with the naked eye, you’ll see the following pattern:
|
1 2 3 4 |
01:14:15.658173 In 08:60:6e:e7:a9:94 ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23873, length 40 01:14:15.658187 Out 00:1b:21:1c:47:ad ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23873, length 40 01:14:15.658397 In f4:6d:04:2a:67:56 ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23873, length 40 01:14:15.658409 Out 00:e0:81:74:0f:1b ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23873, length 40 |
…which repeats…
|
1 2 3 4 |
01:14:16.673167 In 08:60:6e:e7:a9:94 ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23874, length 40 01:14:16.673180 Out 00:1b:21:1c:47:ad ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23874, length 40 01:14:16.673456 In f4:6d:04:2a:67:56 ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23874, length 40 01:14:16.673468 Out 00:e0:81:74:0f:1b ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23874, length 40 |
…and repeats…
|
1 2 3 4 |
01:14:17.686607 In 08:60:6e:e7:a9:94 ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23875, length 40 01:14:17.686620 Out 00:1b:21:1c:47:ad ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23875, length 40 01:14:17.687496 In f4:6d:04:2a:67:56 ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23875, length 40 01:14:17.687514 Out 00:e0:81:74:0f:1b ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23875, length 40 |
And if you look carefully to this pattern, you’ll see that there are 4 obvious phases.
A request came:
|
1 |
01:14:15.658173 In 08:60:6e:e7:a9:94 ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23873, length 40 |
The request gone:
|
1 |
01:14:15.658187 Out 00:1b:21:1c:47:ad ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 23873, length 40 |
The reply came:
|
1 |
01:14:15.658397 In f4:6d:04:2a:67:56 ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23873, length 40 |
The reply gone:
|
1 |
01:14:15.658409 Out 00:e0:81:74:0f:1b ethertype IPv4 (0x0800), length 76: 192.168.1.196 > 192.168.0.223: ICMP echo reply, id 14, seq 23873, length 40 |
So you can sit and watch the packets being received and transmitted, but it can be really boring, so you can run the following script:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
#!/bin/bash function warn { echo -e "$(date +"%F %T") $*" >&2 } function die { warn "$*" exit 13 } while getopts ":s:d:v" OPT; do case "${OPT}" in s) SOURCE="${OPTARG}" ;; d) DESTINATION="${OPTARG}" ;; v) VERBOSE=1 ;; \?) die "Invalid option: -${OPTARG}" ;; :) die "Option -${OPTARG} requires an argument" ;; esac done [[ -z "${SOURCE}" ]] && die "The source address is undefined" [[ -z "${DESTINATION}" ]] && die "The destination address is undefined" NEXT_PHASE=0 NEXT_SEQ=0 tcpdump -l -n -i any -e "icmp and host ${SOURCE} and host ${DESTINATION}" 2>/dev/null | while :; do [[ "${VERBOSE}" -eq 1 ]] && echo -n ">${NEXT_PHASE}" read LINE INPUT=($(echo "${LINE}" | sed -n -r 's/^[0-9:\.]+ ?(In|Out) .+ ([0-9\.]+) > ([0-9\.]+): ICMP echo (request|reply), id [0-9]+, seq ([0-9]+).*, length [0-9]+$/\1 \2 \3 \4 \5/gp')) NEW_SEQ="${INPUT[4]}" case "${INPUT[3]}" in 'request') NEW_PHASE=1 ([[ "${INPUT[1]}" == "${SOURCE}" ]] && [[ "${INPUT[2]}" == "${DESTINATION}" ]]) || continue 3 ;; 'reply') NEW_PHASE=3 ([[ "${INPUT[2]}" == "${SOURCE}" ]] && [[ "${INPUT[1]}" == "${DESTINATION}" ]]) || continue 3 ;; *) die "(1) The following line looks strange:\n${LINE}" ;; esac case "${INPUT[0]}" in 'In') ;; 'Out') NEW_PHASE=$((${NEW_PHASE} + 1)) ;; *) die "(2) The following line looks strange:\n${LINE}" ;; esac [[ "${NEXT_PHASE}" -eq 0 ]] && NEXT_PHASE="${NEW_PHASE}" [[ "${NEXT_SEQ}" -eq 0 ]] && NEXT_SEQ="${NEW_SEQ}" [[ "${NEXT_PHASE}" -ne "${NEW_PHASE}" ]] && warn "Wrong phase, expected ${NEXT_PHASE}, got ${NEW_PHASE}:\n${LINE}" [[ "${NEXT_SEQ}" -ne "${NEW_SEQ}" ]] && warn "Wrong seq, expected ${NEXT_SEQ}, got ${NEW_SEQ}:\n${LINE}" if [[ ${NEXT_PHASE} -eq 4 ]]; then NEXT_PHASE=1 NEXT_SEQ=$((${NEXT_SEQ} + 1)) else NEXT_PHASE=$((${NEXT_PHASE} + 1)) fi done |
It reads the output of tcpdump and prints an alert message when the usual pattern gets broken. When some request or some reply is absent or the seq-number is unexpected, it will print something like that:
|
1 2 |
2016-06-14 23:37:51 Wrong seq, expected 18167, got 18168: 23:37:51.919246 Out 00:1b:21:1c:47:ad ethertype IPv4 (0x0800), length 76: 192.168.0.223 > 192.168.1.196: ICMP echo request, id 14, seq 18168, length 40 |
The latest version is available at GitHub: https://gist.github.com/anonymous/2e8b6883c93326de280124c077424cc6.