Stop sniff() asynchronously

[HenningCash]

Hi there,
we are using scapy to built something similar to tcpdump.
The capturing process runs in its own thread:

def run(self):
    try:
        sniff(iface=self.interface, filter=self.filter, prn=self._loop_callback, store=0)
    except BaseException as e:
        log.exception(e)
    finally:
        self.output.close()

def _loop_callback(self, pkg: Packet):
    if self._stop_flag is True:
        raise KeyboardInterrupt
    else:
        self.output.write(pkg)

Basically we have a flag that is set externally from the main thread if the capturing process should stop. This works as long as a package is received after the stop flag is set. But if there is no package the thread (recv()) hangs until the next package is received and the callback is fired.

We could built something like this by passing the timeout to sniff():

    while not self._stop_flag:
        sniff(iface=self.interface, filter=self.filter, prn=self._loop_callback, store=0, timeout=1)

But I think with this approach there might be packages that get lost between the sniffing calls, am I right?

Are there any other approaches to stop sniff() async without losing packages?

[gpotter2]

This is actually quite easy to do, using the stop_filter function:

Obviously, this will have to wait for a new packet to be sniffed (otherwise recieving on the socket is stuck)

https://github.com/secdev/scapy/blob/master/scapy/sendrecv.py#L622-L625

stop_filter: Python function applied to each packet to determine if
we have to stop the capture after this packet.
Ex: stop_filter = lambda x: x.haslayer(TCP)

[HenningCash]

Your example works, if there is a packet. As the documentation says stop_filter is a "Python function applied to each packet". If there is no packet (e.g. if you use a very restrictive filter and/or there is very little traffic) the process/thread will wait and exit only after the next captured packet when stop_filter is applied.

#!/usr/bin/env python3

import time, threading
from scapy.all import sniff
e = threading.Event()
def _sniff(e):
    a = sniff(filter="tcp port 80", stop_filter=lambda p: e.is_set())
    print("Stopped after %i packets" % len(a))

print("Start capturing thread")
t = threading.Thread(target=_sniff, args=(e,))
t.start()

time.sleep(3)
print("Try to shutdown capturing...")
e.set()

# This will run until you send a HTTP request somewhere
# There is no way to exit clean if no package is received
while True:
	t.join(2)
	if t.is_alive():
		print("Thread is still running...")
	else:
		break

print("Shutdown complete!")

[gpotter2]

Well we could add the stop_event check when every packet is recieved, but this would actually slow down scapy :/
Would this be a proper option to you ?

I don't think they would be a proper way we could internally abort sniffing if no packets are recieved, as we rely on the select (unix) or winpcap (windows) calls, which we cannot abort...

[guedou]

I will suggest that you build your own packet capture logic using Scapy SuperSocket and select, such as:

from scapy.all import *
import select

s = conf.L2listen()
rlist = select.select([s], [], [])
if rlist:
    p = s.recv()
    print p.summary()
s.close()

[HenningCash]

@guedou that looks promising. I will give it a try within the next week.

Do you think its worth the effort implementing this as new feature for scapy? Passing in a threading.Event and checking for isSet() should be simple

[guedou]

That way too complicated. I think that adding a callback that is triggered when select and its friends do not return file descriptors is the way to go.

[guedou]

Feel free to reopen if needed.

[HenningCash]

The suggested custom implementation with SuperSocket worked perfectly fine, thank you!

[SkypLabs]

I just published a blog post about a way to address this issue: http://blog.skyplabs.net/2018/03/01/python-sniffing-inside-a-thread-with-scapy/ . I hope it will help you.

[guedou]

Thanks!

…

Sent from my iPhone

On 2 Mar 2018, at 13:12, Skyper ***@***.***> wrote: I just published a blog post about a way to address this issue: http://blog.skyplabs.net/2018/03/01/python-sniffing-inside-a-thread-with-scapy/ . I hope it will help you. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[louisabraham]

@SkypLabs Your post is about stop_filter.

A little observation: one could also use stop_callback which is called more often.

Apart from that, I wrote a custom version of sniff to address this issue optimally (using select). Feel free to use it.
It should also be possible to integrate it in the code, but I don't know if it is a common issue.

def sniff(store=False, prn=None, lfilter=None,
          stop_event=None, refresh=.1, *args, **kwargs):
    """Sniff packets
sniff([count=0,] [prn=None,] [store=1,] [offline=None,] [lfilter=None,] + L2ListenSocket args)

  store: wether to store sniffed packets or discard them
    prn: function to apply to each packet. If something is returned,
         it is displayed. Ex:
         ex: prn = lambda x: x.summary()
lfilter: python function applied to each packet to determine
         if further action may be done
         ex: lfilter = lambda x: x.haslayer(Padding)
stop_event: Event that stops the function when set
refresh: check stop_event.set() every refresh seconds
    """
    s = conf.L2listen(type=ETH_P_ALL, *args, **kwargs)
    remain = None
    lst = []
    try:
        while True:
            if stop_event and stop_event.is_set():
                break
            sel = select([s], [], [], refresh)
            if s in sel[0]:
                p = s.recv(MTU)
                if p is None:
                    break
                if lfilter and not lfilter(p):
                    continue
                if store:
                    lst.append(p)
                c += 1
                if prn:
                    r = prn(p)
                    if r is not None:
                        print(r)
    except KeyboardInterrupt:
        pass
    finally:
        s.close()

    return plist.PacketList(lst, "Sniffed")

[SkypLabs]

@SkypLabs Your post is about stop_filter.

No, using stop_filter was a design choice for "letting the time to the sniff function to terminate its job by itself, after which the sniffing thread will be force-stopped and its socket properly closed from the main thread.". My post is not about stop_filter but about a way to stop the sniffing loop programmatically at any time.

A little observation: one could also use stop_callback which is called more often.

stop_callback is not part of Scapy but has been added to the fork scapy3k.

Apart from that, I wrote a custom version of sniff to address this issue optimally (using select). Feel free to use it.

As I have already said in this issue, "using a SuperSocket with a timeout works fine but this method is resource-consuming since you have to check every X seconds if the sniffing loop should be stopped or not."