python-scapy no longer works with kernel 1.20170703-1 and higher using integrated wifi on pi0w and pi3 b+

[dliloch]

When sniffing for arp packets using python-scapy only the router and the machine scapy is running on show up. Before upgrading..ie. Kernel 1.20170405-1 scapy would return all arp packets. If I use a tenda wifi dongle scapy works properly.
I can provide a simple python example to demonstrate this. I will also be glad to provide any further doc you request..ie traces.. thanks

[pelwell]

Can you get wireshark (sudo apt-get install wireshark) traces for a success case and a failure case?

Alternatively give clear and complete steps to create and demonstrate a working system, and then the steps needed to break it.

[dliloch]

Thanks for responding so quickly..
I will do both! The wireshark will have to wait until tonight when I have access to the pi.. but to install and break it..

1. Never worked on stretch..
2. Install April 10 Jessie-lite.zip distro.
3. sudo apt-get install python-scapy tcpdump
4. sudo python scapy2tst.py
5. While the above is running take another pi or iPad ... and connect to your network..you will see your MAC address of the new device..
   To break
6. sudo apt-get update
   If you do an upgrade you will see some 70+ modules..I got it down to the kernel
7. sudo apt-get install kernel
8. Reboot
9. sudo python scapy2tst.py
10. While the code is running try to connect another wifi device ..pi, iPad etc to network
11. You will no longer get the new connected device just your MAC address and the router
    You can use any python code but here is my small test example.

#! /usr/bin/env python
from scapy.all import sniff
from scapy.all import ARP

def arp_monitor_callback(pkt):
if ARP in pkt and pkt[ARP].op in (1,2): #who-has or is-at
return pkt.sprintf("%ARP.hwsrc% %ARP.psrc%")

sniff(prn=arp_monitor_callback, filter="arp", store=0)

[dliloch]

In addition after it is “broken” you can plug in an external wifi dongle and it should work again.

[pelwell]

Thanks - I can reproduce the problem with the first 4.9 kernel release and not with the last 4.4 kernel. I won't need those traces.

[dliloch]

Wow that’s great..I was posting on the forum for about 2 months and nothing..I thought I was the only one having this issue...actually people were complaining that the dash button examples in the magpi did not work..but they do..it all depends on the distro..
So if you need anything further let me know..
thanks again! 😀

[pelwell]

The bad news is that there are 158 commits to the brcmfmac driver between those two releases, including a wholesale directory move, and the required Pi-specific patches appear after the last brcmfmac commit, so a simple git bisect is not going to work. But we have a good test case and @JamesH65 is back online soon, so we may be able to make some progress.

[AndyTaylorTweet]

In addition to the above, and probably for the same reason, I see issues with Pi0W WiFi and Multicast groups. In Wireless networks with IGMP snooping ON, the Pi-Zero W doesn't join the multicast groups, and does not get any Multicast packets, for example mDNS (Avahi-Daemon) fails to work on Pi0W onboard WiFi - but does work on other WiFi devices.

This may or may not help push you in the correct direction.

[pelwell]

I think the change is something to do with the ARP offload support, but I haven't found exactly which commit is responsible. There are a few workarounds possible, but without some feedback from the brcmfmac maintainers I don't know which (if any) is the best.

If you can build your own kernel, try this patch:

commit c35c007e91a7ccb256daf90df210c4cbae522e36
Author: Phil Elwell <[email protected]>
Date:   Thu Aug 24 16:16:16 2017 +0100

    brcmfmac: Set BRCMF_ARP_OL_SNOOP when promiscuous.
    
    This is a test patch to attempt to fix the inability to sniff ARP
    packets in promiscuous mode. Do not apply to a production kernel.
    
    See: https://github.com/raspberrypi/linux/issues/2171

diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
index f877301..682760a 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
@@ -80,6 +80,7 @@ static void _brcmf_set_multicast_list(struct work_struct *work)
        __le32 cnt_le;
        char *buf, *bufp;
        u32 buflen;
+       u32 mode;
        s32 err;
 
        ifp = container_of(work, struct brcmf_if, multicast_work);
@@ -134,6 +135,15 @@ static void _brcmf_set_multicast_list(struct work_struct *work)
        if (err < 0)
                brcmf_err("Setting BRCMF_C_SET_PROMISC failed, %d\n",
                          err);
+
+       if ((brcmf_fil_iovar_int_get(ifp, "arp_ol", &mode) == 0) && mode) {
+               if (ndev->flags & IFF_PROMISC)
+                       mode |= BRCMF_ARP_OL_SNOOP;
+               else
+                       mode &= ~BRCMF_ARP_OL_SNOOP;
+
+               brcmf_fil_iovar_int_set(ifp, "arp_ol", mode);
+       }
 }
 
 #if IS_ENABLED(CONFIG_IPV6)

[dliloch]

I’ve never built a kernel..can you point me to the instructions I should use...it’s time to learn..
Thanks...

[pelwell]

These are our standard instructions: https://www.raspberrypi.org/documentation/linux/kernel/building.md

Apply the patch after the git clone and sometime before the main make (obviously).

[pelwell]

Broadcom have provided an alternative patch that also restores ARP sniffing. It's now in the 4.9 tree and will be in the next rpi-update release.

[dliloch]

Phil, That’s great news ... thanks again for the follow up on this!

…

Sent from my iPad Air 2

On Aug 30, 2017, at 8:42 AM, Phil Elwell ***@***.***> wrote: Broadcom have provided an alternative patch that also restores ARP sniffing. It's now in the 4.9 tree and will be in the next rpi-update release. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[dliloch]

by the way I tried to patch the kernel using the code you gave me above and I got this error:
pi@raspistreatch:~/linux/drivers/net/wireless/broadcom/brcm80211/brcmfmac $ cat /home/pi/linux/patch-arp.txt | patch
patching file core.c
Hunk #1 FAILED at 80.
patch unexpectedly ends in middle of line
Hunk #2 FAILED at 134.
2 out of 2 hunks FAILED -- saving rejects to file core.c.rej

then ...
looking in core.c.rej
pi@raspistreatch:~/linux/drivers/net/wireless/broadcom/brcm80211/brcmfmac $ cat core.c.rej
--- core.c
+++ core.c
@@ -80,6 +80,7 @@ static void _brcmf_set_multicast_list(struct work_struct *work)
__le32 cnt_le;
char *buf, *bufp;
u32 buflen;

•   u32 mode;
    s32 err;
  
    ifp = container_of(work, struct brcmf_if, multicast_work);
  

@@ -134,6 +135,15 @@ static void _brcmf_set_multicast_list(struct work_struct *work)
if (err < 0)
brcmf_err("Setting BRCMF_C_SET_PROMISC failed, %d\n",
err);
+

•   if ((brcmf_fil_iovar_int_get(ifp, "arp_ol", &mode) == 0) && mode) {
  

•           if (ndev->flags & IFF_PROMISC)
  

•                   mode |= BRCMF_ARP_OL_SNOOP;
  

•           else
  

•                   mode &= ~BRCMF_ARP_OL_SNOOP;
  

•           brcmf_fil_iovar_int_set(ifp, "arp_ol", mode);
  

•   }
  

}

any ideas?

[pelwell]

You've almost certainly got whitespace/line wrap problems. Rather than waste your time on that, update your kernel to include the new patch and build that instead.

[dliloch]

Just downloaded the latest stretch update 4.9.41v7 #1023 sept. 7th and then ran rbi-update moved kernel to 4.9.48+ now it works again...so i’m Closing the issue .. thanks again for the great support..