Bluetooth LE Sensor Nodes to Raspberry Pi WiFi Bridge - Stuck at scanning

[wildestpixel]

From the guide by John Park at https://learn.adafruit.com/bluetooth-le-broadcastnet-sensor-node-raspberry-pi-wifi-bridge/overview ......

Following through the guide and executing on an upto date Pi OS on Zero W everything goes fine until you actually execute python3 ble_broadcastnet_blinka_bridge.py

After this , instead of finding the Clue (which has the prescribed basic broadcast script running and is tested with both CP5.3 and CP5.4BETA), the output is stuck at

This is BroadcastNet bridge: b827eb2d289a

Fetching existing feeds.
{}
scanning

until after about 15 minutes an error (memory error) stops python 3.7 running on the Pi Zero W and the following extra output is given :

Traceback (most recent call last):
  File "ble_broadcastnet_blinka_bridge.py", line 113, in <module>
    adafruit_ble_broadcastnet.AdafruitSensorMeasurement, interval=0.5
  File "/home/pi/.local/lib/python3.7/site-packages/adafruit_ble/__init__.py", line 243, in start_scan
    active=active,
  File "/home/pi/.local/lib/python3.7/site-packages/_bleio/adapter_.py", line 201, in start_scan
    prefixes, timeout=timeout, minimum_rssi=minimum_rssi, active=active,
  File "/home/pi/.local/lib/python3.7/site-packages/_bleio/adapter_.py", line 299, in _start_scan_hcitool
    buffered, prefixes, minimum_rssi, active
  File "/home/pi/.local/lib/python3.7/site-packages/_bleio/adapter_.py", line 261, in _parse_hcidump_data
    if scan_entry.matches(prefixes, all=False):
  File "/home/pi/.local/lib/python3.7/site-packages/_bleio/scan_entry.py", line 79, in matches
    fields = self._advertisement_fields
  File "/home/pi/.local/lib/python3.7/site-packages/_bleio/scan_entry.py", line 138, in _advertisement_fields
    fields.append(self._advertisement_bytes[idx + 1 : idx + field_length])
MemoryError

This same fault was raised by another use in Adafruit Customer forums in MArch at https://forums.adafruit.com/viewtopic.php?f=50&t=163964 and by myself earlier today at https://forums.adafruit.com/viewtopic.php?f=65&t=166701

Would like to get this to work, and given I can see the CLUE in a bluetooth scan from Bluefruit Connect iOS app I'm at a loss as to whether this is down to Adafruit_CircuitPython_BLE_BroadcastNet or the CP library adafruit_ble_broadcastnet.mpy

[tannewt]

Hi hi, I'm using this at my house and wrote it originally. Sorry I didn't see those posts, I only look in the CircuitPython forum.

Does hcidump show anything if you run it at the same time as the bridge Python is going?

[wildestpixel]

Hi,

The output seems to mirror that of the python script as in seeing nothing in a scan

pi@raspberrypiBlinkaBridge:~ $ hcidump
HCI sniffer - Bluetooth packet analyzer ver 5.50
device: hci0 snap_len: 1500 filter: 0xffffffff

and nothing else unfortunately

Conversely - to know Bluetooth is working on the device, albeit not through the tools used for this CP example , I checked bluetoothctl and got a working scan

pi@raspberrypiBlinkaBridge:~ $ bluetoothctl
Agent registered
[bluetooth]# agent on
Agent is already registered
[bluetooth]# default-agent
Default agent request successful
[bluetooth]# scan on
Discovery started
[CHG] Controller B8:27:EB:2D:28:9A Discovering: yes
[NEW] Device 52:EF:83:60:FC:60 52-EF-83-60-FC-60
[NEW] Device 53:5D:F5:6C:CB:65 53-5D-F5-6C-CB-65
[NEW] Device 5A:E2:54:6C:EC:C3 5A-E2-54-6C-EC-C3
[NEW] Device 7F:C0:C3:DF:7D:9E 7F-C0-C3-DF-7D-9E

Doesn't see the Clue in this scan although I'd imagine its broadcasting in a non-pairable state.

[kevinjwalters]

Does the python process bloat over time? Watching it in top or similar might be interesting to see if it's gradual growth or some werid spurt.

[dhalbert]

EDIT: SEE BELOW re hcitool lescan --duplicates

I reproduced the essence of this problem, also on a freshly installed Pi Zero W. I installed the latest bluez-hcidump, and also gave hcitool and hcidump the proper permissions.

RPi Zero W, v5.50: If I run hcitool lescan in one terminal and hcidump in another, both will print a small number of advertisements and then stop. Tens of seconds can go by before another small batch of advertisements is printed. I repeated this with both running under sudo, just to make sure, and the problem still exists. Both are version 5.50.

If I initiate the scan with bluetoothctl ... scan on I see more advertisements, though they are still sporadic.

RPi 4: Same problem; I thought it might be the weak processor on the RPi zero.

x64 Ubuntu 20.04, v5.53: By comparison, running the same commands on an adjacent x64 Ubuntu 20.04 machine prints advertisements continuously, in my rich Bluetooth environment. On this machine, the commands are version 5.53.

I will try an Ubuntu build on an RPi, since I can get up to v5.53.
EDIT: bluetooth on RPi under ubuntu is a rathole. You need to install pi-bluetooth, but then you lose the serial console.

[wildestpixel]

Does the python process bloat over time? Watching it in top or similar might be interesting to see if it's gradual growth or some werid spurt.

Htop sees the python process running over 95% cpu on the zero w and hcitool lescan forms many duplicates

[tannewt]

Does dmesg give any errors? On some versions of Pi3 I know the UART to the BLE modules is clocked too fast and causes dmesg errors.

[dhalbert]

No errors in dmesg or syslog, but I realize that I forgot to do hcitool lescan --duplicates. Once I do that on the RPi Zero W, I see plenty of traffic.

I'm a little confused as to why I didn't have to do that on the Ubuntu x64 box. When I was testing hcitool for another reason last week, I was doing that.

[dhalbert]

For bluetoothctl this will present duplicate advertisements:

$ sudo bluetoothctl             # sudo not necessary if ACL permissions have been assigned.
[bluetooth]# menu data
[bluetooth]# duplicate-data on
[bluetooth]# menu back
[bluetooth]# scan on

[dhalbert]

@wildestpixel did you follow the instructions in the guide here about adding your user to the bluetooth group, and then running some other commands to make hcitool and hcidump privileged? If you don't do that, then duplicate advertisements will be suppressed, and you will see little traffic.

[wildestpixel]

Yes - all of the following

sudo chown :bluetooth /usr/bin/hcidump /usr/bin/hcitool
sudo chmod o-x /usr/bin/hcidump /usr/bin/hcitool
sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/bin/hcitool
sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/bin/hcidump

and user pi is present in the bluetooth user group

Ref my dmesg , no bluetooth errors

[   32.296722] Bluetooth: HCI device and connection manager initialized
[   32.296747] Bluetooth: HCI socket layer initialized
[   32.296759] Bluetooth: L2CAP socket layer initialized
[   32.296816] Bluetooth: SCO socket layer initialized
[   32.323164] Bluetooth: HCI UART driver ver 2.3
[   32.323182] Bluetooth: HCI UART protocol H4 registered
[   32.323374] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   32.325679] Bluetooth: HCI UART protocol Broadcom registered
[   33.291877] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   33.291891] Bluetooth: BNEP filters: protocol multicast
[   33.291915] Bluetooth: BNEP socket layer initialized
[   33.828218] Bluetooth: RFCOMM TTY layer initialized
[   33.828256] Bluetooth: RFCOMM socket layer initialized
[   33.828306] Bluetooth: RFCOMM ver 1.11

However some errors elsewhere , look wlan related

[   16.476344] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[   16.476813] usbcore: registered new interface driver brcmfmac
[   16.749008] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[   16.749208] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available (err=-2), device may have limited channels available
[   16.750447] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Oct 22 2019 01:59:28 version 7.45.98.94 (r723000 CY) FWID 01-3b33decd
[   25.658141] random: crng init done
[   25.658167] random: 7 urandom warning(s) missed due to ratelimiting
[   25.801004] 8021q: 802.1Q VLAN Support v1.8
[   27.870452] uart-pl011 20201000.serial: no DMA platform data
[   28.396423] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   28.396457] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled

And this is a very clean install of latest PI OS, just apt updated and upgraded and the guide followed.

[tennesteve]

Hi - I've been attempting the same project with a new CLUE and fresh install on Raspberry Pi 4 and get a similar response...
`
pi@YobiSenseAlpha:~/Adafruit_CircuitPython_BLE_BroadcastNet/examples $ python3 ble_broadcastnet_blinka_bridge.py
This is BroadcastNet bridge: dca63251a708

Fetching existing feeds.
{}
scanning

Traceback (most recent call last):
File "ble_broadcastnet_blinka_bridge.py", line 113, in
adafruit_ble_broadcastnet.AdafruitSensorMeasurement, interval=0.5
File "/home/pi/.local/lib/python3.7/site-packages/adafruit_ble/init.py", line 249, in start_scan
active=active,
File "/home/pi/.local/lib/python3.7/site-packages/bleio/adapter.py", line 201, in start_scan
prefixes, timeout=timeout, minimum_rssi=minimum_rssi, active=active,
File "/home/pi/.local/lib/python3.7/site-packages/bleio/adapter.py", line 299, in _start_scan_hcitool
buffered, prefixes, minimum_rssi, active
File "/home/pi/.local/lib/python3.7/site-packages/bleio/adapter.py", line 261, in _parse_hcidump_data
if scan_entry.matches(prefixes, all=False):
File "/home/pi/.local/lib/python3.7/site-packages/_bleio/scan_entry.py", line 92, in matches
fields = self._advertisement_fields
File "/home/pi/.local/lib/python3.7/site-packages/_bleio/scan_entry.py", line 151, in _advertisement_fields
fields.append(self._advertisement_bytes[idx + 1 : idx + field_length])
MemoryError
`
I'm relatively new so don't have the expertise of the author, but was curious to know if there has been a resolution yet?

[wildestpixel]

As this is tagged as a bug, once fixed it will be referenced here. Maybe days, maybe months, it maybe it’s dependant on the actions of other individuals or packages that Adafruit have no direct control over.

[leadmocha]

So looking at _bleio/scan_entry.py around line 151, I printed out the list of bytes that it receives from the Clue and see what causes it to enter an infinite loop inside a while loop that looks at index idx.

So, here is an example:
self._advertisement_bytes has 30 bytes: 0x1d 0xff 0x22 0x8 0x6 0x4 0xa 0x30 0x35 0xf6 0x41 0xe 0x1 0xa 0xf4 0x24 0x3a 0x41 0x24 0x3a 0xa3 0xc0 0x34 0x87 0x85 0x41 0x3 0x3 0x0 0x0

First one should be the length, but sice 0x1d == 29, it only increments idx by 29. Then it looks at the last byte, which is 0, so it sits there incrementing the index by zero. All the while it is appending 0 to a list until it runs out of memory and crashes.

That's as far as I got. I don't know enough about the libraries to understand if it's the Clue sending the wrong number of bytes, or the raspberry pi reading the last byte wrong.