Support for HDX device?

[kubaraczkowski]

Hi,

Any idea if the new OO HDX can be supported?
The PID is 2003.
It looks like it has the same 'interface electronics' as the FlameX, the PID is "so close" (2003 vs 2001 of FX), they are both provided in "offline" installation folder of omnidriver, the clues are all pointing to being compatible at least.

Thanks for making pyseabreeze! It's great!

[ap--]

hi @kubaraczkowski,

You need to look for a datasheet that contains the full usb command set. I couldnt find one just now.
The fact that the product Ids are "so close" does not tell you anything. The protocol could be completely different.

report back if you find one. Then I can give you more advice and tell you if it's complicated to get it running.

Best,
Andreas

[kubaraczkowski]

ok, deal. we're considering to buy the hdx vs an older model, one thing to take into account is availability of low lever drivers ... kuba

…

On Fri, Jul 20, 2018, 00:05 Andreas Poehlmann ***@***.***> wrote: hi @kubaraczkowski <https://github.com/kubaraczkowski>, You need to look for a datasheet that contains the full usb command set. I couldnt find one just now. The fact that the product Ids are "so close" does not tell you anything. The protocol could be completely different. report back if you find one. Then I can give you more advice and tell you if it's complicated to get it running. Best, Andreas — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/ap--/python-seabreeze/issues/60#issuecomment-406428851>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAh2nNWaTWw-BxjxaVzwkUXOqgBdxmwOks5uIQKdgaJpZM4VKrVn> .

[jochen-scheuer]

Are there any news about the support of the HDX spectrometers? The seabreeze package works like a charm for the QE-Pro. Thanks for the great work!

[ap--]

@jochen-scheuer
so ocean optic's seabreeze library doesn't support it. Without knowing about the low level usb protocol I can't help with implementing support via pyseabreeze

[toddkarin]

I hope this helps develop support:

https://files.oceanoptics.com/Downloads/Ocean-HDX/Firmware/Ocean%20HDX%20Firmware%20and%20Advanced%20Communications%20Rev2.pdf

Please let us know if/when you plan to add full support for the HDX!

[ap--]

@toddkarin
I don't own an HDX spectrometer, so it's unlikely i'll be adding it, because I can't test it.
But looking at the datasheet, getting basic spectrometer functionality should be as easy as following https://python-seabreeze.readthedocs.io/en/latest/contributing.html#adding-a-new-spectrometer and changing the specific config values.

Let me know if I can help somehow, or if something's unclear.
Also if you implement support, feel free to make a pull request :)

Best,
-Andreas

[toddkarin]

Great, I just need basic functionality so I can try that. By the way, I'm planning to buy one soon and would be happy to run a test script if you feel like adding full support. (It seems to be a popular request!)

[ap--]

Nice 👍 A test script is included via pytest and running the spectrometer tests is explained here: https://python-seabreeze.readthedocs.io/en/latest/contributing.html#testing-spectrometer-models

[toddkarin]

It would help if you made some more comments on that page for explaining what the parameters are, e.g.

• What units are used for integration time min and max (ms, us)?
• What does integration_time_base mean?
• What is spectrum_raw_length?
• How to find the usb_product_id?

etc.

Thanks!

[ap--]

I currently don't have good access to my development computer, so updating the docs will have to wait a bit.
Regarding your questions:
1.) micro seconds
2.) internally the code handles everything in micro seconds. But some spectrometers expect the intergration time in milli seconds. base let's you provide either us (=1) or ms (=1000) to the spectrometer. look at the code in https://github.com/ap--/python-seabreeze/blob/master/src/seabreeze/pyseabreeze/features/spectrometer.py
3.) that's the number of bytes that the spectrometer returns when it transfers a raw spectrum. It's number of pixels * bytesOfDataType (uint16, uint32) + optional metadata byte overhead.
4.) look at page 2 in the PDF you posted (section USB Port)

I believe most of your questions should be answered after thoroughly reading the datasheet :)
And I'll update the docs and make them more verbose as soon as I have time :)

Best,
-Andreas

[toddkarin]

Thanks for the update. I'm having trouble. I added the following to devices.py:

class HDX(SeaBreezeDevice):

    model_name = 'HDX'

    # communication config
    transport = (USBTransport, )
    usb_product_id = 0x2003
    usb_endpoint_map = EndPointMap(ep_out=0x01, lowspeed_in=0x81)  # XXX: we'll ignore the alternative EPs
    usb_protocol = OBPProtocol

    # spectrometer config
    dark_pixel_indices = DarkPixelIndices.from_ranges()
    integration_time_min = 6000
    integration_time_max = 10000000
    integration_time_base = 1
    spectrum_num_pixel = 2068
    spectrum_raw_length = (2068 * 2) + 64  # XXX: Metadata
    spectrum_max_value = 65535
    trigger_modes = TriggerMode.supported('OBP_NORMAL', 'OBP_EXTERNAL', 'OBP_INTERNAL')

    # features
    feature_classes = (
        sbf.spectrometer.SeaBreezeSpectrometerFeatureHDX,  # need to implement this
        sbf.rawusb.SeaBreezeRawUSBBusAccessFeature,
    )

Then I added the following to spectrometer.py

class SeaBreezeSpectrometerFeatureHDX(SeaBreezeSpectrometerFeatureOBP):
    pass

and tried to import using:

import numpy as np
import pandas as pd
import usb
import seabreeze
seabreeze.use('pyseabreeze')

from seabreeze.spectrometers import list_devices, Spectrometer

devices = list_devices()
print(devices)

spec = Spectrometer.from_serial_number("HDX00626")

The returned output is:

[<SeaBreezeDevice HDX:HDX00626>]
Traceback (most recent call last):
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-c753eedb85c2>", line 1, in <module>
    runfile('/Users/toddkarin/Dropbox/LBL/Projects/arcoatings/test_seabreeze.py', wdir='/Users/toddkarin/Dropbox/LBL/Projects/arcoatings')
  File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/toddkarin/Dropbox/LBL/Projects/arcoatings/test_seabreeze.py", line 17, in <module>
    spec = Spectrometer.from_serial_number("HDX00626")
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/seabreeze/spectrometers.py", line 117, in from_serial_number
    return cls(dev)
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/seabreeze/spectrometers.py", line 57, in __init__
    self.open()  # always open the device here to allow caching values
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/seabreeze/spectrometers.py", line 325, in open
    self._dev.open()
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/seabreeze/pyseabreeze/devices.py", line 283, in open
    self._transport.open_device(self._raw_device)
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/seabreeze/pyseabreeze/transport.py", line 132, in open_device
    self._device.set_configuration()
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/usb/core.py", line 869, in set_configuration
    self._ctx.managed_set_configuration(self, configuration)
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/usb/core.py", line 102, in wrapper
    return f(self, *args, **kwargs)
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/usb/core.py", line 148, in managed_set_configuration
    self.backend.set_configuration(self.handle, cfg.bConfigurationValue)
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/usb/backend/libusb1.py", line 794, in set_configuration
    _check(self.lib.libusb_set_configuration(dev_handle.handle, config_value))
  File "/Users/toddkarin/anaconda3/envs/py3/lib/python3.7/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 19] No such device (it may have been disconnected)

Any suggestions?

[ap--]

Hmm, have you updated the udev rules and included the HDX?
could be just a permission problem.

It's easiest if you fork the repository, push your changes and make a pull request.
Then I can review your code and find potential issues faster. 😊

-Andreas

[Lavikjo]

@toddkarin Did you ever get it working? I used your code together with udev-rule and made connection few times.

Most of the time I'm running into timeout issues.. dunno if it is related to running code inside VM and passing USB-connection there.

I'm running the code you have above with the following results:

[<SeaBreezeDevice HDX:HDX00767>]
Traceback (most recent call last):
  File "test_communication.py", line 12, in <module>
    spec = Spectrometer.from_serial_number("HDX00767")
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/spectrometers.py", line 114, in from_serial_number
    for dev in list_devices():
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/spectrometers.py", line 35, in list_devices
    return list_devices._api.list_devices()
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/api.py", line 98, in list_devices
    dev.open()
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/devices.py", line 331, in open
    self._serial_number = self.get_serial_number()
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/devices.py", line 371, in get_serial_number
    return protocol.query(0x00000100).decode("utf8")
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/protocol.py", line 361, in query
    return self.receive(timeout_ms=timeout_ms)
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/protocol.py", line 307, in receive
    response = self.transport.read(size=64, timeout_ms=timeout_ms)
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/seabreeze/pyseabreeze/transport.py", line 171, in read
    return self._device.read(endpoint, size, timeout=timeout_ms)
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/usb/core.py", line 988, in read
    self.__get_timeout(timeout))
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
    timeout)
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/usb/backend/libusb1.py", line 936, in __read
    _check(retval)
  File "/home/kokeilu/.local/share/virtualenvs/dippa-_rkJLSL6/lib/python3.7/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 110] Operation timed out
double free or corruption (!prev)
Aborted

I added
ATTR{idVendor}=="2457", ATTR{idProduct}=="2003", SYMLINK+="hdx-%n", MODE:="0666"
to udev rules file.

Any ideas?

[ap--]

HDX spectrometers should now be supported with pyseabreeze thanks to @Lavikjo 👍