undetected armhf setuid issues in at least versions 0.4.20894 and 0.4.20868 (for emcapp python2 gui stuff)

[the-snowwhite]

Following the new guide for how to build EMCApplacation for armhf
I then built a fresh new mksocfpga armhf buster sd-image for testing on a DE0-Nano-SoC Kit/Atlas-SoC Kit

Installing the debs I also had to include:

python3-avahi_0.8-3_armhf.deb

libavahi-common-data_0.8-3_armhf.deb

from "Bullseye"

For some reason emcapplication_2.9.0-pre0.23507.git1dcbe183f-buster_armhf.deb asked for machinekit-hal_0.4.20996

so I wgot that and ended up with:

machinekit@mksocfpga-ob-ox:~/current_emc$ ls
emcapplication_2.9.0~pre0.23507.git1dcbe183f~buster_armhf.deb  machinekit-hal_0.4.20894-1.gitebe1344a0~buster_armhf.deb             python3-avahi_0.8-3_armhf.deb
libavahi-common-data_0.8-3_armhf.deb                           machinekit-hal-rt-preempt_0.4.20894-1.gitebe1344a0~buster_armhf.deb

That I installed with:

sudo apt install ./*.deb

testing then turned into an immediate terrible experience:

running linuxcnc gave some error messages (about BACKGROUND color) that could be remedied with:

xrdb -load /dev/null
xrdb -query

Then linuxcnc gui ran (in a ssh -X terminal) however selecting the axis sim configuration lead to a foul error loop:

Investigating further I found out running halrun gave error messages: (with both version 0.4.20894 and 0.4.20868)
However not with the latest version:0.4.20996

machinekit@mksocfpga-ob-ox:~/recomended_emc$ ls
machinekit-hal_0.4.20996-1.git2524ccd02~buster_armhf.deb  machinekit-hal-rt-preempt_0.4.20996-1.git2524ccd02~buster_armhf.deb

machinekit@mksocfpga-ob-ox:~/recomended_emc$ halrun
msgd:0 stopped
rtapi:0 stopped
rtapi_msgd command:  /usr/libexec/machinekit/rtapi_msgd --instance=0 --rtmsglevel=1 --usrmsglevel=1 --debug=1 --halsize=524288
warning: removing unused HAL shm segment /hal-0-00414c32
rtapi_app command:  /usr/libexec/machinekit/rtapi_app --instance=0 --debug=1
3::4345:rt INFO:  Picked default flavor 'rt-preempt' automatically
1:rtapi_app:4345:user cannot gain I/O privileges - forgot 'sudo make setuid'?
1:rtapi_app:4345:user rtapi_app:0 failed to setup realtime environment - 'sudo make setuid' missing?
halcmd: cant connect to rtapi_app: -1 (uri= uuid=18cabec7-96ae-4413-85e0-7f75eed5aa92): rtapi_rpc(): reply timeout

halcmd: the rtapi:0 RT demon is not running - please investigate /var/log/hal.log
halcmd: the msgd:0 logger demon is not running - please investigate /var/log/hal.log
E: 20-10-14 14:50:37 [4346]dangling 'DEALER' socket created at hal/utils/halcmd_rtapiapp.cc:284

machinekit@mksocfpga-ob-ox:~/recomended_emc$ cat /var/log/hal.log

Oct 14 14:50:31 localhost rtapi:0: 1:rtapi_app:4345:user cannot gain I/O privileges - forgot 'sudo make setuid'?
Oct 14 14:50:31 localhost rtapi:0: 1:rtapi_app:4345:user rtapi_app:0 failed to setup realtime environment - 'sudo make setuid' missing?
Oct 14 14:50:31 localhost msgd:0: rtapi_app exit detected - scheduled shutdown
Oct 14 14:50:33 localhost msgd:0: msgd shutting down

same reuult for:

machinekit@mksocfpga-ob-ox:~/recomended_emc$ ls
machinekit-hal_0.4.20868-1.gitca75c54aa~buster_armhf.deb  machinekit-hal-rt-preempt_0.4.20868-1.gitca75c54aa~buster_armhf.deb

Since the guide specifies max or = 0.4.20868 the setuid problem in these packages is a big showstopper.

---

On the other side I prefer the Machineface gui qtquickvcp gui setup instead of any builtin linuxcnc gui's so I wonder if its possible to
build against 0.4.20996 and ignore the builtin lcnc gui's and get the Machineface gui to work instead ?

[the-snowwhite]

I can confirm that this issue is not relevant for the arm64 packages.
I just sucessfully ran the emcapplication on an Ultra96v1 after recompiling for machinekit-hal_0.4.20868-1.gitca75c54aa

[cerna]

@the-snowwhite (@Holotronic, sorry I have just now noted that you opened the machinekit/machinekit-docs#321 with different account, and I am not sure which one are you using),

Installing the debs I also had to include:
python3-avahi_0.8-3_armhf.deb
libavahi-common-data_0.8-3_armhf.deb
from "Bullseye"

These packages should be available from the Machinekit/Machinekit repository: Python3-avahi and Libavahi-common-data and actually should be installed automatically as dependencies of the machinekit-hal* packages. But I actually have never tested them with real armhf hardware. Does it mean that they are not working?

For some reason emcapplication_2.9.0-pre0.23507.git1dcbe183f-buster_armhf.deb asked for machinekit-hal_0.4.20996

The reason is that I haven't yet regenerated the emcapplication packages yet and so far there is no CI service. More info can be had in machinekit/EMCApplication#2 and machinekit/EMCApplication#6.

Investigating further I found out running halrun gave error messages: (with both version 0.4.20894 and 0.4.20868) However not with the latest version:0.4.20996

This is interesting. So can I understand it that the whole machinekit-hal package at version 0.4.20894 does not run on armhf architecture? @jallwine reported some time back the same thing, I recommended commenting out code in rtapi_app.cc, which now I can see was included in his latest patch, so that is the reason why it is now working. (That code - well, at least in my opinion - has no place in that place, it should complain lot lower in the module where I/O access is actually requested.)

Can you try to comment it out and rebuild the package at the 20894 (ebe1344a0) state?

On the other side I prefer the Machineface gui qtquickvcp gui setup instead of any builtin linuxcnc gui's so I wonder if its possible to
build against 0.4.20996 and ignore the builtin lcnc gui's and get the Machineface gui to work instead ?

It should be possible, but so far I haven't looked into it. (I have been procrastinating on the EMCApplication's issue, so...). But still it will need first the Python3 packaging in upstream LinuxCNC@master (they are doing it at the same time).

The best way how to attack this is probably to create a new package for the whole qtquickvcp against Machinekit-HAL. (If I remember correctly, it was never tested with Machinekit-HAL, only the monorepo Machinekit.)

[the-snowwhite]

@cerna sorry about the @Holotronic mishap I created that account for some research help work that unfortunately has to be in a private repo.

Can you try to comment it out and rebuild the package at the 20894 (ebe1344a0) state?

Sure: I can test out the armhf commenting out tomorrow, however I have been testing out the mklauncher, qtquickvcp, Cetus, MachinekitClient stuff out on my arm64 based Ultra96 so far without any success:

1. Does'nt seem to accept python based hal config. --> attempted to mold a traditional hal file.
2. latest debug info I'm stuck at is: ./ox.hal:430: haltalk exited without becoming ready. (the very last line don't know what to do about that)

I suspect any armhf based approach will strand the same place.

BTW: The qtquickvcp, python-hal and MachinekitClient stuff did make it past the monorepo Machinekit:

Just before you stepped in armhf qtquickvcp was already working in the machinekit-hal machinekit-cnc repo combo and .deb packaged and in distribution.

The arm64 mk-hal / mk-cnc combo was just made to work correctly and actually committed when unfortunately exactly then the plug was simultaneously pulled on the old build system's, so the arm64 packages were never built and distributed officially, likewise with any work in progress with arm64 mksocfpga building and packaging.

[the-snowwhite]

about:
2.
I see following messages in hal.log:

...
Oct 14 23:48:20 localhost haltalk: FATAL - 'MACHINEKIT_INI' missing in environment
Oct 14 23:48:22 localhost msgd:0: hal_lib:3794:rt hm2/hm2_ultr.0: unregistered
Oct 14 23:48:22 localhost msgd:0: hal_lib:3794:rt hm2: unloading
...

Edit:
Its possible to get the configuration to take hold by doing:

export MACHINEKIT_INI=/etc/machinekit/machinekit.ini

before running mklauncher.

However MachinekitClient is only able to find the file service and the rest of needed services are missing.

[the-snowwhite]

I did the commenting out and rebuilt the mk-hal armhf packages and then the emcapplication and then ran it on
my DE0_Nano_Soc / Atlas.

linuxcnc ran the default axis sim after doing:

xrdb -load /dev/null

However stepconf and pncconf are no-go.

Edit:
I wonder if it would be possible to rebuild the armhf 0.4.20868, etc cloudsmith packages and get them online instead ?

[cerna]

@the-snowwhite,

However stepconf and pncconf are no-go.

Actually, that's a feature, not a bug. 😛

Both these programs are manipulating the HAL and as such both come under the LinuxCNC HAL implementation, not Machinekit-HAL.

Sure: I can test out the armhf commenting out tomorrow, however I have been testing out the mklauncher, qtquickvcp, Cetus, MachinekitClient stuff out on my arm64 based Ultra96 so far without any success:

Hmm, I understood it that the arm64 architecture (Machinekit-HAL stack) is working flawlessly and only the armhf is causing problems.

Both Travis and Drone Cloud are testing the arm* build code natively, but both are using actually arm64 hardware with only Drone testing the armhf under faked environment. So it might happen that for some reason tests are green but actually the application is not.

The arm64 mk-hal / mk-cnc combo was just made to work correctly and actually committed when unfortunately exactly then the plug was simultaneously pulled on the old build system's, so the arm64 packages were never built and distributed officially, likewise with any work in progress with arm64 mksocfpga building and packaging.

Hopefully we will be able to get it all working now too! 👍

I wonder if it would be possible to rebuild the armhf 0.4.20868, etc cloudsmith packages and get them online instead ?

You mean the EMCApplication requiring the 0.4.20868 Machinekit-HAL or Machinekit-HAL at 0.4.20868 state with that code commented out?

BTW, can you merge pull requests?

[the-snowwhite]

I meant

Machinekit-HAL at 0.4.20868 state with that code commented out

BTW, can you merge pull requests?

I think I can ... I just havn't been brave enough to try..

[the-snowwhite]

Hmm, I understood it that the arm64 architecture (Machinekit-HAL stack) is working flawlessly and only the armhf is causing problems.

What does flawlessly mean ?

1. are python based hal configs supposed to work ?
2. are mklauncher based configs with f.ex Cetus/Machineface and MachinekitClient supposed to work ?

[the-snowwhite]

I got a step closer:
It's possible to patch the linuxcnc script to get it to parse python based hal config files:

mib@kdeneon-ws3:~/Developer/ext-repos_git/emcapplication$ git diff scripts/linuxcnc.in
diff --git a/scripts/linuxcnc.in b/scripts/linuxcnc.in
index 7cc32928e..2c013bcaf 100644
--- a/scripts/linuxcnc.in
+++ b/scripts/linuxcnc.in
@@ -847,6 +847,12 @@ else
                exit -1
            fi
        ;;
+        *.py)
+            if ! python2 $CFGFILE ; then
+                Cleanup
+                exit -1
+            fi
+        ;;
        *)
            if ! $HALCMD -i "$INIFILE" -f $CFGFILE && [ "$DASHK" = "" ]; then
                Cleanup

And then run a mklauncer run.py config
This however leads to a new problem loading the hm2_soc_ol driver workes fine from a hal fil but running mklauncher gives this:

starting mklauncher... done
starting configserver... done
starting linuxcnc... LINUXCNC - 2.9.0~pre0
Machine configuration directory is '/home/machinekit/Hm2-soc_FDM/Cramps/PY/OX'
Machine configuration file is 'ox.ini'
Starting LinuxCNC...
done
rtapi_msgd command:  /usr/libexec/machinekit/rtapi_msgd --instance=0 --rtmsglevel=1 --usrmsglevel=1 --debug=1 --halsize=524288
rtapi_app command:  /usr/libexec/machinekit/rtapi_app --instance=0 --debug=1
3::4686:rt INFO:  Picked default flavor 'rt-preempt' automatically
Found file(REL): ./ox.py
1:rtapi_app:4686:user hal_call_usrfunct(newinst,--  "num_pwmgens=1 num_stepgens=4") failed: -1 - Operation not permitted
Traceback (most recent call last):
File "./ox.py", line 18, in <module>
    motion.setup_motion()
File "/home/machinekit/Hm2-soc_FDM/Cramps/PY/OX/fdm_local/config/motion.py", line 18, in setup_motion
    c.find('HOSTMOT2', 'CONFIG'))
File "hal/cython/machinekit/rtapi.pyx", line 272, in machinekit.rtapi.RTAPIcommand.newinst
RuntimeError: rtapi_newinst '('hm2_soc_ol', '"hm2-socfpga0 already_programmed=1"', '-- ', '"num_pwmgens=1 num_stepgens=4"')' failed: Operation not permitted
Shutting down and cleaning up LinuxCNC...
LinuxCNC terminated with an error.  You can find more information in the log:
    /home/machinekit/linuxcnc_debug.txt
and
    /home/machinekit/linuxcnc_print.txt
as well as in the output of the shell command 'dmesg' and in the terminal
stopping mklauncher... done
stopping configserver... done

[cerna]

@the-snowwhite,

are python based hal configs supposed to work ?

I am trying to get the Python based Machinekit-HAL application working and so far having a problem:

./run.py
stopping realtime... done
cleaning up leftover session... done
starting realtime...rtapi_msgd command:  /usr/libexec/machinekit/rtapi_msgd --instance=0 --rtmsglevel=1 --usrmsglevel=1 --debug=1 --halsize=524288
rtapi_app command:  /usr/libexec/machinekit/rtapi_app --instance=0 --debug=1
3::20528:rt INFO:  Picked default flavor 'posix' automatically
done
loading main.py... done
starting mklauncher... INFO:mklauncher:Remote communication is deactivated, configserver will use the loopback interfaces
INFO:mklauncher:set REMOTE in /etc/machinekit/machinekit.ini to 1 to enable remote communication
ERROR:dbus.connection:Unable to set arguments (0, 0, dbus.UInt32(0), 'Machinekit Launcher on mars.local', '_machinekit._tcp', '', '', dbus.UInt16(46869), ['dsn=tcp://127.0.0.1:46869', 'uuid=39b7d24d-a7a3-486e-947e-be1db9a3b8a7', 'instance=12456472-10b1-11eb-83a3-0c5415b56747', 'service=launcher']) according to signature 'iiussssqaay': <class 'TypeError'>: an integer is required (got type str)
ERROR:mklauncher:cannot register DNS servicean integer is required (got type str)
stopping realtime... stopping realtime... halcmd: cant connect to rtapi_app: -1 (uri= uuid=39b7d24d-a7a3-486e-947e-be1db9a3b8a7): rtapi_rpc(): reply timeout

halcmd: the rtapi:0 RT demon is not running - please investigate /var/log/hal.log
halcmd: the msgd:0 logger demon is not running - please investigate /var/log/hal.log
E: 20-10-17 21:43:52 [20583]dangling 'DEALER' socket created at hal/utils/halcmd_rtapiapp.cc:284
done

Based on the python-hal-seed (BTW, why these useful tutorials are not in the main Machinekit repository is beyond me, I discovered it just yesterday).

Can you confirm that you are seeing the same problem? Or is your problem different?

Looking back, you have probably different problem...

[the-snowwhite]

Oh this is a normal problem you just have to:

set REMOTE in /etc/machinekit/machinekit.ini to 1

with nano or some other text editor.

[cerna]

@the-snowwhite,

Oh this is a normal problem you just have to:

Yes, but the real problem is actually hidden in:

starting mklauncher... ERROR:dbus.connection:Unable to set arguments (-1, 0, dbus.UInt32(0), 'Machinekit Launcher on mars.local', '_machinekit._tcp', '', '', dbus.UInt16(34681), ['dsn=tcp://mars.local:34681', 'uuid=39b7d24d-a7a3-486e-947e-be1db9a3b8a7', 'instance=e5e6f16a-1133-11eb-83ac-0c5415b56747', 'service=launcher']) according to signature 'iiussssqaay': <class 'TypeError'>: an integer is required (got type str)
ERROR:mklauncher:cannot register DNS servicean integer is required (got type str)

Which comes from file lib/python/machinekit/service.py. I have tried this file outside Machinekit and yes, there is some (type) problem stemming from the Python 2 -> Python 3 port and how the action is requested from Avahi by DBus. I have been able to create the announcement by mucking the code little bit, but unfortunately I will have to study the DBus and Avahi more.

Have no idea if @zultron or @rene-dev looked at this portion of code in their Python3 marathon.

[rene-dev]

I only ported hal, and mentioned serveral times that the UIs will be broken after this.
I will happily look into the issue, but could not figure out how to run the repo in place, with non-packaged MK HAL