Jailer: vsock: Failure to launch VM with vsock even with vsock disabled: code: 25

[mcastelino]

When using jailer with vsock even with the seccomp filters disabled VM launch fails.

Firecracker version: commit f4349ca
Firecracker build process: ./tools/devtool --unattended build --debug -- --features vsock

The error reported by firecracker is

[anonymous-instance:ERROR:api_server/src/http_service.rs:586] Received Error on synchronous Put request on "/actions" with body "{\n        \"action_type\": \"InstanceStart\"\n     }". Status code: 500 Internal Server Error.

Error reported to the caller of the API is on InstanceStart is

{
  "fault_message": "Cannot create vsock device. VhostGetFeatures(IoctlError(Os { code: 25, kind: Other, message: Not a tty }))"
}

A simple test case

Launch Jailer

Note: The vhost-vsock bind mounting is done later in the VM launch script

sudo ip netns add nsfire
sudo ip tuntap add dev fire0 mode tap
sudo ip link set fire0 netns nsfire
mkdir -p /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/
ln $HOME/firecracker/hello-vmlinux.bin /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/vmlinux
ln $HOME/firecracker/hello-rootfs.ext4 /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/rootfs.img
sudo $HOME/firecracker/build/debug/jailer --id 551e7604-e35c-42b3-b825-416853441234 --node 0 --exec-file $HOME/firecracker/build/debug/firecracker --uid 0 --gid 0 --chroot-base-dir /tmp/firecracker --netns /var/run/netns/nsfire --seccomp-level 0

Launch the VM

SOCKET=/tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/api.socket
sudo -E curl --unix-socket "$SOCKET" -i \
    -X PUT 'http://localhost/boot-source'   \
    -H 'Accept: application/json'           \
    -H 'Content-Type: application/json'     \
    -d '{
        "kernel_image_path": "./vmlinux",
        "boot_args": "console=ttyS0 reboot=k panic=1 pci=off init=/bin/ash"
    }'

sudo -E curl --unix-socket "$SOCKET" -i \
    -X PUT 'http://localhost/drives/rootfs' \
    -H 'Accept: application/json'           \
    -H 'Content-Type: application/json'     \
    -d '{
        "drive_id": "rootfs",
        "path_on_host": "./rootfs.img",
        "is_root_device": true,
        "is_read_only": false
    }'

# Allow vhost-vsock access
sudo touch /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/dev/vhost-vsock
sudo mount --bind /dev/vhost-vsock /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/dev/vhost-vsock
sudo ls -alp /tmp/firecracker/firecracker/551e7604-e35c-42b3-b825-416853441234/root/dev/vhost-vsock
sudo ls -alp /dev/vhost-vsock

sudo -E curl --unix-socket "$SOCKET" -i \
     -X PUT "http://localhost/vsocks/root" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{
            \"id\": \"root\",
            \"guest_cid\": 8
         }"


sudo -E curl --unix-socket "$SOCKET" -i \
     -X GET "http://localhost/" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \

sudo -E curl --unix-socket "$SOCKET" -i \
    -X PUT 'http://localhost/actions'       \
    -H  'Accept: application/json'          \
    -H  'Content-Type: application/json'    \
    -d '{
        "action_type": "InstanceStart"
     }'

sudo -E curl --unix-socket "$SOCKET" -i \
     -X GET "http://localhost/" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \

[mcastelino]

/cc @andreeaflorescu @xibz

#897 indicates that this should work with seccomp disabled, but I am not seeing it succeed.

Also I am using the root uid/gid for ease of reproduction.

[andreeaflorescu]

This issue and #897 are caused by different problems.

The #897 is caused by the fact that starting with 0.14.0 we enable seccomp by default even if we are not running in the jailer. Another Firecracker change that caused the #897 issue is that we now crash on bad syscalls instead of just logging the syscall that was not whitelisted. The solution for this should be relatively easy (whitelist a syscall), but the work also involves running integration tests to make sure we are not going to break it again.

We will also need to investigate why we cannot run firecracker+vsock using the jailer. Once we find out what is the root cause of this problem, we will let you know.

[acatangiu]

Fixed by #918