website-serve infinite loop

I just realized that the "website" module does not work any more on linux. Minimal example:

(website-serve #f 8080)

The code does nothing but to start the local server. It should do nothing but to recat to http requests. But it results in one process occupying one core 100%. That process does not generate any output not does it response to any request.

[mgorges]

Sorry, I can't seem to reproduce this easily; this is from a fresh clone of the current main branch:

mgorges@ECEM01:~/lambdanative$ uname -a
Linux ECEM01 5.xxxxx-generic #xxxxx-Ubuntu SMP xxxxx x86_64 x86_64 x86_64 GNU/Linux
mgorges@ECEM01:~/lambdanative$ git log --pretty=format:'%h' -n 1
d06be02
mgorges@ECEM01:~/lambdanative$ git diff apps/DemoConsole/
diff --git a/apps/DemoConsole/MODULES b/apps/DemoConsole/MODULES
index 668f264..457c15e 100644
--- a/apps/DemoConsole/MODULES
+++ b/apps/DemoConsole/MODULES
@@ -1 +1 @@
-config ln_core 
+config ln_core generalized-arrays website 
diff --git a/apps/DemoConsole/main.scm b/apps/DemoConsole/main.scm
index c0250af..aa55fc8 100644
--- a/apps/DemoConsole/main.scm
+++ b/apps/DemoConsole/main.scm
@@ -38,6 +38,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 (display "DemoConsole\n")
 
+(define db (website->table "example-website"))
+(website-addhook db "/index.html" (lambda (x) "AAA\n"))
+(website-serve db 8888)
+
 (let loop () 
   (with-exception-catcher (lambda (e) 
     (for-each display (list (exception->string e) "\n")) #f) 
mgorges@ECEM01:~/lambdanative$ make; make install
=== using profile PART BC Children's [/home/mgorges/lambdanative/PROFILE]..
=== configured to build DemoConsole version 1.0 for linux on linux in normal mode

==> checking for required tools..
==> checking for required libraries..
==> checking for lambdanative tools..
==> creating libraries needed for DemoConsole..
 => liblambdanative..
 => cleaning up..
 => exploding library liblambdanative..
==> updating embedded files for DemoConsole..
==> updating web assets for DemoConsole..
 => compiling scheme payload..
    /home/mgorges/lambdanative/apps/DemoConsole/main.scm ..
 => done compiling scheme payload
 => generating hook..
 => assembling payload..
 == /home/mgorges/.cache/lambdanative/linux/lib/libpayload.a
==> creating linux loader needed for DemoConsole..
 => processing sounds needed for DemoConsole..
 => compiling application..
 => cleaning up..
=== /home/mgorges/.cache/lambdanative/linux/DemoConsole
==> making package..
 => making generic zip archive /home/mgorges/.cache/lambdanative/packages/DemoConsole-1.0-linux.zip..
=== /home/mgorges/.cache/lambdanative/packages/DemoConsole-1.0-linux.zip
=== using profile PART BC Children's [/home/mgorges/lambdanative/PROFILE]..
=== configured to build DemoConsole version 1.0 for linux on linux in normal mode

==> attempting to install linux application DemoConsole to local desktop..
==> attempting to copy .desktop file to /home/mgorges/.local/share/applications/
cp: cannot create regular file '/home/mgorges/.local/share/applications/': Not a directory
==> Starting application..
DemoConsole
>

Here is a complete testproject that shows the problem - on linux, that is, I don't know if it happens on other OS.
DemoWebTemplate.zip

I dug a bit deeper: The server hangs at the read in modules/website/website.scm line 129 (

lambdanative/modules/website/website.scm

Line 129 in b9f302a

(let loop () (let ((connection (read accept-port)))

)

Wireshark says:

• GET / is sent to :8080
• :8080 responds with TCP Keeep-alive
• firefox answer with TCP Keeep-alive
  This TCP Keep-Alive ping-pong repeats ad infinitum. Well, not eactly, after ~ 30 minutes sometimes the read finishes.

Don't know what to do about it.

[mgorges]

Your test project example runs fine when built against libgambit, but I can also make it fail on Gentoo with libgambc. Interestingly, if I comment the line with (website-addsubstitution "[PROC]" (let ((a 0)) (lambda () (set! a (+ a 1)) (number->string a)))) it oftentimes runs and sometimes fails. I originally thought it might have something to do with (glgui-timings-at-10msec!), but that doesn't fully fix it, which makes me suspicious that the problem is with the lack of render-loop calls in your hybrid app.

If I try apps/DemoHybridApp/main.scm and don't give focus to the small loader window, it also doesn't do anything; yet when I give it focus, it renders the website. Applying the same idea to your app, it doesn't render the website, but once I click the black box in the 'DemoWebTemplate' app window the website renders. I can't quite point to where exactly this happens but it is lack of redraw events coming from OpenGL and is a known problem (see #86 (comment)).

Found it!

loaders/x11/x11_microgl.c line 253 says:

lambdanative/loaders/x11/x11_microgl.c

Line 253 in b9f302a

expose = expose || (event.xexpose.width && event.xexpose.height) ? 1 : 0;

Now this removes all Expose Events, as nowadays event.xexpose.width and event.xexpose.height are always 0.

Bugfix: Replace the line with this:

expose = 1;

... as an afterthought: if the application is only a webserver without GUI, wouldn't it be nice to remove the opengl window? Maybe a target called console like android? For my usecase the code would run as a small webserver without GUI, so I'd need to figure out how that would work anyway.

[mgorges]

The loaders/x11/x11_microgl.c#L253 code used to be that way since inception until b0a16b3, and I can't recall why we changed it specifically? - Added: It was part of pull request #417 which we partially implemented; I'd be happy to revert that change but I didn't author the original change.

Nothing requires you to use a web server with a GUI (my earliest example was a ConsoleApp). - It is solely that you used modules/hybridapp/hybridapp.scm#L68 which has the GUI requirement so people can use it to test applications that run in a mobile device - if you look at modules/hybridapp/MODULES it either applies an eventloop approach on mobile platforms or a ln_glgui one everywhere else?

The patch is flaged as "avoid some of the needless redraw events" - that might have been true in earlier days, but while they might end in eventloop.scm but do nothing. A redraw is explictly done when EVENT_REDRAW is sent down the line, so no problem there.

@hybridapp: Yes, I missed that part. I was unter the impression that hybridapp works the same on X11 as on iOS/Android - till I realised that the browser part is totally independent from the scheme part.

[mgorges]

Before I undo b0a16b3 could you test and confirm that this fixes your problem completely; I recall from earlier data collection applications we had built that processing stops after a few seconds when the focus is not on the app - this is a Linux-specific problem, as Windows has a timer that triggers it at least every second (fdc3626), and similar heartbeat things exist on iOS and Android.

At least on my system it works, even when the dummy window is never exposed (= it's hidden behind forground windows all the time) and never get the focus. I don't think the timer is needed on linux - at least on Xorg, things might be different on Wayland. I could add a timer if you find that processing stops after some seconds - alarm(1) will most likely do.