Enforce YAJL mandatory dependency in configuration and codebase

[eduar-hte]

The goal of this PR is to enforce YAJL being a mandatory dependencies during configuration and simplify the codebase. Work on this PR originated in discussions about mandatory dependencies in PR #3144 and initially in this comment.

Summary of the changes:

• updated the m4 files that control how the library is configured to make the dependency mandatory so that if the library is not available, configuration will now fail.
• removed #ifdef blocks for the scenario when the library is not present (and is thus unused).
• updated configure.ac to log the library and its version in the mandatory section.

[airween]

Thanks for this great PR again.

I know I suggested to remove the optional state of libxml and libyaml, but now I thought it again - I think we should check the most popular architectures that each of them has both libxml and libyaml support (also pcre2).

Eg. libmodsecurity is available on all platform in Debian, but libxml2 does not supported on ia64.

I don't think this is a big problem, but may be we have to consider it. (I just checked libxml2 and only in case of Debian).

May be we should ask the community - not just here, but on Slack.

What do you think?

[eduar-hte]

Of course, I think it makes sense to check.

With regards to ia64 (Itanium), it was discontinued by Intel back in 2020. Debian last release with support for ia64 was Debian 7 (wheezy) from 2016. They're now on Debian 12 and have support for just their last three releases (external paid support extends to Debian 8). So I guess this in particular won't be an issue. :-)

[eduar-hte]

A quick note on performance comparing the pcre & pcre2 builds (v3/master vs this branch), running the benchmark test with 100.000 requests (instead of 1.000.000). These were run on a couple of local environments, so they're provided just as a reference.

• Mac mini m1
  • v3/master: 64.78 secs
  • this branch: 63.84 secs
• Snapdragon 8cx Gen 3 (arm64) (Debian 11 -bullseye- container running on Windows Subsystem for Linux)
  • v3/master: 95.25 secs
  • this branch: 95.40 secs

I think these indicate that there's no significant difference in performance between these two versions of the library.

NOTE: I couldn't include a run on Windows from a different environment because work on PR #3132 only includes support to build with pcre2 already.

[airween]

Of course, I think it makes sense to check.

With regards to ia64 (Itanium), it was discontinued by Intel back in 2020. Debian last release with support for ia64 was Debian 7 (wheezy) from 2016. They're now on Debian 12 and have support for just their last three releases (external paid support extends to Debian 8). So I guess this in particular won't be an issue. :-)

Sure, I don't think that's a real issue (moreover check the build results of the package in Debian SID here; rows with darker background means that architecture is supported, but not mandatory; so it's no problem if the build fails there).

I just thought we should think it over again, and perhaps we might ask the community - eg. ask them on Slack (or Twitter? 😄)

[airween]

Thanks for these tests!

I think these indicate that there's no significant difference in performance between these two versions of the library.

yes, I didn't expect that there are any significant differences. But yes, it's good to see the fact.

NOTE: I couldn't include a run on Windows from a different environment because work on PR #3132 only includes support to build with pcre2 already.

Even though this PR is not actual at the moment, would you mind to create a separated PR where the PCRE2 will be the default regex engine?

[eduar-hte]

I'm updating the PR to simplify it and just focus on YAJL. The changes to make libxml2 and pcre2 mandatory can be resumed when there's feedback on that.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[eduar-hte]

@airween, I don't think there's anything missing to merge this PR (after limiting its scope to YAJL).

I have a follow-up to build ModSecurity on mac systems running on Apple Silicon, which depends on the changes to yajl.m4, and is thus blocked by this one.

[airween]

I'm updating the PR to simplify it and just focus on YAJL. The changes to make libxml2 and pcre2 mandatory can be resumed when there's feedback on that.

Sorry for the delay, let me check this next week - I'll start with this.

[airween]

In a regular comment I'll explain why is this not a good idea.

[airween]

And this too.

[airween]

Hi @eduar-hte,

thanks for your time and your work again.

This PR was made at the end of May. After that, we had a ModSecurity co-leader meeting (in person) in early June where we discussed a few things. I raised this question whether it is necessary to make the functions optional or whether we can expect them to be mandatory.

@marcstern had a very good argument for the optional requirement, because what if someone doesn't want to work with JSON (or XML). With this step, we completely deprive users of the possibility to reduce the size of the compiled code (think embedded systems).

I'm really sorry to say this, but could you retract just the part of your PR where you made the YAJL library mandatory?

(and it seems like there is a conflict which should be resolve - sorry for that too).

I'm really sorry again and thanks for your patience.

[eduar-hte]

@marcstern had a very good argument for the optional requirement, because what if someone doesn't want to work with JSON (or XML). With this step, we completely deprive users of the possibility to reduce the size of the compiled code (think embedded systems).

That makes sense. The goal of this PR was to align with the documentation in the repository's README, which states that YAJL is mandatory (that whole sentence could be rephrased then to 'other non-mandatory dependencies include YAJL ..., libpcre ..., libXML2).

I'll close this PR then because after the previous changes related to pcre, the remaining changes were related to making YAJL mandatory.