Document language inspection is flaky

[php-coder]

I see that 0.27 version has started to report the following warnings:

WARNING:html5validator.validator:"file:/home/travis/build/php-coder/mystamps/src/main/webapp/WEB-INF/views/error/403.html":1.16-5.73: info warning: This document appears to be written in Danish but the "html" start tag has "lang="en"". Consider using "lang="da"" (or variant) instead.
"file:/home/travis/build/php-coder/mystamps/src/main/webapp/WEB-INF/views/error/500.html":1.16-5.73: info warning: This document appears to be written in Danish but the "html" start tag has "lang="en"". Consider using "lang="da"" (or variant) instead.

Here are these files:

• https://github.com/php-coder/mystamps/blob/be8d102327bcc68ca256bde8aed489886bd86835/src/main/webapp/WEB-INF/views/error/403.html
• https://github.com/php-coder/mystamps/blob/be8d102327bcc68ca256bde8aed489886bd86835/src/main/webapp/WEB-INF/views/error/500.html
  They're in English and not in Danish so these warnings are false positives.

[php-coder]

I forgot to mention how I'm executing html5validator:

html5validator \
	--root src/main/webapp/WEB-INF/views \
	--ignore-re 'Attribute “(th|sec|togglz|xmlns):[a-z]+” not allowed' \
		'Attribute “(th|sec|togglz):[a-z]+” is not serializable' \
		'Attribute with the local name “xmlns:[a-z]+” is not serializable' \
		'An "img" element must have an "alt" attribute' \
		'The first child "option" element of a "select" element with a "required" attribute' \
	--show-warnings

[php-coder]

I couldn't reproduce it locally under MacOS. But it 100% reproducible in TravisCi.

[php-coder]

Could be related to validator/validator#493

[php-coder]

It reproduces on Linux.

[php-coder]

@sideshowbarker Could you look at it? I suspect that it's not html5validator problem but rather validator related.

[sideshowbarker]

@php-coder I’m not able to reproduce the problem when I directly check the documents using the W3C HTML Checker:

• https://validator.w3.org/nu/?doc=https%3A%2F%2Frawgit.com%2Fphp-coder%2Fmystamps%2Fbe8d102327bcc68ca256bde8aed489886bd86835%2Fsrc%2Fmain%2Fwebapp%2FWEB-INF%2Fviews%2Ferror%2F403.html

• https://validator.w3.org/nu/?doc=https%3A%2F%2Frawgit.com%2Fphp-coder%2Fmystamps%2Fbe8d102327bcc68ca256bde8aed489886bd86835%2Fsrc%2Fmain%2Fwebapp%2FWEB-INF%2Fviews%2Ferror%2F500.html

[svenkreiss]

@php-coder Can you link to a Travis build where this error occurred?

[php-coder]

Yes, here they're:

• https://travis-ci.org/php-coder/mystamps/jobs/222440191
• https://travis-ci.org/php-coder/mystamps/jobs/222606169

[php-coder]

@sideshowbarker @svenkreiss Hi, this issue has started to appear again time to time, but now it says that the document in French.

Could you suggest me a way of debugging it? Is enabling verbose mode would help? Where the sources of this check, so I can read it/play with the code? Thanks!

[sideshowbarker]

My only suggestion as far as debugging is to see if you can reproduce it with the vnu.jar directly or with https://checker.html5.org/ or https://validator.w3.org/nu/

[svenkreiss]

I agree, you will have to look into the Java part for debugging this.

Also just had a look at your 403.html file. It actually is a template file and not pure HTML. I wouldn't be surprised if the special characters trip up the language detection.

[sideshowbarker]

Also just had a look at your 403.html file. It actually is a template file and not pure HTML. I wouldn't be surprised if the special characters trip up the language detection.

Yeah, if that’s the case, then all bets are off as far as the HTML checker backend behavior goes — and not just specifically for language detection. The checker isn’t a tool for checking pre-parsed PHP or template content. It’s intended for checking the HTML contents as they would be sent over the wire.

[php-coder]

It actually is a template file and not pure HTML.

It should be valid HTML that have a bunch of non-standard th:* attributes.

@svenkreiss Could you add an option to expose --no-langdetect to html5validator then?

[svenkreiss]

Yes, that sounds like a good idea.

…

On Thu, Jan 4, 2018 at 6:44 AM, Vyacheslav Semushin < ***@***.***> wrote: It actually is a template file and not pure HTML. It should be valid HTML that have a bunch of non-standard th:* attributes. @svenkreiss <https://github.com/svenkreiss> Could you add an option to expose --no-langdetect to html5validator then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#32 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACpYJpuz_TEyopQwreQcWyUPtNwCLZJjks5tHLmQgaJpZM4M-cg4> .

[svenkreiss]

Version 0.2.10 is now on pypi that has this command line option. Does that help?

[php-coder]

Thanks! Looks like it worked!