Support for last-modified resolution in Tomcat's unpackwar=false mode [SPR-13393]

[spring-projects-issues]

Gaurav Rawat opened SPR-13393 and commented

Hi I am using spring data rest provided HAL browser to view my sprign data rest HAL based api .So far things have been good when I am running it via eclipse or as a spring boot app for testing in my local tomcat on windows.Though when I deploy on aws on a tomcat container (as a spring boot war) I get this weird error as below when I browse to the root or the address of the hal browser /browser/index.html#

Whitelabel Error Page

This application has no explicit mapping for /error, so you are seeing this as a fallback.

Mon Aug 24 07:05:46 UTC 2015
There was an unexpected error (type=Internal Server Error, status=500).
Jar URL cannot be resolved to absolute file path because it does not reside in the file system: war:jar:file:/deployment/wars/hfds.1.3.war!/WEB-INF/lib/spring-data-rest-hal-browser-2.4.0.BUILD-SNAPSHOT.jar

-I am still not able to find a valid justification on why its not being able to find the provided hal browser though things work fine in local .Am I missing something can someone please help .Is this a bug with spring framework or otherwise-

This works if unpackwar=true but doesnt if unpackwar is set to false - That seems to cause the real issue
UPDATE

The stack trace shows the issue coming from the code below when unpackw=false ,I get war:jar:file which causes the issue due to this code section in org.springframework.util.ResourceUtils

public static File getFile(URL resourceUrl, String description) throws FileNotFoundException {
    Assert.notNull(resourceUrl, "Resource URL must not be null");
    if (!URL_PROTOCOL_FILE.equals(resourceUrl.getProtocol())) {
        throw new FileNotFoundException(
                description + " cannot be resolved to absolute file path " +
                "because it does not reside in the file system: " + resourceUrl);
    }
    try {
        return new File(toURI(resourceUrl).getSchemeSpecificPart());
    }
    catch (URISyntaxException ex) {
        // Fallback for URLs that are not valid URIs (should hardly ever happen).
        return new File(resourceUrl.getFile());
    }
}

Not sure it should go to the Spring data bucket or Spring framework in general .

+UPDATE NEW+

ALso after more analysis seems this is not an environment specific issue but it occurs if a resource like HAL_Browser is packed inside a jar and tomcat unpackwar is set to false .In this situation somehow spring is not able to extract contents from the jar file and render them .

---

Affects: 4.1.7, 4.2 GA

Reference URL: http://stackoverflow.com/questions/32176383/not-able-to-open-spring-default-hal-browser-provided-by-spring-data-rest-on-aws

Issue Links:

• PathMatchingResourcePatternResolver fails to work under Tomcat 8.0.41 with unpackWARs=false [SPR-15332] #19895 PathMatchingResourcePatternResolver fails to work under Tomcat 8.0.41 with unpackWARs=false

Referenced from: commits 302a069, 1c3a668, 155bbf5

Backported to: 4.1.8

[spring-projects-issues]

Stéphane Nicoll commented

This is the issue tracker for the Spring framework. That integration is provided by Spring Boot so please open an issue in the Spring Boot tracker

[spring-projects-issues]

Gaurav Rawat commented

sure spring-projects/spring-boot#3826

[spring-projects-issues]

Andy Wilkinson commented

Having investigated the Spring Boot issue a little more, I think this actually was the appropriate place to report the issue. The problem affects any use ResourceHttpRequestHandler when using an unpacked war file. For example, the problem can be recreated by deploying https://github.com/spring-projects/spring-petclinic to Tomcat 8.0.26 configured with unpackWARs=false in server.xml. Navigating around the app will produce numerous exceptions similar to the following:

java.io.FileNotFoundException: Jar URL cannot be resolved to absolute file path because it does not reside in the file system: war:jar:file:/Users/awilkinson/dev/runtimes/tomcat/8.0.26/webapps/petclinic.war!/WEB-INF/lib/jquery-ui-1.10.3.jar
	at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:212) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.core.io.AbstractFileResolvingResource.getFileForLastModifiedCheck(AbstractFileResolvingResource.java:67) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.core.io.AbstractResource.lastModified(AbstractResource.java:153) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.core.io.AbstractFileResolvingResource.lastModified(AbstractFileResolvingResource.java:169) ~[spring-core-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.resource.ResourceHttpRequestHandler.handleRequest(ResourceHttpRequestHandler.java:231) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:51) ~[spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) [servlet-api.jar:na]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) [spring-webmvc-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [servlet-api.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-websocket.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at com.github.dandelion.datatables.core.web.filter.DatatablesFilter.doFilter(DatatablesFilter.java:74) [datatables-core-1.0.1.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at com.github.dandelion.core.web.DandelionFilter.doFilter(DandelionFilter.java:128) [dandelion-core-1.0.1.jar:na]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.1.7.RELEASE.jar:4.1.7.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.26]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.26]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) [catalina.jar:8.0.26]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [catalina.jar:8.0.26]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [catalina.jar:8.0.26]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.0.26]
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) [catalina.jar:8.0.26]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:8.0.26]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518) [catalina.jar:8.0.26]
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091) [tomcat-coyote.jar:8.0.26]
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:673) [tomcat-coyote.jar:8.0.26]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1526) [tomcat-coyote.jar:8.0.26]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1482) [tomcat-coyote.jar:8.0.26]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.26]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]

[spring-projects-issues]

Juergen Hoeller commented

Fixed through accepting Tomcat's war:jar: URL format in ResourceUtils.isJarURL/extractJarFileURL.

Juergen

[spring-projects-issues]

Gaurav Rawat commented

great thought that was the issue .Can this be taken in the latest 4.2 build snapshot or we have to wait

Regards

[spring-projects-issues]

Juergen Hoeller commented

I've just pushed it, so the upcoming 4.2.2.BUILD-SNAPSHOT will include it already! 4.1.8 following in about half an hour.

Juergen

[spring-projects-issues]

Juergen Hoeller commented

Unfortunately, the fix turns out to be incomplete... reopening the issue for another pass.

Juergen

[spring-projects-issues]

Juergen Hoeller commented

I'm resolving this in a slightly different way, with a new ResourceUtils.extractArchiveURL method being able to parse Tomcat's war file paths and always returning the outermost archive (instead of the nested jar URL that extractJarFileURL would return). AbstractFileResolvingResource uses that extractArchiveURL method within its getFileForLastModifiedCheck method now.

This will show up in the upcoming 4.1.8.BUILD-SNAPSHOT first now and in 4.2.2.BUILD-SNAPSHOT in about an hour.

Juergen

[spring-projects-issues]

Gaurav Rawat commented

Hi @Juergen the errors seems to be still coming in 4.2.3.RELEASE has it also been rolled of there ? trace below

java.io.FileNotFoundException: Jar URL cannot be resolved to absolute file path because it does not reside in the file system: war:jar:file:/deployment/wars/hfds.0.2.war!/WEB-INF/lib/spring-data-rest-hal-browser-2.4.0.RELEASE.jar at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:218) ~[spring-core-4.2.3.RELEASE.jar:4.2.3.RELEASE] at org.springframework.core.io.AbstractFileResolvingResource.getFileForLastModifiedCheck(AbstractFileResolvingResource.java:67) ~[spring-core-4.2.3.RELEASE.jar:4.2.3.RELEASE]

[spring-projects-issues]

Juergen Hoeller commented

Andy Wilkinson, we did test this against 4.2.3, didn't we? Are you aware of any limitations there still?

Juergen

[spring-projects-issues]

Andy Wilkinson commented

We did indeed. I'm not aware of any limitations. I've just tested a small sample app and it was able to serve static content both from directly within the war and from jars in /WEB-INF/lib. I used Spring Boot's spring-boot-sample-web-static sample and added HAL browser to it (org.springframework.data:spring-data-hap-browser).

[spring-projects-issues]

Gaurav Rawat commented

HI @Juergen amd @Andy it still seems to be broken .Seems my app on ec2 running on tomcat 8 is picking the getFile method 218 as below and then the check there fails due to the condition below as its not a file resource I reckon

public static File getFile(URL resourceUrl, String description) throws FileNotFoundException {
		Assert.notNull(resourceUrl, "Resource URL must not be null");
		if (!URL_PROTOCOL_FILE.equals(resourceUrl.getProtocol())) {
			throw new FileNotFoundException(
					description + " cannot be resolved to absolute file path " +
					"because it does not reside in the file system: " + resourceUrl);
		}
		try {
			return new File(toURI(resourceUrl).getSchemeSpecificPart());
		}
		catch (URISyntaxException ex) {
			// Fallback for URLs that are not valid URIs (should hardly ever happen).
			return new File(resourceUrl.getFile());
		}
	}

Full stack trace goes like

java.io.FileNotFoundException: Jar URL cannot be resolved to absolute file path because it does not reside in the file system: war:jar:file:/deployment/wars/foo_1.10.war!/WEB-INF/lib/spring-data-rest-hal-browser-2.4.4.RELEASE.jar
        at org.springframework.util.ResourceUtils.getFile(ResourceUtils.java:218)
        at org.springframework.core.io.AbstractFileResolvingResource.getFileForLastModifiedCheck(AbstractFileResolvingResource.java:67)
        at org.springframework.core.io.AbstractResource.lastModified(AbstractResource.java:153)
        at org.springframework.core.io.AbstractFileResolvingResource.lastModified(AbstractFileResolvingResource.java:169)
        at org.springframework.web.servlet.resource.ResourceHttpRequestHandler.handleRequest(ResourceHttpRequestHandler.java:240)
        at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:51)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:968)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:859)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:618)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:844)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:237)