Support JavaScript Templating [SPR-12266]

[spring-projects-issues]

marc schipperheyn opened SPR-12266 and commented

These days platforms like react.js, webcomponents, angular.js and Node.js are pushing the boundaries of our conception of what a web client is to new levels.

In recent years, web clients have gotten more and more powerful and consequently pushed more functionality to that frontend.

Json has become the defacto standard for interacting with a backend such as provided by Spring.

With Node.js, javascript moved to the serverside offering some significant advantages in performance and the use of identical content templates for both frontend and serverside processing.

Micro templates have also become relevant. One challenge that we are dealing with is that our freemarker heavy backend is lifting a lot of processing weight that we also want to happen on the javascript frontend, requiring in essence: duplication of logic.

If you have a heavy investment in Spring with template languages such as freemarker and JSP, you will find that Spring is not keeping up with this revolution. It's still fully "page oriented". Sure, if we're just talking about json, @ResponseBody and all the supporting infrastructure help you process json excellently.

But when it comes to avoiding duplication in the combination of server-side template code and client side rendering, ie Node.js style processing, and basically: how Spring MVC fits into a heavy Javascript client world, you'll be hard put to find any documentation at all. It's either @ResponseBody or Freemarker/Velocity/JSP/etc.

Spring is not bridging the gap between the raw json response and serverside rendering of the template as Node.js is doing.

I think Spring should offer a strategy and guidance in this respect. IMHO this doesn't mean offering a prescribed technology stack, although an opinionated approach in this area would be a relevant "add-on". But I do think that offering both technology and documentation on server-side processing vs client-side processing, micro template languages (DUST, Handlebars, JSX, etc) would make a lot of sense in order to keep Spring relevant. It could even be documenting how to integrate Node.js with a Spring backoffice.

Spring has always excelled in making complex combinations of technologies easy to use. In this new arena of javascript heavy clients, there's a gap in this aspect. Let's face it, the View technologies part of the Spring documentation is hopelessly outdated and represents a view of the web that is rapidly disappearing.

---

Affects: 4.1.5

Issue Links:

• Server-side JavaScript improvements [SPR-13508] #18086 Server-side JavaScript improvements
• ScriptTemplateView configuration option for JavaScript concurrency issues [SPR-13034] #17626 ScriptTemplateView configuration option for JavaScript concurrency issues
• Add JavaScript support for Spring dynamic languages [SPR-7592] #12248 Add JavaScript support for Spring dynamic languages

9 votes, 26 watchers

[spring-projects-issues]

Juergen Hoeller commented

I'm not sure I agree with the "fully page oriented" statement but you certainly do have a point that there is next to no documentation on Spring MVC's role in modern JavaScript stack arrangements. At the very least, we need to completely revise the story that our web documentation tells...

Juergen

[spring-projects-issues]

Rossen Stoyanchev commented

First of all thanks for the thoughtful write-up! The JavaScript client side is and continues to evolve rapidly so it is an ongoing effort. For 4.1 we actually invested a significant amount of time and thought towards a comprehensive strategy for working with static resources for 4.1 in a modern client application. The Sagan project is our real-world example of that and there is a sample for the SpringOne presentation, which I highly recommend checking out for the full context of our thought process there. Also in 4.1 is the Groovy MarkupTemplateEngine support which provides a DRY markup that's missing in Java view templating and in my mind provides a true new alternative next to FreeMarker and Velocity. The sample by the way has handlebars and Groovy templates.

That said I'm also curious to hear more related to the "fully page oriented" comment? And more specifics on what you would like to see, around avoiding duplication, micro templates, etc. Are you using JavaScript templating and server-side templating in the same app or across different apps?

[spring-projects-issues]

marc schipperheyn commented

I had a quick look at the sample and the Sagan source. It looks nice and modular but I had a bit of a hard time understanding the overall architecture and how it integrates with backoffice. Is there any kind of framework in play here (apart from jquery)? It looks that way but it's not entirely clear.

I also encourage you to have a look at http://reactjs.net/ which offers some of the technologies such as server side rendering of jsx that I was talking about.

From what I can see at first glance, the Sagan code is basically about generating a page and then enhancing it with Javascript in a very structured way. What I think the industry is moving towards is: generating UIs through Javascript supported by server side rendering of these UIs as to not have an empty UI with a busy icon while the Javascript-json-template combo is executed after the page skeleton has been rendered.

So, I think Spring needs to answer the question: how do I fit in this new structure? Looking at stuff like reactjs.net, I think technologically there are a number of steps that Spring can deliver to enable Spring developers. I think these are part rendering technologies (handlebars, jsx, etc). Some of this stuff is already available, such as Handlebars Java and easy to integrate. But when you do something like Project Sagan with Handlebars you will find, as I did, that it's not as easy as all that, since simplistic template languages like Handlebars assume that you will use Javascript functions in the actual model and so called "helpers" to deal with the limitations of the template language. Here's where it gets icky and again you are duplicating functionality on the server (Java helpers) and the client (Javascript helpers).

To deal with these challenges I think a certain level of prescriptiveness may be required. Which can't be handled in a generic manner and which might lead to something like Spring-Angular, Spring-Reactjs, which BTW would be GREAT marketing. So, following in this line of thought, Spring has to pick the winners, like it did by supporting Freemarker, JSP etc.

Right now, the major front end players (Facebook, Yahoo, AirBnb among others) are moving to React.js/Flux, simply because everybody recognizes it as a revolution. But it's early days. Angular.js is considered way too complicated, unmaintainable and slow to compete with it. AFAIC Angular is dead. That doesn't mean it's not interesting for Spring. Since only now the enterprise is starting to pick up on it and it will have years of play before it's supplanted by or morphed into something based on Polymer.

As a developer, I want to get started with React.js. I would like to start moving Freemarker macros to JSX templates and have a processing pipeline that makes sense and that allows me to start the migration and have some of the advantage that Node.js offers (server-side rendering, smooth client-server code integration) but in Spring.

[spring-projects-issues]

John Hann commented

Hey Marc ... so much to discuss in this thread ... where to start? :)

I'm one of the resident JavaScript experts on the front-end team within the greater Spring team. We've been spending a considerable amount of effort in this area over the past few years. We have released several JavaScript Architectural-level tools under the cujoJS github organization (homepage: cujojs.com). We have some very exciting new projects in development, too.

Split Rendering

We have dubbed the combination of server-side rendering and client-side rendering, "split rendering". Split rendering looks pretty awesome at first glance, but we've found it less compelling than some other alternatives for many use cases. Here's why:

tl;dr: split rendering doesn't scale with app size and is too complicated for small apps.

1. As you described, the technique is to render the HTML at the server, and let the client rendezvous on special markers in the HTML. While this renders the UI much sooner, the behavior of the app lags. This can cause the UI to be unresponsive for an unacceptable period of time for all but the most trivial apps. (The "busy icon" that you mention only gets deferred.) This is most evident on mobile devices.

2. Sophisticated templates require increased code, memory, and CPU. Delegating sophisticated templating to the client slows down the app at load time and run time. Again, mobile devices suffer the worst. We advocate static HTML snippets, rather than sophisticated templates.

3. REACT.net and other split rendering solutions don't address the real problems: efficient loading of resources and code; efficient, scalable rendering; and efficient communications. Minimizing the size and quantity of HTTP requests is critical.

Let's take the Sagan app as an example. From a front-end perspective, it's very simple. When we were asked to update it to modern practices, we contemplated using split rendering. We thought we might be able to render the HTML on the server and load the behavior async because users would unlikely be able to interact with the page before the JavaScript loaded and bound to the DOM. However, we began to notice that there was already a noticeable lag! If we added the split rendering code, the load time be much larger, which would make it lag even more.

Instead, we devised a simple mechanism to activate only the features available on the current page. This reduced the lag a bit.

However, there was a bigger problem: script tags are slow because they're sync. So we introduced an async module loader. We chose curl.js + cram.js because they would allow us to bundle the entire JavaScript, CSS, and static HTML snippets into a single file. The resulting app loaded about 8x faster using only two HTTP requests (excluding fonts and images). (I do not recall the reasons, but the sagan devs later decided to split out most of the css and to break the JavaScript into two files. Yet, the load time was still up to 5x faster than when using script tags.)

Imho, the lesson to be learned from sagan -- as well as dozens of other, larger apps we've worked on -- is that split rendering is the best solution for very few use cases. That said, I can see it being alluring for folks who want to leverage server-side templates.

To be clear: we're not ignoring split rendering. It's on our list of goals, but it's not our top priority, atm. Read on...

Client-side rendering

I wholeheartedly agree that client-side/JavaScript rendering is the currently accepted best practice for most use cases. Despite their drawbacks, AngularJS and React are the current faves. Angular is fast at rendering, but lacks decent architecture for larger apps. React is bulky and requires a complicated set up. We are working on an alternative that is forward-compatible with WebComponent, but it's on the back burner since the industry seems generally happy with Angular and React, atm. :)

For now, we're concentrating on the connectivity and load-time problems that neither AngularJS or React address at all. The short-term goal is to provide the tools to easily create an ultra-efficient communications layer between an Angular or React front-end and a Spring back end. I encourage you to take a look at the following projects:

• rave - Zero-configuration application bootstrap and development inspired by Spring Boot (in development)
• fabulous - Differential Sync (ultra-efficient communication) (in development)
• jiff - JSON diff and patch based on rfc6902
• most - Monadic streams
• rest - RESTful HTTP client library

These projects have already made immediate and tangible impacts on modern web apps. The next step will be to make them easier to incorporate into Spring-based apps as you suggested. I'd love to see Spring-Angular and Spring-React projects soon.

I haven't vetted this community-led project, yet, but you might find rave-react-starter interesting.

Thoughts?

• John

[spring-projects-issues]

marc schipperheyn commented

Ok, thx for that extensive reply.

Just a little about my background. I developed a platform based on Spring, Hibernate and YUI. I stand on the shoulders of giants. YUI was recently discontinued because Yahoo fell in love with React.js so much, they killed their own framework. So, naturally, I got curious.

Our website has a sophisticated front end with lots and lots of javascript modules. We load HTML first and then "add on" functionality through javascript. YUI has problably the most sophisticated loader in the market today and we use a comboloader. Even so, lag is a problem, especially on mobile.

My biggest issue: I want to develop an HTML 5 mobile app and even though we have a significantly mobile optimized website, found that I have to rewrite whole parts of the application for this, allowing the client to render the app in stead of the server.

My experience with YUI has taught me a few things:

• It's good to have a prescriptive architecture because it helps you be productive and maintainable.
• Only the popular kids on the block will deliver true productivity, since all the advanced ui tools (e.g. autocomplete within a textarea) usually get developed by the community who will only do that if they see traction.
• A great framework has a mobile strategy
• Frameworks that are open sourced by companies that build and maintain those frameworks primarily to support their own web presence, often lack vision and key functionalities to make them truly successful (i.e. YUI, React.js)

YUI became a somewhat popular kid, more or less delivering on advanced ui components primarily through the community, but for great selection of great functionality, it was JQuery that ruled. It also didn't have a solid mobile strategy which accelerated its demise.

Split rendering
Ad 1. This is not exactly what I was suggesting. I suggested that the server should prerender the end page and allow Javascript to "take it from there", e.g. by rendering the results of a "more results" click. I was not suggesting the "render a frame" and the post load the content. Perhaps we are saying the same thing here

Ad 2 Working with static HTML snippets as opposed to using JSON responses, is an approach that I've been using. What I don't like about it is that there is "logic" in the returned content. If I post a comment to a post, and I get a validation exception, I want to process this. If I get a redirect I want to process this. If I have a successful I could have a static HTML snippet containing the resulting post so I can insert this content, but given the desire to have processing logic to deal with exceptions etc, this is far from ideal. Conceivably, a processed HTML result could be a "key" in my json response, however, this is not a "regular flow" in spring right now. It's possible but not standard.

Ad 3 I don't understand this. React.js helps you process content through Json communication with the option of serverside rendering. On top of that, their diff algorithm allows you to minimize slow DOM interactions. What could be more efficient than that? It's certainly is more efficient than doing a page reload or receiving an HTML snippet.

Client side rendering
I think it's very relevant to deliver solid server side solutions that optimize the client interaction. But of course, it doesn't address the core of my expressed concern: having a great development flow from the backend all the way to the frontend.

When you suggest that you guys are working on your own javascript frontend solution, I get a little worried. Primarily because you will try to compete with frameworks that were built by companies (Google, Facebook) that base their frameworks on their experiences running some of the biggest web portals in the world. And also because you have developed awesome serverside solutions but not clientside. I would rather see you guys deliver awesome integration with the existing success stories: React.js and Angular.

Cheers,
Marc

[spring-projects-issues]

Tor Arne Kvaløy commented

As I see it React with it's state-based rendering of components, virtual DOM, and support for server-side rendering, has pretty much killed every other frontend-framework that is currently on the market.

The use case for server-side rendering is providing the initial page with fully rendered HTML, so that the user does not need to wait for any javascripts to run before the page is fully viewable. Without server-side rendering, the browser has first to run the javascript that generates the HTML and this creates a short delay, most notable for mobile users, and secondly is an issue for SEO.

So for server-side rendering of React-components we need to run javascript on the server. We are currently working on two ways of getting to run javascript in our Java/Spring stack:

• One approach is to delegate execution of javascript to Node running outside Tomcat. A JSP tag would do a HTTP request to Node with the javascript code to execute and return the HTML snippet that the tag insert into the page.
• Another approach is to have the javascript run in Nashorn and insert the HTML snippet in the same way.

We will implement these two approaches and measure performance, and analyse the overall complexity in terms of solution and deployment.

Best wishes,
Tor

[spring-projects-issues]

marc schipperheyn commented

Looking forward to seeing this come to fruition. I also think server side rendering of Javascript is inevitable for a relevant solution. Spring offering the fluidity of development that it's famous for all the way to the frontend would certainly have a huge impact.

Especially if this translates to a uniform development experience for both websites and mobile apps (Phonegap style).

[spring-projects-issues]

John Hann commented

Hey Marc,

You rock. Our team had a great conversation at our standup after reviewing this thread. :)

I am not advocating HTML snippets instead of JSON payloads. JSON is the way to go. Rather, I was advocating HTML snippets over sophisticated template engines. We have found that template engines cause much of the lag (not just the loading) when rendering a UI -- especially on mobile. Using plain old HTML is orders of magnitude faster. Handlebars, mustache, etc. -- they are all too slow.

From what we've seen so far, React.js seems to be fast. Perhaps the lag caused by rendering will be less of an issue. Until I see a large app using React and split rendering together, I'm skeptical, though. I am confident you will have to partition the app into several bundles that get loaded as needed instead of at page load time. Does React give any guidance about how to do this?

We don't plan to compete with AngularJS or React. As you know, a project doesn't have to be technically elegant to become popular in the JavaScript world these days, it just has to be attached to a big name. Even though we've got something that's faster, lighter, and more extensible than AngularJS or React, we simply can't compete with Google or Facebook.

React, though -- as well as AngularJS imho -- lacks a decent communication layer. This is where we have a good solution and can work with them, rather than against them. :)

We concluded that the best short term strategy is to create some Getting Started Guides, tutorials, and blog post about using our communications libraries with AngularJS and React.

Again, your thoughts and feedback are very valuable!

• John

[spring-projects-issues]

marc schipperheyn commented

Thanks for the kind thoughts.

Until I see a large app using React and split rendering together, I'm skeptical, though.

Facebook and Instagram use it for everything AFAIK. I also believe that Facebook mobile website uses it, giving it very fluid interactivity.

Does React give any guidance about how to do this?
React.js sells itself as just offering a highly efficient view layer. As such it plugs in to existing MVC front end architectures such as Backbone, Knockout etc. The Flux architecture is meant to complement that by offering a dispatcher infrastructure on top of that. React.js ultimately offers the same "magical" updating of Angular without the performance cost and maintenance headache.

We concluded that the best short term strategy is to create some Getting Started Guides, tutorials, and blog post about using our communications libraries with AngularJS and React.

I agree. The concept of the central datastore offers a very clear interaction point for Spring.

An interesting experiment for you guys might be to create proof of concept with a React.js frontend and a Spring backend, and next to it, the same proof of concept with the same React.js frontend and a Node.js backend, and take both apps to Phonegap, and experience the pros and cons of both. Without having tried it myself, I would imagine the painpoint to become the server side rendering as the Node.js backend would conceivably make it much easier to just re-use the same template code and secondly transference of code to a mobile app.

I'm not suggesting you should compete with Facebook or Google. I personally see you guys rather competing with Node.js. I guess in recent years MVC frameworks have moved to the front end with Backbone, Knockout etc, so, I guess an obvious approach would be find a way to easily mirror the server side MVC to the client MVC, however, I think Flux is showing a direction away from MVC
http://www.infoq.com/news/2014/05/facebook-mvc-flux

When I opened this issue, I was thinking more about server-side rendering but as the discussion evolved, I realized my bigger painpoint is the mobile app. The steps of many existing platforms can probably be seen over time:

1. Create a website: Spring, Freemarker, JQuery
2. Create a mobile enabled version: Spring, Freemarker, JQuery, Mobile JQuery
3. Create a mobile app with Phonegap: Spring, completely rewritten frontend, darn

For the people using something like React.js with Node.js, it would probably start mobile first these days, but assuming this isn't so, it would still be something like

1. Create a website: Node.js, Backbone.js, JSX, React.js
2. Create a mobile enabled version: Node.js, Backbone.js, JSX, React.js
3. Create a mobile app with Phonegap: Node.js, Backbone.js, JSX, React.js

The key take away being that it's probably possible to have a once size fits all code base for all environments.

So, I think it's interesting to explore the way to create a Spring based web environment that transfers easily to these target environments.

Looking forward to those blog posts!

[spring-projects-issues]

marc schipperheyn commented

Something interesting to review perhaps is the work Yahoo is doing in abstracting server-side and client side data fetching and rendering with React

https://github.com/yahoo/flux-examples
https://github.com/yahoo/fetchr

[spring-projects-issues]

marc schipperheyn commented

A Server Side implementation of React.js https://github.com/KnisterPeter/jreact

[spring-projects-issues]

marc schipperheyn commented

It's interesting to note that since React.js uses a virtual DOM, it is the perfect candidate for server side support as the "next Spring template language" in Spring since any framework that would require interaction with the real DOM for operation would be problematic.

[spring-projects-issues]

Sébastien Deleuze commented

marc schipperheyn Good point, I like the idea of template rendering that could be done on client and server side.
It could be useful for SEO, performances (like Twitter did), etc.

[spring-projects-issues]

marc schipperheyn commented

React.js uses a very clear, kinda MVC, structure. Where the model or "Store" is the filter for access to the backend.
From the View's and Controller's point of view, the backend basically doesn't exist. The Store determines whether a data request can be served from the local cache or has to be retrieved from the backend (json => Spring MVC).

At the same time, using the concept of server side Javascript rendering, any page request can be rendered using the identical structure.

So, we have to appreciate the power of this concept. Creating a mobile app, would mean installing the MVC Javascript structure on a phone (using e.g. Phonegap).
This EXACT same mobile app could be installed on a webserver and rendered as a website.

The interaction point with the backend would always be the same (the Store). The only difference would be that a page render on the webserver would require a server side render action which on a phone would not exist.

This is the nirvana that I want to achieve. Right now, I have the website rendered through Freemarker plus heavy Javascript functionality on top of that. There is simply no way to take that beyond a mobile optimized website to a mobile app.

If Spring can deliver on a server side Javascript render functionality, this would become possible.

[spring-projects-issues]

marc schipperheyn commented

I hope you guys are taking notice of the enormous growth of Reactjs. Netflix, AirBnb, BBC, Facebook, to name a few.

Judging from some blog posts you seem to be interested in Angular, which would be an understandable mistake, since they are currently still big in the enterprise space. But a mistake nonetheless. AngularJs is on the way out, ReactJs is it right now. Bet on the winning horse.