Natively convert values to multi-line json documents

[guoshimin]

When a value is coerced to a string, it is converted to a single-line JSON document in C++. If you want a multi-line JSON document, you can use the recently-added stdlib function manifestJson. However, in my case, manifestJson has a 5x performance penalty compared to the native conversion. It would be nice to provide a way to convert values to multi-line JSON documents natively.

[sparkprime]

How much does the 5x performance penalty affect you in real terms (i.e. the total run time goes from x seconds to y seconds)?

[guoshimin]

4 seconds vs 20 seconds.
On May 19, 2016 5:32 AM, "Dave Cunningham" notifications@github.com wrote:

How much does the 5x performance penalty affect you in real terms (i.e.
the total run time goes from x seconds to y seconds)?

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#187 (comment)

[sparkprime]

Ok that's significant enough to do something. I wonder if there is a more general fix that would improve the execution performance without adding more native functionality. E.g. I think std.join() can be made faster.

How would you characterize the JSON in this case? Is it long arrays, long strings, big objects, or just very deep? Even better is if you can give a realistic sample that takes that long.

[guoshimin]

Actually, the performance numbers cited above was using a version of
jsonnet between 0.8.6 and 0.8.7. Using 0.8.7 and master, the numbers were
2.x seconds vs 7.x seconds. Great job on the performance improvement!

In my jsonnet document, I call manifestJson ten times on objects of various
sizes. A typical object has around 100 elements at all levels. Some are
pretty flat, with at most 2 levels, and some have as many as 5 levels.
There are no exceptionally long strings or arrays.

On Thu, May 19, 2016 at 7:13 AM, Dave Cunningham notifications@github.com
wrote:

Ok that's significant enough to do something. I wonder if there is a more
general fix that would improve the execution performance without adding
more native functionality. E.g. I think std.join() can be made faster.

How would you characterize the JSON in this case? Is it long arrays, long
strings, big objects, or just very deep? Even better is if you can give a
realistic sample that takes that long.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#187 (comment)

[guoshimin]

Oh actually the performance improvement was due to compilation mode: When
building with -O3 (the default when using make), it's 2.x seconds vs 7.x
seconds, and when building without -O3 (the default when using bazel), it's
4s vs 20s. Sorry for the confusion!

On Thu, May 19, 2016 at 2:01 PM, Shimin Guo shimin@databricks.com wrote:

Actually, the performance numbers cited above was using a version of
jsonnet between 0.8.6 and 0.8.7. Using 0.8.7 and master, the numbers were
2.x seconds vs 7.x seconds. Great job on the performance improvement!

In my jsonnet document, I call manifestJson ten times on objects of
various sizes. A typical object has around 100 elements at all levels. Some
are pretty flat, with at most 2 levels, and some have as many as 5 levels.
There are no exceptionally long strings or arrays.

On Thu, May 19, 2016 at 7:13 AM, Dave Cunningham <notifications@github.com

wrote:

Ok that's significant enough to do something. I wonder if there is a more
general fix that would improve the execution performance without adding
more native functionality. E.g. I think std.join() can be made faster.

How would you characterize the JSON in this case? Is it long arrays, long
strings, big objects, or just very deep? Even better is if you can give a
realistic sample that takes that long.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#187 (comment)

[sparkprime]

Ah thanks for figuring it out. The only performance improvement between those two releases was related to std.format() and the % string formatting operator which didn't sound like it would match your case. Are you ok with 7 seconds for now? I am currently investing all my energy in updating documentation which has fallen far behind.

[guoshimin]

Yeah it's fine for now.
On May 21, 2016 4:32 AM, "Dave Cunningham" notifications@github.com wrote:

Ah thanks for figuring it out. The only performance improvement between
those two releases was related to std.format() and the % string formatting
operator which didn't sound like it would match your case. Are you ok with
7 seconds for now? I am currently investing all my energy in updating
documentation which has fallen far behind.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#187 (comment)

[sparkprime]

Feel free to re-open if you're spending an unreasonable amount of time waiting for Jsonnet :)