HTML rendering of the spec (bikeshed/ReSpec)

[MikeRalphson]

There has been some discussion, on twitter and within the TSC about providing a HTML-rendered version of the specification (in addition to, and not replacing the markdown source-of-truth). This would provide an easier-to-use ToC, referenceable section numbers and a bibliography.

This also builds on the earlier discussion in #829 - cc @apowers313 @adrianhopebailie & @philsturgeon

Two examples have been generated, using Bikeshed and ReSpec.

The following pros/cons/differences are noted, but further feedback is greatly welcomed. Particular areas of interest are accessibility (s.508 and WCAG compliance) - please recommend good tools in this area, browser/mobile compatibility and ToC/section number rendering. Minor formatting tweaks can probably be accommodated, so these would not be a show-stopper, but please do call them out in comments.

I have also added a comment for each option, purely to gauge GitHub reactions. Please feel free to use 👍 👎 ❤️ etc. Please avoid one-line comments simply voting for either option - these may be deleted.

Note	Bikeshed version	ReSpec version
a11y	pending	pending
Size	342k	160k + 394k JS
w/o javascript	Renders fully	Renders without header/ToC
Anchors	Inline	ToC only Inline
Version number	Further down	Prominent
Logo	Further down	Prominent
Copyright	Present	Pending
Former Editors	Supported	Not supported Supported in later versions
Front matter format	Custom	JSON
Conformance	Repeated	Once
GitHub Links	Repo / Issues	Repo / Issues / Commit history
Mobile preview	Bikeshed	ReSpec
Implementation	Python 2	Node.js

[MikeRalphson]

Prefer Bikeshed version.

[MikeRalphson]

Prefer ReSpec version.

[darrelmiller]

So far Bikeshed is my preferred choice. Beyond the points listed above, Bikeshed includes anchors in the text for getting a link to the text and most importantly the tables in Respec look terrible in Edge ;-)

But that "Living Standard" is going to have to go.

[MikeRalphson]

most importantly the tables in Respec look terrible in Edge ;-)

The CSS styling should be easy to duplicate from one to the other. Edit: I've added some more 'github-y' CSS to the ReSpec version.

But that "Living Standard" is going to have to go.

This may be trickier as Bikeshed only seems to allow one from a fixed set of 'document statuses'.

I'm going to come out in favour of ReSpec, as the JSON header would be much easier to work with in Travis and the Python 2 dependency may come back to bite us.

[philsturgeon]

Oh boy, bikeshed is rather protective over statuses. Can we regex the HTML?

They both seem really similar other than style. Although bikeshed initially seemed better (and currently looks to have better style), perhaps we can slap enough paint on ReSpec that it starts to look like a RESPECtful enough option?

[apowers313]

Just FYI, I ended up having to solve most of these problems for FIDO's use of Bikeshed. We started with regex'ing the HTML to replace statuses; then we forked bikeshed and updated our statuses there (which was honestly about 5 minutes worth of work and much easier than regex); finally, it turns out that @tabatkins is a very nice and accessible guy and willing to collaborate and include statuses from other organizations.

As far as rendering issues go, you can use bikeshed boilerplate to update any of the formatting, logos, etc. For FIDO we ended up having a copyright.include, header.include and logo.include. Note that the .include files have to be in the same directory as the .bs file. You can find boilerplate for different groups in the bikeshed source.

One last thing, which you don't seem to have stumbled upon yet is managing references. If you include a biblio.json in the same directory as your .bs file, then you can add references that aren't included in the specref database. (Although, submitting specs to specref is always the preferred route and I've found that the w3c team is pretty easy to collaborate with.)

Hope that helps. If you need help throwing together a proof of concept, let me know.

[MikeRalphson]

Thanks @apowers313 - all good to know!

[tabatkins]

I'm also happy to host your boilerplate files in Bikeshed, if you're going to be using them for several specs. That way you don't have to keep track of them yourself.

If you go that route, just PR me with the additions to the bikeshed/boilerplate folder.

[MikeRalphson]

Thanks @tabatkins - initially it would be just multiple versions of the one OAS spec.

[DavidBiesack]

I prefer the Bikeshed version numbering and depth in the TOC. Can ReSpec version be updated so that the sections are not all subsections of 1., and that subsections are numbered? i.e. 1.3.1 Versions should appear as 3.1 Versions, and in the document, the sections numbers also show up ("3.1 Versions" not just "Versions") uniformly in the body. Why is Definitions expanded in the BikeShed TOC but not in ReSpec TOC? TOC / navigation is an important factor.

[MikeRalphson]

@DavidBiesack - both tools have specific mechanisms for generating ToCs. From memory, bikeshed uses the actual headings (but insists they step in and out by one level each time - I believe this is not always the case in the spec. markdown, where headings have sometimes been chosen for their visual appearance, so they are massaged by a pre-processing step). ReSpec requires the use of <section> tags, so these are also added by the pre-processor. No attempt was made to alter the markdown header level nesting to create a ToC with particular numbering - though I'm sure we can iterate on this.

Note: the Bikeshed ToC seems to stutter within the 'Definitions' (sections 3.1.1 and 3.1.1.1) - I'm not sure why this is.

and in the document, the sections numbers also show up ("3.1 Versions" not just "Versions")

I'm not sure I follow this. Both renderings show the section numbers in both the ToC and in the text.

Why is Definitions expanded in the BikeShed TOC but not in ReSpec TOC?

This just seems to be one of the differences between the tools. Both understand definitions and create links to them, but obviously choose to render them in the ToCs differently - some specs may of course have many more definitions than the OAS, perhaps this is a factor?

[MikeRalphson]

@DavidBiesack I've attempted to re-align the ReSpec ToC somewhat. Please take another look.

@darrelmiller It turns out permalink anchors was an optional feature in ReSpec which is off by default. I've enabled this now and re-rendered.

But that "Living Standard" is going to have to go.

@OAI/tsc - do we have a view on the correct 'document status' description for a published version of the OAS?

[darrelmiller]

@MikeRalphson Anchors are excellent. And the new version of Edge made the table rendering in Respec look much better :-)

[tabatkins]

Note: the Bikeshed ToC seems to stutter within the 'Definitions' (sections 3.1.1 and 3.1.1.1) - I'm not sure why this is.

In the Bikeshed HTML output the "Path Templating" heading is an h4, while the subsequent two headings are both h5, thus they're interpreted as nested subsections, as is appropriate.

(Just afterwards, the "Specification" heading is an h4 as well, rather than an h2 as it should be, which makes for a very weird/bad ToC organization.)

This just seems to be one of the differences between the tools. Both understand definitions and create links to them, but obviously choose to render them in the ToCs differently - some specs may of course have many more definitions than the OAS, perhaps this is a factor?

Bikeshed just renders all of the headings into the ToC. You can exclude individual headings by adding a no-toc class (docs), or automatically exclude all headings below a certain level by adding a Max ToC Depth metadata.

I don't know the details of ReSpec's rendering, but it probably either auto-excludes headings below a certain level, or is excluding them because the source jumps from the "Definitions" h2 straight to h5s for the subsections.

But that "Living Standard" is going to have to go.

This is a consequence of generating the spec with Status: LS in your metadata, and using a header.include file based off of the ones in Bikeshed's repo. There's a [LONGSTATUS] macro in there that gets filled in according to the Status. You can just change that manually if you like, to say whatever you want; Bikeshed doesn't care.

Similarly, in the comparison table one difference you cite is that ReSpec puts the version number up at the top, while Bikeshed doesn't - this is something you can customize just by adding a [LEVEL] macro into your header.include, along with whatever supporting text you want.

And same with the logo, also noted as a difference - it's just markup in your header.include file, you can style it however you want. It looks in the source like the logo boilerplate section was just given an explicit style="display:none", and then a logo <img> manually inserted in right after it. If you instead use the Logo metadata and don't display:none the section, it'll automatically style itself into the upper-right corner, like ReSpec does. ^_^

[apowers313]

Here's an example of what 3.0.1 would look like in Bikeshed:
https://apowers313.github.io/openapi3-bikeshed-test/

And here's the repo with the source:
https://github.com/apowers313/openapi3-bikeshed-test

I started with 3.0.1.md and made a few changes to get it to work with Bikeshed. Note that I did some pretty ham-fisted things just to get Bikeshed to build (like deleting some references and anchors). You can look at the commit history to see what I did.

This also has Travis CI setup on it so that it automatically builds and pushes to GitHub pages whenever a commit is pushed. I pretty much copied those scripts from the W3C with some minor tweaks.

If you guys want to play around with it, I can add you as contributors on the repo so that you can push changes and see how it works.

[tabatkins]

👍 The only big issue is that Bikeshed doesn't support Markdown-ish tables, and I don't particularly plan on doing so; they're not part of CommonMark, and using HTML tables is nearly as lightweight if you take proper advantage of omitting end tags for the table elements.

[DavidBiesack]

@MikeRalphson Thanks for looking into my comments. I think whatever issue is causing the "Definitions" sections to deviate in the ReSpec TOC output may also cause it to not number those sections, even though the rest of the sections are numbered in the body:

[MikeRalphson]

I think whatever issue is causing the "Definitions" sections to deviate in the ReSpec TOC output may also cause it to not number those sections

@DavidBiesack noted. Perhaps @marcoscaceres from ReSpec could clarify?

[MikeRalphson]

Here's an example of what 3.0.1 would look like in Bikeshed:

@apowers313 thanks. But... the Bikeshed version linked to above is already an example of what 3.0.1 would look like in Bikeshed. 😄

I started with 3.0.1.md and made a few changes to get it to work with Bikeshed. Note that I did some pretty ham-fisted things just to get Bikeshed to build (like deleting some references and anchors). You can look at the commit history to see what I did.

Also as above, there is, I believe, no appetite currently to move to Bikeshed .bs format being the source format of the specification. It does not render automatically in GitHub and there is no built-in syntax highlighting in edit mode.

The only big issue is that Bikeshed doesn't support Markdown-ish tables (@tabatkins)

The other issues I spotted:

• Links not being rendered correctly
• Backticks with associated text/markdown not rendering correctly
• Bolding not rendering correctly
• YAML syntax highlighting not working
• JSON syntax highlighting sometimes not being picked up
• Some headers not rendering (may be a depth-setting related issue)

All of the above, plus the table issue are resolved in the version linked to above, via an automated pre-processing step.

This also has Travis CI setup on it so that it automatically builds and pushes to GitHub pages whenever a commit is pushed. I pretty much copied those scripts from the W3C with some minor tweaks.

This, though, is very interesting. Could you add an explicit license to your repo? Apache 2.0 would be perfect, but whatever is appropriate to the W3C source.

Both the Travis setup and the the .include boilerplate would be valuable contributions if we did go down the Bikeshed route. Thanks again.

[apowers313]

@apowers313 thanks. But... the Bikeshed version linked to above is already an example of what 3.0.1 would look like in Bikeshed. 😄

Doh! I should have been paying attention...

Also as above, there is, I believe, no appetite currently to move to Bikeshed .bs format

I've used both Respec and Bikeshed. One of the reasons FIDO switched was that we heard that Respec was being deprecated for Bikeshed (although I don't think I've ever seen an official statement from W3C to that effect). Regardless, I prefer Bikeshed and I would advocate for submitting PRs for Bikeshed for your use cases rather than using Respec. I doubt that will keep you from going down the Respec path, but I would feel guilty if I let you suffer without warning you first. :)

This, though, is very interesting. Could you add an explicit license to your repo? Apache 2.0 would be perfect, but whatever is appropriate to the W3C source.

I'm not sure the license is mine to give. Most of the code was borrowed from the W3C. The Travis CI code is here and lots of examples of the Bikeshed boilerplate are here.

Note that I've created a rather sophisticated set of Makfiles for publishing Respec as well that kicks out files like this:
https://fidoalliance.org/specs/fido-uaf-v1.2-rd-20171128/

It includes PDF, a complete PDF of all specs, updating references, renaming files and links, etc. It wasn't fun to create, but I might be able to convince FIDO to share their code if it's of any use to you.

[MikeRalphson]

After much wailing and gnashing of teeth, I believe I have got both tools to output equivalent ToCs. Examples re-rendered. @DavidBiesack does this address your issue re: Definitions?

I'm not sure the license is mine to give. Most of the code was borrowed from the W3C

Not being a lawyer, I can't say whether the W3C Document License is compatible with Apache 2.0 - it looks safer to re-implement the deployment scripts (which shouldn't be difficult).

@tabatkins I'm still using the cgi version of Bikeshed, so I don't know if I can easily feed that the .include boilerplate to fix the logo etc. If we go with Bikeshed I'll fix that up.

[DavidBiesack]

yes, ReSpec "Definitions" section looks good now.

[tedepstein]

I was wondering how these two formats differ in their ability to override default CSS styles, and whether one format or the other is easier to work with, w.r.t. custom styling.

I see a comment about from @philsturgeon about ReSpec styling, and some comments from @MikeRalphson describing some success with overriding default styles.

I also took a quick look at the Bikeshed and ReSpec docs. ReSpec has a general guideline about this:

Extra Styling

If you wish to add your own additional styles to your document, just use the regular link and style elements. Be warned however that the W3C styles will always be added after yours, so if you wish to override them you will need to use more specific selectors.

@MikeRalphson , having done this, was there significant extra effort required to "use more specific selectors," as prescribed? Is it correct to assume that this requirement would make it harder to reuse existing stylesheets that aren't ReSpec-aware?

Bikeshed docs don't seem to have a definitive "Here's how to override the default styles" section. Is it as simple as plugging in your own stylesheet?

[tabatkins]

• Links not being rendered correctly
• Backticks with associated text/markdown not rendering correctly
• Bolding not rendering correctly

These are all because Bikeshed does not, by default, support inline Markdown. You have to set Markup Shorthands: markdown on in your metadata, then all of these work. (I submitted a PR in apowers313/openapi3-bikeshed-test#1 for this.)

• YAML syntax highlighting not working
• JSON syntax highlighting sometimes not being picked up

Both of these should work just fine if you tag them with json or yaml, per the normal block-syntax-highlighting rules (or if you're using HTML <pre> in the source, add a highlight=json or highlight=yaml attribute). There's no reasonable way to auto-detect highlighting language. (If your document generally uses the same language everywhere, you can set a default highlighting language with Default Highlight: json in your metadata (and you can still override that on individual code blocks with the aforementioned mechanisms).

• Some headers not rendering (may be a depth-setting related issue)

Ah, yup, because the document's own title claims the h1 spot, that only leaves h2-h6 actually available for the rest of the headings in your source. (And it interprets the markdown top-level headings (=== underline, or single #) as being an h2 in the document, and adjust the rest accordingly.) Thus, using sixth-level markdown headings ###### have nothing to render as in HTML.

(I should detect this and flag an error, thanks for running into this.)

I've used both Respec and Bikeshed. One of the reasons FIDO switched was that we heard that Respec was being deprecated for Bikeshed (although I don't think I've ever seen an official statement from W3C to that effect).

It's not being deprecated, but Bikeshed is overall more popular and mostly used for new documents, and a lot of older documents are converting from ReSpec to Bikeshed. (There are some fundamental weaknesses of a live-rendered JS processor like ReSpec that can't be reasonably worked around, which don't bother an offline processor like Bikeshed - in particular, Bikeshed can maintain large databases for things like autolinking to other specifications.)

I was wondering how these two formats differ in their ability to override default CSS styles, and whether one format or the other is easier to work with, w.r.t. custom styling.

I think they're equivalent - you can just put <link> and <style> into your document to do whatever you want. Like ReSpec says, if you're preparing a document for /TR publication, Bikeshed will ensure that the W3C styles show up last in the cascade, as that's required by publication rules. The W3C stylesheets are generally easy to override if necessary, tho: https://www.w3.org/StyleSheets/TR/2016/base.css

I'm still using the cgi version of Bikeshed, so I don't know if I can easily feed that the .include boilerplate to fix the logo etc. If we go with Bikeshed I'll fix that up.

Ah, no, extra files can't be used with the cgi version. You'd have to commit your boilerplate to Bikeshed's repository so it's available to the CGI.

[marcoscaceres]

Just noting that ReSpec supports both "Former Editors" and per section markdown and globally. However, markdown itself is quite limited - so it's not really recommended for complex documents (not because ReSpec can't handle it, it's just not a good format for writing technical documents).

Also, just to put rumors to rest, "Respec being deprecated for Bikeshed" is not the case. Both are in active development and serve different communities and purposes.

If you have many JS devs in your community, then they might appreciate the customizability and hackability of ReSpec... similarly, if you have folks that know Python (and are comfortable with CLI interfaces), then BikeShed is a really good option.

Because BikeShed has a lot of really powerful capabilities, it's my personal experience that BikeShed has a higher learning curve than ReSpec... This can put off contributions from folks who don't want to learn new syntax, read a lot of documentation, or know how to change their favorite editor to treat .bs as HTML, etc. However, if you have editors willing to learn BikeShed, and you are not expecting many random external contributions, then BikeShed can work really well.

It really comes down to the community around the spec. Hope that helps!

[namdeirf]

Action Items

• registration work - @darrelmiller
• deployment @MikeRalphson
• CNAME - @namdeirf

[uxweb]

Some small style adjustments to the body element of the docs will make them look nicer and more readable:

body {
    line-height: 2rem;
    font-weight: 300;
    color: #30353a;
}

And using other color for the code elements makes it look more bright to the eyes:

code {
    color: #d14;
}

[tedepstein]

Now that we've chosen Respec, do we have a designated site/URL for this, and a publishing process to maintain a current version? I see three to-do items here, looking like they're still pending completion, but I'm not sure if that's the current status.

Also, I'm interested in this for two reasons:

1. All the benefits of a published spec: TOC navigation, better presentation, etc.
2. Having a single URL that always points to the current version of the spec.

That second point is something I can break out into its own issue if we don't intend to address it as part of the Respec publishing process. Ideally, we'd have virtual symlinks or aliases arranged like this:

URL Type	URL	Description
Specific Versions	.../spec/v1.2	Specification for version 1.2
	.../spec/v2.0	Specification for version 2.0
	.../spec/v3.0.0	Specification for version 3.0.0
	.../spec/v3.0.1	Specification for version 3.0.1
	.../spec/v3.0.2	Specification for version 3.0.2
	.../spec/v3.1.0	Specification for version 3.1.0
	.../spec/v3.1.1	Specification for version 3.1.1
Minor Version Symlinks	.../spec/v3.0	Latest 3.0 patch --> .../spec/v3.0.2
	.../spec/v3.1	Latest 3.1 patch --> .../spec/v3.1.1
Major Version Symlinks	.../spec/v1	Latest v1 minor --> .../spec/v1.2
	.../spec/v2	Latest v2 minor --> .../spec/v2.0
	.../spec/v3	Latest v3 minor --> .../spec/v3.1.1
Latest Version Symlink	.../spec	Current spec --> .../spec/v3.1.1

Maybe we'd choose to publish only a subset of these. Currently lack of this one is causing a bit of pain:

URL Type	URL	Description
Minor Version Symlinks	.../spec/v3.0	Latest 3.0 patch --> .../spec/v3.0.2

We're often referring to the v3 specification in emails, articles, etc. My intent is to direct the reader to the latest patch release, whatever that is. But if I use a direct link, it will go stale as soon as a new patch release of the spec is published.

I get around this now with a branded link shortener from GoDaddy: http://rzen.io/OAS3Spec

I can edit that redirect whenever there's a new patch of the 3.0 spec. It works OK, but it would be nice if we all had an easy way to do this. Also nice if updating these symlinks were a routine part of the specification release process.

Maybe publishing with Respec is a good opportunity to address this. But if it's out of scope, I'll open a separate issue.

[MikeRalphson]

💯 I just need to work out how to do symlinks or invisible redirects on github pages. Publishing will be via Travis CI on changes to the master branch. It's all ready to go, and links like https://spec.openapis.org/oas/v3.0.1.html are working now. 3.0.2 isn't up yet though.

[tedepstein]

Awesome. Thanks, @MikeRalphson !

[cebe]

The current rendering has the same issue as we had in #1726 with github.

https://spec.openapis.org/oas/v3.0.1.html#parameterObject Does not link to the headline in the document. It seems to navigate to the entry in the left navigation though.

[handrews]

Looks like this all got finished including the parameterObject link issue in the last comment here.