http2: introducing HTTP/2

At long last: The initial *experimental* implementation of HTTP/2.

This is an accumulation of the work that has been done in the nodejs/http2
repository, squashed down to a couple of commits. The original commit
history has been preserved in the nodejs/http2 repository.

This PR introduces the nghttp2 C library as a new dependency. This library
provides the majority of the HTTP/2 protocol implementation, with the rest
of the code here providing the mapping of the library into a usable JS API.

Within src, a handful of new node_http2_*.c and node_http2_*.h files are
introduced. These provide the internal mechanisms that interface with nghttp
and define the `process.binding('http2')` interface.

The JS API is defined within `internal/http2/*.js`.

There are two APIs provided: Core and Compat.

The Core API is HTTP/2 specific and is designed to be as minimal and as
efficient as possible.

The Compat API is intended to be as close to the existing HTTP/1 API as
possible, with some exceptions.

Tests, documentation and initial benchmarks are included.

The `http2` module is gated by a new `--expose-http2` command line flag.
When used, `require('http2')` will be exposed to users. Note that there
is an existing `http2` module on npm that would be impacted by the introduction
of this module, which is the main reason for gating this behind a flag.

When using `require('http2')` the first time, a process warning will be
emitted indicating that an experimental feature is being used.

To run the benchmarks, the `h2load` tool (part of the nghttp project) is
required: `./node benchmarks/http2/simple.js benchmarker=h2load`. Only
two benchmarks are currently available.

Additional configuration options to enable verbose debugging are provided:

```
$ ./configure --debug-http2 --debug-nghttp2
$ NODE_DEBUG=http2 ./node
```

The `--debug-http2` configuration option enables verbose debug statements
from the `src/node_http2_*` files. The `--debug-nghttp2` enables the nghttp
library's own verbose debug output. The `NODE_DEBUG=http2` enables JS-level
debug output.

The following illustrates as simple HTTP/2 server and client interaction:

(The HTTP/2 client and server support both plain text and TLS connections)

```jt client = http2.connect('http://localhost:80');
const req = client.request({ ':path': '/some/path' });
req.on('data', (chunk) => { /* do something with the data */ });
req.on('end', () => {
  client.destroy();
});

// Plain text (non-TLS server)
const server = http2.createServer();
server.on('stream', (stream, requestHeaders) => {
  stream.respond({ ':status': 200 });
  stream.write('hello ');
  stream.end('world');
});
server.listen(80);
```

```js
const http2 = require('http2');
const client = http2.connect('http://localhost');

```

Author: Anna Henningsen <[email protected]>
Author: Colin Ihrig <[email protected]>
Author: Daniel Bevenius <[email protected]>
Author: James M Snell <[email protected]>
Author: Jun Mukai
Author: Kelvin Jin
Author: Matteo Collina <[email protected]>
Author: Robert Kowalski <[email protected]>
Author: Santiago Gimeno <[email protected]>
Author: Sebastiaan Deckers <[email protected]>
Author: Yosuke Furukawa <[email protected]>

Backport-PR-URL: #14813
Backport-Reviewed-By: Anna Henningsen <[email protected]>
Backport-Reviewed-By: Timothy Gu <[email protected]>

PR-URL: #14239
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>

Author: jasnell

configure

@@ -64,6 +64,8 @@ shared_optgroup = optparse.OptionGroup(parser, "Shared libraries",
 intl_optgroup = optparse.OptionGroup(parser, "Internationalization",
     "Flags that lets you enable i18n features in Node.js as well as which "
     "library you want to build against.")
+http2_optgroup = optparse.OptionGroup(parser, "HTTP2",
+    "Flags that allows you to control HTTP2 features in Node.js")
 
 # Options should be in alphabetical order but keep --prefix at the top,
 # that's arguably the one people will be looking for most.
@@ -397,6 +399,16 @@ intl_optgroup.add_option('--download-path',
 
 parser.add_option_group(intl_optgroup)
 
+http2_optgroup.add_option('--debug-http2',
+    action='store_true',
+    dest='debug_http2',
+    help='build with http2 debug statements on (default is false)')
+
+http2_optgroup.add_option('--debug-nghttp2',
+    action='store_true',
+    dest='debug_nghttp2',
+    help='build nghttp2 with DEBUGBUILD (default is false)')
+
 parser.add_option('--with-perfctr',
     action='store_true',
     dest='with_perfctr',
@@ -898,6 +910,16 @@ def configure_node(o):
   if options.enable_static:
     o['variables']['node_target_type'] = 'static_library'
 
+  if options.debug_http2:
+    o['variables']['debug_http2'] = 1
+  else:
+    o['variables']['debug_http2'] = 'false'
+
+  if options.debug_nghttp2:
+    o['variables']['debug_nghttp2'] = 1
+  else:
+    o['variables']['debug_nghttp2'] = 'false'
+
   o['variables']['node_no_browser_globals'] = b(options.no_browser_globals)
   o['variables']['node_shared'] = b(options.shared)
   node_module_version = getmoduleversion.get_version()

doc/api/_toc.md

@@ -24,6 +24,7 @@
 * [File System](fs.html)
 * [Globals](globals.html)
 * [HTTP](http.html)
+* [HTTP/2](http2.html)
 * [HTTPS](https.html)
 * [Inspector](inspector.html)
 * [Internationalization](intl.html)

doc/api/cli.md

@@ -170,6 +170,13 @@ added: v6.0.0
 
 Silence all process warnings (including deprecations).
 
+### `--expose-http2`
+<!-- YAML
+added: REPLACEME
+-->
+
+Enable the experimental `'http2'` module.
+
 ### `--napi-modules`
 <!-- YAML
 added: v8.0.0

doc/api/errors.md

@@ -587,6 +587,212 @@ with `for...of` loops) is required, but not provided to a Node.js API.
 Used by the `util.callbackify()` API when a callbackified `Promise` is rejected
 with a falsy value (e.g. `null`).
 
+<a id="ERR_HTTP_HEADERS_SENT"></a>
+### ERR_HTTP_HEADERS_SENT
+
+Used when headers have already been sent and another attempt is made to add
+more headers.
+
+<a id="ERR_HTTP_INVALID_STATUS_CODE"></a>
+### ERR_HTTP_INVALID_STATUS_CODE
+
+Used for status codes outside the regular status code ranges (100-999).
+
+<a id="ERR_HTTP_TRAILER_INVALID"></a>
+### ERR_HTTP_TRAILER_INVALID
+
+Used when the `Trailer` header is set even though the transfer encoding does not
+support that.
+
+<a id="ERR_HTTP2_CONNECT_AUTHORITY"></a>
+### ERR_HTTP2_CONNECT_AUTHORITY
+
+For HTTP/2 requests using the `CONNECT` method, the `:authority` pseudo-header
+is required.
+
+<a id="ERR_HTTP2_CONNECT_PATH"></a>
+### ERR_HTTP2_CONNECT_PATH
+
+For HTTP/2 requests using the `CONNECT` method, the `:path` pseudo-header is
+forbidden.
+
+<a id="ERR_HTTP2_CONNECT_SCHEME"></a>
+### ERR_HTTP2_CONNECT_SCHEME
+
+The HTTP/2 requests using the `CONNECT` method, the `:scheme` pseudo-header is
+forbidden.
+
+<a id="ERR_HTTP2_ERROR"></a>
+### ERR_HTTP2_ERROR
+
+A non-specific HTTP/2 error has occurred.
+
+<a id="ERR_HTTP2_FRAME_ERROR"></a>
+### ERR_HTTP2_FRAME_ERROR
+
+Used when a failure occurs sending an individual frame on the HTTP/2
+session.
+
+<a id="ERR_HTTP2_HEADERS_OBJECT"></a>
+### ERR_HTTP2_HEADERS_OBJECT
+
+Used when an HTTP/2 Headers Object is expected.
+
+<a id="ERR_HTTP2_HEADERS_SENT"></a>
+### ERR_HTTP2_HEADERS_SENT
+
+Used when an attempt is made to send multiple response headers.
+
+<a id="ERR_HTTP2_HEADER_SINGLE_VALUE"></a>
+### ERR_HTTP2_HEADER_SINGLE_VALUE
+
+Used when multiple values have been provided for an HTTP header field that
+required to have only a single value.
+
+<a id="ERR_HTTP2_INFO_HEADERS_AFTER_RESPOND"></a>
+### ERR_HTTP2_INFO_HEADERS_AFTER_RESPOND
+
+HTTP/2 Informational headers must only be sent *prior* to calling the
+`Http2Stream.prototype.respond()` method.
+
+<a id="ERR_HTTP2_INFO_STATUS_NOT_ALLOWED"></a>
+### ERR_HTTP2_INFO_STATUS_NOT_ALLOWED
+
+Informational HTTP status codes (`1xx`) may not be set as the response status
+code on HTTP/2 responses.
+
+<a id="ERR_HTTP2_INVALID_CONNECTION_HEADERS"></a>
+### ERR_HTTP2_INVALID_CONNECTION_HEADERS
+
+HTTP/1 connection specific headers are forbidden to be used in HTTP/2
+requests and responses.
+
+<a id="ERR_HTTP2_INVALID_HEADER_VALUE"></a>
+### ERR_HTTP2_INVALID_HEADER_VALUE
+
+Used to indicate that an invalid HTTP/2 header value has been specified.
+
+<a id="ERR_HTTP2_INVALID_INFO_STATUS"></a>
+### ERR_HTTP2_INVALID_INFO_STATUS
+
+An invalid HTTP informational status code has been specified. Informational
+status codes must be an integer between `100` and `199` (inclusive).
+
+<a id="ERR_HTTP2_INVALID_PACKED_SETTINGS_LENGTH"></a>
+
+Input `Buffer` and `Uint8Array` instances passed to the
+`http2.getUnpackedSettings()` API must have a length that is a multiple of
+six.
+
+<a id="ERR_HTTP2_INVALID_PSEUDOHEADER"></a>
+### ERR_HTTP2_INVALID_PSEUDOHEADER
+
+Only valid HTTP/2 pseudoheaders (`:status`, `:path`, `:authority`, `:scheme`,
+and `:method`) may be used.
+
+<a id="ERR_HTTP2_INVALID_SESSION"></a>
+### ERR_HTTP2_INVALID_SESSION
+
+Used when any action is performed on an `Http2Session` object that has already
+been destroyed.
+
+<a id="ERR_HTTP2_INVALID_SETTING_VALUE"></a>
+### ERR_HTTP2_INVALID_SETTING_VALUE
+
+An invalid value has been specified for an HTTP/2 setting.
+
+<a id="ERR_HTTP2_INVALID_STREAM"></a>
+### ERR_HTTP2_INVALID_STREAM
+
+Used when an operation has been performed on a stream that has already been
+destroyed.
+
+<a id="ERR_HTTP2_MAX_PENDING_SETTINGS_ACK"></a>
+### ERR_HTTP2_MAX_PENDING_SETTINGS_ACK
+
+Whenever an HTTP/2 `SETTINGS` frame is sent to a connected peer, the peer is
+required to send an acknowledgement that it has received and applied the new
+SETTINGS. By default, a maximum number of un-acknowledged `SETTINGS` frame may
+be sent at any given time. This error code is used when that limit has been
+reached.
+
+<a id="ERR_HTTP2_OUT_OF_STREAMS"></a>
+### ERR_HTTP2_OUT_OF_STREAMS
+
+Used when the maximum number of streams on a single HTTP/2 session have been
+created.
+
+<a id="ERR_HTTP2_PAYLOAD_FORBIDDEN"></a>
+### ERR_HTTP2_PAYLOAD_FORBIDDEN
+
+Used when a message payload is specified for an HTTP response code for which
+a payload is forbidden.
+
+<a id="ERR_HTTP2_PSEUDOHEADER_NOT_ALLOWED"></a>
+### ERR_HTTP2_PSEUDOHEADER_NOT_ALLOWED
+
+Used to indicate that an HTTP/2 pseudo-header has been used inappropriately.
+Pseudo-headers are header key names that begin with the `:` prefix.
+
+<a id="ERR_HTTP2_PUSH_DISABLED"></a>
+### ERR_HTTP2_PUSH_DISABLED
+
+Used when push streams have been disabled by the client but an attempt to
+create a push stream is made.
+
+<a id="ERR_HTTP2_SEND_FILE"></a>
+### ERR_HTTP2_SEND_FILE
+
+Used when an attempt is made to use the
+`Http2Stream.prototype.responseWithFile()` API to send a non-regular file.
+
+<a id="ERR_HTTP2_SOCKET_BOUND"></a>
+### ERR_HTTP2_SOCKET_BOUND
+
+Used when an attempt is made to connect a `Http2Session` object to a
+`net.Socket` or `tls.TLSSocket` that has already been bound to another
+`Http2Session` object.
+
+<a id="ERR_HTTP2_STATUS_101"></a>
+### ERR_HTTP2_STATUS_101
+
+Use of the `101` Informational status code is forbidden in HTTP/2.
+
+<a id="ERR_HTTP2_STATUS_INVALID"></a>
+### ERR_HTTP2_STATUS_INVALID
+
+An invalid HTTP status code has been specified. Status codes must be an integer
+between `100` and `599` (inclusive).
+
+<a id="ERR_HTTP2_STREAM_CLOSED"></a>
+### ERR_HTTP2_STREAM_CLOSED
+
+Used when an action has been performed on an HTTP/2 Stream that has already
+been closed.
+
+<a id="ERR_HTTP2_STREAM_ERROR"></a>
+### ERR_HTTP2_STREAM_ERROR
+
+Used when a non-zero error code has been specified in an RST_STREAM frame.
+
+<a id="ERR_HTTP2_STREAM_SELF_DEPENDENCY"></a>
+### ERR_HTTP2_STREAM_SELF_DEPENDENCY
+
+When setting the priority for an HTTP/2 stream, the stream may be marked as
+a dependency for a parent stream. This error code is used when an attempt is
+made to mark a stream and dependent of itself.
+
+<a id="ERR_HTTP2_UNSUPPORTED_PROTOCOL"></a>
+### ERR_HTTP2_UNSUPPORTED_PROTOCOL
+
+Used when `http2.connect()` is passed a URL that uses any protocol other than
+`http:` or `https:`.
+
+<a id="ERR_INDEX_OUT_OF_RANGE"></a>
+### ERR_INDEX_OUT_OF_RANGE
+
+Used when a given index is out of the accepted range (e.g. negative offsets).
+
 <a id="ERR_INVALID_ARG_TYPE"></a>
 ### ERR_INVALID_ARG_TYPE