WASD VMS Web Services - Features and Facilities

5 - HTTP/2

5.1 - WASD HTTP/2 5.2 - HTTP/2 and Performance 5.3 - HTTP/2 Configuration 5.3.1 - Global Configuration 5.3.2 - Service Configuration 5.3.3 - Mapping Set Rules 5.4 - HTTP/2 Detection 5.5 - HTTP/2 References
next previous contents full-page

HTTP/2 is the most recent standard (RFC 7540, 2015) for implementing how HTTP is represented by, and transported between, client and server. It is not a ground-up rewrite of the established standard, HTTP/1.1 (RFC 2616, 1999). Those elements and semantics remain substantially the same. Instead HTTP/2 modifies how the data is encapsulated (framed) and transferred between agents, abstracting the complexity of this within the new protocol layer, leaving the application level largely insulated from change. As a result all existing HTTP/1.1 web-based environments should be able to continue without modification.

The focus of the protocol is on performance, in particular end-user perceived page rendering and web application responsiveness. With the original web use case being a relatively simple, single resource request-response, and early markup involving text with a few illustrative images, the single network connection, back-to-back request-response paradigm was simple to implement and worked well enough. In short time this moved to multiple network connections, each loading elements in parallel as the complexity and density of the individual elements on the pages increased, and to the introduction of HTTP/1.1 pipelining (back-to-back requests over a single connection) in an attempt to avoid request-response-request latency. Modern web documents and applications tend to have dozens of fine-grained elements that dynamically load resources based on the content of the page and/or user interaction. The single, then multiple network connections, each with its round-trip TCP connection establishment overhead and request-response blocking of resources, did not scale effectively. HTTP/2 replaces it with a single TCP connection on which multiple resources concurrently can be requested, pushed, and transferred. A more rigorous and effective implementation of the pipeline concept.

While multiplexing communication over a single network connection is a core performance technology there are other contributing elements. The framing layer uses binary tokens and parameters. The plain-text request and response headers of HTTP/1.n are replaced with tokenised, encoded and dynamically cached equivalents, commonly providing compression in excess of eighty percent. The relationship and priority of resources can be established allowing inferior resources to be delivered after or dependent on superior ones. The HTTP/2 server can send multiple responses to a single request. Known as server push it can be used to pre-load the browser (cache) with resources it has not encountered yet.

HTTP/2 has the potential to place additional load on the client and server in comparison to HTTP/1.n. One particular consideration for WASD sites is the stream concurrency setting of the HTTP/2 connection. The server specifies to the client the maximum number of concurent request-response (and server push) streams it will accept. RFC 7540 contains, "This limit is directional: it applies to the number of streams that the sender permits the receiver to create. Initially, there is no limit to this value. It is recommended that this value be no smaller than 100, so as to not unnecessarily limit parallelism." This translates to a hypothetical ten browsers connected to the site each with up to one hundred concurrent streams, or potentially one thousand active requests! Time to check those server configuration and SYSGEN parameters...

Note that HTTP/1.1 has recently been revisited with RFC 7230 family of specifications (2014) providing some clarifications and refinements on the original.

5.1 - WASD HTTP/2

WASD HTTP/2 implements all of the essential requirements of RFC 7540 (naturally enough). This includes the framing protocol, datagram (message) and stream management, header compression (RFC 7541), connection settings and flow control, along with HTTP/2 connection establishment and termination (TLS ALPN and HTTP upgrade). It does not ((perhaps) currently) provide server-push or stream prioritisation and dependency.

Prior to the introduction of HTTP/2, WASD's fundamental abstraction was the request, with each request interfacing directly with the network stack. With an HTTP/2 protocol connection somewhat supplanting the role of a Transmission Control Protocol (TCP) connection in HTTP/1.n, a new level of communication abstraction was required between the request processing and the network processing. It should be noted that HTTP/2 itself is transported on TCP.

Another new layer of abstraction required interfacing each protocol's request/response header formats with the underlying server processing (avoiding excessive duplication of code). HTTP/1.n has a plain-text, carriage-control separated format, while HTTP/2 has a binary, compressed, lookup-table oriented format (RFC 7541). The layer was implemented using a key-value dictionary.

The accomodations for handling both HTTP/2 and HTTP/1.1, along with related and ancilliary design and code changes, have not measurably impacted overall WASD performance, although as noted below there is a server process CPU impost associated with HTTP/2.


WATCH reports have a new network item: [x]HTTP/2. This provides a detailed overview of the underlying framing and connection management exchanges between client and server. WATCH reports are available to HTTP/2 connected clients with one consideration. Due to multiplexed requests over the single network connection, WATCHing the [x]HTTP/2 item of another request in the same browser (using the same HTTP/2 connection - and there can be multiple from a single browser) is not possible (or at least more code than it's worth). The HTTP/2 activity of the WATCHing generates more report items which generate ... a descent into reporting oblivion.

WASD detects when a request is initiated on the same HTTP/2 connection as an [x]HTTP/2 WATCHing client and advises

|22:00:55.22 WATCH    1823 0004 CONNECT    HTTP/2 with,62446 on https://klaatu.private,443 (|
|22:00:55.22 WATCH    1454 0004 CONNECT    HTTP/2 rabbit hole|
Such a request is not reported on further.


5.2 - HTTP/2 and Performance

With HTTP/2 not modifying the fundamentals of HTTP/1.1 semantics the commonly touted payoff for all the additional complexity (in implementation) is performance. While this is often stated in terms of page rendering speeds or web application responsiveness there is another significant measure of performance - efficiency. HTTP/2 much more efficiently utilises each network (TCP) connection, as well as reducing the (time and processing) overhead of setting-up and tearing-down of each of these required for parallelism under HTTP/1.1.

Is it all worth it?  As might be expected - that depends.

There are a number of sufficiently good analyses of both the factors that affect HTTP/2 performance and the actual performance relative to HTTP/1.1. See the references section and search the Web. This section contains some observations made during WASD HTTP/2 development. All of these seem to correspond with others' observations, as well as what might reasonably be expected considering the strategies employed by the protocol.

Performance Assessment

The simplest tool for getting a feel for, and elementary measurement of HTTP/2 may be found in the WASD_ROOT:[EXERCISE] directory. The document DOTTY.HTML and its companion files provide a page that loads a selectable number of resources (images) in a consistent and reproducible manner. This DOTTY.HTML can be accessed via unencrypted HTTP (http://), encrypted HTTP (https://) and services configured to provide HTTP/2 or HTTP/1.1. Using these combinations with the selectable volume of resources, elementary comparisons may be made in target environments.

The Server Admin, HTTP Report (9 - Server Administration) contains comparative duration and bytes-per-second minimum/maximum/average for total server HTTP/2 and HTTP/1.n requests. These cannot simply be taken at face value without some consideration of the respective load profile but under controlled conditions can provide useful metrics.

Other development and load/performance tools were employed from a Linux platform. For someone educated in computing during the (19)70s, the availability of VM technology for such purposes is just brilliant!   "But you know, we were happy in those days, though we were poor."

Indispensible were:

Many thanks to the developer(s) of this package.

5.3 - HTTP/2 Configuration

While effectively transparent to the end-user, HTTP/2 has some aspects that need to be carefully considered by the server administrator.

5.3.1 - Global Configuration

HTTP/2 and its features are globally enabled and configured using directives contained in the WASD_CONFIG_GLOBAL configuration file.

HTTP/2 Global Configuration
[Http2Protocol]enabled or disabled on a whole-of-server basisdisabled
[Http2FrameSizeMax]maximum frame size in octets (bytes) the server is prepared to receive16384
[Http2HeaderListMax]maximum number of octets (bytes) permitted in a received header once uncompressed65535
[Http2HeaderTableMax]maximum number of bytes permitted in the server-end header cache4096
[Http2PingSeconds]number of seconds between connection RTT pings300
[Http2StreamsMax]maximum number of concurrent streams (requests) the server permits on the connection32
[Http2InitWindowSize]initial window size (number of octets in transit) for flow-control purposes6291456

These largely reflect settings and defaults from RFC 7540 6.5.1

5.3.2 - Service Configuration

Using the WASD_CONFIG_SERVICE directive [ServiceHttp2Protocol] HTTP/2 may be disabled on a per-service basis. The default is enabled if HTTP/2 is enabled globally.

5.3.3 - Mapping Set Rules

WASD request processing rules may be used on a per-path basis to modify (some) global configuration settings and provide other WevDAV configuation. See "WASD VMS Web Services - Install and Config"; 12 - Request Processing Configuration WASD Web Services - Install and Config .

HTTP/2 Set Rules
HTTP2=PROTOCOL=1.1send a "HTTP_1_1_REQUIRED" error causing the client to use HTTP/1.1 (RFC 7540 section 7)
HTTP2=SEND=GOAWAYsend a "GOAWAY" frame to the client resulting in it dropping the HTTP/2 connection
HTTP2=SEND=PINGsend a "PING" frame to the client calculating the Round Trip Time (RTT) of the connection
HTTP2=SEND=RESETsend a "RST_STREAM" frame to the client causing it to drop the HTTP/2 stream (request in progress)
HTTP2=STREAMS=MAX=integerset the maximum concurrent streams on a per-path basis
HTTP2=WRITE=low|normal|highWhen request data is written it is queued at the specified priority, where high priority are written before normal (default) and low priority, and normal priority before low. This is only for associated stream (request) and is not a connection or whole-of-server prioritisation.

Use path SETings to prioritise some resources (e.g. CSS and JavaScript) over others (e.g. images) and potentially improve page rendering speed. Where multiple concurrent requests are being serviced on the one HTTP/2 connection this will deliver the higher priority content before others.

SET **.css http2=write=high
SET **.js  http2=write=high

5.4 - HTTP/2 Detection

A request using HTTP/2 may be detected during processing with the http2: conditional.

if (http2:)
   do this

See "WASD VMS Web Services - Install and Config"; 7 - Conditional Configuration WASD Web Services - Install and Config .

A script may detect HTTP/2 using the REQUEST_PROTOCOL CGI variable with the value "HTTP/2". Other protocol versions are similarly represented.

A Server-Side Includes (SSI) document can use variations on the following construct (and similar to the script suggestion immediately above) to detect and process the request protocol.

<!--#if var={request_protocol} eqs="HTTP/2" -->
This is demonstrated in the example SSI document: WASD_ROOT:[EXERCISE]SHTML.SHTML

At the time of writing there is no browser-supported mechanism for a dynamic document (i.e. JavaScript) determining the underlying HTTP protocol used to access a resource. To access this information the server must be used. The suggested method, and the one employed by the DOTTY.HTML tool described above, is to provide one JavaScript source for HTTP/2 and another for everything else.

The document would contain

<script type="text/javascript" src="/example-path/http.js"></script>
and the server configuration
if (http2:)
   map /example-path/http.js /example-path/http2.js
   map /example-path/http.js /example-path/http1.js
where each contains a minimum variable setting or similar flag detectable by the document.

5.5 - HTTP/2 References

The following provide a starting-point for investigating HTTP/2 (verified available at time of publication).

next previous contents full-page