Chris@0: <?php
Chris@0: 
Chris@0: namespace Drupal\big_pipe\Render;
Chris@0: 
Chris@0: use Drupal\Component\Utility\Crypt;
Chris@0: use Drupal\Component\Utility\Html;
Chris@0: use Drupal\Core\Ajax\AjaxResponse;
Chris@0: use Drupal\Core\Ajax\ReplaceCommand;
Chris@0: use Drupal\Core\Asset\AttachedAssets;
Chris@0: use Drupal\Core\Asset\AttachedAssetsInterface;
Chris@0: use Drupal\Core\Config\ConfigFactoryInterface;
Chris@0: use Drupal\Core\Render\HtmlResponse;
Chris@0: use Drupal\Core\Render\RendererInterface;
Chris@0: use Symfony\Component\EventDispatcher\EventDispatcherInterface;
Chris@0: use Symfony\Component\HttpFoundation\Request;
Chris@0: use Symfony\Component\HttpFoundation\RequestStack;
Chris@0: use Symfony\Component\HttpFoundation\Response;
Chris@0: use Symfony\Component\HttpFoundation\Session\SessionInterface;
Chris@0: use Symfony\Component\HttpKernel\Event\FilterResponseEvent;
Chris@0: use Symfony\Component\HttpKernel\HttpKernelInterface;
Chris@0: use Symfony\Component\HttpKernel\KernelEvents;
Chris@0: 
Chris@0: /**
Chris@0:  * Service for sending an HTML response in chunks (to get faster page loads).
Chris@0:  *
Chris@0:  * At a high level, BigPipe sends a HTML response in chunks:
Chris@0:  * 1. one chunk: everything until just before </body> — this contains BigPipe
Chris@0:  *    placeholders for the personalized parts of the page. Hence this sends the
Chris@0:  *    non-personalized parts of the page. Let's call it The Skeleton.
Chris@0:  * 2. N chunks: a <script> tag per BigPipe placeholder in The Skeleton.
Chris@0:  * 3. one chunk: </body> and everything after it.
Chris@0:  *
Chris@0:  * This is conceptually identical to Facebook's BigPipe (hence the name).
Chris@0:  *
Chris@0:  * @see https://www.facebook.com/notes/facebook-engineering/bigpipe-pipelining-web-pages-for-high-performance/389414033919
Chris@0:  *
Chris@0:  * The major way in which Drupal differs from Facebook's implementation (and
Chris@0:  * others) is in its ability to automatically figure out which parts of the page
Chris@0:  * can benefit from BigPipe-style delivery. Drupal's render system has the
Chris@0:  * concept of "auto-placeholdering": content that is too dynamic is replaced
Chris@0:  * with a placeholder that can then be rendered at a later time. On top of that,
Chris@0:  * it also has the concept of "placeholder strategies": by default, placeholders
Chris@0:  * are replaced on the server side and the response is blocked on all of them
Chris@0:  * being replaced. But it's possible to add additional placeholder strategies.
Chris@0:  * BigPipe is just another placeholder strategy. Others could be ESI, AJAX …
Chris@0:  *
Chris@0:  * @see https://www.drupal.org/developing/api/8/render/arrays/cacheability/auto-placeholdering
Chris@0:  * @see \Drupal\Core\Render\PlaceholderGeneratorInterface::shouldAutomaticallyPlaceholder()
Chris@0:  * @see \Drupal\Core\Render\Placeholder\PlaceholderStrategyInterface
Chris@0:  * @see \Drupal\Core\Render\Placeholder\SingleFlushStrategy
Chris@0:  * @see \Drupal\big_pipe\Render\Placeholder\BigPipeStrategy
Chris@0:  *
Chris@0:  * There is also one noteworthy technical addition that Drupal makes. BigPipe as
Chris@0:  * described above, and as implemented by Facebook, can only work if JavaScript
Chris@0:  * is enabled. The BigPipe module also makes it possible to replace placeholders
Chris@0:  * using BigPipe in-situ, without JavaScript. This is not technically BigPipe at
Chris@0:  * all; it's just the use of multiple flushes. Since it is able to reuse much of
Chris@0:  * the logic though, we choose to call this "no-JS BigPipe".
Chris@0:  *
Chris@0:  * However, there is also a tangible benefit: some dynamic/expensive content is
Chris@0:  * not HTML, but for example a HTML attribute value (or part thereof). It's not
Chris@0:  * possible to efficiently replace such content using JavaScript, so "classic"
Chris@0:  * BigPipe is out of the question. For example: CSRF tokens in URLs.
Chris@0:  *
Chris@0:  * This allows us to use both no-JS BigPipe and "classic" BigPipe in the same
Chris@0:  * response to maximize the amount of content we can send as early as possible.
Chris@0:  *
Chris@0:  * Finally, a closer look at the implementation, and how it supports and reuses
Chris@0:  * existing Drupal concepts:
Chris@0:  * 1. BigPipe placeholders: 1 HtmlResponse + N embedded AjaxResponses.
Chris@0:  *   - Before a BigPipe response is sent, it is just a HTML response that
Chris@0:  *     contains BigPipe placeholders. Those placeholders look like
Chris@0:  *     <span data-big-pipe-placeholder-id="…"></span>. JavaScript is used to
Chris@0:  *     replace those placeholders.
Chris@0:  *     Therefore these placeholders are actually sent to the client.
Chris@0:  *   - The Skeleton of course has attachments, including most notably asset
Chris@0:  *     libraries. And those we track in drupalSettings.ajaxPageState.libraries —
Chris@0:  *     so that when we load new content through AJAX, we don't load the same
Chris@0:  *     asset libraries again. A HTML page can have multiple AJAX responses, each
Chris@0:  *     of which should take into account the combined AJAX page state of the
Chris@0:  *     HTML document and all preceding AJAX responses.
Chris@0:  *   - BigPipe does not make use of multiple AJAX requests/responses. It uses a
Chris@0:  *     single HTML response. But it is a more long-lived one: The Skeleton is
Chris@0:  *     sent first, the closing </body> tag is not yet sent, and the connection
Chris@0:  *     is kept open. Whenever another BigPipe Placeholder is rendered, Drupal
Chris@0:  *     sends (and so actually appends to the already-sent HTML) something like
Chris@0:  *     <script type="application/vnd.drupal-ajax">[{"command":"settings","settings":{…}}, {"command":…}.
Chris@0:  *   - So, for every BigPipe placeholder, we send such a <script
Chris@0:  *     type="application/vnd.drupal-ajax"> tag. And the contents of that tag is
Chris@0:  *     exactly like an AJAX response. The BigPipe module has JavaScript that
Chris@0:  *     listens for these and applies them. Let's call it an Embedded AJAX
Chris@0:  *     Response (since it is embedded in the HTML response). Now for the
Chris@0:  *     interesting bit: each of those Embedded AJAX Responses must also take
Chris@0:  *     into account the cumulative AJAX page state of the HTML document and all
Chris@0:  *     preceding Embedded AJAX responses.
Chris@0:  * 2. No-JS BigPipe placeholders: 1 HtmlResponse + N embedded HtmlResponses.
Chris@0:  *   - Before a BigPipe response is sent, it is just a HTML response that
Chris@0:  *     contains no-JS BigPipe placeholders. Those placeholders can take two
Chris@0:  *     different forms:
Chris@0:  *     1. <span data-big-pipe-nojs-placeholder-id="…"></span> if it's a
Chris@0:  *        placeholder that will be replaced by HTML
Chris@0:  *     2. big_pipe_nojs_placeholder_attribute_safe:… if it's a placeholder
Chris@0:  *        inside a HTML attribute, in which 1. would be invalid (angle brackets
Chris@0:  *        are not allowed inside HTML attributes)
Chris@0:  *     No-JS BigPipe placeholders are not replaced using JavaScript, they must
Chris@0:  *     be replaced upon sending the BigPipe response. So, while the response is
Chris@0:  *     being sent, upon encountering these placeholders, their corresponding
Chris@0:  *     placeholder replacements are sent instead.
Chris@0:  *     Therefore these placeholders are never actually sent to the client.
Chris@0:  *   - See second bullet of point 1.
Chris@0:  *   - No-JS BigPipe does not use multiple AJAX requests/responses. It uses a
Chris@0:  *     single HTML response. But it is a more long-lived one: The Skeleton is
Chris@0:  *     split into multiple parts, the separators are where the no-JS BigPipe
Chris@0:  *     placeholders used to be. Whenever another no-JS BigPipe placeholder is
Chris@0:  *     rendered, Drupal sends (and so actually appends to the already-sent HTML)
Chris@0:  *     something like
Chris@0:  *     <link rel="stylesheet" …><script …><content>.
Chris@0:  *   - So, for every no-JS BigPipe placeholder, we send its associated CSS and
Chris@0:  *     header JS that has not already been sent (the bottom JS is not yet sent,
Chris@0:  *     so we can accumulate all of it and send it together at the end). This
Chris@0:  *     ensures that the markup is rendered as it was originally intended: its
Chris@0:  *     CSS and JS used to be blocking, and it still is. Let's call it an
Chris@0:  *     Embedded HTML response. Each of those Embedded HTML Responses must also
Chris@0:  *     take into account the cumulative AJAX page state of the HTML document and
Chris@0:  *     all preceding Embedded HTML responses.
Chris@0:  *   - Finally: any non-critical JavaScript associated with all Embedded HTML
Chris@0:  *     Responses, i.e. any footer/bottom/non-header JavaScript, is loaded after
Chris@0:  *     The Skeleton.
Chris@0:  *
Chris@0:  * Combining all of the above, when using both BigPipe placeholders and no-JS
Chris@0:  * BigPipe placeholders, we therefore send: 1 HtmlResponse + M Embedded HTML
Chris@0:  * Responses + N Embedded AJAX Responses. Schematically, we send these chunks:
Chris@0:  *  1. Byte zero until 1st no-JS placeholder: headers + <html><head /><span>…</span>
Chris@0:  *  2. 1st no-JS placeholder replacement: <link rel="stylesheet" …><script …><content>
Chris@0:  *  3. Content until 2nd no-JS placeholder: <span>…</span>
Chris@0:  *  4. 2nd no-JS placeholder replacement: <link rel="stylesheet" …><script …><content>
Chris@0:  *  5. Content until 3rd no-JS placeholder: <span>…</span>
Chris@0:  *  6. [… repeat until all no-JS placeholder replacements are sent …]
Chris@0:  *  7. Send content after last no-JS placeholder.
Chris@0:  *  8. Send script_bottom (markup to load bottom i.e. non-critical JS).
Chris@0:  *  9. 1st placeholder replacement: <script type="application/vnd.drupal-ajax">[{"command":"settings","settings":{…}}, {"command":…}
Chris@0:  * 10. 2nd placeholder replacement: <script type="application/vnd.drupal-ajax">[{"command":"settings","settings":{…}}, {"command":…}
Chris@0:  * 11. [… repeat until all placeholder replacements are sent …]
Chris@0:  * 12. Send </body> and everything after it.
Chris@0:  * 13. Terminate request/response cycle.
Chris@0:  *
Chris@0:  * @see \Drupal\big_pipe\EventSubscriber\HtmlResponseBigPipeSubscriber
Chris@0:  * @see \Drupal\big_pipe\Render\Placeholder\BigPipeStrategy
Chris@0:  */
Chris@0: class BigPipe {
Chris@0: 
Chris@0:   /**
Chris@0:    * The BigPipe placeholder replacements start signal.
Chris@0:    *
Chris@0:    * @var string
Chris@0:    */
Chris@0:   const START_SIGNAL = '<script type="application/vnd.drupal-ajax" data-big-pipe-event="start"></script>';
Chris@0: 
Chris@0:   /**
Chris@0:    * The BigPipe placeholder replacements stop signal.
Chris@0:    *
Chris@0:    * @var string
Chris@0:    */
Chris@0:   const STOP_SIGNAL = '<script type="application/vnd.drupal-ajax" data-big-pipe-event="stop"></script>';
Chris@0: 
Chris@0:   /**
Chris@0:    * The renderer.
Chris@0:    *
Chris@0:    * @var \Drupal\Core\Render\RendererInterface
Chris@0:    */
Chris@0:   protected $renderer;
Chris@0: 
Chris@0:   /**
Chris@0:    * The session.
Chris@0:    *
Chris@0:    * @var \Symfony\Component\HttpFoundation\Session\SessionInterface
Chris@0:    */
Chris@0:   protected $session;
Chris@0: 
Chris@0:   /**
Chris@0:    * The request stack.
Chris@0:    *
Chris@0:    * @var \Symfony\Component\HttpFoundation\RequestStack
Chris@0:    */
Chris@0:   protected $requestStack;
Chris@0: 
Chris@0:   /**
Chris@0:    * The HTTP kernel.
Chris@0:    *
Chris@0:    * @var \Symfony\Component\HttpKernel\HttpKernelInterface
Chris@0:    */
Chris@0:   protected $httpKernel;
Chris@0: 
Chris@0:   /**
Chris@0:    * The event dispatcher.
Chris@0:    *
Chris@0:    * @var \Symfony\Component\EventDispatcher\EventDispatcherInterface
Chris@0:    */
Chris@0:   protected $eventDispatcher;
Chris@0: 
Chris@0:   /**
Chris@0:    * The config factory.
Chris@0:    *
Chris@0:    * @var \Drupal\Core\Config\ConfigFactoryInterface
Chris@0:    */
Chris@0:   protected $configFactory;
Chris@0: 
Chris@0:   /**
Chris@0:    * Constructs a new BigPipe class.
Chris@0:    *
Chris@0:    * @param \Drupal\Core\Render\RendererInterface $renderer
Chris@0:    *   The renderer.
Chris@0:    * @param \Symfony\Component\HttpFoundation\Session\SessionInterface $session
Chris@0:    *   The session.
Chris@0:    * @param \Symfony\Component\HttpFoundation\RequestStack $request_stack
Chris@0:    *   The request stack.
Chris@0:    * @param \Symfony\Component\HttpKernel\HttpKernelInterface $http_kernel
Chris@0:    *   The HTTP kernel.
Chris@0:    * @param \Symfony\Component\EventDispatcher\EventDispatcherInterface $event_dispatcher
Chris@0:    *   The event dispatcher.
Chris@0:    * @param \Drupal\Core\Config\ConfigFactoryInterface $config_factory
Chris@0:    *   The config factory.
Chris@0:    */
Chris@0:   public function __construct(RendererInterface $renderer, SessionInterface $session, RequestStack $request_stack, HttpKernelInterface $http_kernel, EventDispatcherInterface $event_dispatcher, ConfigFactoryInterface $config_factory) {
Chris@0:     $this->renderer = $renderer;
Chris@0:     $this->session = $session;
Chris@0:     $this->requestStack = $request_stack;
Chris@0:     $this->httpKernel = $http_kernel;
Chris@0:     $this->eventDispatcher = $event_dispatcher;
Chris@0:     $this->configFactory = $config_factory;
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Performs tasks before sending content (and rendering placeholders).
Chris@0:    */
Chris@0:   protected function performPreSendTasks() {
Chris@0:     // The content in the placeholders may depend on the session, and by the
Chris@0:     // time the response is sent (see index.php), the session is already
Chris@0:     // closed. Reopen it for the duration that we are rendering placeholders.
Chris@0:     $this->session->start();
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Performs tasks after sending content (and rendering placeholders).
Chris@0:    */
Chris@0:   protected function performPostSendTasks() {
Chris@0:     // Close the session again.
Chris@0:     $this->session->save();
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends a chunk.
Chris@0:    *
Chris@0:    * @param string|\Drupal\Core\Render\HtmlResponse $chunk
Chris@0:    *   The string or response to append. String if there's no cacheability
Chris@0:    *   metadata or attachments to merge.
Chris@0:    */
Chris@0:   protected function sendChunk($chunk) {
Chris@0:     assert(is_string($chunk) || $chunk instanceof HtmlResponse);
Chris@0:     if ($chunk instanceof HtmlResponse) {
Chris@0:       print $chunk->getContent();
Chris@0:     }
Chris@0:     else {
Chris@0:       print $chunk;
Chris@0:     }
Chris@0:     flush();
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends an HTML response in chunks using the BigPipe technique.
Chris@0:    *
Chris@0:    * @param \Drupal\big_pipe\Render\BigPipeResponse $response
Chris@0:    *   The BigPipe response to send.
Chris@0:    *
Chris@0:    * @internal
Chris@0:    *   This method should only be invoked by
Chris@0:    *   \Drupal\big_pipe\Render\BigPipeResponse, which is itself an internal
Chris@0:    *   class.
Chris@0:    */
Chris@0:   public function sendContent(BigPipeResponse $response) {
Chris@0:     $content = $response->getContent();
Chris@0:     $attachments = $response->getAttachments();
Chris@0: 
Chris@0:     // First, gather the BigPipe placeholders that must be replaced.
Chris@0:     $placeholders = isset($attachments['big_pipe_placeholders']) ? $attachments['big_pipe_placeholders'] : [];
Chris@0:     $nojs_placeholders = isset($attachments['big_pipe_nojs_placeholders']) ? $attachments['big_pipe_nojs_placeholders'] : [];
Chris@0: 
Chris@0:     // BigPipe sends responses using "Transfer-Encoding: chunked". To avoid
Chris@0:     // sending already-sent assets, it is necessary to track cumulative assets
Chris@0:     // from all previously rendered/sent chunks.
Chris@0:     // @see http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.41
Chris@0:     $cumulative_assets = AttachedAssets::createFromRenderArray(['#attached' => $attachments]);
Chris@0:     $cumulative_assets->setAlreadyLoadedLibraries($attachments['library']);
Chris@0: 
Chris@0:     $this->performPreSendTasks();
Chris@0: 
Chris@0:     // Find the closing </body> tag and get the strings before and after. But be
Chris@0:     // careful to use the latest occurrence of the string "</body>", to ensure
Chris@0:     // that strings in inline JavaScript or CDATA sections aren't used instead.
Chris@0:     $parts = explode('</body>', $content);
Chris@0:     $post_body = array_pop($parts);
Chris@0:     $pre_body = implode('', $parts);
Chris@0: 
Chris@0:     $this->sendPreBody($pre_body, $nojs_placeholders, $cumulative_assets);
Chris@0:     $this->sendPlaceholders($placeholders, $this->getPlaceholderOrder($pre_body, $placeholders), $cumulative_assets);
Chris@0:     $this->sendPostBody($post_body);
Chris@0: 
Chris@0:     $this->performPostSendTasks();
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends everything until just before </body>.
Chris@0:    *
Chris@0:    * @param string $pre_body
Chris@0:    *   The HTML response's content until the closing </body> tag.
Chris@0:    * @param array $no_js_placeholders
Chris@0:    *   The no-JS BigPipe placeholders.
Chris@0:    * @param \Drupal\Core\Asset\AttachedAssetsInterface $cumulative_assets
Chris@0:    *   The cumulative assets sent so far; to be updated while rendering no-JS
Chris@0:    *   BigPipe placeholders.
Chris@0:    */
Chris@0:   protected function sendPreBody($pre_body, array $no_js_placeholders, AttachedAssetsInterface $cumulative_assets) {
Chris@0:     // If there are no no-JS BigPipe placeholders, we can send the pre-</body>
Chris@0:     // part of the page immediately.
Chris@0:     if (empty($no_js_placeholders)) {
Chris@0:       $this->sendChunk($pre_body);
Chris@0:       return;
Chris@0:     }
Chris@0: 
Chris@0:     // Extract the scripts_bottom markup: the no-JS BigPipe placeholders that we
Chris@0:     // will render may attach additional asset libraries, and if so, it will be
Chris@0:     // necessary to re-render scripts_bottom.
Chris@0:     list($pre_scripts_bottom, $scripts_bottom, $post_scripts_bottom) = explode('<drupal-big-pipe-scripts-bottom-marker>', $pre_body, 3);
Chris@0:     $cumulative_assets_initial = clone $cumulative_assets;
Chris@0: 
Chris@0:     $this->sendNoJsPlaceholders($pre_scripts_bottom . $post_scripts_bottom, $no_js_placeholders, $cumulative_assets);
Chris@0: 
Chris@0:     // If additional asset libraries or drupalSettings were attached by any of
Chris@0:     // the placeholders, then we need to re-render scripts_bottom.
Chris@0:     if ($cumulative_assets_initial != $cumulative_assets) {
Chris@0:       // Create a new HtmlResponse. Ensure the CSS and (non-bottom) JS is sent
Chris@0:       // before the HTML they're associated with.
Chris@0:       // @see \Drupal\Core\Render\HtmlResponseSubscriber
Chris@0:       // @see template_preprocess_html()
Chris@0:       $js_bottom_placeholder = '<nojs-bigpipe-placeholder-scripts-bottom-placeholder token="' . Crypt::randomBytesBase64(55) . '">';
Chris@0: 
Chris@0:       $html_response = new HtmlResponse();
Chris@0:       $html_response->setContent([
Chris@0:         '#markup' => BigPipeMarkup::create($js_bottom_placeholder),
Chris@0:         '#attached' => [
Chris@0:           'drupalSettings' => $cumulative_assets->getSettings(),
Chris@0:           'library' => $cumulative_assets->getAlreadyLoadedLibraries(),
Chris@0:           'html_response_attachment_placeholders' => [
Chris@0:             'scripts_bottom' => $js_bottom_placeholder,
Chris@0:           ],
Chris@0:         ],
Chris@0:       ]);
Chris@0:       $html_response->getCacheableMetadata()->setCacheMaxAge(0);
Chris@0: 
Chris@0:       // Push a fake request with the asset libraries loaded so far and dispatch
Chris@0:       // KernelEvents::RESPONSE event. This results in the attachments for the
Chris@0:       // HTML response being processed by HtmlResponseAttachmentsProcessor and
Chris@0:       // hence the HTML to load the bottom JavaScript can be rendered.
Chris@0:       $fake_request = $this->requestStack->getMasterRequest()->duplicate();
Chris@0:       $html_response = $this->filterEmbeddedResponse($fake_request, $html_response);
Chris@0:       $scripts_bottom = $html_response->getContent();
Chris@0:     }
Chris@0: 
Chris@0:     $this->sendChunk($scripts_bottom);
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends no-JS BigPipe placeholders' replacements as embedded HTML responses.
Chris@0:    *
Chris@0:    * @param string $html
Chris@0:    *   HTML markup.
Chris@0:    * @param array $no_js_placeholders
Chris@0:    *   Associative array; the no-JS BigPipe placeholders. Keys are the BigPipe
Chris@0:    *   selectors.
Chris@0:    * @param \Drupal\Core\Asset\AttachedAssetsInterface $cumulative_assets
Chris@0:    *   The cumulative assets sent so far; to be updated while rendering no-JS
Chris@0:    *   BigPipe placeholders.
Chris@0:    *
Chris@0:    * @throws \Exception
Chris@0:    *   If an exception is thrown during the rendering of a placeholder, it is
Chris@0:    *   caught to allow the other placeholders to still be replaced. But when
Chris@0:    *   error logging is configured to be verbose, the exception is rethrown to
Chris@0:    *   simplify debugging.
Chris@0:    */
Chris@0:   protected function sendNoJsPlaceholders($html, $no_js_placeholders, AttachedAssetsInterface $cumulative_assets) {
Chris@0:     // Split the HTML on every no-JS placeholder string.
Chris@0:     $prepare_for_preg_split = function ($placeholder_string) {
Chris@0:       return '(' . preg_quote($placeholder_string, '/') . ')';
Chris@0:     };
Chris@0:     $preg_placeholder_strings = array_map($prepare_for_preg_split, array_keys($no_js_placeholders));
Chris@0:     $fragments = preg_split('/' . implode('|', $preg_placeholder_strings) . '/', $html, NULL, PREG_SPLIT_NO_EMPTY | PREG_SPLIT_DELIM_CAPTURE);
Chris@0: 
Chris@0:     // Determine how many occurrences there are of each no-JS placeholder.
Chris@0:     $placeholder_occurrences = array_count_values(array_intersect($fragments, array_keys($no_js_placeholders)));
Chris@0: 
Chris@0:     // Set up a variable to store the content of placeholders that have multiple
Chris@0:     // occurrences.
Chris@0:     $multi_occurrence_placeholders_content = [];
Chris@0: 
Chris@0:     foreach ($fragments as $fragment) {
Chris@0:       // If the fragment isn't one of the no-JS placeholders, it is the HTML in
Chris@0:       // between placeholders and it must be printed & flushed immediately. The
Chris@0:       // rest of the logic in the loop handles the placeholders.
Chris@0:       if (!isset($no_js_placeholders[$fragment])) {
Chris@0:         $this->sendChunk($fragment);
Chris@0:         continue;
Chris@0:       }
Chris@0: 
Chris@0:       // If there are multiple occurrences of this particular placeholder, and
Chris@0:       // this is the second occurrence, we can skip all calculations and just
Chris@0:       // send the same content.
Chris@0:       if ($placeholder_occurrences[$fragment] > 1 && isset($multi_occurrence_placeholders_content[$fragment])) {
Chris@0:         $this->sendChunk($multi_occurrence_placeholders_content[$fragment]);
Chris@0:         continue;
Chris@0:       }
Chris@0: 
Chris@0:       $placeholder = $fragment;
Chris@14:       assert(isset($no_js_placeholders[$placeholder]));
Chris@0:       $token = Crypt::randomBytesBase64(55);
Chris@0: 
Chris@0:       // Render the placeholder, but include the cumulative settings assets, so
Chris@0:       // we can calculate the overall settings for the entire page.
Chris@0:       $placeholder_plus_cumulative_settings = [
Chris@0:         'placeholder' => $no_js_placeholders[$placeholder],
Chris@0:         'cumulative_settings_' . $token => [
Chris@0:           '#attached' => [
Chris@0:             'drupalSettings' => $cumulative_assets->getSettings(),
Chris@0:           ],
Chris@0:         ],
Chris@0:       ];
Chris@0:       try {
Chris@0:         $elements = $this->renderPlaceholder($placeholder, $placeholder_plus_cumulative_settings);
Chris@0:       }
Chris@0:       catch (\Exception $e) {
Chris@0:         if ($this->configFactory->get('system.logging')->get('error_level') === ERROR_REPORTING_DISPLAY_VERBOSE) {
Chris@0:           throw $e;
Chris@0:         }
Chris@0:         else {
Chris@0:           trigger_error($e, E_USER_ERROR);
Chris@0:           continue;
Chris@0:         }
Chris@0:       }
Chris@0: 
Chris@0:       // Create a new HtmlResponse. Ensure the CSS and (non-bottom) JS is sent
Chris@0:       // before the HTML they're associated with. In other words: ensure the
Chris@0:       // critical assets for this placeholder's markup are loaded first.
Chris@0:       // @see \Drupal\Core\Render\HtmlResponseSubscriber
Chris@0:       // @see template_preprocess_html()
Chris@0:       $css_placeholder = '<nojs-bigpipe-placeholder-styles-placeholder token="' . $token . '">';
Chris@0:       $js_placeholder = '<nojs-bigpipe-placeholder-scripts-placeholder token="' . $token . '">';
Chris@0:       $elements['#markup'] = BigPipeMarkup::create($css_placeholder . $js_placeholder . (string) $elements['#markup']);
Chris@0:       $elements['#attached']['html_response_attachment_placeholders']['styles'] = $css_placeholder;
Chris@0:       $elements['#attached']['html_response_attachment_placeholders']['scripts'] = $js_placeholder;
Chris@0: 
Chris@0:       $html_response = new HtmlResponse();
Chris@0:       $html_response->setContent($elements);
Chris@0:       $html_response->getCacheableMetadata()->setCacheMaxAge(0);
Chris@0: 
Chris@0:       // Push a fake request with the asset libraries loaded so far and dispatch
Chris@0:       // KernelEvents::RESPONSE event. This results in the attachments for the
Chris@0:       // HTML response being processed by HtmlResponseAttachmentsProcessor and
Chris@0:       // hence:
Chris@0:       // - the HTML to load the CSS can be rendered.
Chris@0:       // - the HTML to load the JS (at the top) can be rendered.
Chris@0:       $fake_request = $this->requestStack->getMasterRequest()->duplicate();
Chris@0:       $fake_request->request->set('ajax_page_state', ['libraries' => implode(',', $cumulative_assets->getAlreadyLoadedLibraries())]);
Chris@0:       try {
Chris@0:         $html_response = $this->filterEmbeddedResponse($fake_request, $html_response);
Chris@0:       }
Chris@0:       catch (\Exception $e) {
Chris@0:         if ($this->configFactory->get('system.logging')->get('error_level') === ERROR_REPORTING_DISPLAY_VERBOSE) {
Chris@0:           throw $e;
Chris@0:         }
Chris@0:         else {
Chris@0:           trigger_error($e, E_USER_ERROR);
Chris@0:           continue;
Chris@0:         }
Chris@0:       }
Chris@0: 
Chris@0:       // Send this embedded HTML response.
Chris@0:       $this->sendChunk($html_response);
Chris@0: 
Chris@0:       // Another placeholder was rendered and sent, track the set of asset
Chris@0:       // libraries sent so far. Any new settings also need to be tracked, so
Chris@0:       // they can be sent in ::sendPreBody().
Chris@0:       $cumulative_assets->setAlreadyLoadedLibraries(array_merge($cumulative_assets->getAlreadyLoadedLibraries(), $html_response->getAttachments()['library']));
Chris@0:       $cumulative_assets->setSettings($html_response->getAttachments()['drupalSettings']);
Chris@0: 
Chris@0:       // If there are multiple occurrences of this particular placeholder, track
Chris@0:       // the content that was sent, so we can skip all calculations for the next
Chris@0:       // occurrence.
Chris@0:       if ($placeholder_occurrences[$fragment] > 1) {
Chris@0:         $multi_occurrence_placeholders_content[$fragment] = $html_response->getContent();
Chris@0:       }
Chris@0:     }
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends BigPipe placeholders' replacements as embedded AJAX responses.
Chris@0:    *
Chris@0:    * @param array $placeholders
Chris@0:    *   Associative array; the BigPipe placeholders. Keys are the BigPipe
Chris@0:    *   placeholder IDs.
Chris@0:    * @param array $placeholder_order
Chris@0:    *   Indexed array; the order in which the BigPipe placeholders must be sent.
Chris@0:    *   Values are the BigPipe placeholder IDs. (These values correspond to keys
Chris@0:    *   in $placeholders.)
Chris@0:    * @param \Drupal\Core\Asset\AttachedAssetsInterface $cumulative_assets
Chris@0:    *   The cumulative assets sent so far; to be updated while rendering BigPipe
Chris@0:    *   placeholders.
Chris@0:    *
Chris@0:    * @throws \Exception
Chris@0:    *   If an exception is thrown during the rendering of a placeholder, it is
Chris@0:    *   caught to allow the other placeholders to still be replaced. But when
Chris@0:    *   error logging is configured to be verbose, the exception is rethrown to
Chris@0:    *   simplify debugging.
Chris@0:    */
Chris@0:   protected function sendPlaceholders(array $placeholders, array $placeholder_order, AttachedAssetsInterface $cumulative_assets) {
Chris@0:     // Return early if there are no BigPipe placeholders to send.
Chris@0:     if (empty($placeholders)) {
Chris@0:       return;
Chris@0:     }
Chris@0: 
Chris@0:     // Send the start signal.
Chris@0:     $this->sendChunk("\n" . static::START_SIGNAL . "\n");
Chris@0: 
Chris@0:     // A BigPipe response consists of a HTML response plus multiple embedded
Chris@0:     // AJAX responses. To process the attachments of those AJAX responses, we
Chris@0:     // need a fake request that is identical to the master request, but with
Chris@0:     // one change: it must have the right Accept header, otherwise the work-
Chris@0:     // around for a bug in IE9 will cause not JSON, but <textarea>-wrapped JSON
Chris@0:     // to be returned.
Chris@0:     // @see \Drupal\Core\EventSubscriber\AjaxResponseSubscriber::onResponse()
Chris@0:     $fake_request = $this->requestStack->getMasterRequest()->duplicate();
Chris@0:     $fake_request->headers->set('Accept', 'application/vnd.drupal-ajax');
Chris@0: 
Chris@0:     foreach ($placeholder_order as $placeholder_id) {
Chris@0:       if (!isset($placeholders[$placeholder_id])) {
Chris@0:         continue;
Chris@0:       }
Chris@0: 
Chris@0:       // Render the placeholder.
Chris@0:       $placeholder_render_array = $placeholders[$placeholder_id];
Chris@0:       try {
Chris@0:         $elements = $this->renderPlaceholder($placeholder_id, $placeholder_render_array);
Chris@0:       }
Chris@0:       catch (\Exception $e) {
Chris@0:         if ($this->configFactory->get('system.logging')->get('error_level') === ERROR_REPORTING_DISPLAY_VERBOSE) {
Chris@0:           throw $e;
Chris@0:         }
Chris@0:         else {
Chris@0:           trigger_error($e, E_USER_ERROR);
Chris@0:           continue;
Chris@0:         }
Chris@0:       }
Chris@0: 
Chris@0:       // Create a new AjaxResponse.
Chris@0:       $ajax_response = new AjaxResponse();
Chris@0:       // JavaScript's querySelector automatically decodes HTML entities in
Chris@0:       // attributes, so we must decode the entities of the current BigPipe
Chris@0:       // placeholder ID (which has HTML entities encoded since we use it to find
Chris@0:       // the placeholders).
Chris@0:       $big_pipe_js_placeholder_id = Html::decodeEntities($placeholder_id);
Chris@0:       $ajax_response->addCommand(new ReplaceCommand(sprintf('[data-big-pipe-placeholder-id="%s"]', $big_pipe_js_placeholder_id), $elements['#markup']));
Chris@0:       $ajax_response->setAttachments($elements['#attached']);
Chris@0: 
Chris@0:       // Push a fake request with the asset libraries loaded so far and dispatch
Chris@0:       // KernelEvents::RESPONSE event. This results in the attachments for the
Chris@0:       // AJAX response being processed by AjaxResponseAttachmentsProcessor and
Chris@0:       // hence:
Chris@0:       // - the necessary AJAX commands to load the necessary missing asset
Chris@0:       //   libraries and updated AJAX page state are added to the AJAX response
Chris@0:       // - the attachments associated with the response are finalized, which
Chris@0:       //   allows us to track the total set of asset libraries sent in the
Chris@0:       //   initial HTML response plus all embedded AJAX responses sent so far.
Chris@0:       $fake_request->request->set('ajax_page_state', ['libraries' => implode(',', $cumulative_assets->getAlreadyLoadedLibraries())] + $cumulative_assets->getSettings()['ajaxPageState']);
Chris@0:       try {
Chris@0:         $ajax_response = $this->filterEmbeddedResponse($fake_request, $ajax_response);
Chris@0:       }
Chris@0:       catch (\Exception $e) {
Chris@0:         if ($this->configFactory->get('system.logging')->get('error_level') === ERROR_REPORTING_DISPLAY_VERBOSE) {
Chris@0:           throw $e;
Chris@0:         }
Chris@0:         else {
Chris@0:           trigger_error($e, E_USER_ERROR);
Chris@0:           continue;
Chris@0:         }
Chris@0:       }
Chris@0: 
Chris@0:       // Send this embedded AJAX response.
Chris@0:       $json = $ajax_response->getContent();
Chris@0:       $output = <<<EOF
Chris@0:     <script type="application/vnd.drupal-ajax" data-big-pipe-replacement-for-placeholder-with-id="$placeholder_id">
Chris@0:     $json
Chris@0:     </script>
Chris@0: EOF;
Chris@0:       $this->sendChunk($output);
Chris@0: 
Chris@0:       // Another placeholder was rendered and sent, track the set of asset
Chris@0:       // libraries sent so far. Any new settings are already sent; we don't need
Chris@0:       // to track those.
Chris@0:       if (isset($ajax_response->getAttachments()['drupalSettings']['ajaxPageState']['libraries'])) {
Chris@0:         $cumulative_assets->setAlreadyLoadedLibraries(explode(',', $ajax_response->getAttachments()['drupalSettings']['ajaxPageState']['libraries']));
Chris@0:       }
Chris@0:     }
Chris@0: 
Chris@0:     // Send the stop signal.
Chris@0:     $this->sendChunk("\n" . static::STOP_SIGNAL . "\n");
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Filters the given embedded response, using the cumulative AJAX page state.
Chris@0:    *
Chris@0:    * @param \Symfony\Component\HttpFoundation\Request $fake_request
Chris@0:    *   A fake subrequest that contains the cumulative AJAX page state of the
Chris@0:    *   HTML document and all preceding Embedded HTML or AJAX responses.
Chris@0:    * @param \Symfony\Component\HttpFoundation\Response|\Drupal\Core\Render\HtmlResponse|\Drupal\Core\Ajax\AjaxResponse $embedded_response
Chris@0:    *   Either a HTML response or an AJAX response that will be embedded in the
Chris@0:    *   overall HTML response.
Chris@0:    *
Chris@0:    * @return \Symfony\Component\HttpFoundation\Response
Chris@0:    *   The filtered response, which will load only the assets that $fake_request
Chris@0:    *   did not indicate to already have been loaded, plus the updated cumulative
Chris@0:    *   AJAX page state.
Chris@0:    */
Chris@0:   protected function filterEmbeddedResponse(Request $fake_request, Response $embedded_response) {
Chris@14:     assert($embedded_response instanceof HtmlResponse || $embedded_response instanceof AjaxResponse);
Chris@0:     return $this->filterResponse($fake_request, HttpKernelInterface::SUB_REQUEST, $embedded_response);
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Filters the given response.
Chris@0:    *
Chris@0:    * @param \Symfony\Component\HttpFoundation\Request $request
Chris@0:    *   The request for which a response is being sent.
Chris@0:    * @param int $request_type
Chris@0:    *   The request type. Can either be
Chris@0:    *   \Symfony\Component\HttpKernel\HttpKernelInterface::MASTER_REQUEST or
Chris@0:    *   \Symfony\Component\HttpKernel\HttpKernelInterface::SUB_REQUEST.
Chris@0:    * @param \Symfony\Component\HttpFoundation\Response $response
Chris@0:    *   The response to filter.
Chris@0:    *
Chris@0:    * @return \Symfony\Component\HttpFoundation\Response
Chris@0:    *   The filtered response.
Chris@0:    */
Chris@0:   protected function filterResponse(Request $request, $request_type, Response $response) {
Chris@14:     assert($request_type === HttpKernelInterface::MASTER_REQUEST || $request_type === HttpKernelInterface::SUB_REQUEST);
Chris@0:     $this->requestStack->push($request);
Chris@0:     $event = new FilterResponseEvent($this->httpKernel, $request, $request_type, $response);
Chris@0:     $this->eventDispatcher->dispatch(KernelEvents::RESPONSE, $event);
Chris@0:     $filtered_response = $event->getResponse();
Chris@0:     $this->requestStack->pop();
Chris@0:     return $filtered_response;
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Sends </body> and everything after it.
Chris@0:    *
Chris@0:    * @param string $post_body
Chris@0:    *   The HTML response's content after the closing </body> tag.
Chris@0:    */
Chris@0:   protected function sendPostBody($post_body) {
Chris@0:     $this->sendChunk('</body>' . $post_body);
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Renders a placeholder, and just that placeholder.
Chris@0:    *
Chris@0:    * BigPipe renders placeholders independently of the rest of the content, so
Chris@0:    * it needs to be able to render placeholders by themselves.
Chris@0:    *
Chris@0:    * @param string $placeholder
Chris@0:    *   The placeholder to render.
Chris@0:    * @param array $placeholder_render_array
Chris@0:    *   The render array associated with that placeholder.
Chris@0:    *
Chris@0:    * @return array
Chris@0:    *   The render array representing the rendered placeholder.
Chris@0:    *
Chris@0:    * @see \Drupal\Core\Render\RendererInterface::renderPlaceholder()
Chris@0:    */
Chris@0:   protected function renderPlaceholder($placeholder, array $placeholder_render_array) {
Chris@0:     $elements = [
Chris@0:       '#markup' => $placeholder,
Chris@0:       '#attached' => [
Chris@0:         'placeholders' => [
Chris@0:           $placeholder => $placeholder_render_array,
Chris@0:         ],
Chris@0:       ],
Chris@0:     ];
Chris@0:     return $this->renderer->renderPlaceholder($placeholder, $elements);
Chris@0:   }
Chris@0: 
Chris@0:   /**
Chris@0:    * Gets the BigPipe placeholder order.
Chris@0:    *
Chris@0:    * Determines the order in which BigPipe placeholders must be replaced.
Chris@0:    *
Chris@0:    * @param string $html
Chris@0:    *   HTML markup.
Chris@0:    * @param array $placeholders
Chris@0:    *   Associative array; the BigPipe placeholders. Keys are the BigPipe
Chris@0:    *   placeholder IDs.
Chris@0:    *
Chris@0:    * @return array
Chris@0:    *   Indexed array; the order in which the BigPipe placeholders must be sent.
Chris@0:    *   Values are the BigPipe placeholder IDs. Note that only unique
Chris@0:    *   placeholders are kept: if the same placeholder occurs multiple times, we
Chris@0:    *   only keep the first occurrence.
Chris@0:    */
Chris@0:   protected function getPlaceholderOrder($html, $placeholders) {
Chris@0:     $fragments = explode('<span data-big-pipe-placeholder-id="', $html);
Chris@0:     array_shift($fragments);
Chris@0:     $placeholder_ids = [];
Chris@0: 
Chris@0:     foreach ($fragments as $fragment) {
Chris@0:       $t = explode('"></span>', $fragment, 2);
Chris@0:       $placeholder_id = $t[0];
Chris@0:       $placeholder_ids[] = $placeholder_id;
Chris@0:     }
Chris@0:     $placeholder_ids = array_unique($placeholder_ids);
Chris@0: 
Chris@0:     // The 'status messages' placeholder needs to be special cased, because it
Chris@0:     // depends on global state that can be modified when other placeholders are
Chris@0:     // being rendered: any code can add messages to render.
Chris@0:     // This violates the principle that each lazy builder must be able to render
Chris@0:     // itself in isolation, and therefore in any order. However, we cannot
Chris@17:     // change the way \Drupal\Core\Messenger\MessengerInterface::addMessage()
Chris@17:     // works in the Drupal 8 cycle. So we have to accommodate its special needs.
Chris@0:     // Allowing placeholders to be rendered in a particular order (in this case:
Chris@0:     // last) would violate this isolation principle. Thus a monopoly is granted
Chris@0:     // to this one special case, with this hard-coded solution.
Chris@0:     // @see \Drupal\Core\Render\Element\StatusMessages
Chris@0:     // @see \Drupal\Core\Render\Renderer::replacePlaceholders()
Chris@0:     // @see https://www.drupal.org/node/2712935#comment-11368923
Chris@0:     $message_placeholder_ids = [];
Chris@0:     foreach ($placeholders as $placeholder_id => $placeholder_element) {
Chris@0:       if (isset($placeholder_element['#lazy_builder']) && $placeholder_element['#lazy_builder'][0] === 'Drupal\Core\Render\Element\StatusMessages::renderMessages') {
Chris@0:         $message_placeholder_ids[] = $placeholder_id;
Chris@0:       }
Chris@0:     }
Chris@0: 
Chris@0:     // Return placeholder IDs in DOM order, but with the 'status messages'
Chris@0:     // placeholders at the end, if they are present.
Chris@0:     $ordered_placeholder_ids = array_merge(
Chris@0:       array_diff($placeholder_ids, $message_placeholder_ids),
Chris@0:       array_intersect($placeholder_ids, $message_placeholder_ids)
Chris@0:     );
Chris@0:     return $ordered_placeholder_ids;
Chris@0:   }
Chris@0: 
Chris@0: }