Russian version
Add to Del.icio.us
English version
Digg It!

 Old-School dkLab | Constructor | HTTP_StaticMerger: Automatical "merging" of CSS and JS files for faster load 

Site map :: Project Orphus :: Constructor


2008-11-23
Discuss at the Forum

You may help to develop and improve this library at GitHub

The library HTTP_StaticMerger merges "on the air" a set of static files (CSS or JS) and speedups page loading (lower number of HTTP queries). The nearest analog is minify.

Of course, it is recommended use the library together with caching reverse-proxy (e.g. nginx) to minimize the response time.

Now you could view usage samples.

For dummies 

For heavy loaded sites: please read Nginx: gzip compression and caching chapter below to know how to use HTTP_StaticMerger properly.

The problem

Commonly complex site has a large number of included CSS and JS files:

Listing 1: a number of static files loading
<head>
    <script type="text/javascript" src="/js/jquery.js"></script>
    <script type="text/javascript" src="/js/jquery-dimensions.js"></script>
    <script type="text/javascript" src="/js/left-menu.js"></script>
    ...
</head>

Loading of each file is done in a separated HTTP query, so the browser cannot show the page content before all of CSS and JS files are completely loaded. Of course, it slows down the page displaying.

The HTTP_StaticMerger library allows you to use a single URL to load a set of CSS or JS files:

Listing 2: merged query samples
<head>
    <script src="/merge.php/!!a2de6253!!12345678!!js/jquery.js!!js/jquery-dimensions.js!!..."></script>
    <link rel="stylesheet" type="text/css" href="/merge.php/!!3ad3b1f8!!12345678!!css/common.css!!css/menu.css!!..."></script>
</head>

Usage samples: the simpliest case

First, create the file merge.php in your DOCUMENT_ROOT:

Listing 3: /merge.php — merging handler
<?php
require_once "../lib/HTTP/StaticMerger.php";
$merger = new HTTP_StaticMerger('a-secret-and-constant-string');
$merger->main('windows-1251'); // It is desirable to specify files encoding.

Second, in your page displaying code enumerate CSS or JS files to be merged:

Listing 4: insertion or merging URL
<head>
  <?
  $merger = new HTTP_StaticMerger('a-secret-and-constant-string');
  echo $merger->getHtml("/merge.php/", array("js/jquery.js", "js/jquery-dimensions.js", ...));
  ?>
</head>

While debugging it is useful to temporarily disable static files merging. To do that, pass the third pamameter to getHtml() method equal to true. Then, each file will be represented as a separated <script> or <link> tag:

Listing 5: the third argument is true — no merging at all
echo $merger->getHtml("/merge.php/", array("js/jquery.js", ...), !empty($_GET['no_merge']));

That was an essential minimum.

Nginx: gzip compression and caching

If you plan to use HTTP_StaticMerger in a heavy loaded project, please do not try to do it with a "brute force". Note that merging procedure is performed in each HTTP query and tooks significant time (usual server performs about 100-200 of merging queries per second).

To lower the server load, use caching reverse proxy in front of web-server (e.g. nginx proxy). Let it cache "forever" all requests to merge.php:

Modify merging handler

Listing 6: /merge.php — merging handler with gzip compression
<?php
require_once "../lib/HTTP/StaticMerger.php";
ob_start();
$merger = new HTTP_StaticMerger('a-secret-and-constant-string');
$merger->main('windows-1251');
$c = ob_get_clean();
if ($c && empty($_SERVER['HTTP_NOGZIP'])) ob_start(array('ob_gzhandler', 9));
echo $c;

Creating your cache directory

Listing 7: First create your cache directory.
mkdir -p /var/cache/nginx
chown nginx:nginx /var/cache/nginx

Configuration for nginx+apache+mod_php

Listing 8: Configuration for nginx+apache+mod_php
# Enable caching for merged files.
proxy_cache_path /var/cache/nginx levels= keys_zone=merge:10m;
location /merge.php {
    set $no_gzip 0;
    if ($http_user_agent ~ "MSIE [4-6]\.|Safari|Konqueror") {
        # These browsers have bugs while processing gzip-encoded files with LONG URLs.
        set $no_gzip 1;
    }
    if ($http_accept_encoding !~ "gzip") {
        set $no_gzip 1;
    }
    proxy_pass   http://127.0.0.1:80;
    proxy_set_header Host $host;
    
    # Nginx caching.
    proxy_cache merge;
    proxy_cache_valid 200 304 404 240h;
    proxy_cache_key "$no_gzip|$request_method|$http_if_modified_since|$http_if_none_match|$host|$request_uri";
    proxy_set_header NOGZIP $no_gzip;
}

For dummies 

Attention! MS Internet Explorer 6.0 (and older) has problems while caching gzip-encoding content, if a CSS/JS file URL is longer than 215 characters. We turn off the compression in these browsers.

Or - configuration for FastCGI PHP version

Listing 9: Configuration for FastCGI PHP version
# Enable caching for merged files.
fastcgi_cache_path /var/cache/nginx levels= keys_zone=merge:10m;
location /merge.php {
    set $no_gzip 0;
    if ($http_user_agent ~ "MSIE [4-6]\.|Safari|Konqueror") {
        # These browsers have bugs while processing gzip-encoded files with long URLs.
        set $no_gzip 1;
    }
    if ($http_accept_encoding !~ "gzip") {
        set $no_gzip 1;
    }
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    
    # Nginx caching.
    fastcgi_cache merge;
    fastcgi_cache_valid 200 304 404 240h;
    fastcgi_cache_key "$no_gzip|$request_method|$http_if_modified_since|$http_if_none_match|$host|$request_uri";
    fastcgi_param HTTP_NOGZIP $no_gzip;
}

For dummies 

See details in
nginx documentation.

Using together with mod_rewrite

Of course you may beautify merged URLs with Apache mod_php or nginx "rewrite" directive, like these:

/merge.php?some=param&other=!!a2de6253!!12345678!!js/jquery.js!!...
/merge/!!a2de6253!!12345678!!js/jquery.js!!...
/m/!!a2de6253!!12345678!!js/jquery.js!!...

The last sample implies that you use the following mod_rewrite directive:

Listing 10: better merging URL format because of mod_rewrite
RewriteRule ^m/(.*) merge.php/$1

If it does not work, try using "?" in resulting URL:

Listing 11: better merging URL format because of mod_rewrite
RewriteRule ^m/(.*) merge.php?$1

For dummies 

Try to avoid "?..." part in merged URLs, because some of old browsers disable client caching for such URLs.





Dmitry Koterov, Dk lab. ©1999-2016
GZip
Add to Del.icio.us   Digg It!   Reddit