Understanding if Using a Public CDN is Right for You?

I created an ASP.NET 5 project recently, and one of the things that caught my eye was this segment of code in the _Layout.html page.

First, in the document head for our CSS (click on to enlarge and be able to read)

And then later for our scripts at the end of the body

What each of these code segments is doing is using the environment tag helper class to insert the appropriate CSS and JavaScript into the document respectively.  In Development, all of the CSS and JavaScript will be served from the local web server.  However, in Staging and Production, the third party libraries of Bootstrap, jQuery, hammer and the Bootstrap touch carousel will be served from the ASP.NET public CDN.  This is what the URL's to the aspnetcdn.com domain are all about.

This project was created with the default template that ships with the release candidate of Visual Studio 2015.  So things could change by the final release, but since this is a release candidate, one would think this code would closely resemble what will actually ship with the final version.

So what is a public CDN and when is it appropriate to use one?  If you have read The Pragmatic Programmer, you will remember one of the tips in the book is that when code is generated by a wizard, you need to fully understand what that code does, because now that code is in your project and you are responsible for it.  In the case, the code comes from the default template, but the principle is the same.  We need to understand what a public CDN is and if this is really the right choice for our project to use.

What is a Public CDN

A public CDN is basically, it is a website that hosts popular CSS frameworks and JavaScript libraries that you can link to and use in your pages.  Take for example a library like jQuery.  We are accustomed to seeing a script tag like this:

<script src="/scripts/external/jquery-1.11.3.min.js"></script>

When the browser processes this script tag, it will download jQuery from your web server.

However, with a public CDN, we are going to change the URL of our script tag to point at the location of the correct version of jQuery on the CDN server.  So now we will have something that looks like this:

<script src="//ajax.aspnetcdn.com/ajax/jQuery/jquery-1.11.3.min.js"></script>

The difference is that when the browser encounters this script tag in our HTML document, it is not going to download jQuery from our web server, but from a Microsoft web server for the ASP.NET CDN.

Microsoft is not the only company to have a public CDN server that is available to be used.  There are others, and I'm listed some of the most popular below.  They vary in terms of what libraries each on has available, with Cloudflare having the most extensive collection at the time of writing:

• ASP.NET (Microsoft) - http://www.asp.net/ajax/cdn
• Google - https://developers.google.com/speed/libraries/
• cdnjs - https://cdnjs.com/
• jQuery - https://code.jquery.com/

So in a nutshell, you are handing over the hosting of these popular libraries to one of these CDN providers rather than hosting the files yourself.  And there are some clear benefits to that as we will see in the next section

Benefits From Using a Public CDN

There are some clear benefits to using a public CDN.

• We have shifted the workload for serving these files from our web server to the CDN server, which saves us a little bit of load on our server and most importantly network bandwidth.  
• Browsers limit the number of simultaneous connections they will make to a single web server.  So by having some resources loaded from a CDN server, we are increasing the number of simultaneous resources that can be downloaded.
• Every major CDN provider has CDN servers distributed around the world.  So this means the user will be downloading that copy of jQuery from a server that is relatively nearby, not one across the country or across the world.  This helps to decrease network latency, the time it takes for packets to travel over the network from one place to another.
• All of the major CDN's will add the appropriate cache headers to their responses, so that any library loaded from the CDN will be cached for a year, which will save bandwidth on subsequent requests the user makes to the site.
• If multiple sites use all use the same library version on the same CDN, then this library will be cached by the browser when the user visits the first site.  Then, when the user visits site 2 and sees the same URL for that library, it doesn't need to download the library because it already has the contents of that URL cached in the browser.  In this way, the more sites that use a CDN the better, because there is a possibility that one of the common JavaScript libraries you use on your site is already cached in the browser due to the user requesting that exact same CDN URL from another site they have visited.

So in a nutshell, a CDN is a just a web server that hosts these popular libraries that you can then reference from your web pages, which saves your web server from having to serve these files yourself.

Is There a Downside?

When I first learned about public CDN's, I felt there was literally no reason not to use a public CDN for every website that I worked on.  As I have learned more and carefully studied the issues related to web performance, my enthusiasm for public CDN's is much more measured, and I'll describe why below.

The first thing we need to understand is that we have now introduced an additional network server into our hosting picture, and this is a server that we do not control.  It is true that CDN's are engineered for very high reliability, but as we have learned time and time again, every system will experience some down time.  Google goes down.  Microsoft goes down.  Facebook goes down.  Not very much, but it does happen.  And in the case of a CDN going down, this means that your pages are probably not going to be functional because they are most likely dependent on the JavaScript you are loading.

In this blog post by Scott Hanselman, he describes a situation where the ASP.NET CDN server in the Northeastern United States went down for a period of time.  The good news is that because CDN's are distributed around the globe, this failure only affected user's in the Northeastern United States.  The bad news about this type of failure is that they are much harder to diagnose.  Why does it work for the user in California and not for the user in Massachusetts?  When these types of partial failures happen, they are difficult to diagnose.

So the bottom line is that there will be some small amount of down time if you use a public CDN.  It won't be much, but making use of a public CDN means that there is another component in your system that needs to be up in order for your system to be up.

If you read the article, you will see some JavaScript code that will check to see if a library loaded and if not, server a local copy of the library to the browser.  This code generally looks something like this:

<script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-2.0.0.min.js"></script>
<script>
if (typeof jQuery == 'undefined') {
    document.write(unescape("%3Cscript src='/js/jquery-2.0.0.min.js' type='text/javascript'%3E%3C/script%3E"));
}
</script>

If you search the web, you will find many similar code fragments.  What is important though is to understand how a public CDN fails and how this code fragment responds.  If a web server is down (and this includes a public CDN), usually you do not get a failure status back right away.  Instead, the browser waits for the request to the server (the public CDN) to time out.  Depending on the browser, it will wait between 20 and 120 seconds to timeout, and only then fallback to the local script.

One of the important rules about web performance is that the browser will not render any DOM elements that are below a script tag that has not finished loading.  So if you have the code above in your document head, that means if the public CDN is unreachable that the user will be staring at a blank screen for 20 to 120 seconds before the fallback script fires and serves the local version of the script.  Even if your script tags are at the bottom of your HTML body section as they should be, it means your page is probably missing major functionality for 20 - 120 seconds.  The user of course will now know that the public CDN is not reachable, so to them, the page will appear partially (or perhaps completely) non-functional, and they may very well decide to leave in that period of time.

Again, the public CDN's are designed to be highly available, but the point is, if for any reason the CDN server becomes unreachable of the network, there are going to be serious consequences for your page.  Network connectivity is something we tend to take for granted until we don't have it.

Re-evaluating the Benefits

The main benefits we outlined above was that we save our web server from needing to serve these common CSS frameworks and JavaScript libraries, which ultimately saves load and bandwidth on our web server.  But here is another important point to consider.  CSS and JavaScript files are static files and as such, we should be setting the caching headers in the responses for these files such that the only time they have to be loaded is on the very first page view they make on our site.  After this, on every other page the user browses to, the browser will issue a conditional request for these resources of which the web server will send back an HTTP 304 response (Resource Not Changed) which is only a single packet.  So using a public CDN really means that we are going to have some savings on the very first page the user visit's on our site, but after that, the savings are basically negligible.

So what about the advantage of public CDN's allowing more files to be downloaded simultaneously due to resources being loaded from multiple servers.  This is still an advantage.  However, you can (and should) look at bundling together your CSS and JavaScript files, which has the effect of reducing the number of requests the browser has to make to download your page.  So this is really just a different way of solving the problem.

Finally, what about the fact that if many sites on the web use the same library version from the same CDN, there is a probability that the user will already have the library cached when they visit your site.  This is true that this can still happen, but in reality, there are so many different versions of a library like jQuery in use today, many different CDN's and many sites that still server the library themselves that in practice, the probability of the library from the CDN already being cached is very low.

What we are left with is that CDN's are geographically distributed around the world.  If you are running a public site on the internet that receives hits from around the country or around the world, this is an important advantage.  If most all of the traffic from your site comes from a local area, then this is not so important.

The point here is that if you implement good practices like caching the static content on your website and bundling and minifying your JavaScript, the advantages of using a public CDN are not nearly so great.  So what we are left with then is evaluating these smaller advantages against the risk of the CDN being unreachable.

So When Is a Public CDN the Right Choice

If you are running a public website, especially a website that is hosted in the cloud, and you have visitors from across the country or across the world, then I think a public CDN makes sense.  You will get some lift from the geographic distribution that public CDN's offer.  You also have to consider that in many cloud hosting scenarios, you are charged based on the bandwidth that you use, so any amount of bandwidth you can off load is important.

Likewise, if you are hosting a site that has bandwidth limits on the amount of data you serve, a public CDN again makes a lot of sense because you want to conserve every KB of bandwidth you can.  And if you have a web site that is nearing capacity in terms of the load it serves, using a CDN can be a quick way to transfer some of the load to another server while you work on a longer term solution.

If you are developing an internally facing (intranet) web application, I think that using a public CDN is the wrong choice.  Your internal web app probably accesses mostly other internal resources (think databases and web services within your firewall).  If you add in a public CDN, you are now reliant upon a component outside of your firewall.  If for any reason your data center experiences network connectivity issues with the internet, your internally facing app is now going to be impacted.  This doesn't have to be the CDN going down.  It could be your ISP having issues or even a component failing in your network room.  In this scenario, I think all you are doing is adding risk by using a public CDN for these apps.

Plus, for an internal web application, you have a set of user's that is nearly constant, and most likely in only a couple of locations.  So all of your CSS and JavaScript will already be cached in these user's browsers and we don't really have to solve the problem of first page views from different geographic locations.  So in the internal application scenario, I don't think using a public CDN makes any sense.

There are clearly many other scenarios, and ultimately you will end up having to analyze for yourself if a public CDN makes sense or not for your application.  This post should help you think through what some of the advantages are and if those advantages will really apply to you.  If you have already implemented caching of your static content and bundling of your CSS and JavaScript, then any lift you get from a CDN is probably going to be on the smaller side.  So what you have to do is balance this against the potential implications of if the CDN is unreachable for any reason.  For cloud hosted apps or apps that server users across a wide geographic region, this tradeoff is probably worth making.  In other cases though. the potential gain is so small that you will probably be better off just serving these files on your own.

An Approach to Bundling and Minification is ASP.NET 5

Microsoft has made significant changes in how ASP.NET 5 works, and one of these changes is around how bundling and minification is handled.  The bundling and minification API is no longer available, and instead, you need to set up gulp tasks to bundle and minify your files.

Lets quickly summarize the capabilities the bundling and minification API in ASP.NET 4.5 offered us:

1. CSS and JavaScript files could be combined into bundles
2. CSS and JavaScript files could be minified
3. An Expires header was added to each bundle response so that the bundle would be cached by the browser
4. A SHA1 hash was added to the URL of the bundle to act as a cache breaker.

So far, the approaches I have seen to this problem only cover points 1 and 2.  They bundle and minify the files, but they do not address anything with caching.  And this is really important.  CSS and JavaScrpt files change somewhat infrequently, so we want the browser to cache them, especially since we are using more and more JavaScript in our apps to deliver highly interactive experiences. 

What is also important though is that any solution we come up with addresses point #4 above.  By taking a SHA1 hash of the bundle and incorporating it into the URL, if any file in the bundle is changes (or if a file is added to or removed from the bundle), then the URL of the bundle changes and the browser will know to download the new version of the bundle.  This is critical, because CSS and JavaScript that we write will inevitable have bugs or need features added, and when we do this, we need to make sure the browser knows to download the new bundle.

What follows is the approach that I developed for my new Pluralsight course "Improving Website Performance with PageSpeed Insights" (to be release mid-July 2015).  I do expect that as time goes on, I'll update this approach.  But this will at the minimum provide you a starting point for how to solve this problem.

Step 1 - Setting up Gulp

The first thing you need to do is edit your packages.json file in order to pull in the needed npm packages.  This file is in your project root, and when done should look like this

{
  "name": "ASP.NET",
  "version": "0.0.0",
  "devDependencies": {
    "gulp": "3.8.11",
    "rimraf": "2.2.8",
    "gulp-concat-css": "2.2.0",
    "gulp-concat": "2.5.2",
    "gulp-minify-css": "1.1.6",
    "gulp-uglify": "1.2.0",
    "gulp-hash": "2.0.4",
    "gulp-rename": "1.2.2",
    "event-stream": "3.3.1",
    "gulp-extend": "0.2.0",
    "gulp-clean": "0.3.1"
  }
}

What is important here are the lines in red, as that is what you are needing to add.  Note that by the time you read this, version numbers may have changed.  Each of these packages though will perform a very specific task, and what we will do in the next step is set the pipeline to do this.  For now though, lets understand why we are including each of these packages.

• gulp-concat-css - Used to concatenate (bundle) CSS files together
• gulp-concat - Used to concatenate (bundle) JavaScript files together
• gulp-minify-css - Used to minify CSS files
• gulp-uglify - Used to minify JavaScript files
• gulp-hash - Used to generate a SHA1 hash for a file and embed that hash in the filename
• gulp-rename - Used to copy files (the name says rename, but it really copies files)
• event-stream - Used to help create the pipeline of events below.
• gulp-extend - Used to merge the contents of JSON files.  Used to create our manifest below
• gulp-clean - Used to clean directories so we can start fresh for every build

Once we save this file, Visual Studio 2015 will automatically download these packages into your solution.

Step 2 - Creating a Gulp Task to Bundle and Minify Our Files

Next, we need to edit the file gulpfile.js (also in the project root) to create a new gulp task.  Eventually, we'll set this task up to run every time we build, but for now, lets concentrate on the code.

I added the following code at the end of the file

paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];



function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}


function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}


function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}


gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});

This is kind of a long segment of code, so lets break it down into some smaller segments and take it step by step to see what it does.

The first section is just setting up some directory paths that we are going to need.  The paths object is actually defined earlier in the file, I am just adding some paths to it.

paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

The next section is pulling in all of those packages that we added to our project earlier and storing them in a variable so we can make use of them later.

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

This third section is setting up the options for our hashing package.  We are going to use the SHA1 algorithm and we want all 40 bytes of the hash to be embedded into the filename.  This is important to include the entire hash to make sure we avoid any hash collisions.

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

In the fourth section, I am defining some JSON objects that define my bundles.  In this example, I have just one CSS and one JavaScript bundle, but a real project would of course have more.  It would also be better in my opinion to load this out of a separate config file, but for this demo, I didn't go that far.

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];

Next, we have two helper functions that create the CSS and JavaScript bundles respectively.  If you have done any shell scripting in Unix or Powershell, this is pretty similar.  We have a bunch of small, individual commands that pipe their output to one another to accomplish a bigger task.

So what each of these do is

• read in the source files
• concatenate them together
• output a bundled file (not minified yet -- we'll use this version while developing)
• make a copy of the file with the ".min" in it that we can operate on
• minify this file
• use the hash package to create a SHA1 hash of the file and embed that hash in the filename
• output that file to our bundles directory

All this is wrapped in the addToManifest() helper function so we can add an entry to our manifest file.

function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}

function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}

The next function is the helper function that is used to create the manifest file.  We'll need this later, because we will need to translate a bundle name into the name of the file with the hash embedded into it.  This code I took from documentation page for the gulp-hash project on npm.

function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}

Now finally, we have our gulp tasks.  The first of these cleans (removes) all of our existing bundles, and the second is the task that will build the bundles.  One thing to note here.  The cleanBundles task is a dependency of the bundleFiles task, so every time bundleFiles gets called, cleanBundles will be called automatically.

gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});

Step 3 - Running the bundleFiles task and Adding to the Build Process

Visual Studio 2015 contains a new window called Task Runner Explorer where you run can run these tasks from.  To open this window, in the Visual Studio menu go to View --> Other Windows --> Task Runner Explorer (its about in the middle).

From there, you will see all of your Gulp tasks.  If you want to run your task directly, just click on it and say run.

What you really want to do though is set your tasks up to run whenever you build, and to do that, you again right click on the task, go to bindings and make sure that "After Build" is clicked for the bundleFiles task.  

Step 4 - Creating a TagHelper To Get the Bundles Into Our Pages

Where we are at now is that our bundles are created and minified and they are sitting in a bundles directory below our application.  What we have to do now is get our pages to use these bundles.  

I've seen a couple different approaches taken here.  Many people are using the environment tag helper.  This wasn't going to work for me because I have different filenames I need to account for.  And I think it is going to be a bit of a stretch that everyone will set an environment variable on their web servers at this point.

So I need a couple of pieces here.

First, somewhere to tell my app that it should use the minified or unminified content.  For this, I chose to put a value in appSettings.  Low tech, yes, but this works.  So here is my config.json file:

{
  "AppSettings": {
    "SiteTitle": "AspNet5Bundling",
    "UseMinifiedContent":  true
  },
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=aspnet5-AspNet5Bundling-c7120031-310d-47a6-b645-970e20afc890;Trusted_Connection=True;MultipleActiveResultSets=true"
    }
  }
}

Then, I need something that is going to look at this and output the appropriate HTML based on what I need.  This is where a custom tag helper comes in.  Basically, a custom tag helper just outputs some HTML for you.  So what I am going to have in my cshtml files will look like

        <cssBundle bundle-name="site-css-bundle" bundle-dir="~/bundles"></cssBundle>

So in this case, cssBundle corresponds to the class CssBundleTagHelper, and we are passing in two items, the name of the bundle we want and the virtual path of where that bundle is.

So lets take a look at the tag helper class that processes this.

using System;
using Microsoft.AspNet.Razor.Runtime.TagHelpers;
using Microsoft.AspNet.Mvc;
using AspNet5Bundling.Util;

namespace AspNet5Bundling.Tags
{
    [TargetElement("cssBundle", Attributes = BUNDLE_ATTRIBUTE_NAME)]
    [TargetElement("cssBundle", Attributes = BUNDLE_DIRECTORY)]
    public class CssBundleTagHelper : TagHelper
    {

        public const String BUNDLE_ATTRIBUTE_NAME = "bundle-name";

        public const String BUNDLE_DIRECTORY = "bundle-dir";

        private const String LINK_TAG_TEMPLATE = "<link href=\"{0}\" rel=\"stylesheet\"/>";

        [Activate]
        public BundleConfig BundleConfig { get; set; }


        [HtmlAttributeName(BUNDLE_ATTRIBUTE_NAME)]
        public string Name { get; set; }

        [HtmlAttributeName(BUNDLE_DIRECTORY)]
        public String Directory { get; set; }

        
        public override void Process(TagHelperContext context, TagHelperOutput output)
        {
            // Always strip the outer tag name as we never want <cssBundle> to render
            output.TagName = null;
           
            String bundleFileName = this.BundleConfig.GetBundleFileName(this.Name, BundleType.css);
            String cssBundlePath = String.Format("{0}/{1}", this.Directory, bundleFileName);
            output.Content.SetContent(String.Format(LINK_TAG_TEMPLATE, cssBundlePath));            
        }

    }
}

Starting off, we have two TargetElement attributes on the class, and these help define the attributes that we can pass into this tag.  This is what tells ASP.NET what attributes this tag can accept.  Further down in the class, you see two properties called Name and Directory that have an attribute of HtmlAttributeName, and this is what links up the HTML attributes with the properties.  So this is how we get those parameters supplied on our web page into the specific TagHelper object that is running.

Then what happens in the Process() method will get called, and the job of the Process() method is to output whatever HTML you need to into the TagHelperOutput object.  By setting the TagName equal to null, this tells ASP.NET to throw away the cssBundle part of the tag.  In this case, we don't want cssBundle at all, we want to completely replace it.  Then, it is using a helper class to get the path you our CSS bundle (more on that in a minute) and it just interpolates that into a <link> tag.

So the TagHelper isn't that complicated.  It is taking a bundle name, using a helper class to look up the name of the bundle file it wants to use and then producing a link tag.  I have a jsBundle tag helper class that works exactly the same, but produces a <script> tag instead.

So what about this BundleConfig  class and the GetBundleFilename() method.  What do they look like?

using Microsoft.Framework.ConfigurationModel;
using System;
using System.Collections.Generic;

namespace AspNet5Bundling.Util
{
    public class BundleConfig
    {

        public BundleConfig(IConfiguration siteConfiguration, IConfiguration bundleConfig)
        {
            String useMinified = siteConfiguration.GetSubKey("AppSettings")["UseMinifiedContent"];
            if (!String.IsNullOrWhiteSpace(useMinified) && String.Equals(useMinified, "true", StringComparison.CurrentCultureIgnoreCase))
                this.enableMinification = true;

            this.bundleMapping = new Dictionary<string, string>();
            foreach (var item in bundleConfig.GetSubKeys())
            {
                String key = item.Key;
                String value = bundleConfig[key];
                this.bundleMapping.Add(key, value);
            }
        }


        private bool enableMinification = false;
        private Dictionary<String, String> bundleMapping;


        public String GetBundleFileName(String bundleName, BundleType bundleType)
        {
            if ( enableMinification )
            {
                String baseBundleName = bundleName + ".min." + bundleType.ToString();
                if ( this.bundleMapping.ContainsKey(baseBundleName) )
                {
                    
                    String bundleNameWIthHash = this.bundleMapping[baseBundleName];
                    return bundleNameWIthHash;
                }
                else
                {
                    throw new Exception(String.Format("Unable to find {0} bundle with name {1}", 
                        bundleType.ToString(), bundleName));
                }
            }
            else
            {
                String bundleFile = bundleName + "." + bundleType.ToString();
                return bundleFile;
            }
        }
    }
}

In the Startup.cs file, I am creating a BundleConfig object as a Singleton so the same object will always get used when a class needs it.  As you can see, in the constructor, it is looking fo rthe appSettings property to see if we should use minification and stores that as a member variable.  It also reads the manifest file in the bundles subdirectory, so it knows what bundle name goes with what file.

In the GetBundledFileName() method, it looks to see if minification is enabled.  If so, it needs to look the name of the file up in the manifest entries.  This has to be done at runtime, because the hash could be different, and of course we don't want to go update all of our pages everything a bundles hash changes.  If the file is not minified, it knows the pattern of the files, so it just constructs the appropriate filename based on this pattern.

What This Gives You

We can now bundle and minify our CSS and JavaScript files.  Every time we build, these bundles will be created, and the minifed bundles will have a SHA1 hash embedded in them to act as a cachebuster, so if we update a CSS or JavaScript file, we know the browser will get the new version.

Analysis

I'll be the first to say, this is one approach, and I think I'll refine this approach over time.  There are things I am happy with and things I think could be better.

I am happy that I was able to embed the SHA1 hash as a cachebuster in the filenames.  I think this is an oversight in other solutions I have seen, and it may be a surprise to people when the browser is not loading their latest JavaScript.  So I think this is really important.

I like the custom tags of cssBundle and jsBundle I created.  I think these are succinct and to the point of what I am doing.  Put this bundle of files here.  Simple and easy to understand.

I'm not happy that in my BundleConfig class, I am constructing some Strings to look up the bundle or just create the filename based on a pattern.  I think by revising my gulp tasks, I could produce a better manifest that would take the magic strings out of this, so when I get some time, I want to revisit this.

As I said above, I would like to pull my bundle definitions out of the gulpfile.js and into a config file, maybe bundle.config.  Right now I am mixing code and configuration, which is never good.

So there is some work to do, and as I revise this, I'll update this post so you can see the revisions (this post will remain though, as the link in my Pluralsight course points here).  But this will hopefully give you a good start as you tackle this problem.  Many have argued that the approach ASP.NET 5 is taking in this regard is better.  I am not ready to make that assertion.  I think ultimately, this is more flexible.  But the old approach was simple and you could be up and running in 5-10 minutes.  This new approach is not that way, at least not yet.  I spent a lot of time figuring this out, part of which was due things still being changed in ASP.NET 5, part of it due to the new approach.  So if nothing else, I hope this saves you some of the trouble that that I went through in figuring this out.  I think better solutions will emerge in time, but better solutions are often built on earlier solutions, so we have to get that conversation started.

Using Cassette to Bundle and Minify Files in MVC2, MVC3 and Older WebForms Apps

Over the last few years, we have seen the emergence of highly interactive web applications.  In order to achieve this level of interactivity, most web pages now contain multiple JavaScript and CSS files that power this behavior.

As the number of files required by a web page goes up though, the performance of the page can be degraded, because the browser has to load each one of these dependent files from the web server.  This impacts performance in three ways

• Number of Downloads - Each separate file is a round trip from the browser to the server, and each round trip will incur network latency, the amount of time it takes for a packet to travel from the browser to server and back.  Downloading many small files is generally sower than downloading a single large file due to the impact of network latency.
• Download Size - Multiple files to download adds up in terms of the amount of data that must be downloaded by the browser from the web server.  Simply put, larger pages take longer to download and ultimately display to the user, so we want to trim download size wherever possible,
• Number of Concurrent Downloads - This is something that is not realized by most developers, but your browser will only download so many files concurrently from a given host at a time.  For most browsers, the limit is 6 concurrent downloads.  For IE 11, the number is 13 (check out all browsers at http://www.browserscope.org/).  What this means is that if you have 20 files that need to be downloaded from your website, only N will downloaded at a time, and the remaining files will queue behind these N files.  When a file from that group finishes, then the next download can start.  So if we can minimize the number of items that need to be queued, we can download and render our page faster.

Minification and Bundling

So how does minification and bundling help these problems?

Mnification - Looking at any JavaScript file, you will notice that there are lots of spaces, line feeds and comments in the file.  As a developer, these are good because this is what makes the file readable for when we have to work with it.  But from the point of view of the JavaScript interpreter in the browser, this information is extraneous.  It doesn't care if there is one space or eight, so long as it can parse the file.  So by removing these bytes from the file, we can shrink the size of the file, which means fewer bytes have to be sent over the wire from the server to the browser.  And these savings can be dramatic.  For example, the minified version of jQuery 2.1.3 is only 34 KB, where as the non-minified version is over 87 KB (click on the links to see the differences between a minified and non-minified file).  And minification doesn't just apply to JavaScript files.  You can see dramatic savings in CSS as well.

Note that minification is not compression.  Compression is the process of encoding information such that statistical redundancies in the data are reduced such that data can be represented in a shorter, more concise format.  Minification is the process of removing extraneous information from the file.  So what we ultimately want to do is first minify our JavaScript and CSS files and then compress them using HTTP compression so that we minimize the number of bytes that need to be sent over the wire.

Bundling - Bundling is the process of concatenating multiple files together so they can be downloaded as a single file.  Lets say that you site makes use of three CSS files to define styles.  And we want to keep these as three separate files because this makes the editing and maintenance of these files easier.  But, this means that the browser now has to perform three separate downloads in order to get each of these files.  As we said above, we are going to incur additional network latency by having these as three separate downloads, and these three files will all count against our concurrent download limit, which may block the browser from starting to download other resources required by the page.

The answer here is bundling.  These three files can be concatenated together such that now only one file needs to be downloaded by the browser.  The total size of the file will be just the sum of the size of each individual file, but we save in terms of not having separate downloads for each one and by freeing up some of the concurrent network connections the browser has to perform other work, like downloading other resources

How Do I Accomplish This?

You could manually minify your JavaScript by using a tool like UglifyJS.  And many people do just that in their build process.  Bundling presents a bigger challenge though, because we really want separate files when we are developing, and then only to combine them at deployment or run time.   It is possible to build this into your deployment process with a number of scripts, but this is a hassle to maintain.

In ASP.NET 4.5, Microsoft introduced the Bundling and Minification API, which is well covered in this excellent article by Rick Anderson.  The bundling and minification allows you to define JavaScript and CSS files that should be bundled together in your code, and then at run time, it will automatically minimize, concatenate and serve these files for your site.  The beauty of this is that it all happens transparently to you.  You work with files as you normally would in development, and then with some simple configuration, they are automatically optimized at run time.

However, many web projects exist today that are on older versions of Microsft frameworks.  Yes, there are many MVC2, MVC3 and older WebForms apps out there.  And due to constrained IT budgets, higher priority projects and most of all time, it is not always possible to simply lift these projects up to the latest version of the framework.  So these new features are out of reach for your apps using older frameworks.  Or are they?

Enter Cassette

Cassette is a package created by Andrew Davey that brings the same bundling and minification functionality available in ASP.NET 4.5 to prior versions of ASP.NET.  While the syntax varies somewhat, the concept is the same.  We can define 'bundles' of either CSS or JavaScript files in our ASP.NET application, and at runtime, these files will be bundled together and minified, thereby increasing the performance of our web site.  Lets take a look how this happens.

First, we want to add Cassette to our application.  This is most easily done with the NuGet.  The package you want is called Cassette.AspNet.  You can install this package using the GUI:

Or you can use the Package Manager Console where you will see something like this when you install:

Once Cassette is installed, you will see a new file in the root directory of your ASP.NET called CassetteConfiguration.  You want to edit this file to configure the various bundles needed by your project.

Defining Bundles

The next step is to define how you want to bundle various files together for your site.  A bundle can be either a Stylesheet bundle or a Script bundle, but not both.  What you want to do is think about how your stylesheet and JavaScript files logically map to pages.  If you have multiple stylesheets that are included in your master page or master layout view, then a bundle that contains all of these files makes sense.  A similar bundle for all of your JavaScript files that are on your master page makes sense as well.  Then you may have additional bundles that represent scripts that are only present on a subset of your pages.

Bundles are defined in the CassetteConfiguration.cs file that was shown above.  There are a number of different ways to define a bundle, but I prefer to create a List of the files to be included in the bundle and then add that list to the BundlesCollection object as shown below.

Here, I am defining three bundles.  The first contains all of the CSS files that are in my master layout view, the second all of the JavaScript I include in my master layout view and the third a single script used for working with Google Maps on one of the pages in my site.

You might ask, why would I create a bundle for a single file, because after all, the word bundle implies there should be multiple files.  What I am after here is minification of this JavaScript file.  By including this file in a bindle, Cassette will automtically minify the file for me at runtime.  So now, I can work with a readable file like normal in Visual Studio, but be assured that Cassette will take care of minification when the time comes.  Further, we will see in a bit that Cassette also adds a cache header for each bundle, which further improves performance.

When you are calling the bundles.Add() method, you will see intellisense in Visual Studio as follows:

The first argument is what Cassette calls applicationRelativePath.  Indeed, you can use this to point to a path in your application in order to include files in the bundle.  However, also note that it says this does not to be a real directory path.  And that is what I am doing.  I am using this parameter to give a meaningful name to the bundle, a meaningful name we will use in a moment when we go to include the bundle in one of our web pages.

There are a number of other ways to create your bundles, like providing a subdirectory name and allowing Cassette to create a bundle of all of the files in that subdirectory.  I like this method though, where I explicitly define the files in a collection and then add them to the bundles, giving each bundle a meaningful name.  I think this makes it very easy for someone else to follow what I am doing.  Do know though, there are other options available, and these are covered in the Cassette Documentation.

Adding Bundles To Our Web Pages

In this article, I am working with an MVC3 project.  You can however use Cassette on MVC2 and WebForms projects.  The syntax will be slightly different, but the concepts are the same.

In my View, first you need to add a code block at the top that defines the stylesheet and script bundles you are going to use in the view.  Note, you can do this in both a master layout view and an individual page view (the example below is actually in my _Layout.cshtml file).

Then, you call Bundles.RenderStylesheets() and Bundles.RenderScripts() at the points in your page where you want the stylesheet and script bundles to appear respectively.

For stylesheets, these go in the <head> tag

And it is best for performance is your scripts are placed just before the closing body tag.

There is a way that you can have scripts render in different places in your HTML page, but I will save how to accomplish that for a later blog post.

Turning on Cassette

There is one last step, and that is to turn on Cassette.  For Cassette to bundle and minify your files, you need to set the debug flag to false.



If the debug flag is set to true (as it might be for building on your local machine), then Cassette will just output the links to the regular version each CSS and JavaScript file.  This is useful for when you are debugging and might need to debug through some JavaScript code.  But in your production environments, you would have debug set to false to get the full benefit of bundling and minification (among other things).

So how does this look when things are working.  Here is the view from the Network tab in the Chrome Developer Tools.

So we can see what Cassette is doing.  It is using an HTTP Handler to service the request, and this handler bundles and minifies all of the associated files for this bundle.  Again, the nice thing is this all happens to us transparently as developers.  Cassette puts the correct links in each of our pages.  All we have to do is set the bundles up correctly.

But Wait, There is More...

One of the things Cassette also does is add the appropriate caching headers to your bundles that are sent down to the browser.  This means once a browser has downloaded the bundle the first time, it will not have to download it again for another one year.

So in the case above where we have bundles named MasterCss and MasterScripts containing the CSS and JavaScript files we use on every page in our site, these bundles will only be loaded once, not on every page a user navigates to while visiting our site.  And if they come back and visit our site again tomorrow or next week, again, these bundles will be caches locally on their browser and not need to be reloaded.  By caching these assets on the browser after the first page load, we'll save significant bandwidth on ever subsequent page load.

What happens though if we need to change one of these files?  Perhaps we find a bug in our JavaScript or need to change our stylesheets to support a new color scheme?  How does the browser know that a particular bundle has changed so that it should be downloaded again?

What Cassette does is calculate a SHA1 hash over the contents of the bundle, base 64 encodes this hash and then embeds this base 64 encoded hash value in the name of the bundle.  That is the long string in the name of the bundle in the screeenshot in the previous section.

In this way, if the contents of any of the files in the bundle changes, the computed SHA1 hash will change and hence the name of the bundle will change.  When the browser sees this new bundle name, it will realize it does not have this version of the bundle cached and download the new bundle from the web server.  This makes sure that for any changes that you need to make to your files, the browser will always download the correct version, and saves us from manually having to version our files and manage this process within our web application.

Summary

Using Cassette, we get all of the benefits of bundling and minification in our ASP.NET projects on earlier versions of the framework.  We can continue to work with the unminified, separate versions of each file while developing the project, and Cassette will automatically optimize these at runtime.  Further, each bundle will include the appropriate cache headers such that the bundles contents will be cached in the browser.

From a performance standpoint, we achieve better performance in by three major elements:

• CSS and JavaScript files sent to the browser will be minimized, thereby removing unnecessary information like spaces and comments and reducing the number of overall bytes that need to be sent down to the client.
• We create bundles of CSS and JavaScript files that can contain multiple files concatenated together.  This reduces the total number of files the browser must download, which reduces the penalty associated with network latency that you have to pay for each individual file download.  Also, all browsers have a limit on the number of files they will concurrently download from a site, so bundling files together helps reduce queuing of files that must be downloaded by the browser.
• By including an aggressive cache header, a bundle will only have to be downloaded once by a browser and then can be served from cache on all subsequent page views that need that bundle.  This again reduces the number of bytes that need to be sent down to the browser.  Further, in the case where a file does change in the bundle, it is already built into Cassette to calculate a new hash value and embed the value in the name such that the browser will automatically know that it needs to download a new version of the bundle.

The example I showed here was from the MVC Music Store application.  However, I have used Cassette in production on a large consumer website that received hundreds of thousands of hits every month and it has worked flawlessly.  Of course, you want to testing of any new component like you normally would, but I can say from experience that I have had success with the package in some large scale environments.

We all know that many of these ASP.NET applications on older versions will still be around for a few more years.  So if you have one of these applications that you are responsible for, I urge you to use Cassette to get the advantages of bundling and minification to improve the performance of that application.

ASP.NET Response Headers and Unnecessary Information Disclosure

I've been listening to Troy Hunt's Pluralsight course entitled "Hack Yourself First: How to go on the Cyber-Offensive" and it has been pretty interesting.  One of the topics he covers is how ASP.NET actually discloses quite a bit of information on every request view HTTP headers.  What do I mean?  Pop open the developer tools on your browser, make a request to an ASP.NET website, and more than likely, you are going to see something like this:

This is a capture from a test app I had on my local IIS instance, but notice what is circled in red.  By looking at the response headers, you now know exactly what version of IIS the site is running, the exact version of the .NET Framework and the exact version of MVC.

Just so you don't think this is just an artifact of hitting the localhost address or something I conjured up, go to http://msdn.microsoft.com/ and use your developer tools to inspect the response headers.  This is what I get.

By default, for any ASP.NET site, all of this information is made available.  So what is the problem with that?  Well, a lot actually.  Now a potential attacker knows exactly what version of software you are running.  And they can use this information to understand what vulnerabilities have been reported against that version of software and use this information to refine their attack vectors.

How do you do this?  You can use a CVE (Common Vulnerabilities and Exposures) database, of which are freely available on the web.  For example, there is one at https://cve.mitre.org/cve/index.html, and a Google search will review numerous others.  These databases are used by security professionals to share information about vulnerabilities and measure risk exposure.  Unfortunately, they can also be used by the bad guys to understand what vulnerabilities exist in a product and plan an attack against that product.

So for an example, an attacker learns from your response headers that you are running IIS version 7.0.  Now they can go look up all the vulnerabilities in version 7.0 and craft an attack around each one.  If you have been negligent about patching, you will more than likely be exploited in such an attack.

But wait, it gets worse.  Mr. Hunt points out in his course that there are search engines out there that don't index documents and web pages, but rather devices connected to the Internet.  Once such example is http://www.shodanhq.com/.  Using such a search engine, someone can easily easily find a list of sites that are running a particular version of software.

So lets say that there is a known exploit in a particular version of ASP.NET.  What an attacker can do is get a list of all of the websites running that vulnerable version and then script an attack against all of those sites.  Sure, some of those sites may have been patched or otherwise mitigated the threat so the attack does not work.  But some of those sites will be vulnerable.  The more recent the exploit, the more sites that will be vulnerable because there will have been less time for patches to be applied or the threat to be mitigated otherwise.

All of this goes back to these headers, which making publicly available information that there is no legitimate use for.  What version of IIS, ASP.NET or MVC you are running makes zero difference to the browser.  These headers are purely informational.  The problem is that this information can be used against you.  So the best course of action is to turn these headers off.  Just like in the TV shows, if you are given the right to remain silent, then remain silent.

Turning Off Unnecessary Response Headers

I will first of all say that I really, really wish Microsoft would turn these headers off by default in future versions of IIS and ASP.NET.  That is what secure by default means.  Out of the box, we don't do things that could compromise security.  But that may never happen, so in the meantime, we have to resort to other means.

There are two blog posts I want to point out tat also contain information on this, and they are worth a read as well.

• Troy Hunt -- Shhh… don’t let your response headers talk too loudly
• Scott Mitchell -- Removing Unnecessary HTTP Headers in IIS and ASP.NET

Both Mr. Hunt and Mr. Mitchell take the approach of disabling The X-AspNet-Version header in the config and the X-AspNetMvc-Version via setting a property in code.  I'm taking a different approach here and I am going to remove the X-AspNet-Version, X-AspNetMvc-Version and Server headers all in an HttpModule.

The reason why I am going to remove all three headers in a module is because I think it is useful to build a module that can applied at the machine level in IIS rather than individually to applications.  My reason for this is that it is too easy for someone to forget to turn some of these headers off when creating a new application, and then we are right back to the unnecessary information disclosure problem.  I want something I can set up when I build an IIS machine, and then I know it is being applied to all applications on the server.  This is just a different approach I am taking.  Ultimately, you have to decide what works best for you and your environment.

So how do we develop an IIS Module to do this?  It is actually crazy simple.

1:    public class RemoveServerHeadersModule : IHttpModule  
2:    {  
3:        
4:      public void Dispose()  
5:      {  
6:    
7:      }  
8:    
9:      public void Init(HttpApplication context)  
10:      {  
11:        context.PreSendRequestHeaders += context_PreSendRequestHeaders;  
12:      }  
13:    
14:    
15:      void context_PreSendRequestHeaders(object sender, EventArgs e)  
16:      {  
17:        var headers = HttpContext.Current.Response.Headers;  
18:    
19:        headers.Remove("Server");  
20:        headers.Remove("X-AspNet-Version");  
21:        headers.Remove("X-AspNetMvc-Version");  
22:      }  
23:    }  

You just write a class that implements the IHttpModule interface.  And then, the event you want to act upon is the PreSendRequestHeaders event.  Then, it is just a matter of removing the unnecessary headers as you see.

One thing that you will notice is that this code does not remove the "X-Powered-By" header.  That is because this response header is set in IIS after ASP.NET processing is completed.  If you want to remove this header, you have to do so in IIS Manager.  How to do so is covered in both of the blog posts mentioned above, so I won't repeat how to do it here.

So now we have to add this module to an application.  You do so with the following config element (this assumes an assembly name of WebSecurity -- adjust for whatever you call your assembly or locate the module).

1:   <system.webServer>  
2:    <modules runAllManagedModulesForAllRequests="true">  
3:     <add name="RemoveServerHeaders" type="WebSecurity.RemoveServerHeadersModule, WebSecurity" />  
4:    </modules>  
5:   </system.webServer>  

All of this works just fine.  We could include the class above in one of our applications, include the config element in our Web.config file and we are off and running.

As I alluded to earlier, I wanted to be able to apply this site wide.  So how do we do that?

1. You need to be using an App Pool with the IIS Integrated Pipeline.  This is so that we can use an HttpModule.
2. Create a class library project in Visual Studio that will contain the HttpModule written above.  Note that I had to create this as a .NET 3.5 module, because for my version of IIS (IIS 7.5), it only seems to see .NET 2.0/3.0/3.5 modules in the IIS configuration.  When I created the assembly as a 4.0 assembly, IIS would not pick it up.  Again, I've only tried this on IIS 7.5, so later versions may be different.
3. Code the class above in the class library project you just created
4. Configure the class library to be strongly named (right click on the project --> Properties --> Look in the Signing tab).  We need to do this because we will have to put the assembly in the GAC, which requires the assembly to be strongly named.
5. Compile the project.  
6. Get the release version of the DLL and add it to the GAC using the "gacutil -u YourAssemblyName.dll"
7. Configure IIS to run the HttpModule for the entire site.  To do this, go into IIS Manager, click on the name of the server (item #1 in the diagram) and then double click on the Modules icon (item #2).  This will bring up all of the Http modules that are configured server wide, and you can add your module as a managed module from here.

If you are wondering, the configuration file that IIS uses for machine wide configuration is the ApplicationHost.config file, which is located in %WINDIR%\system32\inetsrv\config directory.  If you want to, you can also edit this file directly to add in the module.

What I found is that I needed to then stop and restart IIS for the module to take effect.  Of course, it is always good to test things out and make sure things are working as you expect them to.

Am I Safe Now

You have probably already thought of this, but there are other ways someone can figure out if you are running ASP.NET, the most obvious being the aspx extension applied to pages in Web Forms.  And that is true.  And if they know you are running ASP.NET, its a pretty safe guess that you are running on top of IIS.

But at least now someone does not have detailed information about exactly what version of software you are running.  And this is the point.  We don't want to give away any more information than we have to.  We want to make things a little harder for a potential attacker and not hand them information about our site on a silver platter.

Of course, I would always advocate that you make these changes in DEV first, then QA and finally move them to your production environment.  But take a little time in your next sprint to either build this module into your site or otherwise turn off these headers.  It takes just a few minutes, and it helps keeps private information that should have never been available anyway.

Useful Websites for Web Developers

Many times a blog posts functions as a place where you can write something down so you will be able to find the information later, and that is what this blog post is.  There are a number of useful websites that I have bookmarked or sometimes not, but I want to keep this list all in one place.  If you are a web developer, you probably already know about most or all of these sites.  But if you don't, you might want to check them out as well.

Browserscope (http://www.browserscope.org/)

This site allows you to test the capabilities of your browser as well as see the capabilities for other browsers people have tested.  This does test browser capability, but also gets a little more down into the details than other sites on this list.  For example, it you are wondering how many concurrent network connections a particular browser supports, then this is the site to go to.

HTML5 Test (https://html5test.com/)

More and more browsers support HTML 5 these days, but there are still important differences out there.  This site will not only test your browser, but compare different browsers to each other so you know what capabilities are and are not supported.

Can I Use (http://caniuse.com/)

Another site about what features are available in what browsers.  In this site, you select the feature from the main page and it then takes you to a page showing you what browser versions support that feature.

Modern IE (https://www.modern.ie/en-us)

This site by Microsoft allows you to scan a page to see if you are using any old or deprecated features that you should be uplifting.  What is even better is they have an interface to Browserstack which will take screenshots of your website right from a webpage.

Google PageSpeed Insights (https://developers.google.com/speed/pagespeed/insights/)

Web performance is a big deal, and PageSpeed Insights quickly scans a target website for front end best practices.  You will get a score for both mobile and desktop page speed as well as mobile usability.  In addition, you will get specific recommendations about how the performance of your site can be improved, along with links to Google's documentation explaining exactly what needs to be done.

The online version of PageSpeed Insights can only target publicly available web sites.  You can also check out the Chrome Extension.  One of the nice things about the extension is that for things like optimized images, the extension will provide you with an already optimized version.

Web Page Test (http://www.webpagetest.org/)

This site allows you to run a performance test against your website from locations around the world.  This is very useful to get some real data about what your performance might look like for someone across the country or across the world.  You get back detailed reports including waterfall charts and a breakdown of the type of data that was loaded.

Alexa (http://www.alexa.com/)

Alexa provides site ranking and demographic data about who is viewing a site.  This is useful for comparing your site to competitors in your categories.  The data provided has an SEO bent.  There is a pay version, but still a good amount of interesting data available for free.

Built With (http://builtwith.com/)

Ever wonder what technologies a web site was using.  Built With scans the site and delivers you a nice report on everything ranging from the web server to JavaScript and CSS libraries used.  Could you get all of this information yourself by inspecting HTTP headers and HTML?  Sure.  But why do that when this site can do it for you.