An Approach to Bundling and Minification is ASP.NET 5

Microsoft has made significant changes in how ASP.NET 5 works, and one of these changes is around how bundling and minification is handled.  The bundling and minification API is no longer available, and instead, you need to set up gulp tasks to bundle and minify your files.

Lets quickly summarize the capabilities the bundling and minification API in ASP.NET 4.5 offered us:

1. CSS and JavaScript files could be combined into bundles
2. CSS and JavaScript files could be minified
3. An Expires header was added to each bundle response so that the bundle would be cached by the browser
4. A SHA1 hash was added to the URL of the bundle to act as a cache breaker.

So far, the approaches I have seen to this problem only cover points 1 and 2.  They bundle and minify the files, but they do not address anything with caching.  And this is really important.  CSS and JavaScrpt files change somewhat infrequently, so we want the browser to cache them, especially since we are using more and more JavaScript in our apps to deliver highly interactive experiences. 

What is also important though is that any solution we come up with addresses point #4 above.  By taking a SHA1 hash of the bundle and incorporating it into the URL, if any file in the bundle is changes (or if a file is added to or removed from the bundle), then the URL of the bundle changes and the browser will know to download the new version of the bundle.  This is critical, because CSS and JavaScript that we write will inevitable have bugs or need features added, and when we do this, we need to make sure the browser knows to download the new bundle.

What follows is the approach that I developed for my new Pluralsight course "Improving Website Performance with PageSpeed Insights" (to be release mid-July 2015).  I do expect that as time goes on, I'll update this approach.  But this will at the minimum provide you a starting point for how to solve this problem.

Step 1 - Setting up Gulp

The first thing you need to do is edit your packages.json file in order to pull in the needed npm packages.  This file is in your project root, and when done should look like this

{
  "name": "ASP.NET",
  "version": "0.0.0",
  "devDependencies": {
    "gulp": "3.8.11",
    "rimraf": "2.2.8",
    "gulp-concat-css": "2.2.0",
    "gulp-concat": "2.5.2",
    "gulp-minify-css": "1.1.6",
    "gulp-uglify": "1.2.0",
    "gulp-hash": "2.0.4",
    "gulp-rename": "1.2.2",
    "event-stream": "3.3.1",
    "gulp-extend": "0.2.0",
    "gulp-clean": "0.3.1"
  }
}

What is important here are the lines in red, as that is what you are needing to add.  Note that by the time you read this, version numbers may have changed.  Each of these packages though will perform a very specific task, and what we will do in the next step is set the pipeline to do this.  For now though, lets understand why we are including each of these packages.

• gulp-concat-css - Used to concatenate (bundle) CSS files together
• gulp-concat - Used to concatenate (bundle) JavaScript files together
• gulp-minify-css - Used to minify CSS files
• gulp-uglify - Used to minify JavaScript files
• gulp-hash - Used to generate a SHA1 hash for a file and embed that hash in the filename
• gulp-rename - Used to copy files (the name says rename, but it really copies files)
• event-stream - Used to help create the pipeline of events below.
• gulp-extend - Used to merge the contents of JSON files.  Used to create our manifest below
• gulp-clean - Used to clean directories so we can start fresh for every build

Once we save this file, Visual Studio 2015 will automatically download these packages into your solution.

Step 2 - Creating a Gulp Task to Bundle and Minify Our Files

Next, we need to edit the file gulpfile.js (also in the project root) to create a new gulp task.  Eventually, we'll set this task up to run every time we build, but for now, lets concentrate on the code.

I added the following code at the end of the file

paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];



function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}


function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}


function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}


gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});

This is kind of a long segment of code, so lets break it down into some smaller segments and take it step by step to see what it does.

The first section is just setting up some directory paths that we are going to need.  The paths object is actually defined earlier in the file, I am just adding some paths to it.

paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

The next section is pulling in all of those packages that we added to our project earlier and storing them in a variable so we can make use of them later.

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

This third section is setting up the options for our hashing package.  We are going to use the SHA1 algorithm and we want all 40 bytes of the hash to be embedded into the filename.  This is important to include the entire hash to make sure we avoid any hash collisions.

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

In the fourth section, I am defining some JSON objects that define my bundles.  In this example, I have just one CSS and one JavaScript bundle, but a real project would of course have more.  It would also be better in my opinion to load this out of a separate config file, but for this demo, I didn't go that far.

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];

Next, we have two helper functions that create the CSS and JavaScript bundles respectively.  If you have done any shell scripting in Unix or Powershell, this is pretty similar.  We have a bunch of small, individual commands that pipe their output to one another to accomplish a bigger task.

So what each of these do is

• read in the source files
• concatenate them together
• output a bundled file (not minified yet -- we'll use this version while developing)
• make a copy of the file with the ".min" in it that we can operate on
• minify this file
• use the hash package to create a SHA1 hash of the file and embed that hash in the filename
• output that file to our bundles directory

All this is wrapped in the addToManifest() helper function so we can add an entry to our manifest file.

function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}

function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}

The next function is the helper function that is used to create the manifest file.  We'll need this later, because we will need to translate a bundle name into the name of the file with the hash embedded into it.  This code I took from documentation page for the gulp-hash project on npm.

function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}

Now finally, we have our gulp tasks.  The first of these cleans (removes) all of our existing bundles, and the second is the task that will build the bundles.  One thing to note here.  The cleanBundles task is a dependency of the bundleFiles task, so every time bundleFiles gets called, cleanBundles will be called automatically.

gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});

Step 3 - Running the bundleFiles task and Adding to the Build Process

Visual Studio 2015 contains a new window called Task Runner Explorer where you run can run these tasks from.  To open this window, in the Visual Studio menu go to View --> Other Windows --> Task Runner Explorer (its about in the middle).

From there, you will see all of your Gulp tasks.  If you want to run your task directly, just click on it and say run.

What you really want to do though is set your tasks up to run whenever you build, and to do that, you again right click on the task, go to bindings and make sure that "After Build" is clicked for the bundleFiles task.  

Step 4 - Creating a TagHelper To Get the Bundles Into Our Pages

Where we are at now is that our bundles are created and minified and they are sitting in a bundles directory below our application.  What we have to do now is get our pages to use these bundles.  

I've seen a couple different approaches taken here.  Many people are using the environment tag helper.  This wasn't going to work for me because I have different filenames I need to account for.  And I think it is going to be a bit of a stretch that everyone will set an environment variable on their web servers at this point.

So I need a couple of pieces here.

First, somewhere to tell my app that it should use the minified or unminified content.  For this, I chose to put a value in appSettings.  Low tech, yes, but this works.  So here is my config.json file:

{
  "AppSettings": {
    "SiteTitle": "AspNet5Bundling",
    "UseMinifiedContent":  true
  },
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=aspnet5-AspNet5Bundling-c7120031-310d-47a6-b645-970e20afc890;Trusted_Connection=True;MultipleActiveResultSets=true"
    }
  }
}

Then, I need something that is going to look at this and output the appropriate HTML based on what I need.  This is where a custom tag helper comes in.  Basically, a custom tag helper just outputs some HTML for you.  So what I am going to have in my cshtml files will look like

        <cssBundle bundle-name="site-css-bundle" bundle-dir="~/bundles"></cssBundle>

So in this case, cssBundle corresponds to the class CssBundleTagHelper, and we are passing in two items, the name of the bundle we want and the virtual path of where that bundle is.

So lets take a look at the tag helper class that processes this.

using System;
using Microsoft.AspNet.Razor.Runtime.TagHelpers;
using Microsoft.AspNet.Mvc;
using AspNet5Bundling.Util;

namespace AspNet5Bundling.Tags
{
    [TargetElement("cssBundle", Attributes = BUNDLE_ATTRIBUTE_NAME)]
    [TargetElement("cssBundle", Attributes = BUNDLE_DIRECTORY)]
    public class CssBundleTagHelper : TagHelper
    {

        public const String BUNDLE_ATTRIBUTE_NAME = "bundle-name";

        public const String BUNDLE_DIRECTORY = "bundle-dir";

        private const String LINK_TAG_TEMPLATE = "<link href=\"{0}\" rel=\"stylesheet\"/>";

        [Activate]
        public BundleConfig BundleConfig { get; set; }


        [HtmlAttributeName(BUNDLE_ATTRIBUTE_NAME)]
        public string Name { get; set; }

        [HtmlAttributeName(BUNDLE_DIRECTORY)]
        public String Directory { get; set; }

        
        public override void Process(TagHelperContext context, TagHelperOutput output)
        {
            // Always strip the outer tag name as we never want <cssBundle> to render
            output.TagName = null;
           
            String bundleFileName = this.BundleConfig.GetBundleFileName(this.Name, BundleType.css);
            String cssBundlePath = String.Format("{0}/{1}", this.Directory, bundleFileName);
            output.Content.SetContent(String.Format(LINK_TAG_TEMPLATE, cssBundlePath));            
        }

    }
}

Starting off, we have two TargetElement attributes on the class, and these help define the attributes that we can pass into this tag.  This is what tells ASP.NET what attributes this tag can accept.  Further down in the class, you see two properties called Name and Directory that have an attribute of HtmlAttributeName, and this is what links up the HTML attributes with the properties.  So this is how we get those parameters supplied on our web page into the specific TagHelper object that is running.

Then what happens in the Process() method will get called, and the job of the Process() method is to output whatever HTML you need to into the TagHelperOutput object.  By setting the TagName equal to null, this tells ASP.NET to throw away the cssBundle part of the tag.  In this case, we don't want cssBundle at all, we want to completely replace it.  Then, it is using a helper class to get the path you our CSS bundle (more on that in a minute) and it just interpolates that into a <link> tag.

So the TagHelper isn't that complicated.  It is taking a bundle name, using a helper class to look up the name of the bundle file it wants to use and then producing a link tag.  I have a jsBundle tag helper class that works exactly the same, but produces a <script> tag instead.

So what about this BundleConfig  class and the GetBundleFilename() method.  What do they look like?

using Microsoft.Framework.ConfigurationModel;
using System;
using System.Collections.Generic;

namespace AspNet5Bundling.Util
{
    public class BundleConfig
    {

        public BundleConfig(IConfiguration siteConfiguration, IConfiguration bundleConfig)
        {
            String useMinified = siteConfiguration.GetSubKey("AppSettings")["UseMinifiedContent"];
            if (!String.IsNullOrWhiteSpace(useMinified) && String.Equals(useMinified, "true", StringComparison.CurrentCultureIgnoreCase))
                this.enableMinification = true;

            this.bundleMapping = new Dictionary<string, string>();
            foreach (var item in bundleConfig.GetSubKeys())
            {
                String key = item.Key;
                String value = bundleConfig[key];
                this.bundleMapping.Add(key, value);
            }
        }


        private bool enableMinification = false;
        private Dictionary<String, String> bundleMapping;


        public String GetBundleFileName(String bundleName, BundleType bundleType)
        {
            if ( enableMinification )
            {
                String baseBundleName = bundleName + ".min." + bundleType.ToString();
                if ( this.bundleMapping.ContainsKey(baseBundleName) )
                {
                    
                    String bundleNameWIthHash = this.bundleMapping[baseBundleName];
                    return bundleNameWIthHash;
                }
                else
                {
                    throw new Exception(String.Format("Unable to find {0} bundle with name {1}", 
                        bundleType.ToString(), bundleName));
                }
            }
            else
            {
                String bundleFile = bundleName + "." + bundleType.ToString();
                return bundleFile;
            }
        }
    }
}

In the Startup.cs file, I am creating a BundleConfig object as a Singleton so the same object will always get used when a class needs it.  As you can see, in the constructor, it is looking fo rthe appSettings property to see if we should use minification and stores that as a member variable.  It also reads the manifest file in the bundles subdirectory, so it knows what bundle name goes with what file.

In the GetBundledFileName() method, it looks to see if minification is enabled.  If so, it needs to look the name of the file up in the manifest entries.  This has to be done at runtime, because the hash could be different, and of course we don't want to go update all of our pages everything a bundles hash changes.  If the file is not minified, it knows the pattern of the files, so it just constructs the appropriate filename based on this pattern.

What This Gives You

We can now bundle and minify our CSS and JavaScript files.  Every time we build, these bundles will be created, and the minifed bundles will have a SHA1 hash embedded in them to act as a cachebuster, so if we update a CSS or JavaScript file, we know the browser will get the new version.

Analysis

I'll be the first to say, this is one approach, and I think I'll refine this approach over time.  There are things I am happy with and things I think could be better.

I am happy that I was able to embed the SHA1 hash as a cachebuster in the filenames.  I think this is an oversight in other solutions I have seen, and it may be a surprise to people when the browser is not loading their latest JavaScript.  So I think this is really important.

I like the custom tags of cssBundle and jsBundle I created.  I think these are succinct and to the point of what I am doing.  Put this bundle of files here.  Simple and easy to understand.

I'm not happy that in my BundleConfig class, I am constructing some Strings to look up the bundle or just create the filename based on a pattern.  I think by revising my gulp tasks, I could produce a better manifest that would take the magic strings out of this, so when I get some time, I want to revisit this.

As I said above, I would like to pull my bundle definitions out of the gulpfile.js and into a config file, maybe bundle.config.  Right now I am mixing code and configuration, which is never good.

So there is some work to do, and as I revise this, I'll update this post so you can see the revisions (this post will remain though, as the link in my Pluralsight course points here).  But this will hopefully give you a good start as you tackle this problem.  Many have argued that the approach ASP.NET 5 is taking in this regard is better.  I am not ready to make that assertion.  I think ultimately, this is more flexible.  But the old approach was simple and you could be up and running in 5-10 minutes.  This new approach is not that way, at least not yet.  I spent a lot of time figuring this out, part of which was due things still being changed in ASP.NET 5, part of it due to the new approach.  So if nothing else, I hope this saves you some of the trouble that that I went through in figuring this out.  I think better solutions will emerge in time, but better solutions are often built on earlier solutions, so we have to get that conversation started.

Documenting Your Tables and Columns in Oracle

If you ask most developers if they would prefer to get a root canal or write software documentation, most would choose the root canal.  Almost everyone universally hates to write documentation.  But in many cases, at least some amount of documentation for a system is essential.

This is very much true for the tables and columns in your database.  At some point, someone is going to need to know what data is stored in your database, what assumptions you made and what some non-intuitive column name really means.  Ideally, yes, we would like every table and column to have a meaningful name that was descriptive enough such that our database was self documenting.  But we all know this is rarely the case.

In Oracle, you can use the COMMENT ON command to put comments on tables and columns.  The exact syntax is shown below:

-- Setting comments on a table
COMMENT ON TABLE <<table name>> IS ‘<<comment text>>’;


-- Setting a comment on a columns
COMMENT ON COLUMN <<table name>>.<<column name>>
    IS ‘<<comment text>>’;

To add comments in this way, you need to either be the schema owner on the object or have the COMMENT ANY TABLE privilege.

It is pretty simple, the comment text that you enter will be stored as a comment with the table or column respectively.

Where is this comment stored?  It is stored in Oracle's data dictionary and you can access it through two different views, all_tab_comments and all_col_comments.  (You can also use the user or dba version of these views as well).

SELECT * FROM all_tab_comments;

SELECT * FROM all_col_comments
    WHERE table_name = ‘<<table name>>’;

By themselves, these views only provide the comments.  But you can easily join these views to other views in the data dictionary to get a more complete view of what is going on in your database.  Take for example the following query which joins the user_tab_columns and user_col_comments views in order to list all of the columns for a table with their data type, if the column is nullable and any comments entered for the column.

SELECT tc.column_name, tc.data_type, tc.data_length,
    tc.data_precision, tc.data_scale, tc.nullable,
    cc.comments
FROM user_tab_columns tc
LEFT OUTER JOIN user_col_comments cc
    ON tc.table_name = cc.table_name
    AND tc.column_name = cc.column_name
WHERE tc.table_name = ‘<<table name>>’
ORDER BY tc.column_id;

Now, you have not just the column name and whatever data in the column to go on, you potentially have comments left by who designed this table or added this column about what their intent for this column was.  If you have ever taken over supporting an application and its database that has been around for a while, you know how useful this can be.

Success Factors

So that is the mechanics of how to add a comment for a table or column.  But how do we use this to solve the question posed in the beginning, the fact that everyone hates doing documentation.  Here are some tips:

• DO focus on commenting tables or columns whose meaning is not immediately obvious.  Focus on columns that have cryptic names and the parts of your database that are difficult to understand.  Maximize your time by spending it on places where concepts or intent is not obvious and people may trouble understanding a year down the road.
• DON'T comment every column just to say you have every column commented.  We all know what the purpose of the FIRST_NAME column is.  Spend your time where there is value.
• DO store comments for tables and columns in the database like this.  The problem with putting data definitions in a project document is they get filed away with the project.  And then someone has to remember what project added a table or column and go look up a document in a separate system.  Use Oracle's built in commenting feature so the definitions you create will be at your fingertips when ever you need them.
• DON'T ignore existing tables in your database for commenting.  We've all been assigned to take over an application and had occasion when we had to reverse engineer what a table or column really meant.  That is the hard work.  If you go through all of this effort, take the simple step of storing what you discovered as a comment on the table or column.  This way it will be written down somewhere for three months from now when you need to remember the definition again.
• DO realize that documenting your system is an important part of your job.  Hey, not every part of our job is fun.  That is why it is called work.  But you do have a responsibility to other professionals in your organization to record some basic notes about how you designed and built your database.  You don't have to write Shakespeare, but some basic, functional docmentation is not too much to ask.

The full documentation for the COMMENT statement in Oracle 12c is here.

Why I Love Being a Pluralsight Author

Pluralsight was in the news today for a couple of reasons.  First, Lynda.com, a competitor was bought by Linked In.  Second, Pluralsight announced they had raised some additional capital.  The combination of the two stories led to inevitable talk about company valuations.  I think this is bound to happen when 1) there is M&A activity going on in the broader market and 2) you are private company who has seen spectacular growth over your lifespan.

Consequently, financial measures were in the headlines today.  And when people find out you are an author, they usually associate being an author with a lot of financial success and large royalty payments.  There is no doubt, being part of a company that has been as successful from a financial standpoint as Pluralsight is great.  And while I am not a top royalty earning author, the income I earn from my Pluralsight courses is a nice stream of extra income.   In our society, we tend to associate financial success with overall personal success and satisfaction.  That is a simplistic view though that often is simply not true.

What I love about being a Pluralsight author, what I really love has nothing to do with money.  It is that in some way, I'm helping hundreds, maybe thousands of people realize their career dreams, to be better at their jobs and to achieve their goals.  Those of us who have been in technology know that it is hard.  It is hard to learn new things and it is hard when you just can't figure out why something doesn't work.  If the courses I do help someone get over that hurdle, get through that rough patch, then I have made a difference for someone, and that is the most valuable reward that I can ever ask for.

Six weeks ago, I attended the Pluralsight Author's Summit in Salt Lake City, and what I found was that I am not alone in feeling that.  Every author I talked to took real interest in wanting to help whoever would click play on their course be a better developer, a better IT Pro, a better creative professional.  You could describe it as a passion, yes, but that word gets a little overused these days.  A better way to describe it might be that embedded in all of us who are associated with Pluralsight, there is a deep sense of purpose to help our colleagues throughout the industry grow and learn and most importantly, realize their dreams.

Financial success is such an easy yardstick to use that sometimes, it becomes the only measuring stick we use to measure the value both of companies and individuals.  Much harder to measure is the little differences someone has made to all of the people who have watched a course and become a better person from what they learned in that course.  Maybe that means someone gets a raise at their job or gets assigned to a project that they really wanted to be on.  Maybe someone who watched a course I or someone else did has more confidence at work tomorrow or has a deeper satisfaction about the project they created.

Those measures matter.  The fact that I get to touch the lives of others all around the world and help them be better at what they do might not ever make headlines, but it means the world to me, and it is what I think about every day.  And I think anyone associated with Pluralsight would tell you the same thing.

That is what I love about having the opportunity to be a Pluralsight author.

Archiving Updated and Deleted Rows on a Table in SQL Server

I previously wrote about how to archive rows in Oracle using a trigger.  Today, I want to discuss how to do this in SQL Server.  Again, the syntax varies between the two databases, but you can still accomplish the same functionality.

So to recap on what we want to accomplish:

• Any time a row in a our table is UPDATED or DELETED, we want to store a copy of the old row in the archive table
• We also want to store with the archived record date and times of when that version of the row was the active version of the row in the table
• We would like to store some information about what user ran the DML operation and the client machine it came from
• We want all of this to happen transparently and automatically.  That is, we don't want a user or application to have to do anything except run their SQL statement, and the archival of the old version of the row happens automatically.

The last requirement is what drives this solution to use a trigger.  While it often causes a lot of trouble to start putting business logic in a trigger, a requirement like this is the perfect job for a trigger.  Having a trigger archive the data makes sure that it always is done, regardless if the DML comes from an application, stored procedure or a user running an ad-hoc SQL statement.  And a trigger can perform this archiving transparently in the background, the the application or user running the DML statement does not even have to think about it happening.

A Sample Table

So lets take a look at a sample table that we will archive data for:

CREATE TABLE dbo.Contacts
(
    ContactId          INT           NOT NULL,
    FirstName          VARCHAR(20)   NOT NULL,
    LastName           VARCHAR(20)   NOT NULL,
    Email              VARCHAR(40)   NULL,
    Telephone          VARCHAR(20)   NULL,
    CreatedDate        DATETIME      NULL,
    LastModifiedDate   DATETIME      NULL,
    CONSTRAINT pk_contacts PRIMARY KEY (ContactId)
);

And here will be our archive table

CREATE TABLE dbo.ContactsArchive
(
    ArchiveId          INT IDENTITY  NOT NULL,
    ContactId          INT           NOT NULL,
    FirstName          VARCHAR(20)   NOT NULL,
    LastName           VARCHAR(20)   NOT NULL,
    Email              VARCHAR(40)   NULL,
    Telephone          VARCHAR(20)   NULL,
    OperationType      VARCHAR(1)    NOT NULL,
    VersionStartTime   DATETIME      NOT NULL,
    VersionEndTime     DATETIME      NOT NULL,
    ChangeUser         VARCHAR(30)   NULL,
    ChangeIp           VARCHAR(20)   NULL,
    CONSTRAINT pk_contacts_archive PRIMARY KEY (ArchiveId)
);

We of course see our data columns from the original table in our archive table, but lets quickly explain the purpose of the other columns in this table.

• ArchiveId - The primary key of our archive table.  Just a sequential integer
• OperationType - A column containing a code of the DML operation that resulted in the row being archived.  For updates, the code will be 'U' and for deletes 'D' will be used
• VersionStartTime - The date/time of when this version of the row became active in the table.
• VersionEndTime - The date/time when this version of the row was removed/replaced from the table.  Together with VersionStartTime, this is when this version of the row was in the main table.
• ChangeUser - The SQL Server identity of the user that ran the DML operation
• ChangeIp - The IP Address the SQL client was connected from when this change was performed.

The last two columns will help us understand who performed the DML statement that changed the data in the row.  This is not perfect.  Say we have a web application, we get the name of the SQL Server user used to log into the database, not the username of who was logged into the web application.  But this is still useful information to have because we can differentiate changes done by different applications (assuming the app use different SQL logins) and changes done by an app versus a user with ad-hoc SQL.

I also want to point out that in this case, I am using the two triggers from this previous blog post to automatically update the CreatedDate and LastModifiedDate columns of the contacts table.  Again, we want these dates to be updated automatically, transparently and to always be correct, so using these triggers makes sense.  For completeness, here are the triggers I am using for this post:

CREATE TRIGGER dbo.TRG_Contacts_On_Insert_Row_Dates
    ON dbo.Contacts
    FOR INSERT
    AS
    BEGIN
        SET NOCOUNT ON
        
        UPDATE dbo.Contacts
            SET CreatedDate = GETUTCDATE(),
                LastModifiedDate = GETUTCDATE()
        FROM dbo.Contacts INNER JOIN inserted 
            ON dbo.Contacts.ContactId = inserted.ContactId;
    END
GO


CREATE TRIGGER TRG_Contacts_On_Update_Row_Dates
    ON dbo.Contacts
    FOR UPDATE
    AS
    BEGIN
        SET NOCOUNT ON
        
        IF ( (SELECT trigger_nestlevel() ) > 1 )
            RETURN
            
        UPDATE dbo.Contacts
            SET CreatedDate = deleted.CreatedDate,
                LastModifiedDate = GETUTCDATE()
        FROM dbo.Contacts 
        INNER JOIN deleted
            ON dbo.Contacts.ContactId = deleted.ContactId;
        
    END
GO

Triggers to Archive Changes

The first case to deal with is when data in a row is changed, so for this scenario, we'll use the following trigger.

CREATE TRIGGER TRG_Contacts_Archive_Updates
    ON dbo.Contacts
    FOR UPDATE
    AS
    BEGIN
        SET NOCOUNT ON
        
        IF ( (SELECT trigger_nestlevel() ) > 1 )
            RETURN
            
        INSERT INTO dbo.ContactsArchive
            (ContactId, FirstName, LastName, Email, Telephone, 
             OperationType, VersionStartTime, VersionEndTime, 
             ChangeUser, ChangeIp)
        SELECT
            deleted.ContactId, deleted.FirstName, deleted.LastName, deleted.Email, deleted.Telephone,
            'U', deleted.LastModifiedDate, GETUTCDATE(),
             CURRENT_USER, CONVERT(varchar(20), CONNECTIONPROPERTY('client_net_address'))
        FROM deleted
        
    END
GO

We see that the first thing we do is look at the nesting level of the trigger and only execute the code for this trigger for the original DML statement.  If we didn't do this, we would actually get two rows in the archive table, because this trigger would also fire when the other trigger on the table (the one that updates the LastModifiedDate) executes.  But we just want one record, one that captures the original state of the row before the triggering UPDATE statement ran.

Then, we are dealing with a straightforward insert statement into our ContactsArchive table.  We can use the DELETED pseudo-table to get access to the original values in each row and to determine what rows were updated by the statement.  The original LastModified date of the row becomes the VersionStartTime of the row.  That is when this particular version of the row became active in the database.  For the VersionEndTime, we use the current UTC date.

Finally, we use a couple of SQL Server functions to get the current user logged into SQL Server and the IP Address that user is logged in from.

Now lets take a look at the FOR DELETE trigger

CREATE TRIGGER TRG_Contacts_Archive_Deletes
    ON dbo.Contacts
    FOR DELETE
    AS
    BEGIN
        SET NOCOUNT ON
        
        IF ( (SELECT trigger_nestlevel() ) > 1 )
            RETURN
            
        INSERT INTO dbo.ContactsArchive
            (ContactId, FirstName, LastName, Email, Telephone, 
             OperationType, VersionStartTime, VersionEndTime, 
             ChangeUser, ChangeIp)
        SELECT
            deleted.ContactId, deleted.FirstName, deleted.LastName, deleted.Email, deleted.Telephone,
            'D', deleted.LastModifiedDate, GETUTCDATE(),
             CURRENT_USER, CONVERT(varchar(20), CONNECTIONPROPERTY('client_net_address'))
        FROM deleted
        
    END
GO

This trigger is exactly the same as the trigger above, except it inserts an operation type code of 'D' into the OperationType column.  There are some ways within a trigger in SQL Server to determine if the triggering DML statement is an INSERT, UPDATE or DELETE, but these aren't nearly as clear as the syntax that Oracle gives you to do this.  So I decided to make this a completely separate trigger.

Trying Things Out

Lets put some data into our sample table and try things out.  Here are some sample rows that we will insert:

INSERT INTO Contacts (ContactId, FirstName, LastName, Email, Telephone)
    VALUES (1, 'Christine', 'Jankowski', 'ChristineJJankowski@dayrep.com', '985-555-1212');

INSERT INTO Contacts (ContactId, FirstName, LastName, Email, Telephone)
    VALUES (2, 'Dwight', 'Robertson', 'DwightSRobertson@jourrapide.com', '734-555-1212');

INSERT INTO Contacts (ContactId, FirstName, LastName, Email, Telephone)
    VALUES (3, 'Maria', 'Beck', 'MariaBBeck@teleworm.us', '859-555-1212');

INSERT INTO Contacts (ContactId, FirstName, LastName, Email, Telephone)
    VALUES (4, 'Michael', 'Harrington', 'MichaelCHarrington@dayrep.com', '718-555-1212');

INSERT INTO Contacts (ContactId, FirstName, LastName, Email, Telephone)
    VALUES (5, 'Andrea', 'Alexander', 'AndreaMAlexander@rhyta.com', '972-555-1212');

And now, we'll change one of those rows with this UPDATE statement:

UPDATE Contacts
    SET Telephone = '213-555-1234'
    WHERE ContactId = 2;

So we should be able to take a look at our archive table and find one record in there of the old version of the row.  And indeed we do.

Now let's try out deleting a row from our sample table with this DELETE statement.

DELETE FROM Contacts 
    WHERE ContactId = 5;

And again, we'll look in our archive table to find a copy of the deleted row.

Conclusion

Many times, it is important to keep a copy of the old version of data from a table whenever data in that table is updated or deleted.  This can be easily done with the triggers found above, and it makes sure that this archiving of data is done automatically each time a change occurs to the table.  Further, this process is completely transparent to the user or application executing the SQL statement.  They don't have to take any additional steps outside of their normal DML statement.

Feel free to use these triggers or variants of them in your own database as the need arises.

Populating Created Date and Last Modified Date Columns in SQL Server Using Triggers

I wrote a blog post a while back on updating what I call "administrative columns" like created date and last modified date in Oracle.  However, I don't just work in environments with Oracle as the backend RDBMS, but also work with SQL Server quite a bit.  Recently, I needed to write the same functionality for SQL Server, and as it turns out, you can achieve the same result in SQL Server, but the approach is quite a bit different.  So I think it is good to blog about how to do this, so if anyone else has this need, they can see my approach and either use it straight away or adapt it to their needs.

Lets briefly summarize the goals before we get started so it is clear what we are trying to accomplish.

• The CreatedDate column should reflect the date and time when the row was first inserted into the table.
• The CreatedDate column should never be able to change.  That is, we want to prevent anyone from updating the value in this column, either accidentally or for some malicious reason.
• The LastModifiedDate column reflects the last time the row was changed.  For a new row, the LastModifiedDate is the CreatedDate.  Once a row is updated, the LastModifiedDate value is the time that update occured.
• We want to populate both of these columns automatically from the current database time.  We don't want to rely on user supplied values for these columns.  This is for two reasons:
• It is easier if the user doesn't have to worry about supplying values for these administrative type columns so they can just focus on the business data
• We want to make sure the values are correct.  We don't want to use a user supplied date/time that is in the past or future or a value that might be in a different time zone.  Using the database value, we know the value is correct and consistent.

So first of all, lets define our target table.  For this example, I am going to use a table that captures restaurant information.  This table is simplified for our purposes, but will work just fine to demonstrate what we need to.  So here is the table.

CREATE TABLE Restaurants
(
    RestaurantId       INT           NOT NULL,
    RestaurantName     VARCHAR(40)   NOT NULL,
    Cuisine            VARCHAR(25)   NOT NULL,
    City               VARCHAR(30)   NOT NULL,
    State              VARCHAR(2)    NOT NULL,
    CreatedDate        DATETIME      NULL,
    LastModifiedDate   DATETIME      NULL,
    CONSTRAINT PK_Restaurants_RestaurantsId
        PRIMARY KEY (RestaurantId),         
);

Note that I am defining both the CreatedDate and LastModifiedDate columns as nullable columns.  This is important because in SQL Server, a trigger always runs after the DML statement that it fires for.  Unlike in Oracle where you can have a BEFORE trigger to intercept the statement and modify or replace values, in SQL Server, the trigger always fires after the DML statement.  So even though these columns accept NULL values, we'll take care of that in the triggers that follow.

You could define these columns as NOT NULL, but if you do so, then you need to provide a default value for the column (say GetUtcDate() ).  Again, the reason for this is that in SQL Server, the trigger fires after the DML statement has completed.  If the column is NOT NULL and you don't have any value (either one in the statement or a default value), then the statement will fail due to the NOT NULL constraints.  So if you choose to go the NOT NULL route, then provide a default value.

OK, lets take a look at the triggers that can accomplish this.  We'll need two triggers, and the first one to look at is the trigger that will fire after an INSERT statement.

CREATE TRIGGER TRG_Restaurants_After_Insert
    ON dbo.Restaurants
    FOR INSERT
    AS
    BEGIN
        SET NOCOUNT ON
        
        UPDATE dbo.Restaurants
            SET CreatedDate = GETUTCDATE(),
                LastModifiedDate = GETUTCDATE()
        FROM dbo.Restaurants INNER JOIN inserted 
            ON dbo.Restaurants.RestaurantID = inserted.RestaurantID;
    END
GO

In SQL Server, the pseudo table INSERTED lists all of the 'after' values of the rows that were affected by the DML statement, so in this case, like its name implies, the inserted rows.  So what we need to do in order to update these columns is write an UPDATE statement like above where we join our target table to the INSERTED pseudo table in order to get the correct set of rows, and then set the values of CreatedDate and LastModifiedDate to the correct time, which in this case we are using UTC time.  So basically what this does is get the set of affected rows, and go back to the table and update the times to the times that we want.  In this way, even if someone supplies a value for CreatedDate or LastModifiedDate, we don't care.  We'll make sure the correct date/time value gets put into the column via this trigger.

Now for the trigger that will fire after an UPDATE statement:

CREATE TRIGGER TRG_Restaurants_After_Update
    ON dbo.Restaurants
    FOR UPDATE
    AS
    BEGIN
        SET NOCOUNT ON
        
        IF ( (SELECT trigger_nestlevel() ) > 1 )
            RETURN
            
        UPDATE dbo.Restaurants
            SET CreatedDate = deleted.CreatedDate,
                LastModifiedDate = GETUTCDATE()
        FROM dbo.Restaurants 
        INNER JOIN deleted
            ON dbo.Restaurants.RestaurantId = deleted.RestaurantID;
        
    END
GO

The first thing you might notice about this trigger is the IF block wrapped around the SELECT trigger_nestlevel() function.  This is necessary to prevent this trigger firing as a nested trigger.  For example, during an INSERT statement, the TRG_Restaurants_After_Insert trigger defined above will fire, which in turn does an UPDATE statement against the table.  Without this code block, the TRG_Restaurants_After_Update statement would also complete on the update statement fired by the first trigger, and that is not what we want.  We only want this trigger to complete for user/application executed UPDATE statements against the table.

Next, we again need to update the CreatedDate and LastModifiedDate columns in our table.  This time, we'll use the DELETED pseudo table to join to, which includes the old values of all of the rows affected by the UPDATE statement.  Joining to the DELETED pseudo table gives us the proper set of rows, and we can pull the existing (old) CreatedDate value out of these rows and use it in our trigger's UPDATE statement as shown above.  What this does is keeps someone from updating the value of CreatedDate.  As we'll show below, even if someone tries to update the value of CreatedDate, it doesn't matter, because our trigger will insure we retain the existing value in our table.

Testing It All Out

Lets test everything out and make sure it is working.  At this point, I have created the table above and both of my triggers.  So lets insert a few rows into the table.

INSERT INTO Restaurants (RestaurantId, RestaurantName, Cuisine, City, State)
    VALUES (1, 'Crepes de Lux', 'French', 'Iowa City', 'IA');
 
INSERT INTO Restaurants (RestaurantId, RestaurantName, Cuisine, City, State)
    VALUES (2, 'Ginger Rootz', 'Chinese', 'Appleton', 'WI');
  
INSERT INTO Restaurants (RestaurantId, RestaurantName, Cuisine, City, State, 
        CreatedDate, LastModifiedDate)
    VALUES (3, 'House of Pho', 'Asian', 'Schaumburg', 'IL', 
        '01/01/2020', '01/01/2010');    

We see that in the third row, I am supplying a value for CreatedDate and LastModifiedDate.  However, as I am executing this statement, it is most certainly not January 1st, 2020 nor January 1st, 2010.  This is exactly the type of situation we are designing for, to make sure we always have correct values in these columns and not allow them to be overridden by user supplied values like this that may potentially be incorrect.

So lets see what is on our table at this point.

This is exactly what we want.  Even though we did not supply values in the first two INSERT statements, our trigger has automatically populated these columns for use with the correct value.  And for the third statement when incorrect values were supplied, these values were not used but again, we have the correct values in these two columns.

Lets run an UPDATE statement now.  And to start with, we'll just perform the most basic UPDATE statement, changing the Cuisine for House of Pho from Asian to Vietnamese.

UPDATE Restaurants 
    SET Cuisine = 'Vietnamese'
    WHERE RestaurantId = 3;

And now we will check our table data once again.

We see here that for "House of Pho", the LastModifiedDate has been automatically updated by our trigger to the appropriate time for when the row was changed.  The CreatedDate record was left unchanged, as were the records for the other rows in the table.  So this is exactly what we were looking for.

Finally, lets test the scenario where someone tries to run an UPDATE statement that will modify the CreatedDate column, which as we have discussed, we do not want to allow.  So to test this out, we'll run the following statement:

UPDATE Restaurants 
    SET Cuisine = 'Pan-Asian',
        CreatedDate = '01/01/2010'
    WHERE RestaurantId = 2;

So we can see here, we are trying to backdate the CreatedDate of the row to a different value.  However, with our trigger, we don't allow this to happen and we just retain the existing value.  Here is the data:

We see here that row 2 for "Ginger Rootz" has a new value for LastModifiedDate (and this value is correct).  But even though a value for CreatedDate was supplied in the UPDATE statement, the supplied value was ignored and the original CreatedDate value retained, which is the correct behavior.

Why All of This Matters

What we are essentially trying to do is establish an audit trail for our data.  We want to know when a row came into existence and when it was last changed.  For this audit trail to be effective, we need to make sure that these columns always contain the correct values.  We don't want to burden a developer or someone writing a SQL statement to worry about how to populate these columns, because we can do that automatically.  At the same time though, we want to prevent these columns from being changed, whether maliciously or accidentally so we are confident the values in these columns are always correct.  Using these two triggers, we accomplish both of those goals.  

When researching this problem, I did see a number of other blog posts and postings on StackOverflow that said just to use a DEFAULT value to populate the CreatedDate column.  This solves half of the problem, in that a developer no longer has to worry about providing a value.  But it does not address the problem of preventing the column to be modified.  And I feel this is an equally important aspect to solve.

Lately, I've been involved in researching a number of security related items.  I am not able to give away many details beyond saying I was involved in researching some issues, but what I can say is that two of the questions that always come up are "when did this row come into existence?" and "when was it last modified?".  Having audit columns that you know are correct and you know cannot be tampered with is critical when researching these sorts of issues.  

I have to admit, I myself am sometimes lazy about including columns like CreatedDate and LastModifiedDate in my tables.  I need to be better about that, and about making sure these columns are always properly updated.  With these triggers, now I don't have an excuse any more.

Using Cassette to Bundle and Minify Files in MVC2, MVC3 and Older WebForms Apps

Over the last few years, we have seen the emergence of highly interactive web applications.  In order to achieve this level of interactivity, most web pages now contain multiple JavaScript and CSS files that power this behavior.

As the number of files required by a web page goes up though, the performance of the page can be degraded, because the browser has to load each one of these dependent files from the web server.  This impacts performance in three ways

• Number of Downloads - Each separate file is a round trip from the browser to the server, and each round trip will incur network latency, the amount of time it takes for a packet to travel from the browser to server and back.  Downloading many small files is generally sower than downloading a single large file due to the impact of network latency.
• Download Size - Multiple files to download adds up in terms of the amount of data that must be downloaded by the browser from the web server.  Simply put, larger pages take longer to download and ultimately display to the user, so we want to trim download size wherever possible,
• Number of Concurrent Downloads - This is something that is not realized by most developers, but your browser will only download so many files concurrently from a given host at a time.  For most browsers, the limit is 6 concurrent downloads.  For IE 11, the number is 13 (check out all browsers at http://www.browserscope.org/).  What this means is that if you have 20 files that need to be downloaded from your website, only N will downloaded at a time, and the remaining files will queue behind these N files.  When a file from that group finishes, then the next download can start.  So if we can minimize the number of items that need to be queued, we can download and render our page faster.

Minification and Bundling

So how does minification and bundling help these problems?

Mnification - Looking at any JavaScript file, you will notice that there are lots of spaces, line feeds and comments in the file.  As a developer, these are good because this is what makes the file readable for when we have to work with it.  But from the point of view of the JavaScript interpreter in the browser, this information is extraneous.  It doesn't care if there is one space or eight, so long as it can parse the file.  So by removing these bytes from the file, we can shrink the size of the file, which means fewer bytes have to be sent over the wire from the server to the browser.  And these savings can be dramatic.  For example, the minified version of jQuery 2.1.3 is only 34 KB, where as the non-minified version is over 87 KB (click on the links to see the differences between a minified and non-minified file).  And minification doesn't just apply to JavaScript files.  You can see dramatic savings in CSS as well.

Note that minification is not compression.  Compression is the process of encoding information such that statistical redundancies in the data are reduced such that data can be represented in a shorter, more concise format.  Minification is the process of removing extraneous information from the file.  So what we ultimately want to do is first minify our JavaScript and CSS files and then compress them using HTTP compression so that we minimize the number of bytes that need to be sent over the wire.

Bundling - Bundling is the process of concatenating multiple files together so they can be downloaded as a single file.  Lets say that you site makes use of three CSS files to define styles.  And we want to keep these as three separate files because this makes the editing and maintenance of these files easier.  But, this means that the browser now has to perform three separate downloads in order to get each of these files.  As we said above, we are going to incur additional network latency by having these as three separate downloads, and these three files will all count against our concurrent download limit, which may block the browser from starting to download other resources required by the page.

The answer here is bundling.  These three files can be concatenated together such that now only one file needs to be downloaded by the browser.  The total size of the file will be just the sum of the size of each individual file, but we save in terms of not having separate downloads for each one and by freeing up some of the concurrent network connections the browser has to perform other work, like downloading other resources

How Do I Accomplish This?

You could manually minify your JavaScript by using a tool like UglifyJS.  And many people do just that in their build process.  Bundling presents a bigger challenge though, because we really want separate files when we are developing, and then only to combine them at deployment or run time.   It is possible to build this into your deployment process with a number of scripts, but this is a hassle to maintain.

In ASP.NET 4.5, Microsoft introduced the Bundling and Minification API, which is well covered in this excellent article by Rick Anderson.  The bundling and minification allows you to define JavaScript and CSS files that should be bundled together in your code, and then at run time, it will automatically minimize, concatenate and serve these files for your site.  The beauty of this is that it all happens transparently to you.  You work with files as you normally would in development, and then with some simple configuration, they are automatically optimized at run time.

However, many web projects exist today that are on older versions of Microsft frameworks.  Yes, there are many MVC2, MVC3 and older WebForms apps out there.  And due to constrained IT budgets, higher priority projects and most of all time, it is not always possible to simply lift these projects up to the latest version of the framework.  So these new features are out of reach for your apps using older frameworks.  Or are they?

Enter Cassette

Cassette is a package created by Andrew Davey that brings the same bundling and minification functionality available in ASP.NET 4.5 to prior versions of ASP.NET.  While the syntax varies somewhat, the concept is the same.  We can define 'bundles' of either CSS or JavaScript files in our ASP.NET application, and at runtime, these files will be bundled together and minified, thereby increasing the performance of our web site.  Lets take a look how this happens.

First, we want to add Cassette to our application.  This is most easily done with the NuGet.  The package you want is called Cassette.AspNet.  You can install this package using the GUI:

Or you can use the Package Manager Console where you will see something like this when you install:

Once Cassette is installed, you will see a new file in the root directory of your ASP.NET called CassetteConfiguration.  You want to edit this file to configure the various bundles needed by your project.

Defining Bundles

The next step is to define how you want to bundle various files together for your site.  A bundle can be either a Stylesheet bundle or a Script bundle, but not both.  What you want to do is think about how your stylesheet and JavaScript files logically map to pages.  If you have multiple stylesheets that are included in your master page or master layout view, then a bundle that contains all of these files makes sense.  A similar bundle for all of your JavaScript files that are on your master page makes sense as well.  Then you may have additional bundles that represent scripts that are only present on a subset of your pages.

Bundles are defined in the CassetteConfiguration.cs file that was shown above.  There are a number of different ways to define a bundle, but I prefer to create a List of the files to be included in the bundle and then add that list to the BundlesCollection object as shown below.

Here, I am defining three bundles.  The first contains all of the CSS files that are in my master layout view, the second all of the JavaScript I include in my master layout view and the third a single script used for working with Google Maps on one of the pages in my site.

You might ask, why would I create a bundle for a single file, because after all, the word bundle implies there should be multiple files.  What I am after here is minification of this JavaScript file.  By including this file in a bindle, Cassette will automtically minify the file for me at runtime.  So now, I can work with a readable file like normal in Visual Studio, but be assured that Cassette will take care of minification when the time comes.  Further, we will see in a bit that Cassette also adds a cache header for each bundle, which further improves performance.

When you are calling the bundles.Add() method, you will see intellisense in Visual Studio as follows:

The first argument is what Cassette calls applicationRelativePath.  Indeed, you can use this to point to a path in your application in order to include files in the bundle.  However, also note that it says this does not to be a real directory path.  And that is what I am doing.  I am using this parameter to give a meaningful name to the bundle, a meaningful name we will use in a moment when we go to include the bundle in one of our web pages.

There are a number of other ways to create your bundles, like providing a subdirectory name and allowing Cassette to create a bundle of all of the files in that subdirectory.  I like this method though, where I explicitly define the files in a collection and then add them to the bundles, giving each bundle a meaningful name.  I think this makes it very easy for someone else to follow what I am doing.  Do know though, there are other options available, and these are covered in the Cassette Documentation.

Adding Bundles To Our Web Pages

In this article, I am working with an MVC3 project.  You can however use Cassette on MVC2 and WebForms projects.  The syntax will be slightly different, but the concepts are the same.

In my View, first you need to add a code block at the top that defines the stylesheet and script bundles you are going to use in the view.  Note, you can do this in both a master layout view and an individual page view (the example below is actually in my _Layout.cshtml file).

Then, you call Bundles.RenderStylesheets() and Bundles.RenderScripts() at the points in your page where you want the stylesheet and script bundles to appear respectively.

For stylesheets, these go in the <head> tag

And it is best for performance is your scripts are placed just before the closing body tag.

There is a way that you can have scripts render in different places in your HTML page, but I will save how to accomplish that for a later blog post.

Turning on Cassette

There is one last step, and that is to turn on Cassette.  For Cassette to bundle and minify your files, you need to set the debug flag to false.



If the debug flag is set to true (as it might be for building on your local machine), then Cassette will just output the links to the regular version each CSS and JavaScript file.  This is useful for when you are debugging and might need to debug through some JavaScript code.  But in your production environments, you would have debug set to false to get the full benefit of bundling and minification (among other things).

So how does this look when things are working.  Here is the view from the Network tab in the Chrome Developer Tools.

So we can see what Cassette is doing.  It is using an HTTP Handler to service the request, and this handler bundles and minifies all of the associated files for this bundle.  Again, the nice thing is this all happens to us transparently as developers.  Cassette puts the correct links in each of our pages.  All we have to do is set the bundles up correctly.

But Wait, There is More...

One of the things Cassette also does is add the appropriate caching headers to your bundles that are sent down to the browser.  This means once a browser has downloaded the bundle the first time, it will not have to download it again for another one year.

So in the case above where we have bundles named MasterCss and MasterScripts containing the CSS and JavaScript files we use on every page in our site, these bundles will only be loaded once, not on every page a user navigates to while visiting our site.  And if they come back and visit our site again tomorrow or next week, again, these bundles will be caches locally on their browser and not need to be reloaded.  By caching these assets on the browser after the first page load, we'll save significant bandwidth on ever subsequent page load.

What happens though if we need to change one of these files?  Perhaps we find a bug in our JavaScript or need to change our stylesheets to support a new color scheme?  How does the browser know that a particular bundle has changed so that it should be downloaded again?

What Cassette does is calculate a SHA1 hash over the contents of the bundle, base 64 encodes this hash and then embeds this base 64 encoded hash value in the name of the bundle.  That is the long string in the name of the bundle in the screeenshot in the previous section.

In this way, if the contents of any of the files in the bundle changes, the computed SHA1 hash will change and hence the name of the bundle will change.  When the browser sees this new bundle name, it will realize it does not have this version of the bundle cached and download the new bundle from the web server.  This makes sure that for any changes that you need to make to your files, the browser will always download the correct version, and saves us from manually having to version our files and manage this process within our web application.

Summary

Using Cassette, we get all of the benefits of bundling and minification in our ASP.NET projects on earlier versions of the framework.  We can continue to work with the unminified, separate versions of each file while developing the project, and Cassette will automatically optimize these at runtime.  Further, each bundle will include the appropriate cache headers such that the bundles contents will be cached in the browser.

From a performance standpoint, we achieve better performance in by three major elements:

• CSS and JavaScript files sent to the browser will be minimized, thereby removing unnecessary information like spaces and comments and reducing the number of overall bytes that need to be sent down to the client.
• We create bundles of CSS and JavaScript files that can contain multiple files concatenated together.  This reduces the total number of files the browser must download, which reduces the penalty associated with network latency that you have to pay for each individual file download.  Also, all browsers have a limit on the number of files they will concurrently download from a site, so bundling files together helps reduce queuing of files that must be downloaded by the browser.
• By including an aggressive cache header, a bundle will only have to be downloaded once by a browser and then can be served from cache on all subsequent page views that need that bundle.  This again reduces the number of bytes that need to be sent down to the browser.  Further, in the case where a file does change in the bundle, it is already built into Cassette to calculate a new hash value and embed the value in the name such that the browser will automatically know that it needs to download a new version of the bundle.

The example I showed here was from the MVC Music Store application.  However, I have used Cassette in production on a large consumer website that received hundreds of thousands of hits every month and it has worked flawlessly.  Of course, you want to testing of any new component like you normally would, but I can say from experience that I have had success with the package in some large scale environments.

We all know that many of these ASP.NET applications on older versions will still be around for a few more years.  So if you have one of these applications that you are responsible for, I urge you to use Cassette to get the advantages of bundling and minification to improve the performance of that application.

ASP.NET Response Headers and Unnecessary Information Disclosure

I've been listening to Troy Hunt's Pluralsight course entitled "Hack Yourself First: How to go on the Cyber-Offensive" and it has been pretty interesting.  One of the topics he covers is how ASP.NET actually discloses quite a bit of information on every request view HTTP headers.  What do I mean?  Pop open the developer tools on your browser, make a request to an ASP.NET website, and more than likely, you are going to see something like this:

This is a capture from a test app I had on my local IIS instance, but notice what is circled in red.  By looking at the response headers, you now know exactly what version of IIS the site is running, the exact version of the .NET Framework and the exact version of MVC.

Just so you don't think this is just an artifact of hitting the localhost address or something I conjured up, go to http://msdn.microsoft.com/ and use your developer tools to inspect the response headers.  This is what I get.

By default, for any ASP.NET site, all of this information is made available.  So what is the problem with that?  Well, a lot actually.  Now a potential attacker knows exactly what version of software you are running.  And they can use this information to understand what vulnerabilities have been reported against that version of software and use this information to refine their attack vectors.

How do you do this?  You can use a CVE (Common Vulnerabilities and Exposures) database, of which are freely available on the web.  For example, there is one at https://cve.mitre.org/cve/index.html, and a Google search will review numerous others.  These databases are used by security professionals to share information about vulnerabilities and measure risk exposure.  Unfortunately, they can also be used by the bad guys to understand what vulnerabilities exist in a product and plan an attack against that product.

So for an example, an attacker learns from your response headers that you are running IIS version 7.0.  Now they can go look up all the vulnerabilities in version 7.0 and craft an attack around each one.  If you have been negligent about patching, you will more than likely be exploited in such an attack.

But wait, it gets worse.  Mr. Hunt points out in his course that there are search engines out there that don't index documents and web pages, but rather devices connected to the Internet.  Once such example is http://www.shodanhq.com/.  Using such a search engine, someone can easily easily find a list of sites that are running a particular version of software.

So lets say that there is a known exploit in a particular version of ASP.NET.  What an attacker can do is get a list of all of the websites running that vulnerable version and then script an attack against all of those sites.  Sure, some of those sites may have been patched or otherwise mitigated the threat so the attack does not work.  But some of those sites will be vulnerable.  The more recent the exploit, the more sites that will be vulnerable because there will have been less time for patches to be applied or the threat to be mitigated otherwise.

All of this goes back to these headers, which making publicly available information that there is no legitimate use for.  What version of IIS, ASP.NET or MVC you are running makes zero difference to the browser.  These headers are purely informational.  The problem is that this information can be used against you.  So the best course of action is to turn these headers off.  Just like in the TV shows, if you are given the right to remain silent, then remain silent.

Turning Off Unnecessary Response Headers

I will first of all say that I really, really wish Microsoft would turn these headers off by default in future versions of IIS and ASP.NET.  That is what secure by default means.  Out of the box, we don't do things that could compromise security.  But that may never happen, so in the meantime, we have to resort to other means.

There are two blog posts I want to point out tat also contain information on this, and they are worth a read as well.

• Troy Hunt -- Shhh… don’t let your response headers talk too loudly
• Scott Mitchell -- Removing Unnecessary HTTP Headers in IIS and ASP.NET

Both Mr. Hunt and Mr. Mitchell take the approach of disabling The X-AspNet-Version header in the config and the X-AspNetMvc-Version via setting a property in code.  I'm taking a different approach here and I am going to remove the X-AspNet-Version, X-AspNetMvc-Version and Server headers all in an HttpModule.

The reason why I am going to remove all three headers in a module is because I think it is useful to build a module that can applied at the machine level in IIS rather than individually to applications.  My reason for this is that it is too easy for someone to forget to turn some of these headers off when creating a new application, and then we are right back to the unnecessary information disclosure problem.  I want something I can set up when I build an IIS machine, and then I know it is being applied to all applications on the server.  This is just a different approach I am taking.  Ultimately, you have to decide what works best for you and your environment.

So how do we develop an IIS Module to do this?  It is actually crazy simple.

1:    public class RemoveServerHeadersModule : IHttpModule  
2:    {  
3:        
4:      public void Dispose()  
5:      {  
6:    
7:      }  
8:    
9:      public void Init(HttpApplication context)  
10:      {  
11:        context.PreSendRequestHeaders += context_PreSendRequestHeaders;  
12:      }  
13:    
14:    
15:      void context_PreSendRequestHeaders(object sender, EventArgs e)  
16:      {  
17:        var headers = HttpContext.Current.Response.Headers;  
18:    
19:        headers.Remove("Server");  
20:        headers.Remove("X-AspNet-Version");  
21:        headers.Remove("X-AspNetMvc-Version");  
22:      }  
23:    }  

You just write a class that implements the IHttpModule interface.  And then, the event you want to act upon is the PreSendRequestHeaders event.  Then, it is just a matter of removing the unnecessary headers as you see.

One thing that you will notice is that this code does not remove the "X-Powered-By" header.  That is because this response header is set in IIS after ASP.NET processing is completed.  If you want to remove this header, you have to do so in IIS Manager.  How to do so is covered in both of the blog posts mentioned above, so I won't repeat how to do it here.

So now we have to add this module to an application.  You do so with the following config element (this assumes an assembly name of WebSecurity -- adjust for whatever you call your assembly or locate the module).

1:   <system.webServer>  
2:    <modules runAllManagedModulesForAllRequests="true">  
3:     <add name="RemoveServerHeaders" type="WebSecurity.RemoveServerHeadersModule, WebSecurity" />  
4:    </modules>  
5:   </system.webServer>  

All of this works just fine.  We could include the class above in one of our applications, include the config element in our Web.config file and we are off and running.

As I alluded to earlier, I wanted to be able to apply this site wide.  So how do we do that?

1. You need to be using an App Pool with the IIS Integrated Pipeline.  This is so that we can use an HttpModule.
2. Create a class library project in Visual Studio that will contain the HttpModule written above.  Note that I had to create this as a .NET 3.5 module, because for my version of IIS (IIS 7.5), it only seems to see .NET 2.0/3.0/3.5 modules in the IIS configuration.  When I created the assembly as a 4.0 assembly, IIS would not pick it up.  Again, I've only tried this on IIS 7.5, so later versions may be different.
3. Code the class above in the class library project you just created
4. Configure the class library to be strongly named (right click on the project --> Properties --> Look in the Signing tab).  We need to do this because we will have to put the assembly in the GAC, which requires the assembly to be strongly named.
5. Compile the project.  
6. Get the release version of the DLL and add it to the GAC using the "gacutil -u YourAssemblyName.dll"
7. Configure IIS to run the HttpModule for the entire site.  To do this, go into IIS Manager, click on the name of the server (item #1 in the diagram) and then double click on the Modules icon (item #2).  This will bring up all of the Http modules that are configured server wide, and you can add your module as a managed module from here.

If you are wondering, the configuration file that IIS uses for machine wide configuration is the ApplicationHost.config file, which is located in %WINDIR%\system32\inetsrv\config directory.  If you want to, you can also edit this file directly to add in the module.

What I found is that I needed to then stop and restart IIS for the module to take effect.  Of course, it is always good to test things out and make sure things are working as you expect them to.

Am I Safe Now

You have probably already thought of this, but there are other ways someone can figure out if you are running ASP.NET, the most obvious being the aspx extension applied to pages in Web Forms.  And that is true.  And if they know you are running ASP.NET, its a pretty safe guess that you are running on top of IIS.

But at least now someone does not have detailed information about exactly what version of software you are running.  And this is the point.  We don't want to give away any more information than we have to.  We want to make things a little harder for a potential attacker and not hand them information about our site on a silver platter.

Of course, I would always advocate that you make these changes in DEV first, then QA and finally move them to your production environment.  But take a little time in your next sprint to either build this module into your site or otherwise turn off these headers.  It takes just a few minutes, and it helps keeps private information that should have never been available anyway.