Sunday, August 9, 2015

Slides from That Conference for my Talk on Personal Development and Becoming a Lifelong Learner

On Monday, August 10th, I will be speaking at That Conference on the topic of personal development and how to become a lifelong learner.  It is a great honor for me to be chosen to speak at That Conference.  I also would like to extend a thank you to everyone who decides to attend my session.  There are so many great sessions that are available, I am truly humbled that you have chosen to attend my talk.  I hope in return I can share with you some practical tips that help you become a more effective personal learner.

Of course, many times attendees are interested in the slides from the talk, so here those are.




Also, I reference posts on the blog at a few points in the talk.  You can search around for those, but I'll make those easy for you and just list them here.



Thanks again for attending.  I am looking at turning the talk into a series of YouTube videos at some point, so those who are not able to attend conferences or code camps can still get the benefit of the talk.  Check back here or follow me on Twitter for news on how that is going.


Saturday, August 8, 2015

Using Trello to Create a Personal Skills Matrix

When creating a person development plan, you need to know what skills you are good at and what items you need improvement on.  Furthermore, you want to be tracking this information over the long term.  As you work on a skill, it will improve.  Also though, new needs will emerge and you need to be able to add these to your matrix and decide where they fit in.

What we are talking about here is conducting a skills assessment and getting our skills into a skills matrix.  There are a number of tools that we could use for this, but I am going to show you how it can be done in a tool called Trello.

Trello is a free online Kanban board, but it is also easily adapted to a variety of other purposes.  For our purpose, we are really just using the fact that we can create cards in the board, group them together in columns and do some basic commenting and sorting.  For our skills matrix, we don't need to move cards between columns.  Still, everything we need to do is well within Trello's capabilities.

Creating Your Skills Matrix

This is what we want our end product to look like:



As you can see, each column forms a group of related skills.  I have columns for .NET Development, Web Development, Database and Professional Skills.  In reality, you would probably have a few more columns, I just want to keep this example to the point.  Don't worry though, Trello will let you create as many columns as you need to.  Then, within each column, you create an item (a card in Trello terminology) for each skill and give yourself a rating for that skill.

OK, so how do we get to this point.

Creating Your Skills Matrix

Lets walk through step by step how to do this.

Create a Trello Account
Go to Trello.com and click on the big green button in the middle of the page.  You can create an independent Trello account or login with your Google account.




Create Your First Board
Click on the grey box that says "Create new board...".  When prompted, call this board "Skills Matrix or something similar.





Add Columns To Your Board
On the left hand side of the board you will see a box that says "Add a list...".  Click on the textbox here and add your first column.  Once you have added this column, you will see this same box, just moved to the right, so keep clicking it to add all of the columns you need.


Remember, columns are just groupings of skills, and you can group these skills any way you want.  You can also add columns at any time if a new skill group emerges or you want to break an existing column into two groups to make things easier to manage and visualize.

Here are some sample groups you might consider:

  • C#/.NET Development
  • Web Development (focusing on front end aspects like HTML, CSS, JavaScript and JS frameworks)
  • Database Development (SQL, Data Modeling, Query Tuning, etc)
  • Servers (IIS, Windows Server)
  • Cloud Technologies
  • Data Analysis and Reporting
  • Professional Skills (Soft Skills)

Don't feel like you have to have all of these categories or use the same ones I do.  Come up with what works best for you.  If you need to change it up later, no worries.  Probably the only one that everyone should have is professional skills.  The others depend on what role you have and what technologies you work with.


Add Labels To Your Board
We need to be able to rate ourselves on each of these skills so we know where we are strong and where we need to improve.  We are going to use a four point rating system as follows:

  • Mastery (Green) - I am highly proficient at this skill, so much that I can teach it to someone else.  This is really something I know inside and out.
  • Proficient (Blue) - This is something I am very good at.  I can usually perform tasks involving this skill without asking questions or the help of others.  I'm able to quickly perform these tasks and rarely have defects in my work
  • Developing (Orange) - I have some experience with this skill, but these are items that tend to take longer than others as I am still learning this skill.  I may have quite a few questions or need the assistance of others when doing these items.  I tend to look a lot of things up.  I may also have received feedback that this is an area I need to improve.
  • Novice  (Red) - This is a new skill to me.  I have heard of it, but I have worked with it very little or not at all.  But this is something I want to track because I feel it will be important for me to develop in this area at some point.

You actually can choose any color scheme you want, it is just that we want to break things down so we have quick visual cues of where we are at on our skills.

To do this in Trello, look on the right hand sidebar and click the link that says "More"


Then, again in the right hand sidebar, click the link that says "Labels"



Finally, you will see the Label colors, and to give these a name, you just click on each one and type the names you want (the names that we gave above -- Mastery, Proficient, Developing, Novice)


Now we will be able to add these to our cards when we start creating them below.

Start Adding Cards for Each Skill You Have, Need or Want to Acquire
Now, in each column, click on the link that says "Add a card..." and starting adding cards for each skill.



How Do I Know What Skills I Should Add

Glad you asked.  The first thing is to ask yourself what are the skills needed for your current role.  These are usually things like languages, frameworks and features of languages.  For example, if you are a C# developer, you don't just want to put C# in the .NET Develpment column.  Break this down by features that you need to use in your role.  For example, you could have cards for each of the following:

  • General C# Constructs
  • Object Orientated Programming
  • Generics
  • Parallel Execution/Threading/Async Programming
  • Entity Framework
  • WCF
  • Design Patterns
  • SOA

And so forth.  You want to break things down such that each one is a self contained group of knowledge.



One good way to get an idea of the skills needed for your role is to look at job descriptions for titles similar to yours and see what skills employers are asking for.  If you are a web developer and keep seeing Angular.js over and over again in various job postings, that is a pretty good indication that is an important skill and should be on your matrix somewhere.

You also want to consider how your job is changing and where you want to go with your career.  Ask yourself, if I am in my same role 12-24 months from now, what new technologies are emerging that I will need to know to remain effective in this job.  If you are looking to move into a different role, look at others already in that role and ask yourself what skills do they have that make them successful in that role.

You are likely to end up with quite a list, and that is the point.  We will narrow down what we will work on later, but what we want to do is capture what knowledge we have or could need so we can prioritize what we need to work on.

This List is Meant to Be Dynamic

You just went to a user group meeting and everyone was talking about this great new JavaScript framework!  Or cloud technology!  Or super cool shiny fancy gizmo!  This is going to happen, and this is one of the reasons we use Trello.  It handles these dynamic situations well.

Create a new card for this technology under the appropriate grouping.  In the Card, add some notes about why you are excited about the technology and how it relates to the work you are doing.  And now, you are tracking this skill along with all of your other skills.  As you prioritize what you want to learn, you can evaluate your need for this skill with all of your other development opportunities.  But the point is that you have written it down, so now it won't get lost or forgotten about.  Maybe this is so important that you really will start working on learning it next month.  Maybe it will be six months.  No matter.  The point is as you learn about any new skill that you need to learn, get it on the board so it can be tracked.

Rating Yourself in Terms of Proficiency

Now for each skill, you want to rate yourself using the scale described above.  If you are unsure about a skill, a good way to determine where you are really at is to go look at the table of contents for a couple Pluralsight courses on the topic.  Go through each major area and ask yourself if you really know that area well enough that you can perform that function without help.  If there are a lot of areas you are unfamiliar with or feel you need help in, this is probably pointing to rating yourself lower.

The important thing here is to be honest with yourself.  This is not a job interview.  This is for you, to know the areas you really need more practice in.  No one else is going to see this other than you, so don't be afraid to be candid with where you need to improve.

For items that you rate yourself lower in, you can also go into the card and add some comments about areas that you feel you need to improve.  As we will see below, you aren't going to work on all of these at once, so it is useful to have some notes about what specific parts of the skill you think you need to improve at.  Again, this is an advantage of Trello in that it supports this ability to store some notes along with each card.

Where Do I Go From Here?

At this point, you have lots of different skills in different categories, and more than a few of these will have ratings of novice or developing on them.

So pick at most two of those skills to work on.  And then create a plan to work on those one to two skills over the next six to eight weeks (I'll go into how to create this plan in another post).  What we want is a focused effort for a relatively short duration of time so that we can move the one or two skills up to the next level.  Then, we'll come back to the board and look at what the next set of skills are that we should focus on and repeat the process all over again.  We might pick a specific skill a couple of times in a row if it is really important and an area we need to grow in, or we might pick different skills.  But the point is we are really focusing in one one or two items at a time and improving ourselves in those areas.

Wrapping Up

You don't have to use Trello if there is another Kanban type board that you like and want to use.  The important point is that we are cataloging our skills and getting a visual representation of where we are and what we need to work on.  At a glance, we can tell where we need to be spending our time.  And this helps us spend our development time more effectively, because we are spending our time improving on one of these topics, not wondering what we should work on next.

This approach is also flexible.  We can store notes to ourselves on each card, like specific areas we feel we need improvement in or the names of resources someone has recommended to us.  When we hear about something new we think we might need to learn, we just have to add a card to our board, and not it is included in the big picture of how we need to develop.  And through Trello, we have related skills grouped together, can filter on skills or search all of our cards.  So this electronic version really makes a lot of sense.

I hope this has been useful to you, and if you come up with any additional pointers, leave them in the comments so others can benefit from your learning's.








Using MOOC's for Your Personal Development

We have more resources than ever available for personal development today, and one of the resources that people often overlook are MOOCs.  MOOC stands for Massive Open Online Course.  They are courses that are offered by universities through a provider like Coursera or Edx that anyone can sign up for and take over the internet.  When I say anyone, I mean anyone.  Classes often start out with tens of thousands of students from all over the world.




Generally, you will watch 1-2 hours of videos each week, and often within the video there will be some practice quizzes.  Videos can be watched at any time, so this is helpful for busy professionals who don't have time to take out of their day to go to a class at a specific time.  You also typically have an assignment to complete each week.  In many cases, these are peer graded, meaning other students will grade your work and provide feedback, while you do the same for their work.  This actually turns out to be pretty effective and you get multiple perspectives on how you are learning.

Usually just taking a class is free or you can pay a small fee ($50 - $100) to earn a verified certificate.   At this time, these classes still don't carry the same weight as a standard university program, but attitudes on this may be starting to change.  I also would say that the primary reason you should be doing this is for personal enrichment.  The fact that you are learning is the most important thing, and in the end, if you are learning and improving yourself, that always shines through regardless of whether or not you earned a certificate.

What Types of Courses Are Available

Just about everything.  This includes computer science courses, math courses, engineering courses and business courses.

What many of these courses do is provide you a good introduction to the subject.  Think of it like attending a 3 day seminar in the topic.  That isn't going to be a full semester's worth of depth, but it will get you familiar with the fundamentals of the topic.  Often times, I have seen similar such seminars advertised and these cost $1500-$2000.  So it is really a good deal to be able to take courses like this for low or no cost.  Once you take a course and know you are interested in the topic, you can always pursue additional studies as well.

That doesn't mean that there aren't courses that get to depth.  A friend of mine took the Algorithms course taught by Princeton on Coursera.  He showed me the homework and I can say it was a very in-depth, difficult course.  There are also Calculus courses that look like the equivalent to taking the same course on campus.  So these are great if for some reason you need to go back to school and brush up on some of these topics.


Some Specific Recommendations

I want to recommend two courses that I have taken that I found to be very good.  Both of these courses are in the business/management area, and I found them useful from a sense that they have helped me on gaining new perspectives about how decisions are made from a business perspective.


Developing Innovative Ideas for New Companies

This course is aimed at people who are interested in starting their own business.  By this I mean starting any business, not just a technology business.  It is an introductory level course, but it does give you some exposure to some things you need to consider when thinking about starting a new business and putting together a business plan.

I think what I found most useful was that this course introduced me to something called the Business Model Canvas.  The business model canvas is a visual way to answer nine key questions about a new business you intend to start.  These include areas like what is your value proposition, who are your customers, who are your partners and what is your cost structure.  These are all factors that would go into a business plan, but by organizing this information visually, you can get a better idea of how it fits together and what areas really might need more thought.



One thing to understand though is that a tool like the business model canvas is not just for someone who is starting a business.  These are really questions that any business or non-profit should be able to answer at any time and clearly communicate to everyone involved.  So using this tool, you can better understand how your company or your department fits into the larger business ecosystem and what are the real drivers behind the decisions that are made.  To be honest, I wish every company would fill out and regularly update a business model canvas of their own, and then hang an extra large version up on a wall somewhere where everyone could see it and contribute to it.  I think this would really help to bring a lot of clarity to what the mission was and how everything fits together.


Foundations of Business Strategy

My degree is in engineering, so I took relatively few business courses while in college.  As we progress as professionals, we understand that technology is rarely the sole driver of any decision.  There are marketplace dynamics and an overall strategy of our organization to be considers.  This course provides an introduction to different strategies that a firm can choose and why a firm might choose to adopt them.

One of my takeaways from the course was that even though two firms are in the same market, they may not be competitors because they are targeting different segments of that market.  What this course does is give you some of the tools to answer those questions about competitive dynamics and how markets are structured.  What is useful here is to understand what segment of the market your firm competes in and the competitive forces that surround that segment, and this course gives you the tools to answer those questions.


Summary

Don't overlook these online courses for your personal learning.  These courses are especially useful for helping you to learn about a topic that you have minimal familiarity with, and you can do so on your own time at very low cost.

If you have a particular course that you have taken and found useful, feel free to leave a comment.  I'd be interested to hear what other's experiences has been with an MOOCs they have taken.

Some Books to Help Develop You Professional Skills

We often here the term "soft skills" when talking about personal development, but I prefer to call these professional skills, because I think that better describes the skills we are trying to develop.  When we talk about skills like time management, communication, the ability to work in a team or work independently, the ability to make good decisions, I think what we are really talking about is someone's ability to work in a professional environment and carry themselves as a professional.

So we will often hear someone say "This person needs to work on their soft skills".  We might even hear someone mention a specific soft skill, like a person needs to work on their communication skills.  But what is often missing from the discussion are the specifics of how to work on these skills.  Without the how, these phrases are just empty advice.

I read not only technical material, but also quite a bit of material on what I would consider professional skills.  As you would expect, some is good, some is OK and some didn't meet my expectations.  What I have done below is summarize the best of the material I have read and spelled out why I think each particular resource is worth the investment of your time.

As a technical professional, we have to devote a lot of our learning time to technical skills, whether that be development languages and techniques, IT Pro skills or different techniques around analysis and project management.  But everyone still needs to set aside some time to work on these professional skills.  So with that in mind, lets take a look at the list:


Personal Kanban


We've all heard "You need to work smarter, not harder".  Ever notice that no one ever tells you how to do that?

Kanban has its roots in the Toyota Production System and has two essential rules.  Limit your work in progress and visualize your workflow.  By limiting your work in progress, you actually get things done faster because you are focused and not constantly expending cycles to switch tasks.  By visualizing your workflow, you you can better understand the work you have in front of you an prioritize the right things to be worked on first.  Personal Kanban shows you in detail to apply these principles you your personal workflow  The result is that you feel more in control of what you are working on and can make the right choices about what needs to get done when.



The 7 Habits of Highly Effective People

This is a book that every professional in every field should read, and then probably re-read every few years.  The seven habits laid out by Dr. Covey (Be Proactive, Begin with the end in mind, Put first things first, Think win/win, Seek first to understand, then to be understood, Synergize and Sharpen the saw) are habits that none of us will ever truly master, but we must always be improving in order to improve our personal effectiveness.  Dr. Covey also describes how the goal is to have a team of interdependent people working together, who combine their skills to be more than the sum of their individual talents.  But to do so, everyone must first grow from a stage of dependence to interdependence, and then achieve a level of trust with others that make us interdependent.

This is not a book filled with cliche's and pie in the sky pictures.  There is real advice in this book about how to improve yourself in these areas and as a result improve your professional life.  The seven habits are timeless and serve as the underpinning of all of our achievements, which is why it is so important that we are aware of these habits and constantly striving to improve in these areas.


Decisive

We make decisions all of the time.  However, most of us don't have a good process for making decisions.  As the first chapter of this book discusses, neither do most companies.  As a result, the success rate of the decisions we make is much lower than it should be.

Decisive is all about how to refine your decision making process so that you can make more informed and ultimately better decisions.  The author's first explore the reasons why we tend to make bad decisions, like limiting our options or favoring information that supports a position we are pre-disposed to.  Then, they discuss techniques that help you overcome these shortcomings in our decision making process.

After reading Decisive, you will have the tools to be a much more analytical decision maker, who considers a wider variety or options and knows how to really test if your assumptions are valid or not.  And you will learn how to set tripwires after you have made a decision to make sure that you have indeed made the right choice or you need to reconsider.

Delivering Happiness


As I continue in my career, I have realized how important customer focus is.  Everyone has a customer, and if that customer isn't happy, they will find someone else that can better meet their needs.  This is even true for internal customers.

Today though, customer satisfaction is about much more than just "checking all the boxes" or "we delivered what the customer asked for".  Tony Hsieh (CEO of Zappos) introduces us to the concept that we have to delight our customers.  And this is so true.  When we delight our customers, they become our biggest supporters.

There is much more to this book, and a lot of it is about new thinking in terms of leadership.  Zappos doesn't track the time its customer service associates spends on any one call because it trusts them to do the right thing.  And for associates who join and decide Zappos isn't the place for them, they will actually give them a lump sum to walk away, no hard feelings.  These are refreshing new ideas about how to lead a company, and will challenge you to think about leadership differently.


Drive

I strongly recommend that anyone in a leadership position, whether formal or informal read Drive.

Mr. Pink describes how we have managed people for the last hundred years or so in a top down, hierarchical fashion and why this mode of management is outdated today, especially for knowledge workers.  He describes the differences between extrinsic motivation (e.g. financial incentives, carrot and stick approaches) and intrinsic motivation (people who are self motivated to do a good job based on the purpose), and he explains why intrinsic motivation is always superior to extrinsic motivation.

Pink describes three factors that lead to true motivation: autonomy, purpose and mastery.  It is these three factors you need to make sure everyone has in order to insure they are truly motivated to do their best work.  Making sure these three factors are present is the key to making sure that people are intrinsically motivated, and ultimately to performing their best.  This isn't about motivating people, but about creating the right environment where people are intrinsically motivated to do their best work.



Turn the Ship Around

There are a lot of books that have been written about leadership and many of them are filled with empty cliches and advice.  This book is the real deal though.

Written by a former US Navy submarine Captain David Marquet, this book talks about creating leadership at all levels and about how you can effectively move away from the command and control structures common in most organizations.  In the beginning of the book, he talks about so often someone starts a new job with excitement and the feeling that they can make a difference, only to be completely burned out and disillusioned 6 months later when it is clear that they have no power and decisions flow from the top.

This book describes the practices and challenges envountered by Captain Marquet as he took the USS Sante Fe from the worst performing ship in the fleet to the best.  What I liked most about the book is that Captain Marquet doesn't just tell us his success stories, he also tells us about the failures along the way, how he was tempted to go back to the old command model and how he ultimately overcame them.

Whether you are in a formal or informal leadership position, this book is well worth your time.  We so much need everyone to feel as though they can take the initiative, to contribute their ideas and that their voice matters, and this book shows you how giving away control to those around you can result in a better organization that truly does value everyone's contributions.


The Essential Drucker


Peter Drucker is known as the father of modern management, and his advice today is as relevant as it was 50 years ago.  You might think that because much of these writings come from decades ago, they would be out of date in today's world.  Not so.  Mr. Drucker was a man who was truly ahead of his time.

This work covers every aspect of leadership in an organization, fro having a clear objective to strategy to hiring to problem solving.  There are not many areas it does not touch on.

This was one of the first business books that I read and I still consider it one of the best.  Probably what I like most is how plain spoken the advice is.  Again, as with all of these books, there aren't any unicorns.  There is simply real hands on advice to real, every day problems that are encountered by people at every level of the organization.





As time goes on, I'll add more books to this list, so you may want to check back occasionally.  Most of these books you can get an eBook copy from Amazon for around $10-$15.  I would recommend this approach because many of these books you may want to refer back to or re-read at different times to refresh yourself on the principles they share.  If cost is an issue though, don't hesitate to check your local public library for a copy.

I hope you enjoy the list, and if you have any reading recommendations for me, put them in comments below and I'll try to get to them.



Wednesday, August 5, 2015

Why the Standard Corporate Development Plan Doesn't Work

About a week from now, I will be speaking at That Conference in Wisconsin Dells on the topic of personal learning and becoming a lifelong learner.  I have an interest in this topic for a couple of reasons.  First, as a Pluralsight author, I have a natural interest in helping others learn and helping them learn more effectively.

Second, I have done just about every job in an IT department, and this includes time where I have been an IT manager, that is, I had staff that reported to me.  One of the things that I learned is that while we encourage people to actively learn new topics and keep their skills up to date, many individuals are at a loss how to put together a personal development plan and then execute on that plan to acquire the skills they need.  For this reason, I decided to put together a talk that I will deliver at That Conference and hopefully other code camps in the area to help others learn how they can put together a development plan for themselves.  I'll also be posting much of the same information as blog posts so that attendees or any other interested party can view additional information on the topic.

The Standard Corporate Development Plan

If you work for any mid-size to large company, you probably go through an annual goal setting and review process.  Often as part of this process or as a compliment process, you will sit down with your manager and define a personal development plan for the year, and that plan will be a document that looks something like this.



I've went through this process both as an employee and as a manager working with my direct reports to complete this form.  You go through at the beginning of the year listing out all of the things that you want to improve upon in the coming year.  Acronyms like setting S.M.A.R.T. objectives get thrown around (Specific, Measurable, Attainable, Relevant and Time bound) and all of this is done with the very best of intentions.

The problem is that from what I have seen, this rarely works.  Too often we put a lot of effort in creating this document in January only to see it forgotten by March.  And that is a real shame, because all of us, no matter who we are, have skills we need to improve on.  Setting up learning goals is a good idea, and so is writing them down.  So where did we go wrong here, and more importantly, what needs to change about this process to make sure that we are getting the personal development we need.

Where the Above Process Goes Wrong

From my experience, there are two primary places this process breaks down.  They are as follows:

The Time Frame Is Too Long
Typically, the time frame for such a document is one year.  This is too long.  The problem is that when we have a very long time to do something, most of us tend to procrastinate to at least some degree.  After all, we have a year.  I can get going on that next month.  Since we have a year to complete all of these items, they don't seem urgent, certainly not compared to the other responsibilities we have at work.  So we end up working on our more "urgent" day to day priorities rather than our "important" personal development objectives.

At some point, we realize that we don't have all year any more but only a month or two left to meet these objectives, to which there is usually one of three outcomes.

The first outcome is that we try to cram all of our learning into a short time period.  Yes, I know I should have been watching Pluralsight videos all year, but instead I'll binge watch all Friday and Saturday and I'll "complete" the courses I was supposed to.  The problem is this isn't a very effective way to learn.  Yes, you may have completed the videos, but watching 20 hours of videos over two days doesn't give you time to really digest and retain the content you just watched.  Just like most of us learned in college, we needed to keep up with our assignments in a course and study throughout the semester, not just cram at finals time.  So this is really not a good outcome.

The second possible outcome is that rather than spending a significant amount of time on each objective and really getting to depth with the topic, we instead have to choose a shallower approach to learning the topic.  For example, you may have intended to read a book on time management skills for busy professionals, but since you don't have time for that, now you decide to just read a couple of articles online.  So instead reading a book over 5 or 6 hours where you have time to digest the information and see different perspectives and techniques that you could use, you are spending 30 minutes to read a couple of quick articles so you can basically check something off of your list. The problem here is that we aren't really getting to depth with our topic and our personal development need at this point, so likely, this topic will remain something we have to work on.

The third possible outcome is that we simple just don't complete some or even all of our goals.  And of course if this is the outcome, then we have to ask why these were goals in the first place.


Too Many Objectives
The second issue with the standard template and process most companies use is that it compels us to list too many different objectives.  The template I created above has six lines.  Upon being handed that template, most of use would feel compelled to put down 5 or 6 different development objectives for the year.  After all, if we only put down two, that would really look awkward.

In Jim Collins book "Good to Great" he remarks "If you have more than three priorities, then you don't have any".  This certainly applies to creating a personal development plan.  Having five, six or even more objectives almost assures that our efforts will be unfocused.  We'll end just scratching the surface of many of the objectives we have outlined for ourselves rather than really getting the the depth we need.  And without getting to that depth, it is difficult to see how we can really improve.

A Better Way

Fortunately, there is a solution here, and it is very easy.  We need to do the following:
  • Define our learning goals on a shorter time frame, preferably 2 months or less
  • Limit ourselves to a maximum of two objectives at any one time
What we are trying to do here is focus.  Rather than take on a large number of goals over a long time frame, we want to focus on just a couple of goals over a short period of time.  We want to make sure that we are focus on these goals so we become proficient in what we are trying to learn.  We don't want to dabble in 10 things.  We want to master a couple of topic areas, and then at the end of this 4-8 week period, evaluate what our next set of goals are and repeat this process again.

If you have been involved in any sort of Agile project management, this thinking is really no different.  In an Agile project environment, we define a couple of critical items that we need to get done in the next iteration, and we focus all of our energy on completing those items.  What we don't do is get distracted by other items not in the iteration, not because those items aren't important, but because we know the best way to make progress is to focus on just a couple of items at a time and get those knocked out.  It is the same with the development plan.  We are defining a shorter time period and keeping our goals very focused for that time period.

What I will do is take the document above and modify it slightly so that there are only two rows for goals.  And that is on purpose.  If there are only two rows, then you can only put two goals.  My document looks like this:



We have a lot of the same columns as before, but I have added one additional column:  Total Estimated Time.  This is an estimate, it doesn't have to be exact.  If you are taking Pluralsight courses, add up the time for the courses and then add in another 50-100% for time to do some exercises with the technology and get some hands on experience.  If you are reading a book, figure out how many pages you can read an hour and estimate the time to read the book.

The reason for this column is that if we come up with a skill that lets say we estimate will take 60 total hours to learn, then we want to break this into smaller pieces and do several iterations over these pieces.  Again, this is just like agile project management where we want to break these really big tasks down into smaller more consumable ones.  So in this case I would advise someone to identify the first 15-20 hours of training and some goals around those, and do that in the first iteration of the plan (the first 6-8 weeks, depending how many hours they are spending on development per week).  Then, in the second one of these documents, you handle the second part of the plan an so on.


Breaking Things Down Even Further

For myself, the above breakdown is usually sufficient.  Sometimes though, it is helpful to break your plan down even further like so.
All I have done here is create a simple Word document and then take the individual tasks I am going to do to learn the topic and break them down by what I am going to do on a week to week basis.  In this case, There is a Pluralsight course I am going to watch roughly a module a week from and then I am also allocating some time where I'll apply what I learned in a simple demonstration website I am going to create.

The main goal of breaking the tasks down by week is to make sure that I am not underloading or overloading a week.  And it continues to give detail and structure to what I am trying to accomplish.  Now I know exactly what I do to stay on track.  This just helps break things down into some smaller hurdles I have each week, so it is more like "learn this concept".  OK, now learn this concept.  So if this further breakdown helps you, it might be something you want to do.


But Wait, My Company Makes Me Do an Annual Development Plan

At many companies, you are going to be required to do an annual development plan and you will be required to use the standard template for your company.

In this case, go ahead and complete the standard company template.  But then, break these goals down into smaller pieces such that you can focus on 1-2 goals at a time in 4-8 week periods like was talked about above.  Once you start working on a couple of goals, keep your focus there and don't worry about all the other goals that are on the standard development plan document.  All you are really doing here is breaking those big goals down into smaller chunks.

Summary

The main point here is focus.  Focus one on or two items at a time, and focus the amount of time you spend on these items so there is a short, well defined period where you work on these topics and master them.  The real goal of having a development plan is so that you acquire new skills and improve in areas that you need to.  We are trying to achieve depth and mastery.  What I have found is focused effort is a much better way to achieve this mastery than a large number of goals that end up not really being time bound because of the period they are planned over.






Friday, July 17, 2015

Understanding if Using a Public CDN is Right for You?

I created an ASP.NET 5 project recently, and one of the things that caught my eye was this segment of code in the _Layout.html page.

First, in the document head for our CSS (click on to enlarge and be able to read)

And then later for our scripts at the end of the body
What each of these code segments is doing is using the environment tag helper class to insert the appropriate CSS and JavaScript into the document respectively.  In Development, all of the CSS and JavaScript will be served from the local web server.  However, in Staging and Production, the third party libraries of Bootstrap, jQuery, hammer and the Bootstrap touch carousel will be served from the ASP.NET public CDN.  This is what the URL's to the aspnetcdn.com domain are all about.

This project was created with the default template that ships with the release candidate of Visual Studio 2015.  So things could change by the final release, but since this is a release candidate, one would think this code would closely resemble what will actually ship with the final version.

So what is a public CDN and when is it appropriate to use one?  If you have read The Pragmatic Programmer, you will remember one of the tips in the book is that when code is generated by a wizard, you need to fully understand what that code does, because now that code is in your project and you are responsible for it.  In the case, the code comes from the default template, but the principle is the same.  We need to understand what a public CDN is and if this is really the right choice for our project to use.



What is a Public CDN

A public CDN is basically, it is a website that hosts popular CSS frameworks and JavaScript libraries that you can link to and use in your pages.  Take for example a library like jQuery.  We are accustomed to seeing a script tag like this:

<script src="/scripts/external/jquery-1.11.3.min.js"></script>

When the browser processes this script tag, it will download jQuery from your web server.

However, with a public CDN, we are going to change the URL of our script tag to point at the location of the correct version of jQuery on the CDN server.  So now we will have something that looks like this:

<script src="//ajax.aspnetcdn.com/ajax/jQuery/jquery-1.11.3.min.js"></script>


The difference is that when the browser encounters this script tag in our HTML document, it is not going to download jQuery from our web server, but from a Microsoft web server for the ASP.NET CDN.

Microsoft is not the only company to have a public CDN server that is available to be used.  There are others, and I'm listed some of the most popular below.  They vary in terms of what libraries each on has available, with Cloudflare having the most extensive collection at the time of writing:
So in a nutshell, you are handing over the hosting of these popular libraries to one of these CDN providers rather than hosting the files yourself.  And there are some clear benefits to that as we will see in the next section

Benefits From Using a Public CDN

There are some clear benefits to using a public CDN.
  • We have shifted the workload for serving these files from our web server to the CDN server, which saves us a little bit of load on our server and most importantly network bandwidth.  
  • Browsers limit the number of simultaneous connections they will make to a single web server.  So by having some resources loaded from a CDN server, we are increasing the number of simultaneous resources that can be downloaded.
  • Every major CDN provider has CDN servers distributed around the world.  So this means the user will be downloading that copy of jQuery from a server that is relatively nearby, not one across the country or across the world.  This helps to decrease network latency, the time it takes for packets to travel over the network from one place to another.
  • All of the major CDN's will add the appropriate cache headers to their responses, so that any library loaded from the CDN will be cached for a year, which will save bandwidth on subsequent requests the user makes to the site.
  • If multiple sites use all use the same library version on the same CDN, then this library will be cached by the browser when the user visits the first site.  Then, when the user visits site 2 and sees the same URL for that library, it doesn't need to download the library because it already has the contents of that URL cached in the browser.  In this way, the more sites that use a CDN the better, because there is a possibility that one of the common JavaScript libraries you use on your site is already cached in the browser due to the user requesting that exact same CDN URL from another site they have visited.
So in a nutshell, a CDN is a just a web server that hosts these popular libraries that you can then reference from your web pages, which saves your web server from having to serve these files yourself.

Is There a Downside?

When I first learned about public CDN's, I felt there was literally no reason not to use a public CDN for every website that I worked on.  As I have learned more and carefully studied the issues related to web performance, my enthusiasm for public CDN's is much more measured, and I'll describe why below.

The first thing we need to understand is that we have now introduced an additional network server into our hosting picture, and this is a server that we do not control.  It is true that CDN's are engineered for very high reliability, but as we have learned time and time again, every system will experience some down time.  Google goes down.  Microsoft goes down.  Facebook goes down.  Not very much, but it does happen.  And in the case of a CDN going down, this means that your pages are probably not going to be functional because they are most likely dependent on the JavaScript you are loading.

In this blog post by Scott Hanselman, he describes a situation where the ASP.NET CDN server in the Northeastern United States went down for a period of time.  The good news is that because CDN's are distributed around the globe, this failure only affected user's in the Northeastern United States.  The bad news about this type of failure is that they are much harder to diagnose.  Why does it work for the user in California and not for the user in Massachusetts?  When these types of partial failures happen, they are difficult to diagnose.

So the bottom line is that there will be some small amount of down time if you use a public CDN.  It won't be much, but making use of a public CDN means that there is another component in your system that needs to be up in order for your system to be up.

If you read the article, you will see some JavaScript code that will check to see if a library loaded and if not, server a local copy of the library to the browser.  This code generally looks something like this:


<script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-2.0.0.min.js"></script>
<script>
if (typeof jQuery == 'undefined') {
    document.write(unescape("%3Cscript src='/js/jquery-2.0.0.min.js' type='text/javascript'%3E%3C/script%3E"));
}
</script>

If you search the web, you will find many similar code fragments.  What is important though is to understand how a public CDN fails and how this code fragment responds.  If a web server is down (and this includes a public CDN), usually you do not get a failure status back right away.  Instead, the browser waits for the request to the server (the public CDN) to time out.  Depending on the browser, it will wait between 20 and 120 seconds to timeout, and only then fallback to the local script.

One of the important rules about web performance is that the browser will not render any DOM elements that are below a script tag that has not finished loading.  So if you have the code above in your document head, that means if the public CDN is unreachable that the user will be staring at a blank screen for 20 to 120 seconds before the fallback script fires and serves the local version of the script.  Even if your script tags are at the bottom of your HTML body section as they should be, it means your page is probably missing major functionality for 20 - 120 seconds.  The user of course will now know that the public CDN is not reachable, so to them, the page will appear partially (or perhaps completely) non-functional, and they may very well decide to leave in that period of time.

Again, the public CDN's are designed to be highly available, but the point is, if for any reason the CDN server becomes unreachable of the network, there are going to be serious consequences for your page.  Network connectivity is something we tend to take for granted until we don't have it.

Re-evaluating the Benefits

The main benefits we outlined above was that we save our web server from needing to serve these common CSS frameworks and JavaScript libraries, which ultimately saves load and bandwidth on our web server.  But here is another important point to consider.  CSS and JavaScript files are static files and as such, we should be setting the caching headers in the responses for these files such that the only time they have to be loaded is on the very first page view they make on our site.  After this, on every other page the user browses to, the browser will issue a conditional request for these resources of which the web server will send back an HTTP 304 response (Resource Not Changed) which is only a single packet.  So using a public CDN really means that we are going to have some savings on the very first page the user visit's on our site, but after that, the savings are basically negligible.

So what about the advantage of public CDN's allowing more files to be downloaded simultaneously due to resources being loaded from multiple servers.  This is still an advantage.  However, you can (and should) look at bundling together your CSS and JavaScript files, which has the effect of reducing the number of requests the browser has to make to download your page.  So this is really just a different way of solving the problem.

Finally, what about the fact that if many sites on the web use the same library version from the same CDN, there is a probability that the user will already have the library cached when they visit your site.  This is true that this can still happen, but in reality, there are so many different versions of a library like jQuery in use today, many different CDN's and many sites that still server the library themselves that in practice, the probability of the library from the CDN already being cached is very low.

What we are left with is that CDN's are geographically distributed around the world.  If you are running a public site on the internet that receives hits from around the country or around the world, this is an important advantage.  If most all of the traffic from your site comes from a local area, then this is not so important.

The point here is that if you implement good practices like caching the static content on your website and bundling and minifying your JavaScript, the advantages of using a public CDN are not nearly so great.  So what we are left with then is evaluating these smaller advantages against the risk of the CDN being unreachable.

So When Is a Public CDN the Right Choice

If you are running a public website, especially a website that is hosted in the cloud, and you have visitors from across the country or across the world, then I think a public CDN makes sense.  You will get some lift from the geographic distribution that public CDN's offer.  You also have to consider that in many cloud hosting scenarios, you are charged based on the bandwidth that you use, so any amount of bandwidth you can off load is important.

Likewise, if you are hosting a site that has bandwidth limits on the amount of data you serve, a public CDN again makes a lot of sense because you want to conserve every KB of bandwidth you can.  And if you have a web site that is nearing capacity in terms of the load it serves, using a CDN can be a quick way to transfer some of the load to another server while you work on a longer term solution.

If you are developing an internally facing (intranet) web application, I think that using a public CDN is the wrong choice.  Your internal web app probably accesses mostly other internal resources (think databases and web services within your firewall).  If you add in a public CDN, you are now reliant upon a component outside of your firewall.  If for any reason your data center experiences network connectivity issues with the internet, your internally facing app is now going to be impacted.  This doesn't have to be the CDN going down.  It could be your ISP having issues or even a component failing in your network room.  In this scenario, I think all you are doing is adding risk by using a public CDN for these apps.

Plus, for an internal web application, you have a set of user's that is nearly constant, and most likely in only a couple of locations.  So all of your CSS and JavaScript will already be cached in these user's browsers and we don't really have to solve the problem of first page views from different geographic locations.  So in the internal application scenario, I don't think using a public CDN makes any sense.

There are clearly many other scenarios, and ultimately you will end up having to analyze for yourself if a public CDN makes sense or not for your application.  This post should help you think through what some of the advantages are and if those advantages will really apply to you.  If you have already implemented caching of your static content and bundling of your CSS and JavaScript, then any lift you get from a CDN is probably going to be on the smaller side.  So what you have to do is balance this against the potential implications of if the CDN is unreachable for any reason.  For cloud hosted apps or apps that server users across a wide geographic region, this tradeoff is probably worth making.  In other cases though. the potential gain is so small that you will probably be better off just serving these files on your own.





Wednesday, July 8, 2015

Sample Code for Pluralsight Course "Improving Website Performance with Google PageSpeed Insights"

My latest Pluralsight course is in its final stages of review and should be published within the next week or so.  To support the course and make sure that everyone can follow along as I analyze and make improvements to the sample website used in the course, I am making the code for that sample website available here.

Download Source Code Here


Inside of the zip file, you will find two sub directories.

  • FoodForTechiesSlow - This is the "initial" state of the code when the analysis is done in module 2.  Therefore, this code has all of the poor performance practices present.  The PageSpeed score for this version should be around 29 or so.
  • FoodForTechiesOptimized - This is the final state of the code at the end of module 8 after the recommendations covered in the course are addressed.  This version of the home page should result in a PageSpeed score of around 99.
I say around, because there may be instances where an IIS server already has compression turned on, a file gets cached, etc.  But this gives you some general guidelines for where things start and end up at.

As I say in the course, this is by no means a fully functional website.  Really, it is only the first page that works, but the intent of this code is to demonstrate how to identify and fix web performance issues, and it is sufficient for that.

You will notice that one image in the download is different than what appears in the video.  This is the chef image used for the Articles section.  For the image used in the video, the license I purchased allows me to use the image, but not redistribute it.  So in the download, I found a public domain image on pixabay.com and substituted it in the download.  This is intentional and doesn't change the analysis.  The image still needs to be optimized.

As for the other images, they are from public domain sources or if attribution is required, I have done so in the readme file in the zip file.  I believe I have got all of these correct.  If you find an image that you are the owner of and I haven't got the licensing correct, let me know so I can get things straightened out.

If you enjoy the course, I would appreciate if you would tweet about it or otherwise let others know about it.  And thank you for watching.



Tuesday, June 30, 2015

An Approach to Bundling and Minification is ASP.NET 5

Microsoft has made significant changes in how ASP.NET 5 works, and one of these changes is around how bundling and minification is handled.  The bundling and minification API is no longer available, and instead, you need to set up gulp tasks to bundle and minify your files.

Lets quickly summarize the capabilities the bundling and minification API in ASP.NET 4.5 offered us:
  1. CSS and JavaScript files could be combined into bundles
  2. CSS and JavaScript files could be minified
  3. An Expires header was added to each bundle response so that the bundle would be cached by the browser
  4. A SHA1 hash was added to the URL of the bundle to act as a cache breaker.
So far, the approaches I have seen to this problem only cover points 1 and 2.  They bundle and minify the files, but they do not address anything with caching.  And this is really important.  CSS and JavaScrpt files change somewhat infrequently, so we want the browser to cache them, especially since we are using more and more JavaScript in our apps to deliver highly interactive experiences. 

What is also important though is that any solution we come up with addresses point #4 above.  By taking a SHA1 hash of the bundle and incorporating it into the URL, if any file in the bundle is changes (or if a file is added to or removed from the bundle), then the URL of the bundle changes and the browser will know to download the new version of the bundle.  This is critical, because CSS and JavaScript that we write will inevitable have bugs or need features added, and when we do this, we need to make sure the browser knows to download the new bundle.

What follows is the approach that I developed for my new Pluralsight course "Improving Website Performance with PageSpeed Insights" (to be release mid-July 2015).  I do expect that as time goes on, I'll update this approach.  But this will at the minimum provide you a starting point for how to solve this problem.

Step 1 - Setting up Gulp

The first thing you need to do is edit your packages.json file in order to pull in the needed npm packages.  This file is in your project root, and when done should look like this

{
  "name": "ASP.NET",
  "version": "0.0.0",
  "devDependencies": {
    "gulp": "3.8.11",
    "rimraf": "2.2.8",
    "gulp-concat-css": "2.2.0",
    "gulp-concat": "2.5.2",
    "gulp-minify-css": "1.1.6",
    "gulp-uglify": "1.2.0",
    "gulp-hash": "2.0.4",
    "gulp-rename": "1.2.2",
    "event-stream": "3.3.1",
    "gulp-extend": "0.2.0",
    "gulp-clean": "0.3.1"
  }
}

What is important here are the lines in red, as that is what you are needing to add.  Note that by the time you read this, version numbers may have changed.  Each of these packages though will perform a very specific task, and what we will do in the next step is set the pipeline to do this.  For now though, lets understand why we are including each of these packages.


  • gulp-concat-css - Used to concatenate (bundle) CSS files together
  • gulp-concat - Used to concatenate (bundle) JavaScript files together
  • gulp-minify-css - Used to minify CSS files
  • gulp-uglify - Used to minify JavaScript files
  • gulp-hash - Used to generate a SHA1 hash for a file and embed that hash in the filename
  • gulp-rename - Used to copy files (the name says rename, but it really copies files)
  • event-stream - Used to help create the pipeline of events below.
  • gulp-extend - Used to merge the contents of JSON files.  Used to create our manifest below
  • gulp-clean - Used to clean directories so we can start fresh for every build
Once we save this file, Visual Studio 2015 will automatically download these packages into your solution.

Step 2 - Creating a Gulp Task to Bundle and Minify Our Files

Next, we need to edit the file gulpfile.js (also in the project root) to create a new gulp task.  Eventually, we'll set this task up to run every time we build, but for now, lets concentrate on the code.

I added the following code at the end of the file


paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];



function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}


function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}


function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}


gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});


This is kind of a long segment of code, so lets break it down into some smaller segments and take it step by step to see what it does.

The first section is just setting up some directory paths that we are going to need.  The paths object is actually defined earlier in the file, I am just adding some paths to it.

paths.webroot = "./" + project.webroot;
paths.css = "./" + project.webroot + "/css/";
paths.bundles = "./" + project.webroot + "/bundles/";
var manifestPath = paths.webroot + '/bundle-hashes.json';

The next section is pulling in all of those packages that we added to our project earlier and storing them in a variable so we can make use of them later.

var concatCss = require("gulp-concat-css"),
    concat = require("gulp-concat"),
    minifyCss = require("gulp-minify-css"),
    uglify = require("gulp-uglify"),
    hash = require("gulp-hash"),
    rename = require("gulp-rename"),
    es = require('event-stream'),
    extend = require('gulp-extend');

This third section is setting up the options for our hashing package.  We are going to use the SHA1 algorithm and we want all 40 bytes of the hash to be embedded into the filename.  This is important to include the entire hash to make sure we avoid any hash collisions.

var hashOptions = {
    algorithm: 'sha1',
    hashLength: 40,
    template: '<%= name %>.<%= hash %><%= ext %>'
};

In the fourth section, I am defining some JSON objects that define my bundles.  In this example, I have just one CSS and one JavaScript bundle, but a real project would of course have more.  It would also be better in my opinion to load this out of a separate config file, but for this demo, I didn't go that far.

var cssBundleConfig =
    [
        {
            name: "site-css-bundle",
            files: [
                paths.lib + "bootstrap/css/bootstrap.css",
                paths.lib + "bootstrap-touch-carousel/css/bootstrap-touch-carousel.css",
                paths.css + "site.css"
            ]
        }
    ];


var jsBundleConfig =
    [
        {
            name: "scripts-bundle",
            files: [
                paths.lib + "jquery/jquery.js",
                paths.lib + "bootstrap/js/bootstrap.js",
                paths.lib + "hammer.js/hammer.js",
                paths.lib + "bootstrap-touch-carousel/js/bootstrap-touch-carousel.js"
            ]
        }
    ];


Next, we have two helper functions that create the CSS and JavaScript bundles respectively.  If you have done any shell scripting in Unix or Powershell, this is pretty similar.  We have a bunch of small, individual commands that pipe their output to one another to accomplish a bigger task.

So what each of these do is
  • read in the source files
  • concatenate them together
  • output a bundled file (not minified yet -- we'll use this version while developing)
  • make a copy of the file with the ".min" in it that we can operate on
  • minify this file
  • use the hash package to create a SHA1 hash of the file and embed that hash in the filename
  • output that file to our bundles directory

All this is wrapped in the addToManifest() helper function so we can add an entry to our manifest file.

function createCssBundle(bundleName, cssFiles, bundlePath) {
    return addToManifest(
        gulp.src(cssFiles)
          .pipe(concatCss(bundleName + ".css"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.css"))
          .pipe(minifyCss())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );

}

function createJsBundle(bundleName, jsFiles, bundlePath)  {
    return addToManifest(
        gulp.src(jsFiles)
          .pipe(concat(bundleName + ".js"))
          .pipe(gulp.dest(bundlePath))
          .pipe(rename(bundleName + ".min.js"))
          .pipe(uglify())
          .pipe(hash(hashOptions))
          .pipe(gulp.dest(bundlePath))
        );
}

The next function is the helper function that is used to create the manifest file.  We'll need this later, because we will need to translate a bundle name into the name of the file with the hash embedded into it.  This code I took from documentation page for the gulp-hash project on npm.

function addToManifest(srcStream) {
    return es.concat(
        gulp.src(manifestPath),
        srcStream
            .pipe(hash.manifest(manifestPath))
    )
    .pipe(extend(manifestPath, false, 4))
    .pipe(gulp.dest('.'));
}

Now finally, we have our gulp tasks.  The first of these cleans (removes) all of our existing bundles, and the second is the task that will build the bundles.  One thing to note here.  The cleanBundles task is a dependency of the bundleFiles task, so every time bundleFiles gets called, cleanBundles will be called automatically.

gulp.task("cleanBundles", function (cb) {
    rimraf(paths.bundles, cb);
});


gulp.task("bundleFiles", ['cleanBundles'], function () {
    for(var i=0; i < cssBundleConfig.length; i++) {
        var item = cssBundleConfig[i];
        createCssBundle(item.name, item.files, paths.bundles);
    }

    for (var i = 0; i < jsBundleConfig.length; i++) {
        var item = jsBundleConfig[i];
        createJsBundle(item.name, item.files, paths.bundles);
    }
});


Step 3 - Running the bundleFiles task and Adding to the Build Process

Visual Studio 2015 contains a new window called Task Runner Explorer where you run can run these tasks from.  To open this window, in the Visual Studio menu go to View --> Other Windows --> Task Runner Explorer (its about in the middle).

From there, you will see all of your Gulp tasks.  If you want to run your task directly, just click on it and say run.



What you really want to do though is set your tasks up to run whenever you build, and to do that, you again right click on the task, go to bindings and make sure that "After Build" is clicked for the bundleFiles task.  




Step 4 - Creating a TagHelper To Get the Bundles Into Our Pages

Where we are at now is that our bundles are created and minified and they are sitting in a bundles directory below our application.  What we have to do now is get our pages to use these bundles.  

I've seen a couple different approaches taken here.  Many people are using the environment tag helper.  This wasn't going to work for me because I have different filenames I need to account for.  And I think it is going to be a bit of a stretch that everyone will set an environment variable on their web servers at this point.

So I need a couple of pieces here.

First, somewhere to tell my app that it should use the minified or unminified content.  For this, I chose to put a value in appSettings.  Low tech, yes, but this works.  So here is my config.json file:

{
  "AppSettings": {
    "SiteTitle": "AspNet5Bundling",
    "UseMinifiedContent":  true
  },
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=aspnet5-AspNet5Bundling-c7120031-310d-47a6-b645-970e20afc890;Trusted_Connection=True;MultipleActiveResultSets=true"
    }
  }
}


Then, I need something that is going to look at this and output the appropriate HTML based on what I need.  This is where a custom tag helper comes in.  Basically, a custom tag helper just outputs some HTML for you.  So what I am going to have in my cshtml files will look like

        <cssBundle bundle-name="site-css-bundle" bundle-dir="~/bundles"></cssBundle>


So in this case, cssBundle corresponds to the class CssBundleTagHelper, and we are passing in two items, the name of the bundle we want and the virtual path of where that bundle is.

So lets take a look at the tag helper class that processes this.

using System;
using Microsoft.AspNet.Razor.Runtime.TagHelpers;
using Microsoft.AspNet.Mvc;
using AspNet5Bundling.Util;

namespace AspNet5Bundling.Tags
{
    [TargetElement("cssBundle", Attributes = BUNDLE_ATTRIBUTE_NAME)]
    [TargetElement("cssBundle", Attributes = BUNDLE_DIRECTORY)]
    public class CssBundleTagHelper : TagHelper
    {

        public const String BUNDLE_ATTRIBUTE_NAME = "bundle-name";

        public const String BUNDLE_DIRECTORY = "bundle-dir";

        private const String LINK_TAG_TEMPLATE = "<link href=\"{0}\" rel=\"stylesheet\"/>";

        [Activate]
        public BundleConfig BundleConfig { get; set; }


        [HtmlAttributeName(BUNDLE_ATTRIBUTE_NAME)]
        public string Name { get; set; }

        [HtmlAttributeName(BUNDLE_DIRECTORY)]
        public String Directory { get; set; }

        
        public override void Process(TagHelperContext context, TagHelperOutput output)
        {
            // Always strip the outer tag name as we never want <cssBundle> to render
            output.TagName = null;
           
            String bundleFileName = this.BundleConfig.GetBundleFileName(this.Name, BundleType.css);
            String cssBundlePath = String.Format("{0}/{1}", this.Directory, bundleFileName);
            output.Content.SetContent(String.Format(LINK_TAG_TEMPLATE, cssBundlePath));            
        }

    }
}



Starting off, we have two TargetElement attributes on the class, and these help define the attributes that we can pass into this tag.  This is what tells ASP.NET what attributes this tag can accept.  Further down in the class, you see two properties called Name and Directory that have an attribute of HtmlAttributeName, and this is what links up the HTML attributes with the properties.  So this is how we get those parameters supplied on our web page into the specific TagHelper object that is running.

Then what happens in the Process() method will get called, and the job of the Process() method is to output whatever HTML you need to into the TagHelperOutput object.  By setting the TagName equal to null, this tells ASP.NET to throw away the cssBundle part of the tag.  In this case, we don't want cssBundle at all, we want to completely replace it.  Then, it is using a helper class to get the path you our CSS bundle (more on that in a minute) and it just interpolates that into a <link> tag.

So the TagHelper isn't that complicated.  It is taking a bundle name, using a helper class to look up the name of the bundle file it wants to use and then producing a link tag.  I have a jsBundle tag helper class that works exactly the same, but produces a <script> tag instead.

So what about this BundleConfig  class and the GetBundleFilename() method.  What do they look like?

using Microsoft.Framework.ConfigurationModel;
using System;
using System.Collections.Generic;

namespace AspNet5Bundling.Util
{
    public class BundleConfig
    {

        public BundleConfig(IConfiguration siteConfiguration, IConfiguration bundleConfig)
        {
            String useMinified = siteConfiguration.GetSubKey("AppSettings")["UseMinifiedContent"];
            if (!String.IsNullOrWhiteSpace(useMinified) && String.Equals(useMinified, "true", StringComparison.CurrentCultureIgnoreCase))
                this.enableMinification = true;

            this.bundleMapping = new Dictionary<string, string>();
            foreach (var item in bundleConfig.GetSubKeys())
            {
                String key = item.Key;
                String value = bundleConfig[key];
                this.bundleMapping.Add(key, value);
            }
        }


        private bool enableMinification = false;
        private Dictionary<String, String> bundleMapping;


        public String GetBundleFileName(String bundleName, BundleType bundleType)
        {
            if ( enableMinification )
            {
                String baseBundleName = bundleName + ".min." + bundleType.ToString();
                if ( this.bundleMapping.ContainsKey(baseBundleName) )
                {
                    
                    String bundleNameWIthHash = this.bundleMapping[baseBundleName];
                    return bundleNameWIthHash;
                }
                else
                {
                    throw new Exception(String.Format("Unable to find {0} bundle with name {1}", 
                        bundleType.ToString(), bundleName));
                }
            }
            else
            {
                String bundleFile = bundleName + "." + bundleType.ToString();
                return bundleFile;
            }
        }
    }
}


In the Startup.cs file, I am creating a BundleConfig object as a Singleton so the same object will always get used when a class needs it.  As you can see, in the constructor, it is looking fo rthe appSettings property to see if we should use minification and stores that as a member variable.  It also reads the manifest file in the bundles subdirectory, so it knows what bundle name goes with what file.

In the GetBundledFileName() method, it looks to see if minification is enabled.  If so, it needs to look the name of the file up in the manifest entries.  This has to be done at runtime, because the hash could be different, and of course we don't want to go update all of our pages everything a bundles hash changes.  If the file is not minified, it knows the pattern of the files, so it just constructs the appropriate filename based on this pattern.

What This Gives You

We can now bundle and minify our CSS and JavaScript files.  Every time we build, these bundles will be created, and the minifed bundles will have a SHA1 hash embedded in them to act as a cachebuster, so if we update a CSS or JavaScript file, we know the browser will get the new version.

Analysis

I'll be the first to say, this is one approach, and I think I'll refine this approach over time.  There are things I am happy with and things I think could be better.

I am happy that I was able to embed the SHA1 hash as a cachebuster in the filenames.  I think this is an oversight in other solutions I have seen, and it may be a surprise to people when the browser is not loading their latest JavaScript.  So I think this is really important.

I like the custom tags of cssBundle and jsBundle I created.  I think these are succinct and to the point of what I am doing.  Put this bundle of files here.  Simple and easy to understand.

I'm not happy that in my BundleConfig class, I am constructing some Strings to look up the bundle or just create the filename based on a pattern.  I think by revising my gulp tasks, I could produce a better manifest that would take the magic strings out of this, so when I get some time, I want to revisit this.

As I said above, I would like to pull my bundle definitions out of the gulpfile.js and into a config file, maybe bundle.config.  Right now I am mixing code and configuration, which is never good.

So there is some work to do, and as I revise this, I'll update this post so you can see the revisions (this post will remain though, as the link in my Pluralsight course points here).  But this will hopefully give you a good start as you tackle this problem.  Many have argued that the approach ASP.NET 5 is taking in this regard is better.  I am not ready to make that assertion.  I think ultimately, this is more flexible.  But the old approach was simple and you could be up and running in 5-10 minutes.  This new approach is not that way, at least not yet.  I spent a lot of time figuring this out, part of which was due things still being changed in ASP.NET 5, part of it due to the new approach.  So if nothing else, I hope this saves you some of the trouble that that I went through in figuring this out.  I think better solutions will emerge in time, but better solutions are often built on earlier solutions, so we have to get that conversation started.