Tuesday, April 3, 2018

Building Enterprise Grade Web APIs - Use Feature Folders

One of the most important attributes of any code we write is maintainability.  Over the lifetime of a piece of code, it will be debugged, modified and upgraded numerous times, often by developers who were not on the original team that wrote the code. 

One of the most important aspects of maintainability is that the developer maintaining the code needs to be able to easily find what they are looking for.  This is where feature folders come in.  When using feature folders in our API, we don't use the traditional folders named Models and Controllers.  Instead, we name our folders after the endpoints of our API (the features) and place all the code related to to that endpoint (feature) within that folder.  Here is an example:




As you have probably guessed, this is an API related to Food Trucks.  If you looked at the endpoints for the API you would see URLs like /api/FoodTrucks and /api/Locations.  It is pretty easy to guess where the code is for those endpoints, in the FoodTrucks and Locations folders respectively.  For child endpoints, that is, endpoints like /api/FoodTrucks/{foodTruckId}/Reviews, I use a dot notation on the folder name like FoodTrucks.Reviews.  As you can see though, it is very easy to map from what an endpoint is to where the code is that supports that endpoint.

As you see in the screenshot above, I have expanded the Locations folder so we can have a look inside.  We see the following:
  • The controller - LocationsController
  • The view models used by this feature - LocationModel, CreateLocationModel and UpdateLocationModel
  • Two validator classes that are used to implement validation rules related to this feature  - CreateLocationModelValidator and UpdateLocationModelValidator
  • The AutoMapper profile used to map objects related to this feature - LocationsAutoMapperProfile.
Without even looking inside the files at the code, you already have a pretty good idea about what is going on.  And you know where everything is related to this feature.  Contrast this with the traditional folder layout in MVC projects:




Here if we need to debug and then make a change to the Locations endpoint, we have to navigate multiple folders to do so.  In fact, I'm not even showing the validator classes or the AutoMapper profile classes in this screenshot because they don't fit on the screen.  Sure, we could put our validators in with our models in the ApiModels folder, but for even for this relatively API, we are starting to get some folders that are unwieldy.

There is another benefit of using feature folders.  Since all of the code used for a feature is in one folder, the folder (and thereby the namespace) forms a boundary in our code that delineates this feature.   If we find ourselves needing to reach across folders, we are crossing a boundary in our code base and we have an indication that something is wrong.  This helps to discourage us from sharing model objects across two or more controllers where the object would have more than one responsibility.  Having boundaries defined like this in our code helps us be more modular and keep everything where it belongs.

Summary

I think feature folders are the way to go.  This is a pattern that you actually see quite a bit in the Angular and React communities, and I think there is no reason not to adopt the same in ASP.NET projects.  I think we end up with a much cleaner organization of the code and it is easier to find what you are looking for.  So for your next project, give feature folders a try.  I think once you complete a project using feature folders you will agree it is the way to go.

Sunday, April 1, 2018

Implementing HATEOAS Functionality in an ASP.NET Core Web API

HATEOAS (Hypertext As The Engine Of Application State) is a feature of REST based APIs in which the server provides not just data in its responses, but hyperlinks to related data and/or actions that a client can be accessed on the API.  This allows the client to self discover what operations are available on an API and dynamically navigate through them.

An API response that implements HATEOAS would look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
    "foodTruckId": 4,
    "name": "Rice Bowl",
    "description": "Asian favorites served in a bowl of rice",
    "website": "http://foodtrucknation.com/RiceBowl",
    "lastModifiedDate": "2017-10-23T01:57:37.185",
    "tags": [
        "Asian",
        "Chinese Food"
    ],
    "reviewCount": 4,
    "reviewAverage": 4,
    "socialMediaAccounts": [],
    "meta": {
        "self": "http://localhost:8000/api/FoodTrucks/4",
        "reviews": "http://localhost:8000/api/FoodTrucks/4/Reviews",
        "schedules": "http://localhost:8000/api/FoodTrucks/4/Schedules"
    }
}

In this case, I've grouped all of the hyperlinks under a property named meta, and we can see in this case I'm providing links to additional data about the food truck, namely where to find the reviews and schedule for this food truck as well as the URL for this food truck itself.

Why would you want to implement HATEOAS, when after all many APIs work just fine without it?  One reason is because it is part of the REST spec, but I think there are better reasons than simply "it is part of the spec".  I think what is nice is the self discovery aspect, especially for a developer who is new to your API.  They can easily see how the different endpoints and objects relate to each other.  They can follow these links in their browser and walk through your API discovering the various relationships as they go.  Yes, there are tools like Swagger, but it can be really powerful to walk through an API and see real data as you click on links to different end points.  The hope is that one day there are automated clients can do this, but today, HATEOAS still helps the most important client of your API, the developer who is consuming it.

That said, one of the barriers to implementing HATEOAS is the ability to generate the correct URLs in your response.  In this post, I am going to show how this can be easily done in ASP.NET Core.

Sample Project

All of this code is implemented in my Food Truck Nation API that is available on Github at the following URL.


Assumptions

Lets set the stage with some assumptions so we are all talking the same language.

  • This blog post and the sample code is currently using ASP.NET Core 2.0.  The solution should work for ASP.NET Core 1.1 as well, but you may need a minor tweak here and there.
  • I am assuming you have separate view model objects that you use to return data back to the client and you aren't just returning your domain objects.  By domain objects, I mean whatever you are querying out of your database and working with in your application.
  • This post and the sample code uses AutoMapper to map between the domain objects and the view models.  If you roll your own mapping code or use a different mapping framework, many of the lessons should still apply, but of course the implementation will be different.

Model Objects

The first thing we need to do is define properties in the model objects that we will return to the client to hold the hyperlinks we want to include.  For my Food Truck Model, here is what my response model object looks like:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
    /// <summary>
    /// Model class to represent the food truck data sent back to the client
    /// </summary>
    public class FoodTruckModel
    {
        public int FoodTruckId { get; set; }
        public String Name { get; set; }
        public String Description { get; set; }
        public String Website { get; set; }
        public List<String> Tags { get; set; }
        public int ReviewCount { get; set; }
        public double ReviewAverage { get; set; }
        public DateTime LastModifiedDate { get; set; }
        public FoodTruckLinks Meta { get; set; }

        #region Nested Types

        public class FoodTruckLinks
        {
            public String Self { get; set; }
            public String Reviews { get; set; }
            public String Schedules { get; set; }
        }

        #endregion
    }

The Meta property is the property we want to pay attention to, as it contains a FoodTruckLinks object that groups together all of the hyperlinks we want to send back with this model object.  The Reviews and Schedules properties on the FoodTruckLinks object will all contain hyperlinks to their respective resources while the Self property will contain a link (the resource identifier) for this object.

The idea of using the property name Meta came from another talk I saw, though I don't remember which one.  Whatever name you use though, I think it is a good idea to group all of the links you want to provide together in an object like this and to give them a consistent name on your model objects.  This way the client knows where to look for this information and it is all grouped together in one place.

Finally, you can see that I model my FoodTruckLinks object as a nested (inner) class.  This is because the FoodTruckLinks object (and all my other Link objects) really don't have any purpose outside of the context of their parent class.  Therefore, I modeled them as nested classes to reinforce the notion that this object really belongs in and exists just in the context of its parent (owning) object.  You do not have to model you link objects this way, but I find that this a useful technique to use.

Creating URLs in ASP.NET Core

One of the major challenges is how to generate correct URLs in your application.  Fortunately, ASP.NET Core includes a built in Url Helper class that will help create URLs for us.  This class knows about the protocol, server name and the directory where our API is deployed so it can create a proper URL for us without us having to put all of these details together ourselves and then perform some string concatenation to create a URL.  Being able to use a built in class to create our URLs not only saves us a bunch of work, but is also more reliable because we can be confident that Microsoft has properly handled all of the corner cases that we might encounter.

The Url Helper class is exposed as an interface, IUrlHelper, and we can access the interface from the Url property of our Controller classes.   To generate an absolute URL, we want to use the RouteUrl method like this.
1
2
3
4
5
6
  String selfUrl = this.Url.Link(GET_FOOD_TRUCK_BY_ID, 
      new { foodTruckId = foodTruckId });
  String reviewsUrl = this.Url.Link(FoodTruckReviewsController.GET_ALL_FOOD_TRUCK_REVIEWS, 
      new { foodTruckId = foodTruckId });
  String schedulesUrl = this.Url.Link(FoodTruckSchedulesController.GET_FOOD_TRUCK_SCHEDULE, 
      new { foodTruckId = foodTruckId });

You might be tempted to use the Action method on IUrlHelper, but the Action method will only generate an absolute path of the URL (like /api/FoodTrucks/4), not the full URL including the server name and directory path unless you include these as arguments, and that is what we want to avoid.  So Link is the method we want to use.

The first argument is the name of the route.  If you want to implement HATEOAS in your API, you are going to need to name your routes.  To do this, you include the Name property on your HttpGetHttpPostHttpPut and HttpDelete attributes that you use to decorate your like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    /// <summary>
    /// Route name constant for route that gets an individual food truck
    /// </summary>
    public const String GET_FOOD_TRUCK_BY_ID = "GetFoodTruckById";


    [HttpGet("{foodTruckId:int}", Name = GET_FOOD_TRUCK_BY_ID)]
    public IActionResult Get(int foodTruckId)
    {
        // Action Code Here
    }

In this case, I'm using a constant to hold the value of the route name.  More importantly, we see that we are including setting the Name property for the route in the HttpGet attribute so we can refer to this route in other parts of our code, namely the places we need to use the URL Resolver to create URLs for us.

The second argument is an anonymous object of the route parameters.  In our case, each of the three routes take just one parameter, foodTruckId, so that is the only parameter we include in our anonymous object.  If however your route required multiple parameters, your anonymous object would have one property for each parameter in the route.  Also note that the name of the property in the anonymous object must match the name of the parameter exactly.  Our FoodTruckReviewsController expects a parameter of foodTruckId, so that is what we name our property.  We can't name it just id and have things work.  We need to match the names exactly.

If we place the code snippet above in one of controllers and debug through it, we can see we get an actual URL:
Since I am debugging on my local machine, I get an address that includes localhost and the port number I am running on.  But rest assured that when we are running on a server, IUrlResolver will create a URL with that servers name and the path to where the application is deployed correctly.

Using code like this, we could map from our entity object to a model object in our controller like this.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
    [HttpGet("{foodTruckId:int}", Name = GET_FOOD_TRUCK_BY_ID)]
    public IActionResult Get(int foodTruckId)
    {
        FoodTruck foodTruck = this.foodTruckService.GetFoodTruck(foodTruckId);

        if (foodTruck == null)
        {
            return this.NotFound(new ApiMessageModel() { Message = $"No food truck found with id {foodTruckId}" });
        }
        else
        {
            String selfUrl = this.Url.Link(GET_FOOD_TRUCK_BY_ID,
                    new { foodTruckId = foodTruckId });
            String reviewsUrl = this.Url.Link(FoodTruckReviewsController.GET_ALL_FOOD_TRUCK_REVIEWS,
                new { foodTruckId = foodTruckId });
            String schedulesUrl = this.Url.Link(FoodTruckSchedulesController.GET_FOOD_TRUCK_SCHEDULE,
                new { foodTruckId = foodTruckId });

            var model = new FoodTruckModel()
            {
                FoodTruckId = foodTruck.FoodTruckId,
                Name = foodTruck.Name,
                Description = foodTruck.Description,
                Website = foodTruck.Website,
                Meta = new FoodTruckModel.FoodTruckLinks()
                {
                    Self = selfUrl,
                    Reviews = reviewsUrl,
                    Schedules = schedulesUrl
                }

            };

            return this.Ok(model);
        }
    }

This code will return a model object like we saw at the beginning of this article which includes the hyperlinks to this resource (the Food Truck), the reviews for the food truck and the schedule for the food truck.

What is not ideal is that this code requires us to do the mapping in directly in our controller action, and we would need to do this in each and every one of our actions that returned a model.  Typically though, we use a library like AutoMapper to translate our data from domain objects into model objects.  This is where things get a little tricky, so lets take a look at how we can make that work.

Creating Hyperlinks Using AutoMapper

When we created the URLs for our hyperlinks above, we were in an Action method of our Controller, so we had access to the IUrlHelper object.  However, when our objects are being mapped inside of AutoMapper, but default, AutoMapper (or any other mapping library) will not know anything about IUrlHelper or how to create URLs.  Fortunately though, there is a way we can inject both the IUrlHelper object and some custom mapping code into the mapping process such that we can create proper URLs when we are mapping our objects.

One of the features of AutoMapper is the ability to define Custom Value Resolvers.  These allow us to take control of the mapping process by writing a custom class that will handle the mapping process.  If you look at the FoodTruckAutoMapperProfile class, you will see this is exactly what I am doing.  Here is the relevant code snippet:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    this.CreateMap<FoodTruck, FoodTruckModel.FoodTruckLinks>()
        .ForMember(
            dest => dest.Self,
            opt => opt.ResolveUsing<UrlResolver, RouteUrlInfo>(src =>
                new RouteUrlInfo()
                {
                    RouteName = FoodTrucksController.GET_FOOD_TRUCK_BY_ID,
                    RouteParams = new { id = src.FoodTruckId }
                }
            )
        )
        .ForMember(
            dest => dest.Reviews,
            opt => opt.ResolveUsing<UrlResolver, RouteUrlInfo>(src =>
                new RouteUrlInfo()
                {
                    RouteName = Reviews.FoodTruckReviewsController.GET_ALL_FOOD_TRUCK_REVIEWS,
                    RouteParams = new { foodTruckId = src.FoodTruckId }
                }
            )
        )
        .ForMember(
            dest => dest.Schedules,
            opt => opt.ResolveUsing<UrlResolver, RouteUrlInfo>(src =>
                new RouteUrlInfo()
                {
                    RouteName = Schedules.FoodTruckSchedulesController.GET_FOOD_TRUCK_SCHEDULE,
                    RouteParams = new { foodTruckId = src.FoodTruckId }
                }
            )
        );

Rather than thinking of this code as mapping a FoodTruck object to a FoodTruckLinks object (line 1), think of this code as the code that will create the FoodTruckLinks object, and we need to use the FoodTruck object as input into this process for some of the data that we need.  The three ForMember calls (lines 2, 12 and 22) all do the same process, just for different URLs, so lets walk through the first mapping, the Self property that gets the hyperlink for the current object.

Line 3 designates that we are populating the self property.  Line 4 is where things get interesting.  The opt.ResolveUsing call tells AutoMapper we want to map this value using a Custom Value Resolver class.  The name of that Custom Value Resolver class is UrlHelper, which we see as the first generic parameter and we'll take a look at in a moment.  The second generic parameter allows us to specify a custom source object of data we need to pass into the value resolver.  This is critical, because now we can pass additional information into the resolver like the name of the route and the any route parameters that we need in our mapping process.  For our purposes, I've defined an object called RouteUrlInfo that just acts as a container for the data we need in our custom mapping process.

Finally, on lines 5 through 9 we see a lambda function that tells AutoMapper how to populate this custom source information before it calls the custom value resolver.  In this case, we are creating and populating the RouteUrlInfo object with the route name and the parameters needed for the route.

That may seem complicated on the surface, but all that is really happening is that we are writing a class with some custom mapping code that we need (UrlHelper) and then AutoMapper will invoke a method named Resolve() on that object when it needs to map that property.   In addition, we get the chance to pass some custom data into this mapping process, so we do that by using a RouteUrlInfo object that is created via a lambda function right before the mapping is to occur.

So lets take a look at the code for UrlResolver and see how what the custom mapping process looks like for creating a link.

UrlResolver

Below is the code for the UrlResolver class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
   public class UrlResolver : IMemberValueResolver<object, object, RouteUrlInfo, String>
    {

        public UrlResolver(IHttpContextAccessor httpContextAccessor)
        {
            var httpContext = httpContextAccessor.HttpContext;
            this.urlHelper = (IUrlHelper)httpContext.Items["URL_HELPER"];
        }

  
        private readonly IUrlHelper urlHelper;

  
        public virtual string Resolve(object source, object destination, 
      RouteUrlInfo sourceMember, string destMember, ResolutionContext context)
        {
            return this.urlHelper.Link(sourceMember.RouteName, sourceMember.RouteParams);
        }
    }

As you see, this class implements AutoMapper's  IMemberValueResolver interface.  Classes implementing IMemberValueResolver are defined with four generic parameters which are as follows.
  • The type of the source object being mapped from.  Since we want our resolver class to work for any domain object, we use the type object.
  • The type of the destination object being mapped to.  Since we want to allow any model or links object, again we use the type object.
  • The type of the custom source object AutoMapper will pass in.  This is our custom data carrier object RouteUrlInfo that was discussed earlier.  This parameter allows us to design a custom object that has any other information we need for our mapping and pass it into the custom resolver.  
  • The type of the object to be returned from this custom mapping process.  In our case, we want a URL which is just a string.

The Resolve() method is what gets called when AutoMapper needs to resolve the custom value during the mapping of these objects.  We see that this class has  IUrlHelper object that it grabs out of the HttpContext and then it is able call the Link() method on IUrlHelper just like before to create the hyperlink.  To call the Link() method, we need the name of the route and any parameters for the route, and these come our of the RouteUrlInfo class, which is a class we define to help pass additional information into the mapping process.

Here is the source code for the RouteUrlInfo class.
1
2
3
4
5
6
7
8
9
    public class RouteUrlInfo
    {

        public String RouteName { get; set; }


        public object RouteParams { get; set; }

    }

The RouteName parameter is obvious enough, just the name of the route we want to create a Hyperlink for.  The RouteParams property should be populated with a C# anonymous object that contains a property for each parameter needed in the route.  If a route needs just one parameter, then the anonymous object will have just one property.  If the route needs two parameters, then the object will have two properties and so on.  Further, the property names in the anonymous object need to exactly match the names of the parameters on the route.  In this way, this simple object can deliver all of the information needed to let our custom value resolver object know what it needs to know to create our route.

There is one item that we have still not discussed, and that is of how the IUrlHelper got into the HttpContext in the first place.

Making IUrlHelper Available to Your Custom Value Resolver Object

ASP.NET exposes an IUrlHelper object to us as a property on our Controller classes as we saw in the first part of this post, so our challenge is to get this instance from our controller over to our custom value resolver (UrlResolver).  The vehicle for sharing data throughout a request life cycle in ASP.NET is the Items property off of HttpContext.  What we need to do is every time before an action method runs, put a copy of IUrlHelper in Items collection of the current HttpContext.  The best way to do this is to override the OnActionExecuting method on your Controller class:
1
2
3
4
5
6
    public override void OnActionExecuting(ActionExecutingContext context)
    {
        base.OnActionExecuting(context);

        context.HttpContext.Items.Add("URL_HELPER", this.Url);
    }

I actually do this in a BaseController class that I define, and then have all of my controller classes derive from the BaseController class, so I know this automatically done for every action method I might define.  We see all this is doing is grabbing the IUrlHelper object in the Url property of the controller and putting it in HttpContext's Items collection.

Then, UrlResolver can pull the pull the reference out in its constructor so it is available when the Resolve() method gets called.  Here is the constructor for IUrlResolver:


1
2
3
4
5
    public UrlResolver(IHttpContextAccessor httpContextAccessor)
    {
        var httpContext = httpContextAccessor.HttpContext;
        this.urlHelper = (IUrlHelper)httpContext.Items["URL_HELPER"];
    }

We actually can't inject HttpContext directly, but rather have to inject an IHttpContextAccessor object.  The good news is though that ASP.NET Core's built in DI framework knows how to take care of everything, so we just need to define our constructor like this and everything else is taken care of.  And now, our custom value resolver (UrlResolver) will have access to the IUrlHelper object it needs to create hyperlinks.

Summary

This has been a long journey, but as when you look back, it actually isn't that hard to implement HATEOAS in your ASP.NET Core APIs.  And you can actually do this rather seamlessly, where the incremental cost for each response model is just defining the appropriate mappings in you  AutoMapper profile.  So to summarize, here are the major steps.

  • Leverage the built in IUrlHelper to create your hyperlinks.  This class knows how to form proper URLs and will take care of creating a link with the correct protocol, server, port, app directory, path and parameters for you.
  • Override the OnActionExecuting() method on your Controller class to put a reference to the IUrlHelper into the Items collection of HttpContext.  This way it will be available to our AutoMapper custom value resolver object later.  I suggest you define a base controller class and perform this logic there so it is consistently done for all your controllers and actions.
  • Create an AutoMapper custom value resolver object that contains the logic for how to create hyperlinks given a route name and its parameters.  This is the UrlResolver class, and this is where the custom mapping code lives that gets run when we need to create a hyperlink during the mapping process.
  • Define a simple data carrier object (RouteUrlInfo)  that we can use to passes additional information like the route name and any parameters down to our mapping process when we need to map a domain object to a model object
  • Name your routes.  This needs to be done so IUrlHelper can find your routes by name.  You do this by simply including the Name property in the HttpGet, HttpPost, HttpPut or HttpDelete attributes on your action methods.
  • Define our custom mappings in the our AutoMapper profile objects.  Basically, this is just some syntax to tell AutoMapper for each property in our Links object what route goes with the link and what the parameters are.
As you have seen, a lot of the code is code that you write once and just include in your project (or you can just copy it out of my project).  Now the only thing you have to do each time is to name your routes and implement to correct mapping code in your AutoMapper profile.  And that becomes very boiler plate.  So for not a lot of work, your APIs can support HATEOAS.

Is It Worth It?

This code actually took me a very long Saturday to develop as there were lots of pitfalls along the way.  But now I have it, so all the hard work has been done.  I like having the links in my responses just because I think it makes self-discovery a little easier for a new user of the API.  Being able to walk the tree and understand how everything relates is really invaluable when you are trying to figure out how everything fits together.  There are a few more bytes that end up going over the wire, but I will take that trade-off in order to make the API a little easier to use and understand.  This is especially true since the heavy lifting is done and I have all of the supporting framework developed and ready to go.

So what are your thoughts?  Leave them in the comments below or reach out on Twitter.  I'd love to know if you find this as a useful approach.