Integration Testing ASP.NET Core APIs incl. auth and database
Automated tests are pretty awesome to be honest! They make life a lot simpler in many ways. And even if a lot of people are talking about how we need to do unit testing, I find integration testing much more valuable to be honest.
“Automated tests helps you discover bugs in your code” is a pretty common thing I keep hearing, and that is true to an extent. But to me, the benefit of automated tests isn’t really finding bugs during coding. It is the fact that it allows me to verify that a change I have made, hasn’t caused a deviation in the expected functionality. And with integration tests, I can also verify that it hasn’t caused a deviation somewhere else in the system. This makes me much more confident when releasing new code into production, as I can be fairly certain that I haven’t broken anything that used to work at least.
Not to mention, that with automated integration testing of APIs, I can also easily make sure that I haven’t made any backwards incompatible changes.
Problems with integration testing
The test pyramid says that we should have a lot of unit tests, fewer integration tests, and even fewer end-to-end tests that include the UI. The reason for this, is that unit tests are extremely easy to write and maintain, as they only depend on a single unit. Integration testing is harder, as it comes with dependencies that might make them harder to run and maintain. And finally, end-to-end tests are generally really hard to work with. Not only because the UI tends to change more often than the back-end, but also because the tools seem a bit more fragile to be honest.
I get the reasons behind this. However, I still believe, that if we can build and maintain integration tests in a reasonably simple way, they provide a lot more value. Especially if we write them to verify the real usage of the system, and not just that the parts work together as expected. Also, with good integration tests, we might not need as many unit tests either to be honest, as the integration tests will validate the units as well.
The problem is the “build and maintain integration tests in a reasonably simple way” part. The problem with integration tests is just that, that they integrate with other things. Things that might be hard to use during testing. Things like external systems, databases, storage etc.
Some of these dependencies will need to be mocked, as it just isn’t possible to call them over and over again in rapid succession in tests. However, we should definitely aim to mock as few of the dependencies as possible. The more things we mock, the further away from the real system we get. And the further away from the real system we get, the less confidence can be put in the test.
One dependency that keeps on popping up in most solutions is a database. For quite obvious reasons… It is unfortunately also a somewhat hard dependency to work with, which is why I see a lot of developers mocking the data access layer.
Unfortunately, if you mock away the database, you are also not testing a bunch of tings in your system. For example, you aren’t testing that your code isn’t violating constraints in the database. Or that incorrect transaction handling isn’t causing problems in the system. Or even that your ORM mappings are actually correct. And so on, and so on. As you can see, it removes a lot of the confidence that the tests are supposed to provide us.
On the other hand, if you do decide to include the database, you have a whole heap of other problems that need to be sorted out. Things like how to set up the database schema, what data to populate it with, and how to handle changing data as the tests might add or remove data as part of the tests.
Options for integration testing with a database
The first problem, which is “how to set up the database schema”, is fairly simple so solve, as long as you use some form of database migration solution. For example EF Core migrations, Fluent Migrator or even handwritten SQL-scripts (not really recommended).
Populating the database with data, and maintaining it over time, is a much harder thing to solve…
Simply populating the database with data isn’t a hard problem as such. However, being able to populate it with data that gives you every single combination of data that you need for your tests, that is different story. A story that becomes even more complicated when you need to test a new combination later on.
A new combination of data, will likely require a new set of data to be added, which might cause a bunch of existing tests to fail, as they are not expecting the new data to be there.
On top of figuring out what data to add to the database, you also need to figure out how to handle the fact that some tests add new data as part of the test. And some remove data…
It is quite easy to see why a lot of developers ignore these problems and simply mock the data access layer. Even if it might make the integration tests a little less valid.
If you do decide to keep the database in the loop, you have a couple of options to choose from to solve the mentioned problems.
You could try and create a perfect database that have all the data required for your test suite, and add it to a Docker image. This allows you to spin up a new container every time you run the tests. However, you are still stuck with the fact that the data will change over time, as new tests are added. And you would also have to update the Docker image every time a new migration was added. And even if Docker is fast, it would still slow down your tests a LOT if you decided to restart the container for each test. So, you would need to make sure that the container contained everything needed for a full test run, or at least a partial test run, and then make sure to restart the container as few times as possible.
As you can see, this solves some of the problems, but far from all of them in my opinion.
Another solution is to clean out the database after every run, using something like Jimmy Bogard’s Respawn. This allows you to set up the database using the migrations in your project, add the data needed for your specific test, run the test, empty out the database, and start over again.
I find this solution to be great. It gives us controlled data for each test, and makes sure that adding and removing data doesn’t cause any problems over time. The biggest downside is that it requires the tests to be run sequentially, as each test is dependent on the database being reset before it is run.
To get away from this limitation, I started using a solution that uses database transactions around my tests instead. This allows the tests to be run in parallel, while still having access to only the database specified for that test.
Authentication/Authorization
Another often complicated thing to solve in integration tests, is authentication/authorization. During tests, we do not want to depend on any external, or complicated auth services. Not only because they might be unreliable dependencies, but also because they might be slow, and sometimes not even possible to access from the location where the tests are run.
On the other hand, auth is a standardized thing in ASP.NET Core. So, replacing the chosen implementation with another one during testing should not cause any problems. Because of this, I think the easiest way around this problem, is to simply replace the system’s auth with a simple Basic auth implementation during testing. Something that is actually fairly simple to do.
The tested application
To show how to do all of this, we need an application to test. So, I have created a very simple one called AspNetCoreTesting.Api, which is an almost empty ASP.NET Core MVC application. It contains a single DbContext called ApiContext, an IUsers service that is used to access the DbContext, and an INotificationService that is used to notify other parts of the system when a user is added. The actual API consists of a single UsersController, that allows us to retrieve all, or individual users, as well as add a user.
Finally, it uses JwtBearer authentication, coupled with a default policy and an AuthorizeFilter to perform authentication of all requests.
Note: Yes, this is simplified. And yes, the Auth isn’t actually configured in the application. But, as I only care about the testing, that doesn’t matter…
The Program.cs file looks like this
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers(options => {
options.Filters.Add(new AuthorizeFilter());
});
builder.Services.AddDbContext<ApiContext>(x => {
x.UseSqlServer(builder.Configuration.GetConnectionString("Sql"));
});
builder.Services.AddScoped<IUsers, Users>();
builder.Services.AddSingleton<INotificationService, DummyNotificationService>();
builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddJwtBearer();
builder.Services.AddAuthorization(config =>
{
config.DefaultPolicy = new AuthorizationPolicyBuilder()
.RequireAuthenticatedUser()
.Build();
});
var app = builder.Build();
app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();
app.Run();
The ApiContext looks like this
public class ApiContext : DbContext
{
public ApiContext(DbContextOptions<ApiContext> options) : base(options)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<User>(x => {
x.ToTable("Users");
});
}
#nullable disable
public DbSet<User> Users { get; set; }
#nullable enable
}
The implementation of the IUsers interface looks like this
public class Users : IUsers
{
private readonly ApiContext _context;
private readonly INotificationService _notificationService;
public Users(ApiContext context, INotificationService notificationService)
{
_context = context;
_notificationService = notificationService;
}
public Task<User[]> All()
{
return _context.Users.ToArrayAsync();
}
public Task<User?> WithId(int id)
{
return _context.Users.FirstOrDefaultAsync(x => x.Id == id);
}
public async Task<User> Add(string firstName, string lastName)
{
var user = User.Create(firstName, lastName);
await _context.AddAsync(user);
await _context.SaveChangesAsync();
await _notificationService.SendUserCreatedNotification(user);
return user;
}
}
And finally, the UserController looks like this
[Route("[controller]")]
[ApiController]
public class UsersController : ControllerBase
{
private readonly IUsers _users;
public UsersController(IUsers users)
{
_users = users;
}
[HttpGet()]
public async Task<ActionResult<User>> GetUsers()
{
return Ok(await _users.All());
}
[HttpGet("{id}")]
public async Task<ActionResult<User>> GetUserById(int id)
{
var user = await _users.WithId(id);
return user != null ? Ok(user) : NotFound();
}
[HttpPut("")]
public async Task<ActionResult<User>> AddUser(AddUserModel model)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
var user = await _users.Add(model.FirstName!, model.LastName!);
return CreatedAtAction("GetUserById", new { id = user.Id }, user);
}
}
As you can see, there isn’t a whole lot going on. But that is kind of the point. I don’t care too much about the application to test in this case. The important part is the testing of it.
Note: Yes, I skipped the implementation of the INotificationService interface, and that’s just because it isn’t important at all. I just wanted to show how to handle the dependencies that we have to mock.
The test project
To test the application, I added an xUnit test project, and removed the default test class that was in there.
Next I added a reference to the AspNetCoreTesting.Api project, as well as to Microsoft.AspNetCore.Mvc.Testing, Microsoft.EntityFrameworkCore.SqlServer, Bazinga.AspNetCore.Authentication.Basic and FakeItEasy.
The first problem to tackle is the database set up. This is quite easily handled in xUnit by using the Xunit.TestFrameworkAttribute. This allows you to “tag” a class that inherits from Xunit.Sdk.XunitTestFramework, which is then called whenever a test run is started. Inside this class, I can use EF to run the migrations needed, like this
[assembly: Xunit.TestFramework("AspNetCoreTesting.Api.Tests.TestRunStart", "AspNetCoreTesting.Api.Tests")]
namespace AspNetCoreTesting.Api.Tests
{
public class TestRunStart : XunitTestFramework
{
public TestRunStart(IMessageSink messageSink) : base(messageSink)
{
var options = new DbContextOptionsBuilder<ApiContext>()
.UseSqlServer("Server=localhost,14331;Database=AspNetCoreTesting;User Id=sa;Password=P@ssword123");
var dbContext = new ApiContext(options.Options);
dbContext.Database.Migrate();
}
}
}
This code will ensure that the database AspNetCoreTesting exists, and then run the migrations.
It might be worth noting that I use a SQL Server Docker container to host my database. And to make sure that it doesn’t collide with my regular SQL Server set up, I have mapped it to port 14331 instead of the default 1433. To do the same on your machine, you just need to run
> docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P@ssword123" \
-p 14331:1433 --name sql_test --hostname sql_test \
-d mcr.microsoft.com/mssql/server:2019-latest
Note: The migrations are run for every test run, but they will not do anything if no new migrations have been added. I don’t put a transaction around that part, as all tests need the same database structure.
Anyhow…now that I have a database that is being set up before every run, it is time to have a look at how to run the tests
Running a test
The first step is to create a new class and a new test. Like this
public class UsersControllerTests
{
[Fact]
public async Task Get_returns_401_Unauthorized_if_not_authenticated()
{
}
}
As the name implies, the test will verify that a GET request to the UsersController will return an HTTP 401 Unauthorized if the user is not authenticated.
Inside that test, the first thing I need is a server to call. But instead of spinning up an instance of my application, I want to use an in-memory test server that allows me to run an ASP.NET Core application in-memory, while still making requests to it using an HttpClient.
To create this test server, I use the Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory<T> class (from the Microsoft.AspNetCore.Mvc.Testing NuGet package), where T is the startup class from the web application I want to run. “Unfortunately”, in the new templates, the Program.cs file uses “top level statements” when setting up the application. This means that there is no Program type for me to reference. To fix that, I add a partial Program class to the Program.cs file. Like this
...
app.Run();
public partial class Program
{
}
This fixes that issue, and allows me to create the WebApplicationFactory<T> instance.
Comment: Yes, this is weird, but it works!
The next step is to make some changes to the services that are added to the application during start up. For example, I need to replace the existing ApiContext registration with one that uses the connection string I want to use. I also need to make it singleton, so that I can work with it in my test.
Note: Making the context singleton is fine, as each test method gets its own implementation anyway, as they all create their own WebApplicationFactory<T>. Just don’t start sharing the WebApplicationFactory<T> instance between tests…
To do this, I use the WithWebHostBuilder() method to reconfigure the host. Inside this method, I can register the services I need, using a method called ConfigureTestServices()
It looks like this
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureTestServices(services => {
var options = new DbContextOptionsBuilder<ApiContext>()
.UseSqlServer(TestingConstants.SqlConnection)
.Options;
services.AddSingleton(options);
services.AddSingleton<ApiContext>();
services.AddSingleton(NotificationServiceFake);
});
});
As you can see, I am also adding a NotificationServiceFake service. This is a globally available INotificationService fake created using FakeItEasy like this
private INotificationService NotificationServiceFake = A.Fake<INotificationService>();
Note: The ConfigureTestServices method is an extension method in the Microsoft.AspNetCore.TestHost namespace, which allows us to replace services that were added in the Program class with versions that we want to use in the test.
Once the web server has been set up, it is time to call the API and see if it returns a 401.
To do this, I need to get hold of an HttpClient. This is quite easily done by calling CreateClient() on the WebApplicationFactory instance. And when I have an HttpClient instance, it is just a matter or calling the API, and verifying the result. Like this
var client = application.CreateClient();
var response = await client.GetAsync("/users");
Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
As you can see, the interaction with the server, using the HttpClient, is just like you would “in the real world”.
Running a test with authentication
The next step is to get a test that passes the required authentication data to actually get a response.
For this, I create a new test called Get_returns_all_users(), and copy over the WebApplicationFactory<T> set up from the previous test.
However, for this to work, the set-up of the web server has to be a bit different. Not only do I need to set up the ApiContext and INotificationService, I also need to set up some new authentication infrastructure.
Luckily, this is just a matter of adding a new authentication scheme in the ConfigureTestServices() method. In my case one that supports Basic authentication.
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureTestServices(services => {
// DB and INotificationService set up
services.AddAuthentication()
.AddBasicAuthentication(credentials =>
Task.FromResult(credentials.username == "Test" && credentials.password == "test")
);
});
});
One this new authentication scheme is in place, I also need to tell the auth policy to take it into account when evaluating the user. To do this, all I really need to do, is to set the DefaultPolicy to a copy of itself, but with the new authentication scheme added. Like this
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureTestServices(services => {
// DB and INotificationService set up
// Basic Auth code
services.AddAuthorization(config =>
{
config.DefaultPolicy = new AuthorizationPolicyBuilder(config.DefaultPolicy)
.AddAuthenticationSchemes(BasicAuthenticationDefaults.AuthenticationScheme)
.Build();
});
});
});
That’s it! The server should now take into account Basic authentication.
The next step is to set up the required database stuff to support the transactional functionality I have been talking about.
The first step, is to add a transaction to the ApiContext. This requires me to get hold of the ApiContext instance being used in the application, which is done using the following code
using (var services = application.Services.CreateScope())
{
IDbContextTransaction? transaction = null;
try
{
var ctx = services.ServiceProvider.GetRequiredService<ApiContext>();
transaction = ctx.Database.BeginTransaction();
... // More code
}
finally
{
transaction?.Rollback();
}
}
As you can see, I need to use the WebApplicationFactory.Services property to create what is called an IServiceScope. This is what gives me access to the services in the IoC container.
Note: This is the same thing that happens every time a web request comes into the web server, which is why the ApiContext needs to be a singleton. Otherwise, the test would get its own scoped ApiContext instance, which would be different from the one being used when handling my test request.
Once the ApiContext has been retrieved, it is just a matter of calling and extension method called Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.BeginTransaction() on the Database property to start a transaction. This transaction is then stored throughout the test, and rolled back at the end, resetting the database.
Once the transaction part has been configured, the next step is to populate the database with the required data. This can be done by asking the ApiContext for a database connection, and then using that connection to create a database command. Like this
var conn = ctx.Database.GetDbConnection();
using (var cmd = conn.CreateCommand())
{
cmd.Transaction = transaction.GetDbTransaction();
cmd.CommandText = "SET IDENTITY_INSERT Users ON; " +
"INSERT INTO Users (Id, FirstName, LastName) VALUES" +
"(1, 'John', 'Doe'), " +
"(2, 'Jane', 'Doe'); " +
"SET IDENTITY_INSERT Users OFF;";
await cmd.ExecuteNonQueryAsync();
}
As you can see, the command needs to have the current transaction added to it, to work. However, the transaction property is of type IDbContextTransaction, and the command needs a DbTransaction. To fix this, I use the Microsoft.EntityFrameworkCore.Storage.DbContextTransactionExtensions.GetDbTransaction() extension method to get hold of the correct type.
Important: It is important to not add a using statement to the connection, as this needs to stay open even after we are done with adding the data.
Note: You might also want to note the SET IDENTITY_INSERT statements. These need to be there to allow me to insert a specific ID for the users, which simplifies the testing.
You might wonder why I use SQL commands instead of the DbContext to set up the database. And the reason for that, is that I want the data to come from the database in the same way that it would in the real world. Adding it using the DbContext also causes the context to cache it, which is not what I want for my test…
Once the data is in place, I can create an HttpClient instance by calling CreateClient() on the WebApplicationFactory instance. And then finally, make the actual request, and verify the result. However, as there is now authentication in place, I also need to make sure that I add the required Authorize header
var client = application.CreateClient();
var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes("Test:test"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodedAuthenticationString);
var response = await client.GetAsync("/users");
dynamic users = JArray.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal(2, users.Count);
Assert.Equal("John", (string)users[0].firstName);
Assert.Equal("Doe", (string)users[1].lastName);
I also choose to parse the response as a dynamic JSON array, instead of deserializing it to an array of C# objects. The reason for this, is that I want to make sure that the format and casing of the JSON is correct. This is important for the clients talking to this API.
That’s it!
Adding multiple tests
There are obviously a lot more tests I want to run on the UserController. For example, I might want to test if I can add a user. That would look something like this
[Fact]
public async Task Put_returns_Created_if_successful()
{
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureTestServices(services => {
var options = new DbContextOptionsBuilder<ApiContext>()
.UseSqlServer(TestingConstants.SqlConnection)
.Options;
services.AddSingleton(options);
services.AddSingleton<ApiContext>();
services.AddSingleton(NotificationServiceFake);
});
});
using (var services = application.Services.CreateScope())
{
IDbContextTransaction? transaction = null;
try
{
var ctx = services.ServiceProvider.GetRequiredService<ApiContext>();
transaction = ctx.Database.BeginTransaction();
var client = application.CreateClient();
var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes("Test:test"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodedAuthenticationString);
var response = await client.PutAsJsonAsync("/users/", new { firstName = "John", lastName = "Doe" });
dynamic user = JObject.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal("John", (string)user.firstName);
Assert.Equal("Doe", (string)user.lastName);
Assert.Equal(System.Net.HttpStatusCode.Created, response.StatusCode);
Assert.Matches("^http:\\/\\/localhost\\/users\\/\\d+$", response.Headers.Location!.AbsoluteUri.ToLower());
var userId = int.Parse(response.Headers.Location!.PathAndQuery.Substring(response.Headers.Location!.PathAndQuery.LastIndexOf("/") + 1));
var conn = ctx.Database.GetDbConnection();
using (var cmd = conn.CreateCommand())
{
cmd.Transaction = transaction.GetDbTransaction();
cmd.CommandText = $"SELECT TOP 1 * FROM Users WHERE Id = {userId}";
using (var rs = await cmd.ExecuteReaderAsync())
{
Assert.True(await rs.ReadAsync());
Assert.Equal("John", rs["FirstName"]);
Assert.Equal("Doe", rs["LastName"]);
}
}
}
finally
{
transaction?.Rollback();
}
}
}
There are a couple of things to note here. First of all, there is a lot of repetition between the tests, and I will get back to that. But I especially want you to note the last part of the test. This parses the ID of the created user from the Location HTTP header, and then uses that ID to query the database to see if the user has been added properly.
Note: Not everyone agrees with verifying the contents of the database, but I actually like doing that in some cases, just to verify that it did add what was expected. In this case, it doesn’t do a lot, but in more complex scenarios, you might want to verify that dependent objects are added etc.
Verifying use of mocked services
The last thing to have a look at, is how I can verify that removed dependencies are actually called as expected. For this, I added the NotificationServiceFake that I mentioned before. This allows me to verify that a call to that service has been made as expected, even if I can’t rely on that external service during testing.
A test to verify that could look like this
[Fact]
public async Task Put_returns_sends_notification_if_successful()
{
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureTestServices(services => {
var options = new DbContextOptionsBuilder<ApiContext>()
.UseSqlServer(TestingConstants.SqlConnection)
.Options;
services.AddSingleton(options);
services.AddSingleton<ApiContext>();
services.AddSingleton(NotificationServiceFake);
});
});
using (var services = application.Services.CreateScope())
{
IDbContextTransaction? transaction = null;
try
{
var ctx = services.ServiceProvider.GetRequiredService<ApiContext>();
transaction = ctx.Database.BeginTransaction();
var client = application.CreateClient();
var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes("Test:test"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", base64EncodedAuthenticationString);
var response = await client.PutAsJsonAsync("/users/", new { firstName = "John", lastName = "Doe" });
A.CallTo(() =>
NotificationServiceFake.SendUserCreatedNotification(A<User>.That.Matches(x => x.FirstName == "John" && x.LastName == "Doe"))
).MustHaveHappened();
}
finally
{
transaction?.Rollback();
}
}
}
Once again, lots of repetition, but also a new verification. Using FakeItEasy, or more or less any other mocking framework, you can easily verify that a call has been made as expected during the execution of your request.
Removing repetition
Now, getting back to that repetition. Obviously, with set up like this, there will be a lot of repetition. The easiest way to fix this, is to move the repetitive code into a helper method, or into extension methods. Like this
public class UsersControllerTests
{
public async Task Get_returns_401_Unauthorized_if_not_authenticated()
{
var application = GetWebApplication();
var client = application.CreateClient();
client.AddAuthentication("Test", "test") // Extension method
// Make request and validate
}
private WebApplicationFactory<Program> GetWebApplication()
=> new WebApplicationFactory<Program>().WithWebHostBuilder(builder =>
{
builder.ConfigureTestServices(services =>
{
var options = new DbContextOptionsBuilder<ApiContext>()
.UseSqlServer(SqlConnectionString)
.Options;
services.AddSingleton(options);
services.AddSingleton<ApiContext>();
services.AddSingleton(NotificationServiceFake);
services.AddAuthentication()
.AddBasicAuthentication(credentials => Task.FromResult(credentials.username == Username && credentials.password == Password));
services.AddAuthorization(config =>
{
config.DefaultPolicy = new AuthorizationPolicyBuilder(config.DefaultPolicy)
.AddAuthenticationSchemes(BasicAuthenticationDefaults.AuthenticationScheme)
.Build();
});
});
});
}
And with many test classes, you might want to move that into a base class to make sure you don’t add repetition back in.
However, I have played around with 2 other approaches as well.
The first one is to use a base class with a twist. Instead of just exposing a GetWebApplication() method, it can expose a bit funkier syntax. Something that ends up looking like this
public class UsersControllerTestsWithTestBase : UserControllerTestBase
{
[Fact]
public Task Get_returns_401_Unauthorized_if_not_authenticated()
=> RunTest(
test: async client =>
{
var response = await client.GetAsync("/users");
Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode);
}
, addAuth: false);
public Task Get_returns_all_users()
=> RunTest(
populateDatabase: async cmd =>
{
cmd.CommandText = "SET IDENTITY_INSERT Users ON; " +
"INSERT INTO Users (Id, FirstName, LastName) VALUES" +
"(1, 'John', 'Doe'), " +
"(2, 'Jane', 'Doe'); " +
"SET IDENTITY_INSERT Users OFF;";
await cmd.ExecuteNonQueryAsync();
},
test: async client =>
{
var response = await client.GetAsync("/users");
dynamic users = JArray.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal(2, users.Count);
Assert.Equal("John", (string)users[0].firstName);
Assert.Equal("Doe", (string)users[1].lastName);
}
);
[Fact]
public Task Put_returns_Created_if_successful()
{
var userId = -1;
return RunTest(
test: async client =>
{
var response = await client.PutAsJsonAsync("/users/", new { firstName = "John", lastName = "Doe" });
dynamic user = JObject.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal("John", (string)user.firstName);
Assert.Equal("Doe", (string)user.lastName);
Assert.Equal(System.Net.HttpStatusCode.Created, response.StatusCode);
Assert.Matches("^http:\\/\\/localhost\\/users\\/\\d+$", response.Headers.Location!.AbsoluteUri.ToLower());
userId = int.Parse(response.Headers.Location!.PathAndQuery.Substring(response.Headers.Location!.PathAndQuery.LastIndexOf("/") + 1));
},
validateDatabase: async cmd =>
{
cmd.CommandText = $"SELECT TOP 1 * FROM Users WHERE Id = {userId}";
using (var rs = await cmd.ExecuteReaderAsync())
{
Assert.True(await rs.ReadAsync());
Assert.Equal("John", rs["FirstName"]);
Assert.Equal("Doe", rs["LastName"]);
}
}
);
}
}
As you can see, the actual test becomes a lot cleaner. Allowing me to do the pre-test preparation of the database, the actual test and finally the post-test database validation, using a single base class method called RunTest().
Note: There are actually 2 base classes involved. One to do the actual test stuff, and one that adds the stuff specific for these tests. This gives us the ability to re-use a bit more of the code across multiple test classes, even if they have different service requirements.
Comment: All code is available on GitHub. The link is provided further down in the post.
The second approach is to use a helper class, with a fluent syntax, which allows for a bit more flexibility. And cool syntax. However, the code for the helper class becomes a lot more complex than the base class approach. I still like it a lot though, due to the added flexibility. And the ability to extend it using extension methods etc. It ends up looking like this
public class UsersControllerTestsWithTestHelper
{
private INotificationService NotificationServiceFake = A.Fake<INotificationService>();
[Fact]
public async Task Get_returns_all_users()
{
await GetTestRunner()
.PrepareDb<ApiContext>(async cmd => {
await cmd.AddUser(1, "John", "Doe");
await cmd.AddUser(2,"Jane", "Doe");
})
.Run(async client => {
var response = await client.GetAsync("/users");
dynamic users = JArray.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal(2, users.Count);
Assert.Equal("John", (string)users[0].firstName);
Assert.Equal("Doe", (string)users[1].lastName);
});
}
[Fact]
public async Task Put_returns_Created_if_successful()
{
var userId = -1;
await GetTestRunner()
.Run(async client => {
var response = await client.PutAsJsonAsync("/users/", new { firstName = "John", lastName = "Doe" });
dynamic user = JObject.Parse(await response.Content.ReadAsStringAsync());
Assert.Equal("John", (string)user.firstName);
Assert.Equal("Doe", (string)user.lastName);
Assert.Equal(System.Net.HttpStatusCode.Created, response.StatusCode);
Assert.Matches("^http:\\/\\/localhost\\/users\\/\\d+$", response.Headers.Location!.AbsoluteUri.ToLower());
userId = int.Parse(response.Headers.Location!.PathAndQuery.Substring(response.Headers.Location!.PathAndQuery.LastIndexOf("/") + 1));
})
.ValidatePostTestDb<ApiContext>(async cmd => {
cmd.CommandText = $"SELECT TOP 1 * FROM Users WHERE Id = {userId}";
using (var rs = await cmd.ExecuteReaderAsync())
{
Assert.True(await rs.ReadAsync());
Assert.Equal("John", rs["FirstName"]);
Assert.Equal("Doe", rs["LastName"]);
}
});
}
private TestHelper<Program> GetTestRunner(bool addClientAuth = true)
{
var helper = new TestHelper<Program>()
.AddDbContext<ApiContext>(SqlConnectionString)
.AddFakeAuth("Test", "test")
.OverrideServices(services => {
services.AddSingleton(NotificationServiceFake);
});
if (addClientAuth)
helper.AddFakeClientAuth("Test", "test");
return helper;
}
}
As you can see, once again the code includes a lot less repetition, and becomes a lot cleaner. It’s also a bit more flexible than the base class approach, as I can for example add multiple DbContexts for individual tests if needed. The fluent syntax also makes for easy reading and understanding of what happens in my opinion.
All 3 approaches are available on my GitHub account. The “standard” implementation is in the UsersControllerTests.cs file, while the base class and helper versions are available in the UsersControllerTestsWithTestBase.cs and UsersControllerTestsWithTestHelper.cs files. They all test the same things, just in different ways.
Configuration
One question that keeps popping up when it comes to this kind of testing is “how do you handle configuration?”. And to that question, there are basically 2 answers.
The first one is to set the environment name to something unique, like “Test” for example, and let the web application factory load in a config file based on that.
Note: ASP.NET Core will by default load configuration data from a file called appsettings.json, as well as one called appsettings.{EnvironmentName}.json. So, if you set the environment name to “Test”, it will load the appsettings.Test.json file as well.
It would look something like this
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.UseEnvironment("Test");
...
});
The second one is to add your configuration through code. This is done by using the AddInMemoryCollection() method on the IConfigurationBuilder instance. It would look something like this
var application = new WebApplicationFactory<Program>().WithWebHostBuilder(builder => {
builder.ConfigureAppConfiguration((ctx, config) => {
config.AddInMemoryCollection(new Dictionary<string, string> {
...
});
});
...
});
Both version work equally well. However, the in-memory version obviously has the benefit of allowing you to test your application using multiple different configurations.
The sample project on GitHub has a simple SettingsController and some tests to demonstrate the use the different options.
Conclusion
Integration testing ASP.NET Core APIs including a database and authentication doesn’t have to be hard. Sure, it adds a few extra things to for us solve, but in the end, it offers so much more confidence, that I really think it should be used a lot more than it is. I would definitely be willing to sacrifice quite a few unit tests for having more integration tests like this.
However, I can’t stress how important it is that you use some form of set up that allows you to write your tests easily, and without too much repetition. And, since there is a bit of set-up that needs to be done, keeping it somewhat “DRY” is important. This is why I wanted to show the examples with the base class and with the helper class, as they both help out quite a bit in this area.
And sure, the example is very basic, and you probably need a lot more stuff in your test setup to get it to work. But this should be a pretty good starting point.
If you have any thoughts or comments, please reach out! I’m on Twitter as @ZeroKoll. I’m particularly curious about your opinion regarding base class vs helper for this kind of stuff.
