Database

#CASBAN6: the DTOs and mappings

#CASBAN6: the DTOs and mappings

We already have created our database and our entities, so let’s have a look at how we bring the data to our API consuming applications.

If we recap, our entity models contain all the relations and identifiers. This could lead to some issues like circular references during serialization and unnecessary data repetition. Luckily for us, there is already a solution for this—it’s called data transfer object (DTO). The main purposes of a DTO is to serve data while being serializable (see also Wikipedia).

The DTO project

If you have been following along, you might already have guessed that I have created a separate project for the DTO model classes. The overall structure is similar to what you have already seen in my last post, where I showed you the entity model.

Implementation

Let’s have an exemplary look at the Medium entity class:

using System;
using System.Collections.Generic;

namespace MSiccDev.ServerlessBlog.EntityModel
{
    public class Medium
    {
        public Guid MediumId { get; set; }

        public Uri MediumUrl { get; set; }

        public string AlternativeText { get; set; }

        public string Description { get; set; }

        public Guid MediumTypeId { get; set; }
        public MediumType MediumType { get; set; }

        public Guid BlogId { get; set; }
        public Blog Blog { get; set; }

        public ICollection<Post> Posts { get; set; }
        public ICollection<Author> Authors { get; set; }

        public List<PostMediumMapping> PostMediumMappings { get; set; }

    }
}

The entity contains all relationships on the database. Our API will constrain a lot of them already down (we will see in a later post how), for example by requiring the BlogId for every call as primary identifier. There are a lot of other connection points, but we also want to be able to use the Medium endpoint just for managing media.

Here is the Medium DTO:

using System;
namespace MSiccDev.ServerlessBlog.DtoModel
{
    public class Medium
    {
        public Guid MediumId { get; set; }

        public Uri MediumUrl { get; set; }

        public string AlternativeText { get; set; }

        public string Description { get; set; }

        public MediumType MediumType { get; set; }

        public bool? IsPostImage { get; set; } = null;
    }
}


The class contains all the information we need. With this DTO, we will be able to manage media files alone but also in its usage context (which is mostly within posts of a blog).

Mapping helpers

To convert entity objects to data transfer objects and vice versa, we are using mappings. Mappings are converters that bring the data into the desired shape. On the contrary to our model classes, mappings are allowed to modify data during the conversion.

No library this time

If you are wondering why I am not using one of the established libraries for mappings, there are several reasons. When I came to the point of DTO implementation in the developing process, I evaluated the options for the mappings.

All of them had quite a learning curve, in the end, I was faster writing my own mappings. On bigger systems like shops or similar projects, I would probably have chosen the other path. There is also a small chance I change my mind one day, which would result in a refactoring session then.

Both mapping helper classes are, once again, in their own project.

Converting entities to DTOs

As you can see in the EntityToDtoMapExtensions class, I created extension methods for all entity objects. To remain on the Medium class, here are the particular implementations (there should be no surprise):

public static DtoModel.Medium ToDto(this EntityModel.Medium entity)
{
    return new DtoModel.Medium()
    {
        MediumId = entity.MediumId,
        MediumType = entity.MediumType.ToDto(),
        MediumUrl = entity.MediumUrl,
        AlternativeText = entity.AlternativeText,
        Description = entity.Description
    };
}

public static DtoModel.MediumType ToDto(this EntityModel.MediumType entity)
{
    return new DtoModel.MediumType()
    {
        MediumTypeId = entity.MediumTypeId,
        MimeType = entity.MimeType,
        Name = entity.Name,
        Encoding = entity.Encoding
    };
}

You may have noticed that I am not setting the IsPostImage property from within the extension. The information is only important in the context of a post, which is why the ToDto method for the post is setting it to true or false. Otherwise, it will be null and can be omitted in the API response.

Converting DTOs to entities

There are two scenarios where we need to convert DTOs to entities: one is the creation of new entities, the other is updating existing entities. Being very creative with the names, I implemented a CreateFrom and an UpdateWith method for each DTO type.

You can have a look at all implementations on Github, like above, here we are focusing on the Medium DTO extensions:

public static EntityModel.Medium CreateFrom(this DtoModel.Medium dto, Guid blogId)
{
    return new EntityModel.Medium()
    {
        BlogId = blogId,
        MediumId = dto.MediumId,
        MediumTypeId = dto.MediumType?.MediumTypeId ?? default,
        MediumUrl = dto.MediumUrl,
        AlternativeText = dto.AlternativeText,
        Description = dto.Description,
    };
}

public static EntityModel.Medium UpdateWith(this EntityModel.Medium existingMedium, DtoModel.Medium updatedMedium)
{
    if (existingMedium.MediumId != updatedMedium.MediumId)
        throw new ArgumentException("MediumId must be equal in UPDATE operation.");

    if (existingMedium.AlternativeText != updatedMedium.AlternativeText)
        existingMedium.AlternativeText = updatedMedium.AlternativeText;

    if (existingMedium.Description != updatedMedium.Description)
        existingMedium.Description = updatedMedium.Description;

    if (existingMedium.MediumTypeId != updatedMedium.MediumType.MediumTypeId)
        existingMedium.MediumTypeId = updatedMedium.MediumType.MediumTypeId;

    if (existingMedium.MediumUrl != updatedMedium.MediumUrl)
        existingMedium.MediumUrl = updatedMedium.MediumUrl;

    return existingMedium;
}

Once again, there should be no surprise in the implementation. If you have a look at the other methods, you will find them implemented similarly.

Conclusion

In this post, I explained why we need DTOs and showed you how I implemented them. We also had a look at the mapping extensions to convert the entities to data transfer objects and vice versa. Now that we have them in place, we are able to start implementing our Azure Functions, which is where we are heading to next in the #CASBAN6 blog series.

Until the next post, happy coding, everyone!

Posted by msicc in Azure, Database, Dev Stories, 2 comments
#CASBAN6: Implementing the data model using EntityFramework Core (separate libraries)

#CASBAN6: Implementing the data model using EntityFramework Core (separate libraries)

EntityModel library

When I started the project, I started with creating the classes for all the tables that I need for my blog engine. I have put them into their own library to keep things clean.

The classes also use ICollection references for relationships whenever required. Let’s have a look at the Blog class:

public class Blog 
{
    public Guid BlogId { get; set; }

    public string Name { get; set; }

    public string Slogan { get; set; }

    public Uri LogoUrl { get; set; }

    public ICollection<Post> Posts { get; set; }

    public ICollection<Author> Authors { get; set; }

    public ICollection<Tag> Tags { get; set; }

    public ICollection<Medium> Media { get; set; }

}

The other classes are implemented similarly to reflect the data model I showed you in my last post. You can have a look at the other class implementations in the GitHub repo (folder: EntityModel).

EFCore library

The EFCore library has three main components:

  1. BlogContext
  2. Configurations
  3. Seed extension

BlogContext

The BlogContext is straight forward and follows the pattern described here in the documentation for using a factory (spoiler: we will do that later):

public sealed class BlogContext : DbContext
{
    public BlogContext(DbContextOptions<BlogContext> options) : base(options)
    {

    }

    public DbSet<Blog> Blogs { get; set; }
    public DbSet<Post> Posts { get; set; }
    public DbSet<Author> Authors { get; set; }
    public DbSet<MSiccDev.ServerlessBlog.EntityModel.Medium> Media { get; set; }
    public DbSet<MediumType> MediaTypes { get; set; }
    public DbSet<Tag> Tags { get; set; }
}

The class is declaring a constructor that uses the DBContextOptions<DBContext> parameter that allows our factory to configure the context later on for the migrations. Of course, we need references to all the possible DbSets as well to be able to access them via the BlogContext instance.

Configurations

In Entity Framework, we can configure our tables with configurations. By implementing the IEntityTypeConfiguration interface for all of our models in a separate file for each, we are continuing to keep our code clean and easily maintainable. Here is how the implementation for the Blog table:

public class BlogConfiguration : IEntityTypeConfiguration<Blog>
{
    public void Configure(EntityTypeBuilder<Blog> builder)
    {
        builder.Property(nameof(Blog.BlogId)).
            IsRequired();

        builder.Property(nameof(Blog.BlogId)).
            ValueGeneratedOnAdd();

        builder.HasKey(blog => blog.BlogId).
            HasName($"PK_{nameof(Blog.BlogId)}");

        builder.Property(nameof(Blog.Name)).
            HasMaxLength(255).
            IsRequired();

        builder.Property(nameof(Blog.Slogan)).
            HasMaxLength(255).
            IsRequired();

        builder.Property(nameof(Blog.LogoUrl)).
            IsRequired();
    }
}

Most of the fluent implementations above are self-explaining. The other classes of the project are implemented in a similar way, you can find them here in the Github repository.

I implemented my configurations by following the docs, which I absolutely recommend reading:

To apply the configurations, we need to override the OnModelCreating method:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.ApplyConfiguration(new BlogConfiguration());
    modelBuilder.ApplyConfiguration(new MediumypeConfiguration());
    modelBuilder.ApplyConfiguration(new MediumConfiguration());
    modelBuilder.ApplyConfiguration(new AuthorConfiguration());
    modelBuilder.ApplyConfiguration(new TagConfiguration());
    modelBuilder.ApplyConfiguration(new PostConfiguration());
}

Seed extension

To verify our configurations are working, we need some test data. This is where the seeding feature of EF Core comes in handy, and it helped me to improve my data model a lot. I am using the extension method approach here to implement a blog with three test posts, including all relations, constraints, and property configurations. You can find the full implementation here in the Github repository.

Besides the docs on EF Core data seeding, these links helped me to understand and write my seed implementation:

With this extension method in place, applying the seed is just one line of code at the end of the OnModelCreating override away:

modelBuilder.Seed();

Now we have everything together, we finally can turn to actually migrate our code to database.

EFCore.DesignDummy library

Because I am running this whole thing on a Mac, I need to use the CLI tools for all migrations. To keep also this step in its own library, I created a DesignDummy library which I use for pushing the migrations to my local database.

The library requires a reference to the EFCore library as well as to the EntityModel library. On top of that, we need the Microsoft.EntityFrameworkCore.Design NuGet package.

Now that our dependencies are in place, we just need to create an implementation of the IDesignTimeDbContextFactory interface as described here in the docs:

public class BlogContextFactory : IDesignTimeDbContextFactory<BlogContext>
{
    public BlogContext CreateDbContext(string[] args)
    {
        BlogContext? instance = null;

        var optionsBuilder = new DbContextOptionsBuilder<BlogContext>();

        optionsBuilder.UseSqlServer(dbContextBuilder =>
            dbContextBuilder.MigrationsAssembly("EFCore.DesignDummy")).
            EnableSensitiveDataLogging();

        instance = new BlogContext(optionsBuilder.Options);

        return instance;
    }
}

Now let’s create our first migration and push it to the database (find the docs here). In your terminal window, change to the folder of your dummy project. Once you’re in the correct folder, create a new migration with the add command:

dotnet ef migrations add {MigrationName}

To push the migration you just created to the database, use the update command with the connection parameter:

dotnet ef database update --connection 'Data Source=localhost;Initial Catalog=localDB;User ID=sa;Password=thisShouldB3Stronger!'

If all goes well, you should now be able to view your database with the seeded data:

database seeded

Conclusion

In this post, I showed you how to create the model for the database and their matching IEntityTypeConfiguration implementations. We learned how to create a IDesignTimeDbContextFactory and how to add migrations and push them to the database. The full code is on GitHub for your further exploration.

As always, I hope this post is helpful for some of you.

Until the next post, happy coding, everyone!

Posted by msicc in Azure, Database, Dev Stories, 3 comments
#CASBAN6: the data model explained

#CASBAN6: the data model explained

Preface

Initially, this post should have been about the direct implementation and design of the data model for my serverless blog engine. As the data model became a bit more complex, I decided to split the data model post into two posts. The first aims to explain the data model, while the second post is for the implementation with Entity Framework Core.

The data model

A picture is worth a thousand words, they say. So here is a complete picture of the data model:

CASBAN6 data model

I will go through the model table by table and tell you a sentence or two on each of it for the rest of this post.

The tables

__EFMigrationsHistory

This table just stores all the MigrationIds and is handled by the Entity Framework. I recommend you to not touch this table.

Blogs

In theory, the database could hold more than one blog. By adding a new row to this table, you are creating a new blog in the database. The BlogId is essential for a range of other tables.

Authors

To be able to issue blog posts, our blog needs at least one author. As you may have more than one person to fill your blog, a collection of authors can be saved within this table. One author can only be assigned to one blog (at least for now, this is intentional).

Posts

The content fuelling our blog is in the posts table. One post can only be on one blog. The published date will be set on insert, all subsequent changes will modify the LastModified column. Also, one post can only have one author. The author can be replaced on updating the row in the table.

The slug can be used to create a human-readable URL for the post (instead of the PostId, which is a GUID). The slug must be unique across all blogs.

Tags

In order to group and categorize posts, I decided to go with a tags-only approach (unlike other platforms, which allow both categories and tags). Tags are unique to a blog, but can be used in multiple posts.

Media

Of course, our blog should support also media content like images, videos and other types. I opted in for a URL-based approach, which is making it easier to add content from other platforms (like videos hosted on dedicated platforms). A medium can be used in several posts.

MediaTypes

To make it a bit easier to determine a medium’s type, I added the MediaTypes table. It holds information about the MIME-Type and possibly also the encoding of a file. The uniqueness is based off the MIME-Type.

Mapping tables

As we learned already above, both tags and media can be used in multiple posts. At the same time, posts can have multiple media and also multiple tags. To cover this many-to-many relationships, I use two mapping-tables with a composite primary key to ensure their uniqueness across the blog.

Conclusion

In this post, I outlined the data model of my serverless blog engine. In the next post, I will show you the implementation of this model with Entity Framework Core.

Until the next post, happy coding, everyone!

Posted by msicc in Azure, Database, Dev Stories, 2 comments
#CASBAN6: How to set up a local Microsoft SQL database on macOS

#CASBAN6: How to set up a local Microsoft SQL database on macOS

Microsoft’s SQL Server cannot be installed directly on macOS, like on Windows machines. Luckily, there is a not so complicated solution using a Docker container – provided by Microsoft themselves.

Install Docker

Obviously, the first step is to download Docker and install it on your Mac. Just head over to the Docker website and download the appropriate version. Install the app by opening the disk image and follow the instructions.

Install SQL Server

After installing the Docker desktop client, head over to the docker hub of Microsoft’s SQL Server. You can choose between SQL Server 2017, 2019 and 2022, with the latter one being preview (as of publishing time of this post). To download the image, we need to open a terminal and download it with the pull command:

sudo docker pull mcr.microsoft.com/mssql/server:2019-latest

I am selecting 2019 here as it is closest to what Azure SQL databases uses as of publishing time of this post.

Create a server instance

Microsoft makes it quite easy to create a server instance, we just need to copy the appropriate run command from the docker hub website. I am using SQL Server Express for my testing purposes:

 docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=thisShouldB3Stronger!' -e 'MSSQL_PID=Express' -p 1433:1433 --name mssql  -d mcr.microsoft.com/mssql/server:2019-latest

Once you run this command without any error, type in docker ps to verify the image is up and running. If all goes well, you should see something like this:

Create a database

Now that we have a running server instance, we can finally create a database for our purposes. We are using this terminal command to achieve our goal:

docker exec -i mssql /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P 'thisShouldB3Stronger!' -Q 'CREATE DATABASE localDB'

We are logging into our server with this and send the command to create our database. Alternatively, we could already connect using DBeaver(link below) to create the Database. In both cases, we have our local database up and running by now.

Connect

There are several ways to connect to this database. The one we are going to use with Entity Framework Core is the good old connection string:

//template: Data Source=localhost;Initial Catalog=<database>;User ID=sa;Password=<password>

Data Source=localhost;Initial Catalog=localDB;User ID=sa;Password=thisShouldB3Stronger!

If you want to access your database with a GUI, I recommend using either Visual Studio Code with the Azure and SQL workload installed or the Community Edition of DBeaver.

DBeaver Community screenshot

Visual Studio allows connecting on a database level, while DBeaver can be used to connect at server level as well. Both of them also support access to Azure SQL databases, which will be helpful later on.

Conclusion

Microsoft’s SQL Server is not available for macOS. Nonetheless, we are able to quickly set up a Docker container that runs MS SQL and set up a local database for testing. I wrote this post for completeness of the series.

Useful links

Until the next post – happy coding, everyone!

Posted by msicc in Azure, Database, Dev Stories, 3 comments