24
Theme:

Run Umbraco on Google Cloud

posted on by Erik-Jan Westendorp

Tagged with:
Developer
Fun
GitHub
v15

Over the past two years, I’ve experimented with running Umbraco on a Kubernetes (K8S) clusters and on AWS. This year, I’m shifting focus to explore the feasibility of running Umbraco on Google Cloud Platform (GCP). Is it achievable? And if so, what are the necessary steps to set it up?

In this blog, I’ll focus on three key areas:

  1. Setting up a local/development environment.
  2. Creating a test environment.
  3. Establishing a CI/CD pipeline using GitHub Actions.

Let’s start by setting up the development environment.

Setting Up the Development Environment

To begin, we need to set up a local development environment for our Umbraco project. This involves installing Umbraco 15 with the Clean Starter Kit and configuring the required Google Cloud resources. For this setup, we’ll utilize Google Cloud Secret Manager for secure storage of secrets and Google Cloud Storage for managing Umbraco media files.

Installing Umbraco
To install Umbraco, we’ll use the script below, created with Paul Seal’s Package Script Writer tool. This script installs Umbraco with the Clean Starter Kit and uSync for configuration synchronization.


# Ensure we have the latest Umbraco templates
dotnet new install Umbraco.Templates --force

# Create solution/project
dotnet new sln --name "24-days"
dotnet new umbraco --force -n "24Days" --friendly-name "admin" --email "admin@admin.com" --password "0123456789" --development-database-type SQLite
dotnet sln add "24Days"

#Add Packages
dotnet add "24Days" package uSync
dotnet add "24Days" package Clean

dotnet run --project "24Days"
#Running

Install Script

Creating a Google Cloud Project

With Umbraco running locally, it’s time to set up a development project in Google Cloud and add the necessary resources.

  1. Visit Google Cloud Console, sign in with your Google account, and create a new project.
  2. For this example, I named the project 24-days-dev, but you can choose any name you prefer.
Screenshot of the Google Cloud Console New Project creation screen, with fields for Project Name, Project ID, and Location. A 'Create' button is visible at the bottom

Creating a new project in the Google Cloud Console

Configuring Secret Manager

  1. Navigate to Security → Secret Manager in the Google Cloud Console.
  2. Enable the Secret Manager API (you may need to set up billing first).

Enable Secret Manager API

Creating a Service Account

  1. Go to IAM & Admin → Service Accounts and create a new service account. (Example: dev-24days-service-account)
  2. Assign the role: Owner

Create Service Account

Generating a Service Account Key

  1. After creating the service account, navigate to it and open the Keys tab.
  2. Create a new key and select JSON format.
  3. This action will generate and download a JSON file, which you’ll use for authentication.

Create Service Account Key

Adding the Secret Manager NuGet Package

Add the following NuGet package to your project for interacting with Google Cloud Secret Manager:

<PackageReference Include="GCP.DotNet.Extensions.SecretManager" Version="1.0.0-beta2" />

Note

The `GCP.DotNet.Extensions.SecretManager` package is in beta. Feedback and contributions are welcome on the GitHub repository.

Configuring the Application

To use Google Cloud Secret Manager effectively in your application, we need to make some changes to ensure it integrates seamlessly into your project. This step involves creating helper methods, configuration classes, and setting up constants for better organization and maintainability. Here's what we do:

Extension Method for Configuration
We start by adding an extension method to simplify the process of retrieving configuration settings. This method ensures we can bind settings from configuration files (e.g., appsettings.json) directly into strongly-typed objects:


namespace _24Days.Extensions;

public static class ConfigurationExtensions
{
    public static T GetConfiguredInstance<T>(this IConfiguration configuration, string sectionName) where T : new()
    {
        var instance = new T();

        var section = configuration.GetSection(sectionName);
        section.Bind(instance);

        return instance;
    }
}

ConfigurationExtensions.cs

Creating a AppSettings class

Next, we create an AppSettings class to define a strongly-typed structure for our configuration. This class contains a nested SecretManager class, which holds properties for the Google Cloud project ID and credentials path.


namespace _24Days.Configuration;

public class AppSettings
{
    public class SecretManager
    {
        public string ProjectId { get; set; } = string.Empty;
        public string CredentialsPath { get; set; } = string.Empty;
    }
}

AppSettings.cs

Adding Constants for Reusability

The ProjectConstants class is added to centralize constant values used in the project, such as configuration section names. This makes it easy to update or reuse these values without scattering magic strings throughout the codebase.


namespace _24Days;

public static class ProjectConstants
{
    public static class SettingsSections
    {
        public const string SecretManager = nameof(SecretManager);
    }
}

ProjectConstants.cs

Extending WebApplicationBuilder

The WebApplicationBuilderExtensions.cs class contains a method to configure Google Cloud Secret Manager for use in the application. Here’s the method:


using _24Days.Configuration;
using _24Days.Extensions;
using GCP.DotNet.Extensions.SecretManager;

namespace _24Days.Compose;

public static class WebApplicationBuilderExtensions
{
    public static WebApplicationBuilder ConfigureGoogleCloudSecretManagerDefault(this WebApplicationBuilder builder)
    {
        var projectId = builder.Configuration.GetConfiguredInstance<AppSettings.SecretManager>(ProjectConstants.SettingsSections.SecretManager).ProjectId;

        if (string.IsNullOrWhiteSpace(projectId))
        {
            throw new InvalidOperationException("ProjectId is not configured in appsettings.json under 'SecretManager:ProjectId'.");
        }

        builder.Configuration.AddGoogleCloudSecretManager(projectId);

        return builder;
    }
}

WebApplicationBuilderExtensions.cs

Updating Configuration Files

Finally, we update the appsettings.Development.json file to include the Secret Manager configuration. For example:


{
  "$schema": "appsettings-schema.json",
  ...
  "SecretManager": {
    "ProjectId": "your_project_id"
  }
  ...
}

appsettings.Development.json

Additionally, we set up an environment variable for the Google Cloud credentials file (in the launchSettings.json):


{
  "$schema": "https://json.schemastore.org/launchsettings.json",
  ...
  "profiles": {
    "IIS Express": {
	  ...
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "GOOGLE_APPLICATION_CREDENTIALS": "path_secrets.json"
      }
    },
    "Umbraco.Web.UI": {
	  ...
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "GOOGLE_APPLICATION_CREDENTIALS": "path_secrets.json"
      }      
    }
  }
}

launchSettings.json

Update Program.cs

The final step in configuring Secret Manager is to update the Program.cs class to ensure that the extension method is invoked:


using _24Days.Compose;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.ConfigureGoogleCloudSecretManagerDefault();

builder.CreateUmbracoBuilder()
    .AddBackOffice()
    .AddWebsite()
    .AddComposers()
    .Build();

WebApplication app = builder.Build();

await app.BootUmbracoAsync();


app.UseUmbraco()
    .WithMiddleware(u =>
    {
        u.UseBackOffice();
        u.UseWebsite();
    })
    .WithEndpoints(u =>
    {
        u.UseBackOfficeEndpoints();
        u.UseWebsiteEndpoints();
    });

await app.RunAsync();

Program.cs

Adding a Secret in Google Cloud

  1. In the Google Cloud Console, navigate back to Secret Manager.
  2. Add a new secret named ConnectionStrings--umbracoDbDSN with the value from your appsettings.Development.json file.
  3. Remove the connection string from appsettings.Development.json to ensure it’s securely managed.

Testing the Setup

Run your application. If everything is configured correctly, it should retrieve the connection string from Secret Manager and function as expected.

Secret Details

Setting Up Google Cloud Storage for Media Files

With Secret Manager configured, the next step is to set up storage for media files. Instead of storing media locally, we’ll use Google Cloud Storage. Here’s how to configure it:

  1. Log in to the Google Cloud Console.
  2. Navigate to Cloud Storage and click the Create Bucket button. (If prompted, you may need to enable the Cloud Storage API.)
  3. Configure your bucket:
    • Bucket Name: Choose a name, e.g., 24-days-media.
    • Region: Select a region that’s closest to your users (e.g., Netherlands).
  4. Click Create to finish setting up the bucket.

Storage Configuration

Configure Umbraco to Use Google Cloud Storage

To ensure Umbraco stores media files in your Cloud Storage bucket, you need to install and configure a storage provider package.

Install the Storage Provider Package

Add the package `Our.Umbraco.Community.StorageProviders.GoogleCloud` to your project by adding this line to your .csproj file:

<PackageReference Include="Our.Umbraco.Community.StorageProviders.GoogleCloud" Version="1.0.0-alpha1" />

Note

This package is currently in alpha. If you encounter issues, please report them on the issue tracker.

Implement a Composer

To register the Google Cloud Storage provider with Umbraco, create a Composer. A Composer allows you to extend Umbraco’s configuration during startup. Follow these steps:

  1. Create a class named AppComposer in your project.
  2. Implement the IComposer interface.
  3. Use the AddGoogleCloudMediaFileSystem extension method from the installed package to configure media file storage.

Here’s an example:


using Our.Umbraco.Community.StorageProviders.GoogleCloud.DependencyInjection;
using Umbraco.Cms.Core.Composing;

namespace _24Days.Compose;

public class AppComposer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder.AddGoogleCloudMediaFileSystem();
    }
}

AppComposer.cs

Update AppSettings

Next, update your appsettings.Development.json file to include the storage configuration. Replace the placeholders with your bucket name and credentials path:


{
  "$schema": "appsettings-schema.json",
  ...
  "Umbraco": {
    "Storage": {
      "GoogleCloud": {
        "Media": {
          "BucketName": "my-bucket",
          "CredentialPath": "my-secret-file.json"
        }
      }
    }
  }
  ...
}

appsettings.Development.json

Note

While the service account currently has Owner permissions, it's best practice to follow the principle of least privilege by assigning only the necessary permissions. In this case, to manage the bucket effectively, the service account requires the Storage Admin role. This role provides sufficient permissions to perform bucket-level actions without granting excessive access.

Move Local Media Files to the Cloud

If you already have media files stored locally, you’ll need to move them to the Google Cloud bucket.

Test the Configuration

  1. Start your application.
  2. Try saving, uploading, or deleting media files in Umbraco’s backoffice.
  3. Verify that the changes are reflected in your Google Cloud Storage bucket.

By completing this setup, you have successfully configured Umbraco to store media files in Google Cloud Storage, ensuring that media assets are securely stored and accessible

Storage Overview

In the development environment setup, we first integrated Google Cloud Secret Manager to securely manage sensitive information, such as connection strings, without exposing them in source control. Next, we configured Google Cloud Storage for media file management. This involved creating a storage bucket, installing a Google Cloud storage provider package, and updating the application configuration. With these steps completed, we now have a secure and cloud-integrated development environment. Now it's time to setup a test environment.

Setting Up the Test Environment

Now that our development environment is fully configured, it’s time to set up a dedicated test environment. This will allow us to validate our application in a cloud-hosted environment before deploying to production. Here’s how we’ll do it:

Create a New Google Cloud Project

  1. Navigate to the Google Cloud Console.
  2. Create a new project and name it 24-days-tst.

This project will host all the resources needed for the test environment, including:

  • Secret Manager: To securely store application secrets.
  • Cloud Storage: To store media files.
  • Artifact Registry: To host Docker images of your application.
  • SQL Database: To manage your application’s data.

Set Up the Artifact Registry

  1. Switch to the 24-days-tst project in the Google Cloud Console.
  2. Search for Artifact Registry and enable the API if it’s not already active.
  3. Create a new repository with the following details:
    • Name: umbraco
    • Docker
    • Region: Choose a region close to your users (e.g., Netherlands).

This repository will store the Docker images of your application that will be deployed using Google Cloud Run.

Artifact Registry Configuration

Configure GitHub Workflow

To automate the process of building and pushing Docker images to Artifact Registry, we’ll create a GitHub Actions workflow.

Add a Dockerfile to Your Project

Include a Dockerfile in your project with the following content:


FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
ENV APP_UID=1000
USER $APP_UID
WORKDIR /app
EXPOSE 8080
EXPOSE 8081

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src

COPY ["24Days/24Days.csproj", "24Days/"]
RUN dotnet restore "./24Days/24Days.csproj"

COPY . .
WORKDIR "/src/24Days"

RUN dotnet build "./24Days.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./24Days.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .

USER root
RUN mkdir -p /app/wwwroot/media && chown -R $APP_UID:$APP_UID /app/wwwroot/media

USER $APP_UID
ENTRYPOINT ["dotnet", "24Days.dll"]

Dockerfile

This Dockerfile defines a build for the Umbraco application:

  1. It starts by setting up a lightweight runtime image (base) for the app, configuring environment variables, user permissions, working directory, and ports.
  2. Then, it uses a build image (build) to restore dependencies, build the application, and publish its output to a directory.
  3. Finally, the published files are copied to the runtime image (final), permissions for specific directories are set, and the container is configured to run the application with the .NET runtime.

Add a GitHub Workflow File

Create a workflow file .github/workflows/containerize.yml with the following content:


name: Build and Push to GCP Artifact Registry

on:
  workflow_dispatch:
    inputs:
      image_tag:
        description: 'The tag of the Docker image to deploy'
        required: true

jobs:
  build-and-push:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v3

      - name: Setup .NET SDK
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '9.0'

      - name: Build Application
        run: |
          cd src/24Days
          dotnet restore
          dotnet publish -c Release -o out

      # Authenticate to GCP
      - name: Authenticate to Google Cloud
        env:
          GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
        run: |
          echo "${GCP_SA_KEY}" | base64 --decode > gcloud-key.json
          gcloud auth activate-service-account --key-file=gcloud-key.json
          gcloud config set project ${{ secrets.GCP_PROJECT_ID }}
          gcloud auth configure-docker europe-west4-docker.pkg.dev

      - name: Configure Docker for Artifact Registry
        run: |
          gcloud auth configure-docker

      - name: Build Docker Image
        run: |
          docker build -t ${{ secrets.GCP_ARTIFACT_REGISTRY }}/app:${{ github.event.inputs.image_tag }} -f src/24Days/Dockerfile src

      - name: Push Docker Image to Artifact Registry
        run: |
          docker push ${{ secrets.GCP_ARTIFACT_REGISTRY }}/app:${{ github.event.inputs.image_tag }}

      - name: Cleanup GCP Credentials
        run: |
          rm -f gcloud-key.json

containerize.yml

This GitHub Actions workflow automates the process of building a .NET application and pushing its Docker image to Google Cloud Artifact Registry:

  1. Trigger and Inputs: The workflow is triggered manually via workflow_dispatch, requiring the user to provide a Docker image tag as input.
  2. Build the App: It checks out the code, sets up the .NET SDK, restores dependencies, and publishes the application in release mode.
  3. Authenticate and Push: It authenticates with Google Cloud using a service account key, builds a Docker image from the application, and pushes the image to the specified Artifact Registry.
  4. Cleanup: Finally, it securely removes the GCP credentials file to prevent exposure.

Configure Secrets for GitHub Workflow

As you can see the GitHub workflow requires several secrets. Here’s how to configure them:

  1. Create a Service Account:
    • Go to IAM & Admin → Service Accounts in the Cloud Console.
    • Create a service account (e.g., tst-24days-service-account) and assign the Artifact Registry Administrator role.
    • Generate a JSON key file for this account and download it. (As we did before)
  2. Encode the Key File:
  3. Convert the JSON key file to a base64 string using PowerShell
  4. Add Secrets to GitHub:
    • GCP_SA_KEY: Paste the base64 string of the JSON key.
    • GCP_PROJECT_ID: Add the test project ID (24-days-tst).
    • GCP_ARTIFACT_REGISTRY: Add the Artifact Registry path (e.g., europe-west4-docker.pkg.dev/24-days-tst/umbraco).

$bytes = [System.Text.Encoding]::UTF8.GetBytes((Get-Content -Path "path_to_json_file.json" -Raw))
[convert]::ToBase64String($bytes) > gcp-key-base64.txt

PowerShell

GitHub Secrets

Create a SQL Database

  1. In the Cloud Console, navigate to SQL and enable the API if necessary.
  2. Create a new SQL Server instance:
    • Instance ID: sql-tst-24-days
    • Region: Choose a region close to your other resources (e.g., europe-west4).
    • Database Type: Select SQL Server and choose Enterprise, Sandbox for demo purposes.
    • Password: Generate and save the password for later use.
  3. Wait for the instance to be created. Once ready, navigate to Databases and create a database for your application.

SQL Configuration

Deploying to Google Cloud Run

The final step in setting up the test environment is deploying your application to Google Cloud Run.

  1. Navigate to Cloud Run
  2. Click "Deploy Container"
    • Choose the Artifact Registry option as the source of your container image.
    • Select the image from the Artifact Registry repository (umbraco) we created earlier.
  3. Configure the Service
    • Service Name: Set a name for your service, such as umbraco-app.
    • Region: Select the same region where your other resources are located (e.g., Netherlands or europe-west4).
    • Allow Unauthenticated Invocations: Enable this option to make your application publicly accessible.
    • Number of Instances: Set the maximum number of instances to 1 for this test environment.
  4. Environment Variables
    Add your Google Cloud Project ID as an environment variable. This is essential for the application to retrieve secrets from Secret Manager.
    GOOGLE_CLOUD_PROJECT = 24-days-tst
  5. Deploy the Service
    Click Create and Deploy to deploy the containerized application to Cloud Run.

Add Secrets to Secret Manager

Before running your application, add the required secrets to the Secret Manager in your tst project. These secrets will allow your application to connect to the SQL database etc.

ConnectionStrings--umbracoDbDSN:
"Server={server_ip};Database=db-tst-24-days;User Id={user_id};Password={password};TrustServerCertificate=true;"

ConnectionStrings--umbracoDbDSN_ProviderName
"Microsoft.Data.SqlClient"

Umbraco--CMS--WebRouting--UmbracoApplicationUrl:
"https://app-{the_cloud_run_id}.europe-west4.run.app/"

Final steps
After starting the application, you will be prompted to complete the Umbraco installation. 

Ensure the connection to the SQL database is correctly configured (using the secret ConnectionStrings--umbracoDbDSN from Secret Manager).


Umbraco on Goole Cloud

In this blog, we successfully ran Umbraco on Google Cloud Platform. We integrated Google Cloud Secret Manager for secure secret storage and configured Google Cloud Storage for managing media files. We then moved to the test environment, creating essential resources such as Artifact Registry, SQL Database, and additional secrets, while automating deployment with a GitHub Actions workflow. Finally, we manually deployed the application to Google Cloud Run

Erik-Jan Westendorp
Erik-Jan Westendorp
Twitter
ewestendorp on Twitter
LinkedIn
erik-jan-westendorp on LinkedIn
GitHub
erikjanwestendorp/ on GitHub
Mastodon
@ewestendorp@umbracocommunity.social on Mastodon
Bluesky
erikjanwestendorp.bsky.social on Bluesky
UMB.FYI
@erikjanwestendorp on UMB.FYI