Creating a CI/CD pipeline with Azure Pipelines and Compute Engine

This tutorial shows how to use Azure Pipelines and Compute Engine to create a continuous integration/continuous deployment (CI/CD) pipeline.

The tutorial uses Orchard CMS, an open source content management system, as a sample application. Orchard CMS is based on ASP.NET MVC, uses Microsoft Internet Information Services (IIS), and runs on Windows Server 2016.

The CI/CD pipeline uses two separate environments, one for testing and one for production, as the following diagram shows.

Conceptual diagram of CI/CD pipeline showing how developers and end users interact with the application

At the beginning of the pipeline, developers commit changes to the example codebase. This action triggers the pipeline to create a new Windows Server 2016–based virtual machine (VM) image by using Packer. The new image is then automatically released to the development environment by using a rolling update. After testing, a release manager can then promote the release so that it's deployed into the production environment.

This tutorial is intended for developers and DevOps engineers. It assumes that you have basic knowledge of .NET Framework, Windows Server, IIS, Azure Pipelines, and Compute Engine. The tutorial also requires you to have administrative access to an Azure DevOps account.

Objectives

  • Run a private Azure Pipelines agent on Compute Engine and connect it to Azure Pipelines.
  • Use Packer with Compute Engine to create Windows images.
  • Use Compute Engine Managed Instance Groups to implement rolling deployments.
  • Set up a CI/CD pipeline in Azure Pipelines to orchestrate the building, creating, and deployment processes.

Costs

This tutorial uses the following billable components of Google Cloud:

  • Compute Engine
  • Cloud Load Balancing
  • Image storage for custom VM images
  • Cloud Storage

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Clean up.

Check the Azure DevOps pricing page for any fees that might apply to using Azure DevOps.

Before you begin

It's usually advisable to use separate projects for development and production workloads so that identity and access management (IAM) roles and permissions can be granted individually. For the sake of simplicity, this tutorial uses a single project for the CI, development, and production environments.

  1. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  2. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  3. Make sure you have an Azure DevOps account and have administrator access to it. If you don't yet have an Azure DevOps account, you can sign up on the Azure DevOps home page.

Creating an Azure DevOps project

You use Azure DevOps to manage the source code, run builds and tests, and orchestrate the deployment to GKE. To begin, you create a project in your Azure DevOps account.

  1. Go to the Azure DevOps home page (https://dev.azure.com/YOUR_AZURE_DEVOPS_ACCOUNT_NAME).
  2. Click New Project.
  3. Enter a project name, such as Orchard.
  4. Set Visibility to Private, and then click Create.
  5. After you create the project, in the menu on the left, click Repos.
  6. Click Import to fork the Orchard CMS repository from GitHub, and then set the following values:
    • Source type: Git
    • Clone URL: https://github.com/OrchardCMS/Orchard.git

  7. Click Import.

    When the import process is done, you see the source code of Orchard CMS.

Building continuously

You can now use Azure Pipelines to set up continuous integration. For each commit that you push to the Git repository, Azure Pipelines will build the code and publish the resulting build artifact to internal Azure Pipelines storage.

Create a testing branch

To make sure that the instructions in this tutorial work, you need to create a branch that's based on a specific version of the source code. This step will help make sure that future changes to the code on GitHub don't break this tutorial.

  1. In the Azure DevOps menu, select Repos > Tags.
  2. In the list of tags, hold the pointer over the tag named 1.10.2 and click the ... button that appears on the right.
  3. Select New branch.
  4. In the Name box, enter testing as the branch name, and confirm by clicking Create.

By default, Azure Pipelines expects your code to reside in the master branch. In order to have it use the testing branch, you need to change the default branch:

  1. In the Azure DevOps menu, select Repos > Branches.
  2. In the list of branches, hold the pointer over the testing branch and click the ... button that appears on the right.
  3. Select Set as default branch.

Build the code

After you create the branch, you can begin to automate the build. Because Orchard CMS is an ASP.NET application that's written in Visual Studio, defining the build includes the following steps:

  • Restoring the NuGet package dependencies.
  • Building the solution (src\Orchard.sln).
  • Publishing the build artifacts of the Orchard.Web project.

Later you'll add steps to deploy to Compute Engine. Because the application requires a Windows-based environment, you'll set up the entire build process to run on Windows-based build agents.

  1. Using Visual Studio or a command-line git client, clone your new Git repository.
  2. In the root of the repository, create a file named azure-pipelines.yml.

  3. Copy the following code and paste into the file:

    resources: - repo: self   fetchDepth: 1 trigger: - testing variables:   artifactName: 'Orchard.Web' jobs: - job: Build   displayName: Build application   condition: succeeded()   pool:     vmImage: vs2017-win2016     demands:     - msbuild     - visualstudio   variables:     solution: 'src\Orchard.sln'     buildPlatform: 'Any CPU'     buildConfiguration: 'Release'   steps:   - task: NuGetToolInstaller@0     displayName: 'Use NuGet 4.4.1'     inputs:       versionSpec: 4.4.1   - task: NuGetCommand@2     displayName: 'NuGet restore'     inputs:       restoreSolution: '$(solution)'   - task: VSBuild@1     displayName: 'Build solution'     inputs:       solution: '$(solution)'       msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"'       platform: '$(buildPlatform)'       configuration: '$(buildConfiguration)'   - task: PublishBuildArtifacts@1     displayName: 'Publish Artifact'     inputs:       PathtoPublish: '$(build.artifactstagingdirectory)/Orchard.Web.zip'       ArtifactName: '$(artifactName)'

  4. Commit your changes and push them to Azure Pipelines.

    Visual Studio

    1. Open Team Explorer and click the Home icon.
    2. Click Changes.
    3. Enter a commit message like Add build definition.
    4. Click Commit All and Push.

    Command line

    1. Stage all modified files:

                                          git add -A

    2. Commit the changes to the local repository:

                                          git commit -m "Add build definition"

    3. Push the changes to Azure DevOps:

                                          git push

  5. In Azure, select Pipelines.

    A build definition is created based on the YAML file that you committed to the Git repository.

The build takes up to seven minutes to complete. At the end of the build, the file Orchard.Web.zip, which contains all files of the web application, will be available in the internal Azure Pipelines artifact storage area.

With the source code being built continuously, the next step is to create the VM images automatically. This automation includes the following steps:

  • Launching a new, temporary VM instance that uses the Windows Server 2016 image.
  • Installing and configuring IIS.
  • Deploying Orchard CMS.
  • Stopping the VM.
  • Creating an image.
  • Deleting the temporary VM.

You use Packer to automate this process. Packer needs access to the VM instance, but you don't want to expose the instance to the public internet. Therefore, you run Packer on a private Azure Pipelines agent within the Google Cloud network.

Deploy a private Azure Pipelines agent

To minimize the overhead of administering a private agent, you deploy the agent in a managed instance group. Using this approach has these advantages:

  • A managed instance group allows you to re-create the VM, install all required components, and register with Azure Pipelines automatically.
  • You can apply Windows updates by deleting the VM and letting the managed instance group re-create the environment with the latest, fully patched Windows image as a base.

Follow these steps:

  1. In the Cloud Console, switch to your newly created project.

  2. Open Cloud Shell.

    Go to Cloud Shell

  3. To save time, set default values for your project ID and Compute Engine zone:

    gcloud config set project                            PROJECT_ID                            gcloud config set compute/zone                            ZONE

    Replace PROJECT_ID with the ID of your Google Cloud project, and replace ZONE with the name of the zone that you're going to use for creating resources. If you are unsure about which zone to pick, use us-central1-a.

    Example:

    gcloud config set project devops-test-project-12345 gcloud config set compute/zone us-central1-a

  4. Enable the Compute Engine API:

    gcloud services enable compute.googleapis.com

  5. Switch to the Azure DevOps menu, select Project settings, and then select Pipelines > Agent pools.

  6. Click Add pool.

  7. Configure the following settings:

    • Pool to link: Self-hosted
    • Name: Google Cloud

  8. Click Create.

  9. In the list of agent pools, select Google Cloud.

  10. Click New agent.

  11. Under Download the agent, click Copy to copy the download URL.

  12. In Cloud Shell, initialize an environment variable. For the value, paste the URL that you copied in the previous step.

    export AZURE_DEVOPS_AGENT_URL=PASTE TOKEN FROM CLIPBOARD

  13. Switch back to Azure Pipelines, click the user settings button in the upper-right corner of the screen, and select Personal access tokens.

  14. Select New Token.

  15. Configure the following settings:

    • Name: Google Cloud Agent
    • Scopes: Full access

  16. Click Create.

  17. Copy the token to the clipboard.

  18. In Cloud Shell, initialize another environment variable by running the following command. For the value, paste the token that you just copied.

    export AZURE_DEVOPS_TOKEN=PASTE TOKEN FROM CLIPBOARD

  19. Initialize another environment variable to contain the Azure Pipelines URL. Replace ACCOUNT with your account name, as displayed in the address bar in Azure Pipelines.

    export AZURE_DEVOPS_URL=https://dev.azure.com/ACCOUNT

  20. Run the following command to create a specialize script. The script downloads and installs the Azure Pipelines agent package, Packer, and Cloud SDK, and registers the agent with Azure Pipelines.

    cat | envsubst '$AZURE_DEVOPS_AGENT_URL $AZURE_DEVOPS_TOKEN $AZURE_DEVOPS_URL' > specialize.ps1 << 'EOF' $ErrorActionPreference = "Stop"  # Create an installation directory for the Azure Pipelines agent New-Item -ItemType directory -Path $env:programfiles\vsts-agent  # Create a work directory for the Azure Pipelines agent New-Item -ItemType directory -Path $env:programdata\vsts-agent  # Download and install the Azure Pipelines agent package [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Invoke-WebRequest `   -Uri "$AZURE_DEVOPS_AGENT_URL" `   -OutFile $env:TEMP\vsts-agent.zip Add-Type -AssemblyName System.IO.Compression.FileSystem [System.IO.Compression.ZipFile]::ExtractToDirectory( `   "$env:TEMP\vsts-agent.zip", `   "$env:programfiles\vsts-agent")  # Download and install Packer [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Invoke-WebRequest `   -Uri "https://releases.hashicorp.com/packer/1.5.5/packer_1.5.5_windows_amd64.zip" `   -OutFile $env:TEMP\packer.zip [System.IO.Compression.ZipFile]::ExtractToDirectory( `   "$env:TEMP\packer.zip", `   "$env:programfiles\packer")  # Add Packer and the Cloud SDK installation directory to global path [Environment]::SetEnvironmentVariable( `   "Path", $env:Path + ";$env:programfiles\packer;${env:ProgramFiles(x86)}\Google\Cloud SDK\google-cloud-sdk\bin", `   [System.EnvironmentVariableTarget]::Machine)  # Configure the Azure Pipelines agent & $env:programfiles\vsts-agent\bin\Agent.Listener configure `   --url $AZURE_DEVOPS_URL `   --agent "GCE Agent" `   --work $env:programdata\vsts-agent `   --pool "Google Cloud" `   --replace `   --runAsService `   --windowsLogonAccount "NT AUTHORITY\NETWORK SERVICE" `   --auth PAT `   --token $AZURE_DEVOPS_TOKEN EOF

  21. Create an instance template for the Azure Pipelines agent. Configure the instance template so that the VM instance runs specialize.ps1 as a specialize script during startup.

    gcloud compute instance-templates create vsts-agent \     --machine-type n1-standard-1 \     --image-family windows-2016-core \     --image-project windows-cloud \     --metadata-from-file sysprep-specialize-script-ps1=specialize.ps1 \     --scopes "https://www.googleapis.com/auth/compute,https://www.googleapis.com/auth/devstorage.read_write"

  22. Create a managed instance group that's based on this instance template. It might take around three minutes for the new VM instance to start and for the Azure Pipelines agent to register.

    gcloud compute instance-groups managed create vsts-agent \     --template=vsts-agent \     --size=1

  23. In Azure DevOps, navigate back to your project.

  24. In the menu, select Project settings, and then select Pipelines > Agent pools.

  25. Select the Google Cloud pool, and then verify that the agent is registered and its state shows as Online.

If the agent doesn't register within 10 minutes, there might be an issue. Try the following:

  1. In the Cloud Console, navigate to Compute Engine > VM Instances.
  2. Open the details for the VM that has the prefix vsts-agent.
  3. Click Serial port 1 (console).

The console log contains the output of the specialize script, which might help with troubleshooting. Even after the agent shows up in Azure Pipelines, it might initially appear as Offline. In that case, wait another two minutes for the status to change to Online.

Create VM images

To automate the VM image creation process, you need to create a Packer template. This template is a JSON file that you should keep with the project's source code.

  1. In Visual Studio, open Solution Explorer.
  2. In the root of the solution, create a new file named packer.json.

  3. Copy the following code into the newly created file, and then save the file:

    {   "variables": {     "gcp_project": "",     "gcp_zone": "",     "windows_user": "packer_user",     "windows_password": "Packer123",     "image_name": "vsts",     "image_family": "vsts",     "app_package": ""   },   "builders": [     {       "type": "googlecompute",       "project_id": "{{user `gcp_project`}}",       "source_image_family": "windows-2016",       "disk_size": "50",       "instance_name": "{{user `image_name`}}",       "image_name": "{{user `image_name`}}",       "image_family": "{{user `image_family`}}",       "machine_type": "n1-standard-2",       "communicator": "winrm",       "winrm_username": "{{user `windows_user`}}",       "winrm_password": "{{user `windows_password`}}",       "winrm_insecure": true,       "winrm_use_ssl": true,       "winrm_port": 5986,       "metadata": {         "windows-startup-script-cmd": "winrm quickconfig -quiet & net user /add {{user `windows_user`}} {{user `windows_password`}} & net localgroup administrators {{user `windows_user`}} /add & winrm set winrm/config/service/auth @{Basic=\"true\"}"       },       "zone": "{{user `gcp_zone`}}",       "use_internal_ip": true,       "state_timeout": "8m",       "scopes": [ "https://www.googleapis.com/auth/devstorage.read_only" ]     }   ],   "provisioners": [     {       "type": "powershell",       "inline": [         "$ErrorActionPreference = \"Stop\"",          "# Download application package from Cloud Storage",         "gsutil cp {{user `app_package`}} $env:TEMP\\app.zip",          "# Install IIS",         "Enable-WindowsOptionalFeature -Online -FeatureName `",         "        NetFx4Extended-ASPNET45, `",         "        IIS-WebServerRole, `",         "        IIS-WebServer, `",         "        IIS-CommonHttpFeatures, `",         "        IIS-HttpErrors, `",         "        IIS-HttpRedirect, `",         "        IIS-ApplicationDevelopment, `",         "        IIS-HealthAndDiagnostics, `",         "        IIS-HttpLogging, `",         "        IIS-LoggingLibraries, `",         "        IIS-RequestMonitor, `",         "        IIS-HttpTracing, `",         "        IIS-Security, `",         "        IIS-RequestFiltering, `",         "        IIS-Performance, `",         "        IIS-WebServerManagementTools, `",         "        IIS-IIS6ManagementCompatibility, `",         "        IIS-Metabase, `",         "        IIS-DefaultDocument, `",         "        IIS-ApplicationInit, `",         "        IIS-NetFxExtensibility45, `",         "        IIS-ISAPIExtensions, `",         "        IIS-ISAPIFilter, `",         "        IIS-ASPNET45, `",         "        IIS-HttpCompressionStatic",          "# Extract application package to wwwroot",         "New-Item -ItemType directory -Path $env:TEMP\\app",         "Add-Type -AssemblyName System.IO.Compression.FileSystem",         "[System.IO.Compression.ZipFile]::ExtractToDirectory(\"$env:TEMP\\app.zip\", \"$env:TEMP\\app\")",         "Remove-Item $env:TEMP\\app.zip",         "Move-Item -Path $(dir -recurse $env:TEMP\\app\\**\\PackageTmp | % { $_.FullName }) -Destination c:\\inetpub\\wwwroot\\app -force",          "# Configure IIS web application pool and application",         "Import-Module WebAdministration",         "New-WebAppPool orchard-net4",         "Set-ItemProperty IIS:\\AppPools\\orchard-net4 managedRuntimeVersion v4.0",         "New-WebApplication -Name Orchard -Site 'Default Web Site' -PhysicalPath c:\\inetpub\\wwwroot\\app -ApplicationPool orchard-net4",          "# Grant read/execute access to the application pool user",         "&icacls C:\\inetpub\\wwwroot\\app\\ /grant \"IIS AppPool\\orchard-net4:(OI)(CI)(RX)\"",          "# Create data folder and grant write access to the application pool user",         "New-Item -ItemType directory -Path C:\\inetpub\\wwwroot\\app\\App_Data\\",         "&icacls C:\\inetpub\\wwwroot\\app\\App_Data\\ /grant \"IIS AppPool\\orchard-net4:(OI)(CI)M\"",          "# Disable searching for Windows updates",         "New-ItemProperty -Path HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\WindowsUpdate\\AU -Name NoAutoUpdate -Value 1 -PropertyType DWORD -Force",          "# Disable provisioning user",         "disable-localuser {{user `windows_user`}}",          "# Generalize the image",         "& \"$Env:Programfiles\\Google\\Compute Engine\\sysprep\\gcesysprep\""       ]     }   ] }

The Packer template has three sections:

  • The variables section defines the variables that are used in the template. The values for these variables will be passed as command-line arguments to Packer.
  • The builders section contains the settings for creating and communicating with the temporary VM instance.
  • The provisioners section contains a sequence of commands that are run on the VM to configure it after it has been created.

The preceding template uses the Packer Google Compute Builder to create a VM instance that runs Windows Server 2016. When you specify the source image for Packer to use, it's important to use an image that already includes the latest security patches. To prevent you from having to change the source_image setting every time that Google releases a new version of the Windows Server 2016 image, the configuration obtains the most recent image from the windows-2016-core image family.

Using a startup script, the template enables Windows Remote Management (WinRM) and creates a temporary user. After the VM is up and running, Packer uses WinRM and this user to run the PowerShell commands that are specified in the provisioners section. Using the use_internal_ip setting, you ensure that WinRM communicates over the local network rather than over the internet, which is possible because the corresponding build step will run on the private build agent that you provisioned earlier. Finally, the builders section also defines an IAM scope for the VM, which permits access to Cloud Storage.

The PowerShell commands that are in the provisioners section configure the VM to run Orchard CMS. This configuration involves the following:

  • Downloading the application package from Cloud Storage.
  • Installing IIS.
  • Creating an IIS application pool and application for Orchard CMS.
  • Extracting the application package to the IIS webroot folder and configuring the access control lists (ACLs) for the folder.
  • Disabling the search for Windows updates, which is unnecessary if you apply updates by rerunning the CI/CD pipeline periodically.
  • Disabling the provisioning user.
  • Generalizing the image by using GCESysprep to ensure that each VM that's created from the image is assigned a unique security identifier.

To commit the file to Git, do the following:

Visual Studio

  1. Open Team Explorer and click the Home icon at upper left to switch to the Home view.
  2. Under Changes, right-click packer.json and then click Stage.
  3. Enter a commit message like Add Packer template.
  4. Click Commit Staged.

Command line

  1. Stage all modified files:

                                    git add packer.json

  2. Commit the changes to the local repository:

                                    git commit -m "Add Packer template"

Create a Cloud Storage bucket for build artifacts

Using the File Provisioner in Packer, you can copy files from the machine that's running Packer to the VM. If you use WinRM, this operation can be slow. Therefore, the template that you created in the previous section looks for the Orchard CMS application package on Cloud Storage, which offers substantially better performance.

To create a bucket in Cloud Storage for this purpose, run the following command in Cloud Shell:

gsutil mb gs://$(gcloud config get-value core/project)-artifacts

If you don't want to keep the artifacts of all builds, you might consider configuring an object lifecycle rule to delete files that are past a certain age.

Extend the build definition

You have now checked the Packer template in to Git and created the Cloud Storage bucket. You can now integrate Packer into your Azure Pipelines build process.

So far, the Azure Pipelines build definition that you've created uses a single phase. In Azure Pipelines, all tasks that are part of the same phase run on the same build agent. In your case, that agent is Hosted VS2017. To start using the private build agent that is provisioned in Compute Engine, you must add a second phase to the build definition.

Within a single phase, tasks share a working directory and have access to artifacts from previous tasks. This access is not possible across phases, because to consume artifacts from previous stages, you must publish those artifacts to Azure Pipelines in the first phase and then download them in the next phase. Therefore, extend the build definition as follows:

  1. In Visual Studio, open azure-pipelines.yml.

  2. Extend the build definition by appending the following piece of code to the file. Replace PROJECT_ID with the name of your Google Cloud project, and replace ZONE with the name of the zone that you're going to use for creating resources. If you are unsure about which zone to pick, use us-central1-a.

    - job: CreateVM   displayName: Create VM image   dependsOn: Build   condition: succeeded()   pool:     name: 'Google Cloud'   variables:     Packer.Project:                            PROJECT_ID                            Packer.Zone:                            ZONE                            steps:   - task: DownloadBuildArtifacts@0     displayName: 'Download Build Artifacts'     inputs:       artifactName: '$(artifactName)'   - task: CmdLine@2     displayName: 'Publish artifact to Cloud Storage'     inputs:       script: 'gsutil cp $(System.ArtifactsDirectory)\$(artifactName)\Orchard.Web.zip gs://$(Packer.Project)-artifacts/Orchard.Web-$(Build.BuildId).zip'   - task: CmdLine@2     displayName: 'Create image'     inputs:       workingDirectory: '$(Build.SourcesDirectory)'       script: 'packer build -var "gcp_project=$(Packer.Project)" -var "gcp_zone=$(Packer.Zone)" -var "image_family=orchard" -var "image_name=orchard-$(Build.BuildId)" -var "app_package=gs://$(Packer.Project)-artifacts/Orchard.Web-$(Build.BuildId).zip" packer.json'

To commit and push the changes to Azure DevOps, do the following:

  1. Commit your changes and push them to Azure Pipelines.

    Visual Studio

    1. Open Team Explorer and click the Home icon.
    2. Click Changes.
    3. Under Changes, right-click azure-pipelines.yml and then click Stage.
    4. Enter a commit message like Extend build definition to create VM image.
    5. Click Commit Staged and Push.

    Command line

    1. Stage all modified files:

                                          git add azure-pipelines.yml

    2. Commit the changes to the local repository:

                                          git commit -m "Extend build definition to create VM image"

    3. Push the changes to Azure DevOps:

                                          git push

  2. In the Azure DevOps menu, select Pipelines. You should see that a new build has automatically been triggered. Allow 10 to 15 minutes for the build to complete. Note that if the build fails with the error message Could not find a pool with name Google Cloud, you may need to re-save your pipeline.

  3. In Google Cloud, navigate to Compute Engine > Images and confirm that an image named orchard-N has been created, where N represents the Azure Pipelines build ID.

Deploying continuously

Now that Azure Pipelines is automatically building your code and creating a new VM image for each commit, you can turn your attention toward deployment.

Configure the development environment

With Orchard CMS, you can use either SQL Server or an embedded database that stores data locally. For the sake of simplicity, use the default configuration that relies on the embedded database, although it comes with two restrictions:

  • Only a single VM instance can run at a time. Otherwise, users might see different data depending on which VM instance is serving content to them.
  • Any data changes are lost whenever the VM instance is restarted, unless you change the deployment to use Filestore for data storage. (We do not cover this scenario in the tutorial.)

Before you can configure the steps in Azure Pipelines to automate the deployment, you must prepare the development environment. This preparation includes creating a managed instance group that will manage the web server VM instances. It also includes creating an HTTP load balancer.

  1. In Cloud Shell, create an instance template that uses a standard Windows Server 2016 Core image (not one that's been customized). You will use this template only initially, because each build will produce a new template.

    gcloud compute instance-templates create orchard-initial \     --machine-type n1-standard-2 \     --image-family windows-2016-core \     --image-project windows-cloud

  2. Create an HTTP health check. Because Orchard CMS does not have a dedicated health check endpoint, you can query the path /.

    gcloud compute http-health-checks create orchard-dev-http \     --check-interval=10s --unhealthy-threshold=10 \     --request-path=/

  3. Create a managed instance group that's based on the initial instance template. For simplicity's sake, the following commands create a zonal managed instance group. However, you can use the same approach for regional managed instance groups that distribute VM instances across more than one zone.

    gcloud compute instance-groups managed create orchard-dev \     --template=orchard-initial \     --http-health-check=orchard-dev-http \     --initial-delay=2m \     --size=1 && \ gcloud compute instance-groups set-named-ports orchard-dev --named-ports http:80

  4. Create a load balancer backend service that uses the HTTP health check and managed instance group that you created previously:

    gcloud compute backend-services create orchard-dev-backend \     --http-health-checks orchard-dev-http \     --port-name http --protocol HTTP --global && \ gcloud compute backend-services add-backend orchard-dev-backend \     --instance-group orchard-dev --global \     --instance-group-zone=$(gcloud config get-value compute/zone)

  5. Create a load balancer frontend:

    gcloud compute url-maps create orchard-dev --default-service orchard-dev-backend && \ gcloud compute target-http-proxies create orchard-dev-proxy --url-map=orchard-dev && \ gcloud compute forwarding-rules create orchard-dev-fw-rule --global --target-http-proxy orchard-dev-proxy --ports=80

  6. Create a firewall rule that allows the Google load balancer to send HTTP requests to instances that have been annotated with the gclb-backend tag. You will later apply this tag to the web service VM instances.

    gcloud compute firewall-rules create gclb-backend --source-ranges=130.211.0.0/22,35.191.0.0/16 --target-tags=gclb-backend --allow tcp:80

Configure the production environment

Setting up the production environment requires a sequence of steps similar to those for configuring the development environment.

  1. In Cloud Shell, create an HTTP health check. Because Orchard CMS does not have a dedicated health check endpoint, you can query the path /.

    gcloud compute http-health-checks create orchard-prod-http \     --check-interval=10s --unhealthy-threshold=10 \     --request-path=/

  2. Create another managed instance group that is based on the initial instance template that you created earlier:

    gcloud compute instance-groups managed create orchard-prod \     --template=orchard-initial \     --http-health-check=orchard-prod-http \     --initial-delay=2m \     --size=1 && \ gcloud compute instance-groups set-named-ports orchard-prod --named-ports http:80

  3. Create a load balancer backend service that uses the HTTP health check and managed instance group that you created previously:

    gcloud compute backend-services create orchard-prod-backend --http-health-checks orchard-prod-http --port-name http --protocol HTTP --global && \ gcloud compute backend-services add-backend orchard-prod-backend --instance-group orchard-prod --global --instance-group-zone=$(gcloud config get-value compute/zone)

  4. Create a load balancer frontend:

    gcloud compute url-maps create orchard-prod --default-service orchard-prod-backend && \ gcloud compute target-http-proxies create orchard-prod-proxy --url-map=orchard-prod && \ gcloud compute forwarding-rules create orchard-prod-fw-rule --global --target-http-proxy orchard-prod-proxy --ports=80

Configure the release pipeline

Unlike some other continuous integration systems, Azure Pipelines distinguishes between building and deploying, and it provides a specialized set of tools labeled Release Management for all deployment-related tasks.

Azure Pipelines Release Management is built around these concepts:

  • A release refers to a set of artifacts that make up a specific version of your app and that are usually the result of a build process.
  • Deployment refers to the process of taking a release and deploying it into a specific environment.
  • A deployment performs a set of tasks, which can be grouped in jobs.
  • Stages let you segment your pipeline and can be used to orchestrate deployments to multiple environments—for example, development and testing environments.

Usually, an Azure Pipelines release consumes an artifact such as a zip file from a build. In this tutorial, the build produces a VM image in GCP, so there is no artifact like that to consume. However, by using the build ID, which is passed to the release as an environment variable, you can locate the corresponding VM image and use it for the deployment.

Create a release definition

The first step is to create a new release definition.

  1. In the Azure DevOps menu, select Pipelines > Releases.
  2. Click New pipeline.
  3. From the list of templates, select Empty job.
  4. When you're prompted for a name for the stage, enter Dev.
  5. At the top of the screen, name the release Orchard-ComputeEngine.
  6. In the pipeline diagram, next to Artifacts, click Add.

  7. Select Build and add the following settings:

    • Source: Select the Git repository that contains the azure-pipelines.yml file.
    • Default version: Latest
    • Source alias: Orchard

  8. Click Add.

  9. On the Artifact box, click Continuous deployment trigger (the lightning bolt icon) to add a deployment trigger.

  10. Under Continuous deployment trigger, set the switch to Enabled.

  11. Click Save.

  12. Enter a comment if you want, and then confirm by clicking OK.

The pipeline now looks like this:

Screenshot of the pipeline in Azure Pipelines

Deploy the development environment

Now that you have created the release definition, you can add the steps to initiate a rolling deployment to the development environment.

  1. In Azure Pipelines, switch to the Tasks tab.
  2. Click Agent job.
  3. Change the Agent pool value to Private > Google Cloud.
  4. Next to Agent job, click the + icon to add a step to the phase.

  5. Select the Command Line task, click Add, and configure the following settings:

    • Task Version: 2.*
    • Display name: 
      Create instance template
    • Script: 
                                      gcloud compute instance-templates create orchard-$(Build.BuildId)-$(Release.ReleaseId) --machine-type n1-standard-2 --image orchard-$(Build.BuildId) --image-project $(Packer.Project) --tags=gclb-backend

    This command creates a new instance template that uses the VM image that you previously built with Packer. The command applies the gclb-backend tag so that the load balancer can reach instances that are created from this template.

  6. Add another Command Line task and configure the following settings:

    • Version: 2.*
    • Display name: 
      Associate instance template
    • Script: 
                                      gcloud compute instance-groups managed set-instance-template orchard-dev --template=orchard-$(Build.BuildId)-$(Release.ReleaseId) --zone $(Deployment.Dev.Zone)

    This command updates the existing instance group to use the new instance template. Note that this command does not yet cause any of the existing VMs to be replaced or updated. Instead, it ensures that any future VMs in this instance group are created from the new template.

  7. Add another Command Line task and configure the following settings:

    • Version: 2.*
    • Display name: 
      Start rolling update
    • Script: 
                                      gcloud compute instance-groups managed rolling-action start-update orchard-dev --version template=orchard-$(Build.BuildId)-$(Release.ReleaseId) --type proactive --max-unavailable 0 --zone $(Deployment.Dev.Zone)

    This command causes the existing instance group to replace existing VMs with new VMs in a rolling fashion.

  8. Click the Variables tab, and add the following variables:

    Name Value
    Packer.Project The name of your Google Cloud project.
    Deployment.Dev.Zone The zone that you specified earlier when running gcloud config set compute/zone (for example, us-central1-a)

  9. Click Save.

  10. Enter a comment if you want, and confirm by clicking OK.

Deploy the production environment

Finally, you need to configure the deployment to the production environment.

  1. In Azure Pipelines, switch to the Pipeline tab.
  2. In the Stages box, select Add > New stage.
  3. From the list of templates, select Empty job.
  4. When you're prompted for a name for the stage, enter Prod.
  5. Click the lightning bolt icon of the newly created stage.

  6. Configure the following settings:

    • Select trigger: After stage
    • Stages: Dev
    • Pre-deployment approvals: (enabled)
    • Approvers: Select your own user name.

  7. Hold the mouse over the Tasks tab and click Tasks > Prod.

  8. Click Agent job.

  9. Change the Agent pool value to Private > Google Cloud.

  10. Next to Agent phase, click the + icon to add a step to the phase.

  11. Add a Command Line task and configure the following settings:

    • Version: 2.*
    • Display name: 
      Associate instance template
    • Script: 
                                      gcloud compute instance-groups managed set-instance-template orchard-prod --template=orchard-$(Build.BuildId)-$(Release.ReleaseId) --zone $(Deployment.Prod.Zone)

    This command updates the existing instance group to use the instance template that you created during the deployment to the Dev environment. Reusing the same instance template ensures that you are deploying the exact same image.

  12. Add another Command Line task, and configure the following settings:

    • Version: 2.*
    • Display name: 
      Start rolling update
    • Script: 
                                      gcloud compute instance-groups managed rolling-action start-update orchard-prod --version template=orchard-$(Build.BuildId)-$(Release.ReleaseId) --type proactive --max-unavailable 0 --zone $(Deployment.Prod.Zone)

    This command causes the existing instance group to replace existing VMs with new VMs in a rolling fashion.

  13. Switch to the Variables tab.

  14. Add a variable:

    • Name: Deployment.Prod.Zone
    • Value: Zone that you specified earlier when running gcloud config set compute/zone (for example: us-central1-a)

  15. Click Save.

  16. Enter a comment if you want, and confirm by clicking OK.

Run the pipeline

Now that you've configured the entire pipeline, it's time to test it. In the Packer template that you created previously, you used windows-2016 as the source image family. When you want to run a web server, running a full-featured Windows Server distribution might increase resource consumption with no benefit. Instead you will change the template to use Windows Server 2016 Core, and use this change to exercise the entire CI/CD pipeline.

  1. Open the file packer.json.

  2. Change the source image family:

    "source_image_family": "windows-2016-core",

  3. Commit your changes and push them to Azure Pipelines.

    Visual Studio

    1. Open Team Explorer and click the Home icon.
    2. Click Changes.
    3. Under Changes, right-click packer.json and then click Stage.
    4. Enter a commit message like Use Windows Server Core.
    5. Click Commit Staged and Push.

    Command line

    1. Stage all modified files:

                                          git add packer.json

    2. Commit the changes to the local repository:

                                          git commit -m "Use Windows Server Core"

    3. Push the changes to Azure DevOps:

                                          git push

  4. In the Azure DevOps menu, select Pipelines. A build is triggered.

  5. After the build is finished, select Pipelines > Releases. A release process is initiated.

  6. Click Release-1 to open the details page, and wait for the status of the Development stage to switch to Succeeded.

  7. In Cloud Shell, run the following command to obtain the IP address of the load balancer for the development environment:

    gcloud compute forwarding-rules list | grep orchard-dev | awk '{print $2}'

  8. In the browser, go to the Orchard installation using the URL that you got in the previous step:

    http://IP-ADDRESS/orchard/

    You might see an error at first because the load balancer takes a few minutes to become available. When it's ready, observe that Orchard CMS has been deployed successfully:

    Screenshot showing the Orchard CMS app running in a browser page

  9. In Azure Pipelines, below the Prod stage, click Approve to trigger the deployment to the production environment:

    Screenshot showing the release page and a message 'A pre-deployment approval is pending ... Approve or Reject'

  10. Wait for the status of the Prod stage to switch to Succeeded. You might need to manually refresh the page in your browser.

  11. In Cloud Shell, run the following command to obtain the IP address of the load balancer for the production environment:

    gcloud compute forwarding-rules list | grep orchard-prod | awk '{print $2}'

  12. In the browser, go to the orchard installation using the URL that you got in the previous step:

    http://IP-ADDRESS/orchard/

    Again, you might see an error at first because the load balancer takes a few minutes to become available. When it's ready, you see the Orchard CMS page again, this time running in the production cluster.

Clean up

To avoid incurring further costs after you complete this tutorial, delete the entities that you created.

Delete the Azure Pipelines project

To delete the Azure Pipelines project, see the Azure DevOps Services documentation. Deleting the Azure Pipelines project causes all source code changes to be lost.

Delete the Google Cloud development and production projects

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

  • Read about best practices for managing images.
  • Learn how to deploy a highly available SQL Server group on Compute Engine.
  • Read about .NET on Google Cloud Platform.
  • Install Cloud Tools for Visual Studio.
  • Explore reference architectures, diagrams, tutorials, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2018-08-06 UTC.