Deploying a .Net AppEngine using AppVeyor

So, I've written a dotnet core application in C# and deployed it to AppEngine to run it as a web service. Problem is, it doesn't feel like I'm really in control of it until I can put it through a Continuous Integration pipeline where I can run my tests and push it to the Google Cloud Platform without any manual intervention.

Here's my first stab, using AppVeyor. AppVeyor is a Windows CI pipeline that allows free use for open source projects. I don't need Windows for this project, as AppEngine requires dotnet core deployed on Linux, but I only want to learn one CI system, and the dual use of .Net in core and traditional is ideal for me.

My test app is a little implementation of an algorithm, outlined by Gillespie1, that simulates chemical reactions. It is open source, written in C# for deployment to AppEngine, and can be found here.

Now, as good as AppVeyor is, it is not integrated with the Google Cloud Platform so we have to script the deployment instructions ourselves. These instructions live in a configuration file called appveyor.yml. This is where it got a little tricky. It's easy to script a deployment interactively, as the gcloud command integrates so easily with the GCP using a mobile phone to authenticate. We need a couple more things to achieve this in script.

Encrypted service account credentials

In order to deploy to the GCP project we need to authenticate. To do this we first need a service account. We can create this in the GCP IAM console, like this:

creating an admin service account

Note that although there is a role "App Engine Deployer" it doesn't have enough permissions to deploy to AppEngine. Or at least not enough to deploy to mine. I created a service account with the role "App Engine Admin."

You can then download the json file. Keep it safe. You'll need to encrypt it, add the encrypted version to your git repo and save the encryption secret as an AppVeyor environment variable, as outlined here.

Installing the Google Cloud SDK on the CI server

For AppVeyor to deploy it needs to run the deployment script non-interactively. First we install theSDK using the Windows package manager Chocolately:

  - choco install gcloudsdk

Then a couple of undocumented gotchas:

  - refreshenv
  - gcloud.cmd components copy-bundled-python>>python_path.txt && SET /p CLOUDSDK_PYTHON=<python_path.txt && DEL python_path.txt

These restart the Powershell session and set the Python path.

Finally, we install the SDK:

  - gcloud.cmd components update --quiet
  - gcloud.cmd components install beta --quiet

Note that the --quiet flag is important. It is still verbose in the sense that logging will still appear on the command line, but the session will not pause for a Y/N response. Without this the build will pause for an hour and then time out.

Here is the full working config:

version: 1.0.{build}  
image: Visual Studio 2015  
environment:  
  gcp_app_engine_service_account:
    secure: ybv1IQS2tSs+dxUqDCGqIcQ2SIZcup4fj5JYZiAdGf7a3Ol1b+k/FdaZyGfRO8OyWxgM7/F6iaBUivHWtYrpt1kZcavundHf38GbjRC79T4=
build_script:  
- cmd: >-
    dotnet restore

    dotnet publish -c Release
test_script:  
- cmd: dotnet test test/kinetics.library.tests/kinetics.library.tests.csproj
install:  
  - nuget install secure-file -ExcludeVersion
  - choco install gcloudsdk
  - refreshenv
  - gcloud.cmd components copy-bundled-python>>python_path.txt && SET /p CLOUDSDK_PYTHON=<python_path.txt && DEL python_path.txt
  - gcloud.cmd components update --quiet
  - gcloud.cmd components install beta --quiet
  - secure-file\tools\secure-file -decrypt "My First Project-4c1a58f53f53.json.enc" -secret %gcp_app_engine_service_account%
deploy_script:  
- cmd: >-

    dotnet publish -c Release

    gcloud config set project crypto-monolith-156009

    gcloud auth activate-service-account crypto-monolith-156009@appspot.gserviceaccount.com --key-file="My First Project-4c1a58f53f53.json"

    gcloud beta app deploy src/kinetics.webapi/bin/Release/netcoreapp1.1/publish/app.yaml --quiet

And to show that it works, here is the log output from AppVeyor :

successful deploy log

To test it yourself, feel free to fire a POST request at my app with the json

{"timeLimit":40.0,"steps":50,"reactions":[{"equation":"2A -> 0", "rate":0.001},{"equation":"A + B -> 0","rate": 0.01},{"equation":"0 -> A", "rate":1.2},{"equation":"0 -> B","rate": 1.0}],"initialPopulations":[{"species":"A","count":0},{"species":"B","count":0}]}
Note

This config appears as a StackOverflow answer here

1 Daniel T. Gillespie Exact stochastic simulation of coupled chemical reactions, J. Phys. Chem., 1977, 81 (25), pp 2340–2361