BLOG

Using a single environment variables file for both Google cloud function and local dev

July 11, 2020

Whenever I need to write a cloud function for a Google Cloud Platform (GCP) project, I prefer to develop and test my code locally and deploy it as a cloud function only when it's done. It's often the case that I need to use environment variables to store important and sensitive information. For instance, if my code needs to access a Google Sheet via the Google Sheet API, then I prefer to store the sheet ID in an environment variable, instead of hard-coding it in the code.

There are many ways to set environment variables both locally and for a cloud function, but since I prefer to store a variable only once, I am inclined to use a YAML file that I reference when I deploy the cloud function.

The YAML file simply contains key/value pairs like this:

SPREADSHEETID: xxxxxxxxxxxxxxxxxxx
SHEETNAME: sheet1

I do not like to store the YAML file in my version control system, so I place it in a "config" folder and I call out the folder in my .gitignore file. (I also store GCP service account key files in the folder.) Since, by default, GCP doesn't deploy folders and files that are listed in .gitignore, I have to explicitly call out the folder in the .gcloudignore file to get the folder deployed:

!config

Eventually, when I'm ready to deploy the code as a cloud function, I issue the following command, referencing the YAML file:

gcloud functions deploy myFunction --runtime nodejs8 \
--trigger-topic some_topic_name \
--env-vars-file config/.env.yaml

This is great for the cloud function, but how can I use that file for local development? I did find an NPM package that parses YAML files to set environment variables, but I always prefer to set the variables outside of my code (i.e, in package.json).

To achieve that, I wrote a small utility that reads the YAML file, sets the variables, and then spawns a child process to run my main script. Here's the code:

// set-env-var.js:
const fs = require('fs');
const spawn = require('cross-spawn');
const [_, __, envFileName, command, scriptName] = process.argv;
fs.readFile(envFileName, 'utf-8', (err, vars) => {
  if (err) throw err;
  vars.split('\n').forEach((va) => {
    const [k, v] = va.split(/:(.+)/);
    if (k) {
      process.env[k.trim()] = v.trim();
    }
  });
  spawn(command, [scriptName], {
    stdio: 'inherit',
  });
});

The utility takes advantage of the cross-spawn package, which you'll need to install:

npm i -D cross-spawn

I run the utility via the "scripts" attribute in package.json:

  "scripts": {
    "dev": "node set-env-vars.js config/.env.yaml nodemon index.js"
  }

The dev script runs the utility, which inspects the command line arguments. The first argument is the name of the YAML file that stores the environment variables.

The utility reads the file and splits it into separate lines. Each line is then set as an environment variable. Once that's done, the utility launches the main script (in this case using nodemon).

The child process has access to the environment variables:

// index.js:
console.log(`Spreadsheet ID: ${process.env.SPREADSHEETID}`);
// xxxxxxxxxxxxxxxxxxx

This utility is meant only to make local development simpler. Please do NOT deploy this code in a production environment. You should also include the utility in your .gcloudignore file.

With this approach, I can use a single file to set up variables for both my local dev and the cloud function. I don't have to implement special code to differentiate between Dev and Prod: the variables are set independently, and the code simply uses them as-is.

I'll be interested to learn about other possible strategies to handle such cases. Please leave a comment about how you handle environment variables for both local and cloud.

– Ben