itemis Blog

Secure your Travis CI releases – part 1: checksum with SHA

Written by Jan Mosig | Sep 10, 2019

When fiddling around with Travis CI a very popular CI tool for GitHub projects, I was wondering: How to secure my release artifacts against forgery, e.g., malware injection? I already knew the solution: SHA and OpenPGP. But how to include those into my Travis CI build? Signing requires the private key to exist in the repository. How to keep it secret? I found a solution, which I gladly share in this series of articles. Part 1 is about assuring release artifact integrity with SHA-512.

Creating a release with Travis GitHub release provider

In mid 2019 I released a bunch of shell scripts that ought to make OpenPGP key generation and Yubikey manipulation easier: Yubiset. In the Yubiset world, “release” means: Zip all scripts and put the created ZIP file on GitHub. This is rather easy with Travis CI. Using the GitHub releases provider, just put this file in the root of your GitHub repository:

.travis.yml

# Make sure build does only run if a tag is pushed, not if a branch is pushed.
if: NOT tag IS blank
 
# A minimal Linux environment with general tools, not tailored to one specific language.
language: minimal
 
notifications:
  email:
    on_success: never
    on_failure: always
 
# Use Trusty Linux distribution
matrix:
  include:
  - os: linux
    dist: trusty
 
# Skip installation phase. We do not need any dependencies to be installed.
install: true
 
# Zip stuff and provide ZIP file's name in $release_zip.
# This is also where a possible software build, e. g., with mvn, could go into.
script:
    - . ./.ci/script.sh
 
# skip-cleanup: true -> Prevent deletion of created artifacts after install and before deployment
# overwrite: true -> Allow overwriting "old" tags (a.k.a. releases) with the same name.
# prerelease: true -> Mark release as prerelease. Adjust to your needs.
# repo: GitHubAccountName/RepoName
# tags: true -> deploy only on tagged builds
deploy:
  - provider: releases
    api_key:
      secure: $GITHUB_API_KEY
    file:
    - $release_zip
    skip-cleanup: true
    overwrite: true
    draft: false
    prerelease: true
    on:
     repo: JanMosigItemis/yubiset
     tags: true

.ci/script.sh

#!/bin/bash
 
# Prints error message and exit with error code 1.
# Reroutes stderr to stdout.
# Arg 1: Error message - If empty, default message will be used.
end_with_error()
{
    echo "ERROR: ${1:-"Unknown Error"} Exiting." 1>&2
    exit 1
}
 
export TZ=Europe/Berlin
export TIMESTAMP=`(date +'%Y%m%d%H%M')`

# see https://graysonkoonce.com/getting-the-current-branch-name-during-a-pull-request-in-travis-ci/
export BRANCH=$(if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then echo $TRAVIS_BRANCH; else echo $TRAVIS_PULL_REQUEST_BRANCH; fi)
 
# Guard against usage scenarios where no tag is available, e. g. build of a pushed branch.
if [[ -z "${TRAVIS_TAG}" ]] ; then
    TRAVIS_TAG="no_tag"
fi
 
export release="release_${TRAVIS_TAG}.${TIMESTAMP}"
export release_zip="${release}.zip"
 
echo
echo "Creating ${release_zip}.."
# -r.. Walk through directories (recursive).
# -T.. Test ZIP file integrity at end of process.
# -9.. Use best compression (smallest file size).
# -x.. Ignore
{ zip -rT9 -x".git/*" -x"*.gitignore" -x"*.travis.yml" -x".ci/*" "${release_zip}" * ; } || { end_with_error "Creating ZIP file failed." ; }
echo "Success!"

How to obtain GITHUB_API_KEY?

$GITHUB_API_KEY is a Travis CI environment variable. It contains an encoded OAuth token Travis CI may use to access your GitHub repository. The token may be generated like this:

          • Get and install a copy of Travis CI CLI.
          • cd into your repository clone.
          • Run: travis setup releases
          • This will probably alter your .travis.yml like this:

            .travis.yml

            deploy:
  provider: releases
  api_key:
    secure:
# copy this value
P9Nvrvch87YjHFOVuKruK9x3oCeGPs7BmLAJllZfw9yMNpoXo3lAQ+EfhrXa4b9v20Ju9516nE7YnloW8kGM77+AVZJ6Xolyo3qwrY1zq7HTC2yN6Hju37PTzPAg0gfsPvAyFSgADMfIR6J9+ocBZP6MTWkwqsWXIPEUNCDKdJKSVeBzM0vrAVV4afNxxejo+u7/DWB3IXYtIKXARDG8lHkxPY+IhL/bsNmurrEziXa/h14eJtG99Yb5IoKlUniH2YlkXs0xgNPLMj0Ilitox/kipLFQhY2oWRzeHmzy3Rf6EomulUc+NOzP6jrMjo+GjQNHOJN14EeCUaTvB4rct0sCIwvbM1ayG/hAuOXFeW+wvZBClt3ri6nDnNCZJbqTuPbgRXDUa6YO37X7Z8XQCewwn+vrHmM5PM6DCHLk7zczEeJPux1PAokYvIuu1C6+0Ao8jiNsQkRjAe8RXzXRJYzim9axlJ5J3x0BJWkwh4x+0ROAgmnPuYritG3ncfRIJL7AkCwqjrU5oz9Maw2gc19V6HhGY8kvYhMafukYbDJf2pA5N7kFopLRCzqBUlwJ6mmx0YO6nPgt6vXt4qzAGn+dFkGDZkBnAoEvu7LYKGkI3IKoWE8QqlIb6hsuYe/gzhT7YqUDAZ+F0YyWkNklWs4rsgDRxzyizhbhUzKPlUw=
  file: $release_zip
  on:
    repo: JanMosigItemis/yubiset
          • Copy the secure value and create a new environment variable named $GITHUB_API_KEY in your Travis build’s settings, accessible under https://travis-ci.org/GitHubName/RepoName/settings
          • Revert your .travis.yml
          • This will create an access token in your GitHub account (see https://github.com/settings/tokens), and put it into the Travis environment variable $GITHUB_API_KEY, which will be accessible from .travis.yml at build time only.

Creating a first test release

When ready, add and commit everything, but do not push yet. In order for Travis to actually run the script, we need to create and push a Git tag like this:

        • git tag -a -m"Tag message"
        • git push origin --follow-tags

That’s all you need to trigger build and release. Using annotated tags (git tag -a) makes all the valuable metadata, like commit message, committer, etc. available along with the tag. The result will be a new release in your GitHub repository’s release section:

Assuring release integrity with SHA-512

We’ve got a release. Great! Now let’s make sure downloaders have a way of telling whether or not their download is legit, i.e., the officially released ZIP file from our build, not one that has been tampered with or is broken. This so-called file integrity may be checked by calculating a hash code a.k.a. checksum. Because a hash code based on a secure hash function cannot be easily reproduced with a manipulated ZIP file as input, it can be used to check file integrity like this:

  • File provider (Travis CI release build in our example) calculates a hash code for the ZIP file.
  • Downloader recalculates the hash code on his local machine.
  • Original and local code are compared. If equal → file integrity is okay.

Since our Travis build is running on Linux, we can use the sha512sum tool, which comes with many distributions as part of the GNU core utilities. This tool is also available on Windows via git-bash. It calculates a SHA-512-based hash code.

Extend the file .ci/script.sh like this.

.ci/script.sh extended by hash calculation

export release_hash="${release}.sha512"
 
echo
echo "Creating ${release_hash}.."
{ sha512sum "${release_zip}" > "${release_hash}" ; } || { end_with_error "Creating hash file failed." ; }
echo Success!

This will create a file called release_TAG.TIMESTAMP.sha512 with the following contents:

release_TAG.TIMESTAMP.sha512:

E1AAA728E002F462D74DD7E7F2D107AAAF6038C345F5E1F849B6DB8647A15C170F7CC2D65F45E602570A1752C4B0E0D4C9549639DDA8607960A40063AAB84748 *release_TAG.TIMESTAMP.zip

It contains the hash code in hexadecimal form and the filename it belongs to, which is the ZIP file in our case. The filename will be the value of the new environment variable $release_hash, which must be included into the “deploy” section of .travis.yml, like this:

“deploy” section of .travis.yml with hash code file

deploy:
  - provider: releases
    api_key:
      secure: $GITHUB_API_KEY
    file:
    - $release_zip
    - $release_hash
    skip-cleanup: true
    overwrite: true
    draft: false
    prerelease: true
    on:
     repo: GitHubName/RepoName
     tags: true

Checking download integrity

Imagine you have downloaded the ZIP file and the sha512 file into the same directory. The integrity of the ZIP file may then be checked like this:

Checking ZIP file integrity

# The file mentioned in the sha512 file (this is release_TAG.TIMESTAMP.zip in this case) must be accessible from here.
# -c Check integrity
sha512sum -c release_TAG.TIMESTAMP.sha512
release_TAG.TIMESTAMP.zip: OK # Expected output.

Of course it is always possible to just recalculate the hash and compare both hashes manually.

On Windows without git-bash you must either use a third-party tool with appropriate functionality or you may use the following batch script, which makes use of the PowerShell:

check_hash.bat

@ECHO OFF
SETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
 
if [%1] == [] (
    echo ERROR: Please specify SHA512 file to use.
    exit /b 1
)
 
for /F "delims=* tokens=1" %%i in ('type %~1') do set remote_hash=%%i
for /F "delims=* tokens=2" %%i in ('type %~1') do set remote_zip=%%i
for /F %%i in ('powershell -command "(Get-FileHash -Path %remote_zip% -Algorithm SHA512).hash.ToLower()"') do set local_hash=%%i
 
if %local_hash% == %remote_hash% (
    echo %remote_zip%: OK
) else (
    echo %remote_zip%: FAILED
)
 
endlocal

Usage is like this cmd.exe:

Checking file integrity on Windows

check_hash release_TAG.TIMESTAMP.sha512
release_TAG.TIMESTAMP.zip: OK # Expected output.

Let’s summarize

Alright, that’s it for the first part. Thank you very much for spending your time here. We do now have an automated Travis CI build that provides a ZIP file and an accompanying hash code file that can be used to check ZIP file integrity.

In the next part of our series, we are going to secure our release with signatures from OpenPGP against hash code forgery.
View full post