Git pull automation: how to deal with modified files?

The idea of not forcing updates would be to keep a specific version, as sometimes daily Chromium binaries are unstable - thus latest ≠ not always greatest. For instance, I read in the Snappdf github issues tab that a recent version would not interpret CSS and JS, so one might want to wait before updating Chrome blindlessly. I guess that’s why Snappdf keeps several versions by default, too: if after a manual upgrade a version is unstable, it’s possible to revert to a previous one by editing revision.txt, but that would be trickier if the script erased previous versions.

So, besides having chrome-stable installed at the system level and Snappdf pointing to it (which is not the default configurationi), keeping the installed Chrome binaries would be the safest best. That being said, I guess major bugs are not that common, especially for “basic stuff” like printing PDFs.

It’s just a custom name I chose for the logs created by my Supervisor service (which is indeed installed on the server rather than running within the app itself). I also decided to put it in the Invoice Ninja root, but I could have put it anywhere; I just thought it would make sense to not bury it too far. Supervisor workers can stay “on” all the time, compared to running a cron job at specific intervals. It’s the recommended way of keeping php artisan queue:work alive on systems with root access, as per the self-host install docs. More info here:

Edit:
In your script I also saw that you added this:

# Uncomment the line below if you use snappdf
# vendor/bin/snappdf download

I think the whole command line should rather be, in your structure:
./public_html/vendor/bin/snappdf download with ./ (to tell the shell snappdf is executable) and the full path.

Thanks for all the info, that’s good to know. It sounds like it the safest thing to do would just be to preserve the contents of /snappdf/versions/ folder so I’ve added a line for that.

Good catch with the snappdf directory structure, I’d goosed that up by not including public_html. It should only need the ./ when executing from the same directory as the file but I’ve removed that now that it’s set to preserve the snappdf releases.

I’ve also updated it to set the installation directory, temp update directory, and php-cli command as variables at the start so that it’s much easier to configure. It’s also set to preserve the /storage/log/ files by default now.

1 Like

Hi Josh,

I worked a bit further on the Chrome/Snappdf thing, and I made a bit of code to detect if Chrome is installed and if so, back it up with the rest, and if not, install it before backing it up. I created a variable to bypass the process, and that variable will also be dynamically updated to reflect variables in .env that might instruct to either bypass the Chrome download or use a locally installed version on the system.

I also reworded a bunch of stuff, added some more error handling, added more comments, and optimized the code here and there. Feel free to look at the result here:

#!/bin/bash

### Update Invoice Ninja ###

# Set the -euo pipefail option to exit immediately if any command fails or any undefined variable is used
set -euo pipefail

# Define the path to the Invoice Ninja installation directory
ninja_dir="public_html/admin"

# Define other variables
update_dir="invoiceninja_temp_update"
chrome_installed_in_invoice_ninja=false
chrome_download_bypassed=false
chrome_downloaded=false

# Define a function to handle errors and print error messages
function handle_error {
    # Get the error message
    local error_message="${1}"
    # Get the line number of the error
    local line_number="${2}"
    # Print the error message and line number
    echo "Error on line ${line_number}: ${error_message}"
    # Exit with a non-zero status code
    exit 1
}

# Check if curl is installed on the system. Curl is a command line tool for transferring data with URL syntax.
if ! command -v curl >/dev/null 2>&1; then
    # If curl is not installed, exit with an error message using handle_error function defined above.
    handle_error "Curl is required but not installed. The script will end." $LINENO
fi

# Query GitHub for the latest release URL of Invoice Ninja and extract the tag name using sed. This will be used to determine which version of Invoice Ninja should be downloaded and installed.
latest_url="$(curl -f -sL -o /dev/null -w '%{url_effective}' https://github.com/invoiceninja/invoiceninja/releases/latest)"
version="$(printf "${latest_url}" | sed 's#.*/tag/\(.*\)$#\1#')"

# Check if the version number was successfully obtained from GitHub. If not, exit with an error message using handle_error function defined above.
if [ -z "${version}" ]; then
    handle_error "Failed to obtain the latest version number from GitHub. The script will end." $LINENO
fi

# Print out the latest version of Invoice Ninja obtained from GitHub.
printf "The latest version of Invoice Ninja is: %s\n" "${version}"

# Read contents of VERSION.txt file in ninja_dir and add 'v' at beginning of line. This file contains information about currently installed version of Invoice Ninja.
version_from_file="$(sed 's/^/v/' "${ninja_dir}/VERSION.txt")"

# Check if current installed version matches with latest available version obtained from  GitHub API.
if [ "${version_from_file}" = "${version}" ]; then
    # If versions match, print message indicating that latest version is already installed
    printf "Latest version already installed! \nInstalled version: %s \nLatest version: %s\n" "${version_from_file}" "${version}"
    exit 1
else
    # If versions do not match, print message indicating that download of latest release will start
    printf "Downloading latest release %s\n" "${version}"

    # Construct the download URL for the zip file using obtained value of $version variable
    zip_url="https://github.com/invoiceninja/invoiceninja/releases/download/${version}/invoiceninja.zip"

    # Download zip file from constructed URL using curl command and check if the download was successful. The -f flag will cause curl to fail silently if the HTTP code returned is not 200 OK. The -L flag will follow any redirects.
    if curl -s -f -L --location-trusted -O "${zip_url}"; then
        # Create a temporary directory using mkdir command to store downloaded files
        mkdir -p "${update_dir}"

        # Unzip downloaded invoiceninja.zip file into the update directory
        printf "Extracting zip file, this can take while... \n"
        unzip -qq -o invoiceninja.zip -d "${update_dir}"
        rm invoiceninja.zip
    else
        # If the download was unsuccessful, exit the script with an error message using handle_error function defined above.
        handle_error "Failed to download the latest release. The script will end." $LINENO
    fi
fi

# Backup configuration files with all file attributes preserved
# The '-a' option is equivalent to '-dR --preserve=all', which copies files and directories recursively while preserving symbolic links and special files
# This ensures that all file attributes, such as permissions and timestamps, are preserved when copying the files
cp -a "${ninja_dir}/.env" "${update_dir}"
cp -a "${ninja_dir}/laravel_worker.log" "${update_dir}"
cp -a "${ninja_dir}/public/storage" "${update_dir}/public/"

#BEGIN SNAPPDF/CHROME

# Check if the Chrome version revision text file (revision.txt) exists in the Invoice Ninja installation directory
if [[ -f ${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt ]]; then
  # Read the content of revision.txt and store it in a variable
  existing_chromerevision="$(cat "${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt")"

  # Check if a folder named $existing_chromerevision exists at the same level as revision.txt,
  # if there is a chrome executable inside the chrome-linux subfolder, and if the content of revision.txt contains numbers and ends with -Linux_x64
  if [[ -d "${ninja_dir}/vendor/beganovich/snappdf/versions/${existing_chromerevision}" ]] && [[ -x "${ninja_dir}/vendor/beganovich/snappdf/versions/${existing_chromerevision}/chrome-linux/chrome" ]] && [[ "{$existing_chromerevision}" =~ [0-9]+-Linux_x64$ ]]; then
    printf "Chrome installation detected in Invoice Ninja, version ${existing_chromerevision}. \n"
    chrome_installed_in_invoice_ninja=true
  fi  
fi

# Check if Chrome installation was not detected in Invoice Ninja. If not, print message indicating that Chrome was not found.
if [[ ! "$chrome_installed_in_invoice_ninja" ]]; then
  printf "Chrome installation not detected in Invoice Ninja. \n"
fi

# If Chrome was not found and download has not been bypassed, download Chrome using snappdf command line tool. The --force argument is to make sure to overwrite any possible previous installation of the same version that would not have been picked up by the script.
if [[ "${chrome_installed_in_invoice_ninja}" == false && "${chrome_download_bypassed}" == false ]]; then
  printf "Downloading Chrome... \n"
  ./${ninja_dir}/vendor/bin/snappdf download --force
  chrome_downloaded=true

  # Check if Chrome has been downloaded successfully by checking for the presence of an executable file named 'chrome'
  new_chromerevision="$(cat "${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt")"
  if [[ -x "${ninja_dir}/vendor/beganovich/snappdf/versions/${new_chromerevision}/chrome-linux/chrome" ]]; then 
    printf "Chrome has been successfully installed. \n"
    chrome_installed_in_invoice_ninja=true 
  else 
    printf "Error: Chrome could not be installed. \n" 
  fi 

  if [[ "${chrome_installed_in_invoice_ninja}" == true ]]; then
    # If Chrome was installed successfully, backup the revision.txt file and the Chrome installation directory to the update directory
    printf "Backing up Chrome... \n"
    cp -a ${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt ${ninja_dir}/vendor/beganovich/snappdf/versions/${existing_chromerevision}/ ${update_dir}/vendor/beganovich/snappdf/versions/
  elif [[ "${chrome_downloaded}" == true ]]; then
    # If Chrome was not found and download was attempted but failed, print message indicating that manual intervention is required.
    printf "No chrome installation found. Please manually check and install Chrome. \n"
  fi
fi

#END SNAPPDF/CHROME

# Replace the existing installation directory with the updated version
printf "Replacing previous installation with updated version... \n"
# Create a variable to hold the name of the old installation directory, by appending "_old" to the original directory name
old_ninja_dir="${ninja_dir}_old"
mv "${ninja_dir}" "${old_ninja_dir}"
mv "${update_dir}" "${ninja_dir}"

# Update configuration and clear caches
# Display message to user
printf "Updating config and clearing caches...\n"

# Define an array of commands to be executed
commands=("clear-compiled" "route:clear" "view:clear" "migrate --force" "optimize")

# Iterate over each command in the array
for cmd in "${commands[@]}"; do
    # Execute the command using the system function
    php "${ninja_dir}/artisan" $cmd
done

# Remove the temporary directory
printf "Cleaning up temporary directory...\n"
rm -rf "${old_ninja_dir}"

# Check if the contents of VERSION.txt match the latest version number. If so, print success message. Otherwise, print failure message.
check_version_from_file=$(<"${ninja_dir}/VERSION.txt")
check_version_from_file="v${check_version_from_file}"

if [ "$check_version_from_file" = "$version" ]; then
  printf "Invoice Ninja successfully updated! \nInstalled version: ${check_version_from_file} \n"
else
  printf "Update failed! \nInstalled version: ${check_version_from_file} \nLatest version: ${version} \n"
fi

Edit: spelling

1 Like

Nice work, looks good!

Have you tested that this command works to copy the chrome binary to the update directory rather than copying revisions.txt to both directories?

cp -a ${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt ${ninja_dir}/vendor/beganovich/snappdf/versions/${existing_chromerevision}/ ${update_dir}/vendor/beganovich/snappdf/versions/

Tanks! Most of the heavy lifting was done by Chat-GPT and Bing Chat. I was merely the “director” and tester. Those are wonderful toos, but sometimes they make mistakes and contradict themselves, so I had to stay focused :slight_smile:

That’s a good question, I had to double check. The script seemed to work fine for that part, but I investigated further by chatting with Bing and I think you’re right. Thanks for picking that up! the slash after ${existing_chromerevision}/ would make it a destination directory instead a source directory, so I should remove the slash so that it behaves as a source as intended. Here is the fix:

cp -a ${ninja_dir}/vendor/beganovich/snappdf/versions/revision.txt ${ninja_dir}/vendor/beganovich/snappdf/versions/${existing_chromerevision} ${update_dir}/vendor/beganovich/snappdf/versions/

Better yet, and more readable, would be

cp -a ${ninja_dir}/vendor/beganovich/snappdf/versions/{revision.txt,${existing_chromerevision}} ${update_dir}/vendor/beganovich/snappdf/versions/

I updated my code with the second option :slight_smile:

I also optimized the first (general) backup commands to make it an editable list, so that stuff can be easily be added or removed:

for item in ".env" "laravel_worker.log" "public/storage"; do
    cp -a "${ninja_dir}/${item}" "${update_dir}/${item}"
done

Ill post a new version here with the .env checks when I have the time.

Thanks!