Logging to Seq from shell scripts

A lot of scripting can be involved in running an IT operation. While your business applications may be logging to Seq, is it any less important that you have visibility of key scripts? What happens when they fail and endanger your critical SLAs?

At a fundamental level, logging is really important. Often, shell scripts already have logging - but this is to a text log file trapped somewhere on disk. You might send it to a SIEM or other logging server for ingestion, but what if you could treat your shell script like any other application?

Enter seqcli, a multi-function and multi-faceted command line tool. I touched before on its functionality that I initially modelled against for Seq Reporter, but one of its most basic functions is logging to Seq.

As a .NET Core app, you can run seqcli on Windows, Linux, and OS X .. and it works great! I've integrated a number of Linux shell scripts to Seq using it. The key thing to remember is that Seq is at its best with well structured logs; exposing variables and properties to Seq for indexing and querying is always ideal. You can do that readily with seqcli ... and suddenly, getting an alert to something such as OpsGenie becomes simple!

In the sample script below, you'll note that I tend to try to pull in relevant environment variables from the Linux scripts for sending to Seq- this in turn allows script troubleshooting and debugging without having to directly log in. 

Logging is only as good as you make it, of course - if you aren't logging each stage of a script (start, processing, end), for example, you won't necessarily find it easy to diagnose where something failed. And more information is better - okay, a file transfer failed, but what file? Where from? Where was it being sent to?

As a principle, try to standardise logging as far as possible. You'll see a familiar AppName property in the below script, which I use in Lurgle.Logging, the Log4j appender, Seq Reporter, Seq Client for Windows Logins, and Seq apps like Event Timeout, Event Threshold, and OpsGenie Heartbeat. This is valuable data that helps build signals, dashboards, and alerts - being able to differentiate between applications that log to Seq is important. Equally, I often include an Environment or MachineName property that allows differentiation between environments for similar reasons.

The logEntry function below is structured to default to Information events, with the ability to pass other log levels such as Warning or Error. It can be pasted into a shell script (eg. Bash, Ksh, etc), but it's always worth taking a look at what environment variables are being set. Including those as Seq properties is simple - just add another -p PropertyName="$VariableName" - and they're invaluable for debugging. 

The command line can wind up quite long with this approach, but of course you can split it into multiple lines (like below) and I can't recommend doing this enough.

# function <logEntry> :: Log to Seq                          #
function logEntry {
if [ -z "$3" ]

if [ -z "$2" ]
messageTemplate="{Summary} - {Description}"

/etc/seq/seqcli log -l "$errorlevel" -m "{Summary} - {Description}" \
      -p Summary="$1" -p Description="$2" -p AppName="Linux Script Name" \
-p Property="$Property" -p Property2="$Property2" -p Property3="$Property3" \
      -s https://seq.domain.com -a "<apikey>"

Property="Important Property"
Property2="Stuff used in script"
Property3="Relevant info"

logEntry "An error occurred" "Stuff happened!" "Error"
And it's as simple as that. A function added to your script that calls the seqcli app on your machine. Obviously you need to download seqcli and place it in a location where the script has execute permissions - but it's reasonably easy to have a common and standard approach to your shell scripts that can help transform your operations with monitoring and alerting that really works.


You may also like:

Lurgle.Logging v1.1.14 and Lurgle.Alerting v1.1.9 Released

I've pushed out updates to Lurgle.Logging and Lurgle.Alerting today. The Lurgle.Logging update is minor - I noticed that Log.Add wasn't correctly passing the calling method, source file, and line number. Lurgle.Alerting has received a more substantial update: This helps to make Lurgle.Alerting even more useful and reliable! You can get...

Lurgle.Logging v1.2.0 - Multi-threaded correlation ids are now a thing

Multi-threaded correlation ids were not a thing Following on from my work on Seq.Client.WindowsLogins and the subsequent realisation that EventLog's EntryWritten event handler is bad and should feel bad, I contemplated whether I could apply some of my efforts to solve another issue that had been bugging me. Lurgle.Logging was...

Passing Priority, Responder, and Tags from Seq to OpsGenie!

Building up the Seq app for OpsGenie Over the past few weeks, I've worked with Nicholas Blumhardt to enhance the Seq.App.OpsGenie application for Seq. Nicholas is the founder and CEO of Datalust, the company behind Seq, and is very active in the community - which is awesome, and has meant that...