Updates to Event Timeout, Event Threshold, and Event Schedule for Seq ... and introducing Lurgle.Dates!

MattMofDoom August 2nd, 2021 17:11

Grr ... Bugses. We hates them precious.
Lurgle.Dates
Updates to EventX Trilogy
Get them!

Grr ... Bugses. We hates them precious.

There's nothing worse than code that doesn't quite work as intended. Well, that's not quite true. If you have 3 products with common code that doesn't work as intended, that's probably worse.

Anyway, I noticed that Event Schedule fired prematurely on the last day of the month, instead of the first day of the month as scheduled. Which, of course, meant that when I checked Event Timeout, the instances that had settings for the first day of the month (different timeouts) were firing on both the last and first day of the month. How annoying, and wrong.

When I blogged about Event Schedule being released, I foreshadowed that the rapid development of Event Threshold and Event Schedule had led to features being duplicated from Event Timeout, and that I would likely move to a common library.

This bug provided the rationale to do so. Rather than fixing a bug three times in three codebases, it made sense to consolidate to that single library and fix it once.

So Lurgle.Dates was born. I'm going to dive into the functionality available for Lurgle.Dates, but if you're only interested in the EventX updates, skip to Updates to EventX Trilogy.

Lurgle.Dates

The first thing to go into Lurgle.Dates is, of course, the common date logic that is used by the "EventX Trilogy" - Event Timeout, Event Threshold, and Event Schedule. This is logic that, given a string of date expressions, a start time, format of the start time, and the current date/time, will give you a list of dates for those date expressions.

The Dates class

This was where the bug fix lay. The mistake I'd made in Event Timeout, and duplicated into Event Threshold and Event Schedule, was to rely on a list of integers for the day of month. It "works" but it can fail at various date boundaries (because not all months are the same length).

I'd put off the "real" fix when I only had one codebase to work with, and instead worked on tweaking the logic. But in short - the real fix is to always deal in datetime, so that the date and time you're relying on is correct.

So if I pass

dates = Dates.GetDaysOfMonth("first", "9:00", "H:mm", DateTime.Now);

I should be able to expect that the single date returned in the list is the 1st of next month, 9:00am. Moving to a List<DateTime> does just that.

Now, in moving to a common library, it makes sense to cater for both local and UTC dates. So if I pass

dates = Dates.GetUtcDaysOfMonth("first", "9:00", "H:mm", DateTime.Now);

I'll get that first day of the month in UTC+0, rather than local time. That's what we need when working with Seq.

This is the guts of the EventX date inclusion and exclusion capabilities. This has some strong flexibility in what you can provide as a date expression, eg.

first - first day of month
last - last day of month
first weekday
last weekday
first Monday
second Tuesday
third Wednesday
fourth Thursday
fifth Friday
1 - 31 (day of month)

The Dates class also includes a method to return a list of days of week, given a string of day names, start time, and format of the start time. We provide methods for local and UTC calculations here as well;

daysOfWeek = Dates.GetDaysOfWeek("Monday,Tuesday,Wednesday,Thursday,Friday", "9:00", "H:mm");
daysOfWeek = daysOfWeek = Dates.GetUtcDaysOfWeek("Monday,Tuesday,Wednesday,Thursday,Friday", "9:00", "H:mm");

The GetUtcDaysOfWeek call is particularly useful for Seq purposes, because it will shift the list to reflect the correct day in UTC time given your start time (eg. Monday 9:00am would be Sunday 11:00pm when converted to UTC from the AEST+10 timezone). Otherwise - it's a fancy string to enum converter.

The WebClient and Holidays class

Of course, the EventX Trilogy have the AbstractAPI Holidays API integrated, to allow automatically excluding public holidays in your locale. This has now been moved into Lurgle.Dates.

This is implemented using Flurl.Http and includes the ability to specify a proxy configuration when you don't have direct connectivity. This is simply done via

WebClient.SetConfig(AppName, useProxy, proxyUrl, proxyUserName, proxyPass, bypassLocal, localAddresses);

where useProxy and bypassLocal are boolean, and localAddresses is an array of URLs to exclude from proxy. Only AppName and UseProxy have to be specified, so if you don't need proxy, just pass

WebClient.SetConfig(AppName, useProxy)

The AppName is used to set the UserAgent field in the HTTP request.

To retrieve the AbstractAPI Holidays for today, you'll call

var result = WebClient.GetHolidays(ApiKey, "AU", DateTime.Today).Result;

And to handle the resulting List<AbstractApiHolidays>:

var holidays = Holidays.ValidateHolidays(result, "National,Local", "Australia,New South Wales", includeBank, includeWeekends);

which will validate the result using your rules for holiday type (only match National and Local in the above sample) and locations (only match Australia and New South Wales). IncludeBank and IncludeWeekends are boolean to direct whether to include holidays that match "Bank Holiday", and holidays that fall on weekends.

The end result is a List<AbstractAPIHolidays> that you can reference with;

foreach (var holiday in holidays)
{
var utcStart = holiday.UtcStart
var utcEnd = holiday.UtcEnd
var localStart = holiday.LocalStart
}

which are DateTime values for the holidays that are matched by your rules.

The DateTokens class

Introduced with Event Schedule, I added the ability to include Date tokens and expressions in the Summary, Description, and Tags properties that are logged when a schedule triggers.

DateTokens.ValidDate(DateString);

This simply validates that a string has been passed as "yyyy-MM-dd" format via a simple regex, and returns true/false.

DateTokens.ValidDateExpression(DateExpressionString);

This validates that a date expression has been passed with at least one of the following values (spaces are optional, because we can correct those with the next method).

1d 1h 1m

where d = day, h = hour, and m = minute. Obviously you can specify any valid numeric value for day, hour, and minute. You want a date expression of 9000h? Go for it.

DateTokens.SetValidExpression(DateExpressionString));

This simply makes sure that we have a valid date expression that includes spaces between the day, hour, and month expressions. This is needed when you send these expressions to Jira, for example.

DateTokens.HandleTokens(valueList, tokenKeyPair);
DateTokens.HandleTokens(message, tokenKeyPair);
DateTokens.HandleTokens(message);

This is where the magic of Event Schedule comes from. You can specify more or less any valid .NET custom date string as a token in the Summary, Description, and Tags fields. Optionally, you can append modifiers to the end to subtract days, months, or years.

The examples I gave when introducing this feature still apply;

{d M y+10d} - Add 10 days, with an example output 1 1 21.
{dd MM yy+10m} - Add 10 months, with an example output 01 11 21.
{dd MMM yyy+10y} - Add 10 years, with an example output 01 Jan 2021.
{MMMM yyyy-1m} - Subtract 1 month, with an example output December 2020.
{ddd dd MMM yyyy} - Sun 01 Aug 2021

And you can specify simple date tokens as well, to return parts of a date and optional add/subtract:

{d} {MMM} {yyyy} - 3 Aug 2021
{d-1} {MMM} {yyyy} - 2 Aug 2021

but bear in mind that if you try to calculate last months' dates with simple date tokens, you'll wind up disappointed. That's what the .NET custom date logic with modifiers is for.

As of v1.0.7, you can easily turn these expressions into datetimes, using DateTokens.CalculateDateExpression(dateToken);

On top of this, we include {LogToken} and {LogTokenLong} tags that can optionally be passed to HandleTokens as a keypair. Simply put, if your message includes those tags, whatever you pass in the keypair will be applied as follows:

{LogToken} - tokenKeyPair.Key
{LogTokenLong} - tokenKeyPair.Value

Which makes the Event Schedule functionality of multi-log tokens possible, since it can maintain a dictionary of LogToken=LogTokenLong values.

I dropped the initial "custom tokens" idea from this method, because frankly, if since I already had the ability to set a single template summary/description/tag with multiple date expressions and the log token keypair, the static list of custom tokens really added nothing useful.

The DateParse class

This is where I branched out and pulled in code that had been created for Seq Reporter. I needed a fairly flexible date expression scheme that would allow me to specify a period and time. So I created something that allowed expressions like;

1s - 1 second ago
1m - 1 minute ago
1h - 1 hour ago
1d - 1 day ago
1w - 1 week ago
1M - 1 month ago
now - current date and time

which is nice enough if you want to calculate based on the current time, but I also allowed for hybrid date and time expressions;

1M 9:00 - 1 month ago at 9:00
1d 9:00 - 1 day ago at 9:00
1w 9:00 - 1 week ago at 9:00

As well as a best effort to parse a specific date and time string into a DateTime, using the local culture rules.

For the uplift to Lurgle.Dates, I created both UTC and local methods. Seq.Client.Reporter operates in UTC to be consistent with Seq, and converts local to UTC as part of its methods, but having clarity between UTC or local is useful - hence the calls look like;

DateParse.GetDateTime("30d"))
DateParse.GetDateTimeUtc("30d 9:00")
DateParse.GetDateTime("-30d"))
DateParse.GetDateTimeUtc("-30d 9:00")
DateParse.GetDateTime("+30d")
DateParse.GetDateTimeUtc("+30d 9:00")

Note that for Lurgle.Dates, I've added the ability to specify a "+" or "-" operator, to instruct whether to add or subtract the date expression. Since the original functionality was to subtract, subtract is the default without these operators - you'll get 30 days ago.

Updates to EventX Trilogy

So that's Lurgle.Dates in a nutshell. It's a powerful library of date expressions, tokens, and parsing that powers multiple projects - starting with Event Timeout, Event Threshold, and Event Schedule.

Event Timeout and Event Threshold particularly benefit, both from the bug fix and from uplifting to a level of feature parity with Event Schedule where it makes sense. This means that integration with other apps, like Seq.App.OpsGenie and Seq.App.Atlassian.Jira, can potentially benefit - although I need to circle back on the OpsGenie app and see if I can pass those properties in a way that's useful to OpsGenie (unlikely with how OpsGenie APIs are structured, though).

You may want to read this important note before updating these two - this issue "shouldn't" occur, but if it does, it's a simple fix by editing and saving your instance settings.

You may need to update your Nuget API feed to see the Event Timeout update, due to an issue with the Nuget v2 API.

Event Timeout

Switch to Lurgle.Dates as common date expression library
Add the ProjectKey, Initial Time Estimate, Remaining Time Estimate, and Due Date properties from Event Schedule, for integration with other Seq apps
Add date expression functionality from Event Schedule to allow inserting dates to your Message, Description, and Tags.
Including the Log Description with the message is now optional, per Event Schedule

Event Threshold

Switch to Lurgle.Dates as common date expression library
Add the ProjectKey, Initial Time Estimate, Remaining Time Estimate, and Due Date properties from Event Schedule, for integration with other Seq apps
Add date expression functionality from Event Schedule to allow inserting dates to your Message, Description, and Tags.
Including the Log Description with the message is now optional, per Event Schedule

Event Schedule

Switch to Lurgle.Dates as common date expression library
Remove custom tokens ("I do not think they mean what you think they mean")

Get them!

Powerful capabilities and consistent date handling via a common library - hard to argue with, right? Here's a collection of fancy links, but of course to install or update the EventX apps in Nuget, all you need is the Nuget ID which is also shown below.