Regular Transactions

We run a nightly batch job to group the transactions belonging to an account into regular series, based on patterns in the transaction dates and similarities in their descriptions and amount. When a new series is detected or a new transaction is detected for an existing series the details can be sent to the regular transactions Kafka egress topic. All existing series for a user or account can also be retrieved through the API.

In order to do this we require a fixed identifier for a transaction that does not change when the transaction status changes, eg. from pending to booked.

Requirements

A minimum number of transactions are required to detect a series, this depends on the frequency as listed below. There are also a maximum number of transactions that will be stored in a series.

Frequency	Minimum number of transactions	Maximum number of transactions
Weekly	8	16
Fortnightly	6	12
Monthly	3	18
Quarterly	4	12
Yearly	3	6

Date and amount predictions

Due to the nature of Direct Debits (can be over 4 working days) and card payments (within 3 working days) there is some tolerance in the predicted date. Where consistency is demonstrated the earliest and latest date will be the same.

Some series are for a regular amount (such as rent and mortgage), whereas some are for regular purposes where the amount may differ (such as a varying mobile phone bill based on usage). Using previous transactions we predict the next payment value. We also provide the upper and lower limit of the prediction for reference.

Series Statistics

Along with the transactions, we provide powerful insights for each regular transaction series:

The number of transactions in a series is a count of transactions detected.
The gap length is the number of times a payment has not been detected at the expected time.
Date anomalies are when a transaction occurs outside the usual cadence of the series.
Returned payments is a count where the payment is returned to the account shortly after leaving on the expected day, this is most likely due to insufficient funds in the account.
Whether the predicted payment is overdue.
The predicted level 2 category of the next payment due.