From 7d14d68bfa33bc018ef44f9462af747f2da407aa Mon Sep 17 00:00:00 2001 From: Ilian Iliev Date: Fri, 6 Jun 2025 13:18:24 +0300 Subject: [PATCH 01/16] Moving time formatting examples to separate page Updating examples of how to set keys TTL --- .../formatting-date-and-time-values.md | 122 ++++++++++++++++++ .../redis-expiration-example.md | 100 ++------------ 2 files changed, 133 insertions(+), 89 deletions(-) create mode 100644 content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md new file mode 100644 index 000000000..5032481fa --- /dev/null +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -0,0 +1,122 @@ +--- +Title: Formatting date and time values +alwaysopen: false +categories: +- docs +- integrate +- rs +- rdi +description: null +group: di +linkTitle: Formatting date and time values +summary: Redis Data Integration keeps Redis in sync with a primary database in near + real time. +type: integration +weight: 40 +--- + +How to format date and time values depends on the source database and the data type of the field, and how they are represented in the incoming record. Below are examples for different databases and data types. + +## Oracle + +Oracle supports the following date and time data types: + +- `DATE` - represented by debezium as a 64-bit integer representing the milliseconds since epoch + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_date + language: sql + # Date is stored as a Unix timestamp in milliseconds so you need to + # divide it by 1000 to convert it to seconds. + expression: STRFTIME('%Y-%m-%d %H:%M:%S', DATE / 1000, 'unixepoch') + # Example: 1749047572000 is transformed to 2025-06-04 14:32:52 + ``` +- `TIMESTAMP` - the value is represented by Debezium as a 64-bit integer and depends on the number of decimal places of precision of the column, representing fractions of a second. For example, if the column is defined as `TIMESTAMP(6)`, there are six decimal places and so the value is represented as microseconds since epoch (since there are 10^6 microseconds in each second). +You can format it similarly to `DATE`, but you need to divide the value by the appropriate factor based on the precision. + +- `TIMESTAMP WITH TIME ZONE` - the value is represented as string representation of the timestamp with time zone information. + +- `TIMESTAMP WITH LOCAL TIME ZONE` - the value is represented as string representation of the timestamp with local time zone information. + + Both `TIMESTAMP WITH TIME ZONE` and `TIMESTAMP WITH LOCAL TIME ZONE` are supported by SQLite and can be formatted using the `STRFTIME` function. + + ```yaml + transform: + - uses: add_field + with: + fields: + - field: seconds_since_epoch + language: sql + # Convert the timestamp with local time zone to seconds since epoch. + expression: STRFTIME('%s', TIMESTAMP_FIELD) + + - field: date_from_timestamp + language: sql + # Convert the timestamp with local time zone to date and time. + expression: STRFTIME('%Y-%m-%d %H:%M:%S', TIMESTAMP_FIELD) + ``` + +---- + +## SQL Server +SQL Server supports the following date and time data types: + +- `date` - represented by Debezium as number of days since epoch (1970-01-01). You can multiply the value by 86400 (the number of seconds in a day) to convert it to seconds since epoch and then use the `STRFTIME` or `DATE` functions to format it. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: with_default_date_format + language: sql + # Uses the default DATE format + expression: DATE(event_date * 86400, 'unixepoch') + + - field: with_custom_date_format + language: sql + # Uses the default DATE format + expression: STRFTIME('%Y/%m/%d', event_date * 86400, 'unixepoch') + ``` + +- `datetime`, `smalldatetime` - represented by Debezium as number of milliseconds since epoch. You have to divide the value by 1000 to convert it to seconds since epoch and then use the `STRFTIME` function to format it. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_datetime + language: sql + expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_datetime / 1000, 'unixepoch') + ``` + +- `datetime2` - similar to `datetime` but with higher precision. For `datetime2(0-3)` the representation is the same as for `datetime`. For `datetime2(4-6)` it is the number of microseconds since epoch. and for `datetime2(7)` it is the number of nanoseconds since epoch. You can use the same approach as for `datetime` but you need to divide by 1000, 1000000 or 1000000000 depending on the precision. + +- `time` - the time of milliseconds since midnight. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_time + language: sql + expression: TIME(event_time, 'unixepoch', 'utc') + ``` + +- `datetimeoffset` - represented as a timestamp with timezone information e.g. `2025-05-27T15:21:42.864Z` and `2025-01-02T14:45:30.123+05:00`. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_datetimeoffset + language: sql + expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_datetimeoffset) + ``` + + + + + diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md index b63e19a00..c522e2962 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md @@ -53,112 +53,34 @@ output: In some cases, you can also set the expiration time based on a field that contains a date, datetime, or timestamp value, but it depends on the source database and the data types it supports. See the examples below for your specific source database and data type. -### Oracle examples +There are two main approaches you can use to set the expiration time based on a date, datetime, or timestamp field: -The transformation depends on the data type of the field in the source database: +- For values representing a passed amount of time e.g. milliseconds since epoch start, you have to convert the value to seconds since epoch and subtracting the current time in seconds since epoch from it. -- `DATE` - represented by debezium as a 64-bit integer representing the milliseconds since epoch - ```yaml - output: - - uses: redis.write - with: - data_type: hash - expire: - # To set the expiration time to a date field, convert the value to seconds and subtract the current time in seconds since epoch - expression: EXPIRES_DATE / 1000 - STRFTIME('%s', 'now') - language: sql - ``` -- `TIMESTAMP` - the value is represented by Debezium as a 64-bit integer and depends on the number of decimal places of precision of the column, representing fractions of a second. For example, if the column is defined as `TIMESTAMP(6)`, there are six decimal places and so the value is represented as microseconds since epoch (since there are 10^6 microseconds in each second). - ```yaml - output: - - uses: redis.write - with: - data_type: hash - expire: - # To set the expiration time to a date field, convert the value to seconds (divider differs based on the fractional second precision) and subtract the current time in seconds since epoch. Example below is for 6 digits of precision. - expression: EXPIRES_TIMESTAMP / 1000000 - STRFTIME('%s', 'now') - language: sql - ``` -- `TIMESTAMP WITH TIME ZONE` - the value is represented as string representation of the timestamp with time zone information. -- `TIMESTAMP WITH LOCAL TIME ZONE` - the value is represented as string representation of the timestamp with local time zone information. - - For both `TIMESTAMP WITH TIME ZONE` and `TIMESTAMP WITH LOCAL TIME ZONE`, a two-step approach is needed. First, calculate the difference between the given time and now in seconds and then invert the value. ```yaml - transform: - - uses: add_field - with: - fields: - - field: expire_seconds - language: jmespath - expression: time_delta_seconds(EXPIRES_TS_TZ) - output: + output: - uses: redis.write with: data_type: hash expire: - # `time_delta_seconds` Returns the number of seconds between a given dt and now. - # A negative value means that the given dt is in the future, so we need to invert it. - # A positive value means that the given dt is in the past, so set the expiration to -1 (expire immediately). - expression: CASE WHEN expire_seconds < 0 THEN -expire_seconds ELSE -1 END + # To set the expiration time to a date field, convert the value to + # seconds (e.g. divide it by 1000 if the fields has milliseconds precision) + # and subtract the current time in seconds since epoch. + expression: EXPIRES_TIMESTAMP / 1000 - STRFTIME('%s', 'now') language: sql ``` ----- - -### SQL Server examples -SQL Server supports the following date and time data types: +- For values matching the subset of ISO 8601 supported by SQLite e.g. `2023-10-01T12:00:00`, `2023-10-01T12:00:00Z` or `2025-06-05T13:40:14.784000+02:00` you can use the `STRFTIME` function to convert the value to seconds since epoch and subtract the current time in seconds since epoch from it. -- `date` - represented in Debezium as number of days since epoch (1970-01-01). Please note that due to the lack of time information, this method is not very accurate. - ```yaml - output: - - uses: redis.write - with: - data_type: hash - expire: - # Calculate the number of seconds equivalent to the number of days and subtract the current time in seconds since epoch. - expression: (event_date * 86400) - strftime('%s', 'now') - language: sql - ``` - -- `datetime`, `smalldatetime` - represented in Debezium as number of milliseconds since epoch. - ```yaml - output: - - uses: redis.write - with: - data_type: hash - expire: - # Since event_datetime is in miliseconds, you must divide it by 1000 to convert it to seconds. - expression: event_datetime / 1000 - strftime('%s', 'now') - language: sql - ``` -- `datetime2` - similar to `datetime` but with higher precision. For `datetime2(0-3)` the representation is the same as for `datetime`. For `datetime2(4-6)` it is the number of microseconds since epoch. and for `datetime2(7)` it is the number of nanoseconds since epoch. You can use the same approach as for `datetime` but you need to divide by 1000, 1000000 or 1000000000 depending on the precision. - -- `time` - the time of milliseconds since midnight. - ```yaml - output: - - uses: redis.write - with: - data_type: hash - expire: - # Convert the time to seconds and subtract the current time in seconds since midnight. - expression: (event_time / 1000.0) - - ( - CAST(strftime('%H', 'now') AS INTEGER) * 3600 + - CAST(strftime('%M', 'now') AS INTEGER) * 60 + - CAST(strftime('%S', 'now') AS INTEGER) - ) - language: sql - ``` -- `datetimeoffset` - represented as a timestamp with timezone information, where the timezone is GMT ```yaml output: - uses: redis.write with: data_type: hash expire: - # Convert the time to seconds and subtract the current time in seconds since epoch. - expression: strftime('%s', event_datetimeoffset) - strftime('%s', 'now') language: sql + expression: STRFTIME('%s', EXPIRATION_TS) - STRFTIME('%s', 'now') ``` - +For more examples on how to manipulate date and time values, see the [Formatting date and time values]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values/">}}) page. + From 27a1286d3c0ce9d112b198d3a52bbf50ad29a799 Mon Sep 17 00:00:00 2001 From: Ilian Iliev Date: Fri, 6 Jun 2025 14:48:18 +0300 Subject: [PATCH 02/16] Adding PostgreSQL examples --- .../formatting-date-and-time-values.md | 67 +++++++++++++++++++ .../transform-examples/map-example.md | 2 +- 2 files changed, 68 insertions(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 5032481fa..c3cbf12ee 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -120,3 +120,70 @@ SQL Server supports the following date and time data types: + + + +---- + +## PostgreSQL + +PostgreSQL supports the following date and time data types: + +- `date` - represented by Debezium as number of days since epoch (1970-01-01). You can multiply the value by 86400 (the number of seconds in a day) to convert it to seconds since epoch and then use the `STRFTIME` or `DATE` functions to format it. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: with_default_date_format + language: sql + # Uses the default DATE format + expression: DATE(event_date * 86400, 'unixepoch') + ``` + +- `time` - the time of microseconds since midnight. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_time + language: sql + # Divide by 1000000 to convert microseconds to seconds + expression: TIME(event_time / 1000000, 'unixepoch', 'utc') + ``` + +- `time with time zone` - a string representation of the time with timezone information, where the timezone is GMT, example `07:15:00Z`. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_time_with_tz + language: sql + expression: STRFTIME('%H:%M:%S', event_time_with_time_zone) + ``` + +- `timestamp` - represented by Debezium as a 64-bit integer representing the microseconds since epoch. You can use the `STRFTIME` function to format it. + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_timestamp + language: sql + # Divide by 1000000 to convert microseconds to seconds + expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_timestamp / 1000000, 'unixepoch') + ``` + +- `timestamp with time zone` - represented by Debezium as a string representation of the timestamp with time zone information, where the timezone is GMT, e.g. `2025-06-07T10:15:00.000000Z` + ```yaml + transform: + - uses: add_field + with: + fields: + - field: formatted_timestamp_with_tz + language: sql + # Divide by 1000000 to convert microseconds to seconds + expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_timestamp_with_time_zone) + ``` diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/map-example.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/map-example.md index 296297828..6eb25b841 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/map-example.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/map-example.md @@ -167,4 +167,4 @@ like the following: 6) "3:35.0" 7) "storagesize" 8) "6.71MB" -``` \ No newline at end of file +``` From 35f096842abb66b719e209e33c5622b7d738737f Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:25:01 +0300 Subject: [PATCH 03/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index c3cbf12ee..2d74da488 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -15,7 +15,7 @@ type: integration weight: 40 --- -How to format date and time values depends on the source database and the data type of the field, and how they are represented in the incoming record. Below are examples for different databases and data types. +The way you format date and time values depends on the source database, the data type of the field, and how it is represented in the incoming record. Below are some examples for different databases and data types. ## Oracle From 51ff78933516dcdcf33415eac39de8c62d841549 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:25:09 +0300 Subject: [PATCH 04/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 2d74da488..35ddd7567 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -21,7 +21,7 @@ The way you format date and time values depends on the source database, the data Oracle supports the following date and time data types: -- `DATE` - represented by debezium as a 64-bit integer representing the milliseconds since epoch +- `DATE` - represented by Debezium as a 64-bit integer representing the milliseconds since epoch ```yaml transform: - uses: add_field From f5472d0ee0cbae8054a518055ebb1f5654d52d2e Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:25:30 +0300 Subject: [PATCH 05/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 35ddd7567..8a223de27 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -37,9 +37,9 @@ Oracle supports the following date and time data types: - `TIMESTAMP` - the value is represented by Debezium as a 64-bit integer and depends on the number of decimal places of precision of the column, representing fractions of a second. For example, if the column is defined as `TIMESTAMP(6)`, there are six decimal places and so the value is represented as microseconds since epoch (since there are 10^6 microseconds in each second). You can format it similarly to `DATE`, but you need to divide the value by the appropriate factor based on the precision. -- `TIMESTAMP WITH TIME ZONE` - the value is represented as string representation of the timestamp with time zone information. +- `TIMESTAMP WITH TIME ZONE` - the value is represented as a string containing the timestamp and time zone. -- `TIMESTAMP WITH LOCAL TIME ZONE` - the value is represented as string representation of the timestamp with local time zone information. +- `TIMESTAMP WITH LOCAL TIME ZONE` - the value is represented as a string containing the timestamp and local time zone. Both `TIMESTAMP WITH TIME ZONE` and `TIMESTAMP WITH LOCAL TIME ZONE` are supported by SQLite and can be formatted using the `STRFTIME` function. From 2adb15b3719d263ba36e1482a649f8813d3cbc0e Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:25:58 +0300 Subject: [PATCH 06/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 8a223de27..cc26c1c97 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -41,7 +41,7 @@ You can format it similarly to `DATE`, but you need to divide the value by the a - `TIMESTAMP WITH LOCAL TIME ZONE` - the value is represented as a string containing the timestamp and local time zone. - Both `TIMESTAMP WITH TIME ZONE` and `TIMESTAMP WITH LOCAL TIME ZONE` are supported by SQLite and can be formatted using the `STRFTIME` function. + SQLite supports both `TIMESTAMP WITH TIME ZONE` and `TIMESTAMP WITH LOCAL TIME ZONE`. You can format them using the `STRFTIME` function. ```yaml transform: From 60e71132fc1194309ca44a92cb2db8e167b2e144 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:26:07 +0300 Subject: [PATCH 07/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index cc26c1c97..ff6094118 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -81,7 +81,7 @@ SQL Server supports the following date and time data types: expression: STRFTIME('%Y/%m/%d', event_date * 86400, 'unixepoch') ``` -- `datetime`, `smalldatetime` - represented by Debezium as number of milliseconds since epoch. You have to divide the value by 1000 to convert it to seconds since epoch and then use the `STRFTIME` function to format it. +- `datetime`, `smalldatetime` - represented by Debezium as number of milliseconds since epoch. Divide the value by 1000 to convert it to seconds since epoch and then use the `STRFTIME` function to format it. ```yaml transform: - uses: add_field From 0aa0f55def5182c15a1f4fe5b2c84e66a9f32568 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:26:38 +0300 Subject: [PATCH 08/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index ff6094118..0e38011cd 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -92,7 +92,7 @@ SQL Server supports the following date and time data types: expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_datetime / 1000, 'unixepoch') ``` -- `datetime2` - similar to `datetime` but with higher precision. For `datetime2(0-3)` the representation is the same as for `datetime`. For `datetime2(4-6)` it is the number of microseconds since epoch. and for `datetime2(7)` it is the number of nanoseconds since epoch. You can use the same approach as for `datetime` but you need to divide by 1000, 1000000 or 1000000000 depending on the precision. +- `datetime2` - similar to `datetime` but with higher precision. For `datetime2(0-3)`, the representation is the same as for `datetime`. For `datetime2(4-6)`, it is the number of microseconds since epoch. For `datetime2(7)`, it is the number of nanoseconds since epoch. To convert to another time unit, you can use the same approach as for `datetime` but you need to divide by 1000, 1000000 or 1000000000 depending on the precision. - `time` - the time of milliseconds since midnight. ```yaml From 2d0395009b79363cf0f0579b3c00d91fddf87dbd Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:27:00 +0300 Subject: [PATCH 09/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 0e38011cd..5bac3ff60 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -94,7 +94,7 @@ SQL Server supports the following date and time data types: - `datetime2` - similar to `datetime` but with higher precision. For `datetime2(0-3)`, the representation is the same as for `datetime`. For `datetime2(4-6)`, it is the number of microseconds since epoch. For `datetime2(7)`, it is the number of nanoseconds since epoch. To convert to another time unit, you can use the same approach as for `datetime` but you need to divide by 1000, 1000000 or 1000000000 depending on the precision. -- `time` - the time of milliseconds since midnight. +- `time` - the number of milliseconds since midnight. ```yaml transform: - uses: add_field From fd453941f809b88b27edb934118823b983f46eab Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:27:09 +0300 Subject: [PATCH 10/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 5bac3ff60..ec6ab50ba 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -164,7 +164,7 @@ PostgreSQL supports the following date and time data types: expression: STRFTIME('%H:%M:%S', event_time_with_time_zone) ``` -- `timestamp` - represented by Debezium as a 64-bit integer representing the microseconds since epoch. You can use the `STRFTIME` function to format it. +- `timestamp` - represented by Debezium as a 64-bit integer containing the microseconds since epoch. You can use the `STRFTIME` function to format it. ```yaml transform: - uses: add_field From 2f49c22bf97793c4e915dedd78a30b8c4d6638f7 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:46:44 +0300 Subject: [PATCH 11/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index ec6ab50ba..9143bbe4c 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -176,7 +176,7 @@ PostgreSQL supports the following date and time data types: expression: STRFTIME('%Y-%m-%d %H:%M:%S', event_timestamp / 1000000, 'unixepoch') ``` -- `timestamp with time zone` - represented by Debezium as a string representation of the timestamp with time zone information, where the timezone is GMT, e.g. `2025-06-07T10:15:00.000000Z` +- `timestamp with time zone` - represented by Debezium as a string containing the timestamp with time zone information, where the timezone is GMT (for example, `2025-06-07T10:15:00.000000Z`). ```yaml transform: - uses: add_field From a374f8b0e6a7d41fc474dafd488067ff7d2172a8 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:47:34 +0300 Subject: [PATCH 12/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/redis-expiration-example.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md index c522e2962..190829355 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md @@ -55,7 +55,7 @@ In some cases, you can also set the expiration time based on a field that contai There are two main approaches you can use to set the expiration time based on a date, datetime, or timestamp field: -- For values representing a passed amount of time e.g. milliseconds since epoch start, you have to convert the value to seconds since epoch and subtracting the current time in seconds since epoch from it. +- For values representing an elapsed time since epoch start (in milliseconds, for example), you have to convert the value to seconds since epoch and then subtract the current time (also in seconds since epoch). The difference between the two is the time until expiration. ```yaml output: From 56618085f9527bb73f3ec2a3ee1411b18b5b955b Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:47:45 +0300 Subject: [PATCH 13/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/redis-expiration-example.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md index 190829355..2ab060f69 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md @@ -70,7 +70,7 @@ There are two main approaches you can use to set the expiration time based on a language: sql ``` -- For values matching the subset of ISO 8601 supported by SQLite e.g. `2023-10-01T12:00:00`, `2023-10-01T12:00:00Z` or `2025-06-05T13:40:14.784000+02:00` you can use the `STRFTIME` function to convert the value to seconds since epoch and subtract the current time in seconds since epoch from it. +- For values matching the subset of ISO 8601 supported by SQLite (for example, `2023-10-01T12:00:00`, `2023-10-01T12:00:00Z`, or `2025-06-05T13:40:14.784000+02:00`), you can use the `STRFTIME` function to convert the value to seconds since epoch and subtract the current time in seconds since epoch from it. ```yaml output: From 8960a3f07857c78a2aa976c27293ef5842be2978 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:48:04 +0300 Subject: [PATCH 14/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/redis-expiration-example.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md index 2ab060f69..8ecfe10c3 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-expiration-example.md @@ -82,5 +82,5 @@ There are two main approaches you can use to set the expiration time based on a expression: STRFTIME('%s', EXPIRATION_TS) - STRFTIME('%s', 'now') ``` -For more examples on how to manipulate date and time values, see the [Formatting date and time values]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values/">}}) page. +For more examples of how to manipulate date and time values, see [Formatting date and time values]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values/">}}). From cd632847dc6535124cd679009ef646e145c2082f Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:48:17 +0300 Subject: [PATCH 15/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index 9143bbe4c..b1ddad2e7 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -105,7 +105,7 @@ SQL Server supports the following date and time data types: expression: TIME(event_time, 'unixepoch', 'utc') ``` -- `datetimeoffset` - represented as a timestamp with timezone information e.g. `2025-05-27T15:21:42.864Z` and `2025-01-02T14:45:30.123+05:00`. +- `datetimeoffset` - represented as a timestamp with timezone information (for example, `2025-05-27T15:21:42.864Z` or `2025-01-02T14:45:30.123+05:00`). ```yaml transform: - uses: add_field From df6f801a9a883ca177a3bc97cac1e30d183417f8 Mon Sep 17 00:00:00 2001 From: ilianiliev-redis Date: Wed, 11 Jun 2025 14:48:26 +0300 Subject: [PATCH 16/16] Update content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> --- .../transform-examples/formatting-date-and-time-values.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md index b1ddad2e7..b4b4b02ad 100644 --- a/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md +++ b/content/integrate/redis-data-integration/data-pipelines/transform-examples/formatting-date-and-time-values.md @@ -153,7 +153,7 @@ PostgreSQL supports the following date and time data types: expression: TIME(event_time / 1000000, 'unixepoch', 'utc') ``` -- `time with time zone` - a string representation of the time with timezone information, where the timezone is GMT, example `07:15:00Z`. +- `time with time zone` - a string representation of the time with timezone information, where the timezone is GMT (for example, `07:15:00Z`). ```yaml transform: - uses: add_field