Converts an Java object to days.

Converts an Java object to microseconds.

Converts the timestamp micros from one timezone to another.

Time-zone rules, such as daylight savings, mean that not every local date-time is valid for the toZone time zone, thus the local date-time may be adjusted.

Obtains an instance of java.time.LocalDate from the epoch day count.

Converts days since 1970-01-01 at the given zone ID to microseconds since 1970-01-01 00:00:00Z.

Converts a local date at the default JVM time zone to the number of days since 1970-01-01 in the hybrid calendar (Julian + Gregorian) by discarding the time part. The resulted days are rebased from the hybrid to Proleptic Gregorian calendar. The days rebasing is performed via UTC time zone for simplicity because the difference between two calendars is the same in any given time zone and UTC time zone.

Note: The date is shifted by the offset of the default JVM time zone for backward compatibility with Spark 2.4 and earlier versions. The goal of the shift is to get a local date derived from the number of days that has the same date fields (year, month, day) as the original date at the default JVM time zone.

Converts an instance of java.sql.Timestamp to the number of microseconds since 1970-01-01T00:00:00.000000Z. It extracts date-time fields from the input, builds a local timestamp in Proleptic Gregorian calendar from the fields, and binds the timestamp to the system time zone. The resulted instant is converted to microseconds since the epoch.

The conversion is performed via the system time zone because it is used internally in java.sql.Timestamp while extracting date-time fields.

The goal of the function is to have the same local date-time in the original calendar

• the hybrid calendar (Julian + Gregorian) and in the target calendar which is Proleptic Gregorian calendar, see SPARK-26651.

Converts an instance of java.sql.Timestamp to the number of microseconds since 1970-01-01T00:00:00.000000Z.

Gets the number of microseconds since the epoch of 1970-01-01 00:00:00Z from the given instance of java.time.Instant. The epoch microsecond count is a simple incrementing count of microseconds where microsecond 0 is 1970-01-01 00:00:00Z.

Converts the local date to the number of days since 1970-01-01.

Converts microseconds since 1970-01-01 00:00:00Z to days since 1970-01-01 at the given zone ID.

Obtains an instance of java.time.Instant using microseconds from the epoch of 1970-01-01 00:00:00Z.

Converts the timestamp to milliseconds since epoch. In Spark timestamp values have microseconds precision, so this conversion is lossy.

Converts milliseconds since the epoch to microseconds.

Trims and parses a given UTF8 timestamp string to the corresponding timestamp segments, time zone id and whether it is just time without a date. value. The return type is Option in order to distinguish between 0L and null. The following formats are allowed:

[+-]yyyy* [+-]yyyy*-[m]m [+-]yyyy*-[m]m-[d]d [+-]yyyy*-[m]m-[d]d [+-]yyyy*-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id] [+-]yyyy*-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id] [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id] T[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]

where zone_id should have one of the forms:

• Z - Zulu time zone UTC+0
• +|-[h]h:[m]m
• A short id, see https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html#SHORT_IDS
• An id with one of the prefixes UTC+, UTC-, GMT+, GMT-, UT+ or UT-, and a suffix in the formats:
  • +|-h[h]
  • +|-hh[:]mm
  • +|-hh:mm:ss
  • +|-hhmmss
• Region-based zone IDs in the form area/city, such as Europe/Paris

Trims and parses a given UTF8 date string to a corresponding Int value. The return type is Option in order to distinguish between 0 and null. The following formats are allowed:

[+-]yyyy* [+-]yyyy*-[m]m [+-]yyyy*-[m]m-[d]d [+-]yyyy*-[m]m-[d]d [+-]yyyy*-[m]m-[d]d * [+-]yyyy*-[m]m-[d]dT*

Trims and parses a given UTF8 timestamp string to the corresponding a corresponding Long value. The return type is Option in order to distinguish between 0L and null. Please refer to parseTimestampString for the allowed formats

Trims and parses a given UTF8 string to a corresponding Long value which representing the number of microseconds since the epoch. The result will be independent of time zones.

If the input string contains a component associated with time zone, the method will return None if allowTimeZone is set to false. If allowTimeZone is set to true, the method will simply discard the time zone component. Enable the check to detect situations like parsing a timestamp with time zone as TimestampNTZType.

The return type is Option in order to distinguish between 0L and null. Please refer to parseTimestampString for the allowed formats.

Converts days since the epoch 1970-01-01 in Proleptic Gregorian calendar to a local date at the default JVM time zone in the hybrid calendar (Julian + Gregorian). It rebases the given days from Proleptic Gregorian to the hybrid calendar at UTC time zone for simplicity because the difference between two calendars doesn't depend on any time zone. The result is shifted by the time zone offset in wall clock to have the same date fields (year, month, day) at the default JVM time zone as the input daysSinceEpoch in Proleptic Gregorian calendar.

Note: The date is shifted by the offset of the default JVM time zone for backward compatibility with Spark 2.4 and earlier versions.

Converts microseconds since the epoch to an instance of java.sql.Timestamp via creating a local timestamp at the system time zone in Proleptic Gregorian calendar, extracting date and time fields like year and hours, and forming new timestamp in the hybrid calendar from the extracted fields.

The conversion is based on the JVM system time zone because the java.sql.Timestamp uses the time zone internally.

The method performs the conversion via local timestamp fields to have the same date-time representation as year, month, day, ..., seconds in the original calendar and in the target calendar.

Converts microseconds since the epoch to an instance of java.sql.Timestamp.