This repository has been archived by the owner on Dec 4, 2023. It is now read-only.

Commits

Commits on Jul 28, 2021

[SPARK-36314][SS] Update Sessionization examples to use native support of session window

HeartSaVioR
authored and
viirya
committed

Commits on Jul 21, 2021

[SPARK-36172][SS] Document session window into Structured Streaming guide doc
HeartSaVioR
committed

Commits on Jul 16, 2021

[SPARK-34893][SS] Support session window natively

committed

Commits on Jul 15, 2021

[SPARK-36157][SQL][SS] TimeWindow expression: apply filter before project

HeartSaVioR
authored and
viirya
committed

Commits on Jul 14, 2021

[SPARK-34892][SS] Introduce MergingSortWithSessionWindowStateIterator sorting input rows and rows in state efficiently
HeartSaVioR
committed

Commits on Jul 13, 2021

[SPARK-34891][SS] Introduce state store manager for session window in streaming query

HeartSaVioR
authored and
viirya
committed

Commits on Jul 12, 2021

[SPARK-35861][SS] Introduce "prefix match scan" feature on state store

HeartSaVioR
authored and
viirya
committed

Commits on Jun 26, 2021

[SPARK-35894][BUILD] Introduce new style enforce to not import scala.collection.Seq/IndexedSeq
HeartSaVioR
committed

Commits on Jun 23, 2021

[SPARK-34889][SS] Introduce MergingSessionsIterator merging elements directly which belong to the same session

HeartSaVioR
authored and
viirya
committed

Commits on Jun 21, 2021

[SPARK-35611][SS] Introduce the strategy on mismatched offset for start offset timestamp on Kafka data source

HeartSaVioR
authored and
viirya
committed

Commits on May 25, 2021

[SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source
HeartSaVioR
committed

Commits on May 17, 2021

[SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements
HeartSaVioR
committed

Commits on Mar 29, 2021

[SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

HeartSaVioR
authored and
cloud-fan
committed

Commits on Mar 22, 2021

[SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR
authored and
gaborgsomogyi
committed

Commits on Feb 5, 2021

[SPARK-34339][CORE][SQL] Expose the number of total paths in Utils.buildLocationMetadata()

HeartSaVioR
authored and
HyukjinKwon
committed

Commits on Feb 3, 2021

[SPARK-34326][CORE][SQL] Fix UTs added in SPARK-31793 depending on the length of temp path
HeartSaVioR
committed

Commits on Feb 2, 2021

[SPARK-34326][CORE][SQL] Fix UTs added in SPARK-31793 depending on the length of temp path
HeartSaVioR
committed

Commits on Jan 6, 2021

[SPARK-33635][SS] Adjust the order of check in KafkaTokenUtil.needTokenUpdate to remedy perf regression

HeartSaVioR
authored and
dongjoon-hyun
committed

Commits on Dec 21, 2020

[SPARK-33836][SS][PYTHON] Expose DataStreamReader.table and DataStreamWriter.toTable

HeartSaVioR
authored and
HyukjinKwon
committed

Commits on Dec 4, 2020

[SPARK-27237][SS] Introduce State schema validation among query restart

HeartSaVioR
authored and
HyukjinKwon
committed

Commits on Dec 1, 2020

Commits on Nov 30, 2020

[SPARK-33440][CORE] Use current timestamp with warning log in HadoopFSDelegationTokenProvider when the issue date for token is not set up properly
HeartSaVioR
committed

Commits on Nov 25, 2020

[SPARK-33224][SS][WEBUI] Add watermark gap information into SS UI page
HeartSaVioR
committed

Commits on Nov 21, 2020

[SPARK-33223][SS][FOLLOWUP] Clarify the meaning of "number of rows dropped by watermark" in SS UI page
HeartSaVioR
committed

Commits on Nov 5, 2020

[SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore
HeartSaVioR
committed

Commits on Oct 28, 2020

Commits on Oct 24, 2020

[SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

HeartSaVioR
authored and
viirya
committed

Commits on Oct 14, 2020

[SPARK-33136][SQL] Fix mistakenly swapped parameter in V2WriteCommand.outputResolved

HeartSaVioR
authored and
dongjoon-hyun
committed

Commits on Oct 9, 2020

[SPARK-32896][SS] Add DataStreamWriter.table API

HeartSaVioR
authored and
dongjoon-hyun
committed

Commits on Sep 28, 2020

[SPARK-33011][ML] Promote the stability annotation to Evolving for MLEvent traits/classes

HeartSaVioR
authored and
HyukjinKwon
committed

Commits on Sep 17, 2020

[SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption
HeartSaVioR
committed

Commits on Sep 10, 2020