22 Pull requests opened by 17 people

• [SPARK-50622] RocksDB Refactor

  #49242 opened Dec 19, 2024

• [SPARK-50623][SQL] respect table lookup error from the catalog

  #49243 opened Dec 19, 2024

• [SPARK-50624][SQL] Add TimestampNTZType to ColumnarRow

  #49244 opened Dec 19, 2024

• [SPARK-49565][SQL] Improve auto-generated expression aliases with pipe SQL operators

  #49245 opened Dec 19, 2024

• [SPARK-50625][BUILD] simplify shell scripts using mvn help:evaluate

  #49246 opened Dec 19, 2024

• [SPARK-50628] fix stop error on windows

  #49247 opened Dec 19, 2024

• [SPARK-50613][CONNECT] Support Configurable Idle Timeout of Connect Server

  #49249 opened Dec 20, 2024

• [SPARK-50632][BUILD] Upgrade tink to 1.16.0

  #49250 opened Dec 20, 2024

• [SPARK-50633][INFRA] Fix `codecov/codecov-action@v4` daily scheduling failure

  #49251 opened Dec 20, 2024

• [SPARK-50642][CONNECT][SS]Fix the state schema for FlatMapGroupsWithState in spark connect when there is no initial state

  #49260 opened Dec 21, 2024

• [SPARK-48568][SQL] Improve Performance of CodeFormatter

  #49261 opened Dec 21, 2024

• [SPARK-50648][CORE] when the job is cancelled during shuffle retry in parent stage, might leave behind zombie running tasks

  #49270 opened Dec 23, 2024

• [SPARK-49087][SQL][CONNECT] Distinguish UnresolvedFunction calling internal functions

  #49274 opened Dec 23, 2024

• [SPARK-50652][SS] Add checks to RocksDB V2 backward compatibility

  #49275 opened Dec 23, 2024

• [SPARK-50639][SQL] Improve warning logging in CacheManager

  #49276 opened Dec 23, 2024

• Ssm

  #49277 opened Dec 23, 2024

• [SPARK-50654][SS] CommitMetadata should set stateUniqueIds to None in V1

  #49278 opened Dec 24, 2024

• [SPARK-50612][SQL][FOLLOWUP] Put normalization of inner project lists under a flag

  #49285 opened Dec 24, 2024

• [SPARK-50662][PYTHON][WINDOWS] Escape ampersands in command arguments on windows

  #49286 opened Dec 24, 2024

• [SPARK-50669][SQL] Change the signature of TimestampAdd expression

  #49291 opened Dec 25, 2024

• [SPARK-48334] [CORE] Release RpcEnv when SparkContext Initialization Fails

  #49292 opened Dec 25, 2024

• [SPARK-50644][FOLLOWUP][SQL] Fix scalar cast in the shredded reader.

  #49293 opened Dec 25, 2024

34 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-50589][SQL] Avoid extra expression duplication when push filter

  #49202 commented on Dec 25, 2024 • 44 new comments

• [SPARK-50573][SS] Adding State Schema ID to State Rows to schema evolution

  #49171 commented on Dec 26, 2024 • 19 new comments

• [SPARK-50541] Describe Table As JSON

  #49139 commented on Dec 25, 2024 • 17 new comments

• [SPARK-50578][PYTHON][SS] Add support for new version of state metadata for TransformWithStateInPandas

  #49156 commented on Dec 24, 2024 • 16 new comments

• [SPARK-50561][SQL] Improve type coercion and boundary checking for UNIFORM SQL function

  #49237 commented on Dec 23, 2024 • 15 new comments

• [SQL] Table and view level collations support

  #49084 commented on Dec 25, 2024 • 13 new comments

• [TEST-ONLY][SQL] Make `QueryPlan` lockless

  #49212 commented on Dec 25, 2024 • 12 new comments

• [SPARK-50614][SQL] Add Variant shredding support for Parquet

  #49234 commented on Dec 25, 2024 • 10 new comments

• [WIP][SPARK-50636][SQL] Extending CTESubstitution.scala to make it aware of recursion

  #49232 commented on Dec 20, 2024 • 10 new comments

• [SPARK-50134][SPARK-50130][SQL][CONNECT] Support DataFrame API for SCALAR and EXISTS subqueries in Spark Connect

  #49182 commented on Dec 20, 2024 • 10 new comments

• [SPARK-50529][SQL] Change char/varchar behavior under the `spark.sql.preserveCharVarcharTypeInfo` config

  #49128 commented on Dec 25, 2024 • 10 new comments

• [SPARK-49886][SS] Query level failure test for rocksDB format V2

  #48460 commented on Dec 24, 2024 • 8 new comments

• [SPARK-50392][PYTHON] DataFrame conversion to table argument in Spark Classic

  #49055 commented on Dec 19, 2024 • 8 new comments

• [SPARK-50493][SPARK-37687][BUILD] Migrate kubernetes-client from `6.x` to `7.x`

  #49159 commented on Dec 26, 2024 • 7 new comments

• [SPARK-50618][SS][SQL] Make DataFrameReader and DataStreamReader leverage the analyzer more

  #49238 commented on Dec 23, 2024 • 6 new comments

• [SPARK-50560][SQL] Improve type coercion and boundary checking for RANDSTR SQL function

  #49210 commented on Dec 24, 2024 • 6 new comments

• [SPARK-48728][SQL] Support ignoreNulls for collect_list and collect_set

  #47149 commented on Dec 24, 2024 • 4 new comments

• [SPARK-50608][SQL][DOCS] Fix malformed configuration page caused by unclosed tags

  #49223 commented on Dec 26, 2024 • 4 new comments

• [SPARK-50616][SQL] Add File Extension Option to CSV DataSource Writer

  #49233 commented on Dec 20, 2024 • 3 new comments

• [SPARK-50469][SQL] V1Writes should respect the output ordering

  #49027 commented on Dec 24, 2024 • 3 new comments

• [SPARK-40993][CONNECT][PYTHON][DOCS] Migrate markdown style README to PySpark Development Documentation

  #49221 commented on Dec 19, 2024 • 1 new comment

• [WIP] state schema management

  #49218 commented on Dec 20, 2024 • 1 new comment

• [SPARK-43415][CONNECT][SQL] Implement `KVGDS.agg` with custom `mapValues` function

  #49111 commented on Dec 19, 2024 • 1 new comment

• [SPARK-50593][SQL] SPJ: Support truncate transform

  #49211 commented on Dec 24, 2024 • 0 new comments

• [Test Only][BUILD] Upgrade `sbt` to `1.10.6`

  #48972 commented on Dec 23, 2024 • 0 new comments

• [SPARK-50263][CONNECT] Replace `System.currentTimeMillis` with `System.nanoTime`

  #49003 commented on Dec 23, 2024 • 0 new comments

• [SPARK-50522][SQL] Support for indeterminate collation

  #49103 commented on Dec 25, 2024 • 0 new comments

• [SPARK-49883][SS][TESTS][FOLLOWUP] RocksDB Fault Tolerance Test

  #49175 commented on Dec 23, 2024 • 0 new comments

• [SPARK-50634][INFRA] Upgrade `codecov/codecov-action` to v5

  #49228 commented on Dec 20, 2024 • 0 new comments

• [SPARK-49783][YARN] Fix resource leak of yarn allocator

  #48238 commented on Dec 23, 2024 • 0 new comments

• [WIP][SPARK-49631][SQL] Remove the ANSI config suggestion in ARITHMETIC_OVERFLOW

  #48206 commented on Dec 20, 2024 • 0 new comments

• [SPARK-50600][Connect][SQL] Send optimized plan during ReadyForExecution event

  #49236 commented on Dec 19, 2024 • 0 new comments

• [DO NOT REVIEW] Table and schema level collations

  #48090 commented on Dec 26, 2024 • 0 new comments

• [SPARK-50525][SQL] Prohibit partitioning by MapType

  #49144 commented on Dec 19, 2024 • 0 new comments