You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DELETE fails with AnalysisException on tables with CHAR(N) columns
Steps to reproduce
importio.delta.tables.DeltaTable
spark.sql("create table test (a char(5)) using delta")
DeltaTable.forName(spark, "test").delete()
Observed results
org.apache.spark.sql.delta.DeltaAnalysisException: [DELTA_UNSUPPORTED_SOURCE] DELETE destination only supports Delta sources.
Some(DeleteFromTable true
+- SubqueryAlias spark_catalog.default.test
+- Project [staticinvoke(class org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType, readSidePadding, a#51, 5, true, false, true) AS a#52]
+- Relation spark_catalog.default.test[a#51] parquet
)
at org.apache.spark.sql.delta.DeltaErrorsBase.notADeltaSourceException(DeltaErrors.scala:536)
at org.apache.spark.sql.delta.DeltaErrorsBase.notADeltaSourceException$(DeltaErrors.scala:534)
at org.apache.spark.sql.delta.DeltaErrors$.notADeltaSourceException(DeltaErrors.scala:3382)
at org.apache.spark.sql.delta.DeltaAnalysis$$anonfun$apply$1.applyOrElse(DeltaAnalysis.scala:465)
at org.apache.spark.sql.delta.DeltaAnalysis$$anonfun$apply$1.applyOrElse(DeltaAnalysis.scala:81)
Expected results
DELETE is performed.
Further details
It fails because the following code returns multiple indices:
valindices= newTarget.collect {
caseDeltaFullTable(_, index) => index
}
when newTarget is a Project with a Relation as a child. The project node is added by Spark to add CharVarcharCodegenUtils.readSidePadding.
Environment information
Delta Lake version: 3.2.1
Spark version: 3.5.4
Scala version: 2.12.18
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
Yes. I can contribute a fix for this bug independently.
Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
No. I cannot contribute a bug fix at this time.
The text was updated successfully, but these errors were encountered:
Bug
Which Delta project/connector is this regarding?
Describe the problem
DELETE fails with AnalysisException on tables with CHAR(N) columns
Steps to reproduce
Observed results
Expected results
DELETE
is performed.Further details
It fails because the following code returns multiple indices:
when
newTarget
is aProject
with aRelation
as a child. The project node is added by Spark to addCharVarcharCodegenUtils.readSidePadding
.Environment information
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
The text was updated successfully, but these errors were encountered: