Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Schemas comparison fails if we have added comments inside each column metadata in our delta table irrespective of fact that schemas are similar #3975

Open
Kartik031299 opened this issue Dec 16, 2024 · 1 comment

Comments

@Kartik031299
Copy link

private boolean areSchemasEqual(@Nullable StructType first,

@Kartik031299
Copy link
Author

Looks like StructField allows you to add some metadata: https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/types/Metadata.html but in flink row type it's not possible to add metadata with each RowField and hence this comparison fails when there is some metadata attached to column in delta table even though column names and column types are same

@Kartik031299 Kartik031299 changed the title Schemas comparison fails if we have added comments inside each column metadata in our delta table irrespective of fact that schemas are similar [Bug] Schemas comparison fails if we have added comments inside each column metadata in our delta table irrespective of fact that schemas are similar Dec 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant