clickhouse-jdbc 0.8.x incompatible with liquibase-core

[zeroEntropyy]

It seems that clickhouse-jdbc after the 0.8.1 version is no longer compatible with liquibase.
We were successfully using 0.6.0-patch5 version successfully with liquibase. We tried to upgrade to 0.8.1 and facing certain issues with liquibase-core library. We are using liquibase-core 4.29.2 version

In the DatabaseMetaData.java class while fetching the columns metadata, the query to get columns takes system.columns.type as the DATA_TYPE, which returns a String, but liquibase-core accepts DATA_TYPE only as an Integer. This is throwing a NumberFormatException, and it seems there is no workaround for this issue.

DatabaseMetaData ->
String sql = "SELECT " + this.catalogPlaceholder + " AS TABLE_CAT, database AS TABLE_SCHEM, table AS TABLE_NAME, name AS COLUMN_NAME, system.columns.type AS DATA_TYPE, type AS TYPE_NAME, " + JdbcUtils.generateSqlTypeSizes("system.columns.type") + " AS COLUMN_SIZE, toInt32(0) AS BUFFER_LENGTH, IF (numeric_scale == 0, NULL, numeric_scale) as DECIMAL_DIGITS, toInt32(numeric_precision_radix) AS NUM_PREC_RADIX, toInt32(position(type, 'Nullable(') >= 1 ?" + 1 + " : " + 0 + ") as NULLABLE, system.columns.comment AS REMARKS, system.columns.default_expression AS COLUMN_DEF, toInt32(0) AS SQL_DATA_TYPE, toInt32(0) AS SQL_DATETIME_SUB, character_octet_length AS CHAR_OCTET_LENGTH, toInt32(system.columns.position) AS ORDINAL_POSITION, position(upper(type), 'NULLABLE') >= 1 ? 'YES' : 'NO' AS IS_NULLABLE,NULL AS SCOPE_CATALOG, NULL AS SCOPE_SCHEMA, NULL AS SCOPE_TABLE, NULL AS SOURCE_DATA_TYPE, 'NO' as IS_AUTOINCREMENT, 'NO' as IS_GENERATEDCOLUMN FROM system.columns WHERE database LIKE '" + (schemaPattern == null ? "%" : schemaPattern) + "' AND table LIKE '" + (tableNamePattern == null ? "%" : tableNamePattern) + "' AND name LIKE '" + (columnNamePattern == null ? "%" : columnNamePattern) + "' ORDER BY TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION";

liquibase-core ->

int dataType = columnMetadataResultSet.getInt("DATA_TYPE");

Is there a way, we may be able to override the getColumns method and change the query? Or some workaround for the issue?

[peter-yf]

This is really bad, and will break any application/process that uses the JDBC meta-data to drive data access. DatabaseMetaData.getColumns() should return this, as defined by Java Spec:

This should be fixed in the driver to adhere to the JDBC specification. It needs to return the JDBC type as defined in java.sql.Types. The driver is now just returning the name of the data type, like: "UInt32"

[YellowStallion]

Looks like there's a fix for this, developed for #2281

[chernser]

@YellowStallion thank you very much for pointing it!
@zeroEntropyy would you please confirm that the problem is solved? Thanks!

[zeroEntropyy]

I need to find some time to work on this issue again, will keep the thread updated

[Paultagoras]

Hi @zeroEntropyy - closing this out as it seems fixed, but if if your tests still run into it please feel free to reopen and let us know!

[peter-yf]

The fix for this is causing another Exception.
This occurs when fetching the DataType column from the Resultset supplied by DatabseMetaData.getColumns().
This below error happens when an Unsigned Integer is present in the table that we are listing columns for, but on further investigation it seems to be happening for all DataTypes.

java.sql.SQLException: Failed to convert UInt64 to java.lang.Integer
at com.clickhouse.jdbc.internal.JdbcUtils.convert(JdbcUtils.java:269) ~[?:?]
at com.clickhouse.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:1534) ~[?:?]
at com.clickhouse.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:461) ~[?:?]
at com.hof.pool.DBUtil.getColumns(DBUtil.java:249) ~[classes/:?]

[chernser]

@peter-yf thank you for reporting!
We will look into it.

[Paultagoras]

I'll look more into it, but it's because the underlying code is calling getObject rather than getInteger

[peter-yf]

Thanks @Paultagoras

[chernser]

@peter-yf we have looked into the problem and seems need to address it with bigger changes.
Here is the issue #2371 .
The problem here is that MetadataResultSet is limited (we do have to override schema on the fly and do other adjustments what make something not working). So instead of just fixing this particular problem we will rework class.
Sorry about that.

I'm closing the issue because there is another addressing the problem.