Skip to content

Commit

Permalink
Python: Add Hive Catalog (#5391)
Browse files Browse the repository at this point in the history
  • Loading branch information
Fokko committed Sep 30, 2023
1 parent 365f307 commit 69e37c7
Show file tree
Hide file tree
Showing 19 changed files with 119,125 additions and 194 deletions.
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
# under the License.
---
files: ^python/
exclude: ^python/vendor/

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
Expand Down
20 changes: 20 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -213,3 +213,23 @@ Home page: https://avro.apache.org/
License: https://www.apache.org/licenses/LICENSE-2.0

--------------------------------------------------------------------------------

This product includes code from Apache Thrift.

* Uses the fb303.thrift file that's part of Hive's thrift service in vendor/fb303/

Copyright: 2006-2022 The Apache Software Foundation.
Home page: https://thrift.apache.org/
License: https://www.apache.org/licenses/LICENSE-2.0

--------------------------------------------------------------------------------

This product includes code from Apache Hive.

* Uses hive_metastore.thrift to generate the Hive Metastore client in vendor/hive_metastore/

Copyright: 2008-2022 The Apache Software Foundation.
Home page: https://hive.apache.org/
License: https://www.apache.org/licenses/LICENSE-2.0

--------------------------------------------------------------------------------
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

install:
pip install poetry
poetry install -E pyarrow
poetry install -E pyarrow -E hive

lint:
poetry run pre-commit run --all-files
Expand Down
236 changes: 48 additions & 188 deletions poetry.lock

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions pyiceberg/catalog/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def create_table(
def load_table(self, identifier: str | Identifier) -> Table:
"""Loads the table's metadata and returns the table instance.
You can also use this method to check for table existence using 'try catalog.table() except TableNotFoundError'
You can also use this method to check for table existence using 'try catalog.table() except NoSuchTableError'
Note: This method doesn't scan data stored in the table.
Args:
Expand Down Expand Up @@ -166,7 +166,7 @@ def drop_namespace(self, namespace: str | Identifier) -> None:
"""

@abstractmethod
def list_tables(self, namespace: str | Identifier | None = None) -> list[Identifier]:
def list_tables(self, namespace: str | Identifier) -> list[Identifier]:
"""List tables under the given namespace in the catalog.
If namespace not provided, will list all tables in the catalog.
Expand Down
Loading

0 comments on commit 69e37c7

Please sign in to comment.