feat: Moved to a lark-based parser (georgia-tech-db#492)

* feat: adding lark grammar * updates to grammar * introduce partially complete lark parser * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * restore tutorials * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * checkpoint * docs: remove references to antlr and JAVA * checkpoint * checkpoint * checkpoint * ci check * ci check
luoj1 · Dec 30, 2022 · 2ec9ddd · 2ec9ddd
1 parent 97dfa95
commit 2ec9ddd
Show file tree

Hide file tree

Showing 70 changed files with 2,560 additions and 2,864 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -34,27 +34,26 @@ jobs:
       - checkout
       - restore_cache:
           keys:
-            - v13-model_cache-{{ checksum "setup.py" }}
+            - v1-model_cache-{{ checksum "setup.py" }}
       # removing it - no significant advnatage
       # - restore_cache:
       #     keys:
-      #       - v13-pip-cache-{{ checksum "setup.py" }}
+      #       - v1-pip-cache-{{ checksum "setup.py" }}
 
-      - run:
-          name: Install only EVA package from pip
-          command: |
-            sudo apt-get update          
-            sudo apt-get install lsof 
-            pip install --upgrade pip
-            pip install evadb
-            bash script/test/package.sh
+      #- run:
+      #    name: Install only EVA package from pip
+      #    command: |
+      #      sudo apt-get update          
+      #      sudo apt-get install lsof 
+      #      pip install --upgrade pip
+      #      pip install evadb
+      #      bash script/test/package.sh
 
       - run:
           name: Install EVA package from repo with all dependencies
           command: |
             "python<< parameters.v >>" -m venv test_evadb
             source test_evadb/bin/activate
-            sh script/antlr4/generate_parser.sh
             pip install ".[dev]"
 
       - run:
@@ -65,12 +64,12 @@ jobs:
             coveralls
 
       - save_cache:
-          key: v13-model_cache-{{ checksum "setup.py" }}
+          key: v1-model_cache-{{ checksum "setup.py" }}
           paths:
             - /home/circleci/.cache/torch/hub/
             - /home/circleci/.EasyOCR/
 
       # - save_cache:
-      #     key: v13-pip-cache-{{ checksum "setup.py" }}
+      #     key: v1-pip-cache-{{ checksum "setup.py" }}
       #     paths:
       #       - ./test_evadb/
diff --git a/Jenkinsfile b/Jenkinsfile
@@ -29,12 +29,6 @@ pipeline {
               '''
           }
         }
-        stage('Generate Parser Files') {
-
-          steps {
-            sh 'sh script/antlr4/generate_parser.sh'
-          }
-        }
       }
     }
 

diff --git a/Jenkinsfile_weekly b/Jenkinsfile_weekly
@@ -30,12 +30,6 @@ pipeline {
               '''
           }
         }
-        stage('Generate Parser Files') {
-
-          steps {
-            sh 'sh script/antlr4/generate_parser.sh'
-          }
-        }
       }
     }
 

diff --git a/api-docs/_toc.yml b/api-docs/_toc.yml
@@ -1,6 +1,10 @@
 format: jb-book
 root: index
 parts:
+  - caption: Version
+    chapters:
+      - file: source/version
+
   - caption: Overview
     chapters:
       - file: source/overview/video

diff --git a/api-docs/conf.py b/api-docs/conf.py
@@ -95,7 +95,7 @@
         "sidebar/brand.html",
         "sidebar/search.html",
         "sidebar/navigation.html",
-        "sidebar/scroll-end.html",
+        "sidebar/scroll-end.html"
     ]
 }
 

diff --git a/api-docs/source/contribute/index.rst b/api-docs/source/contribute/index.rst
@@ -14,28 +14,16 @@ Setting up the Development Environment
 First, you will need to checkout the repository from GitHub and build EVA from
 the source. Follow the following instructions to build EVA locally. We recommend using a virtual environment and the pip package manager. 
 
-EVA requires JAVA 11 for generating the ANTLR-based EVAQL parser.
-
 .. code-block:: bash
 
    git clone https://github.com/georgia-tech-db/eva.git && cd eva
-   python3 -m venv test_eva_db                            # to create a virtual environment
-   . test_eva_db/bin/activate
-   pip install --upgrade pip
-   sudo -E apt install -y openjdk-11-jdk openjdk-11-jre   # to install JAVA
-   sh script/antlr4/generate_parser.sh                    # to generate the EVA parser
-   pip install -e ".[dev]"
-   bash script/test/test.sh                               # to run the test suite
-
-For developers using an M1 Mac, here are some pointers for installing JAVA and to resolve multi-threading issues:
-
-.. code-block:: bash      
-
-   brew install openjdk@11                         # to install openjdk 11
-   export JAVA_HOME="/opt/homebrew/opt/openjdk@11" # add this command in ~/.zshrc or ~/.bashrc
-   export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES  # to resolve multi-threading issues in macOS
+   python3 -m venv test_eva_db       # create a virtual environment
+   source test_eva_db/bin/activate   # activate the virtual environment
+   pip install --upgrade pip         # upgrade pip
+   pip install -e ".[dev]"           # build and install the EVA package
+   bash script/test/test.sh          # run the eva EVA suite
    
-After you have installed the package locally, you can make changes in the code base and examine their impact.
+After installing the package locally, you can make changes and run the test cases to check their impact.
 
 .. code-block:: bash
 

diff --git a/api-docs/source/contribute/new_command.rst b/api-docs/source/contribute/new_command.rst
@@ -33,28 +33,21 @@ We discuss each part separately.
 The parser firstly generate **syntax tree** from the input string, and
 then tansform syntax tree into **statement**.
 
-The first part of Parser is build from two ANTLR file.
+The first part of Parser is build from a LARK grammar file.
 
-parser/evaql
+parser/eva
 ~~~~~~~~~~~~
 
--  ``evaql_lexer.g4`` - add keywords(eg. CREATE, TABLE) under **Common
+-  ``eva.lark`` - add keywords(eg. CREATE, TABLE) under **Common
    Keywords**
--  ``evaql_parser.g4``
 
-   -  Add new grammar name(eg. createTable) under **ddlStatement**
+   -  Add new grammar rule (eg. create_table)
    -  Write a new grammar, for example:
 
    ::
 
-      createTable
-      : CREATE TABLE
-        ifNotExists?
-        tableName createDefinitions                                  #columnCreateTable
-      ;
+      create_table: CREATE TABLE if_not_exists? table_name create_definitions 
 
--  **Run ``sh script/antlr4/generate_parser.sh`` after modify g4 file**
-   to generate python file for the parser.
 
 --------------
 

diff --git a/api-docs/source/version.rst b/api-docs/source/version.rst
@@ -0,0 +1,7 @@
+► Versions 
+=====
+
+- `stable version <https://evadb.readthedocs.io/>`_
+- `v0.1.2 <https://evadb.readthedocs.io/en/v0.1.2>`_
+- `v0.1.1 <https://evadb.readthedocs.io/en/v0.1.1>`_
+- `developer version <https://evadb.readthedocs.io/en/dev>`_
diff --git a/eva/catalog/catalog_type.py b/eva/catalog/catalog_type.py
@@ -12,45 +12,47 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-from enum import Enum, IntEnum, auto
 
+from eva.utils.generic_utils import EVAEnum
 
-class Dimension(IntEnum):
-    ANYDIM = -1
 
+class Dimension(EVAEnum):
+    ANYDIM  # noqa: F821
+
+
+class TableType(EVAEnum):
+    STRUCTURED_DATA  # noqa: F821
+    VIDEO_DATA  # noqa: F821
+    IMAGE_DATA  # noqa: F821
 
-class TableType(IntEnum):
-    STRUCTURED_DATA = auto()
-    VIDEO_DATA = auto()
-    IMAGE_DATA = auto()
     # reserved for system generated tables
     # cannot be accessed/modified directly by user
-    SYSTEM_STRUCTURED_DATA = auto()
-
-
-class ColumnType(Enum):
-    BOOLEAN = 1
-    INTEGER = 2
-    FLOAT = 3
-    TEXT = 4
-    NDARRAY = 5
-    ANY = 6
-
-
-class NdArrayType(Enum):
-    INT8 = auto()
-    UINT8 = auto()
-    INT16 = auto()
-    INT32 = auto()
-    INT64 = auto()
-    UNICODE = auto()
-    BOOL = auto()
-    FLOAT32 = auto()
-    FLOAT64 = auto()
-    DECIMAL = auto()
-    STR = auto()
-    DATETIME = auto()
-    ANYTYPE = auto()
+    SYSTEM_STRUCTURED_DATA  # noqa: F821
+
+
+class ColumnType(EVAEnum):
+    BOOLEAN  # noqa: F821
+    INTEGER  # noqa: F821
+    FLOAT  # noqa: F821
+    TEXT  # noqa: F821
+    NDARRAY  # noqa: F821
+    ANY  # noqa: F821
+
+
+class NdArrayType(EVAEnum):
+    INT8  # noqa: F821
+    UINT8  # noqa: F821
+    INT16  # noqa: F821
+    INT32  # noqa: F821
+    INT64  # noqa: F821
+    UNICODE  # noqa: F821
+    BOOL  # noqa: F821
+    FLOAT32  # noqa: F821
+    FLOAT64  # noqa: F821
+    DECIMAL  # noqa: F821
+    STR  # noqa: F821
+    DATETIME  # noqa: F821
+    ANYTYPE  # noqa: F821
 
     @classmethod
     def to_numpy_type(cls, t):
@@ -90,10 +92,5 @@ def to_numpy_type(cls, t):
         return np_type
 
 
-class IndexType(Enum):
-    HNSW = auto()
-
-    def __str__(self):
-        # Overload the str method by getting ride of IndexType.
-        res_str = Enum.__str__(self)
-        return res_str.split(".")[-1]
+class IndexType(EVAEnum):
+    HNSW  # noqa: F821
-Original file line number
+Diff line change
@@ Expand Up / @@ -29,12 +29,6 @@ pipeline { @@
                   '''
               }
             }
-            stage('Generate Parser Files') {
-              steps {
-                sh 'sh script/antlr4/generate_parser.sh'
-              }
-            }
           }
         }
@@ Expand Down @@