Tutorial 02 Nautobot setup: use poetry as installation method

02-Nautobot-setup/README.md

@@ -4,45 +4,72 @@
 
 This tutorial describes a quick way of setting up Nautobot on the local system.
 
-After spinning up containers, the Nautobot database will be filled with some basic data, so that it does not have to be handled manually. 
+After spinning up Nautobot containers, the database will be filled with some generic data, like credentials, so that user does not have to add it manually. 
 
 
 ## Setup Nautobot
 
-The version of the Nautobot to be used: `2.1.1`.
-
-Clone nautobot repo into the `source` directory and then copy poetry dependency files adequately:
+The version of the Nautobot to be used: `2.2.7` and python `3.9`
 
+Before starting Nautobot, create Docker network that containers will be using to communicate with each other:
 ```
-$ git clone https://github.com/nautobot/nautobot.git --branch v2.1.1 source
-$ cp nautobot/pyproject.toml nautobot/poetry.lock ./source
+$ docker network create --driver bridge automation_net
 ```
 
-Make sure to setup ENV variables as such:
+Make sure to install `invoke` on your local system and setup python version to 3.9:
 ```
-$ export NAUTOBOT_VERSION=2.1.1
+$ pip install invoke==2.2.0
 $ export PYTHON_VER=3.9
 ```
 
-These ones are used by `docker-compose` when building Nautobot image on the local system.
+Now, docker images have to be build with `docker-compose build` but we can take advantage of `tasks.py` and run `invoke build`:
 
-Before starting Nautobot, create Docker network that containers will be using to communicate with each other:
 ```
-$ docker network create --driver bridge automation_net
-```
-
-Now, docker images have to be build with `docker-compose build` command and some arguments:
+$ invoke build
+Building Nautobot with Python 3.9...
+Running docker compose command "build"
+redis uses an image, skipping
+selenium uses an image, skipping
+db uses an image, skipping
+celery_beat uses an image, skipping
+celery_worker uses an image, skipping
+Building nautobot
+#2 [internal] load build definition from Dockerfile
+#2 sha256:7278356e40f728c6282b8018e4afb12601a906d00c6574fa90d3bf6eff024c57
+#2 transferring dockerfile: 997B done
+#2 DONE 0.0s
+
+#1 [internal] load .dockerignore
+#1 sha256:4af1373924805dbe623198cd0b7e22555f2c5f32ea4970c0dfadc4f539b212ca
+#1 transferring context: 2B done
+#1 DONE 0.0s
+
+#3 [internal] load metadata for ghcr.io/nautobot/nautobot-dev:2.2.7-py3.11
+#3 sha256:c656460b0c17c61b03e1e7b00d4541daa593ea14b6291fceaaf23deaffe90605
+#3 DONE 0.8s
+
+#12 [base 1/5] FROM ghcr.io/nautobot/nautobot-dev:2.2.7-py3.11@sha256:1a93f873b672146cd526d3d9772d5ed2b113818e9b9f33d898f856522ad0b62e
+#12 sha256:902149c9fad999e3035f48015c069ed7507623143f9dba9b5e8abfe1d7b6d6a2
+#12 DONE 0.0s
+
+#10 [internal] load build context
+#10 sha256:a1de697bd746d9e9b374f64c45a81188ad4b59d73f0b6bead7164977336f27f6
+#10 transferring context: 676B done
+#10 DONE 0.0s
+
+...
+<skipped>
 
 ```
-$ docker-compose build --build-arg NAUTOBOT_VERSION="$NAUTOBOT_VERSION" --build-arg PYTHON_VER="$PYTHON_VER"
-```
 
-Start Nautobot by using `docker-compose up` command:
+`invoke` command helps us manage local Nautobot instance. We can build image, start or stop instance and so on.
+
+Once image was successfully build, Nautobot can be started using following command:
 ```
-$ docker-compose up
+$ invoke start
 ```
 
-At this step you can navigate to [Nautobot UI](http://localhost:8080/) and login with credentials `admin/admin`. There is no any data in Nautobot so far, so let's populate some data into Nautobot.
+At this step, after successful start, you can navigate to [Nautobot UI](http://localhost:8080/) and login with credentials `admin/admin`. There is no any data in Nautobot so far, so let's populate some data into Nautobot database.
 
 ### Pre-populate data into Nautobot
 ----------------------------------
@@ -54,10 +81,10 @@ This step generates:
 - platform
 - secrets
 
-Enter the Nautobot shell inside container: `docker exec -it nautobot_lab nautobot-server nbshell`
+Enter the Nautobot shell inside container: `invoke nbshell`
 
 ```
-$ docker exec -it nautobot_lab nautobot-server nbshell
+$ invoke nbshell
 # Shell Plus Model Imports
 from constance.backends.database.models import Constance
 from django.contrib.admin.models import LogEntry
@@ -110,7 +137,7 @@ Type "help", "copyright", "credits" or "license" for more information.
 
 ```
 
-Onced in Python/nbshell shell, execute following command `exec(open('/source/scripts/populate_data.py').read())`:
+Once in Python/nbshell shell, execute following command `exec(open('/source/scripts/populate_data.py').read())`:
 ```
 >>> 
 >>> exec(open('/source/scripts/populate_data.py').read())
@@ -129,4 +156,4 @@ Created platform: Arista
 
 ## Conclusion
 
-At this stage Nautobot is able to communicate with our VMs and gather some basic information about them. Next let's look at the details of `device onboarding` app in Nautobot, but now you can already explore Nautobot UI from the browser.
+At this stage Nautobot is up and running, but also is able to communicate with our VMs. In the next part let's look at the details of `device onboarding` app in Nautobot, but before that you can explore Nautobot UI from the browser.

02-Nautobot-setup/nautobot/Dockerfile

@@ -0,0 +1,47 @@
+ARG NAUTOBOT_VER="2.2.7"
+ARG PYTHON_VER="3.9"
+
+FROM ghcr.io/nautobot/nautobot-dev:${NAUTOBOT_VER}-py${PYTHON_VER} AS base
+
+
+ARG NAUTOBOT_ROOT=/opt/nautobot
+
+ENV prometheus_multiproc_dir=/prom_cache
+ENV NAUTOBOT_ROOT=${NAUTOBOT_ROOT}
+
+
+RUN which poetry || curl -sSL https://install.python-poetry.org | python3 - && \
+    poetry config virtualenvs.create false
+
+
+COPY pyproject.toml poetry.lock /source/
+WORKDIR /source
+
+ENV PATH="${POETRY_HOME}/bin:${PATH}"
+
+RUN --mount=type=cache,target="/root/.cache",sharing=locked \
+    poetry install --no-root --no-ansi --no-interaction --only main
+
+
+FROM base AS apps
+
+RUN --mount=type=cache,target="/root/.cache",sharing=locked \
+    poetry install --no-root --no-ansi --no-interaction --only apps
+
+
+FROM apps AS dev
+
+COPY pkg /source/pkg
+
+RUN --mount=type=cache,target="/root/.cache",sharing=locked \
+    poetry install --no-root --no-ansi --no-interaction --only dev
+
+RUN cd /source/pkg/custom_napalm && \
+    poetry install --no-ansi --no-interaction --only main
+
+RUN cd /source/pkg/onboarding_extensions && \ 
+    poetry install --no-ansi --no-interaction --only main
+
+
+COPY nautobot_config.py ${NAUTOBOT_ROOT}/nautobot_config.py
+

02-Nautobot-setup/docker-compose.yml → 02-Nautobot-setup/nautobot/docker-compose.yml

@@ -6,17 +6,17 @@ services:
     build:
       args:
         PYTHON_VER: "${PYTHON_VER}"
-      context: ./source
-      dockerfile: docker/Dockerfile
+      context: .
+      dockerfile: Dockerfile
       target: dev
-    image: "nautobot:${NAUTOBOT_VERSION}"
+    image: "nautobot:latest"
     ports:
       - "8080:8080"
     volumes:
       - "media_root:/opt/nautobot/media"
-      - "./nautobot/populate_data.py:/source/scripts/populate_data.py"
-      - "./nautobot/nautobot_config.py:/opt/nautobot/nautobot_config.py"
-      - ./source/:/source
+      - "./populate_data.py:/source/scripts/populate_data.py"
+      - "./nautobot_config.py:/opt/nautobot/nautobot_config.py"
+      - "./pkg:/source/pkg"
     networks:
       - automation_net
     depends_on:
@@ -27,7 +27,7 @@ services:
       db:
         condition: service_healthy
     env_file:
-      - "./nautobot/local.env"
+      - "./local.env"
     tty: true
     healthcheck:
       interval: 5s
@@ -41,13 +41,13 @@ services:
         - "http://localhost:8080/health/"
   celery_worker:
     container_name: celery_worker_lab
-    image: "nautobot:${NAUTOBOT_VERSION}"
+    image: "nautobot:latest"
     ports:
       - "8081:8080"
     volumes:
       - media_root:/opt/nautobot/media
-      - "./nautobot/nautobot_config.py:/opt/nautobot/nautobot_config.py"
-      - ./source/:/source
+      - "./nautobot_config.py:/opt/nautobot/nautobot_config.py"
+      - "./pkg:/source/pkg"
     networks:
       - automation_net
     entrypoint: "watchmedo auto-restart --directory './' --pattern '*.py' --recursive -- nautobot-server celery worker -l INFO --events"
@@ -67,15 +67,15 @@ services:
       redis:
         condition: service_started
     env_file:
-      - "./nautobot/local.env"
+      - "./local.env"
     tty: true
   celery_beat:
     container_name: celery_beat_lab
-    image: "nautobot:${NAUTOBOT_VERSION}"
+    image: "nautobot:latest"
     entrypoint: "nautobot-server celery beat -l INFO"
     volumes:
-      - "./nautobot/nautobot_config.py:/opt/nautobot/nautobot_config.py"
-      - ./source/:/source
+      - "./nautobot_config.py:/opt/nautobot/nautobot_config.py"
+      - "./pkg:/source/pkg"
     networks:
       - automation_net
     healthcheck:
@@ -86,7 +86,7 @@ services:
       redis:
         condition: service_started
     env_file:
-      - "./nautobot/local.env"
+      - "./local.env"
     tty: true
   redis:
     container_name: redis_lab
@@ -98,7 +98,7 @@ services:
     networks:
       - automation_net
     env_file:
-      - "./nautobot/local.env"
+      - "./local.env"
   selenium:
     container_name: selenium_lab
     image: selenium/standalone-firefox:4.9.1
@@ -112,7 +112,7 @@ services:
     container_name: db_lab
     image: postgres:13
     env_file:
-      - "./nautobot/local.env"
+      - "./local.env"
     volumes:
       - pgdata_nautobot:/var/lib/postgresql/data
     networks:
@@ -126,7 +126,7 @@ services:
     container_name: mkdocs_lab
     profiles:
       - mkdocs
-    image: "local/nautobot-dev:local-py${PYTHON_VER}"
+    image: "nautobot:latest"
     entrypoint: "mkdocs serve -a '0.0.0.0:8001'"
     networks:
       - automation_net

02-Nautobot-setup/nautobot/local.env

@@ -1,8 +1,8 @@
 # ---------------------
 # PYTHON_VER is used for which version of Python to use. Check hub.docker.com for the available versions
 # ---------------------
-NAUTOBOT_PYTHON_VERSION=3.9
-NAUTOBOT_VERSION=2.1.1
+NAUTOBOT_PYTHON_VERSION=3.11
+NAUTOBOT_VERSION=2.2.7
 
 # This should be limited to the hosts that are going to be the web app.
 # https://docs.djangoproject.com/en/3.2/ref/settings/#allowed-hosts
@@ -59,3 +59,6 @@ NAUTOBOT_SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567
 
 ARISTA_USERNAME=admin
 ARISTA_PASSWORD=admin
+
+NAPALM_USERNAME=admin
+NAPALM_PASSWORD=admin

02-Nautobot-setup/nautobot/nautobot_config.py

@@ -46,8 +46,52 @@
     "nautobot_ssot",
     "nautobot_plugin_nornir",
     "nautobot_golden_config",
+    "nautobot_capacity_metrics",
+    "nautobot_device_lifecycle_mgmt",
 ]
 
+
+PLUGINS_CONFIG = {
+    "nautobot_capacity_metrics": {
+        "enable": True
+    },
+    "nautobot_ssot": {
+        "enable": True
+    },
+    "nautobot_plugin_nornir": {
+        "enable": True
+    },
+    "nautobot_golden_config": {
+        "enable": True
+    },
+    "nautobot_device_lifecycle_mgmt": {
+        "enable": True
+    },
+    "nautobot_device_onboarding": {
+        "create_platform_if_missing": False,
+        "create_manufacturer_if_missing": False,
+        "create_device_type_if_missing": True,
+        "create_device_role_if_missing": True,
+        "create_management_interface_if_missing": True,
+        "default_device_status": "Active",
+        "default_device_role": "unknown",
+        "default_device_role_color": "ff0000",
+        "skip_device_type_on_update": False,
+        "skip_manufacturer_on_update": False,
+        "platform_map": {
+            "cisco_ios": "cisco_ios",
+            "cisco_eos": "cisco_eos",
+        },
+        "onboarding_extensions_map": {
+            "custom_eos": "onboarding_extensions.custom_eos",
+        },
+        "object_match_strategy": "strict",
+        "enable": True,
+    },
+}
+
+
+
 CORS_ALLOWED_ORIGINS = ["http://localhost:3000"]
 CORS_ALLOW_CREDENTIALS = True
 SESSION_COOKIE_SAMESITE = None

02-Nautobot-setup/nautobot/pkg/custom_napalm/custom_napalm/custom_eos.py

@@ -0,0 +1,22 @@
+from napalm.eos.eos import EOSDriver
+
+
+class CustomEOSDriver(EOSDriver):
+    """Custom NAPALM Arista EOS Handler."""
+
+    def get_my_banner(self):
+        command = 'show banner motd'
+        output = self._send_command(command)
+
+        return_vars = {}
+        for line in output.splitlines():
+            split_line = line.split()
+            if "Site:" == split_line[0]:
+                return_vars["site"] = split_line[1]
+            elif "Device:" == split_line[0]:
+                return_vars["device"] = split_line[1]
+            elif "Floor:" == split_line[0]:
+                return_vars["floor"] = split_line[1]
+            elif "Room:" == split_line[0]:
+                return_vars["room"] = split_line[1]
+        return return_vars

02-Nautobot-setup/nautobot/pkg/custom_napalm/pyproject.toml

@@ -0,0 +1,16 @@
+[tool.poetry]
+name = "custom-napalm"
+version = "0.1.0"
+description = "custom napalm"
+authors = ["Michal Spiez <[email protected]>"]
+license = "Apache-2.0"
+packages = [
+    { include = "custom_napalm" }
+]
+
+[tool.poetry.dependencies]
+python = ">=3.8,<3.12"
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"