Docs and README update for ZenML Cloud (zenml-io#1723)

* new docs pages

* removing 3 files

* GITBOOK-24: big changes

* GITBOOK-25: new new changes

* new docs structure with the right directory structure

* GITBOOK-26: change request with no subject merged in GitBook

* GITBOOK-28: change request with no subject merged in GitBook

* GITBOOK-29: change request with no subject merged in GitBook

* GITBOOK-30: change request with no subject merged in GitBook

* GITBOOK-33: change request with no subject merged in GitBook

* GITBOOK-34: change request with no subject merged in GitBook

* GITBOOK-35: change request with no subject merged in GitBook

* GITBOOK-36: change request with no subject merged in GitBook

* missing scarf

* Optimised images with calibre/image-actions

* GITBOOK-38: change request with no subject merged in GitBook

* GITBOOK-39: change request with no subject merged in GitBook

* small readme changes

* formatting

* fixing more links

* pr review suggestions

* more fixes

* Update docs/book/deploying-zenml/zenml-cloud/get-started.md

Co-authored-by: Alex Strick van Linschoten <[email protected]>

* Update docs/book/user-guide/advanced-guide/custom-stack-solutions/implement-a-custom-integration.md

Co-authored-by: Alex Strick van Linschoten <[email protected]>

* Update docs/book/reference/faq.md

Co-authored-by: Alex Strick van Linschoten <[email protected]>

* review changes

* removed one question

* adding links

* adding links

* GITBOOK-41: change request with no subject merged in GitBook

* fixing more markdown link errors

* more link problems

* GITBOOK-42: change request with no subject merged in GitBook

* GITBOOK-43: intro cards

* cleaning up assets

* more asset cleanup

* in code link fixes

* custom step operator fix

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Alex Strick van Linschoten <[email protected]>

.github/pull_request_template.md

@@ -5,7 +5,7 @@ I implemented/fixed _ to achieve _.
 Please ensure you have done the following:
 - [ ] I have read the **CONTRIBUTING.md** document.
 - [ ] If my change requires a change to docs, I have updated the documentation accordingly.
-- [ ] If I have added an integration, I have updated the [integrations](https://docs.zenml.io/component-gallery/integrations) table and the [corresponding website section](https://zenml.io/integrations).
+- [ ] If I have added an integration, I have updated the [integrations](https://docs.zenml.io/stacks-and-components/component-guide) table and the [corresponding website section](https://zenml.io/integrations).
 - [ ] I have added tests to cover my changes.
 
 ## Types of changes

CONTRIBUTING.md

@@ -217,7 +217,7 @@ for detailed step-by-step instructions.
 [Examples README](examples/README.md) 
 to find out what to do.
 3. All integrations deserve to be documented. Make sure to pay a visit to the
-[Component Guide](https://docs.zenml.io/user-guide/component-guide)
+[Component Guide](https://docs.zenml.io/stacks-and-components/component-guide)
 in the docs and add your implementations. 
 
 ## 🆘 Getting Help

README.md

@@ -89,10 +89,11 @@
     ·
     <a href="#-meet-the-team">Meet the Team</a>
     <br />
+    <b>ZenML Cloud</b> is now available in beta. <a href="https://cloud.zenml.io">Sign up</a> to see it in action.  
+    <br />
     🎉 Version 0.42.1 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
-    <br />
     <a href="https://www.linkedin.com/company/zenml/">
     <img src="https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555" alt="Logo">
     </a>
@@ -185,17 +186,17 @@ enable collaborative features as the central MLOps interface for teams.
 
 ![ZenML Architecture Diagram.](docs/book/.gitbook/assets/Scenario3.png)
 
-In case your machine is authenticated with one of the big three cloud 
-providers, this command will do the full deployment for you.
+Currently, there are two main options to deploy ZenML:
 
-```bash
-zenml deploy --provider aws  # aws, gcp and azure are supported providers
-```
+- **ZenML Cloud**: With [ZenML Cloud](https://docs.zenml.io/deploying-zenml/zenml-cloud), 
+you can utilize a control plane to create ZenML servers, also known as tenants. 
+These tenants are managed and maintained by ZenML's dedicated team, alleviating 
+the burden of server management from your end. 
 
-You can also choose to deploy with docker or helm with full control over
-the configuration and deployment. Check out the
-[docs](https://docs.zenml.io/platform-guide/set-up-your-mlops-platform/deploy-zenml)
-to find out how.
+- **Self-hosted deployment**: Alternatively, you have the flexibility to [deploy 
+ZenML on your own self-hosted environment](https://docs.zenml.io/deploying-zenml/zenml-self-hosted). 
+This can be achieved through various methods, including using our CLI, Docker, 
+Helm, or HuggingFace Spaces.
 
 ## 👨‍🍳 2. Deploy Stack Components
 
@@ -219,7 +220,7 @@ zenml stack register production_stack --orchestrator kubernetes_orchestrator --a
 When you run a pipeline with this stack set, it will be running on your deployed
 Kubernetes cluster.
 
-You can also [deploy your own tooling manually](https://docs.zenml.io/platform-guide/set-up-your-mlops-platform/deploy-and-set-up-a-cloud-stack).
+You can also [deploy your own tooling manually](https://docs.zenml.io/stacks-and-components/stack-deployment).
 
 ## 🏇 3. Create a Pipeline
 
@@ -282,7 +283,7 @@ and you can directly influence the roadmap as follows:
   board](https://zenml.io/discussion).
 - Start a thread in our [Slack channel](https://zenml.io/slack-invite).
 - [Create an issue](https://github.com/zenml-io/zenml/issues/new/choose) on our
-  Github repo.
+  GitHub repo.
 
 # 🙌 Contributing and Community
 

RELEASE_NOTES.md

@@ -172,14 +172,14 @@ now be assigned to any step output using `typing_extensions.Annotated`.
 
 This is a minor ZenML release that introduces a couple of new features:
 
-* the [Azure Service Connector](https://docs.zenml.io/platform-guide/set-up-your-mlops-platform/connect-to-your-cloud-provider/azure-service-connector) is now available in addition to the AWS and GCP ones. It can be used to connect ZenML and Stack Components to Azure cloud infrastructure resources like Azure Blob Storage, Azure Container Registry and Azure Kubernetes Service.
+* the [Azure Service Connector](https://docs.zenml.io/stacks-and-components/auth-management/azure-service-connector) is now available in addition to the AWS and GCP ones. It can be used to connect ZenML and Stack Components to Azure cloud infrastructure resources like Azure Blob Storage, Azure Container Registry and Azure Kubernetes Service.
 * Service Connectors can now also be managed through the ZenML Dashboard
 * adds `zenml secret export` CLI command to export secrets from the ZenML Secret Store to a local file
 * adds the ability to create/update ZenML secrets from JSON/YAML files or command line arguments (courtesy of @bhatt-priyadutt)
 
 In addition to that, this release also contains a couple of bug fixes and improvements, including:
 
-* better documentation and fixes for the ZenML [Vertex AI Orchestrator](https://docs.zenml.io/user-guide/component-guide/orchestrators/vertex) and [Vertex AI Step Operator](https://docs.zenml.io/user-guide/component-guide/step-operators/vertex)
+* better documentation and fixes for the ZenML [Vertex AI Orchestrator](https://docs.zenml.io/stacks-and-components/component-guide/orchestrators/vertex) and [Vertex AI Step Operator](https://docs.zenml.io/stacks-and-components/component-guide/step-operators/vertex)
 * adjust Seldon and BentoML Steps and Examples to new pipeline interface
 
 ## What's Changed
@@ -347,7 +347,7 @@ Here are just a few ways you could use ZenML Service Connectors:
 - Assisted setup with security in mind: Our Service Connectors come with features for configuration validation and verification, the generation of temporary, low-privilege credentials, and pre-authenticated and pre-configured clients for Python libraries.
 - Easy local configuration transfer: ZenML's Service Connectors aim to resolve the reproducibility issue in ML pipelines. They do this by automatically transferring authentication configurations and credentials from your local machine, storing them securely, and allowing for effortless sharing across different environments.
 
-[Visit our documentation pages](https://docs.zenml.io/platform-guide/set-up-your-mlops-platform/connect-zenml-to-infrastructure) to learn more about ZenML Connectors and how you can use them in a way that supports your ML workflows.
+[Visit our documentation pages](https://docs.zenml.io/stacks-and-components/auth-management) to learn more about ZenML Connectors and how you can use them in a way that supports your ML workflows.
 
 ## What's Changed
 

docker/zenml-server-hf-spaces.Dockerfile

@@ -11,7 +11,7 @@ ENV ZENML_SERVER_DEPLOYMENT_TYPE="hf_spaces"
 # localstorage in a SQLite database. If you would like to make your storage
 # persistent, use the appropriate environment variables below to configure the
 # image to use a MySQL-compatible database service that is reachable from the
-# container. See https://docs.zenml.io/getting-started/deploying-zenml/docker
+# container. See https://docs.zenml.io/getting-started/deploying-zenml/zenml-self-hosted/deploy-with-docker
 # for more information on how to configure these environment variables.
 
 # You can also configure the secrets store to use for your ZenML server. Be 

docs/book/deploying-zenml/zenml-cloud/get-started.md

@@ -0,0 +1,35 @@
+---
+description: Your first steps to join ZenML Cloud and set up your first tenant.
+---
+
+# Get started
+
+This page will walk you through the simple steps to sign up for the ZenML Cloud and gain access to a managed ZenML server instance, called a tenant, with exciting new features.
+
+<figure><img src="../../.gitbook/assets/zenml-cloud-signup-page.png" alt=""><figcaption></figcaption></figure>
+
+## Step 1: Sign-Up
+
+Visit [the ZenML website](https://www.zenml.io/home) and navigate to the ZenML Cloud sign-up page. You can find it in the top-right corner.
+
+On the ZenML Cloud sign-up page, you will have the option to sign up using either your Google account or your GitHub account. Click on the respective button to proceed with your preferred sign-up method.
+
+After successfully signing in with your Google or GitHub account, you will be asked to complete your ZenML Cloud profile. Provide any additional information required to set it up, such as your name, email address, and organization details.
+
+<figure><img src="../../.gitbook/assets/zenml-cloud-form.png" alt=""><figcaption></figcaption></figure>
+
+## Step 2: Automatic tenant creation
+
+Upon completing the sign-up process, you will gain access to a free trial period lasting for 30 days. You will have access to a ZenML tenant (automatically created for you behind the scenes) for the duration of this trial period. This setup process typically takes a few minutes to complete, and once it's ready you will receive an email notification. This will mark the beginning of your 30-day trial, allowing you to explore and experience the benefits of ZenML's features firsthand.
+
+<figure><img src="../../.gitbook/assets/zenml-cloud-tenant-creation.png" alt=""><figcaption></figcaption></figure>
+
+## Step 3: Run your first pipeline
+
+Once your tenant is ready, you will be provided with a set of instructions. Following these instructions will guide you through the process of running your first pipeline, ensuring that your tenant is fully functional and operational.
+
+<figure><img src="../../.gitbook/assets/zenml-cloud-tenant-details.png" alt=""><figcaption></figcaption></figure>
+
+Congratulations! You are now a part of the ZenML Cloud and have successfully set up your first managed ZenML instance. We appreciate your participation in the beta phase and value your feedback to improve the ZenML Cloud experience further. If you have any questions or need assistance, please don't hesitate to reach out to our dedicated support team.
+
+<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>

docs/book/deploying-zenml/zenml-cloud/zenml-cloud.md

@@ -0,0 +1,31 @@
+---
+description: Your one-stop MLOps control plane.
+---
+
+# ☁ ZenML Cloud
+
+ZenML Cloud is a Software-as-a-Service (SaaS) platform that extends the capabilities of the open-source ZenML product. It provides you with a centralized control plane to effortlessly launch and manage their ZenML server instances. The core foundation of ZenML Cloud remains the powerful open-source offering, but with ZenML Cloud, you gain access to a host of additional features designed to streamline the machine learning workflow.
+
+<div data-full-width="true">
+
+<figure><img src="../../.gitbook/assets/image.png" alt=""><figcaption><p>an architectural overview of the ZenML Cloud</p></figcaption></figure>
+
+</div>
+
+## Key features
+
+ZenML Cloud brings convenience to your machine learning workflows by allowing you to deploy a managed instance of ZenML servers with a single click. This means you can set up and manage your machine learning pipelines effortlessly without the hassle of dealing with infrastructure complexities. We take care of all the necessary upgrades and backups, ensuring that your system remains up-to-date and resilient while you focus on your core MLOps tasks. As a valued cloud customer, you'll also benefit from priority support, ensuring that you receive the assistance you need to make the most of our platform.
+
+Data security and privacy are of utmost importance at ZenML Cloud. You can connect your infrastructure safely and securely and we track only metadata through an encrypted connection, ensuring that your sensitive information remains confidential. Our platform offers seamless integration with your cloud services through service connectors, making it easy to connect and utilize various cloud resources without compromising data security. Additionally, we store your secrets in a safe and secure isolated environment, providing an extra layer of protection for your valuable data.
+
+As a ZenML Cloud user, you also gain early access to a powerful control plane that centralizes user management and streamlines your workflows. With centralized management of users, handling access and permissions becomes efficient and organized. Moreover, ZenML Cloud users enjoy exclusive access to a range of cloud-only features, providing you with a competitive edge and the opportunity to stay ahead in the rapidly evolving field of machine learning.
+
+{% hint style="info" %}
+ZenML Cloud is currently in the beta phase, offering users the opportunity to host a managed ZenML instance and gain early access to the exciting new features mentioned above. Beta users will receive priority access to the enhanced functionalities and dedicated support to ensure a smooth onboarding experience.
+{% endhint %}
+
+## Coming soon...
+
+<table data-card-size="large" data-view="cards" data-full-width="true"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><mark style="color:purple;"><strong>CI/CD/CT with Triggers</strong></mark></td><td>Continuous Integration, Continuous Deployment, and Continuous Training are all integrated into your repository.</td><td><a href="https://zenml.io/cloud-features/continuous-integration-and-delivery-ci-cd">https://zenml.io/cloud-features/continuous-integration-and-delivery-ci-cd</a></td></tr><tr><td><mark style="color:purple;"><strong>ML Models WatchTower</strong></mark></td><td>All your models are conveniently gathered in one place. Up-to-date information about your training, deployments, and endpoints all within one view.</td><td><a href="https://zenml.io/cloud-features/ml-models-watch-tower">https://zenml.io/cloud-features/ml-models-watch-tower</a></td></tr><tr><td><mark style="color:purple;"><strong>Managed MLOps infrastructure</strong></mark></td><td>ZenML Cloud also includes hosted instances of support stack components like MLflow.</td><td><a href="https://zenml.io/cloud-features/managed-ml-ops-infrastructure">https://zenml.io/cloud-features/managed-ml-ops-infrastructure</a></td></tr><tr><td><mark style="color:purple;"><strong>Tenants and Workspaces</strong></mark></td><td>Create multiple ZenML tenants and workspaces within tenants to subdivide your team's efforts neatly.</td><td><a href="https://zenml.io/cloud-features/tenants-and-workspaces">https://zenml.io/cloud-features/tenants-and-workspaces</a></td></tr><tr><td><mark style="color:purple;"><strong>User Management</strong></mark></td><td>User permission management and project-specific configurations</td><td><a href="https://zenml.io/cloud-features/user-management">https://zenml.io/cloud-features/user-management</a></td></tr><tr><td><mark style="color:purple;"><strong>Role Based Access Control</strong></mark></td><td>Manage permissions and roles at an organizational level</td><td><a href="https://zenml.io/cloud-features/role-based-access-control">https://zenml.io/cloud-features/role-based-access-control</a></td></tr></tbody></table>
+
+<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>

docs/book/platform-guide/set-up-your-mlops-platform/deploy-zenml/deploy-using-huggingface-spaces.md → docs/book/deploying-zenml/zenml-self-hosted/deploy-using-huggingface-spaces.md

@@ -12,10 +12,10 @@ any infrastructure overhead.
 {% hint style="info" %}
 Note that it is not recommended to use HuggingFace Spaces for production use as by default the data stored there is
 non-persistent and the underlying machine is not as available to you as a dedicated machine. See
-our [other deployment options](/docs/book/platform-guide/set-up-your-mlops-platform/deploy-zenml/deploy-zenml.md) if you want to use ZenML in production.
+our [other deployment options](/docs/book/deploying-zenml/zenml-self-hosted/zenml-self-hosted.md) if you want to use ZenML in production.
 {% endhint %}
 
-![ZenML on HuggingFace Spaces -- default deployment](../../../.gitbook/assets/hf_spaces_chart.png)
+![ZenML on HuggingFace Spaces -- default deployment](../../.gitbook/assets/hf_spaces_chart.png)
 
 In this diagram, you can see what the default deployment of ZenML on HuggingFace looks like.
 
@@ -29,7 +29,7 @@ To set up your ZenML app, you need to specify three main components: the Owner (
 organization), a Space name, and the Visibility (a bit lower down the page). Note that the space visibility needs to be
 set to 'Public' if you wish to connect to the ZenML server from your local machine.
 
-![HuggingFace Spaces SDK interface](../../../.gitbook/assets/hf-spaces-sdk.png)
+![HuggingFace Spaces SDK interface](../../.gitbook/assets/hf-spaces-sdk.png)
 
 You have the option here to select a higher-tier machine to use for your server. The advantage of selecting a paid CPU
 instance is that it is not subject to auto-shutdown policies and thus will stay up as long as you leave it up. In order