*** We are hiring! *** Looking for backend / network / cloud engineering supports for deployment and scale. ᓚᘏᗢ Please contact [email protected]
We are driven by a humble desire to explore the immense potential of MEC and edge computing for Large Language Models Applications. We are building an ecosystem. We aim to provide the necessary infrastructure while leveraging the advantages of MEC. This involves optimizing data consumption, enhancing security and privacy, addressing the challenges between cloud, edge, and end-users, and achieving improved redundancy and resilience. We're also leveraging the wealth of additional information offered by MEC and synergizing it with large language models to enhance the overall user experience. HW/SW co-design diagrams:- 💻 Device side
we seamlessly integrate various user data types, including live information, location data, and inputs in multiple formats, such as images, text, audio, JSON, and video saved in the vector database. These inputs undergo embedding searches for relevant information and are then processed within our Flow Client before being transmitted to the MEC as inputs.
- 🌐 MEC side
We leverage MEC APIs and local vector databases, each housing data in diverse formats. These rich local data, along with the user inputs from the device side, empower our Large Language Models with user-specific insights and abundant local knowledge. We are also facilitating seamless streaming audio interactions through our streaming ASR TTS services, which operate on the MEC.
- 🔧 Edge Computing Front
We are striving to maximize the potential of edge resources, including computing, storage, and server engines, which opens up opportunities in the future such as federated learning on the edge. This approach enables us to deliver highly customized and localized services, ultimately shaping the future of our platform.
pip install -r requirements.txt create a file named ".env" and add OPENAI_API_KEY=YOUR OPENAI API KEYrun app.py