Merge branch 'develop' of github_qinkai:THUDM/CodeGeeX into develop

README.md

@@ -4,7 +4,7 @@
     🏠 <a href="https://models.aminer.cn/codegeex" target="_blank">Homepage</a> | 📖 <a href="https://models.aminer.cn/codegeex/blog/" target="_blank">Blog</a> | 🪧 <a href="https://models.aminer.cn/codegeex/playground" target="_blank">DEMO</a> | 🤖 <a href="https://models.aminer.cn/codegeex/download/request" target="_blank">Download Model</a> | 📃 Paper(Coming soon!) |
 </p>
 <p align="center">
-    🛠 <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code Extension</a> | 👋 Join our <a href="https://join.slack.com/t/codegeexworkspace/shared_invite/zt-1jxpygozo-GuB40XQPiyfrCflupyLKKw"target="_blank">Slack</a> or <a href="https://models.aminer.cn/codegeex/static/xdaivscodegeex.b65f1404.png"target="_blank">WeChat</a> | 🌐 <a href="README_zh.md" target="_blank">中文</a>
+    🛠 <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code Extension</a> | 👋 Join our <a href="https://join.slack.com/t/codegeexworkspace/shared_invite/zt-1jxpygozo-GuB40XQPiyfrCflupyLKKw"target="_blank">Slack</a> or <a href="https://t.me/+IipIayJ32B1jOTg1"target="_blank">Telegram</a> or <a href="https://wj.qq.com/s2/11274205/a15b/"target="_blank">WeChat</a> | 🌐 <a href="README_zh.md" target="_blank">中文</a>
 </p>
 
 
@@ -42,6 +42,8 @@ We introduce CodeGeeX, a large-scale multilingual code generation model with 13
 
 ## News
 
+* **2022-12-04**: We release source code of quantization (requires less GPU RAM: 27GB -> 15GB) and model parallelism (possible to run on multiple GPUs with <8G RAM).
+
 * **2022-09-30**: We release the cross-platform source code and models weights for both Ascend and NVIDIA platforms. 
 
 ## Getting Started
@@ -61,7 +63,7 @@ pip install -e .
 Apply and download model weights through this [link](https://models.aminer.cn/codegeex/download/request). You'll receive by mail ```urls.txt``` that contains temporary download links. We recommend you to use [aria2](https://aria2.github.io/) to download it via the following command (Please make sure you have enough disk space to download the checkpoint (~26GB)):
 ```bash
 aria2c -x 16 -s 16 -j 4 --continue=true -i urls.txt 
-``` 
+```
 Run the following command to get the full model weights:
 ```bash
 cat codegeex_13b.tar.gz.* > codegeex_13b.tar.gz
@@ -72,7 +74,15 @@ tar xvf codegeex_13b.tar.gz
 
 Have a try on generating the first program with CodeGeeX. First, specify the path of the model weights in ``configs/codegeex_13b.sh``. Second, write the prompt (natural language description or code snippet) into a file, e.g., ``tests/test_prompt.txt``, then run the following script:
 ```bash
+# On a single GPU (with more than 27GB RAM)
 bash ./scripts/test_inference.sh <GPU_ID> ./tests/test_prompt.txt
+
+# With quantization (with more than 15GB RAM)
+bash ./scripts/test_inference_quantized.sh <GPU_ID> ./tests/test_prompt.txt
+
+# On multiple GPUs (with more than 6GB RAM, need to first convert ckpt to MP_SIZE partitions)
+bash ./scripts/convert_ckpt_parallel.sh <LOAD_CKPT_PATH> <SAVE_CKPT_PATH> <MP_SIZE>
+bash ./scripts/test_inference_parallel.sh <MP_SIZE> ./tests/test_prompt.txt
 ```
 
 ### VS Code Extension Guidance

README_zh.md

@@ -4,7 +4,7 @@
     🏠 <a href="https://models.aminer.cn/codegeex/zh-CN" target="_blank">主页</a> | 📖 <a href="https://models.aminer.cn/codegeex/blog/index_zh.html" target="_blank">博客</a> | 🪧 <a href="https://models.aminer.cn/codegeex/zh-CN/playground" target="_blank">示例</a> | 🤖 <a href="https://models.aminer.cn/codegeex/download/request" target="_blank">模型下载</a> | 📃 论文（即将推出！）
 </p>
 <p align="center">
-    🛠 <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code插件</a> | 📒 <a href="https://github.com/THUDM/CodeGeeX/blob/main/api/README_zh.md" target="_blank">API申请</a> | 👋 欢迎加入 <a href="https://join.slack.com/t/codegeexworkspace/shared_invite/zt-1jxpygozo-GuB40XQPiyfrCflupyLKKw"target="_blank">Slack</a> 或 <a href="https://models.aminer.cn/codegeex/static/xdaivscodegeex.b65f1404.png"target="_blank">微信开发者交流群</a> | 🌐 <a href="https://github.com/THUDM/CodeGeeX/blob/main/README.md" target="_blank">English</a>
+    🛠 <a href="https://marketplace.visualstudio.com/items?itemName=aminer.codegeex" target="_blank">VS Code插件</a> | 📒 <a href="https://github.com/THUDM/CodeGeeX/blob/main/api/README_zh.md" target="_blank">API申请</a> | 👋 欢迎加入 <a href="https://join.slack.com/t/codegeexworkspace/shared_invite/zt-1jxpygozo-GuB40XQPiyfrCflupyLKKw"target="_blank">Slack</a> 或 <a href="https://t.me/+IipIayJ32B1jOTg1"target="_blank">Telegram</a> 或 <a href="https://wj.qq.com/s2/11274205/a15b/"target="_blank">微信开发者交流群</a> | 🌐 <a href="https://github.com/THUDM/CodeGeeX/blob/main/README.md" target="_blank">English</a>
 </p>
 
 
@@ -41,6 +41,8 @@ CodeGeeX是一个具有130亿参数的多编程语言代码生成预训练模型
 
 ## 新闻
 
+* **2022-12-04**: 我们开源了量化代码（需要更少的显存：27GB -> 15GB）以及模型并行代码（可以运行在多个显存至少8GB的GPUs上）。
+
 * **2022-09-30**: 我们开源了跨平台代码和模型权重，同时支持昇腾和英伟达平台。
 ## 使用指南
 
@@ -70,7 +72,15 @@ tar xvf codegeex_13b.tar.gz
 
 尝试使用CodeGeeX模型生成第一个程序吧！首先，在配置文件``configs/codegeex_13b.sh``中写明存放权重的路径。其次，将提示（可以是任意描述或代码片段）写入文件``tests/test_prompt.txt``，运行以下脚本即可开始推理（需指定GPU序号）：
 ```bash
+# On a single GPU (with more than 27GB RAM)
 bash ./scripts/test_inference.sh <GPU_ID> ./tests/test_prompt.txt
+
+# With quantization (with more than 15GB RAM)
+bash ./scripts/test_inference_quantized.sh <GPU_ID> ./tests/test_prompt.txt
+
+# On multiple GPUs (with more than 6GB RAM, need to first convert ckpt to MP_SIZE partitions)
+bash ./scripts/convert_ckpt_parallel.sh <LOAD_CKPT_PATH> <SAVE_CKPT_PATH> <MP_SIZE>
+bash ./scripts/test_inference_parallel.sh <MP_SIZE> ./tests/test_prompt.txt
 ```
 
 ### VS Code插件使用指南
@@ -173,4 +183,4 @@ HumanEval-X中每个语言的样本，包含了声明、描述和解答，它们
 
 ### 许可证
 
-代码使用[Apache-2.0许可证](LICENSE)
+代码使用[Apache-2.0许可证](LICENSE)

codegeex/paddle/__init__.py

@@ -0,0 +1 @@
+from .codegeex_model import CodeGeeXModel