Bringing BERT into modernity via both architecture changes and scaling

记录大模型相关的一些知识和方法

「大模型」3小时完全从0训练26M的小参数GPT，个人显卡即可推理训练！

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

📖 Full Stack Practice of the Large Language Model Training @ RLChina 2024

Transformers 库快速入门教程

The simplest, fastest repository for training/finetuning medium-sized GPTs.

从0到1构建一个MiniLLM (pretrain+sft+dpo实践中)

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Awesome-LLM: a curated list of Large Language Model

Explore the world of Large Language Models (LLMs) with this introductory repository. Learn about LLM architectures, pre-training, fine-tuning techniques, popular models, and their applications. Del…

总结梳理自然语言处理工程师(NLP)需要积累的各方面知识，包括面试题，各种基础知识，工程能力等等，提升核心竞争力

Llama中文社区，Llama3在线体验和微调模型已开放，实时汇总最新Llama3学习资料，已将所有代码更新适配Llama3，构建最好的中文Llama大模型，完全开源可商用

Natural Language Processing Tutorial for Deep Learning Researchers

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and…

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

此项目是机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现，也是作为一个算法工程师必会的理论基础知识。

NLP新手入门教程

Transformer论文Attention is All You Need的代码中文注释实现，翻译自harvardnlp/annotated-transformer

[CVPR'24] Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities

NLP上手教程

A comprehensive collection of PPG-related resources, including libraries, datasets, tutorials, papers, and more, for researchers and developers in the Photoplethysmography(HR/SpO2/BP) field.

An annotated implementation of the Transformer paper.

Code and written solutions of the assignments of the Stanford CS224N: Natural Language Processing with Deep Learning course from winter 2022/2023

Reading list for research topics in multimodal machine learning

Use ChatGPT to summarize the arXiv papers. 全流程加速科研，利用chatgpt进行论文全文总结+专业翻译+润色+审稿+审稿回复

Tutorials on using the MIMIC Waveform Database

关于domain generalization，domain adaptation，causality，robutness，prompt，optimization，generative model各式各样研究的阅读笔记