This is the homepage of a new book entitled "Mathematical Foundations of Reinforcement Learning."

LLM training code for Databricks foundation models

When it comes to optimizers, it's always better to be safe than sorry

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643

functorch is JAX-like composable function transforms for PyTorch.

LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters

中国科学院大学2019-2020课程（秋季，春季，夏季）

Code for NeurIPS 2019 paper: "Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes"

[NeurIPS 2024 Spotlight] Official repository of the CycleNet paper: "CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns". This work is developed by the Lab of Professor …

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

LaTeX Thesis Template for the University of Chinese Academy of Sciences

Chinese GPT2: pre-training and fine-tuning framework for text generation

Fine-tuned pre-trained GPT2 for custom topic specific text generation. Such system can be used for Text Augmentation.

Understanding Training Dynamics of Deep ReLU Networks

A Library for Advanced Deep Time Series Models.

The official code for "One Fits All: Power General Time Series Analysis by Pretrained LM (NeurIPS 2023 Spotlight)"

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Code and documentation to train Stanford's Alpaca models, and generate the data.

LDAdam - Adaptive Optimization from Low-Dimensional Gradient Statistics

Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)

Video+code lecture on building nanoGPT from scratch

Code for the paper "Language Models are Unsupervised Multitask Learners"

EasyLiterature is an open-sourced, Python-based command line tool for automatic literature management.

Deep Residual Learning in Spiking Neural Networks