- 入门指南/经验类:
- 博客/博主:
- Lil’log: https://lilianweng.github.io/
- Sebastian Raschka博客(深度学习/大模型前沿知识&课程):https://magazine.sebastianraschka.com/
- **Jay Alammar博客:**https://jalammar.github.io/
- Andrej主页(一系列演讲和博客): https://karpathy.ai/
- 知乎苏剑林(前沿内容博主)
- 知乎张俊林
- 知乎时间旅客(LLM前沿知识解答)
- 小红书 小黄搞AI (每一篇都是一个面试问题)
- 公众号:
- AIGC小白入门记:各种公司面经
- 智荐阁:推荐系统论文速递
- B站博主:
- 数学家是我理想:主要方向有NLP,BERT-base模型
- Statquest:深度学习讲解
- Studio-Borolo:NLP,LLM
- 网课:
- 从深度学习入门到构建GPT (Andrej课程): https://karpathy.ai/zero-to-hero.html
- NLP:
- B站:NLP教程完全版(搬运youtube上Wang Shusen老师的NLP课程)(评价非常好,可以试着听一下)
- 生成模型相关:
- 何凯明深度生成模型课程
- Andrej开发的LLM101 course:
- 课程主页(暂时还没开课): https://github.com/karpathy/LLM101n
- 陆续更新的一系列模型底层实现代码: github EurekaLabsAI: https://github.com/EurekaLabsAI
- 面经:
- 公众号:AIGC小白入门记
- 其他:
- 带读AI经典论文:https://b23.tv/K3GbuVQ
- 关于深度学习训练:
- A recipe for training neural network: https://karpathy.github.io/2019/04/25/recipe/
- https://karpathy.github.io/2019/04/25/recipe/
- 如何加快模型训练:
- 多卡训练
- 单机多卡:
- 归一化/标准化:
- transformer
- 对比学习:
- B站李沐对比学习论文串讲
- B站up数学家是我理想的视频:contrastive learning
- diffusion model, stable diffusion:
- Diffusion model学习资源总结:https://github.com/diff-usion/Awesome-Diffusion-Models
- 李宏毅网课
- 知乎:https://zhuanlan.zhihu.com/p/563661713
- Lillog‘s blog: https://lilianweng.github.io/posts/2021-07-11-diffusion-models/
- 教程+代码实现:https://github.com/mikonvergence/DiffusionFastForward
- 代码实现:https://huggingface.co/blog/annotated-diffusion
- Score-based model:
- https://yang-song.net/blog/2021/score/
- diffusion model and score-based generative model: https://cbmm.mit.edu/video/diffusion-and-score-based-generative-models
- Chain of Thoughts:
- 李沐B站视频
- 大模型相关:
- 大模型入门tutorial:https://www.bilibili.com/video/BV1pF4m1V7FB/?spm_id_from=333.999.0.0&vd_source=c739db1ebdd361d47af5a0b8497417db
- awsome-LLM-resources (大模型学习资源汇总):https://github.com/WangRongsheng/awesome-LLM-resourses?tab=readme-ov-file#课程-Course
- 《大语言模型》书籍网站:大语言模型 | LLMBook-zh
- 《Hands-on LLM》:这本书是有结合代码的
- awsome-LLM-papers & project: https://github.com/Hannibal046/Awesome-LLM?tab=readme-ov-file
- 微软一系列大模型:https://github.com/microsoft/unilm
- 课程:
- 上交动手学大模型
- Andrej Karpathy课程网址(包含搭建GPT):https://karpathy.ai/zero-to-hero.html
- 阅读:知乎张俊林《通向AGI之路:大型语言模型技术精要》
- GPT2教程:https://zhuanlan.zhihu.com/p/79714797?utm_psn=1837110080388067329
- 综述: A survey of large language models, A survey of in-context learning, Explainability for large language models: A survey
- Code-LLM: https://github.com/huybery/Awesome-Code-LLM
- 大模型视频课程和资源汇总:https://zhuanlan.zhihu.com/p/654693985?utm_psn=1837107148598960130
- 大模型论文阅读清单&博客汇总:https://sebastianraschka.com/blog/2023/llm-reading-list.html
- Sebastian Raschka博客(深度学习/大模型前沿知识&课程):https://magazine.sebastianraschka.com/
- GPT相关:
- Illustrated GPT-2:The Illustrated GPT-2 (Visualizing Transformer Language Models) – Jay Alammar – Visualizing machine learning one concept at a time.
- andrej关于GPT的演讲: https://b23.tv/AwnzGnx
- 李沐GPT系列串讲:https://b23.tv/K7kqIKj
- 3b1b关于GPT的几个视频
- StatQuest关于GPT的几个视频
- andrej build GPT from scratch:
- stanford cs25 transformer united course:
- 从零训练1B以下小模型汇总:https://zhuanlan.zhihu.com/p/693252663?utm_psn=1837110354259361795
- 开源大模型:
- Meta OPT (Open Pretrained Transformer一系列decoder only Transformer):
- 马毅
- 论文整理/讲座整理:https://people.eecs.berkeley.edu/~yima/Talks.html
- 简约与自洽讲座:https://www.youtube.com/watch?v=LiFN_6i_oQA&t=1479s
- 相关论文:
-
Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction (2021)
-
ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction (2021)
-
On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence(2022)
-
unsupervised learning of structured representations via closed-loop transcription (2023)
-
- 神经科学与人工智能交叉(neuroAI)
- 综述文章:Data science opportunities of large language models for neuroscience and biomedcine
- 集智neuroAI读书会
- 深度学习网络的宽度和深度怎么理解:
- WANDB:实时los,acc可视化工具?
- labml:各种模型架构实现+逐行代码讲解