GlassRoom

Public

8 repositories

torch_parallel_scan
Public
A simple implementation of parallel scan over sequences of tensors, with custom binary associative functions, for PyTorch (Franz A. Heinsen, 2024)
pytorch parallel-scan pscan parallel-scanning
Python
•
MIT License
•0•1•0•0•Updated Jan 20, 2025Jan 20, 2025
heinsen_sequence
Public
Code implementing "Efficient Parallelization of a Ubiquitious Sequential Computation" (Heinsen, 2023)
parallel-computing pytorch heinsen-sequence
MIT License
•4•89•0•0•Updated Dec 5, 2024Dec 5, 2024
heinsen_position_embeddings
Public
Reference implementation of "Encoding Position by Decaying and Updating Different Exponentiated States Differently," as proposed by Franz A. Heinsen in March 2024
Python
•
MIT License
•0•5•0•0•Updated Jun 17, 2024Jun 17, 2024
heinsen_attention
Public
Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
attention attention-mechanism attention-model linear-attention linear-attention-model heinsen-attention
Python
•
MIT License
•1•24•0•0•Updated Jun 6, 2024Jun 6, 2024
heinsen_routing
Public
Reference implementation of "An Algorithm for Routing Vectors in Sequences" (Heinsen, 2022) and "An Algorithm for Routing Capsules in All Domains" (Heinsen, 2019), for composing deep neural networks.
deep-learning paper pytorch routing-algorithm associative-memory capsule-network capsules em-routing vector-routing dense-associative-memory
Python
•
MIT License
•17•169•0•0•Updated Apr 13, 2023Apr 13, 2023
torch_train_test_loop
Public
Composable training/testing of PyTorch models with minimal overhead
Python
•
MIT License
•0•1•0•0•Updated Jan 24, 2023Jan 24, 2023
heinsen_tree
Public
Reference implementation of "Tree Methods for Hierarchical Classification in Parallel" (Heinsen, 2022) in PyTorch.
tree paper pytorch wordnet classification hierarchical hierarchical-classification heinsen-tree semantic-tree wordnet-classification
Python
•
MIT License
•0•14•0•0•Updated Dec 16, 2022Dec 16, 2022
torch_lsuv_init
Public
Layer Sequential Unit Variance (LSUV) Initialization for PyTorch
Python
•
MIT License
•1•2•0•0•Updated Aug 1, 2019Aug 1, 2019