[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71580":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":9,"rankLanguage":9,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":9,"pushedAt":9,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":15,"starSnapshotCount":15,"syncStatus":16,"lastSyncTime":29,"discoverSource":30},71580,"deeplearning-models","rasbt\u002Fdeeplearning-models","rasbt","A collection of various deep learning architectures, models, and tips",null,"Jupyter Notebook",17527,4106,591,8,0,2,7,29,6,45,"MIT License",false,"master",true,[],"2026-06-12 02:02:54","\n# Deep Learning Models\n\nA collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks.\n\n\n\n## Traditional Machine Learning\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| Perceptron | 2D toy data | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fbasic-ml\u002Fperceptron.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fbasic-ml\u002Fperceptron.ipynb) |\n| Logistic Regression | 2D toy data | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fbasic-ml\u002Flogistic-regression.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fbasic-ml\u002Flogistic-regression.ipynb)|\n| Softmax Regression (Multinomial Logistic Regression) | MNIST | TBD |  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fbasic-ml\u002Fsoftmax-regression.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fbasic-ml\u002Fsoftmax-regression.ipynb) |\n| Softmax Regression with MLxtend's plot_decision_regions on Iris | Iris | TBD |  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fbasic-ml\u002Fsoftmax-regression-mlxtend-1.ipynb) |\n\n\n## Multilayer Perceptrons\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| Multilayer Perceptron | MNIST | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fmlp\u002Fmlp-basic.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmlp\u002Fmlp-basic.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmlp\u002Fmlp-basic.ipynb) |\n| Multilayer Perceptron with Dropout | MNIST | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fmlp\u002Fmlp-dropout.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmlp\u002Fmlp-dropout.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmlp\u002Fmlp-dropout.ipynb) |\n|Multilayer Perceptron with Batch Normalization | MNIST | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fmlp\u002Fmlp-batchnorm.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmlp\u002Fmlp-batchnorm.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmlp\u002Fmlp-batchtnorm.ipynb) |\n|Multilayer Perceptron with Backpropagation from Scratch | MNIST | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmlp\u002Fmlp-fromscratch__sigmoid-mse.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmlp\u002Fmlp-fromscratch__sigmoid-mse.ipynb)|\n\n\n\n## Convolutional Neural Networks\n\n\n#### Basic\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| Convolutional Neural Network | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-basic.ipynb)  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-basic.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fcnn\u002Fcnn-basic.ipynb) |\n| CNN with He Initialization | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-he-init.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-he-init.ipynb)  |\n\n\n\n#### Concepts\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Replacing Fully-Connected by Equivalent Convolutional Layers | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Ffc-to-conv.ipynb)  |\n\n\n\n---\n\n\n#### AlexNet\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| AlexNet Trained on CIFAR-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-alexnet-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-alexnet-cifar10.ipynb) |\n| AlexNet with Grouped Convolutions Trained on CIFAR-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-alexnet-grouped-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-alexnet-grouped-cifar10.ipynb) |\n\n\n\n#### DenseNet\n\n|Title | Description | Daset | Notebooks |\n| --- | --- | --- | --- | \n| DenseNet-121 Digit Classifier Trained on MNIST | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-densenet121-mnist.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-densenet121-mnist.ipynb)  |\n| DenseNet-121 Image Classifier Trained on CIFAR-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-densenet121-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-densenet121-cifar10.ipynb)  |\n\n\n\n#### Fully Convolutional\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| \"All Convolutionl Net\" -- A Fully Convolutional Neural Network | TBD |  TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-allconv.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-allconv.ipynb) |\n\n#### LeNet\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| LeNet-5 on MNIST | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-lenet5-mnist.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-lenet5-mnist.ipynb)  |\n| LeNet-5 on CIFAR-10  | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-lenet5-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-lenet5-cifar10.ipynb)  |\n| LeNet-5 on QuickDraw | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-lenet5-quickdraw.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-lenet5-quickdraw.ipynb) |\n\n\n\n\n#### MobileNet\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| MobileNet-v2 on Cifar-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-mobilenet-v2-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-mobilenet-v2-cifar10.ipynb)  |\n| MobileNet-v3 small on Cifar-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-mobilenet-v3-small-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-mobilenet-v3-small-cifar10.ipynb)  |\n| MobileNet-v3 large on Cifar-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-mobilenet-v3-large-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-mobilenet-v3-large-cifar10.ipynb)   |\n| MobileNet-v3 large on MNIST via Embetter | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-embetter-mobilenet.ipynb)   |\n\n\n\n\n#### Network in Network\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Network in Network Trained on CIFAR-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-nin-cifar10.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fnin-cifar10.ipynb)  |\n\n\n#### VGG\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Convolutional Neural Network VGG-16 Trained on CIFAR-10 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-vgg16.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-vgg16.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fcnn\u002Fcnn-vgg16.ipynb) |\n| VGG-16 Smile Classifier | [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-vgg16-celeba.ipynb) [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-vgg16-celeba.ipynb)  |\n| VGG-16 Dogs vs Cats Classifier | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-vgg16-cats-dogs.ipynb)  |\n| Convolutional Neural Network VGG-19 | TBD | TBD | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fcnn\u002Fcnn-vgg19.ipynb)  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-vgg19.ipynb)  |\n\n\n\n\n#### ResNet\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| ResNet and Residual Blocks | [MNIST](http:\u002F\u002Fyann.lecun.com\u002Fexdb\u002Fmnist\u002F) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fresnet-ex-1.ipynb)  |\n| ResNet-18 Digit Classifier| [MNIST](http:\u002F\u002Fyann.lecun.com\u002Fexdb\u002Fmnist\u002F) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet18-mnist.ipynb)  |\n| ResNet-18 Gender Classifier | [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet18-celeba-dataparallel.ipynb)  |\n| ResNet-34 Digit Classifier | [MNIST](http:\u002F\u002Fyann.lecun.com\u002Fexdb\u002Fmnist\u002F) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet34-mnist.ipynb)  |\n| ResNet-34 Object Classifier | [QuickDraw](https:\u002F\u002Fquickdraw.withgoogle.com) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet34-quickdraw.ipynb)  |\n| ResNet-34 Gender Classifier| [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet34-celeba-dataparallel.ipynb)  |\n| ResNet-50 Digit Classifier| [MNIST](http:\u002F\u002Fyann.lecun.com\u002Fexdb\u002Fmnist\u002F) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet50-mnist.ipynb)  |\n| ResNet-50 Gender Classifier | [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet50-celeba-dataparallel.ipynb)  |\n| ResNet-101 Gender Classifier| [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet101-celeba.ipynb)  |\n| ResNet-101| [CIFAR-10](https:\u002F\u002Fwww.cs.toronto.edu\u002F~kriz\u002Fcifar.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet101-cifar10.ipynb)  |\n| ResNet-152 Gender Classifier| [CelebA](https:\u002F\u002Fmmlab.ie.cuhk.edu.hk\u002Fprojects\u002FCelebA.html) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet152-celeba.ipynb)  |\n\n\n---\n\n## Transformers\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Multilabel DistilBERT | [Jigsaw Toxic Comment Challenge](https:\u002F\u002Fwww.kaggle.com\u002Fcompetitions\u002Fjigsaw-toxic-comment-classification-challenge) | DistilBERT classifier fine-tuning | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftransformer\u002Fdistilbert-multilabel.ipynb)  |\n| DistilBERT as feature extractor | [IMDB movie review](https:\u002F\u002Fai.stanford.edu\u002F~amaas\u002Fdata\u002Fsentiment\u002F) | DistilBERT classifier with sklearn random forest and logistic regression | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftransformer\u002F1_distilbert-as-feature-extractor.ipynb)  |\n| DistilBERT as feature extractor using `embetter` | [IMDB movie review](https:\u002F\u002Fai.stanford.edu\u002F~amaas\u002Fdata\u002Fsentiment\u002F) | DistilBERT classifier with sklearn random forest and logistic regression using the scikit-learn `embetter` library | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftransformer\u002Fdistilbert-embetter-feature-extractor.ipynb)  |\n| Fine-tune DistilBERT I | [IMDB movie review](https:\u002F\u002Fai.stanford.edu\u002F~amaas\u002Fdata\u002Fsentiment\u002F) | Fine-tune only the last 2 layers of DistilBERT classifier |  [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Ftransformer\u002Fdistilbert-finetune-last-layers.ipynb) |\n| Fine-tune DistilBERT II | [IMDB movie review](https:\u002F\u002Fai.stanford.edu\u002F~amaas\u002Fdata\u002Fsentiment\u002F) | Fine-tune the whole DistilBERT classifier | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftransformer\u002Fdistilbert-hf-finetuning.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Ftransformer\u002Fdistilbert-finetuning-ii.ipynb)  |\n\n---\n\n## Ordinal Regression and Deep Learning\n\nPlease note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Baseline multilayer perceptron | Cement | A baseline multilayer perceptron for classification trained with the standard cross entropy loss | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002Fbaseline_cement.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002Fbaseline-light_cement.ipynb) |\n| CORAL multilayer perceptron | Cement | Implementation of [Rank Consistent Ordinal Regression for Neural Networks with Application to Age Estimation](https:\u002F\u002Fwww.sciencedirect.com\u002Fscience\u002Farticle\u002Fpii\u002FS016786552030413X) 2020 | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002FCORAL_cement.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002FCORAL-light_cement.ipynb) |\n| CORN multilayer perceptron | Cement | Implementation of [Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities](https:\u002F\u002Farxiv.org\u002Fabs\u002F2111.08851) 2022 | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002FCORN_cement.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002FCORN-light_cement.ipynb) |\n| Binary extension multilayer perceptron | Cement | Implementation of [Ordinal Regression with Multiple Output CNN for Age Estimation](https:\u002F\u002Fwww.cv-foundation.org\u002Fopenaccess\u002Fcontent_cvpr_2016\u002Fpapers\u002FNiu_Ordinal_Regression_With_CVPR_2016_paper.pdf) 2016 | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002Fniu2016_cement.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002Fniu2016-light_cement.ipynb) |\n| Reformulated squared-error multilayer perceptron | Cement | Implementation of [A simple squared-error reformulation for ordinal classification](https:\u002F\u002Farxiv.org\u002Fabs\u002F1612.00775) 2016 | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002Fbeckham2016_cement.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002Fbeckham2016-light_cement.ipynb) |\n| Class distance weighted cross-entropy loss | Cement | Implementation of [Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation](https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.05167) 2022 | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fordinal\u002Fpolat2022_cement.ipynb)  [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fordinal\u002Fpolat2022-light_cement.ipynb) |\n\n---\n\n\n\n## Normalization Layers\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fnin-cifar10_batchnorm.ipynb)  |\n| Filter Response Normalization for Network-in-Network CIFAR-10 Classifier | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fnin-cifar10_filter-response-norm.ipynb)  |\n\n\n\n## Metric Learning\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Siamese Network with Multilayer Perceptrons | TBD | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmetric\u002Fsiamese-1.ipynb) |\n\n## Autoencoders\n\n#### Fully-connected Autoencoders\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Autoencoder (MNIST) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-basic.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fautoencoder\u002Fae-basic.ipynb) |\n| Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-basic-with-rf.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fautoencoder\u002Fae-basic-with-rf.ipynb) |\n\n\n\n\n\n\n#### Convolutional Autoencoders\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- | \n| Convolutional Autoencoder with Deconvolutions \u002F Transposed Convolutions | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-deconv.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fautoencoder\u002Fae-deconv.ipynb) |\n| Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-deconv-jaccard.ipynb)  |\n| Convolutional Autoencoder with Deconvolutions (without pooling operations) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-deconv-nopool.ipynb) |\n| Convolutional Autoencoder with Nearest-neighbor Interpolation | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-conv-nneighbor.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fautoencoder\u002Fae-conv-nneighbor.ipynb) |\n| Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-conv-nneighbor-celeba.ipynb)  |\n| Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-conv-nneighbor-quickdraw-1.ipynb)  |\n\n\n\n#### Variational Autoencoders\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Variational Autoencoder | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-var.ipynb)  |\n| Convolutional Variational Autoencoder | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-conv-var.ipynb)  |\n\n\n\n#### Conditional Variational Autoencoders\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Conditional Variational Autoencoder (with labels in reconstruction loss) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-cvae.ipynb)  |\n| Conditional Variational Autoencoder (without labels in reconstruction loss) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-cvae_no-out-concat.ipynb)  |\n| Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-cnn-cvae.ipynb) |\n| Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fautoencoder\u002Fae-cnn-cvae_no-out-concat.ipynb)  |\n\n\n\n\n## Generative Adversarial Networks (GANs)\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Fully Connected GAN on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fgan.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fgan\u002Fgan.ipynb) |\n| Fully Connected Wasserstein GAN on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fwgan-1.ipynb)  |\n| Convolutional GAN on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fgan-conv.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fgan\u002Fgan-conv.ipynb) |\n| Convolutional GAN on MNIST with Label Smoothing | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fgan-conv-smoothing.ipynb) [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fgan\u002Fgan-conv-smoothing.ipynb) |\n| Convolutional Wasserstein GAN on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fdc-wgan-1.ipynb)  |\n| Deep Convolutional GAN (DCGAN) on Cats and Dogs Images | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fdcgan-cats-and-dogs.ipynb) |\n| Deep Convolutional GAN (DCGAN) on CelebA Face Images | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgan\u002Fdcgan-celeba.ipynb)  |\n\n\n## Graph Neural Networks (GNNs)\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Most Basic Graph Neural Network with Gaussian Filter on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgnn\u002Fgnn-basic-1.ipynb)  |\n| Basic Graph Neural Network with Edge Prediction on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgnn\u002Fgnn-basic-edge-1.ipynb)  |\n| Basic Graph Neural Network with Spectral Graph Convolution on MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fgnn\u002Fgnn-basic-graph-spectral-1.ipynb)  |\n\n\n\n## Recurrent Neural Networks (RNNs)\n\n\n\n\n#### Many-to-one: Sentiment Analysis \u002F Classification\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| A simple single-layer RNN (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_simple_imdb.ipynb)  |\n| A simple single-layer RNN with packed sequences to ignore padding characters (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_simple_packed_imdb.ipynb)  |\n| RNN with LSTM cells (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_lstm_packed_imdb.ipynb) |\n| RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_lstm_packed_imdb-glove.ipynb)  |\n| RNN with LSTM cells and Own Dataset in CSV Format (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_lstm_packed_own_csv_imdb.ipynb) |\n| RNN with GRU cells (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_gru_packed_imdb.ipynb) |\n| Multilayer bi-directional RNN (IMDB) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_lstm_bi_imdb.ipynb) |\n| Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_bi_multilayer_lstm_own_csv_agnews.ipynb)  |\n\n\n\n#### Many-to-Many \u002F Sequence-to-Sequence\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| A simple character RNN to generate new text (Charles Dickens) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Fchar_rnn-charlesdickens.ipynb) |\n\n\n\n## Model Evaluation\n\n### K-Fold Cross-Validation\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Baseline CNN  | MNIST | A simple baseline with traditional train\u002Fvalidation\u002Ftest splits | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fkfold\u002Fbaseline-cnn-mnist.ipynb) [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fkfold\u002Fbaseline-light-cnn-mnist.ipynb) |\n| K-fold with `pl_cross` | MNIST | A 5-fold cross-validation run using the `pl_cross` library | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](pytorch-lightning_ipynb\u002Fkfold\u002Fkfold-light-cnn-mnist.ipynb)  |\n\n\n\n## Data Augmentation\n\n| Title                      | Dataset | Description | Notebooks                                                    |\n| -------------------------- | ------- | ----------- | ------------------------------------------------------------ |\n| AutoAugment & TrivialAugment for Image Data | CIFAR-10     | Trains a ResNet-18 using AutoAugment and TrivialAugment         | [![PyTorch Lightning](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-Lightning-blueviolet)](.\u002Fpytorch-lightning_ipynb\u002Fdata-augmentation\u002Fautoaugment) |\n\n\n\n\n## Tips and Tricks\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Cyclical Learning Rate  | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftricks\u002Fcyclical-learning-rate.ipynb)  |\n| Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftricks\u002Fcnn-alexnet-cifar10-batchincrease.ipynb)  |\n| Gradient Clipping (w. MLP on MNIST)  | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftricks\u002Fgradclipping_mlp.ipynb)  |\n\n\n\n\n## Transfer Learning\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10)  | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Ftransfer\u002Ftransferlearning-vgg16-cifar10-1.ipynb)  |\n\n\n## Visualization and Interpretation\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Vanilla Loss Gradient (wrt Inputs) Visualization (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fviz\u002Fcnns\u002Fcats-and-dogs\u002Fcnn-viz-grad__vgg16-cats-dogs.ipynb)  |\n| Guided Backpropagation (Based on a VGG16 Convolutional Neural Network for Kaggle's Cats and Dogs Images) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fviz\u002Fcnns\u002Fcats-and-dogs\u002Fcnn-viz-guided-backprop__vgg16-cats-dogs.ipynb)  |\n\n\n\n## PyTorch Workflows and Mechanics\n\n\n#### PyTorch Lightning Examples\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| MLP in Lightning with TensorBoard  -- continue training the last model | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Flightning\u002Flightning-mlp.ipynb)  |\n| MLP in Lightning with TensorBoard  -- checkpointing best model | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Flightning\u002Flightning-mlp-best-model)  |\n\n\n\n\n#### Custom Datasets\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Custom Data Loader Example for PNG Files | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-dataloader-png\u002Fcustom-dataloader-example.ipynb)  |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- CSV files converted to HDF5 | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-csv.ipynb)  |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Face Images from CelebA | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-celeba.ipynb) |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from Quickdraw | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-quickdraw.ipynb) |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Drawings from the Street View House Number (SVHN) Dataset | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-svhn.ipynb) |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Asian Face Dataset (AFAD) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-afad.ipynb)  |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Dating Historical Color Images | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader_dating-historical-color-images.ipynb) |\n| Using PyTorch Dataset Loading Utilities for Custom Datasets -- Fashion MNIST | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcustom-data-loader-quickdraw.ipynb)  |\n\n\n\n#### Training and Preprocessing\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| PyTorch DataLoader State and Nested Iterations | Toy | Explains DataLoader behavior when in nested functions | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fdataloader-nesting.ipynb)|\n| Generating Validation Set Splits | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fvalidation-splits.ipynb)  |\n| Dataloading with Pinned Memory | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-resnet34-cifar10-pinmem.ipynb) |\n| Standardizing Images | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-standardized.ipynb) |\n| Image Transformation Examples | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Ftorchvision-transform-examples.ipynb) |\n| Char-RNN with Own Text File | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Fchar_rnn-charlesdickens.ipynb) |\n| Sentiment Classification RNN with Own CSV File | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frnn\u002Frnn_lstm_packed_own_csv_imdb.ipynb)  |\n\n\n\n#### Improving Memory Efficiency\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10) | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fgradient-checkpointing-nin.ipynb)  |\n\n#### Parallel Computing\n\n|Title | Description | Notebooks |\n| --- | --- | --- | \n| Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fcnn\u002Fcnn-vgg16-celeba-data-parallel.ipynb)  |\n| Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example) | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fmodel-pipeline-vgg16.ipynb)  |\n\n\n#### Other \n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| PyTorch with and without Deterministic Behavior -- Runtime Benchmark | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fdeterministic_benchmark.ipynb)  |\n| Sequential API and hooks | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fmlp-sequential.ipynb)  |\n| Weight Sharing Within a Layer | TBD | TBD |  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fcnn-weight-sharing.ipynb)  |\n| Plotting Live Training Performance in Jupyter Notebooks with just Matplotlib | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fplot-jupyter-matplotlib.ipynb)  |\n\n\n\n#### Autograd\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Getting Gradients of an Intermediate Variable in PyTorch | TBD | TBD | [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Fmechanics\u002Fmanual-gradients.ipynb)  |\n\n\n## TensorFlow Workflows and Mechanics\n\n#### Custom Datasets\n\n|Title | Description | Notebooks |\n| --- | --- | --- | \n| Chunking an Image Dataset for Minibatch Training using NumPy NPZ Archives | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Fimage-data-chunking-npz.ipynb) |\n| Storing an Image Dataset for Minibatch Training using HDF5 | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Fimage-data-chunking-hdf5.ipynb) |\n| Using Input Pipelines to Read Data from TFRecords Files | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Ftfrecords.ipynb) |\n| Using Queue Runners to Feed Images Directly from Disk  | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Ffile-queues.ipynb) |\n| Using TensorFlow's Dataset API | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Fdataset-api.ipynb) |\n\n\n\n\n#### Training and Preprocessing\n\n|Title | Dataset | Description | Notebooks |\n| --- | --- | --- | --- |\n| Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives | TBD | TBD | [![TensorFlow](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensor-Flow1.0-orange)](tensorflow1_ipynb\u002Fmechanics\u002Fsaving-and-reloading-models.ipynb) |\n\n## Related Libraries\n\n|Title | Description |  Notebooks |\n| --- | --- | --- | \n| TorchMetrics | How do we use it, and what's the difference between .update() and .forward()? |  [![PyTorch](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPy-Torch-red)](pytorch_ipynb\u002Frelated-libraries\u002Ftorchmetrics-update-forward.ipynb)  |\n\n","该项目是一个包含多种深度学习架构、模型及技巧的集合，主要使用Jupyter Notebook编写。它涵盖了从传统的机器学习算法如感知机、逻辑回归到多层感知器和卷积神经网络等深度学习模型，并提供了基于TensorFlow和PyTorch框架的实现版本。特别地，项目中还包含了使用PyTorch Lightning进行加速训练的示例。通过提供丰富的代码示例与实践指南，本项目非常适合希望深入理解不同深度学习技术及其在实际问题中应用的数据科学家、研究人员以及学生参考学习。","2026-06-11 03:38:38","high_star"]