machine-learning

This is one of the continuously updated repositories that documents my own personal journey on learning data science related topics. Currently, contents are organized into two separate repositories based on the following table's description.

Repository	Documentation Focus
machine-learning	Machine learning, algorithm and programming in R / Python
Business-Analytics	All other data analytic related stuffs, e.g. concepts, statistics, articles, visualizations

Within each section, documentations are listed in reverse chronological order of the start date and each of them are independent of one another unless specified.

Documentation Listings

recsys : 2016.12.17

Recommendation System.

• Alternating Least Squares. [nbviewer]

trees : 2016.12.10

Tree-based models for both regression and classification tasks.

• Decision Tree from scratch. [nbviewer]
• Random Forest from scratch and Extra Trees. [nbviewer]
• Gradient Boosting from scratch. [nbviewer]
• Xgboost API walkthrough (includes hyperparmeter tuning via scikit-learn like API). [nbviewer]

association_rule : 2016.09.16

Also known as market-basket analysis.

• Apriori from scratch. [nbviewer]
• Using R's arules package (aprori) on tabular data. [R markdown]

clustering : 2016.08.16

TF-IDF and Topic Modeling are techniques specifically used for text analytics.

• TF-IDF (text frequency - inverse document frequency) from scratch. [nbviewer]
• K-means, K-means++ from scratch; Elbow method for choosing K. [nbviewer]
• Gaussian Mixture Model from scratch; AIC and BIC for choosing the number of Gaussians. [nbviewer]
• Topic Modeling with gensim's Latent Dirichlet Allocation(LDA). [nbviewer]

data_science_is_software : 2016.08.01

SciPy 2016: Data Science is Software. Best practices for doing data science (in Python).

• View [nbviewer]

deep_learning : 2016.07.23

Curated notes on deep learning. Tensorflow is used to implement some of the models.

• Softmax Regression from scratch. [nbviewer]
• Softmax Regression using Tensorflow (includes Tensorflow hello world). [nbviewer]
• Multi-layers Neural Network. [nbviewer]
• Convolutional Neural Network. [nbviewer]

keras : 2016.06.29

Walking through keras, a deep learning library. Note that this is only a API walkthrough, NOT a tutorial on the details of deep learning.

• Multi-layers Neural Network (keras basics). [nbviewer]
• Multi-layers Neural Network hyperparameter tuning via scikit-learn like API. [nbviewer]
• Convolutional Neural Network (image classification). [nbviewer]
• Convolutional Neural Network and Glove word embedding (text classification). [nbviewer]

text_classification : 2016.06.15

Naive Bayes and Logistic Regression for text classification.

• Building intuition with spam classification using scikit-learn. [nbviewer]
• Bernoulli and Multinomial Naive Bayes from scratch. [nbviewer]
• Logistic Regression (stochastic gradient descent) from scratch. [nbviewer]
• Chi-square feature selection. [nbviewer]

networkx : 2016.06.13

PyCon 2016: Practical Network Analysis Made Simple. Quickstart to networkx's API. Includes some basic graph plotting and algorithms.

• View [nbviewer]

regularization : 2016.05.25

Building intuition on Ridge and Lasso regularization using scikit-learn.

• View [nbviewer]

ga : 2016.04.25

Genetic Algorithm. Math-free explanation and code from scratch.

• Start from a simple optimization problem and extending it to traveling salesman problem (tsp).
• View [nbviewer]

h2o : 2016.01.24

Walking through H2O 2015 World Training GitBook.The walkthrough does basically zero feature engineering with the example dataset, as it is just browsing through its function calls and parameters. Apart from that, H2o Resources also contains booklets on each of the models.

• R's API:
  • h2o’s deep learning. [R markdown]
  • h2o’s Ensemble Tree. [R markdown]
  • h2o’s Generalized Linear Model. [R markdown]
  • h2o’s super learner. [R code]
• Python's API:
  • h2o's deep learning, Ensemble Tree. [nbviewer]

unbalanced : 2015.11.25

Choosing the optimal cutoff value for logistic regression using cost-sensitive mistakes (meaning when the cost of misclassification might differ between the two classes) when your dataset consists of unbalanced binary classes. e.g. Majority of the data points in the dataset have a positive outcome, while few have negative, or vice versa. The notion can be extended to any other classification algorithm that can predict class’s probability, this documentation just uses logistic regression for illustration purpose.

• Visualize two by two standard confusion matrix and ROC curve with costs using ggplot2.
• View [R markdown]

clustering_old

A collection of scattered old clustering documents in R.

• 2015.12.08 | Toy sample code of the LDA algorithm (gibbs sampling) and the topicmodels library. [R markdown]
• 2015.11.19 | k-shingle, Minhash and Locality Sensitive Hashing for solving the problem of finding textually similar documents. [R markdown]
• 2015.11.17 | Introducing tf-idf (term frequency-inverse document frequency), a text mining technique. Also uses it to perform text clustering via hierarchical clustering. [R markdown]
• 2015.11.06 | Some useful evaluations when working with hierarchical clustering and K-means clustering (K-means++ is used here). Including Calinski-Harabasz index for determine the right K (cluster number) for clustering and boostrap evaluation of the clustering result’s stability. [R markdown]

linear_regression : 2015.10.30

Training Linear Regression with gradient descent in R.

• Briefly covers the interpretation and visualization of linear regression's summary output.
• View [R markdown]

General Programming

Python

• 2016.12.26 | Walking through the free online tutorial, Problem Solving with Algorithms and Data Structures, that introduces basic data structure, algorithms from scratch. [folder]
• 2016.12.22 | Cython and Numba quickstart for high performance python. [nbviewer]
• 2016.06.22 | pandas's category type. [nbviewer]
• 2016.06.10 | unittest. [python code]
• 2016.04.26 | Some pre-implemented data structure and algorithm. [nbviewer]
• 2016.04.26 | Tricks with strings and text. [nbviewer]
• 2016.04.17 | python's decorators (useful script for logging and timing function). [nbviewer]
• 2016.03.18 | pandas's pivot table. [nbviewer]
• 2016.03.02 | @classmethod, @staticmethod and @property. [nbviewer]
• 2016.02.22 | sorting with itemgetter and attrgetter. [nbviewer]
• 2016.02.19 | for .. else .. statement. [nbviewer]
• 2016.02.18 | namedtuple and defaultdict. [nbviewer]

R

• 2016.04.15 | data.table joining and other tricks. [R markdown]