Machine Learning

Scikit Learn

• Splitting and scaling the data (Standard and MinMax scalers)

• Grid search and cross validation

• Pipelines, saving and loading models

• Supervised Learning

  • Linear regression, Ridge, Lasso, Elastic net. Polynomial features

  • Logistic regression

  • K Nearest Neighbors

  • Support Vector Machines (SVM)

  • Decision Trees, Random Forests, Boosting

  • Naive Bayes for features from text data

  • Regression and classification performance metrics

• Unsupervised Learning

  • K means clustering

  • Hierarchical clustering

  • DBSCAN

  • Principal Component Analysis (PCA)

Generic code:

from sklearn.model_family import ModelAlgo
mymodel = ModelAlgo(param1,param2)
mymodel.fit(X_train,y_train)
predictions = mymodel.predict(X_test)

from sklearn.metrics import error_metric
performance = error_metric(y_test,predictions)

Keras / Tensorflow

Choosing the activation function of the last layer

Last layer of network:

• Regression: no activation function with one neuron

• Binary classification: sigmoid with one neuron

• Multi-class classification: softmax with as many neurons as classes to predict

Choosing an optimizer and loss

Keep in mind what kind of problem you are trying to solve:

# For a multi-class classification problem
model.compile(optimizer='rmsprop',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

# For a binary classification problem
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# For a mean squared error regression problem
model.compile(optimizer='rmsprop',
              loss='mse')

• Feed-forward NN and saving model

• Convolutional NN (CNN) for Image Classification

• Recurring NN (RNN) for Time Series Forecasting

• Gated Recurrent Unit (GRU) for NLP

• Autoencoders

• Generative Adversarial Network (GAN) - 1D example

• Generative Adversarial Network (GAN) - Computer vision example

Important Libraries

• Numpy

• Pandas

• Matplotlib

• Seaborn

Other useful links

• Dealing with imbalanced datasets

• Selecting number of layers and nodes in feed-forward NN.

• Selecting hyperparameters for convolutional NN

• Activation functions

• (YOUTUBE) Data Analysis with Python Course - Numpy, Pandas, Data Visualization

• Universal Approximation Theorem