top of page

Machine Learning Algorithms As Part of Digital Analytics Deliverables

Machine learning algorithms take us beyond the simple generation of statistical and mathematical equations. It is a form of artificial intelligence where these equations which allow us to extract patterns out of data are representation of raw materials that we can turn to business rule engines.

Types of Machine Learning

Supervised Learning

The equation can be generated from supervised learning which imply the simple fitting of the data to generate approximation. Supervised learning involves prediction of category through classification, prediction of order through ranking and prediction of value through different types of Ordinary Least Squares Regression (OLS) and Generalized Least Squares (GLS) among others - See Below.

Unsupervised Learning

There are many situations whereby attempting to fit the data may not be the option, we attempt at this point to see if we can group the data into similar grouping to determine the factors that can help to group them. Unsupervised learning, more of ad hoc, learns data points which have no labels through algorithms and can be used to remove anomaly, reduce data dimensionality, group data based on similarity and estimate probability distribution.

Reinforcement Learning

Rewards and payoffs are the basis of running risk mitigation environments. The questions to ask include: Are there opportunities of reinforcing the factors that enhance the mitigation through ensuring the continuation of the reward? So if the past work, in a Markov process modeling, why not reinforce with reward to keep it going? What then becomes the type of reward and how best to implement the reward tend to combine the learning methods with strategic implementation which generate the structures and processes for sustaining the behavior

Artificial Neural Network Learning Algorithms

  • Feed-Forward Neural Network Models

  • Perceptrons

    • Single-Layer Perceptrons (SLP)

    • Multi-Layer Perceptrons (MLP)

  • Back-Propagation Algorithms

  • Hidden Neurons

  • Radial Basis Functions (RBF)

  • Multilayer Perceptron - Artificial Neural Networks (ANNs)

  • Hopfield Networks

  • Recurrent Neural Networks

  • Building Self-Organizing Maps (SOM)

  • Radial Basis Function Network (RBFN)

 

Decision Tree Learning Algorithms

  • ID3 (Iterative Dichotomiser 3 - Ross Quinlan)

  • C4.5 and C5.0 Algorithms - Successor of ID3

  • Classification and Regression Tree (CART)

  • Chi-Squared Automatic Interaction Detection (CHAID)

  • Conditional Decision Trees

  • Decision Stump

  • M5

  • MARS (Multivariate Adaptive Regression Splines) Algorithms

 

Bayesian Learning Algorithms

  • Naive Bayes Optimal Classifier

  • Gaussian Naive Bayes

  • Brute-Force Maximum a Posteriori (MAP) Hypotheses and Consistent Learners

  • Type-II Maximum Likelihood

  • Markov Chain Monte Carlo (MCMC)

  • Averaged One-Dependence Estimators (AODE)

  • Expectation Propagation (EP)

  • Laplace Approximations

  • Bayesian Belief Network (BBN)

  • Variational Approximations

  • Multinomial Naive Bayes

  • Bayesian Information Criterion (BIC)

  • Bayesian Networks (BN)

  • Exact Sampling

Regression Learning Algorithms

  • Ordinary Least Squares (OLS)

  • Linear Regression

  • Logistic Regression

  • Stepwise Regression

  • Generalized Least Squares (GLS)

    • Feasible GLS (FGLS) or Estimated GLS (EGLS)

    • Full Estimated GLS (FEGLS)

  • Multivariate Adaptive Regression Splines (MARS)

  • Locally Estimated Scatterplot Smoothing (LOESS)

  • Ordinal

  • Poisson

  • Negative Binomial

  • Log Gamma

  • Quantile

  • Decision Forest

  • Linear Probability Model (LPM)

  • Random Features Regularized Least Squares (RFRLS)

  • Locally Weighted Regression algorithm (LWR)

  • Locally Weighted Projection Regression algorithm (LWPR)

     

Nonlinear Regression Learning Algorithms (NLRM)

  • Log-Linear Models

  • Gaussian Mixture Regression (GMR)

Support Vector Machines (SVMs) Regression

  • Linear SVMs Regression

  • Binary and Multiclass Classification

  • Nonlinear SVMs Regression

  • SVMs Regression Optimization

     

Support Vector Machines (SVMs)

  • Support Vector Machine (SVMs) Solvers Algorithm

  • Support Vector Machine Classification

  • Large-Scale Learning with Kernels

  • Sequential Solver for Large-Scale SVMs

Dimensionality Reduction Algorithms

  • Multi-Dimensional Scaling (MDS)

  • Factor Analysis

  • Multivariate Statistical Analysis

    • Principal Component Analysis (PCA)

    • Rotated PCA

    • PCA for Vectors

    • Canonical Correlation Analysis (CCA)

  • Nonnegative Matrix Factorization (NMF)

  • Procrustes Function Analysis

  • Discriminant Analysis

    • Linear

    • Mixture 

    • Quadratic 

    • Flexible

       

Non-Linear Optimization

  • Gradient Descent Methods

  • Conjugate Gradient Methods

  • Quasi-Newton Methods

  • Nonlinear Least Squares Methods

  • Evolutionary Computation and Genetic Algorithms

Kernel Methods

  • Primal and Dual Solutions for Linear Regression

  • Kernel Ridge Regression

     

Non-Linear Classification

  • Multi-Layer Perceptron Classifier

  • Multi-Class Classification

  • Bayesian Neural Network (BNN) Classifier

  • Support Vector Machine (SVM) Classifier

  • Forecast Verification

Clustering Algorithms

Clustering is mainly an unsupervised classification learning technique which identifies inherent commonality in structures which can be found in a set of objects which show similarity measure. Some of the use include vector quantization features extraction, function approximation, pattern recognition, data mining and image segmentation.

Some of the common approaches include:

  • Partitioning Algorithms

    • K-means Algorithms

    • K-medoids Algorithms

    • Density-Based Algorithms

      • Density Functions Clustering

      • Density-Based Connectivity Clustering

    • Probabilistic Algorithms

  • Grid-Based Algorithms

  • Evolutionary Algorithms

  • Constraint-Based Clustering

  • Co-Occurrence of Categorical Data

  • Hierarchical Clustering Algorithms

  • Agglomerative Algorithms

    • Divisive Algorithms

    • Algorithms for High Dimensional Data

  • Scalable Clustering Algorithms

  • Expectation–Maximization (EM) Clustering

  • Support Vector Clustering (SVC)

     

Instance-Based Learning (IBL) Algorithms

  • k-Nearest Neighbour (kNN)

    • k-NN Classification

    • k-NN Regression

  • IB1

  • IB2

  • IB3

  • Learning Vector Quantization (LVQ)

  • Radial Basis Function Network (RBFN) 

  • Locally Weighted Learning (LWL)

  • Kernel Methods – Some of which include:

    • Polynomial kernel

    • Graph Kernel

    • Fisher Kernel

Control Chart Pattern Recognition Algorithms

  • Learning Vector Quantization (LVQ) Network

  • Multi-Layer Perceptrons (MLP)

  • Generalized Learning Vector Quantization

 

Association Rules Learning Algorithms

  • Apriori Algorithms

  • Equivalence Class Transformation (ECLAT)

  • Frequent Pattern Growth Algorithms (FP-growth)

  • Node-Set-Based Algorithms

  • OPUS Search

  • Multi-Relation Association Rules (MRAR)

  • Context Based Association Rules

  • Generalized Association Rules

  • Sequential Pattern

  • Generalized Association Rules

  • K-optimal Pattern

Regularization Learning Algorithms

  • Least-Squares Methods

  • Lasso or Ridge Regression

  • Polynomial Regression   

  • Kernel Regression 

  • Conditional Random Fields

  • Matrix Decomposition

     

Ensemble Learning Algorithms

  • Bayes Optimal Classifier

  • Bootstrap Aggregating (Bagging)

  • Random Forest

  • AdaBoost

  • Bayesian Parameter Averaging (BPA)

  • Gradient Boosting Machines (GBM)

  • Bayesian Model Combination (BMC)

  • Boosting

  • Randomized Trees

  • Random Subspace

Using Python

  • Clustering

  • Latent Dirichlet Allocation (LDA)

  • Classification

    • Using Logistic Regression

  • Sentiment Analysis

  • Regression

    • Penalized Regression

    • Basket Analysis

  • Computer Vision – Pattern Recognition

bottom of page