Key Algorithms

These articles represent what I think of as the things every data scientist should know. Each gives a short summary of an algorithm or group of related algorithms. While the presentation is consise, I do go into enough mathematical detail to properly understand the algorithm. Wherever possible I have discussed my own experience of using them, and I also highlight the connections

They were originally written as a series of blog posts between November 2023 and July 2024, but in porting them to this site I have decided to organise them thematically.

Measurements

• Similarity and Difference Metrics
• TF-IDF
• Data Reduction
• Latent Semantic Indexing
• Information Theory

Model Evaluation

• Evaluation Metrics for Classifiers
• Evaluation Metrics for Regression
• Cross Validation

Bayesian Models

• Bayes’ Theorem
• Hidden Markov Models
• The Viterbi Algorithm
• Markov Chain Monte Carlo

Simple Supervised Learning Models

• Logistic Regression
• Linear Regression
• Random Forests

Missing and Anomalous Data

• Outlier Detection
• Imputation

Unsupervised Learning

• Hierarchical Clustering
• K-Means Clustering
• Expectation Maximisation

Collective Intelligence

• Collaborative Filtering
• PageRank

Search and Navigation

• Vector Search Trees
• Priority Queues
• Graph Search Algorithms

Components of Neural Networks

• The Chain Rule and Backpropogation
• Activation Functions
• Loss Functions
• Gradient Descent
• Transfer Learning
• Tokenizers

Neural Network Architectures

• Multilayer Perceptron
• Convolutional Networks
• Recurrent Neural Networks
• The Attention Mechanism
• Transformers

by Dr Peter J Bleackley