Pattern Recognition at TU München

accuracy = (TP + TN)/(TP +FP + TN +FN)

Specificity describes how reliably negative samples are labeled as such:

specificity = TN/(TN +FP)

Another way of looking at classifier performance is the predictive value of a label.
As the name implies, this metric describes the probability of a sample actually belonging to class X if it was classified as such:
PPV = Positive Predictive Value = TP/(TP +FP)
NPV = Negative Predictive Value = TN/(TN +FN)

F1 measure = 2 ×precision × recall/(precision + recall)

With e.g. k = 5, the data are split into 5 equal pieces. In the first fold, pieces 1–4 are used for training and piece 5 for testing; in the second fold, piece 4 is used for testing and 1–3, 5 for training; etc.
• every data point is tested exactly once
• but still expensive

We use it to generalize well on new data and to avoid overfitting on the available training data.

1. Random split: Randomly sample a certain proportion (e.g. 50%, 70%) of the data set as the training set, use the remainder for testing.
   • simple
   • potentially wasteful
2. Leave-one-out, leave-one-pair-out (LOO/LOPO): Training is performed on all data but one point (or one pair of 1 positive, 1 negative sample). This is repeated for every possible test set.
   • very expensive
   • maximizes available training data
3. Bootstrapping: Uses random splits with replacement.
   • allows estimation of bias and variance

Error due to model complexity is called the variance error. Error introduced due to some biases in the data is called bias error.

High bias means the data is being underfitted. The decision boundary is not usually complex enough. High variance happens due to overfitting, the decision boundary is more complex than what it should be.  

High bias high variance happens when you fit a complex decision boundary that is also not fitting the training set correctly in several places. 

• Disadvantages: Naive Bayes is based on the conditional independence of features assumption – an assumption that is not valid in many real-world scenarios. Hence it sometimes oversimplifies the problem by saying features are independent and gives a sub-par performance. There are chances of under-fitting due to this assumption. 
• Advantages: However, Naive Bayes is very efficient. It is a model you can train in a single iteration and hence fast to execute. It can be parallelized easily. Naive Bayes works when there are fewer data and lots of features, like bag of words with text data. Due to independence assumption, the number of parameters is less and constant w.r.t data (unlike other algorithms like decision trees). There are fewer chances of overfitting.

• Point outliers are single data points that lay far from the rest of the distribution. 
• Contextual outliers can be noise in data, such as punctuation symbols when realizing text analysis or background noise signal when doing speech recognition. 
• Collective outliers can be subsets of novelties in data such as a signal that may indicate the discovery of new phenomena.

• To build a tree, the algorithm randomly picks a feature from the feature space and a random split value ranging between the maximums and minimums. This is made for all the observations in the training set. 
• To build the forest, a tree ensemble is made averaging all the trees in the forest.
• Then for prediction, it compares an observation against that splitting value in a “node”, that node will have two node children on which another random comparison will be made. The number of “splittings” made by the algorithm for an instance is named: “path length”. 
• As expected, outliers will have shorter path lengths than the rest of the observations.

Naive Bayes is a supervised learning algorithm for classification so the task is to find the class of observation (data point) given the values of features. Naive Bayes classifier calculates the posterior probabilities using Bayes Theorem of it being a specific class when specific features appear. This classifier assumes the features (e.g. words as input) are independent, so the calculation of the class given the features is easier to compute.

It used for

• Real-time Prediction
• Text classification/ Spam Filtering
• Recommendation System