Class Metrics

A diverse set of metrics to evaluate various kinds of predictors.

Inheritance

Object

Metrics

Inherited Members

Object.Equals(Object)

Object.Equals(Object, Object)

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ReferenceEquals(Object, Object)

Object.ToString()

Namespace: Microsoft.ML.Probabilistic.Learners

Assembly: Microsoft.ML.Probabilistic.Learners.dll

Syntax

public static class Metrics

Fields

LinearDiscountFunc

Linear position discount function.

Declaration

public static readonly Func<int, double> LinearDiscountFunc

Field Value

Type	Description
Func<Int32, Double>

LogarithmicDiscountFunc

Logarithmic position discount function.

Declaration

public static readonly Func<int, double> LogarithmicDiscountFunc

Field Value

Type	Description
Func<Int32, Double>

Methods

AbsoluteError(Boolean, Boolean)

Computes the absolute difference between a prediction and the ground truth.

Declaration

public static double AbsoluteError(bool groundTruth, bool prediction)

Parameters

Type	Name	Description
Boolean	groundTruth	The ground truth.
Boolean	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

AbsoluteError(Double, Double)

Computes the absolute difference between a prediction and the ground truth.

Declaration

public static double AbsoluteError(double groundTruth, double prediction)

Parameters

Type	Name	Description
Double	groundTruth	The ground truth.
Double	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

AbsoluteError(Double, Int32)

Computes the absolute difference between a prediction and the ground truth.

Declaration

public static double AbsoluteError(double groundTruth, int prediction)

Parameters

Type	Name	Description
Double	groundTruth	The ground truth.
Int32	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

AbsoluteError(Int32, Int32)

Computes the absolute difference between a prediction and the ground truth.

Declaration

public static double AbsoluteError(int groundTruth, int prediction)

Parameters

Type	Name	Description
Int32	groundTruth	The ground truth.
Int32	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

AreaUnderRocCurve<TInstance>(IEnumerable<TInstance>, IEnumerable<KeyValuePair<TInstance, Double>>)

Computes the area under the receiver operating characteristic curve.

The implementation follows Algorithm 3 as described in Fawcett, T. (2004): ROC Graphs: Notes and Practical Considerations for Researchers.

Declaration

public static double AreaUnderRocCurve<TInstance>(IEnumerable<TInstance> positiveInstances, IEnumerable<KeyValuePair<TInstance, double>> instanceScores)

Parameters

Type	Name	Description
IEnumerable<TInstance>	positiveInstances	The instances with 'positive' ground truth labels.
IEnumerable<KeyValuePair<TInstance, Double>>	instanceScores	The predicted instance scores. The larger a predicted score, the more likely the instance is to belong to the 'positive' class.

Returns

Type	Description
Double	AUC - the area under the receiver operating characteristic curve.

Type Parameters

Name	Description
TInstance	The type of an instance.

Remarks

All instances not contained in positiveInstances are assumed to belong to the 'negative' class.

CosineSimilarity(Vector, Vector)

Computes cosine similarity between two given vectors.

Declaration

public static double CosineSimilarity(Vector vector1, Vector vector2)

Parameters

Type	Name	Description
Vector	vector1	The first vector.
Vector	vector2	The second vector.

Returns

Type	Description
Double	The computed similarity value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if the similarity is not defined for the given pair of vectors. It can happen if the vectors are of different length or one of them has zero magnitude.

Dcg(IEnumerable<Double>)

Computes discounted cumulative gain for the given list of gains using LogarithmicDiscountFunc discount function.

Declaration

public static double Dcg(IEnumerable<double> orderedGains)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.

Dcg(IEnumerable<Double>, Func<Int32, Double>)

Computes discounted cumulative gain for the given list of gains.

Declaration

public static double Dcg(IEnumerable<double> orderedGains, Func<int, double> discountFunc)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.
Func<Int32, Double>	discountFunc	Position discount function.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.

GradedAveragePrecision(IEnumerable<Double>)

Computes graded average precision for the given list of relevance values.

Declaration

public static double GradedAveragePrecision(IEnumerable<double> orderedRelevances)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedRelevances	List of relevance values ordered according to some external criteria.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.

LinearDcg(IEnumerable<Double>)

Computes discounted cumulative gain for the given list of gains using LinearDiscountFunc discount function.

Declaration

public static double LinearDcg(IEnumerable<double> orderedGains)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.

LinearNdcg(IEnumerable<Double>, IEnumerable<Double>)

Computes discounted cumulative gain for the given list of gains normalized given another list of gains using LinearDiscountFunc.

Declaration

public static double LinearNdcg(IEnumerable<double> orderedGains, IEnumerable<double> bestOrderedGains)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.
IEnumerable<Double>	bestOrderedGains	List of gains used to compute a normalizer for discounted cumulative gain.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if orderedGains and bestOrderedGains are of different size.

Ndcg(IEnumerable<Double>, IEnumerable<Double>)

Computes discounted cumulative gain for the given list of gains normalized given another list of gains using LogarithmicDiscountFunc.

Declaration

public static double Ndcg(IEnumerable<double> orderedGains, IEnumerable<double> bestOrderedGains)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.
IEnumerable<Double>	bestOrderedGains	List of gains used to compute a normalizer for discounted cumulative gain.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if orderedGains and bestOrderedGains are of different size.

Ndcg(IEnumerable<Double>, IEnumerable<Double>, Func<Int32, Double>)

Computes discounted cumulative gain for the given list of gains normalized given another list of gains.

Declaration

public static double Ndcg(IEnumerable<double> orderedGains, IEnumerable<double> bestOrderedGains, Func<int, double> discountFunc)

Parameters

Type	Name	Description
IEnumerable<Double>	orderedGains	List of gains ordered according to some external criteria.
IEnumerable<Double>	bestOrderedGains	List of gains used to compute a normalizer for discounted cumulative gain.
Func<Int32, Double>	discountFunc	Position discount function.

Returns

Type	Description
Double	The computed metric value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if orderedGains and bestOrderedGains are of different size.

NegativeLogProbability(Boolean, Bernoulli)

Returns the negative natural logarithm of the probability of ground truth for a given predictive distribution.

Declaration

public static double NegativeLogProbability(bool groundTruth, Bernoulli prediction)

Parameters

Type	Name	Description
Boolean	groundTruth	The ground truth.
Bernoulli	prediction	The prediction as a Discrete distribution.

Returns

Type	Description
Double	The negative natural logarithm of the probability of groundTruth.

NegativeLogProbability(Int32, Discrete)

Returns the negative natural logarithm of the probability of ground truth for a given predictive distribution.

Declaration

public static double NegativeLogProbability(int groundTruth, Discrete prediction)

Parameters

Type	Name	Description
Int32	groundTruth	The ground truth.
Discrete	prediction	The prediction as a Discrete distribution.

Returns

Type	Description
Double	The negative natural logarithm of the probability of groundTruth.

NegativeLogProbability<TLabel>(TLabel, IDictionary<TLabel, Double>)

Returns the negative natural logarithm of the probability of ground truth for a given predictive distribution.

Declaration

public static double NegativeLogProbability<TLabel>(TLabel groundTruth, IDictionary<TLabel, double> prediction)

Parameters

Type	Name	Description
TLabel	groundTruth	The ground truth.
IDictionary<TLabel, Double>	prediction	The prediction as a discrete distribution over labels.

Returns

Type	Description
Double	The negative natural logarithm of the probability of groundTruth.

Type Parameters

Name	Description
TLabel	The type of a label.

NormalizedEuclideanSimilarity(Vector, Vector)

Computes the similarity measure for a given pair of vectors based on the Euclidean distance between them. The similarity is computed as 1.0 / (1.0 + D(vector1, vector2), where D stands for the Euclidean distance divided by the square root of the dimensionality.

Declaration

public static double NormalizedEuclideanSimilarity(Vector vector1, Vector vector2)

Parameters

Type	Name	Description
Vector	vector1	The first vector.
Vector	vector2	The second vector.

Returns

Type	Description
Double	The computed similarity value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if the distance is not defined for the given pair of vectors. It can happen if the vectors are of different or zero length.

NormalizedManhattanSimilarity(Vector, Vector)

Computes the similarity measure for a given pair of vectors based on Manhattan distance between them. The similarity is computed as 1.0 / (1.0 + D(vector1, vector2), where D stands for the Manhattan distance divided by the dimensionality.

Declaration

public static double NormalizedManhattanSimilarity(Vector vector1, Vector vector2)

Parameters

Type	Name	Description
Vector	vector1	The first vector.
Vector	vector2	The second vector.

Returns

Type	Description
Double	The computed similarity value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if the distance is not defined for the given pair of vectors. It can happen if the vectors are of different or zero length.

PearsonCorrelation(Vector, Vector)

Computes Pearson's correlation coefficient for a given pair of vectors.

Declaration

public static double PearsonCorrelation(Vector vector1, Vector vector2)

Parameters

Type	Name	Description
Vector	vector1	The first vector.
Vector	vector2	The second vector.

Returns

Type	Description
Double	The computed correlation value.

Exceptions

Type	Condition
ArgumentNullException	Thrown if one of the arguments is null.
ArgumentException	Thrown if the correlation is not defined for the given pair of vectors. It can happen if the vectors are of different length or one of them has zero variance.

PrecisionRecallCurve<TInstance>(IEnumerable<TInstance>, IEnumerable<KeyValuePair<TInstance, Double>>)

Computes the precision-recall curve.

Declaration

public static IEnumerable<PrecisionRecall> PrecisionRecallCurve<TInstance>(IEnumerable<TInstance> positiveInstances, IEnumerable<KeyValuePair<TInstance, double>> instanceScores)

Parameters

Type	Name	Description
IEnumerable<TInstance>	positiveInstances	The instances with 'positive' ground truth labels.
IEnumerable<KeyValuePair<TInstance, Double>>	instanceScores	The predicted instance scores. The larger a predicted score, the more likely the instance is to belong to the 'positive' class.

Returns

Type	Description
IEnumerable<PrecisionRecall>	The points on the precision-recall curve, increasing by recall.

Type Parameters

Name	Description
TInstance	The type of an instance.

Remarks

All instances not contained in positiveInstances are assumed to belong to the 'negative' class.

ReceiverOperatingCharacteristicCurve<TInstance>(IEnumerable<TInstance>, IEnumerable<KeyValuePair<TInstance, Double>>)

Computes the receiver operating characteristic curve.

The implementation follows Algorithm 2 as described in Fawcett, T. (2004): ROC Graphs: Notes and Practical Considerations for Researchers.

Declaration

public static IEnumerable<FalseAndTruePositiveRate> ReceiverOperatingCharacteristicCurve<TInstance>(IEnumerable<TInstance> positiveInstances, IEnumerable<KeyValuePair<TInstance, double>> instanceScores)

Parameters

Type	Name	Description
IEnumerable<TInstance>	positiveInstances	The instances with 'positive' ground truth labels.
IEnumerable<KeyValuePair<TInstance, Double>>	instanceScores	The predicted instance scores. The larger a predicted score, the more likely the instance is to belong to the 'positive' class.

Returns

Type	Description
IEnumerable<FalseAndTruePositiveRate>	The points on the receiver operating characteristic curve, increasing by false positive rate.

Type Parameters

Name	Description
TInstance	The type of an instance.

Remarks

All instances not contained in positiveInstances are assumed to belong to the 'negative' class.

SquaredError(Boolean, Boolean)

Computes squared difference between a prediction and the ground truth.

Declaration

public static double SquaredError(bool groundTruth, bool prediction)

Parameters

Type	Name	Description
Boolean	groundTruth	The ground truth.
Boolean	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

SquaredError(Double, Double)

Computes the squared difference between a prediction and the ground truth.

Declaration

public static double SquaredError(double groundTruth, double prediction)

Parameters

Type	Name	Description
Double	groundTruth	The ground truth.
Double	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

SquaredError(Double, Int32)

Computes squared difference between a prediction and the ground truth.

Declaration

public static double SquaredError(double groundTruth, int prediction)

Parameters

Type	Name	Description
Double	groundTruth	The ground truth.
Int32	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

SquaredError(Int32, Int32)

Computes squared difference between a prediction and the ground truth.

Declaration

public static double SquaredError(int groundTruth, int prediction)

Parameters

Type	Name	Description
Int32	groundTruth	The ground truth.
Int32	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

ZeroOneError(Int32, Int32)

Returns 0 if a prediction and the ground truth are the same and 1 otherwise.

Declaration

public static double ZeroOneError(int groundTruth, int prediction)

Parameters

Type	Name	Description
Int32	groundTruth	The ground truth.
Int32	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

ZeroOneError<TLabel>(TLabel, TLabel)

Returns 0 if a prediction and the ground truth are the same and 1 otherwise.

Declaration

public static double ZeroOneError<TLabel>(TLabel groundTruth, TLabel prediction)

Parameters

Type	Name	Description
TLabel	groundTruth	The ground truth.
TLabel	prediction	The prediction.

Returns

Type	Description
Double	The computed metric value.

Type Parameters

Name	Description
TLabel	The type of a label.