A’ is the probability that if you give a model an example from each category, it will accurately identify which is which. It is the percentage of time your model can differentiate between the negative and positive.
Given here is a small example of how to calculate AUC of a model without actually drawing the curve. The example given here has been done manually and is only for illustration purpose. AUC can be automatically computed using most software packages.
Let us say we have an output from a model as depicted here:
On summarizing the positive and negatives in Column A (Data) we get the following :
Y = 12; N = 13
Where Y = Total number of positives and N = Total Number of negatives.
AUC is given by the following formula. Here S j is the rank of the jth negative element in sequence and Sj – j is the number of positives before that jth negative element.
Steps for calculation of A’ are given below:
Step 1: Sort the dataset in decreasing order of magnitude on column B.
Step 2: For every negative entry in the sorted data set computer the total number of positive entries above it as shown below:
Step 3: calculate the sum of all the items calculated in step above. It comes out to be 144 in this example.
Step 4: Total Number of Positives, Y = 12 and Total Number of negatives, N = 13 as shown in the data summary.
Step 5: Compute AUC as follows: AUC = (Sum of all items in Step 3) / Y * N = 144/ 12* 13 = 0.92
