Understanding Decision Trees

Learn the basics of decision trees in machine learning.

Calculate entropy:

import math

def entropy(p):
    return -p * math.log2(p) - (1-p) * math.log2(1-p)

Calculate information gain:

def information_gain(parent, children):
    total = sum(len(c) for c in children)
    return entropy(parent) - sum((len(c)/total)*entropy(c) for c in children)