Decision-Trees

Understanding Decision Trees

Learn the basics of decision trees in machine learning.

  1. Calculate entropy:
import math

def entropy(p):
    return -p * math.log2(p) - (1-p) * math.log2(1-p)
  1. Calculate information gain:
def information_gain(parent, children):
    total = sum(len(c) for c in children)
    return entropy(parent) - sum((len(c)/total)*entropy(c) for c in children)

Read more: Decision Trees