Machine-Learning

Understanding PCA for Dimensionality Reduction

Learn the basics of Principal Component Analysis (PCA).

Standardize data:

def standardize(data):
    mean = sum(data) / len(data)
    std = (sum((x-mean)**2 for x in data) / len(data)) ** 0.5
    return [(x-mean)/std for x in data]

Calculate covariance matrix:

def covariance_matrix(data):
    n = len(data[0])
    return [[sum(data[i][k]*data[j][k] for k in range(n)) for j in range(n)] for i in range(n)]

Implementing K-Nearest Neighbors

Learn how to implement the K-Nearest Neighbors algorithm.

Calculate Euclidean distance:

import math

def euclidean_distance(p1, p2):
    return math.sqrt(sum((x-y)**2 for x,y in zip(p1,p2)))

Find nearest neighbors:

def get_neighbors(train, test, k):
    distances = [(euclidean_distance(test, t), t) for t in train]
    distances.sort(key=lambda x: x[0])
    return [d[1] for d in distances[:k]]

Understanding Decision Trees

Learn the basics of decision trees in machine learning.

Calculate entropy:

import math

def entropy(p):
    return -p * math.log2(p) - (1-p) * math.log2(1-p)

Calculate information gain:

def information_gain(parent, children):
    total = sum(len(c) for c in children)
    return entropy(parent) - sum((len(c)/total)*entropy(c) for c in children)

Implementing Linear Regression

Learn how to implement linear regression using Python’s math library.

Calculate mean:

def mean(values):
    return sum(values) / float(len(values))

Calculate variance:

def variance(values, mean):
    return sum((x-mean)**2 for x in values)

Calculate covariance:

def covariance(x, mean_x, y, mean_y):
    return sum((x[i]-mean_x)*(y[i]-mean_y) for i in range(len(x)))

Getting Started with Neural Networks

Learn the basics of neural networks with this simple tutorial. We’ll create a basic feedforward neural network using Python’s built-in math capabilities.

Understand the basic structure of a neural network
Implement a simple neuron with sigmoid activation
Create a network layer
Train with basic gradient descent
Visualize the learning process

import math

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

class Neuron:
    def __init__(self, weights, bias):
        self.weights = weights
        self.bias = bias
    
    def feedforward(self, inputs):
        total = sum(w*i for w,i in zip(self.weights, inputs)) + self.bias
        return sigmoid(total)

class NeuralNetwork:
    def __init__(self):
        self.hidden_layer = [Neuron([0.5, -0.6], 0.1)]
        self.output_layer = [Neuron([0.1], -0.3)]
    
    def train(self, inputs, targets, learning_rate=0.1):
        # Training logic here
        pass

To better understand neural networks, let’s break down the key components: