Coding Linear Regression from Scratch

Coding Linear Regression from Scratch

ยท

5 min read


๐Ÿ‘‹ Hey there!

So, you're curious about Machine Learning? Awesome! Let's start with something cool and not too complex โ€“ Linear Regression. Trust me; it's not as intimidating as it sounds!

๐Ÿค– What is Linear Regression?

Linear Regression is like the superhero of Machine Learning โ€“ it's the good guy, the "Hello, World!" of the ML world. At its core, it's all about drawing a straight line through data points. That line helps us make predictions โ€“ think of it as the magic wand of predictions!

๐Ÿ“ˆ The Challenge

Here's the game: you have a bunch of data with features (X) and some target values (y). Your job is to find a line (an equation, really) that predicts y based on X.

๐Ÿ’ป Let's Get Our Hands Dirty

1. The Setup

Okay, let's roll up our sleeves and start coding! We need some tools โ€“ Python and a couple of libraries:

import numpy as np
import matplotlib.pyplot as plt

2. The Data

We'll create our dataset for this experiment :

X_train = np.array([[1], [2], [3], [4], [5]])  # 2D array
y_train = np.array([2, 4, 6, 8, 10])

3. Our Linear Regression Class

Time to dive into the code! Let's create our very own LinearRegression class. This is where the magic happens:

class LinearRegression:
    def __init__(self, lr=0.001, n_iters=1000):
        self.lr = lr
        self.n_iters = n_iters
        self.weights = None
        self.bias = None

Explanation: We're building a special toolbox here. Our LinearRegression class has some essential tools.

  • lr stands for the learning rate. It's like the "step size" our model takes when learning from data. You can think of it as how fast our model learns. ๐Ÿƒโ€โ™‚๏ธ

  • n_iters is the number of iterations. We can't learn everything in one go, so this tells our model how many times it needs to learn from the data. It's like saying, "Let's do this again and again until we get it right." ๐Ÿ”

  • weights and bias are the parameters our model will adjust while learning. Think of weights as the coefficients for each feature, and bias as the intercept. They're like the building blocks of our model. ๐Ÿงฑ

Now, let's move on and see how we make our model learn:

    def fit(self, X, y):
        n_samples, n_features = X.shape
        self.weights = np.zeros(n_features)  # As many zeros as the number of features.
        self.bias = 0

Explanation: In the fit method, we roll up our sleeves and get to work.

  • First, we find out how much data we have. n_samples tells us how many data points we've got, and n_features counts our data's characteristics.

  • We start with empty weights and a bias set to zero. This is like starting with a clean slate. We'll adjust these to make our predictions better.

  • Now, it's time to learn from the data. We do this by running through the data n_iters times. With each iteration, we get better at making predictions. It's like practicing a musical instrument. ๐ŸŽถ

Let's continue to the exciting part where our model gets smarter:

        for _ in range(self.n_iters):
            y_pred = np.dot(X, self.weights) + self.bias
            dw = (1/n_samples) * np.dot(X.T, (y_pred - y))
            db = (1/n_samples) * np.sum(y_pred - y)
            self.weights = self weights - self.lr * dw
            self.bias = self.bias - self.lr * db

Explanation: This is where our model learns and adapts. It's like the plot of a thrilling movie.

  • In each iteration, we make predictions (y_pred) based on our current weights and biases.

  • We figure out how wrong we are using the y_pred - y calculation. This difference tells us how much we need to adjust our parameters. It's like checking how off our guesses were. ๐Ÿคจ

  • With dw and db, we calculate how much to change our weights and bias to get closer to the correct predictions. We're fine-tuning the model. It's like adjusting the steering wheel to stay on the right path.

  • The self.weights and self.bias gets updated with smaller steps (controlled by self.lr, our learning rate) to reach better predictions. It's like taking baby steps to get better.

Our model learns by repeating this process, getting better and better with each iteration. It's like practicing a sport and improving with every game.

Now, let's see the model in action:

    def predict(self, X):
        y_pred = np.dot(X, self.weights) + self.bias
        return y_pred

Explanation: With the predict method, we're all set to make predictions.

  • We use our learned weights and bias to calculate y_pred. It's like applying what we've learned to new data points. It's like using the lessons we've learned to answer new questions. ๐Ÿ“š

  • Finally, we return y_pred, which are our model's predictions. This is where the magic happens. We feed new data, and our model tells us what it thinks the output should be. It's like a wise mentor providing answers. ๐ŸŽ“

In a nutshell, the LinearRegression class is the heart of our model. It learns from data, makes predictions, and gets better with practice. It's like teaching a robot to understand the world!


4. It's Training Time!

The machine needs to learn, right? So we teach it using our data:

reg = LinearRegression()
reg.fit(X_train, y_train)

5. Let's Predict, shall we?

Now that our model has learned, it's prediction time! We'll use it to predict values for new data points:

X_test = np.array([[6], [7], [8]])  # Test data
predictions = reg.predict(X_test)

6. See It with Your Own Eyes

Visualizing your results is the cherry on top. We'll use Matplotlib to do this.

plt.scatter(X_train, y_train, color='blue', label='Training Data')
plt.plot(X_test, predictions, color='red', label='Predictions')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.show()

๐Ÿ“ That's a Wrap!

There you go! You've just tackled Linear Regression in Machine Learning. This is where we start, folks โ€“ it's the foundation.


Did you find this article valuable?

Support Kanishk's Kaleidoscope by becoming a sponsor. Any amount is appreciated!

ย