Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Implementing Multiple Layer Neural Network from Scratch
| Date | Stars |
|---|---|
| 2026-07-31 | 329 |
| 2026-08-03 | 329 |
| 2026-08-04 | 329 |
| 2026-08-06 | 329 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Implementing Multiple Layer Neural Network from Scratch This post is inspired by <http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch>. In this post, we will implement a multiple layer neural network from scratch. You can regard the number of layers and dimension of each layer as parameter. For example, `[2, 3, 2]` represents inputs with 2 dimension, one hidden layer with 3 dimension and output with 2 dimension (binary classification) (using softmax as output). We won’t derive all the math that’s required, but I will try to give an intuitive explanation of what we are doing. I will also point to resources for you read up on the details. ## Generating a dataset Let’s start by generating a dataset we can play with. Fortunately, [scikit-learn](http://scikit-learn.org/) has some useful dataset generators, so we don’t need to write the code ourselves. We will go with the [make_moons](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html) function. ```python # Generate a dataset and plot it np.random.seed(0) X, y = sklearn.datasets.make_moons(200, noise=0.20) plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.Spectral) ```  The dataset we generated has two classes, plotted as red and blue points. Our goal is to train a Machine Learning classifier that predicts the correct class given the x- and y- coordinates. Note that the data is not linearly separable, we can’t draw a straight line that separates the two classes. This means that linear classifiers, such as Logistic Regression, won’t be able to fit the data unless you hand-engineer non-linear features (such as polynomials) that work well for the given dataset. In fact, that’s one of the major advantages of Neural Networks. You don’t need to worry about feature engineering. The hidden layer of a neural network will learn features for you. ## Neural Network ### Neural Network Architecture You can read this tutorial (<http://cs231n.github.io/neural-networks-1/>) to learn the basic concepts of neural network. Like activation functions, feed-forward computation and so on. Because we want our network to output probabilities the activation function for the output layer will be the [softmax](https://en.wikipedia.org/wiki/Softmax_function), which is simply a way to convert raw scores to probabilities. If you’re familiar with the logistic function you can think of softmax as its generalization to multiple classes. When you choose softmax as output, you can use [cross-entropy loss](https://en.wikipedia.org/wiki/Cross_entropy#Cross-entropy_error_function_and_logistic_regression) (also known as negative log likelihood) as loss function. More about Loss Function can be find in <http://cs231n.github.io/neural-networks-2/#losses>. ### Learning the Parameters Learning the parameters for our network means finding parameters (such as (W_1, b_1, W_2, b_2)) that minimize the error on our training data (loss function). We can use [gradient descent](http://cs231n.github.io/optimization-1/) to find the minimum and I will implement the most vanilla version of gradient descent, also called batch gradient descent with a fixed learning rate. Variations such as SGD (stochastic gradient descent) or minibatch gradient descent typically perform better in practice. So if you are serious you’ll want to use one of these, and ideally you would also [decay the learning rate over time](http://cs231n.github.io/neural-networks-3/#anneal). The key of gradient descent method is how to calculate the gradient of loss function by the parameters. One approach is called [Back Propagation](https://en.wikipedia.org/wiki/Backpropagation). You can learn it more from <http://colah.github.io/posts/2015-08-Backprop/> and <http://cs231n.github.io/optimization-2/>. ### Implementation We start by given the computation graph of neural network. ![](https://github.com/pan
Excerpt of 12,566 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:95b1b649346eac28, topic:neural-network