Running Deeper C++/C/Java

Using the MNIST data set, I will be trying to detect numbers based on images, and get the accuracy as high as possible.

The first step: the normal Flattened Neural Network. This method is much faster, but also less accurate, since the image's 2D properties turn into 1D, and lose the 2D shape. This causes a drastic loss in accuracy. 

The neural net I will be using is a Multi Layer Perceptron neural network, and consists of activation functions, layers(y=wx+b form), and input/output layers.

import tensorflow as tf

import numpy as np

mn=tf.keras.datasets.mnist

(x_train,y_train),(x_test,y_test)=mn.load_data()

x_train,x_test=x_train/255.,x_test/255.

lr=0.001

x=tf.placeholder(dtype=tf.float32,shape=[None,28,28],name="in")

y=tf.placeholder(dtype=tf.uint8,shape=[None],name="out")

flx=tf.layers.Flatten()(x)

h=tf.layers.Dense(1000,activation=tf.nn.sigmoid)(flx)

h=tf.layers.Dense(10,activation=None)(h)

output=tf.nn.softmax(h)

target=tf.one_hot(y,depth=10)

loss=tf.reduce_mean(-tf.reduce_sum(target*tf.log(output),axis=1))

opt=tf.train.AdamOptimizer(lr)

train_opt=opt.minimize(loss)

sess=tf.Session()

sess.run(tf.global_variables_initializer())

cp=tf.equal(tf.argmax(output,1),tf.argmax(target,1))

ac=tf.reduce_mean(tf.cast(cp,tf.float32))

for i in range(10000):

sess.run(train_opt,feed_dict={x:x_train,y:y_train})

tmp1, tmp2 = sess.run([output,ac],feed_dict={x:x_test,y:y_test})

print(np.argmax(tmp1,axis=1), tmp2)

However, I haven't implemented a save mekanism yet. I will do that when I reach CNNs.

Result:

The Max accuracy changes a lot depending on what I set my learning rate to. However, if I set the learning rate too low, the AI will take forever to improve. Currently, I reached around 91% with this MLP Neural Network.

Using the MNIST data set, I will be trying to detect numbers based on images, and get the accuracy as high as possible.

The first step: the normal Flattened Neural Network. This method is much faster, but also less accurate, since the image's 2D properties turn into 1D, and lose the 2D shape. This causes a drastic loss in accuracy. 

The neural net I will be using is a Multi Layer Perceptron neural network, and consists of activation functions, layers(y=wx+b form), and input/output layers.

import tensorflow as tf

import numpy as np

mn=tf.keras.datasets.mnist

(x_train,y_train),(x_test,y_test)=mn.load_data()

x_train,x_test=x_train/255.,x_test/255.

lr=0.001

x=tf.placeholder(dtype=tf.float32,shape=[None,28,28],name="in")

y=tf.placeholder(dtype=tf.uint8,shape=[None],name="out")

flx=tf.layers.Flatten()(x)

h=tf.layers.Dense(1000,activation=tf.nn.sigmoid)(flx)

h=tf.layers.Dense(10,activation=None)(h)

output=tf.nn.softmax(h)

target=tf.one_hot(y,depth=10)

loss=tf.reduce_mean(-tf.reduce_sum(target*tf.log(output),axis=1))

opt=tf.train.AdamOptimizer(lr)

train_opt=opt.minimize(loss)

sess=tf.Session()

sess.run(tf.global_variables_initializer())

cp=tf.equal(tf.argmax(output,1),tf.argmax(target,1))

ac=tf.reduce_mean(tf.cast(cp,tf.float32))

for i in range(10000):

sess.run(train_opt,feed_dict={x:x_train,y:y_train})

tmp1, tmp2 = sess.run([output,ac],feed_dict={x:x_test,y:y_test})

print(np.argmax(tmp1,axis=1), tmp2)

However, I haven't implemented a save mekanism yet. I will do that when I reach CNNs.

Result:

The Max accuracy changes a lot depending on what I set my learning rate to. However, if I set the learning rate too low, the AI will take forever to improve. Currently, I reached around 91% with this MLP Neural Network.