a simple mnist example based on Keras(Keras Intro)

Keras

https://keras.io/

Simple. Flexible. Powerful.

Deep learning for humans.

Keras is an API designed for human beings, not machines.

Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear & actionable error messages.

It also has extensive documentation and developer guides.

mnist

https://keras.io/api/datasets/mnist/

load_data function

tf.keras.datasets.mnist.load_data(path="mnist.npz")

Loads the MNIST dataset.

This is a dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images. More info can be found at the MNIST homepage.

Arguments

• path: path where to cache the dataset locally (relative to ~/.keras/datasets).

Returns

• Tuple of Numpy arrays: (x_train, y_train), (x_test, y_test).
• x_train, x_test: uint8 arrays of grayscale image data with shapes (num_samples, 28, 28).
• y_train, y_test: uint8 arrays of digit labels (integers in range 0-9) with shapes (num_samples,).

License: Yann LeCun and Corinna Cortes hold the copyright of MNIST dataset, which is a derivative work from original NIST datasets. MNIST dataset is made available under the terms of the Creative Commons Attribution-Share Alike 3.0 license.

http://yann.lecun.com/exdb/mnist/

The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.

It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.

安装

https://www.tensorflow.org/install/pip

pip install --upgrade tensorflow

Simple MNIST convnet

https://keras.io/examples/vision/mnist_convnet/

https://github.com/keras-team/keras-io/blob/master/examples/vision/mnist_convnet.py

"""
Title: Simple MNIST convnet
Author: [fchollet](https://twitter.com/fchollet)
Date created: 2015/06/19
Last modified: 2020/04/21
Description: A simple convnet that achieves ~99% test accuracy on MNIST.
"""

"""
## Setup
"""

import numpy as np
from tensorflow import keras
from tensorflow.keras import layers

"""
## Prepare the data
"""

# Model / data parameters
num_classes = 10
input_shape = (28, 28, 1)

# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

# Scale images to the [0, 1] range
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255
# Make sure images have shape (28, 28, 1)
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
print("x_train shape:", x_train.shape)
print(x_train.shape[0], "train samples")
print(x_test.shape[0], "test samples")


# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

"""
## Build the model
"""

model = keras.Sequential(
    [
        keras.Input(shape=input_shape),
        layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
        layers.MaxPooling2D(pool_size=(2, 2)),
        layers.Flatten(),
        layers.Dropout(0.5),
        layers.Dense(num_classes, activation="softmax"),
    ]
)

model.summary()

"""
## Train the model
"""

batch_size = 128
epochs = 15

model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])

model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)

"""
## Evaluate the trained model
"""

score = model.evaluate(x_test, y_test, verbose=0)
print("Test loss:", score[0])
print("Test accuracy:", score[1])

output

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 14s 1us/step
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
WARNING:tensorflow:From /root/.pyenv/versions/3.6.8/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d (Conv2D)              (None, 26, 26, 32)        320
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 11, 11, 64)        18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 1600)              0
_________________________________________________________________
dropout (Dropout)            (None, 1600)              0
_________________________________________________________________
dense (Dense)                (None, 10)                16010
=================================================================
Total params: 34,826
Trainable params: 34,826
Non-trainable params: 0
_________________________________________________________________
Train on 54000 samples, validate on 6000 samples
2021-03-15 16:39:38.250130: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-03-15 16:39:38.260145: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2112000000 Hz
2021-03-15 16:39:38.262184: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ffff254b980 executing computations on platform Host. Devices:
2021-03-15 16:39:38.262538: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2021-03-15 16:39:38.312421: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Epoch 1/15
54000/54000 [==============================] - 28s 513us/sample - loss: 0.3700 - acc: 0.8849 - val_loss: 0.0806 - val_acc: 0.9788
Epoch 2/15
54000/54000 [==============================] - 24s 447us/sample - loss: 0.1105 - acc: 0.9656 - val_loss: 0.0541 - val_acc: 0.9845
Epoch 3/15
54000/54000 [==============================] - 26s 476us/sample - loss: 0.0840 - acc: 0.9746 - val_loss: 0.0457 - val_acc: 0.9873
Epoch 4/15
54000/54000 [==============================] - 24s 450us/sample - loss: 0.0727 - acc: 0.9784 - val_loss: 0.0431 - val_acc: 0.9878
Epoch 5/15
54000/54000 [==============================] - 31s 576us/sample - loss: 0.0636 - acc: 0.9804 - val_loss: 0.0392 - val_acc: 0.9883
Epoch 6/15
54000/54000 [==============================] - 24s 453us/sample - loss: 0.0573 - acc: 0.9824 - val_loss: 0.0344 - val_acc: 0.9905
Epoch 7/15
54000/54000 [==============================] - 26s 478us/sample - loss: 0.0526 - acc: 0.9834 - val_loss: 0.0324 - val_acc: 0.9913
Epoch 8/15
54000/54000 [==============================] - 24s 438us/sample - loss: 0.0492 - acc: 0.9845 - val_loss: 0.0339 - val_acc: 0.9905
Epoch 9/15
54000/54000 [==============================] - 23s 429us/sample - loss: 0.0452 - acc: 0.9855 - val_loss: 0.0311 - val_acc: 0.9905
Epoch 10/15
54000/54000 [==============================] - 23s 430us/sample - loss: 0.0432 - acc: 0.9864 - val_loss: 0.0299 - val_acc: 0.9913
Epoch 11/15
54000/54000 [==============================] - 23s 428us/sample - loss: 0.0412 - acc: 0.9867 - val_loss: 0.0291 - val_acc: 0.9922
Epoch 12/15
54000/54000 [==============================] - 23s 421us/sample - loss: 0.0394 - acc: 0.9875 - val_loss: 0.0306 - val_acc: 0.9902
Epoch 13/15
54000/54000 [==============================] - 23s 418us/sample - loss: 0.0376 - acc: 0.9880 - val_loss: 0.0268 - val_acc: 0.9917
Epoch 14/15
54000/54000 [==============================] - 22s 404us/sample - loss: 0.0347 - acc: 0.9889 - val_loss: 0.0296 - val_acc: 0.9912
Epoch 15/15
54000/54000 [==============================] - 22s 399us/sample - loss: 0.0360 - acc: 0.9884 - val_loss: 0.0269 - val_acc: 0.9917
Test loss: 0.02560416207953822
Test accuracy: 0.9908

Epoch vs Iteration

https://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks

618

 

In the neural network terminology:

• one epoch = one forward pass and one backward pass of all the training examples
• batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.
• number of iterations = number of passes, each pass using [batch size] number of examples. To be clear, one pass = one forward pass + one backward pass (we do not count the forward pass and backward pass as two different passes).

Example: if you have 1000 training examples, and your batch size is 500, then it will take 2 iterations to complete 1 epoch.

FYI: Tradeoff batch size vs. number of iterations to train a neural network