使用Keras和Tensorflow实现CNN

我没有看到你的代码有任何错误。请注意,对于这样一个简单的问题,由于 Dense 引入超过260k可训练参数的层:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_3 (Conv2D)            (None, 24, 24, 64)        1664      
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 12, 12, 64)        0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 8, 8, 64)          102464    
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 4, 4, 64)          0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 1024)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 256)               262400    
_________________________________________________________________
dense_3 (Dense)              (None, 10)                2570      
=================================================================
Total params: 369,098
Trainable params: 369,098
Non-trainable params: 0
_________________________________________________________________

下面,我将使用以下命令运行您的代码:

• keras.datasets
• 一个简化模型:基本上我移除了256节点的密集层,大大减少了可训练参数的数量,并引入了一些用于正则化的退出。

通过这些更改,两个模型在第一个历元后都达到了90%以上的验证集精度。 因此,您遇到的问题似乎与导致高度可变结果的不适定优化问题有关,而与代码中的错误无关

# Import the datasets
import numpy as np
from keras.datasets import mnist
from keras.utils import to_categorical

(x_train, y_train), (x_test, y_test) = mnist.load_data()

# Add batch dimension
x_train = np.expand_dims(x_train, axis=-1)
x_test = np.expand_dims(x_test, axis=-1)

# One-hot encode the labels
y_train = to_categorical(y_train, num_classes=None)
y_test = to_categorical(y_test, num_classes=None)

batch_size = 64

# Fit model using Keras
import keras
import numpy as np
from keras.layers import Conv2D, MaxPool2D, Flatten, Dense, Dropout
from keras.models import Sequential

model = Sequential()
model.add(Conv2D(32, 5, activation="relu", input_shape=(28, 28, 1)))
model.add(MaxPool2D())
model.add(Conv2D(32, 5, activation="relu"))
model.add(MaxPool2D())
model.add(Flatten())
model.add(Dense(10, activation='softmax'))
model.compile(optimizer='adam', 
    loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, 
    batch_size=32, validation_data=(x_test, y_test), epochs=1)

结果:

Train on 60000 samples, validate on 10000 samples
Epoch 1/1
60000/60000 [==============================] - 35s 583us/step - loss: 1.5217 - acc: 0.8736 - val_loss: 0.0850 - val_acc: 0.9742

请注意,可训练参数的数量现在只是模型中数量的一小部分:

model.summary()
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_3 (Conv2D)            (None, 24, 24, 32)        832       
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 12, 12, 32)        0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 8, 8, 32)          25632     
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 4, 4, 32)          0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 512)               0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 10)                5130      
=================================================================
Total params: 31,594
Trainable params: 31,594
Non-trainable params: 0

现在,对TensorFlow执行相同的操作:

# Fit model using TensorFlow
import tensorflow as tf

X = tf.placeholder(shape=[None, 28, 28, 1], dtype=tf.float32, name="X")
Y = tf.placeholder(shape=[None, 10], dtype=tf.float32, name="Y")

net = tf.layers.conv2d(
  X, filters=32, kernel_size=5, padding="valid", activation=tf.nn.relu)
net = tf.layers.max_pooling2d(net, pool_size=2, strides=2)
net = tf.layers.conv2d(
  net, filters=32, kernel_size=5, padding="valid", activation=tf.nn.relu)
net = tf.layers.max_pooling2d(net, pool_size=2, strides=2)
net = tf.contrib.layers.flatten(net)
net = tf.layers.dropout(net, rate=0.25)
model = tf.layers.dense(net, name="output", units=10)

loss = tf.nn.softmax_cross_entropy_with_logits_v2(labels=Y, logits=model)
opt = tf.train.AdamOptimizer().minimize(loss)
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(model, 1), tf.argmax(Y, 1)), tf.float32))

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    L = []
    l_ = 0
    for i in range(x_train.shape[0] // batch_size):
        x, y = x_train[i*batch_size:(i+1)*batch_size],\
            y_train[i*batch_size:(i+1)*batch_size]
        l, _ = sess.run([loss, opt], feed_dict={X: x, Y: y})
        l_ += np.mean(l)
    L.append(l_ / (x_train.shape[0] // batch_size))
    print('Training loss: {:.3f}'.format(L[-1]))

    acc = []
    for j in range(x_test.shape[0] // batch_size):
        x, y = x_test[j*batch_size:(j+1)*batch_size],\
            y_test[j*batch_size:(j+1)*batch_size]
        acc.append(sess.run(accuracy, feed_dict={X: x, Y: y}))
    print('Test set accuracy: {:.3f}'.format(np.mean(acc)))

结果:

Training loss: 0.519
Test set accuracy: 0.968

可能会改进您的模型。

我在不同的问题上使用了CNN网络,并且通过正则化技术总是得到了很好的效果改进,最好的是退出。

我建议在密集层和卷积层概率较低的情况下使用压降。

此外,对输入数据的数据扩充也非常重要,但适用性取决于问题域。