我有同样的问题,我只有两个数据集,一个用于测试和训练。我使用测试数据作为
路缘石模型.fit
像这样工作。
model.fit(X_train,y_train,
batch_size=64,
epochs=epochs,
callbacks=[es,reduceLROnPlateau,modelCheckpoint],
validation_data=(X_test,y_test),
verbose=2)
我在用
早期
停止,
还原隆坪
和
模型检查点
. 一切都在
val_loss
. 我的目标是让训练停止在测试精度或AUC得分最高的地方。既然验证数据对培训没有贡献,这样做可以吗?更详细地说,我的测试数据与列车数据包含完全不同/不可见的对象,因此即使我通过分割列车数据来制作验证数据,它也不是我的网络对测试数据的泛化性能的一个很好的指标。所以我得到的结果是科学上公平还是
作弊
?这方面的最佳做法是什么?
Train on 39840 samples, validate on 7440 samples
Epoch 1/30
- 257s - loss: 0.6046 - acc: 0.7782 - val_loss: 0.6797 - val_acc: 0.8055
Epoch 00001: val_loss improved from inf to 0.67972, saving model to keras_densenet_simple_wt_28Sept.h5
Epoch 2/30
- 253s - loss: 0.5055 - acc: 0.8236 - val_loss: 0.5370 - val_acc: 0.8085
Epoch 00002: val_loss improved from 0.67972 to 0.53701, saving model to keras_densenet_simple_wt_28Sept.h5
Epoch 3/30
- 253s - loss: 0.4506 - acc: 0.8518 - val_loss: 0.6623 - val_acc: 0.7937
Epoch 00003: val_loss did not improve from 0.53701
Epoch 4/30
- 253s - loss: 0.4110 - acc: 0.8711 - val_loss: 0.6583 - val_acc: 0.7995
Epoch 00004: val_loss did not improve from 0.53701
Epoch 5/30
- 253s - loss: 0.3726 - acc: 0.8908 - val_loss: 0.5282 - val_acc: 0.8237
Epoch 00005: val_loss improved from 0.53701 to 0.52820, saving model to keras_densenet_simple_wt_28Sept.h5
Epoch 6/30
- 253s - loss: 0.3427 - acc: 0.9035 - val_loss: 0.4683 - val_acc: 0.8422
Epoch 00006: val_loss improved from 0.52820 to 0.46830, saving model to keras_densenet_simple_wt_28Sept.h5
Epoch 7/30
- 253s - loss: 0.3148 - acc: 0.9152 - val_loss: 0.5580 - val_acc: 0.8109
Epoch 00007: val_loss did not improve from 0.46830
Epoch 8/30
- 253s - loss: 0.2928 - acc: 0.9241 - val_loss: 0.5347 - val_acc: 0.8290
Epoch 00008: val_loss did not improve from 0.46830
Epoch 9/30
- 252s - loss: 0.2719 - acc: 0.9334 - val_loss: 0.7429 - val_acc: 0.7800
Epoch 00009: val_loss did not improve from 0.46830
Epoch 10/30
- 252s - loss: 0.2563 - acc: 0.9389 - val_loss: 0.8768 - val_acc: 0.7226
Epoch 00010: val_loss did not improve from 0.46830
Epoch 11/30
- 252s - loss: 0.2413 - acc: 0.9449 - val_loss: 0.9852 - val_acc: 0.7405
Epoch 00011: ReduceLROnPlateau reducing learning rate to 9.486833431105525e-05.
Epoch 00011: val_loss did not improve from 0.46830
Epoch 00011: early stopping
7440/7440 [==============================] - 15s 2ms/step
Present Test accuracy: 0.740
Present auc_score ------------------> 0.896
7440/7440 [==============================] - 17s 2ms/step
Loading Best saved model by model checkpoint
Best saved model Test accuracy: 0.842
best saved model Auc_score ------------------> 0.922
======================================================================
