代码之家 › 专栏 › 技术社区 › Ric S

cifar10随机训练和测试集

numpy-ndarray keras random numpy python

Ric S · 技术社区 · 7 年前

keras.数据集 图书馆我知道,为了构造一个神经网络,它可能没有那么重要,但我是一个Python新手,我想学习用这种编程语言处理数据。

因此,要导入数据集,我运行

from keras.datasets import cifar10
(X_train, Y_train), (X_test, Y_test) = cifar10.load_data()

我想采取的步骤是:

X Y 形状(60000,1)
X 和例如,一个50000 obs的训练集和一个10000 obs的测试集中的数据集
恩达雷 火车 X_检验 你的火车 , Y_检验 使用与原始形状相同的形状,这样我就可以开始训练我的卷积神经网络

但也许有更快捷的方法。

我已经尝试了几个小时不同的方法,但没有取得任何成果。有人能帮我吗?我会非常感激的,谢谢。

2 回复 | 直到 7 年前

A Kruger 7 年前

你可以用 sklearn.model_selection.train_test_split random_state

from keras.datasets import cifar10
(X_train, Y_train), (X_test, Y_test) = cifar10.load_data()

# View first image
import matplotlib.pyplot as plt
plt.imshow(X_train[0])
plt.show()

import numpy as np
from sklearn.model_selection import train_test_split

# Concatenate train and test images
X = np.concatenate((X_train,X_test))
y = np.concatenate((Y_train,Y_test))

# Check shape
print(X.shape) # (60000, 32, 32, 3)

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=10000, random_state=1234)

# Check shape
print(X_train.shape) # (50000, 32, 32, 3)

# View first image
plt.imshow(X_train[0])
plt.show()

kmario23 Mazdak 7 年前

这是您要求的完整演示。首先,我们下载数据并随机化一次,然后先进行50K训练,然后进行10K验证。

In [21]: import tensorflow  
In [22]: import tensorflow.keras.datasets as datasets    
In [23]: cifar10 = datasets.cifar10.load_data() 
In [24]: (X_train, Y_train), (X_test, Y_test) = datasets.cifar10.load_data() 

In [25]: X_train.shape, Y_train.shape 
Out[25]: ((50000, 32, 32, 3), (50000, 1))

In [26]: X_test.shape, Y_test.shape 
Out[26]: ((10000, 32, 32, 3), (10000, 1)) 

In [27]: import numpy as np
In [28]: X, Y = np.vstack((X_train, X_test)), np.vstack((Y_train, Y_test))  

In [29]: X.shape, Y.shape 
Out[29]: ((60000, 32, 32, 3), (60000, 1)) 

In [30]: # Shuffle only the training data along axis 0 
    ...: def shuffle_train_data(X_train, Y_train): 
    ...:     """called after each epoch""" 
    ...:     perm = np.random.permutation(len(Y_train)) 
    ...:     Xtr_shuf = X_train[perm] 
    ...:     Ytr_shuf = Y_train[perm] 
    ...:      
    ...:     return Xtr_shuf, Ytr_shuf 


In [31]: X_shuffled, Y_shuffled = shuffle_train_data(X, Y) 

In [32]: (X_train_new, Y_train_new) = X_shuffled[:50000, ...], Y_shuffled[:50000, ...] 

In [33]: (X_test_new, Y_test_new) = X_shuffled[50000:, ...], Y_shuffled[50000:, ...] 

In [34]: X_train_new.shape, Y_train_new.shape 
Out[34]: ((50000, 32, 32, 3), (50000, 1))

In [35]: X_test_new.shape, Y_test_new.shape 
Out[35]: ((10000, 32, 32, 3), (10000, 1))

shuffle_train_data 它不断地洗牌数据,使示例及其标签保持相同的顺序。