代码之家  ›  专栏  ›  技术社区  ›  Rohit Pandey

使用tensorflow在简单分类数据上获得惊人的精度

  •  1
  • Rohit Pandey  · 技术社区  · 8 年前

    我开始学习如何使用tensorflow。所以,我从提供的最基本的教程开始 here .

    该示例在mnist数据上训练两层感知器。我决定用我自己的数据来代替它。因此,我创建了一个方法,可以生成属于四个类之一的二维数据(参见非常简单的实现, method called get_data here ).这些类很明显是线性可分的(见下图)。所以,我希望任何分级机都能把这件事搞砸。

    然后我 modify the tensorflow sample 这样它就可以读取我的数据。

    结果太可怕了。这一损失似乎与准确度完全无关,而准确度在训练过程中差异很大。

    对任何人来说,我可能遗漏了什么是显而易见的吗?我所做的修改非常简单,我已经复习过很多次了。

    Step 1, Minibatch Loss= 2167311.5000, Training Accuracy= 0.250
    Step 100, Minibatch Loss= 19227.4277, Training Accuracy= 0.250
    Step 200, Minibatch Loss= 5008.3398, Training Accuracy= 0.180
    Step 300, Minibatch Loss= 1909.1743, Training Accuracy= 0.461
    Step 400, Minibatch Loss= 1811.5497, Training Accuracy= 0.398
    Step 500, Minibatch Loss= 2363.8840, Training Accuracy= 0.414
    Step 600, Minibatch Loss= 2374.1931, Training Accuracy= 0.195
    Step 700, Minibatch Loss= 173.2211, Training Accuracy= 0.352
    Step 800, Minibatch Loss= 1286.1042, Training Accuracy= 0.250
    Step 900, Minibatch Loss= 560.9091, Training Accuracy= 0.023
    Step 1000, Minibatch Loss= 163.1450, Training Accuracy= 0.195
    Step 1100, Minibatch Loss= 412.8525, Training Accuracy= 0.023
    Step 1200, Minibatch Loss= 155.7486, Training Accuracy= 0.094
    Step 1300, Minibatch Loss= 137.8443, Training Accuracy= 0.078
    Step 1400, Minibatch Loss= 59.5813, Training Accuracy= 0.062
    Step 1500, Minibatch Loss= 74.8706, Training Accuracy= 0.180
    Step 1600, Minibatch Loss= 7.7829, Training Accuracy= 0.250
    Step 1700, Minibatch Loss= 18.4251, Training Accuracy= 0.250
    Step 1800, Minibatch Loss= 76.1630, Training Accuracy= 0.211
    Step 1900, Minibatch Loss= 2147.1362, Training Accuracy= 0.250
    Step 2000, Minibatch Loss= 8275.0098, Training Accuracy= 0.242
    Step 2100, Minibatch Loss= 36986.7539, Training Accuracy= 0.250
    Step 2200, Minibatch Loss= 2482.1418, Training Accuracy= 0.164
    Step 2300, Minibatch Loss= 8919.1445, Training Accuracy= 0.250
    Step 2400, Minibatch Loss= 2694.6621, Training Accuracy= 0.172
    Step 2500, Minibatch Loss= 262.8948, Training Accuracy= 0.172
    Step 2600, Minibatch Loss= 655.5334, Training Accuracy= 0.148
    Step 2700, Minibatch Loss= 278.0427, Training Accuracy= 0.250
    Step 2800, Minibatch Loss= 2314.9653, Training Accuracy= 0.242
    Step 2900, Minibatch Loss= 570.7736, Training Accuracy= 0.180
    Step 3000, Minibatch Loss= 4217.2334, Training Accuracy= 0.250
    Step 3100, Minibatch Loss= 1161.3817, Training Accuracy= 0.250
    Step 3200, Minibatch Loss= 2473.6438, Training Accuracy= 0.234
    Step 3300, Minibatch Loss= 2483.4707, Training Accuracy= 0.250
    Step 3400, Minibatch Loss= 720.1823, Training Accuracy= 0.070
    Step 3500, Minibatch Loss= 1411.0126, Training Accuracy= 0.188
    Step 3600, Minibatch Loss= 1034.0898, Training Accuracy= 0.250
    Step 3700, Minibatch Loss= 2143.2910, Training Accuracy= 0.258
    Step 3800, Minibatch Loss= 2471.9592, Training Accuracy= 0.242
    Step 3900, Minibatch Loss= 932.8969, Training Accuracy= 0.250
    Step 4000, Minibatch Loss= 2762.5869, Training Accuracy= 0.180
    Step 4100, Minibatch Loss= 2132.0295, Training Accuracy= 0.250
    Step 4200, Minibatch Loss= 6322.4692, Training Accuracy= 0.250
    Step 4300, Minibatch Loss= 6657.2842, Training Accuracy= 0.242
    Step 4400, Minibatch Loss= 343629.0312, Training Accuracy= 0.195
    Step 4500, Minibatch Loss= 19370.7188, Training Accuracy= 0.234
    Step 4600, Minibatch Loss= 1008.6259, Training Accuracy= 0.227
    Step 4700, Minibatch Loss= 952.9269, Training Accuracy= 0.125
    Step 4800, Minibatch Loss= 390.7108, Training Accuracy= 0.211
    Step 4900, Minibatch Loss= 846.1492, Training Accuracy= 0.188
    Step 5000, Minibatch Loss= 218.7473, Training Accuracy= 0.250
    Optimization Finished!
    Testing Accuracy: 0.3275
    

    Radially separable data with four classes generated

    1 回复  |  直到 8 年前
        1
  •  1
  •   dennlinger    8 年前

    你的学习率相当高;常用值更接近于0.001-0.01的范围(不过,这可能因具体任务而异)。此外,您可能希望随着时间的推移降低学习率(即所谓的“学习率衰减”(详见 Andrey Karpathy's great lecture ).

    此外,仅培训5000次迭代可能不是您想要的。通常,像“过度拟合”这样的东西会妨碍你准确地学习一门语言 全体的 代表权。这意味着,您非常擅长识别(和分类)之前看到的测试数据,但不太擅长对未看到的数据进行分类。
    为此,最好使用第三个数据集进行验证(~测试集的大小)。然后使用该验证集来查看经过训练的模型在保留数据方面的性能。您将在大约每100次迭代后进行评估,然后查看此数据集上的损失是如何变化的。

    一种常见的技巧是“提前停止”,这意味着一旦你的算法在测试集上不再改进,你就停止训练过程。下图显示了一个与上下文密切相关的良好可视化效果 this Stackexchange post .

    最后,256个神经元相当多,在你的情况下甚至可能不需要。我不太清楚你的输入有多少维度,但一般来说,每层大约有16个神经元,你应该做得很好(也许可以尝试一下,看看不同大小的网络表现如何)。

    很多结果完全来自于知道要设置哪些参数,所以祝你好运!