代码之家  ›  专栏  ›  技术社区  ›  angelokh

批量插入到Hyperledger结构中可保持超时

  •  7
  • angelokh  · 技术社区  · 6 年前

    我们正在把记录批量插入超级分类账结构。然而,我们正在讨论超时问题。即使我们不断增加超时时间,我们也会在稍后发生这个错误。

    每个事务使用 PutState 在所有这些记录的循环中(盲插入,读取集中没有任何内容)。我们还将BatchTimeout增加到3s,MaxMessageCount增加到100,这样我们就得到了更大的块(我们看到每个块有4个事务,所以每个块有4000条[4x1000条记录/事务]记录被插入到分类账中)。

    当CouchDB的批量更新失败,并且对等方必须分别重试每个(每个事务1000条记录中的)记录时,查询花费的时间太长,超过了超时时间。但这是我们的假设。我们还发现: https://jira.hyperledger.org/browse/FAB-10558 ,但它说它已经在v1.2.0中修复了,这是我们正在使用的版本。

    我们的错误是 net/http: request canceled (Client.Timeout exceeded while reading body) 从以下日志中:

    我们尝试在对等容器中设置以下环境变量:

    CORE_CHAINCODE_EXECUTETIMEOUT=120s

    还有 req.setProposalWaitTime(120 * 1000) 使用Java SDK时。

    但是,我们只是在稍后得到相同的超时错误。所以我们可以继续增加超时变量到一个更大的数字,但我们相信它会在以后再次发生。插入CouchDB所需的时间是否与CouchDB中的记录数成比例?当文档数量增加时,更新索引可能需要更多的时间?


        October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/core/committer.(*LedgerCommitter).CommitWithPvtData(0xc4222db8c0, 0xc451e4f470, 0xc4312ddd40, 0xdf8475800)
        October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/state.(*GossipStateProviderImpl).deliverPayloads(0xc4220c5a00)
        October 5th 2018, 04:36:38.646  goroutine 283 [running]:
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/core/committer/committer_impl.go:105 +0x6b
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/privdata/coordinator.go:236 +0xc3b
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:771 +0x6c
        October 5th 2018, 04:36:38.646  
        October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/core/ledger/kvledger.(*kvLedger).CommitWithPvtData(0xc421fb1860, 0xc451e4f470, 0x0, 0x0)
        October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/privdata.(*coordinator).StoreBlock(0xc422286e60, 0xc42462cd80, 0x0, 0x0, 0x0, 0xc4312dde78, 0x7329db)
        October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/state.(*GossipStateProviderImpl).commitBlock(0xc4220c5a00, 0xc42462cd80, 0x0, 0x0, 0x0, 0x0, 0x0)
        October 5th 2018, 04:36:38.646  panic: Error during commit to txmgr:net/http: request canceled (Client.Timeout exceeded while reading body)
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/core/ledger/kvledger/kv_ledger.go:273 +0x870
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:558 +0x3c5
        October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:239 +0x681
        October 5th 2018, 04:36:38.646  created by github.com/hyperledger/fabric/gossip/state.NewGossipStateProvider
        October 5th 2018, 04:36:03.645  2018-10-04 20:36:00.783 UTC [kvledger] CommitWithPvtData -> INFO 466e[0m Channel [mychannel]: Committed block [1719] with 4 transaction(s)
        October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.807 UTC [statecouchdb] commitUpdates -> WARN 465c[0m CouchDB batch document update encountered an problem. Retrying update for document ID:32216027-da66-4ecd-91a1-a37bdf47f07d
        October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.866 UTC [statecouchdb] commitUpdates -> WARN 4663[0m CouchDB batch document update encountered an problem. Retrying update for document ID:6eaed2ae-e5c4-48b1-b063-20eb3009969b
        October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.870 UTC [statecouchdb] commitUpdates -> WARN 4664[0m CouchDB batch document update encountered an problem. Retrying update for document ID:2ca2fbcc-e78f-4ed0-be70-2c4d7ecbee69
        October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.904 UTC [statecouchdb] commitUpdates -> WARN 4667[0m CouchDB batch document update encountered an problem. ... and so on
    
    1 回复  |  直到 6 年前
        1
  •  4
  •   Vishesh Chauhan    6 年前

    [33m2018-10-04 20:35:55.870 UTC [statecouchdb] commitUpdates -> WARN 4664[0m CouchDB batch document update encountered an problem. Retrying update for document ID:2ca2fbcc-e78f-4ed0-be70-2c4d7ecbee69

    以上说明 POST http://localhost:5984/db/_bulk_docks 失败,因此单独尝试各个文档。

    查看可配置的不同参数,增加 requestTimeout ledger section 也许值得一试。

    这可以通过在docker compose中为对等容器设置以下环境变量来实现:

    CORE_LEDGER_STATE_COUCHDBCONFIG_REQUESTTIMEOUT=100s

    与配置参数关联的环境变量的名称可以通过查看 this 回答。

    配置 CORE_CHAINCODE_EXECUTETIMEOUT proposalWaitime 同龄人 库奇达 )正在超时,然后超时异常正在向上传播。