代码之家 › 专栏 › 技术社区 › riorio

当传入线程数增加时,couchbase get操作将减慢。

couchbase spring-data spring-boot

riorio · 技术社区 · 7 年前

总结:

我们的主要性能问题是 Spring-Boot 2.0.4 和 Couchbase server 5.5.1

当线程数增加时,数据库响应时间性能会迅速下降。 Here is another report about the issue .

详情:

弹簧套以500个螺纹运行:

server:
  tomcat:
    max-threads: 500
    max-connections: 500

我们使用以下依赖项:

     <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-couchbase</artifactId>
        <version>3.0.9.RELEASE</version>
    </dependency>

我们的“从数据库中选择”是使用Spring数据存储库执行的:

Cat findFirstByOwnerIdAndNameAndColor(String ownerId, String name, String color);

我们有一个特别针对此查询的索引:

CREATE INDEX `cat_by_ownerId_name_and_color_idx` ON `pets`(`ownerId`,`name`,`color`) WHERE (`_class` = "com.example.Cat")

随着请求数的增加,我们可以看到数据库在回答查询时所用的时间快速下降。

例如,当每秒运行300个请求时,99%的响应时间大约是 10秒!! 50的百分位数大约是5秒。

返回文档的平均大小约为 300字节 . 这意味着我们正试图每秒提取大约90千字节。数量相对较低。

编辑:

我在这里添加了在couchbase的用户界面中运行相同查询的结果: (在UI中,完成查询需要1.75毫秒)。

{
 "plan": {
  "#operator": "Sequence",
  "~children": [
  {
    "#operator": "IndexScan3",
    "index": "cats_by_ownerId_name_and_color_idx",
    "index_id": "c061141c2d373067",
    "index_projection": {
      "primary_key": true
    },
    "keyspace": "pets",
    "namespace": "default",
    "spans": [
      {
        "exact": true,
        "range": [
          {
            "high": "\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\"",
            "inclusion": 3,
            "low": "\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\""
          },
          {
            "high": "\"Oscar\"",
            "inclusion": 3,
            "low": "\"Oscar\""
          },
          {
            "high": "\"red\"",
            "inclusion": 3,
            "low": "\"red\""
          }
        ]
      }
    ],
    "using": "gsi"
  },
  {
    "#operator": "Fetch",
    "keyspace": "pets",
    "namespace": "default"
  },
  {
    "#operator": "Parallel",
    "~child": {
      "#operator": "Sequence",
      "~children": [
        {
          "#operator": "Filter",
          "condition": "(((((`pets`.`_class`) = \"com.example.Cat\") and ((`pets`.`ownerId`) = \"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\")) and ((`pets`.`name`) = \"Oscar\")) and ((`pets`.`color`) = \"red\"))"
        },
        {
          "#operator": "InitialProject",
          "result_terms": [
            {
              "expr": "self",
              "star": true
            }
          ]
        },
        {
          "#operator": "FinalProject"
        }
      ]
    }
  }
]
 },
 "text": "select * from pets where _class=\"com.example.Cat\" and projectId=\"bf23fa4c-22c3-42ac-b141-39cdc76bb2x5\" and name=\"Oscar\" and color=\"red\""

}

编辑2

我们还尝试隐式地编写N1QL查询,但结果是相同的。和以前一样,我们会得到很多TimeoutExceptions:

   Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.dao.QueryTimeoutException: java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"}; nested exception is java.lang.RuntimeException: java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"}] with root cause

java.util.concurrent.TimeoutException: {"b":"pets","s":"n1ql","t":7500000,"i":"f8cdf670-d32a-4d74-858c-f9dd9789d264"}
   at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:131) ~[java-client-2.7.0.jar:na]
   at com.couchbase.client.java.bucket.api.Utils$1.call(Utils.java:127) ~[java-client-2.7.0.jar:na]
   at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140) ~[rxjava-1.3.8.jar:1.3.8]
   at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber.onTimeout(OnSubscribeTimeoutTimedWithFallback.java:166) ~[rxjava-1.3.8.jar:1.3.8]
   at rx.internal.operators.OnSubscribeTimeoutTimedWithFallback$TimeoutMainSubscriber$TimeoutTask.call(OnSubscribeTimeoutTimedWithFallback.java:191) ~[rxjava-1.3.8.jar:1.3.8]
   at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55) ~[rxjava-1.3.8.jar:1.3.8]
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_161]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_161]
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_161]
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_161]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_161]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_161]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

有没有办法解决这个问题,或者我们需要一个不同的数据库?

1 回复 | 直到 7 年前

riorio 7 年前

因此,经过进一步的调查,在 Spring-Data 组件。

我们得搬到 无阻塞机构 .

我们做了两件事:

从控制器层到服务和存储库层的所有调用都已更改为 CompleteableFuture<Cat>

为了绕过到couchbase的Spring数据连接,我们使用实现代码创建了一个我们自己的知识库类,代码如下所示:

Statement statement = select("*")
        .from(i(bucket.name()))
        .where(x("name").eq(s(name))
                .and(x("ownerId").eq(s(ownerId)))
                .and(x("color").eq(s(color)))
                .and(x("_class").eq(s("com.example.Cat"))));

CompletableFuture<Cat> completableFuture = new CompletableFuture();
bucket.async().query(statement)
...

在我们这样做之后,延迟问题消失了,性能 2毫秒 对于查询,即使在大约几百个并发请求中也是如此。