代码之家 › 专栏 › 技术社区 › aSemy

当stringstream.str()有8192个字符的神秘限制时,我如何获取std::stringstream中包含的完整字符串?

esp32 c++

aSemy · 技术社区 · 1 年前

我在用 https://github.com/biaks/prometheus-cpp-lite/ 在ESP32项目中(即我是C++新手)。

我正在将指标提取到 std::stringstream :

std::stringstream bodyStream;
prometheus::TextSerializer::Serialize(bodyStream, metrics::Registry.Collect());

// get the string
const std::string bodyString = bodyStream.str();
const char *bodyChars = bodyString.c_str();

// debug printing
Serial.print("bodyString.length(): ");
Serial.println(bodyString.length()); // prints at most 8192
Serial.println(bodyChars); // prints at most 8192 characters

然而,这永远不会打印完整的字符串——它总是限制在最多8192个字符。这意味着指标永远无法被抓取,因为它们总是被切到指标的一半,所以Prometheus无法解析它们。

为什么字符串的长度限制为8192?我怎样才能得到完整的字符串?

文件 stringstream.str() 和 bodyStream.rdbuf()->str() 是相当无益的:

/**
 *  @brief  Copying out the string buffer.
 *  @return  A copy of one of the underlying sequences.
 *
 *  <em>If the buffer is only created in input mode, the underlying
 *  character sequence is equal to the input sequence; otherwise, it
 *  is equal to the output sequence.</em> [27.7.1.2]/1
 */

我不知道这份文件是什么意思。。。

我试着构建 std::stringstream 并提供一个预先设定大小的较大字符串作为其缓冲区:

std::string bufStr (10000, 'x');
std::stringstream bodyStream(bufStr);
prometheus::TextSerializer::Serialize(bodyStream, metrics::Registry.Collect());

然而,这会触发一个错误(我无法破译)

abort() was called at PC 0x4015f537 on core 1

Backtrace: 0x400838fd:0x3ffdb900 0x4008d40d:0x3ffdb920 0x40092e5d:0x3ffdb940 0x4015f537:0x3ffdb9c0 0x4015f57e:0x3ffdb9e0 0x4015f531:0x3ffdba00 0x400d97a7:0x3ffdba20 0x400da826:0x3ffdba60 0x400d35cd:0x3ffdbbb0 0x400dc5cd:0x3ffdbc00 0x400de6fd:0x3ffdc180 0x400f18af:0x3ffdc1a0 0x400f18ea:0x3ffdc1d0 0x400eef99:0x3ffdc220 0x400ef059:0x3ffdc270 0x400ef311:0x3ffdc2d0 0x400ead27:0x3ffdc2f0 0x400ead76:0x3ffdc320 0x400eadb9:0x3ffdc340 0x400eaf01:0x3ffdc360 0x400eaf86:0x3ffdc380

  #0  0x400838fd:0x3ffdb900 in panic_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/panic.c:408
  #1  0x4008d40d:0x3ffdb920 in esp_system_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/esp_system.c:137
  #2  0x40092e5d:0x3ffdb940 in abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/newlib/abort.c:46
  #3  0x4015f537:0x3ffdb9c0 in __cxxabiv1::__terminate(void (*)()) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
  #4  0x4015f57e:0x3ffdb9e0 in std::terminate() at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
  #5  0x4015f531:0x3ffdba00 in __cxa_rethrow at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:133
  #6  0x400d97a7:0x3ffdba20 in prometheus::ClientMetric* std::__uninitialized_copy<false>::__uninit_copy<__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*>(__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, __gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*) at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_uninitialized.h:89
      (inlined by) prometheus::ClientMetric* std::uninitialized_copy<__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*>(__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, __gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*) at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_uninitialized.h:134
      (inlined by) prometheus::ClientMetric* std::__uninitialized_copy_a<__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*, prometheus::ClientMetric>(__gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, __gnu_cxx::__normal_iterator<prometheus::ClientMetric const*, std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > >, prometheus::ClientMetric*, std::allocator<prometheus::ClientMetric>&) at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_uninitialized.h:289
      (inlined by) std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> >::vector(std::vector<prometheus::ClientMetric, std::allocator<prometheus::ClientMetric> > const&) at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_vector.h:463
      (inlined by) prometheus::MetricFamily::MetricFamily(prometheus::MetricFamily const&) at src/lib/prometheus/metric_family.h:11
  #7  0x400da826:0x3ffdba60 in prometheus::Family::Collect() const at src/lib/prometheus/family.h:252
  #8  0x400d35cd:0x3ffdbbb0 in prometheus::Registry::Collect() const at src/lib/prometheus/registry.h:74
  #9  0x400dc5cd:0x3ffdbc00 in initWebServer()::{lambda(AsyncWebServerRequest*)#2}::operator()(AsyncWebServerRequest*) const at src/main.cpp:187
  #10 0x400de6fd:0x3ffdc180 in std::_Function_handler<void (AsyncWebServerRequest*), initWebServer()::{lambda(AsyncWebServerRequest*)#2}>::_M_invoke(std::_Any_data const&, AsyncWebServerRequest*&&) at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
  #11 0x400f18af:0x3ffdc1a0 in std::function<void (AsyncWebServerRequest*)>::operator()(AsyncWebServerRequest*) const at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
  #12 0x400f18ea:0x3ffdc1d0 in AsyncCallbackWebHandler::handleRequest(AsyncWebServerRequest*) at .pio/libdeps/esp32dev/ESP Async WebServer/src/WebHandlerImpl.h:132
  #13 0x400eef99:0x3ffdc220 in AsyncWebServerRequest::_parseLine() at .pio/libdeps/esp32dev/ESP Async WebServer/src/WebRequest.cpp:581 (discriminator 1)
  #14 0x400ef059:0x3ffdc270 in AsyncWebServerRequest::_onData(void*, unsigned int) at .pio/libdeps/esp32dev/ESP Async WebServer/src/WebRequest.cpp:123
  #15 0x400ef311:0x3ffdc2d0 in std::_Function_handler<void (void*, AsyncClient*, void*, unsigned int), AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::{lambda(void*, AsyncClient*, void*, unsigned int)#8}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&, std::_Any_data const&, unsigned int&&) at .pio/libdeps/esp32dev/ESP Async WebServer/src/WebRequest.cpp:76
      (inlined by) _M_invoke at d:\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297
  #16 0x400ead27:0x3ffdc2f0 in std::function<void (void*, AsyncClient*, void*, unsigned int)>::operator()(void*, AsyncClient*, void*, unsigned int) const at d:\user\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687
  #17 0x400ead76:0x3ffdc320 in AsyncClient::_recv(tcp_pcb*, pbuf*, signed char) at .pio/libdeps/esp32dev/AsyncTCP/src/AsyncTCP.cpp:915
  #18 0x400eadb9:0x3ffdc340 in AsyncClient::_s_recv(void*, tcp_pcb*, pbuf*, signed char) at .pio/libdeps/esp32dev/AsyncTCP/src/AsyncTCP.cpp:1191
  #19 0x400eaf01:0x3ffdc360 in _handle_async_event(lwip_event_packet_t*) at .pio/libdeps/esp32dev/AsyncTCP/src/AsyncTCP.cpp:159
  #20 0x400eaf86:0x3ffdc380 in _async_service_task(void*) at .pio/libdeps/esp32dev/AsyncTCP/src/AsyncTCP.cpp:194

值得注意的是,其中一些错误与 https://github.com/me-no-dev/ESPAsyncWebServer 代码,因为Prometheus必须通过端点抓取指标。

#include <ESPAsyncWebServer.h>
#include "main.h"
#include "metrics.h"
#include "utils.h"

AsyncWebServer httpServer(80);

void initWebServer() {
  httpServer.on("/metrics", HTTP_GET, [](AsyncWebServerRequest *request) {
    request->client()->setRxTimeout(10);
    Serial.println("[httpServer] /metrics...");

    std::string bufStr (10000, 'x');
    std::stringstream bodyStream(bufStr);
    prometheus::TextSerializer::Serialize(bodyStream, metrics::Registry.Collect());
    const std::string bodyString = bodyStream.str();
    Serial.print("bodyString.length(): ");
    Serial.println(bodyString.length());
    const char *bodyChars = bodyString.c_str();
    //Serial.println(bodyChars);

    request->send_P(
        200,
        "text/plain",
        bodyChars
    );
  });
}

void setup() {
  initWebServer();
}

0 回复 | 直到 1 年前

aSemy 1 年前

我无法确定8192限制的原因,但我已经实施了一种解决方法。

首先,我更新了 text_serializer.h 将内容写入 Stream ,不是a std::stringstream ,因为文件是流的一种类型。

例如,我更改了 WriteValue() 功能从

  static void WriteValue(std::ostream &out, const std::string &value) {
    for (auto c: value) {
      switch (c) {
        case '\n':
          out << '\\' << 'n';
          break;
        case '\\':
            out << '\\' << c;
            break;
        case '"':
          out << '\\' << c;
          break;
        default:
          out << c;
          break;
      }
    }
  }

到

  static void WriteValue(Stream &out, const std::string &value) {
    for (auto c: value) {
      switch (c) {
        case '\n':
          out.print("\\n");
          break;
        case '\\':
        case '"':
          out.print("\\");
          out.print(c);
          break;
        default:
          out.print(c);
          break;
      }
    }
  }

这仍然对字符串的大小有限制(略小于8192,这很奇怪,但我没有进一步调查)。

我还创建了一个单独的任务,使用LittleFS收集指标并将其写入文件。它不是写入另一个任务将读取的同一文件,如果在写入过程中读取文件,则会有并发问题的风险,而是写入一个临时文件并重新命名该文件。

void collectMetricsTask(void *) {
  while (true) {
    File file = LittleFS.open("/metrics-tmp.txt", FILE_WRITE);
    if (!file) {
      Serial.println("There was an error opening metrics-tmp.txt for writing");
    } else {
      prometheus::TextSerializerStream::Serialize(file, metrics::Registry.Collect());
      file.close();
    }
    const auto renamed = LittleFS.rename("/metrics-tmp.txt", "/metrics.txt");
    if (!renamed) {
      Serial.println("failed to rename metrics-tmp.txt to metrics.txt");
    }
    delay(10000); // 10 seconds
    yield();
  }
}

然后,可以在单独的任务中读取指标,并且可以获得完整的内容。不再有8192个上限!

例如,ESPAsyncWebServer可以读取指标文件:

void initWebServer() {
  httpServer.on("/metrics", HTTP_GET, [](AsyncWebServerRequest *request) {
    Serial.println("[httpServer] /metrics...");
    request->send(LittleFS, "/metrics.txt", "text/plain");
  });