代码之家 › 专栏 › 技术社区 › bortzmeyer

使用aiohttp转储请求头

aiohttp python-3.x python

bortzmeyer · 技术社区 · 7 年前

我想显示一个请求的所有HTTP头(我添加的头和自动生成的头)。我试过用痕迹( https://aiohttp.readthedocs.io/en/stable/tracing_reference.html#aiohttp-client-tracing-reference ):

#!/usr/bin/env python3                                                                                                      

import aiohttp
import asyncio

async def on_request_start(session, trace_config_ctx, params):
    print("Starting %s request for %s. I will send: %s" % (params.method, params.url, params.headers))

async def on_request_end(session, trace_config_ctx, params):
    print("Ending %s request for %s. I sent: %s" % (params.method, params.url, params.headers))

async def fetch(session, url):
    async with session.get(url) as response:
        return response

async def main():
    trace_config = aiohttp.TraceConfig()
    trace_config.on_request_start.append(on_request_start)
    trace_config.on_request_end.append(on_request_end)
    async with aiohttp.ClientSession(trace_configs=[trace_config]) as session:
        r = await fetch(session, 'http://stackoverflow.com')
        print(r)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

通过这段代码,我得到了方法和URL,但是头的dict总是空的:

% ./test-debug.py
Starting GET request for http://stackoverflow.com. I will send: <CIMultiDict()>
Ending GET request for https://stackoverflow.com/. I sent: <CIMultiDict()>

我错过了什么?

Python 3.7.2

% pip show aiohttp
Name: aiohttp
Version: 3.5.4
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /usr/lib/python3.7/site-packages
Requires: async-timeout, attrs, multidict, yarl, chardet
Required-by:

2 回复 | 直到 7 年前

Patrick Mevzek James Dean 7 年前

仔细阅读了库源代码之后, request_start 太早了,即使在创建请求对象之前也会调用它,因此它将永远看不到完整的请求及其头;计时器将在之后启动,并循环发送内容。

但在 request_end 您可以访问完全响应对象,该对象绑定到请求对象,因此也可以访问所有头。

有了这个变化:

async def on_request_end(session, trace_config_ctx, params):
    print("Ending %s request for %s. I sent: %s" % (params.method, params.url, params.headers))
    print('Sent headers: %s' % params.response.request_info.headers)

我得到:

Sent headers: <CIMultiDictProxy('Host': 'stackoverflow.com', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Python/3.7 aiohttp/3.5.4', 'Cookie': 'prov=f4fad342-c1f7-bcc2-5d25-0e30ae5cdbf6')>

你可能还需要看看 params.response.history 在重定向的情况下。这是一系列 ClientResponse 对象,因此您应该能够调用 request_info.headers 在他们每个人身上。

Sylvain Lesage 7 年前

我也一样:

$ ./test-debug.py
Starting GET request for http://stackoverflow.com. I will send: <CIMultiDict()>
Ending GET request for https://stackoverflow.com/. I sent: <CIMultiDict()>
<ClientResponse(https://stackoverflow.com/) [200 OK]>
<CIMultiDictProxy('Cache-Control': 'private', 'Content-Type': 'text/html; charset=utf-8', 'Content-Encoding': 'gzip', 'X-Frame-Options': 'SAMEORIGIN', 'X-Request-Guid': 'c89dd68d-cb88-43c1-b08d-f2a07bf81043', 'Strict-Transport-Security': 'max-age=15552000', 'Content-Security-Policy': 'upgrade-insecure-requests', 'Content-Length': '52698', 'Accept-Ranges': 'bytes', 'Date': 'Tue, 15 Jan 2019 08:06:32 GMT', 'Via': '1.1 varnish', 'Connection': 'keep-alive', 'X-Served-By': 'cache-cdg20748-CDG', 'X-Cache': 'MISS', 'X-Cache-Hits': '0', 'X-Timer': 'S1547539592.382231,VS0,VE120', 'Vary': 'Accept-Encoding,Fastly-SSL', 'X-DNS-Prefetch-Control': 'off')>

$ python --version
Python 3.7.1

$ python -c "import aiohttp; print(aiohttp.__version__)"
3.4.4

如果我明确地向clientsession添加一个头部,

    async with aiohttp.ClientSession(trace_configs=[trace_config], headers={"Host": "stackoverflow.com"}) as session:

我从痕迹中看到:

$ ./test-debug.py
Starting GET request for http://stackoverflow.com. I will send: <CIMultiDict('Host': 'stackoverflow.com')>
Ending GET request for http://stackoverflow.com. I sent: <CIMultiDict('Host': 'stackoverflow.com')>