代码之家 › 专栏 › 技术社区 › knowledge_seeker

无法解析从Python中的HTML标记中的属性获得的JSON字符串

beautifulsoup json python

knowledge_seeker · 技术社区 · 2 年前

我正在对一个端点进行AJAX调用(我没有创建这个API),其中响应是JSON形式的 content 的类型 string 。在我看来,这些内容是HTML数据,其中包含一些JSON。我希望能够解析HTML数据中包含的JSON,但当我尝试执行 json.loads() 字符串的:

{JSONDecodeError}JSONDecodeError('Expecting property name enclosed in double quotes: line 1 column 2 (char 1)')

我真的不明白为什么我会犯这个错误

以下是我试图解析的JSON字符串:

{\"name\":\"ThreadMainListItemNormalizer\",\"props\":{\"thread\":{\"threadId\":4369992,\"threadTypeId\":1,\"titleSlug\":\"sebamed-sale-extra-soft-baby-cream-ps239-anti-dandruff-shampoo-ps387\",\"title\":\"Sebamed sale - extra soft baby cream \Â£2.39 / anti dandruff shampoo \Â£3.87\",\"currentUserVoteDirection\":\"\",\"commentCount\":0,\"status\":\"Activated\",\"isExpired\":false,\"isNew\":true,\"isPinned\":false,\"isTrending\":null,\"isBookmarked\":false,\"isLocal\":false,\"temperature\":0,\"temperatureLevel\":\"\",\"type\":\"Deal\",\"nsfw\":false,\"deletedAt\":null,\"publishedAt\":1720003748,\"voucherCode\":\"\",\"link\":\"https://www.justmylook.com/sebamed-m583\",\"merchant\":{\"merchantId\":45518,\"merchantName\":\"Justmylook\",\"merchantUrlName\":\"justmylook.co.uk\",\"isMerchantPageEnabled\":true},\"price\":2.39,\"nextBestPrice\":0,\"percentage\":0,\"discountType\":null,\"shipping\":{\"isFree\":1,\"price\":0},\"user\":{\"userId\":2701300,\"username\":\"Manish_N\",\"title\":\"\",\"avatar\":{\"path\":\"users/raw/default\",\"name\":\"2701300_6\",\"slotId\":\"default\",\"width\":0,\"height\":0,\"version\":6,\"unattached\":false,\"uid\":\"2701300_6.raw\",\"ext\":\"raw\"},\"persona\":{\"text\":null,\"type\":null},\"isBanned\":false,\"isDeletedOrPendingDeletion\":false,\"isUserProfileHidden\":false}}}}

如果我将上面的JSON字符串粘贴在 this online JSON validator tool 它说它是无效的JSON,但是,当我取消捕获 JSON using this tool 我得到以下输出:

"name":"ThreadMainListItemNormalizer","props":{"thread":{"threadId":4369991,"threadTypeId":1,"titleSlug":"samsung-55-qn700c-neo-qled-8k-hdr-smart-tv","title":"Samsung 55\" QN700C Neo QLED 8K HDR Smart TV Sold by Reliant Direct FBA","currentUserVoteDirection":"","commentCount":0,"status":"Activated","isExpired":false,"isNew":true,"isPinned":false,"isTrending":null,"isBookmarked":false,"isLocal":false,"temperature":0.59,"temperatureLevel":"Hot1","type":"Deal","nsfw":false,"deletedAt":null,"publishedAt":1720003637,"voucherCode":"","link":"https://www.amazon.co.uk/dp/B0BWFNLPTP?smid=A2CN43WDI0AWCL","merchant":{"merchantId":1650,"merchantName":"Amazon","merchantUrlName":"amazon-uk","isMerchantPageEnabled":true},"price":999,"nextBestPrice":1198,"percentage":0,"discountType":null,"shipping":{"isFree":1,"price":0},"user":{"userId":2679277,"username":"ben.jammin","title":"","avatar":{"path":"users/raw/default","name":"2679277_1","slotId":"default","width":0,"height":0,"version":1,"unattached":false,"uid":"2679277_1.raw","ext":"raw"},"persona":{"text":null,"type":null},"isBanned":false,"isDeletedOrPendingDeletion":false,"isUserProfileHidden":false}}}}

这实际上是有效的JSON。然后,当我尝试复制unescape工具并尝试在Python中对字符串进行unescape时,我的问题就出现了。

我尝试了以下解决方案

Using ast.literal_eval() 但我得到以下错误

{SyntaxError}SyntaxError('unexpected character after line continuation character', ('<unknown>', 1, 3, '{\\"name\\":\\"ThreadMainListItemNo...:null,\\"type\\":null},\\"isBanned\\":false,\\"isDeletedOrPendingDeletion\\":false,\\"isUserProfileHidden\\":false}}}}', 1, 0))

使用 .encode('raw_unicode_escape').decode('unicode_escape') 概述的方法 here 但在做了一个 json.loads() 我得到以下错误
```
{JSONDecodeError}JSONDecodeError('Invalid \\escape: line 1 column 224 (char 223)')
```

Here is the full API response as requested. 我对的价值感兴趣 所容纳之物 钥匙

更新:

我认为问题是字符串中有一些无效的转义字符。 \Â£ 。我遵循了解决方案 here 它解决了我的问题。

有人知道为什么这个API可能包含一个escaped Â£ 象征

2 回复 | 直到 2 年前

Barry the Platipus 2 年前

以下是一种处理方法:

import json

string = "{\"name\":\"ThreadMainListItemNormalizer\",\"props\":{\"thread\":{\"threadId\":4369992,\"threadTypeId\":1,\"titleSlug\":\"sebamed-sale-extra-soft-baby-cream-ps239-anti-dandruff-shampoo-ps387\",\"title\":\"Sebamed sale - extra soft baby cream \Â£2.39 / anti dandruff shampoo \Â£3.87\",\"currentUserVoteDirection\":\"\",\"commentCount\":0,\"status\":\"Activated\",\"isExpired\":false,\"isNew\":true,\"isPinned\":false,\"isTrending\":null,\"isBookmarked\":false,\"isLocal\":false,\"temperature\":0,\"temperatureLevel\":\"\",\"type\":\"Deal\",\"nsfw\":false,\"deletedAt\":null,\"publishedAt\":1720003748,\"voucherCode\":\"\",\"link\":\"https://www.justmylook.com/sebamed-m583\",\"merchant\":{\"merchantId\":45518,\"merchantName\":\"Justmylook\",\"merchantUrlName\":\"justmylook.co.uk\",\"isMerchantPageEnabled\":true},\"price\":2.39,\"nextBestPrice\":0,\"percentage\":0,\"discountType\":null,\"shipping\":{\"isFree\":1,\"price\":0},\"user\":{\"userId\":2701300,\"username\":\"Manish_N\",\"title\":\"\",\"avatar\":{\"path\":\"users/raw/default\",\"name\":\"2701300_6\",\"slotId\":\"default\",\"width\":0,\"height\":0,\"version\":6,\"unattached\":false,\"uid\":\"2701300_6.raw\",\"ext\":\"raw\"},\"persona\":{\"text\":null,\"type\":null},\"isBanned\":false,\"isDeletedOrPendingDeletion\":false,\"isUserProfileHidden\":false}}}}"
string = string.replace('false', '"False"').replace('true', '"True"').replace('null', '"None"').replace('\\', '')
result = json.loads(string)
print(result)

终端中的结果:

{'name': 'ThreadMainListItemNormalizer',
 'props': {'thread': {'threadId': 4369992,
   'threadTypeId': 1,
   'titleSlug': 'sebamed-sale-extra-soft-baby-cream-ps239-anti-dandruff-shampoo-ps387',
   'title': 'Sebamed sale - extra soft baby cream Â£2.39 / anti dandruff shampoo Â£3.87',
   'currentUserVoteDirection': '',
   'commentCount': 0,
   'status': 'Activated',
   'isExpired': 'False',
   'isNew': 'True',
   'isPinned': 'False',
   'isTrending': 'None',
   'isBookmarked': 'False',
   'isLocal': 'False',
   'temperature': 0,
   'temperatureLevel': '',
   'type': 'Deal',
   'nsfw': 'False',
   'deletedAt': 'None',
   'publishedAt': 1720003748,
   'voucherCode': '',
   'link': 'https://www.justmylook.com/sebamed-m583',
   'merchant': {'merchantId': 45518,
    'merchantName': 'Justmylook',
    'merchantUrlName': 'justmylook.co.uk',
    'isMerchantPageEnabled': 'True'},
   'price': 2.39,
   'nextBestPrice': 0,
   'percentage': 0,
   'discountType': 'None',
   'shipping': {'isFree': 1, 'price': 0},
   'user': {'userId': 2701300,
    'username': 'Manish_N',
    'title': '',
    'avatar': {'path': 'users/raw/default',
     'name': '2701300_6',
     'slotId': 'default',
     'width': 0,
     'height': 0,
     'version': 6,
     'unattached': 'False',
     'uid': '2701300_6.raw',
     'ext': 'raw'},
    'persona': {'text': 'None', 'type': 'None'},
    'isBanned': 'False',
    'isDeletedOrPendingDeletion': 'False',
    'isUserProfileHidden': 'False'}}}}

jsbueno 2 年前

只需使用一个简单的str.replace-在这种情况下似乎足够好了。即使有一个转义的反斜杠,如 a\\"b ,替换策略仍将保留一个反斜杠字符:

new_json = old_json.replace(r'\"', '"')