json.loads 错误：“utf-8”编解码器无法解码位置 0 的字节 0x83：无效的起始字节
Posted
技术标签:
【中文标题】json.loads 错误：“utf-8”编解码器无法解码位置 0 的字节 0x83：无效的起始字节【英文标题】：json.loads Error: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte 【发布时间】：2021-07-10 21:55:16 【问题描述】：突然间，我开始收到来自此 URL 的 JSON 响应的解码错误：
https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte
Unicode error hint
The string that could not be encoded/decoded was: �� ��
我找不到任何不符合 UTF-8 的字符。
Python 3.8：
import json
import requests
url = "https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product"
headers =
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,'
'application/signed-exchange;v=b3;q=0.9',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.9',
'Connection': 'keep-alive',
'Host': 'stockx.com',
'TE': 'Trailers',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/89.0.4389.128 Safari/537.36'
my_response = requests.get(url, headers=headers)
if my_response.ok:
j_data = json.loads(my_response.content)
print(j_data)
注意，my_response.encoding 返回 'utf-8'
【问题讨论】：
我可以确认我可以解析来自该 URL 的响应（不同的语言，不同的操作系统，但 JSON 似乎不是问题） 我也尝试过并为我工作！ @Everett，更像是当我直接访问编码时是“内容编码br”。所以唯一的区别就是环境！ 【参考方案1】：从我的测试来看，“Accept-Encoding”标头似乎将其强制转换为不可读的字节。如果您希望以纯文本形式发出请求，只需删除此标头并打印 my_response.text，否则，只需学习如何处理您请求的编码
【讨论】：
【参考方案2】：从标题中删除Accept-Encoding。此外，您可以在响应对象上使用.json() 方法：
import json
import requests
url = "https://stockx.com/api//browse?productCategory=sneakers&sort=featured&order=DESC&_search=CQ4227-030&dataType=product"
headers =
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,"
"application/signed-exchange;v=b3;q=0.9",
# "Accept-Encoding": "gzip, deflate, br", # <--- remove the Accept-Encoding header
"Accept-Language": "en-US,en;q=0.9",
"Connection": "keep-alive",
"Host": "stockx.com",
"TE": "Trailers",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/89.0.4389.128 Safari/537.36",
my_response = requests.get(url, headers=headers)
if my_response.ok:
j_data = my_response.json()
# j_data = json.loads(my_response.content)
print(json.dumps(j_data, indent=4))
打印：
"Pagination":
"limit": 40,
"page": 1,
"total": 1,
"lastPage": "/api/v3/browse?_search=CQ4227-030&dataType=product&order=DESC&productCategory=sneakers&sort=featured&page=1",
"sort": [
"featured"
],
"order": [
"DESC"
],
"currentPage": "/api/v3/browse?_search=CQ4227-030&dataType=product&order=DESC&productCategory=sneakers&sort=featured",
"nextPage": null,
"previousPage": null
,
"Facets":
"year":
"2021": 1
,
"brand":
"Jordan": 1
,
"gender":
"men": 1
,
"dataType":
"product": 1
,
"releaseTime":
"1610064000": 1
,
"market.featured":
"0": 1
,
"productCategory":
"sneakers": 1
,
"market.lowestAsk":
"144": 1
,
"market.highestBid":
"170": 1
,
"market.volatility":
"0.190516": 1
,
"market.pricePremium":
"-0.206": 1
,
"market.deadstockSold":
"359": 1
,
"market.changePercentage":
"-0.046667": 1
,
"market.salesLast72Hours":
"5": 1
,
"market.lastLowestAskTime":
"1618144284": 1
,
"market.lastHighestBidTime":
"1618029125": 1
,
"market.averageDeadstockPrice":
"158": 1
,
"Products": [
"id": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
"uuid": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
"brand": "Jordan",
"breadcrumbs": [],
"category": "Air Jordan 35",
"charityCondition": 0,
"childId": null,
"colorway": "Black/Fire Red-Reflective Silver",
"condition": "New",
"countryOfManufacture": "CN",
"dataType": "product",
"description": "",
"hidden": false,
"listingType": "standard",
"minimumBid": 25,
"gender": "men",
"doppelgangers": [],
"media":
"imageUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=700&h=500&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
"smallImageUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=300&h=214&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
"thumbUrl": "https://images.stockx.com/images/Air-Jordan-XXXV-Bred-Product.jpg?fit=fill&bg=FFFFFF&w=140&h=100&auto=format,compress&trim=color&q=90&dpr=2&updated_at=1615910682",
"gallery": [],
"hidden": false
,
"name": "Bred",
"productCategory": "sneakers",
"releaseDate": "2021-01-08",
"releaseTime": 1610064000,
"belowRetail": true,
"retailPrice": 180,
"shoe": "Jordan XXXV",
"shortDescription": "Air-Jordan-XXXV-Bred",
"styleId": "CQ4227-030/CQ4228-030",
"tickerSymbol": "JB-JOXBBF",
"title": "Jordan XXXV Bred",
"traits": [
"name": "Style",
"value": "CQ4227-030/CQ4228-030",
"filterable": true,
"visible": true,
"highlight": true
,
"name": "Colorway",
"value": "Black/Fire Red-Reflective Silver",
"filterable": true,
"visible": true,
"highlight": true
,
"name": "Retail Price",
"value": 180,
"filterable": true,
"visible": true,
"highlight": false
,
"name": "Release Date",
"value": "2021-01-08",
"filterable": true,
"visible": true,
"highlight": false
],
"type": 0,
"urlKey": "air-jordan-xxxv-bred",
"year": 2021,
"shoeSize": null,
"market":
"productId": 0,
"skuUuid": "",
"productUuid": "d1e93a15-3b48-431f-a98d-6f0b0ca80135",
"lowestAsk": 144,
"lowestAskSize": "10.5",
"parentLowestAsk": 0,
"numberOfAsks": 148,
"hasAsks": 1,
"salesThisPeriod": 5,
"salesLastPeriod": 0,
"highestBid": 170,
"highestBidSize": "15",
"numberOfBids": 96,
"hasBids": 1,
"annualHigh": 305,
"annualLow": 88,
"deadstockRangeLow": 116,
"deadstockRangeHigh": 170,
"volatility": 0.190516,
"deadstockSold": 359,
"pricePremium": -0.206,
"averageDeadstockPrice": 158,
"lastSale": 143,
"lastSaleSize": "8.5",
"salesLast72Hours": 5,
"changeValue": -7,
"changePercentage": -0.046667,
"absChangePercentage": 0.046667,
"totalDollars": 56819,
"lastLowestAskTime": 1618144284,
"lastHighestBidTime": 1618029125,
"lastSaleDate": "2021-04-15T13:38:23+00:00",
"createdAt": "2021-01-06T19:31:31+00:00",
"updatedAt": 1618495984,
"deadstockSoldRank": 29,
"pricePremiumRank": 96,
"averageDeadstockPriceRank": 86,
"featured": 0
,
"_tags": [
"sneakers",
"jordan",
"air jordan",
"35",
"release_date|2021-01-08",
"colorway|black/fire red-reflective silver",
"retail_price|180",
"style_id|cq4227-030/cq4228-030"
],
"lock_selling": false,
"selling_countries": [
"AD",
"AE",
"AG",
"AL",
"AR",
"AS",
"AT",
"AU",
"BA",
"BB",
"BE",
"BG",
"BH",
"BM",
"BR",
"BS",
"BW",
"BZ",
"CH",
"CL",
"CN",
"CO",
"CR",
"CY",
"CZ",
"DE",
"DK",
"DM",
"DO",
"DZ",
"EC",
"EE",
"FI",
"FJ",
"FO",
"GD",
"GE",
"GF",
"GI",
"GL",
"GP",
"GR",
"GT",
"GU",
"HK",
"HN",
"HR",
"HU",
"ID",
"IE",
"IL",
"IS",
"JM",
"JO",
"JP",
"KE",
"KN",
"KR",
"KW",
"KY",
"KZ",
"LC",
"LI",
"LS",
"LT",
"LU",
"LV",
"MA",
"MC",
"MO",
"MP",
"MQ",
"MT",
"MU",
"MW",
"MX",
"MY",
"MZ",
"NC",
"NI",
"NL",
"NO",
"NZ",
"OM",
"PA",
"PE",
"PF",
"PH",
"PL",
"PR",
"PW",
"QA",
"RE",
"RO",
"RS",
"SA",
"SC",
"SE",
"SG",
"SI",
"SK",
"SN",
"SV",
"TC",
"TH",
"TW",
"UY",
"VA",
"VE",
"VI",
"VN",
"ZA",
"IT",
"PT",
"CA",
"FR",
"GB",
"GG",
"JE",
"ES",
"US",
"IN"
],
"buying_countries": [
"AD",
"AE",
"AG",
"AI",
"AL",
"AM",
"AO",
"AR",
"AS",
"AT",
"AU",
"AW",
"BA",
"BB",
"BE",
"BF",
"BG",
"BH",
"BI",
"BJ",
"BM",
"BO",
"BR",
"BS",
"BT",
"BW",
"BY",
"BZ",
"CH",
"CI",
"CK",
"CL",
"CM",
"CN",
"CO",
"CR",
"CV",
"CY",
"CZ",
"DE",
"DJ",
"DK",
"DM",
"DO",
"DZ",
"EC",
"EE",
"ER",
"ET",
"FI",
"FJ",
"FO",
"GD",
"GE",
"GF",
"GI",
"GL",
"GM",
"GN",
"GP",
"GR",
"GT",
"GU",
"GW",
"GY",
"HK",
"HN",
"HR",
"HU",
"ID",
"IE",
"IL",
"IS",
"JM",
"JO",
"JP",
"KE",
"KG",
"KH",
"KI",
"KM",
"KN",
"KR",
"KW",
"KY",
"KZ",
"LA",
"LC",
"LI",
"LK",
"LS",
"LT",
"LU",
"LV",
"MA",
"MC",
"ME",
"MG",
"MH",
"MK",
"ML",
"MN",
"MO",
"MP",
"MQ",
"MR",
"MS",
"MT",
"MU",
"MV",
"MW",
"MX",
"MY",
"MZ",
"NC",
"NE",
"NF",
"NG",
"NI",
"NL",
"NO",
"NP",
"NZ",
"OM",
"PA",
"PE",
"PF",
"PG",
"PH",
"PL",
"PR",
"PW",
"PY",
"QA",
"RE",
"RO",
"RS",
"RW",
"SA",
"SB",
"SC",
"SE",
"SG",
"SI",
"SK",
"SL",
"SN",
"SR",
"ST",
"SV",
"SZ",
"TC",
"TD",
"TG",
"TH",
"TJ",
"TM",
"TN",
"TO",
"TV",
"TW",
"TZ",
"UA",
"UG",
"UY",
"VA",
"VC",
"VE",
"VG",
"VI",
"VN",
"VU",
"WF",
"WS",
"YE",
"YT",
"ZA",
"ZM",
"ZW",
"AZ",
"BN",
"CD",
"CG",
"EG",
"FM",
"GA",
"IT",
"MD",
"NA",
"PT",
"RU",
"SM",
"TR",
"TT",
"CA",
"FR",
"GB",
"GG",
"JE",
"ES",
"US",
"XK",
"IN"
],
"objectID": "d1e93a15-3b48-431f-a98d-6f0b0ca80135"
]
【讨论】：
以上是关于json.loads 错误：“utf-8”编解码器无法解码位置 0 的字节 0x83：无效的起始字节的主要内容，如果未能解决你的问题，请参考以下文章