簡介: uncurl是一個庫,允許您將curl請求轉換為使用requests 的python代碼。由於Chrome網路檢查器具有的“copy as cURL”,因此該工具對於用python重新創建瀏覽器請求很有用。 安裝: (venv) zhangdeMacBook-Pro:pythonClass ...
簡介:
uncurl是一個庫,允許您將curl請求轉換為使用requests 的python代碼。由於Chrome網路檢查器具有的“copy as cURL”,因此該工具對於用python重新創建瀏覽器請求很有用。
安裝:
(venv) zhangdeMacBook-Pro:pythonClass test$ pip install uncurl
使用步驟:(Google Chrome瀏覽器) 抓取百度新聞數據
google瀏覽器選擇開發者工具,進入開發者模式
3.打開pycharm,安裝過uncurl
uncurl "複製百度界面的curl格式數據"
(venv) zhangdeMacBook-Pro:pythonClass test$ uncurl "curl 'http://news.baidu.com/passport' -H 'Connection: keep-alive' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'X-Requested-With: XMLHttpRequest' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36' -H 'Referer: http://news.baidu.com/' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: zh-CN,zh;q=0.9' -H 'Cookie: BIDUPSID=B4118D1EAC1C90F87904BEAF72F66215; PSTM=1578995919; BAIDUID=B4118D1EAC1C90F8DFFC8D1114C47930:FG=1; LOCALGX=%u5317%u4EAC%7C%30%7C%u5317%u4EAC%7C%30; BDORZ=B490B5EBF6F3CD402E515D22BCDA1598; H_PS_PSSID=1423_21119_30492_26350_30499; delPer=0; PSINO=2; Hm_lvt_e9e114d958ea263de46e080563e254c4=1579054369,1579055803,1579139547,1579169647; Hm_lpvt_e9e114d958ea263de46e080563e254c4=1579169647' --compressed --insecure"
requests.get("http://news.baidu.com/passport",
headers={
"Accept": "application/json, text/javascript, */*; q=0.01",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "zh-CN,zh;q=0.9",
"Connection": "keep-alive",
"Referer": "http://news.baidu.com/",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36",
"X-Requested-With": "XMLHttpRequest"
},
cookies={
"BAIDUID": "B4118D1EAC1C90F8DFFC8D1114C47930:FG=1",
"BDORZ": "B490B5EBF6F3CD402E515D22BCDA1598",
"BIDUPSID": "B4118D1EAC1C90F87904BEAF72F66215",
"H_PS_PSSID": "1423_21119_30492_26350_30499",
"Hm_lpvt_e9e114d958ea263de46e080563e254c4": "1579169647",
"Hm_lvt_e9e114d958ea263de46e080563e254c4": "1579054369,1579055803,1579139547,1579169647",
"LOCALGX": "%u5317%u4EAC%7C%30%7C%u5317%u4EAC%7C%30",
"PSINO": "2",
"PSTM": "1578995919",
"delPer": "0"
},
verify=False
)
(venv) zhangdeMacBook-Pro:pythonClass test$