Skip to content

Commit

Permalink
新增 Web API 模式
Browse files Browse the repository at this point in the history
  • Loading branch information
JoeanAmier committed Jun 5, 2024
1 parent e26d487 commit bd0780c
Show file tree
Hide file tree
Showing 12 changed files with 166 additions and 18 deletions.
61 changes: 57 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<li>✅ 支持命令行下载作品文件</li>
<li>✅ 从浏览器读取 Cookie</li>
<li>✅ 自定义文件名称格式</li>
<li>☑️ 支持 API 调用功能</li>
<li> 支持 API 调用功能</li>
</ul>
<ul><b>脚本功能</b>
<li>✅ 下载小红书无水印作品文件</li>
Expand Down Expand Up @@ -70,14 +70,66 @@
<li>运行 <code>main.py</code> 即可使用</li>
</ol>
<h1>🛠 命令行模式</h1>
<p>项目支持命令行运行模式,若想要下载图文作品的部分图片,可以使用此模式传入需要下载的图片序号!</p>
<p>可以使用命令行从浏览器读取 Cookie 并写入配置文件!注意需要关闭对应浏览器才能读取数据!</p>
<p><code>bool</code> 类型参数支持使用 <code>true</code>、<code>false</code>、<code>1</code>、<code>0</code>、<code>yes</code>、<code>no</code>、<code>on</code> 或 <code>off</code>(不区分大小写)来设置。</p>
<p>项目支持命令行运行模式,若想要下载图文作品的部分图片,可以使用此模式设置需要下载的图片序号!</p>
<p>可以使用命令行<b>从浏览器读取 Cookie 并写入配置文件</b>!注意需要关闭浏览器才能读取数据!</p>
<p>命令示例:<code>python .\main.py --browser_cookie Chrome --update_settings</code></p>
<p><code>bool</code> 类型参数支持使用 <code>true</code>、<code>false</code>、<code>1</code>、<code>0</code>、<code>yes</code>、<code>no</code>、<code>on</code> 或 <code>off</code>(不区分大小写)来设置。</p>
<hr>
<img src="static/screenshot/命令行模式截图1.png" alt="">
<hr>
<img src="static/screenshot/命令行模式截图2.png" alt="">
<h1>🖥 服务器模式</h1>
<p><b>启动:</b>运行命令:<code>python .\main.py server</code></p>
<p><b>关闭:</b>按下 <code>Ctrl</code> + <code>C</code> 关闭服务器</p>
<p><b>请求接口:</b><code>/xhs/</code></p>
<p><b>请求类型:</b><code>POST</code></p>
<p><b>请求参数:</b></p>
<table>
<thead>
<tr>
<th align="center">参数</th>
<th align="center">类型</th>
<th align="center">含义</th>
<th align="center">默认值</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center">url</td>
<td align="center">str</td>
<td align="center">小红书作品链接,自动提取,不支持多链接</td>
<td align="center">无</td>
</tr>
<tr>
<td align="center">download</td>
<td align="center">bool</td>
<td align="center">是否下载作品文件;设置为 <code>true</code> 将会耗费更多时间</td>
<td align="center">false</td>
</tr>
<tr>
<td align="center">index</td>
<td align="center">str</td>
<td align="center">下载指定序号的图片文件,仅对图文作品生效;多个序号之间使用空格分隔;<code>download</code> 参数设置为 <code>false</code> 时不生效</td>
<td align="center">null</td>
</tr>
<tr>
<td align="center">skip</td>
<td align="center">bool</td>
<td align="center">是否跳过存在下载记录的作品;设置为 <code>true</code> 将不会返回存在下载记录的作品数据</td>
<td align="center">false</td>
</tr>
</tbody>
</table>
<p><b>代码示例:</b></p>
<pre>
def api_demo():
server = "http://127.0.0.1:8080"
data = {
"url": "https://www.xiaohongshu.com/explore/123456789",
}
response = requests.post(server, data=data)
print(response.json())
</pre>
<h1>🕹 用户脚本</h1>
<img src="static/screenshot/用户脚本截图1.png" alt="">
<hr>
Expand Down Expand Up @@ -235,6 +287,7 @@ async def example():
<h1>🌐 Cookie</h1>
<ol>
<li>打开浏览器(可选无痕模式启动),访问 <code>https://www.xiaohongshu.com/explore</code></li>
<li>登录小红书账号(可跳过)</li>
<li>按下 <code>F12</code> 打开开发人员工具</li>
<li>选择 <code>网络</code> 选项卡</li>
<li>勾选 <code>保留日志</code></li>
Expand Down
Binary file modified locale/en_GB/LC_MESSAGES/xhs.mo
Binary file not shown.
15 changes: 15 additions & 0 deletions locale/en_GB/LC_MESSAGES/xhs.po
Original file line number Diff line number Diff line change
Expand Up @@ -255,3 +255,18 @@ msgstr "Format of works file name"

msgid "邀请链接:"
msgstr "Invitation link: "

msgid "获取小红书作品数据成功"
msgstr "Successfully obtained data on Xiaohongshu's works"

msgid "获取小红书作品数据失败"
msgstr "Failed to obtain data on Xiaohongshu's works"

msgid "Web API 服务器已启动!"
msgstr "Web API server started!"

msgid "Web API 服务器已关闭!"
msgstr "Web API server has been shut down!"

msgid "服务器主机及端口: {0}"
msgstr "Server host and port: {0}"
15 changes: 15 additions & 0 deletions locale/zh_CN/LC_MESSAGES/xhs.po
Original file line number Diff line number Diff line change
Expand Up @@ -255,3 +255,18 @@ msgstr ""

msgid "邀请链接:"
msgstr ""

msgid "获取小红书作品数据成功"
msgstr ""

msgid "获取小红书作品数据失败"
msgstr ""

msgid "Web API 服务器已启动!"
msgstr ""

msgid "Web API 服务器已关闭!"
msgstr ""

msgid "服务器主机及端口: {0}"
msgstr ""
16 changes: 12 additions & 4 deletions main.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from asyncio import run
from sys import argv

from source import Settings
from source import XHS
from source import XHSDownloader
from source import cli
Expand Down Expand Up @@ -47,13 +48,20 @@ async def example():
print(await xhs.extract(multiple_links, download, ))


async def main():
async def app():
async with XHSDownloader() as xhs:
await xhs.run_async()


async def server():
async with XHS(**Settings().run()) as xhs:
await xhs.run_server()


if __name__ == '__main__':
if len(argv) > 1:
cli()
if len(argv) == 1:
run(app())
elif argv[1] == "server":
run(server())
else:
run(main())
cli()
2 changes: 1 addition & 1 deletion source/TUI/setting.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ def save_settings(self):
"language": self.query_one("#language").value,
"image_download": self.query_one("#image_download").value,
"video_download": self.query_one("#video_download").value,
"server": False,
# "server": False,
})

@on(Button.Pressed, "#abandon")
Expand Down
3 changes: 2 additions & 1 deletion source/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from .CLI import cli
from .TUI import XHSDownloader
from .application import XHS
from .module import Settings

__all__ = ['XHS', 'XHSDownloader', 'cli']
__all__ = ['XHS', 'XHSDownloader', 'cli', 'Settings', ]
56 changes: 54 additions & 2 deletions source/application/app.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from asyncio import CancelledError
from asyncio import Event
from asyncio import Queue
from asyncio import QueueEmpty
Expand All @@ -9,6 +10,7 @@
from typing import Callable
from urllib.parse import urlparse

from aiohttp import web
from pyperclip import paste

from source.expansion import BrowserCookie
Expand All @@ -22,6 +24,7 @@
ERROR,
WARNING,
MASTER,
REPOSITORY,
)
from source.module import Translate
from source.module import logging
Expand Down Expand Up @@ -62,7 +65,7 @@ def __init__(
video_download=True,
folder_mode=False,
language="zh_CN",
server=False,
# server=False,
transition: Callable[[str], str] = None,
read_cookie: int | str = None,
*args,
Expand All @@ -85,6 +88,7 @@ def __init__(
image_download,
video_download,
folder_mode,
# server,
self.message,
)
self.html = Html(self.manager)
Expand All @@ -98,7 +102,8 @@ def __init__(
self.clipboard_cache: str = ""
self.queue = Queue()
self.event = Event()
self.server = server
self.runner = self.init_server()
self.site = None

def __extract_image(self, container: dict, data: Namespace):
container["下载地址"] = self.image.get_image_link(
Expand Down Expand Up @@ -279,3 +284,50 @@ async def close(self):
def read_browser_cookie(value: str | int) -> str:
return BrowserCookie.get(
value, domain="xiaohongshu.com") if value else ""

@staticmethod
async def index(request):
return web.HTTPFound(REPOSITORY)

async def handle(self, request):
data = await request.post()
url = data.get("url")
download = data.get("download", False)
index = data.get("index")
skip = data.get("skip", False)
url = await self.__extract_links(url, None)
if not url:
msg = self.message("提取小红书作品链接失败")
data = None
else:
if data := await self.__deal_extract(url[0], download, index, None, None, not skip, ):
msg = self.message("获取小红书作品数据成功")
else:
msg = self.message("获取小红书作品数据失败")
data = None
return web.json_response(dict(message=msg, url=url[0], data=data))

def init_server(self, ):
app = web.Application(debug=True)
app.router.add_get('/', self.index)
app.router.add_post('/xhs/', self.handle)
return web.AppRunner(app)

async def run_server(self, log=None, ):
try:
await self.start_server(log)
while True:
await sleep(3600) # 保持服务器运行
except (CancelledError, KeyboardInterrupt):
await self.close_server(log)

async def start_server(self, log=None, ):
await self.runner.setup()
self.site = web.TCPSite(self.runner, "0.0.0.0")
await self.site.start()
logging(log, self.message("Web API 服务器已启动!"))
logging(log, self.message("服务器主机及端口: {0}".format(self.site.name, )))

async def close_server(self, log=None, ):
await self.runner.cleanup()
logging(log, self.message("Web API 服务器已关闭!"))
2 changes: 1 addition & 1 deletion source/application/video.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ class Video:

@classmethod
def get_video_link(cls, data: Namespace) -> list:
return [Html.format_url(f"https://sns-video-hw.xhscdn.com/{t}")] if (
return [Html.format_url(f"https://sns-video-bd.xhscdn.com/{t}")] if (
t := data.safe_extract(".".join(cls.VIDEO_LINK))) else []
2 changes: 2 additions & 0 deletions source/module/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ def __init__(
image_download: bool,
video_download: bool,
folder_mode: bool,
# server: bool,
transition: Callable[[str], str],
):
self.root = root
Expand Down Expand Up @@ -77,6 +78,7 @@ def __init__(
self.message = transition
self.image_download = self.check_bool(image_download, True)
self.video_download = self.check_bool(video_download, True)
# self.server = self.check_bool(server, False)

def __check_path(self, path: str) -> Path:
if not path:
Expand Down
6 changes: 4 additions & 2 deletions source/module/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
from pathlib import Path
from platform import system

from .static import ROOT

__all__ = ['Settings']


Expand All @@ -23,11 +25,11 @@ class Settings:
"video_download": True,
"folder_mode": False,
"language": "zh_CN",
"server": False,
# "server": False,
}
encode = "UTF-8-SIG" if system() == "Windows" else "UTF-8"

def __init__(self, root: Path):
def __init__(self, root: Path = ROOT):
self.file = root.joinpath("./settings.json")

def run(self):
Expand Down
6 changes: 3 additions & 3 deletions static/XHS-Downloader.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// ==UserScript==
// @name XHS-Downloader
// @namespace https://github.com/JoeanAmier/XHS-Downloader
// @version 1.4.3
// @version 1.4.4
// @description 提取小红书作品/用户链接,下载小红书无水印图文/视频作品文件
// @author JoeanAmier
// @match http*://xhslink.com/*
Expand Down Expand Up @@ -42,7 +42,7 @@
2. 提取账号发布、收藏、点赞作品链接时,脚本会尝试自动滚动屏幕直至加载全部作品,滚动检测间隔:2.5 秒
3. 提取搜索结果作品、用户链接时,脚本会自动滚动屏幕以尝试加载更多内容,滚动屏幕次数:10 次
4. 可以修改滚动检测间隔、滚动屏幕次数,修改后立即生效;亦可关闭自动滚动屏幕功能,手动滚动屏幕加载内容
5. XHS-Downloader 用户脚本仅实现可见即可得的数据采集功能,无任何破解功能
5. XHS-Downloader 用户脚本仅实现可见即可得的数据采集功能,无任何收费功能和破解功能
项目开源地址:https://github.com/JoeanAmier/XHS-Downloader
`
Expand Down Expand Up @@ -132,7 +132,7 @@

const generateVideoUrl = note => {
try {
return [`https://sns-video-hw.xhscdn.com/${note.video.consumer.originVideoKey}`];
return [`https://sns-video-bd.xhscdn.com/${note.video.consumer.originVideoKey}`];
} catch (error) {
console.error("Error generating video URL:", error);
return [];
Expand Down

0 comments on commit bd0780c

Please sign in to comment.