Compare commits

...

143 Commits

Author SHA1 Message Date
jxxghp
9f5e1b8dd7 更新 version.py 2025-06-14 14:45:58 +08:00
jxxghp
c86ed20c34 fix 2025-06-14 08:23:48 +08:00
jxxghp
c32c37e66a Merge pull request #4444 from cddjr/fix_doh_reload 2025-06-14 08:22:13 +08:00
jxxghp
7b100d3cdb Merge pull request #4446 from wikrin/v2 2025-06-14 07:05:20 +08:00
Attente
95a2362885 fix(db): 修复系统配置更新时内存共享问题
- 在更新系统配置时,使用 deepcopy 复制新值以避免内存共享
2025-06-13 23:03:13 +08:00
jxxghp
d8b14b9a9f Merge pull request #4445 from cddjr/feat_nettest 2025-06-13 19:06:02 +08:00
景大侠
c45953f63a feat 网络测试支持加速代理以及GitHub Token
fix 测试耗时大于1秒时,时间差计算错误
2025-06-13 18:35:49 +08:00
景大侠
e3d3087a5d fix GitHub请求头补上UA 2025-06-13 18:06:17 +08:00
景大侠
e162bd1168 fix DoH热加载 2025-06-13 17:43:45 +08:00
jxxghp
db5d81d7f0 Merge pull request #4442 from wumode/fix_download_api 2025-06-13 14:24:22 +08:00
wumode
f737f1287b fix(api): 无法设置非默认下载器状态 2025-06-13 08:43:34 +08:00
jxxghp
1ffa5178db Merge pull request #4440 from wikrin/v2 2025-06-13 06:35:24 +08:00
Attente
49cb43488c feat(plugin): 优化插件同步和安装逻辑
- 优化 sync 函数,考虑插件版本因素
- 更新 is_plugin_exists 函数,增加版本比较
2025-06-13 00:19:08 +08:00
jxxghp
fd7a6f8ddd Merge pull request #4438 from H1dery/v2 2025-06-12 20:05:00 +08:00
Cais1
7979ce0f0a File reading fixes
File reading fixes
2025-06-12 19:58:47 +08:00
Cais1
2ba5d9484d Update plugin.py
File reading fixes
2025-06-12 19:57:26 +08:00
jxxghp
23b981c5ac fix #4434 2025-06-12 18:41:46 +08:00
jxxghp
86ab2c8c05 Merge pull request #4434 from alfchao/v2 2025-06-12 16:14:59 +08:00
xuchao3
9ea0bc609a feat:增加telegram api代理地址
#4266
2025-06-12 13:56:36 +08:00
jxxghp
5366c2844a Merge pull request #4433 from wikrin/v2 2025-06-12 08:47:56 +08:00
Attente
eac4d703c7 fix(plugins_initializer): 优化插件恢复的容错处理
- 添加单个插件恢复失败的异常处理,使用 continue 跳过
- 确保单个插件恢复失败不影响其他插件继续恢复
2025-06-12 07:56:44 +08:00
jxxghp
8ed87294e2 v2.5.5-1
- 修复下载器监控问题
2025-06-12 07:08:19 +08:00
jxxghp
b343c601be v2.5.5
- 支持更精细的用户权限控制
- 高级设置中增加了刮削内容设定
2025-06-11 20:27:49 +08:00
jxxghp
e56d7006b4 init users 2025-06-11 20:24:59 +08:00
jxxghp
1b7bcd7784 init users 2025-06-11 19:57:21 +08:00
jxxghp
4cb9025b6c fix season_nfo 2025-06-11 19:48:02 +08:00
jxxghp
f8864ab053 fix reload 2025-06-11 07:11:50 +08:00
jxxghp
64eba46a67 fix 2025-06-11 07:07:55 +08:00
jxxghp
35d9cc1d40 remove jiaba 2025-06-11 00:00:08 +08:00
jxxghp
3036107dac fix user api 2025-06-10 23:42:57 +08:00
jxxghp
214089b4ea Merge pull request #4423 from lonelyman0108/v2 2025-06-10 18:04:13 +08:00
LM
95b7ba28e4 update: 添加fanart环境变量 2025-06-10 17:59:25 +08:00
LM
880272f96e update: 优化fanart获取逻辑,支持设定语言 2025-06-10 17:59:03 +08:00
LM
7ed26fadb6 update: 更新fanart刮削逻辑,优先获取中文、英文内容 2025-06-10 17:25:58 +08:00
jxxghp
f0d25a02a6 feat:支持刮削详细设定 2025-06-10 16:37:15 +08:00
jxxghp
162ba9307d fix restart 2025-06-10 07:09:59 +08:00
jxxghp
49dae92b8e fix flag path 2025-06-09 21:58:02 +08:00
jxxghp
b484a52b6d v2.5.4
- 插件市场支持手动刷新
- 优化了重置容器时已安装插件的恢复策略
2025-06-09 20:57:44 +08:00
jxxghp
d754091a7c fix log 2025-06-09 20:44:48 +08:00
jxxghp
e2febc24ae feat:插件市场支持强制刷新 2025-06-09 20:33:06 +08:00
jxxghp
d0677edaaa fix 优雅停止 2025-06-09 15:39:11 +08:00
jxxghp
f0aaecd0c7 fix #4413 2025-06-09 14:45:26 +08:00
jxxghp
3518940fec Merge pull request #4413 from cddjr/fix_plugin
修复分身的一些BUG
2025-06-09 14:42:54 +08:00
jxxghp
2e5c92ae0c fix 优雅停止 2025-06-09 13:09:16 +08:00
jxxghp
4ad699dbe6 fix 优雅停止 2025-06-09 13:06:27 +08:00
景大侠
931be9e6aa fix 分身复用原插件配置 2025-06-09 09:54:55 +08:00
景大侠
9656d6fbd0 fix 分身类名使用小写后缀
避免与分身ID不一致,导致误判没有安装
2025-06-09 09:51:06 +08:00
景大侠
c7cbb13044 fix 插件卸载后从系统模块中移除
避免分身时误报插件已存在
2025-06-09 09:50:55 +08:00
jxxghp
327d30dcc2 feat:识别容器是否重置 2025-06-09 09:15:58 +08:00
jxxghp
e4e2079917 fix:插件恢复安全性 2025-06-09 08:30:24 +08:00
jxxghp
0427506572 fix:移除Action类静态属性 2025-06-09 08:18:43 +08:00
jxxghp
ea168edb43 fix:移除Oper类静态属性 2025-06-09 08:08:55 +08:00
jxxghp
aa039c6c05 feat:启停插件自动备份与恢复 2025-06-09 08:04:44 +08:00
jxxghp
3de998051a fix memory snapshot 2025-06-08 21:57:49 +08:00
jxxghp
69ade1ae37 更新内存快照间隔为30分钟,保留的内存快照文件数量减少至20个 2025-06-08 21:48:37 +08:00
jxxghp
1d6133e3b1 fix plugins遍历 2025-06-08 21:39:37 +08:00
jxxghp
203a111d1a remove gc 2025-06-08 21:24:26 +08:00
jxxghp
0a20234268 remove gc 2025-06-08 21:19:15 +08:00
jxxghp
7f8e50f83d fix memory helper 2025-06-08 21:13:37 +08:00
jxxghp
443ef7d41b fix 2025-06-08 21:06:27 +08:00
jxxghp
059ae6595d fix 2025-06-08 20:37:42 +08:00
jxxghp
19c3dad338 fix 2025-06-08 19:41:46 +08:00
jxxghp
81bc51c972 fix pympler 2025-06-08 19:02:25 +08:00
jxxghp
6c17868744 add pympler 2025-06-08 18:55:02 +08:00
jxxghp
a18040ccfa add pympler 2025-06-08 18:54:35 +08:00
jxxghp
0835a75503 更新 thread.py 2025-06-08 14:43:13 +08:00
jxxghp
3ee32757e5 rollback 2025-06-08 14:35:59 +08:00
jxxghp
344abfa8d8 fix memory helper 2025-06-08 14:03:01 +08:00
jxxghp
906b2a3485 fix memory statistics 2025-06-08 11:36:15 +08:00
jxxghp
e0d2b87ed3 wallpaper cache skip empty 2025-06-08 11:30:57 +08:00
jxxghp
83a8c8b42b fix memory threshold 2025-06-08 11:14:16 +08:00
jxxghp
d840ed6c5a fix memory log 2025-06-08 11:08:01 +08:00
jxxghp
0112087be4 refactor #4407 2025-06-08 10:51:59 +08:00
jxxghp
7320084e11 rollback #4379 2025-06-07 22:26:51 +08:00
jxxghp
23929f5eaa fix pool size 2025-06-07 22:00:09 +08:00
jxxghp
c002d4619a 更新 scheduler.py 2025-06-07 20:11:31 +08:00
jxxghp
f60a909bba 更新 version.py 2025-06-07 11:43:04 +08:00
jxxghp
c2c22e3968 Merge pull request #4399 from cddjr/fix_subscribe 2025-06-07 11:42:25 +08:00
jxxghp
f10299b2de Merge pull request #4403 from cddjr/fix_systemconfig 2025-06-07 11:41:36 +08:00
景大侠
1d3563ed97 fix(config): 修复新装的插件会消失的问题 2025-06-07 11:33:28 +08:00
景大侠
f3eb2caa4e fix(subscribe): 避免重复下载已入库的剧集 2025-06-07 02:48:22 +08:00
jxxghp
2364dacd52 添加对 GitHub 容器注册 2025-06-06 22:02:04 +08:00
jxxghp
883f7451c3 fix event log 2025-06-06 21:45:14 +08:00
jxxghp
a534c9bca1 fix 设置保存失败提示 2025-06-06 21:30:11 +08:00
jxxghp
b14202a324 fix logger 2025-06-06 21:18:31 +08:00
jxxghp
a6fae48f07 更新 system.py 2025-06-06 17:15:25 +08:00
jxxghp
963caf2afe fix logger reload 2025-06-06 16:31:00 +08:00
jxxghp
50b0268531 v2.5.3-1 2025-06-06 15:37:44 +08:00
jxxghp
f484b64be3 fix 2025-06-06 15:37:02 +08:00
jxxghp
349535557f 更新 subscribe.py 2025-06-06 14:04:12 +08:00
jxxghp
de4973a270 feat:内存监控开关 2025-06-06 13:49:52 +08:00
jxxghp
e42d2baf8a fix lint 2025-06-05 22:14:14 +08:00
jxxghp
eac435b233 fix lint 2025-06-05 22:13:33 +08:00
jxxghp
447b8564e9 更新 GitHub Actions 工作流 2025-06-05 22:02:52 +08:00
jxxghp
97cee657bd 更新 .gitignore 文件以包含 Pylint 相关文件,并修改 system.py 中的成功返回逻辑 2025-06-05 21:58:50 +08:00
jxxghp
fe894754cf 更新 system.py 2025-06-05 21:39:13 +08:00
jxxghp
9ffb1d1931 更新 wallpaper.py 2025-06-05 21:03:21 +08:00
jxxghp
a16bd30903 更新 wallpaper.py 2025-06-05 21:00:18 +08:00
jxxghp
13f9ea8be4 v2.5.3 2025-06-05 20:28:43 +08:00
jxxghp
304af5e980 fix:仪表盘内存只显示当前程序占用 2025-06-05 17:09:11 +08:00
jxxghp
dc180c09e9 fix wallpaper 2025-06-05 17:03:29 +08:00
jxxghp
8e20e26565 fix:捕捉插件停止异常 2025-06-05 14:07:31 +08:00
jxxghp
11075a4012 fix:增加更多内存控制 2025-06-05 13:33:39 +08:00
jxxghp
a9300faaf8 fix:优化单例模式和类引用 2025-06-05 13:22:16 +08:00
jxxghp
504827b7e5 fix:memory use 2025-06-05 09:57:41 +08:00
jxxghp
e180130b38 fix:memory use 2025-06-05 08:32:24 +08:00
jxxghp
faaee09827 fix:memory use 2025-06-05 08:18:26 +08:00
jxxghp
99334795b6 fix rsshelper 2025-06-04 22:00:46 +08:00
jxxghp
8c9c59ef64 fix rsshelper 2025-06-04 21:42:03 +08:00
jxxghp
7a112000c9 更新 memory.py 2025-06-04 18:46:55 +08:00
jxxghp
1424087d5a fix:memory use 2025-06-04 18:34:49 +08:00
jxxghp
984f4731cd 更新 log.py 2025-06-04 15:33:58 +08:00
jxxghp
3a3de64b0f fix:重构配置热加载 2025-06-04 08:21:14 +08:00
jxxghp
0911854e9d fix Config reload 2025-06-04 07:17:47 +08:00
jxxghp
2af8b6f445 fix Config reload 2025-06-03 23:10:48 +08:00
jxxghp
bbfd8ca3f5 fix Config reload 2025-06-03 23:08:58 +08:00
jxxghp
b4ed2880f7 refactor:重构配置热加载 2025-06-03 20:56:21 +08:00
jxxghp
5f18a21e86 fix:整理失败时也打上已整理标签 2025-06-03 17:48:30 +08:00
jxxghp
5d188e3877 fix module close 2025-06-03 17:11:44 +08:00
jxxghp
90f113a292 remove ttl cache 2025-06-03 16:31:16 +08:00
jxxghp
eecfe58297 fix memory manager startup 2025-06-03 16:27:51 +08:00
jxxghp
079a747210 fix memory manager startup 2025-06-03 16:19:38 +08:00
jxxghp
4be8c70f23 fix memory log 2025-06-03 16:05:49 +08:00
jxxghp
d9aee4df77 fix memory log 2025-06-03 16:03:05 +08:00
jxxghp
225de87d4d fix torrents chain 2025-06-03 15:48:43 +08:00
jxxghp
2ce7cedfbd fix 2025-06-03 12:30:26 +08:00
jxxghp
cfb163d904 fix 2025-06-03 12:27:50 +08:00
jxxghp
de7c9be11b 优化内存管理,增加最大内存配置项,改进内存使用检查逻辑。 2025-06-03 12:25:13 +08:00
jxxghp
841209adc9 fix 2025-06-03 11:49:16 +08:00
jxxghp
e48d51fe6e 优化内存管理和垃圾回收机制 2025-06-03 11:45:17 +08:00
jxxghp
9d436ec7ed fix #4382 2025-06-03 08:19:15 +08:00
jxxghp
fb2b29d088 fix #4382 2025-06-03 07:07:40 +08:00
jxxghp
1c46b0bc20 更新 subscribe.py 2025-06-02 16:23:09 +08:00
jxxghp
81d0e4696a Merge pull request #4379 from jtcymc/v2 2025-06-02 10:48:36 +08:00
shaw
f9a287b52b feat(core): 增加剧集交集最小置信度设置
新增了剧集交集最小置信度的配置项,用于过滤掉包含过多不需要剧集的种子。实现了以下功能:

- 在 config.py 中添加了 EPISODE_INTERSECTION_MIN_CONFIDENCE 配置项,默认值为 0.0
- 修改了 download.py 中的下载逻辑,增加了计算种子与目标缺失集之间交集比例的函数
- 使用交集比例来筛选和排序种子,优先下载与缺失集交集较大的种子
-可以通过配置项设置交集比例的阈值,低于阈值的种子将被跳过

这个改动可以提高下载效率,避免下载过多不必要的剧集。
2025-06-02 00:38:10 +08:00
jxxghp
0f0072abea Merge pull request #4375 from awsl1110/v2 2025-05-31 20:08:10 +08:00
awsl1110
312933a259 fix(indexer): 修正 DiscuzX 站点名称
- 将 Discuz! 站点名称修改为 DiscuzX
2025-05-31 19:18:25 +08:00
jxxghp
288854b8f1 Merge pull request #4374 from awsl1110/v2 2025-05-31 19:04:51 +08:00
awsl1110
7f5991aa34 refactor(core): 优化配置项和模型定义
- 为配置项添加类型注解,提高代码可读性和安全性
- 为模型字段添加默认值,优化数据处理
- 更新验证器使用新语法,以适应Pydantic库的变更
2025-05-31 16:38:06 +08:00
jxxghp
361df95d50 Merge pull request #4372 from cddjr/fix_4371 2025-05-31 13:34:48 +08:00
景大侠
fc1ade32d7 更新蓝光测试用例 2025-05-31 11:05:02 +08:00
景大侠
b74c7531d9 fix #4371 递归判断蓝光目录 2025-05-31 02:37:14 +08:00
景大侠
7e3be3325a fix #4294 更新测试用例 2025-05-31 01:52:31 +08:00
129 changed files with 3288 additions and 1397 deletions

View File

@@ -25,7 +25,9 @@ jobs:
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ secrets.DOCKER_USERNAME }}/moviepilot-v2
images: |
${{ secrets.DOCKER_USERNAME }}/moviepilot-v2
ghcr.io/${{ github.repository }}
tags: |
type=raw,value=${{ env.app_version }}
type=raw,value=latest
@@ -42,6 +44,13 @@ jobs:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build Image
uses: docker/build-push-action@v5
with:

30
.github/workflows/issues.yml vendored Normal file
View File

@@ -0,0 +1,30 @@
name: Close inactive issues
on:
workflow_dispatch:
schedule:
# Github Action 只支持 UTC 时间。
# '0 18 * * *' 对应 UTC 时间的 18:00也就是中国时区 (UTC+8) 的第二天凌晨 02:00。
- cron: "0 18 * * *"
jobs:
close-issues:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v5
with:
# 标记 stale 标签时间
days-before-issue-stale: 30
# 关闭 issues 标签时间
days-before-issue-close: 14
# 自定义标签名
stale-issue-label: "stale"
stale-issue-message: "此问题已过时,因为它已打开 30 天且没有任何活动。"
close-issue-message: "此问题已关闭,因为它在标记为 stale 后,已处于无更新状态 14 天。"
# 忽略所有的 Pull Request只处理 Issue
days-before-pr-stale: -1
days-before-pr-close: -1
repo-token: ${{ secrets.GITHUB_TOKEN }}

91
.github/workflows/pylint.yml vendored Normal file
View File

@@ -0,0 +1,91 @@
name: Pylint Code Quality Check
on:
# 允许手动触发
workflow_dispatch:
jobs:
pylint:
runs-on: ubuntu-latest
name: Pylint Code Quality Check
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/requirements.in') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install pylint
# 安装项目依赖
if [ -f requirements.txt ]; then
echo "📦 安装 requirements.txt 中的依赖..."
pip install -r requirements.txt
elif [ -f requirements.in ]; then
echo "📦 安装 requirements.in 中的依赖..."
pip install -r requirements.in
else
echo "⚠️ 未找到依赖文件,仅安装 pylint"
fi
- name: Verify pylint config
run: |
# 检查项目中的pylint配置文件是否存在
if [ -f .pylintrc ]; then
echo "✅ 找到项目配置文件: .pylintrc"
echo "配置文件内容预览:"
head -10 .pylintrc
else
echo "❌ 未找到 .pylintrc 配置文件"
exit 1
fi
- name: Run pylint
run: |
# 运行pylint检查主要的Python文件
echo "🚀 运行 Pylint 错误检查..."
# 检查主要目录 - 只关注错误,如果有错误则退出
echo "📂 检查 app/ 目录..."
pylint app/ --output-format=colorized --reports=yes --score=yes
# 检查根目录的Python文件
echo "📂 检查根目录 Python 文件..."
for file in $(find . -name "*.py" -not -path "./.*" -not -path "./.venv/*" -not -path "./build/*" -not -path "./dist/*" -not -path "./tests/*" -not -path "./docs/*" -not -path "./__pycache__/*" -maxdepth 1); do
echo "检查文件: $file"
pylint "$file" --output-format=colorized || exit 1
done
# 生成详细报告
echo "📊 生成 Pylint 详细报告..."
pylint app/ --output-format=json > pylint-report.json || true
# 显示评分(仅供参考)
echo "📈 Pylint 评分(仅供参考):"
pylint app/ --score=yes --reports=no | tail -2 || true
- name: Upload pylint report
uses: actions/upload-artifact@v4
if: always()
with:
name: pylint-report
path: pylint-report.json
- name: Summary
run: |
echo "🎉 Pylint 检查完成!"
echo "✅ 没有发现语法错误或严重问题"
echo "📊 详细报告已保存为构建工件"

6
.gitignore vendored
View File

@@ -23,4 +23,8 @@ config/cache/
*.pyc
*.log
.vscode
venv
venv
# Pylint
pylint-report.json
.pylint.d/

83
.pylintrc Normal file
View File

@@ -0,0 +1,83 @@
[MASTER]
# 指定Python路径
init-hook='import sys; sys.path.append(".")'
# 忽略的文件和目录
ignore=.git,__pycache__,.venv,build,dist,tests,docs
# 并行作业数量
jobs=0
[MESSAGES CONTROL]
# 只关注错误级别的问题,禁用警告、约定和重构建议
# E = Error (错误) - 会导致构建失败
# W = Warning (警告) - 仅显示,不会失败
# R = Refactor (重构建议) - 仅显示,不会失败
# C = Convention (约定) - 仅显示,不会失败
# I = Information (信息) - 仅显示,不会失败
# 禁用大部分警告、约定和重构建议,只保留错误和重要警告
disable=all
enable=error,
syntax-error,
undefined-variable,
used-before-assignment,
unreachable,
return-outside-function,
yield-outside-function,
continue-in-finally,
nonlocal-without-binding,
undefined-loop-variable,
redefined-builtin,
not-callable,
assignment-from-no-return,
no-value-for-parameter,
too-many-function-args,
unexpected-keyword-arg,
redundant-keyword-arg,
import-error,
relative-beyond-top-level
[REPORTS]
# 设置报告格式
output-format=colorized
reports=yes
score=yes
[FORMAT]
# 最大行长度
max-line-length=120
# 缩进大小
indent-string=' '
[DESIGN]
# 最大参数数量
max-args=10
# 最大本地变量数量
max-locals=20
# 最大分支数量
max-branches=15
# 最大语句数量
max-statements=50
# 最大父类数量
max-parents=7
# 最大属性数量
max-attributes=10
# 最小公共方法数量
min-public-methods=1
# 最大公共方法数量
max-public-methods=25
[SIMILARITIES]
# 最小相似行数
min-similarity-lines=6
# 忽略注释
ignore-comments=yes
# 忽略文档字符串
ignore-docstrings=yes
# 忽略导入
ignore-imports=yes
[TYPECHECK]
# 生成缺失成员提示的类列表
generated-members=requests.packages.urllib3

View File

@@ -26,37 +26,31 @@ class AddDownloadAction(BaseAction):
添加下载资源
"""
# 已添加的下载
_added_downloads = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.downloadchain = DownloadChain()
self.mediachain = MediaChain()
self._added_downloads = []
self._has_error = False
@classmethod
@property
def name(cls) -> str: # noqa
def name(cls) -> str: # noqa
return "添加下载"
@classmethod
@property
def description(cls) -> str: # noqa
def description(cls) -> str: # noqa
return "根据资源列表添加下载任务"
@classmethod
@property
def data(cls) -> dict: # noqa
def data(cls) -> dict: # noqa
return AddDownloadParams().dict()
@property
def success(self) -> bool:
return not self._has_error
def execute(self, workflow_id: int, params: dict, context: ActionContext) -> ActionContext:
def execute(self, workflow_id: int, params: dict, context: ActionContext) -> ActionContext:
"""
将上下文中的torrents添加到下载任务中
"""
@@ -73,13 +67,13 @@ class AddDownloadAction(BaseAction):
if not t.meta_info:
t.meta_info = MetaInfo(title=t.torrent_info.title, subtitle=t.torrent_info.description)
if not t.media_info:
t.media_info = self.mediachain.recognize_media(meta=t.meta_info)
t.media_info = MediaChain().recognize_media(meta=t.meta_info)
if not t.media_info:
self._has_error = True
logger.warning(f"{t.torrent_info.title} 未识别到媒体信息,无法下载")
continue
if params.only_lack:
exists_info = self.downloadchain.media_exists(t.media_info)
exists_info = DownloadChain().media_exists(t.media_info)
if exists_info:
if t.media_info.type == MediaType.MOVIE:
# 电影
@@ -96,14 +90,15 @@ class AddDownloadAction(BaseAction):
exists_episodes = exists_seasons.get(t.meta_info.begin_season)
if exists_episodes:
if set(t.meta_info.episode_list).issubset(exists_episodes):
logger.warning(f"{t.meta_info.title}{t.meta_info.begin_season} 季第 {t.meta_info.episode_list} 集已存在,跳过")
logger.warning(
f"{t.meta_info.title}{t.meta_info.begin_season} 季第 {t.meta_info.episode_list} 集已存在,跳过")
continue
_started = True
did = self.downloadchain.download_single(context=t,
downloader=params.downloader,
save_path=params.save_path,
label=params.labels)
did = DownloadChain().download_single(context=t,
downloader=params.downloader,
save_path=params.save_path,
label=params.labels)
if did:
self._added_downloads.append(did)
# 保存缓存

View File

@@ -19,29 +19,24 @@ class AddSubscribeAction(BaseAction):
添加订阅
"""
_added_subscribes = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.subscribechain = SubscribeChain()
self.subscribeoper = SubscribeOper()
self._added_subscribes = []
self._has_error = False
@classmethod
@property
def name(cls) -> str: # noqa
def name(cls) -> str: # noqa
return "添加订阅"
@classmethod
@property
def description(cls) -> str: # noqa
def description(cls) -> str: # noqa
return "根据媒体列表添加订阅"
@classmethod
@property
def data(cls) -> dict: # noqa
def data(cls) -> dict: # noqa
return AddSubscribeParams().dict()
@property
@@ -63,19 +58,20 @@ class AddSubscribeAction(BaseAction):
continue
mediainfo = MediaInfo()
mediainfo.from_dict(media.dict())
if self.subscribechain.exists(mediainfo):
subscribechain = SubscribeChain()
if subscribechain.exists(mediainfo):
logger.info(f"{media.title} 已存在订阅")
continue
# 添加订阅
_started = True
sid, message = self.subscribechain.add(mtype=mediainfo.type,
title=mediainfo.title,
year=mediainfo.year,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season,
doubanid=mediainfo.douban_id,
bangumiid=mediainfo.bangumi_id,
username=settings.SUPERUSER)
sid, message = subscribechain.add(mtype=mediainfo.type,
title=mediainfo.title,
year=mediainfo.year,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season,
doubanid=mediainfo.douban_id,
bangumiid=mediainfo.bangumi_id,
username=settings.SUPERUSER)
if sid:
self._added_subscribes.append(sid)
# 保存缓存
@@ -84,7 +80,7 @@ class AddSubscribeAction(BaseAction):
if self._added_subscribes:
logger.info(f"已添加 {len(self._added_subscribes)} 个订阅")
for sid in self._added_subscribes:
context.subscribes.append(self.subscribeoper.get(sid))
context.subscribes.append(SubscribeOper().get(sid))
elif _started:
self._has_error = True

View File

@@ -16,11 +16,8 @@ class FetchDownloadsAction(BaseAction):
获取下载任务
"""
_downloads = []
def __init__(self, action_id: str):
super().__init__(action_id)
self.chain = ActionChain()
self._downloads = []
@classmethod
@@ -51,7 +48,7 @@ class FetchDownloadsAction(BaseAction):
if global_vars.is_workflow_stopped(workflow_id):
break
logger.info(f"获取下载任务 {download.download_id} 状态 ...")
torrents = self.chain.list_torrents(hashs=[download.download_id])
torrents = ActionChain().list_torrents(hashs=[download.download_id])
if not torrents:
download.completed = True
continue

View File

@@ -27,10 +27,6 @@ class FetchMediasAction(BaseAction):
获取媒体数据
"""
_inner_sources = []
_medias = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)

View File

@@ -29,29 +29,24 @@ class FetchRssAction(BaseAction):
获取RSS资源列表
"""
_rss_torrents = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.rsshelper = RssHelper()
self.chain = ActionChain()
self._rss_torrents = []
self._has_error = False
@classmethod
@property
def name(cls) -> str: # noqa
def name(cls) -> str: # noqa
return "获取RSS资源"
@classmethod
@property
def description(cls) -> str: # noqa
def description(cls) -> str: # noqa
return "订阅RSS地址获取资源"
@classmethod
@property
def data(cls) -> dict: # noqa
def data(cls) -> dict: # noqa
return FetchRssParams().dict()
@property
@@ -74,10 +69,10 @@ class FetchRssAction(BaseAction):
if params.ua:
headers["User-Agent"] = params.ua
rss_items = self.rsshelper.parse(url=params.url,
proxy=settings.PROXY if params.proxy else None,
timeout=params.timeout,
headers=headers)
rss_items = RssHelper().parse(url=params.url,
proxy=settings.PROXY if params.proxy else None,
timeout=params.timeout,
headers=headers)
if rss_items is None or rss_items is False:
logger.error(f'RSS地址 {params.url} 请求失败!')
self._has_error = True
@@ -103,7 +98,7 @@ class FetchRssAction(BaseAction):
meta = MetaInfo(title=torrentinfo.title, subtitle=torrentinfo.description)
mediainfo = None
if params.match_media:
mediainfo = self.chain.recognize_media(meta)
mediainfo = ActionChain().recognize_media(meta)
if not mediainfo:
logger.warning(f"{torrentinfo.title} 未识别到媒体信息")
continue

View File

@@ -29,26 +29,23 @@ class FetchTorrentsAction(BaseAction):
搜索站点资源
"""
_torrents = []
def __init__(self, action_id: str):
super().__init__(action_id)
self.searchchain = SearchChain()
self._torrents = []
@classmethod
@property
def name(cls) -> str: # noqa
def name(cls) -> str: # noqa
return "搜索站点资源"
@classmethod
@property
def description(cls) -> str: # noqa
def description(cls) -> str: # noqa
return "搜索站点种子资源列表"
@classmethod
@property
def data(cls) -> dict: # noqa
def data(cls) -> dict: # noqa
return FetchTorrentsParams().dict()
@property
@@ -60,9 +57,10 @@ class FetchTorrentsAction(BaseAction):
搜索站点,获取资源列表
"""
params = FetchTorrentsParams(**params)
searchchain = SearchChain()
if params.search_type == "keyword":
# 按关键字搜索
torrents = self.searchchain.search_by_title(title=params.name, sites=params.sites)
torrents = searchchain.search_by_title(title=params.name, sites=params.sites)
for torrent in torrents:
if global_vars.is_workflow_stopped(workflow_id):
break
@@ -74,7 +72,7 @@ class FetchTorrentsAction(BaseAction):
continue
# 识别媒体信息
if params.match_media:
torrent.media_info = self.searchchain.recognize_media(torrent.meta_info)
torrent.media_info = searchchain.recognize_media(torrent.meta_info)
if not torrent.media_info:
logger.warning(f"{torrent.torrent_info.title} 未识别到媒体信息")
continue
@@ -84,10 +82,10 @@ class FetchTorrentsAction(BaseAction):
for media in context.medias:
if global_vars.is_workflow_stopped(workflow_id):
break
torrents = self.searchchain.search_by_id(tmdbid=media.tmdb_id,
doubanid=media.douban_id,
mtype=MediaType(media.type),
sites=params.sites)
torrents = searchchain.search_by_id(tmdbid=media.tmdb_id,
doubanid=media.douban_id,
mtype=MediaType(media.type),
sites=params.sites)
for torrent in torrents:
self._torrents.append(torrent)

View File

@@ -22,8 +22,6 @@ class FilterMediasAction(BaseAction):
过滤媒体数据
"""
_medias = []
def __init__(self, action_id: str):
super().__init__(action_id)
self._medias = []

View File

@@ -27,12 +27,8 @@ class FilterTorrentsAction(BaseAction):
过滤资源数据
"""
_torrents = []
def __init__(self, action_id: str):
super().__init__(action_id)
self.torrenthelper = TorrentHelper()
self.chain = ActionChain()
self._torrents = []
@classmethod
@@ -62,7 +58,7 @@ class FilterTorrentsAction(BaseAction):
for torrent in context.torrents:
if global_vars.is_workflow_stopped(workflow_id):
break
if self.torrenthelper.filter_torrent(
if TorrentHelper().filter_torrent(
torrent_info=torrent.torrent_info,
filter_params={
"quality": params.quality,
@@ -73,7 +69,7 @@ class FilterTorrentsAction(BaseAction):
"size": params.size
}
):
if self.chain.filter_torrents(
if ActionChain().filter_torrents(
rule_groups=params.rule_groups,
torrent_list=[torrent.torrent_info],
mediainfo=torrent.media_info

View File

@@ -20,8 +20,6 @@ class InvokePluginAction(BaseAction):
调用插件
"""
_success = False
def __init__(self, action_id: str):
super().__init__(action_id)
self._success = False

View File

@@ -24,12 +24,8 @@ class ScanFileAction(BaseAction):
整理文件
"""
_fileitems = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.storagechain = StorageChain()
self._fileitems = []
self._has_error = False
@@ -59,12 +55,13 @@ class ScanFileAction(BaseAction):
params = ScanFileParams(**params)
if not params.storage or not params.directory:
return context
fileitem = self.storagechain.get_file_item(params.storage, Path(params.directory))
storagechain = StorageChain()
fileitem = storagechain.get_file_item(params.storage, Path(params.directory))
if not fileitem:
logger.error(f"目录不存在: 【{params.storage}{params.directory}")
self._has_error = True
return context
files = self.storagechain.list_files(fileitem, recursion=True)
files = storagechain.list_files(fileitem, recursion=True)
for file in files:
if global_vars.is_workflow_stopped(workflow_id):
break

View File

@@ -21,13 +21,8 @@ class ScrapeFileAction(BaseAction):
刮削文件
"""
_scraped_files = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.storagechain = StorageChain()
self.mediachain = MediaChain()
self._scraped_files = []
self._has_error = False
@@ -61,7 +56,7 @@ class ScrapeFileAction(BaseAction):
break
if fileitem in self._scraped_files:
continue
if not self.storagechain.exists(fileitem):
if not StorageChain().exists(fileitem):
continue
# 检查缓存
cache_key = f"{fileitem.path}"
@@ -69,12 +64,13 @@ class ScrapeFileAction(BaseAction):
logger.info(f"{fileitem.path} 已刮削过,跳过")
continue
meta = MetaInfoPath(Path(fileitem.path))
mediainfo = self.mediachain.recognize_media(meta)
mediachain = MediaChain()
mediainfo = mediachain.recognize_media(meta)
if not mediainfo:
_failed_count += 1
logger.info(f"{fileitem.path} 未识别到媒体信息,无法刮削")
continue
self.mediachain.scrape_metadata(fileitem=fileitem, meta=meta, mediainfo=mediainfo)
mediachain.scrape_metadata(fileitem=fileitem, meta=meta, mediainfo=mediainfo)
self._scraped_files.append(fileitem)
# 保存缓存
self.save_cache(workflow_id, cache_key)

View File

@@ -4,7 +4,7 @@ from pydantic import Field
from app.actions import BaseAction, ActionChain
from app.schemas import ActionParams, ActionContext, Notification
from core.config import settings
from app.core.config import settings
class SendMessageParams(ActionParams):
@@ -22,7 +22,6 @@ class SendMessageAction(BaseAction):
def __init__(self, action_id: str):
super().__init__(action_id)
self.chain = ActionChain()
@classmethod
@property
@@ -60,7 +59,7 @@ class SendMessageAction(BaseAction):
if not params.client:
params.client = [""]
for client in params.client:
self.chain.post_message(
ActionChain().post_message(
Notification(
source=client,
userid=params.userid,

View File

@@ -26,30 +26,24 @@ class TransferFileAction(BaseAction):
整理文件
"""
_fileitems = []
_has_error = False
def __init__(self, action_id: str):
super().__init__(action_id)
self.transferchain = TransferChain()
self.storagechain = StorageChain()
self.transferhis = TransferHistoryOper()
self._fileitems = []
self._has_error = False
@classmethod
@property
def name(cls) -> str: # noqa
def name(cls) -> str: # noqa
return "整理文件"
@classmethod
@property
def description(cls) -> str: # noqa
def description(cls) -> str: # noqa
return "整理队列中的文件"
@classmethod
@property
def data(cls) -> dict: # noqa
def data(cls) -> dict: # noqa
return TransferFileParams().dict()
@property
@@ -72,6 +66,9 @@ class TransferFileAction(BaseAction):
params = TransferFileParams(**params)
# 失败次数
_failed_count = 0
storagechain = StorageChain()
transferchain = TransferChain()
transferhis = TransferHistoryOper()
if params.source == "downloads":
# 从下载任务中整理文件
for download in context.downloads:
@@ -85,16 +82,16 @@ class TransferFileAction(BaseAction):
if self.check_cache(workflow_id, cache_key):
logger.info(f"{download.path} 已整理过,跳过")
continue
fileitem = self.storagechain.get_file_item(storage="local", path=Path(download.path))
fileitem = storagechain.get_file_item(storage="local", path=Path(download.path))
if not fileitem:
logger.info(f"文件 {download.path} 不存在")
continue
transferd = self.transferhis.get_by_src(fileitem.path, storage=fileitem.storage)
transferd = transferhis.get_by_src(fileitem.path, storage=fileitem.storage)
if transferd:
# 已经整理过的文件不再整理
continue
logger.info(f"开始整理文件 {download.path} ...")
state, errmsg = self.transferchain.do_transfer(fileitem, background=False)
state, errmsg = transferchain.do_transfer(fileitem, background=False)
if not state:
_failed_count += 1
logger.error(f"整理文件 {download.path} 失败: {errmsg}")
@@ -112,13 +109,13 @@ class TransferFileAction(BaseAction):
if self.check_cache(workflow_id, cache_key):
logger.info(f"{fileitem.path} 已整理过,跳过")
continue
transferd = self.transferhis.get_by_src(fileitem.path, storage=fileitem.storage)
transferd = transferhis.get_by_src(fileitem.path, storage=fileitem.storage)
if transferd:
# 已经整理过的文件不再整理
continue
logger.info(f"开始整理文件 {fileitem.path} ...")
state, errmsg = self.transferchain.do_transfer(fileitem, background=False,
continue_callback=check_continue)
state, errmsg = transferchain.do_transfer(fileitem, background=False,
continue_callback=check_continue)
if not state:
_failed_count += 1
logger.error(f"整理文件 {fileitem.path} 失败: {errmsg}")

View File

@@ -7,9 +7,9 @@ from app.core.event import eventmanager
from app.core.security import verify_token
from app.schemas import DiscoverSourceEventData
from app.schemas.types import ChainEventType, MediaType
from chain.bangumi import BangumiChain
from chain.douban import DoubanChain
from chain.tmdb import TmdbChain
from app.chain.bangumi import BangumiChain
from app.chain.douban import DoubanChain
from app.chain.tmdb import TmdbChain
router = APIRouter()

View File

@@ -94,22 +94,22 @@ def add(
@router.get("/start/{hashString}", summary="开始任务", response_model=schemas.Response)
def start(
hashString: str,
hashString: str, name: Optional[str] = None,
_: schemas.TokenPayload = Depends(verify_token)) -> Any:
"""
开如下载任务
"""
ret = DownloadChain().set_downloading(hashString, "start")
ret = DownloadChain().set_downloading(hashString, "start", name=name)
return schemas.Response(success=True if ret else False)
@router.get("/stop/{hashString}", summary="暂停任务", response_model=schemas.Response)
def stop(hashString: str,
def stop(hashString: str, name: Optional[str] = None,
_: schemas.TokenPayload = Depends(verify_token)) -> Any:
"""
暂停下载任务
"""
ret = DownloadChain().set_downloading(hashString, "stop")
ret = DownloadChain().set_downloading(hashString, "stop", name=name)
return schemas.Response(success=True if ret else False)
@@ -125,10 +125,10 @@ def clients(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
@router.delete("/{hashString}", summary="删除下载任务", response_model=schemas.Response)
def delete(hashString: str,
def delete(hashString: str, name: Optional[str] = None,
_: schemas.TokenPayload = Depends(verify_token)) -> Any:
"""
删除下载任务
"""
ret = DownloadChain().remove_downloading(hashString)
ret = DownloadChain().remove_downloading(hashString, name=name)
return schemas.Response(success=True if ret else False)

View File

@@ -1,12 +1,10 @@
from typing import List, Any, Optional
import jieba
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app import schemas
from app.chain.storage import StorageChain
from app.core.config import settings
from app.core.event import eventmanager
from app.core.security import verify_token
from app.db import get_db
@@ -59,9 +57,6 @@ def transfer_history(title: Optional[str] = None,
status = True
if title:
if settings.TOKENIZED_SEARCH:
words = jieba.cut(title, HMM=False)
title = "%".join(words)
total = TransferHistory.count_by_title(db, title=title, status=status)
result = TransferHistory.list_by_title(db, title=title, page=page,
count=count, status=status)

View File

@@ -5,9 +5,7 @@ from fastapi import APIRouter, Depends, Form, HTTPException
from fastapi.security import OAuth2PasswordRequestForm
from app import schemas
from app.chain.tmdb import TmdbChain
from app.chain.user import UserChain
from app.chain.mediaserver import MediaServerChain
from app.core import security
from app.core.config import settings
from app.helper.sites import SitesHelper
@@ -45,7 +43,8 @@ def login_access_token(
user_id=user_or_message.id,
user_name=user_or_message.name,
avatar=user_or_message.avatar,
level=level
level=level,
permissions= user_or_message.permissions or {},
)
@@ -54,14 +53,7 @@ def wallpaper() -> Any:
"""
获取登录页面电影海报
"""
if settings.WALLPAPER == "bing":
url = WallpaperHelper().get_bing_wallpaper()
elif settings.WALLPAPER == "mediaserver":
url = MediaServerChain().get_latest_wallpaper()
elif settings.WALLPAPER == "customize":
url = WallpaperHelper().get_customize_wallpaper()
else:
url = TmdbChain().get_random_wallpager()
url = WallpaperHelper().get_wallpaper()
if url:
return schemas.Response(
success=True,
@@ -75,13 +67,4 @@ def wallpapers() -> Any:
"""
获取登录页面电影海报
"""
if settings.WALLPAPER == "bing":
return WallpaperHelper().get_bing_wallpapers()
elif settings.WALLPAPER == "mediaserver":
return MediaServerChain().get_latest_wallpapers()
elif settings.WALLPAPER == "tmdb":
return TmdbChain().get_trending_wallpapers()
elif settings.WALLPAPER == "customize":
return WallpaperHelper().get_customize_wallpapers()
else:
return []
return WallpaperHelper().get_wallpapers()

View File

@@ -137,7 +137,7 @@ def register_plugin(plugin_id: str):
@router.get("/", summary="所有插件", response_model=List[schemas.Plugin])
def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
state: Optional[str] = "all") -> List[schemas.Plugin]:
state: Optional[str] = "all", force: bool = False) -> List[schemas.Plugin]:
"""
查询所有插件清单包括本地插件和在线插件插件状态installed, market, all
"""
@@ -151,7 +151,7 @@ def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
# 未安装的本地插件
not_installed_plugins = [plugin for plugin in local_plugins if not plugin.installed]
# 在线插件
online_plugins = PluginManager().get_online_plugins()
online_plugins = PluginManager().get_online_plugins(force)
if not online_plugins:
# 没有获取在线插件
if state == "market":
@@ -348,7 +348,7 @@ def plugin_static_file(plugin_id: str, filepath: str):
获取插件静态文件
"""
# 基础安全检查
if ".." in filepath or ".." in filepath:
if ".." in filepath or ".." in plugin_id:
logger.warning(f"Static File API: Path traversal attempt detected: {plugin_id}/{filepath}")
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Forbidden")

View File

@@ -1,6 +1,7 @@
import asyncio
import io
import json
import re
import tempfile
from collections import deque
from datetime import datetime
@@ -20,20 +21,21 @@ from app.core.config import global_vars, settings
from app.core.metainfo import MetaInfo
from app.core.module import ModuleManager
from app.core.security import verify_apitoken, verify_resource_token, verify_token
from app.core.event import eventmanager
from app.db.models import User
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_superuser
from app.helper.mediaserver import MediaServerHelper
from app.helper.message import MessageHelper, MessageQueueManager
from app.helper.message import MessageHelper
from app.helper.progress import ProgressHelper
from app.helper.rule import RuleHelper
from app.helper.sites import SitesHelper
from app.helper.subscribe import SubscribeHelper
from app.helper.system import SystemHelper
from app.log import logger
from app.monitor import Monitor
from app.scheduler import Scheduler
from app.schemas.types import SystemConfigKey
from app.schemas import ConfigChangeEventData
from app.schemas.types import SystemConfigKey, EventType
from app.utils.crypto import HashUtils
from app.utils.http import RequestUtils
from app.utils.security import SecurityUtils
@@ -219,18 +221,27 @@ def set_env_setting(env: dict,
result = settings.update_settings(env=env)
# 统计成功和失败的结果
success_updates = {k: v for k, v in result.items() if v[0]}
failed_updates = {k: v for k, v in result.items() if not v[0]}
failed_updates = {k: v for k, v in result.items() if v[0] is False}
if failed_updates:
return schemas.Response(
success=False,
message="部分配置项更新失败",
message=f"{', '.join([v[1] for v in failed_updates.values()])}",
data={
"success_updates": success_updates,
"failed_updates": failed_updates
}
)
if success_updates:
for key in success_updates.keys():
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=getattr(settings, key, None),
change_type="update"
))
return schemas.Response(
success=True,
message="所有配置项更新成功",
@@ -284,12 +295,28 @@ def set_setting(key: str, value: Union[list, dict, bool, int, str] = None,
"""
if hasattr(settings, key):
success, message = settings.update_setting(key=key, value=value)
if success:
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=value,
change_type="update"
))
elif success is None:
success = True
return schemas.Response(success=success, message=message)
elif key in {item.value for item in SystemConfigKey}:
if isinstance(value, list):
value = list(filter(None, value))
value = value if value else None
SystemConfigOper().set(key, value)
success = SystemConfigOper().set(key, value)
if success:
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=value,
change_type="update"
))
return schemas.Response(success=True)
else:
return schemas.Response(success=False, message=f"配置项 '{key}' 不存在")
@@ -420,30 +447,55 @@ def ruletest(title: str,
@router.get("/nettest", summary="测试网络连通性")
def nettest(url: str,
proxy: bool,
_: schemas.TokenPayload = Depends(verify_token)):
def nettest(
url: str,
proxy: bool,
include: Optional[str] = None,
_: schemas.TokenPayload = Depends(verify_token),
):
"""
测试网络连通性
"""
# 记录开始的毫秒数
start_time = datetime.now()
headers = None
if "github" in url or "{GITHUB_PROXY}" in url:
# 这是github的连通性测试
url = url.replace(
"{GITHUB_PROXY}", UrlUtils.standardize_base_url(settings.GITHUB_PROXY or "")
)
headers = settings.GITHUB_HEADERS
url = url.replace("{TMDBAPIKEY}", settings.TMDB_API_KEY)
result = RequestUtils(proxies=settings.PROXY if proxy else None,
ua=settings.USER_AGENT).get_res(url)
url = url.replace(
"{PIP_PROXY}",
UrlUtils.standardize_base_url(settings.PIP_PROXY or "https://pypi.org/simple/"),
)
result = RequestUtils(
proxies=settings.PROXY if proxy else None,
headers=headers,
timeout=10,
ua=settings.USER_AGENT,
).get_res(url)
# 计时结束的毫秒数
end_time = datetime.now()
time = round((end_time - start_time).total_seconds() * 1000)
# 计算相关秒数
if result and result.status_code == 200:
return schemas.Response(success=True, data={
"time": round((end_time - start_time).microseconds / 1000)
})
elif result:
return schemas.Response(success=False, message=f"错误码:{result.status_code}", data={
"time": round((end_time - start_time).microseconds / 1000)
})
if result is None:
return schemas.Response(success=False, message="无法连接", data={"time": time})
elif result.status_code == 200:
if include and not re.search(r"%s" % include, result.text, re.IGNORECASE):
# 通常是被加速代理跳转到其它页面了
logger.error(f"{url} 的响应内容不匹配包含规则 {include}")
return schemas.Response(
success=False,
message=f"无效响应,不匹配 {include}",
data={"time": time},
)
return schemas.Response(success=True, data={"time": time})
else:
return schemas.Response(success=False, message="网络连接失败!")
return schemas.Response(
success=False, message=f"错误码:{result.status_code}", data={"time": time}
)
@router.get("/modulelist", summary="查询已加载的模块ID列表", response_model=schemas.Response)
@@ -483,18 +535,6 @@ def restart_system(_: User = Depends(get_current_active_superuser)):
return schemas.Response(success=ret, message=msg)
@router.get("/reload", summary="重新加载模块", response_model=schemas.Response)
def reload_module(_: User = Depends(get_current_active_superuser)):
"""
重新加载模块(仅管理员)
"""
MessageQueueManager().init_config()
ModuleManager().reload()
Scheduler().init()
Monitor().init()
return schemas.Response(success=True)
@router.get("/runscheduler", summary="运行服务", response_model=schemas.Response)
def run_scheduler(jobid: str,
_: User = Depends(get_current_active_superuser)):

View File

@@ -1,5 +1,4 @@
import copy
import gc
import pickle
import traceback
from abc import ABCMeta
@@ -43,7 +42,6 @@ class ChainBase(metaclass=ABCMeta):
self.messagequeue = MessageQueueManager(
send_callback=self.run_module
)
self.useroper = UserOper()
self.pluginmanager = PluginManager()
@staticmethod
@@ -70,10 +68,6 @@ class ChainBase(metaclass=ABCMeta):
pickle.dump(cache, f) # noqa
except Exception as err:
logger.error(f"保存缓存 {filename} 出错:{str(err)}")
finally:
# 主动资源回收
del cache
gc.collect()
@staticmethod
def remove_cache(filename: str) -> None:
@@ -575,26 +569,27 @@ class ChainBase(metaclass=ABCMeta):
# 是否已发送管理员标志
admin_sended = False
send_orignal = False
useroper = UserOper()
for action in actions:
send_message = copy.deepcopy(message)
if action == "admin" and not admin_sended:
# 仅发送管理员
logger.info(f"{send_message.mtype} 的消息已设置发送给管理员")
# 读取管理员消息IDS
send_message.targets = self.useroper.get_settings(settings.SUPERUSER)
send_message.targets = useroper.get_settings(settings.SUPERUSER)
admin_sended = True
elif action == "user" and send_message.username:
# 发送对应用户
logger.info(f"{send_message.mtype} 的消息已设置发送给用户 {send_message.username}")
# 读取用户消息IDS
send_message.targets = self.useroper.get_settings(send_message.username)
send_message.targets = useroper.get_settings(send_message.username)
if send_message.targets is None:
# 没有找到用户
if not admin_sended:
# 回滚发送管理员
logger.info(f"用户 {send_message.username} 不存在,消息将发送给管理员")
# 读取管理员消息IDS
send_message.targets = self.useroper.get_settings(settings.SUPERUSER)
send_message.targets = useroper.get_settings(settings.SUPERUSER)
admin_sended = True
else:
# 管理员发过了,此消息不发了

View File

@@ -3,12 +3,11 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.core.context import MediaInfo
from app.utils.singleton import Singleton
class BangumiChain(ChainBase, metaclass=Singleton):
class BangumiChain(ChainBase):
"""
Bangumi处理链,单例运行
Bangumi处理链
"""
def calendar(self) -> Optional[List[MediaInfo]]:

View File

@@ -2,10 +2,9 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.utils.singleton import Singleton
class DashboardChain(ChainBase, metaclass=Singleton):
class DashboardChain(ChainBase):
"""
各类仪表板统计处理链
"""

View File

@@ -4,12 +4,11 @@ from app import schemas
from app.chain import ChainBase
from app.core.context import MediaInfo
from app.schemas import MediaType
from app.utils.singleton import Singleton
class DoubanChain(ChainBase, metaclass=Singleton):
class DoubanChain(ChainBase):
"""
豆瓣处理链,单例运行
豆瓣处理链
"""
def person_detail(self, person_id: int) -> Optional[schemas.MediaPerson]:

View File

@@ -16,11 +16,12 @@ from app.core.metainfo import MetaInfo
from app.db.downloadhistory_oper import DownloadHistoryOper
from app.db.mediaserver_oper import MediaServerOper
from app.helper.directory import DirectoryHelper
from app.helper.message import MessageHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import ExistMediaInfo, NotExistMediaInfo, DownloadingTorrent, Notification, ResourceSelectionEventData, ResourceDownloadEventData
from app.schemas.types import MediaType, TorrentStatus, EventType, MessageChannel, NotificationType, ContentType, ChainEventType
from app.schemas import ExistMediaInfo, NotExistMediaInfo, DownloadingTorrent, Notification, ResourceSelectionEventData, \
ResourceDownloadEventData
from app.schemas.types import MediaType, TorrentStatus, EventType, MessageChannel, NotificationType, ContentType, \
ChainEventType
from app.utils.http import RequestUtils
from app.utils.string import StringUtils
@@ -30,14 +31,6 @@ class DownloadChain(ChainBase):
下载处理链
"""
def __init__(self):
super().__init__()
self.torrent = TorrentHelper()
self.downloadhis = DownloadHistoryOper()
self.mediaserver = MediaServerOper()
self.directoryhelper = DirectoryHelper()
self.messagehelper = MessageHelper()
def download_torrent(self, torrent: TorrentInfo,
channel: MessageChannel = None,
source: Optional[str] = None,
@@ -120,7 +113,7 @@ class DownloadChain(ChainBase):
logger.error(f"{torrent.title} 无法获取下载地址:{torrent.enclosure}")
return None, "", []
# 下载种子文件
torrent_file, content, download_folder, files, error_msg = self.torrent.download_torrent(
torrent_file, content, download_folder, files, error_msg = TorrentHelper().download_torrent(
url=torrent_url,
cookie=site_cookie,
ua=torrent.site_ua or settings.USER_AGENT,
@@ -218,7 +211,7 @@ class DownloadChain(ChainBase):
else:
content = torrent_file
# 获取种子文件的文件夹名和文件清单
_folder_name, _file_list = self.torrent.get_torrent_info(torrent_file)
_folder_name, _file_list = TorrentHelper().get_torrent_info(torrent_file)
# 下载目录
if save_path:
@@ -226,7 +219,7 @@ class DownloadChain(ChainBase):
download_dir = Path(save_path)
else:
# 根据媒体信息查询下载目录配置
dir_info = self.directoryhelper.get_dir(_media, storage="local", include_unsorted=True)
dir_info = DirectoryHelper().get_dir(_media, storage="local", include_unsorted=True)
# 拼装子目录
if dir_info:
# 一级目录
@@ -276,7 +269,8 @@ class DownloadChain(ChainBase):
_save_path = download_dir if _layout == "NoSubfolder" or not _folder_name else download_path
# 登记下载记录
self.downloadhis.add(
downloadhis = DownloadHistoryOper()
downloadhis.add(
path=str(download_path),
type=_media.type.value,
title=_media.title,
@@ -324,7 +318,7 @@ class DownloadChain(ChainBase):
"torrentname": _meta.org_string,
})
if files_to_add:
self.downloadhis.add_files(files_to_add)
downloadhis.add_files(files_to_add)
# 下载成功发送消息
self.post_message(
@@ -538,7 +532,7 @@ class DownloadChain(ChainBase):
if isinstance(content, str):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法确定种子文件集数")
continue
torrent_episodes = self.torrent.get_torrent_episodes(torrent_files)
torrent_episodes = TorrentHelper().get_torrent_episodes(torrent_files)
logger.info(f"{meta.org_string} 解析种子文件集数为 {torrent_episodes}")
if not torrent_episodes:
continue
@@ -712,7 +706,7 @@ class DownloadChain(ChainBase):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法解析种子文件集数")
continue
# 种子全部集
torrent_episodes = self.torrent.get_torrent_episodes(torrent_files)
torrent_episodes = TorrentHelper().get_torrent_episodes(torrent_files)
logger.info(f"{torrent.site_name} - {meta.org_string} 解析种子文件集数:{torrent_episodes}")
# 选中的集
selected_episodes = set(torrent_episodes).intersection(set(need_episodes))
@@ -801,11 +795,12 @@ class DownloadChain(ChainBase):
if not totals:
totals = {}
mediaserver = MediaServerOper()
if mediainfo.type == MediaType.MOVIE:
# 电影
itemid = self.mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id)
itemid = mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id)
exists_movies: Optional[ExistMediaInfo] = self.media_exists(mediainfo=mediainfo, itemid=itemid)
if exists_movies:
logger.info(f"媒体库中已存在电影:{mediainfo.title_year}")
@@ -825,10 +820,10 @@ class DownloadChain(ChainBase):
logger.error(f"媒体信息中没有季集信息:{mediainfo.title_year}")
return False, {}
# 电视剧
itemid = self.mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season)
itemid = mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season)
# 媒体库已存在的剧集
exists_tvs: Optional[ExistMediaInfo] = self.media_exists(mediainfo=mediainfo, itemid=itemid)
if not exists_tvs:
@@ -927,7 +922,7 @@ class DownloadChain(ChainBase):
return []
ret_torrents = []
for torrent in torrents:
history = self.downloadhis.get_by_hash(torrent.hash)
history = DownloadHistoryOper().get_by_hash(torrent.hash)
if history:
# 媒体信息
torrent.media = {
@@ -944,21 +939,21 @@ class DownloadChain(ChainBase):
ret_torrents.append(torrent)
return ret_torrents
def set_downloading(self, hash_str, oper: str) -> bool:
def set_downloading(self, hash_str, oper: str, name: Optional[str] = None) -> bool:
"""
控制下载任务 start/stop
"""
if oper == "start":
return self.start_torrents(hashs=[hash_str])
return self.start_torrents(hashs=[hash_str], downloader=name)
elif oper == "stop":
return self.stop_torrents(hashs=[hash_str])
return self.stop_torrents(hashs=[hash_str], downloader=name)
return False
def remove_downloading(self, hash_str: str) -> bool:
def remove_downloading(self, hash_str: str, name: Optional[str] = None) -> bool:
"""
删除下载任务
"""
return self.remove_torrents(hashs=[hash_str])
return self.remove_torrents(hashs=[hash_str], downloader=name)
@eventmanager.register(EventType.DownloadFileDeleted)
def download_file_deleted(self, event: Event):

View File

@@ -10,11 +10,11 @@ from app.core.context import Context, MediaInfo
from app.core.event import eventmanager, Event
from app.core.meta import MetaBase
from app.core.metainfo import MetaInfo, MetaInfoPath
from app.db.systemconfig_oper import SystemConfigOper
from app.log import logger
from app.schemas import FileItem
from app.schemas.types import EventType, MediaType, ChainEventType
from app.schemas.types import EventType, MediaType, ChainEventType, SystemConfigKey
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
recognize_lock = Lock()
@@ -22,14 +22,53 @@ scraping_lock = Lock()
scraping_files = []
class MediaChain(ChainBase, metaclass=Singleton):
class MediaChain(ChainBase):
"""
媒体信息处理链,单例运行
"""
def __init__(self):
super().__init__()
self.storagechain = StorageChain()
@staticmethod
def _get_scraping_switchs() -> dict:
"""
获取刮削开关配置
"""
switchs = SystemConfigOper().get(SystemConfigKey.ScrapingSwitchs) or {}
# 默认配置
default_switchs = {
'movie_nfo': True, # 电影NFO
'movie_poster': True, # 电影海报
'movie_backdrop': True, # 电影背景图
'movie_logo': True, # 电影Logo
'movie_disc': True, # 电影光盘图
'movie_banner': True, # 电影横幅图
'movie_thumb': True, # 电影缩略图
'tv_nfo': True, # 电视剧NFO
'tv_poster': True, # 电视剧海报
'tv_backdrop': True, # 电视剧背景图
'tv_banner': True, # 电视剧横幅图
'tv_logo': True, # 电视剧Logo
'tv_thumb': True, # 电视剧缩略图
'season_nfo': True, # 季NFO
'season_poster': True, # 季海报
'season_banner': True, # 季横幅图
'season_thumb': True, # 季缩略图
'episode_nfo': True, # 集NFO
'episode_thumb': True # 集缩略图
}
# 合并用户配置和默认配置
for key, default_value in default_switchs.items():
if key not in switchs:
switchs[key] = default_value
return switchs
@staticmethod
def set_scraping_switchs(switchs: dict) -> bool:
"""
设置刮削开关配置
:param switchs: 开关配置字典
:return: 是否设置成功
"""
return SystemConfigOper().set(SystemConfigKey.ScrapingSwitchs, switchs)
def metadata_nfo(self, meta: MetaBase, mediainfo: MediaInfo,
season: Optional[int] = None, episode: Optional[int] = None) -> Optional[str]:
@@ -337,6 +376,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
:param overwrite: 是否覆盖已有文件
"""
storagechain = StorageChain()
def is_bluray_folder(_fileitem: schemas.FileItem) -> bool:
"""
判断是否为原盘目录
@@ -346,7 +387,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 蓝光原盘目录必备的文件或文件夹
required_files = ['BDMV', 'CERTIFICATE']
# 检查目录下是否存在所需文件或文件夹
for item in self.storagechain.list_files(_fileitem):
for item in storagechain.list_files(_fileitem):
if item.name in required_files:
return True
return False
@@ -355,7 +396,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
"""
列出下级文件
"""
return self.storagechain.list_files(fileitem=_fileitem)
return storagechain.list_files(fileitem=_fileitem)
def __save_file(_fileitem: schemas.FileItem, _path: Path, _content: Union[bytes, str]):
"""
@@ -371,7 +412,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
tmp_file.write_bytes(_content)
# 获取文件的父目录
try:
item = self.storagechain.upload_file(fileitem=_fileitem, path=tmp_file, new_name=_path.name)
item = storagechain.upload_file(fileitem=_fileitem, path=tmp_file, new_name=_path.name)
if item:
logger.info(f"已保存文件:{item.path}")
else:
@@ -407,37 +448,47 @@ class MediaChain(ChainBase, metaclass=Singleton):
if not mediainfo:
logger.warn(f"{filepath} 无法识别文件媒体信息!")
return
# 获取刮削开关配置
scraping_switchs = self._get_scraping_switchs()
logger.info(f"开始刮削:{filepath} ...")
if mediainfo.type == MediaType.MOVIE:
# 电影
if fileitem.type == "file":
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 电影文件
movie_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if movie_nfo:
# 保存或上传nfo文件到上级目录
__save_file(_fileitem=parent, _path=nfo_path, _content=movie_nfo)
else:
logger.warn(f"{filepath.name} nfo文件生成失败")
else:
logger.info(f"已存在nfo文件{nfo_path}")
else:
# 电影目录
if is_bluray_folder(fileitem):
# 原盘目录
nfo_path = filepath / (filepath.name + ".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 生成原盘nfo
# 检查电影NFO开关
if scraping_switchs.get('movie_nfo', True):
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 电影文件
movie_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if movie_nfo:
# 保存或上传nfo文件到当前目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=movie_nfo)
# 保存或上传nfo文件到上级目录
__save_file(_fileitem=parent, _path=nfo_path, _content=movie_nfo)
else:
logger.warn(f"{filepath.name} nfo文件生成失败")
else:
logger.info(f"已存在nfo文件{nfo_path}")
else:
logger.info("电影NFO刮削已关闭跳过")
else:
# 电影目录
if is_bluray_folder(fileitem):
# 原盘目录
if scraping_switchs.get('movie_nfo', True):
nfo_path = filepath / (filepath.name + ".nfo")
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 生成原盘nfo
movie_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if movie_nfo:
# 保存或上传nfo文件到当前目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=movie_nfo)
else:
logger.warn(f"{filepath.name} nfo文件生成失败")
else:
logger.info(f"已存在nfo文件{nfo_path}")
else:
logger.info("电影NFO刮削已关闭跳过")
else:
# 处理目录内的文件
files = __list_files(_fileitem=fileitem)
@@ -452,16 +503,35 @@ class MediaChain(ChainBase, metaclass=Singleton):
image_dict = self.metadata_img(mediainfo=mediainfo)
if image_dict:
for image_name, image_url in image_dict.items():
image_path = filepath.with_name(image_name)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 写入图片到当前目录
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
# 根据图片类型检查开关
if 'poster' in image_name.lower():
should_scrape = scraping_switchs.get('movie_poster', True)
elif 'backdrop' in image_name.lower() or 'fanart' in image_name.lower():
should_scrape = scraping_switchs.get('movie_backdrop', True)
elif 'logo' in image_name.lower():
should_scrape = scraping_switchs.get('movie_logo', True)
elif 'disc' in image_name.lower() or 'cdart' in image_name.lower():
should_scrape = scraping_switchs.get('movie_disc', True)
elif 'banner' in image_name.lower():
should_scrape = scraping_switchs.get('movie_banner', True)
elif 'thumb' in image_name.lower():
should_scrape = scraping_switchs.get('movie_thumb', True)
else:
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath.with_name(image_name)
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 写入图片到当前目录
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
else:
logger.info(f"电影图片刮削已关闭,跳过:{image_name}")
else:
# 电视剧
if fileitem.type == "file":
@@ -475,38 +545,45 @@ class MediaChain(ChainBase, metaclass=Singleton):
if not file_mediainfo:
logger.warn(f"{filepath.name} 无法识别文件媒体信息!")
return
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 获取集的nfo文件
episode_nfo = self.metadata_nfo(meta=file_meta, mediainfo=file_mediainfo,
season=file_meta.begin_season,
episode=file_meta.begin_episode)
if episode_nfo:
# 保存或上传nfo文件到上级目录
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=nfo_path, _content=episode_nfo)
else:
logger.warn(f"{filepath.name} nfo文件生成失败")
else:
logger.info(f"已存在nfo文件{nfo_path}")
# 获取集的图片
image_dict = self.metadata_img(mediainfo=file_mediainfo,
season=file_meta.begin_season, episode=file_meta.begin_episode)
if image_dict:
for episode, image_url in image_dict.items():
image_path = filepath.with_suffix(Path(image_url).suffix)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
# 检查集NFO开关
if scraping_switchs.get('episode_nfo', True):
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 获取集的nfo文件
episode_nfo = self.metadata_nfo(meta=file_meta, mediainfo=file_mediainfo,
season=file_meta.begin_season,
episode=file_meta.begin_episode)
if episode_nfo:
# 保存或上传nfo文件到上级目录
if not parent:
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=nfo_path, _content=episode_nfo)
else:
logger.info(f"已存在图片文件:{image_path}")
logger.warn(f"{filepath.name} nfo文件生成失败")
else:
logger.info(f"已存在nfo文件{nfo_path}")
else:
logger.info("集NFO刮削已关闭跳过")
# 获取集的图片
if scraping_switchs.get('episode_thumb', True):
image_dict = self.metadata_img(mediainfo=file_mediainfo,
season=file_meta.begin_season, episode=file_meta.begin_episode)
if image_dict:
for episode, image_url in image_dict.items():
image_path = filepath.with_suffix(Path(image_url).suffix)
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
else:
logger.info("集缩略图刮削已关闭,跳过")
else:
# 当前为目录,处理目录内的文件
files = __list_files(_fileitem=fileitem)
@@ -524,71 +601,95 @@ class MediaChain(ChainBase, metaclass=Singleton):
if filepath.name in settings.RENAME_FORMAT_S0_NAMES:
season_meta.begin_season = 0
if season_meta.begin_season is not None:
# 是否已存在
nfo_path = filepath / "season.nfo"
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有季号生成季nfo
season_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo,
season=season_meta.begin_season)
if season_nfo:
# 写入nfo到根目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=season_nfo)
else:
logger.warn(f"无法生成电视剧季nfo文件{meta.name}")
else:
logger.info(f"已存在nfo文件{nfo_path}")
# TMDB季poster图片
image_dict = self.metadata_img(mediainfo=mediainfo, season=season_meta.begin_season)
if image_dict:
for image_name, image_url in image_dict.items():
image_path = filepath.with_name(image_name)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到剧集目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
# 检查季NFO开关
if scraping_switchs.get('season_nfo', True):
# 是否已存在
nfo_path = filepath / "season.nfo"
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有季号生成季nfo
season_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo,
season=season_meta.begin_season)
if season_nfo:
# 写入nfo到根目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=season_nfo)
else:
logger.info(f"已存在图片文件:{image_path}")
logger.warn(f"无法生成电视剧季nfo文件{meta.name}")
else:
logger.info(f"已存在nfo文件{nfo_path}")
else:
logger.info("季NFO刮削已关闭跳过")
# TMDB季poster图片
if scraping_switchs.get('season_poster', True):
image_dict = self.metadata_img(mediainfo=mediainfo, season=season_meta.begin_season)
if image_dict:
for image_name, image_url in image_dict.items():
image_path = filepath.with_name(image_name)
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到剧集目录
if content:
if not parent:
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
else:
logger.info("季海报刮削已关闭,跳过")
# 额外fanart季图片poster thumb banner
image_dict = self.metadata_img(mediainfo=mediainfo)
if image_dict:
for image_name, image_url in image_dict.items():
if image_name.startswith("season"):
image_path = filepath.with_name(image_name)
# 只下载当前刮削季的图片
image_season = "00" if "specials" in image_name else image_name[6:8]
if image_season != str(season_meta.begin_season).rjust(2, '0'):
logger.info(f"当前刮削季为:{season_meta.begin_season},跳过文件:{image_path}")
continue
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
# 根据季图片类型检查开关
if 'poster' in image_name.lower():
should_scrape = scraping_switchs.get('season_poster', True)
elif 'banner' in image_name.lower():
should_scrape = scraping_switchs.get('season_banner', True)
elif 'thumb' in image_name.lower():
should_scrape = scraping_switchs.get('season_thumb', True)
else:
logger.info(f"已存在图片文件:{image_path}")
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath.with_name(image_name)
# 只下载当前刮削季的图片
image_season = "00" if "specials" in image_name else image_name[6:8]
if image_season != str(season_meta.begin_season).rjust(2, '0'):
logger.info(f"当前刮削季为:{season_meta.begin_season},跳过文件:{image_path}")
continue
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
else:
logger.info(f"季图片刮削已关闭,跳过:{image_name}")
# 判断当前目录是不是剧集根目录
if not season_meta.season:
# 是否已存在
nfo_path = filepath / "tvshow.nfo"
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有名称,生成tvshow nfo 和 tv图片
tv_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if tv_nfo:
# 写入tvshow nfo到根目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=tv_nfo)
# 检查电视剧NFO开关
if scraping_switchs.get('tv_nfo', True):
# 是否已存在
nfo_path = filepath / "tvshow.nfo"
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有名称生成tvshow nfo 和 tv图片
tv_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if tv_nfo:
# 写入tvshow nfo到根目录
__save_file(_fileitem=fileitem, _path=nfo_path, _content=tv_nfo)
else:
logger.warn(f"无法生成电视剧nfo文件{meta.name}")
else:
logger.warn(f"无法生成电视剧nfo文件{meta.name}")
logger.info(f"已存在nfo文件{nfo_path}")
else:
logger.info(f"已存在nfo文件{nfo_path}")
logger.info("电视剧NFO刮削已关闭跳过")
# 生成目录图片
image_dict = self.metadata_img(mediainfo=mediainfo)
if image_dict:
@@ -596,14 +697,31 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 不下载季图片
if image_name.startswith("season"):
continue
image_path = filepath / image_name
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
# 根据电视剧图片类型检查开关
if 'poster' in image_name.lower():
should_scrape = scraping_switchs.get('tv_poster', True)
elif 'backdrop' in image_name.lower() or 'fanart' in image_name.lower():
should_scrape = scraping_switchs.get('tv_backdrop', True)
elif 'banner' in image_name.lower():
should_scrape = scraping_switchs.get('tv_banner', True)
elif 'logo' in image_name.lower():
should_scrape = scraping_switchs.get('tv_logo', True)
elif 'thumb' in image_name.lower():
should_scrape = scraping_switchs.get('tv_thumb', True)
else:
logger.info(f"已存在图片文件:{image_path}")
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath / image_name
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
else:
logger.info(f"电视剧图片刮削已关闭,跳过:{image_name}")
logger.info(f"{filepath.name} 刮削完成")

View File

@@ -2,7 +2,6 @@ import threading
from typing import List, Union, Optional, Generator, Any
from app.chain import ChainBase
from app.core.cache import cached
from app.core.config import global_vars
from app.db.mediaserver_oper import MediaServerOper
from app.helper.service import ServiceConfigHelper
@@ -17,10 +16,6 @@ class MediaServerChain(ChainBase):
媒体服务器处理链
"""
def __init__(self):
super().__init__()
self.dboper = MediaServerOper()
def librarys(self, server: str, username: Optional[str] = None,
hidden: bool = False) -> List[MediaServerLibrary]:
"""
@@ -96,7 +91,6 @@ class MediaServerChain(ChainBase):
"""
return self.run_module("mediaserver_latest", count=count, server=server, username=username)
@cached(maxsize=1, ttl=3600)
def get_latest_wallpapers(self, server: Optional[str] = None, count: Optional[int] = 10,
remote: bool = True, username: Optional[str] = None) -> List[str]:
"""
@@ -131,7 +125,8 @@ class MediaServerChain(ChainBase):
# 汇总统计
total_count = 0
# 清空登记薄
self.dboper.empty()
dboper = MediaServerOper()
dboper.empty()
# 遍历媒体服务器
for mediaserver in mediaservers:
if not mediaserver:
@@ -175,7 +170,7 @@ class MediaServerChain(ChainBase):
item_dict = item.dict()
item_dict["seasoninfo"] = seasoninfo
item_dict["item_type"] = item_type
self.dboper.add(**item_dict)
dboper.add(**item_dict)
logger.info(f"{server_name} 媒体库 {library.name} 同步完成,共同步数量:{library_count}")
# 总数累加
total_count += library_count

View File

@@ -1,4 +1,4 @@
import copy
import gc
import re
from typing import Any, Optional, Dict, Union
@@ -9,10 +9,8 @@ from app.chain.search import SearchChain
from app.chain.subscribe import SubscribeChain
from app.core.config import settings
from app.core.context import MediaInfo, Context
from app.core.event import EventManager
from app.core.meta import MetaBase
from app.db.message_oper import MessageOper
from app.helper.message import MessageHelper
from app.db.user_oper import UserOper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import Notification, NotExistMediaInfo, CommingMessage
@@ -36,19 +34,8 @@ class MessageChain(ChainBase):
# 每页数据量
_page_size: int = 8
def __init__(self):
super().__init__()
self.downloadchain = DownloadChain()
self.subscribechain = SubscribeChain()
self.searchchain = SearchChain()
self.mediachain = MediaChain()
self.eventmanager = EventManager()
self.torrenthelper = TorrentHelper()
self.messagehelper = MessageHelper()
self.messageoper = MessageOper()
@staticmethod
def __get_noexits_info(
self,
_meta: MetaBase,
_mediainfo: MediaInfo) -> Dict[Union[int, str], Dict[int, NotExistMediaInfo]]:
"""
@@ -57,10 +44,10 @@ class MessageChain(ChainBase):
if _mediainfo.type == MediaType.TV:
if not _mediainfo.seasons:
# 补充媒体信息
_mediainfo = self.mediachain.recognize_media(mtype=_mediainfo.type,
tmdbid=_mediainfo.tmdb_id,
doubanid=_mediainfo.douban_id,
cache=False)
_mediainfo = MediaChain().recognize_media(mtype=_mediainfo.type,
tmdbid=_mediainfo.tmdb_id,
doubanid=_mediainfo.douban_id,
cache=False)
if not _mediainfo:
logger.warn(f"{_mediainfo.tmdb_id or _mediainfo.douban_id} 媒体信息识别失败!")
return {}
@@ -173,7 +160,7 @@ class MessageChain(ChainBase):
elif text.isdigit():
# 用户选择了具体的条目
# 缓存
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
# 选择项目
if not cache_data \
or not cache_data.get('items') \
@@ -186,15 +173,15 @@ class MessageChain(ChainBase):
# 缓存类型
cache_type: str = cache_data.get('type')
# 缓存列表
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
# 选择
if cache_type in ["Search", "ReSearch"]:
# 当前媒体信息
mediainfo: MediaInfo = cache_list[_choice]
_current_media = mediainfo
# 查询缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(meta=_current_meta,
mediainfo=_current_media)
exist_flag, no_exists = DownloadChain().get_no_exists_info(meta=_current_meta,
mediainfo=_current_media)
if exist_flag and cache_type == "Search":
# 媒体库中已存在
self.post_message(
@@ -234,8 +221,8 @@ class MessageChain(ChainBase):
title=f"开始搜索 {mediainfo.type.value} {mediainfo.title_year} ...",
userid=userid))
# 开始搜索
contexts = self.searchchain.process(mediainfo=mediainfo,
no_exists=no_exists)
contexts = SearchChain().process(mediainfo=mediainfo,
no_exists=no_exists)
if not contexts:
# 没有数据
self.post_message(Notification(
@@ -246,7 +233,7 @@ class MessageChain(ChainBase):
userid=userid))
return
# 搜索结果排序
contexts = self.torrenthelper.sort_torrents(contexts)
contexts = TorrentHelper().sort_torrents(contexts)
# 判断是否设置自动下载
auto_download_user = settings.AUTO_DOWNLOAD_USER
# 匹配到自动下载用户
@@ -283,8 +270,8 @@ class MessageChain(ChainBase):
best_version = False
# 查询缺失的媒体信息
if cache_type == "Subscribe":
exist_flag, _ = self.downloadchain.get_no_exists_info(meta=_current_meta,
mediainfo=mediainfo)
exist_flag, _ = DownloadChain().get_no_exists_info(meta=_current_meta,
mediainfo=mediainfo)
if exist_flag:
self.post_message(Notification(
channel=channel,
@@ -296,18 +283,18 @@ class MessageChain(ChainBase):
else:
best_version = True
# 转换用户名
mp_name = self.useroper.get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
mp_name = UserOper().get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
# 添加订阅状态为N
self.subscribechain.add(title=mediainfo.title,
year=mediainfo.year,
mtype=mediainfo.type,
tmdbid=mediainfo.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
best_version=best_version)
SubscribeChain().add(title=mediainfo.title,
year=mediainfo.year,
mtype=mediainfo.type,
tmdbid=mediainfo.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
best_version=best_version)
elif cache_type == "Torrent":
if int(text) == 0:
# 自动选择下载,强制下载模式
@@ -320,12 +307,12 @@ class MessageChain(ChainBase):
# 下载种子
context: Context = cache_list[_choice]
# 下载
self.downloadchain.download_single(context, channel=channel, source=source,
userid=userid, username=username)
DownloadChain().download_single(context, channel=channel, source=source,
userid=userid, username=username)
elif text.lower() == "p":
# 上一页
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
if not cache_data:
# 没有缓存
self.post_message(Notification(
@@ -341,7 +328,7 @@ class MessageChain(ChainBase):
_current_page -= 1
cache_type: str = cache_data.get('type')
# 产生副本,避免修改原值
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
if _current_page == 0:
start = 0
end = self._page_size
@@ -367,7 +354,7 @@ class MessageChain(ChainBase):
elif text.lower() == "n":
# 下一页
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
if not cache_data:
# 没有缓存
self.post_message(Notification(
@@ -375,7 +362,7 @@ class MessageChain(ChainBase):
return
cache_type: str = cache_data.get('type')
# 产生副本,避免修改原值
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
total = len(cache_list)
# 加一页
cache_list = cache_list[
@@ -434,7 +421,7 @@ class MessageChain(ChainBase):
if action in ["Search", "ReSearch", "Subscribe", "ReSubscribe"]:
# 搜索
meta, medias = self.mediachain.search(content)
meta, medias = MediaChain().search(content)
# 识别
if not meta.name:
self.post_message(Notification(
@@ -475,15 +462,22 @@ class MessageChain(ChainBase):
# 保存缓存
self.save_cache(user_cache, self._cache_file)
# 清理内存
user_cache.clear()
del user_cache
gc.collect()
def __auto_download(self, channel: MessageChannel, source: str, cache_list: list[Context],
userid: Union[str, int], username: str,
no_exists: Optional[Dict[Union[int, str], Dict[int, NotExistMediaInfo]]] = None):
"""
自动择优下载
"""
downloadchain = DownloadChain()
if no_exists is None:
# 查询缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(
exist_flag, no_exists = downloadchain.get_no_exists_info(
meta=_current_meta,
mediainfo=_current_media
)
@@ -492,12 +486,12 @@ class MessageChain(ChainBase):
no_exists = self.__get_noexits_info(_current_meta, _current_media)
# 批量下载
downloads, lefts = self.downloadchain.batch_download(contexts=cache_list,
no_exists=no_exists,
channel=channel,
source=source,
userid=userid,
username=username)
downloads, lefts = downloadchain.batch_download(contexts=cache_list,
no_exists=no_exists,
channel=channel,
source=source,
userid=userid,
username=username)
if downloads and not lefts:
# 全部下载完成
logger.info(f'{_current_media.title_year} 下载完成')
@@ -512,19 +506,19 @@ class MessageChain(ChainBase):
else:
note = None
# 转换用户名
mp_name = self.useroper.get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
mp_name = UserOper().get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
# 添加订阅状态为R
self.subscribechain.add(title=_current_media.title,
year=_current_media.year,
mtype=_current_media.type,
tmdbid=_current_media.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
state="R",
note=note)
SubscribeChain().add(title=_current_media.title,
year=_current_media.year,
mtype=_current_media.type,
tmdbid=_current_media.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
state="R",
note=note)
def __post_medias_message(self, channel: MessageChannel, source: str,
title: str, items: list, userid: str, total: int):

View File

@@ -29,12 +29,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
推荐处理链,单例运行
"""
def __init__(self):
super().__init__()
self.tmdbchain = TmdbChain()
self.doubanchain = DoubanChain()
self.bangumichain = BangumiChain()
self.cache_max_pages = 5
# 推荐数据的缓存页数
cache_max_pages = 5
def refresh_recommend(self):
"""
@@ -174,16 +170,16 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB热门电影
"""
movies = self.tmdbchain.tmdb_discover(mtype=MediaType.MOVIE,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
movies = TmdbChain().tmdb_discover(mtype=MediaType.MOVIE,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
return [movie.to_dict() for movie in movies] if movies else []
@log_execution_time(logger=logger)
@@ -200,16 +196,16 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB热门电视剧
"""
tvs = self.tmdbchain.tmdb_discover(mtype=MediaType.TV,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
tvs = TmdbChain().tmdb_discover(mtype=MediaType.TV,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
return [tv.to_dict() for tv in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -218,7 +214,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB流行趋势
"""
infos = self.tmdbchain.tmdb_trending(page=page)
infos = TmdbChain().tmdb_trending(page=page)
return [info.to_dict() for info in infos] if infos else []
@log_execution_time(logger=logger)
@@ -227,7 +223,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
Bangumi每日放送
"""
medias = self.bangumichain.calendar()
medias = BangumiChain().calendar()
return [media.to_dict() for media in medias[(page - 1) * count: page * count]] if medias else []
@log_execution_time(logger=logger)
@@ -236,7 +232,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣正在热映
"""
movies = self.doubanchain.movie_showing(page=page, count=count)
movies = DoubanChain().movie_showing(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -246,8 +242,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣最新电影
"""
movies = self.doubanchain.douban_discover(mtype=MediaType.MOVIE,
sort=sort, tags=tags, page=page, count=count)
movies = DoubanChain().douban_discover(mtype=MediaType.MOVIE,
sort=sort, tags=tags, page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -257,8 +253,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣最新电视剧
"""
tvs = self.doubanchain.douban_discover(mtype=MediaType.TV,
sort=sort, tags=tags, page=page, count=count)
tvs = DoubanChain().douban_discover(mtype=MediaType.TV,
sort=sort, tags=tags, page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -267,7 +263,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣电影TOP250
"""
movies = self.doubanchain.movie_top250(page=page, count=count)
movies = DoubanChain().movie_top250(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -276,7 +272,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣国产剧集榜
"""
tvs = self.doubanchain.tv_weekly_chinese(page=page, count=count)
tvs = DoubanChain().tv_weekly_chinese(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -285,7 +281,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣全球剧集榜
"""
tvs = self.doubanchain.tv_weekly_global(page=page, count=count)
tvs = DoubanChain().tv_weekly_global(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -294,7 +290,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门动漫
"""
tvs = self.doubanchain.tv_animation(page=page, count=count)
tvs = DoubanChain().tv_animation(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -303,7 +299,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门电影
"""
movies = self.doubanchain.movie_hot(page=page, count=count)
movies = DoubanChain().movie_hot(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -312,5 +308,5 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门电视剧
"""
tvs = self.doubanchain.tv_hot(page=page, count=count)
tvs = DoubanChain().tv_hot(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []

View File

@@ -27,13 +27,6 @@ class SearchChain(ChainBase):
__result_temp_file = "__search_result__"
def __init__(self):
super().__init__()
self.siteshelper = SitesHelper()
self.progress = ProgressHelper()
self.systemconfig = SystemConfigOper()
self.torrenthelper = TorrentHelper()
def search_by_id(self, tmdbid: Optional[int] = None, doubanid: Optional[str] = None,
mtype: MediaType = None, area: Optional[str] = "title", season: Optional[int] = None,
sites: List[int] = None, cache_local: bool = False) -> List[Context]:
@@ -184,19 +177,20 @@ class SearchChain(ChainBase):
return []
# 开始新进度
self.progress.start(ProgressKey.Search)
progress = ProgressHelper()
progress.start(ProgressKey.Search)
# 开始过滤
self.progress.update(value=0, text=f'开始过滤,总 {len(torrents)} 个资源,请稍候...',
key=ProgressKey.Search)
progress.update(value=0, text=f'开始过滤,总 {len(torrents)} 个资源,请稍候...',
key=ProgressKey.Search)
# 匹配订阅附加参数
if filter_params:
logger.info(f'开始附加参数过滤,附加参数:{filter_params} ...')
torrents = [torrent for torrent in torrents if self.torrenthelper.filter_torrent(torrent, filter_params)]
torrents = [torrent for torrent in torrents if TorrentHelper().filter_torrent(torrent, filter_params)]
# 开始过滤规则过滤
if rule_groups is None:
# 取搜索过滤规则
rule_groups: List[str] = self.systemconfig.get(SystemConfigKey.SearchFilterRuleGroups)
rule_groups: List[str] = SystemConfigOper().get(SystemConfigKey.SearchFilterRuleGroups)
if rule_groups:
logger.info(f'开始过滤规则/剧集过滤,使用规则组:{rule_groups} ...')
torrents = __do_filter(torrents)
@@ -206,7 +200,7 @@ class SearchChain(ChainBase):
logger.info(f"过滤规则/剧集过滤完成,剩余 {len(torrents)} 个资源")
# 过滤完成
self.progress.update(value=50, text=f'过滤完成,剩余 {len(torrents)} 个资源', key=ProgressKey.Search)
progress.update(value=50, text=f'过滤完成,剩余 {len(torrents)} 个资源', key=ProgressKey.Search)
# 开始匹配
_match_torrents = []
@@ -215,17 +209,19 @@ class SearchChain(ChainBase):
# 已处理数
_count = 0
torrenthelper = TorrentHelper()
if mediainfo:
# 英文标题应该在别名/原标题中,不需要再匹配
logger.info(f"开始匹配结果 标题:{mediainfo.title},原标题:{mediainfo.original_title},别名:{mediainfo.names}")
self.progress.update(value=51, text=f'开始匹配,总 {_total} 个资源 ...', key=ProgressKey.Search)
progress.update(value=51, text=f'开始匹配,总 {_total} 个资源 ...', key=ProgressKey.Search)
for torrent in torrents:
if global_vars.is_system_stopped:
break
_count += 1
self.progress.update(value=(_count / _total) * 96,
text=f'正在匹配 {torrent.site_name},已完成 {_count} / {_total} ...',
key=ProgressKey.Search)
progress.update(value=(_count / _total) * 96,
text=f'正在匹配 {torrent.site_name},已完成 {_count} / {_total} ...',
key=ProgressKey.Search)
if not torrent.title:
continue
@@ -236,10 +232,9 @@ class SearchChain(ChainBase):
logger.info(f"种子名称应用识别词后发生改变:{torrent.title} => {torrent_meta.org_string}")
# 季集数过滤
if season_episodes \
and not self.torrenthelper.match_season_episodes(
torrent=torrent,
meta=torrent_meta,
season_episodes=season_episodes):
and not torrenthelper.match_season_episodes(torrent=torrent,
meta=torrent_meta,
season_episodes=season_episodes):
continue
# 比对IMDBID
if torrent.imdbid \
@@ -250,17 +245,17 @@ class SearchChain(ChainBase):
continue
# 比对种子
if self.torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent):
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent):
# 匹配成功
_match_torrents.append((torrent, torrent_meta))
continue
# 匹配完成
logger.info(f"匹配完成,共匹配到 {len(_match_torrents)} 个资源")
self.progress.update(value=97,
text=f'匹配完成,共匹配到 {len(_match_torrents)} 个资源',
key=ProgressKey.Search)
progress.update(value=97,
text=f'匹配完成,共匹配到 {len(_match_torrents)} 个资源',
key=ProgressKey.Search)
else:
_match_torrents = [(t, MetaInfo(title=t.title, subtitle=t.description)) for t in torrents]
@@ -273,17 +268,17 @@ class SearchChain(ChainBase):
meta_info=t[1]) for t in _match_torrents]
# 排序
self.progress.update(value=99,
text=f'正在对 {len(contexts)} 个资源进行排序,请稍候...',
key=ProgressKey.Search)
contexts = self.torrenthelper.sort_torrents(contexts)
progress.update(value=99,
text=f'正在对 {len(contexts)} 个资源进行排序,请稍候...',
key=ProgressKey.Search)
contexts = torrenthelper.sort_torrents(contexts)
# 结束进度
logger.info(f'搜索完成,共 {len(contexts)} 个资源')
self.progress.update(value=100,
text=f'搜索完成,共 {len(contexts)} 个资源',
key=ProgressKey.Search)
self.progress.end(ProgressKey.Search)
progress.update(value=100,
text=f'搜索完成,共 {len(contexts)} 个资源',
key=ProgressKey.Search)
progress.end(ProgressKey.Search)
# 返回
return contexts
@@ -307,9 +302,9 @@ class SearchChain(ChainBase):
# 配置的索引站点
if not sites:
sites = self.systemconfig.get(SystemConfigKey.IndexerSites) or []
sites = SystemConfigOper().get(SystemConfigKey.IndexerSites) or []
for indexer in self.siteshelper.get_indexers():
for indexer in SitesHelper().get_indexers():
# 检查站点索引开关
if not sites or indexer.get("id") in sites:
indexer_sites.append(indexer)
@@ -318,7 +313,8 @@ class SearchChain(ChainBase):
return []
# 开始进度
self.progress.start(ProgressKey.Search)
progress = ProgressHelper()
progress.start(ProgressKey.Search)
# 开始计时
start_time = datetime.now()
# 总数
@@ -326,9 +322,9 @@ class SearchChain(ChainBase):
# 完成数
finish_count = 0
# 更新进度
self.progress.update(value=0,
text=f"开始搜索,共 {total_num} 个站点 ...",
key=ProgressKey.Search)
progress.update(value=0,
text=f"开始搜索,共 {total_num} 个站点 ...",
key=ProgressKey.Search)
# 结果集
results = []
# 多线程
@@ -356,18 +352,18 @@ class SearchChain(ChainBase):
if result:
results.extend(result)
logger.info(f"站点搜索进度:{finish_count} / {total_num}")
self.progress.update(value=finish_count / total_num * 100,
text=f"正在搜索{keywords or ''},已完成 {finish_count} / {total_num} 个站点 ...",
key=ProgressKey.Search)
progress.update(value=finish_count / total_num * 100,
text=f"正在搜索{keywords or ''},已完成 {finish_count} / {total_num} 个站点 ...",
key=ProgressKey.Search)
# 计算耗时
end_time = datetime.now()
# 更新进度
self.progress.update(value=100,
text=f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}",
key=ProgressKey.Search)
progress.update(value=100,
text=f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}",
key=ProgressKey.Search)
logger.info(f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}")
# 结束进度
self.progress.end(ProgressKey.Search)
progress.end(ProgressKey.Search)
# 返回
return results

View File

@@ -16,7 +16,6 @@ from app.helper.browser import PlaywrightHelper
from app.helper.cloudflare import under_challenge
from app.helper.cookie import CookieHelper
from app.helper.cookiecloud import CookieCloudHelper
from app.helper.message import MessageHelper
from app.helper.rss import RssHelper
from app.helper.sites import SitesHelper
from app.log import logger
@@ -34,13 +33,6 @@ class SiteChain(ChainBase):
def __init__(self):
super().__init__()
self.siteoper = SiteOper()
self.siteshelper = SitesHelper()
self.rsshelper = RssHelper()
self.cookiehelper = CookieHelper()
self.message = MessageHelper()
self.cookiecloud = CookieCloudHelper()
self.systemconfig = SystemConfigOper()
# 特殊站点登录验证
self.special_site_test = {
@@ -62,9 +54,9 @@ class SiteChain(ChainBase):
"""
userdata: SiteUserData = self.run_module("refresh_userdata", site=site)
if userdata:
self.siteoper.update_userdata(domain=StringUtils.get_url_domain(site.get("domain")),
name=site.get("name"),
payload=userdata.dict())
SiteOper().update_userdata(domain=StringUtils.get_url_domain(site.get("domain")),
name=site.get("name"),
payload=userdata.dict())
# 发送事件
EventManager().send_event(EventType.SiteRefreshed, {
"site_id": site.get("id")
@@ -100,7 +92,7 @@ class SiteChain(ChainBase):
"""
刷新所有站点的用户数据
"""
sites = self.siteshelper.get_indexers()
sites = SitesHelper().get_indexers()
any_site_updated = False
result = {}
for site in sites:
@@ -303,21 +295,24 @@ class SiteChain(ChainBase):
return sub_domain
logger.info("开始同步CookieCloud站点 ...")
cookies, msg = self.cookiecloud.download()
cookies, msg = CookieCloudHelper().download()
if not cookies:
logger.error(f"CookieCloud同步失败{msg}")
if manual:
self.message.put(msg, title="CookieCloud同步失败", role="system")
self.messagehelper.put(msg, title="CookieCloud同步失败", role="system")
return False, msg
# 保存Cookie或新增站点
_update_count = 0
_add_count = 0
_fail_count = 0
siteshelper = SitesHelper()
siteoper = SiteOper()
rsshelper = RssHelper()
for domain, cookie in cookies.items():
# 索引器信息
indexer = self.siteshelper.get_indexer(domain)
indexer = siteshelper.get_indexer(domain)
# 数据库的站点信息
site_info = self.siteoper.get_by_domain(domain)
site_info = siteoper.get_by_domain(domain)
if site_info and site_info.is_active == 1:
# 站点已存在,检查站点连通性
status, msg = self.test(domain)
@@ -327,7 +322,7 @@ class SiteChain(ChainBase):
# 更新站点rss地址
if not site_info.public and not site_info.rss:
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(
rss_url, errmsg = rsshelper.get_rss_link(
url=site_info.url,
cookie=cookie,
ua=site_info.ua or settings.USER_AGENT,
@@ -335,13 +330,13 @@ class SiteChain(ChainBase):
)
if rss_url:
logger.info(f"更新站点 {domain} RSS地址 ...")
self.siteoper.update_rss(domain=domain, rss=rss_url)
siteoper.update_rss(domain=domain, rss=rss_url)
else:
logger.warn(errmsg)
continue
# 更新站点Cookie
logger.info(f"更新站点 {domain} Cookie ...")
self.siteoper.update_cookie(domain=domain, cookies=cookie)
siteoper.update_cookie(domain=domain, cookies=cookie)
_update_count += 1
elif indexer:
if settings.COOKIECLOUD_BLACKLIST and any(
@@ -396,21 +391,21 @@ class SiteChain(ChainBase):
rss_url = None
if not indexer.get("public") and domain_url:
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(url=domain_url,
cookie=cookie,
ua=settings.USER_AGENT,
proxy=proxy)
rss_url, errmsg = rsshelper.get_rss_link(url=domain_url,
cookie=cookie,
ua=settings.USER_AGENT,
proxy=proxy)
if errmsg:
logger.warn(errmsg)
# 插入数据库
logger.info(f"新增站点 {indexer.get('name')} ...")
self.siteoper.add(name=indexer.get("name"),
url=domain_url,
domain=domain,
cookie=cookie,
rss=rss_url,
proxy=1 if proxy else 0,
public=1 if indexer.get("public") else 0)
siteoper.add(name=indexer.get("name"),
url=domain_url,
domain=domain,
cookie=cookie,
rss=rss_url,
proxy=1 if proxy else 0,
public=1 if indexer.get("public") else 0)
_add_count += 1
# 通知站点更新
@@ -423,7 +418,7 @@ class SiteChain(ChainBase):
if _fail_count > 0:
ret_msg += f"{_fail_count}个站点添加失败,下次同步时将重试,也可以手动添加"
if manual:
self.message.put(ret_msg, title="CookieCloud同步成功", role="system")
self.messagehelper.put(ret_msg, title="CookieCloud同步成功", role="system")
logger.info(f"CookieCloud同步成功{ret_msg}")
return True, ret_msg
@@ -442,29 +437,31 @@ class SiteChain(ChainBase):
if str(domain).startswith("http"):
domain = StringUtils.get_url_domain(domain)
# 站点信息
siteinfo = self.siteoper.get_by_domain(domain)
siteoper = SiteOper()
siteshelper = SitesHelper()
siteinfo = siteoper.get_by_domain(domain)
if not siteinfo:
logger.warn(f"未维护站点 {domain} 信息!")
return
# Cookie
cookie = siteinfo.cookie
# 索引器
indexer = self.siteshelper.get_indexer(domain)
indexer = siteshelper.get_indexer(domain)
if not indexer:
logger.warn(f"站点 {domain} 索引器不存在!")
return
# 查询站点图标
site_icon = self.siteoper.get_icon_by_domain(domain)
site_icon = siteoper.get_icon_by_domain(domain)
if not site_icon or not site_icon.base64:
logger.info(f"开始缓存站点 {indexer.get('name')} 图标 ...")
icon_url, icon_base64 = self.__parse_favicon(url=indexer.get("domain"),
cookie=cookie,
ua=settings.USER_AGENT)
if icon_url:
self.siteoper.update_icon(name=indexer.get("name"),
domain=domain,
icon_url=icon_url,
icon_base64=icon_base64)
siteoper.update_icon(name=indexer.get("name"),
domain=domain,
icon_url=icon_url,
icon_base64=icon_base64)
logger.info(f"缓存站点 {indexer.get('name')} 图标成功")
else:
logger.warn(f"缓存站点 {indexer.get('name')} 图标失败")
@@ -484,11 +481,12 @@ class SiteChain(ChainBase):
# 获取主域名中间那段
domain_host = StringUtils.get_url_host(domain)
# 查询以"site.domain_host"开头的配置项,并清除
site_keys = self.systemconfig.all().keys()
systemconfig = SystemConfigOper()
site_keys = systemconfig.all().keys()
for key in site_keys:
if key.startswith(f"site.{domain_host}"):
logger.info(f"清理站点配置:{key}")
self.systemconfig.delete(key)
systemconfig.delete(key)
@eventmanager.register(EventType.SiteUpdated)
def cache_site_userdata(self, event: Event):
@@ -504,7 +502,7 @@ class SiteChain(ChainBase):
return
if str(domain).startswith("http"):
domain = StringUtils.get_url_domain(domain)
indexer = self.siteshelper.get_indexer(domain)
indexer = SitesHelper().get_indexer(domain)
if not indexer:
return
# 刷新站点用户数据
@@ -518,7 +516,8 @@ class SiteChain(ChainBase):
"""
# 检查域名是否可用
domain = StringUtils.get_url_domain(url)
site_info = self.siteoper.get_by_domain(domain)
siteoper = SiteOper()
site_info = siteoper.get_by_domain(domain)
if not site_info:
return False, f"站点【{url}】不存在"
@@ -535,9 +534,9 @@ class SiteChain(ChainBase):
# 统计
seconds = (datetime.now() - start_time).seconds
if state:
self.siteoper.success(domain=domain, seconds=seconds)
siteoper.success(domain=domain, seconds=seconds)
else:
self.siteoper.fail(domain)
siteoper.fail(domain)
return state, message
except Exception as e:
return False, f"{str(e)}"
@@ -593,7 +592,7 @@ class SiteChain(ChainBase):
"""
查询所有站点,发送消息
"""
site_list = self.siteoper.list()
site_list = SiteOper().list()
if not site_list:
self.post_message(Notification(
channel=channel,
@@ -633,7 +632,8 @@ class SiteChain(ChainBase):
if not arg_str.isdigit():
return
site_id = int(arg_str)
site = self.siteoper.get(site_id)
siteoper = SiteOper()
site = siteoper.get(site_id)
if not site:
self.post_message(Notification(
channel=channel,
@@ -641,7 +641,7 @@ class SiteChain(ChainBase):
userid=userid))
return
# 禁用站点
self.siteoper.update(site_id, {
siteoper.update(site_id, {
"is_active": False
})
# 重新发送消息
@@ -655,25 +655,27 @@ class SiteChain(ChainBase):
if not arg_str:
return
arg_strs = str(arg_str).split()
siteoper = SiteOper()
for arg_str in arg_strs:
arg_str = arg_str.strip()
if not arg_str.isdigit():
continue
site_id = int(arg_str)
site = self.siteoper.get(site_id)
site = siteoper.get(site_id)
if not site:
self.post_message(Notification(
channel=channel,
title=f"站点编号 {site_id} 不存在!", userid=userid))
return
# 禁用站点
self.siteoper.update(site_id, {
siteoper.update(site_id, {
"is_active": True
})
# 重新发送消息
self.remote_list(channel=channel, userid=userid, source=source)
def update_cookie(self, site_info: Site,
@staticmethod
def update_cookie(site_info: Site,
username: str, password: str, two_step_code: Optional[str] = None) -> Tuple[bool, str]:
"""
根据用户名密码更新站点Cookie
@@ -684,7 +686,7 @@ class SiteChain(ChainBase):
:return: (是否成功, 错误信息)
"""
# 更新站点Cookie
result = self.cookiehelper.get_site_cookie_ua(
result = CookieHelper().get_site_cookie_ua(
url=site_info.url,
username=username,
password=password,
@@ -695,7 +697,7 @@ class SiteChain(ChainBase):
cookie, ua, msg = result
if not cookie:
return False, msg
self.siteoper.update(site_info.id, {
SiteOper().update(site_info.id, {
"cookie": cookie,
"ua": ua
})
@@ -737,7 +739,7 @@ class SiteChain(ChainBase):
# 站点ID
site_id = int(site_id)
# 站点信息
site_info = self.siteoper.get(site_id)
site_info = SiteOper().get(site_id)
if not site_info:
self.post_message(Notification(
channel=channel,

View File

@@ -14,10 +14,6 @@ class StorageChain(ChainBase):
存储处理链
"""
def __init__(self):
super().__init__()
self.directoryhelper = DirectoryHelper()
def save_config(self, storage: str, conf: dict) -> None:
"""
保存存储配置
@@ -192,7 +188,7 @@ class StorageChain(ChainBase):
# 检查和删除上级目录
if dir_item and len(Path(dir_item.path).parts) > 2:
# 如何目录是所有下载目录、媒体库目录的上级,则不处理
for d in self.directoryhelper.get_dirs():
for d in DirectoryHelper().get_dirs():
if d.download_path and Path(d.download_path).is_relative_to(Path(dir_item.path)):
logger.debug(f"{dir_item.storage}{dir_item.path} 是下载目录本级或上级目录,不删除")
return True

View File

@@ -24,35 +24,20 @@ from app.db.models.subscribe import Subscribe
from app.db.site_oper import SiteOper
from app.db.subscribe_oper import SubscribeOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.message import MessageHelper
from app.helper.subscribe import SubscribeHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import MediaRecognizeConvertEventData
from app.schemas.types import MediaType, SystemConfigKey, MessageChannel, NotificationType, EventType, ChainEventType, ContentType
from app.utils.singleton import Singleton
from app.schemas.types import MediaType, SystemConfigKey, MessageChannel, NotificationType, EventType, ChainEventType, \
ContentType
class SubscribeChain(ChainBase, metaclass=Singleton):
class SubscribeChain(ChainBase):
"""
订阅管理处理链
"""
def __init__(self):
super().__init__()
self._rlock = threading.RLock()
self.downloadchain = DownloadChain()
self.downloadhis = DownloadHistoryOper()
self.searchchain = SearchChain()
self.subscribeoper = SubscribeOper()
self.subscribehelper = SubscribeHelper()
self.torrentschain = TorrentsChain()
self.mediachain = MediaChain()
self.tmdbchain = TmdbChain()
self.message = MessageHelper()
self.systemconfig = SystemConfigOper()
self.torrenthelper = TorrentHelper()
self.siteoper = SiteOper()
_rlock = threading.RLock()
def add(self, title: str, year: str,
mtype: MediaType = None,
@@ -86,11 +71,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if event and event.event_data:
event_data: MediaRecognizeConvertEventData = event.event_data
if event_data.media_dict:
mediachain = MediaChain()
new_id = event_data.media_dict.get("id")
if event_data.convert_type == "themoviedb":
return self.mediachain.recognize_media(meta=_meta, tmdbid=new_id)
return mediachain.recognize_media(meta=_meta, tmdbid=new_id)
elif event_data.convert_type == "douban":
return self.mediachain.recognize_media(meta=_meta, doubanid=new_id)
return mediachain.recognize_media(meta=_meta, doubanid=new_id)
return None
logger.info(f'开始添加订阅,标题:{title} ...')
@@ -110,7 +96,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if not tmdbid:
if doubanid:
# 将豆瓣信息转换为TMDB信息
tmdbinfo = self.mediachain.get_tmdbinfo_by_doubanid(doubanid=doubanid, mtype=mtype)
tmdbinfo = MediaChain().get_tmdbinfo_by_doubanid(doubanid=doubanid, mtype=mtype)
if tmdbinfo:
mediainfo = MediaInfo(tmdb_info=tmdbinfo)
elif mediaid:
@@ -213,7 +199,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"filter_groups") else kwargs.get("filter_groups")
})
# 操作数据库
sid, err_msg = self.subscribeoper.add(mediainfo=mediainfo, season=season, username=username, **kwargs)
sid, err_msg = SubscribeOper().add(mediainfo=mediainfo, season=season, username=username, **kwargs)
if not sid:
logger.error(f'{mediainfo.title_year} {err_msg}')
if not exist_ok and message:
@@ -252,7 +238,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"mediainfo": mediainfo.to_dict(),
})
# 统计订阅
self.subscribehelper.sub_reg_async({
SubscribeHelper().sub_reg_async({
"name": title,
"year": year,
"type": metainfo.type.value,
@@ -270,13 +256,14 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 返回结果
return sid, ""
def exists(self, mediainfo: MediaInfo, meta: MetaBase = None):
@staticmethod
def exists(mediainfo: MediaInfo, meta: MetaBase = None):
"""
判断订阅是否已存在
"""
if self.subscribeoper.exists(tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=meta.begin_season if meta else None):
if SubscribeOper().exists(tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=meta.begin_season if meta else None):
return True
return False
@@ -290,11 +277,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"""
with self._rlock:
logger.debug(f"search lock acquired at {datetime.now()}")
subscribeoper = SubscribeOper()
if sid:
subscribe = self.subscribeoper.get(sid)
subscribe = subscribeoper.get(sid)
subscribes = [subscribe] if subscribe else []
else:
subscribes = self.subscribeoper.list(self.get_states_for_search(state))
subscribes = subscribeoper.list(self.get_states_for_search(state))
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
@@ -349,20 +337,20 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups) or []
or SystemConfigOper().get(SystemConfigKey.BestVersionFilterRuleGroups) or []
else:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups) or []
or SystemConfigOper().get(SystemConfigKey.SubscribeFilterRuleGroups) or []
# 搜索,同时电视剧会过滤掉不需要的剧集
contexts = self.searchchain.process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
contexts = SearchChain().process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
if not contexts:
logger.warn(f'订阅 {subscribe.keyword or subscribe.name} 未搜索到资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
@@ -372,6 +360,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 过滤搜索结果
matched_contexts = []
for context in contexts:
if global_vars.is_system_stopped:
break
torrent_meta = context.meta_info
torrent_info = context.torrent_info
torrent_mediainfo = context.media_info
@@ -403,7 +393,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
continue
# 自动下载
downloads, lefts = self.downloadchain.batch_download(
downloads, lefts = DownloadChain().batch_download(
contexts=matched_contexts,
no_exists=no_exists,
userid=subscribe.username,
@@ -414,7 +404,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
)
# 同步外部修改,更新订阅信息
subscribe = self.subscribeoper.get(subscribe.id)
subscribe = subscribeoper.get(subscribe.id)
# 判断是否应完成订阅
if subscribe:
@@ -423,17 +413,17 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
finally:
# 如果状态为N则更新为R
if subscribe and subscribe.state == 'N':
self.subscribeoper.update(subscribe.id, {'state': 'R'})
subscribeoper.update(subscribe.id, {'state': 'R'})
# 手动触发时发送系统消息
if manual:
if subscribes:
if sid:
self.message.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
self.messagehelper.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
else:
self.message.put('所有订阅搜索完成!', title="订阅搜索", role="system")
self.messagehelper.put('所有订阅搜索完成!', title="订阅搜索", role="system")
else:
self.message.put('没有找到订阅!', title="订阅搜索", role="system")
self.messagehelper.put('没有找到订阅!', title="订阅搜索", role="system")
logger.debug(f"search Lock released at {datetime.now()}")
def update_subscribe_priority(self, subscribe: Subscribe, meta: MetaBase,
@@ -448,7 +438,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 当前下载资源的优先级
priority = max([item.torrent_info.pri_order for item in downloads])
# 订阅存在待定策略,不管是否已完成,均需更新订阅信息
self.subscribeoper.update(subscribe.id, {
SubscribeOper().update(subscribe.id, {
"current_priority": priority,
"last_update": datetime.now().strftime('%Y-%m-%d %H:%M:%S')
})
@@ -505,17 +495,18 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if sites is None:
return
self.match(
self.torrentschain.refresh(sites=sites)
TorrentsChain().refresh(sites=sites)
)
def get_sub_sites(self, subscribe: Subscribe) -> List[int]:
@staticmethod
def get_sub_sites(subscribe: Subscribe) -> List[int]:
"""
获取订阅中涉及的站点清单
:param subscribe: 订阅信息对象
:return: 涉及的站点清单
"""
# 从系统配置获取默认订阅站点
default_sites = self.systemconfig.get(SystemConfigKey.RssSites) or []
default_sites = SystemConfigOper().get(SystemConfigKey.RssSites) or []
# 如果订阅未指定站点,直接返回默认站点
if not subscribe.sites:
return default_sites
@@ -535,7 +526,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
:return: 返回[]代表所有站点命中返回None代表没有订阅
"""
# 查询所有订阅
subscribes = self.subscribeoper.list(self.get_states_for_search('R'))
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
if not subscribes:
return None
ret_sites = []
@@ -560,26 +551,28 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
with self._rlock:
logger.debug(f"match lock acquired at {datetime.now()}")
# 所有订阅
subscribes = self.subscribeoper.list(self.get_states_for_search('R'))
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
# 预识别所有未识别的种子
processed_torrents = {}
processed_torrents: Dict[str, List[Context]] = {}
for domain, contexts in torrents.items():
if global_vars.is_system_stopped:
break
processed_torrents[domain] = []
for context in contexts:
# 复制上下文避免修改原始数据
_context = copy.deepcopy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
if global_vars.is_system_stopped:
break
# 如果种子未识别,尝试识别
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
torrent_mediainfo = self.recognize_media(meta=torrent_meta)
if torrent_mediainfo:
if not context.media_info or (not context.media_info.tmdb_id
and not context.media_info.douban_id):
re_mediainfo = self.recognize_media(meta=context.meta_info)
if re_mediainfo:
# 清理多余信息
re_mediainfo.clear()
# 更新种子缓存
context.media_info = torrent_mediainfo
context.media_info = re_mediainfo
# 添加已预处理
processed_torrents[domain].append(_context)
processed_torrents[domain].append(context)
# 遍历订阅
for subscribe in subscribes:
@@ -599,7 +592,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 订阅的站点域名列表
domains = []
if subscribe.sites:
domains = self.siteoper.get_domains_by_ids(subscribe.sites)
domains = SiteOper().get_domains_by_ids(subscribe.sites)
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
@@ -618,6 +611,9 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if exist_flag:
continue
# 清理多余信息
mediainfo.clear()
# 订阅识别词
if subscribe.custom_words:
custom_words_list = subscribe.custom_words.split("\n")
@@ -626,6 +622,9 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 遍历预识别后的种子
_match_context = []
torrenthelper = TorrentHelper()
systemconfig = SystemConfigOper()
wordsmatcher = WordsMatcher()
for domain, contexts in processed_torrents.items():
if global_vars.is_system_stopped:
break
@@ -633,8 +632,10 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
continue
logger.debug(f'开始匹配站点:{domain},共缓存了 {len(contexts)} 个种子...')
for context in contexts:
if global_vars.is_system_stopped:
break
# 提取信息
_context = copy.deepcopy(context)
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
@@ -648,8 +649,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = WordsMatcher().prepare(torrent_meta.org_string,
custom_words=custom_words_list)
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
@@ -657,27 +658,29 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
context.meta_info = torrent_meta
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
context.media_info = torrent_mediainfo
_context.media_info = torrent_mediainfo
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
logger.warn(
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if self.torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
# 更新种子缓存
context.media_info = mediainfo
_context.media_info = mediainfo
else:
continue
@@ -735,17 +738,17 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
continue
# 匹配订阅附加参数
if not self.torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
@@ -781,22 +784,23 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 开始批量择优下载
logger.info(f'{mediainfo.title_year} 匹配完成,共匹配到{len(_match_context)}个资源')
downloads, lefts = self.downloadchain.batch_download(contexts=_match_context,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
downloads, lefts = DownloadChain().batch_download(contexts=_match_context,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 同步外部修改,更新订阅信息
subscribe = self.subscribeoper.get(subscribe.id)
subscribe = SubscribeOper().get(subscribe.id)
# 判断是否要完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
logger.debug(f"match Lock released at {datetime.now()}")
def check(self):
@@ -804,7 +808,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
定时检查订阅,更新订阅信息
"""
# 查询所有订阅
subscribes = self.subscribeoper.list()
subscribeoper = SubscribeOper()
subscribes = subscribeoper.list()
if not subscribes:
# 没有订阅不运行
return
@@ -843,7 +848,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
total_episode = subscribe.total_episode
lack_episode = subscribe.lack_episode
# 更新TMDB信息
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"name": mediainfo.title,
"year": mediainfo.year,
"vote": mediainfo.vote_average,
@@ -857,28 +862,32 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
})
logger.info(f'{subscribe.name} 订阅元数据更新完成')
def follow(self):
@staticmethod
def follow():
"""
刷新follow的用户分享并自动添加订阅
"""
follow_users: List[str] = self.systemconfig.get(SystemConfigKey.FollowSubscribers)
follow_users: List[str] = SystemConfigOper().get(SystemConfigKey.FollowSubscribers)
if not follow_users:
return
share_subs = self.subscribehelper.get_shares()
share_subs = SubscribeHelper().get_shares()
logger.info(f'开始刷新follow用户分享订阅 ...')
success_count = 0
subscribeoper = SubscribeOper()
for share_sub in share_subs:
if global_vars.is_system_stopped:
break
uid = share_sub.get("share_uid")
if uid and uid in follow_users:
# 订阅已存在则跳过
if self.subscribeoper.exists(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
if subscribeoper.exists(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
continue
# 已经订阅过跳过
if self.subscribeoper.exist_history(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
if subscribeoper.exist_history(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
continue
# 去除无效属性
for key in list(share_sub.keys()):
@@ -919,7 +928,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
logger.error(f'follow用户分享订阅 {title} 添加失败:{message}')
logger.info(f'follow用户分享订阅刷新完成共添加 {success_count} 个订阅')
def __update_subscribe_note(self, subscribe: Subscribe, downloads: Optional[List[Context]]):
@staticmethod
def __update_subscribe_note(subscribe: Subscribe, downloads: Optional[List[Context]]):
"""
更新已下载信息到note字段
"""
@@ -951,7 +961,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
note = list(set(note).union(set(items)))
# 更新订阅
if note:
self.subscribeoper.update(subscribe.id, {
SubscribeOper().update(subscribe.id, {
"note": note
})
@@ -975,7 +985,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
return note
return []
def __update_lack_episodes(self, lefts: Dict[Union[int, str], Dict[int, schemas.NotExistMediaInfo]],
@staticmethod
def __update_lack_episodes(lefts: Dict[Union[int, str], Dict[int, schemas.NotExistMediaInfo]],
subscribe: Subscribe,
mediainfo: MediaInfo,
update_date: Optional[bool] = False):
@@ -1008,7 +1019,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
update_data["lack_episode"] = lack_episode
# 更新数据库
if update_data:
self.subscribeoper.update(subscribe.id, update_data)
SubscribeOper().update(subscribe.id, update_data)
def __finish_subscribe(self, subscribe: Subscribe, mediainfo: MediaInfo, meta: MetaBase):
"""
@@ -1021,9 +1032,10 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
msgstr = "订阅" if not subscribe.best_version else "洗版"
logger.info(f'{mediainfo.title_year} 完成{msgstr}')
# 新增订阅历史
self.subscribeoper.add_history(**subscribe.to_dict())
subscribeoper = SubscribeOper()
subscribeoper.add_history(**subscribe.to_dict())
# 删除订阅
self.subscribeoper.delete(subscribe.id)
subscribeoper.delete(subscribe.id)
# 发送通知
if mediainfo.type == MediaType.TV:
link = settings.MP_DOMAIN('#/subscribe/tv?tab=mysub')
@@ -1050,7 +1062,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"mediainfo": mediainfo.to_dict(),
})
# 统计订阅
self.subscribehelper.sub_done_async({
SubscribeHelper().sub_done_async({
"tmdbid": mediainfo.tmdb_id,
"doubanid": mediainfo.douban_id
})
@@ -1060,7 +1072,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"""
查询订阅并发送消息
"""
subscribes = self.subscribeoper.list()
subscribes = SubscribeOper().list()
if not subscribes:
self.post_message(schemas.Notification(channel=channel,
source=source,
@@ -1094,20 +1106,22 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"[id]为订阅编号", userid=userid))
return
arg_strs = str(arg_str).split()
subscribeoper = SubscribeOper()
subscribehelper = SubscribeHelper()
for arg_str in arg_strs:
arg_str = arg_str.strip()
if not arg_str.isdigit():
continue
subscribe_id = int(arg_str)
subscribe = self.subscribeoper.get(subscribe_id)
subscribe = subscribeoper.get(subscribe_id)
if not subscribe:
self.post_message(schemas.Notification(channel=channel, source=source,
title=f"订阅编号 {subscribe_id} 不存在!", userid=userid))
return
# 删除订阅
self.subscribeoper.delete(subscribe_id)
subscribeoper.delete(subscribe_id)
# 统计订阅
self.subscribehelper.sub_done_async({
subscribehelper.sub_done_async({
"tmdbid": subscribe.tmdbid,
"doubanid": subscribe.doubanid
})
@@ -1171,6 +1185,9 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
new_episodes = list(range(max(start_episode, start), total_episode + 1))
# 与原集列表取交集
episodes = list(set(episode_list).intersection(set(new_episodes)))
# 交集为空时,说明订阅的剧集均已入库
if not episodes:
return True, {}
# 更新集合
no_exists[mediakey][begin_season] = schemas.NotExistMediaInfo(
season=begin_season,
@@ -1233,13 +1250,14 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
site_id = event_data.get("site_id")
if not site_id:
return
subscribeoper = SubscribeOper()
if site_id == "*":
# 站点被重置
SystemConfigOper().set(SystemConfigKey.RssSites, [])
for subscribe in self.subscribeoper.list():
for subscribe in subscribeoper.list():
if not subscribe.sites:
continue
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"sites": []
})
return
@@ -1249,14 +1267,14 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
selected_sites.remove(site_id)
SystemConfigOper().set(SystemConfigKey.RssSites, selected_sites)
# 查询所有订阅
for subscribe in self.subscribeoper.list():
for subscribe in subscribeoper.list():
if not subscribe.sites:
continue
sites = subscribe.sites or []
if site_id not in sites:
continue
sites.remove(site_id)
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"sites": sites
})
@@ -1281,12 +1299,13 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
return None
return value.get(default_config_key) or None
def get_params(self, subscribe: Subscribe):
@staticmethod
def get_params(subscribe: Subscribe):
"""
获取订阅默认参数
"""
# 默认过滤规则
default_rule = self.systemconfig.get(SystemConfigKey.SubscribeDefaultParams) or {}
default_rule = SystemConfigOper().get(SystemConfigKey.SubscribeDefaultParams) or {}
return {
key: value for key, value in {
"include": subscribe.include or default_rule.get("include"),
@@ -1314,7 +1333,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
episodes: Dict[int, schemas.SubscribeEpisodeInfo] = {}
if subscribe.tmdbid and subscribe.type == MediaType.TV.value:
# 查询TMDB中的集信息
tmdb_episodes = self.tmdbchain.tmdb_episodes(
tmdb_episodes = TmdbChain().tmdb_episodes(
tmdbid=subscribe.tmdbid,
season=subscribe.season,
episode_group=subscribe.episode_group
@@ -1339,11 +1358,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
episodes[0] = info
# 所有下载记录
download_his = self.downloadhis.get_by_mediaid(tmdbid=subscribe.tmdbid, doubanid=subscribe.doubanid)
downloadhis = DownloadHistoryOper()
download_his = downloadhis.get_by_mediaid(tmdbid=subscribe.tmdbid, doubanid=subscribe.doubanid)
if download_his:
for his in download_his:
# 查询下载文件
files = self.downloadhis.get_files_by_hash(his.download_hash)
files = downloadhis.get_files_by_hash(his.download_hash)
if files:
for file in files:
# 识别文件名
@@ -1437,7 +1457,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
subscribe.season: subscribe.total_episode
}
# 查询媒体库缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(
exist_flag, no_exists = DownloadChain().get_no_exists_info(
meta=meta,
mediainfo=mediainfo,
totals=totals

View File

@@ -8,24 +8,18 @@ from app.core.config import settings
from app.log import logger
from app.schemas import Notification, MessageChannel
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.system import SystemUtils
from helper.system import SystemHelper
from app.helper.system import SystemHelper
from version import FRONTEND_VERSION, APP_VERSION
class SystemChain(ChainBase, metaclass=Singleton):
class SystemChain(ChainBase):
"""
系统级处理链
"""
_restart_file = "__system_restart__"
def __init__(self):
super().__init__()
# 重启完成检测
self.restart_finish()
def remote_clear_cache(self, channel: MessageChannel, userid: Union[int, str], source: Optional[str] = None):
"""
清理系统缓存
@@ -38,6 +32,8 @@ class SystemChain(ChainBase, metaclass=Singleton):
"""
重启系统
"""
from app.core.config import global_vars
if channel and userid:
self.post_message(Notification(channel=channel, source=source,
title="系统正在重启,请耐心等候!", userid=userid))
@@ -46,6 +42,8 @@ class SystemChain(ChainBase, metaclass=Singleton):
"channel": channel.value,
"userid": userid
}, self._restart_file)
# 设置停止标志,通知所有模块准备停止
global_vars.stop_system()
# 重启
SystemHelper.restart()

View File

@@ -3,13 +3,11 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.core.cache import cached
from app.core.context import MediaInfo
from app.schemas import MediaType
from app.utils.singleton import Singleton
class TmdbChain(ChainBase, metaclass=Singleton):
class TmdbChain(ChainBase):
"""
TheMovieDB处理链单例运行
"""
@@ -145,7 +143,6 @@ class TmdbChain(ChainBase, metaclass=Singleton):
"""
return self.run_module("tmdb_person_credits", person_id=person_id, page=page)
@cached(maxsize=1, ttl=3600)
def get_random_wallpager(self) -> Optional[str]:
"""
获取随机壁纸缓存1个小时
@@ -159,7 +156,6 @@ class TmdbChain(ChainBase, metaclass=Singleton):
return info.backdrop_path
return None
@cached(maxsize=1, ttl=3600)
def get_trending_wallpapers(self, num: Optional[int] = 10) -> List[str]:
"""
获取所有流行壁纸

View File

@@ -2,8 +2,6 @@ import re
import traceback
from typing import Dict, List, Union, Optional
from cachetools import cached, TTLCache
from app.chain import ChainBase
from app.chain.media import MediaChain
from app.core.config import settings, global_vars
@@ -17,11 +15,10 @@ from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import Notification
from app.schemas.types import SystemConfigKey, MessageChannel, NotificationType, MediaType
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
class TorrentsChain(ChainBase, metaclass=Singleton):
class TorrentsChain(ChainBase):
"""
站点首页或RSS种子处理链服务于订阅、刷流等
"""
@@ -29,15 +26,6 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
_spider_file = "__torrents_cache__"
_rss_file = "__rss_cache__"
def __init__(self):
super().__init__()
self.siteshelper = SitesHelper()
self.siteoper = SiteOper()
self.rsshelper = RssHelper()
self.systemconfig = SystemConfigOper()
self.mediachain = MediaChain()
self.torrenthelper = TorrentHelper()
@property
def cache_file(self) -> str:
"""
@@ -81,39 +69,37 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
self.remove_cache(self._rss_file)
logger.info(f'种子缓存数据清理完成')
@cached(cache=TTLCache(maxsize=128, ttl=595))
def browse(self, domain: str, keyword: Optional[str] = None, cat: Optional[str] = None,
page: Optional[int] = 0) -> List[TorrentInfo]:
"""
浏览站点首页内容返回种子清单TTL缓存10分钟
浏览站点首页内容返回种子清单TTL缓存5分钟
:param domain: 站点域名
:param keyword: 搜索标题
:param cat: 搜索分类
:param page: 页码
"""
logger.info(f'开始获取站点 {domain} 最新种子 ...')
site = self.siteshelper.get_indexer(domain)
site = SitesHelper().get_indexer(domain)
if not site:
logger.error(f'站点 {domain} 不存在!')
return []
return self.refresh_torrents(site=site, keyword=keyword, cat=cat, page=page)
@cached(cache=TTLCache(maxsize=128, ttl=295))
def rss(self, domain: str) -> List[TorrentInfo]:
"""
获取站点RSS内容返回种子清单TTL缓存5分钟
获取站点RSS内容返回种子清单TTL缓存3分钟
:param domain: 站点域名
"""
logger.info(f'开始获取站点 {domain} RSS ...')
site = self.siteshelper.get_indexer(domain)
site = SitesHelper().get_indexer(domain)
if not site:
logger.error(f'站点 {domain} 不存在!')
return []
if not site.get("rss"):
logger.error(f'站点 {domain} 未配置RSS地址')
return []
rss_items = self.rsshelper.parse(site.get("rss"), True if site.get("proxy") else False,
timeout=int(site.get("timeout") or 30))
rss_items = RssHelper().parse(site.get("rss"), True if site.get("proxy") else False,
timeout=int(site.get("timeout") or 30))
if rss_items is None:
# rss过期尝试保留原配置生成新的rss
self.__renew_rss_url(domain=domain, site=site)
@@ -156,7 +142,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 刷新站点
if not sites:
sites = self.systemconfig.get(SystemConfigKey.RssSites) or []
sites = SystemConfigOper().get(SystemConfigKey.RssSites) or []
# 读取缓存
torrents_cache = self.get_torrents()
@@ -164,12 +150,13 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 缓存过滤掉无效种子
for _domain, _torrents in torrents_cache.items():
torrents_cache[_domain] = [_torrent for _torrent in _torrents
if not self.torrenthelper.is_invalid(_torrent.torrent_info.enclosure)]
if not TorrentHelper().is_invalid(_torrent.torrent_info.enclosure)]
# 所有站点索引
indexers = self.siteshelper.get_indexers()
indexers = SitesHelper().get_indexers()
# 需要刷新的站点domain
domains = []
# 遍历站点缓存资源
for indexer in indexers:
if global_vars.is_system_stopped:
@@ -188,13 +175,13 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 按pubdate降序排列
torrents.sort(key=lambda x: x.pubdate or '', reverse=True)
# 取前N条
torrents = torrents[:settings.CACHE_CONF["refresh"]]
torrents = torrents[:settings.CONF["refresh"]]
if torrents:
# 过滤出没有处理过的种子
# 过滤出没有处理过的种子 - 优化:使用集合查找,避免重复创建字符串列表
cached_signatures = {f'{t.torrent_info.title}{t.torrent_info.description}'
for t in torrents_cache.get(domain) or []}
torrents = [torrent for torrent in torrents
if f'{torrent.title}{torrent.description}'
not in [f'{t.torrent_info.title}{t.torrent_info.description}'
for t in torrents_cache.get(domain) or []]]
if f'{torrent.title}{torrent.description}' not in cached_signatures]
if torrents:
logger.info(f'{indexer.get("name")}{len(torrents)} 个新种子')
else:
@@ -213,12 +200,12 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
and torrent.category == MediaType.TV.value:
meta.type = MediaType.TV
# 识别媒体信息
mediainfo: MediaInfo = self.mediachain.recognize_by_meta(meta)
mediainfo: MediaInfo = MediaChain().recognize_by_meta(meta)
if not mediainfo:
logger.warn(f'{torrent.title} 未识别到媒体信息')
# 存储空的媒体信息
mediainfo = MediaInfo()
# 清理多余数据
# 清理多余数据,减少内存占用
mediainfo.clear()
# 上下文
context = Context(meta_info=meta, media_info=mediainfo, torrent_info=torrent)
@@ -228,10 +215,8 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
else:
torrents_cache[domain].append(context)
# 如果超过了限制条数则移除掉前面的
if len(torrents_cache[domain]) > settings.CACHE_CONF["torrents"]:
torrents_cache[domain] = torrents_cache[domain][-settings.CACHE_CONF["torrents"]:]
# 回收资源
del torrents
if len(torrents_cache[domain]) > settings.CONF["torrents"]:
torrents_cache[domain] = torrents_cache[domain][-settings.CONF["torrents"]:]
else:
logger.info(f'{indexer.get("name")} 没有获取到种子')
@@ -244,6 +229,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 去除不在站点范围内的缓存种子
if sites and torrents_cache:
torrents_cache = {k: v for k, v in torrents_cache.items() if k in domains}
return torrents_cache
def __renew_rss_url(self, domain: str, site: dict):
@@ -254,7 +240,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# RSS链接过期
logger.error(f"站点 {domain} RSS链接已过期正在尝试自动获取")
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(
rss_url, errmsg = RssHelper().get_rss_link(
url=site.get("url"),
cookie=site.get("cookie"),
ua=site.get("ua") or settings.USER_AGENT,
@@ -268,7 +254,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 获取过期rss除去passkey部分
new_rss = re.sub(r'&passkey=([a-zA-Z0-9]+)', f'&passkey={new_passkey}', site.get("rss"))
logger.info(f"更新站点 {domain} RSS地址 ...")
self.siteoper.update_rss(domain=domain, rss=new_rss)
SiteOper().update_rss(domain=domain, rss=new_rss)
else:
# 发送消息
self.post_message(

View File

@@ -328,7 +328,8 @@ class JobManager:
# 计算状态为完成的任务数
if __mediaid__ not in self._job_view:
return 0
return sum([task.fileitem.size for task in self._job_view[__mediaid__].tasks if task.state == "completed" and task.fileitem.size is not None])
return sum([task.fileitem.size for task in self._job_view[__mediaid__].tasks if
task.state == "completed" and task.fileitem.size is not None])
def total(self) -> int:
"""
@@ -371,14 +372,6 @@ class TransferChain(ChainBase, metaclass=Singleton):
def __init__(self):
super().__init__()
self.downloadhis = DownloadHistoryOper()
self.transferhis = TransferHistoryOper()
self.progress = ProgressHelper()
self.mediachain = MediaChain()
self.tmdbchain = TmdbChain()
self.storagechain = StorageChain()
self.systemconfig = SystemConfigOper()
self.directoryhelper = DirectoryHelper()
self.jobview = JobManager()
# 启动整理任务
@@ -397,11 +390,12 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
整理完成后处理
"""
transferhis = TransferHistoryOper()
if not transferinfo.success:
# 转移失败
logger.warn(f"{task.fileitem.name} 入库失败:{transferinfo.message}")
# 新增转移失败历史记录
self.transferhis.add_fail(
transferhis.add_fail(
fileitem=task.fileitem,
mode=transferinfo.transfer_type if transferinfo else '',
downloader=task.downloader,
@@ -428,7 +422,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.info(f"{task.fileitem.name} 入库成功:{transferinfo.target_diritem.path}")
# 新增转移成功历史记录
self.transferhis.add_success(
transferhis.add_success(
fileitem=task.fileitem,
mode=transferinfo.transfer_type if transferinfo else '',
downloader=task.downloader,
@@ -457,6 +451,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
tasks = self.jobview.success_tasks(task.mediainfo, task.meta.begin_season)
# 记录已处理的种子hash
processed_hashes = set()
storagechain = StorageChain()
for t in tasks:
# 下载器hash
if t.download_hash and t.download_hash not in processed_hashes:
@@ -465,7 +460,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.info(f"移动模式删除种子成功:{t.download_hash} ")
# 删除残留目录
if t.fileitem:
self.storagechain.delete_media_file(t.fileitem, delete_self=False)
storagechain.delete_media_file(t.fileitem, delete_self=False)
# 整理完成且有成功的任务时
if self.jobview.is_finished(task):
# 发送通知,实时手动整理时不发
@@ -543,6 +538,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 失败数量
fail_num = 0
progress = ProgressHelper()
while not global_vars.is_system_stopped:
try:
item: TransferQueue = self._queue.get(block=False)
@@ -556,24 +553,24 @@ class TransferChain(ChainBase, metaclass=Singleton):
if __queue_start:
logger.info("开始整理队列处理...")
# 启动进度
self.progress.start(ProgressKey.FileTransfer)
progress.start(ProgressKey.FileTransfer)
# 重置计数
processed_num = 0
fail_num = 0
total_num = self.jobview.total()
__process_msg = f"开始整理队列处理,当前共 {total_num} 个文件 ..."
logger.info(__process_msg)
self.progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
# 队列已开始
__queue_start = False
# 更新进度
__process_msg = f"正在整理 {fileitem.name} ..."
logger.info(__process_msg)
self.progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
# 整理
state, err_msg = self.__handle_transfer(task=task, callback=item.callback)
if not state:
@@ -583,18 +580,18 @@ class TransferChain(ChainBase, metaclass=Singleton):
processed_num += 1
__process_msg = f"{fileitem.name} 整理完成"
logger.info(__process_msg)
self.progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
except queue.Empty:
if not __queue_start:
# 结束进度
__end_msg = f"整理队列处理完成,共整理 {processed_num} 个文件,失败 {fail_num}"
logger.info(__end_msg)
self.progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
self.progress.end(ProgressKey.FileTransfer)
progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
# 重置计数
processed_num = 0
fail_num = 0
@@ -614,6 +611,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
try:
# 识别
transferhis = TransferHistoryOper()
if not task.mediainfo:
mediainfo = None
download_history = task.download_history
@@ -633,7 +631,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
mediainfo.category = download_history.media_category
else:
# 识别媒体信息
mediainfo = self.mediachain.recognize_by_meta(task.meta)
mediainfo = MediaChain().recognize_by_meta(task.meta)
# 更新媒体图片
if mediainfo:
@@ -641,7 +639,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not mediainfo:
# 新增整理失败历史记录
his = self.transferhis.add_fail(
his = transferhis.add_fail(
fileitem=task.fileitem,
mode=task.transfer_type,
meta=task.meta,
@@ -661,8 +659,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 如果未开启新增已入库媒体是否跟随TMDB信息变化则根据tmdbid查询之前的title
if not settings.SCRAP_FOLLOW_TMDB:
transfer_history = self.transferhis.get_by_type_tmdbid(tmdbid=mediainfo.tmdb_id,
mtype=mediainfo.type.value)
transfer_history = transferhis.get_by_type_tmdbid(tmdbid=mediainfo.tmdb_id,
mtype=mediainfo.type.value)
if transfer_history:
mediainfo.title = transfer_history.title
@@ -682,7 +680,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 默认值1
if season_num is None:
season_num = 1
task.episodes_info = self.tmdbchain.tmdb_episodes(
task.episodes_info = TmdbChain().tmdb_episodes(
tmdbid=task.mediainfo.tmdb_id,
season=season_num,
episode_group=task.mediainfo.episode_group
@@ -692,15 +690,15 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not task.target_directory:
if task.target_path:
# 指定目标路径,`手动整理`场景下使用,忽略源目录匹配,使用指定目录匹配
task.target_directory = self.directoryhelper.get_dir(media=task.mediainfo,
dest_path=task.target_path,
target_storage=task.target_storage)
task.target_directory = DirectoryHelper().get_dir(media=task.mediainfo,
dest_path=task.target_path,
target_storage=task.target_storage)
else:
# 启用源目录匹配时,根据源目录匹配下载目录,否则按源目录同盘优先原则,如无源目录,则根据媒体信息获取目标目录
task.target_directory = self.directoryhelper.get_dir(media=task.mediainfo,
storage=task.fileitem.storage,
src_path=Path(task.fileitem.path),
target_storage=task.target_storage)
task.target_directory = DirectoryHelper().get_dir(media=task.mediainfo,
storage=task.fileitem.storage,
src_path=Path(task.fileitem.path),
target_storage=task.target_storage)
if not task.target_storage and task.target_directory:
task.target_storage = task.target_directory.library_storage
@@ -784,7 +782,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 全局锁,避免重复处理
with downloader_lock:
# 获取下载器监控目录
download_dirs = self.directoryhelper.get_download_dirs()
download_dirs = DirectoryHelper().get_download_dirs()
# 如果没有下载器监控的目录则不处理
if not any(dir_info.monitor_type == "downloader" and dir_info.storage == "local"
for dir_info in download_dirs):
@@ -820,7 +818,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.debug(f"文件 {file_path} 不在下载器监控目录中,不通过下载器进行整理")
continue
# 查询下载记录识别情况
downloadhis: DownloadHistory = self.downloadhis.get_by_hash(torrent.hash)
downloadhis: DownloadHistory = DownloadHistoryOper().get_by_hash(torrent.hash)
if downloadhis:
# 类型
try:
@@ -859,26 +857,31 @@ class TransferChain(ChainBase, metaclass=Singleton):
)
# 设置下载任务状态
if state:
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
if not state:
logger.warn(f"整理下载器任务失败:{torrent.hash} - {errmsg}")
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
# 结束
logger.info("所有下载器中下载完成的文件已整理完成")
return True
def __get_trans_fileitems(self, fileitem: FileItem) -> List[Tuple[FileItem, bool]]:
def __get_trans_fileitems(
self, fileitem: FileItem, depth: int = 1
) -> List[Tuple[FileItem, bool]]:
"""
获取整理目录或文件列表
:param fileitem: 文件项
:param depth: 递归深度默认为1
"""
storagechain = StorageChain()
def __is_bluray_dir(_fileitem: FileItem) -> bool:
def __contains_bluray_sub(_fileitems: List[FileItem]) -> bool:
"""
判断是不是蓝光目录
判断是否包含蓝光目录
"""
subs = self.storagechain.list_files(_fileitem)
if subs:
for sub in subs:
if _fileitems:
for sub in _fileitems:
if sub.type == "dir" and sub.name in ["BDMV", "CERTIFICATE"]:
return True
return False
@@ -895,10 +898,10 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
for p in _path.parents:
if p.name == "BDMV":
return self.storagechain.get_file_item(storage=_storage, path=p.parent)
return storagechain.get_file_item(storage=_storage, path=p.parent)
return None
if not self.storagechain.get_item(fileitem):
if not storagechain.get_item(fileitem):
logger.warn(f"目录或文件不存在:{fileitem.path}")
return []
@@ -913,25 +916,22 @@ class TransferChain(ChainBase, metaclass=Singleton):
return [(fileitem, False)]
# 蓝光原盘根目录
if __is_bluray_dir(fileitem):
sub_items = storagechain.list_files(fileitem) or []
if __contains_bluray_sub(sub_items):
return [(fileitem, True)]
# 需要整理的文件项列表
trans_items = []
# 先检查当前目录的下级目录,以支持合集的情况
for sub_dir in self.storagechain.list_files(fileitem):
for sub_dir in sub_items if depth >= 1 else []:
if sub_dir.type == "dir":
if __is_bluray_dir(sub_dir):
trans_items.append((sub_dir, True))
else:
trans_items.append((sub_dir, False))
trans_items.extend(self.__get_trans_fileitems(sub_dir, depth=depth - 1))
if not trans_items:
# 没有有效子目录,直接整理当前目录
trans_items.append((fileitem, False))
else:
# 有子目录时,把当前目录的文件添加到整理任务中
sub_items = self.storagechain.list_files(fileitem)
if sub_items:
trans_items.extend([(f, False) for f in sub_items if f.type == "file"])
@@ -993,11 +993,13 @@ class TransferChain(ChainBase, metaclass=Singleton):
offset=epformat.offset) if epformat else None
# 整理屏蔽词
transfer_exclude_words = self.systemconfig.get(SystemConfigKey.TransferExcludeWords)
transfer_exclude_words = SystemConfigOper().get(SystemConfigKey.TransferExcludeWords)
# 汇总错误信息
err_msgs: List[str] = []
# 待整理目录或文件项
trans_items = self.__get_trans_fileitems(fileitem)
trans_items = self.__get_trans_fileitems(
fileitem, depth=2 # 为解决 issue#4371 深度至少需要>=2
)
# 待整理的文件列表
file_items: List[Tuple[FileItem, bool]] = []
@@ -1010,7 +1012,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 如果是目录且不是⼀蓝光原盘,获取所有文件并整理
if trans_item.type == "dir" and not bluray_dir:
# 遍历获取下载目录所有文件(递归)
if files := self.storagechain.list_files(trans_item, recursion=True):
if files := StorageChain().list_files(trans_item, recursion=True):
file_items.extend([(file, False) for file in files])
else:
file_items.append((trans_item, bluray_dir))
@@ -1059,7 +1061,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 整理成功的不再处理
if not force:
transferd = self.transferhis.get_by_src(file_item.path, storage=file_item.storage)
transferd = TransferHistoryOper().get_by_src(file_item.path, storage=file_item.storage)
if transferd:
if not transferd.status:
all_success = False
@@ -1095,14 +1097,15 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 根据父路径获取下载历史
download_history = None
downloadhis = DownloadHistoryOper()
if bluray_dir:
# 蓝光原盘,按目录名查询
download_history = self.downloadhis.get_by_path(str(file_path))
download_history = downloadhis.get_by_path(str(file_path))
else:
# 按文件全路径查询
download_file = self.downloadhis.get_file_by_fullpath(str(file_path))
download_file = downloadhis.get_file_by_fullpath(str(file_path))
if download_file:
download_history = self.downloadhis.get_by_hash(download_file.download_hash)
download_history = downloadhis.get_by_hash(download_file.download_hash)
# 获取下载Hash
if download_history and (not downloader or not download_hash):
@@ -1145,12 +1148,13 @@ class TransferChain(ChainBase, metaclass=Singleton):
fail_num = 0
# 启动进度
self.progress.start(ProgressKey.FileTransfer)
progress = ProgressHelper()
progress.start(ProgressKey.FileTransfer)
__process_msg = f"开始整理,共 {total_num} 个文件 ..."
logger.info(__process_msg)
self.progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
for transfer_task in transfer_tasks:
if global_vars.is_system_stopped:
@@ -1160,9 +1164,9 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 更新进度
__process_msg = f"正在整理 {processed_num + fail_num + 1}/{total_num}{transfer_task.fileitem.name} ..."
logger.info(__process_msg)
self.progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
state, err_msg = self.__handle_transfer(
task=transfer_task,
callback=self.__default_callback
@@ -1178,10 +1182,10 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 整理结束
__end_msg = f"整理队列处理完成,共整理 {total_num} 个文件,失败 {fail_num}"
logger.info(__end_msg)
self.progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
self.progress.end(ProgressKey.FileTransfer)
progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
return all_success, "".join(err_msgs)
@@ -1236,7 +1240,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
:param mediaid: TMDB ID/豆瓣ID
"""
# 查询历史记录
history: TransferHistory = self.transferhis.get(logid)
history: TransferHistory = TransferHistoryOper().get(logid)
if not history:
logger.error(f"整理记录不存在ID{logid}")
return False, "整理记录不存在"
@@ -1252,7 +1256,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 更新媒体图片
self.obtain_images(mediainfo=mediainfo)
else:
mediainfo = self.mediachain.recognize_by_path(str(src_path), episode_group=history.episode_group)
mediainfo = MediaChain().recognize_by_path(str(src_path), episode_group=history.episode_group)
if not mediainfo:
return False, f"未识别到媒体信息,类型:{mtype.value}id{mediaid}"
# 重新执行整理
@@ -1262,7 +1266,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if history.dest_fileitem:
# 解析目标文件对象
dest_fileitem = FileItem(**history.dest_fileitem)
self.storagechain.delete_file(dest_fileitem)
StorageChain().delete_file(dest_fileitem)
# 强制整理
if history.src_fileitem:
@@ -1317,18 +1321,19 @@ class TransferChain(ChainBase, metaclass=Singleton):
if tmdbid or doubanid:
# 有输入TMDBID时单个识别
# 识别媒体信息
mediainfo: MediaInfo = self.mediachain.recognize_media(tmdbid=tmdbid, doubanid=doubanid,
mtype=mtype, episode_group=episode_group)
mediainfo: MediaInfo = MediaChain().recognize_media(tmdbid=tmdbid, doubanid=doubanid,
mtype=mtype, episode_group=episode_group)
if not mediainfo:
return False, f"媒体信息识别失败tmdbid{tmdbid}doubanid{doubanid}type: {mtype.value}"
else:
# 更新媒体图片
self.obtain_images(mediainfo=mediainfo)
# 开始进度
self.progress.start(ProgressKey.FileTransfer)
self.progress.update(value=0,
text=f"开始整理 {fileitem.path} ...",
key=ProgressKey.FileTransfer)
progress = ProgressHelper()
progress.start(ProgressKey.FileTransfer)
progress.update(value=0,
text=f"开始整理 {fileitem.path} ...",
key=ProgressKey.FileTransfer)
# 开始整理
state, errmsg = self.do_transfer(
fileitem=fileitem,
@@ -1349,7 +1354,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not state:
return False, errmsg
self.progress.end(ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
logger.info(f"{fileitem.path} 整理完成")
return True, ""
else:
@@ -1370,7 +1375,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
return state, errmsg
def send_transfer_message(self, meta: MetaBase, mediainfo: MediaInfo,
transferinfo: TransferInfo, season_episode: Optional[str] = None, username: Optional[str] = None):
transferinfo: TransferInfo, season_episode: Optional[str] = None,
username: Optional[str] = None):
"""
发送入库成功的消息
"""

View File

@@ -1,9 +1,9 @@
from typing import List
from app.chain import ChainBase
from app.utils.singleton import Singleton
class TvdbChain(ChainBase, metaclass=Singleton):
class TvdbChain(ChainBase):
"""
Tvdb处理链单例运行
"""

View File

@@ -10,20 +10,15 @@ from app.log import logger
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import ChainEventType
from app.utils.otp import OtpUtils
from app.utils.singleton import Singleton
PASSWORD_INVALID_CREDENTIALS_MESSAGE = "用户名或密码或二次校验码不正确"
class UserChain(ChainBase, metaclass=Singleton):
class UserChain(ChainBase):
"""
用户链,处理多种认证协议
"""
def __init__(self):
super().__init__()
self.user_oper = UserOper()
def user_authenticate(
self,
username: Optional[str] = None,
@@ -90,7 +85,8 @@ class UserChain(ChainBase, metaclass=Singleton):
logger.debug(f"辅助认证未启用,认证类型 {grant_type} 未实现")
return False, "不支持的认证类型"
def password_authenticate(self, credentials: AuthCredentials) -> Tuple[bool, Union[User, str]]:
@staticmethod
def password_authenticate(credentials: AuthCredentials) -> Tuple[bool, Union[User, str]]:
"""
密码认证
@@ -103,7 +99,7 @@ class UserChain(ChainBase, metaclass=Singleton):
logger.info("密码认证失败,认证类型不匹配")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
user = self.user_oper.get_by_name(name=credentials.username)
user = UserOper().get_by_name(name=credentials.username)
if not user:
logger.info(f"密码认证失败,用户 {credentials.username} 不存在")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -131,8 +127,9 @@ class UserChain(ChainBase, metaclass=Singleton):
return False, "认证凭证无效"
# 检查是否因为用户被禁用
useroper = UserOper()
if credentials.username:
user = self.user_oper.get_by_name(name=credentials.username)
user = useroper.get_by_name(name=credentials.username)
if user and not user.is_active:
logger.info(f"用户 {user.name} 已被禁用,跳过后续身份校验")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -156,7 +153,7 @@ class UserChain(ChainBase, metaclass=Singleton):
success = self._process_auth_success(username=credentials.username, credentials=credentials)
if success:
logger.info(f"用户 {credentials.username} 辅助认证通过")
return True, self.user_oper.get_by_name(credentials.username)
return True, useroper.get_by_name(credentials.username)
else:
logger.warning(f"用户 {credentials.username} 辅助认证未通过")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -213,7 +210,8 @@ class UserChain(ChainBase, metaclass=Singleton):
return False
# 检查用户是否存在,如果不存在且当前为密码认证时则创建新用户
user = self.user_oper.get_by_name(name=username)
useroper = UserOper()
user = useroper.get_by_name(name=username)
if user:
# 如果用户存在,但是已经被禁用,则直接响应
if not user.is_active:
@@ -226,8 +224,8 @@ class UserChain(ChainBase, metaclass=Singleton):
return True
else:
if credentials.grant_type == "password":
self.user_oper.add(name=username, is_active=True, is_superuser=False,
hashed_password=get_password_hash(secrets.token_urlsafe(16)))
useroper.add(name=username, is_active=True, is_superuser=False,
hashed_password=get_password_hash(secrets.token_urlsafe(16)))
logger.info(f"用户 {username} 不存在,已通过 {credentials.grant_type} 认证并已创建普通用户")
return True
else:

View File

@@ -2,10 +2,9 @@ from typing import Any
from app.chain import ChainBase
from app.schemas.types import EventType
from app.utils.singleton import Singleton
class WebhookChain(ChainBase, metaclass=Singleton):
class WebhookChain(ChainBase):
"""
Webhook处理链
"""

View File

@@ -188,16 +188,14 @@ class WorkflowChain(ChainBase):
工作流链
"""
def __init__(self):
super().__init__()
self.workflowoper = WorkflowOper()
def process(self, workflow_id: int, from_begin: Optional[bool] = True) -> Tuple[bool, str]:
@staticmethod
def process(workflow_id: int, from_begin: Optional[bool] = True) -> Tuple[bool, str]:
"""
处理工作流
:param workflow_id: 工作流ID
:param from_begin: 是否从头开始默认为True
"""
workflowoper = WorkflowOper()
def save_step(action: Action, context: ActionContext):
"""
@@ -207,16 +205,16 @@ class WorkflowChain(ChainBase):
serialized_data = pickle.dumps(context)
# 使用Base64编码字节流
encoded_data = base64.b64encode(serialized_data).decode('utf-8')
self.workflowoper.step(workflow_id, action_id=action.id, context={
workflowoper.step(workflow_id, action_id=action.id, context={
"content": encoded_data
})
# 重置工作流
if from_begin:
self.workflowoper.reset(workflow_id)
workflowoper.reset(workflow_id)
# 查询工作流数据
workflow = self.workflowoper.get(workflow_id)
workflow = workflowoper.get(workflow_id)
if not workflow:
logger.warn(f"工作流 {workflow_id} 不存在")
return False, "工作流不存在"
@@ -228,7 +226,7 @@ class WorkflowChain(ChainBase):
return False, "工作流无流程"
logger.info(f"开始处理 {workflow.name},共 {len(workflow.actions)} 个动作 ...")
self.workflowoper.start(workflow_id)
workflowoper.start(workflow_id)
# 执行工作流
executor = WorkflowExecutor(workflow, step_callback=save_step)
@@ -236,15 +234,16 @@ class WorkflowChain(ChainBase):
if not executor.success:
logger.info(f"工作流 {workflow.name} 执行失败:{executor.errmsg}")
self.workflowoper.fail(workflow_id, result=executor.errmsg)
workflowoper.fail(workflow_id, result=executor.errmsg)
return False, executor.errmsg
else:
logger.info(f"工作流 {workflow.name} 执行完成")
self.workflowoper.success(workflow_id)
workflowoper.success(workflow_id)
return True, ""
def get_workflows(self) -> List[Workflow]:
@staticmethod
def get_workflows() -> List[Workflow]:
"""
获取工作流列表
"""
return self.workflowoper.list_enabled()
return WorkflowOper().list_enabled()

View File

@@ -196,7 +196,7 @@ class CacheToolsBackend(CacheBackend):
return None
return region_cache.get(key)
def delete(self, key: str, region: Optional[str] = DEFAULT_CACHE_REGION) -> None:
def delete(self, key: str, region: Optional[str] = DEFAULT_CACHE_REGION):
"""
删除缓存
@@ -205,7 +205,7 @@ class CacheToolsBackend(CacheBackend):
"""
region_cache = self.__get_region_cache(region)
if region_cache is None:
return None
return
with lock:
del region_cache[key]

View File

@@ -1,12 +1,11 @@
import copy
import json
import os
import re
import secrets
import sys
import threading
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple, Type, Union
from typing import Any, Dict, List, Optional, Tuple, Type
from dotenv import set_key
from pydantic import BaseModel, BaseSettings, validator, Field
@@ -25,7 +24,7 @@ class ConfigModel(BaseModel):
extra = "ignore" # 忽略未定义的配置项
# 项目名称
PROJECT_NAME = "MoviePilot"
PROJECT_NAME: str = "MoviePilot"
# 域名 格式https://movie-pilot.org
APP_DOMAIN: str = ""
# API路径
@@ -70,8 +69,8 @@ class ConfigModel(BaseModel):
DB_MAX_OVERFLOW: int = 500
# SQLite 的 busy_timeout 参数,默认为 60 秒
DB_TIMEOUT: int = 60
# SQLite 是否启用 WAL 模式,默认关闭
DB_WAL_ENABLE: bool = False
# SQLite 是否启用 WAL 模式,默认开启
DB_WAL_ENABLE: bool = True
# 缓存类型,支持 cachetools 和 redis默认使用 cachetools
CACHE_BACKEND_TYPE: str = "cachetools"
# 缓存连接字符串,仅外部缓存(如 Redis、Memcached需要
@@ -115,6 +114,8 @@ class ConfigModel(BaseModel):
TVDB_V4_API_PIN: str = ""
# Fanart开关
FANART_ENABLE: bool = True
# Fanart语言
FANART_LANG: str = "zh,en"
# Fanart API Key
FANART_API_KEY: str = "d2d31f9ecabea050fc7d68aa3146015f"
# 115 AppId
@@ -123,8 +124,6 @@ class ConfigModel(BaseModel):
ALIPAN_APP_ID: str = "ac1bf04dc9fd4d9aaabb65b4a668d403"
# 元数据识别缓存过期时间(小时)
META_CACHE_EXPIRE: int = 0
# 电视剧动漫的分类genre_ids
ANIME_GENREIDS = [16]
# 用户认证站点
AUTH_SITE: str = ""
# 重启自动升级
@@ -140,6 +139,7 @@ class ConfigModel(BaseModel):
"api.github.com,"
"github.com,"
"raw.githubusercontent.com,"
"codeload.github.com,"
"api.telegram.org")
# DOH 解析服务器列表
DOH_RESOLVERS: str = "1.0.0.1,1.1.1.1,9.9.9.9,149.112.112.112"
@@ -204,7 +204,7 @@ class ConfigModel(BaseModel):
# CookieCloud同步黑名单多个域名,分割
COOKIECLOUD_BLACKLIST: Optional[str] = None
# CookieCloud对应的浏览器UA
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
USER_AGENT: str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
# 电影重命名格式
MOVIE_RENAME_FORMAT: str = "{{title}}{% if year %} ({{year}}){% endif %}" \
"/{{title}}{% if year %} ({{year}}){% endif %}{% if part %}-{{part}}{% endif %}{% if videoFormat %} - {{videoFormat}}{% endif %}" \
@@ -242,12 +242,18 @@ class ConfigModel(BaseModel):
GITHUB_TOKEN: Optional[str] = None
# Github代理服务器格式https://mirror.ghproxy.com/
GITHUB_PROXY: Optional[str] = ''
# pip镜像站点格式https://pypi.tuna.tsinghua.edu.cn/simple
# pip镜像站点格式https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
PIP_PROXY: Optional[str] = ''
# 指定的仓库Github token多个仓库使用,分隔,格式:{user1}/{repo1}:ghp_****,{user2}/{repo2}:github_pat_****
REPO_GITHUB_TOKEN: Optional[str] = None
# 大内存模式
BIG_MEMORY_MODE: bool = False
# 是否启用内存监控
MEMORY_ANALYSIS: bool = False
# 内存快照间隔(分钟)
MEMORY_SNAPSHOT_INTERVAL: int = 60
# 保留的内存快照文件数量
MEMORY_SNAPSHOT_KEEP_COUNT: int = 20
# 全局图片缓存,将媒体图片缓存到本地
GLOBAL_IMAGE_CACHE: bool = False
# 是否启用编码探测的性能模式
@@ -255,32 +261,26 @@ class ConfigModel(BaseModel):
# 编码探测的最低置信度阈值
ENCODING_DETECTION_MIN_CONFIDENCE: float = 0.8
# 允许的图片缓存域名
SECURITY_IMAGE_DOMAINS: List[str] = Field(
default_factory=lambda: ["image.tmdb.org",
"static-mdb.v.geilijiasu.com",
"bing.com",
"doubanio.com",
"lain.bgm.tv",
"raw.githubusercontent.com",
"github.com",
"thetvdb.com",
"cctvpic.com",
"iqiyipic.com",
"hdslb.com",
"cmvideo.cn",
"ykimg.com",
"qpic.cn"]
)
SECURITY_IMAGE_DOMAINS: list = Field(default=[
"image.tmdb.org",
"static-mdb.v.geilijiasu.com",
"bing.com",
"doubanio.com",
"lain.bgm.tv",
"raw.githubusercontent.com",
"github.com",
"thetvdb.com",
"cctvpic.com",
"iqiyipic.com",
"hdslb.com",
"cmvideo.cn",
"ykimg.com",
"qpic.cn"
])
# 允许的图片文件后缀格式
SECURITY_IMAGE_SUFFIXES: List[str] = Field(
default_factory=lambda: [".jpg", ".jpeg", ".png", ".webp", ".gif", ".svg", ".avif"]
)
SECURITY_IMAGE_SUFFIXES: list = Field(default=[".jpg", ".jpeg", ".png", ".webp", ".gif", ".svg", ".avif"])
# 重命名时支持的S0别名
RENAME_FORMAT_S0_NAMES: List[str] = Field(
default_factory=lambda: ["Specials", "SPs"]
)
# 启用分词搜索
TOKENIZED_SEARCH: bool = False
RENAME_FORMAT_S0_NAMES: list = Field(default=["Specials", "SPs"])
# 为指定默认字幕添加.default后缀
DEFAULT_SUB: Optional[str] = "zh-cn"
# Docker Client API地址
@@ -331,6 +331,7 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
raise_exception: bool = False) -> Tuple[Any, bool]:
"""
通用类型转换函数,根据预期类型转换值。如果转换失败,返回默认值
:return: 元组 (转换后的值, 是否需要更新)
"""
if isinstance(value, (list, dict, set)):
value = copy.deepcopy(value)
@@ -371,12 +372,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
converted = float(value)
return converted, str(converted) != str(original_value)
elif expected_type is str:
# 清理 value 中所有空白字符的字段
fields_not_keep_spaces = {"AUTO_DOWNLOAD_USER", "REPO_GITHUB_TOKEN", "PLUGIN_MARKET"}
if field_name in fields_not_keep_spaces:
value = re.sub(r"\s+", "", value)
return value, str(value) != str(original_value)
# 支持 list 类型的处理
converted = str(value).strip()
return converted, converted != str(original_value)
elif expected_type is list:
if isinstance(value, list):
return value, str(value) != str(original_value)
@@ -386,7 +383,6 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return items, items != original_value
else:
return items, str(items) != str(original_value)
# 可根据需要添加更多类型处理
else:
return value, str(value) != str(original_value)
except (ValueError, TypeError) as e:
@@ -438,9 +434,12 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
logger.info(f"配置项 '{field.name}' 已自动修正并写入到 'app.env' 文件")
return True, message
def update_setting(self, key: str, value: Any) -> Tuple[bool, str]:
def update_setting(self, key: str, value: Any) -> Tuple[Optional[bool], str]:
"""
更新单个配置项
:param key: 配置项的名称
:param value: 配置项的新值
:return: (是否成功 True 成功/False 失败/None 无需更新, 错误信息)
"""
if not hasattr(self, key):
return False, f"配置项 '{key}' 不存在"
@@ -451,8 +450,11 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
if field.name == "API_TOKEN":
converted_value, needs_update = self.validate_api_token(value, original_value)
else:
converted_value, needs_update = self.generic_type_converter(value, original_value, field.type_,
field.default, key)
converted_value, needs_update = self.generic_type_converter(value,
original_value,
field.type_,
field.default,
key)
# 如果没有抛出异常,则统一使用 converted_value 进行更新
if needs_update or str(value) != str(converted_value):
success, message = self.update_env_config(field, value, converted_value)
@@ -462,30 +464,17 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
if hasattr(log_settings, key):
setattr(log_settings, key, converted_value)
return success, message
return True, ""
return None, ""
except Exception as e:
return False, str(e)
def update_settings(self, env: Dict[str, Any]) -> Dict[str, Tuple[bool, str]]:
def update_settings(self, env: Dict[str, Any]) -> Dict[str, Tuple[Optional[bool], str]]:
"""
更新多个配置项
"""
results = {}
log_updated, plugin_monitor_updated = False, False
for k, v in env.items():
results[k] = self.update_setting(k, v)
if hasattr(log_settings, k):
log_updated = True
if k in ["PLUGIN_AUTO_RELOAD", "DEV"]:
plugin_monitor_updated = True
# 本次更新存在日志配置项更新,需要重新加载日志配置
if log_updated:
logger.update_loggers()
# 本次更新存在插件监控配置项更新,需要重新加载插件监控
if plugin_monitor_updated:
# 解决顶层循环导入问题
from app.core.plugin import PluginManager
PluginManager().reload_monitor()
return results
@property
@@ -534,7 +523,7 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return self.CONFIG_PATH / "cookies"
@property
def CACHE_CONF(self):
def CONF(self):
"""
{
"torrents": "缓存种子数量",
@@ -542,7 +531,10 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"tmdb": "TMDB请求缓存数量",
"douban": "豆瓣请求缓存数量",
"fanart": "Fanart请求缓存数量",
"meta": "元数据缓存过期时间(秒)"
"meta": "元数据缓存过期时间(秒)",
"memory": "最大占用内存MB",
"scheduler": "调度器缓存数量"
"threadpool": "线程池数量"
}
"""
if self.BIG_MEMORY_MODE:
@@ -553,7 +545,9 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"douban": 512,
"bangumi": 512,
"fanart": 512,
"meta": (self.META_CACHE_EXPIRE or 24) * 3600
"meta": (self.META_CACHE_EXPIRE or 24) * 3600,
"scheduler": 100,
"threadpool": 100
}
return {
"torrents": 100,
@@ -562,7 +556,9 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"douban": 256,
"bangumi": 256,
"fanart": 128,
"meta": (self.META_CACHE_EXPIRE or 2) * 3600
"meta": (self.META_CACHE_EXPIRE or 2) * 3600,
"scheduler": 50,
"threadpool": 50
}
@property
@@ -589,7 +585,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"""
if self.GITHUB_TOKEN:
return {
"Authorization": f"Bearer {self.GITHUB_TOKEN}"
"Authorization": f"Bearer {self.GITHUB_TOKEN}",
"User-Agent": self.USER_AGENT,
}
return {}
@@ -617,7 +614,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
print(f"无效的令牌或仓库信息: {token_pair}")
continue
headers[repo_info] = {
"Authorization": f"Bearer {token}"
"Authorization": f"Bearer {token}",
"User-Agent": self.USER_AGENT,
}
except Exception as e:
print(f"处理令牌对 '{token_pair}' 时出错: {e}")
@@ -638,6 +636,10 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return UrlUtils.combine_url(host=self.APP_DOMAIN, path=url)
# 实例化配置
settings = Settings()
class GlobalVar(object):
"""
全局标识
@@ -695,8 +697,5 @@ class GlobalVar(object):
return self.is_system_stopped or workflow_id in self.EMERGENCY_STOP_WORKFLOWS
# 实例化配置
settings = Settings()
# 全局标识
global_vars = GlobalVar()

View File

@@ -10,7 +10,6 @@ from functools import lru_cache
from queue import Empty, PriorityQueue
from typing import Callable, Dict, List, Optional, Union
from app.helper.message import MessageHelper
from app.helper.thread import ThreadHelper
from app.log import logger
from app.schemas import ChainEventData
@@ -75,7 +74,6 @@ class EventManager(metaclass=Singleton):
__event = threading.Event()
def __init__(self):
self.__messagehelper = MessageHelper()
self.__executor = ThreadHelper() # 动态线程池,用于消费事件
self.__consumer_threads = [] # 用于保存启动的事件消费者线程
self.__event_queue = PriorityQueue() # 优先级队列
@@ -140,11 +138,12 @@ class EventManager(metaclass=Singleton):
"""
event = Event(etype, data, priority)
if isinstance(etype, EventType):
self.__trigger_broadcast_event(event)
return self.__trigger_broadcast_event(event)
elif isinstance(etype, ChainEventType):
return self.__trigger_chain_event(event)
else:
logger.error(f"Unknown event type: {etype}")
return None
def add_event_listener(self, event_type: Union[EventType, ChainEventType], handler: Callable,
priority: Optional[int] = DEFAULT_EVENT_PRIORITY):
@@ -293,7 +292,7 @@ class EventManager(metaclass=Singleton):
# 对于类实例(实现了 __call__ 方法)
if not inspect.isfunction(handler) and hasattr(handler, "__call__"):
handler_cls = handler.__class__ # noqa
handler_cls = handler.__class__ # noqa
return cls.__get_handler_identifier(handler_cls)
# 对于未绑定方法、静态方法、类方法,使用 __qualname__ 提取类信息
@@ -303,6 +302,7 @@ class EventManager(metaclass=Singleton):
module = inspect.getmodule(handler)
module_name = module.__name__ if module else "unknown_module"
return f"{module_name}.{class_name}"
return None
def __is_handler_enabled(self, handler: Callable) -> bool:
"""
@@ -398,16 +398,28 @@ class EventManager(metaclass=Singleton):
try:
from app.core.plugin import PluginManager
from app.core.module import ModuleManager
if class_name in PluginManager().get_plugin_ids():
# 定义一个插件调用函数
def plugin_callable():
"""
插件调用函数
"""
PluginManager().run_plugin_method(class_name, method_name, event_to_process)
if is_broadcast_event:
self.__executor.submit(plugin_callable)
else:
plugin_callable()
elif class_name in ModuleManager().get_module_ids():
module = ModuleManager().get_running_module(class_name)
if module:
method = getattr(module, method_name, None)
if method:
if is_broadcast_event:
self.__executor.submit(method, event_to_process)
else:
method(event_to_process)
else:
# 获取全局对象或模块类的实例
class_obj = self.__get_class_instance(class_name)
@@ -438,22 +450,25 @@ class EventManager(metaclass=Singleton):
# 如果类不在全局变量中,尝试动态导入模块并创建实例
try:
if class_name == "Command":
module_name = "app.command"
if class_name.endswith("Manager"):
module_name = f"app.core.{class_name[:-7].lower()}"
module = importlib.import_module(module_name)
elif class_name.endswith("Chain"):
module_name = f"app.chain.{class_name[:-5].lower()}"
module = importlib.import_module(module_name)
elif class_name.endswith("Helper"):
module_name = f"app.helper.{class_name[:-6].lower()}"
module = importlib.import_module(module_name)
else:
logger.debug(f"事件处理出错:无效的 Chain 类名: {class_name},类名必须以 'Chain' 结尾")
return None
module_name = f"app.{class_name.lower()}"
module = importlib.import_module(module_name)
if hasattr(module, class_name):
class_obj = getattr(module, class_name)()
return class_obj
else:
logger.debug(f"事件处理出错:模块 {module_name} 中没有找到类 {class_name}")
except Exception as e:
logger.error(f"事件处理出错:{str(e)} - {traceback.format_exc()}")
logger.debug(f"事件处理出错:{str(e)} - {traceback.format_exc()}")
return None
def __broadcast_consumer_loop(self):
@@ -491,9 +506,11 @@ class EventManager(metaclass=Singleton):
names = handler.__qualname__.split(".")
class_name, method_name = names[0], names[1]
self.__messagehelper.put(title=f"{event.event_type} 事件处理出错",
message=f"{class_name}.{method_name}{str(e)}",
role="system")
# 发送系统错误通知
from app.helper.message import MessageHelper
MessageHelper().put(title=f"{event.event_type} 事件处理出错",
message=f"{class_name}.{method_name}{str(e)}",
role="system")
self.send_event(
EventType.SystemError,
{

View File

@@ -81,7 +81,6 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
}
def __init__(self):
self.systemconfig = SystemConfigOper()
release_groups = []
for site_groups in self.RELEASE_GROUPS.values():
for release_group in site_groups:
@@ -98,7 +97,7 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
return ""
if not groups:
# 自定义组
custom_release_groups = self.systemconfig.get(SystemConfigKey.CustomReleaseGroups)
custom_release_groups = SystemConfigOper().get(SystemConfigKey.CustomReleaseGroups)
if isinstance(custom_release_groups, list):
custom_release_groups = list(filter(None, custom_release_groups))
if custom_release_groups:

View File

@@ -197,67 +197,3 @@ def find_metainfo(title: str) -> Tuple[str, dict]:
elif metainfo.get('begin_episode') and not metainfo.get('end_episode'):
metainfo['total_episode'] = 1
return title, metainfo
def test_find_metainfo():
"""
测试find_metainfo函数的各种ID识别格式
"""
test_cases = [
# 测试 [tmdbid=xxxx] 格式
("The Vampire Diaries (2009) [tmdbid=18165]", "18165"),
# 测试 [tmdbid-xxxx] 格式
("Inception (2010) [tmdbid-27205]", "27205"),
# 测试 [tmdb=xxxx] 格式
("Breaking Bad (2008) [tmdb=1396]", "1396"),
# 测试 [tmdb-xxxx] 格式
("Interstellar (2014) [tmdb-157336]", "157336"),
# 测试 {tmdbid=xxxx} 格式
("Stranger Things (2016) {tmdbid=66732}", "66732"),
# 测试 {tmdbid-xxxx} 格式
("The Matrix (1999) {tmdbid-603}", "603"),
# 测试 {tmdb=xxxx} 格式
("Game of Thrones (2011) {tmdb=1399}", "1399"),
# 测试 {tmdb-xxxx} 格式
("Avatar (2009) {tmdb-19995}", "19995"),
]
for title, expected_tmdbid in test_cases:
cleaned_title, metainfo = find_metainfo(title)
found_tmdbid = metainfo.get('tmdbid')
print(f"原标题: {title}")
print(f"清理后标题: {cleaned_title}")
print(f"期望的tmdbid: {expected_tmdbid}")
print(f"识别的tmdbid: {found_tmdbid}")
print(f"结果: {'通过' if found_tmdbid == expected_tmdbid else '失败'}")
print("-" * 50)
def test_meta_info_path():
"""
测试MetaInfoPath函数
"""
# 测试文件路径
path_tests = [
# 文件名中包含tmdbid
Path("/movies/The Vampire Diaries (2009) [tmdbid=18165]/The.Vampire.Diaries.S01E01.1080p.mkv"),
# 目录名中包含tmdbid
Path("/movies/Inception (2010) [tmdbid-27205]/Inception.2010.1080p.mkv"),
# 父目录名中包含tmdbid
Path("/movies/Breaking Bad (2008) [tmdb=1396]/Season 1/Breaking.Bad.S01E01.1080p.mkv"),
# 祖父目录名中包含tmdbid
Path("/tv/Game of Thrones (2011) {tmdb=1399}/Season 1/Game.of.Thrones.S01E01.1080p.mkv"),
]
for path in path_tests:
meta = MetaInfoPath(path)
print(f"测试路径: {path}")
print(f"识别结果: tmdbid={meta.tmdbid}")
print("-" * 50)
if __name__ == "__main__":
# 运行测试函数
# test_find_metainfo()
test_meta_info_path()

View File

@@ -1,5 +1,5 @@
import traceback
from typing import Generator, Optional, Tuple, Any, Union
from typing import Generator, Optional, Tuple, Any, Union, List
from app.core.config import settings
from app.core.event import eventmanager
@@ -164,3 +164,9 @@ class ModuleManager(metaclass=Singleton):
获取模块列表
"""
return self._modules
def get_module_ids(self) -> List[str]:
"""
获取模块id列表
"""
return list(self._modules.keys())

View File

@@ -16,7 +16,7 @@ from watchdog.observers import Observer
from app import schemas
from app.core.config import settings
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.db.plugindata_oper import PluginDataOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.module import ModuleHelper
@@ -87,7 +87,6 @@ class PluginManager(metaclass=Singleton):
"""
插件管理器
"""
systemconfig: SystemConfigOper = None
# 插件列表
_plugins: dict = {}
@@ -99,10 +98,6 @@ class PluginManager(metaclass=Singleton):
_observer: Observer = None
def __init__(self):
self.siteshelper = SitesHelper()
self.pluginhelper = PluginHelper()
self.systemconfig = SystemConfigOper()
self.plugindata = PluginDataOper()
# 开发者模式监测插件修改
if settings.DEV or settings.PLUGIN_AUTO_RELOAD:
self.__start_monitor()
@@ -141,7 +136,7 @@ class PluginManager(metaclass=Singleton):
filter_func=lambda _, obj: check_module(obj)
)
# 已安装插件
installed_plugins = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 排序
plugins.sort(key=lambda x: x.plugin_order if hasattr(x, "plugin_order") else 0)
for plugin in plugins:
@@ -241,6 +236,19 @@ class PluginManager(metaclass=Singleton):
"""
return self._plugins
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in ['DEV', 'PLUGIN_AUTO_RELOAD']:
return
self.reload_monitor()
def reload_monitor(self):
"""
重新加载插件文件修改监测
@@ -282,12 +290,15 @@ class PluginManager(metaclass=Singleton):
停止插件
:param plugin: 插件实例
"""
# 关闭数据库
if hasattr(plugin, "close"):
plugin.close()
# 关闭插件
if hasattr(plugin, "stop_service"):
plugin.stop_service()
try:
# 关闭数据库
if hasattr(plugin, "close"):
plugin.close()
# 关闭插件
if hasattr(plugin, "stop_service"):
plugin.stop_service()
except Exception as e:
logger.warn(f"停止插件 {plugin.get_name()} 时发生错误: {str(e)}")
def remove_plugin(self, plugin_id: str):
"""
@@ -296,6 +307,13 @@ class PluginManager(metaclass=Singleton):
"""
self.stop(plugin_id)
# 从模块列表中移除插件
from sys import modules
try:
del modules[f"app.plugins.{plugin_id.lower()}"]
except KeyError:
pass
def reload_plugin(self, plugin_id: str):
"""
将一个插件重新加载到内存
@@ -310,12 +328,12 @@ class PluginManager(metaclass=Singleton):
def sync(self) -> List[str]:
"""
安装本地不存在的在线插件
安装本地不存在或需要更新的插件
"""
def install_plugin(plugin):
start_time = time.time()
state, msg = self.pluginhelper.install(pid=plugin.id, repo_url=plugin.repo_url, force_install=True)
state, msg = PluginHelper().install(pid=plugin.id, repo_url=plugin.repo_url, force_install=True)
elapsed_time = time.time() - start_time
if state:
logger.info(
@@ -330,13 +348,14 @@ class PluginManager(metaclass=Singleton):
return []
# 获取已安装插件列表
install_plugins = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
install_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 获取在线插件列表
online_plugins = self.get_online_plugins()
# 确定需要安装的插件
plugins_to_install = [
plugin for plugin in online_plugins
if plugin.id in install_plugins and not self.is_plugin_exists(plugin.id)
if plugin.id in install_plugins
and not self.is_plugin_exists(plugin.id, plugin.plugin_version)
]
if not plugins_to_install:
@@ -366,19 +385,21 @@ class PluginManager(metaclass=Singleton):
)
return sync_plugins
def install_plugin_missing_dependencies(self) -> List[str]:
@staticmethod
def install_plugin_missing_dependencies() -> List[str]:
"""
安装插件中缺失或不兼容的依赖项
"""
pluginhelper = PluginHelper()
# 第一步:获取需要安装的依赖项列表
missing_dependencies = self.pluginhelper.find_missing_dependencies()
missing_dependencies = pluginhelper.find_missing_dependencies()
if not missing_dependencies:
return missing_dependencies
logger.debug(f"检测到缺失的依赖项: {missing_dependencies}")
logger.info(f"开始安装缺失的依赖项,共 {len(missing_dependencies)} 个...")
# 第二步:安装依赖项并返回结果
total_start_time = time.time()
success, message = self.pluginhelper.install_dependencies(missing_dependencies)
success, message = pluginhelper.install_dependencies(missing_dependencies)
total_elapsed_time = time.time() - total_start_time
if success:
logger.info(f"已完成 {len(missing_dependencies)} 个依赖项安装,总耗时:{total_elapsed_time:.2f}")
@@ -393,21 +414,22 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return {}
conf = self.systemconfig.get(self._config_key % pid)
conf = SystemConfigOper().get(self._config_key % pid)
if conf:
# 去掉空Key
return {k: v for k, v in conf.items() if k}
return {}
def save_plugin_config(self, pid: str, conf: dict) -> bool:
def save_plugin_config(self, pid: str, conf: dict, force: bool = False) -> bool:
"""
保存插件配置
:param pid: 插件ID
:param conf: 配置
:param force: 强制保存
"""
if not self._plugins.get(pid):
if not force and not self._plugins.get(pid):
return False
self.systemconfig.set(self._config_key % pid, conf)
SystemConfigOper().set(self._config_key % pid, conf)
return True
def delete_plugin_config(self, pid: str) -> bool:
@@ -417,7 +439,7 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return False
return self.systemconfig.delete(self._config_key % pid)
return SystemConfigOper().delete(self._config_key % pid)
def delete_plugin_data(self, pid: str) -> bool:
"""
@@ -426,7 +448,7 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return False
self.plugindata.del_data(pid)
PluginDataOper().del_data(pid)
return True
def get_plugin_state(self, pid: str) -> bool:
@@ -449,7 +471,9 @@ class PluginManager(metaclass=Singleton):
}]
"""
ret_commands = []
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if pid and pid != plugin_id:
continue
if hasattr(plugin, "get_command") and ObjectUtils.check_method(plugin.get_command):
@@ -509,7 +533,9 @@ class PluginManager(metaclass=Singleton):
}]
"""
ret_services = []
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if pid and pid != plugin_id:
continue
if hasattr(plugin, "get_service") and ObjectUtils.check_method(plugin.get_service):
@@ -532,7 +558,9 @@ class PluginManager(metaclass=Singleton):
}
"""
ret_modules = {}
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if pid and pid != plugin_id:
continue
if hasattr(plugin, "get_module") and ObjectUtils.check_method(plugin.get_module):
@@ -556,7 +584,9 @@ class PluginManager(metaclass=Singleton):
}]
"""
ret_actions = []
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if pid and pid != plugin_id:
continue
if hasattr(plugin, "get_actions") and ObjectUtils.check_method(plugin.get_actions):
@@ -593,7 +623,9 @@ class PluginManager(metaclass=Singleton):
获取插件联邦组件列表
"""
remotes = []
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if pid and pid != plugin_id:
continue
if hasattr(plugin, "get_render_mode"):
@@ -612,7 +644,9 @@ class PluginManager(metaclass=Singleton):
获取所有插件仪表盘元信息
"""
dashboard_meta = []
for plugin_id, plugin in self._running_plugins.items():
# 创建字典快照避免并发修改
running_plugins_snapshot = dict(self._running_plugins)
for plugin_id, plugin in running_plugins_snapshot.items():
if not hasattr(plugin, "get_dashboard") or not ObjectUtils.check_method(plugin.get_dashboard):
continue
try:
@@ -721,7 +755,7 @@ class PluginManager(metaclass=Singleton):
"""
return list(self._running_plugins.keys())
def get_online_plugins(self) -> List[schemas.Plugin]:
def get_online_plugins(self, force: bool = False) -> List[schemas.Plugin]:
"""
获取所有在线插件信息
"""
@@ -742,12 +776,13 @@ class PluginManager(metaclass=Singleton):
if not m:
continue
# 提交任务获取 v1 版本插件,存储 future 到 version 的映射
base_future = executor.submit(self.get_plugins_from_market, m, None)
base_future = executor.submit(self.get_plugins_from_market, m, None, force)
futures_to_version[base_future] = "base_version"
# 提交任务获取高版本插件(如 v2、v3存储 future 到 version 的映射
if settings.VERSION_FLAG:
higher_version_future = executor.submit(self.get_plugins_from_market, m, settings.VERSION_FLAG)
higher_version_future = executor.submit(self.get_plugins_from_market, m,
settings.VERSION_FLAG, force)
futures_to_version[higher_version_future] = "higher_version"
# 按照完成顺序处理结果
@@ -788,7 +823,7 @@ class PluginManager(metaclass=Singleton):
# 返回值
plugins = []
# 已安装插件
installed_apps = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_apps = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
for pid, plugin_class in self._plugins.items():
# 运行状插件
plugin_obj = self._running_plugins.get(pid)
@@ -855,10 +890,11 @@ class PluginManager(metaclass=Singleton):
return plugins
@staticmethod
def is_plugin_exists(pid: str) -> bool:
def is_plugin_exists(pid: str, version: str = None) -> bool:
"""
判断插件是否在本地包中存在
判断插件是否存在,并满足版本要求(有传入version时)
:param pid: 插件ID
:param version: 插件版本
"""
if not pid:
return False
@@ -869,25 +905,38 @@ class PluginManager(metaclass=Singleton):
spec = importlib.util.find_spec(package_name)
package_exists = spec is not None and spec.origin is not None
logger.debug(f"{pid} exists: {package_exists}")
return package_exists
if not package_exists:
return False
local_version = PluginManager().get_plugin_attr(pid=pid, attr="plugin_version")
if not local_version:
return False
if version and not StringUtils.compare_version(local_version, ">=", version):
logger.warn(f"Plugin {pid} version: {local_version} (older than version: {version})")
return False
return True
except Exception as e:
logger.debug(f"获取插件是否在本地包中存在失败,{e}")
return False
def get_plugins_from_market(self, market: str,
package_version: Optional[str] = None) -> Optional[List[schemas.Plugin]]:
package_version: Optional[str] = None,
force: bool = False) -> Optional[List[schemas.Plugin]]:
"""
从指定的市场获取插件信息
:param market: 市场的 URL 或标识
:param package_version: 首选插件版本 (如 "v2", "v3"),如果不指定则获取 v1 版本
:param force: 是否强制刷新(忽略缓存)
:return: 返回插件的列表,若获取失败返回 []
"""
if not market:
return []
# 已安装插件
installed_apps = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_apps = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 获取在线插件
online_plugins = self.pluginhelper.get_plugins(market, package_version)
online_plugins = PluginHelper().get_plugins(market, package_version, force)
if online_plugins is None:
logger.warning(
f"获取{package_version if package_version else ''}插件库失败:{market},请检查 GitHub 网络连接")
@@ -974,7 +1023,8 @@ class PluginManager(metaclass=Singleton):
return ret_plugins
def __set_and_check_auth_level(self, plugin: Union[schemas.Plugin, Type[Any]],
@staticmethod
def __set_and_check_auth_level(plugin: Union[schemas.Plugin, Type[Any]],
source: Optional[Union[dict, Type[Any]]] = None) -> bool:
"""
设置并检查插件的认证级别
@@ -998,7 +1048,8 @@ class PluginManager(metaclass=Singleton):
# 3 - 站点&密钥认证可见
# 99 - 站点&特殊密钥认证可见
# 如果当前站点认证级别大于 1 且插件级别为 99并存在插件公钥说明为特殊密钥认证通过密钥匹配进行认证
if self.siteshelper.auth_level > 1 and plugin.auth_level == 99 and hasattr(plugin, "plugin_public_key"):
siteshelper = SitesHelper()
if siteshelper.auth_level > 1 and plugin.auth_level == 99 and hasattr(plugin, "plugin_public_key"):
plugin_id = plugin.id if isinstance(plugin, schemas.Plugin) else plugin.__name__
public_key = plugin.plugin_public_key
if public_key:
@@ -1006,7 +1057,7 @@ class PluginManager(metaclass=Singleton):
verify = RSAUtils.verify_rsa_keys(public_key=public_key, private_key=private_key)
return verify
# 如果当前站点认证级别小于插件级别,则返回 False
if self.siteshelper.auth_level < plugin.auth_level:
if siteshelper.auth_level < plugin.auth_level:
return False
return True
@@ -1071,7 +1122,7 @@ class PluginManager(metaclass=Singleton):
success, msg = self._modify_plugin_files(
plugin_dir=clone_plugin_dir,
original_id=plugin_id,
suffix=suffix,
suffix=suffix.lower(),
name=name,
description=description,
version=version,
@@ -1085,10 +1136,11 @@ class PluginManager(metaclass=Singleton):
return False, msg
# 将分身插件添加到已安装列表
installed_plugins = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
systemconfig = SystemConfigOper()
installed_plugins = systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
if clone_id not in installed_plugins:
installed_plugins.append(clone_id)
self.systemconfig.set(SystemConfigKey.UserInstalledPlugins, installed_plugins)
systemconfig.set(SystemConfigKey.UserInstalledPlugins, installed_plugins)
# 为分身插件创建初始配置(从原插件复制配置)
logger.info(f"正在为分身插件 {clone_id} 创建初始配置...")
@@ -1100,7 +1152,7 @@ class PluginManager(metaclass=Singleton):
# 默认禁用分身插件,让用户手动配置
clone_config['enable'] = False
clone_config['enabled'] = False
self.save_plugin_config(clone_id, clone_config)
self.save_plugin_config(clone_id, clone_config, force=True)
logger.info(f"已为分身插件 {clone_id} 设置初始配置")
else:
logger.info(f"原插件 {plugin_id} 没有配置,分身插件 {clone_id} 将使用默认配置")

View File

@@ -236,7 +236,6 @@ class DbOper:
"""
数据库操作基类
"""
_db: Session = None
def __init__(self, db: Session = None):
self._db = db

View File

@@ -9,3 +9,4 @@ from .transferhistory import TransferHistory
from .user import User
from .userconfig import UserConfig
from .workflow import Workflow
from .userrequest import UserRequest

View File

@@ -1,4 +1,5 @@
from typing import Any, Union
import copy
from typing import Any, Optional, Union
from app.db import DbOper
from app.db.models.systemconfig import SystemConfig
@@ -7,34 +8,44 @@ from app.utils.singleton import Singleton
class SystemConfigOper(DbOper, metaclass=Singleton):
# 配置对象
__SYSTEMCONF: dict = {}
"""
系统配置管理
"""
def __init__(self):
"""
加载配置到内存
"""
super().__init__()
self.__SYSTEMCONF = {}
for item in SystemConfig.list(self._db):
self.__SYSTEMCONF[item.key] = item.value
def set(self, key: Union[str, SystemConfigKey], value: Any):
def set(self, key: Union[str, SystemConfigKey], value: Any) -> Optional[bool]:
"""
设置系统设置
:param key: 配置键
:param value: 配置值
:return: 是否设置成功True 成功/False 失败/None 无需更新)
"""
if isinstance(key, SystemConfigKey):
key = key.value
# 更新内存
self.__SYSTEMCONF[key] = value
# 旧值
old_value = self.__SYSTEMCONF.get(key)
# 更新内存(deepcopy避免内存共享)
self.__SYSTEMCONF[key] = copy.deepcopy(value)
conf = SystemConfig.get_by_key(self._db, key)
if conf:
if value:
conf.update(self._db, {"value": value})
else:
conf.delete(self._db, conf.id)
if old_value != value:
if value:
conf.update(self._db, {"value": value})
else:
conf.delete(self._db, conf.id)
return True
return None
else:
conf = SystemConfig(key=key, value=value)
conf.create(self._db)
return True
def get(self, key: Union[str, SystemConfigKey] = None) -> Any:
"""
@@ -43,16 +54,18 @@ class SystemConfigOper(DbOper, metaclass=Singleton):
if isinstance(key, SystemConfigKey):
key = key.value
if not key:
return self.__SYSTEMCONF
return self.__SYSTEMCONF.get(key)
return self.all()
# 避免将__SYSTEMCONF内的值引用出去会导致set时误判没有变动
return copy.deepcopy(self.__SYSTEMCONF.get(key))
def all(self):
"""
获取所有系统设置
"""
return self.__SYSTEMCONF or {}
# 避免将__SYSTEMCONF内的值引用出去会导致set时误判没有变动
return copy.deepcopy(self.__SYSTEMCONF)
def delete(self, key: Union[str, SystemConfigKey]):
def delete(self, key: Union[str, SystemConfigKey]) -> bool:
"""
删除系统设置
"""

View File

@@ -7,14 +7,15 @@ from app.utils.singleton import Singleton
class UserConfigOper(DbOper, metaclass=Singleton):
# 配置缓存
__USERCONF: Dict[str, Dict[str, Any]] = {}
"""
用户配置管理
"""
def __init__(self):
"""
加载配置到内存
"""
super().__init__()
self.__USERCONF = {}
for item in UserConfig.list(self._db):
self.__set_config_cache(username=item.username, key=item.key, value=item.value)

View File

@@ -1,2 +1 @@
from .doh import doh_query_json
from .cloudflare import under_challenge

View File

@@ -1,7 +1,8 @@
from typing import Callable, Any, Optional
from playwright.sync_api import sync_playwright, Page
from cf_clearance import sync_cf_retry, sync_stealth
from playwright.sync_api import sync_playwright, Page
from app.log import logger
@@ -35,26 +36,41 @@ class PlaywrightHelper:
:param headless: 是否无头模式
:param timeout: 超时时间
"""
result = None
try:
with sync_playwright() as playwright:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
browser = None
context = None
page = None
try:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
# 回调函数
return callback(page)
result = callback(page)
except Exception as e:
logger.error(f"网页操作失败: {str(e)}")
finally:
browser.close()
# 确保资源被正确清理
if page:
page.close()
if context:
context.close()
if browser:
browser.close()
except Exception as e:
logger.error(f"网页操作失败: {str(e)}")
return None
logger.error(f"Playwright初始化失败: {str(e)}")
return result
def get_page_source(self, url: str,
cookies: Optional[str] = None,
@@ -71,26 +87,40 @@ class PlaywrightHelper:
:param headless: 是否无头模式
:param timeout: 超时时间
"""
source = ""
source = None
try:
with sync_playwright() as playwright:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
browser = None
context = None
page = None
try:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
source = page.content()
except Exception as e:
logger.error(f"获取网页源码失败: {str(e)}")
source = None
finally:
browser.close()
# 确保资源被正确清理
if page:
page.close()
if context:
context.close()
if browser:
browser.close()
except Exception as e:
logger.error(f"获取网页源码失败: {str(e)}")
logger.error(f"Playwright初始化失败: {str(e)}")
return source

View File

@@ -14,7 +14,6 @@ class CookieCloudHelper:
def __init__(self):
self.__sync_setting()
self._req = RequestUtils(content_type="application/json")
def __sync_setting(self):
"""
@@ -46,7 +45,7 @@ class CookieCloudHelper:
return {}, "未从本地CookieCloud服务加载到cookie数据请检查服务器设置、用户KEY及加密密码是否正确"
else:
req_url = UrlUtils.combine_url(host=self._server, path=f"get/{self._key}")
ret = self._req.get_res(url=req_url)
ret = RequestUtils(content_type="application/json").get_res(url=req_url)
if ret and ret.status_code == 200:
try:
result = ret.json()

View File

@@ -13,14 +13,12 @@ class DirectoryHelper:
下载目录/媒体库目录帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_dirs(self) -> List[schemas.TransferDirectoryConf]:
@staticmethod
def get_dirs() -> List[schemas.TransferDirectoryConf]:
"""
获取所有下载目录
"""
dir_confs: List[dict] = self.systemconfig.get(SystemConfigKey.Directories)
dir_confs: List[dict] = SystemConfigOper().get(SystemConfigKey.Directories)
if not dir_confs:
return []
return [schemas.TransferDirectoryConf(**d) for d in dir_confs]

View File

@@ -10,10 +10,15 @@ import socket
import struct
import urllib
import urllib.request
from threading import Lock
from typing import Dict, Optional
from app.core.config import settings
from app.core.event import Event, eventmanager
from app.log import logger
from app.schemas import ConfigChangeEventData
from app.schemas.types import EventType
from app.utils.singleton import Singleton
# 定义一个全局线程池执行器
_executor = concurrent.futures.ThreadPoolExecutor()
@@ -21,41 +26,63 @@ _executor = concurrent.futures.ThreadPoolExecutor()
# 定义默认的DoH配置
_doh_timeout = 5
_doh_cache: Dict[str, str] = {}
_doh_lock = Lock()
# 保存原始的 socket.getaddrinfo 方法
_orig_getaddrinfo = socket.getaddrinfo
def _patched_getaddrinfo(host, *args, **kwargs):
def enable_doh(enable: bool):
"""
socket.getaddrinfo的补丁版本。
socket.getaddrinfo 进行补丁
"""
if host not in settings.DOH_DOMAINS.split(","):
def _patched_getaddrinfo(host, *args, **kwargs):
"""
socket.getaddrinfo的补丁版本。
"""
if host not in settings.DOH_DOMAINS.split(","):
return _orig_getaddrinfo(host, *args, **kwargs)
# 检查主机是否已解析
with _doh_lock:
ip = _doh_cache.get("host", None)
if ip is not None:
logger.info("已解析 [%s] 为 [%s] (缓存)", host, ip)
return _orig_getaddrinfo(ip, *args, **kwargs)
# 使用DoH解析主机
futures = []
for resolver in settings.DOH_RESOLVERS.split(","):
futures.append(_executor.submit(_doh_query, resolver, host))
for future in concurrent.futures.as_completed(futures):
ip = future.result()
if ip is not None:
logger.info("已解析 [%s] 为 [%s]", host, ip)
with _doh_lock:
_doh_cache[host] = ip
host = ip
break
return _orig_getaddrinfo(host, *args, **kwargs)
# 检查主机是否已解析
if host in _doh_cache:
ip = _doh_cache[host]
logger.info("已解析 [%s] 为 [%s] (缓存)", host, ip)
return _orig_getaddrinfo(ip, *args, **kwargs)
if enable:
# 替换 socket.getaddrinfo 方法
socket.getaddrinfo = _patched_getaddrinfo
else:
socket.getaddrinfo = _orig_getaddrinfo
# 使用DoH解析主机
futures = []
for resolver in settings.DOH_RESOLVERS.split(","):
futures.append(_executor.submit(_doh_query, resolver, host))
class DohHelper(metaclass=Singleton):
def __init__(self):
enable_doh(settings.DOH_ENABLE)
for future in concurrent.futures.as_completed(futures):
ip = future.result()
if ip is not None:
logger.info("已解析 [%s] 为 [%s]", host, ip)
_doh_cache[host] = ip
host = ip
break
return _orig_getaddrinfo(host, *args, **kwargs)
# 对 socket.getaddrinfo 进行补丁
if settings.DOH_ENABLE:
_orig_getaddrinfo = socket.getaddrinfo
socket.getaddrinfo = _patched_getaddrinfo
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in ["DOH_ENABLE", "DOH_DOMAINS", "DOH_RESOLVERS"]:
return
with _doh_lock:
# DOH配置有变动的情况下清空缓存
_doh_cache.clear()
enable_doh(settings.DOH_ENABLE)
def _doh_query(resolver: str, host: str) -> Optional[str]:

457
app/helper/memory.py Normal file
View File

@@ -0,0 +1,457 @@
import gc
import sys
import threading
import time
from datetime import datetime
from typing import Optional
import psutil
from pympler import muppy, summary, asizeof
from app.core.config import settings
from app.core.event import eventmanager, Event
from app.log import logger
from app.schemas import ConfigChangeEventData
from app.schemas.types import EventType
from app.utils.singleton import Singleton
class MemoryHelper(metaclass=Singleton):
"""
内存管理工具类,用于监控和优化内存使用
"""
def __init__(self):
# 检查间隔(秒) - 从配置获取默认5分钟
self._check_interval = settings.MEMORY_SNAPSHOT_INTERVAL * 60
self._monitoring = False
self._monitor_thread: Optional[threading.Thread] = None
# 内存快照保存目录
self._memory_snapshot_dir = settings.LOG_PATH / "memory_snapshots"
# 保留的快照文件数量
self._keep_count = settings.MEMORY_SNAPSHOT_KEEP_COUNT
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件,更新内存监控设置
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in ['MEMORY_ANALYSIS', 'MEMORY_SNAPSHOT_INTERVAL', 'MEMORY_SNAPSHOT_KEEP_COUNT']:
return
# 更新配置
if event_data.key == 'MEMORY_SNAPSHOT_INTERVAL':
self._check_interval = settings.MEMORY_SNAPSHOT_INTERVAL * 60
elif event_data.key == 'MEMORY_SNAPSHOT_KEEP_COUNT':
self._keep_count = settings.MEMORY_SNAPSHOT_KEEP_COUNT
self.stop_monitoring()
self.start_monitoring()
def start_monitoring(self):
"""
开始内存监控
"""
if not settings.MEMORY_ANALYSIS:
return
if self._monitoring:
return
# 创建内存快照目录
self._memory_snapshot_dir.mkdir(parents=True, exist_ok=True)
# 初始化内存分析器
self._monitoring = True
self._monitor_thread = threading.Thread(target=self._monitor_loop, daemon=True)
self._monitor_thread.start()
logger.info("内存监控已启动")
def stop_monitoring(self):
"""
停止内存监控
"""
self._monitoring = False
if self._monitor_thread:
self._monitor_thread.join(timeout=5)
logger.info("内存监控已停止")
def _monitor_loop(self):
"""
内存监控循环
"""
logger.info("内存监控循环开始")
while self._monitoring:
try:
# 生成内存快照
self._create_memory_snapshot()
time.sleep(self._check_interval)
except Exception as e:
logger.error(f"内存监控出错: {e}")
# 出错后等待1分钟再继续
time.sleep(60)
logger.info("内存监控循环结束")
def _create_memory_snapshot(self):
"""
创建内存快照并保存到文件
"""
try:
# 获取当前时间戳
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
snapshot_file = self._memory_snapshot_dir / f"memory_snapshot_{timestamp}.txt"
# 获取系统内存使用情况
memory_usage = psutil.Process().memory_info().rss
logger.info(f"开始创建内存快照: {snapshot_file}")
# 第一步:写入基本信息和对象类型统计
self._write_basic_info(snapshot_file, memory_usage)
# 第二步:分析并写入类实例内存使用情况
self._append_class_analysis(snapshot_file)
# 第三步:分析并写入大内存变量详情
self._append_variable_analysis(snapshot_file)
logger.info(f"内存快照已保存: {snapshot_file}, 当前内存使用: {memory_usage / 1024 / 1024:.2f} MB")
# 清理过期的快照文件保留最近30个
self._cleanup_old_snapshots()
except Exception as e:
logger.error(f"创建内存快照失败: {e}")
@staticmethod
def _write_basic_info(snapshot_file, memory_usage):
"""
写入基本信息和对象类型统计
"""
# 获取当前进程的内存使用情况
all_objects = muppy.get_objects()
sum1 = summary.summarize(all_objects)
with open(snapshot_file, 'w', encoding='utf-8') as f:
f.write(f"内存快照时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
f.write(f"当前进程内存使用: {memory_usage / 1024 / 1024:.2f} MB\n")
f.write("=" * 80 + "\n")
f.write("对象类型统计:\n")
f.write("-" * 80 + "\n")
# 写入对象统计信息
for line in summary.format_(sum1):
f.write(line + "\n")
# 立即刷新到磁盘
f.flush()
logger.debug("基本信息已写入快照文件")
def _append_class_analysis(self, snapshot_file):
"""
分析并追加类实例内存使用情况
"""
with open(snapshot_file, 'a', encoding='utf-8') as f:
f.write("\n" + "=" * 80 + "\n")
f.write("类实例内存使用情况 (按内存大小排序):\n")
f.write("-" * 80 + "\n")
f.write("正在分析中...\n")
# 立即刷新,让用户知道这部分开始了
f.flush()
try:
logger.debug("开始分析类实例内存使用情况")
class_objects = self._get_class_memory_usage()
# 重新打开文件,移除"正在分析中..."并写入实际结果
with open(snapshot_file, 'r', encoding='utf-8') as f:
content = f.read()
# 替换"正在分析中..."
content = content.replace("正在分析中...\n", "")
with open(snapshot_file, 'w', encoding='utf-8') as f:
f.write(content)
if class_objects:
# 只显示前100个类
for i, class_info in enumerate(class_objects[:100], 1):
f.write(f"{i:3d}. {class_info['name']:<50} "
f"{class_info['size_mb']:>8.2f} MB ({class_info['count']} 个实例)\n")
else:
f.write("未找到有效的类实例信息\n")
f.flush()
except Exception as e:
logger.error(f"获取类实例信息失败: {e}")
# 即使出错也要更新文件
with open(snapshot_file, 'r', encoding='utf-8') as f:
content = f.read()
content = content.replace("正在分析中...\n", f"获取类实例信息失败: {e}\n")
with open(snapshot_file, 'w', encoding='utf-8') as f:
f.write(content)
f.flush()
logger.debug("类实例分析已完成并写入")
def _append_variable_analysis(self, snapshot_file):
"""
分析并追加大内存变量详情
"""
with open(snapshot_file, 'a', encoding='utf-8') as f:
f.write("\n" + "=" * 80 + "\n")
f.write("大内存变量详情 (前100个):\n")
f.write("-" * 80 + "\n")
f.write("正在分析中...\n")
# 立即刷新,让用户知道这部分开始了
f.flush()
try:
logger.debug("开始分析大内存变量")
large_variables = self._get_large_variables(100)
# 重新打开文件,移除"正在分析中..."并写入实际结果
with open(snapshot_file, 'r', encoding='utf-8') as f:
content = f.read()
# 替换最后的"正在分析中..."
content = content.replace("正在分析中...\n", "")
with open(snapshot_file, 'w', encoding='utf-8') as f:
f.write(content)
if large_variables:
for i, var_info in enumerate(large_variables, 1):
f.write(
f"{i:3d}. {var_info['name']:<30} {var_info['type']:<15} {var_info['size_mb']:>8.2f} MB\n")
else:
f.write("未找到大内存变量\n")
f.flush()
except Exception as e:
logger.error(f"获取大内存变量信息失败: {e}")
# 即使出错也要更新文件
with open(snapshot_file, 'r', encoding='utf-8') as f:
content = f.read()
content = content.replace("正在分析中...\n", f"获取变量信息失败: {e}\n")
with open(snapshot_file, 'w', encoding='utf-8') as f:
f.write(content)
f.flush()
logger.debug("大内存变量分析已完成并写入")
def _cleanup_old_snapshots(self):
"""
清理过期的内存快照文件,只保留最近的指定数量文件
"""
try:
snapshot_files = list(self._memory_snapshot_dir.glob("memory_snapshot_*.txt"))
if len(snapshot_files) > self._keep_count:
# 按修改时间排序,删除最旧的文件
snapshot_files.sort(key=lambda x: x.stat().st_mtime)
for old_file in snapshot_files[:-self._keep_count]:
old_file.unlink()
logger.debug(f"已删除过期内存快照: {old_file}")
except Exception as e:
logger.error(f"清理过期快照失败: {e}")
@staticmethod
def _get_class_memory_usage():
"""
获取所有类实例的内存使用情况,按内存大小排序
"""
class_info = {}
processed_count = 0
error_count = 0
# 获取所有对象
all_objects = muppy.get_objects()
logger.debug(f"开始分析 {len(all_objects)} 个对象的类实例内存使用情况")
for obj in all_objects:
try:
# 跳过类对象本身,统计类的实例
if isinstance(obj, type):
continue
# 获取对象的类名 - 这里可能会出错
obj_class = type(obj)
# 安全地获取类名
try:
if hasattr(obj_class, '__module__') and hasattr(obj_class, '__name__'):
class_name = f"{obj_class.__module__}.{obj_class.__name__}"
else:
class_name = str(obj_class)
except Exception as e:
# 如果获取类名失败,使用简单的类型描述
class_name = f"<unknown_class_{id(obj_class)}>"
logger.debug(f"获取类名失败: {e}")
# 计算对象本身的内存使用(不包括引用对象,避免重复计算)
size_bytes = sys.getsizeof(obj)
if size_bytes < 100: # 跳过太小的对象
continue
size_mb = size_bytes / 1024 / 1024
processed_count += 1
if class_name in class_info:
class_info[class_name]['size_mb'] += size_mb
class_info[class_name]['count'] += 1
else:
class_info[class_name] = {
'name': class_name,
'size_mb': size_mb,
'count': 1
}
except Exception as e:
# 捕获所有可能的异常包括SQLAlchemy、ORM等框架的异常
error_count += 1
if error_count <= 5: # 只记录前5个错误避免日志过多
logger.debug(f"分析对象时出错: {e}")
continue
logger.debug(f"类实例分析完成: 处理了 {processed_count} 个对象, 遇到 {error_count} 个错误")
# 按内存大小排序
sorted_classes = sorted(class_info.values(), key=lambda x: x['size_mb'], reverse=True)
return sorted_classes
def _get_large_variables(self, limit=100):
"""
获取大内存变量信息,按内存大小排序
使用已计算对象集合避免重复计算
"""
large_vars = []
processed_count = 0
calculated_objects = set() # 避免重复计算
# 获取所有对象
all_objects = muppy.get_objects()
logger.debug(f"开始分析 {len(all_objects)} 个对象的内存使用情况")
for obj in all_objects:
# 跳过类对象
if isinstance(obj, type):
continue
# 跳过已经计算过的对象
obj_id = id(obj)
if obj_id in calculated_objects:
continue
try:
# 首先使用 sys.getsizeof 快速筛选
shallow_size = sys.getsizeof(obj)
if shallow_size < 1024: # 只处理大于1KB的对象
continue
# 对于较大的对象,使用 asizeof 进行深度计算
size_bytes = asizeof.asizeof(obj)
# 只处理大于10KB的对象提高分析效率
if size_bytes < 10240:
continue
size_mb = size_bytes / 1024 / 1024
processed_count += 1
calculated_objects.add(obj_id)
# 获取对象信息
var_info = self._get_variable_info(obj, size_mb)
if var_info:
large_vars.append(var_info)
# 如果已经找到足够多的大对象,可以提前结束
if len(large_vars) >= limit * 2: # 多收集一些,后面排序筛选
break
except Exception as e:
# 更广泛的异常捕获
logger.debug(f"分析对象失败: {e}")
continue
logger.debug(f"处理了 {processed_count} 个大对象,找到 {len(large_vars)} 个有效变量")
# 按内存大小排序并返回前N个
large_vars.sort(key=lambda x: x['size_mb'], reverse=True)
return large_vars[:limit]
def _get_variable_info(self, obj, size_mb):
"""
获取变量的描述信息
"""
try:
obj_type = type(obj).__name__
# 尝试获取变量名
var_name = self._get_variable_name(obj)
# 生成描述性信息
if isinstance(obj, dict):
key_count = len(obj)
if key_count > 0:
sample_keys = list(obj.keys())[:3]
var_name += f" ({key_count}项, 键: {sample_keys})"
elif isinstance(obj, (list, tuple, set)):
var_name += f" ({len(obj)}个元素)"
elif isinstance(obj, str):
if len(obj) > 50:
var_name += f" (长度: {len(obj)}, 内容: '{obj[:50]}...')"
else:
var_name += f" ('{obj}')"
elif hasattr(obj, '__class__') and hasattr(obj.__class__, '__name__'):
if hasattr(obj, '__dict__'):
attr_count = len(obj.__dict__)
var_name += f" ({attr_count}个属性)"
return {
'name': var_name,
'type': obj_type,
'size_mb': size_mb
}
except Exception as e:
logger.debug(f"获取变量信息失败: {e}")
return None
@staticmethod
def _get_variable_name(obj):
"""
尝试获取变量名
"""
try:
# 尝试通过gc获取引用该对象的变量名
referrers = gc.get_referrers(obj)
for referrer in referrers:
if isinstance(referrer, dict):
# 检查是否在某个模块的全局变量中
for name, value in referrer.items():
if value is obj and isinstance(name, str):
return name
elif hasattr(referrer, '__dict__'):
# 检查是否在某个实例的属性中
for name, value in referrer.__dict__.items():
if value is obj and isinstance(name, str):
return f"{type(referrer).__name__}.{name}"
# 如果找不到变量名返回对象类型和id
return f"{type(obj).__name__}_{id(obj)}"
except Exception as e:
logger.debug(f"获取变量名失败: {e}")
return f"{type(obj).__name__}_{id(obj)}"

View File

@@ -61,7 +61,8 @@ class TemplateContextBuilder:
self._add_transfer_info(transferinfo)
self._add_torrent_info(torrentinfo)
self._add_file_info(file_extension)
if kwargs: self._context.update(kwargs)
if kwargs:
self._context.update(kwargs)
if include_raw_objects:
self._add_raw_objects(meta, mediainfo, torrentinfo, transferinfo, episodes_info)
@@ -73,7 +74,8 @@ class TemplateContextBuilder:
"""
增加媒体信息
"""
if not mediainfo: return
if not mediainfo:
return
season_fmt = f"S{mediainfo.season:02d}" if mediainfo.season is not None else None
base_info = {
# 标题
@@ -245,7 +247,8 @@ class TemplateContextBuilder:
"""
添加文件信息
"""
if not file_extension: return
if not file_extension:
return
file_info = {
# 文件后缀
"fileExt": file_extension,

View File

@@ -41,12 +41,35 @@ class PluginHelper(metaclass=Singleton):
if self.install_report():
self.systemconfig.set(SystemConfigKey.PluginInstallReport, "1")
@cached(maxsize=1000, ttl=1800)
def get_plugins(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
def get_plugins(self, repo_url: str, package_version: Optional[str] = None,
force: bool = False) -> Optional[Dict[str, dict]]:
"""
获取Github所有最新插件列表
:param repo_url: Github仓库地址
:param package_version: 首选插件版本 (如 "v2", "v3"),如果不指定则获取 v1 版本
:param force: 是否强制刷新,忽略缓存
"""
# 如果强制刷新,直接调用不带缓存的版本
if force:
return self._get_plugins_uncached(repo_url, package_version)
# 正常情况下调用带缓存的版本
return self._get_plugins_cached(repo_url, package_version)
@cached(maxsize=64, ttl=1800)
def _get_plugins_cached(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
"""
获取Github所有最新插件列表使用缓存
:param repo_url: Github仓库地址
:param package_version: 首选插件版本 (如 "v2", "v3"),如果不指定则获取 v1 版本
"""
return self._get_plugins_uncached(repo_url, package_version)
def _get_plugins_uncached(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
"""
获取Github所有最新插件列表不使用缓存
:param repo_url: Github仓库地址
:param package_version: 首选插件版本 (如 "v2", "v3"),如果不指定则获取 v1 版本
"""
if not repo_url:
return None

View File

@@ -3,15 +3,14 @@ from pathlib import Path
from app.core.config import settings
from app.helper.sites import SitesHelper
from app.helper.system import SystemHelper
from app.log import logger
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
from app.utils.system import SystemUtils
from helper.system import SystemHelper
class ResourceHelper(metaclass=Singleton):
class ResourceHelper:
"""
检测和更新资源包
"""
@@ -21,7 +20,6 @@ class ResourceHelper(metaclass=Singleton):
_base_dir: Path = settings.ROOT_PATH
def __init__(self):
self.siteshelper = SitesHelper()
self.check()
@property
@@ -59,10 +57,10 @@ class ResourceHelper(metaclass=Singleton):
# 判断版本号
if rtype == "auth":
# 站点认证资源
local_version = self.siteshelper.auth_version
local_version = SitesHelper().auth_version
elif rtype == "sites":
# 站点索引资源
local_version = self.siteshelper.indexer_version
local_version = SitesHelper().indexer_version
else:
continue
if StringUtils.compare_version(version, ">", local_version):

View File

@@ -1,6 +1,5 @@
import re
import traceback
import xml.dom.minidom
from typing import List, Tuple, Union, Optional
from urllib.parse import urljoin
@@ -10,7 +9,6 @@ from lxml import etree
from app.core.config import settings
from app.helper.browser import PlaywrightHelper
from app.log import logger
from app.utils.dom import DomUtils
from app.utils.http import RequestUtils
from app.utils.string import StringUtils
@@ -19,6 +17,11 @@ class RssHelper:
"""
RSS帮助类解析RSS报文、获取RSS地址等
"""
# RSS解析限制配置
MAX_RSS_SIZE = 50 * 1024 * 1024 # 50MB最大RSS文件大小
MAX_RSS_ITEMS = 1000 # 最大解析条目数
# 各站点RSS链接获取配置
rss_link_conf = {
"default": {
@@ -224,8 +227,8 @@ class RssHelper:
},
}
@staticmethod
def parse(url, proxy: bool = False, timeout: Optional[int] = 15, headers: dict = None) -> Union[List[dict], None, bool]:
def parse(self, url, proxy: bool = False,
timeout: Optional[int] = 15, headers: dict = None) -> Union[List[dict], None, bool]:
"""
解析RSS订阅URL获取RSS中的种子信息
:param url: RSS地址
@@ -238,6 +241,7 @@ class RssHelper:
ret_array: list = []
if not url:
return False
try:
ret = RequestUtils(proxies=settings.PROXY if proxy else None,
timeout=timeout, headers=headers).get_res(url)
@@ -246,11 +250,17 @@ class RssHelper:
except Exception as err:
logger.error(f"获取RSS失败{str(err)} - {traceback.format_exc()}")
return False
if ret:
ret_xml = ""
ret_xml = None
root = None
try:
# 使用chardet检测字符编码
# 检查响应大小避免处理过大的RSS文件
raw_data = ret.content
if raw_data and len(raw_data) > self.MAX_RSS_SIZE:
logger.warning(f"RSS文件过大: {len(raw_data) / 1024 / 1024:.1f}MB跳过解析")
return False
if raw_data:
try:
result = chardet.detect(raw_data)
@@ -269,57 +279,114 @@ class RssHelper:
ret.encoding = ret.apparent_encoding
if not ret_xml:
ret_xml = ret.text
# 解析XML
dom_tree = xml.dom.minidom.parseString(ret_xml)
rootNode = dom_tree.documentElement
items = rootNode.getElementsByTagName("item")
for item in items:
# 使用lxml.etree解析XML
parser = None
try:
# 创建解析器,禁用网络访问以提高安全性和性能
parser = etree.XMLParser(
recover=True, # 容错模式
strip_cdata=False, # 保留CDATA
resolve_entities=False, # 禁用外部实体解析
no_network=True, # 禁用网络访问
huge_tree=False # 禁用大文档解析,避免内存问题
)
root = etree.fromstring(ret_xml.encode('utf-8'), parser=parser)
except etree.XMLSyntaxError:
# 如果XML解析失败尝试作为HTML解析
try:
# 标题
title = DomUtils.tag_value(item, "title", default="")
root = etree.HTML(ret_xml)
if root is not None:
# 查找RSS根节点
rss_root = root.xpath('//rss | //feed')
if rss_root:
root = rss_root[0]
except Exception as e:
logger.error(f"HTML解析也失败{str(e)}")
return False
finally:
if parser is not None:
del parser
if root is None:
logger.error("无法解析RSS内容")
return False
# 查找所有item或entry节点
items = root.xpath('.//item | .//entry')
# 限制处理的条目数量
items_count = min(len(items), self.MAX_RSS_ITEMS)
if len(items) > self.MAX_RSS_ITEMS:
logger.warning(f"RSS条目过多: {len(items)},仅处理前{self.MAX_RSS_ITEMS}")
for item in items[:items_count]:
try:
# 使用xpath提取信息更高效
title_nodes = item.xpath('.//title')
title = title_nodes[0].text if title_nodes and title_nodes[0].text else ""
if not title:
continue
# 描述
description = DomUtils.tag_value(item, "description", default="")
desc_nodes = item.xpath('.//description | .//summary')
description = desc_nodes[0].text if desc_nodes and desc_nodes[0].text else ""
# 种子页面
link = DomUtils.tag_value(item, "link", default="")
link_nodes = item.xpath('.//link')
if link_nodes:
link = link_nodes[0].text if hasattr(link_nodes[0], 'text') and link_nodes[0].text else \
link_nodes[0].get('href', '')
else:
link = ""
# 种子链接
enclosure = DomUtils.tag_value(item, "enclosure", "url", default="")
enclosure_nodes = item.xpath('.//enclosure')
enclosure = enclosure_nodes[0].get('url', '') if enclosure_nodes else ""
if not enclosure and not link:
continue
# 部分RSS只有link没有enclosure
if not enclosure and link:
enclosure = link
# 大小
size = DomUtils.tag_value(item, "enclosure", "length", default=0)
if size and str(size).isdigit():
size = int(size)
else:
size = 0
size = 0
if enclosure_nodes:
size_attr = enclosure_nodes[0].get('length', '0')
if size_attr and str(size_attr).isdigit():
size = int(size_attr)
# 发布日期
pubdate = DomUtils.tag_value(item, "pubDate", default="")
if pubdate:
# 转换为时间
pubdate = StringUtils.get_time(pubdate)
pubdate_nodes = item.xpath('.//pubDate | .//published | .//updated')
pubdate = ""
if pubdate_nodes and pubdate_nodes[0].text:
pubdate = StringUtils.get_time(pubdate_nodes[0].text)
# 获取豆瓣昵称
nickname = DomUtils.tag_value(item, "dc:createor", default="")
nickname_nodes = item.xpath('.//*[local-name()="creator"]')
nickname = nickname_nodes[0].text if nickname_nodes and nickname_nodes[0].text else ""
# 返回对象
tmp_dict = {'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate}
tmp_dict = {
'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate
}
# 如果豆瓣昵称不为空返回数据增加豆瓣昵称供doubansync插件获取
if nickname:
tmp_dict['nickname'] = nickname
ret_array.append(tmp_dict)
except Exception as e1:
logger.debug(f"解析RSS失败{str(e1)} - {traceback.format_exc()}")
logger.debug(f"解析RSS条目失败:{str(e1)} - {traceback.format_exc()}")
continue
except Exception as e2:
logger.error(f"解析RSS失败{str(e2)} - {traceback.format_exc()}")
# RSS过期 观众RSS 链接已过期,您需要获得一个新的! pthome RSS Link has expired, You need to get a new one!
# RSS过期检查
_rss_expired_msg = [
"RSS 链接已过期, 您需要获得一个新的!",
"RSS Link has expired, You need to get a new one!",
@@ -328,6 +395,12 @@ class RssHelper:
if ret_xml in _rss_expired_msg:
return None
return False
finally:
if root is not None:
del root
if ret_xml is not None:
del ret_xml
return ret_array
def get_rss_link(self, url: str, cookie: str, ua: str, proxy: bool = False) -> Tuple[str, str]:
@@ -369,12 +442,20 @@ class RssHelper:
return "", f"获取 {url} RSS链接失败错误码{res.status_code},错误原因:{res.reason}"
else:
return "", f"获取RSS链接失败无法连接 {url} "
# 解析HTML
html = etree.HTML(html_text)
if StringUtils.is_valid_html_element(html):
rss_link = html.xpath(site_conf.get("xpath"))
if rss_link:
return str(rss_link[-1]), ""
if html_text:
html = None
try:
html = etree.HTML(html_text)
if StringUtils.is_valid_html_element(html):
rss_link = html.xpath(site_conf.get("xpath"))
if rss_link:
return str(rss_link[-1]), ""
finally:
if html is not None:
del html
return "", f"获取RSS链接失败{url}"
except Exception as e:
return "", f"获取 {url} RSS链接失败{str(e)}"

View File

@@ -11,14 +11,12 @@ class RuleHelper:
规划帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_rule_groups(self) -> List[FilterRuleGroup]:
@staticmethod
def get_rule_groups() -> List[FilterRuleGroup]:
"""
获取用户所有规则组
"""
rule_groups: List[dict] = self.systemconfig.get(SystemConfigKey.UserFilterRuleGroups)
rule_groups: List[dict] = SystemConfigOper().get(SystemConfigKey.UserFilterRuleGroups)
if not rule_groups:
return []
return [FilterRuleGroup(**group) for group in rule_groups]
@@ -50,11 +48,12 @@ class RuleHelper:
ret_groups.append(group)
return ret_groups
def get_custom_rules(self) -> List[CustomRule]:
@staticmethod
def get_custom_rules() -> List[CustomRule]:
"""
获取用户所有自定义规则
"""
rules: List[dict] = self.systemconfig.get(SystemConfigKey.CustomFilterRules)
rules: List[dict] = SystemConfigOper().get(SystemConfigKey.CustomFilterRules)
if not rules:
return []
return [CustomRule(**rule) for rule in rules]

View File

@@ -10,14 +10,12 @@ class StorageHelper:
存储帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_storagies(self) -> List[schemas.StorageConf]:
@staticmethod
def get_storagies() -> List[schemas.StorageConf]:
"""
获取所有存储设置
"""
storage_confs: List[dict] = self.systemconfig.get(SystemConfigKey.Storages)
storage_confs: List[dict] = SystemConfigOper().get(SystemConfigKey.Storages)
if not storage_confs:
return []
return [schemas.StorageConf(**s) for s in storage_confs]
@@ -49,7 +47,7 @@ class StorageHelper:
if s.type == storage:
s.config = conf
break
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])
def add_storage(self, storage: str, name: str, conf: dict):
"""
@@ -70,7 +68,7 @@ class StorageHelper:
name=name,
config=conf
))
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])
def reset_storage(self, storage: str):
"""
@@ -81,4 +79,4 @@ class StorageHelper:
if s.type == storage:
s.config = {}
break
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])

View File

@@ -50,11 +50,11 @@ class SubscribeHelper(metaclass=Singleton):
]
def __init__(self):
self.systemconfig = SystemConfigOper()
systemconfig = SystemConfigOper()
if settings.SUBSCRIBE_STATISTIC_SHARE:
if not self.systemconfig.get(SystemConfigKey.SubscribeReport):
if not systemconfig.get(SystemConfigKey.SubscribeReport):
if self.sub_report():
self.systemconfig.set(SystemConfigKey.SubscribeReport, "1")
systemconfig.set(SystemConfigKey.SubscribeReport, "1")
self.get_user_uuid()
self.get_github_user()

View File

@@ -1,13 +1,39 @@
import os
import signal
from pathlib import Path
from typing import Tuple
import docker
from app.core.config import settings
from app.core.event import eventmanager, Event
from app.log import logger
from app.schemas import ConfigChangeEventData
from app.schemas.types import EventType
from app.utils.system import SystemUtils
class SystemHelper:
"""
系统工具类,提供系统相关的操作和判断
"""
__system_flag_file = "/var/log/nginx/__moviepilot__"
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件,更新日志设置
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in ['DEBUG', 'LOG_LEVEL', 'LOG_MAX_FILE_SIZE', 'LOG_BACKUP_COUNT',
'LOG_FILE_FORMAT', 'LOG_CONSOLE_FORMAT']:
return
logger.update_loggers()
@staticmethod
def can_restart() -> bool:
"""
@@ -19,17 +45,12 @@ class SystemHelper:
)
@staticmethod
def restart() -> Tuple[bool, str]:
def _get_container_id() -> str:
"""
执行Docker重启操作
获取当前容器ID
"""
if not SystemUtils.is_docker():
return False, "非Docker环境无法重启"
container_id = None
try:
# 创建 Docker 客户端
client = docker.DockerClient(base_url=settings.DOCKER_CLIENT_API)
# 获取当前容器的 ID
container_id = None
with open("/proc/self/mountinfo", "r") as f:
data = f.read()
index_resolv_conf = data.find("resolv.conf")
@@ -45,11 +66,100 @@ class SystemHelper:
data.rfind("/", 0, index_second_slash) + 1
)
container_id = data[index_first_slash:index_second_slash]
except Exception as e:
logger.debug(f"获取容器ID失败: {str(e)}")
return container_id.strip() if container_id else None
@staticmethod
def _check_restart_policy() -> bool:
"""
检查当前容器是否配置了自动重启策略
"""
try:
# 获取当前容器ID
container_id = SystemHelper._get_container_id()
if not container_id:
return False
# 创建 Docker 客户端
client = docker.DockerClient(base_url=settings.DOCKER_CLIENT_API)
# 获取容器信息
container = client.containers.get(container_id)
restart_policy = container.attrs.get('HostConfig', {}).get('RestartPolicy', {})
policy_name = restart_policy.get('Name', 'no')
# 检查是否有有效的重启策略
auto_restart_policies = ['always', 'unless-stopped', 'on-failure']
has_restart_policy = policy_name in auto_restart_policies
logger.info(f"容器重启策略: {policy_name}, 支持自动重启: {has_restart_policy}")
return has_restart_policy
except Exception as e:
logger.warning(f"检查重启策略失败: {str(e)}")
return False
@staticmethod
def restart() -> Tuple[bool, str]:
"""
执行Docker重启操作
"""
if not SystemUtils.is_docker():
return False, "非Docker环境无法重启"
try:
# 检查容器是否配置了自动重启策略
has_restart_policy = SystemHelper._check_restart_policy()
if has_restart_policy:
# 有重启策略,使用优雅退出方式
logger.info("检测到容器配置了自动重启策略,使用优雅重启方式...")
# 发送SIGTERM信号给当前进程触发优雅停止
os.kill(os.getpid(), signal.SIGTERM)
return True, ""
else:
# 没有重启策略使用Docker API强制重启
logger.info("容器未配置自动重启策略使用Docker API重启...")
return SystemHelper._docker_api_restart()
except Exception as err:
logger.error(f"重启失败: {str(err)}")
# 降级为Docker API重启
logger.warning("降级为Docker API重启...")
return SystemHelper._docker_api_restart()
@staticmethod
def _docker_api_restart() -> Tuple[bool, str]:
"""
使用Docker API重启容器并尝试优雅停止
"""
try:
# 创建 Docker 客户端
client = docker.DockerClient(base_url=settings.DOCKER_CLIENT_API)
container_id = SystemHelper._get_container_id()
if not container_id:
return False, "获取容器ID失败"
# 重启当前容器
client.containers.get(container_id.strip()).restart()
# 重启容器
client.containers.get(container_id).restart()
return True, ""
except Exception as err:
print(str(err))
return False, f"重启时发生错误:{str(err)}"
except Exception as docker_err:
return False, f"重启时发生错误:{str(docker_err)}"
def set_system_modified(self):
"""
设置系统已修改标志
"""
try:
if SystemUtils.is_docker():
Path(self.__system_flag_file).touch(exist_ok=True)
except Exception as e:
print(f"设置系统修改标志失败: {str(e)}")
def is_system_reset(self) -> bool:
"""
检查系统是否已被重置
:return: 如果系统已重置,返回 True否则返回 False
"""
if SystemUtils.is_docker():
return not Path(self.__system_flag_file).exists()
return False

View File

@@ -2,14 +2,15 @@ from concurrent.futures import ThreadPoolExecutor
from typing import Optional
from app.utils.singleton import Singleton
from app.core.config import settings
class ThreadHelper(metaclass=Singleton):
"""
线程池管理
"""
def __init__(self, max_workers: Optional[int] = 50):
self.pool = ThreadPoolExecutor(max_workers=max_workers)
def __init__(self):
self.pool = ThreadPoolExecutor(max_workers=settings.CONF['threadpool'])
def submit(self, func, *args, **kwargs):
"""

View File

@@ -28,10 +28,6 @@ class TorrentHelper(metaclass=Singleton):
# 失败的种子:站点链接
_invalid_torrents = []
def __init__(self):
self.system_config = SystemConfigOper()
self.site_oper = SiteOper()
def download_torrent(self, url: str,
cookie: Optional[str] = None,
ua: Optional[str] = None,
@@ -192,7 +188,8 @@ class TorrentHelper(metaclass=Singleton):
file_name = str(datetime.datetime.now())
return file_name
def sort_torrents(self, torrent_list: List[Context]) -> List[Context]:
@staticmethod
def sort_torrents(torrent_list: List[Context]) -> List[Context]:
"""
对种子对行排序torrent、site、upload、seeder
"""
@@ -200,11 +197,11 @@ class TorrentHelper(metaclass=Singleton):
return []
# 下载规则
priority_rule: List[str] = self.system_config.get(
priority_rule: List[str] = SystemConfigOper().get(
SystemConfigKey.TorrentsPriority) or ["torrent", "upload", "seeder"]
# 站点上传量
site_uploads = {
site.name: site.upload for site in self.site_oper.get_userdata_latest()
site.name: site.upload for site in SiteOper().get_userdata_latest()
}
def get_sort_str(_context):

View File

@@ -1,5 +1,7 @@
from typing import Optional, List
from app.chain.mediaserver import MediaServerChain
from app.chain.tmdb import TmdbChain
from app.core.cache import cached
from app.core.config import settings
from app.utils.http import RequestUtils
@@ -7,9 +9,50 @@ from app.utils.singleton import Singleton
class WallpaperHelper(metaclass=Singleton):
"""
壁纸帮助类
"""
def __init__(self):
self.req = RequestUtils(timeout=5)
def get_wallpaper(self) -> Optional[str]:
"""
获取登录页面壁纸
"""
if settings.WALLPAPER == "bing":
url = self.get_bing_wallpaper()
elif settings.WALLPAPER == "mediaserver":
url = self.get_mediaserver_wallpaper()
elif settings.WALLPAPER == "customize":
url = self.get_customize_wallpaper()
else:
url = self.get_tmdb_wallpaper()
return url
def get_wallpapers(self, num: int = 10) -> List[str]:
"""
获取登录页面壁纸列表
"""
if settings.WALLPAPER == "bing":
return self.get_bing_wallpapers(num)
elif settings.WALLPAPER == "mediaserver":
return self.get_mediaserver_wallpapers(num)
elif settings.WALLPAPER == "customize":
return self.get_customize_wallpapers()
else:
return self.get_tmdb_wallpapers(num)
@cached(maxsize=1, ttl=3600)
def get_tmdb_wallpaper(self) -> Optional[str]:
"""
获取TMDB每日壁纸
"""
return TmdbChain().get_random_wallpager()
@cached(maxsize=1, ttl=3600, skip_empty=True)
def get_tmdb_wallpapers(self, num: int = 10) -> List[str]:
"""
获取7天的TMDB每日壁纸
"""
return TmdbChain().get_trending_wallpapers(num)
@cached(maxsize=1, ttl=3600)
def get_bing_wallpaper(self) -> Optional[str]:
@@ -17,7 +60,7 @@ class WallpaperHelper(metaclass=Singleton):
获取Bing每日壁纸
"""
url = "https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1"
resp = self.req.get_res(url)
resp = RequestUtils(timeout=5).get_res(url)
if resp and resp.status_code == 200:
try:
result = resp.json()
@@ -28,13 +71,13 @@ class WallpaperHelper(metaclass=Singleton):
print(str(err))
return None
@cached(maxsize=1, ttl=3600)
@cached(maxsize=1, ttl=3600, skip_empty=True)
def get_bing_wallpapers(self, num: int = 7) -> List[str]:
"""
获取7天的Bing每日壁纸
"""
url = f"https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n={num}"
resp = self.req.get_res(url)
resp = RequestUtils(timeout=5).get_res(url)
if resp and resp.status_code == 200:
try:
result = resp.json()
@@ -44,6 +87,20 @@ class WallpaperHelper(metaclass=Singleton):
print(str(err))
return []
@cached(maxsize=1, ttl=3600)
def get_mediaserver_wallpaper(self) -> Optional[str]:
"""
获取媒体服务器壁纸
"""
return MediaServerChain().get_latest_wallpaper()
@cached(maxsize=1, ttl=3600, skip_empty=True)
def get_mediaserver_wallpapers(self, num: int = 10) -> List[str]:
"""
获取媒体服务器壁纸列表
"""
return MediaServerChain().get_latest_wallpapers(count=num)
@cached(maxsize=1, ttl=3600)
def get_customize_wallpaper(self) -> Optional[str]:
"""
@@ -54,7 +111,7 @@ class WallpaperHelper(metaclass=Singleton):
return wallpaper_list[0]
return None
@cached(maxsize=1, ttl=3600)
@cached(maxsize=1, ttl=3600, skip_empty=True)
def get_customize_wallpapers(self) -> List[str]:
"""
获取自定义壁纸api壁纸
@@ -87,7 +144,7 @@ class WallpaperHelper(metaclass=Singleton):
# 判断是否存在自定义壁纸api
if settings.CUSTOMIZE_WALLPAPER_API_URL:
wallpaper_list = []
resp = self.req.get_res(settings.CUSTOMIZE_WALLPAPER_API_URL)
resp = RequestUtils(timeout=15).get_res(settings.CUSTOMIZE_WALLPAPER_API_URL)
if resp and resp.status_code == 200:
# 如果返回的是图片格式
content_type = resp.headers.get('Content-Type')

View File

@@ -28,7 +28,7 @@ class LogConfigModel(BaseModel):
# 日志文件最大大小单位MB
LOG_MAX_FILE_SIZE: int = 5
# 备份的日志文件数量
LOG_BACKUP_COUNT: int = 3
LOG_BACKUP_COUNT: int = 10
# 控制台日志格式
LOG_CONSOLE_FORMAT: str = "%(leveltext)s[%(name)s] %(asctime)s %(message)s"
# 文件日志格式
@@ -99,6 +99,24 @@ class LoggerManager:
# 线程锁
_lock = threading.Lock()
def get_logger(self, name: str) -> logging.Logger:
"""
获取一个指定名称的、独立的日志记录器。
创建一个独立的日志文件,例如 'diag_memory.log'
:param name: 日志记录器的名称,也将用作文件名。
:return: 一个配置好的 logging.Logger 实例。
"""
# 使用名称作为日志文件名
logfile = f"{name}.log"
with LoggerManager._lock:
# 检查是否已经创建过这个 logger
_logger = self._loggers.get(logfile)
if not _logger:
# 如果没有,就使用现有的 __setup_logger 来创建一个新的
_logger = self.__setup_logger(log_file=logfile)
self._loggers[logfile] = _logger
return _logger
@staticmethod
def __get_caller():
"""

View File

@@ -1,5 +1,6 @@
import multiprocessing
import os
import signal
import sys
import threading
@@ -21,7 +22,7 @@ from app.db.init import init_db, update_db
# uvicorn服务
Server = uvicorn.Server(Config(app, host=settings.HOST, port=settings.PORT,
reload=settings.DEV, workers=multiprocessing.cpu_count(),
timeout_graceful_shutdown=5))
timeout_graceful_shutdown=60))
def start_tray():
@@ -70,7 +71,19 @@ def start_tray():
threading.Thread(target=TrayIcon.run, daemon=True).start()
def signal_handler(signum, frame):
"""
信号处理函数,用于优雅停止服务
"""
print(f"收到信号 {signum},开始优雅停止服务...")
Server.should_exit = True
if __name__ == '__main__':
# 注册信号处理器
signal.signal(signal.SIGTERM, signal_handler)
signal.signal(signal.SIGINT, signal_handler)
# 启动托盘
start_tray()
# 初始化数据库

View File

@@ -191,8 +191,6 @@ class _MessageBase(ServiceBase[TService, NotificationConf]):
:return: 返回消息通知的配置字典
"""
if self._configs is not None:
return self._configs
configs = ServiceConfigHelper.get_notification_configs()
if not self._service_name:
return {}
@@ -260,8 +258,6 @@ class _DownloaderBase(ServiceBase[TService, DownloaderConf]):
:return: 返回下载器配置字典
"""
if self._configs is not None:
return self._configs
configs = ServiceConfigHelper.get_downloader_configs()
if not self._service_name:
return {}
@@ -279,8 +275,6 @@ class _MediaServerBase(ServiceBase[TService, MediaServerConf]):
:return: 返回媒体服务器配置字典
"""
if self._configs is not None:
return self._configs
configs = ServiceConfigHelper.get_mediaserver_configs()
if not self._service_name:
return {}

View File

@@ -18,7 +18,7 @@ class BangumiModule(_ModuleBase):
self.bangumiapi = BangumiApi()
def stop(self):
pass
self.bangumiapi.close()
def test(self) -> Tuple[bool, str]:
"""

View File

@@ -25,19 +25,18 @@ class BangumiApi(object):
"person_credits": "v0/persons/%s/subjects",
}
_base_url = "https://api.bgm.tv/"
_req = RequestUtils(session=requests.Session())
def __init__(self):
pass
self._session = requests.Session()
self._req = RequestUtils(session=self._session)
@classmethod
@cached(maxsize=settings.CACHE_CONF["bangumi"], ttl=settings.CACHE_CONF["meta"])
def __invoke(cls, url, key: Optional[str] = None, **kwargs):
req_url = cls._base_url + url
@cached(maxsize=settings.CONF["bangumi"], ttl=settings.CONF["meta"])
def __invoke(self, url, key: Optional[str] = None, **kwargs):
req_url = self._base_url + url
params = {}
if kwargs:
params.update(kwargs)
resp = cls._req.get_res(url=req_url, params=params)
resp = self._req.get_res(url=req_url, params=params)
try:
if not resp:
return None
@@ -207,3 +206,7 @@ class BangumiApi(object):
return self.__invoke(self._urls["discover"],
key="data",
_ts=datetime.strftime(datetime.now(), '%Y%m%d'), **kwargs)
def close(self):
if self._session:
self._session.close()

View File

@@ -171,14 +171,14 @@ class DoubanApi(metaclass=Singleton):
).digest()
).decode()
@cached(maxsize=settings.CACHE_CONF["douban"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
def __invoke_recommend(self, url: str, **kwargs) -> dict:
"""
推荐/发现类API
"""
return self.__invoke(url, **kwargs)
@cached(maxsize=settings.CACHE_CONF["douban"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
def __invoke_search(self, url: str, **kwargs) -> dict:
"""
搜索类API
@@ -213,7 +213,7 @@ class DoubanApi(metaclass=Singleton):
return resp.json()
return resp.json() if resp else {}
@cached(maxsize=settings.CACHE_CONF["douban"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
def __post(self, url: str, **kwargs) -> dict:
"""
POST请求

View File

@@ -16,7 +16,7 @@ from app.schemas.types import MediaType
lock = RLock()
CACHE_EXPIRE_TIMESTAMP_STR = "cache_expire_timestamp"
EXPIRE_TIMESTAMP = settings.CACHE_CONF["meta"]
EXPIRE_TIMESTAMP = settings.CONF["meta"]
class DoubanCache(metaclass=Singleton):

View File

@@ -2,11 +2,11 @@ from typing import Any, Generator, List, Optional, Tuple, Union
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _MediaServerBase, _ModuleBase
from app.modules.emby.emby import Emby
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
@@ -18,6 +18,19 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
super().init_service(service_name=Emby.__name__.lower(),
service_type=lambda conf: Emby(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Emby"
@@ -269,7 +282,8 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
) for season, episodes in seasoninfo.items()]
def mediaserver_playing(self, server: str,
count: Optional[int] = 20, username: Optional[str] = None) -> List[schemas.MediaServerPlayItem]:
count: Optional[int] = 20, username: Optional[str] = None) -> List[
schemas.MediaServerPlayItem]:
"""
获取媒体服务器正在播放信息
"""
@@ -288,7 +302,8 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
return server_obj.get_play_url(item_id)
def mediaserver_latest(self, server: Optional[str] = None,
count: Optional[int] = 20, username: Optional[str] = None) -> List[schemas.MediaServerPlayItem]:
count: Optional[int] = 20, username: Optional[str] = None) -> List[
schemas.MediaServerPlayItem]:
"""
获取媒体服务器最新入库条目
"""

View File

@@ -13,7 +13,7 @@ from app.log import logger
from app.schemas.types import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Emby:

View File

@@ -399,10 +399,28 @@ class FanartModule(_ModuleBase):
if not mediainfo.get_image(season_image):
mediainfo.set_image(season_image, image_obj.get('url'))
else:
# 其他图片,按欢迎程度倒排
images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
# 取第一张图片
image_obj = images[0]
# 其他图片,优先环境变量指定语言再like最多
def pick_best_image(images):
lang_env = settings.FANART_LANG
if lang_env:
langs = [lang.strip() for lang in lang_env.split(",") if lang.strip()]
for lang in langs:
lang_images = [img for img in images if img.get('lang') == lang]
if lang_images:
lang_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return lang_images[0]
# 没设置或没找到,按原逻辑 zh、en、like最多
zh_images = [img for img in images if img.get('lang') == 'zh']
if zh_images:
zh_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return zh_images[0]
en_images = [img for img in images if img.get('lang') == 'en']
if en_images:
en_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return en_images[0]
images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return images[0]
image_obj = pick_best_image(images)
# 设置图片,没有图片才设置
if not mediainfo.get_image(image_name):
mediainfo.set_image(image_name, image_obj.get('url'))
@@ -420,7 +438,7 @@ class FanartModule(_ModuleBase):
return result
@classmethod
@cached(maxsize=settings.CACHE_CONF["fanart"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["fanart"], ttl=settings.CONF["meta"])
def __request_fanart(cls, media_type: MediaType, queryid: Union[str, int]) -> Optional[dict]:
if media_type == MediaType.MOVIE:
image_url = cls._movie_url % queryid

View File

@@ -3,6 +3,7 @@ from datetime import datetime
from pathlib import Path
from typing import Optional, List, Dict
import requests
from requests import Response
from app import schemas
@@ -529,20 +530,19 @@ class Alist(StorageBase, metaclass=Singleton):
if result["data"]["sign"]:
download_url = download_url + "?sign=" + result["data"]["sign"]
resp = RequestUtils(
headers=self.__get_header_with_token()
).get_res(download_url)
if not path:
new_path = settings.TEMP_PATH / fileitem.name
local_path = settings.TEMP_PATH / fileitem.name
else:
new_path = path / fileitem.name
local_path = path / fileitem.name
with open(new_path, "wb") as f:
f.write(resp.content)
with requests.get(download_url, headers=self.__get_header_with_token(), stream=True) as r:
r.raise_for_status()
with open(local_path, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
if new_path.exists():
return new_path
if local_path.exists():
return local_path
return None
def upload(

View File

@@ -238,7 +238,11 @@ class IndexerModule(_ModuleBase):
cat=cat,
page=page)
return _spider.is_error, _spider.get_torrents()
try:
return _spider.is_error, _spider.get_torrents()
finally:
# 显式清理SiteSpider对象
del _spider
def refresh_torrents(self, site: dict,
keyword: Optional[str] = None, cat: Optional[str] = None, page: Optional[int] = 0) -> Optional[List[TorrentInfo]]:

View File

@@ -17,7 +17,7 @@ from app.utils.site import SiteUtils
# 站点框架
class SiteSchema(Enum):
DiscuzX = "Discuz!"
DiscuzX = "DiscuzX"
Gazelle = "Gazelle"
Ipt = "IPTorrents"
NexusPhp = "NexusPhp"

View File

@@ -2,12 +2,12 @@ from typing import Any, Generator, List, Optional, Tuple, Union
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _MediaServerBase, _ModuleBase
from app.modules.jellyfin.jellyfin import Jellyfin
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class JellyfinModule(_ModuleBase, _MediaServerBase[Jellyfin]):
@@ -19,6 +19,19 @@ class JellyfinModule(_ModuleBase, _MediaServerBase[Jellyfin]):
super().init_service(service_name=Jellyfin.__name__.lower(),
service_type=lambda conf: Jellyfin(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Jellyfin"

View File

@@ -10,7 +10,7 @@ from app.log import logger
from app.schemas import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Jellyfin:

View File

@@ -2,12 +2,12 @@ from typing import Optional, Tuple, Union, Any, List, Generator
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MediaServerBase
from app.modules.plex.plex import Plex
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
@@ -19,6 +19,19 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
super().init_service(service_name=Plex.__name__.lower(),
service_type=lambda conf: Plex(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Plex"
@@ -45,7 +58,12 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
return 3
def stop(self):
pass
"""
停止模块服务
"""
for server in self.get_instances().values():
if server:
server.close()
def test(self) -> Optional[Tuple[bool, str]]:
"""
@@ -273,7 +291,8 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
episodes=episodes
) for season, episodes in seasoninfo.items()]
def mediaserver_playing(self, server: str, count: Optional[int] = 20, **kwargs) -> List[schemas.MediaServerPlayItem]:
def mediaserver_playing(self, server: str, count: Optional[int] = 20,
**kwargs) -> List[schemas.MediaServerPlayItem]:
"""
获取媒体服务器正在播放信息
"""

View File

@@ -14,7 +14,7 @@ from app.log import logger
from app.schemas import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Plex:
@@ -890,3 +890,7 @@ class Plex:
session = Session()
session.headers = headers
return session
def close(self):
if self._session:
self._session.close()

View File

@@ -7,11 +7,12 @@ from torrentool.torrent import Torrent
from app import schemas
from app.core.config import settings
from app.core.metainfo import MetaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _DownloaderBase
from app.modules.qbittorrent.qbittorrent import Qbittorrent
from app.schemas import TransferTorrent, DownloadingTorrent
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType, SystemConfigKey, EventType
from app.utils.string import StringUtils
@@ -24,6 +25,19 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
super().init_service(service_name=Qbittorrent.__name__.lower(),
service_type=Qbittorrent)
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Qbittorrent"
@@ -286,12 +300,13 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
dlspeed=StringUtils.str_filesize(torrent.get('dlspeed')),
upspeed=StringUtils.str_filesize(torrent.get('upspeed')),
left_time=StringUtils.str_secends(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get('dlspeed')) if torrent.get(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get(
'dlspeed')) if torrent.get(
'dlspeed') > 0 else ''
))
else:
return None
return ret_torrents
return ret_torrents # noqa
def transfer_completed(self, hashs: str, downloader: Optional[str] = None) -> None:
"""
@@ -303,6 +318,7 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
if not server:
return None
server.set_torrents_tag(ids=hashs, tags=['已整理'])
return None
def remove_torrents(self, hashs: Union[str, list], delete_file: Optional[bool] = True,
downloader: Optional[str] = None) -> Optional[bool]:

View File

@@ -3,11 +3,12 @@ import re
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.slack.slack import Slack
from app.schemas import MessageChannel, CommingMessage, Notification
from app.schemas.types import ModuleType
from app.schemas import MessageChannel, CommingMessage, Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, SystemConfigKey, EventType
class SlackModule(_ModuleBase, _MessageBase[Slack]):
@@ -20,6 +21,19 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
service_type=Slack)
self._channel = MessageChannel.Slack
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Slack"

View File

@@ -1,11 +1,12 @@
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.synologychat.synologychat import SynologyChat
from app.schemas import MessageChannel, CommingMessage, Notification
from app.schemas.types import ModuleType
from app.schemas import MessageChannel, CommingMessage, Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, SystemConfigKey, EventType
class SynologyChatModule(_ModuleBase, _MessageBase[SynologyChat]):
@@ -18,6 +19,19 @@ class SynologyChatModule(_ModuleBase, _MessageBase[SynologyChat]):
service_type=SynologyChat)
self._channel = MessageChannel.SynologyChat
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Synology Chat"

View File

@@ -1,14 +1,16 @@
import copy
import json
from typing import Optional, Union, List, Tuple, Any, Dict
from typing import Dict
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import Event
from app.core.event import eventmanager
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.telegram.telegram import Telegram
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData
from app.schemas.types import ModuleType, ChainEventType
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData, ConfigChangeEventData
from app.schemas.types import ModuleType, ChainEventType, SystemConfigKey, EventType
from app.utils.structures import DictUtils
@@ -22,6 +24,19 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
service_type=Telegram)
self._channel = MessageChannel.Telegram
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Telegram"

View File

@@ -4,6 +4,7 @@ import uuid
from pathlib import Path
from threading import Event
from typing import Optional, List, Dict
from urllib.parse import urljoin
import telebot
from telebot import apihelper
@@ -17,8 +18,6 @@ from app.utils.common import retry
from app.utils.http import RequestUtils
from app.utils.string import StringUtils
apihelper.proxy = settings.PROXY
class Telegram:
_ds_url = f"http://127.0.0.1:{settings.PORT}/api/v1/message?token={settings.API_TOKEN}"
@@ -38,6 +37,12 @@ class Telegram:
self._telegram_chat_id = TELEGRAM_CHAT_ID
# 初始化机器人
if self._telegram_token and self._telegram_chat_id:
# telegram bot api 地址格式https://api.telegram.org
if kwargs.get("API_URL"):
apihelper.API_URL = urljoin(kwargs["API_URL"], '/bot{0}/{1}')
apihelper.FILE_URL = urljoin(kwargs["API_URL"], '/file/bot{0}/{1}')
else:
apihelper.proxy = settings.PROXY
# bot
_bot = telebot.TeleBot(self._telegram_token, parse_mode="Markdown")
# 记录句柄
@@ -157,7 +162,8 @@ class Telegram:
return False
def send_torrents_msg(self, torrents: List[Context],
userid: Optional[str] = None, title: Optional[str] = None, link: Optional[str] = None) -> Optional[bool]:
userid: Optional[str] = None, title: Optional[str] = None,
link: Optional[str] = None) -> Optional[bool]:
"""
发送列表消息
"""

View File

@@ -406,7 +406,8 @@ class TheMovieDbModule(_ModuleBase):
return None
return self.scraper.get_metadata_nfo(meta=meta, mediainfo=mediainfo, season=season, episode=episode)
def metadata_img(self, mediainfo: MediaInfo, season: Optional[int] = None, episode: Optional[int] = None) -> Optional[dict]:
def metadata_img(self, mediainfo: MediaInfo, season: Optional[int] = None,
episode: Optional[int] = None) -> Optional[dict]:
"""
获取图片名称和url
:param mediainfo: 媒体信息
@@ -506,7 +507,6 @@ class TheMovieDbModule(_ModuleBase):
air_date=sea.get("episodes")[0].get("air_date") if sea.get("episodes") else None,
) for sea in group_seasons]
def tmdb_episodes(self, tmdbid: int, season: int, episode_group: Optional[str] = None) -> List[schemas.TmdbEpisode]:
"""
根据TMDBID查询某季的所有集信息

View File

@@ -15,7 +15,7 @@ from app.schemas.types import MediaType
lock = RLock()
CACHE_EXPIRE_TIMESTAMP_STR = "cache_expire_timestamp"
EXPIRE_TIMESTAMP = settings.CACHE_CONF["meta"]
EXPIRE_TIMESTAMP = settings.CONF["meta"]
class TmdbCache(metaclass=Singleton):

View File

@@ -500,7 +500,7 @@ class TmdbApi:
return ret_info
@cached(maxsize=settings.CACHE_CONF["tmdb"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["tmdb"], ttl=settings.CONF["meta"])
@rate_limit_exponential(source="match_tmdb_web", base_wait=5, max_wait=1800, enable_logging=True)
def match_web(self, name: str, mtype: MediaType) -> Optional[dict]:
"""
@@ -636,7 +636,6 @@ class TmdbApi:
return None
# dict[地区:分级]
ratings = {}
results = []
if results := (tmdb_info.get("release_dates") or {}).get("results"):
"""
[
@@ -1362,8 +1361,6 @@ class TmdbApi:
return group_season
return {}
def get_person_detail(self, person_id: int) -> dict:
"""
获取人物详情

View File

@@ -124,7 +124,7 @@ class TMDb(object):
def cache(self, cache):
self._cache_enabled = bool(cache)
@cached(maxsize=settings.CACHE_CONF["tmdb"], ttl=settings.CACHE_CONF["meta"])
@cached(maxsize=settings.CONF["tmdb"], ttl=settings.CONF["meta"])
def cached_request(self, method, url, data, json,
_ts=datetime.strftime(datetime.now(), '%Y%m%d')):
"""

View File

@@ -7,11 +7,12 @@ from transmission_rpc import File
from app import schemas
from app.core.config import settings
from app.core.metainfo import MetaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _DownloaderBase
from app.modules.transmission.transmission import Transmission
from app.schemas import TransferTorrent, DownloadingTorrent
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType, SystemConfigKey, EventType
from app.utils.string import StringUtils
@@ -24,6 +25,19 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
super().init_service(service_name=Transmission.__name__.lower(),
service_type=Transmission)
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Transmission"
@@ -278,7 +292,7 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
))
else:
return None
return ret_torrents
return ret_torrents # noqa
def transfer_completed(self, hashs: str, downloader: Optional[str] = None) -> None:
"""
@@ -298,6 +312,7 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
else:
tags = ['已整理']
server.set_torrent_tag(ids=hashs, tags=tags)
return None
def remove_torrents(self, hashs: Union[str, list], delete_file: Optional[bool] = True,
downloader: Optional[str] = None) -> Optional[bool]:
@@ -340,7 +355,7 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
server: Transmission = self.get_instance(downloader)
if not server:
return None
return server.start_torrents(ids=hashs)
return server.stop_torrents(ids=hashs)
def torrent_files(self, tid: str, downloader: Optional[str] = None) -> Optional[List[File]]:
"""

Some files were not shown because too many files have changed in this diff Show More