Compare commits

...

143 Commits

Author SHA1 Message Date
jxxghp
97cee657bd 更新 .gitignore 文件以包含 Pylint 相关文件,并修改 system.py 中的成功返回逻辑 2025-06-05 21:58:50 +08:00
jxxghp
fe894754cf 更新 system.py 2025-06-05 21:39:13 +08:00
jxxghp
9ffb1d1931 更新 wallpaper.py 2025-06-05 21:03:21 +08:00
jxxghp
a16bd30903 更新 wallpaper.py 2025-06-05 21:00:18 +08:00
jxxghp
13f9ea8be4 v2.5.3 2025-06-05 20:28:43 +08:00
jxxghp
304af5e980 fix:仪表盘内存只显示当前程序占用 2025-06-05 17:09:11 +08:00
jxxghp
dc180c09e9 fix wallpaper 2025-06-05 17:03:29 +08:00
jxxghp
8e20e26565 fix:捕捉插件停止异常 2025-06-05 14:07:31 +08:00
jxxghp
11075a4012 fix:增加更多内存控制 2025-06-05 13:33:39 +08:00
jxxghp
a9300faaf8 fix:优化单例模式和类引用 2025-06-05 13:22:16 +08:00
jxxghp
504827b7e5 fix:memory use 2025-06-05 09:57:41 +08:00
jxxghp
e180130b38 fix:memory use 2025-06-05 08:32:24 +08:00
jxxghp
faaee09827 fix:memory use 2025-06-05 08:18:26 +08:00
jxxghp
99334795b6 fix rsshelper 2025-06-04 22:00:46 +08:00
jxxghp
8c9c59ef64 fix rsshelper 2025-06-04 21:42:03 +08:00
jxxghp
7a112000c9 更新 memory.py 2025-06-04 18:46:55 +08:00
jxxghp
1424087d5a fix:memory use 2025-06-04 18:34:49 +08:00
jxxghp
984f4731cd 更新 log.py 2025-06-04 15:33:58 +08:00
jxxghp
3a3de64b0f fix:重构配置热加载 2025-06-04 08:21:14 +08:00
jxxghp
0911854e9d fix Config reload 2025-06-04 07:17:47 +08:00
jxxghp
2af8b6f445 fix Config reload 2025-06-03 23:10:48 +08:00
jxxghp
bbfd8ca3f5 fix Config reload 2025-06-03 23:08:58 +08:00
jxxghp
b4ed2880f7 refactor:重构配置热加载 2025-06-03 20:56:21 +08:00
jxxghp
5f18a21e86 fix:整理失败时也打上已整理标签 2025-06-03 17:48:30 +08:00
jxxghp
5d188e3877 fix module close 2025-06-03 17:11:44 +08:00
jxxghp
90f113a292 remove ttl cache 2025-06-03 16:31:16 +08:00
jxxghp
eecfe58297 fix memory manager startup 2025-06-03 16:27:51 +08:00
jxxghp
079a747210 fix memory manager startup 2025-06-03 16:19:38 +08:00
jxxghp
4be8c70f23 fix memory log 2025-06-03 16:05:49 +08:00
jxxghp
d9aee4df77 fix memory log 2025-06-03 16:03:05 +08:00
jxxghp
225de87d4d fix torrents chain 2025-06-03 15:48:43 +08:00
jxxghp
2ce7cedfbd fix 2025-06-03 12:30:26 +08:00
jxxghp
cfb163d904 fix 2025-06-03 12:27:50 +08:00
jxxghp
de7c9be11b 优化内存管理,增加最大内存配置项,改进内存使用检查逻辑。 2025-06-03 12:25:13 +08:00
jxxghp
841209adc9 fix 2025-06-03 11:49:16 +08:00
jxxghp
e48d51fe6e 优化内存管理和垃圾回收机制 2025-06-03 11:45:17 +08:00
jxxghp
9d436ec7ed fix #4382 2025-06-03 08:19:15 +08:00
jxxghp
fb2b29d088 fix #4382 2025-06-03 07:07:40 +08:00
jxxghp
1c46b0bc20 更新 subscribe.py 2025-06-02 16:23:09 +08:00
jxxghp
81d0e4696a Merge pull request #4379 from jtcymc/v2 2025-06-02 10:48:36 +08:00
shaw
f9a287b52b feat(core): 增加剧集交集最小置信度设置
新增了剧集交集最小置信度的配置项,用于过滤掉包含过多不需要剧集的种子。实现了以下功能:

- 在 config.py 中添加了 EPISODE_INTERSECTION_MIN_CONFIDENCE 配置项,默认值为 0.0
- 修改了 download.py 中的下载逻辑,增加了计算种子与目标缺失集之间交集比例的函数
- 使用交集比例来筛选和排序种子,优先下载与缺失集交集较大的种子
-可以通过配置项设置交集比例的阈值,低于阈值的种子将被跳过

这个改动可以提高下载效率,避免下载过多不必要的剧集。
2025-06-02 00:38:10 +08:00
jxxghp
0f0072abea Merge pull request #4375 from awsl1110/v2 2025-05-31 20:08:10 +08:00
awsl1110
312933a259 fix(indexer): 修正 DiscuzX 站点名称
- 将 Discuz! 站点名称修改为 DiscuzX
2025-05-31 19:18:25 +08:00
jxxghp
288854b8f1 Merge pull request #4374 from awsl1110/v2 2025-05-31 19:04:51 +08:00
awsl1110
7f5991aa34 refactor(core): 优化配置项和模型定义
- 为配置项添加类型注解,提高代码可读性和安全性
- 为模型字段添加默认值,优化数据处理
- 更新验证器使用新语法,以适应Pydantic库的变更
2025-05-31 16:38:06 +08:00
jxxghp
361df95d50 Merge pull request #4372 from cddjr/fix_4371 2025-05-31 13:34:48 +08:00
景大侠
fc1ade32d7 更新蓝光测试用例 2025-05-31 11:05:02 +08:00
景大侠
b74c7531d9 fix #4371 递归判断蓝光目录 2025-05-31 02:37:14 +08:00
景大侠
7e3be3325a fix #4294 更新测试用例 2025-05-31 01:52:31 +08:00
jxxghp
7dab7fbe66 更新 transhandler.py 2025-05-30 21:42:50 +08:00
jxxghp
62c06b6593 fix #4216 2025-05-30 17:32:37 +08:00
jxxghp
000b62969f v2.5.2 2025-05-30 17:06:21 +08:00
jxxghp
b4473bb4a7 fix 插件分身服务注册 2025-05-30 16:59:54 +08:00
jxxghp
2c0e06d599 fix 插件分身服务注册 2025-05-30 13:37:40 +08:00
jxxghp
d2c55e8ed3 Merge remote-tracking branch 'origin/v2' into v2 2025-05-30 08:07:57 +08:00
jxxghp
714abaa25a fix rename 2025-05-30 08:07:53 +08:00
jxxghp
0017eb987b Merge pull request #4365 from Aqr-K/fix-modules/thetvdb 2025-05-29 21:17:38 +08:00
Aqr-K
e5a0894692 fix(tvdb): 解决无网络环境时,tvdb 模块初始化时,仍然会进入超长等待的问题
- 改为惰性初始化,启动时不再执行 `auth` ,调用方法时,再进行 `auth` (保留 auth_token 过期检查重新 `auth` 的功能);
- 使用 双重检查锁定 的方式,保证线程安全;
- 统一通过一个 `timeout` 值进行设置,默认值从30秒降为15秒,保持与tmdb相同。
2025-05-29 20:04:18 +08:00
jxxghp
a8e00e9f0f fix apis 2025-05-29 13:35:01 +08:00
jxxghp
77a4c271ae Merge pull request #4361 from madrays/v2
增加缓存管理页面
2025-05-29 09:21:45 +08:00
jxxghp
014b77c3c7 v2.5.1-1 2025-05-29 08:30:31 +08:00
jxxghp
076e241056 fix tvdb 2025-05-29 08:30:14 +08:00
jxxghp
7ce57cc67a fix 2025-05-29 08:22:45 +08:00
jxxghp
da0343283a 支持在插件文件夹中管理分身插件的添加与移除 2025-05-29 08:16:54 +08:00
jxxghp
d5f7f1ba91 fix tvdb api 2025-05-29 08:03:12 +08:00
jxxghp
8761c82afe fix TVDB代理与SSL校验 #4356 2025-05-29 07:14:42 +08:00
madrays
13023141bc 增加缓存管理页面 2025-05-29 00:46:11 +08:00
jxxghp
4dd2038625 Merge pull request #4360 from cddjr/fix_TransHandler 2025-05-29 00:06:32 +08:00
景大侠
06a32b0e9d fix: TransHandler误报success的bug 2025-05-28 23:52:23 +08:00
jxxghp
c91ab7a76b 添加新的设定项 2025-05-28 21:05:29 +08:00
jxxghp
0344aa6a49 更新 version.py 2025-05-28 20:34:59 +08:00
jxxghp
a748c9d750 修复:更新壁纸助手以支持更多图片格式 2025-05-28 08:26:44 +08:00
jxxghp
038dc372b7 更新 config.py 2025-05-28 07:03:22 +08:00
jxxghp
bc8198fb8a Merge pull request #4356 from TimoYoung/v2 2025-05-27 21:06:54 +08:00
TimoYoung
f42275bd83 Merge remote-tracking branch 'origin/v2' into v2 2025-05-27 18:02:21 +08:00
TimoYoung
6bd86a724e fix:区分series和movie id 2025-05-27 17:58:37 +08:00
TimoYoung
fc96cfe8a0 feat:tvdb模块重写,更换tvdbv4 api,增加搜索能力
sonarr /series/lookup接口重写,直接用标题在tvdb查询剧集
2025-05-27 17:32:25 +08:00
jxxghp
a9f25fe7d6 fix bug 2025-05-27 12:31:43 +08:00
jxxghp
f740fed5f2 fix bug 2025-05-26 13:30:30 +08:00
jxxghp
a6d1bd12a2 fix:优化插件分身性能
feat:分身插件删除时清理文件
2025-05-26 13:21:47 +08:00
jxxghp
e8ab20acf2 Merge pull request #4351 from madrays/v2 2025-05-26 11:08:30 +08:00
madrays
ccfe193800 增加插件分身功能 2025-05-26 10:55:40 +08:00
jxxghp
bdccedca59 更新 system.py 2025-05-26 07:45:21 +08:00
DDSRem
9abb1488df Merge pull request #4348 from Aqr-K/fix-sh
fix(sh): 引号格式问题
2025-05-25 23:48:45 +08:00
Aqr-K
195fc1bdc3 fix(sh): 引号格式问题 2025-05-25 23:47:23 +08:00
jxxghp
2a9129f470 更新 version.py 2025-05-25 20:15:44 +08:00
jxxghp
acbfc0cc6e Merge pull request #4343 from Aqr-K/fix-sh 2025-05-25 19:53:58 +08:00
Aqr-K
bfb0c75e95 fix(sh): 补全调用 2025-05-25 18:50:41 +08:00
jxxghp
161a2ddae8 Merge pull request #4344 from DDS-Derek/dev 2025-05-25 18:32:29 +08:00
Aqr-K
99621cfd66 fix(config): 强制指定 quote_mode ,避免后续依赖升级,默认值不再是 always 2025-05-25 18:30:00 +08:00
DDSRem
e6e7234215 fix(u115): get information directly through id 2025-05-25 18:27:06 +08:00
DDSRem
5b7b329279 fix(docker): repair restart judgment
当 DOCKER_CLIENT_API 不等于默认值时代表外部调用重启,无需再映射 `/var/run/docker.sock`
2025-05-25 18:20:04 +08:00
Aqr-K
3abb2c8674 fix(sh): 重启时,无法同时结合 系统变量 与 env 文件,进行变量读取的问题。 2025-05-25 18:15:35 +08:00
jxxghp
39de89254f add Docker Client API地址 2025-05-25 14:55:51 +08:00
jxxghp
ac941968cb 更新 plugin.py 2025-05-25 11:22:08 +08:00
jxxghp
96f603bfd1 Merge pull request #4339 from jtcymc/v2 2025-05-25 08:01:00 +08:00
shaw
677e38c62d fix(SearchChain): with 关闭线程池
- 使用 with 语句管理 ThreadPoolExecutor,确保线程池正确关闭
2025-05-25 00:44:19 +08:00
jxxghp
72fce20905 feat:整理后记录字幕和音频文件 2025-05-24 20:58:46 +08:00
jxxghp
1eb41c20d5 fix TransferInfo 2025-05-24 15:40:03 +08:00
DDSRem
dd0c1d331f Merge pull request #4334 from DDS-Derek/dev
fix(plugin): dependency dynamic refresh
2025-05-24 09:24:41 +08:00
DDSRem
12760a70a1 fix(plugin): dependency dynamic refresh 2025-05-24 09:23:47 +08:00
jxxghp
525d17270f fix #4332 2025-05-24 06:37:59 +08:00
jxxghp
bc9959f5ab Merge pull request #4333 from Aqr-K/fix-log 2025-05-24 06:31:41 +08:00
jxxghp
94a8cd5128 Merge pull request #4331 from madrays/v2 2025-05-24 06:30:59 +08:00
Aqr-K
5a1b2c4938 fix(log): 区分 主程序日志 与 插件日志 2025-05-24 06:20:41 +08:00
madrays
851a2ac03a Delete requirements.in 2025-05-24 04:12:53 +08:00
madrays
34d7707f53 Delete config/plugins/twofahelper/twofahelper_sites.json 2025-05-24 04:12:13 +08:00
madrays
0aac7f62a3 Delete config/app.env 2025-05-24 04:11:54 +08:00
madrays
34379b92d0 重构插件页面,增加文件夹功能 2025-05-24 03:57:04 +08:00
DDSRem
250999f9f5 Merge pull request #4330 from Aqr-K/patch-1
fix(log): 修复 docker 环境下,重复打印日志的问题
2025-05-24 01:18:59 +08:00
Aqr-K
2b3832222b fix(log): 修复 docker 环境下,重复打印日志的问题 2025-05-24 01:16:14 +08:00
jxxghp
c5f6d0e721 更新 config.py 2025-05-23 21:05:50 +08:00
jxxghp
dbb0cf15b8 fix 最新入库条目 2025-05-23 07:12:47 +08:00
jxxghp
ab202ba951 Merge pull request #4324 from wumode/fix_typo 2025-05-23 06:45:55 +08:00
wumode
e2c13aa7ed fix: 确保名称识别正确兜底 2025-05-23 00:23:45 +08:00
jxxghp
c1ab19f3cf 更新 version.py 2025-05-21 21:42:42 +08:00
jxxghp
beebfb2e19 fix 2025-05-21 08:39:04 +08:00
jxxghp
cfca90aa7d fix delay get_item 2025-05-19 20:06:46 +08:00
jxxghp
19fe0a32c8 fix #4308 2025-05-19 12:53:55 +08:00
jxxghp
76659f8837 fix #4308 2025-05-19 12:51:34 +08:00
jxxghp
2254715190 Merge pull request #4308 from k1z/v2
修复重复识别缓存种子的bug
2025-05-19 12:29:13 +08:00
jxxghp
ae1a5460d4 fix FetchMedias Action 2025-05-19 12:26:27 +08:00
k1z
27d9f910ff 修复重复识别缓存种子的bug 2025-05-19 10:35:09 +08:00
k1z
28db4881d7 修复重复识别缓存种子的bug 2025-05-19 10:05:39 +08:00
jxxghp
7c76c3ccd6 rollback #4296 2025-05-18 21:40:06 +08:00
jxxghp
007bd24374 fix message link check 2025-05-18 15:25:45 +08:00
jxxghp
c8dc30287c fix #4294 x26[45] 调整为小写x 2025-05-18 15:15:01 +08:00
jxxghp
360184bbd1 fix 2025-05-18 13:50:43 +08:00
jxxghp
e8ed2454a1 feat:消息为链接时,交由第三方处理 2025-05-18 13:22:42 +08:00
jxxghp
923ecf29b8 fix #4294 2025-05-18 13:16:06 +08:00
jxxghp
a8f8bf5872 增强MetaBase类以支持tmdbid和doubanid的赋值,并为Emby格式ID识别添加测试用例。 2025-05-18 13:03:35 +08:00
jxxghp
bedcd94020 优化find_metainfo函数,增加对Emby格式ID标签的支持,并添加相应的测试用例以验证不同ID格式的识别。 2025-05-18 12:55:25 +08:00
jxxghp
959d4da1f8 Merge pull request #4300 from DDS-Derek/dev 2025-05-18 10:05:14 +08:00
DDSRem
861453c1a8 fix(u115): refresh delay 2025-05-18 10:03:36 +08:00
jxxghp
2f4072da0d Merge pull request #4297 from wikrin/v2 2025-05-17 20:20:30 +08:00
Attente
411b5e0ca6 fix(database): 将下载模板中的 title 变量更改为 torrent_title 2025-05-17 19:45:49 +08:00
Attente
3f03963811 fix(themoviedb): 直接在 API 层次处理剧集组集号
- 移除 season_group_details 中的冗余集号处理
2025-05-17 19:45:49 +08:00
jxxghp
d43f81e118 Merge pull request #4296 from Pollo3470/fix-bluray-match 2025-05-17 18:11:27 +08:00
Pollo
b97dbd2515 fix: 优化 Blu-ray 匹配规则 2025-05-17 17:56:05 +08:00
jxxghp
c6a20a9ed3 Merge pull request #4294 from Miralia/v2 2025-05-16 21:57:19 +08:00
Miralia
27f0f29eef fix(meta): 修复部分格式识别问题 2025-05-16 20:49:23 +08:00
jxxghp
223508ae72 Merge pull request #4292 from Seed680/v2 2025-05-16 15:55:31 +08:00
qiaoyun680
bce0a4b8cd bugfix:如果自定义壁纸API是图片地址,应该返回请求地址 2025-05-16 15:48:37 +08:00
117 changed files with 5462 additions and 3305 deletions

96
.github/workflows/pylint.yml vendored Normal file
View File

@@ -0,0 +1,96 @@
name: Pylint Code Quality Check
on:
# 在推送到任何分支时运行
push:
branches: [ "main", "v2", "develop" ]
paths:
- '**.py'
# 在PR时运行
pull_request:
branches: [ "main", "v2", "develop" ]
paths:
- '**.py'
# 允许手动触发
workflow_dispatch:
jobs:
pylint:
runs-on: ubuntu-latest
name: Pylint Code Quality Check
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Cache pip dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/requirements.in') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint
# 安装项目依赖
if [ -f requirements.txt ]; then
pip install -r requirements.txt
elif [ -f requirements.in ]; then
pip install -r requirements.in
fi
- name: Verify pylint config
run: |
# 检查项目中的pylint配置文件是否存在
if [ -f .pylintrc ]; then
echo "✅ 找到项目配置文件: .pylintrc"
echo "配置文件内容预览:"
head -10 .pylintrc
else
echo "❌ 未找到 .pylintrc 配置文件"
exit 1
fi
- name: Run pylint
run: |
# 运行pylint检查主要的Python文件
echo "🚀 运行 Pylint 错误检查..."
# 检查主要目录 - 只关注错误,如果有错误则退出
echo "📂 检查 app/ 目录..."
pylint app/ --output-format=colorized --reports=yes --score=yes
# 检查根目录的Python文件
echo "📂 检查根目录 Python 文件..."
for file in $(find . -name "*.py" -not -path "./.*" -not -path "./.venv/*" -not -path "./build/*" -not -path "./dist/*" -not -path "./tests/*" -not -path "./docs/*" -not -path "./__pycache__/*" -maxdepth 1); do
echo "检查文件: $file"
pylint "$file" --output-format=colorized || exit 1
done
# 生成详细报告
echo "📊 生成 Pylint 详细报告..."
pylint app/ --output-format=json > pylint-report.json || true
# 显示评分(仅供参考)
echo "📈 Pylint 评分(仅供参考):"
pylint app/ --score=yes --reports=no | tail -2 || true
- name: Upload pylint report
uses: actions/upload-artifact@v3
if: always()
with:
name: pylint-report
path: pylint-report.json
- name: Summary
run: |
echo "🎉 Pylint 检查完成!"
echo "✅ 没有发现语法错误或严重问题"
echo "📊 详细报告已保存为构建工件"

6
.gitignore vendored
View File

@@ -23,4 +23,8 @@ config/cache/
*.pyc
*.log
.vscode
venv
venv
# Pylint
pylint-report.json
.pylint.d/

83
.pylintrc Normal file
View File

@@ -0,0 +1,83 @@
[MASTER]
# 指定Python路径
init-hook='import sys; sys.path.append(".")'
# 忽略的文件和目录
ignore=.git,__pycache__,.venv,build,dist,tests,docs
# 并行作业数量
jobs=0
[MESSAGES CONTROL]
# 只关注错误级别的问题,禁用警告、约定和重构建议
# E = Error (错误) - 会导致构建失败
# W = Warning (警告) - 仅显示,不会失败
# R = Refactor (重构建议) - 仅显示,不会失败
# C = Convention (约定) - 仅显示,不会失败
# I = Information (信息) - 仅显示,不会失败
# 禁用大部分警告、约定和重构建议,只保留错误和重要警告
disable=all
enable=error,
syntax-error,
undefined-variable,
used-before-assignment,
unreachable,
return-outside-function,
yield-outside-function,
continue-in-finally,
nonlocal-without-binding,
undefined-loop-variable,
redefined-builtin,
not-callable,
assignment-from-no-return,
no-value-for-parameter,
too-many-function-args,
unexpected-keyword-arg,
redundant-keyword-arg,
import-error,
relative-beyond-top-level
[REPORTS]
# 设置报告格式
output-format=colorized
reports=yes
score=yes
[FORMAT]
# 最大行长度
max-line-length=120
# 缩进大小
indent-string=' '
[DESIGN]
# 最大参数数量
max-args=10
# 最大本地变量数量
max-locals=20
# 最大分支数量
max-branches=15
# 最大语句数量
max-statements=50
# 最大父类数量
max-parents=7
# 最大属性数量
max-attributes=10
# 最小公共方法数量
min-public-methods=1
# 最大公共方法数量
max-public-methods=25
[SIMILARITIES]
# 最小相似行数
min-similarity-lines=6
# 忽略注释
ignore-comments=yes
# 忽略文档字符串
ignore-docstrings=yes
# 忽略导入
ignore-imports=yes
[TYPECHECK]
# 生成缺失成员提示的类列表
generated-members=requests.packages.urllib3

View File

@@ -40,54 +40,67 @@ class FetchMediasAction(BaseAction):
{
"func": RecommendChain().tmdb_trending,
"name": '流行趋势',
"api_path": "recommend/tmdb_trending"
},
{
"func": RecommendChain().douban_movie_showing,
"name": '正在热映',
"api_path": "recommend/douban_showing"
},
{
"func": RecommendChain().bangumi_calendar,
"name": 'Bangumi每日放送',
"api_path": "recommend/bangumi_calendar"
},
{
"func": RecommendChain().tmdb_movies,
"name": 'TMDB热门电影',
"api_path": "recommend/tmdb_movies"
},
{
"func": RecommendChain().tmdb_tvs,
"name": 'TMDB热门电视剧',
"api_path": "recommend/tmdb_tvs?with_original_language=zh|en|ja|ko"
},
{
"func": RecommendChain().douban_movie_hot,
"name": '豆瓣热门电影',
"api_path": "recommend/douban_movie_hot"
},
{
"func": RecommendChain().douban_tv_hot,
"name": '豆瓣热门电视剧',
"api_path": "recommend/douban_tv_hot"
},
{
"func": RecommendChain().douban_tv_animation,
"name": '豆瓣热门动漫',
"api_path": "recommend/douban_tv_animation"
},
{
"func": RecommendChain().douban_movies,
"name": '豆瓣最新电影',
"api_path": "recommend/douban_movies"
},
{
"func": RecommendChain().douban_tvs,
"name": '豆瓣最新电视剧',
"api_path": "recommend/douban_tvs"
},
{
"func": RecommendChain().douban_movie_top250,
"name": '豆瓣电影TOP250',
"api_path": "recommend/douban_movie_top250"
},
{
"func": RecommendChain().douban_tv_weekly_chinese,
"name": '豆瓣国产剧集榜',
"api_path": "recommend/douban_tv_weekly_chinese"
},
{
"func": RecommendChain().douban_tv_weekly_global,
"name": '豆瓣全球剧集榜',
"api_path": "recommend/douban_tv_weekly_global"
}
]
@@ -124,7 +137,7 @@ class FetchMediasAction(BaseAction):
获取数据源
"""
for s in self.__inner_sources:
if s['name'] == source:
if s['api_path'] == source:
return s
return None
@@ -135,13 +148,14 @@ class FetchMediasAction(BaseAction):
params = FetchMediasParams(**params)
try:
if params.source_type == "ranking":
for name in params.sources:
for api_path in params.sources:
if global_vars.is_workflow_stopped(workflow_id):
break
source = self.__get_source(name)
source = self.__get_source(api_path)
if not source:
continue
logger.info(f"获取媒体数据 {source} ...")
name = source.get("name")
results = []
if source.get("func"):
results = source['func']()

View File

@@ -2,7 +2,7 @@ from fastapi import APIRouter
from app.api.endpoints import login, user, site, message, webhook, subscribe, \
media, douban, search, plugin, tmdb, history, system, download, dashboard, \
transfer, mediaserver, bangumi, storage, discover, recommend, workflow
transfer, mediaserver, bangumi, storage, discover, recommend, workflow, torrent
api_router = APIRouter()
api_router.include_router(login.router, prefix="/login", tags=["login"])
@@ -27,3 +27,4 @@ api_router.include_router(bangumi.router, prefix="/bangumi", tags=["bangumi"])
api_router.include_router(discover.router, prefix="/discover", tags=["discover"])
api_router.include_router(recommend.router, prefix="/recommend", tags=["recommend"])
api_router.include_router(workflow.router, prefix="/workflow", tags=["workflow"])
api_router.include_router(torrent.router, prefix="/torrent", tags=["torrent"])

View File

@@ -5,9 +5,7 @@ from fastapi import APIRouter, Depends, Form, HTTPException
from fastapi.security import OAuth2PasswordRequestForm
from app import schemas
from app.chain.tmdb import TmdbChain
from app.chain.user import UserChain
from app.chain.mediaserver import MediaServerChain
from app.core import security
from app.core.config import settings
from app.helper.sites import SitesHelper
@@ -54,14 +52,7 @@ def wallpaper() -> Any:
"""
获取登录页面电影海报
"""
if settings.WALLPAPER == "bing":
url = WallpaperHelper().get_bing_wallpaper()
elif settings.WALLPAPER == "mediaserver":
url = MediaServerChain().get_latest_wallpaper()
elif settings.WALLPAPER == "customize":
url = WallpaperHelper().get_customize_wallpaper()
else:
url = TmdbChain().get_random_wallpager()
url = WallpaperHelper().get_wallpaper()
if url:
return schemas.Response(
success=True,
@@ -75,13 +66,4 @@ def wallpapers() -> Any:
"""
获取登录页面电影海报
"""
if settings.WALLPAPER == "bing":
return WallpaperHelper().get_bing_wallpapers()
elif settings.WALLPAPER == "mediaserver":
return MediaServerChain().get_latest_wallpapers()
elif settings.WALLPAPER == "tmdb":
return TmdbChain().get_trending_wallpapers()
elif settings.WALLPAPER == "customize":
return WallpaperHelper().get_customize_wallpapers()
else:
return []
return WallpaperHelper().get_wallpapers()

View File

@@ -149,11 +149,12 @@ def seasons(tmdbid: int, _: schemas.TokenPayload = Depends(verify_token)) -> Any
"""
查询媒体剧集组列表themoviedb
"""
mediainfo = MediaChain().recognize_media(tmdbid=tmdbid, mtype=MediaType.TV)
mediainfo = MediaChain().recognize_media(tmdbid=tmdbid, mtype=MediaType.TV)
if not mediainfo:
return []
return mediainfo.episode_groups
@router.get("/seasons", summary="查询媒体季信息", response_model=List[schemas.MediaSeason])
def seasons(mediaid: Optional[str] = None,
title: Optional[str] = None,
@@ -219,14 +220,13 @@ def detail(mediaid: str, type_name: str, title: Optional[str] = None, year: str
)
event = eventmanager.send_event(ChainEventType.MediaRecognizeConvert, event_data)
# 使用事件返回的上下文数据
if event and event.event_data:
if event and event.event_data and event.event_data.media_dict:
event_data: MediaRecognizeConvertEventData = event.event_data
if event_data.media_dict:
new_id = event_data.media_dict.get("id")
if event_data.convert_type == "themoviedb":
mediainfo = MediaChain().recognize_media(tmdbid=new_id, mtype=mtype)
elif event_data.convert_type == "douban":
mediainfo = MediaChain().recognize_media(doubanid=new_id, mtype=mtype)
new_id = event_data.media_dict.get("id")
if event_data.convert_type == "themoviedb":
mediainfo = MediaChain().recognize_media(tmdbid=new_id, mtype=mtype)
elif event_data.convert_type == "douban":
mediainfo = MediaChain().recognize_media(doubanid=new_id, mtype=mtype)
elif title:
# 使用名称识别兜底
meta = MetaInfo(title)

View File

@@ -121,7 +121,7 @@ def not_exists(media_in: schemas.MediaInfo,
@router.get("/latest", summary="最新入库条目", response_model=List[schemas.MediaServerPlayItem])
def latest(server: str, count: Optional[int] = 18,
def latest(server: str, count: Optional[int] = 20,
userinfo: schemas.TokenPayload = Depends(verify_token)) -> Any:
"""
获取媒体服务器最新入库条目

View File

@@ -1,4 +1,5 @@
import mimetypes
import shutil
from typing import Annotated, Any, List, Optional
from fastapi import APIRouter, Depends, Header, HTTPException
@@ -144,11 +145,11 @@ def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
local_plugins = PluginManager().get_local_plugins()
# 已安装插件
installed_plugins = [plugin for plugin in local_plugins if plugin.installed]
# 未安装的本地插件
not_installed_plugins = [plugin for plugin in local_plugins if not plugin.installed]
if state == "installed":
return installed_plugins
# 未安装的本地插件
not_installed_plugins = [plugin for plugin in local_plugins if not plugin.installed]
# 在线插件
online_plugins = PluginManager().get_online_plugins()
if not online_plugins:
@@ -177,6 +178,7 @@ def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
if state == "market":
# 返回未安装的插件
return market_plugins
# 返回所有插件
return installed_plugins + market_plugins
@@ -330,10 +332,11 @@ def reset_plugin(plugin_id: str,
"""
根据插件ID重置插件配置及数据
"""
plugin_manager = PluginManager()
# 删除配置
PluginManager().delete_plugin_config(plugin_id)
plugin_manager.delete_plugin_config(plugin_id)
# 删除插件所有数据
PluginManager().delete_plugin_data(plugin_id)
plugin_manager.delete_plugin_data(plugin_id)
# 重新加载插件
reload_plugin(plugin_id)
return schemas.Response(success=True)
@@ -374,6 +377,71 @@ def plugin_static_file(plugin_id: str, filepath: str):
raise HTTPException(status_code=500, detail="Internal Server Error")
@router.get("/folders", summary="获取插件文件夹配置", response_model=dict)
def get_plugin_folders(_: schemas.TokenPayload = Depends(get_current_active_superuser)) -> dict:
"""
获取插件文件夹分组配置
"""
try:
result = SystemConfigOper().get(SystemConfigKey.PluginFolders) or {}
return result
except Exception as e:
logger.error(f"[文件夹API] 获取文件夹配置失败: {str(e)}")
return {}
@router.post("/folders", summary="保存插件文件夹配置", response_model=schemas.Response)
def save_plugin_folders(folders: dict, _: schemas.TokenPayload = Depends(get_current_active_superuser)) -> Any:
"""
保存插件文件夹分组配置
"""
try:
SystemConfigOper().set(SystemConfigKey.PluginFolders, folders)
return schemas.Response(success=True)
except Exception as e:
logger.error(f"[文件夹API] 保存文件夹配置失败: {str(e)}")
return schemas.Response(success=False, message=str(e))
@router.post("/folders/{folder_name}", summary="创建插件文件夹", response_model=schemas.Response)
def create_plugin_folder(folder_name: str, _: schemas.TokenPayload = Depends(get_current_active_superuser)) -> Any:
"""
创建新的插件文件夹
"""
folders = SystemConfigOper().get(SystemConfigKey.PluginFolders) or {}
if folder_name not in folders:
folders[folder_name] = []
SystemConfigOper().set(SystemConfigKey.PluginFolders, folders)
return schemas.Response(success=True, message=f"文件夹 '{folder_name}' 创建成功")
else:
return schemas.Response(success=False, message=f"文件夹 '{folder_name}' 已存在")
@router.delete("/folders/{folder_name}", summary="删除插件文件夹", response_model=schemas.Response)
def delete_plugin_folder(folder_name: str, _: schemas.TokenPayload = Depends(get_current_active_superuser)) -> Any:
"""
删除插件文件夹
"""
folders = SystemConfigOper().get(SystemConfigKey.PluginFolders) or {}
if folder_name in folders:
del folders[folder_name]
SystemConfigOper().set(SystemConfigKey.PluginFolders, folders)
return schemas.Response(success=True, message=f"文件夹 '{folder_name}' 删除成功")
else:
return schemas.Response(success=False, message=f"文件夹 '{folder_name}' 不存在")
@router.put("/folders/{folder_name}/plugins", summary="更新文件夹中的插件", response_model=schemas.Response)
def update_folder_plugins(folder_name: str, plugin_ids: List[str], _: schemas.TokenPayload = Depends(get_current_active_superuser)) -> Any:
"""
更新指定文件夹中的插件列表
"""
folders = SystemConfigOper().get(SystemConfigKey.PluginFolders) or {}
folders[folder_name] = plugin_ids
SystemConfigOper().set(SystemConfigKey.PluginFolders, folders)
return schemas.Response(success=True, message=f"文件夹 '{folder_name}' 中的插件已更新")
@router.get("/{plugin_id}", summary="获取插件配置")
def plugin_config(plugin_id: str,
_: schemas.TokenPayload = Depends(get_current_active_superuser)) -> dict:
@@ -389,10 +457,11 @@ def set_plugin_config(plugin_id: str, conf: dict,
"""
更新插件配置
"""
plugin_manager = PluginManager()
# 保存配置
PluginManager().save_plugin_config(plugin_id, conf)
plugin_manager.save_plugin_config(plugin_id, conf)
# 重新生效插件
PluginManager().init_plugin(plugin_id, conf)
plugin_manager.init_plugin(plugin_id, conf)
# 注册插件服务
register_plugin(plugin_id)
return schemas.Response(success=True)
@@ -404,18 +473,153 @@ def uninstall_plugin(plugin_id: str,
"""
卸载插件
"""
config_oper = SystemConfigOper()
# 删除已安装信息
install_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
install_plugins = config_oper.get(SystemConfigKey.UserInstalledPlugins) or []
for plugin in install_plugins:
if plugin == plugin_id:
install_plugins.remove(plugin)
break
# 保存
SystemConfigOper().set(SystemConfigKey.UserInstalledPlugins, install_plugins)
config_oper.set(SystemConfigKey.UserInstalledPlugins, install_plugins)
# 移除插件API
remove_plugin_api(plugin_id)
# 移除插件服务
Scheduler().remove_plugin_job(plugin_id)
# 判断是否为分身
plugin_manager = PluginManager()
plugin_class = plugin_manager.plugins.get(plugin_id)
if getattr(plugin_class, "is_clone", False):
# 如果是分身插件,则删除分身数据和配置
plugin_manager.delete_plugin_config(plugin_id)
plugin_manager.delete_plugin_data(plugin_id)
# 删除分身文件
plugin_base_dir = settings.ROOT_PATH / "app" / "plugins" / plugin_id.lower()
if plugin_base_dir.exists():
try:
shutil.rmtree(plugin_base_dir)
plugin_manager.plugins.pop(plugin_id, None)
except Exception as e:
logger.error(f"删除插件分身目录 {plugin_base_dir} 失败: {str(e)}")
# 从插件文件夹中移除该插件
_remove_plugin_from_folders(plugin_id)
# 移除插件
PluginManager().remove_plugin(plugin_id)
plugin_manager.remove_plugin(plugin_id)
return schemas.Response(success=True)
@router.post("/clone/{plugin_id}", summary="创建插件分身", response_model=schemas.Response)
def clone_plugin(plugin_id: str,
clone_data: dict,
_: schemas.TokenPayload = Depends(get_current_active_superuser)) -> Any:
"""
创建插件分身
"""
try:
success, message = PluginManager().clone_plugin(
plugin_id=plugin_id,
suffix=clone_data.get("suffix", ""),
name=clone_data.get("name", ""),
description=clone_data.get("description", ""),
version=clone_data.get("version", ""),
icon=clone_data.get("icon", "")
)
if success:
# 注册插件服务
reload_plugin(message)
# 将分身插件添加到原插件所在的文件夹中
_add_clone_to_plugin_folder(plugin_id, message)
return schemas.Response(success=True, message="插件分身创建成功")
else:
return schemas.Response(success=False, message=message)
except Exception as e:
logger.error(f"创建插件分身失败:{str(e)}")
return schemas.Response(success=False, message=f"创建插件分身失败:{str(e)}")
def _add_clone_to_plugin_folder(original_plugin_id: str, clone_plugin_id: str):
"""
将分身插件添加到原插件所在的文件夹中
:param original_plugin_id: 原插件ID
:param clone_plugin_id: 分身插件ID
"""
try:
config_oper = SystemConfigOper()
# 获取插件文件夹配置
folders = config_oper.get(SystemConfigKey.PluginFolders) or {}
# 查找原插件所在的文件夹
target_folder = None
for folder_name, folder_data in folders.items():
if isinstance(folder_data, dict) and 'plugins' in folder_data:
# 新格式:{"plugins": [...], "order": ..., "icon": ...}
if original_plugin_id in folder_data['plugins']:
target_folder = folder_name
break
elif isinstance(folder_data, list):
# 旧格式:直接是插件列表
if original_plugin_id in folder_data:
target_folder = folder_name
break
# 如果找到了原插件所在的文件夹,则将分身插件也添加到该文件夹中
if target_folder:
folder_data = folders[target_folder]
if isinstance(folder_data, dict) and 'plugins' in folder_data:
# 新格式
if clone_plugin_id not in folder_data['plugins']:
folder_data['plugins'].append(clone_plugin_id)
logger.info(f"已将分身插件 {clone_plugin_id} 添加到文件夹 '{target_folder}'")
elif isinstance(folder_data, list):
# 旧格式
if clone_plugin_id not in folder_data:
folder_data.append(clone_plugin_id)
logger.info(f"已将分身插件 {clone_plugin_id} 添加到文件夹 '{target_folder}'")
# 保存更新后的文件夹配置
config_oper.set(SystemConfigKey.PluginFolders, folders)
else:
logger.info(f"原插件 {original_plugin_id} 不在任何文件夹中,分身插件 {clone_plugin_id} 将保持独立")
except Exception as e:
logger.error(f"处理插件文件夹时出错:{str(e)}")
# 文件夹处理失败不影响插件分身创建的整体流程
def _remove_plugin_from_folders(plugin_id: str):
"""
从所有文件夹中移除指定的插件
:param plugin_id: 要移除的插件ID
"""
try:
config_oper = SystemConfigOper()
# 获取插件文件夹配置
folders = config_oper.get(SystemConfigKey.PluginFolders) or {}
# 标记是否有修改
modified = False
# 遍历所有文件夹,移除指定插件
for folder_name, folder_data in folders.items():
if isinstance(folder_data, dict) and 'plugins' in folder_data:
# 新格式:{"plugins": [...], "order": ..., "icon": ...}
if plugin_id in folder_data['plugins']:
folder_data['plugins'].remove(plugin_id)
logger.info(f"已从文件夹 '{folder_name}' 中移除插件 {plugin_id}")
modified = True
elif isinstance(folder_data, list):
# 旧格式:直接是插件列表
if plugin_id in folder_data:
folder_data.remove(plugin_id)
logger.info(f"已从文件夹 '{folder_name}' 中移除插件 {plugin_id}")
modified = True
# 如果有修改,保存更新后的文件夹配置
if modified:
config_oper.set(SystemConfigKey.PluginFolders, folders)
else:
logger.debug(f"插件 {plugin_id} 不在任何文件夹中,无需移除")
except Exception as e:
logger.error(f"从文件夹中移除插件时出错:{str(e)}")
# 文件夹处理失败不影响插件卸载的整体流程

View File

@@ -5,6 +5,7 @@ from sqlalchemy.orm import Session
from starlette.background import BackgroundTasks
from app import schemas
from app.api.endpoints.plugin import register_plugin_api
from app.chain.site import SiteChain
from app.chain.torrents import TorrentsChain
from app.command import Command
@@ -17,13 +18,13 @@ from app.db.models.site import Site
from app.db.models.siteicon import SiteIcon
from app.db.models.sitestatistic import SiteStatistic
from app.db.models.siteuserdata import SiteUserData
from app.db.site_oper import SiteOper
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_superuser
from app.helper.sites import SitesHelper
from app.scheduler import Scheduler
from app.schemas.types import SystemConfigKey, EventType
from app.utils.string import StringUtils
from startup.plugins_initializer import register_plugin_api
router = APIRouter()
@@ -395,6 +396,21 @@ def auth_site(
return schemas.Response(success=status, message=msg)
@router.get("/mapping", summary="获取站点域名到名称的映射", response_model=schemas.Response)
def site_mapping(_: User = Depends(get_current_active_superuser)):
"""
获取站点域名到名称的映射关系
"""
try:
sites = SiteOper().list()
mapping = {}
for site in sites:
mapping[site.domain] = site.name
return schemas.Response(success=True, data=mapping)
except Exception as e:
return schemas.Response(success=False, message=f"获取映射失败:{str(e)}")
@router.get("/{site_id}", summary="站点详情", response_model=schemas.Site)
def read_site(
site_id: int,

View File

@@ -162,47 +162,50 @@ def rename(fileitem: schemas.FileItem,
"""
if not new_name:
return schemas.Response(success=False, message="新名称为空")
# 重命名目录内文件
if recursive:
transferchain = TransferChain()
media_exts = settings.RMT_MEDIAEXT + settings.RMT_SUBEXT + settings.RMT_AUDIO_TRACK_EXT
# 递归修改目录内文件(智能识别命名)
sub_files: List[schemas.FileItem] = StorageChain().list_files(fileitem)
if sub_files:
# 开始进度
progress = ProgressHelper()
progress.start(ProgressKey.BatchRename)
total = len(sub_files)
handled = 0
for sub_file in sub_files:
handled += 1
progress.update(value=handled / total * 100,
text=f"正在处理 {sub_file.name} ...",
key=ProgressKey.BatchRename)
if sub_file.type == "dir":
continue
if not sub_file.extension:
continue
if f".{sub_file.extension.lower()}" not in media_exts:
continue
sub_path = Path(f"{fileitem.path}{sub_file.name}")
meta = MetaInfoPath(sub_path)
mediainfo = transferchain.recognize_media(meta)
if not mediainfo:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 未识别到媒体信息")
new_path = transferchain.recommend_name(meta=meta, mediainfo=mediainfo)
if not new_path:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 未识别到新名称")
ret: schemas.Response = rename(fileitem=sub_file,
new_name=Path(new_path).name,
recursive=False)
if not ret.success:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 重命名失败!")
progress.end(ProgressKey.BatchRename)
# 重命名自己
result = StorageChain().rename_file(fileitem, new_name)
if result:
if recursive:
transferchain = TransferChain()
media_exts = settings.RMT_MEDIAEXT + settings.RMT_SUBEXT + settings.RMT_AUDIO_TRACK_EXT
# 递归修改目录内文件(智能识别命名)
sub_files: List[schemas.FileItem] = StorageChain().list_files(fileitem)
if sub_files:
# 开始进度
progress = ProgressHelper()
progress.start(ProgressKey.BatchRename)
total = len(sub_files)
handled = 0
for sub_file in sub_files:
handled += 1
progress.update(value=handled / total * 100,
text=f"正在处理 {sub_file.name} ...",
key=ProgressKey.BatchRename)
if sub_file.type == "dir":
continue
if not sub_file.extension:
continue
if f".{sub_file.extension.lower()}" not in media_exts:
continue
sub_path = Path(f"{fileitem.path}{sub_file.name}")
meta = MetaInfoPath(sub_path)
mediainfo = transferchain.recognize_media(meta)
if not mediainfo:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 未识别到媒体信息")
new_path = transferchain.recommend_name(meta=meta, mediainfo=mediainfo)
if not new_path:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 未识别到新名称")
ret: schemas.Response = rename(fileitem=sub_file,
new_name=Path(new_path).name,
recursive=False)
if not ret.success:
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=False, message=f"{sub_path.name} 重命名失败!")
progress.end(ProgressKey.BatchRename)
return schemas.Response(success=True)
return schemas.Response(success=False)

View File

@@ -20,23 +20,24 @@ from app.core.config import global_vars, settings
from app.core.metainfo import MetaInfo
from app.core.module import ModuleManager
from app.core.security import verify_apitoken, verify_resource_token, verify_token
from app.core.event import eventmanager
from app.db.models import User
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_superuser
from app.helper.mediaserver import MediaServerHelper
from app.helper.message import MessageHelper, MessageQueueManager
from app.helper.message import MessageHelper
from app.helper.progress import ProgressHelper
from app.helper.rule import RuleHelper
from app.helper.sites import SitesHelper
from app.helper.subscribe import SubscribeHelper
from app.helper.system import SystemHelper
from app.log import logger
from app.monitor import Monitor
from app.scheduler import Scheduler
from app.schemas.types import SystemConfigKey
from app.schemas import ConfigChangeEventData
from app.schemas.types import SystemConfigKey, EventType
from app.utils.crypto import HashUtils
from app.utils.http import RequestUtils
from app.utils.security import SecurityUtils
from app.utils.system import SystemUtils
from app.utils.url import UrlUtils
from version import APP_VERSION
@@ -219,18 +220,27 @@ def set_env_setting(env: dict,
result = settings.update_settings(env=env)
# 统计成功和失败的结果
success_updates = {k: v for k, v in result.items() if v[0]}
failed_updates = {k: v for k, v in result.items() if not v[0]}
failed_updates = {k: v for k, v in result.items() if v[0] is False}
if failed_updates:
return schemas.Response(
success=False,
message="部分配置项更新失败",
message=f"{', '.join(failed_updates.keys())} 配置项更新失败",
data={
"success_updates": success_updates,
"failed_updates": failed_updates
}
)
if success_updates:
for key in success_updates.keys():
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=getattr(settings, key, None),
change_type="update"
))
return schemas.Response(
success=True,
message="所有配置项更新成功",
@@ -284,12 +294,28 @@ def set_setting(key: str, value: Union[list, dict, bool, int, str] = None,
"""
if hasattr(settings, key):
success, message = settings.update_setting(key=key, value=value)
if success:
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=value,
change_type="update"
))
elif success is None:
success = True
return schemas.Response(success=success, message=message)
elif key in {item.value for item in SystemConfigKey}:
if isinstance(value, list):
value = list(filter(None, value))
value = value if value else None
SystemConfigOper().set(key, value)
success = SystemConfigOper().set(key, value)
if success:
# 发送配置变更事件
eventmanager.send_event(etype=EventType.ConfigChanged, data=ConfigChangeEventData(
key=key,
value=value,
change_type="update"
))
return schemas.Response(success=True)
else:
return schemas.Response(success=False, message=f"配置项 '{key}' 不存在")
@@ -474,27 +500,15 @@ def restart_system(_: User = Depends(get_current_active_superuser)):
"""
重启系统(仅管理员)
"""
if not SystemUtils.can_restart():
if not SystemHelper.can_restart():
return schemas.Response(success=False, message="当前运行环境不支持重启操作!")
# 标识停止事件
global_vars.stop_system()
# 执行重启
ret, msg = SystemUtils.restart()
ret, msg = SystemHelper.restart()
return schemas.Response(success=ret, message=msg)
@router.get("/reload", summary="重新加载模块", response_model=schemas.Response)
def reload_module(_: User = Depends(get_current_active_superuser)):
"""
重新加载模块(仅管理员)
"""
MessageQueueManager().init_config()
ModuleManager().reload()
Scheduler().init()
Monitor().init()
return schemas.Response(success=True)
@router.get("/runscheduler", summary="运行服务", response_model=schemas.Response)
def run_scheduler(jobid: str,
_: User = Depends(get_current_active_superuser)):

View File

@@ -0,0 +1,199 @@
from typing import Optional
from fastapi import APIRouter, Depends
from app import schemas
from app.chain.media import MediaChain
from app.chain.torrents import TorrentsChain
from app.core.config import settings
from app.core.context import MediaInfo
from app.core.metainfo import MetaInfo
from app.db.models import User
from app.db.user_oper import get_current_active_superuser
from app.utils.crypto import HashUtils
router = APIRouter()
@router.get("/cache", summary="获取种子缓存", response_model=schemas.Response)
def torrents_cache(_: User = Depends(get_current_active_superuser)):
"""
获取当前种子缓存数据
"""
torrents_chain = TorrentsChain()
# 获取spider和rss两种缓存
if settings.SUBSCRIBE_MODE == "rss":
cache_info = torrents_chain.get_torrents("rss")
else:
cache_info = torrents_chain.get_torrents("spider")
# 统计信息
torrent_count = sum(len(torrents) for torrents in cache_info.values())
# 转换为前端需要的格式
torrent_data = []
for domain, contexts in cache_info.items():
for context in contexts:
torrent_hash = HashUtils.md5(f"{context.torrent_info.title}{context.torrent_info.description}")
torrent_data.append({
"hash": torrent_hash,
"domain": domain,
"title": context.torrent_info.title,
"description": context.torrent_info.description,
"size": context.torrent_info.size,
"pubdate": context.torrent_info.pubdate,
"site_name": context.torrent_info.site_name,
"media_name": context.media_info.title if context.media_info else "",
"media_year": context.media_info.year if context.media_info else "",
"media_type": context.media_info.type if context.media_info else "",
"season_episode": context.meta_info.season_episode if context.meta_info else "",
"resource_term": context.meta_info.resource_term if context.meta_info else "",
"enclosure": context.torrent_info.enclosure,
"page_url": context.torrent_info.page_url,
"poster_path": context.media_info.get_poster_image() if context.media_info else "",
"backdrop_path": context.media_info.get_backdrop_image() if context.media_info else ""
})
return schemas.Response(success=True, data={
"count": torrent_count,
"sites": len(cache_info),
"data": torrent_data
})
@router.delete("/cache/{domain}/{torrent_hash}", summary="删除指定种子缓存",
response_model=schemas.Response)
def delete_cache(domain: str, torrent_hash: str, _: User = Depends(get_current_active_superuser)):
"""
删除指定的种子缓存
:param domain: 站点域名
:param torrent_hash: 种子hash使用title+description的md5
:param _: 当前用户,必须是超级用户
"""
torrents_chain = TorrentsChain()
try:
# 获取当前缓存
cache_data = torrents_chain.get_torrents()
if domain not in cache_data:
return schemas.Response(success=False, message=f"站点 {domain} 缓存不存在")
# 查找并删除指定种子
original_count = len(cache_data[domain])
cache_data[domain] = [
context for context in cache_data[domain]
if HashUtils.md5(f"{context.torrent_info.title}{context.torrent_info.description}") != torrent_hash
]
if len(cache_data[domain]) == original_count:
return schemas.Response(success=False, message="未找到指定的种子")
# 保存更新后的缓存
torrents_chain.save_cache(cache_data, torrents_chain.cache_file)
return schemas.Response(success=True, message="种子删除成功")
except Exception as e:
return schemas.Response(success=False, message=f"删除失败:{str(e)}")
@router.delete("/cache", summary="清理种子缓存", response_model=schemas.Response)
def clear_cache(_: User = Depends(get_current_active_superuser)):
"""
清理所有种子缓存
"""
torrents_chain = TorrentsChain()
try:
torrents_chain.clear_torrents()
return schemas.Response(success=True, message="种子缓存清理完成")
except Exception as e:
return schemas.Response(success=False, message=f"清理失败:{str(e)}")
@router.post("/cache/refresh", summary="刷新种子缓存", response_model=schemas.Response)
def refresh_cache(_: User = Depends(get_current_active_superuser)):
"""
刷新种子缓存
"""
from app.chain.torrents import TorrentsChain
torrents_chain = TorrentsChain()
try:
result = torrents_chain.refresh()
# 统计刷新结果
total_count = sum(len(torrents) for torrents in result.values())
sites_count = len(result)
return schemas.Response(success=True, message=f"缓存刷新完成,共刷新 {sites_count} 个站点,{total_count} 个种子")
except Exception as e:
return schemas.Response(success=False, message=f"刷新失败:{str(e)}")
@router.post("/cache/reidentify/{domain}/{torrent_hash}", summary="重新识别种子", response_model=schemas.Response)
def reidentify_cache(domain: str, torrent_hash: str,
tmdbid: Optional[int] = None, doubanid: Optional[str] = None,
_: User = Depends(get_current_active_superuser)):
"""
重新识别指定的种子
:param domain: 站点域名
:param torrent_hash: 种子hash使用title+description的md5
:param tmdbid: 手动指定的TMDB ID
:param doubanid: 手动指定的豆瓣ID
:param _: 当前用户,必须是超级用户
"""
torrents_chain = TorrentsChain()
media_chain = MediaChain()
try:
# 获取当前缓存
cache_data = torrents_chain.get_torrents()
if domain not in cache_data:
return schemas.Response(success=False, message=f"站点 {domain} 缓存不存在")
# 查找指定种子
target_context = None
for context in cache_data[domain]:
if HashUtils.md5(f"{context.torrent_info.title}{context.torrent_info.description}") == torrent_hash:
target_context = context
break
if not target_context:
return schemas.Response(success=False, message="未找到指定的种子")
# 重新识别
meta = MetaInfo(title=target_context.torrent_info.title,
subtitle=target_context.torrent_info.description)
if tmdbid or doubanid:
# 手动指定媒体信息
mediainfo = MediaChain().recognize_media(meta=meta, tmdbid=tmdbid, doubanid=doubanid)
else:
# 自动重新识别
mediainfo = media_chain.recognize_by_meta(meta)
if not mediainfo:
# 创建空的媒体信息
mediainfo = MediaInfo()
else:
# 清理多余数据
mediainfo.clear()
# 更新上下文中的媒体信息
target_context.media_info = mediainfo
# 保存更新后的缓存
torrents_chain.save_cache(cache_data, TorrentsChain().cache_file)
return schemas.Response(success=True, message="重新识别完成", data={
"media_name": mediainfo.title if mediainfo else "",
"media_year": mediainfo.year if mediainfo else "",
"media_type": mediainfo.type.value if mediainfo and mediainfo.type else ""
})
except Exception as e:
return schemas.Response(success=False, message=f"重新识别失败:{str(e)}")

View File

@@ -5,6 +5,7 @@ from sqlalchemy.orm import Session
from app import schemas
from app.chain.media import MediaChain
from app.chain.tvdb import TvdbChain
from app.chain.subscribe import SubscribeChain
from app.core.metainfo import MetaInfo
from app.core.security import verify_apikey
@@ -520,87 +521,87 @@ def arr_series_lookup(term: str, _: Annotated[str, Depends(verify_apikey)], db:
"""
# 季信息
seas: List[int] = []
# tvdbid 列表
tvdbids: List[int] = []
# 获取TVDBID
if not term.startswith("tvdb:"):
mediainfo = MediaChain().recognize_media(meta=MetaInfo(term),
mtype=MediaType.TV)
if not mediainfo:
return [SonarrSeries()]
# 季信息
if mediainfo.seasons:
seas = list(mediainfo.seasons)
title = term.replace("+", " ")
tvdbids = TvdbChain().get_tvdbid_by_name(title=title)
else:
tvdbid = int(term.replace("tvdb:", ""))
tvdbids.append(tvdbid)
sonarr_series_list = []
for tvdbid in tvdbids:
# 查询TVDB信息
tvdbinfo = MediaChain().tvdb_info(tvdbid=tvdbid)
if not tvdbinfo:
return [SonarrSeries()]
continue
# 季信息
sea_num = tvdbinfo.get('season')
# 季信息(只取默认季类型,排除特别季)
sea_num = len([season for season in tvdbinfo.get('seasons') if
season['type']['id'] == tvdbinfo.get('defaultSeasonType') and season['number'] > 0])
if sea_num:
seas = list(range(1, int(sea_num) + 1))
# 根据TVDB查询媒体信息
mediainfo = MediaChain().recognize_media(meta=MetaInfo(tvdbinfo.get('seriesName')),
mediainfo = MediaChain().recognize_media(meta=MetaInfo(tvdbinfo.get('name')),
mtype=MediaType.TV)
if not mediainfo:
continue
# 查询是否存在
exists = MediaChain().media_exists(mediainfo)
if exists:
hasfile = True
else:
hasfile = False
# 查询是否存在
exists = MediaChain().media_exists(mediainfo)
if exists:
hasfile = True
else:
hasfile = False
# 查询订阅信息
seasons: List[dict] = []
subscribes = Subscribe.get_by_tmdbid(db, mediainfo.tmdb_id)
if subscribes:
# 已监控
monitored = True
# 已监控季
sub_seas = [sub.season for sub in subscribes]
for sea in seas:
if sea in sub_seas:
seasons.append({
"seasonNumber": sea,
"monitored": True,
})
else:
# 查询订阅信息
seasons: List[dict] = []
subscribes = Subscribe.get_by_tmdbid(db, mediainfo.tmdb_id)
if subscribes:
# 已监控
monitored = True
# 已监控季
sub_seas = [sub.season for sub in subscribes]
for sea in seas:
if sea in sub_seas:
seasons.append({
"seasonNumber": sea,
"monitored": True,
})
else:
seasons.append({
"seasonNumber": sea,
"monitored": False,
})
subid = subscribes[-1].id
else:
subid = None
monitored = False
for sea in seas:
seasons.append({
"seasonNumber": sea,
"monitored": False,
})
subid = subscribes[-1].id
else:
subid = None
monitored = False
for sea in seas:
seasons.append({
"seasonNumber": sea,
"monitored": False,
})
sonarr_series = SonarrSeries(
id=subid,
title=mediainfo.title,
seasonCount=len(seasons),
seasons=seasons,
remotePoster=mediainfo.get_poster_image(),
year=mediainfo.year,
tmdbId=mediainfo.tmdb_id,
tvdbId=tvdbid,
imdbId=mediainfo.imdb_id,
profileId=1,
languageProfileId=1,
monitored=monitored,
hasFile=hasfile,
)
sonarr_series_list.append(sonarr_series)
return [SonarrSeries(
id=subid,
title=mediainfo.title,
seasonCount=len(seasons),
seasons=seasons,
remotePoster=mediainfo.get_poster_image(),
year=mediainfo.year,
tmdbId=mediainfo.tmdb_id,
tvdbId=mediainfo.tvdb_id,
imdbId=mediainfo.imdb_id,
profileId=1,
languageProfileId=1,
qualityProfileId=1,
isAvailable=True,
monitored=monitored,
hasFile=hasfile
)]
return sonarr_series_list if sonarr_series_list else [SonarrSeries()]
@arr_router.get("/series/{tid}", summary="剧集详情")

View File

@@ -43,7 +43,6 @@ class ChainBase(metaclass=ABCMeta):
self.messagequeue = MessageQueueManager(
send_callback=self.run_module
)
self.useroper = UserOper()
self.pluginmanager = PluginManager()
@staticmethod
@@ -67,7 +66,7 @@ class ChainBase(metaclass=ABCMeta):
"""
try:
with open(settings.TEMP_PATH / filename, 'wb') as f:
pickle.dump(cache, f) # noqa
pickle.dump(cache, f) # noqa
except Exception as err:
logger.error(f"保存缓存 {filename} 出错:{str(err)}")
finally:
@@ -374,7 +373,7 @@ class ChainBase(metaclass=ABCMeta):
return self.run_module("search_torrents", site=site, keywords=keywords,
mtype=mtype, page=page)
def refresh_torrents(self, site: dict, keyword: Optional[str] = None,
def refresh_torrents(self, site: dict, keyword: Optional[str] = None,
cat: Optional[str] = None, page: Optional[int] = 0) -> List[TorrentInfo]:
"""
获取站点最新一页的种子,多个站点需要多线程处理
@@ -543,12 +542,12 @@ class ChainBase(metaclass=ABCMeta):
return self.run_module("media_files", mediainfo=mediainfo)
def post_message(self,
message: Optional[Notification] = None,
meta: Optional[MetaBase] = None,
mediainfo: Optional[MediaInfo] = None,
torrentinfo: Optional[TorrentInfo] = None,
transferinfo: Optional[TransferInfo] = None,
**kwargs) -> None:
message: Optional[Notification] = None,
meta: Optional[MetaBase] = None,
mediainfo: Optional[MediaInfo] = None,
torrentinfo: Optional[TorrentInfo] = None,
transferinfo: Optional[TransferInfo] = None,
**kwargs) -> None:
"""
发送消息
:param message: Notification实例
@@ -561,7 +560,7 @@ class ChainBase(metaclass=ABCMeta):
"""
# 渲染消息
message = MessageTemplateHelper.render(message=message, meta=meta, mediainfo=mediainfo,
torrentinfo=torrentinfo, transferinfo=transferinfo, **kwargs)
torrentinfo=torrentinfo, transferinfo=transferinfo, **kwargs)
# 保存消息
self.messagehelper.put(message, role="user", title=message.title)
self.messageoper.add(**message.dict())
@@ -575,26 +574,27 @@ class ChainBase(metaclass=ABCMeta):
# 是否已发送管理员标志
admin_sended = False
send_orignal = False
useroper = UserOper()
for action in actions:
send_message = copy.deepcopy(message)
if action == "admin" and not admin_sended:
# 仅发送管理员
logger.info(f"{send_message.mtype} 的消息已设置发送给管理员")
# 读取管理员消息IDS
send_message.targets = self.useroper.get_settings(settings.SUPERUSER)
send_message.targets = useroper.get_settings(settings.SUPERUSER)
admin_sended = True
elif action == "user" and send_message.username:
# 发送对应用户
logger.info(f"{send_message.mtype} 的消息已设置发送给用户 {send_message.username}")
# 读取用户消息IDS
send_message.targets = self.useroper.get_settings(send_message.username)
send_message.targets = useroper.get_settings(send_message.username)
if send_message.targets is None:
# 没有找到用户
if not admin_sended:
# 回滚发送管理员
logger.info(f"用户 {send_message.username} 不存在,消息将发送给管理员")
# 读取管理员消息IDS
send_message.targets = self.useroper.get_settings(settings.SUPERUSER)
send_message.targets = useroper.get_settings(settings.SUPERUSER)
admin_sended = True
else:
# 管理员发过了,此消息不发了
@@ -643,7 +643,7 @@ class ChainBase(metaclass=ABCMeta):
self.messageoper.add(**message.dict(), note=note_list)
return self.messagequeue.send_message("post_torrents_message", message=message, torrents=torrents)
def metadata_img(self, mediainfo: MediaInfo,
def metadata_img(self, mediainfo: MediaInfo,
season: Optional[int] = None, episode: Optional[int] = None) -> Optional[dict]:
"""
获取图片名称和url

View File

@@ -3,12 +3,11 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.core.context import MediaInfo
from app.utils.singleton import Singleton
class BangumiChain(ChainBase, metaclass=Singleton):
class BangumiChain(ChainBase):
"""
Bangumi处理链,单例运行
Bangumi处理链
"""
def calendar(self) -> Optional[List[MediaInfo]]:

View File

@@ -2,10 +2,9 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.utils.singleton import Singleton
class DashboardChain(ChainBase, metaclass=Singleton):
class DashboardChain(ChainBase):
"""
各类仪表板统计处理链
"""

View File

@@ -4,12 +4,11 @@ from app import schemas
from app.chain import ChainBase
from app.core.context import MediaInfo
from app.schemas import MediaType
from app.utils.singleton import Singleton
class DoubanChain(ChainBase, metaclass=Singleton):
class DoubanChain(ChainBase):
"""
豆瓣处理链,单例运行
豆瓣处理链
"""
def person_detail(self, person_id: int) -> Optional[schemas.MediaPerson]:

View File

@@ -16,11 +16,12 @@ from app.core.metainfo import MetaInfo
from app.db.downloadhistory_oper import DownloadHistoryOper
from app.db.mediaserver_oper import MediaServerOper
from app.helper.directory import DirectoryHelper
from app.helper.message import MessageHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import ExistMediaInfo, NotExistMediaInfo, DownloadingTorrent, Notification, ResourceSelectionEventData, ResourceDownloadEventData
from app.schemas.types import MediaType, TorrentStatus, EventType, MessageChannel, NotificationType, ContentType, ChainEventType
from app.schemas import ExistMediaInfo, NotExistMediaInfo, DownloadingTorrent, Notification, ResourceSelectionEventData, \
ResourceDownloadEventData
from app.schemas.types import MediaType, TorrentStatus, EventType, MessageChannel, NotificationType, ContentType, \
ChainEventType
from app.utils.http import RequestUtils
from app.utils.string import StringUtils
@@ -30,14 +31,6 @@ class DownloadChain(ChainBase):
下载处理链
"""
def __init__(self):
super().__init__()
self.torrent = TorrentHelper()
self.downloadhis = DownloadHistoryOper()
self.mediaserver = MediaServerOper()
self.directoryhelper = DirectoryHelper()
self.messagehelper = MessageHelper()
def download_torrent(self, torrent: TorrentInfo,
channel: MessageChannel = None,
source: Optional[str] = None,
@@ -120,7 +113,7 @@ class DownloadChain(ChainBase):
logger.error(f"{torrent.title} 无法获取下载地址:{torrent.enclosure}")
return None, "", []
# 下载种子文件
torrent_file, content, download_folder, files, error_msg = self.torrent.download_torrent(
torrent_file, content, download_folder, files, error_msg = TorrentHelper().download_torrent(
url=torrent_url,
cookie=site_cookie,
ua=torrent.site_ua or settings.USER_AGENT,
@@ -218,7 +211,7 @@ class DownloadChain(ChainBase):
else:
content = torrent_file
# 获取种子文件的文件夹名和文件清单
_folder_name, _file_list = self.torrent.get_torrent_info(torrent_file)
_folder_name, _file_list = TorrentHelper().get_torrent_info(torrent_file)
# 下载目录
if save_path:
@@ -226,7 +219,7 @@ class DownloadChain(ChainBase):
download_dir = Path(save_path)
else:
# 根据媒体信息查询下载目录配置
dir_info = self.directoryhelper.get_dir(_media, storage="local", include_unsorted=True)
dir_info = DirectoryHelper().get_dir(_media, storage="local", include_unsorted=True)
# 拼装子目录
if dir_info:
# 一级目录
@@ -276,7 +269,8 @@ class DownloadChain(ChainBase):
_save_path = download_dir if _layout == "NoSubfolder" or not _folder_name else download_path
# 登记下载记录
self.downloadhis.add(
downloadhis = DownloadHistoryOper()
downloadhis.add(
path=str(download_path),
type=_media.type.value,
title=_media.title,
@@ -324,7 +318,7 @@ class DownloadChain(ChainBase):
"torrentname": _meta.org_string,
})
if files_to_add:
self.downloadhis.add_files(files_to_add)
downloadhis.add_files(files_to_add)
# 下载成功发送消息
self.post_message(
@@ -451,6 +445,19 @@ class DownloadChain(ChainBase):
return 9999
return no_exist[season].total_episode
def _calculate_intersection_ratio(episodes_set: set, target_set: set) -> Tuple[float, set]:
"""
计算种子与目标缺失集之间的交集比例。
:param episodes_set (Set[int]): 当前种子的集数集合。
:param target_set (Set[int]): 当前季缺失的集数集合。
:return: Tuple[float, Set[int]]: - 交集比例0~1- 交集集合Set[int]
"""
cal_intersection = episodes_set & target_set
if not cal_intersection:
return 0.0, set()
cal_ratio = len(cal_intersection) / len(episodes_set)
return cal_ratio, cal_intersection
# 发送资源选择事件,允许外部修改上下文数据
logger.debug(f"Initial contexts: {len(contexts)} items, Downloader: {downloader}")
event_data = ResourceSelectionEventData(
@@ -538,7 +545,7 @@ class DownloadChain(ChainBase):
if isinstance(content, str):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法确定种子文件集数")
continue
torrent_episodes = self.torrent.get_torrent_episodes(torrent_files)
torrent_episodes = TorrentHelper().get_torrent_episodes(torrent_files)
logger.info(f"{meta.org_string} 解析种子文件集数为 {torrent_episodes}")
if not torrent_episodes:
continue
@@ -609,6 +616,8 @@ class DownloadChain(ChainBase):
# 缺失整季的转化为缺失集进行比较
if not need_episodes:
need_episodes = list(range(start_episode, total_episode + 1))
# 计算每个种子的集数与缺失集数的交集比例 shaw
torrent_ratios = []
# 循环种子
for context in contexts:
if global_vars.is_system_stopped:
@@ -635,24 +644,54 @@ class DownloadChain(ChainBase):
# 整季的不处理
if not torrent_episodes:
continue
# 为需要集的子集则下载
if torrent_episodes.issubset(set(need_episodes)):
# 下载
logger.info(f"开始下载 {meta.title} ...")
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
downloader=downloader)
if download_id:
# 下载成功
logger.info(f"{meta.title} 添加下载成功")
downloaded_list.append(context)
# 更新仍需集数
need_episodes = __update_episodes(_mid=need_mid,
_need=need_episodes,
_sea=need_season,
_current=torrent_episodes)
logger.info(f"{need_season} 剩余需要集:{need_episodes}")
# 计算交集
# 若种子[5-10],[7-10],[9-10] need_episodes=[9,10,11,12,13,14]
# 计算后的交集比例( len(torrent_episodes ∩ need_episodes) / len(torrent_episodes) )分别 0.33 0.66 1.0
ratio, intersection = _calculate_intersection_ratio(torrent_episodes, set(need_episodes))
if ratio <= (settings.EPISODE_INTERSECTION_MIN_CONFIDENCE or 0.05):
# 可以设定阈值
logger.info(
f"{context.meta_info.title} 与当前缺失集数交集比例过低:{ratio:.2%},跳过")
continue
# 收集候选种子
torrent_ratios.append((context, ratio, len(intersection)))
if not torrent_ratios:
continue
# 按交集比例排序
torrent_ratios.sort(key=lambda x: (x[1], x[2]), reverse=True)
# 按排序后的顺序下载
for context, _, _ in torrent_ratios:
if global_vars.is_system_stopped:
break
# 重新计算与当前need_episodes的交集比例
current_episodes = set(context.meta_info.episode_list)
current_ratio, current_intersection = _calculate_intersection_ratio(current_episodes,
set(need_episodes))
if current_ratio <= (settings.EPISODE_INTERSECTION_MIN_CONFIDENCE or 0.05):
# 可以设定阈值
logger.info(
f"{context.meta_info.title} 与当前缺失集数交集比例过低:{current_ratio:.2%},跳过")
continue
# 下载
logger.info(f"开始下载 {context.meta_info.title} ...")
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
downloader=downloader)
if download_id:
# 下载成功
logger.info(f"{context.meta_info.title} 添加下载成功")
downloaded_list.append(context)
# 更新仍需集数
need_episodes = __update_episodes(_mid=need_mid,
_need=need_episodes,
_sea=need_season,
_current=current_intersection)
logger.info(f"{need_season} 剩余需要集:{need_episodes}")
# 如果已经没有需要下载的集数,跳出当前循环
if not need_episodes:
break
# 仍然缺失的剧集从整季中选择需要的集数文件下载仅支持QB和TR
if no_exists:
@@ -712,7 +751,7 @@ class DownloadChain(ChainBase):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法解析种子文件集数")
continue
# 种子全部集
torrent_episodes = self.torrent.get_torrent_episodes(torrent_files)
torrent_episodes = TorrentHelper().get_torrent_episodes(torrent_files)
logger.info(f"{torrent.site_name} - {meta.org_string} 解析种子文件集数:{torrent_episodes}")
# 选中的集
selected_episodes = set(torrent_episodes).intersection(set(need_episodes))
@@ -801,11 +840,13 @@ class DownloadChain(ChainBase):
if not totals:
totals = {}
mediaserver = MediaServerOper()
if mediainfo.type == MediaType.MOVIE:
# 电影
itemid = self.mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id)
itemid = mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id)
exists_movies: Optional[ExistMediaInfo] = self.media_exists(mediainfo=mediainfo, itemid=itemid)
if exists_movies:
logger.info(f"媒体库中已存在电影:{mediainfo.title_year}")
@@ -825,10 +866,10 @@ class DownloadChain(ChainBase):
logger.error(f"媒体信息中没有季集信息:{mediainfo.title_year}")
return False, {}
# 电视剧
itemid = self.mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season)
itemid = mediaserver.get_item_id(mtype=mediainfo.type.value,
title=mediainfo.title,
tmdbid=mediainfo.tmdb_id,
season=mediainfo.season)
# 媒体库已存在的剧集
exists_tvs: Optional[ExistMediaInfo] = self.media_exists(mediainfo=mediainfo, itemid=itemid)
if not exists_tvs:
@@ -927,7 +968,7 @@ class DownloadChain(ChainBase):
return []
ret_torrents = []
for torrent in torrents:
history = self.downloadhis.get_by_hash(torrent.hash)
history = DownloadHistoryOper().get_by_hash(torrent.hash)
if history:
# 媒体信息
torrent.media = {

View File

@@ -14,7 +14,6 @@ from app.log import logger
from app.schemas import FileItem
from app.schemas.types import EventType, MediaType, ChainEventType
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
recognize_lock = Lock()
@@ -22,15 +21,11 @@ scraping_lock = Lock()
scraping_files = []
class MediaChain(ChainBase, metaclass=Singleton):
class MediaChain(ChainBase):
"""
媒体信息处理链,单例运行
"""
def __init__(self):
super().__init__()
self.storagechain = StorageChain()
def metadata_nfo(self, meta: MetaBase, mediainfo: MediaInfo,
season: Optional[int] = None, episode: Optional[int] = None) -> Optional[str]:
"""
@@ -337,6 +332,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
:param overwrite: 是否覆盖已有文件
"""
storagechain = StorageChain()
def is_bluray_folder(_fileitem: schemas.FileItem) -> bool:
"""
判断是否为原盘目录
@@ -346,7 +343,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 蓝光原盘目录必备的文件或文件夹
required_files = ['BDMV', 'CERTIFICATE']
# 检查目录下是否存在所需文件或文件夹
for item in self.storagechain.list_files(_fileitem):
for item in storagechain.list_files(_fileitem):
if item.name in required_files:
return True
return False
@@ -355,7 +352,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
"""
列出下级文件
"""
return self.storagechain.list_files(fileitem=_fileitem)
return storagechain.list_files(fileitem=_fileitem)
def __save_file(_fileitem: schemas.FileItem, _path: Path, _content: Union[bytes, str]):
"""
@@ -371,7 +368,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
tmp_file.write_bytes(_content)
# 获取文件的父目录
try:
item = self.storagechain.upload_file(fileitem=_fileitem, path=tmp_file, new_name=_path.name)
item = storagechain.upload_file(fileitem=_fileitem, path=tmp_file, new_name=_path.name)
if item:
logger.info(f"已保存文件:{item.path}")
else:
@@ -413,7 +410,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
if fileitem.type == "file":
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 电影文件
movie_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if movie_nfo:
@@ -428,7 +425,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
if is_bluray_folder(fileitem):
# 原盘目录
nfo_path = filepath / (filepath.name + ".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 生成原盘nfo
movie_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if movie_nfo:
@@ -453,8 +450,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
if image_dict:
for image_name, image_url in image_dict.items():
image_path = filepath.with_name(image_name)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 写入图片到当前目录
@@ -477,7 +474,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
return
# 是否已存在
nfo_path = filepath.with_suffix(".nfo")
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 获取集的nfo文件
episode_nfo = self.metadata_nfo(meta=file_meta, mediainfo=file_mediainfo,
season=file_meta.begin_season,
@@ -485,7 +482,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
if episode_nfo:
# 保存或上传nfo文件到上级目录
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=nfo_path, _content=episode_nfo)
else:
logger.warn(f"{filepath.name} nfo文件生成失败")
@@ -497,13 +494,13 @@ class MediaChain(ChainBase, metaclass=Singleton):
if image_dict:
for episode, image_url in image_dict.items():
image_path = filepath.with_suffix(Path(image_url).suffix)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=image_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
@@ -526,7 +523,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
if season_meta.begin_season is not None:
# 是否已存在
nfo_path = filepath / "season.nfo"
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有季号生成季nfo
season_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo,
season=season_meta.begin_season)
@@ -542,14 +539,14 @@ class MediaChain(ChainBase, metaclass=Singleton):
if image_dict:
for image_name, image_url in image_dict.items():
image_path = filepath.with_name(image_name)
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到剧集目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
@@ -564,14 +561,14 @@ class MediaChain(ChainBase, metaclass=Singleton):
if image_season != str(season_meta.begin_season).rjust(2, '0'):
logger.info(f"当前刮削季为:{season_meta.begin_season},跳过文件:{image_path}")
continue
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
if content:
if not parent:
parent = self.storagechain.get_parent_item(fileitem)
parent = storagechain.get_parent_item(fileitem)
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
logger.info(f"已存在图片文件:{image_path}")
@@ -579,7 +576,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
if not season_meta.season:
# 是否已存在
nfo_path = filepath / "tvshow.nfo"
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage, path=nfo_path):
# 当前目录有名称生成tvshow nfo 和 tv图片
tv_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo)
if tv_nfo:
@@ -597,8 +594,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
if image_name.startswith("season"):
continue
image_path = filepath / image_name
if overwrite or not self.storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
path=image_path):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录

View File

@@ -2,7 +2,6 @@ import threading
from typing import List, Union, Optional, Generator, Any
from app.chain import ChainBase
from app.core.cache import cached
from app.core.config import global_vars
from app.db.mediaserver_oper import MediaServerOper
from app.helper.service import ServiceConfigHelper
@@ -17,10 +16,6 @@ class MediaServerChain(ChainBase):
媒体服务器处理链
"""
def __init__(self):
super().__init__()
self.dboper = MediaServerOper()
def librarys(self, server: str, username: Optional[str] = None,
hidden: bool = False) -> List[MediaServerLibrary]:
"""
@@ -96,7 +91,6 @@ class MediaServerChain(ChainBase):
"""
return self.run_module("mediaserver_latest", count=count, server=server, username=username)
@cached(maxsize=1, ttl=3600)
def get_latest_wallpapers(self, server: Optional[str] = None, count: Optional[int] = 10,
remote: bool = True, username: Optional[str] = None) -> List[str]:
"""
@@ -131,7 +125,8 @@ class MediaServerChain(ChainBase):
# 汇总统计
total_count = 0
# 清空登记薄
self.dboper.empty()
dboper = MediaServerOper()
dboper.empty()
# 遍历媒体服务器
for mediaserver in mediaservers:
if not mediaserver:
@@ -175,7 +170,7 @@ class MediaServerChain(ChainBase):
item_dict = item.dict()
item_dict["seasoninfo"] = seasoninfo
item_dict["item_type"] = item_type
self.dboper.add(**item_dict)
dboper.add(**item_dict)
logger.info(f"{server_name} 媒体库 {library.name} 同步完成,共同步数量:{library_count}")
# 总数累加
total_count += library_count

View File

@@ -1,4 +1,4 @@
import copy
import gc
import re
from typing import Any, Optional, Dict, Union
@@ -9,10 +9,9 @@ from app.chain.search import SearchChain
from app.chain.subscribe import SubscribeChain
from app.core.config import settings
from app.core.context import MediaInfo, Context
from app.core.event import EventManager
from app.core.meta import MetaBase
from app.db.message_oper import MessageOper
from app.helper.message import MessageHelper
from app.db.user_oper import UserOper
from app.helper.memory import memory_optimized
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import Notification, NotExistMediaInfo, CommingMessage
@@ -36,19 +35,8 @@ class MessageChain(ChainBase):
# 每页数据量
_page_size: int = 8
def __init__(self):
super().__init__()
self.downloadchain = DownloadChain()
self.subscribechain = SubscribeChain()
self.searchchain = SearchChain()
self.mediachain = MediaChain()
self.eventmanager = EventManager()
self.torrenthelper = TorrentHelper()
self.messagehelper = MessageHelper()
self.messageoper = MessageOper()
@staticmethod
def __get_noexits_info(
self,
_meta: MetaBase,
_mediainfo: MediaInfo) -> Dict[Union[int, str], Dict[int, NotExistMediaInfo]]:
"""
@@ -57,10 +45,10 @@ class MessageChain(ChainBase):
if _mediainfo.type == MediaType.TV:
if not _mediainfo.seasons:
# 补充媒体信息
_mediainfo = self.mediachain.recognize_media(mtype=_mediainfo.type,
tmdbid=_mediainfo.tmdb_id,
doubanid=_mediainfo.douban_id,
cache=False)
_mediainfo = MediaChain().recognize_media(mtype=_mediainfo.type,
tmdbid=_mediainfo.tmdb_id,
doubanid=_mediainfo.douban_id,
cache=False)
if not _mediainfo:
logger.warn(f"{_mediainfo.tmdb_id or _mediainfo.douban_id} 媒体信息识别失败!")
return {}
@@ -130,6 +118,7 @@ class MessageChain(ChainBase):
# 处理消息
self.handle_message(channel=channel, source=source, userid=userid, username=username, text=text)
@memory_optimized(force_gc_after=True, log_memory=True)
def handle_message(self, channel: MessageChannel, source: str,
userid: Union[str, int], username: str, text: str) -> None:
"""
@@ -173,7 +162,7 @@ class MessageChain(ChainBase):
elif text.isdigit():
# 用户选择了具体的条目
# 缓存
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
# 选择项目
if not cache_data \
or not cache_data.get('items') \
@@ -186,15 +175,15 @@ class MessageChain(ChainBase):
# 缓存类型
cache_type: str = cache_data.get('type')
# 缓存列表
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
# 选择
if cache_type in ["Search", "ReSearch"]:
# 当前媒体信息
mediainfo: MediaInfo = cache_list[_choice]
_current_media = mediainfo
# 查询缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(meta=_current_meta,
mediainfo=_current_media)
exist_flag, no_exists = DownloadChain().get_no_exists_info(meta=_current_meta,
mediainfo=_current_media)
if exist_flag and cache_type == "Search":
# 媒体库中已存在
self.post_message(
@@ -234,8 +223,8 @@ class MessageChain(ChainBase):
title=f"开始搜索 {mediainfo.type.value} {mediainfo.title_year} ...",
userid=userid))
# 开始搜索
contexts = self.searchchain.process(mediainfo=mediainfo,
no_exists=no_exists)
contexts = SearchChain().process(mediainfo=mediainfo,
no_exists=no_exists)
if not contexts:
# 没有数据
self.post_message(Notification(
@@ -246,7 +235,7 @@ class MessageChain(ChainBase):
userid=userid))
return
# 搜索结果排序
contexts = self.torrenthelper.sort_torrents(contexts)
contexts = TorrentHelper().sort_torrents(contexts)
# 判断是否设置自动下载
auto_download_user = settings.AUTO_DOWNLOAD_USER
# 匹配到自动下载用户
@@ -276,6 +265,10 @@ class MessageChain(ChainBase):
userid=userid,
total=len(contexts))
# 清理内存
del cache_list
del cache_data
elif cache_type in ["Subscribe", "ReSubscribe"]:
# 订阅或洗版媒体
mediainfo: MediaInfo = cache_list[_choice]
@@ -283,8 +276,8 @@ class MessageChain(ChainBase):
best_version = False
# 查询缺失的媒体信息
if cache_type == "Subscribe":
exist_flag, _ = self.downloadchain.get_no_exists_info(meta=_current_meta,
mediainfo=mediainfo)
exist_flag, _ = DownloadChain().get_no_exists_info(meta=_current_meta,
mediainfo=mediainfo)
if exist_flag:
self.post_message(Notification(
channel=channel,
@@ -296,18 +289,18 @@ class MessageChain(ChainBase):
else:
best_version = True
# 转换用户名
mp_name = self.useroper.get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
mp_name = UserOper().get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
# 添加订阅状态为N
self.subscribechain.add(title=mediainfo.title,
year=mediainfo.year,
mtype=mediainfo.type,
tmdbid=mediainfo.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
best_version=best_version)
SubscribeChain().add(title=mediainfo.title,
year=mediainfo.year,
mtype=mediainfo.type,
tmdbid=mediainfo.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
best_version=best_version)
elif cache_type == "Torrent":
if int(text) == 0:
# 自动选择下载,强制下载模式
@@ -320,12 +313,12 @@ class MessageChain(ChainBase):
# 下载种子
context: Context = cache_list[_choice]
# 下载
self.downloadchain.download_single(context, channel=channel, source=source,
userid=userid, username=username)
DownloadChain().download_single(context, channel=channel, source=source,
userid=userid, username=username)
elif text.lower() == "p":
# 上一页
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
if not cache_data:
# 没有缓存
self.post_message(Notification(
@@ -341,7 +334,7 @@ class MessageChain(ChainBase):
_current_page -= 1
cache_type: str = cache_data.get('type')
# 产生副本,避免修改原值
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
if _current_page == 0:
start = 0
end = self._page_size
@@ -364,10 +357,13 @@ class MessageChain(ChainBase):
items=cache_list[start:end],
userid=userid,
total=len(cache_list))
# 清理内存
del cache_list
del cache_data
elif text.lower() == "n":
# 下一页
cache_data: dict = user_cache.get(userid)
cache_data: dict = user_cache.get(userid).copy()
if not cache_data:
# 没有缓存
self.post_message(Notification(
@@ -375,7 +371,7 @@ class MessageChain(ChainBase):
return
cache_type: str = cache_data.get('type')
# 产生副本,避免修改原值
cache_list: list = copy.deepcopy(cache_data.get('items'))
cache_list: list = cache_data.get('items').copy()
total = len(cache_list)
# 加一页
cache_list = cache_list[
@@ -400,6 +396,9 @@ class MessageChain(ChainBase):
source=source,
title=_current_meta.name,
items=cache_list, userid=userid, total=total)
# 清理内存
del cache_list
del cache_data
else:
# 搜索或订阅
@@ -422,15 +421,19 @@ class MessageChain(ChainBase):
or text.find("继续") != -1:
# 聊天
content = text
action = "chat"
action = "Chat"
elif StringUtils.is_link(text):
# 链接
content = text
action = "Link"
else:
# 搜索
content = text
action = "Search"
if action != "chat":
if action in ["Search", "ReSearch", "Subscribe", "ReSubscribe"]:
# 搜索
meta, medias = self.mediachain.search(content)
meta, medias = MediaChain().search(content)
# 识别
if not meta.name:
self.post_message(Notification(
@@ -471,15 +474,22 @@ class MessageChain(ChainBase):
# 保存缓存
self.save_cache(user_cache, self._cache_file)
# 清理内存
user_cache.clear()
del user_cache
gc.collect()
def __auto_download(self, channel: MessageChannel, source: str, cache_list: list[Context],
userid: Union[str, int], username: str,
no_exists: Optional[Dict[Union[int, str], Dict[int, NotExistMediaInfo]]] = None):
"""
自动择优下载
"""
downloadchain = DownloadChain()
if no_exists is None:
# 查询缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(
exist_flag, no_exists = downloadchain.get_no_exists_info(
meta=_current_meta,
mediainfo=_current_media
)
@@ -488,12 +498,12 @@ class MessageChain(ChainBase):
no_exists = self.__get_noexits_info(_current_meta, _current_media)
# 批量下载
downloads, lefts = self.downloadchain.batch_download(contexts=cache_list,
no_exists=no_exists,
channel=channel,
source=source,
userid=userid,
username=username)
downloads, lefts = downloadchain.batch_download(contexts=cache_list,
no_exists=no_exists,
channel=channel,
source=source,
userid=userid,
username=username)
if downloads and not lefts:
# 全部下载完成
logger.info(f'{_current_media.title_year} 下载完成')
@@ -508,19 +518,19 @@ class MessageChain(ChainBase):
else:
note = None
# 转换用户名
mp_name = self.useroper.get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
mp_name = UserOper().get_name(**{f"{channel.name.lower()}_userid": userid}) if channel else None
# 添加订阅状态为R
self.subscribechain.add(title=_current_media.title,
year=_current_media.year,
mtype=_current_media.type,
tmdbid=_current_media.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
state="R",
note=note)
SubscribeChain().add(title=_current_media.title,
year=_current_media.year,
mtype=_current_media.type,
tmdbid=_current_media.tmdb_id,
season=_current_meta.begin_season,
channel=channel,
source=source,
userid=userid,
username=mp_name or username,
state="R",
note=note)
def __post_medias_message(self, channel: MessageChannel, source: str,
title: str, items: list, userid: str, total: int):

View File

@@ -29,12 +29,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
推荐处理链,单例运行
"""
def __init__(self):
super().__init__()
self.tmdbchain = TmdbChain()
self.doubanchain = DoubanChain()
self.bangumichain = BangumiChain()
self.cache_max_pages = 5
# 推荐数据的缓存页数
cache_max_pages = 5
def refresh_recommend(self):
"""
@@ -174,16 +170,16 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB热门电影
"""
movies = self.tmdbchain.tmdb_discover(mtype=MediaType.MOVIE,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
movies = TmdbChain().tmdb_discover(mtype=MediaType.MOVIE,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
return [movie.to_dict() for movie in movies] if movies else []
@log_execution_time(logger=logger)
@@ -200,16 +196,16 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB热门电视剧
"""
tvs = self.tmdbchain.tmdb_discover(mtype=MediaType.TV,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
tvs = TmdbChain().tmdb_discover(mtype=MediaType.TV,
sort_by=sort_by,
with_genres=with_genres,
with_original_language=with_original_language,
with_keywords=with_keywords,
with_watch_providers=with_watch_providers,
vote_average=vote_average,
vote_count=vote_count,
release_date=release_date,
page=page)
return [tv.to_dict() for tv in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -218,7 +214,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
TMDB流行趋势
"""
infos = self.tmdbchain.tmdb_trending(page=page)
infos = TmdbChain().tmdb_trending(page=page)
return [info.to_dict() for info in infos] if infos else []
@log_execution_time(logger=logger)
@@ -227,7 +223,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
Bangumi每日放送
"""
medias = self.bangumichain.calendar()
medias = BangumiChain().calendar()
return [media.to_dict() for media in medias[(page - 1) * count: page * count]] if medias else []
@log_execution_time(logger=logger)
@@ -236,7 +232,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣正在热映
"""
movies = self.doubanchain.movie_showing(page=page, count=count)
movies = DoubanChain().movie_showing(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -246,8 +242,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣最新电影
"""
movies = self.doubanchain.douban_discover(mtype=MediaType.MOVIE,
sort=sort, tags=tags, page=page, count=count)
movies = DoubanChain().douban_discover(mtype=MediaType.MOVIE,
sort=sort, tags=tags, page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -257,8 +253,8 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣最新电视剧
"""
tvs = self.doubanchain.douban_discover(mtype=MediaType.TV,
sort=sort, tags=tags, page=page, count=count)
tvs = DoubanChain().douban_discover(mtype=MediaType.TV,
sort=sort, tags=tags, page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -267,7 +263,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣电影TOP250
"""
movies = self.doubanchain.movie_top250(page=page, count=count)
movies = DoubanChain().movie_top250(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -276,7 +272,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣国产剧集榜
"""
tvs = self.doubanchain.tv_weekly_chinese(page=page, count=count)
tvs = DoubanChain().tv_weekly_chinese(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -285,7 +281,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣全球剧集榜
"""
tvs = self.doubanchain.tv_weekly_global(page=page, count=count)
tvs = DoubanChain().tv_weekly_global(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -294,7 +290,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门动漫
"""
tvs = self.doubanchain.tv_animation(page=page, count=count)
tvs = DoubanChain().tv_animation(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []
@log_execution_time(logger=logger)
@@ -303,7 +299,7 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门电影
"""
movies = self.doubanchain.movie_hot(page=page, count=count)
movies = DoubanChain().movie_hot(page=page, count=count)
return [media.to_dict() for media in movies] if movies else []
@log_execution_time(logger=logger)
@@ -312,5 +308,5 @@ class RecommendChain(ChainBase, metaclass=Singleton):
"""
豆瓣热门电视剧
"""
tvs = self.doubanchain.tv_hot(page=page, count=count)
tvs = DoubanChain().tv_hot(page=page, count=count)
return [media.to_dict() for media in tvs] if tvs else []

View File

@@ -12,6 +12,7 @@ from app.core.context import MediaInfo, TorrentInfo
from app.core.event import eventmanager, Event
from app.core.metainfo import MetaInfo
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.memory import memory_optimized
from app.helper.progress import ProgressHelper
from app.helper.sites import SitesHelper
from app.helper.torrent import TorrentHelper
@@ -27,13 +28,6 @@ class SearchChain(ChainBase):
__result_temp_file = "__search_result__"
def __init__(self):
super().__init__()
self.siteshelper = SitesHelper()
self.progress = ProgressHelper()
self.systemconfig = SystemConfigOper()
self.torrenthelper = TorrentHelper()
def search_by_id(self, tmdbid: Optional[int] = None, doubanid: Optional[str] = None,
mtype: MediaType = None, area: Optional[str] = "title", season: Optional[int] = None,
sites: List[int] = None, cache_local: bool = False) -> List[Context]:
@@ -104,6 +98,7 @@ class SearchChain(ChainBase):
logger.error(f'加载搜索结果失败:{str(e)} - {traceback.format_exc()}')
return []
@memory_optimized(force_gc_after=True, log_memory=True)
def process(self, mediainfo: MediaInfo,
keyword: Optional[str] = None,
no_exists: Dict[int, Dict[int, NotExistMediaInfo]] = None,
@@ -184,19 +179,20 @@ class SearchChain(ChainBase):
return []
# 开始新进度
self.progress.start(ProgressKey.Search)
progress = ProgressHelper()
progress.start(ProgressKey.Search)
# 开始过滤
self.progress.update(value=0, text=f'开始过滤,总 {len(torrents)} 个资源,请稍候...',
key=ProgressKey.Search)
progress.update(value=0, text=f'开始过滤,总 {len(torrents)} 个资源,请稍候...',
key=ProgressKey.Search)
# 匹配订阅附加参数
if filter_params:
logger.info(f'开始附加参数过滤,附加参数:{filter_params} ...')
torrents = [torrent for torrent in torrents if self.torrenthelper.filter_torrent(torrent, filter_params)]
torrents = [torrent for torrent in torrents if TorrentHelper().filter_torrent(torrent, filter_params)]
# 开始过滤规则过滤
if rule_groups is None:
# 取搜索过滤规则
rule_groups: List[str] = self.systemconfig.get(SystemConfigKey.SearchFilterRuleGroups)
rule_groups: List[str] = SystemConfigOper().get(SystemConfigKey.SearchFilterRuleGroups)
if rule_groups:
logger.info(f'开始过滤规则/剧集过滤,使用规则组:{rule_groups} ...')
torrents = __do_filter(torrents)
@@ -206,7 +202,7 @@ class SearchChain(ChainBase):
logger.info(f"过滤规则/剧集过滤完成,剩余 {len(torrents)} 个资源")
# 过滤完成
self.progress.update(value=50, text=f'过滤完成,剩余 {len(torrents)} 个资源', key=ProgressKey.Search)
progress.update(value=50, text=f'过滤完成,剩余 {len(torrents)} 个资源', key=ProgressKey.Search)
# 开始匹配
_match_torrents = []
@@ -215,17 +211,19 @@ class SearchChain(ChainBase):
# 已处理数
_count = 0
torrenthelper = TorrentHelper()
if mediainfo:
# 英文标题应该在别名/原标题中,不需要再匹配
logger.info(f"开始匹配结果 标题:{mediainfo.title},原标题:{mediainfo.original_title},别名:{mediainfo.names}")
self.progress.update(value=51, text=f'开始匹配,总 {_total} 个资源 ...', key=ProgressKey.Search)
progress.update(value=51, text=f'开始匹配,总 {_total} 个资源 ...', key=ProgressKey.Search)
for torrent in torrents:
if global_vars.is_system_stopped:
break
_count += 1
self.progress.update(value=(_count / _total) * 96,
text=f'正在匹配 {torrent.site_name},已完成 {_count} / {_total} ...',
key=ProgressKey.Search)
progress.update(value=(_count / _total) * 96,
text=f'正在匹配 {torrent.site_name},已完成 {_count} / {_total} ...',
key=ProgressKey.Search)
if not torrent.title:
continue
@@ -236,10 +234,9 @@ class SearchChain(ChainBase):
logger.info(f"种子名称应用识别词后发生改变:{torrent.title} => {torrent_meta.org_string}")
# 季集数过滤
if season_episodes \
and not self.torrenthelper.match_season_episodes(
torrent=torrent,
meta=torrent_meta,
season_episodes=season_episodes):
and not torrenthelper.match_season_episodes(torrent=torrent,
meta=torrent_meta,
season_episodes=season_episodes):
continue
# 比对IMDBID
if torrent.imdbid \
@@ -250,17 +247,17 @@ class SearchChain(ChainBase):
continue
# 比对种子
if self.torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent):
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent):
# 匹配成功
_match_torrents.append((torrent, torrent_meta))
continue
# 匹配完成
logger.info(f"匹配完成,共匹配到 {len(_match_torrents)} 个资源")
self.progress.update(value=97,
text=f'匹配完成,共匹配到 {len(_match_torrents)} 个资源',
key=ProgressKey.Search)
progress.update(value=97,
text=f'匹配完成,共匹配到 {len(_match_torrents)} 个资源',
key=ProgressKey.Search)
else:
_match_torrents = [(t, MetaInfo(title=t.title, subtitle=t.description)) for t in torrents]
@@ -273,17 +270,17 @@ class SearchChain(ChainBase):
meta_info=t[1]) for t in _match_torrents]
# 排序
self.progress.update(value=99,
text=f'正在对 {len(contexts)} 个资源进行排序,请稍候...',
key=ProgressKey.Search)
contexts = self.torrenthelper.sort_torrents(contexts)
progress.update(value=99,
text=f'正在对 {len(contexts)} 个资源进行排序,请稍候...',
key=ProgressKey.Search)
contexts = torrenthelper.sort_torrents(contexts)
# 结束进度
logger.info(f'搜索完成,共 {len(contexts)} 个资源')
self.progress.update(value=100,
text=f'搜索完成,共 {len(contexts)} 个资源',
key=ProgressKey.Search)
self.progress.end(ProgressKey.Search)
progress.update(value=100,
text=f'搜索完成,共 {len(contexts)} 个资源',
key=ProgressKey.Search)
progress.end(ProgressKey.Search)
# 返回
return contexts
@@ -307,9 +304,9 @@ class SearchChain(ChainBase):
# 配置的索引站点
if not sites:
sites = self.systemconfig.get(SystemConfigKey.IndexerSites) or []
sites = SystemConfigOper().get(SystemConfigKey.IndexerSites) or []
for indexer in self.siteshelper.get_indexers():
for indexer in SitesHelper().get_indexers():
# 检查站点索引开关
if not sites or indexer.get("id") in sites:
indexer_sites.append(indexer)
@@ -318,7 +315,8 @@ class SearchChain(ChainBase):
return []
# 开始进度
self.progress.start(ProgressKey.Search)
progress = ProgressHelper()
progress.start(ProgressKey.Search)
# 开始计时
start_time = datetime.now()
# 总数
@@ -326,48 +324,48 @@ class SearchChain(ChainBase):
# 完成数
finish_count = 0
# 更新进度
self.progress.update(value=0,
text=f"开始搜索,共 {total_num} 个站点 ...",
key=ProgressKey.Search)
# 多线程
executor = ThreadPoolExecutor(max_workers=len(indexer_sites))
all_task = []
for site in indexer_sites:
if area == "imdbid":
# 搜索IMDBID
task = executor.submit(self.search_torrents, site=site,
keywords=[mediainfo.imdb_id] if mediainfo else None,
mtype=mediainfo.type if mediainfo else None,
page=page)
else:
# 搜索标题
task = executor.submit(self.search_torrents, site=site,
keywords=keywords,
mtype=mediainfo.type if mediainfo else None,
page=page)
all_task.append(task)
progress.update(value=0,
text=f"开始搜索,共 {total_num} 个站点 ...",
key=ProgressKey.Search)
# 结果集
results = []
for future in as_completed(all_task):
if global_vars.is_system_stopped:
break
finish_count += 1
result = future.result()
if result:
results.extend(result)
logger.info(f"站点搜索进度:{finish_count} / {total_num}")
self.progress.update(value=finish_count / total_num * 100,
text=f"正在搜索{keywords or ''},已完成 {finish_count} / {total_num} 个站点 ...",
key=ProgressKey.Search)
# 多线程
with ThreadPoolExecutor(max_workers=len(indexer_sites)) as executor:
all_task = []
for site in indexer_sites:
if area == "imdbid":
# 搜索IMDBID
task = executor.submit(self.search_torrents, site=site,
keywords=[mediainfo.imdb_id] if mediainfo else None,
mtype=mediainfo.type if mediainfo else None,
page=page)
else:
# 搜索标题
task = executor.submit(self.search_torrents, site=site,
keywords=keywords,
mtype=mediainfo.type if mediainfo else None,
page=page)
all_task.append(task)
for future in as_completed(all_task):
if global_vars.is_system_stopped:
break
finish_count += 1
result = future.result()
if result:
results.extend(result)
logger.info(f"站点搜索进度:{finish_count} / {total_num}")
progress.update(value=finish_count / total_num * 100,
text=f"正在搜索{keywords or ''},已完成 {finish_count} / {total_num} 个站点 ...",
key=ProgressKey.Search)
# 计算耗时
end_time = datetime.now()
# 更新进度
self.progress.update(value=100,
text=f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}",
key=ProgressKey.Search)
progress.update(value=100,
text=f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}",
key=ProgressKey.Search)
logger.info(f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}")
# 结束进度
self.progress.end(ProgressKey.Search)
progress.end(ProgressKey.Search)
# 返回
return results

View File

@@ -16,7 +16,6 @@ from app.helper.browser import PlaywrightHelper
from app.helper.cloudflare import under_challenge
from app.helper.cookie import CookieHelper
from app.helper.cookiecloud import CookieCloudHelper
from app.helper.message import MessageHelper
from app.helper.rss import RssHelper
from app.helper.sites import SitesHelper
from app.log import logger
@@ -34,13 +33,6 @@ class SiteChain(ChainBase):
def __init__(self):
super().__init__()
self.siteoper = SiteOper()
self.siteshelper = SitesHelper()
self.rsshelper = RssHelper()
self.cookiehelper = CookieHelper()
self.message = MessageHelper()
self.cookiecloud = CookieCloudHelper()
self.systemconfig = SystemConfigOper()
# 特殊站点登录验证
self.special_site_test = {
@@ -62,9 +54,9 @@ class SiteChain(ChainBase):
"""
userdata: SiteUserData = self.run_module("refresh_userdata", site=site)
if userdata:
self.siteoper.update_userdata(domain=StringUtils.get_url_domain(site.get("domain")),
name=site.get("name"),
payload=userdata.dict())
SiteOper().update_userdata(domain=StringUtils.get_url_domain(site.get("domain")),
name=site.get("name"),
payload=userdata.dict())
# 发送事件
EventManager().send_event(EventType.SiteRefreshed, {
"site_id": site.get("id")
@@ -100,7 +92,7 @@ class SiteChain(ChainBase):
"""
刷新所有站点的用户数据
"""
sites = self.siteshelper.get_indexers()
sites = SitesHelper().get_indexers()
any_site_updated = False
result = {}
for site in sites:
@@ -303,21 +295,24 @@ class SiteChain(ChainBase):
return sub_domain
logger.info("开始同步CookieCloud站点 ...")
cookies, msg = self.cookiecloud.download()
cookies, msg = CookieCloudHelper().download()
if not cookies:
logger.error(f"CookieCloud同步失败{msg}")
if manual:
self.message.put(msg, title="CookieCloud同步失败", role="system")
self.messagehelper.put(msg, title="CookieCloud同步失败", role="system")
return False, msg
# 保存Cookie或新增站点
_update_count = 0
_add_count = 0
_fail_count = 0
siteshelper = SitesHelper()
siteoper = SiteOper()
rsshelper = RssHelper()
for domain, cookie in cookies.items():
# 索引器信息
indexer = self.siteshelper.get_indexer(domain)
indexer = siteshelper.get_indexer(domain)
# 数据库的站点信息
site_info = self.siteoper.get_by_domain(domain)
site_info = siteoper.get_by_domain(domain)
if site_info and site_info.is_active == 1:
# 站点已存在,检查站点连通性
status, msg = self.test(domain)
@@ -327,7 +322,7 @@ class SiteChain(ChainBase):
# 更新站点rss地址
if not site_info.public and not site_info.rss:
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(
rss_url, errmsg = rsshelper.get_rss_link(
url=site_info.url,
cookie=cookie,
ua=site_info.ua or settings.USER_AGENT,
@@ -335,13 +330,13 @@ class SiteChain(ChainBase):
)
if rss_url:
logger.info(f"更新站点 {domain} RSS地址 ...")
self.siteoper.update_rss(domain=domain, rss=rss_url)
siteoper.update_rss(domain=domain, rss=rss_url)
else:
logger.warn(errmsg)
continue
# 更新站点Cookie
logger.info(f"更新站点 {domain} Cookie ...")
self.siteoper.update_cookie(domain=domain, cookies=cookie)
siteoper.update_cookie(domain=domain, cookies=cookie)
_update_count += 1
elif indexer:
if settings.COOKIECLOUD_BLACKLIST and any(
@@ -396,21 +391,21 @@ class SiteChain(ChainBase):
rss_url = None
if not indexer.get("public") and domain_url:
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(url=domain_url,
cookie=cookie,
ua=settings.USER_AGENT,
proxy=proxy)
rss_url, errmsg = rsshelper.get_rss_link(url=domain_url,
cookie=cookie,
ua=settings.USER_AGENT,
proxy=proxy)
if errmsg:
logger.warn(errmsg)
# 插入数据库
logger.info(f"新增站点 {indexer.get('name')} ...")
self.siteoper.add(name=indexer.get("name"),
url=domain_url,
domain=domain,
cookie=cookie,
rss=rss_url,
proxy=1 if proxy else 0,
public=1 if indexer.get("public") else 0)
siteoper.add(name=indexer.get("name"),
url=domain_url,
domain=domain,
cookie=cookie,
rss=rss_url,
proxy=1 if proxy else 0,
public=1 if indexer.get("public") else 0)
_add_count += 1
# 通知站点更新
@@ -423,7 +418,7 @@ class SiteChain(ChainBase):
if _fail_count > 0:
ret_msg += f"{_fail_count}个站点添加失败,下次同步时将重试,也可以手动添加"
if manual:
self.message.put(ret_msg, title="CookieCloud同步成功", role="system")
self.messagehelper.put(ret_msg, title="CookieCloud同步成功", role="system")
logger.info(f"CookieCloud同步成功{ret_msg}")
return True, ret_msg
@@ -442,29 +437,31 @@ class SiteChain(ChainBase):
if str(domain).startswith("http"):
domain = StringUtils.get_url_domain(domain)
# 站点信息
siteinfo = self.siteoper.get_by_domain(domain)
siteoper = SiteOper()
siteshelper = SitesHelper()
siteinfo = siteoper.get_by_domain(domain)
if not siteinfo:
logger.warn(f"未维护站点 {domain} 信息!")
return
# Cookie
cookie = siteinfo.cookie
# 索引器
indexer = self.siteshelper.get_indexer(domain)
indexer = siteshelper.get_indexer(domain)
if not indexer:
logger.warn(f"站点 {domain} 索引器不存在!")
return
# 查询站点图标
site_icon = self.siteoper.get_icon_by_domain(domain)
site_icon = siteoper.get_icon_by_domain(domain)
if not site_icon or not site_icon.base64:
logger.info(f"开始缓存站点 {indexer.get('name')} 图标 ...")
icon_url, icon_base64 = self.__parse_favicon(url=indexer.get("domain"),
cookie=cookie,
ua=settings.USER_AGENT)
if icon_url:
self.siteoper.update_icon(name=indexer.get("name"),
domain=domain,
icon_url=icon_url,
icon_base64=icon_base64)
siteoper.update_icon(name=indexer.get("name"),
domain=domain,
icon_url=icon_url,
icon_base64=icon_base64)
logger.info(f"缓存站点 {indexer.get('name')} 图标成功")
else:
logger.warn(f"缓存站点 {indexer.get('name')} 图标失败")
@@ -484,11 +481,12 @@ class SiteChain(ChainBase):
# 获取主域名中间那段
domain_host = StringUtils.get_url_host(domain)
# 查询以"site.domain_host"开头的配置项,并清除
site_keys = self.systemconfig.all().keys()
systemconfig = SystemConfigOper()
site_keys = systemconfig.all().keys()
for key in site_keys:
if key.startswith(f"site.{domain_host}"):
logger.info(f"清理站点配置:{key}")
self.systemconfig.delete(key)
systemconfig.delete(key)
@eventmanager.register(EventType.SiteUpdated)
def cache_site_userdata(self, event: Event):
@@ -504,7 +502,7 @@ class SiteChain(ChainBase):
return
if str(domain).startswith("http"):
domain = StringUtils.get_url_domain(domain)
indexer = self.siteshelper.get_indexer(domain)
indexer = SitesHelper().get_indexer(domain)
if not indexer:
return
# 刷新站点用户数据
@@ -518,7 +516,8 @@ class SiteChain(ChainBase):
"""
# 检查域名是否可用
domain = StringUtils.get_url_domain(url)
site_info = self.siteoper.get_by_domain(domain)
siteoper = SiteOper()
site_info = siteoper.get_by_domain(domain)
if not site_info:
return False, f"站点【{url}】不存在"
@@ -535,9 +534,9 @@ class SiteChain(ChainBase):
# 统计
seconds = (datetime.now() - start_time).seconds
if state:
self.siteoper.success(domain=domain, seconds=seconds)
siteoper.success(domain=domain, seconds=seconds)
else:
self.siteoper.fail(domain)
siteoper.fail(domain)
return state, message
except Exception as e:
return False, f"{str(e)}"
@@ -593,7 +592,7 @@ class SiteChain(ChainBase):
"""
查询所有站点,发送消息
"""
site_list = self.siteoper.list()
site_list = SiteOper().list()
if not site_list:
self.post_message(Notification(
channel=channel,
@@ -633,7 +632,8 @@ class SiteChain(ChainBase):
if not arg_str.isdigit():
return
site_id = int(arg_str)
site = self.siteoper.get(site_id)
siteoper = SiteOper()
site = siteoper.get(site_id)
if not site:
self.post_message(Notification(
channel=channel,
@@ -641,7 +641,7 @@ class SiteChain(ChainBase):
userid=userid))
return
# 禁用站点
self.siteoper.update(site_id, {
siteoper.update(site_id, {
"is_active": False
})
# 重新发送消息
@@ -655,25 +655,27 @@ class SiteChain(ChainBase):
if not arg_str:
return
arg_strs = str(arg_str).split()
siteoper = SiteOper()
for arg_str in arg_strs:
arg_str = arg_str.strip()
if not arg_str.isdigit():
continue
site_id = int(arg_str)
site = self.siteoper.get(site_id)
site = siteoper.get(site_id)
if not site:
self.post_message(Notification(
channel=channel,
title=f"站点编号 {site_id} 不存在!", userid=userid))
return
# 禁用站点
self.siteoper.update(site_id, {
siteoper.update(site_id, {
"is_active": True
})
# 重新发送消息
self.remote_list(channel=channel, userid=userid, source=source)
def update_cookie(self, site_info: Site,
@staticmethod
def update_cookie(site_info: Site,
username: str, password: str, two_step_code: Optional[str] = None) -> Tuple[bool, str]:
"""
根据用户名密码更新站点Cookie
@@ -684,7 +686,7 @@ class SiteChain(ChainBase):
:return: (是否成功, 错误信息)
"""
# 更新站点Cookie
result = self.cookiehelper.get_site_cookie_ua(
result = CookieHelper().get_site_cookie_ua(
url=site_info.url,
username=username,
password=password,
@@ -695,7 +697,7 @@ class SiteChain(ChainBase):
cookie, ua, msg = result
if not cookie:
return False, msg
self.siteoper.update(site_info.id, {
SiteOper().update(site_info.id, {
"cookie": cookie,
"ua": ua
})
@@ -737,7 +739,7 @@ class SiteChain(ChainBase):
# 站点ID
site_id = int(site_id)
# 站点信息
site_info = self.siteoper.get(site_id)
site_info = SiteOper().get(site_id)
if not site_info:
self.post_message(Notification(
channel=channel,

View File

@@ -14,10 +14,6 @@ class StorageChain(ChainBase):
存储处理链
"""
def __init__(self):
super().__init__()
self.directoryhelper = DirectoryHelper()
def save_config(self, storage: str, conf: dict) -> None:
"""
保存存储配置
@@ -192,7 +188,7 @@ class StorageChain(ChainBase):
# 检查和删除上级目录
if dir_item and len(Path(dir_item.path).parts) > 2:
# 如何目录是所有下载目录、媒体库目录的上级,则不处理
for d in self.directoryhelper.get_dirs():
for d in DirectoryHelper().get_dirs():
if d.download_path and Path(d.download_path).is_relative_to(Path(dir_item.path)):
logger.debug(f"{dir_item.storage}{dir_item.path} 是下载目录本级或上级目录,不删除")
return True

View File

@@ -24,35 +24,21 @@ from app.db.models.subscribe import Subscribe
from app.db.site_oper import SiteOper
from app.db.subscribe_oper import SubscribeOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.message import MessageHelper
from app.helper.memory import memory_optimized
from app.helper.subscribe import SubscribeHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import MediaRecognizeConvertEventData
from app.schemas.types import MediaType, SystemConfigKey, MessageChannel, NotificationType, EventType, ChainEventType, ContentType
from app.utils.singleton import Singleton
from app.schemas.types import MediaType, SystemConfigKey, MessageChannel, NotificationType, EventType, ChainEventType, \
ContentType
class SubscribeChain(ChainBase, metaclass=Singleton):
class SubscribeChain(ChainBase):
"""
订阅管理处理链
"""
def __init__(self):
super().__init__()
self._rlock = threading.RLock()
self.downloadchain = DownloadChain()
self.downloadhis = DownloadHistoryOper()
self.searchchain = SearchChain()
self.subscribeoper = SubscribeOper()
self.subscribehelper = SubscribeHelper()
self.torrentschain = TorrentsChain()
self.mediachain = MediaChain()
self.tmdbchain = TmdbChain()
self.message = MessageHelper()
self.systemconfig = SystemConfigOper()
self.torrenthelper = TorrentHelper()
self.siteoper = SiteOper()
_rlock = threading.RLock()
def add(self, title: str, year: str,
mtype: MediaType = None,
@@ -86,11 +72,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if event and event.event_data:
event_data: MediaRecognizeConvertEventData = event.event_data
if event_data.media_dict:
mediachain = MediaChain()
new_id = event_data.media_dict.get("id")
if event_data.convert_type == "themoviedb":
return self.mediachain.recognize_media(meta=_meta, tmdbid=new_id)
return mediachain.recognize_media(meta=_meta, tmdbid=new_id)
elif event_data.convert_type == "douban":
return self.mediachain.recognize_media(meta=_meta, doubanid=new_id)
return mediachain.recognize_media(meta=_meta, doubanid=new_id)
return None
logger.info(f'开始添加订阅,标题:{title} ...')
@@ -110,7 +97,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if not tmdbid:
if doubanid:
# 将豆瓣信息转换为TMDB信息
tmdbinfo = self.mediachain.get_tmdbinfo_by_doubanid(doubanid=doubanid, mtype=mtype)
tmdbinfo = MediaChain().get_tmdbinfo_by_doubanid(doubanid=doubanid, mtype=mtype)
if tmdbinfo:
mediainfo = MediaInfo(tmdb_info=tmdbinfo)
elif mediaid:
@@ -213,7 +200,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"filter_groups") else kwargs.get("filter_groups")
})
# 操作数据库
sid, err_msg = self.subscribeoper.add(mediainfo=mediainfo, season=season, username=username, **kwargs)
sid, err_msg = SubscribeOper().add(mediainfo=mediainfo, season=season, username=username, **kwargs)
if not sid:
logger.error(f'{mediainfo.title_year} {err_msg}')
if not exist_ok and message:
@@ -252,7 +239,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"mediainfo": mediainfo.to_dict(),
})
# 统计订阅
self.subscribehelper.sub_reg_async({
SubscribeHelper().sub_reg_async({
"name": title,
"year": year,
"type": metainfo.type.value,
@@ -270,16 +257,18 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 返回结果
return sid, ""
def exists(self, mediainfo: MediaInfo, meta: MetaBase = None):
@staticmethod
def exists(mediainfo: MediaInfo, meta: MetaBase = None):
"""
判断订阅是否已存在
"""
if self.subscribeoper.exists(tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=meta.begin_season if meta else None):
if SubscribeOper().exists(tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=meta.begin_season if meta else None):
return True
return False
@memory_optimized(force_gc_after=True, log_memory=True)
def search(self, sid: Optional[int] = None, state: Optional[str] = 'N', manual: Optional[bool] = False):
"""
订阅搜索
@@ -290,11 +279,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"""
with self._rlock:
logger.debug(f"search lock acquired at {datetime.now()}")
subscribeoper = SubscribeOper()
if sid:
subscribe = self.subscribeoper.get(sid)
subscribe = subscribeoper.get(sid)
subscribes = [subscribe] if subscribe else []
else:
subscribes = self.subscribeoper.list(self.get_states_for_search(state))
subscribes = subscribeoper.list(self.get_states_for_search(state))
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
@@ -349,20 +339,20 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups) or []
or SystemConfigOper().get(SystemConfigKey.BestVersionFilterRuleGroups) or []
else:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups) or []
or SystemConfigOper().get(SystemConfigKey.SubscribeFilterRuleGroups) or []
# 搜索,同时电视剧会过滤掉不需要的剧集
contexts = self.searchchain.process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
contexts = SearchChain().process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
if not contexts:
logger.warn(f'订阅 {subscribe.keyword or subscribe.name} 未搜索到资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
@@ -403,7 +393,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
continue
# 自动下载
downloads, lefts = self.downloadchain.batch_download(
downloads, lefts = DownloadChain().batch_download(
contexts=matched_contexts,
no_exists=no_exists,
userid=subscribe.username,
@@ -414,7 +404,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
)
# 同步外部修改,更新订阅信息
subscribe = self.subscribeoper.get(subscribe.id)
subscribe = subscribeoper.get(subscribe.id)
# 判断是否应完成订阅
if subscribe:
@@ -423,17 +413,17 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
finally:
# 如果状态为N则更新为R
if subscribe and subscribe.state == 'N':
self.subscribeoper.update(subscribe.id, {'state': 'R'})
subscribeoper.update(subscribe.id, {'state': 'R'})
# 手动触发时发送系统消息
if manual:
if subscribes:
if sid:
self.message.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
self.messagehelper.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
else:
self.message.put('所有订阅搜索完成!', title="订阅搜索", role="system")
self.messagehelper.put('所有订阅搜索完成!', title="订阅搜索", role="system")
else:
self.message.put('没有找到订阅!', title="订阅搜索", role="system")
self.messagehelper.put('没有找到订阅!', title="订阅搜索", role="system")
logger.debug(f"search Lock released at {datetime.now()}")
def update_subscribe_priority(self, subscribe: Subscribe, meta: MetaBase,
@@ -448,7 +438,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 当前下载资源的优先级
priority = max([item.torrent_info.pri_order for item in downloads])
# 订阅存在待定策略,不管是否已完成,均需更新订阅信息
self.subscribeoper.update(subscribe.id, {
SubscribeOper().update(subscribe.id, {
"current_priority": priority,
"last_update": datetime.now().strftime('%Y-%m-%d %H:%M:%S')
})
@@ -505,17 +495,18 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if sites is None:
return
self.match(
self.torrentschain.refresh(sites=sites)
TorrentsChain().refresh(sites=sites)
)
def get_sub_sites(self, subscribe: Subscribe) -> List[int]:
@staticmethod
def get_sub_sites(subscribe: Subscribe) -> List[int]:
"""
获取订阅中涉及的站点清单
:param subscribe: 订阅信息对象
:return: 涉及的站点清单
"""
# 从系统配置获取默认订阅站点
default_sites = self.systemconfig.get(SystemConfigKey.RssSites) or []
default_sites = SystemConfigOper().get(SystemConfigKey.RssSites) or []
# 如果订阅未指定站点,直接返回默认站点
if not subscribe.sites:
return default_sites
@@ -535,7 +526,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
:return: 返回[]代表所有站点命中返回None代表没有订阅
"""
# 查询所有订阅
subscribes = self.subscribeoper.list(self.get_states_for_search('R'))
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
if not subscribes:
return None
ret_sites = []
@@ -549,6 +540,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
return ret_sites
@memory_optimized(force_gc_after=True, log_memory=True)
def match(self, torrents: Dict[str, List[Context]]):
"""
从缓存中匹配订阅,并自动下载
@@ -560,7 +552,25 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
with self._rlock:
logger.debug(f"match lock acquired at {datetime.now()}")
# 所有订阅
subscribes = self.subscribeoper.list(self.get_states_for_search('R'))
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
# 预识别所有未识别的种子
processed_torrents: Dict[str, List[Context]] = {}
for domain, contexts in torrents.items():
processed_torrents[domain] = []
for context in contexts:
# 如果种子未识别,尝试识别
if not context.media_info or (not context.media_info.tmdb_id
and not context.media_info.douban_id):
re_mediainfo = self.recognize_media(meta=context.meta_info)
if re_mediainfo:
# 清理多余信息
re_mediainfo.clear()
# 更新种子缓存
context.media_info = re_mediainfo
# 添加已预处理
processed_torrents[domain].append(context)
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
@@ -579,7 +589,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 订阅的站点域名列表
domains = []
if subscribe.sites:
domains = self.siteoper.get_domains_by_ids(subscribe.sites)
domains = SiteOper().get_domains_by_ids(subscribe.sites)
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
@@ -598,15 +608,21 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
if exist_flag:
continue
# 清理多余信息
mediainfo.clear()
# 订阅识别词
if subscribe.custom_words:
custom_words_list = subscribe.custom_words.split("\n")
else:
custom_words_list = None
# 遍历缓存种子
# 遍历预识别后的种子
_match_context = []
for domain, contexts in torrents.items():
torrenthelper = TorrentHelper()
systemconfig = SystemConfigOper()
wordsmatcher = WordsMatcher()
for domain, contexts in processed_torrents.items():
if global_vars.is_system_stopped:
break
if domains and domain not in domains:
@@ -614,7 +630,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
logger.debug(f'开始匹配站点:{domain},共缓存了 {len(contexts)} 个种子...')
for context in contexts:
# 提取信息
_context = copy.deepcopy(context)
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
@@ -628,8 +644,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = WordsMatcher().prepare(torrent_meta.org_string,
custom_words=custom_words_list)
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
@@ -637,33 +653,31 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
context.meta_info = torrent_meta
# 媒体信息需要重新识别
torrent_mediainfo = None
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
_context.media_info = torrent_mediainfo
# 先判断是否有没识别的种子,否则重新识别
if not torrent_mediainfo \
or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
# 更新种子缓存
context.media_info = torrent_mediainfo
_context.media_info = mediainfo
else:
# 通过标题匹配兜底
logger.warn(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if self.torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
context.media_info = torrent_mediainfo
else:
continue
continue
# 直接比对媒体信息
if torrent_mediainfo and (torrent_mediainfo.tmdb_id or torrent_mediainfo.douban_id):
@@ -719,17 +733,17 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
continue
# 匹配订阅附加参数
if not self.torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or self.systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
@@ -756,6 +770,10 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
torrent_mediainfo.episode_group = subscribe.episode_group
_match_context.append(_context)
# 清理内存
contexts.clear()
del contexts
if not _match_context:
# 未匹配到资源
logger.info(f'{mediainfo.title_year} 未匹配到符合条件的资源')
@@ -765,22 +783,23 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
# 开始批量择优下载
logger.info(f'{mediainfo.title_year} 匹配完成,共匹配到{len(_match_context)}个资源')
downloads, lefts = self.downloadchain.batch_download(contexts=_match_context,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
downloads, lefts = DownloadChain().batch_download(contexts=_match_context,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 同步外部修改,更新订阅信息
subscribe = self.subscribeoper.get(subscribe.id)
subscribe = SubscribeOper().get(subscribe.id)
# 判断是否要完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
logger.debug(f"match Lock released at {datetime.now()}")
def check(self):
@@ -788,7 +807,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
定时检查订阅,更新订阅信息
"""
# 查询所有订阅
subscribes = self.subscribeoper.list()
subscribeoper = SubscribeOper()
subscribes = subscribeoper.list()
if not subscribes:
# 没有订阅不运行
return
@@ -827,7 +847,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
total_episode = subscribe.total_episode
lack_episode = subscribe.lack_episode
# 更新TMDB信息
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"name": mediainfo.title,
"year": mediainfo.year,
"vote": mediainfo.vote_average,
@@ -841,28 +861,30 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
})
logger.info(f'{subscribe.name} 订阅元数据更新完成')
def follow(self):
@staticmethod
def follow():
"""
刷新follow的用户分享并自动添加订阅
"""
follow_users: List[str] = self.systemconfig.get(SystemConfigKey.FollowSubscribers)
follow_users: List[str] = SystemConfigOper().get(SystemConfigKey.FollowSubscribers)
if not follow_users:
return
share_subs = self.subscribehelper.get_shares()
share_subs = SubscribeHelper().get_shares()
logger.info(f'开始刷新follow用户分享订阅 ...')
success_count = 0
subscribeoper = SubscribeOper()
for share_sub in share_subs:
uid = share_sub.get("share_uid")
if uid and uid in follow_users:
# 订阅已存在则跳过
if self.subscribeoper.exists(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
if subscribeoper.exists(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
continue
# 已经订阅过跳过
if self.subscribeoper.exist_history(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
if subscribeoper.exist_history(tmdbid=share_sub.get("tmdbid"),
doubanid=share_sub.get("doubanid"),
season=share_sub.get("season")):
continue
# 去除无效属性
for key in list(share_sub.keys()):
@@ -903,7 +925,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
logger.error(f'follow用户分享订阅 {title} 添加失败:{message}')
logger.info(f'follow用户分享订阅刷新完成共添加 {success_count} 个订阅')
def __update_subscribe_note(self, subscribe: Subscribe, downloads: Optional[List[Context]]):
@staticmethod
def __update_subscribe_note(subscribe: Subscribe, downloads: Optional[List[Context]]):
"""
更新已下载信息到note字段
"""
@@ -935,7 +958,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
note = list(set(note).union(set(items)))
# 更新订阅
if note:
self.subscribeoper.update(subscribe.id, {
SubscribeOper().update(subscribe.id, {
"note": note
})
@@ -959,7 +982,8 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
return note
return []
def __update_lack_episodes(self, lefts: Dict[Union[int, str], Dict[int, schemas.NotExistMediaInfo]],
@staticmethod
def __update_lack_episodes(lefts: Dict[Union[int, str], Dict[int, schemas.NotExistMediaInfo]],
subscribe: Subscribe,
mediainfo: MediaInfo,
update_date: Optional[bool] = False):
@@ -992,7 +1016,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
update_data["lack_episode"] = lack_episode
# 更新数据库
if update_data:
self.subscribeoper.update(subscribe.id, update_data)
SubscribeOper().update(subscribe.id, update_data)
def __finish_subscribe(self, subscribe: Subscribe, mediainfo: MediaInfo, meta: MetaBase):
"""
@@ -1005,9 +1029,10 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
msgstr = "订阅" if not subscribe.best_version else "洗版"
logger.info(f'{mediainfo.title_year} 完成{msgstr}')
# 新增订阅历史
self.subscribeoper.add_history(**subscribe.to_dict())
subscribeoper = SubscribeOper()
subscribeoper.add_history(**subscribe.to_dict())
# 删除订阅
self.subscribeoper.delete(subscribe.id)
subscribeoper.delete(subscribe.id)
# 发送通知
if mediainfo.type == MediaType.TV:
link = settings.MP_DOMAIN('#/subscribe/tv?tab=mysub')
@@ -1034,7 +1059,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"mediainfo": mediainfo.to_dict(),
})
# 统计订阅
self.subscribehelper.sub_done_async({
SubscribeHelper().sub_done_async({
"tmdbid": mediainfo.tmdb_id,
"doubanid": mediainfo.douban_id
})
@@ -1044,7 +1069,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"""
查询订阅并发送消息
"""
subscribes = self.subscribeoper.list()
subscribes = SubscribeOper().list()
if not subscribes:
self.post_message(schemas.Notification(channel=channel,
source=source,
@@ -1078,20 +1103,22 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
"[id]为订阅编号", userid=userid))
return
arg_strs = str(arg_str).split()
subscribeoper = SubscribeOper()
subscribehelper = SubscribeHelper()
for arg_str in arg_strs:
arg_str = arg_str.strip()
if not arg_str.isdigit():
continue
subscribe_id = int(arg_str)
subscribe = self.subscribeoper.get(subscribe_id)
subscribe = subscribeoper.get(subscribe_id)
if not subscribe:
self.post_message(schemas.Notification(channel=channel, source=source,
title=f"订阅编号 {subscribe_id} 不存在!", userid=userid))
return
# 删除订阅
self.subscribeoper.delete(subscribe_id)
subscribeoper.delete(subscribe_id)
# 统计订阅
self.subscribehelper.sub_done_async({
subscribehelper.sub_done_async({
"tmdbid": subscribe.tmdbid,
"doubanid": subscribe.doubanid
})
@@ -1217,13 +1244,14 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
site_id = event_data.get("site_id")
if not site_id:
return
subscribeoper = SubscribeOper()
if site_id == "*":
# 站点被重置
SystemConfigOper().set(SystemConfigKey.RssSites, [])
for subscribe in self.subscribeoper.list():
for subscribe in subscribeoper.list():
if not subscribe.sites:
continue
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"sites": []
})
return
@@ -1233,14 +1261,14 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
selected_sites.remove(site_id)
SystemConfigOper().set(SystemConfigKey.RssSites, selected_sites)
# 查询所有订阅
for subscribe in self.subscribeoper.list():
for subscribe in subscribeoper.list():
if not subscribe.sites:
continue
sites = subscribe.sites or []
if site_id not in sites:
continue
sites.remove(site_id)
self.subscribeoper.update(subscribe.id, {
subscribeoper.update(subscribe.id, {
"sites": sites
})
@@ -1265,12 +1293,13 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
return None
return value.get(default_config_key) or None
def get_params(self, subscribe: Subscribe):
@staticmethod
def get_params(subscribe: Subscribe):
"""
获取订阅默认参数
"""
# 默认过滤规则
default_rule = self.systemconfig.get(SystemConfigKey.SubscribeDefaultParams) or {}
default_rule = SystemConfigOper().get(SystemConfigKey.SubscribeDefaultParams) or {}
return {
key: value for key, value in {
"include": subscribe.include or default_rule.get("include"),
@@ -1298,7 +1327,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
episodes: Dict[int, schemas.SubscribeEpisodeInfo] = {}
if subscribe.tmdbid and subscribe.type == MediaType.TV.value:
# 查询TMDB中的集信息
tmdb_episodes = self.tmdbchain.tmdb_episodes(
tmdb_episodes = TmdbChain().tmdb_episodes(
tmdbid=subscribe.tmdbid,
season=subscribe.season,
episode_group=subscribe.episode_group
@@ -1323,11 +1352,12 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
episodes[0] = info
# 所有下载记录
download_his = self.downloadhis.get_by_mediaid(tmdbid=subscribe.tmdbid, doubanid=subscribe.doubanid)
downloadhis = DownloadHistoryOper()
download_his = downloadhis.get_by_mediaid(tmdbid=subscribe.tmdbid, doubanid=subscribe.doubanid)
if download_his:
for his in download_his:
# 查询下载文件
files = self.downloadhis.get_files_by_hash(his.download_hash)
files = downloadhis.get_files_by_hash(his.download_hash)
if files:
for file in files:
# 识别文件名
@@ -1421,7 +1451,7 @@ class SubscribeChain(ChainBase, metaclass=Singleton):
subscribe.season: subscribe.total_episode
}
# 查询媒体库缺失的媒体信息
exist_flag, no_exists = self.downloadchain.get_no_exists_info(
exist_flag, no_exists = DownloadChain().get_no_exists_info(
meta=meta,
mediainfo=mediainfo,
totals=totals

View File

@@ -8,23 +8,18 @@ from app.core.config import settings
from app.log import logger
from app.schemas import Notification, MessageChannel
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.system import SystemUtils
from app.helper.system import SystemHelper
from version import FRONTEND_VERSION, APP_VERSION
class SystemChain(ChainBase, metaclass=Singleton):
class SystemChain(ChainBase):
"""
系统级处理链
"""
_restart_file = "__system_restart__"
def __init__(self):
super().__init__()
# 重启完成检测
self.restart_finish()
def remote_clear_cache(self, channel: MessageChannel, userid: Union[int, str], source: Optional[str] = None):
"""
清理系统缓存
@@ -45,7 +40,8 @@ class SystemChain(ChainBase, metaclass=Singleton):
"channel": channel.value,
"userid": userid
}, self._restart_file)
SystemUtils.restart()
# 重启
SystemHelper.restart()
def __get_version_message(self) -> str:
"""

View File

@@ -3,13 +3,11 @@ from typing import Optional, List
from app import schemas
from app.chain import ChainBase
from app.core.cache import cached
from app.core.context import MediaInfo
from app.schemas import MediaType
from app.utils.singleton import Singleton
class TmdbChain(ChainBase, metaclass=Singleton):
class TmdbChain(ChainBase):
"""
TheMovieDB处理链单例运行
"""
@@ -145,7 +143,6 @@ class TmdbChain(ChainBase, metaclass=Singleton):
"""
return self.run_module("tmdb_person_credits", person_id=person_id, page=page)
@cached(maxsize=1, ttl=3600)
def get_random_wallpager(self) -> Optional[str]:
"""
获取随机壁纸缓存1个小时
@@ -159,7 +156,6 @@ class TmdbChain(ChainBase, metaclass=Singleton):
return info.backdrop_path
return None
@cached(maxsize=1, ttl=3600)
def get_trending_wallpapers(self, num: Optional[int] = 10) -> List[str]:
"""
获取所有流行壁纸

View File

@@ -1,9 +1,8 @@
import gc
import re
import traceback
from typing import Dict, List, Union, Optional
from cachetools import cached, TTLCache
from app.chain import ChainBase
from app.chain.media import MediaChain
from app.core.config import settings, global_vars
@@ -11,17 +10,17 @@ from app.core.context import TorrentInfo, Context, MediaInfo
from app.core.metainfo import MetaInfo
from app.db.site_oper import SiteOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.memory import memory_optimized, clear_large_objects
from app.helper.rss import RssHelper
from app.helper.sites import SitesHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import Notification
from app.schemas.types import SystemConfigKey, MessageChannel, NotificationType, MediaType
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
class TorrentsChain(ChainBase, metaclass=Singleton):
class TorrentsChain(ChainBase):
"""
站点首页或RSS种子处理链服务于订阅、刷流等
"""
@@ -29,14 +28,14 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
_spider_file = "__torrents_cache__"
_rss_file = "__rss_cache__"
def __init__(self):
super().__init__()
self.siteshelper = SitesHelper()
self.siteoper = SiteOper()
self.rsshelper = RssHelper()
self.systemconfig = SystemConfigOper()
self.mediachain = MediaChain()
self.torrenthelper = TorrentHelper()
@property
def cache_file(self) -> str:
"""
返回缓存文件列表
"""
if settings.SUBSCRIBE_MODE == 'spider':
return self._spider_file
return self._rss_file
def remote_refresh(self, channel: MessageChannel, userid: Union[str, int] = None):
"""
@@ -72,39 +71,39 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
self.remove_cache(self._rss_file)
logger.info(f'种子缓存数据清理完成')
@cached(cache=TTLCache(maxsize=128, ttl=595))
@memory_optimized(force_gc_after=True, log_memory=True)
def browse(self, domain: str, keyword: Optional[str] = None, cat: Optional[str] = None,
page: Optional[int] = 0) -> List[TorrentInfo]:
"""
浏览站点首页内容返回种子清单TTL缓存10分钟
浏览站点首页内容返回种子清单TTL缓存5分钟
:param domain: 站点域名
:param keyword: 搜索标题
:param cat: 搜索分类
:param page: 页码
"""
logger.info(f'开始获取站点 {domain} 最新种子 ...')
site = self.siteshelper.get_indexer(domain)
site = SitesHelper().get_indexer(domain)
if not site:
logger.error(f'站点 {domain} 不存在!')
return []
return self.refresh_torrents(site=site, keyword=keyword, cat=cat, page=page)
@cached(cache=TTLCache(maxsize=128, ttl=295))
@memory_optimized(force_gc_after=True, log_memory=True)
def rss(self, domain: str) -> List[TorrentInfo]:
"""
获取站点RSS内容返回种子清单TTL缓存5分钟
获取站点RSS内容返回种子清单TTL缓存3分钟
:param domain: 站点域名
"""
logger.info(f'开始获取站点 {domain} RSS ...')
site = self.siteshelper.get_indexer(domain)
site = SitesHelper().get_indexer(domain)
if not site:
logger.error(f'站点 {domain} 不存在!')
return []
if not site.get("rss"):
logger.error(f'站点 {domain} 未配置RSS地址')
return []
rss_items = self.rsshelper.parse(site.get("rss"), True if site.get("proxy") else False,
timeout=int(site.get("timeout") or 30))
rss_items = RssHelper().parse(site.get("rss"), True if site.get("proxy") else False,
timeout=int(site.get("timeout") or 30))
if rss_items is None:
# rss过期尝试保留原配置生成新的rss
self.__renew_rss_url(domain=domain, site=site)
@@ -135,6 +134,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
return ret_torrents
@memory_optimized(force_gc_after=True, log_memory=True)
def refresh(self, stype: Optional[str] = None, sites: List[int] = None) -> Dict[str, List[Context]]:
"""
刷新站点最新资源,识别并缓存起来
@@ -147,7 +147,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 刷新站点
if not sites:
sites = self.systemconfig.get(SystemConfigKey.RssSites) or []
sites = SystemConfigOper().get(SystemConfigKey.RssSites) or []
# 读取缓存
torrents_cache = self.get_torrents()
@@ -155,12 +155,13 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 缓存过滤掉无效种子
for _domain, _torrents in torrents_cache.items():
torrents_cache[_domain] = [_torrent for _torrent in _torrents
if not self.torrenthelper.is_invalid(_torrent.torrent_info.enclosure)]
if not TorrentHelper().is_invalid(_torrent.torrent_info.enclosure)]
# 所有站点索引
indexers = self.siteshelper.get_indexers()
indexers = SitesHelper().get_indexers()
# 需要刷新的站点domain
domains = []
# 遍历站点缓存资源
for indexer in indexers:
if global_vars.is_system_stopped:
@@ -181,11 +182,11 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 取前N条
torrents = torrents[:settings.CACHE_CONF["refresh"]]
if torrents:
# 过滤出没有处理过的种子
# 过滤出没有处理过的种子 - 优化:使用集合查找,避免重复创建字符串列表
cached_signatures = {f'{t.torrent_info.title}{t.torrent_info.description}'
for t in torrents_cache.get(domain) or []}
torrents = [torrent for torrent in torrents
if f'{torrent.title}{torrent.description}'
not in [f'{t.torrent_info.title}{t.torrent_info.description}'
for t in torrents_cache.get(domain) or []]]
if f'{torrent.title}{torrent.description}' not in cached_signatures]
if torrents:
logger.info(f'{indexer.get("name")}{len(torrents)} 个新种子')
else:
@@ -204,12 +205,12 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
and torrent.category == MediaType.TV.value:
meta.type = MediaType.TV
# 识别媒体信息
mediainfo: MediaInfo = self.mediachain.recognize_by_meta(meta)
mediainfo: MediaInfo = MediaChain().recognize_by_meta(meta)
if not mediainfo:
logger.warn(f'{torrent.title} 未识别到媒体信息')
# 存储空的媒体信息
mediainfo = MediaInfo()
# 清理多余数据
# 清理多余数据,减少内存占用
mediainfo.clear()
# 上下文
context = Context(meta_info=meta, media_info=mediainfo, torrent_info=torrent)
@@ -220,9 +221,17 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
torrents_cache[domain].append(context)
# 如果超过了限制条数则移除掉前面的
if len(torrents_cache[domain]) > settings.CACHE_CONF["torrents"]:
# 优化直接删除旧数据无需重复清理数据进缓存前已经clear过
old_contexts = torrents_cache[domain][:-settings.CACHE_CONF["torrents"]]
torrents_cache[domain] = torrents_cache[domain][-settings.CACHE_CONF["torrents"]:]
# 清理旧对象
clear_large_objects(*old_contexts)
# 优化:清理不再需要的临时变量
del meta, mediainfo, context
# 回收资源
torrents.clear()
del torrents
gc.collect()
else:
logger.info(f'{indexer.get("name")} 没有获取到种子')
@@ -234,7 +243,18 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 去除不在站点范围内的缓存种子
if sites and torrents_cache:
old_cache = torrents_cache
torrents_cache = {k: v for k, v in torrents_cache.items() if k in domains}
# 清理不再使用的缓存数据数据进缓存前已经clear过无需重复清理
removed_contexts = []
for domain, contexts in old_cache.items():
if domain not in domains:
removed_contexts.extend(contexts)
# 批量清理
if removed_contexts:
clear_large_objects(*removed_contexts)
del old_cache
return torrents_cache
def __renew_rss_url(self, domain: str, site: dict):
@@ -245,7 +265,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# RSS链接过期
logger.error(f"站点 {domain} RSS链接已过期正在尝试自动获取")
# 自动生成rss地址
rss_url, errmsg = self.rsshelper.get_rss_link(
rss_url, errmsg = RssHelper().get_rss_link(
url=site.get("url"),
cookie=site.get("cookie"),
ua=site.get("ua") or settings.USER_AGENT,
@@ -259,7 +279,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
# 获取过期rss除去passkey部分
new_rss = re.sub(r'&passkey=([a-zA-Z0-9]+)', f'&passkey={new_passkey}', site.get("rss"))
logger.info(f"更新站点 {domain} RSS地址 ...")
self.siteoper.update_rss(domain=domain, rss=new_rss)
SiteOper().update_rss(domain=domain, rss=new_rss)
else:
# 发送消息
self.post_message(

View File

@@ -25,6 +25,7 @@ from app.db.systemconfig_oper import SystemConfigOper
from app.db.transferhistory_oper import TransferHistoryOper
from app.helper.directory import DirectoryHelper
from app.helper.format import FormatParser
from app.helper.memory import memory_optimized
from app.helper.progress import ProgressHelper
from app.log import logger
from app.schemas import TransferInfo, TransferTorrent, Notification, EpisodeFormat, FileItem, TransferDirectoryConf, \
@@ -328,7 +329,8 @@ class JobManager:
# 计算状态为完成的任务数
if __mediaid__ not in self._job_view:
return 0
return sum([task.fileitem.size for task in self._job_view[__mediaid__].tasks if task.state == "completed" and task.fileitem.size is not None])
return sum([task.fileitem.size for task in self._job_view[__mediaid__].tasks if
task.state == "completed" and task.fileitem.size is not None])
def total(self) -> int:
"""
@@ -371,14 +373,6 @@ class TransferChain(ChainBase, metaclass=Singleton):
def __init__(self):
super().__init__()
self.downloadhis = DownloadHistoryOper()
self.transferhis = TransferHistoryOper()
self.progress = ProgressHelper()
self.mediachain = MediaChain()
self.tmdbchain = TmdbChain()
self.storagechain = StorageChain()
self.systemconfig = SystemConfigOper()
self.directoryhelper = DirectoryHelper()
self.jobview = JobManager()
# 启动整理任务
@@ -397,11 +391,12 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
整理完成后处理
"""
transferhis = TransferHistoryOper()
if not transferinfo.success:
# 转移失败
logger.warn(f"{task.fileitem.name} 入库失败:{transferinfo.message}")
# 新增转移失败历史记录
self.transferhis.add_fail(
transferhis.add_fail(
fileitem=task.fileitem,
mode=transferinfo.transfer_type if transferinfo else '',
downloader=task.downloader,
@@ -428,7 +423,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.info(f"{task.fileitem.name} 入库成功:{transferinfo.target_diritem.path}")
# 新增转移成功历史记录
self.transferhis.add_success(
transferhis.add_success(
fileitem=task.fileitem,
mode=transferinfo.transfer_type if transferinfo else '',
downloader=task.downloader,
@@ -457,6 +452,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
tasks = self.jobview.success_tasks(task.mediainfo, task.meta.begin_season)
# 记录已处理的种子hash
processed_hashes = set()
storagechain = StorageChain()
for t in tasks:
# 下载器hash
if t.download_hash and t.download_hash not in processed_hashes:
@@ -465,7 +461,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.info(f"移动模式删除种子成功:{t.download_hash} ")
# 删除残留目录
if t.fileitem:
self.storagechain.delete_media_file(t.fileitem, delete_self=False)
storagechain.delete_media_file(t.fileitem, delete_self=False)
# 整理完成且有成功的任务时
if self.jobview.is_finished(task):
# 发送通知,实时手动整理时不发
@@ -543,6 +539,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 失败数量
fail_num = 0
progress = ProgressHelper()
while not global_vars.is_system_stopped:
try:
item: TransferQueue = self._queue.get(block=False)
@@ -556,24 +554,24 @@ class TransferChain(ChainBase, metaclass=Singleton):
if __queue_start:
logger.info("开始整理队列处理...")
# 启动进度
self.progress.start(ProgressKey.FileTransfer)
progress.start(ProgressKey.FileTransfer)
# 重置计数
processed_num = 0
fail_num = 0
total_num = self.jobview.total()
__process_msg = f"开始整理队列处理,当前共 {total_num} 个文件 ..."
logger.info(__process_msg)
self.progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
# 队列已开始
__queue_start = False
# 更新进度
__process_msg = f"正在整理 {fileitem.name} ..."
logger.info(__process_msg)
self.progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
# 整理
state, err_msg = self.__handle_transfer(task=task, callback=item.callback)
if not state:
@@ -583,18 +581,18 @@ class TransferChain(ChainBase, metaclass=Singleton):
processed_num += 1
__process_msg = f"{fileitem.name} 整理完成"
logger.info(__process_msg)
self.progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=processed_num / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
except queue.Empty:
if not __queue_start:
# 结束进度
__end_msg = f"整理队列处理完成,共整理 {processed_num} 个文件,失败 {fail_num}"
logger.info(__end_msg)
self.progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
self.progress.end(ProgressKey.FileTransfer)
progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
# 重置计数
processed_num = 0
fail_num = 0
@@ -614,6 +612,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
try:
# 识别
transferhis = TransferHistoryOper()
if not task.mediainfo:
mediainfo = None
download_history = task.download_history
@@ -633,7 +632,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
mediainfo.category = download_history.media_category
else:
# 识别媒体信息
mediainfo = self.mediachain.recognize_by_meta(task.meta)
mediainfo = MediaChain().recognize_by_meta(task.meta)
# 更新媒体图片
if mediainfo:
@@ -641,7 +640,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not mediainfo:
# 新增整理失败历史记录
his = self.transferhis.add_fail(
his = transferhis.add_fail(
fileitem=task.fileitem,
mode=task.transfer_type,
meta=task.meta,
@@ -661,8 +660,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 如果未开启新增已入库媒体是否跟随TMDB信息变化则根据tmdbid查询之前的title
if not settings.SCRAP_FOLLOW_TMDB:
transfer_history = self.transferhis.get_by_type_tmdbid(tmdbid=mediainfo.tmdb_id,
mtype=mediainfo.type.value)
transfer_history = transferhis.get_by_type_tmdbid(tmdbid=mediainfo.tmdb_id,
mtype=mediainfo.type.value)
if transfer_history:
mediainfo.title = transfer_history.title
@@ -682,7 +681,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 默认值1
if season_num is None:
season_num = 1
task.episodes_info = self.tmdbchain.tmdb_episodes(
task.episodes_info = TmdbChain().tmdb_episodes(
tmdbid=task.mediainfo.tmdb_id,
season=season_num,
episode_group=task.mediainfo.episode_group
@@ -692,15 +691,15 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not task.target_directory:
if task.target_path:
# 指定目标路径,`手动整理`场景下使用,忽略源目录匹配,使用指定目录匹配
task.target_directory = self.directoryhelper.get_dir(media=task.mediainfo,
dest_path=task.target_path,
target_storage=task.target_storage)
task.target_directory = DirectoryHelper().get_dir(media=task.mediainfo,
dest_path=task.target_path,
target_storage=task.target_storage)
else:
# 启用源目录匹配时,根据源目录匹配下载目录,否则按源目录同盘优先原则,如无源目录,则根据媒体信息获取目标目录
task.target_directory = self.directoryhelper.get_dir(media=task.mediainfo,
storage=task.fileitem.storage,
src_path=Path(task.fileitem.path),
target_storage=task.target_storage)
task.target_directory = DirectoryHelper().get_dir(media=task.mediainfo,
storage=task.fileitem.storage,
src_path=Path(task.fileitem.path),
target_storage=task.target_storage)
if not task.target_storage and task.target_directory:
task.target_storage = task.target_directory.library_storage
@@ -784,7 +783,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 全局锁,避免重复处理
with downloader_lock:
# 获取下载器监控目录
download_dirs = self.directoryhelper.get_download_dirs()
download_dirs = DirectoryHelper().get_download_dirs()
# 如果没有下载器监控的目录则不处理
if not any(dir_info.monitor_type == "downloader" and dir_info.storage == "local"
for dir_info in download_dirs):
@@ -820,7 +819,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.debug(f"文件 {file_path} 不在下载器监控目录中,不通过下载器进行整理")
continue
# 查询下载记录识别情况
downloadhis: DownloadHistory = self.downloadhis.get_by_hash(torrent.hash)
downloadhis: DownloadHistory = DownloadHistoryOper().get_by_hash(torrent.hash)
if downloadhis:
# 类型
try:
@@ -859,26 +858,31 @@ class TransferChain(ChainBase, metaclass=Singleton):
)
# 设置下载任务状态
if state:
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
if not state:
logger.warn(f"整理下载器任务失败:{torrent.hash} - {errmsg}")
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
# 结束
logger.info("所有下载器中下载完成的文件已整理完成")
return True
def __get_trans_fileitems(self, fileitem: FileItem) -> List[Tuple[FileItem, bool]]:
def __get_trans_fileitems(
self, fileitem: FileItem, depth: int = 1
) -> List[Tuple[FileItem, bool]]:
"""
获取整理目录或文件列表
:param fileitem: 文件项
:param depth: 递归深度默认为1
"""
storagechain = StorageChain()
def __is_bluray_dir(_fileitem: FileItem) -> bool:
def __contains_bluray_sub(_fileitems: List[FileItem]) -> bool:
"""
判断是不是蓝光目录
判断是否包含蓝光目录
"""
subs = self.storagechain.list_files(_fileitem)
if subs:
for sub in subs:
if _fileitems:
for sub in _fileitems:
if sub.type == "dir" and sub.name in ["BDMV", "CERTIFICATE"]:
return True
return False
@@ -895,10 +899,10 @@ class TransferChain(ChainBase, metaclass=Singleton):
"""
for p in _path.parents:
if p.name == "BDMV":
return self.storagechain.get_file_item(storage=_storage, path=p.parent)
return storagechain.get_file_item(storage=_storage, path=p.parent)
return None
if not self.storagechain.get_item(fileitem):
if not storagechain.get_item(fileitem):
logger.warn(f"目录或文件不存在:{fileitem.path}")
return []
@@ -913,30 +917,28 @@ class TransferChain(ChainBase, metaclass=Singleton):
return [(fileitem, False)]
# 蓝光原盘根目录
if __is_bluray_dir(fileitem):
sub_items = storagechain.list_files(fileitem) or []
if __contains_bluray_sub(sub_items):
return [(fileitem, True)]
# 需要整理的文件项列表
trans_items = []
# 先检查当前目录的下级目录,以支持合集的情况
for sub_dir in self.storagechain.list_files(fileitem):
for sub_dir in sub_items if depth >= 1 else []:
if sub_dir.type == "dir":
if __is_bluray_dir(sub_dir):
trans_items.append((sub_dir, True))
else:
trans_items.append((sub_dir, False))
trans_items.extend(self.__get_trans_fileitems(sub_dir, depth=depth - 1))
if not trans_items:
# 没有有效子目录,直接整理当前目录
trans_items.append((fileitem, False))
else:
# 有子目录时,把当前目录的文件添加到整理任务中
sub_items = self.storagechain.list_files(fileitem)
if sub_items:
trans_items.extend([(f, False) for f in sub_items if f.type == "file"])
return trans_items
@memory_optimized(force_gc_after=True, log_memory=True)
def do_transfer(self, fileitem: FileItem,
meta: MetaBase = None, mediainfo: MediaInfo = None,
target_directory: TransferDirectoryConf = None,
@@ -993,11 +995,13 @@ class TransferChain(ChainBase, metaclass=Singleton):
offset=epformat.offset) if epformat else None
# 整理屏蔽词
transfer_exclude_words = self.systemconfig.get(SystemConfigKey.TransferExcludeWords)
transfer_exclude_words = SystemConfigOper().get(SystemConfigKey.TransferExcludeWords)
# 汇总错误信息
err_msgs: List[str] = []
# 待整理目录或文件项
trans_items = self.__get_trans_fileitems(fileitem)
trans_items = self.__get_trans_fileitems(
fileitem, depth=2 # 为解决 issue#4371 深度至少需要>=2
)
# 待整理的文件列表
file_items: List[Tuple[FileItem, bool]] = []
@@ -1010,7 +1014,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 如果是目录且不是⼀蓝光原盘,获取所有文件并整理
if trans_item.type == "dir" and not bluray_dir:
# 遍历获取下载目录所有文件(递归)
if files := self.storagechain.list_files(trans_item, recursion=True):
if files := StorageChain().list_files(trans_item, recursion=True):
file_items.extend([(file, False) for file in files])
else:
file_items.append((trans_item, bluray_dir))
@@ -1059,7 +1063,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 整理成功的不再处理
if not force:
transferd = self.transferhis.get_by_src(file_item.path, storage=file_item.storage)
transferd = TransferHistoryOper().get_by_src(file_item.path, storage=file_item.storage)
if transferd:
if not transferd.status:
all_success = False
@@ -1095,14 +1099,15 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 根据父路径获取下载历史
download_history = None
downloadhis = DownloadHistoryOper()
if bluray_dir:
# 蓝光原盘,按目录名查询
download_history = self.downloadhis.get_by_path(str(file_path))
download_history = downloadhis.get_by_path(str(file_path))
else:
# 按文件全路径查询
download_file = self.downloadhis.get_file_by_fullpath(str(file_path))
download_file = downloadhis.get_file_by_fullpath(str(file_path))
if download_file:
download_history = self.downloadhis.get_by_hash(download_file.download_hash)
download_history = downloadhis.get_by_hash(download_file.download_hash)
# 获取下载Hash
if download_history and (not downloader or not download_hash):
@@ -1145,12 +1150,13 @@ class TransferChain(ChainBase, metaclass=Singleton):
fail_num = 0
# 启动进度
self.progress.start(ProgressKey.FileTransfer)
progress = ProgressHelper()
progress.start(ProgressKey.FileTransfer)
__process_msg = f"开始整理,共 {total_num} 个文件 ..."
logger.info(__process_msg)
self.progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
for transfer_task in transfer_tasks:
if global_vars.is_system_stopped:
@@ -1160,9 +1166,9 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 更新进度
__process_msg = f"正在整理 {processed_num + fail_num + 1}/{total_num}{transfer_task.fileitem.name} ..."
logger.info(__process_msg)
self.progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
state, err_msg = self.__handle_transfer(
task=transfer_task,
callback=self.__default_callback
@@ -1178,10 +1184,10 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 整理结束
__end_msg = f"整理队列处理完成,共整理 {total_num} 个文件,失败 {fail_num}"
logger.info(__end_msg)
self.progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
self.progress.end(ProgressKey.FileTransfer)
progress.update(value=100,
text=__end_msg,
key=ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
return all_success, "".join(err_msgs)
@@ -1236,7 +1242,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
:param mediaid: TMDB ID/豆瓣ID
"""
# 查询历史记录
history: TransferHistory = self.transferhis.get(logid)
history: TransferHistory = TransferHistoryOper().get(logid)
if not history:
logger.error(f"整理记录不存在ID{logid}")
return False, "整理记录不存在"
@@ -1252,7 +1258,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 更新媒体图片
self.obtain_images(mediainfo=mediainfo)
else:
mediainfo = self.mediachain.recognize_by_path(str(src_path), episode_group=history.episode_group)
mediainfo = MediaChain().recognize_by_path(str(src_path), episode_group=history.episode_group)
if not mediainfo:
return False, f"未识别到媒体信息,类型:{mtype.value}id{mediaid}"
# 重新执行整理
@@ -1262,7 +1268,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if history.dest_fileitem:
# 解析目标文件对象
dest_fileitem = FileItem(**history.dest_fileitem)
self.storagechain.delete_file(dest_fileitem)
StorageChain().delete_file(dest_fileitem)
# 强制整理
if history.src_fileitem:
@@ -1317,18 +1323,19 @@ class TransferChain(ChainBase, metaclass=Singleton):
if tmdbid or doubanid:
# 有输入TMDBID时单个识别
# 识别媒体信息
mediainfo: MediaInfo = self.mediachain.recognize_media(tmdbid=tmdbid, doubanid=doubanid,
mtype=mtype, episode_group=episode_group)
mediainfo: MediaInfo = MediaChain().recognize_media(tmdbid=tmdbid, doubanid=doubanid,
mtype=mtype, episode_group=episode_group)
if not mediainfo:
return False, f"媒体信息识别失败tmdbid{tmdbid}doubanid{doubanid}type: {mtype.value}"
else:
# 更新媒体图片
self.obtain_images(mediainfo=mediainfo)
# 开始进度
self.progress.start(ProgressKey.FileTransfer)
self.progress.update(value=0,
text=f"开始整理 {fileitem.path} ...",
key=ProgressKey.FileTransfer)
progress = ProgressHelper()
progress.start(ProgressKey.FileTransfer)
progress.update(value=0,
text=f"开始整理 {fileitem.path} ...",
key=ProgressKey.FileTransfer)
# 开始整理
state, errmsg = self.do_transfer(
fileitem=fileitem,
@@ -1349,7 +1356,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
if not state:
return False, errmsg
self.progress.end(ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
logger.info(f"{fileitem.path} 整理完成")
return True, ""
else:
@@ -1370,7 +1377,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
return state, errmsg
def send_transfer_message(self, meta: MetaBase, mediainfo: MediaInfo,
transferinfo: TransferInfo, season_episode: Optional[str] = None, username: Optional[str] = None):
transferinfo: TransferInfo, season_episode: Optional[str] = None,
username: Optional[str] = None):
"""
发送入库成功的消息
"""

13
app/chain/tvdb.py Normal file
View File

@@ -0,0 +1,13 @@
from typing import List
from app.chain import ChainBase
class TvdbChain(ChainBase):
"""
Tvdb处理链单例运行
"""
def get_tvdbid_by_name(self, title: str) -> List[int]:
tvdb_info_list = self.run_module("search_tvdb", title=title)
return [int(item["tvdb_id"]) for item in tvdb_info_list]

View File

@@ -10,20 +10,15 @@ from app.log import logger
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import ChainEventType
from app.utils.otp import OtpUtils
from app.utils.singleton import Singleton
PASSWORD_INVALID_CREDENTIALS_MESSAGE = "用户名或密码或二次校验码不正确"
class UserChain(ChainBase, metaclass=Singleton):
class UserChain(ChainBase):
"""
用户链,处理多种认证协议
"""
def __init__(self):
super().__init__()
self.user_oper = UserOper()
def user_authenticate(
self,
username: Optional[str] = None,
@@ -90,7 +85,8 @@ class UserChain(ChainBase, metaclass=Singleton):
logger.debug(f"辅助认证未启用,认证类型 {grant_type} 未实现")
return False, "不支持的认证类型"
def password_authenticate(self, credentials: AuthCredentials) -> Tuple[bool, Union[User, str]]:
@staticmethod
def password_authenticate(credentials: AuthCredentials) -> Tuple[bool, Union[User, str]]:
"""
密码认证
@@ -103,7 +99,7 @@ class UserChain(ChainBase, metaclass=Singleton):
logger.info("密码认证失败,认证类型不匹配")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
user = self.user_oper.get_by_name(name=credentials.username)
user = UserOper().get_by_name(name=credentials.username)
if not user:
logger.info(f"密码认证失败,用户 {credentials.username} 不存在")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -131,8 +127,9 @@ class UserChain(ChainBase, metaclass=Singleton):
return False, "认证凭证无效"
# 检查是否因为用户被禁用
useroper = UserOper()
if credentials.username:
user = self.user_oper.get_by_name(name=credentials.username)
user = useroper.get_by_name(name=credentials.username)
if user and not user.is_active:
logger.info(f"用户 {user.name} 已被禁用,跳过后续身份校验")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -156,7 +153,7 @@ class UserChain(ChainBase, metaclass=Singleton):
success = self._process_auth_success(username=credentials.username, credentials=credentials)
if success:
logger.info(f"用户 {credentials.username} 辅助认证通过")
return True, self.user_oper.get_by_name(credentials.username)
return True, useroper.get_by_name(credentials.username)
else:
logger.warning(f"用户 {credentials.username} 辅助认证未通过")
return False, PASSWORD_INVALID_CREDENTIALS_MESSAGE
@@ -213,7 +210,8 @@ class UserChain(ChainBase, metaclass=Singleton):
return False
# 检查用户是否存在,如果不存在且当前为密码认证时则创建新用户
user = self.user_oper.get_by_name(name=username)
useroper = UserOper()
user = useroper.get_by_name(name=username)
if user:
# 如果用户存在,但是已经被禁用,则直接响应
if not user.is_active:
@@ -226,8 +224,8 @@ class UserChain(ChainBase, metaclass=Singleton):
return True
else:
if credentials.grant_type == "password":
self.user_oper.add(name=username, is_active=True, is_superuser=False,
hashed_password=get_password_hash(secrets.token_urlsafe(16)))
useroper.add(name=username, is_active=True, is_superuser=False,
hashed_password=get_password_hash(secrets.token_urlsafe(16)))
logger.info(f"用户 {username} 不存在,已通过 {credentials.grant_type} 认证并已创建普通用户")
return True
else:

View File

@@ -2,10 +2,9 @@ from typing import Any
from app.chain import ChainBase
from app.schemas.types import EventType
from app.utils.singleton import Singleton
class WebhookChain(ChainBase, metaclass=Singleton):
class WebhookChain(ChainBase):
"""
Webhook处理链
"""

View File

@@ -188,16 +188,14 @@ class WorkflowChain(ChainBase):
工作流链
"""
def __init__(self):
super().__init__()
self.workflowoper = WorkflowOper()
def process(self, workflow_id: int, from_begin: Optional[bool] = True) -> Tuple[bool, str]:
@staticmethod
def process(workflow_id: int, from_begin: Optional[bool] = True) -> Tuple[bool, str]:
"""
处理工作流
:param workflow_id: 工作流ID
:param from_begin: 是否从头开始默认为True
"""
workflowoper = WorkflowOper()
def save_step(action: Action, context: ActionContext):
"""
@@ -207,16 +205,16 @@ class WorkflowChain(ChainBase):
serialized_data = pickle.dumps(context)
# 使用Base64编码字节流
encoded_data = base64.b64encode(serialized_data).decode('utf-8')
self.workflowoper.step(workflow_id, action_id=action.id, context={
workflowoper.step(workflow_id, action_id=action.id, context={
"content": encoded_data
})
# 重置工作流
if from_begin:
self.workflowoper.reset(workflow_id)
workflowoper.reset(workflow_id)
# 查询工作流数据
workflow = self.workflowoper.get(workflow_id)
workflow = workflowoper.get(workflow_id)
if not workflow:
logger.warn(f"工作流 {workflow_id} 不存在")
return False, "工作流不存在"
@@ -228,7 +226,7 @@ class WorkflowChain(ChainBase):
return False, "工作流无流程"
logger.info(f"开始处理 {workflow.name},共 {len(workflow.actions)} 个动作 ...")
self.workflowoper.start(workflow_id)
workflowoper.start(workflow_id)
# 执行工作流
executor = WorkflowExecutor(workflow, step_callback=save_step)
@@ -236,15 +234,16 @@ class WorkflowChain(ChainBase):
if not executor.success:
logger.info(f"工作流 {workflow.name} 执行失败:{executor.errmsg}")
self.workflowoper.fail(workflow_id, result=executor.errmsg)
workflowoper.fail(workflow_id, result=executor.errmsg)
return False, executor.errmsg
else:
logger.info(f"工作流 {workflow.name} 执行完成")
self.workflowoper.success(workflow_id)
workflowoper.success(workflow_id)
return True, ""
def get_workflows(self) -> List[Workflow]:
@staticmethod
def get_workflows() -> List[Workflow]:
"""
获取工作流列表
"""
return self.workflowoper.list_enabled()
return WorkflowOper().list_enabled()

View File

@@ -1,7 +1,6 @@
import copy
import json
import os
import re
import secrets
import sys
import threading
@@ -25,7 +24,7 @@ class ConfigModel(BaseModel):
extra = "ignore" # 忽略未定义的配置项
# 项目名称
PROJECT_NAME = "MoviePilot"
PROJECT_NAME: str = "MoviePilot"
# 域名 格式https://movie-pilot.org
APP_DOMAIN: str = ""
# API路径
@@ -86,7 +85,7 @@ class ConfigModel(BaseModel):
AUXILIARY_AUTH_ENABLE: bool = False
# API密钥需要更换
API_TOKEN: Optional[str] = None
# 网络代理 IP:PORT
# 网络代理服务器地址
PROXY_HOST: Optional[str] = None
# 登录页面电影海报,tmdb/bing/mediaserver
WALLPAPER: str = "tmdb"
@@ -111,7 +110,8 @@ class ConfigModel(BaseModel):
# TMDB API Key
TMDB_API_KEY: str = "db55323b8d3e4154498498a75642b381"
# TVDB API Key
TVDB_API_KEY: str = "6b481081-10aa-440c-99f2-21d17717ee02"
TVDB_V4_API_KEY: str = "ed2aa66b-7899-4677-92a7-67bc9ce3d93a"
TVDB_V4_API_PIN: str = ""
# Fanart开关
FANART_ENABLE: bool = True
# Fanart API Key
@@ -123,9 +123,11 @@ class ConfigModel(BaseModel):
# 元数据识别缓存过期时间(小时)
META_CACHE_EXPIRE: int = 0
# 电视剧动漫的分类genre_ids
ANIME_GENREIDS = [16]
ANIME_GENREIDS: List[int] = [16]
# 用户认证站点
AUTH_SITE: str = ""
# 重启自动升级
MOVIEPILOT_AUTO_UPDATE: str = 'release'
# 自动检查和更新站点资源包(站点索引、认证等)
AUTO_UPDATE_RESOURCE: bool = True
# 是否启用DOH解析域名
@@ -201,7 +203,7 @@ class ConfigModel(BaseModel):
# CookieCloud同步黑名单多个域名,分割
COOKIECLOUD_BLACKLIST: Optional[str] = None
# CookieCloud对应的浏览器UA
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
USER_AGENT: str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57"
# 电影重命名格式
MOVIE_RENAME_FORMAT: str = "{{title}}{% if year %} ({{year}}){% endif %}" \
"/{{title}}{% if year %} ({{year}}){% endif %}{% if part %}-{{part}}{% endif %}{% if videoFormat %} - {{videoFormat}}{% endif %}" \
@@ -229,7 +231,8 @@ class ConfigModel(BaseModel):
"https://github.com/baozaodetudou/MoviePilot-Plugins,"
"https://github.com/Aqr-K/MoviePilot-Plugins,"
"https://github.com/hotlcc/MoviePilot-Plugins-Third,"
"https://github.com/gxterry/MoviePilot-Plugins")
"https://github.com/gxterry/MoviePilot-Plugins,"
"https://github.com/DzAvril/MoviePilot-Plugins")
# 插件安装数据共享
PLUGIN_STATISTIC_SHARE: bool = True
# 是否开启插件热加载
@@ -251,34 +254,34 @@ class ConfigModel(BaseModel):
# 编码探测的最低置信度阈值
ENCODING_DETECTION_MIN_CONFIDENCE: float = 0.8
# 允许的图片缓存域名
SECURITY_IMAGE_DOMAINS: List[str] = Field(
default_factory=lambda: ["image.tmdb.org",
"static-mdb.v.geilijiasu.com",
"bing.com",
"doubanio.com",
"lain.bgm.tv",
"raw.githubusercontent.com",
"github.com",
"thetvdb.com",
"cctvpic.com",
"iqiyipic.com",
"hdslb.com",
"cmvideo.cn",
"ykimg.com",
"qpic.cn"]
)
SECURITY_IMAGE_DOMAINS: list = Field(default=[
"image.tmdb.org",
"static-mdb.v.geilijiasu.com",
"bing.com",
"doubanio.com",
"lain.bgm.tv",
"raw.githubusercontent.com",
"github.com",
"thetvdb.com",
"cctvpic.com",
"iqiyipic.com",
"hdslb.com",
"cmvideo.cn",
"ykimg.com",
"qpic.cn"
])
# 允许的图片文件后缀格式
SECURITY_IMAGE_SUFFIXES: List[str] = Field(
default_factory=lambda: [".jpg", ".jpeg", ".png", ".webp", ".gif", ".svg", ".avif"]
)
SECURITY_IMAGE_SUFFIXES: list = Field(default=[".jpg", ".jpeg", ".png", ".webp", ".gif", ".svg", ".avif"])
# 重命名时支持的S0别名
RENAME_FORMAT_S0_NAMES: List[str] = Field(
default_factory=lambda: ["Specials", "SPs"]
)
RENAME_FORMAT_S0_NAMES: list = Field(default=["Specials", "SPs"])
# 启用分词搜索
TOKENIZED_SEARCH: bool = False
# 为指定默认字幕添加.default后缀
DEFAULT_SUB: Optional[str] = "zh-cn"
# Docker Client API地址
DOCKER_CLIENT_API: Optional[str] = "tcp://127.0.0.1:38379"
# 剧集交集最小置信度 计算后的交集比例( len(torrent_episodes ∩ need_episodes) / len(torrent_episodes) 低于这个阈值表明包含过多不需要的剧集
EPISODE_INTERSECTION_MIN_CONFIDENCE: float = 0.0
class Settings(BaseSettings, ConfigModel, LogConfigModel):
@@ -325,6 +328,7 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
raise_exception: bool = False) -> Tuple[Any, bool]:
"""
通用类型转换函数,根据预期类型转换值。如果转换失败,返回默认值
:return: 元组 (转换后的值, 是否需要更新)
"""
if isinstance(value, (list, dict, set)):
value = copy.deepcopy(value)
@@ -365,12 +369,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
converted = float(value)
return converted, str(converted) != str(original_value)
elif expected_type is str:
# 清理 value 中所有空白字符的字段
fields_not_keep_spaces = {"AUTO_DOWNLOAD_USER", "REPO_GITHUB_TOKEN", "PLUGIN_MARKET"}
if field_name in fields_not_keep_spaces:
value = re.sub(r"\s+", "", value)
return value, str(value) != str(original_value)
# 支持 list 类型的处理
converted = str(value).strip()
return converted, converted != str(original_value)
elif expected_type is list:
if isinstance(value, list):
return value, str(value) != str(original_value)
@@ -380,7 +380,6 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return items, items != original_value
else:
return items, str(items) != str(original_value)
# 可根据需要添加更多类型处理
else:
return value, str(value) != str(original_value)
except (ValueError, TypeError) as e:
@@ -426,14 +425,18 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
else:
value_to_write = str(converted_value) if converted_value is not None else ""
set_key(SystemUtils.get_env_path(), field.name, value_to_write)
set_key(dotenv_path=SystemUtils.get_env_path(), key_to_set=field.name, value_to_set=value_to_write,
quote_mode="always")
if is_converted:
logger.info(f"配置项 '{field.name}' 已自动修正并写入到 'app.env' 文件")
return True, message
def update_setting(self, key: str, value: Any) -> Tuple[bool, str]:
def update_setting(self, key: str, value: Any) -> Tuple[Optional[bool], str]:
"""
更新单个配置项
:param key: 配置项的名称
:param value: 配置项的新值
:return: (是否成功 True 成功/False 失败/None 无需更新, 错误信息)
"""
if not hasattr(self, key):
return False, f"配置项 '{key}' 不存在"
@@ -444,8 +447,11 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
if field.name == "API_TOKEN":
converted_value, needs_update = self.validate_api_token(value, original_value)
else:
converted_value, needs_update = self.generic_type_converter(value, original_value, field.type_,
field.default, key)
converted_value, needs_update = self.generic_type_converter(value,
original_value,
field.type_,
field.default,
key)
# 如果没有抛出异常,则统一使用 converted_value 进行更新
if needs_update or str(value) != str(converted_value):
success, message = self.update_env_config(field, value, converted_value)
@@ -455,30 +461,17 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
if hasattr(log_settings, key):
setattr(log_settings, key, converted_value)
return success, message
return True, ""
return None, ""
except Exception as e:
return False, str(e)
def update_settings(self, env: Dict[str, Any]) -> Dict[str, Tuple[bool, str]]:
def update_settings(self, env: Dict[str, Any]) -> Dict[str, Tuple[Optional[bool], str]]:
"""
更新多个配置项
"""
results = {}
log_updated, plugin_monitor_updated = False, False
for k, v in env.items():
results[k] = self.update_setting(k, v)
if hasattr(log_settings, k):
log_updated = True
if k in ["PLUGIN_AUTO_RELOAD", "DEV"]:
plugin_monitor_updated = True
# 本次更新存在日志配置项更新,需要重新加载日志配置
if log_updated:
logger.update_loggers()
# 本次更新存在插件监控配置项更新,需要重新加载插件监控
if plugin_monitor_updated:
# 解决顶层循环导入问题
from app.core.plugin import PluginManager
PluginManager().reload_monitor()
return results
@property
@@ -535,7 +528,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"tmdb": "TMDB请求缓存数量",
"douban": "豆瓣请求缓存数量",
"fanart": "Fanart请求缓存数量",
"meta": "元数据缓存过期时间(秒)"
"meta": "元数据缓存过期时间(秒)",
"memory": "最大占用内存MB"
}
"""
if self.BIG_MEMORY_MODE:
@@ -546,7 +540,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"douban": 512,
"bangumi": 512,
"fanart": 512,
"meta": (self.META_CACHE_EXPIRE or 24) * 3600
"meta": (self.META_CACHE_EXPIRE or 24) * 3600,
"memory": 2 * 1024
}
return {
"torrents": 100,
@@ -555,7 +550,8 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
"douban": 256,
"bangumi": 256,
"fanart": 128,
"meta": (self.META_CACHE_EXPIRE or 2) * 3600
"meta": (self.META_CACHE_EXPIRE or 2) * 3600,
"memory": 1024
}
@property
@@ -631,6 +627,10 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return UrlUtils.combine_url(host=self.APP_DOMAIN, path=url)
# 实例化配置
settings = Settings()
class GlobalVar(object):
"""
全局标识
@@ -688,8 +688,5 @@ class GlobalVar(object):
return self.is_system_stopped or workflow_id in self.EMERGENCY_STOP_WORKFLOWS
# 实例化配置
settings = Settings()
# 全局标识
global_vars = GlobalVar()

View File

@@ -10,7 +10,6 @@ from functools import lru_cache
from queue import Empty, PriorityQueue
from typing import Callable, Dict, List, Optional, Union
from app.helper.message import MessageHelper
from app.helper.thread import ThreadHelper
from app.log import logger
from app.schemas import ChainEventData
@@ -75,7 +74,6 @@ class EventManager(metaclass=Singleton):
__event = threading.Event()
def __init__(self):
self.__messagehelper = MessageHelper()
self.__executor = ThreadHelper() # 动态线程池,用于消费事件
self.__consumer_threads = [] # 用于保存启动的事件消费者线程
self.__event_queue = PriorityQueue() # 优先级队列
@@ -140,11 +138,12 @@ class EventManager(metaclass=Singleton):
"""
event = Event(etype, data, priority)
if isinstance(etype, EventType):
self.__trigger_broadcast_event(event)
return self.__trigger_broadcast_event(event)
elif isinstance(etype, ChainEventType):
return self.__trigger_chain_event(event)
else:
logger.error(f"Unknown event type: {etype}")
return None
def add_event_listener(self, event_type: Union[EventType, ChainEventType], handler: Callable,
priority: Optional[int] = DEFAULT_EVENT_PRIORITY):
@@ -293,7 +292,7 @@ class EventManager(metaclass=Singleton):
# 对于类实例(实现了 __call__ 方法)
if not inspect.isfunction(handler) and hasattr(handler, "__call__"):
handler_cls = handler.__class__ # noqa
handler_cls = handler.__class__ # noqa
return cls.__get_handler_identifier(handler_cls)
# 对于未绑定方法、静态方法、类方法,使用 __qualname__ 提取类信息
@@ -303,6 +302,7 @@ class EventManager(metaclass=Singleton):
module = inspect.getmodule(handler)
module_name = module.__name__ if module else "unknown_module"
return f"{module_name}.{class_name}"
return None
def __is_handler_enabled(self, handler: Callable) -> bool:
"""
@@ -398,16 +398,28 @@ class EventManager(metaclass=Singleton):
try:
from app.core.plugin import PluginManager
from app.core.module import ModuleManager
if class_name in PluginManager().get_plugin_ids():
# 定义一个插件调用函数
def plugin_callable():
"""
插件调用函数
"""
PluginManager().run_plugin_method(class_name, method_name, event_to_process)
if is_broadcast_event:
self.__executor.submit(plugin_callable)
else:
plugin_callable()
elif class_name in ModuleManager().get_module_ids():
module = ModuleManager().get_running_module(class_name)
if module:
method = getattr(module, method_name, None)
if method:
if is_broadcast_event:
self.__executor.submit(method, event_to_process)
else:
method(event_to_process)
else:
# 获取全局对象或模块类的实例
class_obj = self.__get_class_instance(class_name)
@@ -438,15 +450,15 @@ class EventManager(metaclass=Singleton):
# 如果类不在全局变量中,尝试动态导入模块并创建实例
try:
if class_name == "Command":
module_name = "app.command"
if class_name.endswith("Manager"):
module_name = f"app.core.{class_name[:-7].lower()}"
module = importlib.import_module(module_name)
elif class_name.endswith("Chain"):
module_name = f"app.chain.{class_name[:-5].lower()}"
module = importlib.import_module(module_name)
else:
logger.debug(f"事件处理出错:无效的 Chain 类名: {class_name},类名必须以 'Chain' 结尾")
return None
module_name = f"app.{class_name.lower()}"
module = importlib.import_module(module_name)
if hasattr(module, class_name):
class_obj = getattr(module, class_name)()
return class_obj
@@ -491,9 +503,11 @@ class EventManager(metaclass=Singleton):
names = handler.__qualname__.split(".")
class_name, method_name = names[0], names[1]
self.__messagehelper.put(title=f"{event.event_type} 事件处理出错",
message=f"{class_name}.{method_name}{str(e)}",
role="system")
# 发送系统错误通知
from app.helper.message import MessageHelper
MessageHelper().put(title=f"{event.event_type} 事件处理出错",
message=f"{class_name}.{method_name}{str(e)}",
role="system")
self.send_event(
EventType.SystemError,
{

View File

@@ -582,6 +582,12 @@ class MetaBase(object):
# Part
if not self.part:
self.part = meta.part
# tmdbid
if not self.tmdbid and meta.tmdbid:
self.tmdbid = meta.tmdbid
# doubanid
if not self.doubanid and meta.doubanid:
self.doubanid = meta.doubanid
def to_dict(self):
"""

View File

@@ -31,7 +31,7 @@ class MetaVideo(MetaBase):
_part_re = r"(^PART[0-9ABI]{0,2}$|^CD[0-9]{0,2}$|^DVD[0-9]{0,2}$|^DISK[0-9]{0,2}$|^DISC[0-9]{0,2}$)"
_roman_numerals = r"^(?=[MDCLXVI])M*(C[MD]|D?C{0,3})(X[CL]|L?X{0,3})(I[XV]|V?I{0,3})$"
_source_re = r"^BLURAY$|^HDTV$|^UHDTV$|^HDDVD$|^WEBRIP$|^DVDRIP$|^BDRIP$|^BLU$|^WEB$|^BD$|^HDRip$|^REMUX$|^UHD$"
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$"
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$"
_resources_type_re = r"%s|%s" % (_source_re, _effect_re)
_name_no_begin_re = r"^[\[【].+?[\]】]"
_name_no_chinese_re = r".*版|.*字幕"
@@ -592,12 +592,12 @@ class MetaVideo(MetaBase):
self._stop_name_flag = True
self._last_token_type = "videoencode"
if not self.video_encode:
if re_res.group(1):
self.video_encode = re_res.group(1).upper()
elif re_res.group(2):
self.video_encode = re_res.group(2).lower()
if re_res.group(2):
self.video_encode = re_res.group(2).upper()
elif re_res.group(3):
self.video_encode = re_res.group(3).lower()
else:
self.video_encode = re_res.group(0).upper()
self.video_encode = re_res.group(1).upper()
self._last_token = self.video_encode
elif self.video_encode == "10bit":
self.video_encode = f"{re_res.group(1).upper()} 10bit"

View File

@@ -81,7 +81,6 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
}
def __init__(self):
self.systemconfig = SystemConfigOper()
release_groups = []
for site_groups in self.RELEASE_GROUPS.values():
for release_group in site_groups:
@@ -98,7 +97,7 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
return ""
if not groups:
# 自定义组
custom_release_groups = self.systemconfig.get(SystemConfigKey.CustomReleaseGroups)
custom_release_groups = SystemConfigOper().get(SystemConfigKey.CustomReleaseGroups)
if isinstance(custom_release_groups, list):
custom_release_groups = list(filter(None, custom_release_groups))
if custom_release_groups:

View File

@@ -120,41 +120,69 @@ def find_metainfo(title: str) -> Tuple[str, dict]:
return title, metainfo
# 从标题中提取媒体信息 格式为{[tmdbid=xxx;type=xxx;s=xxx;e=xxx]}
results = re.findall(r'(?<={\[)[\W\w]+(?=]})', title)
if not results:
return title, metainfo
for result in results:
# 查找tmdbid信息
tmdbid = re.findall(r'(?<=tmdbid=)\d+', result)
if tmdbid and tmdbid[0].isdigit():
metainfo['tmdbid'] = tmdbid[0]
# 查找豆瓣id信息
doubanid = re.findall(r'(?<=doubanid=)\d+', result)
if doubanid and doubanid[0].isdigit():
metainfo['doubanid'] = doubanid[0]
# 查找媒体类型
mtype = re.findall(r'(?<=type=)\w+', result)
if mtype:
if mtype[0] == "movies":
metainfo['type'] = MediaType.MOVIE
elif mtype[0] == "tv":
metainfo['type'] = MediaType.TV
# 查找季信息
begin_season = re.findall(r'(?<=s=)\d+', result)
if begin_season and begin_season[0].isdigit():
metainfo['begin_season'] = int(begin_season[0])
end_season = re.findall(r'(?<=s=\d+-)\d+', result)
if end_season and end_season[0].isdigit():
metainfo['end_season'] = int(end_season[0])
# 查找集信息
begin_episode = re.findall(r'(?<=e=)\d+', result)
if begin_episode and begin_episode[0].isdigit():
metainfo['begin_episode'] = int(begin_episode[0])
end_episode = re.findall(r'(?<=e=\d+-)\d+', result)
if end_episode and end_episode[0].isdigit():
metainfo['end_episode'] = int(end_episode[0])
# 去除title中该部分
if tmdbid or mtype or begin_season or end_season or begin_episode or end_episode:
title = title.replace(f"{{[{result}]}}", '')
if results:
for result in results:
# 查找tmdbid信息
tmdbid = re.findall(r'(?<=tmdbid=)\d+', result)
if tmdbid and tmdbid[0].isdigit():
metainfo['tmdbid'] = tmdbid[0]
# 查找豆瓣id信息
doubanid = re.findall(r'(?<=doubanid=)\d+', result)
if doubanid and doubanid[0].isdigit():
metainfo['doubanid'] = doubanid[0]
# 查找媒体类型
mtype = re.findall(r'(?<=type=)\w+', result)
if mtype:
if mtype[0] == "movies":
metainfo['type'] = MediaType.MOVIE
elif mtype[0] == "tv":
metainfo['type'] = MediaType.TV
# 查找季信息
begin_season = re.findall(r'(?<=s=)\d+', result)
if begin_season and begin_season[0].isdigit():
metainfo['begin_season'] = int(begin_season[0])
end_season = re.findall(r'(?<=s=\d+-)\d+', result)
if end_season and end_season[0].isdigit():
metainfo['end_season'] = int(end_season[0])
# 查找集信息
begin_episode = re.findall(r'(?<=e=)\d+', result)
if begin_episode and begin_episode[0].isdigit():
metainfo['begin_episode'] = int(begin_episode[0])
end_episode = re.findall(r'(?<=e=\d+-)\d+', result)
if end_episode and end_episode[0].isdigit():
metainfo['end_episode'] = int(end_episode[0])
# 去除title中该部分
if tmdbid or mtype or begin_season or end_season or begin_episode or end_episode:
title = title.replace(f"{{[{result}]}}", '')
# 支持Emby格式的ID标签
# 1. [tmdbid=xxxx] 或 [tmdbid-xxxx] 格式
tmdb_match = re.search(r'\[tmdbid[=\-](\d+)\]', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\[tmdbid[=\-](\d+)\]', '', title).strip()
# 2. [tmdb=xxxx] 或 [tmdb-xxxx] 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\[tmdb[=\-](\d+)\]', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\[tmdb[=\-](\d+)\]', '', title).strip()
# 3. {tmdbid=xxxx} 或 {tmdbid-xxxx} 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\{tmdbid[=\-](\d+)\}', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\{tmdbid[=\-](\d+)\}', '', title).strip()
# 4. {tmdb=xxxx} 或 {tmdb-xxxx} 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\{tmdb[=\-](\d+)\}', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\{tmdb[=\-](\d+)\}', '', title).strip()
# 计算季集总数
if metainfo.get('begin_season') and metainfo.get('end_season'):
if metainfo['begin_season'] > metainfo['end_season']:

View File

@@ -1,5 +1,5 @@
import traceback
from typing import Generator, Optional, Tuple, Any, Union
from typing import Generator, Optional, Tuple, Any, Union, List
from app.core.config import settings
from app.core.event import eventmanager
@@ -164,3 +164,9 @@ class ModuleManager(metaclass=Singleton):
获取模块列表
"""
return self._modules
def get_module_ids(self) -> List[str]:
"""
获取模块id列表
"""
return list(self._modules.keys())

View File

@@ -16,7 +16,7 @@ from watchdog.observers import Observer
from app import schemas
from app.core.config import settings
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.db.plugindata_oper import PluginDataOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.module import ModuleHelper
@@ -87,7 +87,6 @@ class PluginManager(metaclass=Singleton):
"""
插件管理器
"""
systemconfig: SystemConfigOper = None
# 插件列表
_plugins: dict = {}
@@ -99,10 +98,6 @@ class PluginManager(metaclass=Singleton):
_observer: Observer = None
def __init__(self):
self.siteshelper = SitesHelper()
self.pluginhelper = PluginHelper()
self.systemconfig = SystemConfigOper()
self.plugindata = PluginDataOper()
# 开发者模式监测插件修改
if settings.DEV or settings.PLUGIN_AUTO_RELOAD:
self.__start_monitor()
@@ -141,7 +136,7 @@ class PluginManager(metaclass=Singleton):
filter_func=lambda _, obj: check_module(obj)
)
# 已安装插件
installed_plugins = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 排序
plugins.sort(key=lambda x: x.plugin_order if hasattr(x, "plugin_order") else 0)
for plugin in plugins:
@@ -226,13 +221,34 @@ class PluginManager(metaclass=Singleton):
logger.info("插件停止完成")
@property
def running_plugins(self):
def running_plugins(self) -> Dict[str, Any]:
"""
获取运行态插件列表
:return: 运行态插件列表
"""
return self._running_plugins
@property
def plugins(self) -> Dict[str, Any]:
"""
获取插件列表
:return: 插件列表
"""
return self._plugins
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in ['DEV', 'PLUGIN_AUTO_RELOAD']:
return
self.reload_monitor()
def reload_monitor(self):
"""
重新加载插件文件修改监测
@@ -274,12 +290,15 @@ class PluginManager(metaclass=Singleton):
停止插件
:param plugin: 插件实例
"""
# 关闭数据库
if hasattr(plugin, "close"):
plugin.close()
# 关闭插件
if hasattr(plugin, "stop_service"):
plugin.stop_service()
try:
# 关闭数据库
if hasattr(plugin, "close"):
plugin.close()
# 关闭插件
if hasattr(plugin, "stop_service"):
plugin.stop_service()
except Exception as e:
logger.warn(f"停止插件 {plugin.get_name()} 时发生错误: {str(e)}")
def remove_plugin(self, plugin_id: str):
"""
@@ -307,7 +326,7 @@ class PluginManager(metaclass=Singleton):
def install_plugin(plugin):
start_time = time.time()
state, msg = self.pluginhelper.install(pid=plugin.id, repo_url=plugin.repo_url, force_install=True)
state, msg = PluginHelper().install(pid=plugin.id, repo_url=plugin.repo_url, force_install=True)
elapsed_time = time.time() - start_time
if state:
logger.info(
@@ -322,7 +341,7 @@ class PluginManager(metaclass=Singleton):
return []
# 获取已安装插件列表
install_plugins = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
install_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 获取在线插件列表
online_plugins = self.get_online_plugins()
# 确定需要安装的插件
@@ -358,19 +377,21 @@ class PluginManager(metaclass=Singleton):
)
return sync_plugins
def install_plugin_missing_dependencies(self) -> List[str]:
@staticmethod
def install_plugin_missing_dependencies() -> List[str]:
"""
安装插件中缺失或不兼容的依赖项
"""
pluginhelper = PluginHelper()
# 第一步:获取需要安装的依赖项列表
missing_dependencies = self.pluginhelper.find_missing_dependencies()
missing_dependencies = pluginhelper.find_missing_dependencies()
if not missing_dependencies:
return missing_dependencies
logger.debug(f"检测到缺失的依赖项: {missing_dependencies}")
logger.info(f"开始安装缺失的依赖项,共 {len(missing_dependencies)} 个...")
# 第二步:安装依赖项并返回结果
total_start_time = time.time()
success, message = self.pluginhelper.install_dependencies(missing_dependencies)
success, message = pluginhelper.install_dependencies(missing_dependencies)
total_elapsed_time = time.time() - total_start_time
if success:
logger.info(f"已完成 {len(missing_dependencies)} 个依赖项安装,总耗时:{total_elapsed_time:.2f}")
@@ -385,7 +406,7 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return {}
conf = self.systemconfig.get(self._config_key % pid)
conf = SystemConfigOper().get(self._config_key % pid)
if conf:
# 去掉空Key
return {k: v for k, v in conf.items() if k}
@@ -399,7 +420,8 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return False
return self.systemconfig.set(self._config_key % pid, conf)
SystemConfigOper().set(self._config_key % pid, conf)
return True
def delete_plugin_config(self, pid: str) -> bool:
"""
@@ -408,7 +430,7 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return False
return self.systemconfig.delete(self._config_key % pid)
return SystemConfigOper().delete(self._config_key % pid)
def delete_plugin_data(self, pid: str) -> bool:
"""
@@ -417,7 +439,7 @@ class PluginManager(metaclass=Singleton):
"""
if not self._plugins.get(pid):
return False
self.plugindata.del_data(pid)
PluginDataOper().del_data(pid)
return True
def get_plugin_state(self, pid: str) -> bool:
@@ -779,7 +801,7 @@ class PluginManager(metaclass=Singleton):
# 返回值
plugins = []
# 已安装插件
installed_apps = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_apps = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
for pid, plugin_class in self._plugins.items():
# 运行状插件
plugin_obj = self._running_plugins.get(pid)
@@ -876,9 +898,9 @@ class PluginManager(metaclass=Singleton):
if not market:
return []
# 已安装插件
installed_apps = self.systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
installed_apps = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 获取在线插件
online_plugins = self.pluginhelper.get_plugins(market, package_version)
online_plugins = PluginHelper().get_plugins(market, package_version)
if online_plugins is None:
logger.warning(
f"获取{package_version if package_version else ''}插件库失败:{market},请检查 GitHub 网络连接")
@@ -965,7 +987,8 @@ class PluginManager(metaclass=Singleton):
return ret_plugins
def __set_and_check_auth_level(self, plugin: Union[schemas.Plugin, Type[Any]],
@staticmethod
def __set_and_check_auth_level(plugin: Union[schemas.Plugin, Type[Any]],
source: Optional[Union[dict, Type[Any]]] = None) -> bool:
"""
设置并检查插件的认证级别
@@ -989,7 +1012,8 @@ class PluginManager(metaclass=Singleton):
# 3 - 站点&密钥认证可见
# 99 - 站点&特殊密钥认证可见
# 如果当前站点认证级别大于 1 且插件级别为 99并存在插件公钥说明为特殊密钥认证通过密钥匹配进行认证
if self.siteshelper.auth_level > 1 and plugin.auth_level == 99 and hasattr(plugin, "plugin_public_key"):
siteshelper = SitesHelper()
if siteshelper.auth_level > 1 and plugin.auth_level == 99 and hasattr(plugin, "plugin_public_key"):
plugin_id = plugin.id if isinstance(plugin, schemas.Plugin) else plugin.__name__
public_key = plugin.plugin_public_key
if public_key:
@@ -997,7 +1021,7 @@ class PluginManager(metaclass=Singleton):
verify = RSAUtils.verify_rsa_keys(public_key=public_key, private_key=private_key)
return verify
# 如果当前站点认证级别小于插件级别,则返回 False
if self.siteshelper.auth_level < plugin.auth_level:
if siteshelper.auth_level < plugin.auth_level:
return False
return True
@@ -1016,3 +1040,334 @@ class PluginManager(metaclass=Singleton):
except Exception as e:
logger.debug(f"获取插件 {plugin_id} 的私钥时发生错误:{e}")
return None
def clone_plugin(self, plugin_id: str, suffix: str, name: str, description: str,
version: str = None, icon: str = None) -> Tuple[bool, str]:
"""
创建插件分身
:param plugin_id: 原插件ID
:param suffix: 分身后缀
:param name: 分身名称
:param description: 分身描述
:param version: 自定义版本号
:param icon: 自定义图标URL
:return: (是否成功, 错误信息)
"""
try:
# 验证参数
if not plugin_id or not suffix:
return False, "插件ID和分身后缀不能为空"
# 检查原插件是否存在
if plugin_id not in self._plugins:
return False, f"原插件 {plugin_id} 不存在"
# 生成分身插件ID
clone_id = f"{plugin_id}{suffix.lower()}"
# 检查分身插件是否已存在
if self.is_plugin_exists(clone_id):
return False, f"分身插件 {clone_id} 已存在"
# 获取原插件目录
original_plugin_dir = Path(settings.ROOT_PATH) / "app" / "plugins" / plugin_id.lower()
if not original_plugin_dir.exists():
return False, f"原插件目录 {original_plugin_dir} 不存在"
# 创建分身插件目录
clone_plugin_dir = Path(settings.ROOT_PATH) / "app" / "plugins" / clone_id.lower()
# 复制插件目录
import shutil
shutil.copytree(original_plugin_dir, clone_plugin_dir)
logger.info(f"已复制插件目录:{original_plugin_dir} -> {clone_plugin_dir}")
# 修改插件文件内容
success, msg = self._modify_plugin_files(
plugin_dir=clone_plugin_dir,
original_id=plugin_id,
suffix=suffix,
name=name,
description=description,
version=version,
icon=icon
)
if not success:
# 如果修改失败,清理已创建的目录
if clone_plugin_dir.exists():
shutil.rmtree(clone_plugin_dir)
return False, msg
# 将分身插件添加到已安装列表
systemconfig = SystemConfigOper()
installed_plugins = systemconfig.get(SystemConfigKey.UserInstalledPlugins) or []
if clone_id not in installed_plugins:
installed_plugins.append(clone_id)
systemconfig.set(SystemConfigKey.UserInstalledPlugins, installed_plugins)
# 为分身插件创建初始配置(从原插件复制配置)
logger.info(f"正在为分身插件 {clone_id} 创建初始配置...")
original_config = self.get_plugin_config(plugin_id)
if original_config:
# 复制原插件配置作为分身插件的初始配置
clone_config = original_config.copy()
# 可以在这里修改一些默认值,比如禁用分身插件
# 默认禁用分身插件,让用户手动配置
clone_config['enable'] = False
clone_config['enabled'] = False
self.save_plugin_config(clone_id, clone_config)
logger.info(f"已为分身插件 {clone_id} 设置初始配置")
else:
logger.info(f"原插件 {plugin_id} 没有配置,分身插件 {clone_id} 将使用默认配置")
# 注册分身插件的API和服务
logger.info(f"正在注册分身插件 {clone_id} ...")
PluginManager().reload_plugin(clone_id)
# 确保分身插件正确初始化配置
if clone_id in self._running_plugins:
clone_instance = self._running_plugins[clone_id]
clone_config = self.get_plugin_config(clone_id)
if clone_config:
logger.info(f"正在为分身插件 {clone_id} 重新初始化配置...")
clone_instance.init_plugin(clone_config)
logger.info(f"分身插件 {clone_id} 配置重新初始化完成")
logger.info(f"插件分身 {clone_id} 创建成功")
return True, clone_id
except Exception as e:
logger.error(f"创建插件分身失败:{str(e)}")
return False, f"创建插件分身失败:{str(e)}"
def _modify_plugin_files(self, plugin_dir: Path, original_id: str, suffix: str,
name: str, description: str, version: str = None,
icon: str = None) -> Tuple[bool, str]:
"""
修改插件文件中的类名和相关信息
:param plugin_dir: 插件目录
:param original_id: 原插件ID
:param suffix: 分身后缀
:param name: 分身名称
:param description: 分身描述
:param version: 自定义版本号
:param icon: 自定义图标URL
:return: (是否成功, 错误信息)
"""
try:
# 获取原插件类
original_plugin_class = self._plugins.get(original_id)
if not original_plugin_class:
return False, f"无法获取原插件类 {original_id}"
# 获取原类名
original_class_name = original_plugin_class.__name__
clone_class_name = f"{original_class_name}{suffix}"
# 修改 __init__.py 文件
init_file = plugin_dir / "__init__.py"
if init_file.exists():
success, msg = self._modify_python_file(
file_path=init_file,
original_class_name=original_class_name,
clone_class_name=clone_class_name,
name=name,
description=description,
version=version,
icon=icon
)
if not success:
return False, msg
# 检查是否为联邦插件存在dist目录
dist_dir = plugin_dir / "dist"
if dist_dir.exists():
success, msg = self._modify_federation_files(
dist_dir=dist_dir,
original_class_name=original_class_name,
clone_class_name=clone_class_name
)
if not success:
return False, msg
return True, "文件修改成功"
except Exception as e:
logger.error(f"修改插件文件失败:{str(e)}")
return False, f"修改插件文件失败:{str(e)}"
@staticmethod
def _modify_python_file(file_path: Path, original_class_name: str,
clone_class_name: str, name: str, description: str,
version: str = None, icon: str = None) -> Tuple[bool, str]:
"""
修改Python文件中的类名和插件信息
"""
try:
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
# 替换类名
content = content.replace(f"class {original_class_name}", f"class {clone_class_name}")
# 替换插件名称和描述
import re
# 替换 plugin_name
if name:
content = re.sub(
r'plugin_name\s*=\s*["\'][^"\']*["\']',
f'plugin_name = "{name}"',
content
)
# 替换 plugin_desc
if description:
content = re.sub(
r'plugin_desc\s*=\s*["\'][^"\']*["\']',
f'plugin_desc = "{description}"',
content
)
# 替换 plugin_config_prefix如果存在
content = re.sub(
r'plugin_config_prefix\s*=\s*["\'][^"\']*["\']',
f'plugin_config_prefix = "{clone_class_name.lower()}_"',
content
)
# 替换 plugin_version如果提供了自定义版本
if version:
content = re.sub(
r'plugin_version\s*=\s*["\'][^"\']*["\']',
f'plugin_version = "{version}"',
content
)
# 替换 plugin_icon如果提供了自定义图标
if icon and icon.strip():
old_content = content
content = re.sub(
r'plugin_icon\s*=\s*["\'][^"\']*["\']',
f'plugin_icon = "{icon}"',
content
)
if old_content != content:
logger.info(f"已替换插件图标为: {icon}")
else:
logger.warning(f"插件图标替换失败,未找到匹配的图标设置")
else:
logger.info("未提供自定义图标,保持原插件图标")
# 添加分身标志
if "def init_plugin(self" in content:
init_index = content.index("def init_plugin(self")
# 在 def init_plugin(self 前添加 is_clone = True
content = content[:init_index] + "is_clone = True\n\n " + content[init_index:]
with open(file_path, 'w', encoding='utf-8') as f:
f.write(content)
logger.debug(f"已修改Python文件{file_path}")
return True, "Python文件修改成功"
except Exception as e:
logger.error(f"修改Python文件失败{str(e)}")
return False, f"修改Python文件失败{str(e)}"
def _modify_federation_files(self, dist_dir: Path, original_class_name: str,
clone_class_name: str) -> Tuple[bool, str]:
"""
修改联邦插件的前端文件
"""
try:
# 获取原始插件名(从类名推导)
original_plugin_name = original_class_name
clone_plugin_name = clone_class_name
# 遍历dist目录下的所有文件
for file_path in dist_dir.rglob("*"):
if not file_path.is_file():
continue
# 处理JS文件
if file_path.suffix == '.js':
try:
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
# 替换类名引用(精确匹配)
content = content.replace(original_class_name, clone_class_name)
# 替换插件名引用(如果存在)
content = content.replace(f'"{original_plugin_name}"', f'"{clone_plugin_name}"')
content = content.replace(f"'{original_plugin_name}'", f"'{clone_plugin_name}'")
# 替换CSS key中的类名联邦插件特有
content = content.replace(f'css__{original_class_name}__', f'css__{clone_class_name}__')
# 替换可能的小写类名引用
content = content.replace(original_class_name.lower(), clone_class_name.lower())
with open(file_path, 'w', encoding='utf-8') as f:
f.write(content)
logger.debug(f"已修改联邦插件JS文件{file_path}")
except Exception as e:
logger.warning(f"修改联邦插件文件 {file_path} 失败:{str(e)}")
continue
# 处理CSS文件
elif file_path.suffix == '.css':
try:
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
# 替换CSS中可能的类名引用
content = content.replace(original_class_name.lower(), clone_class_name.lower())
content = content.replace(original_class_name, clone_class_name)
with open(file_path, 'w', encoding='utf-8') as f:
f.write(content)
logger.debug(f"已修改联邦插件CSS文件{file_path}")
except Exception as e:
logger.warning(f"修改联邦插件CSS文件 {file_path} 失败:{str(e)}")
continue
# 重命名构建文件(如果需要)
self._rename_federation_assets(dist_dir, original_class_name, clone_class_name)
return True, "联邦插件文件修改完成"
except Exception as e:
logger.error(f"修改联邦插件文件失败:{str(e)}")
return False, f"修改联邦插件文件失败:{str(e)}"
@staticmethod
def _rename_federation_assets(dist_dir: Path, original_class_name: str, clone_class_name: str):
"""
重命名联邦插件的资源文件,避免文件名冲突
"""
try:
# 查找包含原类名的文件并重命名
for file_path in dist_dir.glob("*"):
if not file_path.is_file():
continue
file_name = file_path.name
# 如果文件名包含原类名,则重命名
if original_class_name.lower() in file_name.lower():
new_name = file_name.replace(
original_class_name.lower(),
clone_class_name.lower()
)
new_path = file_path.parent / new_name
# 避免重命名冲突
if not new_path.exists():
file_path.rename(new_path)
logger.debug(f"重命名联邦插件文件:{file_name} -> {new_name}")
except Exception as e:
# 重命名失败不影响整体流程
logger.warning(f"重命名联邦插件资源文件失败:{str(e)}")

View File

@@ -1,4 +1,4 @@
from typing import Any, Union
from typing import Any, Union, Optional
from app.db import DbOper
from app.db.models.systemconfig import SystemConfig
@@ -18,23 +18,32 @@ class SystemConfigOper(DbOper, metaclass=Singleton):
for item in SystemConfig.list(self._db):
self.__SYSTEMCONF[item.key] = item.value
def set(self, key: Union[str, SystemConfigKey], value: Any):
def set(self, key: Union[str, SystemConfigKey], value: Any) -> Optional[bool]:
"""
设置系统设置
:param key: 配置键
:param value: 配置值
:return: 是否设置成功True 成功/False 失败/None 无需更新)
"""
if isinstance(key, SystemConfigKey):
key = key.value
# 旧值
old_value = self.__SYSTEMCONF.get(key)
# 更新内存
self.__SYSTEMCONF[key] = value
conf = SystemConfig.get_by_key(self._db, key)
if conf:
if value:
conf.update(self._db, {"value": value})
else:
conf.delete(self._db, conf.id)
if old_value != value:
if value:
conf.update(self._db, {"value": value})
else:
conf.delete(self._db, conf.id)
return True
return None
else:
conf = SystemConfig(key=key, value=value)
conf.create(self._db)
return True
def get(self, key: Union[str, SystemConfigKey] = None) -> Any:
"""
@@ -52,7 +61,7 @@ class SystemConfigOper(DbOper, metaclass=Singleton):
"""
return self.__SYSTEMCONF or {}
def delete(self, key: Union[str, SystemConfigKey]):
def delete(self, key: Union[str, SystemConfigKey]) -> bool:
"""
删除系统设置
"""

View File

@@ -1,4 +1,5 @@
from typing import Callable, Any, Optional
import gc
from playwright.sync_api import sync_playwright, Page
from cf_clearance import sync_cf_retry, sync_stealth
@@ -35,26 +36,43 @@ class PlaywrightHelper:
:param headless: 是否无头模式
:param timeout: 超时时间
"""
result = None
try:
with sync_playwright() as playwright:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
browser = None
context = None
page = None
try:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
# 回调函数
return callback(page)
result = callback(page)
except Exception as e:
logger.error(f"网页操作失败: {str(e)}")
finally:
browser.close()
# 确保资源被正确清理
if page:
page.close()
if context:
context.close()
if browser:
browser.close()
# 强制垃圾回收
gc.collect()
except Exception as e:
logger.error(f"网页操作失败: {str(e)}")
return None
logger.error(f"Playwright初始化失败: {str(e)}")
return result
def get_page_source(self, url: str,
cookies: Optional[str] = None,
@@ -71,26 +89,42 @@ class PlaywrightHelper:
:param headless: 是否无头模式
:param timeout: 超时时间
"""
source = ""
source = None
try:
with sync_playwright() as playwright:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
browser = None
context = None
page = None
try:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
source = page.content()
except Exception as e:
logger.error(f"获取网页源码失败: {str(e)}")
source = None
finally:
browser.close()
# 确保资源被正确清理
if page:
page.close()
if context:
context.close()
if browser:
browser.close()
# 强制垃圾回收
gc.collect()
except Exception as e:
logger.error(f"获取网页源码失败: {str(e)}")
logger.error(f"Playwright初始化失败: {str(e)}")
return source

View File

@@ -14,7 +14,6 @@ class CookieCloudHelper:
def __init__(self):
self.__sync_setting()
self._req = RequestUtils(content_type="application/json")
def __sync_setting(self):
"""
@@ -46,7 +45,7 @@ class CookieCloudHelper:
return {}, "未从本地CookieCloud服务加载到cookie数据请检查服务器设置、用户KEY及加密密码是否正确"
else:
req_url = UrlUtils.combine_url(host=self._server, path=f"get/{self._key}")
ret = self._req.get_res(url=req_url)
ret = RequestUtils(content_type="application/json").get_res(url=req_url)
if ret and ret.status_code == 200:
try:
result = ret.json()

View File

@@ -13,14 +13,12 @@ class DirectoryHelper:
下载目录/媒体库目录帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_dirs(self) -> List[schemas.TransferDirectoryConf]:
@staticmethod
def get_dirs() -> List[schemas.TransferDirectoryConf]:
"""
获取所有下载目录
"""
dir_confs: List[dict] = self.systemconfig.get(SystemConfigKey.Directories)
dir_confs: List[dict] = SystemConfigOper().get(SystemConfigKey.Directories)
if not dir_confs:
return []
return [schemas.TransferDirectoryConf(**d) for d in dir_confs]

176
app/helper/memory.py Normal file
View File

@@ -0,0 +1,176 @@
import gc
import psutil
import threading
import time
from typing import Optional, Callable, Any
from functools import wraps
from app.log import logger
from app.utils.singleton import Singleton
class MemoryManager(metaclass=Singleton):
"""
内存管理工具类,用于监控和优化内存使用
"""
def __init__(self):
self._memory_threshold = 512 # 内存使用阈值(MB)
self._check_interval = 300 # 检查间隔(秒)
self._monitoring = False
self._monitor_thread: Optional[threading.Thread] = None
@staticmethod
def get_memory_usage() -> dict:
"""
获取当前内存使用情况
"""
process = psutil.Process()
memory_info = process.memory_info()
system_memory = psutil.virtual_memory()
return {
'rss': memory_info.rss / 1024 / 1024, # MB
'vms': memory_info.vms / 1024 / 1024, # MB
'percent': process.memory_percent(),
'system_percent': system_memory.percent,
'system_available': system_memory.available / 1024 / 1024 / 1024 # GB
}
def force_gc(self, generation: Optional[int] = None) -> int:
"""
强制执行垃圾回收
:param generation: 垃圾回收代数None表示所有代数
:return: 回收的对象数量
"""
before_memory = self.get_memory_usage()
logger.info(f"开始强制垃圾回收,当前内存使用: {before_memory['rss']:.2f}MB")
if generation is not None:
collected = gc.collect(generation)
else:
collected = gc.collect()
after_memory = self.get_memory_usage()
memory_freed = before_memory['rss'] - after_memory['rss']
if memory_freed > 0:
logger.info(f"垃圾回收完成: 回收对象 {collected} 个, 释放内存 {memory_freed:.2f}MB")
return collected
def check_memory_and_cleanup(self) -> bool:
"""
检查内存使用量,如果超过阈值则执行清理
:return: 是否执行了清理
"""
memory_info = self.get_memory_usage()
current_memory_mb = memory_info['rss']
if current_memory_mb > self._memory_threshold:
logger.warning(f"内存使用超过阈值: {current_memory_mb:.1f}MB > {self._memory_threshold:.1f}MB, 开始清理...")
self.force_gc()
# 再次检查清理效果
after_memory = self.get_memory_usage()
logger.info(f"清理后内存: {after_memory['rss']:.1f}MB")
return True
return False
def start_monitoring(self):
"""
开始内存监控
"""
if self._monitoring:
return
self._monitoring = True
self._monitor_thread = threading.Thread(target=self._monitor_loop, daemon=True)
self._monitor_thread.start()
logger.info(f"内存监控已启动 - 阈值: {self._memory_threshold}MB, 检查间隔: {self._check_interval}")
def stop_monitoring(self):
"""
停止内存监控
"""
self._monitoring = False
if self._monitor_thread:
self._monitor_thread.join(timeout=5)
logger.info("内存监控已停止")
def _monitor_loop(self):
"""
内存监控循环
"""
while self._monitoring:
try:
self.check_memory_and_cleanup()
time.sleep(self._check_interval)
except Exception as e:
logger.error(f"内存监控出错: {e}")
time.sleep(60) # 出错后等待1分钟再继续
def set_threshold(self, threshold_mb: int):
"""
设置内存使用阈值
:param threshold_mb: 内存阈值单位MB500-4096之间
"""
self._memory_threshold = max(512, min(4096, threshold_mb))
logger.info(f"内存阈值已设置为: {self._memory_threshold}MB")
def set_check_interval(self, interval: int):
"""
设置检查间隔
:param interval: 检查间隔单位秒最少60秒
"""
self._check_interval = max(60, interval)
logger.info(f"内存检查间隔已设置为: {self._check_interval}")
def get_threshold(self) -> int:
"""
获取当前内存阈值
:return: 当前阈值(MB)
"""
return self._memory_threshold
def memory_optimized(force_gc_after: bool = False, log_memory: bool = False):
"""
内存优化装饰器
:param force_gc_after: 函数执行后是否强制垃圾回收
:param log_memory: 是否记录内存使用情况
"""
def decorator(func: Callable) -> Callable:
@wraps(func)
def wrapper(*args, **kwargs) -> Any:
memory_manager = MemoryManager()
if log_memory:
before_memory = memory_manager.get_memory_usage()
logger.info(f"{func.__name__} 执行前内存: {before_memory['rss']:.1f}MB")
try:
result = func(*args, **kwargs)
return result
finally:
if force_gc_after:
memory_manager.force_gc()
if log_memory:
after_memory = memory_manager.get_memory_usage()
logger.info(f"{func.__name__} 执行后内存: {after_memory['rss']:.1f}MB")
return wrapper
return decorator
def clear_large_objects(*objects):
"""
清理大型对象的辅助函数
"""
for obj in objects:
if hasattr(obj, 'clear') and callable(obj.clear):
obj.clear()
elif hasattr(obj, '__dict__'):
obj.__dict__.clear()
del obj
gc.collect()

View File

@@ -61,7 +61,8 @@ class TemplateContextBuilder:
self._add_transfer_info(transferinfo)
self._add_torrent_info(torrentinfo)
self._add_file_info(file_extension)
if kwargs: self._context.update(kwargs)
if kwargs:
self._context.update(kwargs)
if include_raw_objects:
self._add_raw_objects(meta, mediainfo, torrentinfo, transferinfo, episodes_info)
@@ -73,7 +74,8 @@ class TemplateContextBuilder:
"""
增加媒体信息
"""
if not mediainfo: return
if not mediainfo:
return
season_fmt = f"S{mediainfo.season:02d}" if mediainfo.season is not None else None
base_info = {
# 标题
@@ -245,7 +247,8 @@ class TemplateContextBuilder:
"""
添加文件信息
"""
if not file_extension: return
if not file_extension:
return
file_info = {
# 文件后缀
"fileExt": file_extension,

View File

@@ -2,6 +2,8 @@ import sys
import json
import shutil
import traceback
import site
import importlib
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple, Set
@@ -39,7 +41,7 @@ class PluginHelper(metaclass=Singleton):
if self.install_report():
self.systemconfig.set(SystemConfigKey.PluginInstallReport, "1")
@cached(maxsize=1000, ttl=1800)
@cached(maxsize=64, ttl=1800)
def get_plugins(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
"""
获取Github所有最新插件列表
@@ -452,7 +454,7 @@ class PluginHelper(metaclass=Singleton):
@staticmethod
def __pip_install_with_fallback(requirements_file: Path) -> Tuple[bool, str]:
"""
使用自动降级策略PIP 安装依赖,优先级依次为镜像站、代理、直连
使用自动降级策略安装依赖,并确保新安装的包可被动态导入
:param requirements_file: 依赖的 requirements.txt 文件路径
:return: (是否成功, 错误信息)
"""
@@ -466,12 +468,25 @@ class PluginHelper(metaclass=Singleton):
strategies.append(("代理", base_cmd + ["--proxy", settings.PROXY_HOST]))
strategies.append(("直连", base_cmd))
# 记录当前已安装的包,以便后续刷新
before_installation = set(sys.modules.keys())
# 遍历策略进行安装
for strategy_name, pip_command in strategies:
logger.debug(f"[PIP] 尝试使用策略:{strategy_name} 安装依赖,命令:{' '.join(pip_command)}")
success, message = SystemUtils.execute_with_subprocess(pip_command)
if success:
logger.debug(f"[PIP] 策略:{strategy_name} 安装依赖成功,输出:{message}")
# 安装成功后刷新Python的模块系统
importlib.reload(site)
# 获取新安装的模块
current_modules = set(sys.modules.keys())
new_modules = current_modules - before_installation
# 重新加载新安装的模块
for module in new_modules:
if module in sys.modules:
del sys.modules[module]
logger.debug(f"[PIP] 已刷新导入系统,新加载的模块: {new_modules}")
return True, message
else:
logger.error(f"[PIP] 策略:{strategy_name} 安装依赖失败,错误信息:{message}")

View File

@@ -3,14 +3,14 @@ from pathlib import Path
from app.core.config import settings
from app.helper.sites import SitesHelper
from app.helper.system import SystemHelper
from app.log import logger
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.string import StringUtils
from app.utils.system import SystemUtils
class ResourceHelper(metaclass=Singleton):
class ResourceHelper:
"""
检测和更新资源包
"""
@@ -20,7 +20,6 @@ class ResourceHelper(metaclass=Singleton):
_base_dir: Path = settings.ROOT_PATH
def __init__(self):
self.siteshelper = SitesHelper()
self.check()
@property
@@ -32,80 +31,80 @@ class ResourceHelper(metaclass=Singleton):
检测是否有更新,如有则下载安装
"""
if not settings.AUTO_UPDATE_RESOURCE:
return
return None
if SystemUtils.is_frozen():
return
return None
logger.info("开始检测资源包版本...")
res = RequestUtils(proxies=self.proxies, headers=settings.GITHUB_HEADERS, timeout=10).get_res(self._repo)
if res:
try:
resource_info = json.loads(res.text)
online_version = resource_info.get("version")
if online_version:
logger.info(f"最新资源包版本v{online_version}")
# 需要更新的资源包
need_updates = {}
# 资源明细
resources: dict = resource_info.get("resources") or {}
for rname, resource in resources.items():
rtype = resource.get("type")
platform = resource.get("platform")
target = resource.get("target")
version = resource.get("version")
# 判断平台
if platform and platform != SystemUtils.platform():
continue
# 判断版本号
if rtype == "auth":
# 站点认证资源
local_version = SitesHelper().auth_version
elif rtype == "sites":
# 站点索引资源
local_version = SitesHelper().indexer_version
else:
continue
if StringUtils.compare_version(version, ">", local_version):
logger.info(f"{rname} 资源包有更新最新版本v{version}")
else:
continue
# 需要安装
need_updates[rname] = target
if need_updates:
# 下载文件信息列表
r = RequestUtils(proxies=settings.PROXY, headers=settings.GITHUB_HEADERS,
timeout=30).get_res(self._files_api)
if r and not r.ok:
return None, f"连接仓库失败:{r.status_code} - {r.reason}"
elif not r:
return None, "连接仓库失败"
files_info = r.json()
for item in files_info:
save_path = need_updates.get(item.get("name"))
if not save_path:
continue
if item.get("download_url"):
logger.info(f"开始更新资源文件:{item.get('name')} ...")
download_url = f"{settings.GITHUB_PROXY}{item.get('download_url')}"
# 下载资源文件
res = RequestUtils(proxies=self.proxies, headers=settings.GITHUB_HEADERS,
timeout=180).get_res(download_url)
if not res:
logger.error(f"文件 {item.get('name')} 下载失败!")
elif res.status_code != 200:
logger.error(f"下载文件 {item.get('name')} 失败:{res.status_code} - {res.reason}")
# 创建插件文件夹
file_path = self._base_dir / save_path / item.get("name")
if not file_path.parent.exists():
file_path.parent.mkdir(parents=True, exist_ok=True)
# 写入文件
file_path.write_bytes(res.content)
logger.info("资源包更新完成,开始重启服务...")
SystemHelper.restart()
else:
logger.info("所有资源已最新,无需更新")
except json.JSONDecodeError:
logger.error("资源包仓库数据解析失败!")
return
return None
else:
logger.warn("无法连接资源包仓库!")
return
online_version = resource_info.get("version")
if online_version:
logger.info(f"最新资源包版本v{online_version}")
# 需要更新的资源包
need_updates = {}
# 资源明细
resources: dict = resource_info.get("resources") or {}
for rname, resource in resources.items():
rtype = resource.get("type")
platform = resource.get("platform")
target = resource.get("target")
version = resource.get("version")
# 判断平台
if platform and platform != SystemUtils.platform():
continue
# 判断版本号
if rtype == "auth":
# 站点认证资源
local_version = self.siteshelper.auth_version
elif rtype == "sites":
# 站点索引资源
local_version = self.siteshelper.indexer_version
else:
continue
if StringUtils.compare_version(version, ">", local_version):
logger.info(f"{rname} 资源包有更新最新版本v{version}")
else:
continue
# 需要安装
need_updates[rname] = target
if need_updates:
# 下载文件信息列表
r = RequestUtils(proxies=settings.PROXY, headers=settings.GITHUB_HEADERS,
timeout=30).get_res(self._files_api)
if r and not r.ok:
return None, f"连接仓库失败:{r.status_code} - {r.reason}"
elif not r:
return None, "连接仓库失败"
files_info = r.json()
for item in files_info:
save_path = need_updates.get(item.get("name"))
if not save_path:
continue
if item.get("download_url"):
logger.info(f"开始更新资源文件:{item.get('name')} ...")
download_url = f"{settings.GITHUB_PROXY}{item.get('download_url')}"
# 下载资源文件
res = RequestUtils(proxies=self.proxies, headers=settings.GITHUB_HEADERS,
timeout=180).get_res(download_url)
if not res:
logger.error(f"文件 {item.get('name')} 下载失败!")
elif res.status_code != 200:
logger.error(f"下载文件 {item.get('name')} 失败:{res.status_code} - {res.reason}")
# 创建插件文件夹
file_path = self._base_dir / save_path / item.get("name")
if not file_path.parent.exists():
file_path.parent.mkdir(parents=True, exist_ok=True)
# 写入文件
file_path.write_bytes(res.content)
logger.info("资源包更新完成,开始重启服务...")
SystemUtils.restart()
else:
logger.info("所有资源已最新,无需更新")
return None

View File

@@ -1,6 +1,6 @@
import gc
import re
import traceback
import xml.dom.minidom
from typing import List, Tuple, Union, Optional
from urllib.parse import urljoin
@@ -10,7 +10,6 @@ from lxml import etree
from app.core.config import settings
from app.helper.browser import PlaywrightHelper
from app.log import logger
from app.utils.dom import DomUtils
from app.utils.http import RequestUtils
from app.utils.string import StringUtils
@@ -19,6 +18,11 @@ class RssHelper:
"""
RSS帮助类解析RSS报文、获取RSS地址等
"""
# RSS解析限制配置
MAX_RSS_SIZE = 50 * 1024 * 1024 # 50MB最大RSS文件大小
MAX_RSS_ITEMS = 1000 # 最大解析条目数
# 各站点RSS链接获取配置
rss_link_conf = {
"default": {
@@ -224,8 +228,7 @@ class RssHelper:
},
}
@staticmethod
def parse(url, proxy: bool = False, timeout: Optional[int] = 15, headers: dict = None) -> Union[List[dict], None, bool]:
def parse(self, url, proxy: bool = False, timeout: Optional[int] = 15, headers: dict = None) -> Union[List[dict], None, bool]:
"""
解析RSS订阅URL获取RSS中的种子信息
:param url: RSS地址
@@ -238,6 +241,7 @@ class RssHelper:
ret_array: list = []
if not url:
return False
try:
ret = RequestUtils(proxies=settings.PROXY if proxy else None,
timeout=timeout, headers=headers).get_res(url)
@@ -246,11 +250,17 @@ class RssHelper:
except Exception as err:
logger.error(f"获取RSS失败{str(err)} - {traceback.format_exc()}")
return False
if ret:
ret_xml = ""
ret_xml = None
root = None
try:
# 使用chardet检测字符编码
# 检查响应大小避免处理过大的RSS文件
raw_data = ret.content
if raw_data and len(raw_data) > self.MAX_RSS_SIZE:
logger.warning(f"RSS文件过大: {len(raw_data)/1024/1024:.1f}MB跳过解析")
return False
if raw_data:
try:
result = chardet.detect(raw_data)
@@ -269,57 +279,117 @@ class RssHelper:
ret.encoding = ret.apparent_encoding
if not ret_xml:
ret_xml = ret.text
# 解析XML
dom_tree = xml.dom.minidom.parseString(ret_xml)
rootNode = dom_tree.documentElement
items = rootNode.getElementsByTagName("item")
for item in items:
# 使用lxml.etree解析XML
parser = None
try:
# 创建解析器,禁用网络访问以提高安全性和性能
parser = etree.XMLParser(
recover=True, # 容错模式
strip_cdata=False, # 保留CDATA
resolve_entities=False, # 禁用外部实体解析
no_network=True, # 禁用网络访问
huge_tree=False # 禁用大文档解析,避免内存问题
)
root = etree.fromstring(ret_xml.encode('utf-8'), parser=parser)
except etree.XMLSyntaxError:
# 如果XML解析失败尝试作为HTML解析
try:
# 标题
title = DomUtils.tag_value(item, "title", default="")
root = etree.HTML(ret_xml)
if root is not None:
# 查找RSS根节点
rss_root = root.xpath('//rss | //feed')
if rss_root:
root = rss_root[0]
except Exception as e:
logger.error(f"HTML解析也失败{str(e)}")
return False
finally:
if parser is not None:
del parser
if root is None:
logger.error("无法解析RSS内容")
return False
# 查找所有item或entry节点
items = root.xpath('.//item | .//entry')
# 限制处理的条目数量
items_count = min(len(items), self.MAX_RSS_ITEMS)
if len(items) > self.MAX_RSS_ITEMS:
logger.warning(f"RSS条目过多: {len(items)},仅处理前{self.MAX_RSS_ITEMS}")
for i, item in enumerate(items[:items_count]):
try:
# 定期执行垃圾回收
if i > 0 and i % 100 == 0:
gc.collect()
# 使用xpath提取信息更高效
title_nodes = item.xpath('.//title')
title = title_nodes[0].text if title_nodes and title_nodes[0].text else ""
if not title:
continue
# 描述
description = DomUtils.tag_value(item, "description", default="")
desc_nodes = item.xpath('.//description | .//summary')
description = desc_nodes[0].text if desc_nodes and desc_nodes[0].text else ""
# 种子页面
link = DomUtils.tag_value(item, "link", default="")
link_nodes = item.xpath('.//link')
if link_nodes:
link = link_nodes[0].text if hasattr(link_nodes[0], 'text') and link_nodes[0].text else link_nodes[0].get('href', '')
else:
link = ""
# 种子链接
enclosure = DomUtils.tag_value(item, "enclosure", "url", default="")
enclosure_nodes = item.xpath('.//enclosure')
enclosure = enclosure_nodes[0].get('url', '') if enclosure_nodes else ""
if not enclosure and not link:
continue
# 部分RSS只有link没有enclosure
if not enclosure and link:
enclosure = link
# 大小
size = DomUtils.tag_value(item, "enclosure", "length", default=0)
if size and str(size).isdigit():
size = int(size)
else:
size = 0
size = 0
if enclosure_nodes:
size_attr = enclosure_nodes[0].get('length', '0')
if size_attr and str(size_attr).isdigit():
size = int(size_attr)
# 发布日期
pubdate = DomUtils.tag_value(item, "pubDate", default="")
if pubdate:
# 转换为时间
pubdate = StringUtils.get_time(pubdate)
pubdate_nodes = item.xpath('.//pubDate | .//published | .//updated')
pubdate = ""
if pubdate_nodes and pubdate_nodes[0].text:
pubdate = StringUtils.get_time(pubdate_nodes[0].text)
# 获取豆瓣昵称
nickname = DomUtils.tag_value(item, "dc:createor", default="")
nickname_nodes = item.xpath('.//*[local-name()="creator"]')
nickname = nickname_nodes[0].text if nickname_nodes and nickname_nodes[0].text else ""
# 返回对象
tmp_dict = {'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate}
tmp_dict = {
'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate
}
# 如果豆瓣昵称不为空返回数据增加豆瓣昵称供doubansync插件获取
if nickname:
tmp_dict['nickname'] = nickname
ret_array.append(tmp_dict)
except Exception as e1:
logger.debug(f"解析RSS失败{str(e1)} - {traceback.format_exc()}")
logger.debug(f"解析RSS条目失败:{str(e1)} - {traceback.format_exc()}")
continue
except Exception as e2:
logger.error(f"解析RSS失败{str(e2)} - {traceback.format_exc()}")
# RSS过期 观众RSS 链接已过期,您需要获得一个新的! pthome RSS Link has expired, You need to get a new one!
# RSS过期检查
_rss_expired_msg = [
"RSS 链接已过期, 您需要获得一个新的!",
"RSS Link has expired, You need to get a new one!",
@@ -328,6 +398,13 @@ class RssHelper:
if ret_xml in _rss_expired_msg:
return None
return False
finally:
if root is not None:
del root
if ret_xml is not None:
del ret_xml
gc.collect()
return ret_array
def get_rss_link(self, url: str, cookie: str, ua: str, proxy: bool = False) -> Tuple[str, str]:
@@ -369,12 +446,20 @@ class RssHelper:
return "", f"获取 {url} RSS链接失败错误码{res.status_code},错误原因:{res.reason}"
else:
return "", f"获取RSS链接失败无法连接 {url} "
# 解析HTML
html = etree.HTML(html_text)
if StringUtils.is_valid_html_element(html):
rss_link = html.xpath(site_conf.get("xpath"))
if rss_link:
return str(rss_link[-1]), ""
if html_text:
html = None
try:
html = etree.HTML(html_text)
if StringUtils.is_valid_html_element(html):
rss_link = html.xpath(site_conf.get("xpath"))
if rss_link:
return str(rss_link[-1]), ""
finally:
if html is not None:
del html
return "", f"获取RSS链接失败{url}"
except Exception as e:
return "", f"获取 {url} RSS链接失败{str(e)}"

View File

@@ -11,14 +11,12 @@ class RuleHelper:
规划帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_rule_groups(self) -> List[FilterRuleGroup]:
@staticmethod
def get_rule_groups() -> List[FilterRuleGroup]:
"""
获取用户所有规则组
"""
rule_groups: List[dict] = self.systemconfig.get(SystemConfigKey.UserFilterRuleGroups)
rule_groups: List[dict] = SystemConfigOper().get(SystemConfigKey.UserFilterRuleGroups)
if not rule_groups:
return []
return [FilterRuleGroup(**group) for group in rule_groups]
@@ -50,11 +48,12 @@ class RuleHelper:
ret_groups.append(group)
return ret_groups
def get_custom_rules(self) -> List[CustomRule]:
@staticmethod
def get_custom_rules() -> List[CustomRule]:
"""
获取用户所有自定义规则
"""
rules: List[dict] = self.systemconfig.get(SystemConfigKey.CustomFilterRules)
rules: List[dict] = SystemConfigOper().get(SystemConfigKey.CustomFilterRules)
if not rules:
return []
return [CustomRule(**rule) for rule in rules]

View File

@@ -10,14 +10,12 @@ class StorageHelper:
存储帮助类
"""
def __init__(self):
self.systemconfig = SystemConfigOper()
def get_storagies(self) -> List[schemas.StorageConf]:
@staticmethod
def get_storagies() -> List[schemas.StorageConf]:
"""
获取所有存储设置
"""
storage_confs: List[dict] = self.systemconfig.get(SystemConfigKey.Storages)
storage_confs: List[dict] = SystemConfigOper().get(SystemConfigKey.Storages)
if not storage_confs:
return []
return [schemas.StorageConf(**s) for s in storage_confs]
@@ -49,7 +47,7 @@ class StorageHelper:
if s.type == storage:
s.config = conf
break
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])
def add_storage(self, storage: str, name: str, conf: dict):
"""
@@ -70,7 +68,7 @@ class StorageHelper:
name=name,
config=conf
))
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])
def reset_storage(self, storage: str):
"""
@@ -81,4 +79,4 @@ class StorageHelper:
if s.type == storage:
s.config = {}
break
self.systemconfig.set(SystemConfigKey.Storages, [s.dict() for s in storagies])
SystemConfigOper().set(SystemConfigKey.Storages, [s.dict() for s in storagies])

View File

@@ -50,11 +50,11 @@ class SubscribeHelper(metaclass=Singleton):
]
def __init__(self):
self.systemconfig = SystemConfigOper()
systemconfig = SystemConfigOper()
if settings.SUBSCRIBE_STATISTIC_SHARE:
if not self.systemconfig.get(SystemConfigKey.SubscribeReport):
if not systemconfig.get(SystemConfigKey.SubscribeReport):
if self.sub_report():
self.systemconfig.set(SystemConfigKey.SubscribeReport, "1")
systemconfig.set(SystemConfigKey.SubscribeReport, "1")
self.get_user_uuid()
self.get_github_user()

59
app/helper/system.py Normal file
View File

@@ -0,0 +1,59 @@
from pathlib import Path
from typing import Tuple
import docker
from app.core.config import settings
from app.utils.system import SystemUtils
class SystemHelper:
"""
系统工具类,提供系统相关的操作和判断
"""
@staticmethod
def can_restart() -> bool:
"""
判断是否可以内部重启
"""
return (
Path("/var/run/docker.sock").exists()
or settings.DOCKER_CLIENT_API != "tcp://127.0.0.1:38379"
)
@staticmethod
def restart() -> Tuple[bool, str]:
"""
执行Docker重启操作
"""
if not SystemUtils.is_docker():
return False, "非Docker环境无法重启"
try:
# 创建 Docker 客户端
client = docker.DockerClient(base_url=settings.DOCKER_CLIENT_API)
# 获取当前容器的 ID
container_id = None
with open("/proc/self/mountinfo", "r") as f:
data = f.read()
index_resolv_conf = data.find("resolv.conf")
if index_resolv_conf != -1:
index_second_slash = data.rfind("/", 0, index_resolv_conf)
index_first_slash = data.rfind("/", 0, index_second_slash) + 1
container_id = data[index_first_slash:index_second_slash]
if len(container_id) < 20:
index_resolv_conf = data.find("/sys/fs/cgroup/devices")
if index_resolv_conf != -1:
index_second_slash = data.rfind(" ", 0, index_resolv_conf)
index_first_slash = (
data.rfind("/", 0, index_second_slash) + 1
)
container_id = data[index_first_slash:index_second_slash]
if not container_id:
return False, "获取容器ID失败"
# 重启当前容器
client.containers.get(container_id.strip()).restart()
return True, ""
except Exception as err:
print(str(err))
return False, f"重启时发生错误:{str(err)}"

View File

@@ -28,10 +28,6 @@ class TorrentHelper(metaclass=Singleton):
# 失败的种子:站点链接
_invalid_torrents = []
def __init__(self):
self.system_config = SystemConfigOper()
self.site_oper = SiteOper()
def download_torrent(self, url: str,
cookie: Optional[str] = None,
ua: Optional[str] = None,
@@ -192,7 +188,8 @@ class TorrentHelper(metaclass=Singleton):
file_name = str(datetime.datetime.now())
return file_name
def sort_torrents(self, torrent_list: List[Context]) -> List[Context]:
@staticmethod
def sort_torrents(torrent_list: List[Context]) -> List[Context]:
"""
对种子对行排序torrent、site、upload、seeder
"""
@@ -200,11 +197,11 @@ class TorrentHelper(metaclass=Singleton):
return []
# 下载规则
priority_rule: List[str] = self.system_config.get(
priority_rule: List[str] = SystemConfigOper().get(
SystemConfigKey.TorrentsPriority) or ["torrent", "upload", "seeder"]
# 站点上传量
site_uploads = {
site.name: site.upload for site in self.site_oper.get_userdata_latest()
site.name: site.upload for site in SiteOper().get_userdata_latest()
}
def get_sort_str(_context):

View File

@@ -1,5 +1,7 @@
from typing import Optional, List
from app.chain.mediaserver import MediaServerChain
from app.chain.tmdb import TmdbChain
from app.core.cache import cached
from app.core.config import settings
from app.utils.http import RequestUtils
@@ -7,9 +9,50 @@ from app.utils.singleton import Singleton
class WallpaperHelper(metaclass=Singleton):
"""
壁纸帮助类
"""
def __init__(self):
self.req = RequestUtils(timeout=5)
def get_wallpaper(self) -> Optional[str]:
"""
获取登录页面壁纸
"""
if settings.WALLPAPER == "bing":
url = self.get_bing_wallpaper()
elif settings.WALLPAPER == "mediaserver":
url = self.get_mediaserver_wallpaper()
elif settings.WALLPAPER == "customize":
url = self.get_customize_wallpaper()
else:
url = self.get_tmdb_wallpaper()
return url
def get_wallpapers(self, num: int = 10) -> List[str]:
"""
获取登录页面壁纸列表
"""
if settings.WALLPAPER == "bing":
return self.get_bing_wallpapers(num)
elif settings.WALLPAPER == "mediaserver":
return self.get_mediaserver_wallpapers(num)
elif settings.WALLPAPER == "customize":
return self.get_customize_wallpapers()
else:
return self.get_tmdb_wallpapers(num)
@cached(maxsize=1, ttl=3600)
def get_tmdb_wallpaper(self) -> Optional[str]:
"""
获取TMDB每日壁纸
"""
return TmdbChain().get_random_wallpager()
@cached(maxsize=1, ttl=3600)
def get_tmdb_wallpapers(self, num: int = 10) -> List[str]:
"""
获取7天的TMDB每日壁纸
"""
return TmdbChain().get_trending_wallpapers(num)
@cached(maxsize=1, ttl=3600)
def get_bing_wallpaper(self) -> Optional[str]:
@@ -17,7 +60,7 @@ class WallpaperHelper(metaclass=Singleton):
获取Bing每日壁纸
"""
url = "https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1"
resp = self.req.get_res(url)
resp = RequestUtils(timeout=5).get_res(url)
if resp and resp.status_code == 200:
try:
result = resp.json()
@@ -34,7 +77,7 @@ class WallpaperHelper(metaclass=Singleton):
获取7天的Bing每日壁纸
"""
url = f"https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n={num}"
resp = self.req.get_res(url)
resp = RequestUtils(timeout=5).get_res(url)
if resp and resp.status_code == 200:
try:
result = resp.json()
@@ -44,6 +87,20 @@ class WallpaperHelper(metaclass=Singleton):
print(str(err))
return []
@cached(maxsize=1, ttl=3600)
def get_mediaserver_wallpaper(self) -> Optional[str]:
"""
获取媒体服务器壁纸
"""
return MediaServerChain().get_latest_wallpaper()
@cached(maxsize=1, ttl=3600)
def get_mediaserver_wallpapers(self, num: int = 10) -> List[str]:
"""
获取媒体服务器壁纸列表
"""
return MediaServerChain().get_latest_wallpapers(count=num)
@cached(maxsize=1, ttl=3600)
def get_customize_wallpaper(self) -> Optional[str]:
"""
@@ -87,12 +144,12 @@ class WallpaperHelper(metaclass=Singleton):
# 判断是否存在自定义壁纸api
if settings.CUSTOMIZE_WALLPAPER_API_URL:
wallpaper_list = []
resp = self.req.get_res(settings.CUSTOMIZE_WALLPAPER_API_URL)
resp = RequestUtils(timeout=15).get_res(settings.CUSTOMIZE_WALLPAPER_API_URL)
if resp and resp.status_code == 200:
# 如果返回的是图片格式
content_type = resp.headers.get('Content-Type')
if content_type and content_type.lower() == 'image/jpeg':
wallpaper_list.append(resp.url)
if content_type and content_type.lower().startswith('image/'):
wallpaper_list.append(settings.CUSTOMIZE_WALLPAPER_API_URL)
else:
try:
result = resp.json()

View File

@@ -28,9 +28,9 @@ class LogConfigModel(BaseModel):
# 日志文件最大大小单位MB
LOG_MAX_FILE_SIZE: int = 5
# 备份的日志文件数量
LOG_BACKUP_COUNT: int = 3
LOG_BACKUP_COUNT: int = 10
# 控制台日志格式
LOG_CONSOLE_FORMAT: str = "%(leveltext)s%(message)s"
LOG_CONSOLE_FORMAT: str = "%(leveltext)s[%(name)s] %(asctime)s %(message)s"
# 文件日志格式
LOG_FILE_FORMAT: str = "%(levelname)s%(asctime)s - %(message)s"
@@ -189,6 +189,9 @@ class LoggerManager:
file_handler.setFormatter(file_formatter)
_logger.addHandler(file_handler)
# 禁止向父级log传递
_logger.propagate = False
return _logger
def update_loggers(self):

View File

@@ -18,7 +18,7 @@ class BangumiModule(_ModuleBase):
self.bangumiapi = BangumiApi()
def stop(self):
pass
self.bangumiapi.close()
def test(self) -> Tuple[bool, str]:
"""

View File

@@ -25,19 +25,18 @@ class BangumiApi(object):
"person_credits": "v0/persons/%s/subjects",
}
_base_url = "https://api.bgm.tv/"
_req = RequestUtils(session=requests.Session())
def __init__(self):
pass
self._session = requests.Session()
self._req = RequestUtils(session=self._session)
@classmethod
@cached(maxsize=settings.CACHE_CONF["bangumi"], ttl=settings.CACHE_CONF["meta"])
def __invoke(cls, url, key: Optional[str] = None, **kwargs):
req_url = cls._base_url + url
def __invoke(self, url, key: Optional[str] = None, **kwargs):
req_url = self._base_url + url
params = {}
if kwargs:
params.update(kwargs)
resp = cls._req.get_res(url=req_url, params=params)
resp = self._req.get_res(url=req_url, params=params)
try:
if not resp:
return None
@@ -207,3 +206,7 @@ class BangumiApi(object):
return self.__invoke(self._urls["discover"],
key="data",
_ts=datetime.strftime(datetime.now(), '%Y%m%d'), **kwargs)
def close(self):
if self._session:
self._session.close()

View File

@@ -2,11 +2,11 @@ from typing import Any, Generator, List, Optional, Tuple, Union
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _MediaServerBase, _ModuleBase
from app.modules.emby.emby import Emby
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
@@ -18,6 +18,19 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
super().init_service(service_name=Emby.__name__.lower(),
service_type=lambda conf: Emby(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Emby"
@@ -269,7 +282,8 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
) for season, episodes in seasoninfo.items()]
def mediaserver_playing(self, server: str,
count: Optional[int] = 20, username: Optional[str] = None) -> List[schemas.MediaServerPlayItem]:
count: Optional[int] = 20, username: Optional[str] = None) -> List[
schemas.MediaServerPlayItem]:
"""
获取媒体服务器正在播放信息
"""
@@ -288,7 +302,8 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
return server_obj.get_play_url(item_id)
def mediaserver_latest(self, server: Optional[str] = None,
count: Optional[int] = 20, username: Optional[str] = None) -> List[schemas.MediaServerPlayItem]:
count: Optional[int] = 20, username: Optional[str] = None) -> List[
schemas.MediaServerPlayItem]:
"""
获取媒体服务器最新入库条目
"""

View File

@@ -13,7 +13,7 @@ from app.log import logger
from app.schemas.types import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Emby:

File diff suppressed because it is too large Load Diff

View File

@@ -435,6 +435,17 @@ class AliPan(StorageBase, metaclass=Singleton):
break
return items
def _delay_get_item(self, path: Path) -> Optional[schemas.FileItem]:
"""
自动延迟重试 get_item 模块
"""
for _ in range(2):
time.sleep(2)
fileitem = self.get_item(path)
if fileitem:
return fileitem
return None
def create_folder(self, parent_item: schemas.FileItem, name: str) -> Optional[schemas.FileItem]:
"""
创建目录
@@ -457,7 +468,7 @@ class AliPan(StorageBase, metaclass=Singleton):
# 缓存新目录
new_path = Path(parent_item.path) / name
self._id_cache[str(new_path)] = (resp.get("drive_id"), resp.get("file_id"))
return self.get_item(new_path)
return self._delay_get_item(new_path)
@staticmethod
def _calculate_pre_hash(file_path: Path):
@@ -676,7 +687,7 @@ class AliPan(StorageBase, metaclass=Singleton):
chunk_size=chunk_size)
if create_res.get('rapid_upload', False):
logger.info(f"【阿里云盘】{target_name} 秒传完成!")
return self.get_item(target_path)
return self._delay_get_item(target_path)
if create_res.get("exist", False):
logger.info(f"【阿里云盘】{target_name} 已存在")
@@ -919,7 +930,7 @@ class AliPan(StorageBase, metaclass=Singleton):
return False
# 重命名
new_path = Path(path) / fileitem.name
new_file = self.get_item(new_path)
new_file = self._delay_get_item(new_path)
self.rename(new_file, new_name)
# 更新缓存
del self._id_cache[fileitem.path]

View File

@@ -3,6 +3,7 @@ from datetime import datetime
from pathlib import Path
from typing import Optional, List, Dict
import requests
from requests import Response
from app import schemas
@@ -529,20 +530,19 @@ class Alist(StorageBase, metaclass=Singleton):
if result["data"]["sign"]:
download_url = download_url + "?sign=" + result["data"]["sign"]
resp = RequestUtils(
headers=self.__get_header_with_token()
).get_res(download_url)
if not path:
new_path = settings.TEMP_PATH / fileitem.name
local_path = settings.TEMP_PATH / fileitem.name
else:
new_path = path / fileitem.name
local_path = path / fileitem.name
with open(new_path, "wb") as f:
f.write(resp.content)
with requests.get(download_url, headers=self.__get_header_with_token(), stream=True) as r:
r.raise_for_status()
with open(local_path, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
if new_path.exists():
return new_path
if local_path.exists():
return local_path
return None
def upload(

View File

@@ -306,6 +306,17 @@ class U115Pan(StorageBase, metaclass=Singleton):
sha1.update(chunk)
return sha1.hexdigest()
def _delay_get_item(self, path: Path) -> Optional[schemas.FileItem]:
"""
自动延迟重试 get_item 模块
"""
for _ in range(2):
time.sleep(2)
fileitem = self.get_item(path)
if fileitem:
return fileitem
return None
def init_storage(self):
pass
@@ -513,7 +524,32 @@ class U115Pan(StorageBase, metaclass=Singleton):
# Step 3: 秒传
if init_result.get("status") == 2:
logger.info(f"【115】{target_name} 秒传成功")
return self.get_item(target_path)
file_id = init_result.get("file_id", None)
if file_id:
logger.debug(f"【115】{target_name} 使用秒传返回ID获取文件信息")
time.sleep(2)
info_resp = self._request_api(
"GET",
"/open/folder/get_info",
"data",
params={
"file_id": int(file_id)
}
)
if info_resp:
return schemas.FileItem(
storage=self.schema.value,
fileid=str(info_resp["file_id"]),
path=str(target_path) + ("/" if info_resp["file_category"] == "0" else ""),
type="file" if info_resp["file_category"] == "1" else "dir",
name=info_resp["file_name"],
basename=Path(info_resp["file_name"]).stem,
extension=Path(info_resp["file_name"]).suffix[1:] if info_resp["file_category"] == "1" else None,
pickcode=info_resp["pick_code"],
size=StringUtils.num_filesize(info_resp['size']) if info_resp["file_category"] == "1" else None,
modify_time=info_resp["utime"]
)
return self._delay_get_item(target_path)
# Step 4: 获取上传凭证
token_resp = self._request_api(
@@ -618,7 +654,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
logger.error(f"【115】{target_name} 上传失败: {e.status}, 错误码: {e.code}, 详情: {e.message}")
return None
# 返回结果
return self.get_item(target_path)
return self._delay_get_item(target_path)
def download(self, fileitem: schemas.FileItem, path: Path = None) -> Optional[Path]:
"""
@@ -783,7 +819,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
return False
if resp["state"]:
new_path = Path(path) / fileitem.name
new_item = self.get_item(new_path)
new_item = self._delay_get_item(new_path)
self.rename(new_item, new_name)
# 更新缓存
del self._id_cache[fileitem.path]
@@ -811,7 +847,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
return False
if resp["state"]:
new_path = Path(path) / fileitem.name
new_file = self.get_item(new_path)
new_file = self._delay_get_item(new_path)
self.rename(new_file, new_name)
# 更新缓存
del self._id_cache[fileitem.path]

View File

@@ -0,0 +1,921 @@
import re
from pathlib import Path
from threading import Lock
from typing import Optional, List, Tuple
from jinja2 import Template
from app.core.config import settings
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.meta import MetaBase
from app.core.metainfo import MetaInfoPath
from app.helper.message import TemplateHelper
from app.log import logger
from app.modules.filemanager.storages import StorageBase
from app.schemas import TransferInfo, TmdbEpisode, TransferDirectoryConf, FileItem, TransferInterceptEventData
from app.schemas.types import MediaType, ChainEventType
from app.utils.system import SystemUtils
from app.schemas import TransferRenameEventData
lock = Lock()
class TransHandler:
"""
文件转移整理类
"""
result: Optional[TransferInfo] = None
inner_lock: Lock = Lock()
def __init__(self):
self.__reset_result()
def __reset_result(self):
"""
重置结果
"""
self.result = TransferInfo()
def __set_result(self, **kwargs):
"""
设置结果
"""
with self.inner_lock:
# 设置值
for key, value in kwargs.items():
if hasattr(self.result, key):
current_value = getattr(self.result, key)
if current_value is None:
current_value = value
elif isinstance(current_value, list):
if isinstance(value, list):
current_value.extend(value)
else:
current_value.append(value)
elif isinstance(current_value, dict):
if isinstance(value, dict):
current_value.update(value)
else:
current_value[key] = value
elif isinstance(current_value, bool):
current_value = value
elif isinstance(current_value, int):
current_value += (value or 0)
else:
current_value = value
setattr(self.result, key, current_value)
def transfer_media(self,
fileitem: FileItem,
in_meta: MetaBase,
mediainfo: MediaInfo,
target_storage: str,
target_path: Path,
transfer_type: str,
source_oper: StorageBase,
target_oper: StorageBase,
need_scrape: Optional[bool] = False,
need_rename: Optional[bool] = True,
need_notify: Optional[bool] = True,
overwrite_mode: Optional[str] = None,
episodes_info: List[TmdbEpisode] = None
) -> TransferInfo:
"""
识别并整理一个文件或者一个目录下的所有文件
:param fileitem: 整理的文件对象,可能是一个文件也可以是一个目录
:param in_meta预识别元数据
:param mediainfo: 媒体信息
:param target_storage: 目标存储
:param target_path: 目标路径
:param transfer_type: 文件整理方式
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param need_scrape: 是否需要刮削
:param need_rename: 是否需要重命名
:param need_notify: 是否需要通知
:param overwrite_mode: 覆盖模式
:param episodes_info: 当前季的全部集信息
:return: TransferInfo、错误信息
"""
# 重置结果
self.__reset_result()
# 重命名格式
rename_format = settings.TV_RENAME_FORMAT \
if mediainfo.type == MediaType.TV else settings.MOVIE_RENAME_FORMAT
# 判断是否为文件夹
if fileitem.type == "dir":
# 整理整个目录,一般为蓝光原盘
if need_rename:
new_path = self.get_rename_path(
path=target_path,
template_string=rename_format,
rename_dict=self.get_naming_dict(meta=in_meta,
mediainfo=mediainfo)
).parent
else:
new_path = target_path / fileitem.name
# 整理目录
new_diritem, errmsg = self.__transfer_dir(fileitem=fileitem,
mediainfo=mediainfo,
source_oper=source_oper,
target_oper=target_oper,
target_storage=target_storage,
target_path=new_path,
transfer_type=transfer_type)
if not new_diritem:
logger.error(f"文件夹 {fileitem.path} 整理失败:{errmsg}")
self.__set_result(success=False,
message=errmsg,
fileitem=fileitem,
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
logger.info(f"文件夹 {fileitem.path} 整理成功")
# 计算目录下所有文件大小
total_size = sum(file.stat().st_size for file in Path(fileitem.path).rglob('*') if file.is_file())
# 返回整理后的路径
self.__set_result(success=True,
fileitem=fileitem,
target_item=new_diritem,
target_diritem=new_diritem,
total_size=total_size,
need_scrape=need_scrape,
need_notify=need_notify,
transfer_type=transfer_type)
return self.result
else:
# 整理单个文件
if mediainfo.type == MediaType.TV:
# 电视剧
if in_meta.begin_episode is None:
logger.warn(f"文件 {fileitem.path} 整理失败:未识别到文件集数")
self.__set_result(success=False,
message=f"未识别到文件集数",
fileitem=fileitem,
fail_list=[fileitem.path],
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
# 文件结束季为空
in_meta.end_season = None
# 文件总季数为1
if in_meta.total_season:
in_meta.total_season = 1
# 文件不可能超过2集
if in_meta.total_episode > 2:
in_meta.total_episode = 1
in_meta.end_episode = None
# 目的文件名
if need_rename:
new_file = self.get_rename_path(
path=target_path,
template_string=rename_format,
rename_dict=self.get_naming_dict(
meta=in_meta,
mediainfo=mediainfo,
episodes_info=episodes_info,
file_ext=f".{fileitem.extension}"
)
)
else:
new_file = target_path / fileitem.name
# 判断是否要覆盖
overflag = False
# 计算重命名中的文件夹层级
rename_format_level = len(rename_format.split("/")) - 1
folder_path = new_file.parents[rename_format_level - 1]
# 目标目录
target_diritem = target_oper.get_folder(folder_path)
if not target_diritem:
logger.error(f"目标目录 {folder_path} 获取失败")
self.__set_result(success=False,
message=f"目标目录 {folder_path} 获取失败",
fileitem=fileitem,
fail_list=[fileitem.path],
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
# 目标文件
target_item = target_oper.get_item(new_file)
if target_item:
# 目标文件已存在
target_file = new_file
if target_storage == "local" and new_file.is_symlink():
target_file = new_file.readlink()
if not target_file.exists():
overflag = True
if not overflag:
# 目标文件已存在
logger.info(f"目的文件系统中已经存在同名文件 {target_file},当前整理覆盖模式设置为 {overwrite_mode}")
if overwrite_mode == 'always':
# 总是覆盖同名文件
overflag = True
elif overwrite_mode == 'size':
# 存在时大覆盖小
if target_item.size < fileitem.size:
logger.info(f"目标文件文件大小更小,将覆盖:{new_file}")
overflag = True
else:
self.__set_result(success=False,
message=f"媒体库存在同名文件,且质量更好",
fileitem=fileitem,
target_item=target_item,
target_diritem=target_diritem,
fail_list=[fileitem.path],
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
elif overwrite_mode == 'never':
# 存在不覆盖
self.__set_result(success=False,
message=f"媒体库存在同名文件,当前覆盖模式为不覆盖",
fileitem=fileitem,
target_item=target_item,
target_diritem=target_diritem,
fail_list=[fileitem.path],
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
elif overwrite_mode == 'latest':
# 仅保留最新版本
logger.info(f"当前整理覆盖模式设置为仅保留最新版本,将覆盖:{new_file}")
overflag = True
else:
if overwrite_mode == 'latest':
# 文件不存在,但仅保留最新版本
logger.info(f"当前整理覆盖模式设置为 {overwrite_mode},仅保留最新版本,正在删除已有版本文件 ...")
self.__delete_version_files(target_oper, new_file)
# 整理文件
new_item, err_msg = self.__transfer_file(fileitem=fileitem,
mediainfo=mediainfo,
target_storage=target_storage,
target_file=new_file,
transfer_type=transfer_type,
over_flag=overflag,
source_oper=source_oper,
target_oper=target_oper)
if not new_item:
logger.error(f"文件 {fileitem.path} 整理失败:{err_msg}")
self.__set_result(success=False,
message=err_msg,
fileitem=fileitem,
fail_list=[fileitem.path],
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
logger.info(f"文件 {fileitem.path} 整理成功")
self.__set_result(success=True,
fileitem=fileitem,
target_item=new_item,
target_diritem=target_diritem,
need_scrape=need_scrape,
transfer_type=transfer_type,
need_notify=need_notify)
return self.result
@staticmethod
def __transfer_command(fileitem: FileItem, target_storage: str,
source_oper: StorageBase, target_oper: StorageBase,
target_file: Path, transfer_type: str,
) -> Tuple[Optional[FileItem], str]:
"""
处理单个文件
:param fileitem: 源文件
:param target_storage: 目标存储
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_file: 目标文件路径
:param transfer_type: 整理方式
"""
def __get_targetitem(_path: Path) -> FileItem:
"""
获取文件信息
"""
return FileItem(
storage=target_storage,
path=str(_path).replace("\\", "/"),
name=_path.name,
basename=_path.stem,
type="file",
size=_path.stat().st_size,
extension=_path.suffix.lstrip('.'),
modify_time=_path.stat().st_mtime
)
if (fileitem.storage != target_storage
and fileitem.storage != "local" and target_storage != "local"):
return None, f"不支持 {fileitem.storage}{target_storage} 的文件整理"
# 加锁
with lock:
if fileitem.storage == "local" and target_storage == "local":
# 创建目录
if not target_file.parent.exists():
target_file.parent.mkdir(parents=True)
# 本地到本地
if transfer_type == "copy":
state = source_oper.copy(fileitem, target_file.parent, target_file.name)
elif transfer_type == "move":
state = source_oper.move(fileitem, target_file.parent, target_file.name)
elif transfer_type == "link":
state = source_oper.link(fileitem, target_file)
elif transfer_type == "softlink":
state = source_oper.softlink(fileitem, target_file)
else:
return None, f"不支持的整理方式:{transfer_type}"
if state:
return __get_targetitem(target_file), ""
else:
return None, f"{fileitem.path} {transfer_type} 失败"
elif fileitem.storage == "local" and target_storage != "local":
# 本地到网盘
filepath = Path(fileitem.path)
if not filepath.exists():
return None, f"文件 {filepath} 不存在"
if transfer_type == "copy":
# 复制
# 根据目的路径创建文件夹
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
# 上传文件
new_item = target_oper.upload(target_fileitem, filepath, target_file.name)
if new_item:
return new_item, ""
else:
return None, f"{fileitem.path} 上传 {target_storage} 失败"
else:
return None, f"{target_storage}{target_file.parent} 目录获取失败"
elif transfer_type == "move":
# 移动
# 根据目的路径获取文件夹
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
# 上传文件
new_item = target_oper.upload(target_fileitem, filepath, target_file.name)
if new_item:
# 删除源文件
source_oper.delete(fileitem)
return new_item, ""
else:
return None, f"{fileitem.path} 上传 {target_storage} 失败"
else:
return None, f"{target_storage}{target_file.parent} 目录获取失败"
elif fileitem.storage != "local" and target_storage == "local":
# 网盘到本地
if target_file.exists():
logger.warn(f"文件已存在:{target_file}")
return __get_targetitem(target_file), ""
# 网盘到本地
if transfer_type in ["copy", "move"]:
# 下载
tmp_file = source_oper.download(fileitem=fileitem, path=target_file.parent)
if tmp_file:
# 创建目录
if not target_file.parent.exists():
target_file.parent.mkdir(parents=True)
# 将tmp_file移动后target_file
SystemUtils.move(tmp_file, target_file)
if transfer_type == "move":
# 删除源文件
source_oper.delete(fileitem)
return __get_targetitem(target_file), ""
else:
return None, f"{fileitem.path} {fileitem.storage} 下载失败"
elif fileitem.storage == target_storage:
# 同一网盘
if transfer_type == "copy":
# 复制文件到新目录
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
if source_oper.move(fileitem, Path(target_fileitem.path), target_file.name):
return target_oper.get_item(target_file), ""
else:
return None, f"{target_storage}{fileitem.path} 复制文件失败"
else:
return None, f"{target_storage}{target_file.parent} 目录获取失败"
elif transfer_type == "move":
# 移动文件到新目录
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
if source_oper.move(fileitem, Path(target_fileitem.path), target_file.name):
return target_oper.get_item(target_file), ""
else:
return None, f"{target_storage}{fileitem.path} 移动文件失败"
else:
return None, f"{target_storage}{target_file.parent} 目录获取失败"
else:
return None, f"不支持的整理方式:{transfer_type}"
return None, "未知错误"
def __transfer_other_files(self, fileitem: FileItem, target_storage: str,
source_oper: StorageBase, target_oper: StorageBase,
target_file: Path, transfer_type: str) -> Tuple[bool, str]:
"""
根据文件名整理其他相关文件
:param fileitem: 源文件
:param target_storage: 目标存储
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_file: 目标路径
:param transfer_type: 整理方式
"""
# 整理字幕
state, errmsg = self.__transfer_subtitles(fileitem=fileitem,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=target_file,
transfer_type=transfer_type)
if not state:
return False, errmsg
# 整理音轨文件
state, errmsg = self.__transfer_audio_track_files(fileitem=fileitem,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=target_file,
transfer_type=transfer_type)
return state, errmsg
def __transfer_subtitles(self, fileitem: FileItem, target_storage: str,
source_oper: StorageBase, target_oper: StorageBase,
target_file: Path, transfer_type: str) -> Tuple[bool, str]:
"""
根据文件名整理对应字幕文件
:param fileitem: 源文件
:param target_storage: 目标存储
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_file: 目标路径
:param transfer_type: 整理方式
"""
# 字幕正则式
_zhcn_sub_re = r"([.\[(](((zh[-_])?(cn|ch[si]|sg|sc))|zho?" \
r"|chinese|(cn|ch[si]|sg|zho?|eng)[-_&]?(cn|ch[si]|sg|zho?|eng)" \
r"|简[体中]?)[.\])])" \
r"|([\u4e00-\u9fa5]{0,3}[中双][\u4e00-\u9fa5]{0,2}[字文语][\u4e00-\u9fa5]{0,3})" \
r"|简体|简中|JPSC|sc_jp" \
r"|(?<![a-z0-9])gb(?![a-z0-9])"
_zhtw_sub_re = r"([.\[(](((zh[-_])?(hk|tw|cht|tc))" \
r"|(cht|eng)[-_&]?(cht|eng)" \
r"|繁[体中]?)[.\])])" \
r"|繁体中[文字]|中[文字]繁体|繁体|JPTC|tc_jp" \
r"|(?<![a-z0-9])big5(?![a-z0-9])"
_eng_sub_re = r"[.\[(]eng[.\])]"
# 比对文件名并整理字幕
org_path = Path(fileitem.path)
# 查找上级文件项
parent_item: FileItem = source_oper.get_parent(fileitem)
if not parent_item:
return False, f"{org_path} 上级目录获取失败"
# 字幕文件列表
file_list: List[FileItem] = source_oper.list(parent_item) or []
file_list = [f for f in file_list if f.type == "file" and f.extension
and f".{f.extension.lower()}" in settings.RMT_SUBEXT]
if len(file_list) == 0:
logger.info(f"{parent_item.path} 目录下没有找到字幕文件...")
else:
logger.info(f"字幕文件清单:{[f.name for f in file_list]}")
# 识别文件名
metainfo = MetaInfoPath(org_path)
for sub_item in file_list:
# 识别字幕文件名
sub_file_name = re.sub(_zhtw_sub_re,
".",
re.sub(_zhcn_sub_re,
".",
sub_item.name,
flags=re.I),
flags=re.I)
sub_file_name = re.sub(_eng_sub_re, ".", sub_file_name, flags=re.I)
sub_metainfo = MetaInfoPath(Path(sub_item.path))
# 匹配字幕文件名
if (org_path.stem == Path(sub_file_name).stem) or \
(sub_metainfo.cn_name and sub_metainfo.cn_name == metainfo.cn_name) \
or (sub_metainfo.en_name and sub_metainfo.en_name == metainfo.en_name):
if metainfo.part and metainfo.part != sub_metainfo.part:
continue
if metainfo.season \
and metainfo.season != sub_metainfo.season:
continue
if metainfo.episode \
and metainfo.episode != sub_metainfo.episode:
continue
new_file_type = ""
# 兼容jellyfin字幕识别(多重识别), emby则会识别最后一个后缀
if re.search(_zhcn_sub_re, sub_item.name, re.I):
new_file_type = ".chi.zh-cn"
elif re.search(_zhtw_sub_re, sub_item.name,
re.I):
new_file_type = ".zh-tw"
elif re.search(_eng_sub_re, sub_item.name, re.I):
new_file_type = ".eng"
# 通过对比字幕文件大小 尽量整理所有存在的字幕
file_ext = f".{sub_item.extension}"
new_sub_tag_dict = {
".eng": ".英文",
".chi.zh-cn": ".简体中文",
".zh-tw": ".繁体中文"
}
new_sub_tag_list = [
(".default" + new_file_type if (
(settings.DEFAULT_SUB == "zh-cn" and new_file_type == ".chi.zh-cn") or
(settings.DEFAULT_SUB == "zh-tw" and new_file_type == ".zh-tw") or
(settings.DEFAULT_SUB == "eng" and new_file_type == ".eng")
) else new_file_type) if t == 0 else "%s%s(%s)" % (new_file_type,
new_sub_tag_dict.get(
new_file_type, ""
),
t) for t in range(6)
]
for new_sub_tag in new_sub_tag_list:
new_file: Path = target_file.with_name(target_file.stem + new_sub_tag + file_ext)
# 如果字幕文件不存在, 直接整理字幕, 并跳出循环
try:
logger.debug(f"正在处理字幕:{sub_item.name}")
new_item, errmsg = self.__transfer_command(fileitem=sub_item,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=new_file,
transfer_type=transfer_type)
if new_item:
logger.info(f"字幕 {sub_item.name} 整理完成")
self.__set_result(
subtitle_list=[sub_item.path],
subtitle_list_new=[new_item.path],
)
break
else:
logger.error(f"字幕 {sub_item.name} 整理失败:{errmsg}")
return False, errmsg
except Exception as error:
logger.info(f"字幕 {new_file} 出错了,原因: {str(error)}")
return True, ""
def __transfer_audio_track_files(self, fileitem: FileItem, target_storage: str,
source_oper: StorageBase, target_oper: StorageBase,
target_file: Path, transfer_type: str) -> Tuple[bool, str]:
"""
根据文件名整理对应音轨文件
:param fileitem: 源文件
:param target_storage: 目标存储
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_file: 目标路径
:param transfer_type: 整理方式
"""
org_path = Path(fileitem.path)
# 查找上级文件项
parent_item: FileItem = source_oper.get_parent(fileitem)
if not parent_item:
return False, f"{org_path} 上级目录获取失败"
file_list: List[FileItem] = source_oper.list(parent_item)
# 匹配音轨文件
pending_file_list: List[FileItem] = [file for file in file_list
if Path(file.name).stem == org_path.stem
and file.type == "file" and file.extension
and f".{file.extension.lower()}" in settings.RMT_AUDIOEXT]
if len(pending_file_list) == 0:
return True, f"{parent_item.path} 目录下没有找到匹配的音轨文件"
logger.debug("音轨文件清单:" + str(pending_file_list))
for track_file in pending_file_list:
track_ext = f".{track_file.extension}"
new_track_file = target_file.with_name(target_file.stem + track_ext)
try:
logger.info(f"正在整理音轨文件:{track_file}{new_track_file}")
new_item, errmsg = self.__transfer_command(fileitem=track_file,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=new_track_file,
transfer_type=transfer_type)
if new_item:
logger.info(f"音轨文件 {org_path.name} 整理完成")
self.__set_result(
audio_list=[track_file.path],
audio_list_new=[new_item.path],
)
else:
logger.error(f"音轨文件 {org_path.name} 整理失败:{errmsg}")
except Exception as error:
logger.error(f"音轨文件 {org_path.name} 整理失败:{str(error)}")
return True, ""
def __transfer_dir(self, fileitem: FileItem, mediainfo: MediaInfo,
source_oper: StorageBase, target_oper: StorageBase,
transfer_type: str, target_storage: str, target_path: Path) -> Tuple[Optional[FileItem], str]:
"""
整理整个文件夹
:param fileitem: 源文件
:param mediainfo: 媒体信息
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param transfer_type: 整理方式
:param target_storage: 目标存储
:param target_path: 目标路径
"""
logger.info(f"正在整理目录:{fileitem.path}{target_path}")
target_item = target_oper.get_folder(target_path)
if not target_item:
return None, f"获取目标目录失败:{target_path}"
event_data = TransferInterceptEventData(
fileitem=fileitem,
mediainfo=mediainfo,
target_storage=target_storage,
target_path=target_path,
transfer_type=transfer_type
)
event = eventmanager.send_event(ChainEventType.TransferIntercept, event_data)
if event and event.event_data:
event_data = event.event_data
# 如果事件被取消,跳过文件整理
if event_data.cancel:
logger.debug(
f"Transfer dir canceled by event: {event_data.source},"
f"Reason: {event_data.reason}")
return None, event_data.reason
# 处理所有文件
state, errmsg = self.__transfer_dir_files(fileitem=fileitem,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_path=target_path,
transfer_type=transfer_type)
if state:
return target_item, errmsg
else:
return None, errmsg
def __transfer_dir_files(self, fileitem: FileItem, target_storage: str,
source_oper: StorageBase, target_oper: StorageBase,
transfer_type: str, target_path: Path) -> Tuple[bool, str]:
"""
按目录结构整理目录下所有文件
:param fileitem: 源文件
:param target_storage: 目标存储
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_path: 目标路径
:param transfer_type: 整理方式
"""
file_list: List[FileItem] = source_oper.list(fileitem)
# 整理文件
for item in file_list:
if item.type == "dir":
# 递归整理目录
new_path = target_path / item.name
state, errmsg = self.__transfer_dir_files(fileitem=item,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
transfer_type=transfer_type,
target_path=new_path)
if not state:
return False, errmsg
else:
# 整理文件
new_file = target_path / item.name
new_item, errmsg = self.__transfer_command(fileitem=item,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=new_file,
transfer_type=transfer_type)
if not new_item:
return False, errmsg
self.__set_result(
file_list=[item.path],
file_list_new=[new_item.path],
)
# 返回成功
return True, ""
def __transfer_file(self, fileitem: FileItem, mediainfo: MediaInfo,
source_oper: StorageBase, target_oper: StorageBase,
target_storage: str, target_file: Path,
transfer_type: str, over_flag: Optional[bool] = False) -> Tuple[Optional[FileItem], str]:
"""
整理一个文件,同时处理其他相关文件
:param fileitem: 原文件
:param mediainfo: 媒体信息
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
:param target_storage: 目标存储
:param target_file: 新文件
:param transfer_type: 整理方式
:param over_flag: 是否覆盖为True时会先删除再整理
:param source_oper: 源存储操作对象
:param target_oper: 目标存储操作对象
"""
logger.info(f"正在整理文件:【{fileitem.storage}{fileitem.path} 到 【{target_storage}{target_file}"
f"操作类型:{transfer_type}")
event_data = TransferInterceptEventData(
fileitem=fileitem,
mediainfo=mediainfo,
target_storage=target_storage,
target_path=target_file,
transfer_type=transfer_type,
options={
"over_flag": over_flag
}
)
event = eventmanager.send_event(ChainEventType.TransferIntercept, event_data)
if event and event.event_data:
event_data = event.event_data
# 如果事件被取消,跳过文件整理
if event_data.cancel:
logger.debug(
f"Transfer file canceled by event: {event_data.source},"
f"Reason: {event_data.reason}")
return None, event_data.reason
if target_storage == "local" and (target_file.exists() or target_file.is_symlink()):
if not over_flag:
logger.warn(f"文件已存在:{target_file}")
return None, f"{target_file} 已存在"
else:
logger.info(f"正在删除已存在的文件:{target_file}")
target_file.unlink()
else:
exists_item = target_oper.get_item(target_file)
if exists_item:
if not over_flag:
logger.warn(f"文件已存在:【{target_storage}{target_file}")
return None, f"{target_storage}{target_file} 已存在"
else:
logger.info(f"正在删除已存在的文件:【{target_storage}{target_file}")
target_oper.delete(exists_item)
# 执行文件整理命令
new_item, errmsg = self.__transfer_command(fileitem=fileitem,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=target_file,
transfer_type=transfer_type)
if new_item:
self.__set_result(
file_list=[fileitem.path],
file_list_new=[new_item.path],
file_count=1,
total_size=fileitem.size,
)
# 处理其他相关文件
self.__transfer_other_files(fileitem=fileitem,
target_storage=target_storage,
source_oper=source_oper,
target_oper=target_oper,
target_file=target_file,
transfer_type=transfer_type)
return new_item, errmsg
return None, errmsg
@staticmethod
def get_dest_path(mediainfo: MediaInfo, target_path: Path,
need_type_folder: Optional[bool] = False, need_category_folder: Optional[bool] = False):
"""
获取目标路径
"""
if need_type_folder:
target_path = target_path / mediainfo.type.value
if need_category_folder and mediainfo.category:
target_path = target_path / mediainfo.category
return target_path
@staticmethod
def get_dest_dir(mediainfo: MediaInfo, target_dir: TransferDirectoryConf,
need_type_folder: Optional[bool] = None, need_category_folder: Optional[bool] = None) -> Path:
"""
根据设置并装媒体库目录
:param mediainfo: 媒体信息
:param target_dir: 媒体库根目录
:param need_type_folder: 是否需要按媒体类型创建目录
:param need_category_folder: 是否需要按媒体类别创建目录
"""
if need_type_folder is None:
need_type_folder = target_dir.library_type_folder
if need_category_folder is None:
need_category_folder = target_dir.library_category_folder
if not target_dir.media_type and need_type_folder:
# 一级自动分类
library_dir = Path(target_dir.library_path) / mediainfo.type.value
elif target_dir.media_type and need_type_folder:
# 一级手动分类
library_dir = Path(target_dir.library_path) / target_dir.media_type
else:
library_dir = Path(target_dir.library_path)
if not target_dir.media_category and need_category_folder and mediainfo.category:
# 二级自动分类
library_dir = library_dir / mediainfo.category
elif target_dir.media_category and need_category_folder:
# 二级手动分类
library_dir = library_dir / target_dir.media_category
return library_dir
@staticmethod
def get_naming_dict(meta: MetaBase, mediainfo: MediaInfo, file_ext: Optional[str] = None,
episodes_info: List[TmdbEpisode] = None) -> dict:
"""
根据媒体信息返回Format字典
:param meta: 文件元数据
:param mediainfo: 识别的媒体信息
:param file_ext: 文件扩展名
:param episodes_info: 当前季的全部集信息
"""
return TemplateHelper().builder.build(meta=meta, mediainfo=mediainfo,
file_extension=file_ext, episodes_info=episodes_info)
@staticmethod
def __delete_version_files(storage_oper: StorageBase, path: Path) -> bool:
"""
删除目录下的所有版本文件
:param storage_oper: 存储操作对象
:param path: 目录路径
"""
# 存储
if not storage_oper:
return False
# 识别文件中的季集信息
meta = MetaInfoPath(path)
season = meta.season
episode = meta.episode
logger.warn(f"正在删除目标目录中其它版本的文件:{path.parent}")
# 获取父目录
parent_item = storage_oper.get_item(path.parent)
if not parent_item:
logger.warn(f"目录 {path.parent} 不存在")
return False
# 检索媒体文件
media_files = storage_oper.list(parent_item)
if not media_files:
logger.info(f"目录 {path.parent} 中没有文件")
return False
# 删除文件
for media_file in media_files:
media_path = Path(media_file.path)
if media_path == path:
continue
if media_file.type != "file":
continue
if f".{media_file.extension.lower()}" not in settings.RMT_MEDIAEXT:
continue
# 识别文件中的季集信息
filemeta = MetaInfoPath(media_path)
# 相同季集的文件才删除
if filemeta.season != season or filemeta.episode != episode:
continue
logger.info(f"正在删除文件:{media_file.name}")
storage_oper.delete(media_file)
return True
@staticmethod
def get_rename_path(template_string: str, rename_dict: dict, path: Path = None) -> Path:
"""
生成重命名后的完整路径,支持智能重命名事件
:param template_string: Jinja2 模板字符串
:param rename_dict: 渲染上下文,用于替换模板中的变量
:param path: 可选的基础路径,如果提供,将在其基础上拼接生成的路径
:return: 生成的完整路径
"""
# 创建jinja2模板对象
template = Template(template_string)
# 渲染生成的字符串
render_str = template.render(rename_dict)
logger.debug(f"Initial render string: {render_str}")
# 发送智能重命名事件
event_data = TransferRenameEventData(
template_string=template_string,
rename_dict=rename_dict,
render_str=render_str,
path=path
)
event = eventmanager.send_event(ChainEventType.TransferRename, event_data)
# 检查事件返回的结果
if event and event.event_data:
event_data: TransferRenameEventData = event.event_data
if event_data.updated and event_data.updated_str:
logger.debug(f"Render string updated by event: "
f"{render_str} -> {event_data.updated_str} (source: {event_data.source})")
render_str = event_data.updated_str
# 目的路径
if path:
return path / render_str
else:
return Path(render_str)

View File

@@ -21,8 +21,8 @@ class FilterModule(_ModuleBase):
rule_set: Dict[str, dict] = {
# 蓝光原盘
"BLU": {
"include": [r'Blu-?Ray.+VC-?1|Blu-?Ray.+AVC|UHD.+blu-?ray.+HEVC|MiniBD'],
"exclude": [r'[Hx].?264|[Hx].?265|WEB-?DL|WEB-?RIP|REMUX']
"include": [r'(?i)(\bBlu-?Ray\b.*\b(?:VC-?1|AVC|MPEG-?2)\b|\b(?:UHD|4K|2160p)\b(?:.*Blu-?Ray)?.*\b(?:HEVC|H\.?265)\b|\bBlu-?Ray\b.*\b(?:UHD|4K|2160p)\b.*\b(?:HEVC|H\.?265)\b|\b(?:COMPLETE|FULL)\b.*\b(?:(?:UHD|4K|2160p)\b.*)?Blu-?Ray\b|\b(BD25|BD50|BD66|BD100|BDMV|MiniBD)\b)'],
"exclude": [r'(?i)(\b[XH]\.?264\b|\b[XH]\.?265\b|\bWEB-?DL\b|\bWEB-?RIP\b|\bHDTV(?:RIP)?\b|\bREMUX\b|\bBDRip\b|\bBRRip\b|\bHDRip\b|\bENCODE\b|\b(?<!WEB-|HDTV)RIP\b)']
},
# 4K
"4K": {

View File

@@ -238,7 +238,11 @@ class IndexerModule(_ModuleBase):
cat=cat,
page=page)
return _spider.is_error, _spider.get_torrents()
try:
return _spider.is_error, _spider.get_torrents()
finally:
# 显式清理SiteSpider对象
del _spider
def refresh_torrents(self, site: dict,
keyword: Optional[str] = None, cat: Optional[str] = None, page: Optional[int] = 0) -> Optional[List[TorrentInfo]]:

View File

@@ -17,7 +17,7 @@ from app.utils.site import SiteUtils
# 站点框架
class SiteSchema(Enum):
DiscuzX = "Discuz!"
DiscuzX = "DiscuzX"
Gazelle = "Gazelle"
Ipt = "IPTorrents"
NexusPhp = "NexusPhp"

View File

@@ -2,12 +2,12 @@ from typing import Any, Generator, List, Optional, Tuple, Union
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _MediaServerBase, _ModuleBase
from app.modules.jellyfin.jellyfin import Jellyfin
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class JellyfinModule(_ModuleBase, _MediaServerBase[Jellyfin]):
@@ -19,6 +19,19 @@ class JellyfinModule(_ModuleBase, _MediaServerBase[Jellyfin]):
super().init_service(service_name=Jellyfin.__name__.lower(),
service_type=lambda conf: Jellyfin(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Jellyfin"

View File

@@ -10,7 +10,7 @@ from app.log import logger
from app.schemas import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Jellyfin:

View File

@@ -2,12 +2,12 @@ from typing import Optional, Tuple, Union, Any, List, Generator
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MediaServerBase
from app.modules.plex.plex import Plex
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType
from app.schemas.types import MediaType, ModuleType, ChainEventType, MediaServerType, SystemConfigKey, EventType
class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
@@ -19,6 +19,19 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
super().init_service(service_name=Plex.__name__.lower(),
service_type=lambda conf: Plex(**conf.config, sync_libraries=conf.sync_libraries))
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Plex"
@@ -45,7 +58,12 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
return 3
def stop(self):
pass
"""
停止模块服务
"""
for server in self.get_instances().values():
if server:
server.close()
def test(self) -> Optional[Tuple[bool, str]]:
"""
@@ -273,7 +291,8 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
episodes=episodes
) for season, episodes in seasoninfo.items()]
def mediaserver_playing(self, server: str, count: Optional[int] = 20, **kwargs) -> List[schemas.MediaServerPlayItem]:
def mediaserver_playing(self, server: str, count: Optional[int] = 20,
**kwargs) -> List[schemas.MediaServerPlayItem]:
"""
获取媒体服务器正在播放信息
"""

View File

@@ -14,7 +14,7 @@ from app.log import logger
from app.schemas import MediaType
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
from schemas import MediaServerItem
from app.schemas import MediaServerItem
class Plex:
@@ -890,3 +890,7 @@ class Plex:
session = Session()
session.headers = headers
return session
def close(self):
if self._session:
self._session.close()

View File

@@ -7,11 +7,12 @@ from torrentool.torrent import Torrent
from app import schemas
from app.core.config import settings
from app.core.metainfo import MetaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _DownloaderBase
from app.modules.qbittorrent.qbittorrent import Qbittorrent
from app.schemas import TransferTorrent, DownloadingTorrent
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType, SystemConfigKey, EventType
from app.utils.string import StringUtils
@@ -24,6 +25,19 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
super().init_service(service_name=Qbittorrent.__name__.lower(),
service_type=Qbittorrent)
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Qbittorrent"
@@ -286,7 +300,8 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
dlspeed=StringUtils.str_filesize(torrent.get('dlspeed')),
upspeed=StringUtils.str_filesize(torrent.get('upspeed')),
left_time=StringUtils.str_secends(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get('dlspeed')) if torrent.get(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get(
'dlspeed')) if torrent.get(
'dlspeed') > 0 else ''
))
else:

View File

@@ -3,11 +3,12 @@ import re
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.slack.slack import Slack
from app.schemas import MessageChannel, CommingMessage, Notification
from app.schemas.types import ModuleType
from app.schemas import MessageChannel, CommingMessage, Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, SystemConfigKey, EventType
class SlackModule(_ModuleBase, _MessageBase[Slack]):
@@ -20,6 +21,19 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
service_type=Slack)
self._channel = MessageChannel.Slack
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Slack"

View File

@@ -1,11 +1,12 @@
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.synologychat.synologychat import SynologyChat
from app.schemas import MessageChannel, CommingMessage, Notification
from app.schemas.types import ModuleType
from app.schemas import MessageChannel, CommingMessage, Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, SystemConfigKey, EventType
class SynologyChatModule(_ModuleBase, _MessageBase[SynologyChat]):
@@ -18,6 +19,19 @@ class SynologyChatModule(_ModuleBase, _MessageBase[SynologyChat]):
service_type=SynologyChat)
self._channel = MessageChannel.SynologyChat
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Synology Chat"

View File

@@ -1,14 +1,16 @@
import copy
import json
from typing import Optional, Union, List, Tuple, Any, Dict
from typing import Dict
from typing import Optional, Union, List, Tuple, Any
from app.core.context import MediaInfo, Context
from app.core.event import Event
from app.core.event import eventmanager
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.telegram.telegram import Telegram
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData
from app.schemas.types import ModuleType, ChainEventType
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData, ConfigChangeEventData
from app.schemas.types import ModuleType, ChainEventType, SystemConfigKey, EventType
from app.utils.structures import DictUtils
@@ -22,6 +24,19 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
service_type=Telegram)
self._channel = MessageChannel.Telegram
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Telegram"

View File

@@ -406,7 +406,8 @@ class TheMovieDbModule(_ModuleBase):
return None
return self.scraper.get_metadata_nfo(meta=meta, mediainfo=mediainfo, season=season, episode=episode)
def metadata_img(self, mediainfo: MediaInfo, season: Optional[int] = None, episode: Optional[int] = None) -> Optional[dict]:
def metadata_img(self, mediainfo: MediaInfo, season: Optional[int] = None,
episode: Optional[int] = None) -> Optional[dict]:
"""
获取图片名称和url
:param mediainfo: 媒体信息
@@ -506,7 +507,6 @@ class TheMovieDbModule(_ModuleBase):
air_date=sea.get("episodes")[0].get("air_date") if sea.get("episodes") else None,
) for sea in group_seasons]
def tmdb_episodes(self, tmdbid: int, season: int, episode_group: Optional[str] = None) -> List[schemas.TmdbEpisode]:
"""
根据TMDBID查询某季的所有集信息

View File

@@ -636,7 +636,6 @@ class TmdbApi:
return None
# dict[地区:分级]
ratings = {}
results = []
if results := (tmdb_info.get("release_dates") or {}).get("results"):
"""
[
@@ -1334,7 +1333,18 @@ class TmdbApi:
return []
try:
logger.debug(f"正在获取剧集组:{group_id}...")
return self.tv.group_episodes(group_id) or []
group_seasons = self.tv.group_episodes(group_id) or []
return [
{
**group_season,
"episodes": [
{**ep, "episode_number": idx}
# 剧集组中每个季的episode_number从1开始
for idx, ep in enumerate(group_season.get("episodes", []), start=1)
]
}
for group_season in group_seasons
]
except Exception as e:
logger.error(str(e))
return []
@@ -1348,14 +1358,9 @@ class TmdbApi:
return {}
for group_season in group_seasons:
if group_season.get('order') == season:
# 剧集组中每个季的episode_number从1开始
for i, e in enumerate(group_season.get('episodes', []), start=1):
e['episode_number'] = i
return group_season
return {}
def get_person_detail(self, person_id: int) -> dict:
"""
获取人物详情

View File

@@ -1,21 +1,92 @@
from threading import Lock
from typing import Optional, Tuple, Union
from app.core.config import settings
from app.log import logger
from app.modules import _ModuleBase
from app.modules.thetvdb import tvdbapi
from app.modules.thetvdb import tvdb_v4_official
from app.schemas.types import ModuleType, MediaRecognizeType
from app.utils.http import RequestUtils
class TheTvDbModule(_ModuleBase):
tvdb: tvdbapi.Tvdb = None
"""
TVDB媒体信息匹配
"""
__timeout: int = 15
tvdb: Optional[tvdb_v4_official.TVDB] = None
__auth_lock = Lock()
def init_module(self) -> None:
self.tvdb = tvdbapi.Tvdb(apikey=settings.TVDB_API_KEY,
cache=False,
select_first=True,
proxies=settings.PROXY)
pass
def _initialize_tvdb_session(self, is_retry: bool = False) -> None:
"""
创建或刷新 TVDB 登录会话。
:param is_retry: 是否是由于token失效后的重试登录
"""
action = "刷新" if is_retry else "创建"
logger.info(f"开始{action}TVDB登录会话...")
try:
if not settings.TVDB_V4_API_KEY:
raise ConnectionError("TVDB API Key 未配置,无法初始化会话。")
self.tvdb = tvdb_v4_official.TVDB(apikey=settings.TVDB_V4_API_KEY,
pin=settings.TVDB_V4_API_PIN,
proxy=settings.PROXY,
timeout=self.__timeout)
if self.tvdb:
logger.info(f"TVDB登录会话{action}成功。")
else:
raise ValueError(f"TVDB登录会话{action}后实例仍为None。")
except Exception as e:
self.tvdb = None
raise ConnectionError(f"TVDB登录会话{action}失败: {str(e)}") from e
def _ensure_tvdb_session(self, is_retry: bool = False) -> None:
"""
确保TVDB会话存在。如果不存在或需要强制重新初始化则进行初始化。
:param is_retry: 是否重新初始化例如token失效时
"""
# 第一次检查 (无锁),提高性能,避免不必要锁竞争
if not self.tvdb or is_retry:
with self.__auth_lock:
# 第二次检查 (有锁),防止多个线程都通过第一次检查后重复初始化
if not self.tvdb or is_retry:
self._initialize_tvdb_session(is_retry=is_retry)
def _handle_tvdb_call(self, method_name: str, *args, **kwargs):
"""
包裹 TVDB 调用,处理 token 失效情况并尝试重新初始化
:param method_name: 要在 self.tvdb 实例上调用的方法的名称 (字符串)
"""
try:
self._ensure_tvdb_session()
actual_method = getattr(self.tvdb, method_name)
return actual_method(*args, **kwargs)
except ValueError as e:
if "Unauthorized" in str(e):
logger.warning("TVDB Token 可能已失效,正在尝试重新登录...")
try:
self._ensure_tvdb_session(is_retry=True)
actual_method = getattr(self.tvdb, method_name)
return actual_method(*args, **kwargs)
except ConnectionError as conn_err:
logger.error(f"TVDB Token失效后重新登录失败: {conn_err}")
raise
elif "NotFoundException" in str(e) or "ID not found" in str(e):
logger.warning(f"TVDB 资源未找到 (调用 {method_name}): {e}")
return None
else:
logger.error(f"TVDB 调用 ({method_name}) 时发生未处理的 ValueError: {str(e)}")
raise
except ConnectionError as e:
logger.error(f"TVDB 连接会话错误: {str(e)}")
raise
except AttributeError as e:
logger.error(f"TVDB 实例上没有方法 '{method_name}': {e}")
raise
except Exception as e:
logger.error(f"TVDB 调用时发生未知错误: {str(e)}", exc_info=True)
raise
@staticmethod
def get_name() -> str:
@@ -43,18 +114,19 @@ class TheTvDbModule(_ModuleBase):
return 4
def stop(self):
self.tvdb.close()
logger.info("TheTvDbModule 停止。正在清除 TVDB 会话。")
with self.__auth_lock:
self.tvdb = None
def test(self) -> Tuple[bool, str]:
"""
测试模块连接性
"""
ret = RequestUtils(proxies=settings.PROXY).get_res("https://api.thetvdb.com/series/81189")
if ret and ret.status_code == 200:
try:
self._handle_tvdb_call("get_series", 81189)
return True, ""
elif ret:
return False, f"无法连接 api.thetvdb.com错误码{ret.status_code}"
return False, "api.thetvdb.com 网络连接失败"
except Exception as e:
return False, str(e)
def init_setting(self) -> Tuple[str, Union[str, bool]]:
pass
@@ -67,6 +139,26 @@ class TheTvDbModule(_ModuleBase):
"""
try:
logger.info(f"开始获取TVDB信息: {tvdbid} ...")
return self.tvdb[tvdbid].data
return self._handle_tvdb_call("get_series_extended", tvdbid)
except Exception as err:
logger.error(f"获取TVDB信息失败: {str(err)}")
return None
def search_tvdb(self, title: str) -> list:
"""
用标题搜索TVDB剧集
:param title: 标题
:return: TVDB信息
"""
try:
logger.info(f"开始用标题搜索TVDB剧集: {title} ...")
res = self._handle_tvdb_call("search", title)
if res is None:
return []
if not isinstance(res, list):
logger.warning(f"TVDB 搜索 '{title}' 未返回列表:{type(res)}")
return []
return [item for item in res if isinstance(item, dict) and item.get("type") == "series"]
except Exception as err:
logger.error(f"用标题搜索TVDB剧集失败 ({title}): {str(err)}")
return []

View File

@@ -0,0 +1,615 @@
"""Official python package for using the tvdb v4 api"""
__author__ = "Weylin Wagnon"
__version__ = "1.0.12"
import json
import urllib.parse
from http import HTTPStatus
from app.utils.http import RequestUtils
class Auth:
"""
TVDB认证类
"""
def __init__(self, url, apikey, pin="", proxy=None, timeout: int = 15):
login_info = {"apikey": apikey}
if pin != "":
login_info["pin"] = pin
login_info_bytes = json.dumps(login_info, indent=2)
try:
# 使用项目统一的RequestUtils类
req_utils = RequestUtils(proxies=proxy, timeout=timeout)
response = req_utils.post_res(
url=url,
data=login_info_bytes,
headers={"Content-Type": "application/json"}
)
if response and response.status_code == 200:
result = response.json()
self.token = result["data"]["token"]
else:
error_msg = f"登录失败,状态码: {response.status_code if response else 'None'}"
if response:
try:
error_data = response.json()
error_msg = f"Code: {response.status_code}, {error_data.get('message', '未知错误')}"
except Exception as err:
error_msg = f"Code: {response.status_code}, 响应解析失败:{err}"
raise Exception(error_msg)
except Exception as e:
raise Exception(f"TVDB认证失败: {str(e)}")
def get_token(self):
"""
获取认证token
"""
return self.token
class Request:
"""
请求处理类
"""
def __init__(self, auth_token, proxy=None, timeout=15):
self.auth_token = auth_token
self.links = None
self.proxy = proxy
self.timeout = timeout
def make_request(self, url, if_modified_since=None):
"""
向指定的 URL 发起请求并返回数据
"""
headers = {"Authorization": f"Bearer {self.auth_token}"}
if if_modified_since:
headers["If-Modified-Since"] = str(if_modified_since)
try:
# 使用项目统一的RequestUtils类
req_utils = RequestUtils(proxies=self.proxy, timeout=self.timeout)
response = req_utils.get_res(url=url, headers=headers)
if response is None:
raise ValueError(f"获取 {url} 失败\n 网络连接失败")
if response.status_code == HTTPStatus.NOT_MODIFIED:
return {
"code": HTTPStatus.NOT_MODIFIED.real,
"message": "Not-Modified",
}
if response.status_code == 200:
result = response.json()
data = result.get("data", None)
if data is not None and result.get("status", "failure") != "failure":
self.links = result.get("links", None)
return data
msg = result.get("message", "未知错误")
raise ValueError(f"获取 {url} 失败\n {str(msg)}")
else:
# 处理其他HTTP错误状态码
try:
error_data = response.json()
msg = error_data.get("message", f"HTTP {response.status_code}")
except Exception as err:
msg = f"HTTP {response.status_code} {err}"
raise ValueError(f"获取 {url} 失败\n {str(msg)}")
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"获取 {url} 失败\n {str(e)}")
class Url:
"""
URL构建类
"""
def __init__(self):
self.base_url = "https://api4.thetvdb.com/v4/"
def construct(self, url_sect, url_id=None, url_subsect=None, url_lang=None, **kwargs):
"""
构建API URL
"""
url = self.base_url + url_sect
if url_id:
url += "/" + str(url_id)
if url_subsect:
url += "/" + url_subsect
if url_lang:
url += "/" + url_lang
if kwargs:
params = {var: val for var, val in kwargs.items() if val is not None}
if params:
url += "?" + urllib.parse.urlencode(params)
return url
class TVDB:
"""
TVDB API主类
"""
def __init__(self, apikey: str, pin="", proxy=None, timeout: int = 15):
self.url = Url()
login_url = self.url.construct("login")
self.auth = Auth(login_url, apikey, pin, proxy, timeout)
auth_token = self.auth.get_token()
self.request = Request(auth_token, proxy, timeout)
def get_req_links(self) -> dict:
"""
获取上一次请求返回的链接信息(例如分页链接)
"""
return self.request.links
def get_artwork_statuses(self, meta=None, if_modified_since=None) -> list:
"""
返回艺术图状态列表
"""
url = self.url.construct("artwork/statuses", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_artwork_types(self, meta=None, if_modified_since=None) -> list:
"""
返回艺术图类型列表
"""
url = self.url.construct("artwork/types", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_artwork(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个艺术图信息的字典
"""
url = self.url.construct("artwork", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_artwork_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个艺术图的扩展信息字典
"""
url = self.url.construct("artwork", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_awards(self, meta=None, if_modified_since=None) -> list:
"""
返回奖项列表
"""
url = self.url.construct("awards", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_award(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个奖项信息的字典
"""
url = self.url.construct("awards", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_award_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个奖项的扩展信息字典
"""
url = self.url.construct("awards", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_award_categories(self, meta=None, if_modified_since=None) -> list:
"""
返回奖项类别列表
"""
url = self.url.construct("awards/categories", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_award_category(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个奖项类别信息的字典
"""
url = self.url.construct("awards/categories", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_award_category_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个奖项类别的扩展信息字典
"""
url = self.url.construct("awards/categories", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_content_ratings(self, meta=None, if_modified_since=None) -> list:
"""
返回内容分级列表
"""
url = self.url.construct("content/ratings", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_countries(self, meta=None, if_modified_since=None) -> list:
"""
返回国家列表
"""
url = self.url.construct("countries", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_companies(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回公司列表 (可分页)
"""
url = self.url.construct("companies", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_company_types(self, meta=None, if_modified_since=None) -> list:
"""
返回公司类型列表
"""
url = self.url.construct("companies/types", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_company(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个公司信息的字典
"""
url = self.url.construct("companies", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_series(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回剧集列表 (可分页)
"""
url = self.url.construct("series", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_series(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个剧集信息的字典
"""
url = self.url.construct("series", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_series_by_slug(self, slug: str, meta=None, if_modified_since=None) -> dict:
"""
通过 slug (别名) 返回单个剧集信息的字典
"""
url = self.url.construct("series/slug", slug, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_series_extended(self, id: int, meta=None, short=False, if_modified_since=None) -> dict:
"""
返回单个剧集的扩展信息字典
"""
url = self.url.construct("series", id, "extended", meta=meta, short=short)
return self.request.make_request(url, if_modified_since)
def get_series_episodes(self, id: int, season_type: str = "default", page: int = 0,
lang: str = None, meta=None, if_modified_since=None, **kwargs) -> dict:
"""
返回指定剧集和季类型的各集信息字典 (可分页,可指定语言)
"""
url = self.url.construct(
"series", id, "episodes/" + season_type, lang, page=page, meta=meta, **kwargs
)
return self.request.make_request(url, if_modified_since)
def get_series_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回剧集的指定语言翻译信息字典
"""
url = self.url.construct("series", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_series_artworks(self, id: int, lang: str, type=None, if_modified_since=None) -> dict:
"""
返回包含艺术图数组的剧集记录 (可指定语言和类型)
"""
url = self.url.construct("series", id, "artworks", lang=lang, type=type)
return self.request.make_request(url, if_modified_since)
def get_series_next_aired(self, id: int, if_modified_since=None) -> dict:
"""
返回剧集的下一播出信息字典
"""
url = self.url.construct("series", id, "nextAired")
return self.request.make_request(url, if_modified_since)
def get_all_movies(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回电影列表 (可分页)
"""
url = self.url.construct("movies", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_movie(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个电影信息的字典
"""
url = self.url.construct("movies", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_movie_by_slug(self, slug: str, meta=None, if_modified_since=None) -> dict:
"""
通过 slug (别名) 返回单个电影信息的字典
"""
url = self.url.construct("movies/slug", slug, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_movie_extended(self, id: int, meta=None, short=False, if_modified_since=None) -> dict:
"""
返回电影的扩展信息字典
"""
url = self.url.construct("movies", id, "extended", meta=meta, short=short)
return self.request.make_request(url, if_modified_since)
def get_movie_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回电影的指定语言翻译信息字典
"""
url = self.url.construct("movies", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_seasons(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回季列表 (可分页)
"""
url = self.url.construct("seasons", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_season(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单季信息的字典
"""
url = self.url.construct("seasons", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_season_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单季的扩展信息字典
"""
url = self.url.construct("seasons", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_season_types(self, meta=None, if_modified_since=None) -> list:
"""
返回季类型列表
"""
url = self.url.construct("seasons/types", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_season_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回季的指定语言翻译信息字典
"""
url = self.url.construct("seasons", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_episodes(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回集列表 (可分页)
"""
url = self.url.construct("episodes", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_episode(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单集信息的字典
"""
url = self.url.construct("episodes", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_episode_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单集的扩展信息字典
"""
url = self.url.construct("episodes", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_episode_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回单集的指定语言翻译信息字典
"""
url = self.url.construct("episodes", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
# 兼容旧函数名。
get_episodes_translation = get_episode_translation
def get_all_genders(self, meta=None, if_modified_since=None) -> list:
"""
返回性别列表
"""
url = self.url.construct("genders", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_genres(self, meta=None, if_modified_since=None) -> list:
"""
返回类型(流派)列表
"""
url = self.url.construct("genres", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_genre(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个类型(流派)信息的字典
"""
url = self.url.construct("genres", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_languages(self, meta=None, if_modified_since=None) -> list:
"""
返回语言列表
"""
url = self.url.construct("languages", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_people(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回人物列表 (可分页)
"""
url = self.url.construct("people", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_person(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个人物信息的字典
"""
url = self.url.construct("people", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_person_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个人物的扩展信息字典
"""
url = self.url.construct("people", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_person_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回人物的指定语言翻译信息字典
"""
url = self.url.construct("people", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_character(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回角色信息的字典
"""
url = self.url.construct("characters", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_people_types(self, meta=None, if_modified_since=None) -> list:
"""
返回人物类型列表
"""
url = self.url.construct("people/types", meta=meta)
return self.request.make_request(url, if_modified_since)
# 兼容旧函数名
get_all_people_types = get_people_types
def get_source_types(self, meta=None, if_modified_since=None) -> list:
"""
返回来源类型列表
"""
url = self.url.construct("sources/types", meta=meta)
return self.request.make_request(url, if_modified_since)
# 兼容旧函数名
get_all_sourcetypes = get_source_types
def get_updates(self, since: int, **kwargs) -> list:
"""
返回更新列表
"""
url = self.url.construct("updates", since=since, **kwargs)
return self.request.make_request(url)
def get_all_tag_options(self, page=None, meta=None, if_modified_since=None) -> list:
"""
返回标签选项列表 (可分页)
"""
url = self.url.construct("tags/options", page=page, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_tag_option(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个标签选项信息的字典
"""
url = self.url.construct("tags/options", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_all_lists(self, page=None, meta=None) -> dict:
"""
返回所有公开的列表信息 (可分页)
"""
url = self.url.construct("lists", page=page, meta=meta)
return self.request.make_request(url)
def get_list(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个列表信息的字典
"""
url = self.url.construct("lists", id, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_list_by_slug(self, slug: str, meta=None, if_modified_since=None) -> dict:
"""
通过 slug (别名) 返回单个列表信息的字典
"""
url = self.url.construct("lists/slug", slug, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_list_extended(self, id: int, meta=None, if_modified_since=None) -> dict:
"""
返回单个列表的扩展信息字典
"""
url = self.url.construct("lists", id, "extended", meta=meta)
return self.request.make_request(url, if_modified_since)
def get_list_translation(self, id: int, lang: str, meta=None, if_modified_since=None) -> dict:
"""
返回列表的指定语言翻译信息字典
"""
url = self.url.construct("lists", id, "translations", lang, meta=meta)
return self.request.make_request(url, if_modified_since)
def get_inspiration_types(self, meta=None, if_modified_since=None) -> dict:
"""
返回灵感类型列表
"""
url = self.url.construct("inspiration/types", meta=meta)
return self.request.make_request(url, if_modified_since)
def search(self, query: str, **kwargs) -> list:
"""
根据查询字符串进行搜索,并返回结果列表
"""
url = self.url.construct("search", query=query, **kwargs)
return self.request.make_request(url)
def search_by_remote_id(self, remoteid: str) -> list:
"""
通过外部 ID 精确匹配搜索,并返回结果列表
"""
url = self.url.construct("search/remoteid", remoteid)
return self.request.make_request(url)
def get_tags(self, slug: str, if_modified_since=None) -> dict:
"""
返回具有指定 slug 的标签实体信息字典 (此方法基于的 /entities/{slug} 端点非标准,请谨慎使用)
"""
url = self.url.construct("entities", url_subsect=slug)
return self.request.make_request(url, if_modified_since)
def get_entities_types(self, if_modified_since=None) -> dict:
"""
返回可用的实体类型列表
"""
url = self.url.construct("entities")
return self.request.make_request(url, if_modified_since)
def get_user_by_id(self, id: int) -> dict:
"""
通过用户 ID 返回用户信息字典
"""
url = self.url.construct("user", id)
return self.request.make_request(url)
def get_user(self) -> dict:
"""
返回当前认证的用户信息字典
"""
url = self.url.construct("user")
return self.request.make_request(url)
def get_user_favorites(self) -> dict:
"""
返回当前认证用户的收藏夹信息字典
"""
url = self.url.construct('user/favorites')
return self.request.make_request(url)

File diff suppressed because it is too large Load Diff

View File

@@ -7,11 +7,12 @@ from transmission_rpc import File
from app import schemas
from app.core.config import settings
from app.core.metainfo import MetaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _DownloaderBase
from app.modules.transmission.transmission import Transmission
from app.schemas import TransferTorrent, DownloadingTorrent
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType
from app.schemas.types import TorrentStatus, ModuleType, DownloaderType, SystemConfigKey, EventType
from app.utils.string import StringUtils
@@ -24,6 +25,19 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
super().init_service(service_name=Transmission.__name__.lower(),
service_type=Transmission)
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "Transmission"

View File

@@ -2,12 +2,12 @@ from typing import Any, Generator, List, Optional, Tuple, Union
from app import schemas
from app.core.context import MediaInfo
from app.core.event import eventmanager
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _MediaServerBase, _ModuleBase
from app.modules.trimemedia.trimemedia import TrimeMedia
from app.schemas import AuthCredentials, AuthInterceptCredentials
from app.schemas.types import ChainEventType, MediaServerType, MediaType, ModuleType
from app.schemas.types import ChainEventType, MediaServerType, MediaType, ModuleType, SystemConfigKey, EventType
class TrimeMediaModule(_ModuleBase, _MediaServerBase[TrimeMedia]):
@@ -23,6 +23,19 @@ class TrimeMediaModule(_ModuleBase, _MediaServerBase[TrimeMedia]):
),
)
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "飞牛影视"

View File

@@ -111,6 +111,7 @@ class Api:
"_api_path",
"_request_utils",
"_version",
"_session"
)
@property
@@ -138,7 +139,8 @@ class Api:
self._apikey = apikey
self._token: Optional[str] = None
self._version: Optional[Version] = None
self._request_utils = RequestUtils(session=requests.Session())
self._session = requests.Session()
self._request_utils = RequestUtils(session=self._session)
def sys_version(self) -> Optional[Version]:
"""
@@ -352,7 +354,7 @@ class Api:
def del_item(self, guid: str, delete_file: bool) -> bool:
"""
删除媒体
:param guid: 媒体GUID
:param delete_file: True删除媒体文件False仅从媒体库移除
"""
if (
@@ -491,3 +493,10 @@ class Api:
if not suppress_log:
logger.error(f"请求接口 {url} 异常:" + str(e))
return None
def close(self):
"""
关闭API会话
"""
if self._session:
self._session.close()

View File

@@ -55,7 +55,8 @@ class TrimeMedia:
"""
return self._api
def __create_api(self, host: Optional[str]) -> Optional[fnapi.Api]:
@staticmethod
def __create_api(host: Optional[str]) -> Optional[fnapi.Api]:
"""
创建一个飞牛API
@@ -76,7 +77,7 @@ class TrimeMedia:
api = fnapi.Api(host, api_key)
return api if api.sys_version() else None
def __del__(self):
def close(self):
self.disconnect()
def is_configured(self) -> bool:
@@ -118,6 +119,7 @@ class TrimeMedia:
"""
if self.is_authenticated():
self._api.logout()
self._api.close()
self._userinfo = None
logger.debug(f"{self._username} 已断开飞牛影视")

View File

@@ -2,11 +2,12 @@ import json
from typing import Optional, Union, List, Tuple, Any, Dict
from app.core.context import Context, MediaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.vocechat.vocechat import VoceChat
from app.schemas import MessageChannel, CommingMessage, Notification
from app.schemas.types import ModuleType
from app.schemas import MessageChannel, CommingMessage, Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, SystemConfigKey, EventType
class VoceChatModule(_ModuleBase, _MessageBase[VoceChat]):
@@ -19,6 +20,19 @@ class VoceChatModule(_ModuleBase, _MessageBase[VoceChat]):
service_type=VoceChat)
self._channel = MessageChannel.VoceChat
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "VoceChat"

View File

@@ -4,10 +4,11 @@ from typing import Union, Tuple
from pywebpush import webpush, WebPushException
from app.core.config import global_vars, settings
from app.core.event import eventmanager, Event
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.schemas import Notification
from app.schemas.types import ModuleType, MessageChannel
from app.schemas import Notification, ConfigChangeEventData
from app.schemas.types import ModuleType, MessageChannel, SystemConfigKey, EventType
class WebPushModule(_ModuleBase, _MessageBase):
@@ -19,6 +20,19 @@ class WebPushModule(_ModuleBase, _MessageBase):
super().init_service(service_name=self.get_name().lower())
self._channel = MessageChannel.WebPush
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "WebPush"

View File

@@ -3,13 +3,13 @@ import xml.dom.minidom
from typing import Optional, Union, List, Tuple, Any, Dict
from app.core.context import Context, MediaInfo
from app.core.event import eventmanager
from app.core.event import Event, eventmanager
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.wechat.WXBizMsgCrypt3 import WXBizMsgCrypt
from app.modules.wechat.wechat import WeChat
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData
from app.schemas.types import ModuleType, ChainEventType
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData, ConfigChangeEventData
from app.schemas.types import ModuleType, ChainEventType, SystemConfigKey, EventType
from app.utils.dom import DomUtils
from app.utils.structures import DictUtils
@@ -24,6 +24,19 @@ class WechatModule(_ModuleBase, _MessageBase[WeChat]):
service_type=WeChat)
self._channel = MessageChannel.Wechat
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
self.init_module()
@staticmethod
def get_name() -> str:
return "微信"

View File

@@ -15,11 +15,13 @@ from app.chain import ChainBase
from app.chain.storage import StorageChain
from app.chain.transfer import TransferChain
from app.core.config import settings
from app.db.systemconfig_oper import SystemConfigOper
from app.core.event import Event, eventmanager
from app.helper.directory import DirectoryHelper
from app.helper.message import MessageHelper
from app.log import logger
from app.schemas import ConfigChangeEventData
from app.schemas import FileItem
from app.schemas.types import SystemConfigKey, EventType
from app.utils.singleton import Singleton
lock = Lock()
@@ -74,17 +76,23 @@ class Monitor(metaclass=Singleton):
def __init__(self):
super().__init__()
self.transferchain = TransferChain()
self.storagechain = StorageChain()
self.directoryhelper = DirectoryHelper()
self.systemmessage = MessageHelper()
self.systemconfig = SystemConfigOper()
self.all_exts = settings.RMT_MEDIAEXT
# 启动目录监控和文件整理
self.init()
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Directories.value]:
return
self.init()
def init(self):
"""
启动监控
@@ -93,7 +101,7 @@ class Monitor(metaclass=Singleton):
self.stop()
# 读取目录配置
monitor_dirs = self.directoryhelper.get_download_dirs()
monitor_dirs = DirectoryHelper().get_download_dirs()
if not monitor_dirs:
return
@@ -103,6 +111,7 @@ class Monitor(metaclass=Singleton):
# 启动定时服务进程
self._scheduler = BackgroundScheduler(timezone=settings.TZ)
messagehelper = MessageHelper()
for mon_dir in monitor_dirs:
if not mon_dir.library_path:
continue
@@ -113,7 +122,7 @@ class Monitor(metaclass=Singleton):
target_path = Path(mon_dir.library_path)
if target_path.is_relative_to(mon_path):
logger.warn(f"{target_path} 是监控目录 {mon_path} 的子目录,无法监控!")
self.systemmessage.put(f"{target_path} 是监控目录 {mon_path} 的子目录,无法监控", title="目录监控")
messagehelper.put(f"{target_path} 是监控目录 {mon_path} 的子目录,无法监控", title="目录监控")
continue
# 启动监控
@@ -143,7 +152,7 @@ class Monitor(metaclass=Singleton):
""")
else:
logger.error(f"{mon_path} 启动目录监控失败:{err_msg}")
self.systemmessage.put(f"{mon_path} 启动目录监控失败:{err_msg}", title="目录监控")
messagehelper.put(f"{mon_path} 启动目录监控失败:{err_msg}", title="目录监控")
else:
# 远程目录监控
self._scheduler.add_job(self.polling_observer, 'interval', minutes=self._snapshot_interval,
@@ -183,7 +192,7 @@ class Monitor(metaclass=Singleton):
"""
with snapshot_lock:
# 快照存储
new_snapshot = self.storagechain.snapshot_storage(storage=storage, path=mon_path)
new_snapshot = StorageChain().snapshot_storage(storage=storage, path=mon_path)
if new_snapshot:
# 比较快照
old_snapshot = self._storage_snapshot.get(storage)
@@ -249,7 +258,7 @@ class Monitor(metaclass=Singleton):
try:
# 开始整理
self.transferchain.do_transfer(
TransferChain().do_transfer(
fileitem=FileItem(
storage=storage,
path=str(event_path).replace("\\", "/"),

View File

@@ -34,6 +34,8 @@ class _PluginBase(metaclass=ABCMeta):
plugin_desc: Optional[str] = ""
# 插件顺序
plugin_order: Optional[int] = 9999
# 是否为插件分身
is_clone: bool = False
def __init__(self):
# 插件数据

View File

@@ -15,21 +15,20 @@ from app.chain.mediaserver import MediaServerChain
from app.chain.recommend import RecommendChain
from app.chain.site import SiteChain
from app.chain.subscribe import SubscribeChain
from app.chain.tmdb import TmdbChain
from app.chain.transfer import TransferChain
from app.chain.workflow import WorkflowChain
from app.core.config import settings
from app.core.event import EventManager
from app.core.event import EventManager, eventmanager, Event
from app.core.plugin import PluginManager
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.sites import SitesHelper
from app.helper.wallpaper import WallpaperHelper
from app.log import logger
from app.schemas import Notification, NotificationType, Workflow
from app.schemas import Notification, NotificationType, Workflow, ConfigChangeEventData
from app.schemas.types import EventType, SystemConfigKey
from app.utils.singleton import Singleton
from app.utils.timer import TimerUtils
lock = threading.Lock()
@@ -57,6 +56,20 @@ class Scheduler(metaclass=Singleton):
def __init__(self):
self.init()
@eventmanager.register(EventType.ConfigChanged)
def handle_config_changed(self, event: Event):
"""
处理配置变更事件
:param event: 事件对象
"""
if not event:
return
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in ['DEV', 'COOKIECLOUD_INTERVAL', 'MEDIASERVER_SYNC_INTERVAL', 'SUBSCRIBE_SEARCH',
'SUBSCRIBE_MODE', 'SUBSCRIBE_RSS_INTERVAL', 'SITEDATA_REFRESH_INTERVAL']:
return
self.init()
def init(self):
"""
初始化定时服务
@@ -135,7 +148,7 @@ class Scheduler(metaclass=Singleton):
},
"random_wallpager": {
"name": "壁纸缓存",
"func": TmdbChain().get_trending_wallpapers,
"func": WallpaperHelper().get_wallpapers,
"running": False,
},
"sitedata_refresh": {
@@ -540,17 +553,18 @@ class Scheduler(metaclass=Singleton):
self.remove_plugin_job(pid)
# 获取插件服务列表
with self._lock:
plugin_manager = PluginManager()
try:
plugin_services = PluginManager().get_plugin_services(pid=pid)
plugin_services = plugin_manager.get_plugin_services(pid=pid)
except Exception as e:
logger.error(f"运行插件 {pid} 服务失败:{str(e)} - {traceback.format_exc()}")
return
# 获取插件名称
plugin_name = PluginManager().get_plugin_attr(pid, "plugin_name")
plugin_name = plugin_manager.get_plugin_attr(pid, "plugin_name")
# 开始注册插件服务
for service in plugin_services:
try:
sid = f"{service['id']}"
sid = f"{pid}_{service['id']}"
job_id = sid.split("|")[0]
self.remove_plugin_job(pid, job_id)
self._jobs[job_id] = {
@@ -599,8 +613,8 @@ class Scheduler(metaclass=Singleton):
name = service.get("name")
provider_name = service.get("provider_name")
if service.get("running") and name and provider_name:
if name not in added:
added.append(name)
if job_id not in added:
added.append(job_id)
schedulers.append(schemas.ScheduleInfo(
id=job_id,
name=name,
@@ -609,11 +623,11 @@ class Scheduler(metaclass=Singleton):
))
# 获取其他待执行任务
for job in jobs:
if job.name not in added:
added.append(job.name)
job_id = job.id.split("|")[0]
if job_id not in added:
added.append(job_id)
else:
continue
job_id = job.id.split("|")[0]
service = self._jobs.get(job_id)
if not service:
continue

View File

@@ -22,6 +22,15 @@ class BaseEventData(BaseModel):
pass
class ConfigChangeEventData(BaseEventData):
"""
ConfigChange 事件的数据模型
"""
key: str = Field(..., description="配置项的键")
value: Optional[Any] = Field(default=None, description="配置项的新值")
change_type: str = Field(default="update", description="配置项的变更类型,如 'add', 'update', 'delete'")
class ChainEventData(BaseEventData):
"""
链式事件数据的基类,所有具体事件数据类应继承自此类
@@ -306,7 +315,7 @@ class MediaRecognizeConvertEventData(ChainEventData):
convert_type: str = Field(..., description="转换类型themoviedb/douban")
# 输出参数
media_dict: dict = Field(default=dict, description="转换后的媒体信息TheMovieDb/豆瓣)")
media_dict: dict = Field(default_factory=dict, description="转换后的媒体信息TheMovieDb/豆瓣)")
class StorageOperSelectionEventData(ChainEventData):

View File

@@ -5,7 +5,7 @@ from pydantic import BaseModel, Field
class FileItem(BaseModel):
# 存储类型
storage: Optional[str] = "local"
storage: Optional[str] = Field(default="local")
# 类型 dir/file
type: Optional[str] = None
# 文件路径

View File

@@ -11,7 +11,7 @@ class ExistMediaInfo(BaseModel):
媒体服务器存在媒体信息
"""
# 类型 电影、电视剧
type: Optional[MediaType]
type: Optional[MediaType] = None
# 季
seasons: Optional[Dict[int, list]] = Field(default_factory=dict)
# 媒体服务器类型plex、jellyfin、emby、trimemedia

View File

@@ -102,7 +102,7 @@ class Subscription(BaseModel):
"""
客户端消息订阅
"""
endpoint: Optional[str]
endpoint: Optional[str] = None
keys: Optional[dict] = Field(default_factory=dict)
@@ -110,8 +110,8 @@ class SubscriptionMessage(BaseModel):
"""
客户端订阅消息体
"""
title: Optional[str]
body: Optional[str]
icon: Optional[str]
url: Optional[str]
title: Optional[str] = None
body: Optional[str] = None
icon: Optional[str] = None
url: Optional[str] = None
data: Optional[dict] = Field(default_factory=dict)

View File

@@ -115,15 +115,23 @@ class TransferInfo(BaseModel):
# 整理方式
transfer_type: Optional[str] = None
# 处理文件数
file_count: Optional[int] = 0
file_count: Optional[int] = Field(default=0)
# 处理文件清单
file_list: Optional[list] = Field(default_factory=list)
# 目标文件清单
file_list_new: Optional[list] = Field(default_factory=list)
# 总文件大小
total_size: Optional[float] = 0.0
total_size: Optional[int] = Field(default=0)
# 失败清单
fail_list: Optional[list] = Field(default_factory=list)
# 处理字幕文件清单
subtitle_list: Optional[list] = Field(default_factory=list)
# 目标字幕文件清单
subtitle_list_new: Optional[list] = Field(default_factory=list)
# 处理音频文件清单
audio_list: Optional[list] = Field(default_factory=list)
# 目标音频文件清单
audio_list_new: Optional[list] = Field(default_factory=list)
# 错误信息
message: Optional[str] = None
# 是否需要刮削

View File

@@ -61,6 +61,8 @@ class EventType(Enum):
MetadataScrape = "metadata.scrape"
# 模块需要重载
ModuleReload = "module.reload"
# 配置项更新
ConfigChanged = "config.updated"
# 同步链式事件
@@ -141,6 +143,8 @@ class SystemConfigKey(Enum):
UserInstalledPlugins = "UserInstalledPlugins"
# 插件安装统计
PluginInstallReport = "PluginInstallReport"
# 插件文件夹分组配置
PluginFolders = "PluginFolders"
# 默认电影订阅规则
DefaultMovieSubscribeConfig = "DefaultMovieSubscribeConfig"
# 默认电视剧订阅规则

Some files were not shown because too many files have changed in this diff Show More