Compare commits

..

49 Commits

Author SHA1 Message Date
jxxghp
647c0929c5 v2.6.2 2025-07-06 08:28:33 +08:00
jxxghp
a61533a131 Merge pull request #4536 from cddjr/fix_local_exists 2025-07-05 22:02:16 +08:00
景大侠
bc5e682308 fix 本地媒体检查潜在的额外扫盘问题 2025-07-05 21:46:21 +08:00
jxxghp
25a481df12 Merge pull request #4534 from jxxghp/cursor/bc-55af1137-dea1-4191-9033-64ea5fcaa43a-d338
修复文件整理快照处理问题
2025-07-05 15:44:51 +08:00
Cursor Agent
764c10fae4 Fix snapshot handling logic to correctly process files during monitoring
Co-authored-by: jxxghp <jxxghp@163.com>
2025-07-05 07:22:44 +00:00
Cursor Agent
d8249d4e38 Fix snapshot handling logic to correctly process files during monitoring
Co-authored-by: jxxghp <jxxghp@163.com>
2025-07-05 07:19:53 +00:00
jxxghp
0e3e42b398 Merge pull request #4531 from Aqr-K/feat-process 2025-07-05 06:33:57 +08:00
Aqr-K
7d3b64dcf9 Update requirements.in 2025-07-05 03:16:49 +08:00
Aqr-K
2c8d525796 feat: 增加进程名设置 2025-07-05 03:14:54 +08:00
jxxghp
4869f071ab fix error message 2025-07-04 21:34:31 +08:00
jxxghp
3029eeaf6f fix error message 2025-07-04 21:33:32 +08:00
jxxghp
33fb692aee 更新 plugin.py 2025-07-03 22:20:04 +08:00
jxxghp
6a075d144f 更新 version.py 2025-07-03 20:19:36 +08:00
jxxghp
aa23315599 rollback transmission-rpc 2025-07-03 19:16:36 +08:00
jxxghp
8d0bb35505 add 网络流量API 2025-07-03 19:05:43 +08:00
jxxghp
32e76bc6ce Merge pull request #4529 from cddjr/add_ctx_mgr_proto 2025-07-03 18:47:08 +08:00
景大侠
6c02766000 AutoCloseResponse支持上下文管理协议,避免部分插件报错 2025-07-03 18:38:48 +08:00
jxxghp
52ef390464 图片代理Api增加cache参数 2025-07-03 17:07:54 +08:00
jxxghp
43a557601e fix local usage 2025-07-03 16:48:35 +08:00
jxxghp
82ff7fc090 fix SMB Usage 2025-07-03 15:21:41 +08:00
jxxghp
db40b5105b 修正目录监控模式匹配 2025-07-03 13:55:54 +08:00
jxxghp
b2a379b84b fix SMB Storage 2025-07-03 12:41:44 +08:00
jxxghp
97cbd816fe add SMB Storage 2025-07-03 12:31:59 +08:00
jxxghp
7de3bb2a91 v2.6.0 2025-07-02 21:36:02 +08:00
jxxghp
3a8a2bcab4 Merge pull request #4519 from Aqr-K/patch-2 2025-07-01 19:46:12 +08:00
Aqr-K
eb1adbe992 fix: 错误文案修复,统一文案格式 2025-07-01 19:26:11 +08:00
jxxghp
b55966d42b Merge pull request #4516 from Aqr-K/feat-command
feat(command): 增加 `show` ,用来判断是否注册进菜单里显示
2025-07-01 17:20:59 +08:00
Aqr-K
451ca9cb5a feat(command): 增加 show ,用来判断是否注册进菜单里显示 2025-07-01 17:19:01 +08:00
jxxghp
1e2c607ced fix #4515 流平台不合并到现有标签中,如有需要通过命名模块配置 2025-07-01 17:02:29 +08:00
jxxghp
5ff7da0d19 fix #4515 流平台不合并到现有标签中,如有需要通过命名模块配置 2025-07-01 16:57:45 +08:00
jxxghp
8e06c6f8e6 remove openai 2025-07-01 14:48:16 +08:00
jxxghp
4497cd3904 add site stat api 2025-07-01 11:23:20 +08:00
jxxghp
2945679a94 - 修复Redis缓存问题及站点消息读取问题 2025-07-01 09:20:08 +08:00
jxxghp
1eaf7e3c85 Merge pull request #4513 from cddjr/fix_4511 2025-07-01 06:56:11 +08:00
景大侠
8146b680c6 fix: 修复AutoCloseResponse类在反序列化时无限递归 2025-07-01 01:29:01 +08:00
jxxghp
99e667382f fix #4509 2025-06-30 19:17:36 +08:00
jxxghp
4c03759d3f refactor:优化目录监控 2025-06-30 13:16:05 +08:00
jxxghp
8593a6cdd0 refactor:优化目录监控快照 2025-06-30 12:40:37 +08:00
jxxghp
cd18c31618 fix 订阅匹配 2025-06-30 10:55:10 +08:00
jxxghp
f29c918700 Merge pull request #4505 from wikrin/v2 2025-06-29 23:12:08 +08:00
Attente
0f0c3e660b style: 清理空白字符
移除代码中的 trailing whitespace 和空行缩进, 提升代码整洁度
2025-06-29 22:49:58 +08:00
Attente
1cf4639db3 fix(download): 修复手动下载时下载器选择问题
- 在手动下载模式下,始终使用用户选择的下载器
2025-06-29 22:24:53 +08:00
jxxghp
f5da9b5780 fix log 2025-06-29 22:10:47 +08:00
jxxghp
e4c87c8a96 更新 version.py 2025-06-29 21:56:37 +08:00
jxxghp
4b4bf153f0 fix plugin reload 2025-06-29 21:26:06 +08:00
jxxghp
ec227d0d56 Merge pull request #4500 from Miralia/v2
refactor(meta): 将 web_source 处理逻辑统一到 MetaBase 并添加到消息模板
2025-06-29 11:11:35 +08:00
Miralia
53c8c50779 refactor(meta): 将 web_source 处理逻辑统一到 MetaBase 并添加到消息模板 2025-06-29 11:08:34 +08:00
jxxghp
07b4c8b462 fix #4489 2025-06-29 11:06:36 +08:00
jxxghp
f3cfc5b9f0 fix plex 2025-06-29 08:27:48 +08:00
55 changed files with 1531 additions and 346 deletions

View File

@@ -10,7 +10,7 @@ body:
目的是让协作的开发者间清晰的知道「要做什么」和「具体会怎么做」,以及所有的开发者都能公开透明的参与讨论;
以便评估和讨论产生的影响 (遗漏的考虑、向后兼容性、与现有功能的冲突)
因此提案侧重在对解决问题的 **方案、设计、步骤** 的描述上。
如果仅希望讨论是否添加或改进某功能本身,请使用 -> [Issue: 功能改进](https://github.com/jxxghp/MoviePilot/issues/new?assignees=&labels=feature+request&projects=&template=feature_request.yml&title=%5BFeature+Request%5D%3A+)
- type: textarea
id: background

View File

@@ -8,17 +8,17 @@ jobs:
pylint:
runs-on: ubuntu-latest
name: Pylint Code Quality Check
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Cache pip dependencies
uses: actions/cache@v4
with:
@@ -26,7 +26,7 @@ jobs:
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/requirements.in') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
@@ -41,7 +41,7 @@ jobs:
else
echo "⚠️ 未找到依赖文件,仅安装 pylint"
fi
- name: Verify pylint config
run: |
# 检查项目中的pylint配置文件是否存在
@@ -57,35 +57,35 @@ jobs:
run: |
# 运行pylint检查主要的Python文件
echo "🚀 运行 Pylint 错误检查..."
# 检查主要目录 - 只关注错误,如果有错误则退出
echo "📂 检查 app/ 目录..."
pylint app/ --output-format=colorized --reports=yes --score=yes
# 检查根目录的Python文件
echo "📂 检查根目录 Python 文件..."
for file in $(find . -name "*.py" -not -path "./.*" -not -path "./.venv/*" -not -path "./build/*" -not -path "./dist/*" -not -path "./tests/*" -not -path "./docs/*" -not -path "./__pycache__/*" -maxdepth 1); do
echo "检查文件: $file"
pylint "$file" --output-format=colorized || exit 1
done
# 生成详细报告
echo "📊 生成 Pylint 详细报告..."
pylint app/ --output-format=json > pylint-report.json || true
# 显示评分(仅供参考)
echo "📈 Pylint 评分(仅供参考):"
pylint app/ --score=yes --reports=no | tail -2 || true
- name: Upload pylint report
uses: actions/upload-artifact@v4
if: always()
with:
name: pylint-report
path: pylint-report.json
- name: Summary
run: |
echo "🎉 Pylint 检查完成!"
echo "✅ 没有发现语法错误或严重问题"
echo "📊 详细报告已保存为构建工件"
echo "📊 详细报告已保存为构建工件"

View File

@@ -12,7 +12,7 @@ jobs=0
# 只关注错误级别的问题,禁用警告、约定和重构建议
# E = Error (错误) - 会导致构建失败
# W = Warning (警告) - 仅显示,不会失败
# R = Refactor (重构建议) - 仅显示,不会失败
# R = Refactor (重构建议) - 仅显示,不会失败
# C = Convention (约定) - 仅显示,不会失败
# I = Information (信息) - 仅显示,不会失败
@@ -80,4 +80,4 @@ ignore-imports=yes
[TYPECHECK]
# 生成缺失成员提示的类列表
generated-members=requests.packages.urllib3
generated-members=requests.packages.urllib3

View File

@@ -166,3 +166,19 @@ def memory2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
获取当前内存使用率 API_TOKEN认证?token=xxx
"""
return memory()
@router.get("/network", summary="获取当前网络流量", response_model=List[int])
def network(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
"""
获取当前网络流量上行和下行流量单位bytes/s
"""
return SystemUtils.network_usage()
@router.get("/network2", summary="获取当前网络流量API_TOKEN", response_model=List[int])
def network2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
"""
获取当前网络流量 API_TOKEN认证?token=xxx
"""
return network()

View File

@@ -44,6 +44,8 @@ def download(
# 种子信息
torrentinfo = TorrentInfo()
torrentinfo.from_dict(torrent_in.dict())
# 手动下载始终使用选择的下载器
torrentinfo.site_downloader = downloader
# 上下文
context = Context(
meta_info=metainfo,
@@ -51,7 +53,7 @@ def download(
torrent_info=torrentinfo
)
did = DownloadChain().download_single(context=context, username=current_user.name,
downloader=downloader, save_path=save_path, source="Manual")
save_path=save_path, source="Manual")
if not did:
return schemas.Response(success=False, message="任务添加失败")
return schemas.Response(success=True, data={

View File

@@ -1,5 +1,6 @@
from typing import List, Any, Dict, Optional
from app.helper.sites import SitesHelper
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.orm import Session
from starlette.background import BackgroundTasks
@@ -21,7 +22,6 @@ from app.db.models.siteuserdata import SiteUserData
from app.db.site_oper import SiteOper
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_superuser
from app.helper.sites import SitesHelper
from app.scheduler import Scheduler
from app.schemas.types import SystemConfigKey, EventType
from app.utils.string import StringUtils
@@ -333,8 +333,8 @@ def read_site_by_domain(
return site
@router.get("/statistic/{site_url}", summary="站点统计信息", response_model=schemas.SiteStatistic)
def read_site_by_domain(
@router.get("/statistic/{site_url}", summary="特定站点统计信息", response_model=schemas.SiteStatistic)
def read_statistic_by_domain(
site_url: str,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token)
@@ -349,6 +349,17 @@ def read_site_by_domain(
return schemas.SiteStatistic(domain=domain)
@router.get("/statistic", summary="所有站点统计信息", response_model=List[schemas.SiteStatistic])
def read_statistics(
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token)
) -> Any:
"""
获取所有站点统计信息
"""
return SiteStatistic.list(db)
@router.get("/rss", summary="所有订阅站点", response_model=List[schemas.Site])
def read_rss_sites(db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token)) -> List[dict]:

View File

@@ -144,6 +144,7 @@ def fetch_image(
def proxy_img(
imgurl: str,
proxy: bool = False,
cache: bool = False,
if_none_match: Annotated[str | None, Header()] = None,
_: schemas.TokenPayload = Depends(verify_resource_token)
) -> Response:
@@ -154,7 +155,7 @@ def proxy_img(
hosts = [config.config.get("host") for config in MediaServerHelper().get_configs().values() if
config and config.config and config.config.get("host")]
allowed_domains = set(settings.SECURITY_IMAGE_DOMAINS) | set(hosts)
return fetch_image(url=imgurl, proxy=proxy, use_disk_cache=False,
return fetch_image(url=imgurl, proxy=proxy, use_disk_cache=cache,
if_none_match=if_none_match, allowed_domains=allowed_domains)

View File

@@ -43,7 +43,7 @@ class MediaChain(ChainBase):
'movie_banner': True, # 电影横幅图
'movie_thumb': True, # 电影缩略图
'tv_nfo': True, # 电视剧NFO
'tv_poster': True, # 电视剧海报
'tv_poster': True, # 电视剧海报
'tv_backdrop': True, # 电视剧背景图
'tv_banner': True, # 电视剧横幅图
'tv_logo': True, # 电视剧Logo
@@ -448,7 +448,7 @@ class MediaChain(ChainBase):
if not mediainfo:
logger.warn(f"{filepath} 无法识别文件媒体信息!")
return
# 获取刮削开关配置
scraping_switchs = self._get_scraping_switchs()
logger.info(f"开始刮削:{filepath} ...")
@@ -520,7 +520,7 @@ class MediaChain(ChainBase):
should_scrape = scraping_switchs.get('movie_thumb', True)
else:
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath.with_name(image_name)
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,
@@ -653,7 +653,7 @@ class MediaChain(ChainBase):
should_scrape = scraping_switchs.get('season_thumb', True)
else:
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath.with_name(image_name)
# 只下载当前刮削季的图片
@@ -714,7 +714,7 @@ class MediaChain(ChainBase):
should_scrape = scraping_switchs.get('tv_thumb', True)
else:
should_scrape = True # 未知类型默认刮削
if should_scrape:
image_path = filepath / image_name
if overwrite or not storagechain.get_file_item(storage=fileitem.storage,

View File

@@ -110,11 +110,17 @@ class StorageChain(ChainBase):
"""
return self.run_module("get_parent_item", fileitem=fileitem)
def snapshot_storage(self, storage: str, path: Path) -> Optional[Dict[str, float]]:
def snapshot_storage(self, storage: str, path: Path,
last_snapshot_time: float = None, max_depth: int = 5) -> Optional[Dict[str, Dict]]:
"""
快照存储
:param storage: 存储类型
:param path: 路径
:param last_snapshot_time: 上次快照时间,用于增量快照
:param max_depth: 最大递归深度,避免过深遍历
"""
return self.run_module("snapshot_storage", storage=storage, path=path)
return self.run_module("snapshot_storage", storage=storage, path=path,
last_snapshot_time=last_snapshot_time, max_depth=max_depth)
def storage_usage(self, storage: str) -> Optional[schemas.StorageUsage]:
"""

View File

@@ -434,14 +434,15 @@ class SubscribeChain(ChainBase):
else:
self.messagehelper.put('没有找到订阅!', title="订阅搜索", role="system")
logger.debug(f"search Lock released at {datetime.now()}")
finally:
subscribes.clear()
del subscribes
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
logger.debug(f"search Lock released at {datetime.now()}")
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
def update_subscribe_priority(self, subscribe: Subscribe, meta: MetaBase,
mediainfo: MediaInfo, downloads: Optional[List[Context]]):
@@ -647,153 +648,150 @@ class SubscribeChain(ChainBase):
if domains and domain not in domains:
continue
logger.debug(f'开始匹配站点:{domain},共缓存了 {len(contexts)} 个种子...')
try:
for context in contexts:
if global_vars.is_system_stopped:
break
# 提取信息
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
for context in contexts:
if global_vars.is_system_stopped:
break
# 提取信息
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
# 不在订阅站点范围的不处理
sub_sites = self.get_sub_sites(subscribe)
if sub_sites and torrent_info.site not in sub_sites:
logger.debug(f"{torrent_info.site_name} - {torrent_info.title} 不符合订阅站点要求")
continue
# 不在订阅站点范围的不处理
sub_sites = self.get_sub_sites(subscribe)
if sub_sites and torrent_info.site not in sub_sites:
logger.debug(f"{torrent_info.site_name} - {torrent_info.title} 不符合订阅站点要求")
continue
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
# 重新识别元数据
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
_context.media_info = torrent_mediainfo
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
# 重新识别元数据
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
_context.media_info = mediainfo
else:
continue
_context.media_info = torrent_mediainfo
# 直接比对媒体信息
if torrent_mediainfo and (torrent_mediainfo.tmdb_id or torrent_mediainfo.douban_id):
if torrent_mediainfo.type != mediainfo.type:
continue
if torrent_mediainfo.tmdb_id \
and torrent_mediainfo.tmdb_id != mediainfo.tmdb_id:
continue
if torrent_mediainfo.douban_id \
and torrent_mediainfo.douban_id != mediainfo.douban_id:
continue
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (
not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过媒体信ID匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
# 更新种子缓存
_context.media_info = mediainfo
else:
continue
# 如果是电视剧
if torrent_mediainfo.type == MediaType.TV:
# 有多季的不要
if len(torrent_meta.season_list) > 1:
logger.debug(f'{torrent_info.title} 有多季,不处理')
continue
# 比对季
if torrent_meta.begin_season:
if meta.begin_season != torrent_meta.begin_season:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
elif meta.begin_season != 1:
# 直接比对媒体信息
if torrent_mediainfo and (torrent_mediainfo.tmdb_id or torrent_mediainfo.douban_id):
if torrent_mediainfo.type != mediainfo.type:
continue
if torrent_mediainfo.tmdb_id \
and torrent_mediainfo.tmdb_id != mediainfo.tmdb_id:
continue
if torrent_mediainfo.douban_id \
and torrent_mediainfo.douban_id != mediainfo.douban_id:
continue
logger.info(
f'{mediainfo.title_year} 通过媒体信ID匹配到可选资源{torrent_info.site_name} - {torrent_info.title}')
else:
continue
# 如果是电视剧
if torrent_mediainfo.type == MediaType.TV:
# 有多季的不要
if len(torrent_meta.season_list) > 1:
logger.debug(f'{torrent_info.title} 有多季,不处理')
continue
# 比对季
if torrent_meta.begin_season:
if meta.begin_season != torrent_meta.begin_season:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
# 非洗版
if not subscribe.best_version:
# 不是缺失的剧集不要
if no_exists and no_exists.get(mediakey):
# 缺失集
no_exists_info = no_exists.get(mediakey).get(subscribe.season)
if no_exists_info:
# 是否有交
if no_exists_info.episodes and \
torrent_meta.episode_list and \
not set(no_exists_info.episodes).intersection(
set(torrent_meta.episode_list)
):
logger.debug(
f'{torrent_info.title} 对应剧集 {torrent_meta.episode_list} 未包含缺失的剧集'
)
continue
else:
# 洗版时,非整季不要
if meta.type == MediaType.TV:
if torrent_meta.episode_list:
logger.debug(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
elif meta.begin_season != 1:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
# 非洗版
if not subscribe.best_version:
# 不是缺失的剧集不要
if no_exists and no_exists.get(mediakey):
# 缺失
no_exists_info = no_exists.get(mediakey).get(subscribe.season)
if no_exists_info:
# 是否有交集
if no_exists_info.episodes and \
torrent_meta.episode_list and \
not set(no_exists_info.episodes).intersection(
set(torrent_meta.episode_list)
):
logger.debug(
f'{torrent_info.title} 对应剧集 {torrent_meta.episode_list} 未包含缺失的剧集'
)
continue
# 匹配订阅附加参数
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
mediainfo=torrent_mediainfo)
if result is not None and not result:
# 不符合过滤规则
logger.debug(f"{torrent_info.title} 不匹配过滤规则")
# 洗版时,非整季不要
if meta.type == MediaType.TV:
if torrent_meta.episode_list:
logger.debug(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
continue
# 匹配订阅附加参数
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
mediainfo=torrent_mediainfo)
if result is not None and not result:
# 不符合过滤规则
logger.debug(f"{torrent_info.title} 不匹配过滤规则")
continue
# 洗版时,优先级小于已下载优先级的不要
if subscribe.best_version:
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 洗版时,优先级小于已下载优先级的不要
if subscribe.best_version:
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 匹配成功
logger.info(f'{mediainfo.title_year} 匹配成功:{torrent_info.title}')
# 自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
_match_context.append(_context)
finally:
contexts.clear()
del contexts
# 匹配成功
logger.info(f'{mediainfo.title_year} 匹配成功:{torrent_info.title}')
# 自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
_match_context.append(_context)
if not _match_context:
# 未匹配到资源

View File

@@ -35,7 +35,7 @@ class SystemChain(ChainBase):
重启系统
"""
from app.core.config import global_vars
if channel and userid:
self.post_message(Notification(channel=channel, source=source,
title="系统正在重启,请耐心等候!", userid=userid))

View File

@@ -880,7 +880,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
) -> List[Tuple[FileItem, bool]]:
"""
获取整理目录或文件列表
:param fileitem: 文件项
:param depth: 递归深度默认为1
"""
@@ -1204,7 +1204,8 @@ class TransferChain(ChainBase, metaclass=Singleton):
key=ProgressKey.FileTransfer)
progress.end(ProgressKey.FileTransfer)
return all_success, "".join(err_msgs)
error_msg = "".join(err_msgs[:2]) + (f",等{len(err_msgs)}个文件错误!" if len(err_msgs) > 2 else "")
return all_success, error_msg
def remote_transfer(self, arg_str: str, channel: MessageChannel,
userid: Union[str, int] = None, source: Optional[str] = None):

View File

@@ -225,6 +225,9 @@ class Command(metaclass=Singleton):
添加命令集合
"""
for cmd, command in source.items():
if not command.get("show", True):
continue
command_data = {
"type": command_type,
"description": command.get("description"),
@@ -261,6 +264,7 @@ class Command(metaclass=Singleton):
"func": self.send_plugin_event,
"description": command.get("desc"),
"category": command.get("category"),
"show": command.get("show", True),
"data": {
"etype": command.get("event"),
"data": command.get("data")
@@ -335,7 +339,8 @@ class Command(metaclass=Singleton):
return self._commands.get(cmd, {})
def register(self, cmd: str, func: Any, data: Optional[dict] = None,
desc: Optional[str] = None, category: Optional[str] = None) -> None:
desc: Optional[str] = None, category: Optional[str] = None,
show: bool = True) -> None:
"""
注册单个命令
"""
@@ -344,7 +349,8 @@ class Command(metaclass=Singleton):
"func": func,
"description": desc,
"category": category,
"data": data or {}
"data": data or {},
"show": show
}
def execute(self, cmd: str, data_str: Optional[str] = "",

View File

@@ -150,7 +150,7 @@ class CacheToolsBackend(CacheBackend):
region = self.get_region(region)
return self._region_caches.get(region)
def set(self, key: str, value: Any, ttl: Optional[int] = None,
def set(self, key: str, value: Any, ttl: Optional[int] = None,
region: Optional[str] = DEFAULT_CACHE_REGION, **kwargs) -> None:
"""
设置缓存值支持每个 key 独立配置 TTL 和 Maxsize
@@ -357,7 +357,7 @@ class RedisBackend(CacheBackend):
region = self.get_region(quote(region))
return f"{region}:key:{quote(key)}"
def set(self, key: str, value: Any, ttl: Optional[int] = None,
def set(self, key: str, value: Any, ttl: Optional[int] = None,
region: Optional[str] = DEFAULT_CACHE_REGION, **kwargs) -> None:
"""
设置缓存

View File

@@ -55,6 +55,8 @@ class MetaBase(object):
resource_team: Optional[str] = None
# 识别的自定义占位符
customization: Optional[str] = None
# 识别的流媒体平台
web_source: Optional[str] = None
# 视频编码
video_encode: Optional[str] = None
# 音频编码

View File

@@ -67,7 +67,6 @@ class MetaVideo(MetaBase):
original_title = title
self._source = ""
self._effect = []
self.web_source = None
self._index = 0
# 判断是否纯数字命名
if isfile \
@@ -140,9 +139,6 @@ class MetaVideo(MetaBase):
self.resource_effect = " ".join(self._effect)
if self._source:
self.resource_type = self._source.strip()
# 添加流媒体平台
if self.web_source:
self.resource_type = f"{self.web_source} {self.resource_type}"
# 提取原盘DIY
if self.resource_type and "BluRay" in self.resource_type:
if (self.subtitle and re.findall(r'D[Ii]Y', self.subtitle)) \

View File

@@ -154,35 +154,35 @@ def find_metainfo(title: str) -> Tuple[str, dict]:
# 去除title中该部分
if tmdbid or mtype or begin_season or end_season or begin_episode or end_episode:
title = title.replace(f"{{[{result}]}}", '')
# 支持Emby格式的ID标签
# 1. [tmdbid=xxxx] 或 [tmdbid-xxxx] 格式
tmdb_match = re.search(r'\[tmdbid[=\-](\d+)\]', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\[tmdbid[=\-](\d+)\]', '', title).strip()
# 2. [tmdb=xxxx] 或 [tmdb-xxxx] 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\[tmdb[=\-](\d+)\]', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\[tmdb[=\-](\d+)\]', '', title).strip()
# 3. {tmdbid=xxxx} 或 {tmdbid-xxxx} 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\{tmdbid[=\-](\d+)\}', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\{tmdbid[=\-](\d+)\}', '', title).strip()
# 4. {tmdb=xxxx} 或 {tmdb-xxxx} 格式
if not metainfo['tmdbid']:
tmdb_match = re.search(r'\{tmdb[=\-](\d+)\}', title)
if tmdb_match:
metainfo['tmdbid'] = tmdb_match.group(1)
title = re.sub(r'\{tmdb[=\-](\d+)\}', '', title).strip()
# 计算季集总数
if metainfo.get('begin_season') and metainfo.get('end_season'):
if metainfo['begin_season'] > metainfo['end_season']:

View File

@@ -3,6 +3,7 @@ import concurrent.futures
import importlib.util
import inspect
import os
import sys
import time
import traceback
from concurrent.futures import ThreadPoolExecutor, as_completed
@@ -198,10 +199,14 @@ class PluginManager(metaclass=Singleton):
# 清空指定插件
self._plugins.pop(pid, None)
self._running_plugins.pop(pid, None)
# 清除插件模块缓存,包括所有子模块
self._clear_plugin_modules(pid)
else:
# 清空
self._plugins = {}
self._running_plugins = {}
# 清除所有插件模块缓存
self._clear_plugin_modules()
logger.info("插件停止完成")
@staticmethod
@@ -366,25 +371,51 @@ class PluginManager(metaclass=Singleton):
"""
self.stop(plugin_id)
# 从模块列表中移除插件
from sys import modules
try:
del modules[f"app.plugins.{plugin_id.lower()}"]
except KeyError:
pass
def reload_plugin(self, plugin_id: str):
"""
将一个插件重新加载到内存
:param plugin_id: 插件ID
"""
# 先移除
# 先移除插件实例
self.stop(plugin_id)
# 重新加载
self.start(plugin_id)
# 广播事件
eventmanager.send_event(EventType.PluginReload, data={"plugin_id": plugin_id})
@staticmethod
def _clear_plugin_modules(plugin_id: Optional[str] = None):
"""
清除插件及其所有子模块的缓存
:param plugin_id: 插件ID
"""
# 构建插件模块前缀
if plugin_id:
plugin_module_prefix = f"app.plugins.{plugin_id.lower()}"
else:
plugin_module_prefix = "app.plugins"
# 收集需要删除的模块名(创建模块名列表的副本以避免迭代时修改字典)
modules_to_remove = []
for module_name in list(sys.modules.keys()):
if module_name == plugin_module_prefix or module_name.startswith(plugin_module_prefix + "."):
modules_to_remove.append(module_name)
# 删除模块
for module_name in modules_to_remove:
try:
del sys.modules[module_name]
logger.debug(f"已清除插件模块缓存:{module_name}")
except KeyError:
# 模块可能已经被删除
pass
if plugin_id:
if modules_to_remove:
logger.info(f"插件 {plugin_id} 共清除 {len(modules_to_remove)} 个模块缓存:{modules_to_remove}")
else:
logger.debug(f"插件 {plugin_id} 没有找到需要清除的模块缓存")
def sync(self) -> List[str]:
"""
安装本地不存在或需要更新的插件
@@ -1416,8 +1447,9 @@ class PluginManager(metaclass=Singleton):
content = f.read()
# 替换CSS中可能的类名引用
content = content.replace(original_class_name.lower(), clone_class_name.lower())
content = content.replace(original_class_name, clone_class_name)
content = content.replace(original_class_name.lower(),
clone_class_name.lower()).replace(original_class_name,
clone_class_name)
with open(file_path, 'w', encoding='utf-8') as f:
f.write(content)

View File

@@ -46,17 +46,17 @@ class PlaywrightHelper:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
# 回调函数
result = callback(page)
except Exception as e:
logger.error(f"网页操作失败: {str(e)}")
finally:
@@ -69,7 +69,7 @@ class PlaywrightHelper:
browser.close()
except Exception as e:
logger.error(f"Playwright初始化失败: {str(e)}")
return result
def get_page_source(self, url: str,
@@ -97,16 +97,16 @@ class PlaywrightHelper:
browser = playwright[self.browser_type].launch(headless=headless)
context = browser.new_context(user_agent=ua, proxy=proxies)
page = context.new_page()
if cookies:
page.set_extra_http_headers({"cookie": cookies})
if not self.__pass_cloudflare(url, page):
logger.warn("cloudflare challenge fail")
page.wait_for_load_state("networkidle", timeout=timeout * 1000)
source = page.content()
except Exception as e:
logger.error(f"获取网页源码失败: {str(e)}")
source = None
@@ -120,7 +120,7 @@ class PlaywrightHelper:
browser.close()
except Exception as e:
logger.error(f"Playwright初始化失败: {str(e)}")
return source

View File

@@ -361,7 +361,7 @@ class MemoryHelper(metaclass=Singleton):
# 对于较大的对象,使用 asizeof 进行深度计算
size_bytes = asizeof.asizeof(obj)
# 只处理大于10KB的对象提高分析效率
if size_bytes < 10240:
continue

View File

@@ -183,6 +183,8 @@ class TemplateContextBuilder:
"videoCodec": meta.video_encode,
# 音频编码
"audioCodec": meta.audio_encode,
# 流媒体平台
"webSource": meta.web_source,
}
self._context.update({**meta_info, **tech_metadata, **episode_data})

View File

@@ -9,7 +9,7 @@ class OcrHelper:
_ocr_b64_url = f"{settings.OCR_HOST}/captcha/base64"
def get_captcha_text(self, image_url: Optional[str] = None, image_b64: Optional[str] = None,
def get_captcha_text(self, image_url: Optional[str] = None, image_b64: Optional[str] = None,
cookie: Optional[str] = None, ua: Optional[str] = None):
"""
根据图片地址,获取验证码图片,并识别内容

View File

@@ -53,10 +53,10 @@ class PluginHelper(metaclass=Singleton):
# 如果强制刷新,直接调用不带缓存的版本
if force:
return self._get_plugins_uncached(repo_url, package_version)
# 正常情况下调用带缓存的版本
return self._get_plugins_cached(repo_url, package_version)
@cached(maxsize=64, ttl=1800)
def _get_plugins_cached(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
"""
@@ -65,7 +65,7 @@ class PluginHelper(metaclass=Singleton):
:param package_version: 首选插件版本 (如 "v2", "v3"),如果不指定则获取 v1 版本
"""
return self._get_plugins_uncached(repo_url, package_version)
def _get_plugins_uncached(self, repo_url: str, package_version: Optional[str] = None) -> Optional[Dict[str, dict]]:
"""
获取Github所有最新插件列表不使用缓存
@@ -308,7 +308,7 @@ class PluginHelper(metaclass=Singleton):
return None, "连接仓库失败"
elif res.status_code != 200:
return None, f"连接仓库失败:{res.status_code} - " \
f"{'超出速率限制,请置GITHUB_TOKEN环境变量或稍后重试' if res.status_code == 403 else res.reason}"
f"{'超出速率限制,请置Github Token或稍后重试' if res.status_code == 403 else res.reason}"
try:
ret = res.json()

View File

@@ -246,12 +246,17 @@ class RssHelper:
ret = RequestUtils(proxies=settings.PROXY if proxy else None,
timeout=timeout, headers=headers).get_res(url)
if not ret:
logger.error(f"获取RSS失败请求返回空值URL: {url}")
return False
except Exception as err:
logger.error(f"获取RSS失败{str(err)} - {traceback.format_exc()}")
return False
if ret:
# 检查HTTP状态码
if ret.status_code != 200:
logger.error(f"RSS请求失败状态码: {ret.status_code}, URL: {url}")
return False
ret_xml = None
root = None
try:
@@ -280,6 +285,17 @@ class RssHelper:
if not ret_xml:
ret_xml = ret.text
# 验证RSS内容是否有效
if not ret_xml or not ret_xml.strip():
logger.error("RSS内容为空")
return False
# 检查是否包含基本的RSS/XML结构
ret_xml_stripped = ret_xml.strip()
if not ret_xml_stripped.startswith('<'):
logger.error("RSS内容不是有效的XML格式")
return False
# 使用lxml.etree解析XML
parser = None
try:
@@ -292,7 +308,8 @@ class RssHelper:
huge_tree=False # 禁用大文档解析,避免内存问题
)
root = etree.fromstring(ret_xml.encode('utf-8'), parser=parser)
except etree.XMLSyntaxError:
except etree.XMLSyntaxError as xml_error:
logger.debug(f"XML解析失败{str(xml_error)}尝试HTML解析")
# 如果XML解析失败尝试作为HTML解析
try:
root = etree.HTML(ret_xml)
@@ -304,9 +321,15 @@ class RssHelper:
except Exception as e:
logger.error(f"HTML解析也失败{str(e)}")
return False
except Exception as general_error:
logger.error(f"解析RSS时发生未预期错误{str(general_error)}")
return False
finally:
if parser is not None:
parser.close()
try:
parser.close()
except Exception as close_error:
logger.debug(f"关闭解析器时出错:{str(close_error)}")
del parser
if root is None:

View File

@@ -91,10 +91,10 @@ class SystemHelper:
# 检查是否有有效的重启策略
auto_restart_policies = ['always', 'unless-stopped', 'on-failure']
has_restart_policy = policy_name in auto_restart_policies
logger.info(f"容器重启策略: {policy_name}, 支持自动重启: {has_restart_policy}")
return has_restart_policy
except Exception as e:
logger.warning(f"检查重启策略失败: {str(e)}")
return False
@@ -106,7 +106,7 @@ class SystemHelper:
"""
if not SystemUtils.is_docker():
return False, "非Docker环境无法重启"
try:
# 检查容器是否配置了自动重启策略
has_restart_policy = SystemHelper._check_restart_policy()

View File

@@ -18,14 +18,14 @@ class WallpaperHelper(metaclass=Singleton):
获取登录页面壁纸
"""
if settings.WALLPAPER == "bing":
url = self.get_bing_wallpaper()
return self.get_bing_wallpaper()
elif settings.WALLPAPER == "mediaserver":
url = self.get_mediaserver_wallpaper()
return self.get_mediaserver_wallpaper()
elif settings.WALLPAPER == "customize":
url = self.get_customize_wallpaper()
else:
url = self.get_tmdb_wallpaper()
return url
return self.get_customize_wallpaper()
elif settings.WALLPAPER == "tmdb":
return self.get_tmdb_wallpaper()
return ''
def get_wallpapers(self, num: int = 10) -> List[str]:
"""
@@ -37,8 +37,9 @@ class WallpaperHelper(metaclass=Singleton):
return self.get_mediaserver_wallpapers(num)
elif settings.WALLPAPER == "customize":
return self.get_customize_wallpapers()
else:
elif settings.WALLPAPER == "tmdb":
return self.get_tmdb_wallpapers(num)
return []
@cached(maxsize=1, ttl=3600)
def get_tmdb_wallpaper(self) -> Optional[str]:

View File

@@ -1,5 +1,6 @@
import multiprocessing
import os
import setproctitle
import signal
import sys
import threading
@@ -19,6 +20,9 @@ if SystemUtils.is_frozen():
from app.core.config import settings
from app.db.init import init_db, update_db
# 设置进程名
setproctitle.setproctitle(settings.PROJECT_NAME)
# uvicorn服务
Server = uvicorn.Server(Config(app, host=settings.HOST, port=settings.PORT,
reload=settings.DEV, workers=multiprocessing.cpu_count(),
@@ -83,7 +87,7 @@ if __name__ == '__main__':
# 注册信号处理器
signal.signal(signal.SIGTERM, signal_handler)
signal.signal(signal.SIGINT, signal_handler)
# 启动托盘
start_tray()
# 初始化数据库

View File

@@ -51,7 +51,7 @@ class BangumiModule(_ModuleBase):
获取模块子类型
"""
return MediaRecognizeType.Bangumi
@staticmethod
def get_priority() -> int:
"""

View File

@@ -344,9 +344,14 @@ class FileManagerModule(_ModuleBase):
return None
return storage_oper.get_parent(fileitem)
def snapshot_storage(self, storage: str, path: Path) -> Optional[Dict[str, float]]:
def snapshot_storage(self, storage: str, path: Path,
last_snapshot_time: float = None, max_depth: int = 5) -> Optional[Dict[str, Dict]]:
"""
快照存储
:param storage: 存储类型
:param path: 路径
:param last_snapshot_time: 上次快照时间,用于增量快照
:param max_depth: 最大递归深度,避免过深遍历
"""
if storage not in self._support_storages:
return None
@@ -354,7 +359,7 @@ class FileManagerModule(_ModuleBase):
if not storage_oper:
logger.error(f"不支持 {storage} 的快照处理")
return None
return storage_oper.snapshot(path)
return storage_oper.snapshot(path, last_snapshot_time=last_snapshot_time, max_depth=max_depth)
def storage_usage(self, storage: str) -> Optional[StorageUsage]:
"""
@@ -507,17 +512,31 @@ class FileManagerModule(_ModuleBase):
# 重命名格式
rename_format = settings.TV_RENAME_FORMAT \
if mediainfo.type == MediaType.TV else settings.MOVIE_RENAME_FORMAT
# 元数据补上常用属性,尽可能确保重命名后的路径不出现空白
meta = MetaInfo(mediainfo.title)
if meta.type == MediaType.UNKNOWN and mediainfo.type is not None:
meta.type = mediainfo.type
if meta.year is None:
meta.year = mediainfo.year
if meta.begin_season is None:
meta.begin_season = 1
if meta.begin_episode is None:
meta.begin_episode = 1
# 获取路径(重命名路径)
target_path = handler.get_rename_path(
path=dir_path,
template_string=rename_format,
rename_dict=handler.get_naming_dict(meta=MetaInfo(mediainfo.title),
rename_dict=handler.get_naming_dict(meta=meta,
mediainfo=mediainfo)
)
# 计算重命名中的文件夹层数
rename_format_level = len(rename_format.split("/")) - 1
# 取相对路径的第1层目录
media_path = target_path.parents[rename_format_level - 1]
if dir_path.is_relative_to(media_path):
# 兜底检查,避免不必要的扫盘
logger.warn(f"{media_path} 是媒体库目录 {dir_path} 的父目录,忽略获取媒体文件列表,请检查重命名格式!")
continue
# 检索媒体文件
fileitem = storage_oper.get_item(media_path)
if not fileitem:

View File

@@ -4,6 +4,7 @@ from typing import Optional, List, Dict, Tuple
from app import schemas
from app.helper.storage import StorageHelper
from app.log import logger
class StorageBase(metaclass=ABCMeta):
@@ -135,7 +136,8 @@ class StorageBase(metaclass=ABCMeta):
pass
@abstractmethod
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: Optional[str] = None) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path,
new_name: Optional[str] = None) -> Optional[schemas.FileItem]:
"""
上传文件
:param fileitem: 上传目录项
@@ -192,21 +194,44 @@ class StorageBase(metaclass=ABCMeta):
"""
pass
def snapshot(self, path: Path) -> Dict[str, float]:
def snapshot(self, path: Path, last_snapshot_time: float = None, max_depth: int = 5) -> Dict[str, Dict]:
"""
快照文件系统,输出所有层级文件信息(不含目录)
:param path: 路径
:param last_snapshot_time: 上次快照时间,用于增量快照
:param max_depth: 最大递归深度,避免过深遍历
"""
files_info = {}
def __snapshot_file(_fileitm: schemas.FileItem):
def __snapshot_file(_fileitm: schemas.FileItem, current_depth: int = 0):
"""
递归获取文件信息
"""
if _fileitm.type == "dir":
for sub_file in self.list(_fileitm):
__snapshot_file(sub_file)
else:
files_info[_fileitm.path] = _fileitm.size
try:
if _fileitm.type == "dir":
# 检查递归深度限制
if current_depth >= max_depth:
return
# 增量检查:如果目录修改时间早于上次快照,跳过
if (last_snapshot_time and
_fileitm.modify_time and
_fileitm.modify_time <= last_snapshot_time):
return
# 遍历子文件
sub_files = self.list(_fileitm)
for sub_file in sub_files:
__snapshot_file(sub_file, current_depth + 1)
else:
# 记录文件的完整信息用于比对
files_info[_fileitm.path] = {
'size': _fileitm.size or 0,
'modify_time': getattr(_fileitm, 'modify_time', 0),
'type': _fileitm.type
}
except Exception as e:
logger.debug(f"Snapshot error for {_fileitm.path}: {e}")
fileitem = self.get_item(path)
if not fileitem:

View File

@@ -1,7 +1,7 @@
import json
from datetime import datetime
from pathlib import Path
from typing import Optional, List, Dict
from typing import Optional, List
import requests
@@ -38,7 +38,7 @@ class Alist(StorageBase, metaclass=Singleton):
"""
初始化
"""
pass
self.__generate_token.clear_cache()
@property
def __get_base_url(self) -> str:
@@ -127,7 +127,7 @@ class Alist(StorageBase, metaclass=Singleton):
"""
检查存储是否可用
"""
pass
return True if self.__generate_token else False
def list(
self,
@@ -710,30 +710,6 @@ class Alist(StorageBase, metaclass=Singleton):
"""
pass
def snapshot(self, path: Path) -> Dict[str, float]:
"""
快照文件系统,输出所有层级文件信息(不含目录)
"""
files_info = {}
def __snapshot_file(_fileitm: schemas.FileItem):
"""
递归获取文件信息
"""
if _fileitm.type == "dir":
for sub_file in self.list(_fileitm):
__snapshot_file(sub_file)
else:
files_info[_fileitm.path] = _fileitm.size
fileitem = self.get_item(path)
if not fileitem:
return {}
__snapshot_file(fileitem)
return files_info
@staticmethod
def __parse_timestamp(time_str: str) -> float:
"""

View File

@@ -191,7 +191,8 @@ class LocalStorage(StorageBase):
"""
return Path(fileitem.path)
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: Optional[str] = None) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path,
new_name: Optional[str] = None) -> Optional[schemas.FileItem]:
"""
上传文件
:param fileitem: 上传目录项
@@ -260,8 +261,11 @@ class LocalStorage(StorageBase):
"""
存储使用情况
"""
library_dirs = DirectoryHelper().get_local_library_dirs()
total_storage, free_storage = SystemUtils.space_usage([Path(d.library_path) for d in library_dirs])
directory_helper = DirectoryHelper()
total_storage, free_storage = SystemUtils.space_usage(
[Path(d.download_path) for d in directory_helper.get_local_download_dirs() if d.download_path] +
[Path(d.library_path) for d in directory_helper.get_local_library_dirs() if d.library_path]
)
return schemas.StorageUsage(
total=total_storage,
available=free_storage

View File

@@ -0,0 +1,549 @@
import threading
import time
from pathlib import Path
from typing import List, Optional, Union
import smbclient
from smbclient import ClientConfig, register_session, reset_connection_cache
from smbprotocol.exceptions import SMBException, SMBResponseException, SMBAuthenticationError
from app import schemas
from app.core.config import settings
from app.log import logger
from app.modules.filemanager import StorageBase
from app.schemas.types import StorageSchema
from app.utils.singleton import Singleton
lock = threading.Lock()
class SMBConnectionError(Exception):
"""SMB 连接错误"""
pass
class SMB(StorageBase, metaclass=Singleton):
"""
SMB网络挂载存储相关操作 - 使用 smbclient 高级接口
"""
# 存储类型
schema = StorageSchema.SMB
# 支持的整理方式
transtype = {
"move": "移动",
"copy": "复制",
}
def __init__(self):
super().__init__()
self._connected = False
self._server_path = None
self._host = None
self._username = None
self._password = None
self._init_connection()
def _init_connection(self):
"""
初始化SMB连接配置
"""
try:
conf = self.get_conf()
if not conf:
return
self._host = conf.get("host")
self._username = conf.get("username")
self._password = conf.get("password")
domain = conf.get("domain", "")
share = conf.get("share", "")
port = conf.get("port", 445)
if not all([self._host, share]):
logger.error("【SMB】缺少必要的连接参数host 和 share")
return
# 构建服务器路径
self._server_path = f"\\\\{self._host}\\{share}"
# 配置全局客户端设置
ClientConfig(
username=self._username,
password=self._password,
domain=domain if domain else None,
connection_timeout=60,
port=port,
auth_protocol="negotiate", # 使用协商认证
require_secure_negotiate=False # 匿名访问时可能需要关闭安全协商
)
# 注册会话以启用连接池
register_session(
self._host,
username=self._username,
password=self._password,
port=port,
encrypt=False, # 根据需要启用加密
connection_timeout=60
)
# 测试连接
self._test_connection()
self._connected = True
# 判断是否为匿名访问
if self._is_anonymous_access():
logger.info(f"【SMB】匿名连接成功{self._server_path}")
else:
logger.info(f"【SMB】认证连接成功{self._server_path} (用户:{self._username})")
except Exception as e:
logger.error(f"【SMB】连接初始化失败{e}")
self._connected = False
def _test_connection(self):
"""
测试SMB连接
"""
try:
# 尝试列出根目录来测试连接
smbclient.listdir(self._server_path)
except SMBAuthenticationError as e:
raise SMBConnectionError(f"SMB认证失败{e}")
except SMBResponseException as e:
raise SMBConnectionError(f"SMB响应错误{e}")
except SMBException as e:
raise SMBConnectionError(f"SMB连接错误{e}")
except Exception as e:
raise SMBConnectionError(f"连接测试失败:{e}")
def _is_anonymous_access(self) -> bool:
"""
检查是否为匿名访问
"""
return not self._username and not self._password
def _check_connection(self):
"""
检查SMB连接状态
"""
if not self._connected or not self._server_path:
raise SMBConnectionError("【SMB】连接未建立或已断开请检查配置")
def _normalize_path(self, path: Union[str, Path]) -> str:
"""
标准化路径格式为SMB路径
"""
path_str = str(path)
# 处理根路径
if path_str in ["/", "\\"]:
return self._server_path
# 去除前导斜杠
if path_str.startswith("/"):
path_str = path_str[1:]
# 构建完整的SMB路径
if path_str:
return f"{self._server_path}\\{path_str.replace('/', '\\')}"
else:
return self._server_path
def _create_fileitem(self, stat_result, file_path: str, name: str) -> schemas.FileItem:
"""
创建文件项
"""
try:
# 检查是否为目录
is_directory = smbclient.path.isdir(file_path)
# 处理路径
relative_path = file_path.replace(self._server_path, "").replace("\\", "/")
if not relative_path.startswith("/"):
relative_path = "/" + relative_path
if is_directory and not relative_path.endswith("/"):
relative_path += "/"
# 获取时间戳
try:
modify_time = int(stat_result.st_mtime)
except (AttributeError, TypeError):
modify_time = int(time.time())
if is_directory:
return schemas.FileItem(
storage=self.schema.value,
type="dir",
path=relative_path,
name=name,
basename=name,
modify_time=modify_time
)
else:
return schemas.FileItem(
storage=self.schema.value,
type="file",
path=relative_path,
name=name,
basename=Path(name).stem,
extension=Path(name).suffix[1:] if Path(name).suffix else None,
size=getattr(stat_result, 'st_size', 0),
modify_time=modify_time
)
except Exception as e:
logger.error(f"【SMB】创建文件项失败{e}")
# 返回基本的文件项信息
return schemas.FileItem(
storage=self.schema.value,
type="file",
path=file_path.replace(self._server_path, "").replace("\\", "/"),
name=name,
basename=Path(name).stem,
modify_time=int(time.time())
)
def init_storage(self):
"""
初始化存储
"""
# 重置连接缓存
reset_connection_cache()
self._init_connection()
def check(self) -> bool:
"""
检查存储是否可用
"""
if not self._connected:
return False
try:
self._test_connection()
return True
except Exception as e:
logger.debug(f"【SMB】连接检查失败{e}")
self._connected = False
return False
def list(self, fileitem: schemas.FileItem) -> List[schemas.FileItem]:
"""
浏览文件
"""
try:
self._check_connection()
if fileitem.type == "file":
item = self.detail(fileitem)
if item:
return [item]
return []
# 构建SMB路径
smb_path = self._normalize_path(fileitem.path.rstrip("/"))
# 列出目录内容
try:
entries = smbclient.listdir(smb_path)
except SMBResponseException as e:
logger.error(f"【SMB】列出目录失败: {smb_path} - {e}")
return []
except SMBException as e:
logger.error(f"【SMB】列出目录失败: {smb_path} - {e}")
return []
items = []
for entry in entries:
if entry in [".", ".."]:
continue
entry_path = f"{smb_path}\\{entry}"
try:
stat_result = smbclient.stat(entry_path)
item = self._create_fileitem(stat_result, entry_path, entry)
items.append(item)
except Exception as e:
logger.debug(f"【SMB】获取文件信息失败: {entry_path} - {e}")
continue
return items
except Exception as e:
logger.error(f"【SMB】列出文件失败: {e}")
return []
def create_folder(self, fileitem: schemas.FileItem, name: str) -> Optional[schemas.FileItem]:
"""
创建目录
"""
try:
self._check_connection()
parent_path = self._normalize_path(fileitem.path.rstrip("/"))
new_path = f"{parent_path}\\{name}"
# 创建目录
smbclient.mkdir(new_path)
# 返回创建的目录信息
return schemas.FileItem(
storage=self.schema.value,
type="dir",
path=f"{fileitem.path.rstrip('/')}/{name}/",
name=name,
basename=name,
modify_time=int(time.time())
)
except Exception as e:
logger.error(f"【SMB】创建目录失败: {e}")
return None
def get_folder(self, path: Path) -> Optional[schemas.FileItem]:
"""
获取目录,如目录不存在则创建
"""
# 检查目录是否存在
folder = self.get_item(path)
if folder:
return folder
# 逐级创建目录
parts = path.parts
current_path = Path("/")
for part in parts[1:]: # 跳过根目录
current_path = current_path / part
folder = self.get_item(current_path)
if not folder:
parent_folder = self.get_item(current_path.parent)
if not parent_folder:
logger.error(f"【SMB】父目录不存在: {current_path.parent}")
return None
folder = self.create_folder(parent_folder, part)
if not folder:
return None
return folder
def get_item(self, path: Path) -> Optional[schemas.FileItem]:
"""
获取文件或目录不存在返回None
"""
try:
self._check_connection()
# 处理根目录
if str(path) == "/":
return schemas.FileItem(
storage=self.schema.value,
type="dir",
path="/",
name="",
basename="",
modify_time=int(time.time())
)
smb_path = self._normalize_path(str(path).rstrip("/"))
# 检查路径是否存在
if not smbclient.path.exists(smb_path):
return None
stat_result = smbclient.stat(smb_path)
file_name = Path(path).name
return self._create_fileitem(stat_result, smb_path, file_name)
except Exception as e:
logger.debug(f"【SMB】获取文件项失败: {e}")
return None
def detail(self, fileitem: schemas.FileItem) -> Optional[schemas.FileItem]:
"""
获取文件详情
"""
return self.get_item(Path(fileitem.path))
def delete(self, fileitem: schemas.FileItem) -> bool:
"""
删除文件或目录
"""
try:
self._check_connection()
smb_path = self._normalize_path(fileitem.path.rstrip("/"))
if fileitem.type == "dir":
# 删除目录
smbclient.rmdir(smb_path)
else:
# 删除文件
smbclient.remove(smb_path)
logger.info(f"【SMB】删除成功: {fileitem.path}")
return True
except Exception as e:
logger.error(f"【SMB】删除失败: {e}")
return False
def rename(self, fileitem: schemas.FileItem, name: str) -> bool:
"""
重命名文件
"""
try:
self._check_connection()
old_path = self._normalize_path(fileitem.path.rstrip("/"))
parent_path = Path(fileitem.path).parent
new_path = self._normalize_path(str(parent_path / name))
# 重命名
smbclient.rename(old_path, new_path)
logger.info(f"【SMB】重命名成功: {fileitem.path} -> {name}")
return True
except Exception as e:
logger.error(f"【SMB】重命名失败: {e}")
return False
def download(self, fileitem: schemas.FileItem, path: Path = None) -> Optional[Path]:
"""
下载文件
"""
try:
self._check_connection()
smb_path = self._normalize_path(fileitem.path)
local_path = path or settings.TEMP_PATH / fileitem.name
# 确保本地目录存在
local_path.parent.mkdir(parents=True, exist_ok=True)
# 使用更高效的文件传输方式
with smbclient.open_file(smb_path, mode="rb") as src_file:
with open(local_path, "wb") as dst_file:
# 使用更大的缓冲区提高性能
buffer_size = 1024 * 1024 # 1MB
while True:
chunk = src_file.read(buffer_size)
if not chunk:
break
dst_file.write(chunk)
logger.info(f"【SMB】下载成功: {fileitem.path} -> {local_path}")
return local_path
except Exception as e:
logger.error(f"【SMB】下载失败: {e}")
return None
def upload(self, fileitem: schemas.FileItem, path: Path,
new_name: Optional[str] = None) -> Optional[schemas.FileItem]:
"""
上传文件
"""
try:
self._check_connection()
target_name = new_name or path.name
target_path = Path(fileitem.path) / target_name
smb_path = self._normalize_path(str(target_path))
# 使用更高效的文件传输方式
with open(path, "rb") as src_file:
with smbclient.open_file(smb_path, mode="wb") as dst_file:
# 使用更大的缓冲区提高性能
buffer_size = 1024 * 1024 # 1MB
while True:
chunk = src_file.read(buffer_size)
if not chunk:
break
dst_file.write(chunk)
logger.info(f"【SMB】上传成功: {path} -> {target_path}")
# 返回上传后的文件信息
return self.get_item(target_path)
except Exception as e:
logger.error(f"【SMB】上传失败: {e}")
return None
def copy(self, fileitem: schemas.FileItem, path: Path, new_name: str) -> bool:
"""
复制文件
"""
try:
# 下载到临时文件
temp_file = self.download(fileitem)
if not temp_file:
return False
# 获取目标目录
target_folder = self.get_item(path)
if not target_folder:
return False
# 上传到目标位置
result = self.upload(target_folder, temp_file, new_name)
# 删除临时文件
if temp_file.exists():
temp_file.unlink()
return result is not None
except Exception as e:
logger.error(f"【SMB】复制失败: {e}")
return False
def move(self, fileitem: schemas.FileItem, path: Path, new_name: str) -> bool:
"""
移动文件
"""
try:
# 先复制
if not self.copy(fileitem, path, new_name):
return False
# 再删除原文件
if not self.delete(fileitem):
logger.warn(f"【SMB】删除原文件失败: {fileitem.path}")
return False
return True
except Exception as e:
logger.error(f"【SMB】移动失败: {e}")
return False
def link(self, fileitem: schemas.FileItem, target_file: Path) -> bool:
pass
def softlink(self, fileitem: schemas.FileItem, target_file: Path) -> bool:
pass
def usage(self) -> Optional[schemas.StorageUsage]:
"""
存储使用情况
"""
try:
self._check_connection()
volume_stat = smbclient.stat_volume(self._server_path)
return schemas.StorageUsage(
total=volume_stat.total_size,
available=volume_stat.caller_available_size
)
except Exception as e:
logger.error(f"【SMB】获取存储使用情况失败: {e}")
return None
def __del__(self):
"""
析构函数,清理连接
"""
try:
# smbclient 自动管理连接池,但我们可以重置缓存
if hasattr(self, '_connected') and self._connected:
reset_connection_cache()
except Exception as e:
logger.debug(f"【SMB】清理连接失败: {e}")

View File

@@ -54,7 +54,7 @@ class RuleParser:
if __name__ == '__main__':
# 测试代码
expression_str = """
SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & CNVOI & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & CNVOI & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & CNVOI & 4K & WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & WEBDL & !DOLBY & !3D > SPECSUB & 4K & WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & WEBDL & !DOLBY & !3D > CNSUB & 4K & WEBDL & !DOLBY & !3D > SPECSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & !3D > SPECSUB & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & !BLU & !WEBDL & !DOLBY & !SDR & !3D > CNSUB & 4K & !BLU & !WEBDL & !DOLBY & !SDR & !3D > 4K & !BLU & !REMUX & !DOLBY & HDR & !3D > 4K & !BLURAY & !REMUX & !DOLBY & !3D > SPECSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & 1080P & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & !BLU & !WEBDL & !DOLBY & !3D > CNSUB & 1080P & !BLU & !WEBDL & !DOLBY & !3D > SPECSUB & 1080P & WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & WEBDL & !DOLBY & !3D > CNSUB & 1080P & WEBDL & !DOLBY & !3D > 1080P & !BLU & !REMUX & !DOLBY & HDR & !3D > 1080P & !BLU & !REMUX & !DOLBY & !3D
SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & 4K & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & CNVOI & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & CNVOI & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > CNSUB & 4K & WEBDL & 60FPS & !DOLBY & !SDR & !3D > SPECSUB & CNVOI & 4K & WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & WEBDL & !DOLBY & !3D > SPECSUB & 4K & WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & WEBDL & !DOLBY & !3D > CNSUB & 4K & WEBDL & !DOLBY & !3D > SPECSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & !3D > CNSUB & CNVOI & 4K & !BLU & !WEBDL & !DOLBY & !3D > SPECSUB & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 4K & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 4K & !BLU & !WEBDL & !DOLBY & !SDR & !3D > CNSUB & 4K & !BLU & !WEBDL & !DOLBY & !SDR & !3D > 4K & !BLU & !REMUX & !DOLBY & HDR & !3D > 4K & !BLURAY & !REMUX & !DOLBY & !3D > SPECSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > CNSUB & 1080P & !BLU & !REMUX & !WEBDL & !DOLBY & !3D > SPECSUB & 1080P & !BLU & !WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & !BLU & !WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & !BLU & !WEBDL & !DOLBY & !3D > CNSUB & 1080P & !BLU & !WEBDL & !DOLBY & !3D > SPECSUB & 1080P & WEBDL & !DOLBY & HDR & !3D > CNSUB & 1080P & WEBDL & !DOLBY & HDR & !3D > SPECSUB & 1080P & WEBDL & !DOLBY & !3D > CNSUB & 1080P & WEBDL & !DOLBY & !3D > 1080P & !BLU & !REMUX & !DOLBY & HDR & !3D > 1080P & !BLU & !REMUX & !DOLBY & !3D
"""
for exp in expression_str.split('>'):
parsed_expr = RuleParser().parse(exp.strip())

View File

@@ -302,11 +302,11 @@ class IndexerModule(_ModuleBase):
bonus=site_obj.bonus,
seeding=site_obj.seeding,
seeding_size=site_obj.seeding_size,
seeding_info=site_obj.seeding_info or [],
seeding_info=site_obj.seeding_info.copy() if site_obj.seeding_info else [],
leeching=site_obj.leeching,
leeching_size=site_obj.leeching_size,
message_unread=site_obj.message_unread,
message_unread_contents=site_obj.message_unread_contents or [],
message_unread_contents=site_obj.message_unread_contents.copy() if site_obj.message_unread_contents else [],
updated_day=datetime.now().strftime('%Y-%m-%d'),
err_msg=site_obj.err_msg
)

View File

@@ -788,7 +788,7 @@ class Plex:
# 合并排序
for hub in hubs:
for item in hub.items:
for item in hub.items():
sub_result.append(item)
sub_result.sort(key=lambda x: x.addedAt, reverse=True)

View File

@@ -122,7 +122,7 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
'text': ''
}
}
按钮回调格式:
{
'callback_query': {

View File

@@ -59,7 +59,7 @@ class AsObj:
def __setitem__(self, key, value):
return setattr(self, key, value)
def __str__(self):
return str(self._obj_list) if self._list_only else str(self._dict())
@@ -91,10 +91,10 @@ class AsObj:
def pop(self, key, value=None):
return self.__dict__.pop(key, value)
def popitem(self):
return self.__dict__.popitem()
def setdefault(self, key, value=None):
return self.__dict__.setdefault(key, value)

View File

@@ -4,7 +4,7 @@ from ..tmdb import TMDb
class Collection(TMDb):
_urls = {
"details": "/collection/%s",
"images": "/collection/%s/images",
"images": "/collection/%s/images",
"translations": "/collection/%s/translations"
}

View File

@@ -3,7 +3,7 @@ from ..tmdb import TMDb
class Company(TMDb):
_urls = {
"details": "/company/%s",
"details": "/company/%s",
"alternative_names": "/company/%s/alternative_names",
"images": "/company/%s/images",
"movies": "/company/%s/movies"

View File

@@ -101,11 +101,11 @@ class Movie(TMDb):
:return:
"""
return self._request_obj(self._urls["external_ids"] % movie_id)
def images(self, movie_id, include_image_language=None):
"""
Get the images that belong to a movie.
Querying images with a language parameter will filter the results.
Querying images with a language parameter will filter the results.
If you want to include a fallback language (especially useful for backdrops)
you can use the include_image_language parameter.
This should be a comma separated value like so: include_image_language=en,null.

View File

@@ -55,7 +55,7 @@ class Search(TMDb):
params="query=%s&page=%s" % (quote(term), page),
key="results"
)
def movies(self, term, adult=None, region=None, year=None, release_year=None, page=1):
"""
Search for movies.

View File

@@ -19,7 +19,7 @@ class Transmission:
"peersGettingFromUs", "peersSendingToUs", "uploadRatio", "uploadedEver", "downloadedEver", "downloadDir",
"error", "errorString", "doneDate", "queuePosition", "activityDate", "trackers"]
def __init__(self, host: Optional[str] = None, port: Optional[int] = None,
def __init__(self, host: Optional[str] = None, port: Optional[int] = None,
username: Optional[str] = None, password: Optional[str] = None, **kwargs):
"""
若不设置参数,则创建配置文件设置的下载器

View File

@@ -128,7 +128,7 @@ class WechatModule(_ModuleBase, _MessageBase[WeChat]):
1、消息格式
<xml>
<ToUserName><![CDATA[toUser]]></ToUserName>
<FromUserName><![CDATA[fromUser]]></FromUserName>
<FromUserName><![CDATA[fromUser]]></FromUserName>
<CreateTime>1348831860</CreateTime>
<MsgType><![CDATA[text]]></MsgType>
<Content><![CDATA[this is a test]]></Content>
@@ -143,7 +143,7 @@ class WechatModule(_ModuleBase, _MessageBase[WeChat]):
<MsgType><![CDATA[event]]></MsgType>
<Event><![CDATA[subscribe]]></Event>
<AgentID>1</AgentID>
</xml>
</xml>
"""
dom_tree = xml.dom.minidom.parseString(sMsg.decode('UTF-8'))
root_node = dom_tree.documentElement

View File

@@ -1,10 +1,13 @@
import json
import platform
import re
import subprocess
import threading
import time
import traceback
from pathlib import Path
from threading import Lock
from typing import Any, Optional
from typing import Any, Optional, Dict, List
from apscheduler.schedulers.background import BackgroundScheduler
from cachetools import TTLCache
@@ -65,8 +68,8 @@ class Monitor(metaclass=Singleton):
# 定时服务
_scheduler = None
# 存储快照
_storage_snapshot = {}
# 存储快照缓存目录
_snapshot_cache_dir = None
# 存储过照间隔(分钟)
_snapshot_interval = 5
@@ -77,6 +80,9 @@ class Monitor(metaclass=Singleton):
def __init__(self):
super().__init__()
self.all_exts = settings.RMT_MEDIAEXT
# 初始化快照缓存目录
self._snapshot_cache_dir = settings.TEMP_PATH / "snapshots"
self._snapshot_cache_dir.mkdir(exist_ok=True)
# 启动目录监控和文件整理
self.init()
@@ -94,6 +100,316 @@ class Monitor(metaclass=Singleton):
logger.info("配置变更事件触发,重新初始化目录监控...")
self.init()
def save_snapshot(self, storage: str, snapshot: Dict, file_count: int = 0):
"""
保存快照到文件
:param storage: 存储名称
:param snapshot: 快照数据
:param file_count: 文件数量,用于调整监控间隔
"""
try:
cache_file = self._snapshot_cache_dir / f"{storage}_snapshot.json"
snapshot_data = {
'timestamp': time.time(),
'file_count': file_count,
'snapshot': snapshot
}
with open(cache_file, 'w', encoding='utf-8') as f:
json.dump(snapshot_data, f, ensure_ascii=False, indent=2) # noqa
logger.debug(f"快照已保存到 {cache_file}")
except Exception as e:
logger.error(f"保存快照失败: {e}")
def reset_snapshot(self, storage: str) -> bool:
"""
重置快照,强制下次扫描时重新建立基准
:param storage: 存储名称
:return: 是否成功
"""
try:
cache_file = self._snapshot_cache_dir / f"{storage}_snapshot.json"
if cache_file.exists():
cache_file.unlink()
logger.info(f"快照已重置: {storage}")
return True
logger.debug(f"快照文件不存在,无需重置: {storage}")
return True
except Exception as e:
logger.error(f"重置快照失败: {storage} - {e}")
return False
def force_full_scan(self, storage: str, mon_path: Path) -> bool:
"""
强制全量扫描并处理所有文件(包括已存在的文件)
:param storage: 存储名称
:param mon_path: 监控路径
:return: 是否成功
"""
try:
logger.info(f"开始强制全量扫描: {storage}:{mon_path}")
# 生成快照
new_snapshot = StorageChain().snapshot_storage(
storage=storage,
path=mon_path,
last_snapshot_time=0 # 全量扫描,不使用增量
)
if new_snapshot is None:
logger.warn(f"获取 {storage}:{mon_path} 快照失败")
return False
file_count = len(new_snapshot)
logger.info(f"{storage}:{mon_path} 全量扫描完成,发现 {file_count} 个文件")
# 处理所有文件
processed_count = 0
for file_path, file_info in new_snapshot.items():
try:
logger.info(f"处理文件:{file_path}")
file_size = file_info.get('size', 0) if isinstance(file_info, dict) else file_info
self.__handle_file(storage=storage, event_path=Path(file_path), file_size=file_size)
processed_count += 1
except Exception as e:
logger.error(f"处理文件 {file_path} 失败: {e}")
continue
logger.info(f"{storage}:{mon_path} 全量扫描完成,共处理 {processed_count}/{file_count} 个文件")
# 保存快照
self.save_snapshot(storage, new_snapshot, file_count)
return True
except Exception as e:
logger.error(f"强制全量扫描失败: {storage}:{mon_path} - {e}")
return False
def load_snapshot(self, storage: str) -> Optional[Dict]:
"""
从文件加载快照
:param storage: 存储名称
:return: 快照数据或None
"""
try:
cache_file = self._snapshot_cache_dir / f"{storage}_snapshot.json"
if cache_file.exists():
with open(cache_file, 'r', encoding='utf-8') as f:
data = json.load(f)
logger.debug(f"成功加载快照: {cache_file}, 包含 {len(data.get('snapshot', {}))} 个文件")
return data
logger.debug(f"快照文件不存在: {cache_file}")
return None
except Exception as e:
logger.error(f"加载快照失败: {e}")
return None
@staticmethod
def adjust_monitor_interval(file_count: int) -> int:
"""
根据文件数量动态调整监控间隔
:param file_count: 文件数量
:return: 监控间隔(分钟)
"""
if file_count < 100:
return 5 # 5分钟
elif file_count < 500:
return 10 # 10分钟
elif file_count < 1000:
return 15 # 15分钟
else:
return 30 # 30分钟
@staticmethod
def compare_snapshots(old_snapshot: Dict, new_snapshot: Dict) -> Dict[str, List]:
"""
比对快照,找出变化的文件(只处理新增和修改,不处理删除)
:param old_snapshot: 旧快照
:param new_snapshot: 新快照
:return: 变化信息
"""
changes = {
'added': [],
'modified': []
}
old_files = set(old_snapshot.keys())
new_files = set(new_snapshot.keys())
# 新增文件
changes['added'] = list(new_files - old_files)
# 修改文件(大小或时间变化)
for file_path in old_files & new_files:
old_info = old_snapshot[file_path]
new_info = new_snapshot[file_path]
# 检查文件大小变化
old_size = old_info.get('size', 0) if isinstance(old_info, dict) else old_info
new_size = new_info.get('size', 0) if isinstance(new_info, dict) else new_info
# 检查修改时间变化(如果有的话)
old_time = old_info.get('modify_time', 0) if isinstance(old_info, dict) else 0
new_time = new_info.get('modify_time', 0) if isinstance(new_info, dict) else 0
if old_size != new_size or (old_time and new_time and old_time != new_time):
changes['modified'].append(file_path)
return changes
@staticmethod
def count_directory_files(directory: Path, max_check: int = 10000) -> int:
"""
统计目录下的文件数量(用于检测是否超过系统限制)
:param directory: 目录路径
:param max_check: 最大检查数量,避免长时间阻塞
:return: 文件数量
"""
try:
count = 0
import os
for root, dirs, files in os.walk(str(directory)):
count += len(files)
if count > max_check:
return count
return count
except Exception as err:
logger.debug(f"统计目录文件数量失败: {err}")
return 0
@staticmethod
def check_system_limits() -> Dict[str, Any]:
"""
检查系统限制
:return: 系统限制信息
"""
limits = {
'max_user_watches': 0,
'max_user_instances': 0,
'current_watches': 0,
'warnings': []
}
try:
system = platform.system()
if system == 'Linux':
# 检查 inotify 限制
try:
with open('/proc/sys/fs/inotify/max_user_watches', 'r') as f:
limits['max_user_watches'] = int(f.read().strip())
except Exception as e:
logger.debug(f"读取 inotify 限制失败: {e}")
limits['max_user_watches'] = 8192 # 默认值
try:
with open('/proc/sys/fs/inotify/max_user_instances', 'r') as f:
limits['max_user_instances'] = int(f.read().strip())
except Exception as e:
logger.debug(f"读取 inotify 实例限制失败: {e}")
# 检查当前使用的watches
try:
import subprocess
result = subprocess.run(['find', '/proc/*/fd', '-lname', 'anon_inode:inotify', '-printf', '%h\n'],
capture_output=True, text=True, timeout=5)
if result.returncode == 0:
limits['current_watches'] = len(result.stdout.strip().split('\n'))
except Exception as e:
logger.debug(f"检查当前 inotify 使用失败: {e}")
except Exception as e:
limits['warnings'].append(f"检查系统限制时出错: {e}")
return limits
@staticmethod
def get_system_optimization_tips() -> List[str]:
"""
获取系统优化建议
:return: 优化建议列表
"""
tips = []
system = platform.system()
if system == 'Linux':
tips.extend([
"增加 inotify 监控数量限制:",
"echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf",
"echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf",
"sudo sysctl -p",
"",
"如果在Docker中运行请在宿主机上执行以上命令"
])
elif system == 'Darwin':
tips.extend([
"macOS 系统优化建议:",
"sudo sysctl kern.maxfiles=65536",
"sudo sysctl kern.maxfilesperproc=32768",
"ulimit -n 32768"
])
elif system == 'Windows':
tips.extend([
"Windows 系统优化建议:",
"1. 关闭不必要的实时保护软件对监控目录的扫描",
"2. 将监控目录添加到Windows Defender排除列表",
"3. 确保有足够的可用内存"
])
return tips
def should_use_polling(self, directory: Path, monitor_mode: str,
file_count: int, limits: dict) -> tuple[bool, str]:
"""
判断是否应该使用轮询模式
:param directory: 监控目录
:param monitor_mode: 配置的监控模式
:param file_count: 目录文件数量
:param limits: 系统限制信息
:return: (是否使用轮询, 原因)
"""
if monitor_mode == "compatibility":
return True, "用户配置为兼容模式"
# 检查网络文件系统
if self.is_network_filesystem(directory):
return True, "检测到网络文件系统,建议使用兼容模式"
max_watches = limits.get('max_user_watches')
if max_watches and file_count > max_watches * 0.8:
return True, f"目录文件数量({file_count})接近系统限制({max_watches})"
return False, "使用快速模式"
@staticmethod
def is_network_filesystem(directory: Path) -> bool:
"""
检测是否为网络文件系统
:param directory: 目录路径
:return: 是否为网络文件系统
"""
try:
system = platform.system()
if system == 'Linux':
# 检查挂载信息
result = subprocess.run(['df', '-T', str(directory)],
capture_output=True, text=True, timeout=5)
if result.returncode == 0:
output = result.stdout.lower()
network_fs = ['nfs', 'cifs', 'smbfs', 'fuse', 'sshfs', 'ftpfs']
return any(fs in output for fs in network_fs)
elif system == 'Darwin':
# macOS 检查
result = subprocess.run(['df', '-T', str(directory)],
capture_output=True, text=True, timeout=5)
if result.returncode == 0:
output = result.stdout.lower()
return 'nfs' in output or 'smbfs' in output
elif system == 'Windows':
# Windows 检查网络驱动器
return str(directory).startswith('\\\\')
except Exception as e:
logger.debug(f"检测网络文件系统时出错: {e}")
return False
def init(self):
"""
启动监控
@@ -104,10 +420,12 @@ class Monitor(metaclass=Singleton):
# 读取目录配置
monitor_dirs = DirectoryHelper().get_download_dirs()
if not monitor_dirs:
logger.info("未找到任何目录监控配置")
return
# 按下载目录去重
monitor_dirs = list({f"{d.storage}_{d.download_path}": d for d in monitor_dirs}.values())
logger.info(f"找到 {len(monitor_dirs)} 个目录监控配置")
# 启动定时服务进程
self._scheduler = BackgroundScheduler(timezone=settings.TZ)
@@ -115,9 +433,12 @@ class Monitor(metaclass=Singleton):
messagehelper = MessageHelper()
for mon_dir in monitor_dirs:
if not mon_dir.library_path:
logger.warn(f"跳过监控配置 {mon_dir.download_path}:未设置媒体库目录")
continue
if mon_dir.monitor_type != "monitor":
logger.debug(f"跳过监控配置 {mon_dir.download_path}:监控类型为 {mon_dir.monitor_type}")
continue
# 检查媒体库目录是不是下载目录的子目录
mon_path = Path(mon_dir.download_path)
target_path = Path(mon_dir.library_path)
@@ -129,83 +450,238 @@ class Monitor(metaclass=Singleton):
# 启动监控
if mon_dir.storage == "local":
# 本地目录监控
logger.info(f"正在启动本地目录监控: {mon_path}")
logger.info("*** 重要提示:目录监控只处理新增和修改的文件,不会处理监控启动前已存在的文件 ***")
try:
if mon_dir.monitor_mode == "fast":
observer = self.__choose_observer()
else:
# 统计文件数量并给出提示
file_count = self.count_directory_files(mon_path)
logger.info(f"监控目录 {mon_path} 包含约 {file_count} 个文件")
# 检查系统限制
limits = self.check_system_limits()
# 检查是否需要使用轮询模式
use_polling, reason = self.should_use_polling(mon_path,
monitor_mode=mon_dir.monitor_mode,
file_count=file_count,
limits=limits)
logger.info(f"监控模式决策: {reason}")
if use_polling:
observer = PollingObserver()
logger.info(f"使用兼容模式(轮询)监控 {mon_path}")
else:
observer = self.__choose_observer()
if observer is None:
logger.warn(f"快速模式不可用,自动切换到兼容模式监控 {mon_path}")
observer = PollingObserver()
else:
logger.info(f"使用快速模式监控 {mon_path}")
if limits['warnings']:
for warning in limits['warnings']:
logger.warn(f"系统限制警告: {warning}")
if limits['max_user_watches'] > 0:
usage_percent = (file_count / limits['max_user_watches']) * 100
logger.info(
f"系统监控资源使用率: {usage_percent:.1f}% ({file_count}/{limits['max_user_watches']})")
self._observers.append(observer)
observer.schedule(FileMonitorHandler(mon_path=mon_path, callback=self),
path=str(mon_path),
recursive=True)
observer.daemon = True
observer.start()
logger.info(f"已启动 {mon_path} 的目录监控服务, 监控模式:{mon_dir.monitor_mode}")
mode_name = "兼容模式(轮询)" if use_polling else "快速模式"
logger.info(f"✓ 本地目录监控已启动: {mon_path} [{mode_name}]")
except Exception as e:
err_msg = str(e)
if "inotify" in err_msg and "reached" in err_msg:
logger.warn(
f"目录监控服务启动出现异常:{err_msg}请在宿主机上不是docker容器内执行以下命令并重启"
+ """
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
""")
logger.error(f"启动本地目录监控失败: {mon_path}")
logger.error(f"错误详情: {err_msg}")
if "inotify" in err_msg.lower():
logger.error("inotify 相关错误,这通常是由于系统监控数量限制导致的")
logger.error("解决方案:")
tips = self.get_system_optimization_tips()
for tip in tips:
logger.error(f" {tip}")
logger.error("执行上述命令后重启 MoviePilot")
elif "permission" in err_msg.lower():
logger.error("权限错误,请检查 MoviePilot 是否有足够的权限访问监控目录")
else:
logger.error(f"{mon_path} 启动目录监控失败:{err_msg}")
messagehelper.put(f"{mon_path} 启动目录监控失败:{err_msg}", title="目录监控")
logger.error("建议尝试使用兼容模式进行监控")
messagehelper.put(f"启动本地目录监控失败: {mon_path}\n错误: {err_msg}", title="目录监控")
else:
# 远程目录监控
self._scheduler.add_job(self.polling_observer, 'interval', minutes=self._snapshot_interval,
kwargs={
'storage': mon_dir.storage,
'mon_path': mon_path
})
# 远程目录监控 - 使用智能间隔
# 先尝试加载已有快照获取文件数量
snapshot_data = self.load_snapshot(mon_dir.storage)
file_count = snapshot_data.get('file_count', 0) if snapshot_data else 0
interval = self.adjust_monitor_interval(file_count)
logger.info(f"正在启动远程目录监控: {mon_path} [{mon_dir.storage}]")
logger.info("*** 重要提示:远程目录监控只处理新增和修改的文件,不会处理监控启动前已存在的文件 ***")
logger.info(f"预估文件数量: {file_count}, 监控间隔: {interval}分钟")
self._scheduler.add_job(
self.polling_observer,
'interval',
minutes=interval,
kwargs={
'storage': mon_dir.storage,
'mon_path': mon_path
},
id=f"monitor_{mon_dir.storage}_{mon_dir.download_path}",
replace_existing=True
)
logger.info(f"✓ 远程目录监控已启动: {mon_path} [间隔: {interval}分钟]")
# 启动定时服务
if self._scheduler.get_jobs():
self._scheduler.print_jobs()
self._scheduler.start()
logger.info("定时监控服务已启动")
@staticmethod
def __choose_observer() -> Any:
# 输出监控总结
local_count = len([d for d in monitor_dirs if d.storage == "local" and d.monitor_type == "monitor"])
remote_count = len([d for d in monitor_dirs if d.storage != "local" and d.monitor_type == "monitor"])
logger.info(f"目录监控启动完成: 本地监控 {local_count} 个,远程监控 {remote_count}")
def __choose_observer(self) -> Optional[Any]:
"""
选择最优的监控模式
选择最优的监控模式(带错误处理和自动回退)
"""
system = platform.system()
observers_to_try = []
try:
if system == 'Linux':
from watchdog.observers.inotify import InotifyObserver
return InotifyObserver()
observers_to_try = [
('InotifyObserver',
lambda: self.__try_import_observer('watchdog.observers.inotify', 'InotifyObserver')),
]
elif system == 'Darwin':
from watchdog.observers.fsevents import FSEventsObserver
return FSEventsObserver()
observers_to_try = [
('FSEventsObserver',
lambda: self.__try_import_observer('watchdog.observers.fsevents', 'FSEventsObserver')),
]
elif system == 'Windows':
from watchdog.observers.read_directory_changes import WindowsApiObserver
return WindowsApiObserver()
except Exception as error:
logger.warn(f"导入模块错误:{error},将使用 PollingObserver 监控目录")
return PollingObserver()
observers_to_try = [
('WindowsApiObserver',
lambda: self.__try_import_observer('watchdog.observers.read_directory_changes',
'WindowsApiObserver')),
]
# 尝试每个观察者
for observer_name, observer_func in observers_to_try:
try:
observer_class = observer_func()
if observer_class:
# 尝试创建实例以验证是否可用
test_observer = observer_class()
test_observer.stop() # 立即停止测试实例
logger.debug(f"成功初始化 {observer_name}")
return observer_class()
except Exception as e:
logger.debug(f"初始化 {observer_name} 失败: {e}")
continue
except Exception as e:
logger.debug(f"选择观察者时出错: {e}")
logger.debug("所有快速监控模式都不可用,将使用兼容模式")
return None
@staticmethod
def __try_import_observer(module_name: str, class_name: str):
"""
尝试导入观察者类
"""
try:
module = __import__(module_name, fromlist=[class_name])
return getattr(module, class_name)
except (ImportError, AttributeError) as e:
logger.debug(f"导入 {module_name}.{class_name} 失败: {e}")
return None
def polling_observer(self, storage: str, mon_path: Path):
"""
轮询监控
轮询监控(改进版)
"""
with snapshot_lock:
# 快照存储
new_snapshot = StorageChain().snapshot_storage(storage=storage, path=mon_path)
if new_snapshot:
# 比较快照
old_snapshot = self._storage_snapshot.get(storage)
if old_snapshot:
# 新增的文件
new_files = new_snapshot.keys() - old_snapshot.keys()
for new_file in new_files:
# 添加到待整理队列
self.__handle_file(storage=storage, event_path=Path(new_file),
file_size=new_snapshot.get(new_file))
# 更新快照
self._storage_snapshot[storage] = new_snapshot
try:
logger.debug(f"开始对 {storage}:{mon_path} 进行快照...")
# 加载上次快照数据
old_snapshot_data = self.load_snapshot(storage)
old_snapshot = old_snapshot_data.get('snapshot', {}) if old_snapshot_data else {}
last_snapshot_time = old_snapshot_data.get('timestamp', 0) if old_snapshot_data else 0
# 判断是否为首次快照:检查快照文件是否存在且有效
is_first_snapshot = old_snapshot_data is None
# 生成新快照(增量模式)
new_snapshot = StorageChain().snapshot_storage(
storage=storage,
path=mon_path,
last_snapshot_time=last_snapshot_time
)
if new_snapshot is None:
logger.warn(f"获取 {storage}:{mon_path} 快照失败")
return
file_count = len(new_snapshot)
logger.info(f"{storage}:{mon_path} 快照完成,发现 {file_count} 个文件")
if not is_first_snapshot:
# 比较快照找出变化
changes = self.compare_snapshots(old_snapshot, new_snapshot)
# 处理新增文件
for new_file in changes['added']:
logger.info(f"发现新增文件:{new_file}")
file_info = new_snapshot.get(new_file, {})
file_size = file_info.get('size', 0) if isinstance(file_info, dict) else file_info
self.__handle_file(storage=storage, event_path=Path(new_file), file_size=file_size)
# 处理修改文件
for modified_file in changes['modified']:
logger.info(f"发现修改文件:{modified_file}")
file_info = new_snapshot.get(modified_file, {})
file_size = file_info.get('size', 0) if isinstance(file_info, dict) else file_info
self.__handle_file(storage=storage, event_path=Path(modified_file), file_size=file_size)
if changes['added'] or changes['modified']:
logger.info(
f"{storage}:{mon_path} 发现 {len(changes['added'])} 个新增文件,{len(changes['modified'])} 个修改文件")
else:
logger.debug(f"{storage}:{mon_path} 无文件变化")
else:
logger.info(f"{storage}:{mon_path} 首次快照完成,共 {file_count} 个文件")
logger.info("*** 首次快照仅建立基准,不会处理现有文件。后续监控将处理新增和修改的文件 ***")
# 保存新快照
self.save_snapshot(storage, new_snapshot, file_count)
# 动态调整监控间隔
new_interval = self.adjust_monitor_interval(file_count)
current_job = self._scheduler.get_job(f"monitor_{storage}_{mon_path}")
if current_job and current_job.trigger.interval.total_seconds() / 60 != new_interval:
# 重新安排任务
self._scheduler.modify_job(
f"monitor_{storage}_{mon_path}",
trigger='interval',
minutes=new_interval
)
logger.info(f"{storage}:{mon_path} 监控间隔已调整为 {new_interval} 分钟")
except Exception as e:
logger.error(f"轮询监控 {storage}:{mon_path} 出现错误:{e}")
logger.debug(traceback.format_exc())
def event_handler(self, event, text: str, event_path: str, file_size: float = None):
"""
@@ -217,7 +693,7 @@ class Monitor(metaclass=Singleton):
"""
if not event.is_directory:
# 文件发生变化
logger.debug(f"文件 {event_path} 发生了 {text}")
logger.debug(f"检测到文件变化: {event_path} [{text}]")
# 整理文件
self.__handle_file(storage="local", event_path=Path(event_path), file_size=file_size)
@@ -254,10 +730,12 @@ class Monitor(metaclass=Singleton):
# TTL缓存控重
if self._cache.get(str(event_path)):
logger.debug(f"文件 {event_path} 在缓存中,跳过处理")
return
self._cache[str(event_path)] = True
try:
logger.info(f"开始整理文件: {event_path}")
# 开始整理
TransferChain().do_transfer(
fileitem=FileItem(
@@ -271,7 +749,7 @@ class Monitor(metaclass=Singleton):
)
)
except Exception as e:
logger.error("目录监控发生错误:%s - %s" % (str(e), traceback.format_exc()))
logger.error("目录监控整理文件发生错误:%s - %s" % (str(e), traceback.format_exc()))
def stop(self):
"""
@@ -279,20 +757,22 @@ class Monitor(metaclass=Singleton):
"""
self._event.set()
if self._observers:
logger.info("正在停止本地目录监控服务...")
for observer in self._observers:
try:
logger.info(f"正在停止目录监控服务:{observer}...")
observer.stop()
observer.join()
logger.info(f"{observer} 目录监控已停止")
logger.debug(f"已停止监控服务: {observer}")
except Exception as e:
logger.error(f"停止目录监控服务出现了错误:{e}")
self._observers = []
logger.info("本地目录监控服务已停止")
if self._scheduler:
self._scheduler.remove_all_jobs()
if self._scheduler.running:
try:
self._scheduler.shutdown()
logger.info("定时监控服务已停止")
except Exception as e:
logger.error(f"停止定时服务出现了错误:{e}")
self._scheduler = None

View File

@@ -57,6 +57,8 @@ class MetaInfo(BaseModel):
audio_encode: Optional[str] = None
# 资源类型
edition: Optional[str] = None
# 流媒体平台
web_source: Optional[str] = None
# 应用的识别词信息
apply_words: Optional[List[str]] = None

View File

@@ -290,6 +290,7 @@ class StorageSchema(Enum):
U115 = "u115"
Rclone = "rclone"
Alist = "alist"
SMB = "smb"
# 模块类型

View File

@@ -76,6 +76,15 @@ class AutoCloseResponse:
"""
self._auto_close()
def __setstate__(self, state):
for name, value in state.items():
setattr(self, name, value)
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
class RequestUtils:
@@ -523,7 +532,7 @@ class RequestUtils:
def get_json(self, url: str, params: dict = None, **kwargs) -> Optional[dict]:
"""
发送GET请求并返回JSON数据自动关闭连接
:param url: 请求的URL
:param url: 请求的URL
:param params: 请求的参数
:param kwargs: 其他请求参数
:return: JSON数据若发生异常则返回None

View File

@@ -445,6 +445,24 @@ class SystemUtils:
process_memory_percent = (process_memory / system_memory) * 100
return [process_memory, int(process_memory_percent)]
@staticmethod
def network_usage() -> List[int]:
"""
获取当前网络流量上行和下行流量单位bytes/s
"""
import time
# 获取初始网络统计
net_io_1 = psutil.net_io_counters()
time.sleep(1) # 等待1秒
# 获取1秒后的网络统计
net_io_2 = psutil.net_io_counters()
# 计算1秒内的流量变化
upload_speed = net_io_2.bytes_sent - net_io_1.bytes_sent
download_speed = net_io_2.bytes_recv - net_io_1.bytes_recv
return [upload_speed, download_speed]
@staticmethod
def is_hardlink(src: Path, dest: Path) -> bool:
"""

View File

@@ -1,7 +1,7 @@
"""2.0.0
Revision ID: 294b007932ef
Revises:
Revises:
Create Date: 2024-07-20 08:43:40.741251
"""

View File

@@ -15,25 +15,25 @@ http {
server {
listen 38379;
server_name localhost;
access_log /dev/stdout combined;
error_log /dev/stdout;
location / {
proxy_pass http://docker;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 120;
proxy_read_timeout 120;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;

View File

@@ -131,9 +131,9 @@ function load_config_from_app_env() {
# (例如 envsubst, mp_update.sh, cert.sh)
if declare -gx "${var_name}=${final_value}"; then
if [ -z "${final_value}" ]; then
INFO "变量 ${var_name}, 值为空, 来源: ${value_source})。"
INFO "变量 ${var_name}, 值为空 (来源: ${value_source})。"
else
INFO "变量 ${var_name}, 值: ${final_value} , (来源: ${value_source})。"
INFO "变量 ${var_name}, 值: ${final_value} (来源: ${value_source})。"
fi
# 如果变量不是来自初始环境变量,则记录下来以便稍后 unset
@@ -151,7 +151,7 @@ function load_config_from_app_env() {
fi
fi
else
ERROR "导出变量 ${var_name} (值: '${final_value}', 来源: ${value_source}) 失败。"
ERROR "导出变量 ${var_name}, 值: '${final_value}'失败 (来源: ${value_source}) "
fi
done

View File

@@ -61,7 +61,7 @@ pip install pip-tools
```bash
pip-compile --upgrade-package requests requirements.in
```
3. **全量更新依赖项**
如果你想更新 `requirements.in` 中的所有依赖包,运行以下命令生成或更新 `requirements.txt` 文件:

View File

@@ -25,7 +25,7 @@ pytz~=2025.2
pycryptodome~=3.23.0
qbittorrent-api==2025.5.0
plexapi~=4.17.0
transmission-rpc~=7.0.11
transmission-rpc~=4.3.0
Jinja2~=3.1.6
pyparsing~=3.2.3
func_timeout==4.3.5
@@ -46,7 +46,6 @@ psutil~=7.0.0
python-dotenv~=1.1.1
python-hosts~=1.1.2
watchdog~=6.0.0
openai~=1.92.2
cacheout~=0.16.0
click~=8.2.1
requests-cache~=1.2.1
@@ -70,3 +69,5 @@ oss2~=2.19.1
tqdm~=4.67.1
setuptools~=78.1.0
pympler~=1.1
smbprotocol~=1.15.0
setproctitle~=1.3.6

View File

@@ -1,2 +1,2 @@
APP_VERSION = 'v2.5.9'
FRONTEND_VERSION = 'v2.5.9'
APP_VERSION = 'v2.6.2'
FRONTEND_VERSION = 'v2.6.2'