Compare commits

...

117 Commits

Author SHA1 Message Date
jxxghp
88688672db Merge pull request #939 from KoWming/main 2025-12-18 12:42:40 +08:00
KoWming
ff9c35041e perf: 增加签到检测机制防止重复签到,增强代码健壮性。 2025-12-17 10:13:21 +08:00
KoWming
d9afb64d00 perf: 尝试修复签到失败问题,新增使用代理、Cookie自动更新功能
尝试修复签到失败问题,新增使用代理、Cookie自动更新功能
2025-12-16 11:24:17 +08:00
jxxghp
6d60123272 Merge pull request #938 from wumode/lexiannot 2025-12-13 12:41:26 +08:00
wumode
84fcc3762f fix(LexiAnnot): Array index error 2025-12-12 19:15:21 +08:00
wumode
77b34dba5c feat(lexiannot): Add QueryAnnotationTasksTool 2025-12-12 15:01:56 +08:00
wumode
4d8f36f674 feat(lexiannot): Add exam_distribution to SegmentList schema 2025-12-12 11:19:40 +08:00
wumode
5ccbb412eb feat(tobypasstrackers): Add new trackers and refine IP exclusion logic 2025-12-11 11:34:36 +08:00
wumode
4a0c700e6b feat(imdbsource): Add background image support to metadata and update version to 1.6.4 2025-12-11 11:23:12 +08:00
wumode
00c65a0983 feat(lexiannot): Integrate LLM for advanced vocabulary processing 2025-12-11 11:23:12 +08:00
jxxghp
b961a52440 Merge pull request #937 from Seed680/main 2025-12-10 12:10:57 +08:00
noone
707feedda2 fix(mediaservermsg): 优化剧集图片查询逻辑
- 修改查询条件,仅当项目类型为TV或SHOW时执行剧集图片查询
2025-12-10 10:41:55 +08:00
noone
07c6ee1341 fix(mediaservermsg): 修复媒体服务器消息图片查询逻辑
- 修正了电影图片查询条件,确保仅在有 tmdb_id 时执行
2025-12-10 10:40:55 +08:00
noone
fd360cf21d bugfix:MediaServerMsg1.7.1 2025-12-10 10:36:56 +08:00
noone
a267df9e5d Merge branch 'main' of https://github.com/Seed680/MoviePilot-Plugins-main 2025-12-10 18:18:40 +08:00
noone
8feecbcb42 fix(mediaservermsg): 修复未获取到tmdb信息时的消息发送逻辑
- 当tmdb_id为空时,回退到原有逻辑发送通知消息
- 为电影类型添加海报图片查询支持
2025-12-10 18:18:25 +08:00
jxxghp
4224939f30 Merge pull request #936 from Seed680/main 2025-12-09 11:51:43 +08:00
Seed680
234ceba60c Merge branch 'jxxghp:main' into main 2025-12-09 10:23:50 +08:00
noone
5c8a6647e2 feat(mediaservermsg): 新增TV剧集结入库聚合功能
- 实现TV剧集入库事件的智能聚合,避免消息轰炸
- 添加聚合时间窗口配置,默认15秒
- 支持通过TMDB ID获取剧集详细信息用于消息展示
- 自动合并连续集数信息,优化消息可读性
- 增加聚合功能开关和时间配置项到插件表单
- 完善消息构造逻辑,支持剧集封面和背景图展示
- 优化消息缓存机制,提升重复消息过滤效果
- 补充详细的日志记录便于问题排查
- 更新插件版本至1.7并添加更新说明提示
2025-12-09 10:22:52 +08:00
jxxghp
5b763dff42 Merge pull request #933 from cikezhu/main 2025-12-06 03:52:20 +08:00
cikezhu
ee453841df fix 更新标签而传递旧的种子列表 2025-12-05 22:51:50 +08:00
cikezhu
6768d2c244 add 站点/剧名前缀功能 2025-12-05 22:44:49 +08:00
jxxghp
cb14efcc68 Merge pull request #932 from cikezhu/main 2025-12-05 17:54:08 +08:00
cikezhu
7871dfd0b8 fix gemini-code-assist suggestion‌ 2025-12-05 17:01:33 +08:00
cikezhu
99d1bfe37e fix 2025-12-05 16:24:15 +08:00
cikezhu
b65c1b8bf7 首次运行时初始化rid映射 2025-12-05 16:18:09 +08:00
cikezhu
517a16f0a3 优化采用公共服务自动清理未使用标签 2025-12-05 16:13:38 +08:00
jxxghp
89bfb9750d Merge pull request #930 from cikezhu/main 2025-12-04 22:28:59 +08:00
cikezhu
01eac66a6a fix _get_label 2025-12-04 22:11:57 +08:00
cikezhu
cd53b8d454 add 自动清除未使用标签 2025-12-04 21:56:48 +08:00
jxxghp
d986f45634 Merge pull request #928 from cikezhu/main 2025-12-03 14:40:56 +08:00
cikezhu
0ceb633d96 通过直接解析而不是硬编码解决代码冗余问题 2025-12-03 10:54:12 +08:00
cikezhu
2965743cfe fix 首次更新时页面配置内容 2025-12-03 09:55:16 +08:00
cikezhu
9fa02d62e2 调整界面显示效果 2025-12-03 09:34:18 +08:00
cikezhu
b2bd0f3701 增加tracker映射配置 2025-12-03 09:26:01 +08:00
jxxghp
de0e83f830 Merge pull request #924 from wumode/tobypasstrackers 2025-11-20 17:39:24 +08:00
wumode
94b6df246e fix(LexiAnnot): Handle subtitle duration parsing errors 2025-11-20 17:35:17 +08:00
wumode
6b895919a0 update package.v2.json 2025-11-20 11:53:38 +08:00
wumode
a9830202e8 feat(LexiAnnot): Improve subtitle selection strategy 2025-11-20 10:41:06 +08:00
wumode
e96eece117 feat(ToBypassTrackers): Add Page UI, IP check command, and refactor DNS resolution 2025-11-19 19:34:10 +08:00
jxxghp
107b8e408f 更新 __init__.py 2025-11-19 07:13:59 +08:00
jxxghp
6629aeadef Merge pull request #923 from jxxghp/cursor/update-openai-library-for-chatgpt-plugin-2cf9 2025-11-19 07:08:20 +08:00
Cursor Agent
b0e5680260 Fix: Update OpenAI API compatibility and version
Co-authored-by: jxxghp <jxxghp@qq.com>
2025-11-18 23:06:26 +00:00
Cursor Agent
a322274d77 Refactor: Use new OpenAI client and handle exceptions
Co-authored-by: jxxghp <jxxghp@qq.com>
2025-11-18 23:02:16 +00:00
jxxghp
be2289739a Merge pull request #922 from xslidi/main 2025-11-18 21:03:50 +08:00
xslidi
7536a8782e Support GPW url 2025-11-18 20:34:16 +08:00
jxxghp
4d71a24fbc fix requirements 2025-11-18 16:47:48 +08:00
jxxghp
85ac9dd393 feat(README): 添加get_tool_message方法的说明,要求实现友好的提示消息 2025-11-18 12:52:23 +08:00
jxxghp
75c65b96d4 feat(README): 添加插件中注册智能体工具的说明和示例代码 2025-11-18 11:25:09 +08:00
jxxghp
7d8433b768 Merge pull request #920 from wumode/pydantic_v2 2025-11-17 15:03:31 +08:00
jxxghp
d66413dd7a Merge pull request #918 from Seed680/main 2025-11-17 15:03:08 +08:00
wumode
a0c9afc3ed refactor(ClashRuleProvider, ImdbSource and LexiAnnot): Adapt to Pydantic V2 2025-11-17 14:18:54 +08:00
noone
e0c39170e6 feat(brushflow): 添加RSS支持配置选项 2025-11-06 16:36:22 +08:00
noone
8e199afe24 feat(brushflow): 添加RSS支持配置选项
- 在BrushConfig中新增rss_support字段,默认值为False
- 更新配置描述和默认配置示例,包含rss_support选项
- 在插件页面中添加RSS支持开关组件
- 根据rss_support配置决定使用browse或rss方法获取种子
- 升级插件版本从4.3.3到4.3.4
2025-11-06 16:34:07 +08:00
jxxghp
e68d915f36 Merge pull request #917 from developer-wlj/main 2025-11-04 18:17:56 +08:00
developer-wlj
b3e78c3e5e chore(plugins): 更新媒体同步删除插件版本至1.7.2
- 兼容Windows路径格式,将反斜杠转换为正斜杠
2025-11-04 15:55:32 +08:00
jxxghp
f02b90552b Merge pull request #914 from wumode/imdbsource 2025-11-01 19:51:22 +08:00
jxxghp
e93bfc6667 Merge pull request #913 from KoWming/main 2025-11-01 19:51:06 +08:00
wumode
131463cfbe update(ImdbSource): set delay to 1s 2025-10-26 23:01:05 +08:00
KoWming
b963398987 修复签到失败问题,新增账户登录签到功能、新增签到失败重试机制,美化界面UI
增强代码健壮性。
2025-10-23 11:35:18 +08:00
KoWming
ed395a26a9 Update package.json 2025-10-23 11:34:40 +08:00
wumode
03a2b35930 fix(ImdbSource): API 查询错误重试 2025-10-23 01:19:48 +08:00
KoWming
5a642e1e51 修复签到失败问题,新增账户登录签到功能、新增签到失败重试机制,美化界面UI。 2025-10-22 10:12:42 +08:00
KoWming
a8813b0272 Update package.json 2025-10-22 10:12:05 +08:00
jxxghp
66ce816a31 Merge pull request #908 from wumode/clashruleprovider 2025-10-08 10:30:35 +08:00
wumode
241e3200f8 fix: typo 2025-10-08 00:57:09 +08:00
wumode
19f52d6217 update: ImdbSource & ClashRuleProvider
ImdbSource
- 使用 Pydantic 重构 IMDb API

ClashRuleProvider
- fix: 过早实例化系统 Scheduler
- fix: 缺少 PyYAML
- 配置使用 Pydantic
2025-10-08 00:40:47 +08:00
jxxghp
884efaebbf Merge pull request #907 from wumode/clashruleprovider 2025-10-02 20:57:53 +08:00
wumode
b51ba3d92a fix(ClashRuleProvider): rule comparing 2025-10-02 17:39:35 +08:00
wumode
ec74481160 fix(ClashRuleProvider): rule comparing 2025-10-02 01:33:56 +08:00
jxxghp
c60a4f01aa Merge pull request #906 from wumode/clashruleprovider 2025-10-01 12:48:13 +08:00
wumode
e34cafd641 fix(ClashRuleProvider): typo 2025-10-01 11:57:14 +08:00
wumode
5f8bb72641 update(ClashRuleProvider): 使用secrets.compare_digest() 2025-10-01 10:53:44 +08:00
wumode
df3e42987a update(LexiAnnot): 增加上下文上限 2025-09-30 01:00:34 +08:00
wumode
8a738b7684 refactor: ClashRuleProvider
- 优化插件目录结构和数据结构, 解耦API层和服务层
- 添加了一些Pydantic模型, 用于校验配置
- 支持独立的订阅链接配置
- 新增覆写代理组和出站代理操作
- 支持 smart 组和代理集合
- 代理组回环检测
- 使用异步调度器
- 显示规则更改日期
- 完善了对嵌套逻辑规则和子规则的配置和验证
2025-09-30 00:54:24 +08:00
jxxghp
491f40663b fix logging 2025-09-17 13:37:56 +08:00
jxxghp
fe8a7c6cd2 Merge pull request #897 from hizml/main 2025-09-09 18:17:17 +08:00
ZhaoML
6245940466 fix(Cloudflare IP优选): 修复 IPv6 地址含双引号导致的执行错误
- 移除了 cloudflarespeedtest 插件中执行命令时对 IPv6 地址的双引号
- 该修改解决了当 IPv6 地址包含双引号时,命令无法正确执行的问题
2025-09-09 17:52:06 +08:00
ZhaoML
c86cbc473f feat(cloudflarespeedtest): 适配 CloudflareSpeedTest 新版名称
- 更新插件版本至 1.5
- 修改二进制文件名称从 CloudflareST 到 cfst
- 增加旧版本兼容性处理
- 更新下载链接和安装逻辑以适应新名称
- 在 package.json 中添加新版本历史记录
2025-09-09 17:27:44 +08:00
jxxghp
d93665a572 fix BugReporter 2025-09-08 15:38:21 +08:00
jxxghp
250ee4ada8 Merge pull request #896 from wumode/clashruleprovider 2025-09-08 15:05:11 +08:00
wumode
dfe2247b25 update(ClashRuleProvider): 支持显示节点链接 2025-09-08 13:02:10 +08:00
jxxghp
858261ddcc Merge pull request #895 from JavaZeroo/fix_autosubv2_zero_lang 2025-09-07 12:40:49 +08:00
JavaZeroo
47bf56afe4 feat(AutoSubv2): add auto language detection and improve translation retry logic 2025-09-07 12:12:08 +08:00
jxxghp
af3956d86f Merge pull request #894 from JavaZeroo/fix_autosubv2_zero_division 2025-09-07 06:56:01 +08:00
JavaZeroo
a69feb73ca fix(AutoSubv2): handle empty subtitle files and improve success rate logging 2025-09-07 01:00:30 +08:00
jxxghp
88b29169fc Merge pull request #890 from wumode/clashruleprovider 2025-09-02 18:23:59 +08:00
wumode
2c9e108ac4 fix(ClashRuleProvider): 保持键名一致性 2025-09-02 13:24:24 +08:00
wumode
73b2d778a0 fix(ClashRuleProvider): 配置模板保存问题 2025-09-02 12:39:13 +08:00
jxxghp
bf67d6e567 Merge pull request #889 from wumode/clashruleprovider 2025-09-01 20:38:22 +08:00
wumode
5e9da0802d update(ClashRuleProvider): 优化性能 2025-09-01 20:38:03 +08:00
wumode
2811021996 update(ClashRuleProvider): 优化 UI 2025-09-01 20:21:09 +08:00
jxxghp
8c0a05b2de Merge pull request #888 from wumode/lexiannot 2025-08-29 18:45:54 +08:00
wumode
bb070bf83e 使用字典键直接访问 token 2025-08-29 18:41:27 +08:00
wumode
21aec36ea5 update(LexiAnnot): 避免spaCy模型常驻内存 2025-08-29 15:36:48 +08:00
jxxghp
6019cf92ac fix BugReporter 2025-08-28 08:21:15 +08:00
jxxghp
42d5dd1e89 fix BugReporter 2025-08-27 17:43:58 +08:00
jxxghp
0b3313e078 update PersonMeta 2025-08-27 16:07:01 +08:00
jxxghp
5684ba056a update package.v2.json 2025-08-27 09:59:39 +08:00
jxxghp
44af7dbb78 add BugReporter 2025-08-27 09:53:15 +08:00
jxxghp
2102a03740 Merge pull request #885 from wumode/clashruleprovider 2025-08-24 18:46:30 +08:00
wumode
0a9cadf7ab update(ClashRuleProvider): 通过emoji识别国家 2025-08-24 18:06:43 +08:00
jxxghp
279efe8000 Merge pull request #883 from wumode/lexiannot 2025-08-23 17:18:18 +08:00
wumode
fd92e58f81 update(ImdbSource) 修复错误 2025-08-23 16:58:00 +08:00
wumode
fe93e46e02 update(ImdbSource) 修改UA 2025-08-23 00:01:13 +08:00
wumode
cbf541992f update(LexiAnnot): 添加任务页面 2025-08-22 17:03:07 +08:00
jxxghp
8e1d336250 add 统一缓存使用说明 2025-08-21 16:06:23 +08:00
jxxghp
12e0e2b9f5 Merge pull request #881 from wumode/imdbsource 2025-08-20 00:34:31 +08:00
wumode
ac914f70f3 update: ImdbSource&ToBypassTrackers 2025-08-20 00:10:09 +08:00
jxxghp
a07b8a4f4a Merge pull request #878 from wumode/lexiannot 2025-08-17 20:13:53 +08:00
wumode
6960b3f7aa update(LexiAnnot): 支持考试词汇标注 2025-08-17 19:50:30 +08:00
jxxghp
fe83ff1be8 Merge pull request #876 from liuhangbin/multiclass 2025-08-14 19:40:50 +08:00
Hangbin Liu
6357dc8e4a plugins.v2: 添加多级分类插件
目前MP默认只支持二级分类,但是部分用户有多级目录的需求,比如增加按照年代
或者评分度分类。因此增加一个支持多级分类的插件, 目前仅支持电影多级分类。

Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
2025-08-14 19:16:45 +08:00
jxxghp
f1d94d0aa3 Merge pull request #875 from yelantf/main 2025-08-12 12:08:56 +08:00
夜阑听风
53dd3bc796 Update dingdingmsg in package.json 2025-08-12 10:59:53 +08:00
夜阑听风
a9d528fc05 Update dingdingmsg version 2025-08-12 10:58:25 +08:00
夜阑听风
0388c437b1 update dingdingmsg to support breakline 2025-08-12 10:56:24 +08:00
108 changed files with 29946 additions and 18048 deletions

287
README.md
View File

@@ -23,6 +23,8 @@ MoviePilot官方插件市场https://github.com/jxxghp/MoviePilot-Plugins
- [12. 如何通过插件扩展支持的存储类型?](#12-如何通过插件扩展支持的存储类型)
- [13. 如何将插件功能集成到工作流?](#13-如何将插件功能集成到工作流)
- [14. 如何在插件中通过消息持续与用户交互?](#14-如何在插件中通过消息持续与用户交互)
- [15. 如何在插件中使用系统级统一缓存?](#15-如何在插件中使用系统级统一缓存)
- [16. 如何在插件中注册智能体工具?](#16-如何在插件中注册智能体工具)
- [版本发布](#版本发布)
- [1. 如何发布插件版本?](#1-如何发布插件版本)
- [2. 如何开发V2版本的插件以及实现插件多版本兼容](#2-如何开发v2版本的插件以及实现插件多版本兼容)
@@ -1167,6 +1169,291 @@ def get_actions(self) -> List[Dict[str, Any]]:
- 建议在交互中保存用户状态数据,以支持复杂的多步骤操作
- 可以结合插件数据存储功能保存用户的交互历史和偏好设置
### 15. 如何在插件中使用系统级统一缓存?
**(仅支持 `v2.7.4+` 版本)**
- MoviePilot提供了统一的缓存系统支持内存缓存、文件系统缓存和Redis缓存自动管理当有Redis时优先使用Redis否则使用内存或文件系统。插件可以通过系统提供的缓存接口实现高效的缓存管理无需关心系统设置。
- 1. 使用缓存装饰器:
```python
from app.core.cache import cached
class MyPlugin(_PluginBase):
@cached(region="my_plugin", ttl=3600)
def get_data(self, key: str):
"""
使用缓存装饰器缓存结果1小时
"""
# 复杂的计算或网络请求
return expensive_operation(key)
@cached(region="my_plugin_async", ttl=1800, skip_none=True)
async def get_async_data(self, key: str):
"""
异步函数缓存跳过None值
"""
return await async_expensive_operation(key)
```
- 2. 使用TTLCache类
```python
from app.core.cache import TTLCache
class MyPlugin(_PluginBase):
def __init__(self):
super().__init__()
# 创建缓存实例最大128项TTL 30分钟
self.cache = TTLCache(region="my_plugin", maxsize=128, ttl=1800)
def process_data(self, key: str):
# 检查缓存
if key in self.cache:
return self.cache[key]
# 计算并缓存结果
result = expensive_operation(key)
self.cache[key] = result
return result
def clear_cache(self):
"""
清理插件缓存
"""
self.cache.clear()
```
- 3. 使用文件缓存后端(适用于大文件缓存):
```python
from app.core.cache import FileCache, AsyncFileCache
from pathlib import Path
class MyPlugin(_PluginBase):
def __init__(self):
super().__init__()
# 获取文件缓存后端支持Redis和文件系统
self.file_cache = FileCache(
base=Path("/tmp/my_plugin_cache"),
ttl=86400 # 24小时
)
def cache_large_file(self, key: str, data: bytes):
"""
缓存大文件数据
"""
self.file_cache.set(key, data, region="large_files")
def get_cached_file(self, key: str) -> Optional[bytes]:
"""
获取缓存的文件数据
"""
return self.file_cache.get(key, region="large_files")
async def async_cache_operations(self):
"""
异步文件缓存操作
"""
async_cache = AsyncFileCache(
base=Path("/tmp/my_plugin_async_cache"),
ttl=3600
)
# 异步设置缓存
await async_cache.set("async_key", b"async_data", region="async_files")
# 异步获取缓存
data = await async_cache.get("async_key", region="async_files")
await async_cache.close()
```
- 4. 直接使用缓存后端(高级用法):
```python
from app.core.cache import Cache
class MyPlugin(_PluginBase):
def __init__(self):
super().__init__()
# 直接获取缓存后端实例系统自动选择Redis或内存缓存
self.cache_backend = Cache(maxsize=256, ttl=3600)
def custom_cache_operation(self, key: str, value: Any):
"""
自定义缓存操作
"""
# 设置缓存
self.cache_backend.set(key, value, region="custom_region")
# 检查缓存是否存在
if self.cache_backend.exists(key, region="custom_region"):
# 获取缓存
cached_value = self.cache_backend.get(key, region="custom_region")
return cached_value
return None
def iterate_cache_items(self):
"""
遍历缓存项
"""
for key, value in self.cache_backend.items(region="custom_region"):
print(f"缓存键: {key}, 值: {value}")
def cleanup(self):
"""
清理缓存
"""
self.cache_backend.clear(region="custom_region")
self.cache_backend.close()
```
- 5. 缓存装饰器参数说明:
```python
@cached(
region="my_plugin", # 缓存区域,用于隔离不同插件的缓存
maxsize=512, # 最大缓存条目数(仅内存缓存有效)
ttl=1800, # 缓存存活时间(秒)
skip_none=True, # 是否跳过None值缓存
skip_empty=False # 是否跳过空值缓存(空列表、空字典等)
)
def my_function(self, param):
pass
```
- 6. 缓存管理功能:
```python
class MyPlugin(_PluginBase):
@cached(region="my_plugin")
def cached_function(self, param):
return expensive_operation(param)
def clear_my_cache(self):
"""
清理指定区域的缓存
"""
self.cached_function.cache_clear()
def get_cache_info(self):
"""
获取缓存信息
"""
cache_region = self.cached_function.cache_region
return f"缓存区域: {cache_region}"
```
- 7. 缓存后端自动选择:
- 系统会根据配置自动选择缓存后端:
- `CACHE_BACKEND_TYPE=redis`使用Redis作为缓存后端
- `CACHE_BACKEND_TYPE=memory`使用内存缓存cachetools
- 插件代码无需修改,系统会自动处理缓存后端的切换
- 8. 最佳实践:
- 为每个插件使用独立的缓存区域region避免缓存键冲突
- 合理设置TTL避免缓存过期时间过长导致数据过期
- 对于频繁访问的数据使用较长的TTL对于实时性要求高的数据使用较短的TTL
- 使用`skip_none=True`避免缓存无意义的None值
- 大文件或二进制数据建议使用文件缓存后端
- 在插件卸载时清理相关缓存,避免内存泄漏
### 16. 如何在插件中注册智能体工具?
**(仅支持 `v2.8.0+` 版本)**
- MoviePilot的AI智能体功能支持通过插件扩展工具能力插件可以注册自定义工具供智能体调用实现更丰富的功能扩展。
- 1. 实现 `get_agent_tools()` 方法,返回工具类列表:
```python
def get_agent_tools(self) -> List[Type]:
"""
获取插件智能体工具
返回工具类列表,每个工具类必须继承自 MoviePilotTool
"""
return [MyCustomTool, AnotherTool]
```
- 2. 创建工具类,必须继承自 `MoviePilotTool` 并实现相关要求:
```python
from typing import Optional, Type
from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.chain import ToolChain
class MyToolInput(BaseModel):
"""工具输入参数模型"""
explanation: str = Field(..., description="工具使用说明")
query: str = Field(..., description="查询内容")
limit: Optional[int] = Field(10, description="返回结果数量限制")
class MyCustomTool(MoviePilotTool):
"""自定义工具示例"""
# 工具名称,用于智能体识别和调用
name: str = "my_custom_tool"
# 工具描述,用于智能体理解工具功能,建议详细描述工具用途和使用场景
description: str = "This tool is used to perform custom operations. Use it when you need to query or process specific data."
# 输入参数模型,定义工具接收的参数及其类型和说明
args_schema: Type[BaseModel] = MyToolInput
def get_tool_message(self, **kwargs) -> Optional[str]:
"""根据订阅参数生成友好的提示消息"""
pass
async def run(self, query: str, limit: Optional[int] = None, **kwargs) -> str:
"""
实现工具的核心逻辑(异步方法)
:param query: 查询内容
:param limit: 结果数量限制
:param kwargs: 其他参数,包含 explanation工具使用说明
:return: 工具执行结果,返回字符串格式
"""
try:
# 获取上下文信息(系统自动注入)
session_id = self._session_id
user_id = self._user_id
channel = self._channel
source = self._source
username = self._username
# 执行工具逻辑
result = await self._perform_operation(query, limit)
# 可以通过 send_tool_message 发送消息给用户
await self.send_tool_message(f"操作完成: {result}", title="工具执行")
# 返回执行结果
return f"成功处理查询 '{query}',返回 {len(result)} 条结果"
except Exception as e:
return f"执行失败: {str(e)}"
async def _perform_operation(self, query: str, limit: int):
"""内部方法,执行具体操作"""
# 实现具体业务逻辑
pass
```
- 3. 工具类可用的上下文属性和方法:
- `self._session_id`: 当前会话ID
- `self._user_id`: 用户ID
- `self._channel`: 消息渠道(如 Telegram、Slack 等)
- `self._source`: 消息来源
- `self._username`: 用户名
- `self.send_tool_message(message: str, title: str = "")`: 发送消息给用户
- `ToolChain()`: 访问处理链功能,可调用系统其他功能
- 4. 工具类实现要求:
- **必须继承自 `app.agent.tools.base.MoviePilotTool`**
- **必须实现 `run` 方法**(异步方法),接收参数并返回字符串结果
- **必须实现 `get_tool_message` 方法**,以显示友好的工具执行提示给用户
- **必须定义 `name` 属性**(字符串),工具的唯一标识
- **必须定义 `description` 属性**(字符串),详细描述工具功能,帮助智能体理解何时使用该工具
- **可选定义 `args_schema` 属性**Pydantic模型类用于定义输入参数的结构和验证
- 5. 注意事项:
- 工具的描述(`description`)应该清晰明确,帮助智能体理解工具的功能和使用场景
- 工具的参数模型(`args_schema`)应该包含详细的字段描述,帮助智能体正确构造参数
- 工具执行结果应该返回有意义的字符串,便于智能体理解和向用户展示
- 工具可以通过 `send_tool_message` 方法向用户发送实时消息,提升交互体验
- 工具类在初始化时会自动注入会话和用户信息,可以通过私有属性访问
- 如果工具需要访问插件实例,需要自行通过 `PluginManager` 获取
- 工具执行时间应该尽量短,避免阻塞智能体的响应
- 建议在工具执行过程中添加适当的错误处理和日志记录
## 版本发布

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

After

Width:  |  Height:  |  Size: 35 KiB

View File

@@ -174,11 +174,12 @@
"name": "媒体文件同步删除",
"description": "同步删除历史记录、源文件和下载任务。",
"labels": "文件整理",
"version": "1.7.1",
"version": "1.7.2",
"icon": "mediasyncdel.png",
"author": "thsrite",
"level": 1,
"history": {
"v1.7.2": "兼容windows路径",
"v1.7.1": "修复删除剧集辅种失败报错问题",
"v1.7": "修复重新整理被一并删除问题",
"v1.6": "修复删除辅种",
@@ -217,12 +218,13 @@
"name": "Cloudflare IP优选",
"description": "🌩 测试 Cloudflare CDN 延迟和速度自动优选IP。",
"labels": "网络,站点",
"version": "1.4",
"version": "1.5",
"icon": "cloudflare.jpg",
"author": "thsrite",
"level": 1,
"v2": true,
"history": {
"v1.5": "适配CloudflareSpeedTest新版名称",
"v1.4": "修复立即运行一次",
"v1.3": "调整插件开启状态判断条件",
"v1.2": "增强API安全性"
@@ -319,11 +321,12 @@
"name": "IYUU自动辅种",
"description": "基于IYUU官方Api实现自动辅种。",
"labels": "做种,IYUU",
"version": "1.9.11",
"version": "1.9.12",
"icon": "IYUU.png",
"author": "jxxghp",
"level": 2,
"history": {
"v1.9.12": "修复海豹不能辅种的问题",
"v1.9.11": "修复馒头不能辅种的问题",
"v1.9.10": "Revert 辅种结束后,一起开始所有辅种后暂停的种子(排除了出错的种子)",
"v1.9.9": "修复qb辅种结束后自动开始暂停的种子",
@@ -463,12 +466,16 @@
"name": "药丸签到",
"description": "药丸论坛签到。",
"labels": "站点",
"version": "1.4.1",
"version": "2.0.2",
"icon": "invites.png",
"author": "thsrite",
"level": 2,
"v2": true,
"release": true,
"history": {
"v2.0.2": "增加签到检测机制防止重复签到,增强代码健壮性。",
"v2.0.1": "尝试修复签到失败问题新增使用代理、Cookie自动更新功能",
"v2.0.0": "修复签到失败问题新增账户登录签到功能、新增签到失败重试机制美化界面UI",
"v1.4.1": "更新签到域名前缀",
"v1.4": "自定义保留消息天数"
}
@@ -477,11 +484,12 @@
"name": "演职人员刮削",
"description": "刮削演职人员图片以及中文名称。",
"labels": "媒体库,刮削",
"version": "1.4",
"version": "1.4.1",
"icon": "actor.png",
"author": "jxxghp",
"level": 1,
"history": {
"v1.4.1": "修复异常报错问题",
"v1.4": "人物图片调整为优先从TMDB获取避免douban图片CDN加载过慢的问题",
"v1.3": "修复v1.8.5版本后刮削报错问题"
}
@@ -560,12 +568,13 @@
"name": "TMDB剧集组刮削",
"description": "从TMDB剧集组刮削季集的实际顺序。",
"labels": "刮削",
"version": "2.6",
"version": "2.6.1",
"icon": "Element_A.png",
"author": "叮叮当",
"level": 1,
"v2": true,
"history": {
"v2.6.1": "修复异常报错日志",
"v2.6": "修复无法获取媒体库中季0的问题",
"v2.5": "修复当媒体服务器中剧集的季不完整时会中断的问题",
"v2.3": "修复v2版本无法读取媒体库的问题",
@@ -943,11 +952,14 @@
"name": "钉钉机器人",
"description": "支持使用钉钉机器人发送消息通知。",
"labels": "消息通知,钉钉机器人",
"version": "1.12",
"version": "1.13",
"icon": "Dingding_A.png",
"author": "nnlegenda",
"level": 1,
"v2": true
"v2": true,
"history": {
"v1.13": "优化钉钉消息换行"
}
},
"DynamicWeChat": {
"name": "动态企微可信IP",
@@ -1041,4 +1053,4 @@
"level": 1,
"v2": true
}
}
}

View File

@@ -24,11 +24,12 @@
"name": "站点刷流",
"description": "自动托管刷流,将会提高对应站点的访问频率。",
"labels": "刷流,仪表板",
"version": "4.3.3",
"version": "4.3.4",
"icon": "brush.jpg",
"author": "jxxghp,InfinityPacer",
"author": "jxxghp,InfinityPacer,Seed680",
"level": 2,
"history": {
"v4.3.4": "添加RSS支持配置选项",
"v4.3.2": "增加'删除促销结束的未完成下载'功能",
"v4.3.1": "修复了一些细节问题",
"v4.3": "支持带宽采样并计算平均值,以优化刷流效率",
@@ -61,11 +62,15 @@
"name": "下载任务分类与标签",
"description": "自动给下载任务分类与打站点标签、剧集名称标签",
"labels": "下载管理",
"version": "2.2",
"version": "2.6",
"icon": "Youtube-dl_B.png",
"author": "叮叮当",
"level": 1,
"history": {
"v2.6": "增加站点/剧名前缀功能",
"v2.5": "优化采用公共服务自动清理未使用标签",
"v2.4": "增加自动清理未使用标签",
"v2.3": "增加tracker映射配置",
"v2.2": "MoviePilot V2 版本下载任务分类与标签插件"
}
},
@@ -88,11 +93,13 @@
"name": "媒体库服务器通知",
"description": "发送Emby/Jellyfin/Plex服务器的播放、入库等通知消息。",
"labels": "消息通知,媒体库",
"version": "1.6",
"version": "1.7.1",
"icon": "mediaplay.png",
"author": "jxxghp",
"level": 1,
"history": {
"v1.7.1": "未获取到tmdb信息则按原有逻辑发送电影显示海报",
"v1.7": "对TV剧集入库事件进行聚合避免消息轰炸。更新后如果打不开插件请重置插件",
"v1.6": "查询剧集图片兼容没有季集信息的情况",
"v1.5": "支持独立控制媒体服务器通知",
"v1.4": "MoviePilot V2 版本媒体库服务器通知插件"
@@ -102,11 +109,12 @@
"name": "ChatGPT",
"description": "消息交互支持与ChatGPT对话。",
"labels": "消息通知,识别",
"version": "2.1.7",
"version": "2.1.8",
"icon": "Chatgpt_A.png",
"author": "jxxghp",
"level": 1,
"history": {
"v2.1.8": "修复 OpenAI API >=1.0.0 兼容性问题",
"v2.1.7":"独立安装OpenAi SDK依赖",
"v2.1.6": "支持自定义辅助识别提示词",
"v2.1.5": "兼容一些模型返回json数据信息用markdown语法包裹的情况",
@@ -183,11 +191,13 @@
"name": "演职人员刮削",
"description": "刮削演职人员图片以及中文名称。",
"labels": "媒体库,刮削",
"version": "2.2",
"version": "2.2.2",
"icon": "actor.png",
"author": "jxxghp",
"level": 1,
"history": {
"v2.2.2": "修复异常日志问题",
"v2.2.1": "优化错误数据兼容处理",
"v2.2": "修改使用自定义图片域名时无法下载图片的问题",
"v2.1": "优化执行周期输入需要MoviePilot v2.2.1+",
"v2.0": "兼容MoviePilot V2 版本",
@@ -241,11 +251,12 @@
"name": "IYUU自动辅种",
"description": "基于IYUU官方Api实现自动辅种。",
"labels": "做种,IYUU",
"version": "2.14",
"version": "2.15",
"icon": "IYUU.png",
"author": "jxxghp,CKun",
"level": 2,
"history": {
"v2.15": "修复海豹不能辅种的问题",
"v2.14": "修复馒头不能辅种的问题",
"v2.13": "开启跳过校验后需手动开启自动开始",
"v2.12": "增加qb下载器分类复用配置",
@@ -351,6 +362,18 @@
"v2.0": "适配新的目录结构变化,短剧分类名称调整为配置目录路径,升级后需要重新调整设置后才能使用。"
}
},
"MultiClass": {
"name": "视频多级分类",
"description": "支持视频多级分类",
"labels": "文件整理",
"version": "0.1",
"icon": "Calibreweb_B.png",
"author": "liuhangbin",
"level": 1,
"history": {
"v0.1": "视频多级分类插件, 目前仅支持电影按评分,年代,系列分类。"
}
},
"MoviePilotUpdateNotify": {
"name": "MoviePilot更新推送",
"description": "MoviePilot推送release更新通知、自动重启。",
@@ -420,11 +443,14 @@
"name": "绕过Trackers",
"description": "提供tracker服务器IP地址列表帮助IPv6连接绕过OpenClash。",
"labels": "工具",
"version": "1.4.2",
"version": "1.5.1",
"icon": "Clash_A.png",
"author": "wumode",
"level": 2,
"history": {
"v1.5.1": "新增 Tracker",
"v1.5.0": "新增 Page 界面; 支持通过`/check_ip` 命令检查IP; 改进 UI",
"v1.4.3": "修复 bug",
"v1.4.2": "修复插件动作",
"v1.4.1": "修复通知类型错误",
"v1.4": "异步查询DNS",
@@ -438,11 +464,17 @@
"name": "IMDb源",
"description": "让探索推荐和媒体识别支持IMDb数据源。",
"labels": "探索",
"version": "1.5.6",
"version": "1.6.4",
"icon": "IMDb_IOS-OSX_App.png",
"author": "wumode",
"level": 1,
"history": {
"v1.6.4": "为元数据增加背景图",
"v1.6.3": "优化媒体识别速度; 适配 Pydantic V2 (主程序版本需高于 2.8.1-1)",
"v1.6.2": "修复 API 查询错误重试问题",
"v1.6.1": "添加中文主屏幕组件; 修复 bug",
"v1.5.8": "修改UA",
"v1.5.7": "改进异常处理",
"v1.5.6": "固定仪表盘组件海报比例; 修复 bug",
"v1.5.5": "修复初始化错误",
"v1.5.4": "改进媒体识别",
@@ -468,12 +500,26 @@
"name": "Clash Rule Provider",
"description": "随时为Clash添加一些额外的规则。",
"labels": "工具",
"version": "1.3.2",
"version": "2.0.10",
"icon": "Mihomo_Meta_A.png",
"author": "wumode",
"level": 1,
"release": true,
"history": {
"v2.0.10": "适配 MoviePilot 2.8.4",
"v2.0.9": "适配 Pydantic V2 (主程序版本需高于 2.8.1-1)",
"v2.0.8": "修复已知问题",
"v2.0.7": "修复子规则比较错误",
"v2.0.6": "修复已知问题; 改进对代理组的配置和验证",
"v2.0.5": "完善了对嵌套逻辑规则和子规则的配置和验证",
"v2.0.4": "修复已知问题; 使用异步调度器; 显示规则更改日期",
"v2.0.3": "修复已知问题",
"v2.0.2": "修复分享链接转换问题",
"v2.0.1": "支持独立的订阅链接配置, 覆写代理组和出站代理; 优化数据结构; 修复分享链接解析问题",
"v1.4.2": "优化移动端 UI; 支持显示节点链接",
"v1.4.1": "修复配置模板保存错误, 请重新配置Clash模板",
"v1.4.0": "优化 UI; 支持连接多个 Clash Dashboards",
"v1.3.3": "通过emoji识别国家; 按国家分组节点; mrs格式支持",
"v1.3.2": "注册插件动作",
"v1.3.1": "支持配置 Hosts",
"v1.2.8": "改进导入界面",
@@ -496,11 +542,17 @@
"name": "美剧生词标注",
"description": "根据CEFR等级为英语影视剧标注高级词汇。",
"labels": "英语",
"version": "1.0.1",
"version": "1.2.0",
"icon": "LexiAnnot.png",
"author": "wumode",
"level": 1,
"history": {
"v1.2.0": "引入大模型候选词决策和词义丰富处理链; 支持读取系统智能体配置; 添加智能体工具; 优化通知样式; 改进 UI",
"v1.1.4": "优化字幕选择决策",
"v1.1.3": "适配 Pydantic V2 (主程序版本需高于 2.8.1-1)",
"v1.1.2": "使用子进程避免 spaCy 模型常驻内存",
"v1.1.1": "添加任务页面; 改进 spaCy 模型加载逻辑",
"v1.1.0": "支持考试词汇标注; 优化分词处理; 修复错误",
"v1.0.1": "合并连字符词; 避免ARM平台依赖问题",
"v1.0": "新增LexiAnnot"
}
@@ -517,5 +569,19 @@
"v1.0.0": "首个版本新增MeoW消息通知",
"v1.0.1": "优化代码,修复运行一次按钮没办法自动关闭的问题"
}
},
"BugReporter": {
"name": "Bug反馈",
"description": "自动上报异常,协助开发者发现和解决问题。",
"labels": "开发",
"version": "1.3",
"icon": "Alist_encrypt_A.png",
"author": "jxxghp",
"level": 1,
"history": {
"v1.3": "减少网络异常信息上送",
"v1.2": "优化上报信息量",
"v1.1": "加强脱敏处理"
}
}
}

View File

@@ -79,6 +79,7 @@ class BrushConfig:
self.qb_category = config.get("qb_category")
self.site_hr_active = config.get("site_hr_active", False)
self.site_skip_tips = config.get("site_skip_tips", False)
self.rss_support = config.get("rss_support", False)
self.brush_tag = "刷流"
# 站点独立配置
@@ -123,7 +124,8 @@ class BrushConfig:
"qb_category",
"site_hr_active",
"site_skip_tips",
"del_no_free"
"del_no_free",
"rss_support"
# 当新增支持字段时,仅在此处添加字段名
}
try:
@@ -193,7 +195,9 @@ class BrushConfig:
"del_no_free": false,
"qb_category": "刷流",
"site_hr_active": true,
"site_skip_tips": true
"site_skip_tips": true,
"rss_support": true
"
}]"""
return desc + config
@@ -259,9 +263,9 @@ class BrushFlow(_PluginBase):
# 插件图标
plugin_icon = "brush.jpg"
# 插件版本
plugin_version = "4.3.3"
plugin_version = "4.3.4"
# 插件作者
plugin_author = "jxxghp,InfinityPacer"
plugin_author = "jxxghp,InfinityPacer,Seed680"
# 作者主页
author_url = "https://github.com/InfinityPacer"
# 插件配置项ID前缀
@@ -1638,6 +1642,22 @@ class BrushFlow(_PluginBase):
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 4
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'rss_support',
'label': '启用RSS支持',
}
}
]
}
]
}
@@ -1817,7 +1837,8 @@ class BrushFlow(_PluginBase):
"freeleech": "free",
"hr": "yes",
"enable_site_config": False,
"site_config": BrushConfig.get_demo_site_config()
"site_config": BrushConfig.get_demo_site_config(),
"rss_support": False
}
def get_page(self) -> List[dict]:
@@ -2002,7 +2023,14 @@ class BrushFlow(_PluginBase):
return True
logger.info(f"开始获取站点 {siteinfo.name} 的新种子 ...")
torrents = TorrentsChain().browse(domain=siteinfo.domain)
# 根据rss_support配置决定使用browse还是rss方法获取种子
brush_config = self.__get_brush_config(sitename=siteinfo.name)
if brush_config.rss_support:
torrents = TorrentsChain().rss(domain=siteinfo.domain)
else:
torrents = TorrentsChain().browse(domain=siteinfo.domain)
if not torrents:
logger.info(f"站点 {siteinfo.name} 没有获取到种子")
return True
@@ -3048,6 +3076,7 @@ class BrushFlow(_PluginBase):
"enable_site_config": brush_config.enable_site_config,
"site_config": brush_config.site_config,
"del_no_free": brush_config.del_no_free,
"rss_support": brush_config.rss_support,
"_tabs": self._tabs
}

View File

@@ -0,0 +1,263 @@
import re
from typing import Any, Dict
from typing import List, Tuple
from urllib.parse import urlparse, parse_qs, urlencode, urlunparse
import sentry_sdk
from app.plugins import _PluginBase
from version import APP_VERSION
class SentrySanitizer:
# 常见敏感字段名(可自行扩展)
SENSITIVE_KEYS = {
"password", "passwd", "pwd",
"secret", "token", "access_token", "refresh_token",
"authorization", "api_key", "apikey",
"cookie", "set-cookie", "passkey",
"key", "credential", "auth", "login", "user", "username",
"email", "phone", "address", "ip", "host", "domain"
}
# 匹配包含敏感关键词的正则
SENSITIVE_PATTERN = re.compile(
"|".join(re.escape(key) for key in SENSITIVE_KEYS), re.IGNORECASE
)
# 网络连接错误类异常(不上报)
NETWORK_ERRORS = {
"ConnectionError", "ConnectionRefusedError", "ConnectionAbortedError",
"ConnectionResetError", "TimeoutError", "socket.timeout", "socket.error",
"ssl.SSLError", "ssl.SSLCertVerificationError", "ssl.SSLWantReadError",
"ssl.SSLWantWriteError", "ssl.SSLZeroReturnError", "ssl.SSLSyscallError",
"urllib.error.URLError", "urllib.error.HTTPError", "requests.exceptions.ConnectionError",
"requests.exceptions.Timeout", "requests.exceptions.ConnectTimeout",
"requests.exceptions.ReadTimeout", "requests.exceptions.SSLError",
"aiohttp.ClientConnectionError", "aiohttp.ClientTimeout", "aiohttp.ServerTimeoutError",
"aiohttp.ServerDisconnectedError", "aiohttp.ClientOSError"
}
# 网络连接错误关键词
NETWORK_ERROR_KEYWORDS = [
"connection", "timeout", "network", "dns", "ssl", "certificate",
"refused", "reset", "aborted", "unreachable", "no route to host",
"name or service not known", "temporary failure", "network is unreachable",
"SOCKSHTTPSConnectionPool", "ERR_HTTP_RESPONSE_CODE_FAILURE", "HTTPSConnectionPool",
"网络连接", "无法连接", "请求失败", "下载失败", "请求返回空值", "图片失败", "未获取到返回数据",
"请求返回空值", "返回空响应", "连接出错", "请求错误", "未获取到"
]
@classmethod
def scrub_dict(cls, data: Dict[str, Any]) -> Dict[str, Any]:
"""
递归清洗字典中的敏感信息
"""
if not isinstance(data, dict):
return data
sanitized = {}
for key, value in data.items():
if isinstance(value, dict):
sanitized[key] = cls.scrub_dict(value)
elif isinstance(value, list):
sanitized[key] = [cls.scrub_dict(v) if isinstance(v, dict) else v for v in value]
else:
if cls.SENSITIVE_PATTERN.search(str(key)):
sanitized[key] = "[Filtered]"
else:
sanitized[key] = value
return sanitized
@classmethod
def scrub_url(cls, url: str) -> str:
"""
清理 URL 中的敏感 query 参数
"""
try:
parsed = urlparse(url)
query = parse_qs(parsed.query, keep_blank_values=True)
for key in query:
if cls.SENSITIVE_PATTERN.search(key):
query[key] = ["[Filtered]"]
new_query = urlencode(query, doseq=True)
return urlunparse(parsed._replace(query=new_query))
except Exception as err:
print(str(err))
return url
@classmethod
def is_network_error(cls, event) -> bool:
"""
判断是否为网络连接错误类异常
"""
# 检查异常类型
if "exception" in event:
for exc in event["exception"].get("values", []):
if "type" in exc:
exc_type = exc["type"]
if exc_type in cls.NETWORK_ERRORS:
return True
# 检查异常消息是否包含网络错误关键词
if "value" in exc:
exc_value = exc["value"].lower()
for keyword in cls.NETWORK_ERROR_KEYWORDS:
if keyword in exc_value:
return True
# 检查日志消息
if "message" in event:
message = event["message"].lower()
for keyword in cls.NETWORK_ERROR_KEYWORDS:
if keyword in message:
return True
return False
@classmethod
def before_send(cls, event, hint):
"""
在发送到 Sentry 之前脱敏和过滤
"""
# 如果是网络连接错误,直接返回 None 不上报
if cls.is_network_error(event):
return None
# 处理 request 数据
request = event.get("request", {})
if "url" in request:
request["url"] = cls.scrub_url(request["url"])
if "headers" in request:
request["headers"] = cls.scrub_dict(request["headers"])
if "data" in request:
request["data"] = cls.scrub_dict(request["data"])
if "cookies" in request:
request["cookies"] = cls.scrub_dict(request["cookies"])
# 处理 user 数据
if "user" in event:
event["user"] = cls.scrub_dict(event["user"])
# 处理 extra 数据
if "extra" in event:
event["extra"] = cls.scrub_dict(event["extra"])
# 处理异常信息(避免敏感数据出现在 message 中)
if "exception" in event:
for exc in event["exception"].get("values", []):
if "value" in exc and cls.SENSITIVE_PATTERN.search(exc["value"]):
exc["value"] = "[Filtered Exception Message]"
# 清理异常堆栈中的敏感信息
if "stacktrace" in exc and "frames" in exc["stacktrace"]:
for frame in exc["stacktrace"]["frames"]:
if "vars" in frame:
frame["vars"] = cls.scrub_dict(frame["vars"])
if "context_line" in frame and cls.SENSITIVE_PATTERN.search(frame["context_line"]):
frame["context_line"] = "[Filtered]"
# 清理消息中的敏感信息
if "message" in event and cls.SENSITIVE_PATTERN.search(event["message"]):
event["message"] = "[Filtered Message]"
return event
class BugReporter(_PluginBase):
# 插件名称
plugin_name = "Bug反馈"
# 插件描述
plugin_desc = "自动上报异常,协助开发者发现和解决问题。"
# 插件图标
plugin_icon = "Alist_encrypt_A.png"
# 插件版本
plugin_version = "1.3"
# 插件作者
plugin_author = "jxxghp"
# 作者主页
author_url = "https://github.com/jxxghp"
# 插件配置项ID前缀
plugin_config_prefix = "bugreporter_"
# 加载顺序
plugin_order = 99
# 可使用的用户级别
auth_level = 1
_enable: bool = False
def init_plugin(self, config: dict = None):
self._enable = config.get("enable")
if self._enable:
sentry_sdk.init("https://88da01ad33b4423cb0380620de53efa8@glitchtip.movie-pilot.org/1",
before_send=SentrySanitizer.before_send,
release=APP_VERSION,
send_default_pii=False)
@staticmethod
def get_command() -> List[Dict[str, Any]]:
pass
def get_api(self) -> List[Dict[str, Any]]:
pass
def get_form(self) -> Tuple[List[dict], Dict[str, Any]]:
return [
{
'component': 'VForm',
'content': [
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'enable',
'label': '启用插件',
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'warning',
'variant': 'tonal',
'text': '注意开启插件即代表你同意将部分异常信息自动发送给开发者以帮助改进软件如果你不希望自动发送任何数据请关闭或卸载此插件仅上报系统异常信息不会包含任何个人隐私信息或敏感数据网络连接错误类异常不会上报异常信息采集为使用开源项目解决方案GlitchTip。',
}
}
]
}
]
}
]
}
], {
"enable": self._enable,
}
def get_page(self) -> List[dict]:
pass
def get_state(self) -> bool:
return self._enable
def stop_service(self):
pass

View File

@@ -0,0 +1 @@
sentry_sdk~=2.44.0

View File

@@ -17,7 +17,7 @@ class ChatGPT(_PluginBase):
# 插件图标
plugin_icon = "Chatgpt_A.png"
# 插件版本
plugin_version = "2.1.7"
plugin_version = "2.1.8"
# 插件作者
plugin_author = "jxxghp"
# 作者主页

View File

@@ -14,22 +14,33 @@ class OpenAi:
_api_url: str = None
_model: str = "gpt-3.5-turbo"
_prompt: str = '接下来我会给你一个电影或电视剧的文件名你需要识别文件名中的名称、版本、分段、年份、分瓣率、季集等信息并按以下JSON格式返回{"name":string,"version":string,"part":string,"year":string,"resolution":string,"season":number|null,"episode":number|null}特别注意返回结果需要严格附合JSON格式不需要有任何其它的字符。如果中文电影或电视剧的文件名中存在谐音字或字母替代的情况请还原最有可能的结果。'
_client: openai.OpenAI = None
def __init__(self, api_key: str = None, api_url: str = None, proxy: dict = None, model: str = None, compatible:
bool = False, customize_prompt: str = None):
def __init__(self, api_key: str = None, api_url: str = None,
proxy: dict = None, model: str = None,
compatible: bool = False, customize_prompt: str = None):
self._api_key = api_key
self._api_url = api_url
if compatible:
openai.api_base = self._api_url
else:
openai.api_base = self._api_url + "/v1"
openai.api_key = self._api_key
if proxy and proxy.get("https"):
openai.proxy = proxy.get("https")
if model:
self._model = model
if customize_prompt:
self._prompt = customize_prompt
# 初始化 OpenAI 客户端
if self._api_key and self._api_url:
base_url = self._api_url if compatible else self._api_url + "/v1"
http_client = None
if proxy and proxy.get("https"):
import httpx
proxy_url = proxy.get("https")
# httpx 支持字符串格式的代理 URL
http_client = httpx.Client(proxies=proxy_url, timeout=60.0)
self._client = openai.OpenAI(
api_key=self._api_key,
base_url=base_url,
http_client=http_client
)
def get_state(self) -> bool:
return True if self._api_key else False
@@ -82,6 +93,8 @@ class OpenAi:
"""
获取模型
"""
if not self._client:
raise ValueError("OpenAI client not initialized. Please check API key and API URL.")
if not isinstance(message, list):
if prompt:
message = [
@@ -101,9 +114,10 @@ class OpenAi:
"content": message
}
]
return openai.ChatCompletion.create(
# 新版本 API 不支持 user 参数,需要从 kwargs 中移除
kwargs.pop('user', None)
return self._client.chat.completions.create(
model=self._model,
user=user,
messages=message,
**kwargs
)
@@ -170,11 +184,11 @@ class OpenAi:
if result:
self.__save_session(userid, text)
return result
except openai.error.RateLimitError as e:
except openai.RateLimitError as e:
return f"请求被ChatGPT拒绝了{str(e)}"
except openai.error.APIConnectionError as e:
except openai.APIConnectionError as e:
return f"ChatGPT网络连接失败{str(e)}"
except openai.error.Timeout as e:
except openai.APITimeoutError as e:
return f"没有接收到ChatGPT的返回消息{str(e)}"
except Exception as e:
return f"请求ChatGPT出现错误{str(e)}"

View File

@@ -1 +0,0 @@
openai~=0.27.2

View File

@@ -4,7 +4,8 @@
- 即时通知 Clash 刷新规则集合
- 基于 Meta 内核丰富的代理组配置,提供灵活的路由功能
- 支持按大洲分组节点
- 支持按大洲和国家分组节点
- 支持覆写出站代理
- GEO 规则输入提示
- 支持 [ACL4SSR](https://github.com/ACL4SSR/ACL4SSR) 规则集合

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,264 @@
import asyncio
import json
import secrets
from typing import Any, Dict, List, Callable, Optional, Literal
import websockets
import yaml
from fastapi import HTTPException, Request, status, Response
from fastapi.responses import PlainTextResponse
from sse_starlette.sse import EventSourceResponse
from app import schemas
from app.core.config import settings
from app.log import logger
from .config import PluginConfig
from .models import ProxyGroup
from .models.api import RuleData, Connectivity, Subscription, RuleProviderData, SubscriptionInfo, HostData
from .services import ClashRuleProviderService
class ApiCollection:
def __init__(self):
self.route_definitions = []
def register(self, path: str,
methods: List[Literal['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS', 'HEAD', 'TRACE']],
allow_anonymous: Optional[bool] = None,
auth: Optional[str] = None,
summary: Optional[str] = ''):
def decorator(func: Callable):
route_meta: Dict[str, Any] = {
'path': path,
'methods': methods,
'summary': summary,
'endpoint': func
}
if allow_anonymous is not None:
route_meta['allow_anonymous'] = allow_anonymous
if auth is not None:
route_meta['auth'] = auth
self.route_definitions.append(route_meta)
return func
return decorator
def get_routes(self, instance: Any) -> List[Dict[str, Any]]:
bound_routes = []
for route in self.route_definitions:
func_name = route['endpoint'].__name__
bound_method = getattr(instance, func_name)
bound_routes.append({**route, 'endpoint': bound_method})
return bound_routes
apis = ApiCollection()
class ClashRuleProviderApi:
def __init__(self, services: ClashRuleProviderService, config: PluginConfig):
self.services: ClashRuleProviderService = services
self.config = config
@apis.register(path='/connectivity', methods=['POST'], auth='bear', summary='测试连接')
async def test_connectivity(self, item: Connectivity) -> schemas.Response:
success, message = await self.services.test_connectivity(item.clash_apis, item.sub_links)
return schemas.Response(success=success, message=message)
@apis.register(path="/clash-outbound", methods=["GET"], auth="bear", summary="获取所有出站")
def get_clash_outbound(self) -> schemas.Response:
outbound = self.services.clash_outbound()
return schemas.Response(success=True, data={"outbound": outbound})
@apis.register(path="/status", methods=["GET"], auth="bear", summary="插件状态")
def get_status(self) -> schemas.Response:
data = self.services.get_status()
return schemas.Response(success=True, data=data)
@apis.register(path="/rules/{ruleset}", methods=["GET"], auth="bear", summary="获取指定集合中的规则")
def get_rules(self, ruleset: Literal['ruleset', 'top']) -> schemas.Response:
data = self.services.get_rules(ruleset)
return schemas.Response(success=True, data={'rules': data})
@apis.register(path="/reorder-rules/{ruleset}/{target_priority}", methods=["PUT"], auth="bear",
summary="重新排序规则")
def reorder_rules(self, ruleset: Literal['ruleset', 'top'], target_priority: int,
rule_data: RuleData) -> schemas.Response:
moved_priority = rule_data.priority
success, message = self.services.reorder_rules(ruleset, moved_priority, target_priority)
return schemas.Response(success=success, message=message)
@apis.register(path="/rules/{ruleset}/{priority}", methods=["PATCH"], auth="bear", summary="更新规则")
def update_rule(self, ruleset: Literal['ruleset', 'top'], priority: int, rule_data: RuleData) -> schemas.Response:
success, message = self.services.update_rule(ruleset, priority, rule_data)
return schemas.Response(success=success, message=message)
@apis.register(path="/rules/{ruleset}", methods=["POST"], auth="bear", summary="添加规则")
def add_rule(self, ruleset: Literal['ruleset', 'top'], rule_data: RuleData) -> schemas.Response:
success, message = self.services.add_rule(ruleset, rule_data)
return schemas.Response(success=success, message=message)
@apis.register(path="/rules/{ruleset}/{priority}", methods=["DELETE"], auth="bear", summary="删除规则")
def delete_rule(self, ruleset: Literal['ruleset', 'top'], priority: int) -> schemas.Response:
self.services.delete_rule(ruleset, priority)
return schemas.Response(success=True)
@apis.register(path="/refresh", methods=["PUT"], auth="bear", summary="更新订阅")
async def refresh_subscription(self, subscription: Subscription) -> schemas.Response:
success, message = await self.services.refresh_subscription(subscription.url)
return schemas.Response(success=success, message=message)
@apis.register(path="/rule-providers", methods=["GET"], auth="bear", summary="获取规则集合")
def get_rule_providers(self) -> schemas.Response:
return schemas.Response(success=True, data=self.services.rule_providers())
@apis.register(path="/rule-providers/{name}", methods=["POST"], auth="bear", summary="更新规则集合")
def update_rule_provider(self, name: str, item: RuleProviderData):
success, message = self.services.update_rule_provider(name, item)
return schemas.Response(success=success, message=message)
@apis.register(path="/rule-providers/{name}", methods=["DELETE"], auth="bear", summary="删除规则集合")
def delete_rule_provider(self, name: str):
self.services.delete_rule_provider(name)
return schemas.Response(success=True)
@apis.register(path="/proxies", methods=["GET"], auth="bear", summary="获取出站代理")
def get_proxies(self):
proxies = self.services.get_all_proxies_with_details()
return schemas.Response(success=True, data={'proxies': proxies})
@apis.register(path="/proxies/{name}", methods=["DELETE"], auth="bear", summary="删除出站代理")
def delete_proxy(self, name: str):
self.services.delete_proxy(name)
return schemas.Response(success=True)
@apis.register(path="/proxies", methods=["PUT"], auth="bear", summary="添加出站代理")
def import_proxies(self, params: Dict[str, Any]):
success, message = self.services.import_proxies(params)
return schemas.Response(success=success, message=message)
@apis.register(path="/proxies/{name}", methods=["PATCH"], auth="bear", summary="更新出站代理")
def update_proxy(self, name: str, param: Dict[str, Any]) -> schemas.Response:
success, message = self.services.update_proxy(name, param)
return schemas.Response(success=success, message=message)
@apis.register(path="/proxy-groups", methods=["GET"], auth="bear", summary="获取代理组")
def get_proxy_groups(self):
proxy_groups = self.services.get_all_proxy_groups_with_source()
return schemas.Response(success=True, data={'proxy_groups': proxy_groups})
@apis.register(path="/proxy-groups/{name}", methods=["DELETE"], auth="bear", summary="删除代理组")
def delete_proxy_group(self, name: str):
success, message = self.services.delete_proxy_group(name)
return schemas.Response(success=success, message=message)
@apis.register(path="/proxy-groups", methods=["POST"], auth="bear", summary="添加代理组")
def add_proxy_group(self, item: ProxyGroup):
success, message = self.services.add_proxy_group(item)
return schemas.Response(success=success, message=message)
@apis.register(path="/proxy-groups/{previous_name}", methods=["PATCH"], auth="bear", summary="更新代理组")
def update_proxy_group(self, previous_name: str, item: ProxyGroup):
success, message = self.services.update_proxy_group(previous_name, item)
return schemas.Response(success=success, message=message)
@apis.register(path="/proxy-providers", methods=["GET"], auth="bear", summary="获取代理集合")
def get_proxy_providers(self):
proxy_providers = self.services.all_proxy_providers()
return schemas.Response(success=True, data={'proxy_providers': proxy_providers})
@apis.register(path="/ruleset", methods=["GET"], allow_anonymous=True, summary="获取规则集规则")
def get_ruleset(self, name: str, apikey: str) -> PlainTextResponse:
_apikey = self.config.apikey or settings.API_TOKEN
if not secrets.compare_digest(_apikey, apikey):
raise HTTPException(status_code=403, detail="Invalid API Key")
res = self.services.get_ruleset(name)
if not res:
raise HTTPException(status_code=404, detail=f"Ruleset {name!r} not found")
return PlainTextResponse(content=res, media_type="application/x-yaml")
@apis.register(path="/import", methods=["POST"], auth="bear", summary="导入规则")
def import_rules(self, params: Dict[str, Any]):
self.services.import_rules(params)
return schemas.Response(success=True)
@apis.register(path="/hosts", methods=["GET"], auth="bear", summary="获取 Hosts")
def get_hosts(self):
return schemas.Response(success=True, data={'hosts': self.services.get_hosts()})
@apis.register(path="/hosts", methods=["POST"], auth="bear", summary="更新 Hosts")
def update_hosts(self, host: HostData):
success, message = self.services.update_hosts(host)
return schemas.Response(success=success, message=message)
@apis.register(path="/hosts", methods=["DELETE"], auth="bear", summary="删除 Hosts")
def delete_host(self, host: HostData):
success, message = self.services.delete_host(host)
return schemas.Response(success=success, message=message)
@apis.register(path="/subscription-info", methods=["POST"], auth="bear", summary="更新订阅信息")
def update_subscription_info(self, sub_info: SubscriptionInfo):
self.services.update_subscription_info(sub_info)
return schemas.Response(success=True)
@apis.register(path="/config", methods=["GET"], allow_anonymous=bool(True), summary="获取 Clash 配置")
def get_clash_config(self, apikey: str, request: Request):
_apikey = self.config.apikey or settings.API_TOKEN
if not secrets.compare_digest(apikey, _apikey):
raise HTTPException(status_code=403, detail="Invalid API Key")
logger.info(f"{request.client.host} 正在获取配置")
config = self.services.clash_config()
if not config:
raise HTTPException(status_code=500, detail="配置不可用")
res = yaml.dump(config, allow_unicode=True, sort_keys=False)
sub_info = self.services.get_subscription_user_info()
headers = {'Subscription-Userinfo': f'upload={sub_info["upload"]}; download={sub_info["download"]}; '
f'total={sub_info["total"]}; expire={sub_info["expire"]}'}
return Response(headers=headers, content=res, media_type="text/yaml")
@apis.register(path="/clash/proxy/{path:path}", methods=["GET"], auth="bear", summary="转发 Clash API 请求")
async def clash_proxy(self, path: str):
return await self.services.fetch_clash_data(path)
@apis.register(path="/clash/ws/{endpoint}", methods=["GET"], allow_anonymous=True,
summary="转发 Clash API Websocket 请求")
async def clash_websocket(self, request: Request, endpoint: str, secret: str):
if not secrets.compare_digest(secret, self.config.dashboard_secret):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Secret 校验不通过")
if endpoint not in ['traffic', 'connections', 'memory']:
raise HTTPException(status_code=400, detail="Invalid endpoint")
# This logic is highly coupled with the web framework, so it stays here.
queue = asyncio.Queue()
ws_base = self.config.dashboard_url.replace(
'http://', 'ws://').replace('https://', 'wss://')
url = f"{ws_base}/{endpoint}?token={self.config.dashboard_secret}"
async def clash_ws_listener():
try:
async with websockets.connect(url, ping_interval=None) as ws:
async for message in ws:
await queue.put(json.loads(message))
except Exception as e:
await queue.put({"error": str(e)})
listener_task = asyncio.create_task(clash_ws_listener())
async def event_generator():
try:
while True:
if await request.is_disconnected():
break
try:
data = await queue.get()
yield {'event': endpoint, 'data': json.dumps(data)}
except asyncio.CancelledError:
break
finally:
listener_task.cancel()
return EventSourceResponse(event_generator())

View File

@@ -0,0 +1,34 @@
from abc import ABC
from typing import Final, Literal, Dict
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from app.plugins import _PluginBase
from .config import PluginConfig
from .state import PluginState
from .store import PluginStore
class _ClashRuleProviderBase(_PluginBase, ABC):
# Constants
DEFAULT_CLASH_CONF: Final[
Dict[Literal['rules', 'rule-providers', 'proxies', 'proxy-groups', 'proxy-providers'], dict | list]] = {
'rules': [], 'rule-providers': {},
'proxies': [], 'proxy-groups': [], 'proxy-providers': {}
}
OVERWRITTEN_PROXIES_LIFETIME: Final[int] = 10
ACL4SSR_API: Final[str] = "https://api.github.com/repos/ACL4SSR/ACL4SSR"
METACUBEX_RULE_DAT_API: Final[str] = "https://api.github.com/repos/MetaCubeX/meta-rules-dat"
MISFIRE_GRACE_TIME: Final[int] = 120
KEY_TOP_RULES: Final[str] = "top_rules"
KEY_RULESET_RULES: Final[str] = "ruleset_rules"
KEY_PROXIES: Final[str] = "proxies"
KEY_PROXY_GROUPS: Final[str] = "proxy-groups"
KEY_NAME: Final[str] = "name"
# Runtime variables
state: PluginState
config: PluginConfig
store: PluginStore
scheduler: AsyncIOScheduler = None

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,107 @@
from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field, field_validator
from .models.api import ClashApi
class SubscriptionConfig(BaseModel):
url: str
rules: Optional[bool] = True
rule_providers: Optional[bool] = Field(True, alias='rule-providers')
proxies: Optional[bool] = True
proxy_groups: Optional[bool] = Field(True, alias='proxy-groups')
proxy_providers: Optional[bool] = Field(True, alias='proxy-providers')
@field_validator('url')
@classmethod
def validate_url(cls, v: str) -> str:
return v.strip()
class PluginConfig(BaseModel):
"""
A dataclass to hold all the configuration of the ClashRuleProvider plugin.
"""
enabled: bool = False
proxy: bool = False
notify: bool = False
subscriptions_config: List[SubscriptionConfig] = Field(default_factory=list)
movie_pilot_url: str = ''
cron_string: str = '30 12 * * *'
timeout: int = 10
retry_times: int = 3
filter_keywords: List[str] = Field(default_factory=list)
auto_update_subscriptions: bool = True
ruleset_prefix: str = '📂<='
acl4ssr_prefix: str = '🗂️=>'
group_by_region: bool = False
group_by_country: bool = False
refresh_delay: int = 5
enable_acl4ssr: bool = False
dashboard_components: List[str] = Field(default_factory=list)
clash_template: str = ''
hint_geo_dat: bool = False
best_cf_ip: List[str] = Field(default_factory=list)
apikey: Optional[str] = None
clash_dashboards: List[ClashApi] = Field(default_factory=list)
active_dashboard: Optional[int] = None
@field_validator('clash_dashboards')
@classmethod
def validate_clash_dashboards(cls, v: List[ClashApi]):
for item in v:
url = item.url.rstrip('/')
if not (url.startswith('http://') or url.startswith('https://')):
url = 'http://' + url
item.url = url
return v
@field_validator('movie_pilot_url')
@classmethod
def validate_movie_pilot_url(cls, v: str):
return v.rstrip('/')
@field_validator('ruleset_prefix')
@classmethod
def validate_ruleset_prefix(cls, v: str):
return v.strip()
@field_validator('acl4ssr_prefix')
@classmethod
def validate_acl4ssr_prefix(cls, v: str):
return v.strip()
@staticmethod
def upgrade_conf(conf: Dict[str, Any]) -> Dict[str, Any]:
if conf.get('sub_links'):
subscriptions_config = conf.get('subscriptions_config') or []
subscriptions_config.extend(
[{'url': url, 'rules': True, 'rule-providers': True, 'proxies': True, 'proxy-groups': True,
'proxy-providers': True}
for url in conf['sub_links']]
)
conf['subscriptions_config'] = subscriptions_config
if conf.get('clash_dashboard_url') and conf.get('clash_dashboard_secret'):
clash_dashboards = conf.get('clash_dashboards') or []
clash_dashboards.append({'url': conf.get('clash_dashboard_url'), 'secret': conf.get('clash_dashboard_secret')})
conf['clash_dashboards'] = clash_dashboards
return conf
@property
def sub_links(self) -> List[str]:
return [sub.url for sub in self.subscriptions_config]
@property
def dashboard_url(self) -> str:
dashboard_url = ''
if self.active_dashboard is not None and self.active_dashboard in range(len(self.clash_dashboards)):
dashboard_url = self.clash_dashboards[self.active_dashboard].url
return dashboard_url
@property
def dashboard_secret(self) -> str:
dashboard_secret = ''
if self.active_dashboard is not None and self.active_dashboard in range(len(self.clash_dashboards)):
dashboard_secret = self.clash_dashboards[self.active_dashboard].secret
return dashboard_secret

5136
plugins.v2/clashruleprovider/countries.json Executable file → Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,4 +0,0 @@
.plugin-config[data-v-929102b8] {
margin: 0 auto;
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,4 @@
.plugin-config[data-v-5f383f33] {
margin: 0 auto;
}

View File

@@ -1,50 +0,0 @@
.plugin-page[data-v-d6db167c] {
margin: 0 auto;
}
/* 使卡片等宽并适应移动端 */
.d-flex.flex-wrap[data-v-d6db167c] {
gap: 16px;
}
.url-display[data-v-d6db167c] {
word-break: break-all;
padding: 8px;
background: rgba(0, 0, 0, 0.05);
border-radius: 4px;
}
/* 移动端堆叠布局 */
@media (max-width: 768px) {
.d-flex.flex-wrap[data-v-d6db167c] {
flex-direction: column;
}
}
/* Add visual distinction between sections */
.ruleset-section[data-v-d6db167c] {
border: 1px solid #e0e0e0;
border-radius: 4px;
padding: 16px;
background-color: #f5f5f5;
}
.top-section[data-v-d6db167c] {
border: 1px solid #e0e0e0;
border-radius: 4px;
padding: 16px;
background-color: #f9f9f9;
}
/* Optional: Add different border colors to further distinguish */
.ruleset-section[data-v-d6db167c] {
border-left: 4px solid #2196F3; /* Blue accent */
}
.top-section[data-v-d6db167c] {
border-left: 4px solid #4CAF50; /* Green accent */
}
.drag-handle[data-v-d6db167c] {
cursor: move;
}
.gap-2[data-v-d6db167c] {
gap: 8px;
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,70 @@
.plugin-page[data-v-67d1defe] {
margin: 0 auto;
}
/* 使卡片等宽并适应移动端 */
.d-flex.flex-wrap[data-v-67d1defe] {
gap: 16px;
}
/* 移动端堆叠布局 */
@media (max-width: 768px) {
.d-flex.flex-wrap[data-v-67d1defe] {
flex-direction: column;
}
}
.drag-handle[data-v-67d1defe] {
cursor: move;
}
.toggle-container[data-v-67d1defe] {
display: flex;
align-items: center;
justify-content: space-between;
padding: 0.625rem;
margin-left: 0.75rem;
margin-right: 0.75rem;
}
.subscription-card[data-v-67d1defe] {
border-radius: 12px;
overflow: hidden;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08);
transition: transform 0.3s, box-shadow 0.3s;
background: white;
}
.subscription-card[data-v-67d1defe]:hover {
transform: translateY(-5px);
box-shadow: 0 6px 16px rgba(0, 0, 0, 0.12);
}
.card-title[data-v-67d1defe] {
color: whitesmoke;
}
.card-header[data-v-67d1defe] {
padding: 0.625rem;
background: linear-gradient(135deg, rgba(var(--v-theme-primary), 1) 0%, rgba(var(--v-theme-primary), 0.7) 100%);
color: white;
display: flex;
justify-content: space-between;
align-items: center;
}
.card-refresh-button[data-v-67d1defe] {
background-color: rgba(var(--v-theme-primary), 0.9);
color: whitesmoke;
border: none;
border-radius: 6px;
padding: 0.625rem;
font-weight: 500;
cursor: pointer;
display: flex;
align-items: center;
transition: background-color 0.3s;
}
.search-field[data-v-67d1defe] {
max-width: 25rem;
}
.clash-data-table[data-v-67d1defe] {
max-height: 40rem;
overflow-y: auto;
}

View File

@@ -2,11 +2,11 @@ const currentImports = {};
const exportSet = new Set(['Module', '__esModule', 'default', '_export_sfc']);
let moduleMap = {
"./Page":()=>{
dynamicLoadingCss(["__federation_expose_Page-BOym_1fV.css"], false, './Page');
return __federation_import('./__federation_expose_Page-D5l2MyNA.js').then(module =>Object.keys(module).every(item => exportSet.has(item)) ? () => module.default : () => module)},
dynamicLoadingCss(["__federation_expose_Page-Dx-0nC8K.css"], false, './Page');
return __federation_import('./__federation_expose_Page-CUYOswsP.js').then(module =>Object.keys(module).every(item => exportSet.has(item)) ? () => module.default : () => module)},
"./Config":()=>{
dynamicLoadingCss(["__federation_expose_Config-BrXQaadr.css"], false, './Config');
return __federation_import('./__federation_expose_Config-NH09p1Am.js').then(module =>Object.keys(module).every(item => exportSet.has(item)) ? () => module.default : () => module)},
dynamicLoadingCss(["__federation_expose_Config-D7x82s8Y.css"], false, './Config');
return __federation_import('./__federation_expose_Config-C8YPPEsk.js').then(module =>Object.keys(module).every(item => exportSet.has(item)) ? () => module.default : () => module)},
"./Dashboard":()=>{
dynamicLoadingCss(["__federation_expose_Dashboard-vS9Qm2ZB.css"], false, './Dashboard');
return __federation_import('./__federation_expose_Dashboard-BDSt5WaH.js').then(module =>Object.keys(module).every(item => exportSet.has(item)) ? () => module.default : () => module)},};

View File

@@ -0,0 +1,132 @@
import time
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional, Union, Iterator
from .clashruleparser import ClashRuleParser
from ..models.rule import Action, RoutingRuleType, MatchRule, ClashRule, LogicRule, SubRule
@dataclass
class RuleItem:
"""Clash rule item"""
rule: Union[ClashRule, LogicRule, MatchRule, SubRule]
remark: str = field(default="")
time_modified: float = field(default=0)
class ClashRuleManager:
"""Clash rule manager"""
def __init__(self):
self.rules: List[RuleItem] = []
def import_rules(self, rules_list: List[Dict[str, Any]]):
self.rules = []
for r in rules_list:
rule = ClashRuleParser.parse_rule_line(r['rule'])
if rule is None:
continue
remark = r.get('remark', '')
time_modified = r.get('time_modified', time.time())
self.rules.append(RuleItem(rule=rule, remark=remark, time_modified=time_modified))
def export_rules(self) -> List[Dict[str, str]]:
rules_list = []
for rule in self.rules:
rules_list.append({'rule': str(rule.rule), 'remark': rule.remark, 'time_modified': rule.time_modified})
return rules_list
def append_rules(self, clash_rules: List[RuleItem]):
self.rules.extend(clash_rules)
def insert_rule_at_priority(self, clash_rule: RuleItem, priority: int):
self.rules.insert(priority, clash_rule)
def update_rule_at_priority(self, clash_rule: RuleItem, src_priority: int, dst_priority) -> bool:
if len(self.rules) > src_priority >= 0:
if src_priority == dst_priority:
self.rules[src_priority] = clash_rule
else:
self.remove_rule_at_priority(src_priority)
self.insert_rule_at_priority(clash_rule, dst_priority)
return True
return False
def get_rule_at_priority(self, priority: int) -> Optional[RuleItem]:
"""Get rule item by priority"""
if len(self.rules) > priority >= 0:
return self.rules[priority]
return None
def remove_rule_at_priority(self, priority: int) -> Optional[RuleItem]:
"""Remove rule at specific priority"""
if 0 <= priority < len(self.rules):
return self.rules.pop(priority)
return None
def remove_rules_by_lambda(self, condition: Callable[[RuleItem], bool]):
"""Remove rules by lambda"""
initial_count = len(self.rules)
i = 0
while i < len(self.rules):
if condition(self.rules[i]):
del self.rules[i]
else:
i += 1
return initial_count - len(self.rules)
def move_rule_priority(self, from_priority: int, to_priority: int) -> bool:
"""Move rule priority to priority"""
clash_rule = self.remove_rule_at_priority(from_priority)
if not clash_rule:
return False
self.insert_rule_at_priority(clash_rule, to_priority)
return True
def filter_rules_by_condition(self, condition: Callable[[RuleItem], bool]):
"""Filter rules by condition"""
return [clash_rule for clash_rule in self.rules if condition(clash_rule)]
def filter_rules_by_type(self, rule_type: RoutingRuleType) -> List[RuleItem]:
"""Filter rules by type"""
return [clash_rule for clash_rule in self.rules
if isinstance(clash_rule.rule, ClashRule) and clash_rule.rule.rule_type == rule_type]
def filter_rules_by_action(self, action: Union[Action, str]) -> List[RuleItem]:
"""Filter rules by action"""
return [clash_rule for clash_rule in self.rules if clash_rule.rule.action == action]
def has_rule(self, clash_rule: Union[ClashRule, LogicRule, MatchRule]) -> bool:
"""Check if there is an identical rule"""
return any(r.rule == clash_rule for r in self.rules)
def has_rule_item(self, clash_rule: RuleItem) -> bool:
return any(clash_rule.remark == r.remark and r.rule == clash_rule.rule for r in self.rules)
def reorder_rules(self, moved_priority: int, target_priority: int) -> RuleItem:
"""Reorder the rules"""
if not (0 <= moved_priority < len(self.rules)):
raise IndexError("moved_priority out of range")
if not (0 <= target_priority < len(self.rules)):
raise IndexError("target_priority out of range")
rule = self.rules.pop(moved_priority)
self.rules.insert(target_priority, rule)
return rule
def to_list(self) -> List[Dict[str, Any]]:
"""Convert parsed rules to a list"""
result = []
for priority, rule_item in enumerate(self.rules):
rule_dict = {'remark': rule_item.remark, 'time_modified': rule_item.time_modified,'priority': priority,
**rule_item.rule.to_dict()}
result.append(rule_dict)
return result
def clear(self):
self.rules.clear()
def __len__(self) -> int:
return len(self.rules)
def __iter__(self) -> Iterator[RuleItem]:
return iter(self.rules)

View File

@@ -0,0 +1,334 @@
import re
from typing import List, Dict, Any, Optional, Union
from pydantic import ValidationError
from ..models.rule import RuleType, Action, RoutingRuleType, MatchRule, ClashRule, LogicRule, SubRule, AdditionalParam
class ClashRuleParser:
"""Parser for Clash routing rules"""
@staticmethod
def parse_rule_line(line: str) -> Optional[RuleType]:
"""Parse a single rule line"""
line = line.strip()
try:
# Handle logic rules (AND, OR, NOT)
if line.startswith(('AND,', 'OR,', 'NOT,')):
return ClashRuleParser._parse_logic_rule(line)
elif line.startswith('MATCH'):
return ClashRuleParser._parse_match_rule(line)
elif line.startswith('SUB-RULE'):
return ClashRuleParser._parse_sub_rule(line)
# Handle regular rules
return ClashRuleParser._parse_regular_rule(line)
except (ValidationError, TypeError, ValueError, RecursionError):
return None
@staticmethod
def parse_rule_dict(clash_rule: Dict[str, Any]) -> Optional[RuleType]:
if not clash_rule:
return None
try:
if clash_rule.get("type") in ('AND', 'OR', 'NOT'):
conditions = clash_rule.get("conditions", [])
if not conditions:
return None
conditions = [ClashRuleParser._remove_parenthesis(f"({c})") for c in conditions]
conditions_str = ','.join(conditions)
conditions_str = f"({conditions_str})"
raw_rule = f"{clash_rule.get('type')},{conditions_str},{clash_rule.get('action')}"
rule = ClashRuleParser._parse_logic_rule(raw_rule)
elif clash_rule.get("type") == 'MATCH':
raw_rule = f"{clash_rule.get('type')},{clash_rule.get('action')}"
rule = ClashRuleParser._parse_match_rule(raw_rule)
elif clash_rule.get("type") == 'SUB-RULE':
condition = clash_rule.get("condition")
if not condition:
return None
condition_str = f"({condition})"
condition_str = ClashRuleParser._remove_parenthesis(condition_str)
raw_rule = f"{clash_rule.get('type')},{condition_str},{clash_rule.get('action')}"
rule = ClashRuleParser._parse_sub_rule(raw_rule)
else:
raw_rule = f"{clash_rule.get('type')},{clash_rule.get('payload')},{clash_rule.get('action')}"
if clash_rule.get('additional_params'):
raw_rule += f',{clash_rule.get('additional_params')}'
rule = ClashRuleParser._parse_regular_rule(raw_rule)
except (ValidationError, TypeError, ValueError):
return None
return rule
@staticmethod
def _parse_match_rule(line: str) -> MatchRule:
parts = line.split(',')
if len(parts) < 2:
raise ValueError(f"Invalid rule format: {line}")
action = parts[1].strip()
# Validate rule type
try:
action_enum = Action(action.upper())
final_action = action_enum
except ValueError:
final_action = action
return MatchRule(
action=final_action,
raw_rule=line
)
@staticmethod
def _parse_regular_rule(line: str) -> ClashRule:
"""Parse a regular (non-logic) rule"""
parts = line.split(',')
if len(parts) < 3 or len(parts) > 4:
raise ValueError(f"Invalid rule format: {line}")
rule_type_str = parts[0].upper().strip()
payload = parts[1].strip()
action = parts[2].strip()
if not payload or not rule_type_str:
raise ValueError(f"Invalid rule format: {line}")
additional_params = parts[3].strip() if len(parts) > 3 else None
# Validate rule type
try:
rule_type = RoutingRuleType(rule_type_str)
except ValueError:
raise ValueError(f"Unknown rule type: {rule_type_str}")
# Try to convert action to enum, otherwise keep as string (custom proxy group)
if additional_params is not None:
additional_params = AdditionalParam(additional_params)
try:
action_enum = Action(action.upper())
final_action = action_enum
except ValueError:
final_action = action
return ClashRule(
rule_type=rule_type,
payload=payload,
action=final_action,
additional_params=additional_params,
raw_rule=line
)
@staticmethod
def _parenthesis_balance(s: str) -> Optional[int]:
"""Calculate the balance of parenthesis"""
balance = 0
for i, char in enumerate(s):
if char == '(':
balance += 1
elif char == ')':
balance -= 1
if balance < 0:
return None
return balance
@staticmethod
def _parse_logic_rule(line: str) -> LogicRule:
"""Parse a logic rule (AND, OR, NOT)"""
# Extract logic type
logic_type_str, rest = line.split(',', 1)
logic_type = RoutingRuleType(logic_type_str.upper().strip())
last_comma_index = rest.rfind(',')
if last_comma_index == -1:
raise ValueError(f"Invalid logic rule format: {line}")
action_str = rest[last_comma_index + 1:]
conditions_str = rest[:last_comma_index]
# Find the matching parenthesis for the conditions block to separate conditions from action
balance = ClashRuleParser._parenthesis_balance(conditions_str)
if balance != 0:
raise ValueError(f"Mismatched parentheses in logic rule: {line}")
action = action_str.strip()
# Try to convert action to enum
try:
action_enum = Action(action.upper())
final_action = action_enum
except ValueError:
final_action = action
conditions = ClashRuleParser._parse_logic_conditions(conditions_str)
return LogicRule(
rule_type=logic_type,
conditions=conditions,
action=final_action,
raw_rule=line
)
@staticmethod
def _parse_sub_rule(line: str) -> SubRule:
"""Parse a sub-rule"""
rule_type_str, rest = line.split(',', 1)
rule_type = RoutingRuleType(rule_type_str.upper().strip())
if rule_type != RoutingRuleType.SUB_RULE:
raise ValueError(f"{rule_type.value} is not a sub-rule")
last_comma_index = rest.rfind(',')
if last_comma_index == -1:
raise ValueError(f"Invalid sub-rule format: {line}")
condition_str = rest[:last_comma_index]
action_str = rest[last_comma_index + 1:]
balance = ClashRuleParser._parenthesis_balance(condition_str)
if balance != 0:
raise ValueError(f"Mismatched parentheses in sub-rule: {line}")
conditions = ClashRuleParser._parse_logic_conditions(condition_str)
if len(conditions) != 1:
raise ValueError(f"Invalid sub-rule condition: {condition_str}")
return SubRule(
condition=conditions[0],
action=action_str,
raw_rule=line
)
@staticmethod
def _remove_parenthesis(_con_str: str):
balance = 0
filed_list = []
field = ''
for i, char in enumerate(_con_str):
if char == '(':
balance += 1
elif char == ')':
balance -= 1
elif char == ',':
if balance == 1:
filed_list.append(field)
else:
if balance == 1 and char:
field = field + char
if not any(filed_list):
return ClashRuleParser._remove_parenthesis(_con_str[1:-1])
else:
return _con_str
@staticmethod
def _parse_logic_conditions(conditions_str: str) -> List[Union[ClashRule, LogicRule]]:
"""
Parse conditions within logic rules, supporting nested logic.
The examples of conditions_str:
- (DOMAIN,baidu.com)`
- (AND,(DOMAIN,baidu.com),(NETWORK,TCP))
"""
def __extract_condition_strings(_con_str: str) -> List[str]:
# Split conditions string by top-level commas
_con_str = _con_str.replace(' ', '')
_con_str = ClashRuleParser._remove_parenthesis(_con_str)
_condition_strings = []
balance = 0
start = 0
for i, char in enumerate(_con_str):
if char == '(':
if balance == 0:
start = i
balance += 1
elif char == ')':
balance -= 1
if balance == 0:
_condition_strings.append(_con_str[start:i + 1])
return _condition_strings
conditions = []
if not conditions_str:
return conditions
condition_strings = __extract_condition_strings(conditions_str)
for cond_str in condition_strings:
cond_str = cond_str.strip()
if not cond_str.startswith('(') or not cond_str.endswith(')'):
raise ValueError(f"Invalid nested logic rule format: {cond_str}")
content = cond_str[1:-1] # remove parentheses
if content.upper().startswith(('AND,', 'OR,', 'NOT,')):
# This is a nested logic rule.
parts = content.split(',', 1)
logic_type_str = parts[0].strip().upper()
logic_type = RoutingRuleType(logic_type_str)
nested_conditions_str = parts[1]
nested_conditions = ClashRuleParser._parse_logic_conditions(f'({nested_conditions_str})')
condition = LogicRule(
rule_type=logic_type,
conditions=nested_conditions,
action=Action.COMPATIBLE, # No action for conditions
raw_rule=content
)
conditions.append(condition)
else:
# Simple rule
parts = content.split(',', 1)
if len(parts) == 2:
rule_type_str, payload = parts
try:
rule_type = RoutingRuleType(rule_type_str.upper().strip())
condition = ClashRule(
rule_type=rule_type,
payload=payload.strip(),
action=Action.COMPATIBLE, # Logic conditions don't have actions
raw_rule=content
)
conditions.append(condition)
except ValueError:
raise ValueError(f"Invalid rule format: {content}")
return conditions
@staticmethod
def action_string(action: Union[Action, str]) -> str:
return action.value if isinstance(action, Action) else action
@staticmethod
def parse_rules(rules_text: str) -> List[Union[ClashRule, LogicRule, MatchRule]]:
"""Parse multiple rules from text, preserving order and priority"""
rules = []
lines = rules_text.strip().split('\n')
for line in lines:
rule = ClashRuleParser.parse_rule_line(line)
if rule:
rules.append(rule)
return rules
@staticmethod
def validate_rule(rule: ClashRule) -> bool:
"""Validate a parsed rule"""
try:
# Basic validation based on the rule type
if rule.rule_type in [RoutingRuleType.IP_CIDR, RoutingRuleType.IP_CIDR6]:
# Validate CIDR format
return '/' in rule.payload
elif rule.rule_type == RoutingRuleType.DST_PORT or rule.rule_type == RoutingRuleType.SRC_PORT:
# Validate port number/range
return rule.payload.isdigit() or '-' in rule.payload
elif rule.rule_type == RoutingRuleType.NETWORK:
# Validate the network type
return rule.payload.lower() in ['tcp', 'udp']
elif rule.rule_type == RoutingRuleType.DOMAIN_REGEX or rule.rule_type == RoutingRuleType.PROCESS_PATH_REGEX:
# Try to compile regex
re.compile(rule.payload)
return True
return True
except Exception:
return False

View File

@@ -0,0 +1,298 @@
import base64
import importlib
import json
import os
from typing import List, Dict, Any, Optional, Union
from urllib.parse import quote
from .converters import BaseConverter
class Converter:
"""
A refactored converter for V2Ray subscriptions that uses a strategy pattern.
It dynamically loads protocol-specific converters from the 'converters' directory.
"""
def __init__(self):
self._converters: Dict[str, BaseConverter] = self._load_converters()
def _load_converters(self) -> Dict[str, BaseConverter]:
"""
Dynamically discovers and loads all converter classes from the .py files
in the 'converters' directory.
"""
converters: Dict[str, BaseConverter] = {}
converter_dir = os.path.dirname(__file__)
module_names = [f.replace('.py', '') for f in os.listdir(os.path.join(converter_dir, 'converters'))
if f.endswith('.py') and not f.startswith('__')]
for module_name in module_names:
try:
module = importlib.import_module(f".converters.{module_name}", package=__package__)
class_name = f"{module_name.capitalize()}Converter"
converter_class = getattr(module, class_name, None)
if converter_class and issubclass(converter_class, BaseConverter):
instance = converter_class()
# Determine the protocol scheme based on the module name
scheme = module_name
if scheme == 'http':
converters['http'] = instance
converters['https'] = instance
elif scheme == 'socks':
converters['socks'] = instance
converters['socks5'] = instance
converters['socks5h'] = instance
elif scheme == 'hysteria2':
converters['hysteria2'] = instance
converters['hy2'] = instance
else:
converters[scheme] = instance
except (ImportError, AttributeError) as e:
# Log this error appropriately in a real application
print(f"Could not load converter for {module_name}: {e}")
return converters
def convert_line(self, line: str, names: Optional[Dict[str, int]] = None, skip_exception: bool = True
) -> Optional[Dict[str, Any]]:
"""
Parses a single subscription link and converts it to a proxy dictionary.
"""
if names is None:
names = {}
if "://" not in line:
return None
scheme, _ = line.split("://", 1)
scheme = scheme.lower()
converter = self._converters.get(scheme)
if converter:
try:
return converter.convert(line, names)
except Exception as e:
if not skip_exception:
raise ValueError(f"{scheme.upper()} parse error: {e}") from e
return None
return None
def convert_v2ray(self, v2ray_link: Union[list, bytes], skip_exception: bool = True) -> List[Dict[str, Any]]:
"""
Converts a base64 encoded V2Ray subscription content or a list of links
into a list of proxy dictionaries.
"""
if isinstance(v2ray_link, bytes):
decoded = BaseConverter.decode_base64(v2ray_link).decode("utf-8")
lines = decoded.strip().splitlines()
else:
lines = v2ray_link
proxies = []
names = {}
for line in lines:
line = line.strip()
if not line:
continue
proxy = self.convert_line(line, names, skip_exception=skip_exception)
if proxy:
proxies.append(proxy)
elif not skip_exception:
raise ValueError("Failed to convert one of the links in the subscription.")
return proxies
@staticmethod
def convert_to_share_link(proxy_config: Dict[str, Any]) -> Optional[str]:
proxy_type = proxy_config.get("type")
name = proxy_config.get("name", "proxy")
if proxy_type == "vmess":
vmess_config = {
"v": "2",
"ps": name,
"add": proxy_config.get("server", ""),
"port": str(proxy_config.get("port", "")),
"id": proxy_config.get("uuid", ""),
"aid": str(proxy_config.get("alterId", 0)),
"scy": proxy_config.get("cipher", "auto"),
"net": proxy_config.get("network", "tcp"),
"type": "none",
"tls": "tls" if proxy_config.get("tls") else "",
"host": "",
"path": "/",
}
if proxy_config.get("network") == "http":
vmess_config["type"] = "http"
network = proxy_config.get("network")
if network == "ws":
ws_opts = proxy_config.get("ws-opts", {})
vmess_config["host"] = ws_opts.get("headers", {}).get("Host", "")
vmess_config["path"] = ws_opts.get("path", "/")
elif network == "http":
http_opts = proxy_config.get("http-opts", {})
vmess_config["host"] = http_opts.get("headers", {}).get("Host", "")
vmess_config["path"] = http_opts.get("path", "/")
elif network == "h2":
h2_opts = proxy_config.get("h2-opts", {})
vmess_config["host"] = h2_opts.get("host")[0] if h2_opts.get("host") else ""
vmess_config["path"] = h2_opts.get("path", "/")
# Remove empty values to keep the JSON clean
vmess_config = {k: v for k, v in vmess_config.items() if v not in ["", None]}
encoded_str = base64.b64encode(json.dumps(vmess_config).encode("utf-8")).decode("utf-8")
return f"vmess://{encoded_str}"
elif proxy_type == "ss":
method = proxy_config.get("cipher")
password = proxy_config.get("password")
server = proxy_config.get("server")
port = proxy_config.get("port")
if not all([method, password, server, port]):
return None
credentials = f"{method}:{password}@{server}:{port}"
encoded_credentials = base64.b64encode(credentials.encode("utf-8")).decode("utf-8")
return f"ss://{encoded_credentials}#{quote(name)}"
elif proxy_type == "trojan":
password = proxy_config.get("password")
server = proxy_config.get("server")
port = proxy_config.get("port")
if not all([password, server, port]):
return None
query_params = {}
if proxy_config.get("sni"):
query_params["sni"] = proxy_config["sni"]
if proxy_config.get("alpn"):
query_params["alpn"] = ",".join(proxy_config["alpn"])
if proxy_config.get("skip-cert-verify"):
query_params["allowInsecure"] = "1"
network = proxy_config.get("network")
if network:
query_params["type"] = network
if network == "ws":
ws_opts = proxy_config.get("ws-opts", {})
path = ws_opts.get("path", "/")
host = ws_opts.get("headers", {}).get("Host", "")
# Always add path and host for ws if they exist, even if defaulted, for round-trip consistency
if path:
query_params["path"] = path
if host:
query_params["host"] = host
elif network == "grpc":
grpc_opts = proxy_config.get("grpc-opts", {})
service_name = grpc_opts.get("grpc-service-name", "")
if service_name:
query_params["serviceName"] = service_name
client_fingerprint = proxy_config.get("client-fingerprint")
# Always add fp if it exists, to ensure round-trip consistency, as convert_v2ray defaults to "chrome"
if client_fingerprint:
query_params["fp"] = client_fingerprint
query_string = "&".join([f"{k}={quote(str(v))}" for k, v in query_params.items()])
base_link = f"trojan://{password}@{server}:{port}"
if query_string:
return f"{base_link}?{query_string}#{quote(name)}"
else:
return f"{base_link}#{quote(name)}"
elif proxy_type == "vless":
uuid = proxy_config.get("uuid")
server = proxy_config.get("server")
port = proxy_config.get("port")
if not all([uuid, server, port]):
return None
query_params = {}
name = proxy_config.get("name", f"{server}:{port}")
tls = proxy_config.get("tls", False)
if tls:
if "reality-opts" in proxy_config:
query_params["security"] = "reality"
reality_opts = proxy_config["reality-opts"]
if reality_opts.get("public-key"):
query_params["pbk"] = reality_opts["public-key"]
if reality_opts.get("short-id"):
query_params["sid"] = reality_opts["short-id"]
else:
query_params["security"] = "tls"
if proxy_config.get("client-fingerprint"):
query_params["fp"] = proxy_config["client-fingerprint"]
if proxy_config.get("alpn"):
query_params["alpn"] = ",".join(proxy_config["alpn"])
if proxy_config.get("skip-cert-verify"):
query_params["allowInsecure"] = "1"
if proxy_config.get("servername"):
query_params["sni"] = proxy_config["servername"]
# Network settings
network = proxy_config.get("network", "tcp")
query_params["type"] = network
if network == "ws":
ws_opts = proxy_config.get("ws-opts", {})
path = ws_opts.get("path", "")
host = ws_opts.get("headers", {}).get("Host", "")
if path:
query_params["path"] = path
if host:
query_params["host"] = host
elif network == "grpc":
grpc_opts = proxy_config.get("grpc-opts", {})
service_name = grpc_opts.get("grpc-service-name", "")
if service_name:
query_params["serviceName"] = service_name
if proxy_config.get("flow"):
query_params["flow"] = proxy_config["flow"]
query_string = "&".join([f"{k}={quote(str(v))}" for k, v in query_params.items()])
base_link = f"vless://{uuid}@{server}:{port}"
if query_string:
return f"{base_link}?{query_string}#{quote(name)}"
else:
return f"{base_link}#{quote(name)}"
elif proxy_type == "ssr":
server = proxy_config.get("server")
port = proxy_config.get("port")
protocol = proxy_config.get("protocol", "origin")
cipher = proxy_config.get("cipher")
obfs = proxy_config.get("obfs", "plain")
password = proxy_config.get("password")
name = proxy_config.get("name", f"{server}:{port}")
if not all([server, port, protocol, cipher, obfs, password]):
return None
password_enc = base64.urlsafe_b64encode(password.encode("utf-8")).decode("utf-8").rstrip('=')
ssr_main_part = f"{server}:{port}:{protocol}:{cipher}:{obfs}:{password_enc}"
query_params = {}
if proxy_config.get("obfs-param"):
query_params["obfsparam"] = base64.urlsafe_b64encode(
proxy_config["obfs-param"].encode("utf-8")).decode("utf-8").rstrip('=')
if proxy_config.get("protocol-param"):
query_params["protoparam"] = base64.urlsafe_b64encode(
proxy_config["protocol-param"].encode("utf-8")).decode("utf-8").rstrip('=')
query_params["remarks"] = base64.urlsafe_b64encode(name.encode("utf-8")).decode("utf-8").rstrip('=')
query_params["group"] = base64.urlsafe_b64encode("MoviePilot".encode("utf-8")).decode("utf-8").rstrip('=')
query_string = "&".join([f"{k}={v}" for k, v in query_params.items()])
full_ssr_link_body = f"{ssr_main_part}/?{query_string}"
encoded_full_ssr_link_body = base64.urlsafe_b64encode(
full_ssr_link_body.encode("utf-8")).decode("utf-8").rstrip('=')
return f"ssr://{encoded_full_ssr_link_body}"
return None

View File

@@ -0,0 +1,163 @@
from abc import ABC, abstractmethod
import base64
import binascii
import json
from typing import Dict, Any, Optional
from urllib.parse import unquote, urlparse, parse_qsl
class BaseConverter(ABC):
"""
Abstract base class for all protocol converters.
It defines a common interface and provides shared utility methods.
"""
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome'
@abstractmethod
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
"""
Converts a subscription link to a proxy configuration dictionary.
:param link: The subscription link string (e.g., "vmess://...").
:param names: A dictionary to track and ensure unique proxy names.
:return: A dictionary representing the proxy configuration, or None if conversion fails.
"""
raise NotImplementedError
@staticmethod
def decode_base64(data):
# Add fault tolerance for different padding
data = data.strip()
missing_padding = len(data) % 4
if missing_padding:
data += '=' * (4 - missing_padding)
return base64.b64decode(data)
@staticmethod
def decode_base64_urlsafe(data):
data = data.strip()
missing_padding = len(data) % 4
if missing_padding:
data += '=' * (4 - missing_padding)
return base64.urlsafe_b64decode(data)
@staticmethod
def try_decode_base64_json(data):
try:
return json.loads(BaseConverter.decode_base64(data).decode('utf-8'))
except (binascii.Error, UnicodeDecodeError, json.JSONDecodeError, TypeError):
return None
@staticmethod
def unique_name(name_map: Dict[str, int], name: str) -> str:
index = name_map.get(name, 0)
name_map[name] = index + 1
if index > 0:
return f"{name}-{index:02d}"
return name
@staticmethod
def lower_string(string: Optional[str]) -> Optional[str]:
if isinstance(string, str):
return string.lower()
return string
@staticmethod
def handle_vshare_link(link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
url_info = urlparse(link)
query = dict(parse_qsl(url_info.query))
scheme = url_info.scheme.lower()
if not url_info.hostname or not url_info.port:
return None
proxy: Dict[str, Any] = {
'name': BaseConverter.unique_name(names,
unquote(url_info.fragment or f"{url_info.hostname}:{url_info.port}")),
'type': scheme,
'server': url_info.hostname,
'port': url_info.port,
'uuid': url_info.username,
'udp': True
}
# TLS and Reality settings
tls_mode = BaseConverter.lower_string(query.get('security'))
if tls_mode in ['tls', 'reality']:
proxy['tls'] = True
proxy['client-fingerprint'] = query.get('fp', 'chrome')
if 'alpn' in query:
proxy['alpn'] = query['alpn'].split(',')
if 'sni' in query:
proxy['servername'] = query['sni']
if tls_mode == 'reality':
proxy['reality-opts'] = {
'public-key': query.get('pbk'),
'short-id': query.get('sid')
}
# Network settings
network = BaseConverter.lower_string(query.get('type', 'tcp'))
header_type = BaseConverter.lower_string(query.get('headerType'))
if header_type == 'http':
network = 'http'
elif network == 'http':
network = 'h2'
proxy['network'] = network
if network == 'tcp' and header_type == 'http':
proxy['http-opts'] = {
'method': query.get('method', 'GET'),
'path': [query.get('path', '/')],
'headers': {'Host': [query.get('host', url_info.hostname)]}
}
elif network == 'h2':
proxy["h2-opts"] = {
"path": query.get("path", "/"),
"host": [query.get("host", url_info.hostname)]
}
elif network in ['ws', 'httpupgrade']:
ws_opts: Dict[str, Any] = {
'path': query.get('path', '/'),
'headers': {
'Host': query.get('host', url_info.hostname),
'User-Agent': BaseConverter.user_agent
}
}
if 'ed' in query:
try:
med = int(query['ed'])
if network == 'ws':
ws_opts['max-early-data'] = med
ws_opts['early-data-header-name'] = query.get('eh', 'Sec-WebSocket-Protocol')
elif network == 'httpupgrade':
ws_opts['v2ray-http-upgrade-fast-open'] = True
except (ValueError, TypeError):
pass
proxy['ws-opts'] = ws_opts
elif network == 'grpc':
proxy['grpc-opts'] = {
'grpc-service-name': query.get('serviceName', '')
}
# Packet Encoding
packet_encoding = BaseConverter.lower_string(query.get('packetEncoding'))
if packet_encoding == 'packet':
proxy['packet-addr'] = True
elif packet_encoding != 'none':
proxy['xudp'] = True
# Encryption
if 'encryption' in query and query['encryption']:
proxy['encryption'] = query['encryption']
if 'flow' in query:
proxy['flow'] = query['flow']
return proxy
except Exception:
return None

View File

@@ -0,0 +1,36 @@
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from . import BaseConverter
class AnytlsConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
query = dict(parse_qsl(parsed.query))
username = parsed.username
password = parsed.password or username
server = parsed.hostname
port = parsed.port
insecure = query.get("insecure", "0") == "1"
sni = query.get("sni")
fingerprint = query.get("hpkp")
name = self.unique_name(names, unquote(parsed.fragment or f"{server}:{port}"))
proxy = {
"name": name,
"type": "anytls",
"server": server,
"port": port,
"username": username,
"password": password,
"sni": sni,
"fingerprint": fingerprint,
"skip-cert-verify": insecure,
"udp": True
}
return proxy
except Exception:
return None

View File

@@ -0,0 +1,46 @@
import binascii
from typing import Dict, Any, Optional
from urllib.parse import urlparse, unquote
from . import BaseConverter
class HttpConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
server = parsed.hostname
port = parsed.port
name = self.unique_name(names, unquote(parsed.fragment or f"{server}:{port}"))
username = None
password = None
if parsed.username:
try:
# The userinfo part might be base64 encoded
decoded_userinfo = self.decode_base64(parsed.username.encode('utf-8')).decode('utf-8')
if ":" in decoded_userinfo:
username, password = decoded_userinfo.split(":", 1)
else:
username = decoded_userinfo
except (binascii.Error, UnicodeDecodeError):
# If not base64 encoded, use directly
username = parsed.username
password = parsed.password if parsed.password else ""
proxy = {
"name": name,
"type": "http",
"server": server,
"port": port,
"username": username,
"password": password,
"skip-cert-verify": True
}
if parsed.scheme == "https":
proxy["tls"] = True
return proxy
except Exception:
return None

View File

@@ -0,0 +1,58 @@
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from app.utils.string import StringUtils
from . import BaseConverter
class HysteriaConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
query = dict(parse_qsl(parsed.query))
name = self.unique_name(names, unquote(parsed.fragment or f"{parsed.hostname}:{parsed.port}"))
hysteria: Dict[str, Any] = {
"name": name,
"type": "hysteria",
"server": parsed.hostname,
"port": parsed.port,
}
auth_str = query.get("auth")
if auth_str:
hysteria["auth_str"] = auth_str
obfs = query.get("obfs")
if obfs:
hysteria["obfs"] = obfs
sni = query.get("peer")
if sni:
hysteria["sni"] = sni
protocol = query.get("protocol")
if protocol:
hysteria["protocol"] = protocol
up = query.get("up")
if not up:
up = query.get("upmbps")
if up:
hysteria["up"] = up
down = query.get("down")
if not down:
down = query.get("downmbps")
if down:
hysteria["down"] = down
alpn = query.get("alpn", "")
if alpn:
hysteria["alpn"] = alpn.split(",")
# skip-cert-verify
insecure_str = query.get("insecure", "false")
try:
skip_cert_verify = StringUtils.to_bool(insecure_str)
if skip_cert_verify:
hysteria["skip-cert-verify"] = skip_cert_verify
except ValueError:
pass
return hysteria
except Exception:
return None

View File

@@ -0,0 +1,44 @@
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from app.utils.string import StringUtils
from . import BaseConverter
class Hysteria2Converter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
query = dict(parse_qsl(parsed.query))
user_info = ""
if parsed.username:
if parsed.password:
user_info = f"{parsed.username}:{parsed.password}"
else:
user_info = parsed.username
password = user_info
server = parsed.hostname
port = parsed.port or 443
name = self.unique_name(names, unquote(parsed.fragment or f"{server}:{port}"))
proxy = {
"name": name,
"type": "hysteria2",
"server": server,
"port": port,
"password": password,
"obfs": query.get("obfs"),
"obfs-password": query.get("obfs-password"),
"sni": query.get("sni"),
"skip-cert-verify": StringUtils.to_bool(query.get("insecure", "false")),
"down": query.get("down"),
"up": query.get("up"),
}
if "pinSHA256" in query:
proxy["fingerprint"] = query.get("pinSHA256")
if "alpn" in query:
proxy["alpn"] = query["alpn"].split(",")
return proxy
except Exception:
return None

View File

@@ -0,0 +1,42 @@
import binascii
from typing import Dict, Any, Optional
from urllib.parse import urlparse, unquote
from . import BaseConverter
class SocksConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
server = parsed.hostname
port = parsed.port
name = self.unique_name(names, unquote(parsed.fragment or f"{server}:{port}"))
username = None
password = None
if parsed.username:
try:
# The userinfo part might be base64 encoded
decoded_userinfo = self.decode_base64(parsed.username.encode('utf-8')).decode('utf-8')
if ":" in decoded_userinfo:
username, password = decoded_userinfo.split(":", 1)
else:
username = decoded_userinfo
except (binascii.Error, UnicodeDecodeError):
# If not base64 encoded, use directly
username = parsed.username
password = parsed.password if parsed.password else ""
proxy = {
"name": name,
"type": "socks5",
"server": server,
"port": port,
"username": username,
"password": password,
"skip-cert-verify": True
}
return proxy
except Exception:
return None

View File

@@ -0,0 +1,75 @@
import binascii
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from . import BaseConverter
class SsConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
if parsed.port is None and parsed.netloc:
base64_body = parsed.netloc
decoded_body = self.decode_base64_urlsafe(base64_body).decode('utf-8')
new_line = f"ss://{decoded_body}"
if parsed.fragment:
new_line += f"#{parsed.fragment}"
parsed = urlparse(new_line)
name = self.unique_name(names, unquote(parsed.fragment or f"{parsed.hostname}:{parsed.port}"))
cipher_raw = parsed.username
password = parsed.password
cipher = cipher_raw
if not password and cipher_raw:
try:
decoded_user = self.decode_base64_urlsafe(cipher_raw).decode('utf-8')
except (binascii.Error, UnicodeDecodeError):
decoded_user = self.decode_base64(cipher_raw).decode('utf-8')
if ":" in decoded_user:
cipher, password = decoded_user.split(":", 1)
else:
cipher = decoded_user
server = parsed.hostname
port = parsed.port
query = dict(parse_qsl(parsed.query))
proxy = {
"name": name,
"type": "ss",
"server": server,
"port": port,
"cipher": cipher,
"password": password,
"udp": True
}
if query.get("udp-over-tcp") == "true" or query.get("uot") == "1":
proxy["udp-over-tcp"] = True
plugin = query.get("plugin")
if plugin and ";" in plugin:
query_string = "pluginName=" + plugin.replace(";", "&")
plugin_info = dict(parse_qsl(query_string))
plugin_name = plugin_info.get("pluginName", "")
if "obfs" in plugin_name:
proxy["plugin"] = "obfs"
proxy["plugin-opts"] = {
"mode": plugin_info.get("obfs"),
"host": plugin_info.get("obfs-host"),
}
elif "v2ray-plugin" in plugin_name:
proxy["plugin"] = "v2ray-plugin"
proxy["plugin-opts"] = {
"mode": plugin_info.get("mode"),
"host": plugin_info.get("host"),
"path": plugin_info.get("path"),
"tls": "tls" in plugin,
}
return proxy
except Exception:
return None

View File

@@ -0,0 +1,64 @@
import binascii
from typing import Dict, Any, Optional
from urllib.parse import parse_qsl
from . import BaseConverter
class SsrConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
_, body = link.split("://", 1)
try:
decoded_body = self.decode_base64_urlsafe(body).decode('utf-8')
except (binascii.Error, UnicodeDecodeError):
decoded_body = self.decode_base64(body).decode('utf-8')
parts, _, params_str = decoded_body.partition("/?")
part_list = parts.split(":", 5)
if len(part_list) != 6:
raise ValueError("Invalid SSR link format: incorrect number of parts")
host, port_str, protocol, method, obfs, password_enc = part_list
try:
port = int(port_str)
except ValueError:
raise ValueError("Invalid port in SSR link")
password = self.decode_base64_urlsafe(password_enc).decode('utf-8')
params = dict(parse_qsl(params_str))
remarks_b64 = params.get("remarks", "")
remarks = self.decode_base64_urlsafe(remarks_b64).decode('utf-8') if remarks_b64 else ""
obfsparam_b64 = params.get("obfsparam", "")
obfsparam = self.decode_base64_urlsafe(obfsparam_b64).decode(
'utf-8') if obfsparam_b64 else ""
protoparam_b64 = params.get("protoparam", "")
protoparam = self.decode_base64_urlsafe(protoparam_b64).decode(
'utf-8') if protoparam_b64 else ""
name = self.unique_name(names, remarks or f"{host}:{port}")
proxy = {
"name": name,
"type": "ssr",
"server": host,
"port": port,
"cipher": method,
"password": password,
"obfs": obfs,
"protocol": protocol,
"udp": True
}
if obfsparam:
proxy["obfs-param"] = obfsparam
if protoparam:
proxy["protocol-param"] = protoparam
return proxy
except Exception:
return None

View File

@@ -0,0 +1,60 @@
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from app.utils.string import StringUtils
from . import BaseConverter
class TrojanConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
query = dict(parse_qsl(parsed.query))
name = self.unique_name(names, unquote(parsed.fragment or f"{parsed.hostname}:{parsed.port}"))
trojan: Dict[str, Any] = {
"name": name,
"type": "trojan",
"server": parsed.hostname,
"port": parsed.port or 443,
"password": parsed.username or "",
"udp": True,
"tls": True
}
# skip-cert-verify
try:
trojan["skip-cert-verify"] = StringUtils.to_bool(query.get("allowInsecure", "0"))
except ValueError:
trojan["skip-cert-verify"] = False
# optional fields
if "sni" in query:
trojan["sni"] = query["sni"]
alpn = query.get("alpn")
if alpn:
trojan["alpn"] = alpn.split(",")
network = query.get("type", "").lower()
if network:
trojan["network"] = network
if network == "ws":
headers = {"User-Agent": self.user_agent}
trojan["ws-opts"] = {
"path": query.get("path", "/"),
"headers": headers
}
elif network == "grpc":
trojan["grpc-opts"] = {
"grpc-service-name": query.get("serviceName")
}
fp = query.get("fp")
trojan["client-fingerprint"] = fp if fp else "chrome"
return trojan
except Exception:
return None

View File

@@ -0,0 +1,46 @@
from typing import Dict, Any, Optional
from urllib.parse import urlparse, parse_qsl, unquote
from . import BaseConverter
class TuicConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
parsed = urlparse(link)
query = dict(parse_qsl(parsed.query))
user = parsed.username
password = parsed.password
server = parsed.hostname
port = parsed.port
name = self.unique_name(names, unquote(parsed.fragment or f"{server}:{port}"))
proxy = {
"name": name,
"type": "tuic",
"server": server,
"port": port,
"udp": True
}
if password:
proxy["uuid"] = user
proxy["password"] = password
else:
proxy["token"] = user
if "congestion_control" in query:
proxy["congestion-controller"] = query["congestion_control"]
if "alpn" in query:
proxy["alpn"] = query["alpn"].split(",")
if "sni" in query:
proxy["sni"] = query["sni"]
if query.get("disable_sni", "0") == "1":
proxy["disable-sni"] = True
if "udp_relay_mode" in query:
proxy["udp-relay-mode"] = query["udp_relay_mode"]
return proxy
except Exception:
return None

View File

@@ -0,0 +1,11 @@
from typing import Dict, Any, Optional
from . import BaseConverter
class VlessConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
return self.handle_vshare_link(link, names)
except Exception:
return None

View File

@@ -0,0 +1,106 @@
from typing import Dict, Any, Optional
from . import BaseConverter
class VmessConverter(BaseConverter):
def convert(self, link: str, names: Dict[str, int]) -> Optional[Dict[str, Any]]:
try:
_, body = link.split("://", 1)
vmess_data = self.try_decode_base64_json(body)
# Xray VMessAEAD share link
if vmess_data is None:
return self.handle_vshare_link(link, names)
name = self.unique_name(names, vmess_data.get("ps", "vmess"))
net = self.lower_string(vmess_data.get("net"))
fake_type = self.lower_string(vmess_data.get("type"))
tls_mode = self.lower_string(vmess_data.get("tls"))
cipher = vmess_data.get("scy", "auto") or "auto"
alter_id = vmess_data.get("aid", 0)
# Adjust network type
if fake_type == "http":
net = "http"
elif net == "http":
net = "h2"
proxy = {
"name": name,
"type": "vmess",
"server": vmess_data.get("add"),
"port": vmess_data.get("port"),
"uuid": vmess_data.get("id"),
"alterId": alter_id,
"cipher": cipher,
"tls": tls_mode.endswith("tls") or tls_mode == "reality",
"udp": True,
"xudp": True,
"skip-cert-verify": False,
"network": net
}
# TLS Reality extension
if proxy["tls"]:
proxy["client-fingerprint"] = vmess_data.get("fp", "chrome") or "chrome"
alpn = vmess_data.get("alpn")
if alpn:
proxy["alpn"] = alpn.split(",") if isinstance(alpn, str) else alpn
sni = vmess_data.get("sni")
if sni:
proxy["servername"] = sni
if tls_mode == "reality":
proxy["reality-opts"] = {
"public-key": vmess_data.get("pbk"),
"short-id": vmess_data.get("sid")
}
path = vmess_data.get("path", "/")
host = vmess_data.get("host")
# Extension fields for different networks
if net == "tcp":
if fake_type == "http":
proxy["http-opts"] = {
"path": path,
"headers": {"Host": host} if host else {}
}
elif net == "http":
headers = {}
if host:
headers["Host"] = [host]
proxy["http-opts"] = {"path": [path], "headers": headers}
elif net == "h2":
proxy["h2-opts"] = {
"path": path,
"host": [host] if host else []
}
elif net == "ws":
ws_headers = {"Host": host} if host else {}
ws_headers["User-Agent"] = self.user_agent
ws_opts = {
"path": path,
"headers": ws_headers
}
# Add early-data config
early_data = vmess_data.get("ed")
if early_data:
try:
ws_opts["max-early-data"] = int(early_data)
except ValueError:
pass
early_data_header = vmess_data.get("edh")
if early_data_header:
ws_opts["early-data-header-name"] = early_data_header
proxy["ws-opts"] = ws_opts
elif net == "grpc":
proxy["grpc-opts"] = {
"grpc-service-name": path
}
return proxy
except Exception:
return None

View File

@@ -0,0 +1,105 @@
import copy
from dataclasses import dataclass
from typing import Callable, Dict, List, Optional, Union, Any, Iterator
from ..models.proxy import Proxy, ProxyType
@dataclass
class ProxyItem:
proxy: ProxyType
remark: str = ""
raw: Optional[Union[str, Dict[str, Any]]] = None
class ProxyManager:
"""Proxy Manager"""
def __init__(self):
self.proxies: Dict[str,ProxyItem] = {}
def add(self, proxy: ProxyType, remark: str = "", raw: Optional[str|Dict[str, Any]] = None):
"""Add a proxy to the proxy manager. """
if proxy.name not in self.proxies:
self.proxies[proxy.name] = ProxyItem(proxy, remark, raw=copy.deepcopy(raw))
else:
raise ValueError(f"Proxy with name {proxy.name!r} already exists.")
def add_proxy_dict(self, proxy_dict: Dict[str, Any], remark: str = "", raw: Optional[str] = None):
"""
Add a proxy to the proxies list.
:param proxy_dict: Proxy dict with proxy name as key
:param remark: Proxy remark
:param raw: Proxy raw
:raises: ValueError if proxy name already exists
"""
proxy = Proxy.model_validate(proxy_dict)
raw = raw or proxy_dict
self.add(proxy.root, remark=remark, raw=raw)
def add_from_list(self, proxies: List[Dict[str, Any]], remark: str = "", skip_existing: bool = False):
"""Add proxies from the proxies list. """
proxies_list = []
for proxy in proxies:
p = Proxy.model_validate(proxy)
proxies_list.append(ProxyItem(p.root, remark, raw=proxy))
for proxy_item in proxies_list:
try:
self.add(proxy_item.proxy, remark=remark, raw=proxy_item.raw)
except ValueError:
if skip_existing:
continue
raise
def get_all_proxies(self) -> List[Dict[str, Any]]:
proxies = []
for proxy_item in self.proxies.values():
proxy_dict = proxy_item.proxy.model_dump(by_alias=True, exclude_none=True)
proxies.append(proxy_dict)
return proxies
def remove_proxy(self, name):
if name in self.proxies:
del self.proxies[name]
def remove_proxies_by_condition(self, condition: Callable[[ProxyItem], bool]) -> int:
"""
Removes proxies from the manager based on a given condition.
:param condition: A callable that takes a ProxyItem and returns True if the proxy should be removed.
:return: The number of proxies removed.
"""
initial_count = len(self.proxies)
self.proxies = {
name: item
for name, item in self.proxies.items()
if not condition(item)
}
return initial_count - len(self.proxies)
def filter_proxies_by_condition(self, condition: Callable[[ProxyItem], bool]) -> List[ProxyItem]:
return [proxy for proxy in self.proxies.values() if condition(proxy)]
def clear(self):
self.proxies.clear()
def export_raw(self, condition: Optional[Callable[[ProxyItem], bool]] = None) -> List[str|Dict[str, Any]]:
proxies = []
for proxy in self.proxies.values():
if condition and not condition(proxy):
continue
if proxy.raw:
proxies.append(copy.deepcopy(proxy.raw))
else:
proxies.append(proxy.proxy.model_dump(by_alias=True, exclude_none=True))
return proxies
def proxy_names(self) -> Iterator[str]:
return iter(self.proxies)
def __len__(self) -> int:
return len(self.proxies)
def __iter__(self) -> Iterator[ProxyItem]:
return iter(self.proxies.values())
def __contains__(self, name: str) -> bool:
return name in self.proxies

View File

@@ -0,0 +1,73 @@
import math
import time
from typing import Any, Optional, List, Dict
from urllib.parse import urlparse
class UtilsProvider:
@staticmethod
def filter_empty(original_dict: dict, empty: Optional[List[Any]] = None) -> dict:
"""过滤字典中的空值"""
return {k: v for k, v in original_dict.items() if v not in (empty or [None, '', [], {}])}
@staticmethod
def get_url_domain(url: str) -> str:
"""从 url 中提取域名"""
if not url:
return ""
parsed = urlparse(url)
if not parsed.netloc:
parsed = urlparse("https://" + url)
return parsed.netloc
@staticmethod
def find_cycles(graph: Dict[Any, Any]) -> List[List[Any]]:
"""DFS 检测环,并记录路径"""
visited = set()
stack = []
cycles = []
def dfs(node):
if node in stack:
cycle_index = stack.index(node)
cycles.append(stack[cycle_index:] + [node])
return
if node in visited:
return
visited.add(node)
stack.append(node)
for nei in graph.get(node, []):
dfs(nei)
stack.pop()
for n in graph:
if n not in visited:
dfs(n)
return cycles
@staticmethod
def format_bytes(value_bytes):
if value_bytes == 0:
return '0 B'
k = 1024
sizes = ['B', 'KB', 'MB', 'GB', 'TB']
i = math.floor(math.log(value_bytes) / math.log(k)) if value_bytes > 0 else 0
return f"{value_bytes / math.pow(k, i):.2f} {sizes[i]}"
@staticmethod
def format_expire_time(timestamp):
seconds_left = timestamp - int(time.time())
days = seconds_left // 86400
return f"{days}天后过期" if days > 0 else "已过期"
@staticmethod
def update_with_checking(src_dict: Dict[str, Any], dst_dict: Dict[str, Any]) -> Dict[str, Any]:
"""
跳过存在的键合并字典
"""
for key, value in src_dict.items():
if key in dst_dict:
continue
dst_dict[key] = value
return dst_dict

View File

@@ -0,0 +1,3 @@
from .proxy import *
from .ruleproviders import *
from .proxygroups import *

View File

@@ -0,0 +1,48 @@
from typing import List, Optional, Union, Literal
from pydantic import BaseModel, Field, ConfigDict
from .rule import RoutingRuleType, Action, AdditionalParam
from .ruleproviders import RuleProvider
class RuleData(BaseModel):
priority: int
type: RoutingRuleType
payload: Optional[str] = None
action: Union['Action', str]
additional_params: Optional[AdditionalParam] = None
conditions: Optional[List[str]] = None
condition: Optional[str] = None
model_config = ConfigDict(
use_enum_values=True
)
class ClashApi(BaseModel):
url: str
secret: str
class Connectivity(BaseModel):
clash_apis: List[ClashApi] = Field(default_factory=list)
sub_links: List[str] = Field(default_factory=list)
class Subscription(BaseModel):
url: str
class RuleProviderData(BaseModel):
name: str
rule_provider: RuleProvider
class SubscriptionInfo(BaseModel):
url: str
field: Literal['name', 'enabled']
value: Union[bool, str]
class Host(BaseModel):
domain: str
value: List[str]
using_cloudflare: bool
class HostData(BaseModel):
domain: str
value: Optional[Host] = None

View File

@@ -0,0 +1,48 @@
from typing import Union
from pydantic import Field, RootModel
from .anytlsproxy import AnyTLSProxy
from .directproxy import DirectProxy
from .dnsproxy import DnsProxy
from .httpproxy import HttpProxy
from .hysteriaproxy import HysteriaProxy
from .hysteria2proxy import Hysteria2Proxy
from .mieruproxy import MieruProxy
from .networkmixin import NetworkMixin
from .proxybase import ProxyBase
from .shadowsocksproxy import ShadowsocksProxy
from .shadowsocksrproxy import ShadowsocksRProxy
from .snellproxy import SnellProxy
from .socks5proxy import Socks5Proxy
from .sshproxy import SshProxy
from .tlsmixin import TLSMixin
from .trojanproxy import TrojanProxy
from .tuicproxy import TuicProxy
from .vlessproxy import VlessProxy
from .vmessproxy import VmessProxy
from .wireguardproxy import WireGuardProxy
ProxyType = Union[
AnyTLSProxy,
DirectProxy,
DnsProxy,
HttpProxy,
HysteriaProxy,
Hysteria2Proxy,
MieruProxy,
ShadowsocksProxy,
ShadowsocksRProxy,
SnellProxy,
Socks5Proxy,
SshProxy,
TrojanProxy,
TuicProxy,
VlessProxy,
VmessProxy,
WireGuardProxy,
]
class Proxy(RootModel[ProxyType]):
root: ProxyType = Field(..., discriminator="type")

View File

@@ -0,0 +1,15 @@
from typing import Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
from .networkmixin import NetworkMixin
class AnyTLSProxy(ProxyBase, TLSMixin, NetworkMixin):
type: Literal['anytls'] = 'anytls'
password: str
idle_session_check_interval: Optional[int] = Field(30, alias='idle-session-check-interval')
idle_session_timeout: Optional[int] = Field(30, alias='idle-session-timeout')
min_idle_session: Optional[int] = Field(0, alias='min-idle-session')

View File

@@ -0,0 +1,7 @@
from typing import Literal
from .proxybase import ProxyBase
class DirectProxy(ProxyBase):
type: Literal['direct'] = 'direct'

View File

@@ -0,0 +1,7 @@
from typing import Literal
from .proxybase import ProxyBase
class DnsProxy(ProxyBase):
type: Literal['dns'] = 'dns'

View File

@@ -0,0 +1,11 @@
from typing import Optional, Dict, Literal
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
class HttpProxy(ProxyBase, TLSMixin):
type: Literal['http'] = 'http'
username: Optional[str] = None
password: Optional[str] = None
headers: Optional[Dict[str, str]] = None

View File

@@ -0,0 +1,26 @@
from typing import Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
class Hysteria2Proxy(ProxyBase):
type: Literal['hysteria2'] = 'hysteria2'
password: Optional[str] = None
obfs: Optional[Literal['salamander']] = None
obfs_password: Optional[str] = Field(None, alias='obfs-password')
up: Optional[str] = None
down: Optional[str] = None
hop_interval: Optional[int] = Field(None, alias='hop-interval')
ca: Optional[str] = None
ca_str: Optional[str] = Field(None, alias='ca-str')
cwnd: Optional[int] = None
udp_mtu: Optional[int] = Field(None, alias='udp-mtu')
ports: Optional[str] = None
# QUIC-GO 特殊配置
initial_stream_receive_window: Optional[int] = Field(None, alias='initial-stream-receive-window')
max_stream_receive_window: Optional[int] = Field(None, alias='max-stream-receive-window')
initial_connection_receive_window: Optional[int] = Field(None, alias='initial-connection-receive-window')
max_connection_receive_window: Optional[int] = Field(None, alias='max-connection-receive-window')

View File

@@ -0,0 +1,26 @@
from typing import Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
class HysteriaProxy(ProxyBase):
type: Literal['hysteria'] = 'hysteria'
auth_str: Optional[str] = Field(None, alias='auth-str')
auth: Optional[str] = None
protocol: Optional[Literal['udp','wechat-video', 'faketcp']] = None
up: Optional[str] = None
down: Optional[str] = None
up_speed: Optional[int] = Field(None, alias='up-speed')
down_speed: Optional[int] = Field(None, alias='down-speed')
obfs: Optional[str] = None
obfs_protocol: Optional[str] = Field(None, alias='obfs-protocol')
recv_window_conn: Optional[int] = Field(None, alias='recv-window-conn')
recv_window: Optional[int] = Field(None, alias='recv-window')
disable_mtu_discovery: Optional[bool] = Field(None, alias='disable-mtu-discovery')
fast_open: Optional[bool] = Field(None, alias='fast-open')
hop_interval: Optional[int] = Field(None, alias='hop-interval')
ca: Optional[str] = None
ca_str: Optional[str] = Field(None, alias='ca-str')
ports: Optional[str] = None

View File

@@ -0,0 +1,25 @@
from typing import Optional, Literal
from pydantic import Field, model_validator
from .proxybase import ProxyBase
class MieruProxy(ProxyBase):
type: Literal['mieru'] = 'mieru'
username: str
password: str
port_range: Optional[str] = Field(None, alias='port-range')
transport: Literal['TCP'] = 'TCP'
multiplexing: Optional[Literal[
'MULTIPLEXING_OFF', 'MULTIPLEXING_LOW', 'MULTIPLEXING_MIDDLE', 'MULTIPLEXING_HIGH']] = 'MULTIPLEXING_LOW'
handshake_mode: Optional[Literal['HANDSHAKE_STANDARD', 'HANDSHAKE_NO_WAIT']] = 'HANDSHAKE_STANDARD'
@model_validator(mode='after')
def validate_port_config(self):
"""Pydantic v2 style model-level validation."""
if not getattr(self, 'port', None) and not getattr(self, 'port_range', None):
raise ValueError("either 'port' or 'port-range' must be set")
if getattr(self, 'port', None) and getattr(self, 'port_range', None):
raise ValueError("'port' and 'port-range' cannot be set at the same time")
return self

View File

@@ -0,0 +1,36 @@
from typing import List, Optional, Dict, Literal
from pydantic import BaseModel, Field
class HttpOpts(BaseModel):
method: Optional[str] = None
path: List[str] = ['/']
headers: Optional[Dict[str, List[str]]] = None
class H2Opts(BaseModel):
host: List[str]
path: str = '/'
class GrpcOpts(BaseModel):
grpc_service_name: str = Field(..., alias='grpc-service-name')
class WsOpts(BaseModel):
path: str = '/'
headers: Optional[Dict[str, str]] = None
max_early_data: Optional[int] = Field(None, alias='max-early-data')
early_data_header_name: Optional[str] = Field(None, alias='early-data-header-name')
v2ray_http_upgrade: Optional[bool] = Field(None, alias='v2ray-http-upgrade')
v2ray_http_upgrade_fast_open: Optional[bool] = Field(None, alias='v2ray-http-upgrade-fast-open')
class NetworkMixin(BaseModel):
# Transport settings
network: Optional[Literal['tcp', 'http', 'h2', 'grpc', 'ws', 'kcp']] = None
http_opts: Optional[HttpOpts] = Field(None, alias='http-opts')
h2_opts: Optional[H2Opts] = Field(None, alias='h2-opts')
grpc_opts: Optional[GrpcOpts] = Field(None, alias='grpc-opts')
ws_opts: Optional[WsOpts] = Field(None, alias='ws-opts')

View File

@@ -0,0 +1,38 @@
from typing import Optional, Literal
from pydantic import BaseModel, Field
class SmuxBrutalOpts(BaseModel):
enabled: bool = False
up: Optional[str] = None
down: Optional[str] = None
class Smux(BaseModel):
enabled: bool = False
protocol: Literal['smux', 'yamux', 'h2mux'] = 'h2mux'
max_connections: Optional[int] = Field(None, alias='max-connections')
min_streams: Optional[int] = Field(None, alias='min-streams')
max_streams: Optional[int] = Field(None, alias='max-streams')
statistic: Optional[bool] = None
only_tcp: Optional[bool] = Field(None, alias='only-tcp')
padding: Optional[bool] = None
brutal_opts: Optional[SmuxBrutalOpts] = Field(None, alias='brutal-opts')
class ProxyBase(BaseModel):
name: str
type: Literal['direct', 'dns', 'http', 'ss', 'ssr', 'mieru', 'snell', 'vmess', 'vless', 'trojan', 'anytls',
'hysteria','hysteria2', 'tuic', 'wireguard', 'ssh', 'socks5']
server: str
port: int
ip_version: Optional[Literal['dual', 'ipv4', 'ipv6', 'ipv4-prefer', 'ipv6-prefer']] = Field(None,
alias='ip-version')
udp: bool = False
interface_name: Optional[str] = Field(None, alias='interface-name')
routing_mark: Optional[int] = Field(None, alias='routing-mark')
tfo: Optional[bool] = None
mptcp: Optional[bool] = None
dialer_proxy: Optional[str] = Field(None, alias='dialer-proxy')
smux: Optional[Smux] = None

View File

@@ -0,0 +1,110 @@
from typing import Optional, Dict, Literal, List, Union
from pydantic import Field, BaseModel, field_validator, ValidationInfo
from .proxybase import ProxyBase
ShadowsocksCipherType = Literal[
# AES 相关
'aes-128-ctr', 'aes-192-ctr', 'aes-256-ctr',
'aes-128-cfb', 'aes-192-cfb', 'aes-256-cfb',
'aes-128-gcm', 'aes-192-gcm', 'aes-256-gcm',
'aes-128-com', 'aes-192-com', 'aes-256-com',
'aes-128-gcm-siv', 'aes-256-gcm-siv',
# CHACHA 相关
'chacha20-ietf', 'chacha20', 'xchacha20',
'chacha20-ietf-poly1305', 'xchacha20-ietf-poly1305',
'chacha8-ietf-poly1305', 'xchacha8-ietf-poly1305',
# 2022 Blake3 相关
'2022-blake3-aes-128-gcm', '2022-blake3-aes-256-gcm', '2022-blake3-chacha20-poly1305',
# LEA 相关
'lea-128-gcm', 'lea-192-gcm', 'lea-256-gcm',
# 其他
'rabbit128-poly1305', 'aegis-128l', 'aegis-256', 'aez-384', 'deoxys-ii-256-128', 'rc4-md5', 'none'
]
class ObfsPluginOpts(BaseModel):
mode: Literal['tls', 'http']
host: Optional[str] = Field(default="bing.com")
class V2rayPluginOpts(BaseModel):
mode: Literal['websocket'] = 'websocket'
host: Optional[str] = Field(default="bing.com")
path: Optional[str] = None
tls: Optional[bool] = False
fingerprint: Optional[str] = None
headers: Optional[Dict[str, str]] = None
skip_cert_verify: Optional[bool] = Field(False, alias='skip-cert-verify')
mux: Optional[bool] = True
v2ray_http_upgrade: Optional[bool] = Field(False, alias='v2ray-http-upgrade')
v2ray_http_upgrade_fast_open: Optional[bool] = Field(False, alias='v2ray-http-upgrade-fast-open')
class GostPluginOpts(BaseModel):
mode: Literal['websocket'] = 'websocket'
host: Optional[str] = Field(default="bing.com")
path: Optional[str] = None
tls: Optional[bool] = False
fingerprint: Optional[str] = None
headers: Optional[Dict[str, str]] = None
skip_cert_verify: Optional[bool] = Field(False, alias='skip-cert-verify')
mux: Optional[bool] = True
class ShadowTlsPluginOpts(BaseModel):
password: Optional[str] = None
host: str
fingerprint: Optional[str] = None
skip_cert_verify: Optional[bool] = Field(False, alias='skip-cert-verify')
version: Optional[Literal[1, 2, 3]] = 2
alpn: Optional[List[str]] = None
class RestlsPluginOpts(BaseModel):
password: str
host: str
version_hint: str = Field(alias='version-hint')
restls_script: Optional[str] = Field(None, alias='restls-script')
class ShadowsocksProxy(ProxyBase):
type: Literal['ss'] = 'ss'
cipher: ShadowsocksCipherType
password: str
udp_over_tcp: Optional[bool] = Field(None, alias='udp-over-tcp')
udp_over_tcp_version: Optional[Literal[1, 2]] = Field(1, alias='udp-over-tcp-version')
client_fingerprint: Optional[Literal['chrome', 'ios', 'firefox', 'safari']] = Field(None,
alias='client-fingerprint')
plugin: Optional[Literal['obfs', 'v2ray-plugin', 'shadow-tls', 'restls', 'gost-plugin']] = None
plugin_opts: Optional[Union[
ObfsPluginOpts,
V2rayPluginOpts,
GostPluginOpts,
ShadowTlsPluginOpts,
RestlsPluginOpts,
]] = Field(None, alias='plugin-opts')
@field_validator("plugin_opts")
@classmethod
def validate_plugin_opts(cls, v, info: ValidationInfo):
plugin = info.data.get("plugin")
if plugin and v:
if not isinstance(plugin, str):
raise ValueError("plugin must be a string")
plugin_model_map = {
"obfs": "ObfsPluginOpts",
"v2ray-plugin": "V2rayPluginOpts",
"gost-plugin": "GostPluginOpts",
"shadow-tls": "ShadowTlsPluginOpts",
"restls": "RestlsPluginOpts",
}
expected_model = plugin_model_map.get(plugin)
if expected_model and v.__class__.__name__ != expected_model:
raise ValueError(f"{plugin} plugin requires {expected_model}")
return v

View File

@@ -0,0 +1,14 @@
from pydantic import Field
from typing import Optional, Literal
from .proxybase import ProxyBase
class ShadowsocksRProxy(ProxyBase):
type: Literal['ssr'] = 'ssr'
cipher: str
password: str
obfs: Literal['plain', 'http_simple', 'http_post', 'random_head', 'tls1.2_ticket_auth', 'tls1.2_ticket_fastauth']
obfs_param: Optional[str] = Field(None, alias='obfs-param')
protocol: Literal['origin', 'auth_sha1_v4', 'auth_aes128_md5', 'auth_aes128_sha1', 'auth_chain_a', 'auth_chain_b']
protocol_param: Optional[str] = Field(None, alias='protocol-param')

View File

@@ -0,0 +1,17 @@
from typing import Optional, Literal
from pydantic import BaseModel, Field
from .proxybase import ProxyBase
class SnellObfsOpts(BaseModel):
mode: Optional[Literal['http', 'tls']] = None
host: Optional[str] = None
class SnellProxy(ProxyBase):
type: Literal['snell'] = 'snell'
psk: str
version: Optional[Literal[1,2,3]] = 1
obfs_opts: Optional[SnellObfsOpts] = Field(None, alias='obfs-opts')

View File

@@ -0,0 +1,10 @@
from typing import Optional, Literal
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
class Socks5Proxy(ProxyBase, TLSMixin):
type: Literal['socks5'] = 'socks5'
username: Optional[str] = None
password: Optional[str] = None

View File

@@ -0,0 +1,15 @@
from typing import List, Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
class SshProxy(ProxyBase):
type: Literal['ssh'] = 'ssh'
username: str
password: Optional[str] = None
private_key: Optional[str] = Field(None, alias='privateKey')
private_key_passphrase: Optional[str] = Field(None, alias='private-key-passphrase')
host_key: Optional[List[str]] = Field(None, alias='host-key')
host_key_algorithms: Optional[List[str]] = Field(None, alias='host-key-algorithms')

View File

@@ -0,0 +1,28 @@
from typing import List, Optional, Literal
from pydantic import BaseModel, Field
class RealityOpts(BaseModel):
public_key: str = Field(..., alias='public-key')
short_id: Optional[str] = Field(None, alias='short-id')
support_x25519mlkem768: Optional[bool] = Field(None, alias='support-x25519mlkem768')
class EchOpts(BaseModel):
enable: bool = False
config: str
class TLSMixin(BaseModel):
"""TLS 配置混入类"""
# TLS settings
tls: Optional[bool] = None
sni: Optional[str] = None
servername: Optional[str] = None
fingerprint: Optional[str] = None
alpn: Optional[List[str]] = None
skip_cert_verify: Optional[bool] = Field(None, alias='skip-cert-verify')
client_fingerprint: Optional[Literal['chrome', 'firefox', 'safari', 'ios', 'android', 'edge', '360', 'qq', 'random']] = Field(None, alias='client-fingerprint')
reality_opts: Optional[RealityOpts] = Field(None, alias='reality-opts')
ech_opts: Optional[EchOpts] = Field(None, alias='ech-opts')

View File

@@ -0,0 +1,21 @@
from typing import Optional, Literal
from pydantic import BaseModel, Field
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
from .networkmixin import NetworkMixin
class TrojanSSOption(BaseModel):
enabled: Optional[bool] = None
method: Optional[Literal['aes-128-gcm', 'aes-256-gcm', 'chacha20-ietf-poly1305']] = None
password: Optional[str] = None
class TrojanProxy(ProxyBase, TLSMixin, NetworkMixin):
type: Literal['trojan'] = 'trojan'
password: str
ss_opts: Optional[TrojanSSOption] = Field(None, alias='ss-opts')
network: Optional[Literal['tcp', 'grpc', 'ws']] = None
tls: Optional[bool] = True

View File

@@ -0,0 +1,41 @@
from typing import Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
class TuicProxy(ProxyBase, TLSMixin):
type: Literal['tuic'] = 'tuic'
# TUIC v4/v5 认证
token: Optional[str] = None
uuid: Optional[str] = None
password: Optional[str] = None
# 连接配置
ip: Optional[str] = None
heartbeat_interval: Optional[int] = Field(None, alias='heartbeat-interval')
reduce_rtt: Optional[bool] = Field(None, alias='reduce-rtt')
request_timeout: Optional[int] = Field(None, alias='request-timeout')
udp_relay_mode: Optional[Literal['native', 'quic']] = Field(None, alias='udp-relay-mode')
congestion_controller: Optional[Literal['cubic', 'new_reno', 'bbr']] = Field(None, alias='congestion-controller')
disable_sni: Optional[bool] = Field(None, alias='disable-sni')
max_udp_relay_packet_size: Optional[int] = Field(None, alias='max-udp-relay-packet-size')
# 性能配置
fast_open: Optional[bool] = Field(None, alias='fast-open')
max_open_streams: Optional[int] = Field(None, alias='max-open-streams')
cwnd: Optional[int] = None
recv_window_conn: Optional[int] = Field(None, alias='recv-window-conn')
recv_window: Optional[int] = Field(None, alias='recv-window')
disable_mtu_discovery: Optional[bool] = Field(None, alias='disable-mtu-discovery')
max_datagram_frame_size: Optional[int] = Field(None, alias='max-datagram-frame-size')
# TLS 证书配置
ca: Optional[str] = None
ca_str: Optional[str] = Field(None, alias='ca-str')
# UDP over Stream 扩展
udp_over_stream: Optional[bool] = Field(None, alias='udp-over-stream')
udp_over_stream_version: Optional[int] = Field(None, alias='udp-over-stream-version')

View File

@@ -0,0 +1,16 @@
from pydantic import Field
from typing import Optional, Literal
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
from .networkmixin import NetworkMixin
class VlessProxy(ProxyBase, TLSMixin, NetworkMixin):
type: Literal['vless'] = 'vless'
uuid: str
flow: Optional[str] = None
packet_addr: Optional[bool] = Field(None, alias='packet-addr')
xudp: Optional[bool] = None
packet_encoding: Optional[Literal['packetaddr', 'xudp']] = Field(None, alias='packet-encoding')
encryption: Optional[str] = None

View File

@@ -0,0 +1,18 @@
from typing import Optional, Literal
from pydantic import Field
from .proxybase import ProxyBase
from .tlsmixin import TLSMixin
from .networkmixin import NetworkMixin
class VmessProxy(ProxyBase, TLSMixin, NetworkMixin):
type: Literal['vmess'] = 'vmess'
uuid: str
alter_id: int = Field(0, alias='alterId')
cipher: Literal['auto', 'zero', 'aes-128-gcm', 'chacha20-poly1305', 'none'] = 'auto'
packet_addr: Optional[bool] = Field(None, alias='packet-addr')
xudp: Optional[bool] = None
packet_encoding: Optional[Literal['packetaddr', 'xudp']] = Field(None, alias='packet-encoding')
global_padding: Optional[bool] = Field(None, alias='global-padding')
authenticated_length: Optional[bool] = Field(None, alias='authenticated-length')

View File

@@ -0,0 +1,60 @@
from typing import List, Optional, Literal
from pydantic import BaseModel, Field
from .proxybase import ProxyBase
class WireGuardPeerOption(BaseModel):
server: str
port: int
public_key: str = Field(..., alias='public-key')
pre_shared_key: Optional[str] = Field(None, alias='pre-shared-key')
reserved: Optional[List[int]] = None
allowed_ips: Optional[List[str]] = Field(None, alias='allowed-ips')
class AmneziaWGOption(BaseModel):
jc: Optional[int] = None
jmin: Optional[int] = None
jmax: Optional[int] = None
s1: Optional[int] = None
s2: Optional[int] = None
h1: Optional[int] = None
h2: Optional[int] = None
h3: Optional[int] = None
h4: Optional[int] = None
# AmneziaWG v1.5
i1: Optional[str] = None
i2: Optional[str] = None
i3: Optional[str] = None
i4: Optional[str] = None
i5: Optional[str] = None
j1: Optional[str] = None
j2: Optional[str] = None
j3: Optional[str] = None
itime: Optional[int] = None
class WireGuardProxy(ProxyBase):
type: Literal['wireguard'] = 'wireguard'
ip: Optional[str] = None
ipv6: Optional[str] = None
private_key: str = Field(..., alias='private-key')
public_key: str = Field(..., alias='public-key')
pre_shared_key: Optional[str] = Field(None, alias='pre-shared-key')
reserved: Optional[List[int]] = None
workers: Optional[int] = None
mtu: Optional[int] = None
persistent_keepalive: Optional[int] = Field(None, alias='persistent-keepalive')
# 多 peer 配置
peers: Optional[List[WireGuardPeerOption]] = None
# DNS 配置
remote_dns_resolve: Optional[bool] = Field(None, alias='remote-dns-resolve')
dns: Optional[List[str]] = None
refresh_server_ip_interval: Optional[int] = Field(None, alias='refresh-server-ip-interval')
# AmneziaWG 扩展
amnezia_wg_option: Optional[AmneziaWGOption] = Field(None, alias='amnezia-wg-option')

View File

@@ -0,0 +1,119 @@
import re
from typing import List, Optional, Union, Literal
from pydantic import BaseModel, Field, field_validator, RootModel
class ProxyGroupBase(BaseModel):
"""
包含所有代理组类型共有的通用字段。
"""
# Required field
name: str = Field(..., description="The name of the proxy group.")
# Proxy and provider references
proxies: Optional[List[str]] = Field(None, description="References to outbound proxies or other proxy groups.")
use: Optional[List[str]] = Field(None, description="References to proxy provider sets.")
# Health check fields
url: Optional[str] = Field(None, description="Health check test address.")
interval: Optional[int] = Field(None, description="Health check interval in seconds.")
lazy: Optional[bool] = Field(True, description="If not selected, no health checks are performed.")
timeout: Optional[int] = Field(None, description="Health check timeout in milliseconds.")
max_failed_times: Optional[int] = Field(5, description="Maximum number of failures before a forced health check.",
alias="max-failed-times")
expected_status: Optional[str] = Field('*',
description="Expected HTTP response status code for health checks.",
alias="expected-status")
# Network and routing fields
disable_udp: Optional[bool] = Field(False, description="Disables UDP for this proxy group.", alias="disable-udp")
interface_name: Optional[str] = Field(None, description="DEPRECATED. Specifies the outbound interface.",
alias="interface-name")
routing_mark: Optional[int] = Field(None, description="DEPRECATED. The routing mark for outbound connections.",
alias="routing-mark")
# Dynamic proxy inclusion
include_all: Optional[bool] = Field(False, description="Includes all outbound proxies and proxy sets.",
alias="include-all")
include_all_proxies: Optional[bool] = Field(False, description="Includes all outbound proxies.",
alias="include-all-proxies")
include_all_providers: Optional[bool] = Field(False, description="Includes all proxy provider sets.",
alias="include-all-providers")
# Filtering
filter: Optional[str] = Field(None, description="Regex to filter nodes from providers.")
exclude_filter: Optional[str] = Field(None, description="Regex to exclude nodes.", alias="exclude-filter")
exclude_type: Optional[str] = Field(None, description="Exclude nodes by adapter type, separated by '|'.",
alias="exclude-type")
# UI fields
hidden: Optional[bool] = Field(False, description="Hides the proxy group in the API.")
icon: Optional[str] = Field(None, description="Icon string for the proxy group, for UI use.")
@field_validator('expected_status')
@classmethod
def validate_expected_status(cls, v: Optional[str]) -> Optional[str]:
if v is None or v == '*':
return v
pattern = re.compile(r'^\d{3}([-/]\d{3})*$')
if not pattern.match(v):
raise ValueError("Invalid format for expected-status.")
parts = re.split(r'[/]', v)
for part in parts:
if '-' in part:
start, end = part.split('-')
if not (start.isdigit() and end.isdigit() and 100 <= int(start) < 600 and 100 <= int(end) < 600 and int(
start) <= int(end)):
raise ValueError(f"Invalid status code range: {part}")
elif not (part.isdigit() and 100 <= int(part) < 600):
raise ValueError(f"Invalid status code: {part}")
return v
class SelectGroup(ProxyGroupBase):
type: Literal['select']
class RelayGroup(ProxyGroupBase):
type: Literal['relay']
class FallbackGroup(ProxyGroupBase):
type: Literal['fallback']
class UrlTestGroup(ProxyGroupBase):
type: Literal['url-test']
tolerance: Optional[int] = Field(None, description="proxies switch tolerance, measured in milliseconds (ms).")
class LoadBalanceGroup(ProxyGroupBase):
type: Literal['load-balance']
strategy: Optional[Literal['round-robin', 'consistent-hashing', 'sticky-sessions']] = Field(
'round-robin',
description="Load balancing strategy."
)
class SmartGroup(ProxyGroupBase):
type: Literal['smart']
uselightgbm: bool = Field(..., description="Use LightGBM model predict weight.")
collectdata: bool = Field(..., description="Collect datas for model training.")
policy_priority: Optional[str] = Field("1",
description="<1 means lower priority, >1 means higher priority, "
"the default is 1, pattern support regex and string.",
alias="policy-priority")
strategy: Optional[Literal['round-robin', 'sticky-sessions']] = Field(
'sticky-sessions',
description="Load balancing strategy."
)
sample_rate: Optional[int] = Field(1, description="Data acquisition rate.", alias="sample-rate")
# Discriminated Union
ProxyGroupType = Union[SelectGroup, RelayGroup, FallbackGroup, UrlTestGroup, LoadBalanceGroup, SmartGroup]
class ProxyGroup(RootModel[ProxyGroupType]):
root: ProxyGroupType = Field(..., discriminator='type')

View File

@@ -0,0 +1,195 @@
from enum import Enum
from typing import Any, List, Optional, Union, Dict, Literal
from pydantic import BaseModel, field_validator, ValidationInfo
class AdditionalParam(Enum):
NO_RESOLVE = 'no-resolve'
SRC = 'src'
class RoutingRuleType(Enum):
"""Enumeration of all supported Clash rule types"""
DOMAIN = "DOMAIN"
DOMAIN_SUFFIX = "DOMAIN-SUFFIX"
DOMAIN_KEYWORD = "DOMAIN-KEYWORD"
DOMAIN_REGEX = "DOMAIN-REGEX"
DOMAIN_WILDCARD = "DOMAIN-WILDCARD"
GEOSITE = "GEOSITE"
GEOIP = "GEOIP"
IP_CIDR = "IP-CIDR"
IP_CIDR6 = "IP-CIDR6"
IP_SUFFIX = "IP-SUFFIX"
IP_ASN = "IP-ASN"
SRC_GEOIP = "SRC-GEOIP"
SRC_IP_ASN = "SRC-IP-ASN"
SRC_IP_CIDR = "SRC-IP-CIDR"
SRC_IP_SUFFIX = "SRC-IP-SUFFIX"
DST_PORT = "DST-PORT"
SRC_PORT = "SRC-PORT"
IN_PORT = "IN-PORT"
IN_TYPE = "IN-TYPE"
IN_USER = "IN-USER"
IN_NAME = "IN-NAME"
PROCESS_PATH = "PROCESS-PATH"
PROCESS_PATH_REGEX = "PROCESS-PATH-REGEX"
PROCESS_NAME = "PROCESS-NAME"
PROCESS_NAME_REGEX = "PROCESS-NAME-REGEX"
UID = "UID"
NETWORK = "NETWORK"
DSCP = "DSCP"
RULE_SET = "RULE-SET"
AND = "AND"
OR = "OR"
NOT = "NOT"
SUB_RULE = "SUB-RULE"
MATCH = "MATCH"
class Action(Enum):
"""Enumeration of rule actions"""
DIRECT = "DIRECT"
REJECT = "REJECT"
REJECT_DROP = "REJECT-DROP"
PASS = "PASS"
COMPATIBLE = "COMPATIBLE"
def __str__(self) -> str:
return self.value
class RuleBase(BaseModel):
rule_type: RoutingRuleType
action: Union[Action, str] # Can be Action enum or custom proxy group name
raw_rule: str
def to_dict(self) -> Dict[str, Any]:
pass
def __str__(self) -> str:
pass
def __eq__(self, other: 'RuleBase') -> bool:
if not isinstance(other, RuleBase):
return NotImplemented
return self.__str__() == other.__str__()
class ClashRule(RuleBase):
"""Represents a parsed Clash routing rule"""
rule_type: RoutingRuleType
payload: str
additional_params: Optional[AdditionalParam] = None
def condition_string(self) -> str:
return f"{self.rule_type.value},{self.payload}"
def to_dict(self) -> Dict[str, Any]:
return {
'type': self.rule_type.value,
'payload': self.payload,
'action': self.action.value if isinstance(self.action, Action) else self.action,
'additional_params': self.additional_params.value if self.additional_params else None,
'raw': self.raw_rule
}
def __str__(self) -> str:
rule_str = f"{self.condition_string()},{self.action}"
if self.additional_params:
rule_str += f",{self.additional_params.value}"
return rule_str
@field_validator('payload', mode='after')
@classmethod
def validate_payload(cls, v: Optional[str], info: ValidationInfo) -> Optional[str]:
# 获取其他字段的值
rule_type = info.data['rule_type']
if rule_type == RoutingRuleType.NETWORK and v is not None and v.upper() not in ('TCP', 'UDP'):
raise ValueError('Payload must be TCP or UDP')
return v
class LogicRule(RuleBase):
"""Represents a logic rule (AND, OR, NOT)"""
rule_type: Literal[RoutingRuleType.AND, RoutingRuleType.OR, RoutingRuleType.NOT]
conditions: List[Union[ClashRule, 'LogicRule']]
def condition_string(self) -> str:
conditions_str = ','.join([f"({c.condition_string()})" for c in self.conditions])
return f"{self.rule_type.value},({conditions_str})"
def to_dict(self) -> Dict[str, Any]:
conditions = []
for condition in self.conditions:
conditions.append(condition.condition_string())
return {
'type': self.rule_type.value,
'conditions': conditions,
'action': self.action.value if isinstance(self.action, Action) else self.action,
'raw': self.raw_rule
}
@field_validator('conditions', mode='after')
@classmethod
def validate_conditions(cls, v: List[Union[ClashRule, 'LogicRule']]) -> List[Union[ClashRule, 'LogicRule']]:
if not v:
raise ValueError('A condition list must be provided')
return v
def __str__(self) -> str:
return f"{self.condition_string()},{self.action}"
class SubRule(RuleBase):
rule_type: Literal[RoutingRuleType.SUB_RULE] = RoutingRuleType.SUB_RULE
condition: Union[ClashRule, LogicRule]
action: str
def condition_string(self) -> str:
return f"{self.rule_type.value},({self.condition.condition_string()})"
def to_dict(self) -> Dict[str, Any]:
return {
'type': self.rule_type.value,
'condition': f"({self.condition.condition_string()})",
'action': self.action,
'raw': self.raw_rule
}
def __str__(self) -> str:
return f"{self.condition_string()},{self.action}"
class MatchRule(RuleBase):
"""Represents a match rule"""
rule_type: Literal[RoutingRuleType.MATCH] = RoutingRuleType.MATCH
@staticmethod
def condition_string() -> str:
return "MATCH"
def to_dict(self) -> Dict[str, str]:
return {
'type': 'MATCH',
'action': self.action.value if isinstance(self.action, Action) else self.action,
'raw': self.raw_rule
}
def __str__(self) -> str:
return f"{self.condition_string()},{self.action}"
RuleType = Union[ClashRule, LogicRule, SubRule, MatchRule]

View File

@@ -0,0 +1,59 @@
from typing import List, Optional, Literal, Dict
from pydantic import BaseModel, Field, model_validator, HttpUrl, RootModel
class RuleProvider(BaseModel):
type: Literal["http", "file", "inline"] = Field(..., description="Provider type")
url: Optional[HttpUrl] = Field(None, description="Must be configured if the type is http")
path: Optional[str] = Field(None, description="Optional, file path, must be unique.")
interval: Optional[int] = Field(None, ge=0, description="The update interval for the provider, in seconds.")
proxy: Optional[str] = Field(None, description="Download/update through the specified proxy.")
behavior: Optional[Literal["domain", "ipcidr", "classical"]] = Field(None,
description="Behavior of the rule provider")
format: Literal["yaml", "text", "mrs"] = Field("yaml", description="Format of the rule provider file")
size_limit: int = Field(0, ge=0, description="The maximum size of downloadable files in bytes (0 for no limit)",
alias="size-limit")
payload: Optional[List[str]] = Field(None, description="Content, only effective when type is inline")
@model_validator(mode="before")
@classmethod
def validate_type_relationships(cls, values):
"""Perform cross-field validation before the model is created."""
type_ = values.get('type')
url = values.get('url')
path = values.get('path')
payload = values.get('payload')
format_ = values.get('format', 'yaml')
behavior = values.get('behavior')
# url check
if type_ == "http" and url is None:
raise ValueError("url must be configured if the type is 'http'")
if type_ != "http" and 'url' in values:
values['url'] = None
# path check
if type_ == "file" and path is None:
raise ValueError("path must be configured if the type is 'file'")
if type_ != "file" and 'path' in values:
values['path'] = None
# payload handling
if type_ == "inline":
if payload is None:
raise ValueError("payload must be configured if the type is 'inline'")
if not isinstance(payload, list):
raise ValueError("payload must be a list of strings when type is 'inline'")
elif 'payload' in values:
values['payload'] = None
# format-behavior rule
if format_ == "mrs" and behavior not in {"domain", "ipcidr"}:
raise ValueError("mrs format only supports 'domain' or 'ipcidr' behavior")
return values
class RuleProviders(RootModel[Dict[str, RuleProvider]]):
root: Dict[str, RuleProvider]

View File

@@ -1,2 +1,3 @@
websockets
sse_starlette~=2.3.6
websockets
sse_starlette~=2.3.6
PyYAML~=6.0.2

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,33 @@
from dataclasses import dataclass, field
from typing import Any, Dict, List
from .helper.clashrulemanager import ClashRuleManager
from .helper.proxiesmanager import ProxyManager
@dataclass
class PluginState:
"""
A dataclass to hold all the runtime state of the ClashRuleProvider plugin.
"""
# Rule and Proxy Managers
top_rules_manager: ClashRuleManager = field(default_factory=ClashRuleManager)
ruleset_rules_manager: ClashRuleManager = field(default_factory=ClashRuleManager)
proxies_manager: ProxyManager = field(default_factory=ProxyManager)
# Loaded from saved data
proxy_groups: List[Dict[str, Any]] = field(default_factory=list)
extra_proxies: List[Dict[str, Any]] = field(default_factory=list)
subscription_info: Dict[str, Any] = field(default_factory=dict)
rule_provider: Dict[str, Any] = field(default_factory=dict)
rule_providers: Dict[str, Any] = field(default_factory=dict)
ruleset_names: Dict[str, str] = field(default_factory=dict)
acl4ssr_providers: Dict[str, Any] = field(default_factory=dict)
clash_configs: Dict[str, Any] = field(default_factory=dict)
hosts: List[Dict[str, Any]] = field(default_factory=list)
overwritten_region_groups: Dict[str, Any] = field(default_factory=dict)
overwritten_proxies: Dict[str, Any] = field(default_factory=dict)
clash_template_dict: Dict[str, Any] = field(default_factory=dict)
# Volatile state (generated at runtime)
geo_rules: Dict[str, List[str]] = field(default_factory=lambda: {'geoip': [], 'geosite': []})

View File

@@ -0,0 +1,19 @@
from typing import Any, Optional
from app.db.plugindata_oper import PluginDataOper
class PluginStore:
"""数据持久化"""
def __init__(self, plugin_id: str):
self.plugin_id = plugin_id
self.plugin_data = PluginDataOper()
def get_data(self, key: Optional[str] = None) -> Any:
return self.plugin_data.get_data(self.plugin_id, key)
def save_data(self, key: str, value: Any):
self.plugin_data.save(self.plugin_id, key, value)
def del_data(self, key: str) -> Any:
self.plugin_data.del_data(self.plugin_id, key)

View File

@@ -28,7 +28,7 @@ class DownloadSiteTag(_PluginBase):
# 插件图标
plugin_icon = "Youtube-dl_B.png"
# 插件版本
plugin_version = "2.2"
plugin_version = "2.6"
# 插件作者
plugin_author = "叮叮当"
# 作者主页
@@ -55,12 +55,33 @@ class DownloadSiteTag(_PluginBase):
_enabled_media_tag = False
_enabled_tag = True
_enabled_category = False
_enabled_del_tags = False
_category_movie = None
_category_tv = None
_category_anime = None
_downloaders = None
# 默认的tracker映射字符串用于显示在界面上
_tracker_mappings_default = "\n".join([
"chdbits.xyz -> ptchdbits.co",
"agsvpt.trackers.work -> agsvpt.com",
"tracker.cinefiles.info -> audiences.me",
"# 格式说明tracker域名 -> 映射域名",
"# 使用 -> 作为分隔符",
"# 每行一个映射规则,空行和以#开头的行会被忽略",
"# 站点管理中必须存在对应的域名才能生效"
])
_tracker_mappings_str = ""
_tracker_mappings = {}
_del_tags_task_rid = {}
# 前缀配置
_site_prefix = ""
_media_prefix = ""
def init_plugin(self, config: dict = None):
# 初始化删除标签任务rid映射
self._del_tags_task_rid = {}
# 初始化默认的tracker映射
self._tracker_mappings = self._parse_tracker_mappings(self._tracker_mappings_default)
# 读取配置
if config:
self._enabled = config.get("enabled")
@@ -72,10 +93,29 @@ class DownloadSiteTag(_PluginBase):
self._enabled_media_tag = config.get("enabled_media_tag")
self._enabled_tag = config.get("enabled_tag")
self._enabled_category = config.get("enabled_category")
self._enabled_del_tags = config.get("enabled_del_tags")
self._category_movie = config.get("category_movie") or "电影"
self._category_tv = config.get("category_tv") or "电视"
self._category_anime = config.get("category_anime") or "动漫"
self._downloaders = config.get("downloaders")
self._tracker_mappings_str = config.get("tracker_mappings_str", "")
# 读取前缀配置
self._site_prefix = config.get("site_prefix", "")
self._media_prefix = config.get("media_prefix", "")
# 此设置对于老用户来说缺乏具体说明,因此如果为空,表示用户首次更新,则使用默认配置起到提示作用
if not ("tracker_mappings_str" in config):
config["tracker_mappings_str"] = self._tracker_mappings_default
self.update_config(config)
# 如果用户有配置,解析并合并到默认映射中
elif self._tracker_mappings_str:
user_mappings = self._parse_tracker_mappings(self._tracker_mappings_str)
# 将用户映射合并到默认映射中用户映射会覆盖默认映射中相同的key
self._tracker_mappings.update(user_mappings)
# 首次运行时从下载器初始化rid映射
if self._enabled_del_tags:
self._task_del_unused_tags()
# 停止现有任务
self.stop_service()
@@ -146,7 +186,20 @@ class DownloadSiteTag(_PluginBase):
"kwargs": {} # 定时器参数
}]
"""
# 初始化公共服务列表
tasks = []
if self._enabled:
if self._enabled_del_tags:
# 添加 删除所有未被任何种子使用的标签 任务 每5分钟执行一次
tasks.append({
"id": "DeleteUnusedTags",
"name": "删除下载器中未被使用的标签",
"trigger": "interval",
"func": self._task_del_unused_tags,
"kwargs": {
"minutes": 5
}
})
if self._interval == "计划任务" or self._interval == "固定间隔":
if self._interval == "固定间隔":
if self._interval_unit == "小时":
@@ -163,7 +216,7 @@ class DownloadSiteTag(_PluginBase):
if self._interval_time < 5:
self._interval_time = 5
logger.info(f"{self.LOG_TAG}启动定时服务: 最小不少于5分钟, 防止执行间隔太短任务冲突")
return [{
tasks.append({
"id": "DownloadSiteTag",
"name": "补全下载历史的标签与分类",
"trigger": "interval",
@@ -171,16 +224,16 @@ class DownloadSiteTag(_PluginBase):
"kwargs": {
"minutes": self._interval_time
}
}]
})
else:
return [{
tasks.append({
"id": "DownloadSiteTag",
"name": "补全下载历史的标签与分类",
"trigger": CronTrigger.from_crontab(self._interval_cron),
"func": self._complemented_history,
"kwargs": {}
}]
return []
})
return tasks
@staticmethod
def str_to_number(s: str, i: int) -> int:
@@ -188,10 +241,21 @@ class DownloadSiteTag(_PluginBase):
return int(s)
except ValueError:
return i
@staticmethod
def custom_intersection(indexers, tags):
"""
自定义交集算法, 用于处理标签与分类的交集(后缀匹配模式)
"""
return {
idx for idx in set(indexers)
for tag in set(tags)
if idx == tag or tag.endswith(idx)
}
def _complemented_history(self):
"""
补全下载历史的标签与分类
补全下载历史的标签与分类,且执行清理未使用的标签
"""
if not self.service_infos:
return
@@ -199,15 +263,10 @@ class DownloadSiteTag(_PluginBase):
# 记录处理的种子, 供辅种(无下载历史)使用
dispose_history = {}
# 所有站点索引
indexers = [indexer.get("name") for indexer in SitesHelper().get_indexers()]
indexers = [self._generate_site_tag(i.get("name")) if self._site_prefix else i.get("name") for i in SitesHelper().get_indexers()]
# JackettIndexers索引器支持多个站点, 如果不存在历史记录, 则通过tracker会再次附加其他站点名称
indexers.append("JackettIndexers")
indexers.append(self._generate_site_tag("JackettIndexers"))
indexers = set(indexers)
tracker_mappings = {
"chdbits.xyz": "ptchdbits.co",
"agsvpt.trackers.work": "agsvpt.com",
"tracker.cinefiles.info": "audiences.me",
}
for service in self.service_infos.values():
downloader = service.name
downloader_obj = service.instance
@@ -263,7 +322,7 @@ class DownloadSiteTag(_PluginBase):
trackers = self._get_trackers(torrent=torrent, dl_type=service.type)
for tracker in trackers:
# 检查tracker是否包含特定的关键字并进行相应的映射
for key, mapped_domain in tracker_mappings.items():
for key, mapped_domain in self._tracker_mappings.items():
if key in tracker:
domain = mapped_domain
break
@@ -279,12 +338,17 @@ class DownloadSiteTag(_PluginBase):
# 按设置生成需要写入的标签与分类
_tags = []
_cat = None
# 站点标签, 如果勾选开关的话 因允许torrent_site为空时运行到此, 因此需要判断torrent_site不为空
if self._enabled_tag and history.torrent_site:
_tags.append(history.torrent_site)
site_tag = self._generate_site_tag(history.torrent_site)
_tags.append(site_tag)
# 媒体标题标签, 如果勾选开关的话 因允许title为空时运行到此, 因此需要判断title不为空
if self._enabled_media_tag and history.title:
_tags.append(history.title)
media_tag = self._generate_media_tag(history.title)
_tags.append(media_tag)
# 分类, 如果勾选开关的话 <tr暂不支持> 因允许mtype为空时运行到此, 因此需要判断mtype不为空。为防止不必要的识别, 种子已经存在分类torrent_cat时 也不执行
if service.type == "qbittorrent" and self._enabled_category and not torrent_cat and history.type:
# 如果是电视剧 需要区分是否动漫
@@ -298,24 +362,92 @@ class DownloadSiteTag(_PluginBase):
genre_ids = tmdb_info.get("genre_ids")
_cat = self._genre_ids_get_cat(history.type, genre_ids)
# 识别并清理历史标签
tags_to_remove = []
if torrent_tags:
# 站点标签处理
if history.torrent_site:
# 如果配置了站点前缀
if self._site_prefix:
# 清理无前缀的站点标签
if history.torrent_site in torrent_tags:
tags_to_remove.append(history.torrent_site)
# 清理带旧前缀的站点标签(除了当前前缀)
for tag in torrent_tags:
if tag.endswith(history.torrent_site) and tag != f"{self._site_prefix}{history.torrent_site}":
tags_to_remove.append(tag)
# 如果没有配置站点前缀
else:
# 清理所有带前缀的站点标签
for tag in torrent_tags:
if tag.endswith(history.torrent_site) and tag != history.torrent_site:
tags_to_remove.append(tag)
# 剧名标签处理
if history.title:
# 如果配置了剧名前缀
if self._media_prefix:
# 清理无前缀的剧名标签
if history.title in torrent_tags:
tags_to_remove.append(history.title)
# 清理带旧前缀的剧名标签(除了当前前缀)
for tag in torrent_tags:
if tag.endswith(history.title) and tag != f"{self._media_prefix}{history.title}":
tags_to_remove.append(tag)
# 如果没有配置剧名前缀
else:
# 清理所有带前缀的剧名标签
for tag in torrent_tags:
if tag.endswith(history.title) and tag != history.title:
tags_to_remove.append(tag)
# 去除种子已经存在的标签
if _tags and torrent_tags:
_tags = list(set(_tags) - set(torrent_tags))
# 如果分类一样, 那么不需要修改
if _cat == torrent_cat:
_cat = None
# 判断当前种子是否不需要修改
if not _cat and not _tags:
if not _cat and not _tags and not tags_to_remove:
continue
# 执行通用方法, 设置种子标签与分类
self._set_torrent_info(service=service, _hash=_hash, _torrent=torrent, _tags=_tags, _cat=_cat,
_original_tags=torrent_tags)
_original_tags=torrent_tags, _tags_to_remove=tags_to_remove)
except Exception as e:
logger.error(
f"{self.LOG_TAG}分析种子信息时发生了错误: {str(e)}")
# 执行清理未使用标签
if self._enabled_del_tags:
self._del_unused_tags(service=service)
logger.info(f"{self.LOG_TAG}执行完成")
def _generate_site_tag(self, site_name):
"""
生成带前缀的站点标签
"""
if not site_name:
return ""
if self._site_prefix:
return f"{self._site_prefix}{site_name}"
else:
return site_name
def _generate_media_tag(self, media_title):
"""
生成带前缀的剧名标签
"""
if not media_title:
return ""
if self._media_prefix:
return f"{self._media_prefix}{media_title}"
else:
return media_title
def _genre_ids_get_cat(self, mtype, genre_ids=None):
"""
根据genre_ids判断是否<动漫>分类
@@ -335,6 +467,47 @@ class DownloadSiteTag(_PluginBase):
_cat = self._category_tv
return _cat
@staticmethod
def _parse_tracker_mappings(mapping_str: str) -> dict:
"""
解析tracker映射规则字符串为字典
格式tracker域名 -> 映射域名
例如chdbits.xyz -> ptchdbits.co
使用"->"作为分隔符
"""
tracker_mappings = {}
if not mapping_str:
return tracker_mappings
lines = mapping_str.strip().split('\n')
for line in lines:
line = line.strip()
if not line or line.startswith('#'):
continue # 跳过空行和注释行
# 支持多种分隔符
separators = ['->', '', ':', '']
separator = None
for sep in separators:
if sep in line:
separator = sep
break
if separator:
parts = line.split(separator, 1)
if len(parts) == 2:
key = parts[0].strip()
value = parts[1].strip()
if key and value:
tracker_mappings[key] = value
else:
# 如果没有找到分隔符,尝试按空格分割
parts = line.split()
if len(parts) >= 2:
tracker_mappings[parts[0].strip()] = parts[1].strip()
return tracker_mappings
@staticmethod
def _torrent_key(torrent: Any, dl_type: str) -> Optional[Tuple[int, str]]:
"""
@@ -423,8 +596,12 @@ class DownloadSiteTag(_PluginBase):
获取种子标签
"""
try:
return [str(tag).strip() for tag in torrent.get("tags", "").split(',')] \
if dl_type == "qbittorrent" else torrent.labels or []
if dl_type == "qbittorrent":
tags_str = torrent.get("tags", "")
# 处理空字符串情况,并过滤掉空白标签
return [tag.strip() for tag in tags_str.split(',') if tag.strip()] if tags_str else []
else:
return torrent.labels or []
except Exception as e:
print(str(e))
return []
@@ -441,7 +618,7 @@ class DownloadSiteTag(_PluginBase):
return None
def _set_torrent_info(self, service: ServiceInfo, _hash: str, _torrent: Any = None, _tags=None, _cat: str = None,
_original_tags: list = None):
_original_tags: list = None, _tags_to_remove: list = []):
"""
设置种子标签与分类
"""
@@ -463,7 +640,10 @@ class DownloadSiteTag(_PluginBase):
if _hash and _torrent:
# 下载器api不通用, 因此需分开处理
if service.type == "qbittorrent":
# 设置标签
# 先移除需要删除的标签
if _tags_to_remove:
downloader_obj.remove_torrents_tag(ids=_hash, tag=_tags_to_remove)
# 再添加新标签
if _tags:
downloader_obj.set_torrents_tag(ids=_hash, tags=_tags)
# 设置分类 <tr暂不支持>
@@ -478,16 +658,102 @@ class DownloadSiteTag(_PluginBase):
_torrent.setCategory(category=_cat)
else:
# 设置标签
if _tags:
if _tags or _tags_to_remove:
# _original_tags = None表示未指定, 因此需要获取原始标签
if _original_tags is None:
_original_tags = self._get_label(torrent=_torrent, dl_type=service.type)
# 如果原始标签不是空的, 那么合并原始标签
if _original_tags:
_tags = list(set(_original_tags).union(set(_tags)))
# 移除需要删除的标签
if _tags_to_remove:
_tags = list(set(_tags) - set(_tags_to_remove))
downloader_obj.set_torrent_tag(ids=_hash, tags=_tags)
logger.warn(
f"{self.LOG_TAG}下载器: {service.name} 种子id: {_hash} {(' 标签: ' + ','.join(_tags)) if _tags else ''} {(' 分类: ' + _cat) if _cat else ''}")
def _task_del_unused_tags(self):
"""
公共服务:删除所有未被任何种子使用的标签,遍历全部下载器
"""
if not self.service_infos:
return
for service in self.service_infos.values():
# 仅qb支持删除未使用标签
if service.type != "qbittorrent":
continue
downloader = service.name
downloader_obj = service.instance
if not downloader_obj:
logger.error(f"{self.LOG_TAG} 删除未使用标签公共服务,获取下载器失败 {downloader}")
continue
try:
# 初始化下载器 获取全量数据
if downloader not in self._del_tags_task_rid:
data = downloader_obj.qbc.sync_maindata(rid=0)
logger.info(f"{self.LOG_TAG}初始化删除未使用标签任务 RID for {downloader} full_update: {data.get('full_update', False)}")
self._del_tags_task_rid[downloader] = data.get("rid", 0)
else:
# 提取上次返回的 rid
last_rid = self._del_tags_task_rid[downloader]
data = downloader_obj.qbc.sync_maindata(rid=last_rid)
# 更新 rid 用于下次访问
self._del_tags_task_rid[downloader] = data.get("rid", last_rid)
# 可能服务器重启,或其他原因导致 rid 状态已被重置
if data.get("full_update", False):
logger.info(f"{self.LOG_TAG}重置删除未使用标签任务 RID for {downloader} full_update: {data.get('full_update', False)}")
continue
if data.get('torrents_removed', []):
logger.info(f"{self.LOG_TAG}删除未使用标签任务 RID for {downloader} 发现删除种子,即将执行清理未使用标签操作!")
# 指定下载器服务,执行删除未使用标签
self._del_unused_tags(service=service)
except Exception as e:
logger.error(
f"{self.LOG_TAG}删除未使用标签公共服务,下载器:{downloader} 发生了错误: {str(e)}")
def _del_unused_tags(self, service: ServiceInfo, torrents: Any = None):
"""
删除所有未被任何种子使用的标签, 可指定下载器与种子列表
"""
# 只有qb下载器才需要删除未使用的标签TR下载器未使用标签会自动移除
if not service or not service.instance or service.type != "qbittorrent":
return
downloader_obj = service.instance
try:
# 获取所有现有的标签 调用内部qbc的API
all_tags = downloader_obj.qbc.torrents_tags()
if not all_tags:
logger.info(
f"{self.LOG_TAG}下载器: {service.name} 当前没有任何标签,跳过删除未使用标签操作")
return
# 获取下载器中的种子
if not torrents:
torrents, error = downloader_obj.get_torrents()
# 如果下载器获取种子发生错误 或 没有种子 则跳过
if error or not torrents:
logger.warn(
f"{self.LOG_TAG}删除所有未被任何种子使用的标签时发生了错误或查询不到任何种子!")
return
logger.info(
f"{self.LOG_TAG}删除所有未被任何种子使用的标签: {service.name} 查询到 {len(torrents)} 个种子")
# 收集所有正在被使用的标签
used_tags_set = set()
for torrent in torrents:
tag = self._get_label(torrent=torrent, dl_type=service.type)
if tag: # 确保种子有标签
used_tags_set.update(tag)
# 计算未使用的标签(在全部标签中但不在使用集合中)
unused_tags = [tag for tag in all_tags if tag not in used_tags_set]
# 删除未使用的标签
if unused_tags:
downloader_obj.delete_torrents_tag(ids=None, tag=unused_tags)
logger.info(
f"{self.LOG_TAG}删除所有未被任何种子使用的标签: {",".join(unused_tags)}")
except Exception as e:
logger.error(
f"{self.LOG_TAG}删除所有未被任何种子使用的标签时发生了错误: {str(e)}")
@eventmanager.register(EventType.DownloadAdded)
def download_added(self, event: Event):
@@ -510,7 +776,7 @@ class DownloadSiteTag(_PluginBase):
if not service:
logger.info(f"触发添加下载事件,但没有监听下载器 {downloader},跳过后续处理")
return
context: Context = event.event_data.get("context")
_hash = event.event_data.get("hash")
_torrent = context.torrent_info
@@ -519,10 +785,12 @@ class DownloadSiteTag(_PluginBase):
_cat = None
# 站点标签, 如果勾选开关的话
if self._enabled_tag and _torrent.site_name:
_tags.append(_torrent.site_name)
site_tag = self._generate_site_tag(_torrent.site_name)
_tags.append(site_tag)
# 媒体标题标签, 如果勾选开关的话
if self._enabled_media_tag and _media.title:
_tags.append(_media.title)
media_tag = self._generate_media_tag(_media.title)
_tags.append(media_tag)
# 分类, 如果勾选开关的话 <tr暂不支持>
if self._enabled_category and _media.type:
_cat = self._genre_ids_get_cat(_media.type, _media.genre_ids)
@@ -531,12 +799,13 @@ class DownloadSiteTag(_PluginBase):
self._set_torrent_info(service=service, _hash=_hash, _tags=_tags, _cat=_cat)
except Exception as e:
logger.error(
f"{self.LOG_TAG}分析下载事件时发生了错误: {str(e)}")
f"{self.LOG_TAG}分析添加下载事件时发生了错误: {str(e)}")
def get_form(self) -> Tuple[List[dict], Dict[str, Any]]:
"""
拼装插件配置页面需要返回两块数据1、页面配置2、数据结构
"""
return [
{
'component': 'VForm',
@@ -616,7 +885,8 @@ class DownloadSiteTag(_PluginBase):
{
'component': 'VCol',
'props': {
'cols': 12
'cols': 12,
'md': 6
},
'content': [
{
@@ -627,6 +897,61 @@ class DownloadSiteTag(_PluginBase):
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 3
},
'content': [
{
'component': 'VCheckboxBtn',
'props': {
'model': 'enabled_del_tags',
'label': '自动删除未使用标签',
}
}
]
},
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VTextField',
'props': {
'model': 'site_prefix',
'label': '站点标签前缀',
'placeholder': '留空表示不使用前缀,自动识别历史标签并更新',
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VTextField',
'props': {
'model': 'media_prefix',
'label': '剧名标签前缀',
'placeholder': '留空表示不使用前缀,自动识别历史标签并更新',
}
}
]
}
]
},
@@ -735,6 +1060,27 @@ class DownloadSiteTag(_PluginBase):
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'info',
'variant': 'tonal',
'text': '定时任务:支持两种定时方式,主要针对辅种刷流等种子补全站点信息。如没有对应的需求建议切换为禁用。'
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
@@ -749,7 +1095,8 @@ class DownloadSiteTag(_PluginBase):
'props': {
'model': 'category_movie',
'label': '电影分类名称(默认: 电影)',
'placeholder': '电影'
'placeholder': '电影',
'hint': '请填写下载器里已创建的电影分类名称'
}
}
]
@@ -770,7 +1117,8 @@ class DownloadSiteTag(_PluginBase):
'props': {
'model': 'category_tv',
'label': '电视分类名称(默认: 电视)',
'placeholder': '电视'
'placeholder': '电视',
'hint': '请填写下载器里已创建的电视分类名称'
}
}
]
@@ -791,7 +1139,8 @@ class DownloadSiteTag(_PluginBase):
'props': {
'model': 'category_anime',
'label': '动漫分类名称(默认: 动漫)',
'placeholder': '动漫'
'placeholder': '动漫',
'hint': '请填写下载器里已创建的动漫分类名称'
}
}
]
@@ -804,7 +1153,7 @@ class DownloadSiteTag(_PluginBase):
{
'component': 'VCol',
'props': {
'cols': 12,
'cols': 12
},
'content': [
{
@@ -812,7 +1161,30 @@ class DownloadSiteTag(_PluginBase):
'props': {
'type': 'info',
'variant': 'tonal',
'text': '定时任务:支持两种定时方式,主要针对辅种刷流等种子补全站点信息。如没有对应的需求建议切换为禁用'
'text': '以下为tracker映射规则您可以根据需要修改或添加新的规则'
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12
},
'content': [
{
'component': 'VTextarea',
'props': {
'model': 'tracker_mappings_str',
'label': 'Tracker域名映射规则',
'rows': 8,
'placeholder': '每行一个映射格式tracker域名 -> 映射域名\n例如chdbits.xyz -> ptchdbits.co',
'hint': '支持的分隔符:->, →, :, :,空格'
}
}
]
@@ -827,13 +1199,17 @@ class DownloadSiteTag(_PluginBase):
"enabled_tag": True,
"enabled_media_tag": False,
"enabled_category": False,
"enabled_del_tags": False,
"category_movie": "电影",
"category_tv": "电视",
"category_anime": "动漫",
"interval": "计划任务",
"interval_cron": "5 4 * * *",
"interval_time": "6",
"interval_unit": "小时"
"interval_unit": "小时",
"tracker_mappings_str": self._tracker_mappings_default, # 添加默认的映射规则字符串
"site_prefix": "",
"media_prefix": ""
}
def get_page(self) -> List[dict]:
@@ -852,4 +1228,4 @@ class DownloadSiteTag(_PluginBase):
self._event.clear()
self._scheduler = None
except Exception as e:
print(str(e))
print(str(e))

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,771 @@
from typing import Any, AsyncGenerator, Dict, Generator, List, Optional, Final
import requests
import httpx
from app.core.cache import cached
from app.log import logger
from app.utils.common import retry
from app.utils.http import RequestUtils, AsyncRequestUtils
from .schema.imdbapi import ImdbApiTitle, ImdbApiEpisode, ImdbApiCredit, ImdbapiImage
from .schema.imdbapi import (ImdbApiSearchTitlesResponse, ImdbApiListTitlesResponse, ImdbApiListTitleEpisodesResponse,
ImdbApiListTitleSeasonsResponse, ImdbApiListTitleCreditsResponse,
ImdbapiListTitleAKAsResponse, ImdbApiTitleImagesResponse)
from .schema.imdbtypes import ImdbType
CACHE_LIFESPAN: Final[int] = 86400
class ImdbApiClient:
BASE_URL = 'https://api.imdbapi.dev'
def __init__(self, proxies: Optional[Dict[str, str]] = None, ua: Optional[str] = None) -> None:
self._req = RequestUtils(ua=ua, accept_type="application/json",
proxies=proxies, session=requests.Session())
if proxies:
proxy_url = proxies.get("https") or proxies.get("http")
else:
proxy_url = None
self._free_api_client = httpx.AsyncClient(timeout=10, proxy=proxy_url)
self._async_req = AsyncRequestUtils(
ua=ua,
accept_type="application/json",
client=self._free_api_client
)
@retry(Exception, logger=logger, delay=1)
@cached(maxsize=4096, ttl=CACHE_LIFESPAN)
def _free_imdb_api(self, path: str, params: Optional[dict] = None) -> Optional[dict]:
r = self._req.get_res(url=f"{self.BASE_URL}{path}", params=params, raise_exception=True)
if r is None:
return None
if r.status_code != 200:
try:
logger.warning(
f"Free IMDb API returned non-200 status: {r.status_code} for path={path} params={params}"
)
except requests.exceptions.JSONDecodeError:
return None
return None
return r.json()
@retry(Exception, logger=logger, delay=1)
@cached(maxsize=4096, ttl=CACHE_LIFESPAN)
async def _async_free_imdb_api(self, path: str, params: Optional[dict] = None) -> Optional[dict]:
r = await self._async_req.get_res(url=f"{self.BASE_URL}{path}", params=params, raise_exception=True)
if r is None:
return None
if r.status_code != 200:
try:
logger.warning(
f"Free IMDb API returned non-200 status: {r.status_code} for path={path} params={params}"
)
except requests.exceptions.JSONDecodeError:
return None
return None
return r.json()
def search_titles(self, query: str, limit: Optional[int] = None) -> Optional[ImdbApiSearchTitlesResponse]:
"""
Search for titles using a query string.
:param query: Required. The search query for titles.
:param limit: Optional. Limit the number of results returned. The maximum is 50.
:return: Search results.
"""
path = '/search/titles'
params: Dict[str, Any] = {'query': query}
if limit:
params['limit'] = limit
try:
r = self._free_imdb_api(path=path, params=params)
if r is None:
return None
ret = ImdbApiSearchTitlesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while searching for titles: {e}")
return None
return ret
async def async_search_titles(self, query: str, limit: Optional[int] = None
) -> Optional[ImdbApiSearchTitlesResponse]:
endpoint = '/search/titles'
params: Dict[str, Any] = {'query': query}
if limit:
params['limit'] = limit
try:
r = await self._async_free_imdb_api(path=endpoint, params=params)
if r is None:
return None
ret = ImdbApiSearchTitlesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while searching for titles: {e}")
return None
return ret
def advanced_search(self, query: str, limit: Optional[int] = None,
media_types: Optional[List[ImdbType]] = None,
year: Optional[int] = None) -> Optional[List[ImdbApiTitle]]:
"""
Perform an advanced search for titles using a query string with additional filters.
:param query: The search query for titles.
:param limit: The maximum number of results to return.
:param media_types: The type of titles to filter by.
:param year: The start year for filtering titles.
:return: Search results.
"""
data = self.search_titles(query=query, limit=limit)
if data is None:
return None
ret = data.titles
if year:
ret = [title for title in ret if title.start_year == year]
if media_types:
ret = [title for title in ret if title.type in media_types]
return ret
async def async_advanced_search(self, query: str, limit: Optional[int] = None,
media_types: Optional[List[ImdbType]] = None,
year: Optional[int] = None) -> Optional[List[ImdbApiTitle]]:
"""
Perform an advanced search for titles using a query string with additional filters.
:param query: The search query for titles.
:param limit: The maximum number of results to return.
:param media_types: The type of titles to filter by.
:param year: The start year for filtering titles.
:return: Search results.
"""
res = await self.async_search_titles(query=query, limit=limit)
if res is None:
return None
data = res.titles
if year:
data = [title for title in res.titles if title.start_year == year]
if media_types:
data = [title for title in res.titles if title.type in media_types]
return data
def titles(self,
types: Optional[List[ImdbType]] = None,
genres: Optional[List[str]] = None,
country_codes: Optional[List[str]] = None,
language_codes: Optional[List[str]] = None,
name_ids: Optional[List[str]] = None,
interest_ids: Optional[List[str]] = None,
start_year: Optional[int] = None,
end_year: Optional[int] = None,
min_vote_count: Optional[int] = None,
max_vote_count: Optional[int] = None,
min_aggregate_rating: Optional[float] = None,
max_aggregate_rating: Optional[float] = None,
sort_by: Optional[str] = None,
sort_order: Optional[str] = None,
page_token: Optional[str] = None) -> Optional[ImdbApiListTitlesResponse]:
"""
Retrieve a list of titles with optional filters.
:param types: Optional. The type of titles to filter by. If not specified,
all types are returned.
- MOVIE: Represents a movie title.
- TV_SERIES: Represents a TV series title.
- TV_MINI_SERIES: Represents a TV miniseries title.
- TV_SPECIAL: Represents a TV special title.
- TV_MOVIE: Represents a TV movie title.
- SHORT: Represents a short title.
- VIDEO: Represents a video title.
- VIDEO_GAME: Represents a video game title.
:param genres: Optional. The genres to filter titles by. If not specified,
titles from all genres are returned.
:param country_codes: Optional. The ISO 3166-1 alpha-2 country codes to
filter titles by. If not specified, titles from all countries are
returned. Example: "US" for the United States, "GB" for the United
Kingdom.
:param language_codes: Optional. The ISO 639-1 or ISO 639-2 language codes
to filter titles by. If not specified, titles in all languages are
returned.
:param name_ids: Optional. The IDs of names to filter titles by.
:param interest_ids: Optional. The IDs of interests to filter titles by.
If not specified, titles associated with all interests are returned.
:param start_year: Optional. The start year for filtering titles.
:param end_year: Optional. The end year for filtering titles.
:param min_vote_count: Optional. The minimum number of votes a title must
have to be included. If not specified, titles with any number of votes
are included. The value must be between 0 and 1,000,000,000. Default is 0.
:param max_vote_count: Optional. The maximum number of votes a title can
have to be included. If not specified, titles with any number of votes
are included. The value must be between 0 and 1,000,000,000.
:param min_aggregate_rating: Optional. The minimum rating a title must have
to be included. If not specified, titles with any rating are included.
The value must be between 0.0 and 10.0.
:param max_aggregate_rating: Optional. The maximum rating a title can have
to be included. If not specified, titles with any rating are included.
The value must be between 0.0 and 10.0.
:param sort_by: Optional. The sorting order for the titles. If not
specified, titles are sorted by popularity.
- SORT_BY_POPULARITY: Sort by popularity. Used to rank titles based on
viewership, ratings, or cultural impact.
- SORT_BY_RELEASE_DATE: Sort by release date. Newer titles typically
appear before older ones.
- SORT_BY_USER_RATING: Sort by average user rating, reflecting audience
reception.
- SORT_BY_USER_RATING_COUNT: Sort by number of user ratings, indicating
engagement or popularity.
- SORT_BY_YEAR: Sort by release year, with newer titles typically first.
:param sort_order: Optional. The sorting order for the titles. If not
specified, titles are sorted in ascending order.
- ASC: Sort in ascending order.
- DESC: Sort in descending order.
:param page_token: Optional. Token for pagination, if applicable.
:return: A dictionary containing the list of titles and pagination info.
"""
path = '/titles'
params: Dict[str, Any] = {}
if types:
params['types'] = [t.value for t in types]
if genres:
params['genres'] = genres
if country_codes:
params['countryCodes'] = country_codes
if language_codes:
params['languageCodes'] = language_codes
if name_ids:
params['nameIds'] = name_ids
if interest_ids:
params['interestIds'] = interest_ids
if start_year:
params['startYear'] = start_year
if end_year:
params['endYear'] = end_year
if min_vote_count:
params['minVoteCount'] = min_vote_count
if max_vote_count:
params['maxVoteCount'] = max_vote_count
if min_aggregate_rating:
params['minAggregateRating'] = min_aggregate_rating
if max_aggregate_rating:
params['maxAggregateRating'] = max_aggregate_rating
if sort_by:
params['sortBy'] = sort_by
if sort_order:
params['sortOrder'] = sort_order
if page_token:
params['pageToken'] = page_token
try:
return ImdbApiListTitlesResponse.model_validate(self._free_imdb_api(path=path, params=params))
except Exception as e:
logger.debug(f"An error occurred while listing titles: {e}")
return None
async def async_titles(self,
types: Optional[List[ImdbType]] = None,
genres: Optional[List[str]] = None,
country_codes: Optional[List[str]] = None,
language_codes: Optional[List[str]] = None,
name_ids: Optional[List[str]] = None,
interest_ids: Optional[List[str]] = None,
start_year: Optional[int] = None,
end_year: Optional[int] = None,
min_vote_count: Optional[int] = None,
max_vote_count: Optional[int] = None,
min_aggregate_rating: Optional[float] = None,
max_aggregate_rating: Optional[float] = None,
sort_by: Optional[str] = None,
sort_order: Optional[str] = None,
page_token: Optional[str] = None) -> Optional[ImdbApiListTitlesResponse]:
path = '/titles'
params: Dict[str, Any] = {}
if types:
params['types'] = [t.value for t in types]
if genres:
params['genres'] = genres
if country_codes:
params['countryCodes'] = country_codes
if language_codes:
params['languageCodes'] = language_codes
if name_ids:
params['nameIds'] = name_ids
if interest_ids:
params['interestIds'] = interest_ids
if start_year:
params['startYear'] = start_year
if end_year:
params['endYear'] = end_year
if min_vote_count:
params['minVoteCount'] = min_vote_count
if max_vote_count:
params['maxVoteCount'] = max_vote_count
if min_aggregate_rating:
params['minAggregateRating'] = min_aggregate_rating
if max_aggregate_rating:
params['maxAggregateRating'] = max_aggregate_rating
if sort_by:
params['sortBy'] = sort_by
if sort_order:
params['sortOrder'] = sort_order
if page_token:
params['pageToken'] = page_token
try:
r = await self._async_free_imdb_api(path=path, params=params)
if r is None:
return None
return ImdbApiListTitlesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while listing titles: {e}")
return None
def titles_generator(self,
types: Optional[List[ImdbType]] = None,
genres: Optional[List[str]] = None,
country_codes: Optional[List[str]] = None,
language_codes: Optional[List[str]] = None,
name_ids: Optional[List[str]] = None,
interest_ids: Optional[List[str]] = None,
start_year: Optional[int] = None,
end_year: Optional[int] = None,
min_vote_count: Optional[int] = None,
max_vote_count: Optional[int] = None,
min_aggregate_rating: Optional[float] = None,
max_aggregate_rating: Optional[float] = None,
sort_by: Optional[str] = None,
sort_order: Optional[str] = None,
) -> Generator[ImdbApiTitle, None, None]:
page_token = None
while True:
response = self.titles(
types=types,
genres=genres,
country_codes=country_codes,
language_codes=language_codes,
name_ids=name_ids,
interest_ids=interest_ids,
start_year=start_year,
end_year=end_year,
min_vote_count=min_vote_count,
max_vote_count=max_vote_count,
min_aggregate_rating=min_aggregate_rating,
max_aggregate_rating=max_aggregate_rating,
sort_by=sort_by,
sort_order=sort_order,
page_token=page_token
)
if not response:
return
for title in response.titles:
yield title
if not page_token:
break
async def async_titles_generator(self,
types: Optional[List[ImdbType]] = None,
genres: Optional[List[str]] = None,
country_codes: Optional[List[str]] = None,
language_codes: Optional[List[str]] = None,
name_ids: Optional[List[str]] = None,
interest_ids: Optional[List[str]] = None,
start_year: Optional[int] = None,
end_year: Optional[int] = None,
min_vote_count: Optional[int] = None,
max_vote_count: Optional[int] = None,
min_aggregate_rating: Optional[float] = None,
max_aggregate_rating: Optional[float] = None,
sort_by: Optional[str] = None,
sort_order: Optional[str] = None,
) -> AsyncGenerator[ImdbApiTitle, None]:
page_token = None
while True:
response = await self.async_titles(
types=types,
genres=genres,
country_codes=country_codes,
language_codes=language_codes,
name_ids=name_ids,
interest_ids=interest_ids,
start_year=start_year,
end_year=end_year,
min_vote_count=min_vote_count,
max_vote_count=max_vote_count,
min_aggregate_rating=min_aggregate_rating,
max_aggregate_rating=max_aggregate_rating,
sort_by=sort_by,
sort_order=sort_order,
page_token=page_token
)
if not response:
return
for title in response.titles:
yield title
page_token = response.next_page_token
if not page_token:
break
def title(self, title_id: str) -> Optional[ImdbApiTitle]:
"""
Retrieve a title's details using its IMDb ID.
:param title_id: The IMDb title ID in the format 'tt1234567'.
:return: Details.
"""
path = '/titles/%s'
try:
r = self._free_imdb_api(path=path % title_id)
ret = ImdbApiTitle.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving details: {e}")
return None
return ret
async def async_title(self, title_id: str) -> Optional[ImdbApiTitle]:
path = '/titles/%s'
try:
r = await self._async_free_imdb_api(path=path % title_id)
if r is None:
return None
ret = ImdbApiTitle.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving details: {e}")
return None
return ret
def episodes(self, title_id: str, season: Optional[str] = None,
page_size: Optional[int] = None, page_token: Optional[str] = None) -> Optional[
ImdbApiListTitleEpisodesResponse]:
"""
Retrieve the episodes associated with a specific title.
:param title_id: Required. IMDb title ID in the format "tt1234567".
:param season: Optional. The season number to filter episodes by.
:param page_size: Optional. The maximum number of episodes to return per page.
The value must be between 1 and 50. The default is 20.
:param page_token: Optional. Token for pagination, if applicable.
:return: Episodes.
"""
path = '/titles/%s/episodes'
param: Dict[str, Any] = {}
if season is not None:
param['season'] = season
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = self._free_imdb_api(path=path % title_id, params=param)
ret = ImdbApiListTitleEpisodesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving episodes: {e}")
return None
return ret
async def async_episodes(self, title_id: str, season: Optional[str] = None,
page_size: Optional[int] = None, page_token: Optional[str] = None
) -> Optional[ImdbApiListTitleEpisodesResponse]:
path = '/titles/%s/episodes'
param: Dict[str, Any] = {}
if season is not None:
param['season'] = season
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = await self._async_free_imdb_api(path=path % title_id, params=param)
if r is None:
return None
ret = ImdbApiListTitleEpisodesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving episodes: {e}")
return None
return ret
def episodes_generator(self, title_id: str, season: Optional[str] = None) -> Generator[ImdbApiEpisode, None, None]:
page_token = None
while True:
response = self.episodes(
title_id=title_id,
season=season,
page_size=50,
page_token=page_token
)
if not response:
return
for episode in response.episodes:
yield episode
page_token = response.next_page_token
if not page_token:
break
async def async_episodes_generator(self, title_id: str, season: Optional[str] = None
) -> AsyncGenerator[ImdbApiEpisode, None]:
page_token = None
while True:
response = await self.async_episodes(
title_id=title_id,
season=season,
page_size=50,
page_token=page_token
)
if not response:
return
for episode in response.episodes:
yield episode
page_token = response.next_page_token
if not page_token:
break
def seasons(self, title_id: str) -> Optional[ImdbApiListTitleSeasonsResponse]:
"""
Retrieve the seasons associated with a specific title.
:param title_id: Required. IMDb title ID in the format "tt1234567".
:return: Seasons.
"""
path = '/titles/%s/seasons'
try:
r = self._free_imdb_api(path=path % title_id)
ret = ImdbApiListTitleSeasonsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving seasons: {e}")
return None
return ret
async def async_seasons(self, title_id: str) -> Optional[ImdbApiListTitleSeasonsResponse]:
path = '/titles/%s/seasons'
try:
r = await self._async_free_imdb_api(path=path % title_id)
if r is None:
return None
ret = ImdbApiListTitleSeasonsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving seasons: {e}")
return None
return ret
def credits(self, title_id: str, categories: Optional[List[str]] = None,
page_size: Optional[int] = None, page_token: Optional[str] = None
) -> Optional[ImdbApiListTitleCreditsResponse]:
"""
Retrieve the credits associated with a specific title.
:param title_id: Required. IMDb title ID in the format "tt1234567".
:param categories: Optional. The categories of credits to filter by.
:param page_size: Optional. The maximum number of credits to return per page.
The value must be between 1 and 50. The default is 20.
:param page_token: Optional. Token for pagination, if applicable.
:return: Credits.
"""
path = '/titles/%s/credits'
param: Dict[str, Any] = {}
if categories:
param['categories'] = categories
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = self._free_imdb_api(path=path % title_id, params=param)
ret = ImdbApiListTitleCreditsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving credits: {e}")
return None
return ret
async def async_credits(self, title_id: str, categories: Optional[List[str]] = None,
page_size: Optional[int] = None, page_token: Optional[str] = None) -> Optional[
ImdbApiListTitleCreditsResponse]:
path = '/titles/%s/credits'
param: Dict[str, Any] = {}
if categories:
param['categories'] = categories
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = await self._async_free_imdb_api(path=path % title_id, params=param)
if r is None:
return None
ret = ImdbApiListTitleCreditsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving credits: {e}")
return None
return ret
def credits_generator(self, title_id: str, categories: Optional[List[str]] = None
) -> Generator[ImdbApiCredit, None, None]:
page_token = None
while True:
response = self.credits(
title_id=title_id,
categories=categories,
page_size=50,
page_token=page_token
)
if not response:
return
for credit in response.credits:
yield credit
page_token = response.next_page_token
if not page_token:
break
async def async_credits_generator(self, title_id: str, categories: Optional[List[str]] = None
) -> AsyncGenerator[ImdbApiCredit, None]:
page_token = None
while True:
response = await self.async_credits(
title_id=title_id,
categories=categories,
page_size=50,
page_token=page_token
)
if not response:
return
for credit in response.credits:
yield credit
page_token = response.next_page_token
if not page_token:
break
def akas(self, title_id: str) -> Optional[ImdbapiListTitleAKAsResponse]:
"""
Retrieve the alternative titles (AKAs) associated with a specific title.
:param title_id: Required. IMDb title ID in the format "tt1234567".
:return: AKAs.
"""
path = '/titles/%s/akas'
try:
r = self._free_imdb_api(path=path % title_id)
ret = ImdbapiListTitleAKAsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving alternative titles: {e}")
return None
if r is None:
return None
return ret
async def async_akas(self, title_id: str) -> Optional[ImdbapiListTitleAKAsResponse]:
path = '/titles/%s/akas'
try:
r = await self._async_free_imdb_api(path=path % title_id)
if r is None:
return None
ret = ImdbapiListTitleAKAsResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving alternative titles: {e}")
return None
return ret
def images(self, title_id: str, types: list[str] | None = None, page_size: int | None = None,
page_token: str | None = None) -> ImdbApiTitleImagesResponse | None:
"""
Retrieve the images associated with a specific title.
:param title_id: Required. IMDb title ID in the format "tt1234567".
:param types: Optional. The types of images to filter by.
- 'poster'
- 'behind_the_scenes'
- 'still_frame'
:param page_size: Optional. The maximum number of images to return per page.
The value must be between 1 and 50. The default is 20.
:param page_token: Optional. Token for pagination, if applicable.
"""
path = '/titles/%s/images'
param: Dict[str, Any] = {}
if types:
param['types'] = types
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = self._free_imdb_api(path=path % title_id, params=param)
if r is None:
return None
ret = ImdbApiTitleImagesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving images: {e}")
return None
return ret
async def async_images(self, title_id: str, types: list[str] | None = None, page_size: int = 20,
page_token: str | None = None) -> ImdbApiTitleImagesResponse | None:
path = '/titles/%s/images'
param: Dict[str, Any] = {}
if types:
param['types'] = types
if page_size is not None:
param['pageSize'] = page_size
if page_token is not None:
param['pageToken'] = page_token
try:
r = await self._async_free_imdb_api(path=path % title_id, params=param)
if r is None:
return None
ret = ImdbApiTitleImagesResponse.model_validate(r)
except Exception as e:
logger.debug(f"An error occurred while retrieving images: {e}")
return None
return ret
def images_generator(self, title_id: str, types: list[str] | None = None
) -> Generator[ImdbapiImage, None, None]:
page_token = None
while True:
response = self.images(
title_id=title_id,
types=types,
page_size=50,
page_token=page_token
)
if not response:
return
for image in response.images:
yield image
page_token = response.next_page_token
if not page_token:
break
async def async_images_generator(self, title_id: str, types: list[str] | None = None
) -> AsyncGenerator[ImdbapiImage, None]:
page_token = None
while True:
response = await self.async_images(
title_id=title_id,
types=types,
page_size=50,
page_token=page_token
)
if not response:
return
for image in response.images:
yield image
page_token = response.next_page_token
if not page_token:
break

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,582 @@
import re
from textwrap import dedent
from typing import Any, Dict, List, Optional, Final, AsyncGenerator
import httpx
import requests
from pydantic import ValidationError
from app.core.cache import cached
from app.log import logger
from app.utils.common import retry
from app.utils.http import RequestUtils, AsyncRequestUtils
from .schema.imdbtypes import ImdbType
from .schema import VerticalList, AdvancedTitleSearchResponse, AdvancedTitleSearch, TitleEdge, SearchParams
INTERESTS_ID: Final[Dict[str, Dict[str, str]]] = {
"Action": {
"Action": "in0000001",
"Action Epic": "in0000002",
"B-Action": "in0000003",
"Car Action": "in0000004",
"Disaster": "in0000005",
"Gun Fu": "in0000197",
"Kung Fu": "in0000198",
"Martial Arts": "in0000006",
"One-Person Army Action": "in0000007",
"Samurai": "in0000199",
"Superhero": "in0000008",
"Sword & Sandal": "in0000009",
"War": "in0000010",
"War Epic": "in0000011",
"Wuxia": "in0000200"
},
"Adventure": {
"Adventure": "in0000012",
"Adventure Epic": "in0000015",
"Desert Adventure": "in0000013",
"Dinosaur Adventure": "in0000014",
"Globetrotting Adventure": "in0000016",
"Jungle Adventure": "in0000017",
"Mountain Adventure": "in0000018",
"Quest": "in0000019",
"Road Trip": "in0000020",
"Sea Adventure": "in0000021",
"Swashbuckler": "in0000022",
"Teen Adventure": "in0000023",
"Urban Adventure": "in0000024"
},
"Animation": {
"Adult Animation": "in0000025",
"Animation": "in0000026",
"Computer Animation": "in0000028",
"Hand-Drawn Animation": "in0000029",
"Stop Motion Animation": "in0000030"
},
"Anime": {
"Anime": "in0000027",
"Isekai": "in0000201",
"Iyashikei": "in0000202",
"Josei": "in0000203",
"Mecha": "in0000204",
"Seinen": "in0000205",
"Shōjo": "in0000207",
"Shōnen": "in0000206",
"Slice of Life": "in0000208"
},
"Comedy": {
"Body Swap Comedy": "in0000031",
"Buddy Comedy": "in0000032",
"Buddy Cop": "in0000033",
"Comedy": "in0000034",
"Dark Comedy": "in0000035",
"Farce": "in0000036",
"High-Concept Comedy": "in0000037",
"Mockumentary": "in0000038",
"Parody": "in0000039",
"Quirky Comedy": "in0000040",
"Raunchy Comedy": "in0000041",
"Satire": "in0000042",
"Screwball Comedy": "in0000043",
"Sitcom": "in0000044",
"Sketch Comedy": "in0000045",
"Slapstick": "in0000046",
"Stand-Up": "in0000047",
"Stoner Comedy": "in0000048",
"Teen Comedy": "in0000049"
},
"Crime": {
"Caper": "in0000050",
"Cop Drama": "in0000051",
"Crime": "in0000052",
"Drug Crime": "in0000053",
"Film Noir": "in0000054",
"Gangster": "in0000055",
"Heist": "in0000056",
"Police Procedural": "in0000057",
"True Crime": "in0000058"
},
"Documentary": {
"Crime Documentary": "in0000059",
"Documentary": "in0000060",
"Docuseries": "in0000061",
"Faith & Spirituality Documentary": "in0000062",
"Food Documentary": "in0000063",
"History Documentary": "in0000064",
"Military Documentary": "in0000065",
"Music Documentary": "in0000066",
"Nature Documentary": "in0000067",
"Political Documentary": "in0000068",
"Science & Technology Documentary": "in0000069",
"Sports Documentary": "in0000070",
"Travel Documentary": "in0000071"
},
"Drama": {
"Biography": "in0000072",
"Coming-of-Age": "in0000073",
"Costume Drama": "in0000074",
"Docudrama": "in0000075",
"Drama": "in0000076",
"Epic": "in0000077",
"Financial Drama": "in0000078",
"Historical Epic": "in0000079",
"History": "in0000080",
"Korean Drama": "in0000209",
"Legal Drama": "in0000081",
"Medical Drama": "in0000082",
"Period Drama": "in0000083",
"Political Drama": "in0000084",
"Prison Drama": "in0000085",
"Psychological Drama": "in0000086",
"Showbiz Drama": "in0000087",
"Soap Opera": "in0000088",
"Teen Drama": "in0000089",
"Telenovela": "in0000210",
"Tragedy": "in0000090",
"Workplace Drama": "in0000091"
},
"Family": {
"Animal Adventure": "in0000092",
"Family": "in0000093"
},
"Fantasy": {
"Dark Fantasy": "in0000095",
"Fairy Tale": "in0000097",
"Fantasy": "in0000098",
"Fantasy Epic": "in0000096",
"Supernatural Fantasy": "in0000099",
"Sword & Sorcery": "in0000100",
"Teen Fantasy": "in0000101"
},
"Game Show": {
"Beauty Competition": "in0000102",
"Cooking Competition": "in0000103",
"Game Show": "in0000105",
"Quiz Show": "in0000104",
"Survival Competition": "in0000106",
"Talent Competition": "in0000107"
},
"Horror": {
"B-Horror": "in0000108",
"Body Horror": "in0000109",
"Folk Horror": "in0000110",
"Found Footage Horror": "in0000111",
"Horror": "in0000112",
"Monster Horror": "in0000113",
"Psychological Horror": "in0000114",
"Slasher Horror": "in0000115",
"Splatter Horror": "in0000116",
"Supernatural Horror": "in0000117",
"Teen Horror": "in0000118",
"Vampire Horror": "in0000119",
"Werewolf Horror": "in0000120",
"Witch Horror": "in0000121",
"Zombie Horror": "in0000122"
},
"Lifestyle": {
"Beauty Makeover": "in0000123",
"Cooking & Food": "in0000124",
"Home Improvement": "in0000125",
"Lifestyle": "in0000126",
"News": "in0000211",
"Talk Show": "in0000127",
"Travel": "in0000128"
},
"Music": {
"Concert": "in0000129",
"Music": "in0000130"
},
"Musical": {
"Classic Musical": "in0000131",
"Jukebox Musical": "in0000132",
"Musical": "in0000133",
"Pop Musical": "in0000134",
"Rock Musical": "in0000135"
},
"Mystery": {
"Bumbling Detective": "in0000136",
"Cozy Mystery": "in0000137",
"Hard-boiled Detective": "in0000138",
"Mystery": "in0000139",
"Suspense Mystery": "in0000140",
"Whodunnit": "in0000141"
},
"Reality TV": {
"Business Reality TV": "in0000142",
"Crime Reality TV": "in0000143",
"Dating Reality TV": "in0000144",
"Docusoap Reality TV": "in0000145",
"Hidden Camera": "in0000146",
"Paranormal Reality TV": "in0000147",
"Reality TV": "in0000148"
},
"Romance": {
"Dark Romance": "in0000149",
"Feel-Good Romance": "in0000151",
"Romance": "in0000152",
"Romantic Comedy": "in0000153",
"Romantic Epic": "in0000150",
"Steamy Romance": "in0000154",
"Teen Romance": "in0000155",
"Tragic Romance": "in0000156"
},
"Sci-Fi": {
"Alien Invasion": "in0000157",
"Artificial Intelligence": "in0000158",
"Cyberpunk": "in0000159",
"Dystopian Sci-Fi": "in0000160",
"Kaiju": "in0000161",
"Sci-Fi": "in0000162",
"Sci-Fi Epic": "in0000163",
"Space Sci-Fi": "in0000164",
"Steampunk": "in0000165",
"Time Travel": "in0000166"
},
"Seasonal": {
"Holiday": "in0000192",
"Holiday Animation": "in0000193",
"Holiday Comedy": "in0000194",
"Holiday Family": "in0000195",
"Holiday Romance": "in0000196"
},
"Short": {
"Short": "in0000212"
},
"Sport": {
"Baseball": "in0000167",
"Basketball": "in0000168",
"Boxing": "in0000169",
"Extreme Sport": "in0000170",
"Football": "in0000171",
"Motorsport": "in0000172",
"Soccer": "in0000173",
"Sport": "in0000174",
"Water Sport": "in0000175"
},
"Thriller": {
"Conspiracy Thriller": "in0000176",
"Cyber Thriller": "in0000177",
"Erotic Thriller": "in0000178",
"Giallo": "in0000179",
"Legal Thriller": "in0000180",
"Political Thriller": "in0000181",
"Psychological Thriller": "in0000182",
"Serial Killer": "in0000183",
"Spy": "in0000184",
"Survival": "in0000185",
"Thriller": "in0000186"
},
"Western": {
"Classical Western": "in0000187",
"Contemporary Western": "in0000188",
"Spaghetti Western": "in0000190",
"Western": "in0000191",
"Western Epic": "in0000189"
}
}
CACHE_LIFETIME: Final[int] = 86400
IMDB_GRAPHQL_QUERY: Final[str] = dedent("""
query VerticalListPageItems( $titles: [ID!]! $names: [ID!]! $images: [ID!]! $videos: [ID!]!) {
titles(ids: $titles) { ...TitleParts meterRanking { currentRank meterType rankChange {changeDirection difference} } ratingsSummary { aggregateRating } }
names(ids: $names) { ...NameParts }
videos(ids: $videos) { ...VideoParts }
images(ids: $images) { ...ImageParts }
}
fragment TitleParts on Title {
id
titleText { text }
titleType { id }
releaseYear { year }
akas(first: 50) { edges { node { text country { id text } language { text } } } }
plot { plotText {plainText}}
primaryImage { id url width height }
releaseDate {day month year}
titleGenres {genres {genre { text }}}
certificate { rating }
originalTitleText{ text }
runtime { seconds }
}
fragment NameParts on Name {
id
nameText { text }
primaryImage { id url width height }
}
fragment ImageParts on Image {
id
height
width
url
}
fragment VideoParts on Video {
id
name { value }
contentType { displayName { value } id }
previewURLs { displayName { value } url videoDefinition videoMimeType }
playbackURLs { displayName { value } url videoDefinition videoMimeType }
thumbnail { height url width }
}
""")
class PersistedQueryNotFound(Exception):
def __init__(self, message: str, code: int = None):
super().__init__(message)
self.code = code
class OfficialApiClient:
BASE_URL = "https://caching.graphql.imdb.com/"
def __init__(self, proxies: Optional[Dict[str, str]] = None,
ua: Optional[str] = None):
self._req = RequestUtils(accept_type="application/json",
content_type="application/json",
timeout=10,
ua=ua,
proxies=proxies,
session=requests.Session())
if proxies:
proxy_url = proxies.get("https") or proxies.get("http")
else:
proxy_url = None
self._client = httpx.AsyncClient(timeout=10, proxy=proxy_url)
self._async_req = AsyncRequestUtils(accept_type="application/json", content_type="application/json",
client=self._client, ua=ua)
self.flat_interest_id = {}
for category, value in INTERESTS_ID.items():
for name, in_id in value.items():
self.flat_interest_id[name] = in_id
@cached(maxsize=1024, ttl=CACHE_LIFETIME)
async def _async_request(self, params: Dict[str, Any], sha256: str) -> Optional[Dict]:
params["extensions"] = {"persistedQuery": {"sha256Hash": sha256, "version": 1}}
data = await self._async_req.post_json(f"{self.BASE_URL}", json=params, raise_exception=True)
if not data:
return None
if "errors" in data:
error = data.get("errors")[0] if data.get("errors") else {}
return {'error': error}
return data.get("data")
@retry(Exception, logger=logger, delay=1)
@cached(maxsize=1024, ttl=CACHE_LIFETIME)
def _query_graphql(self, query: str, variables: Dict[str, Any]) -> Optional[dict]:
params = {'query': query, 'variables': variables}
data = self._req.post_json(f"{self.BASE_URL}", json=params, raise_exception=True)
if not data:
return {'error': 'Query failed.'}
if "errors" in data:
error = data.get("errors")[0] if data.get("errors") else {}
return {'error': error}
return data.get("data")
@retry(Exception, logger=logger, delay=1)
@cached(maxsize=1024, ttl=CACHE_LIFETIME)
async def _async_query_graphql(self, query: str, variables: Dict[str, Any]) -> Optional[Dict]:
params = {'query': query, 'variables': variables}
data = await self._async_req.post_json(f"{self.BASE_URL}", json=params, raise_exception=True)
if not data:
return None
if "errors" in data:
error = data.get("errors")[0] if data.get("errors") else {}
return {'error': error}
return data.get("data")
@cached(maxsize=1024, ttl=CACHE_LIFETIME)
def vertical_list_page_items(self,
titles: Optional[List[str]] = None,
names: Optional[List[str]] = None,
images: Optional[List[str]] = None,
videos: Optional[List[str]] = None,
is_registered: bool = False
) -> Optional[VerticalList]:
variables = {'images': images or [],
'titles': titles or [],
'names': names or [],
'videos': videos or [],
'isRegistered': is_registered,
}
try:
data = self._query_graphql(IMDB_GRAPHQL_QUERY, variables)
if 'error' in data:
error = data['error']
if error:
logger.error(f"Error querying VerticalListPageItems: {error}")
return None
ret = VerticalList.model_validate(data)
except Exception as e:
logger.debug(f"An error occurred while querying VerticalListPageItems: {e}")
return None
return ret
@cached(maxsize=1024, ttl=CACHE_LIFETIME)
async def async_vertical_list_page_items(self,
titles: Optional[List[str]] = None,
names: Optional[List[str]] = None,
images: Optional[List[str]] = None,
videos: Optional[List[str]] = None,
is_registered: bool = False
) -> Optional[VerticalList]:
variables = {'images': images or [],
'titles': titles or [],
'names': names or [],
'videos': videos or [],
'isRegistered': is_registered,
}
try:
data = await self._async_query_graphql(IMDB_GRAPHQL_QUERY, variables)
if 'error' in data:
error = data['error']
if error:
logger.error(f"Error querying VerticalListPageItems: {error}")
return None
ret = VerticalList.model_validate(data)
except Exception as e:
logger.debug(f"An error occurred while querying VerticalListPageItems: {e}")
return None
return ret
@retry(Exception, logger=logger, delay=1)
async def async_advanced_title_search(self,
params: SearchParams,
sha256: str,
last_cursor: Optional[str] = None,
) -> Optional[AdvancedTitleSearch]:
variables: Dict[str, Any] = {"first": 50,
"locale": "en-US",
"sortBy": params.sort_by,
"sortOrder": params.sort_order,
}
operation_name = 'AdvancedTitleSearch'
if params.title_types:
title_type_ids = []
for title_type in params.title_types:
if title_type in ImdbType._value2member_map_:
title_type_ids.append(title_type)
if len(title_type_ids):
variables["titleTypeConstraint"] = {"anyTitleTypeIds": title_type_ids}
if params.genres:
variables["genreConstraint"] = {"allGenreIds": params.genres, "excludeGenreIds": []}
if params.countries:
variables["originCountryConstraint"] = {"allCountries": params.countries}
if params.languages:
variables["languageConstraint"] = {"anyPrimaryLanguages": params.languages}
if params.rating_min or params.rating_max:
rating_min = params.rating_min if params.rating_min else 1
rating_min = max(rating_min, 1)
rating_max = params.rating_max if params.rating_max else 10
rating_max = min(rating_max, 10)
variables["userRatingsConstraint"] = {"aggregateRatingRange": {"max": rating_max, "min": rating_min}}
if params.release_date_start or params.release_date_end:
release_dict = {}
if params.release_date_start:
release_dict["start"] = params.release_date_start
if params.release_date_end:
release_dict["end"] = params.release_date_end
variables["releaseDateConstraint"] = {"releaseDateRange": release_dict}
if params.award_constraint:
constraints = []
for award in params.award_constraint:
c = self._award_to_constraint(award)
if c:
constraints.append(c)
variables["awardConstraint"] = {"allEventNominations": constraints}
if params.ranked:
constraints = []
for r in params.ranked:
c = OfficialApiClient._ranked_list_to_constraint(r)
if c:
constraints.append(c)
variables["rankedTitleListConstraint"] = {"allRankedTitleLists": constraints,
"excludeRankedTitleLists": []}
if params.interests:
constraints = []
for interest in params.interests:
in_id = self.flat_interest_id.get(interest)
if in_id:
constraints.append(in_id)
variables["interestConstraint"] = {"allInterestIds": constraints, "excludeInterestIds": []}
if last_cursor:
variables["after"] = last_cursor
params = {"operationName": operation_name,
"variables": variables}
data = await self._async_request(params, sha256)
if not data:
return None
if 'error' in data:
error = data['error']
if error:
if error.get('message') == 'PersistedQueryNotFound':
await self._async_request.cache_clear()
raise PersistedQueryNotFound(error['message'])
return None
try:
ret = AdvancedTitleSearchResponse.model_validate(data)
except ValidationError as err:
logger.error(f"{err}")
return None
return ret.advanced_title_search
async def advanced_title_search_generator(self, params: SearchParams, sha256: str) -> AsyncGenerator[
TitleEdge, None]:
last_cursor = None
while True:
response = await self.async_advanced_title_search(params, sha256, last_cursor=last_cursor)
if not response:
return
for edge in response.edges:
yield edge
last_cursor = response.page_info.end_cursor
if not last_cursor or not response.page_info.has_next_page:
break
@staticmethod
def _ranked_list_to_constraint(ranked: str) -> Optional[Dict]:
"""
"TOP_RATED_MOVIES-100": "IMDb Top 100",
"TOP_RATED_MOVIES-250": "IMDb Top 250",
"TOP_RATED_MOVIES-1000": "IMDb Top 1000",
"LOWEST_RATED_MOVIES-100": "IMDb Bottom 100",
"LOWEST_RATED_MOVIES-250": "IMDb Bottom 250",
"LOWEST_RATED_MOVIES-1000": "IMDb Bottom 1000"
"""
pattern = r'^(TOP_RATED_MOVIES|LOWEST_RATED_MOVIES)-(\d+)$'
match = re.match(pattern, ranked)
if match:
ranked_title_list_type = match.group(1)
rank_range = int(match.group(2))
constraint = {"rankRange": {"max": rank_range}, "rankedTitleListType": ranked_title_list_type}
return constraint
return None
@staticmethod
def _award_to_constraint(award: str) -> Optional[Dict]:
pattern = r'^(ev\d+)(?:-(best\w+))?-(Winning|Nominated)$'
match = re.match(pattern, award)
constraint = {}
if match:
# 第一部分evXXXXXXXX
ev_id = match.group(1)
# 第二部分bestXX可选
best = match.group(2)
# 第三部分Winning/Nominated
status = match.group(3)
constraint["eventId"] = ev_id
if status == "Winning":
constraint["winnerFilter"] = "WINNER_ONLY"
if best:
constraint["searchAwardCategoryId"] = best
return constraint
else:
return None
@property
def interests_id(self) -> Dict[str, str]:
return self.flat_interest_id

View File

@@ -0,0 +1,138 @@
from enum import Enum
from typing import Optional, List, Tuple, Union
from pydantic import BaseModel, Field, ConfigDict
from .imdbapi import ImdbApiTitle, ImdbApiEpisode, ImdbApiCredit, ImdbapiImage
from .imdbtypes import ImdbTitle, ImdbName, ImdbImage, ImdbVideo, AkasNode, TitleEdge
class ErrorType(Enum):
PERSISTED_QUERY_NOT_FOUND = 'PERSISTED_QUERY_NOT_FOUND'
class StaffPickEntry(BaseModel):
name: str
ttconst: str
rmconst: str
detail: Optional[str] = ""
description: Optional[str] = ""
relatedconst: List[str] = Field(default_factory=list)
viconst: Optional[str] = None
class VerticalList(BaseModel):
titles: List[ImdbTitle] = Field(default_factory=list)
names: List[ImdbName] = Field(default_factory=list)
videos: List[ImdbVideo] = Field(default_factory=list)
images: List[ImdbImage] = Field(default_factory=list)
class StaffPickApiResponse(BaseModel):
updated_at: Optional[str]
entries: List[StaffPickEntry] = Field(default_factory=list)
imdb_items: VerticalList
class ImdbMediaInfo(ImdbApiTitle):
akas: List[AkasNode] = Field(default_factory=list)
episodes: List[ImdbApiEpisode] = Field(default_factory=list)
credits: List[ImdbApiCredit] = Field(default_factory=list)
images: List[ImdbapiImage] = Field(default_factory=list)
@classmethod
def from_title(
cls,
title: ImdbApiTitle,
akas: Optional[List[AkasNode]] = None,
episodes: Optional[List[ImdbApiEpisode]] = None,
api_credits: Optional[List[ImdbApiCredit]] = None,
images: Optional[List[ImdbapiImage]] = None
) -> "ImdbMediaInfo":
fields = {
**title.model_dump(exclude_none=True, by_alias=True),
}
if akas is not None:
fields['akas'] = akas
if episodes is not None:
fields['episodes'] = episodes
if api_credits is not None:
fields['credits'] = api_credits
if images is not None:
fields['images'] = images
return cls(**fields)
def backdrop_path(self) -> str | None:
if self.images:
for image in self.images:
if image.url and image.type == 'still_frame':
# replace('@._V1', '@._V1_QL75_UX327_')
return image.url
return None
class ImdbApiHash(BaseModel):
advanced_title_search: str = Field(alias="AdvancedTitleSearch")
class PageInfo(BaseModel):
has_previous_page: Optional[bool] = Field(None, alias="hasPreviousPage")
has_next_page: Optional[bool] = Field(None, alias="hasNextPage")
start_cursor: Optional[str] = Field(None, alias="startCursor")
end_cursor: Optional[str] = Field(None, alias="endCursor")
class FilterInfo(BaseModel):
filter_id: Optional[str] = Field(default=None, alias='filterId')
text: Optional[str] = Field(default=None, alias='text')
total: Optional[int] = Field(default=None, alias='total')
class SearchState(BaseModel):
total: int = 0
page_info: PageInfo = Field(default_factory=PageInfo, alias="pageInfo")
genres: List[FilterInfo] = Field(default_factory=list)
keywords: List[FilterInfo] = Field(default_factory=list)
title_types: List[FilterInfo] = Field(default_factory=list, alias='titleTypes')
class AdvancedTitleSearch(SearchState):
edges: List[TitleEdge] = Field(default_factory=list)
class AdvancedTitleSearchResponse(BaseModel):
advanced_title_search: AdvancedTitleSearch = Field(default_factory=AdvancedTitleSearch, alias="advancedTitleSearch")
class SearchParams(BaseModel):
title_types: Optional[Tuple[str, ...]] = None
genres: Optional[Tuple[str, ...]] = None
sort_by: str = 'POPULARITY'
sort_order: str = 'ASC'
rating_min: Optional[float] = None
rating_max: Optional[float] = None
countries: Optional[Tuple[str, ...]] = None
languages: Optional[Tuple[str, ...]] = None
release_date_end: Optional[str] = None
release_date_start: Optional[str] = None
award_constraint: Optional[Tuple[str, ...]] = None
ranked: Optional[Tuple[str, ...]] = None
interests: Optional[Tuple[str, ...]] = None
model_config = ConfigDict(
frozen=True
)
class ErrorExtension(BaseModel):
code: Union[ErrorType, str]
error_type: str = Field('CLIENT', alias='errorType')
is_retryable: bool = Field(False, alias='isRetryable')
class ErrorValue(BaseModel):
message: Optional[str] = Field(default=None, alias='message')
extensions: Optional[ErrorExtension]
class ErrorResponse(BaseModel):
errors: Optional[List[ErrorValue]] = Field(default_factory=list)

View File

@@ -0,0 +1,156 @@
from typing import Optional, List
from pydantic import BaseModel, Field
from .imdbtypes import ImdbType, RatingsSummary, AkasNode, ImdbDate
class ImdbapiImage(BaseModel):
url: Optional[str] = None
width: Optional[int] = None
height: Optional[int] = None
type: Optional[str] = None
class ImdbApiMetacritic(BaseModel):
url: Optional[str] = None
score: Optional[int] = None
review_count: Optional[int] = Field(None, alias='reviewCount')
class ImdbApiMeterRanking(BaseModel):
current_rank: Optional[int] = Field(None, alias='currentRank')
change_direction: Optional[str] = Field(None, alias='changeDirection')
difference: Optional[int] = None
class ImdbApiPerson(BaseModel):
id: Optional[str] = None
display_name: Optional[str] = Field(None, alias='displayName')
alternative_names: Optional[List[str]] = Field(None, alias='alternativeNames')
primary_image: Optional[ImdbapiImage] = Field(None, alias='primaryImage')
primary_professions: Optional[List[str]] = Field(None, alias='primaryProfessions')
biography: Optional[str] = None
height_cm: Optional[float] = Field(None, alias='heightCm')
birth_name: Optional[str] = Field(None, alias='birthName')
birth_date: Optional[ImdbDate] = Field(None, alias='birthDate')
birth_location: Optional[str] = Field(None, alias='birthLocation')
death_date: Optional[ImdbDate] = Field(None, alias='deathDate')
death_location: Optional[str] = Field(None, alias='deathLocation')
death_reason: Optional[str] = Field(None, alias='deathReason')
meter_ranking: Optional[ImdbApiMeterRanking] = Field(None, alias='meterRanking')
class ImdbApiCountry(BaseModel):
# The ISO 3166-1 alpha-2 country code for the title, (e.g. "US" for the United States, "JP" for Japan)
code: Optional[str] = None
# The name of the country in English.
name: Optional[str] = None
class ImdbApiLanguage(BaseModel):
# The ISO 639-3 language code for the title, (e.g. "eng" for English, "jpn" for Japanese)
code: Optional[str] = None
# The name of the language in English.
name: Optional[str] = None
class ImdbapiPrecisionDate(BaseModel):
year: Optional[int] = None
month: Optional[int] = None
day: Optional[int] = None
class ImdbApiInterest(BaseModel):
id: Optional[str] = None
name: Optional[str] = None
primary_image: Optional[ImdbapiImage] = Field(None, alias='primaryImage')
description: Optional[str] = None
is_subgenre: Optional[bool] = Field(None, alias='isSubgenre')
similar_interests: Optional[List['ImdbApiInterest']] = Field(None, alias='similarInterests')
class ImdbApiTitle(BaseModel):
id: str
type: ImdbType
is_adult: Optional[bool] = Field(None, alias='isAdult')
primary_title: Optional[str] = Field(None, alias='primaryTitle')
original_title: Optional[str] = Field(None, alias='originalTitle')
primary_image: Optional[ImdbapiImage] = Field(None, alias='primaryImage')
start_year: Optional[int] = Field(None, alias='startYear')
end_year: Optional[int] = Field(None, alias='endYear')
runtime_seconds: Optional[int] = Field(None, alias='runtimeSeconds')
genres: Optional[List[str]] = None
rating: Optional[RatingsSummary] = None
metacritic: Optional[ImdbApiMetacritic] = None
plot: Optional[str] = None
directors: Optional[List[ImdbApiPerson]] = Field(default_factory=list)
writers: Optional[List[ImdbApiPerson]] = Field(default_factory=list)
stars: Optional[List[ImdbApiPerson]] = Field(default_factory=list)
origin_countries: Optional[List[ImdbApiCountry]] = Field(default_factory=list, alias='originCountries')
spoken_languages: Optional[List[ImdbApiLanguage]] = Field(default_factory=list, alias='spokenLanguages')
interests: Optional[List[ImdbApiInterest]] = None
class ImdbApiSearchTitlesResponse(BaseModel):
titles: List[ImdbApiTitle]
class ImdbApiListTitlesResponse(BaseModel):
titles: List[ImdbApiTitle] = Field(default_factory=list)
total_count: int = Field(alias='totalCount')
next_page_token: Optional[str] = Field(None, alias='nextPageToken')
class ImdbApiEpisode(BaseModel):
id: str
title: Optional[str] = None
primary_image: Optional[ImdbapiImage] = Field(None, alias='primaryImage')
season: Optional[str] = Field(None, alias='season')
episode_number: Optional[int] = Field(None, alias='episodeNumber')
runtime_seconds: Optional[int] = Field(None, alias='runtimeSeconds')
plot: Optional[str] = Field(None, alias='plot')
rating: Optional[RatingsSummary] = Field(None, alias='rating')
release_date: Optional[ImdbapiPrecisionDate] = Field(None, alias='releaseDate')
class PagedResponse(BaseModel):
total_count: int = Field(alias='totalCount')
next_page_token: Optional[str] = Field(None, alias='nextPageToken')
class ImdbApiListTitleEpisodesResponse(PagedResponse):
episodes: List[ImdbApiEpisode] = Field(default_factory=list)
class ImdbApiSeason(BaseModel):
season: Optional[str] = None
episode_count: Optional[int] = Field(None, alias='episodeCount')
class ImdbApiListTitleSeasonsResponse(BaseModel):
seasons: List[ImdbApiSeason] = Field(default_factory=list)
class ImdbApiCredit(BaseModel):
title: Optional[ImdbApiTitle] = None
name: Optional[ImdbApiPerson] = None
category: Optional[str] = None
characters: Optional[List[str]] = None
episode_count: Optional[int] = Field(None, alias='episodeCount')
class ImdbApiListTitleCreditsResponse(PagedResponse):
credits: List[ImdbApiCredit] = Field(default_factory=list)
class ImdbapiAka(AkasNode):
attributes: List[str] = Field(default_factory=list)
class ImdbapiListTitleAKAsResponse(BaseModel):
akas: List[ImdbapiAka]
class ImdbApiTitleImagesResponse(PagedResponse):
images: List[ImdbapiImage] = Field(default_factory=list)

View File

@@ -0,0 +1,171 @@
from enum import Enum
from typing import Optional, List
from pydantic import BaseModel, Field
class ImdbType(Enum):
TV_SERIES = "tvSeries"
TV_MINI_SERIES = "tvMiniSeries"
MOVIE = "movie"
TV_MOVIE = "tvMovie"
MUSIC_VIDEO = "musicVideo"
TV_SHORT = "tvShort"
SHORT = "short"
TV_EPISODE = "tvEpisode"
TV_SPECIAL = "tvSpecial"
VIDEO_GAME = "videoGame"
VIDEO = "video"
PODCAST_SERIES = "podcastSeries"
PODCAST_EPISODE = "podcastEpisode"
class TitleType(BaseModel):
id: ImdbType
class ReleaseYear(BaseModel):
year: Optional[int] = None
class Country(BaseModel):
id: str
text: str
class TextField(BaseModel):
text: Optional[str] = ''
class ValueField(BaseModel):
value: Optional[str] = None
class SecondsField(BaseModel):
seconds: Optional[int] = None
class AkasNode(BaseModel):
text: Optional[str] = ''
country: Optional[Country] = None
language: Optional[TextField] = None
class AkasEdge(BaseModel):
node: AkasNode
class Akas(BaseModel):
edges: List[AkasEdge] = Field(default_factory=list)
class PlotText(BaseModel):
plain_text: Optional[str] = Field(default='', alias='plainText')
class Plot(BaseModel):
plot_text: Optional[PlotText] = Field(None, alias='plotText')
class ImdbImage(BaseModel):
id: str
url: Optional[str] = None
width: Optional[int] = None
height: Optional[int] = None
def poster_path(self):
if self.url:
return self.url.replace('@._V1', '@._V1_QL75_UY414_CR6,0,280,414_')
return None
class RankChange(BaseModel):
change_direction: Optional[str] = Field(default=None, alias='changeDirection')
difference: Optional[int] = None
class MeterRanking(BaseModel):
current_rank: Optional[int] = Field(default=None, alias='currentRank')
meter_type: Optional[str] = Field(default=None, alias='meterType')
rank_change: Optional[RankChange] = Field(default=None, alias='rankChange')
class RatingsSummary(BaseModel):
aggregate_rating: Optional[float] = Field(default=None, alias='aggregateRating')
vote_count: Optional[int] = Field(None, alias='voteCount')
class ImdbName(BaseModel):
id: str
name_text: TextField = Field(alias='nameText')
primary_image: Optional[ImdbImage] = Field(default=None, alias='primaryImage')
class ContentType(BaseModel):
display_name: ValueField = Field(alias='displayName')
id: str
class VideoUrl(BaseModel):
display_name: ValueField = Field(alias='displayName')
url: str
video_definition: str = Field(alias='videoDefinition')
video_mime_type: str = Field(alias='videoMimeType')
class ImdbDate(BaseModel):
year: Optional[int] = None
month: Optional[int] = None
day: Optional[int] = None
class Genre(BaseModel):
genre: Optional[TextField] = None
class TitleGenre(BaseModel):
genres: List[Genre] = Field(default_factory=list)
class Certificate(BaseModel):
rating: Optional[str] = None
class ImdbTitle(BaseModel):
id: str
title_text: TextField = Field(alias='titleText')
title_type: TitleType = Field(alias='titleType')
release_year: Optional[ReleaseYear] = Field(None, alias='releaseYear')
akas: Optional[Akas] = None
plot: Optional[Plot] = None
primary_image: Optional[ImdbImage] = Field(default=None, alias='primaryImage')
meter_ranking: Optional[MeterRanking] = Field(default=None, alias='meterRanking')
ratings_summary: Optional[RatingsSummary] = Field(default=None, alias='ratingsSummary')
release_date: Optional[ImdbDate] = Field(None, alias='releaseDate')
title_genres: Optional[TitleGenre] = Field(default=None, alias='titleGenres')
certificate: Optional[Certificate] = None
original_title_text: Optional[TextField] = Field(default=None, alias='originalTitleText')
runtime: Optional[SecondsField] = Field(default=None, alias='runtime')
class Thumbnail(BaseModel):
url: str
width: Optional[int] = None
height: Optional[int] = None
class ImdbVideo(BaseModel):
id: str
name: ValueField
content_type: ContentType = Field(alias='contentType')
preview_urls: List[VideoUrl] = Field(default_factory=list, alias='previewURLs')
playback_urls: List[VideoUrl] = Field(default_factory=list, alias='playbackURLs')
thumbnails: Optional[Thumbnail] = None
class TitleNode(BaseModel):
title: ImdbTitle
class TitleEdge(BaseModel):
node: TitleNode

View File

@@ -3,6 +3,7 @@ import re
from datetime import datetime, timedelta
from threading import Event
from typing import Any, Dict, List, Optional, Tuple
from urllib.parse import urljoin
import pytz
from apscheduler.schedulers.background import BackgroundScheduler
@@ -33,7 +34,7 @@ class IYUUAutoSeed(_PluginBase):
# 插件图标
plugin_icon = "IYUU.png"
# 插件版本
plugin_version = "2.14"
plugin_version = "2.15"
# 插件作者
plugin_author = "jxxghp,CKun"
# 作者主页
@@ -1224,6 +1225,12 @@ class IYUUAutoSeed(_PluginBase):
"""
return True if "monikadesign." in url else False
def __is_gpw(url: str):
"""
判断是否为gpw站点
"""
return True if "greatposterwall." in url else False
def __get_mteam_enclosure(tid: str, apikey: str):
"""
获取mteam种子下载链接
@@ -1264,6 +1271,69 @@ class IYUUAutoSeed(_PluginBase):
rsskey = rss_match.group(1)
return f"{site.get('url')}torrents/download/{tid}.{rsskey}"
def __get_gpw_torrent_url_from_page(seed: dict, site: dict):
"""
从详情页面获取下载链接
"""
if not site.get('url'):
logger.warn(f"站点 {site.get('name')} 未获取站点地址,无法获取种子下载链接")
return None
try:
page_url = f"{site.get('url')}torrents.php?torrentid={seed.get('torrent_id')}&hit=1"
logger.info(f"正在获取种子下载链接:{page_url} ...")
res = RequestUtils(
cookies=site.get("cookie"),
ua=site.get("ua") or settings.USER_AGENT,
proxies=settings.PROXY if site.get("proxy") else None
).get_res(url=page_url)
if res is None or res.status_code not in (200, 500):
logger.error(f"获取种子下载链接失败,请求失败:{page_url}{res.status_code if res else ''}")
return None
# Fix encoding
if "charset=utf-8" in res.text or "charset=UTF-8" in res.text:
res.encoding = "UTF-8"
else:
res.encoding = res.apparent_encoding
if not res.text:
logger.warn(f"获取种子下载链接失败,页面内容为空:{page_url}")
return None
# 使用xpath从页面中获取下载链接
html = etree.HTML(res.text)
if html is None:
logger.warning(f"解析页面失败:{page_url}")
return None
xpath = "//a[contains(@href, 'torrents.php?action=download')]/@href"
urls = html.xpath(xpath)
if not urls:
logger.warning(f"获取种子下载链接失败,未找到下载链接:{page_url}")
return None
torrent_id = str(seed.get("torrent_id"))
matched_url = None
# Strict match using regex id=xxxx
for u in urls:
if re.search(rf"id={torrent_id}(?:&|$)", u):
matched_url = u
break
if not matched_url:
logger.warning(f"未找到与 torrent_id={torrent_id} 对应的下载链接")
return None
final_url = urljoin(site['url'], matched_url)
logger.info(f"获取种子下载链接成功:{final_url}")
return final_url
except Exception as e:
logger.warn(f"获取种子下载链接失败:{str(e)}")
return None
def __is_special_site(url: str):
"""
判断是否为特殊站点
@@ -1288,6 +1358,10 @@ class IYUUAutoSeed(_PluginBase):
if __is_monika(site.get('url')):
# 返回种子id和站点配置中所Monika的rss链接
return __get_monika_torrent(tid=seed.get("torrent_id"), rssurl=site.get("rss"))
if __is_gpw(site.get('url')):
# 从详情页面获取下载链接
return __get_gpw_torrent_url_from_page(seed=seed, site=site)
elif __is_special_site(site.get('url')):
# 从详情页面获取下载链接
return self.__get_torrent_url_from_page(seed=seed, site=site)

View File

@@ -1,25 +1,32 @@
# 美剧生词标注
根据CEFR等级为英语影视剧标注高级词汇。
___
在影视剧入库后LexiAnnot 会读取媒体文件的MediaInfo和文件列表如果视频的原始语言为英语并且包含英文文本字幕LexiAnnot将为其生成包含词汇注释的`.en.ass`字幕文件。
在影视剧入库后LexiAnnot会读取媒体文件的MediaInfo和文件列表如果视频的原始语言为英语并且包含英文文本字幕LexiAnnot将为其生成包含词汇注释的.ass字幕文件。
## 主要功能
![](https://images2.imgbox.com/d6/b6/kZu6EH2a_o.png)
![](https://images2.imgbox.com/c8/3a/rEJBWu5v_o.png)
![](https://images2.imgbox.com/97/b7/d6RXFtwD_o.png)
![](https://images2.imgbox.com/56/c0/FBhJMvRD_o.jpg)
![](https://images2.imgbox.com/e8/8c/B1EJwst7_o.jpg)
![](https://images2.imgbox.com/8a/d4/AtgOe265_o.jpg)
# Gemini
- 识别视频的原始语言和字幕语言
- 自动适应原字幕样式
- 俚语 / 自造词 / 熟词生义标注和解释
- **[获取APIKEY](https://aistudio.google.com/app/apikey)**
- **[速率限制](https://ai.google.dev/gemini-api/docs/rate-limits)**
## 使用配置
**确保可以正常访问下面的域名**
- spaCy 模型
- spaCy 用于词形还原、POS 标注和命名实体识别,`en_core_web_sm``en_core_web_md` 已足够满足需求。
- LLM 设置
- 一集影视剧的字幕通常包含数千个单词,建议使用支持长文本输入的模型,选择一个适当的上下文窗口大小。
- 处理 60 min 的影视剧字幕大约会消耗 `60K`~`80K` token具体取决于字幕内容。
- 配置请参考 MoviePilot 智能助手的设置部分。
- Agent 工具
- 在聊天中使用 `/ai` 命令告诉智能助手你要标注的影视剧。
- googleapis.com
- google.dev
- aistudio.google.com
# CEFR
## CEFR
CEFR全称是Common European Framework of Reference for Languages。
@@ -35,20 +42,18 @@ CEFR全称是Common European Framework of Reference for Languages。
- **C1** (高级/Advanced):能够理解各种较长、要求较高的文本,并能识别隐含意义,表达流利、自然,能灵活有效地使用语言来应对各种目的。
- **C2** (精通/Proficient):能够轻松理解几乎所有听到的或读到的内容,能够非常流利、准确、精细地表达自己,即使在复杂的情况下也能区分细微的含义。
# 计划
## 计划
- 双语字幕支持
- 考试词汇标注
- ~~考试词汇标注~~
# FAQ
## FAQ
- **为什么需要用到Gemini**
- LexiAnnot使用的词典仅包含约18000个单词无法覆盖影视剧中的海量的俚语、习语、流行语等更广泛的表达形式
- **只能处理已有字幕的视频吗?**
- 是的,视频需要包含**英文文本字幕**
- **为什么无法处理一些包含字幕视频**
- 目前无法识别基于图片的字幕(通常是特效字幕)
# 感谢
## 感谢
- [coca-vocabulary-20000](https://github.com/llt22/coca-vocabulary-20000)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,134 @@
import asyncio
from typing import Optional, Type
from pydantic import BaseModel
from app.agent.tools.base import MoviePilotTool
from app.core.plugin import PluginManager
from .schemas import VocabularyAnnotatingToolInput, QueryAnnotationTasksToolInput, Task
class VocabularyAnnotatingTool(MoviePilotTool):
"""词汇标注工具"""
# 工具名称
name: str = "vocabulary_annotating_tool"
# 工具描述
description: str = (
"Add new vocabulary annotation task to plugin LexiAnnot's task queue."
)
# 输入参数模型
args_schema: Type[BaseModel] = VocabularyAnnotatingToolInput
def get_tool_message(self, **kwargs) -> Optional[str]:
"""根据订阅参数生成友好的提示消息"""
skip_existing = kwargs.get("skip_existing", False)
video_path = kwargs.get("video_path", "")
message = f"正在添加字幕任务: {video_path!r}"
if skip_existing:
message += "(覆写方式:跳过已存在的字幕文件)"
else:
message += "(覆写方式:覆盖已存在的字幕文件)"
return message
async def run(self, video_path: str, skip_existing: bool = True, **kwargs) -> str:
"""
实现工具的核心逻辑(异步方法)
:param video_path: Path to the video file
:param skip_existing: Whether to skip existing subtitle files
:param kwargs: 其他参数,包含 explanation工具使用说明
:return: 工具执行结果,返回字符串格式
"""
try:
# 执行工具逻辑
result = await self._perform_operation(video_path, skip_existing)
# 返回执行结果
if not result:
return f"成功添加词汇标注任务: {video_path!r}"
else:
return f"添加任务出错: {result}"
except Exception as e:
return f"执行失败: {str(e)}"
async def _perform_operation(
self, video_path: str, skip_existing: bool
) -> str | None:
"""内部方法,执行具体操作"""
# 实现具体业务逻辑
plugins = PluginManager().running_plugins
plugin_instance = plugins.get("LexiAnnot")
if not plugin_instance:
return "LexiAnnot 插件未运行"
res = await asyncio.to_thread(
plugin_instance.add_task, video_file=video_path, skip_existing=skip_existing
)
if not res:
return "任务添加失败"
return None
class QueryAnnotationTasksTool(MoviePilotTool):
"""词汇标注任务查询工具"""
# 工具名称
name: str = "query_annotation_tasks_tool"
# 工具描述
description: str = (
"Query the latest vocabulary annotation tasks from plugin LexiAnnot."
)
# 输入参数模型
args_schema: Type[BaseModel] = QueryAnnotationTasksToolInput
def get_tool_message(self, **kwargs) -> Optional[str]:
"""根据订阅参数生成友好的提示消息"""
count = kwargs.get("count", 5)
return f"正在查询最近的 {count} 条字幕标注任务"
async def run(self, count: int, **kwargs) -> str:
"""
实现工具的核心逻辑(异步方法)
:param count: The max number of returned annotation tasks
:param kwargs: 其他参数,包含 explanation工具使用说明
:return: 工具执行结果,返回字符串格式
"""
try:
# 执行工具逻辑
plugins = PluginManager().running_plugins
plugin_instance = plugins.get("LexiAnnot")
if not plugin_instance:
return "LexiAnnot 插件未运行"
total: list[Task] = plugin_instance.get_tasks()
# Handle potential None in add_time
total.sort(key=lambda t: t.add_time or "", reverse=True)
tasks = total[:count]
if not tasks:
return "未查询到相关任务"
result_lines = [f"最近 {len(tasks)} 条标注任务:"]
for task in tasks:
status_val = (
task.status.value
if hasattr(task.status, "value")
else str(task.status)
)
info = f"\n🎥 **{task.video_path}**"
info += f"\n ID: {task.task_id}"
info += f"\n Status: {status_val}"
info += f"\n Added: {task.add_time or 'N/A'}"
if task.complete_time:
info += f"\n Completed: {task.complete_time}"
if task.message:
info += f"\n Message: {task.message}"
if task.statistics:
info += f"\n Words: {task.statistics.total_words} | Segments: {task.statistics.total_segments}"
result_lines.append(info)
return "\n".join(result_lines)
except Exception as e:
return f"执行失败: {str(e)}"

View File

@@ -0,0 +1,116 @@
from typing import Literal
from pydantic import BaseModel, Field, RootModel
from .schemas import PosDef, Cefr
class CefrEntry(BaseModel):
pos: Literal[
"noun",
"adverb",
"interjection",
"preposition",
"determiner",
"have-verb",
"modal auxiliary",
"adjective",
"number",
"be-verb",
"verb",
"conjunction",
"do-verb",
"infinitive-to",
"vern",
"pos",
"pronoun",
] = Field(..., description="Part of speech")
cefr: Cefr = Field(..., description="CEFR level")
notes: str | None = Field(default=None, description="Notes")
class CefrDictionary(RootModel):
root: dict[str, list[CefrEntry]]
def get(self, word: str) -> list[CefrEntry] | None:
return self.root.get(word)
class Coca20KEntry(BaseModel):
index: int = Field(..., description="Index of the entry")
phonetics_1: str = Field(..., description="Phonetics style 1")
phonetics_2: str = Field(..., description="Phonetics style 2")
pos_defs: list[PosDef] = Field(
..., description="List of part of speech definitions"
)
class Coca20KDictionary(RootModel):
root: dict[str, Coca20KEntry]
def get(self, word: str) -> Coca20KEntry | None:
return self.root.get(word)
class ShanBayDef(BaseModel):
# 'n.', 'v.', 'adv.', 'adj.', 'phrase.', 'int.', 'pron.', 'prep.', '.', 'conj.', 'num.', 'phrase v.', 'linkv.',
# 'det.', 'ordnumber.', 'prefix.', 'un.', 'vt.', 'mod. v.', 'abbr.', 'auxv.', 'modalv.', 'vi.', 'aux. v.',
# 'interj.', 'article.', 'infinitive.', 'suff.', 'ord.', 'art.', 'exclam.', 'n.[C]'
pos: str = Field(..., description="Part of speech")
definition_cn: str = Field(..., description="Definition in Chinese")
class ShanbayEntry(BaseModel):
ipa_uk: str = Field(..., description="UK IPA pronunciation")
ipa_us: str = Field(..., description="US IPA pronunciation")
defs: list[ShanBayDef] = Field(..., description="List of definitions")
class ShanbayDictionary(BaseModel):
"""Dictionary entries for various examinations."""
cet4: dict[str, ShanbayEntry] = Field(
..., alias="CET-4", description="CET-4 dictionary entries"
)
cet6: dict[str, ShanbayEntry] = Field(
..., alias="CET-6", description="CET-6 dictionary entries"
)
npee: dict[str, ShanbayEntry] = Field(
..., alias="NPEE", description="NPEE dictionary entries"
)
ielts: dict[str, ShanbayEntry] = Field(
..., alias="IELTS", description="IELTS dictionary entries"
)
toefl: dict[str, ShanbayEntry] = Field(
..., alias="TOEFL", description="TOEFL dictionary entries"
)
gre: dict[str, ShanbayEntry] = Field(
..., alias="GRE", description="GRE dictionary entries"
)
tem4: dict[str, ShanbayEntry] = Field(
..., alias="TEM-4", description="TEM-4 dictionary entries"
)
tem8: dict[str, ShanbayEntry] = Field(
..., alias="TEM-8", description="TEM-8 dictionary entries"
)
pet: dict[str, ShanbayEntry] = Field(
..., alias="PET", description="PET dictionary entries"
)
def query(self, word: str) -> dict[str, ShanbayEntry]:
result = {}
for field_name, field_info in ShanbayDictionary.model_fields.items():
value = getattr(self, field_name)
if word in value:
result[field_info.alias] = value[word]
return result
class Lexicon(BaseModel):
cefr: CefrDictionary = Field(..., description="CEFR dictionary")
coca20k: Coca20KDictionary = Field(..., description="COCA 20K dictionary")
examinations: ShanbayDictionary = Field(
..., description="Shanbay examinations dictionary"
)
swear_words: list[str] = Field(..., description="List of swear words")
version: str = Field(..., description="Version of the lexicon")

View File

@@ -0,0 +1,733 @@
import re
import threading
from langchain_core.language_models.chat_models import BaseChatModel
from langchain_core.prompts import ChatPromptTemplate
from langchain.output_parsers import PydanticOutputParser
from pydantic import SecretStr
from app.core.config import settings
from app.schemas import Context
from app.schemas.types import MediaType
from app.log import logger
from .lexicon import CefrDictionary, Lexicon, Coca20KDictionary
from .schemas import (
SubtitleSegment,
PosDef,
Word,
Cefr,
WordMetadata,
SegmentList,
LlmFeedback,
UniversalPos,
LlmEnrichmentResult,
LlmTranslationResult,
)
from .spacyworker import SpacyWorker
_patterns = [
r"\d+th|\d?1st|\d?2nd|\d?3rd",
r"\w+'s$",
r"\w+'d$",
r"\w+'t$",
"[Ii]'m$",
r"\w+'re$",
r"\w+'ve$",
r"\w+'ll$",
]
filter_patterns: list[re.Pattern] = [re.compile(p) for p in _patterns]
pos_interests = {"NOUN", "VERB", "ADJ", "ADV", "ADP", "CCONJ", "SCONJ"}
UNIVERSAL_POS_MAP: dict[UniversalPos, str] = {
UniversalPos.ADJ: "adj.",
UniversalPos.ADV: "adv.",
UniversalPos.INTJ: "int.",
UniversalPos.NOUN: "n.",
UniversalPos.PROPN: "n.",
UniversalPos.VERB: "v.",
UniversalPos.AUX: "aux.",
UniversalPos.ADP: "prep.",
UniversalPos.CCONJ: "conj.",
UniversalPos.SCONJ: "conj.",
UniversalPos.DET: "det.",
UniversalPos.NUM: "num.",
UniversalPos.PART: "part.",
UniversalPos.PRON: "pron.",
UniversalPos.PUNCT: None,
UniversalPos.SYM: None,
UniversalPos.X: None,
}
def initialize_llm(
provider: str,
api_key: str,
model_name: str,
base_url: str | None,
temperature: float = 0.1,
max_retries: int = 3,
proxy: bool = False,
) -> BaseChatModel:
"""初始化 LLM"""
if provider == "google":
if proxy:
from langchain_openai import ChatOpenAI
return ChatOpenAI(
model=settings.LLM_MODEL,
api_key=SecretStr(api_key),
max_retries=3,
base_url="https://generativelanguage.googleapis.com/v1beta/openai",
temperature=settings.LLM_TEMPERATURE,
openai_proxy=settings.PROXY_HOST,
)
from langchain_google_genai import ChatGoogleGenerativeAI
return ChatGoogleGenerativeAI(
model=model_name,
google_api_key=api_key, # noqa
max_retries=max_retries,
temperature=temperature,
)
elif provider == "deepseek":
from langchain_deepseek import ChatDeepSeek
return ChatDeepSeek(
model=model_name,
api_key=SecretStr(api_key),
max_retries=max_retries,
temperature=temperature,
)
else:
from langchain_openai import ChatOpenAI
return ChatOpenAI(
model=model_name,
api_key=SecretStr(api_key),
max_retries=max_retries,
base_url=base_url,
temperature=temperature,
openai_proxy=settings.PROXY_HOST if proxy else None,
)
def convert_pos_to_spacy(pos: str):
"""
将给定的词性列表转换为 spaCy 库中使用的词性标签
:param pos: 字符串形式词性
:returns: 一个包含对应spaCy词性标签的列表。对于无法直接映射的词性将返回None
"""
spacy_pos_map = {
"noun": "NOUN",
"adjective": "ADJ",
"adverb": "ADV",
"verb": "VERB",
"preposition": "ADP",
"conjunction": "CCONJ",
"determiner": "DET",
"pronoun": "PRON",
"interjection": "INTJ",
"number": "NUM",
}
pos_lower = pos.lower()
if pos_lower in spacy_pos_map:
spacy_pos = spacy_pos_map[pos_lower]
elif pos_lower == "be-verb":
spacy_pos = "AUX" # Auxiliary verb (e.g., be, do, have)
elif pos_lower == "vern":
spacy_pos = "VERB" # Assuming 'vern' is a typo for 'verb'
elif pos_lower == "modal auxiliary":
spacy_pos = "AUX" # Modal verbs are also auxiliaries
elif pos_lower == "do-verb":
spacy_pos = "AUX"
elif pos_lower == "have-verb":
spacy_pos = "AUX"
elif pos_lower == "infinitive-to":
spacy_pos = "PART" # Particle (e.g., to in "to go")
elif not pos_lower: # Handle empty strings
spacy_pos = None
else:
spacy_pos = None # For unmapped POS tags
return spacy_pos
def convert_spacy_to_universal(spacy_pos: str) -> UniversalPos:
"""
将 spaCy POS 标签转换为 UniversalPos 枚举
"""
# 创建映射字典
pos_mapping = {
"ADJ": UniversalPos.ADJ,
"ADV": UniversalPos.ADV,
"INTJ": UniversalPos.INTJ,
"NOUN": UniversalPos.NOUN,
"PROPN": UniversalPos.PROPN,
"VERB": UniversalPos.VERB,
"AUX": UniversalPos.AUX,
# 介词/后置词
"ADP": UniversalPos.ADP,
# 连词
"CCONJ": UniversalPos.CCONJ,
"SCONJ": UniversalPos.SCONJ,
# 限定词
"DET": UniversalPos.DET,
# 数词
"NUM": UniversalPos.NUM,
# 代词
"PRON": UniversalPos.PRON,
# 小品词
"PART": UniversalPos.PART,
# 标点
"PUNCT": UniversalPos.PUNCT,
# 符号
"SYM": UniversalPos.SYM,
# 其他
"X": UniversalPos.X,
# 特殊处理spaCy 可能返回的其他标签
"SPACE": UniversalPos.PUNCT, # 空格当作标点处理
"CONJ": UniversalPos.CCONJ, # 旧版 spaCy 的连词标签
}
# 转换为大写,确保一致
spacy_pos = spacy_pos.upper()
# 如果直接匹配,返回对应枚举
if spacy_pos in pos_mapping:
return pos_mapping[spacy_pos]
# 处理特殊情况:以特定前缀开头的标签
if spacy_pos.startswith("ADJ"):
return UniversalPos.ADJ
elif spacy_pos.startswith("ADV"):
return UniversalPos.ADV
elif spacy_pos.startswith("NOUN"):
return UniversalPos.NOUN
elif spacy_pos.startswith("VERB"):
return UniversalPos.VERB
elif spacy_pos.startswith("PROPN"):
return UniversalPos.PROPN
elif spacy_pos.startswith("PRON"):
return UniversalPos.PRON
# 默认返回 X未知
return UniversalPos.X
def get_cefr_by_spacy(
lemma_: str, pos_: str, cefr_lexicon: CefrDictionary
) -> Cefr | None:
word = lemma_.lower().strip("-*'")
result = cefr_lexicon.get(word)
if result:
all_cefr: list[Cefr] = []
if len(result) > 0:
for entry in result:
if pos_ == convert_pos_to_spacy(entry.pos):
return entry.cefr
all_cefr.append(entry.cefr)
return min(all_cefr)
return None
def query_coca20k(word: str, coca20k: Coca20KDictionary):
word = word.lower().strip("-*'")
return coca20k.get(word)
def _update_word_via_lexicon(word: Word, lexi: Lexicon) -> Word:
"""
使用词典信息更新单词对象
:param word: 需要更新的单词对象
:param lexi: 词典对象
:returns: 更新后的单词对象
"""
# query dictionary
cefr = get_cefr_by_spacy(word.lemma, word.pos.value, lexi.cefr)
res_of_coca = query_coca20k(word.lemma, lexi.coca20k)
if res_of_coca and not cefr:
cefr = None
res_of_exams = lexi.examinations.query(word.lemma)
exam_tags = [exam_id for exam_id in res_of_exams if exam_id in res_of_exams]
pos_defs = []
phonetics = ""
if res_of_exams:
for exam, value in res_of_exams.items():
phonetics = value.ipa_uk
defs = {}
for pos_def in value.defs:
pos = pos_def.pos
definition_cn = pos_def.definition_cn
defs.setdefault(pos, []).append(definition_cn)
for pos, meanings in defs.items():
pos_defs.append(PosDef(pos=pos, meanings=meanings))
break
elif res_of_coca:
phonetics = res_of_coca.phonetics_1
pos_defs = res_of_coca.pos_defs
word.exams = exam_tags
word.cefr = cefr
word.pos_defs = pos_defs
word.phonetics = phonetics
return word
def extract_advanced_words(segment: SubtitleSegment, lexi: Lexicon, spacy_worker: SpacyWorker,
simple_level: set[Cefr]) -> list[Word]:
text = segment.clean_text
doc = spacy_worker.submit(text)
last_end_pos = 0
lemma_to_query = []
words = []
for token in doc.tokens:
# filter tokens
if (
len(token.text) == 1
or token.is_stop
or token.is_punct
or token.ent_iob_ != "O"
):
continue
if token.pos_ not in pos_interests:
continue
if token.lemma_ in lexi.swear_words:
continue
striped = token.lemma_.strip("-[")
if any(p.match(striped) for p in filter_patterns):
continue
if striped in lemma_to_query:
continue
else:
lemma_to_query.append(striped)
striped_text = token.text.strip("-*[")
start_pos = text.find(striped_text, last_end_pos)
end_pos = start_pos + len(striped_text)
last_end_pos = end_pos
word = Word(
text=striped_text,
lemma=striped,
pos=convert_spacy_to_universal(token.pos_),
meta=WordMetadata(
start_pos=start_pos, end_pos=end_pos, context_id=segment.index
),
)
word = _update_word_via_lexicon(word, lexi)
if word.cefr and word.cefr in simple_level:
continue
words.append(word)
return words
def _find_segment_by_word_id(segments: list[SubtitleSegment], word_id: int) -> SubtitleSegment | None:
for segment in segments:
for word in segment.candidate_words:
if word.meta.word_id == word_id:
return segment
return None
def _update_word_metadata(
new_text: str, meta: WordMetadata, segment: SubtitleSegment
) -> WordMetadata | None:
"""
更新单词的元数据
:param new_text: 新的单词文本
:param meta: 单词的元数据对象
:param segment: 字幕片段对象
"""
text = segment.clean_text
p_end = meta.end_pos
new_len = len(new_text)
i = meta.start_pos - new_len + 1
i = max(0, i)
j = p_end + min(0, (len(text) - (p_end + new_len)))
for x in range(i, j + 1):
text_view = text[x : (x + new_len)]
if text_view == new_text:
return WordMetadata(
start_pos=x,
end_pos=x + new_len,
context_id=segment.index,
word_id=meta.word_id,
)
return None
def format_time_extended(milliseconds: int):
"""
将秒数转换为时间格式
:param milliseconds: 整数,表示毫秒数
:return: 字符串,格式为 HH:MM:SS 或 HH:MM:SS.mmm
"""
if milliseconds < 0:
sign = "-"
milliseconds = abs(milliseconds)
else:
sign = ""
hours = int(milliseconds // 3600000)
minutes = int((milliseconds % 3600000) // 60000)
seconds = (milliseconds % 60000) // 1000
milliseconds_remainder = milliseconds % 1000
return f"{sign}{hours:02d}:{minutes:02d}:{seconds:02d}.{milliseconds_remainder:03d}"
def _context_process_chain(
lexi: Lexicon,
llm: BaseChatModel,
segments: list[SubtitleSegment],
start: int,
end: int,
leaner_level: str = "C1",
media_name: str | None = None,
translate_sentences: bool = False
):
feedback_parser = PydanticOutputParser(pydantic_object=LlmFeedback)
def format_input(segment_list: list[SubtitleSegment]):
media_name_prefix = (
f"The following subtitles are from '{media_name}'.\n" if media_name else ""
)
return {
"media_name_prefix": media_name_prefix,
"context_text": " ".join([seg.clean_text for seg in segment_list]),
"candidate_words": "\n".join(
[
f"- {word.text} (WORD_ID: {word.meta.word_id}, LEMMA: {word.lemma}, CEFR: {word.cefr}, POS: {word.pos})"
for seg in segment_list
for word in seg.candidate_words
]
),
"leaner_level": leaner_level,
"format_instructions": feedback_parser.get_format_instructions(),
}
def refactor_by_feedback(feedback: LlmFeedback):
# Process LLM feedback to update segments
for word in feedback.candidate_words_feedback:
seg = _find_segment_by_word_id(segments, word.word_id)
if not seg or seg.index < start or seg.index > end:
continue
# Update word info based on feedback
if not word.should_keep:
seg.candidate_words = [
w for w in seg.candidate_words if w.meta.word_id != word.word_id
]
continue
for w in seg.candidate_words:
if w.meta.word_id == word.word_id:
word_text = word.text
if word_text is not None and word.text != w.text:
# Update metadata if text changed
if word.text not in seg.clean_text:
# If the word text is not found in the segment, skip updating metadata
continue
new_meta = _update_word_metadata(word_text, w.meta, seg)
if not new_meta:
continue
w.meta = new_meta
w.text = word_text
if word.pos:
w.pos = word.pos
if word.lemma:
w.lemma = word.lemma
# Add new words identified by LLM
for new_word in feedback.llm_identified_words:
for seg in segments:
if seg.index < start or seg.index > end:
continue
start_pos = seg.clean_text.find(new_word.text)
if start_pos == -1:
continue
if any(w.text == new_word.text for w in seg.candidate_words):
continue
if new_word.lemma in lexi.swear_words:
continue
new_meta = WordMetadata(
start_pos=start_pos,
end_pos=start_pos + len(new_word.text),
context_id=seg.index
)
built_word = Word(
text=new_word.text,
lemma=new_word.lemma,
pos=new_word.pos,
meta=new_meta
)
built_word = _update_word_via_lexicon(built_word, lexi)
if built_word.cefr and built_word.cefr < leaner_level:
continue
seg.candidate_words.append(built_word)
prompt_template = ChatPromptTemplate.from_messages(
[
(
"system",
"""You are an expert in linguistics and language learning. Your task is to analyze subtitle segments.
Please perform the following tasks for an English learner at {leaner_level} CEFR level.
**CRITICAL INSTRUCTION**: The learner is advanced. They already know common daily vocabulary.
Your goal is to identify **only** content that helps them reach native-level proficiency.
1. **Review and Evaluate Candidate Words:**
* **Goal**: Filter out simple words and correct any errors in lemma/POS/text.
* **Action**: Return feedback items **ONLY** for words that:
1. Should be **discarded** (too simple, trivial filler, profanity without cultural value). Set `should_keep` to `False`.
2. Need **correction** (wrong lemma, POS, or text boundary). Set `should_keep` to `True` and provide correct values.
* **Implicit Rule**: If a word is appropriate for the learner and has correct info, **DO NOT** include it in the output list.
* **Keep criteria**: Keep simple words **ONLY IF** used in a non-literal, metaphorical, or idiomatic sense.
* **Discard criteria**: Discard trivial conversational fillers ('gonna', 'wanna'), simple interjections, common profanity, and words below {leaner_level} level.
2. **Identify Missed Words:**
* Identify any additional single words or phrases (typically 1-3 words) from the `context_text` that may be important for {leaner_level} learners. This specifically includes:
* **Slang or informal expressions.**
* **Internet terms or modern colloquialisms.**
* **Words or phrases that require specific cultural background knowledge to understand.**
* **Any other words or phrases that are challenging.**
* Avoid repeating words already listed in `candidate_words`.
* Must exist in the exact form in `context_text`.
* Provide lemma and POS.
* **Do NOT include** simple high-frequency words, common fillers ('gonna', 'gotta'), or basic swear words.
-------------------------
You MUST return output strictly matching the provided Pydantic schema.
Return ONLY valid JSON.
**Here are the output format instructions you MUST follow strictly:**
{format_instructions}
""",
),
(
"human",
"""{media_name_prefix}Here is the context from the subtitles:
---
{context_text}
---
Here are the candidate words identified by a basic algorithm:
{candidate_words}
""",
),
]
)
feedback_chain = (
format_input | prompt_template | llm.with_structured_output(LlmFeedback).with_retry(stop_after_attempt=3)
)
result: LlmFeedback = feedback_chain.invoke(segments) # type: ignore
refactor_by_feedback(result)
# 丰富词义
if any(segment.candidate_words for segment in segments):
enrichment_prompt_template = ChatPromptTemplate.from_messages(
[
(
"system",
"""You are a linguistics and English-learning expert. Your goal is to enhance vocabulary learning for Chinese users.\n
For each word (identified by `WORD_ID`), provide:
1. **Translation:** A concise Chinese translation.
2. **Usage or Cultural Context (optional, in Chinese)**:
* **Keep it brief and clear.**
* ONLY include if:
- The word has a specific meaning in this context that differs from its common definition;
- It is slang, idiom, phrasal, metaphorical, or culturally loaded;
* ONLY provide this context when learners would likely struggle to understand the word's usage without it.
3. **Lexical Features**:
* Select the most appropriate tag(s) if applicable.
**For each word, provide the `word_id` to ensure proper mapping.**
**Your judgment should be based strictly on the provided subtitle context. DO NOT fabricate context or forced explanation.**
-------------------------
You MUST return output strictly matching the provided Pydantic schema.
Return ONLY valid JSON.
**Here are the output format instructions you MUST follow strictly:**
{format_instructions}
""",
),
(
"human",
"""{media_name_prefix}Here is the context from the subtitles:
---
{context_text}
---
Here are the words you need to enrich:
{words_to_enrich}
""",
),
]
)
enrichment_parser = PydanticOutputParser(pydantic_object=LlmEnrichmentResult)
def format_enrichment_input(segment_list: list[SubtitleSegment]):
media_name_prefix = (
f"The following subtitles are from '{media_name}'.\n"
if media_name
else ""
)
words_to_enrich = []
for seg in segment_list:
if start <= seg.index <= end:
for w in seg.candidate_words:
words_to_enrich.append(
f"- {w.text} (WORD_ID: {w.meta.word_id}, LEMMA: {w.lemma}, POS: {w.pos}, DEFINITIONS: {w.pos_defs_plaintext})"
)
return {
"media_name_prefix": media_name_prefix,
"context_text": " ".join([seg.clean_text for seg in segment_list]),
"words_to_enrich": "\n".join(words_to_enrich),
"format_instructions": enrichment_parser.get_format_instructions(),
}
enrichment_chain = (
format_enrichment_input
| enrichment_prompt_template
| llm.with_structured_output(LlmEnrichmentResult).with_retry(stop_after_attempt=3)
)
enrichment_result: LlmEnrichmentResult = enrichment_chain.invoke(segments) # type: ignore
for enriched_word_data in enrichment_result.enriched_words:
for segment in segments:
if segment.index < start or segment.index > end:
continue
for candidate_word in segment.candidate_words:
if candidate_word.meta.word_id == enriched_word_data.word_id:
candidate_word.llm_translation = enriched_word_data.translation
candidate_word.llm_usage_context = enriched_word_data.usage_context
candidate_word.lexical_features = enriched_word_data.lexical_features
break
# 整句翻译
if translate_sentences:
translation_parser = PydanticOutputParser(pydantic_object=LlmTranslationResult)
translation_prompt_template = ChatPromptTemplate.from_messages(
[
(
"system",
"""You are a professional subtitle translator. Your task is to translate English subtitle segments into natural, idiomatic Chinese.
**Guidelines:**
1. **Tone & Style:** Maintain the original tone (e.g., casual, formal, humorous, dramatic).
2. **Context:** Use the surrounding segments to ensure continuity and correct meaning.
3. **Conciseness:** Subtitles have space constraints. Keep translations concise but accurate.
4. **Formatting:** Return the result strictly matching the provided JSON schema.
-------------------------
You MUST return output strictly matching the provided Pydantic schema.
Return ONLY valid JSON.
**Here are the output format instructions you MUST follow strictly:**
{format_instructions}
""",
),
(
"human",
"""{media_name_prefix}Here are the segments to translate:
---
{segments_text}
---
""",
),
]
)
def format_translation_input(segment_list: list[SubtitleSegment]):
media_name_prefix = (
f"The following subtitles are from '{media_name}'.\n"
if media_name
else ""
)
# Only translate segments within the current batch range (start to end)
segments_text_lines = []
for seg in segment_list:
if start <= seg.index <= end:
segments_text_lines.append(f"ID {seg.index}: {seg.clean_text}")
return {
"media_name_prefix": media_name_prefix,
"segments_text": "\n".join(segments_text_lines),
"format_instructions": translation_parser.get_format_instructions(),
}
translation_chain = (
format_translation_input
| translation_prompt_template
| llm.with_structured_output(LlmTranslationResult).with_retry(stop_after_attempt=3)
)
try:
translation_result: LlmTranslationResult = translation_chain.invoke(segments) # type: ignore
# Map translations back to segments
trans_map = {
t.index: t.translation for t in translation_result.translations
}
for segment in segments:
if segment.index in trans_map:
segment.Chinese = trans_map[segment.index]
except Exception as e:
logger.error(f"Error during sentence translation: {e}")
return [segment for segment in segments if start <= segment.index <= end]
def llm_process_chain(
lexi: Lexicon,
llm: BaseChatModel,
segments: SegmentList,
shutdown_event: threading.Event,
context_window: int = 30,
leaner_level: str = "C1",
media_context: Context | None = None,
translate_sentences: bool = False,
) -> SegmentList:
"""
根据 LLM 的反馈更新字幕片段中的单词信息
:param lexi: 词典对象
:param llm: LLM 对象
:param segments: 字幕片段
:param shutdown_event: 关闭事件
:param context_window: 上下文窗口大小
:param leaner_level: 学习者的 CEFR 水平
:param media_context: 媒体信息
:param translate_sentences: 是否翻译句子
:returns: 更新后的字幕片段列表
"""
media_name = None
if media_context and media_context.media_info and media_context.meta_info:
media_info = media_context.media_info
if media_info.type == MediaType.TV:
media_name = f"{media_info.title_year} {media_context.meta_info.season_episode}"
else:
media_name = f"{media_info.title_year}"
segments_list = []
for context, (start, end) in segments.context_generator(context_window=context_window, extra_len=2):
if shutdown_event.is_set():
break
logger.info(
f"Processing segments {format_time_extended(context[0].start_time)} ({context[0].index}) ->"
f" {format_time_extended(context[-1].end_time)} ({context[-1].index}) via LLM..."
)
segments_list.extend(
_context_process_chain(
lexi, llm, context, start, end, leaner_level, media_name, translate_sentences
)
)
return SegmentList(root=segments_list)

View File

@@ -1,220 +0,0 @@
import sys
import json
import time
from typing import List, Dict, Any, Type, Union
from pydantic import BaseModel, ValidationError
class Context(BaseModel):
original_text: str
class Vocabulary(BaseModel):
lemma: str
Chinese: str
class VocabularyTranslationTask(BaseModel):
index: int
vocabulary: List[Vocabulary]
context: Context
class DialogueTranslationTask(BaseModel):
index: int
original_text: str
Chinese: str
class GeminiResponse(BaseModel):
tasks: List[Union[VocabularyTranslationTask, DialogueTranslationTask]]
total_token_count: int
success: bool
message: str = ""
def validate_input_data(request_data: Dict[str, Any]) -> None:
"""Validate the input data structure"""
if not isinstance(request_data, dict):
raise ValueError("Input data must be a dictionary")
if "tasks" not in request_data:
raise ValueError("Missing 'tasks' in input data")
if "params" not in request_data:
raise ValueError("Missing 'params' in input data")
params = request_data["params"]
required_params = ["api_key", "system_instruction", "schema"]
for param in required_params:
if param not in params:
raise ValueError(f"Missing required parameter: {param}")
def get_task_schema(schema_name: str) -> Type[BaseModel]:
"""Get the appropriate schema class based on the schema name"""
schema_map = {
'DialogueTranslationTask': DialogueTranslationTask,
'VocabularyTranslationTask': VocabularyTranslationTask
}
if schema_name not in schema_map:
raise ValueError(f"Unknown schema name: {schema_name}")
return schema_map[schema_name]
def query_gemini(
api_key: str,
translation_tasks: List[Dict[str, Any]],
task_schema: Type[Union[VocabularyTranslationTask, DialogueTranslationTask]],
system_instruction: str,
gemini_model: str = "gemini-2.0-flash",
temperature: float = 0.3,
max_retries: int = 3,
retry_delay: int = 10
) -> GeminiResponse:
"""
Query the Gemini API for translation tasks with retry logic.
Args:
api_key: Gemini API key
translation_tasks: List of translation tasks
task_schema: Pydantic model for the task type
system_instruction: System instruction for the model
gemini_model: Model name to use
temperature: Generation temperature
max_retries: Number of retry attempts
retry_delay: Delay between retries in seconds
Returns:
GeminiResponse containing the results
"""
from google import genai
from google.genai import types
from google.genai.types import SchemaUnion
client = genai.Client(api_key=api_key)
messages = []
translation_res = []
total_token_count = 0
# Validate input tasks before sending to API
try:
translation_res = [task_schema(**task) for task in translation_tasks]
except ValidationError as e:
return GeminiResponse(
tasks=[],
total_token_count=0,
success=False,
message=f"Input validation failed: {str(e)}"
)
for attempt in range(1, max_retries + 1):
try:
response = client.models.generate_content(
model=gemini_model,
contents=json.dumps(translation_tasks, ensure_ascii=False),
config=types.GenerateContentConfig(
system_instruction=system_instruction,
response_mime_type="application/json",
response_schema=list[task_schema],
temperature=temperature
),
)
if not response.parsed:
raise ValueError("Empty response from Gemini API")
translation_res = response.parsed
total_token_count = response.usage_metadata.total_token_count
return GeminiResponse(
tasks=translation_res,
total_token_count=total_token_count,
success=True
)
except Exception as e:
messages.append(f"Attempt {attempt} failed: {str(e)}")
if attempt < max_retries:
time.sleep(retry_delay)
return GeminiResponse(
tasks=[],
total_token_count=0,
success=False,
message="All retry attempts failed. " + "\n".join(messages)
)
def main():
try:
# Read and parse input
'''{
"tasks": [{
"index": 0,
"original_text": "That was eight years ago.",
"Chinese": ""
}, {
"index": 1,
"original_text": "Much has changed.",
"Chinese": ""
}],
"params": {
"api_key": "",
"system_instruction": "You are an expert translator. You will be given a list of dialogue translation tasks in JSON format. For each entry, provide the most appropriate translation in Simplified Chinese based on the context. \\nOnly complete the `Chinese` field. Do not include pinyin, explanations, or any additional information.",
"schema": "DialogueTranslationTask"
}
}'''
input_text = sys.stdin.read()
if not input_text:
raise ValueError("No input provided")
request_data = json.loads(input_text)
validate_input_data(request_data)
# Extract parameters
tasks = request_data["tasks"]
params = request_data["params"]
# Get schema and make API call
schema = get_task_schema(params["schema"])
response = query_gemini(
api_key=params["api_key"],
translation_tasks=tasks,
task_schema=schema,
system_instruction=params["system_instruction"],
gemini_model=params.get("model", "gemini-2.0-flash"),
temperature=float(params.get("temperature", 0.3)),
max_retries=int(params.get("max_retries", 3))
)
# Prepare output
if response.success:
result = {
"success": True,
"data": {
"tasks": [task.model_dump() for task in response.tasks],
"total_token_count": response.total_token_count
}
}
else:
result = {
"success": False,
"message": response.message
}
print(json.dumps(result, ensure_ascii=False))
except json.JSONDecodeError as e:
error = {
"success": False,
"message": f"Invalid JSON input: {str(e)}"
}
print(json.dumps(error))
except Exception as e:
error = {
"success": False,
"message": f"Unexpected error: {str(e)}"
}
print(json.dumps(error))
if __name__ == "__main__":
main()

View File

@@ -1,3 +1,4 @@
pysubs2~=1.8.0
langdetect~=1.0.9
pymediainfo~=7.0.1
pymediainfo~=7.0.1
spacy~=3.8.11

View File

@@ -0,0 +1,361 @@
import re
import uuid
from collections import Counter
from enum import Enum
from typing import Literal, Generator, Iterator
from pydantic import BaseModel, Field, RootModel, model_validator
from app.utils.singleton import Singleton
Cefr = Literal["C2", "C1", "B2", "B1", "A2", "A1"]
class UniversalPos(str, Enum):
"""Universal Part-of-Speech tags"""
ADJ = "ADJ" # Adjective
ADV = "ADV" # Adverb
INTJ = "INTJ" # Interjection
NOUN = "NOUN" # Noun
PROPN = "PROPN" # Proper noun
VERB = "VERB" # Verb
ADP = "ADP" # Adposition (preposition/postposition)
AUX = "AUX" # Auxiliary verb
CCONJ = "CCONJ" # Coordinating conjunction
DET = "DET" # Determiner
NUM = "NUM" # Numeral
PART = "PART" # Particle
PRON = "PRON" # Pronoun
SCONJ = "SCONJ" # Subordinating conjunction
PUNCT = "PUNCT" # Punctuation
SYM = "SYM" # Symbol
X = "X" # Other/unknown
class LexicalFeatures(str, Enum):
"""Lexical features for words."""
FORMAL = "formal"
INFORMAL = "informal"
SLANG = "slang"
COLLOQUIAL = "colloquial"
ARCHAIC = "archaic"
DIALECT = "dialect"
TECHNICAL = "technical"
LITERARY = "literary"
ABBREVIATION = "abbreviation"
NAME = "name"
IDIOMATIC = "idiomatic"
NEOLOGISM = "neologism"
GIBBERISH = "gibberish"
COMPOUND = "compound"
class IDGenerator(metaclass=Singleton):
"""Singleton class for generating unique IDs."""
_counter = 0
def next_id(self):
self._counter += 1
return self._counter
def reset(self):
self._counter = 0
class TaskStatus(Enum):
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
CANCELED = "canceled"
IGNORED = "ignored"
class TaskParams(BaseModel):
skip_existing: bool = Field(default=True, description="Whether to skip existing subtitle files")
class TasksApiParams(BaseModel):
operation: Literal["DELETE", "RETRY", "IGNORE"] = Field(
..., description="Operation to perform on the tasks"
)
task_id: str | None = Field(default=None, description="Unique identifier for the task")
class SegmentStatistics(BaseModel):
total_segments: int = Field(default=0, description="Total number of subtitle segments")
total_words: int = Field(default=0, description="Total number of candidate words")
cefr_distribution: dict[str, int] = Field(
default_factory=dict, description="Distribution of words by CEFR level"
)
pos_distribution: dict[str, int] = Field(
default_factory=dict, description="Distribution of words by Part of Speech"
)
exam_distribution: dict[str, int] = Field(
default_factory=dict, description="Distribution of words by Examination"
)
def to_string(self) -> str:
cefr_str = ", ".join(
[f"{level}({count})" for level, count in self.cefr_distribution.items()]
)
pos_str = ", ".join(
[f"{pos}({count})" for pos, count in self.pos_distribution.items()]
)
exam_str = ", ".join([f"{exam}({count})" for exam, count in self.exam_distribution.items()])
return (
f"Total Segments: {self.total_segments}\n"
f"Total Words: {self.total_words}\n"
f"CEFR Distribution: {cefr_str if cefr_str else 'N/A'}\n"
f"POS Distribution: {pos_str if pos_str else 'N/A'}\n"
f"Exam Distribution: {exam_str if exam_str else 'N/A'}"
)
class ProcessResult(BaseModel):
"""Result of processing a task."""
message: str | None = Field(default=None, description="Additional message or error information")
status: TaskStatus = Field(default=TaskStatus.PENDING, description="Current status of the task")
statistics: SegmentStatistics | None = Field(default=None, description="Statistics of the task")
class Task(BaseModel):
video_path: str = Field(..., description="Path to the video file")
task_id: str = Field(
default_factory=lambda: str(uuid.uuid4()),
description="Unique identifier for the task",
)
status: TaskStatus = Field(default=TaskStatus.PENDING, description="Current status of the task")
add_time: str | None = Field(default=None, description="Add time of the task, format %Y-%m-%d %H:%M:%S")
complete_time: str | None = Field(default=None, description="Complete time of the task")
tokens_used: int = Field(default=0, description="Number of used tokens")
message: str | None = Field(default=None, description="Additional message or error information")
params: TaskParams = Field(default_factory=TaskParams, description="Parameters for the task")
statistics: SegmentStatistics | None = Field(default=None, description="Statistics of the task")
class WordMetadata(BaseModel):
start_pos: int = Field(..., description="Start position of the word in the context sentence")
end_pos: int = Field(..., description="End position of the word in the context sentence")
context_id: int = Field(..., description="Identifier of the context sentence")
word_id: int = Field(
default_factory=lambda: IDGenerator().next_id(),
description="Identifier of the word in the context",
)
class PosDef(BaseModel):
# 'art.', 'v.', 'aux.', 'conj.', 'prep.', 'adv.', 'adj.', 'n.', 'vt.', 'pron.', 'det.', 'vi.', 'int.'
# 'num.', 'abbr.', 'na.', 'quant.', 'phr.'
pos: str = Field(..., description="Part of speech")
meanings: list[str] = Field(..., description="List of definitions")
@property
def plaintext(self):
return f"{self.pos} {'; '.join(self.meanings)}"
class WordBase(BaseModel):
text: str = Field(..., description="The word or phrase")
lemma: str = Field(..., description="Lemma form of the word")
pos: UniversalPos = Field(default=UniversalPos.X, description="Universal POS tag of the word")
class Word(WordBase):
phonetics: str | None = Field(default=None, description="Phonetic transcription of the word")
meta: WordMetadata = Field(default_factory=WordMetadata, description="Additional metadata")
cefr: Cefr | None = Field(default=None, description="CEFR level")
exams: list[str] = Field(
default_factory=list,
description="Exams whose vocabulary syllabus include this word",
)
pos_defs: list[PosDef] = Field(default_factory=list, description="Part of speech definitions")
llm_translation: str | None = Field(default=None, description="LLM generated Chinese translation")
llm_usage_context: str | None = Field(default=None, description="LLM generated cultural context")
lexical_features: list[LexicalFeatures] = Field(default_factory=list, description="Lexical features")
llm_example_sentences: list[str] = Field(default_factory=list, description="LLM generated example sentences")
@property
def pos_defs_plaintext(self) -> str:
return " ".join(
[
f"{index}. {pos_def.plaintext}"
for index, pos_def in enumerate(self.pos_defs)
]
)
class SubtitleSegment(BaseModel):
index: int = Field(..., description="Index of the subtitle segment")
start_time: int = Field(
..., description="Start time of the subtitle segment in milliseconds"
)
end_time: int = Field(..., description="End time of the subtitle segment in milliseconds")
plaintext: str = Field(..., description="Text content of the subtitle segment")
Chinese: str | None = Field(default=None, description="Chinese translation of the subtitle segment")
candidate_words: list[Word] = Field(
default_factory=list, description="List of words worth learning in the segment"
)
def words_append(self, word: Word):
"""
向字幕片段中添加一个单词到 words_worth_larning 列表中。
:param word: 要添加的单词对象。
"""
self.candidate_words.append(word)
@staticmethod
def _replace_with_spaces(_text):
"""
使用等长的空格替换文本中的 [xxx] 模式。
例如:"[Hi]" 会被替换成 " " (4个空格)
"""
pattern = r"(\[.*?\])"
return re.sub(pattern, lambda match: " " * len(match.group(1)), _text)
@property
def clean_text(self) -> str:
"""
获取清理后的文本内容,去除换行符并将 [xxx] 模式替换为空格。
"""
return SubtitleSegment._replace_with_spaces(self.plaintext.replace("\n", " "))
def __lt__(self, other: object):
if not isinstance(other, SubtitleSegment):
return NotImplemented
return self.index < other.index
class SegmentList(RootModel):
root: list[SubtitleSegment] = Field(
default_factory=list, description="List of subtitle segments"
)
@property
def statistics(self) -> SegmentStatistics:
all_words = [word for seg in self.root for word in seg.candidate_words]
cefr_counts = Counter(word.cefr if word.cefr else "Other" for word in all_words)
pos_counts = Counter(word.pos.value if word.pos else "Other" for word in all_words)
exam_counts = Counter(exam for word in all_words for exam in word.exams)
return SegmentStatistics(
total_segments=len(self.root),
total_words=len(all_words),
cefr_distribution=dict(cefr_counts),
pos_distribution=dict(pos_counts),
exam_distribution=dict(exam_counts)
)
def context_generator(
self, context_window: int, extra_len: int = 1
) -> Generator[tuple[list[SubtitleSegment], tuple[int, int]], None, None]:
"""
生成包含上下文窗口的字幕片段列表
:param context_window: 上下文窗口大小
:param extra_len: 额外长度,用于调整窗口大小
:yield: 包含上下文的字幕片段列表。
"""
total_segments = len(self.root)
for i in range((total_segments + context_window - 1) // context_window):
real_start = i * context_window
real_end = min(total_segments, (i + 1) * context_window) - 1
start_index = max(0, i * context_window - extra_len)
end_index = min(total_segments, (i + 1) * context_window + extra_len)
yield (
self.root[start_index:end_index],
(self.root[real_start].index, self.root[real_end].index),
)
def sort(self):
self.root.sort()
@model_validator(mode="after")
def sort_root(self):
self.root.sort()
return self
def __iter__(self) -> Iterator[SubtitleSegment]:
return iter(self.root)
class SpacyToken(BaseModel):
lemma_: str = Field(..., description="Lemma form of the word (string)")
pos_: str = Field(..., description="POS tag of the word")
text: str = Field(..., description="Text of the word")
is_stop: bool = Field(default=False, description="Indicates if the word is a stop word")
is_punct: bool = Field(default=False, description="Indicates if the word is punctuation")
ent_iob_: str = Field(..., description="Entity IOB")
class SpacyNamedEntity(BaseModel):
text: str = Field(..., description="Text of the entity")
label_: str = Field(..., description="Label of the entity")
class NlpResult(BaseModel):
tokens: list[SpacyToken] = Field(default_factory=list, description="List of tokens")
entities: list[SpacyNamedEntity] = Field(default_factory=list, description="List of named entities")
class LlmFeedbackAboutCandidateWord(BaseModel):
should_keep: bool = Field(..., description="Indicates whether to keep the candidate word")
# reason: str | None = Field(default=None, description="Concise reason for the decision")
word_id: int = Field(..., description="Identifier of the word in the context")
text: str | None = Field(default=None, description="The vocabulary word or phrase")
lemma: str | None = Field(default=None, description="Lemma form of the word")
pos: UniversalPos | None = Field(
default=None,
description="Universal POS tag of the word. Options: ADJ, ADV, INTJ, NOUN, PROPN, "
"VERB, ADP, AUX, CCONJ, DET, NUM, PART, PRON, SCONJ, PUNCT, SYM, X",
)
class LlmFeedback(BaseModel):
candidate_words_feedback: list[LlmFeedbackAboutCandidateWord] = Field(
default_factory=list, description="Feedback about candidate words."
)
llm_identified_words: list[WordBase] = Field(
default_factory=list, description="List of words identified by the LLM."
)
class LlmWordEnrichment(BaseModel):
word_id: int = Field(..., description="Identifier of the word in the context")
translation: str | None = Field(default=None, description="Chinese translation of the word")
usage_context: str | None = Field(default=None, description="Usage or Cultural Context")
lexical_features: list[LexicalFeatures] = Field(default_factory=list, description="Lexical features")
class LlmEnrichmentResult(BaseModel):
enriched_words: list[LlmWordEnrichment] = Field(default_factory=list, description="List of enriched word data")
class LlmSegmentTranslation(BaseModel):
index: int = Field(..., description="Index of the subtitle segment")
translation: str = Field(..., description="Natural Chinese translation of the segment")
class LlmTranslationResult(BaseModel):
translations: list[LlmSegmentTranslation] = Field(default_factory=list, description="List of segment translations")
class VocabularyAnnotatingToolInput(BaseModel):
explanation: str = Field(
...,
description="This is a tool for adding a new vocabulary-annotating task to AnnotLexi",
)
video_path: str = Field(..., description="Path to the video file")
skip_existing: bool = Field(default=True, description="Whether to skip existing subtitle files")
class QueryAnnotationTasksToolInput(BaseModel):
count: int = Field(default=5, description="The maximum number of returned annotation tasks")
explanation: str = Field(..., description="This is a tool for querying the latest annotation tasks in AnnotLexi")

View File

@@ -0,0 +1,98 @@
from multiprocessing import Process, Queue
import spacy
from spacy.tokenizer import Tokenizer
from app.core.cache import cached
from app.log import logger
from .schemas import SpacyNamedEntity, SpacyToken, NlpResult
class SpacyWorker:
def __init__(self, model="en_core_web_sm"):
self.task_q = Queue()
self.result_q = Queue()
self.status_q = Queue()
self.model = model
# 启动子进程
logger.info("正在启动 SpacyWorker 子进程...")
self.proc = Process(target=self.run, args=(self.model,))
self.proc.start()
# 等待子进程返回模型加载状态
status, info = self.status_q.get()
if status == "error":
self.proc.join()
raise RuntimeError(f"spaCy 模型加载失败: {info}")
else:
logger.info(f"spaCy 模型 `{self.model}` 加载成功")
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self.close()
def run(self, model: str):
try:
nlp = SpacyWorker.load_nlp(model)
infixes = list(nlp.Defaults.infixes)
infixes = [i for i in infixes if "-" not in i]
infix_re = spacy.util.compile_infix_regex(infixes)
nlp.tokenizer = Tokenizer(
nlp.vocab,
prefix_search=nlp.tokenizer.prefix_search,
suffix_search=nlp.tokenizer.suffix_search,
infix_finditer=infix_re.finditer,
token_match=nlp.tokenizer.token_match,
)
except Exception as e:
self.status_q.put(("error", str(e)))
return
# 告诉主进程加载成功
self.status_q.put(("ok", None))
while True:
text = self.task_q.get()
if text is None:
break
doc = nlp(text)
tokens = []
entities = []
for token in doc:
tokens.append(
SpacyToken(
lemma_=token.lemma_,
pos_=token.pos_,
text=token.text,
is_stop=token.is_stop,
is_punct=token.is_punct,
ent_iob_=token.ent_iob_,
)
)
for ent in doc.ents:
entities.append(SpacyNamedEntity(text=ent.text, label_=ent.label_))
self.result_q.put(NlpResult(tokens=tokens, entities=entities))
@staticmethod
@cached(maxsize=1, ttl=3600 * 6)
def load_nlp(model: str) -> spacy.Language:
return spacy.load(model)
def submit(self, text: str) -> NlpResult:
"""
提交任务并等待结果
"""
self.task_q.put(text)
return self.result_q.get()
def close(self):
"""
关闭子进程
"""
if self.proc.is_alive():
self.task_q.put(None)
self.proc.join()
logger.info("SpacyWorker 子进程退出")

View File

@@ -0,0 +1,44 @@
from typing import Generator, Any, overload
from pysubs2 import SSAEvent
from .schemas import SubtitleSegment
class SubtitleProcessor:
def __init__(self):
self._events: list[SSAEvent] = []
def append(self, event: SSAEvent):
self._events.append(event)
def segment_generator(self) -> Generator[SubtitleSegment, None, None]:
for index, event in enumerate(self._events):
yield SubtitleSegment(
index=index,
start_time=event.start,
end_time=event.end,
plaintext=event.plaintext,
)
@overload
def __getitem__(self, item: int) -> SSAEvent:
pass
@overload
def __getitem__(self, s: slice) -> list[SSAEvent]:
pass
def __getitem__(self, item: Any) -> Any:
return self._events[item]
def style_text(style: str, text: str) -> str:
"""
使用指定的样式包装文本。
:param style: 样式名称
:param text: 要包装的文本
:return: 包含样式的文本
"""
return f"{{\\r{style}}}{text}{{\\r}}"

View File

@@ -1,9 +1,14 @@
import re
import threading
import time
from typing import Any, List, Dict, Tuple, Optional
from app.core.cache import cached
from app.core.config import settings
from app.core.event import eventmanager, Event
from app.helper.mediaserver import MediaServerHelper
from app.log import logger
from app.modules.themoviedb import CategoryHelper
from app.plugins import _PluginBase
from app.schemas import WebhookEventInfo, ServiceInfo
from app.schemas.types import EventType, MediaType, MediaImageType, NotificationType
@@ -11,14 +16,28 @@ from app.utils.web import WebUtils
class MediaServerMsg(_PluginBase):
# 插件名称
"""
媒体服务器通知插件
功能:
1. 监听Emby/Jellyfin/Plex等媒体服务器的Webhook事件
2. 根据配置发送播放、入库等通知消息
3. 对TV剧集入库事件进行智能聚合避免消息轰炸
4. 支持多种媒体服务器和丰富的消息类型配置
"""
# 常量定义
DEFAULT_EXPIRATION_TIME = 600 # 默认过期时间(秒)
DEFAULT_AGGREGATE_TIME = 15 # 默认聚合时间(秒)
# 插件基本信息
plugin_name = "媒体库服务器通知"
# 插件描述
plugin_desc = "发送Emby/Jellyfin/Plex服务器的播放、入库等通知消息。"
# 插件图标
plugin_icon = "mediaplay.png"
# 插件版本
plugin_version = "1.6"
plugin_version = "1.7.1"
# 插件作者
plugin_author = "jxxghp"
# 作者主页
@@ -30,17 +49,23 @@ class MediaServerMsg(_PluginBase):
# 可使用的用户级别
auth_level = 1
# 私有属性
_enabled = False
_add_play_link = False
_mediaservers = None
_types = []
_webhook_msg_keys = {}
# 插件运行时状态配置
_enabled = False # 插件是否启用
_add_play_link = False # 是否添加播放链接
_mediaservers = None # 媒体服务器列表
_types = [] # 启用的消息类型
_webhook_msg_keys = {} # Webhook消息去重缓存
_aggregate_enabled = True # 是否启用TV剧集聚合功能
# 拼装消息内容
# TV剧集消息聚合配置
_aggregate_time = DEFAULT_AGGREGATE_TIME # 聚合时间窗口(秒)
_pending_messages = {} # 待聚合的消息 {series_key: [event_info, ...]}
_aggregate_timers = {} # 聚合定时器 {series_key: timer}
# Webhook事件映射配置
_webhook_actions = {
"library.new": "新入库",
"system.webhooktest": "测试",
"system.notificationtest": "测试",
"playback.start": "开始播放",
"playback.stop": "停止播放",
"user.authenticated": "登录成功",
@@ -51,23 +76,44 @@ class MediaServerMsg(_PluginBase):
"PlaybackStop": "停止播放",
"item.rate": "标记了"
}
# 媒体服务器默认图标
_webhook_images = {
"emby": "https://emby.media/notificationicon.png",
"plex": "https://www.plex.tv/wp-content/uploads/2022/04/new-logo-process-lines-gray.png",
"jellyfin": "https://play-lh.googleusercontent.com/SCsUK3hCCRqkJbmLDctNYCfehLxsS4ggD1ZPHIFrrAN1Tn9yhjmGMPep2D9lMaaa9eQi"
}
def init_plugin(self, config: dict = None):
def __init__(self):
super().__init__()
self.category = CategoryHelper()
logger.debug("媒体服务器消息插件初始化完成")
def init_plugin(self, config: dict = None):
"""
初始化插件配置
Args:
config (dict, optional): 插件配置参数
"""
if config:
self._enabled = config.get("enabled")
self._types = config.get("types") or []
self._mediaservers = config.get("mediaservers") or []
self._add_play_link = config.get("add_play_link", False)
self._aggregate_enabled = config.get("aggregate_enabled", False)
self._aggregate_time = int(config.get("aggregate_time", self.DEFAULT_AGGREGATE_TIME))
def service_infos(self, type_filter: Optional[str] = None) -> Optional[Dict[str, ServiceInfo]]:
"""
服务信息
获取媒体服务器信息服务信息
Args:
type_filter (str, optional): 媒体服务器类型过滤器
Returns:
Dict[str, ServiceInfo]: 活跃的媒体服务器服务信息字典
"""
if not self._mediaservers:
logger.warning("尚未配置媒体服务器,请检查配置")
@@ -93,19 +139,45 @@ class MediaServerMsg(_PluginBase):
def service_info(self, name: str) -> Optional[ServiceInfo]:
"""
服务信息
根据名称获取特定媒体服务器服务信息
Args:
name (str): 媒体服务器名称
Returns:
ServiceInfo: 媒体服务器服务信息
"""
service_infos = self.service_infos() or {}
return service_infos.get(name)
def get_state(self) -> bool:
"""
获取插件状态
Returns:
bool: 插件是否启用
"""
return self._enabled
@staticmethod
def get_command() -> List[Dict[str, Any]]:
"""
获取插件命令
(当前未实现)
Returns:
List[Dict[str, Any]]: 空列表
"""
pass
def get_api(self) -> List[Dict[str, Any]]:
"""
获取插件API
(当前未实现)
Returns:
List[Dict[str, Any]]: 空列表
"""
pass
def get_form(self) -> Tuple[List[dict], Dict[str, Any]]:
@@ -210,6 +282,72 @@ class MediaServerMsg(_PluginBase):
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'aggregate_enabled',
'label': '启用TV剧集结入库聚合',
}
}
]
}
]
},
{
'component': 'VRow',
'props': {'show': '{{aggregate_enabled}}'},
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VTextField',
'props': {
'model': 'aggregate_time',
'label': 'TV剧集结入库聚合时间',
'placeholder': '15'
}
}
]
}
]
},
{
'component': 'VRow',
'props': {'show': '{{aggregate_enabled}}'},
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'warning',
'variant': 'tonal',
'text': '请在整理刮削设置中添加tmdbid,以保证准确性。仅保证在Emby和整理刮削添加tmdbid后功能正常。'
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
@@ -235,38 +373,70 @@ class MediaServerMsg(_PluginBase):
}
], {
"enabled": False,
"types": []
"types": [],
"aggregate_enabled": False,
"aggregate_time": 15
}
def get_page(self) -> List[dict]:
"""
获取插件页面
(当前未实现)
Returns:
List[dict]: 空列表
"""
pass
@eventmanager.register(EventType.WebhookMessage)
def send(self, event: Event):
"""
发送通知消息
发送通知消息主入口函数
处理来自媒体服务器的Webhook事件并根据配置决定是否发送通知消息
处理流程:
1. 检查插件是否启用
2. 验证事件数据有效性
3. 检查事件类型是否在支持范围内
4. 检查事件类型是否在用户配置的允许范围内
5. 验证媒体服务器配置
6. 特殊处理TV剧集入库事件聚合处理
7. 处理常规消息事件
8. 构造并发送通知消息
Args:
event (Event): Webhook事件对象
"""
# 检查插件是否启用
if not self._enabled:
logger.debug("插件未启用")
return
# 获取事件数据
event_info: WebhookEventInfo = event.event_data
if not event_info:
logger.debug("事件数据为空")
return
# 不在支持范围不处理
# 打印event_info用于调试
logger.debug(f"收到Webhook事件: {event_info}")
# 检查事件类型是否在支持范围内
if not self._webhook_actions.get(event_info.event):
logger.debug(f"事件类型 {event_info.event} 不在支持范围内")
return
# 不在选中范围不处理
msgflag = False
# 检查事件类型是否在用户配置的允许范围内
# 将配置的类型预处理为一个扁平集合,提高查找效率
allowed_types = set()
for _type in self._types:
if event_info.event in _type.split("|"):
msgflag = True
break
if not msgflag:
allowed_types.update(_type.split("|"))
if event_info.event not in allowed_types:
logger.info(f"未开启 {event_info.event} 类型的消息通知")
return
# 验证媒体服务器配置
if not self.service_infos():
logger.info(f"未开启任一媒体服务器的消息通知")
return
@@ -279,6 +449,30 @@ class MediaServerMsg(_PluginBase):
logger.info(f"未开启媒体服务器类型 {event_info.channel} 的消息通知")
return
# TV剧集结入库聚合处理
logger.debug("检查是否需要进行TV剧集聚合处理")
logger.debug(f"event_info.event={event_info.event}, item_type={event_info.item_type}")
logger.debug(f"json_object存在: {bool(event_info.json_object)}, 类型: {type(event_info.json_object)}")
# 判断是否需要进行TV剧集入库聚合处理
if (self._aggregate_enabled and
event_info.event == "library.new" and
event_info.item_type in ["TV", "SHOW"] and
event_info.json_object and
isinstance(event_info.json_object, dict)):
logger.debug("满足TV剧集聚合条件尝试获取series_id")
series_id = self._get_series_id(event_info)
logger.debug(f"获取到的series_id: {series_id}")
if series_id:
logger.debug(f"开始聚合处理series_id={series_id}")
self._aggregate_tv_episodes(series_id, event_info)
logger.debug("TV剧集消息已处理并返回")
return # TV剧集消息已处理直接返回
else:
logger.debug("未能获取到有效的series_id")
logger.debug("未进行聚合处理,继续普通消息处理流程")
expiring_key = f"{event_info.item_id}-{event_info.client}-{event_info.user_name}"
# 过滤停止播放重复消息
if str(event_info.event) == "playback.stop" and expiring_key in self._webhook_msg_keys.keys():
@@ -286,7 +480,7 @@ class MediaServerMsg(_PluginBase):
self.__add_element(expiring_key)
return
# 消息标题
# 构造消息标题
if event_info.item_type in ["TV", "SHOW"]:
message_title = f"{self._webhook_actions.get(event_info.event)}剧集 {event_info.item_name}"
elif event_info.item_type == "MOV":
@@ -296,7 +490,7 @@ class MediaServerMsg(_PluginBase):
else:
message_title = f"{self._webhook_actions.get(event_info.event)}"
# 消息内容
# 构造消息内容
message_texts = []
if event_info.user_name:
message_texts.append(f"用户:{event_info.user_name}")
@@ -314,39 +508,41 @@ class MediaServerMsg(_PluginBase):
# 消息内容
message_content = "\n".join(message_texts)
# 消息图片
# 处理消息图片
image_url = event_info.image_url
# 查询剧集图片
if event_info.tmdb_id:
season_id = event_info.season_id if event_info.season_id else None
episode_id = event_info.episode_id if event_info.episode_id else None
if not image_url and event_info.tmdb_id:
# 查询电影图片
if event_info.item_type == "MOV" :
image_url = self.chain.obtain_specific_image(
mediaid=event_info.tmdb_id,
mtype=MediaType.MOVIE,
image_type=MediaImageType.Poster
)
specific_image = self.chain.obtain_specific_image(
mediaid=event_info.tmdb_id,
mtype=MediaType.TV,
image_type=MediaImageType.Backdrop,
season=season_id,
episode=episode_id
)
if specific_image:
image_url = specific_image
# 查询剧集图片
elif event_info.item_type in ["TV", "SHOW"]:
season_id = event_info.season_id if event_info.season_id else None
episode_id = event_info.episode_id if event_info.episode_id else None
specific_image = self.chain.obtain_specific_image(
mediaid=event_info.tmdb_id,
mtype=MediaType.TV,
image_type=MediaImageType.Backdrop,
season=season_id,
episode=episode_id
)
if specific_image:
image_url = specific_image
# 使用默认图片
if not image_url:
image_url = self._webhook_images.get(event_info.channel)
# 处理播放链接
play_link = None
if self._add_play_link:
if event_info.server_name:
service = self.service_infos().get(event_info.server_name)
if service:
play_link = service.instance.get_play_url(event_info.item_id)
elif event_info.channel:
services = MediaServerHelper().get_services(type_filter=event_info.channel)
for service in services.values():
play_link = service.instance.get_play_url(event_info.item_id)
if play_link:
break
play_link = self._get_play_link(event_info)
# 更新播放状态缓存
if str(event_info.event) == "playback.stop":
# 停止播放消息,添加到过期字典
self.__add_element(expiring_key)
@@ -358,22 +554,467 @@ class MediaServerMsg(_PluginBase):
self.post_message(mtype=NotificationType.MediaServer,
title=message_title, text=message_content, image=image_url, link=play_link)
def __add_element(self, key, duration=600):
def _get_series_id(self, event_info: WebhookEventInfo) -> Optional[str]:
"""
获取剧集ID用于TV剧集消息聚合
优先级顺序:
1. 从JSON对象的Item中获取SeriesId
2. 从JSON对象的Item中获取SeriesName作为备选
3. 从event_info中直接获取series_idfallback方案
Args:
event_info (WebhookEventInfo): Webhook事件信息
Returns:
Optional[str]: 剧集ID或None如果无法获取
"""
# 从json_object中提取series_id
if event_info.json_object and isinstance(event_info.json_object, dict):
item = event_info.json_object.get("Item", {})
series_id = item.get("SeriesId") or item.get("SeriesName")
if series_id:
return series_id
# fallback到event_info中的series_id
return getattr(event_info, "series_id", None)
def _aggregate_tv_episodes(self, series_id: str, event_info: WebhookEventInfo):
"""
聚合TV剧集结入库消息
当同一剧集的多集在短时间内入库时,将它们聚合为一条消息发送,
避免消息轰炸。通过设置定时器实现延迟发送,定时器时间内到达的
同剧集消息会被聚合在一起。
Args:
series_id (str): 剧集ID
event_info (WebhookEventInfo): Webhook事件信息
"""
try:
logger.debug(f"开始执行聚合处理: series_id={series_id}")
# 初始化该series_id的消息列表
if series_id not in self._pending_messages:
logger.debug(f"为series_id={series_id}初始化消息列表")
self._pending_messages[series_id] = []
# 添加消息到待处理列表
logger.debug(f"添加消息到待处理列表: series_id={series_id}")
self._pending_messages[series_id].append(event_info)
# 如果已经有定时器,取消它并重新设置
if series_id in self._aggregate_timers:
logger.debug(f"取消已存在的定时器: {series_id}")
self._aggregate_timers[series_id].cancel()
# 设置新的定时器
logger.debug(f"设置新的定时器,将在 {self._aggregate_time} 秒后触发")
timer = threading.Timer(self._aggregate_time, self._send_aggregated_message, [series_id])
self._aggregate_timers[series_id] = timer
timer.start()
logger.debug(f"已添加剧集 {series_id} 的消息到聚合队列,当前队列长度: {len(self._pending_messages[series_id])},定时器将在 {self._aggregate_time} 秒后触发")
logger.debug(f"完成聚合处理: series_id={series_id}")
except Exception as e:
logger.error(f"聚合处理过程中出现异常: {str(e)}", exc_info=True)
def _send_aggregated_message(self, series_id: str):
"""
发送聚合后的TV剧集消息
当聚合定时器到期或插件退出时调用此方法,将累积的同剧集消息
合并为一条消息发送给用户。
Args:
series_id (str): 剧集ID
"""
logger.debug(f"定时器触发,准备发送聚合消息: {series_id}")
# 获取该series_id的所有待处理消息
if series_id not in self._pending_messages or not self._pending_messages[series_id]:
logger.debug(f"消息队列为空或不存在: {series_id}")
# 清除定时器引用
if series_id in self._aggregate_timers:
del self._aggregate_timers[series_id]
return
events = self._pending_messages.pop(series_id)
logger.debug(f"从队列中获取 {len(events)} 条消息: {series_id}")
# 清除定时器引用
if series_id in self._aggregate_timers:
del self._aggregate_timers[series_id]
# 构造聚合消息
if not events:
logger.debug(f"事件列表为空: {series_id}")
return
# 使用第一个事件的信息作为基础
first_event = events[0]
# 预计算事件数量避免重复调用len(events)
events_count = len(events)
is_multiple_episodes = events_count > 1
# 尝试从item_path中提取tmdb_id
tmdb_pattern = r'[\[{](?:tmdbid|tmdb)[=-](\d+)[\]}]'
if match := re.search(tmdb_pattern, first_event.item_path):
first_event.tmdb_id = match.group(1)
logger.info(f"从路径提取到tmdb_id: {first_event.tmdb_id}")
else:
logger.info(f"未从路径中提取到tmdb_id: {first_event.item_path}")
# 通过TMDB ID获取详细信息
tmdb_info = None
overview = None
try:
if not first_event.tmdb_id:
logger.debug("tmdb_id为空使用原有逻辑发送消息")
# 使用原有逻辑构造消息
message_title = f"📺 {self._webhook_actions.get(first_event.event)}剧集:{first_event.item_name}"
message_texts = []
message_texts.append(f"⏰ 时间:{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time()))}")
# 收集集数信息
episode_details = []
for event in events:
if event.season_id is not None and event.episode_id is not None:
episode_details.append(f"S{int(event.season_id):02d}E{int(event.episode_id):02d}")
if episode_details:
message_texts.append(f"📺 季集:{', '.join(episode_details)}")
message_content = "\n".join(message_texts)
# 使用默认图片
image_url = first_event.image_url or self._webhook_images.get(first_event.channel)
# 处理播放链接
play_link = None
if self._add_play_link:
play_link = self._get_play_link(first_event)
# 发送消息
self.post_message(mtype=NotificationType.MediaServer,
title=message_title,
text=message_content,
image=image_url,
link=play_link)
return
if first_event.item_type in ["TV", "SHOW"]:
logger.debug("查询TV类型的TMDB信息")
tmdb_info = self._get_tmdb_info(
tmdb_id=first_event.tmdb_id,
mtype=MediaType.TV,
season=first_event.season_id
)
logger.debug(f"从TMDB获取到的信息: {tmdb_info}")
except Exception as e:
logger.debug(f"获取TMDB信息时出错: {str(e)}")
if first_event.overview:
overview = first_event.overview
elif tmdb_info:
if is_multiple_episodes:
if tmdb_info.get('overview'):
overview = tmdb_info.get('overview')
logger.debug(f"从TMDB获取到overview: {overview}")
else:
logger.debug("未能从TMDB获取到有效的overview信息")
else:
if (tmdb_info.get('episodes') and tmdb_info.get('episodes')[int(first_event.episode_id)-1]
and tmdb_info.get('episodes')[int(first_event.episode_id)-1].get('overview')):
overview = tmdb_info.get('episodes')[int(first_event.episode_id)-1].get('overview')
elif tmdb_info.get('overview'):
overview = tmdb_info.get('overview')
else:
logger.debug("未能从TMDB获取到有效的overview信息")
else:
logger.debug("未能从TMDB获取到有效的overview信息")
events[0] = first_event
# 消息标题
message_title = f"📺 {self._webhook_actions.get(first_event.event)}剧集:{first_event.item_name.split(' ', 1)[0]}"
if is_multiple_episodes:
message_title += f"{events_count}个文件"
logger.debug(f"构建消息标题: {message_title}")
# 消息内容
message_texts = []
# 时间信息放在最前面
message_texts.append(f"⏰ 时间:{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time()))}")
# 添加每个集数的信息并合并连续集数
episodes_detail = self._merge_continuous_episodes(events)
message_texts.append(f"📺 季集:{episodes_detail}")
# 确定二级分类
cat = None
if tmdb_info.get('media_type') == MediaType.TV:
cat = self.category.get_tv_category(tmdb_info)
else:
cat = self.category.get_movie_category(tmdb_info)
if cat:
message_texts.append(f"📚 分类:{cat}")
# 评分信息
if tmdb_info and tmdb_info.get('vote_average'):
rating = round(float(tmdb_info.get('vote_average')), 1)
message_texts.append(f"⭐ 评分:{rating}/10")
# 类型信息 - genres可能是字典列表或字符串列表
if tmdb_info.get('genres'):
genres_list = []
for genre in tmdb_info.get('genres')[:3]:
if isinstance(genre, dict):
genres_list.append(genre.get('name', ''))
else:
genres_list.append(str(genre))
if genres_list:
genre_text = ''.join(genres_list)
message_texts.append(f"🎭 类型:{genre_text}")
if overview:
# 限制overview只显示前100个字符超出部分用...代替
if len(overview) > 100:
overview = overview[:100] + "..."
message_texts.append(f"📖 剧情:{overview}")
# 消息内容
message_content = "\n".join(message_texts)
logger.debug(f"构建消息内容: {message_content}")
# 消息图片
image_url = first_event.image_url
logger.debug(f"初始图片URL: {image_url}")
if not image_url and tmdb_info and tmdb_info.get('poster_path') and not is_multiple_episodes:
# 剧集图片
image_url = self.backdrop_path = f"https://{settings.TMDB_IMAGE_DOMAIN}/t/p/original{tmdb_info.get('poster_path')}"
logger.debug(f"使用剧集图片URL: {image_url}")
elif not image_url and tmdb_info and tmdb_info.get('backdrop_path') and is_multiple_episodes:
# 使用TMDB背景
image_url = self.backdrop_path = f"https://{settings.TMDB_IMAGE_DOMAIN}/t/p/original{tmdb_info.get('backdrop_path')}"
logger.debug(f"使用TMDB背景URL: {image_url}")
# 使用默认图片
if not image_url:
image_url = self._webhook_images.get(first_event.channel)
logger.debug(f"使用默认图片URL: {image_url}")
# 处理播放链接
play_link = None
if self._add_play_link:
play_link = self._get_play_link(first_event)
# 发送聚合消息
logger.debug(f"准备发送消息 - 标题: {message_title}, 内容: {message_content}, 图片: {image_url}")
self.post_message(mtype=NotificationType.MediaServer,
title=message_title, text=message_content, image=image_url, link=play_link)
logger.info(f"已发送聚合消息:{message_title}")
def _merge_continuous_episodes(self, events: List[WebhookEventInfo]) -> str:
"""
合并连续的集数信息,使消息展示更美观
将同一季中连续的集数合并为一个区间显示,例如:
S01E01-E03 而不是 S01E01, S01E02, S01E03
Args:
events (List[WebhookEventInfo]): Webhook事件信息列表
Returns:
str: 合并后的集数信息字符串
"""
# 按季分组集数信息
season_episodes = {}
tmdb_info = self._get_tmdb_info(
tmdb_id=events[0].tmdb_id,
mtype=MediaType.TV,
season=events[0].season_id
)
for event in events:
# 提取季号和集号
season, episode = None, None
episode_name = ""
if event.json_object and isinstance(event.json_object, dict):
item = event.json_object.get("Item", {})
season = item.get("ParentIndexNumber")
episode = item.get("IndexNumber")
if episode is not None and int(episode) <= len(tmdb_info.get('episodes')):
episode_name = tmdb_info.get("episodes")[int(episode)-1].get('name')
else:
episode_name = item.get("Name", "")
# 如果无法从json_object获取信息则尝试从event_info直接获取
if season is None:
season = getattr(event, "season_id", None)
if episode is None:
episode = getattr(event, "episode_id", None)
if not episode_name:
episode_name = getattr(event, "item_name", "")
# 确保季号和集号都存在
if season is not None and episode is not None:
if season not in season_episodes:
season_episodes[season] = []
season_episodes[season].append({
"episode": episode,
"name": episode_name
})
# 对每季的集数进行排序并合并连续区间
merged_details = []
for season in sorted(season_episodes.keys()):
episodes = season_episodes[season]
# 按集号排序
episodes.sort(key=lambda x: x["episode"])
# 合并连续集数
if not episodes:
continue
# 初始化第一个区间
start = episodes[0]["episode"]
end = episodes[0]["episode"]
episode_names = [episodes[0]["name"]]
for i in range(1, len(episodes)):
current = episodes[i]["episode"]
# 如果当前集号与上一集连续
if current == end + 1:
end = current
episode_names.append(episodes[i]["name"])
else:
# 保存当前区间
if start == end:
merged_details.append(f"S{season:02d}E{start:02d} {episode_names[0]}")
else:
# 合并区间
merged_details.append(f"S{season:02d}E{start:02d}-E{end:02d}")
# 开始新区间
start = end = current
episode_names = [episodes[i]["name"]]
# 添加最后一个区间
if start == end:
merged_details.append(f"S{season:02d}E{start:02d} {episode_names[-1]}")
else:
merged_details.append(f"S{season:02d}E{start:02d}-E{end:02d}")
return ", ".join(merged_details)
def __add_element(self, key, duration=DEFAULT_EXPIRATION_TIME):
"""
添加元素到过期字典中,用于过滤短时间内的重复消息
Args:
key (str): 元素键值
duration (int, optional): 过期时间默认DEFAULT_EXPIRATION_TIME秒
"""
expiration_time = time.time() + duration
# 如果元素已经存在,更新其过期时间
self._webhook_msg_keys[key] = expiration_time
def __remove_element(self, key):
"""
从过期字典中移除指定元素
Args:
key (str): 要移除的元素键值
"""
self._webhook_msg_keys = {k: v for k, v in self._webhook_msg_keys.items() if k != key}
def __get_elements(self):
"""
获取所有未过期的元素键值列表,并清理过期元素
Returns:
List[str]: 未过期的元素键值列表
"""
current_time = time.time()
# 过滤掉过期的元素
self._webhook_msg_keys = {k: v for k, v in self._webhook_msg_keys.items() if v > current_time}
return list(self._webhook_msg_keys.keys())
# 创建新的字典,只保留未过期的元素
valid_keys = []
expired_keys = []
for key, expiration_time in self._webhook_msg_keys.items():
if expiration_time > current_time:
valid_keys.append(key)
else:
expired_keys.append(key)
# 从字典中移除过期元素
for key in expired_keys:
del self._webhook_msg_keys[key]
return valid_keys
def _get_play_link(self, event_info: WebhookEventInfo) -> Optional[str]:
"""
获取媒体项目的播放链接
Args:
event_info (WebhookEventInfo): 事件信息
Returns:
Optional[str]: 播放链接如果无法获取则返回None
"""
play_link = None
if event_info.server_name:
service = self.service_infos().get(event_info.server_name)
if service:
play_link = service.instance.get_play_url(event_info.item_id)
elif event_info.channel:
services = MediaServerHelper().get_services(type_filter=event_info.channel)
for service in services.values():
play_link = service.instance.get_play_url(event_info.item_id)
if play_link:
break
return play_link
@cached(
region="MediaServerMsg", # 缓存区域,用于隔离不同插件的缓存
maxsize=128, # 最大缓存条目数(仅内存缓存有效)
ttl=600, # 缓存存活时间(秒)
skip_none=True, # 是否跳过None值缓存
skip_empty=False # 是否跳过空值缓存(空列表、空字典等)
)
def _get_tmdb_info(self, tmdb_id: str, mtype: MediaType, season: Optional[int] = None):
"""
获取TMDB信息
Args:
tmdb_id: TMDB ID
mtype: 媒体类型
season: 季数(仅电视剧需要)
Returns:
dict: TMDB信息
"""
if mtype == MediaType.MOVIE:
return self.chain.tmdb_info(tmdbid=tmdb_id, mtype=mtype)
else: # TV类型
tmdb_info = self.chain.tmdb_info(tmdbid=tmdb_id, mtype=mtype, season=season)
tmdb_info2 = self.chain.tmdb_info(tmdbid=tmdb_id, mtype=mtype)
return tmdb_info | tmdb_info2
def stop_service(self):
"""
退出插件
退出插件时的清理工作
在插件被停用或系统关闭时调用,确保:
1. 所有待处理的聚合消息被立即发送出去
2. 所有正在进行的定时器被取消
3. 清空所有内部缓存数据
"""
pass
# 发送所有待处理的聚合消息
for series_id in list(self._pending_messages.keys()):
# 直接发送消息而不依赖定时器
self._send_aggregated_message(series_id)
# 取消所有定时器
for timer in self._aggregate_timers.values():
timer.cancel()
self._aggregate_timers.clear()
self._pending_messages.clear()
self._get_tmdb_info.cache_clear()

View File

@@ -0,0 +1,326 @@
from pathlib import Path
from typing import Any, List, Dict, Tuple
from app.core.context import MediaInfo
from app.core.event import eventmanager, Event
from app.log import logger
from app.plugins import _PluginBase
from app.schemas.types import ChainEventType, MediaType, NotificationType
class MultiClass(_PluginBase):
# 插件名称
plugin_name = "视频多级分类"
# 插件描述
plugin_desc = "支持电影按照评分,年代和系列分类"
# 插件图标
plugin_icon = "Calibreweb_B.png"
# 插件版本
plugin_version = "0.1"
# 插件作者
plugin_author = "liuhangbin"
# 作者主页
author_url = "https://github.com/liuhangbin"
# 插件配置项ID前缀
plugin_config_prefix = "multiclass_"
# 加载顺序
plugin_order = 1
# 可使用的用户级别
auth_level = 1
_enabled = False
_notify = False
_year_class = False
_vote_class = False
_collection_class = False
def init_plugin(self, config: dict = None):
if config:
self._enabled = config.get("enabled", False)
self._notify = config.get("notify", False)
self._year_class = config.get("year_class", False)
self._vote_class = config.get("vote_class", False)
self._collection_class = config.get("collection_class", False)
def get_state(self) -> bool:
return self._enabled
def get_api(self) -> List[Dict[str, Any]]:
pass
def get_form(self) -> Tuple[List[dict], Dict[str, Any]]:
"""
拼装插件配置页面需要返回两块数据1、页面配置2、数据结构
"""
return [
{
'component': 'VForm',
'content': [
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'enabled',
'label': '启用插件',
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'year_class',
'label': '按照年代分类',
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'vote_class',
'label': '按照评分分类',
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'collection_class',
'label': '按照系列分类',
}
}
]
},
{
'component': 'VCol',
'props': {
'cols': 12,
'md': 6
},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'notify',
'label': '发送消息',
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12,
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'info',
'variant': 'tonal',
'text': '插件目前仅支持电影(需要开启智能重命名)。如果按评分分类7-9 高分4-6 一般1-3 垃圾。 系列电影不参与评分, 不按年代分类。'
}
}
]
}
]
}
]
}
], {
"enabled": False,
"notify": False,
"year_class": False,
"vote_class": False,
"collection_class": False
}
def get_page(self) -> List[dict]:
pass
@eventmanager.register(ChainEventType.TransferRename)
def category_handler(self, event: Event):
"""
根据多级分类规则重新分类组装地址
"""
logger.debug(f"多级分类插件触发!")
# 基础验证
if not self.get_state():
logger.debug(f"多级分类插件未启用!")
return
if not event:
logger.warning(f"多级分类异常:事件对象为空")
return
if not hasattr(event, 'event_data'):
logger.warning(f"多级分类异常:事件数据为空")
return
try:
data = event.event_data
# 验证必要的数据字段
if not hasattr(data, 'render_str') or not data.render_str:
logger.warning(f"多级分类异常render_str为空")
return
else:
render_str = data.render_str
# 暂时只支持电影分类
if not hasattr(data, 'rename_dict') or not data.rename_dict:
logger.warning(f"多级分类异常rename_dict为空")
return
else:
rename_dict = data.rename_dict
video_type = rename_dict.get("type", "")
if video_type != "电影":
logger.debug(f"多级分类异常:不支持的媒体类型: {video_type}, 只支持电影分类")
return
# 安全获取数据字段
title = rename_dict.get("title", "")
en_title = rename_dict.get("en_title", "")
year = rename_dict.get("year")
vote_average = rename_dict.get("vote_average")
media_info = rename_dict.get("__mediainfo__")
# 初始化默认值
vote_count = 0
c_name = None
vote_path = "未知评分"
decade = 0
# 安全处理媒体信息
if media_info and hasattr(media_info, 'vote_count'):
try:
vote_count = int(media_info.vote_count) if media_info.vote_count else 0
except (ValueError, TypeError):
vote_count = 0
if hasattr(media_info, 'tmdb_info') and media_info.tmdb_info:
collection = media_info.tmdb_info.get("belongs_to_collection")
if collection and isinstance(collection, dict):
c_name = collection.get("name")
# 安全处理评分数据
try:
if vote_average is not None:
vote_average = float(vote_average)
else:
vote_average = 0
except (ValueError, TypeError):
vote_average = 0
# 评分分类逻辑
if vote_count < 10:
vote_average = 0
vote_path = "评分不足"
elif vote_average >= 7:
vote_path = "高分电影"
elif vote_average >= 4:
vote_path = "一般电影"
else:
vote_path = "垃圾电影"
# 安全处理年份数据
try:
if year and str(year).isdigit():
year_int = int(year)
if 1900 <= year_int <= 2100: # 合理的年份范围
decade = (year_int // 10) * 10
else:
decade = 0
logger.warning(f"年份超出合理范围: {year}")
else:
decade = 0
except (ValueError, TypeError):
decade = 0
logger.warning(f"年份转换失败: {year}")
# 构建分类路径
path_parts = []
if self._collection_class and c_name:
# 当collection为true时只添加collection name
# 清理collection名称移除特殊字符
clean_c_name = str(c_name).strip()
if clean_c_name:
path_parts.append("系列电影")
path_parts.append(clean_c_name)
else:
# 当collection不为true时根据其他配置添加路径
if self._vote_class and vote_path:
path_parts.append(vote_path)
if self._year_class and decade > 0:
path_parts.append(f"{decade}s")
# 构建最终的路径
if path_parts:
# 确保render_str不为空
safe_render_str = str(render_str).strip() if render_str else ""
event.event_data.updated_str = f"{'/'.join(path_parts)}/{safe_render_str}"
# 更新事件数据
event.event_data.updated = True
event.event_data.source = "MultiClass"
# 发送消息
if self._notify:
self.post_message(
mtype=NotificationType.Organize,
title="多级分类完成",
text=f"已重新分类: {event.event_data.updated_str}",
)
else:
event.event_data.updated = False
logger.warning(f"多级分类失败: 未找到分类路径,请检查配置是否已开启")
except Exception as e:
logger.error(f"多级分类异常: {str(e)}", exc_info=True)
# 确保即使出错也不会影响原始数据
if hasattr(event, 'event_data') and event.event_data:
event.event_data.updated = False
event.event_data.updated_str = getattr(data, 'render_str', '') if data else ''
def stop_service(self):
"""
停止服务
"""
pass

View File

@@ -38,7 +38,7 @@ class PersonMeta(_PluginBase):
# 插件图标
plugin_icon = "actor.png"
# 插件版本
plugin_version = "2.2"
plugin_version = "2.2.2"
# 插件作者
plugin_author = "jxxghp"
# 作者主页
@@ -337,6 +337,9 @@ class PersonMeta(_PluginBase):
if not self._enabled:
return
# 事件数据
if not event or not event.event_data:
logger.warn("TransferComplete事件数据为空")
return
mediainfo: MediaInfo = event.event_data.get("mediainfo")
meta: MetaBase = event.event_data.get("meta")
if not mediainfo or not meta:
@@ -406,7 +409,7 @@ class PersonMeta(_PluginBase):
"""
peoples = []
# 更新当前媒体项人物
for people in iteminfo["People"] or []:
for people in iteminfo.get("People", []) or []:
if self._event.is_set():
logger.info(f"演职人员刮削服务停止")
return
@@ -488,7 +491,7 @@ class PersonMeta(_PluginBase):
if not seasons:
logger.warn(f"{item.title} 未找到季媒体项")
return
for season in seasons["Items"]:
for season in seasons.get("Items", []):
# 获取豆瓣演员信息
season_actors = self.__get_douban_actors(mediainfo=mediainfo, season=season.get("IndexNumber"))
# 如果是Jellyfin更新季的人物Emby/Plex季没有人物
@@ -514,7 +517,7 @@ class PersonMeta(_PluginBase):
logger.warn(f"{item.title} 未找到集媒体项")
continue
# 更新集媒体项人物
for episode in episodes["Items"]:
for episode in episodes.get("Items", []):
# 获取集媒体项详情
episodeinfo = self.get_iteminfo(server=server, server_type=server_type,
itemid=episode.get("Id"))
@@ -664,9 +667,13 @@ class PersonMeta(_PluginBase):
# 锁定人物信息
if updated_name:
if "LockedFields" not in personinfo:
personinfo["LockedFields"] = []
if "Name" not in personinfo["LockedFields"]:
personinfo["LockedFields"].append("Name")
if updated_overview:
if "LockedFields" not in personinfo:
personinfo["LockedFields"] = []
if "Overview" not in personinfo["LockedFields"]:
personinfo["LockedFields"].append("Overview")
@@ -1041,9 +1048,12 @@ class PersonMeta(_PluginBase):
res = service.instance.post_data(url=url)
if res and res.status_code in [200, 204]:
return True
else:
elif res is not None:
logger.error(f"更新Jellyfin媒体项图片失败错误码{res.status_code}")
return False
else:
logger.error(f"更新Jellyfin媒体项图片失败返回结果为空")
return False
except Exception as err:
logger.error(f"更新Jellyfin媒体项图片失败{err}")
return False

View File

@@ -3,33 +3,59 @@ import base64
import ipaddress
import json
import socket
import time
from datetime import datetime, timedelta
from typing import Any, List, Dict, Tuple, Optional
from ipaddress import IPv4Network, IPv6Network, IPv4Address, IPv6Address
from typing import Any, List, Dict, Tuple, Optional, Literal, overload
import pytz
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger
from fastapi import Response
from pydantic import BaseModel, Field
from app.core.config import settings
from app.core.event import eventmanager, Event
from app.db.site_oper import SiteOper
from app.log import logger
from app.plugins import _PluginBase
from app.plugins.tobypasstrackers.dns_helper import DnsHelper
from app.schemas.types import EventType, NotificationType
from app.utils.http import RequestUtils
from .dns_helper import DnsHelper
class IpCidrItem(BaseModel):
# IP CIDR
ip_cidr: str
# 解析时间
timestamp: int = Field(default=0)
# DNS
nameserver: str | None = Field(default=None)
# 域名
domain: str | None = Field(default=None)
def to_dict(self) -> dict:
if self.timestamp:
dns_time = datetime.fromtimestamp(int(self.timestamp)).strftime("%Y-%m-%d %H:%M:%S")
else:
dns_time = '-'
return {
'ip_cidr': self.ip_cidr,
'domain': self.domain or '',
'nameserver': self.nameserver or '-',
'datetime': dns_time
}
class ToBypassTrackers(_PluginBase):
# 插件名称
plugin_name = "绕过Trackers"
# 插件描述
plugin_desc = "提供tracker服务器IP地址列表帮助IPv6连接绕过OpenClash。"
plugin_desc = "提供 Tracker 服务器 IP 地址列表,帮助 IPv6 连接绕过 OpenClash。"
# 插件图标
plugin_icon = "Clash_A.png"
# 插件版本
plugin_version = "1.4.2"
plugin_version = "1.5.1"
# 插件作者
plugin_author = "wumode"
# 作者主页
@@ -40,13 +66,15 @@ class ToBypassTrackers(_PluginBase):
plugin_order = 21
# 可使用的用户级别
auth_level = 2
# CN IP lists
chn_route6_lists_url = "https://ispip.clang.cn/all_cn_ipv6.txt"
chn_route_lists_url = "https://ispip.clang.cn/all_cn.txt"
# 定时器
_scheduler: Optional[BackgroundScheduler] = None
# 开关
_enabled: bool = False
_cron: str = ""
_notify = False
_notify: bool = False
_onlyonce: bool = False
_custom_trackers: str = ""
_exempted_domains: str = ""
@@ -55,38 +83,35 @@ class ToBypassTrackers(_PluginBase):
_china_ipv6_route: bool = True
_bypass_ipv4: bool = True
_bypass_ipv6: bool = True
_dns_input: str = ""
ipv6_txt: str = ""
ipv4_txt: str = ""
_dns_input: str | None = None
trackers: Dict[str, List[str]] = {}
def init_plugin(self, config: dict = None):
self.stop_service()
self.trackers = {}
self.ipv6_txt = self.get_data("ipv6_txt") if self.get_data("ipv6_txt") else ""
self.ipv4_txt = self.get_data("ipv4_txt") if self.get_data("ipv4_txt") else ""
try:
with open(f"{settings.ROOT_PATH}/app/plugins/tobypasstrackers/sites/trackers", "r", encoding="utf-8") as f:
site_file = settings.ROOT_PATH/'app'/'plugins'/self.__class__.__name__.lower()/'sites'/'trackers'
with open(site_file, "r", encoding="utf-8") as f:
base64_str = f.read()
self.trackers = json.loads(base64.b64decode(base64_str).decode("utf-8"))
except Exception as e:
logger.error(f"插件加载错误:{e}")
# 配置
if config:
self._enabled = config.get("enabled")
self._cron = config.get("cron")
self._onlyonce = config.get("onlyonce")
self._notify = config.get("notify")
self._custom_trackers = config.get("custom_trackers")
self._exempted_domains = config.get("exempted_domains")
self._enabled = bool(config.get("enabled"))
self._cron = config.get("cron") or "0 4 * * *"
self._onlyonce = bool(config.get("onlyonce"))
self._notify = bool(config.get("notify"))
self._custom_trackers = config.get("custom_trackers") or ""
self._exempted_domains = config.get("exempted_domains") or ""
self._bypassed_sites = config.get("bypassed_sites") or []
self._bypass_ipv4 = config.get("bypass_ipv4")
self._bypass_ipv6 = config.get("bypass_ipv6")
self._dns_input = config.get("dns_input")
self._china_ipv6_route = config.get("china_ipv6_route")
self._china_ip_route = config.get("china_ip_route")
self._bypass_ipv4 = bool(config.get("bypass_ipv4"))
self._bypass_ipv6 = bool(config.get("bypass_ipv6"))
self._dns_input: str | None = config.get("dns_input")
self._china_ipv6_route = bool(config.get("china_ipv6_route"))
self._china_ip_route = bool(config.get("china_ip_route"))
# 过滤掉已删除的站点
all_sites = [site.id for site in SiteOper().list_order_by_pri()]
self._bypassed_sites = [site_id for site_id in all_sites if site_id in self._bypassed_sites]
@@ -94,14 +119,13 @@ class ToBypassTrackers(_PluginBase):
if self._enabled or self._onlyonce:
self._scheduler = BackgroundScheduler(timezone=settings.TZ)
if self._onlyonce:
logger.info(f"立即运行一次")
logger.info("立即运行一次")
self._scheduler.add_job(self.update_ips, "date",
run_date=datetime.now(
tz=pytz.timezone(settings.TZ)) + timedelta(seconds=3)
)
self._onlyonce = False
self.__update_config()
# self._scheduler.print_jobs()
self._scheduler.start()
def get_state(self) -> bool:
@@ -128,14 +152,24 @@ class ToBypassTrackers(_PluginBase):
@staticmethod
def get_command() -> List[Dict[str, Any]]:
return [{
"cmd": "/refresh_tracker_ips",
"event": EventType.PluginAction,
"desc": "更新 Tracker IP 列表",
"data": {
"action": "refresh_tracker_ips"
return [
{
"cmd": "/refresh_tracker_ips",
"event": EventType.PluginAction,
"desc": "更新 Tracker IP 列表",
"data": {
"action": "refresh_tracker_ips"
}
},
{
"cmd": "/check_ip",
"event": EventType.PluginAction,
"desc": "检测 IP 是否在绕过列表中: /check_ip <域名或IP>",
"data": {
"action": "check_ip"
}
}
}]
]
def get_api(self) -> List[Dict[str, Any]]:
"""
@@ -147,13 +181,15 @@ class ToBypassTrackers(_PluginBase):
"summary": "API说明"
}]
"""
return [{
"path": "/bypassed_ips",
"endpoint": self.bypassed_ips,
"methods": ["GET"],
"summary": "绕过的IP",
"description": "绕过Clash核心的IP地址列表",
}]
return [
{
"path": "/bypassed_ips",
"endpoint": self.bypassed_ips,
"methods": ["GET"],
"summary": "绕过的 IP",
"description": "绕过 Clash 核心的 IP 地址列表",
}
]
def get_form(self) -> Tuple[List[dict], Dict[str, Any]]:
site_options = ([{"title": site.name, "value": site.id}
@@ -261,7 +297,7 @@ class ToBypassTrackers(_PluginBase):
'component': 'VSwitch',
'props': {
'model': 'china_ip_route',
'label': '合并中国大陆IPv4列表',
'label': '合并中国大陆 IPv4 列表',
}
}
]
@@ -277,7 +313,7 @@ class ToBypassTrackers(_PluginBase):
'component': 'VSwitch',
'props': {
'model': 'china_ipv6_route',
'label': '合并中国大陆IPv6列表',
'label': '合并中国大陆 IPv6 列表',
}
}
]
@@ -316,7 +352,7 @@ class ToBypassTrackers(_PluginBase):
'props': {
'model': 'dns_input',
'label': 'DNS 服务器',
'placeholder': '留空则使用本地DNS'
'placeholder': '留空则使用本地 DNS'
}
}
]
@@ -357,9 +393,9 @@ class ToBypassTrackers(_PluginBase):
'component': 'VTextarea',
'props': {
'model': 'custom_trackers',
'label': '自定义Tracker服务器',
'label': '自定义 Tracker 服务器',
'rows': 3,
'placeholder': '每行一个域名或IP'
'placeholder': '每行一个域名或 IP'
}
}
]
@@ -375,9 +411,76 @@ class ToBypassTrackers(_PluginBase):
'component': 'VTextarea',
'props': {
'model': 'exempted_domains',
'label': '排除的域名和IP',
'label': '排除的域名和 IP',
'rows': 3,
'placeholder': '每行一个域名或IP'
'placeholder': '每行一个域名或 IP'
}
}
]
}
]
},
{
'component': 'VCard',
'content': [
{
'component': 'VCardItem',
'props': {
'prepend-icon': 'mdi-link-variant',
'title': '订阅 URL',
'subtitle': '请先在 MoviePilot 设置中配置「访问域名」',
'class': 'pb-0'
},
},
{
'component': 'VCardActions',
'props': {
},
'content': [
{
'component': 'VBtn',
'text': 'IPv4',
'props': {
'append-icon': 'mdi-open-in-new',
'href': self.api_url(protocol=4),
'target': '_blank'
},
},
{
'component': 'VBtn',
'text': 'IPv6',
'props': {
'append-icon': 'mdi-open-in-new',
'href': self.api_url(protocol=6),
'target': '_blank'
},
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12
},
'content': [
{
'component': 'VAlert',
'props': {
'title': 'DNS 服务器示例',
'border': 'start',
'variant': 'tonal',
'text': '仅填一个: '
'「223.5.5.5」、'
'「[2400:3200::1]:53」、'
'「quic://dns.alidns.com:853」、'
'「https://dns.alidns.com/dns-query」。'
}
}
]
@@ -396,60 +499,13 @@ class ToBypassTrackers(_PluginBase):
{
'component': 'VAlert',
'props': {
'type': 'info',
'variant': 'tonal',
'text': 'DNS 服务器示例 (仅填一个): '
'「94.140.14.140」、'
'「94.140.14.140:53」、'
'「[2a10:50c0::1:ff]:53」、'
'「https://unfiltered.adguard-dns.com/dns-query」。'
'仅支持UDP和HTTPS方法, 留空使用本地DNS查询'
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'info',
'variant': 'tonal',
'text': '【订阅URL】'
f'「IPv4 API」: /api/v1/plugin/ToBypassTrackers/bypassed_ips?apikey={settings.API_TOKEN}&protocol=4; '
f'「IPv6 API」: /api/v1/plugin/ToBypassTrackers/bypassed_ips?apikey={settings.API_TOKEN}&protocol=6'
}
}
]
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {
'cols': 12
},
'content': [
{
'component': 'VAlert',
'props': {
'type': 'info',
'variant': 'tonal',
'text': '【如何使用】'
'在「OpenClash->插件设置->中国大陆IP路由」选择「绕过中国大陆」; '
'在「OpenClash->插件设置->Chnroute Update」填入「订阅URL」。'
'color': 'info',
'border': 'start',
'title': '如何使用',
'text': '在「OpenClash->插件设置->流量控制->绕过指定区域 IP」选择「绕过中国大陆」; '
'在「OpenClash->插件设置->大陆白名单订阅」填入「订阅 URL」。'
'使用聊天命令`/check_ip <域名或IP>`检查 IP 是否在绕过列表'
}
}
]
@@ -474,7 +530,94 @@ class ToBypassTrackers(_PluginBase):
}
def get_page(self) -> List[dict]:
pass
headers = [
{'title': 'IP CIDR', 'key': 'ip_cidr', 'sortable': True},
{'title': '域名', 'key': 'domain', 'sortable': True},
{'title': 'DNS', 'key': 'nameserver', 'sortable': True},
{'title': '解析时间', 'key': 'datetime', 'sortable': True},
]
items = [IpCidrItem.model_validate(detail).to_dict()
for detail in (self.get_data("cidr_details") or []) if detail.get('domain') != 'CN']
excluded_items = [IpCidrItem.model_validate(detail).to_dict()
for detail in (self.get_data("excluded_cidr_details") or [])]
return [
{
'component': 'VWindow',
'props': {
'show-arrows': 'hover',
},
'content': [
{
'component': 'VWindowItem',
'content': [
{
'component': 'VCard',
'props': {
'class': 'pa-0',
'title': '绕过的 Tracker 服务器 IP 列表',
'subtitle': '以下是已解析并添加到绕过列表中的 Tracker 服务器 IP 地址,'
'请在 OpenClash 中配置「绕过中国大陆 IP」并订阅本列表以实现绕过效果。',
},
'content': [
{
'component': 'VCardText',
'content': [
{
'component': 'VDataTableVirtual',
'props': {
'class': 'text-sm',
'headers': headers,
'items': items,
'height': '30rem',
'density': 'compact',
'fixed-header': True,
'hide-no-data': True,
'hover': True
}
}
]
}
]
}
]
},
{
'component': 'VWindowItem',
'content': [
{
'component': 'VCard',
'props': {
'class': 'pa-0',
'title': '排除的 IP 列表',
'variant': 'elevated',
},
'content': [
{
'component': 'VCardText',
'content': [
{
'component': 'VDataTableVirtual',
'props': {
'class': 'text-sm',
'headers': headers,
'items': excluded_items,
'height': '30rem',
'density': 'compact',
'fixed-header': True,
'hide-no-data': True,
'hover': True
}
}
]
}
]
}
]
},
]
}
]
def get_dashboard(self, key: str = None, **kwargs) -> Optional[Tuple[Dict[str, Any], Dict[str, Any], List[dict]]]:
"""
@@ -491,10 +634,14 @@ class ToBypassTrackers(_PluginBase):
"""
pass
def api_url(self, protocol: int = 4) -> str:
return settings.MP_DOMAIN(f'/api/v1/plugin/{self.__class__.__name__}/bypassed_ips?apikey={settings.API_TOKEN}'
f'&protocol={protocol}')
def stop_service(self):
"""
退出插件
"""
退出插件
"""
try:
if self._scheduler:
self._scheduler.remove_all_jobs()
@@ -525,77 +672,147 @@ class ToBypassTrackers(_PluginBase):
}]
return []
def bypassed_ips(self, protocol: str) -> Response:
if protocol == '6':
return Response(content=self.ipv6_txt, media_type="text/plain")
return Response(content=self.ipv4_txt, media_type="text/plain")
def bypassed_ips(self, protocol: Literal['4', '6']) -> Response:
data_key = "ipv4_txt" if protocol == '4' else "ipv6_txt"
data = self.get_data(data_key) or ""
return Response(content=data, media_type="text/plain")
@eventmanager.register(EventType.PluginAction)
def update_ips(self, event: Optional[Event]=None):
def __is_ip_in_subnet(ip_input: str, su_bnet: str) -> bool:
"""
Check if the given IP address is in the specified subnet.
def check_ip(self, event: Event):
"""检查 IP 地址 是否在绕过列表"""
event_data = event.event_data
if not event_data or event_data.get("action") != "check_ip":
return
host = event_data.get("arg_str")
channel = event_data.get("channel")
userid = event_data.get("userid")
logger.info(f"检查 IP 是否绕过: {host} (来自用户 {userid},渠道 {channel})")
ip_list, bypassed, excluded = self._check_details(host)
if not ip_list:
self.post_message(channel=channel, user=userid, text=f"无法解析 host: {host}", title=f"{host}")
return
message = ""
for ip in ip_list:
detail = bypassed.get(ip)
excluded_detail = excluded.get(ip)
sub_message = f"{ip}"
if excluded_detail is not None:
detail_msg = '\n'.join(f"{k}: {v}" for k,v in excluded_detail.to_dict().items())
sub_message += f" 在排除列表中:\n{detail_msg}\n"
if detail is not None:
detail_msg = '\n'.join(f"{k}: {v}" for k,v in detail.to_dict().items())
sub_message += f" 在绕过列表中:\n{detail_msg}\n"
if detail and not excluded_detail:
sub_message += f"✈️ 会被绕过。\n"
else:
sub_message += f"🛑 不会被绕过。\n"
message += sub_message + "\n"
self.post_message(channel=channel, user=userid, text=message, title=f"{host}")
:param ip_input: IP address as a string (e.g., '192.168.1.1')
:param su_bnet: Subnet in CIDR notation (e.g., '192.168.1.0/24')
:return: True if IP is in the subnet, False otherwise
"""
ip_obj = ipaddress.ip_address(ip_input)
subnet_obj = ipaddress.ip_network(su_bnet, strict=False)
return ip_obj in subnet_obj
@overload
def _load_cn_ip_lists(self, family: type[IPv4Network]) -> list[IPv4Network]: ...
def __search_ip(_ip, ips_list):
i = 0
for ip_range in ips_list:
if __is_ip_in_subnet(_ip, ip_range):
return i
i += 1
return -1
@overload
def _load_cn_ip_lists(self, family: type[IPv6Network]) -> list[IPv6Network]: ...
def __exclude_ip_range(range_b: str, range_a: str):
"""
Exclude IP range A from IP range B and return the remaining subranges.
def _load_cn_ip_lists(self, family: type[IPv4Network] | type[IPv6Network] = IPv4Network
) -> list[IPv4Network | IPv6Network]:
ip_list: list[IPv4Network | IPv6Network] = []
if family is IPv4Network:
url = self.chn_route_lists_url
elif family is IPv6Network:
url = self.chn_route6_lists_url
else:
raise NotImplementedError(f"unknown address family {family}")
res = RequestUtils().get_res(url=url, raise_exception=True)
if res is None or res.status_code != 200:
logger.warn(f"无法获取 CN IP 列表: {url}")
raise ConnectionError
route_list = res.text.strip().split('\n')
for cn_ip_cidr in route_list:
subnet = ipaddress.ip_network(cn_ip_cidr, strict=False)
if isinstance(subnet, family):
ip_list.append(subnet)
return ip_list
:param range_b: The larger IP range in CIDR notation (must include range_a).
:param range_a: The smaller IP range to exclude in CIDR notation.
:return: List of remaining IP subranges in CIDR notation.
"""
net_b = ipaddress.ip_network(range_b, strict=False)
net_a = ipaddress.ip_network(range_a, strict=False)
def _search_details(self, ip_list: list[IPv4Address | IPv6Address], data_key: str) -> dict[str, IpCidrItem | None]:
cidr_details = [IpCidrItem.model_validate(detail) for detail in (self.get_data(data_key) or [])]
ip_cidr_list = [ipaddress.ip_network(item.ip_cidr, strict=False) for item in cidr_details]
details: dict[str, IpCidrItem | None] = {}
for ip in ip_list:
index = ToBypassTrackers._search_ip(ip, ip_cidr_list)
if index == -1:
details[str(ip)] = None
continue
details[str(ip)] = cidr_details[index]
return details
if not (net_a.subnet_of(net_b)):
raise ValueError("Range A is not fully contained within Range B.")
def _check_details(self, host: str) -> tuple[list[str], dict[str, IpCidrItem | None], dict[str, IpCidrItem | None]]:
try:
ip_list = [ipaddress.ip_address(host)]
except ValueError:
dns = DnsHelper(dns_server=self._dns_input)
resolved = asyncio.run(dns.resolve_name(host))
if resolved is None:
return [], {}, {}
ip_list = [ipaddress.ip_address(ip) for ip in resolved]
details = self._search_details(ip_list, "cidr_details")
excluded = self._search_details(ip_list, "excluded_cidr_details")
return [str(ip) for ip in ip_list], details, excluded
remaining_ranges = list(net_b.address_exclude(net_a))
@staticmethod
def _search_ip(ip: IPv4Address | IPv6Address, ips_list: list[IPv4Network | IPv6Network]) -> int:
i = 0
for ip_range in ips_list:
if ip in ip_range:
return i
i += 1
return -1
return [str(sub_net) for sub_net in remaining_ranges]
@staticmethod
def _search_subnet(ip: IPv4Network | IPv6Network, ips_list: list[IPv4Network | IPv6Network]) -> int:
i = 0
for ip_range in ips_list:
if ip.subnet_of(ip_range):
return i
i += 1
return -1
async def resolve_and_check(domain_, results_, failed_msg_, dns_type_, ip_list_):
@eventmanager.register(EventType.PluginAction)
def update_ips(self, event: Optional[Event] = None):
async def resolve_and_check(domain_: str, results_: dict[str, bool], failed_msg_: list[str],
family: int, ip_list_: list[IPv4Network | IPv6Network],
cidr_details_: list[IpCidrItem]):
try:
addresses = await query_helper.query_dns(domain_, dns_type_)
addresses = await query_helper.resolve_name(domain_, family)
if addresses is None:
failed_msg_.append(f"{domain_name_map.get(domain_, domain_)}{domain_}: {dns_type_} 记录查询失败")
dns_type = "AAAA" if family == socket.AF_INET6 else "A"
failed_msg_.append(f"{domain_name_map.get(domain_, domain_)}{domain_}: {dns_type} 记录查询失败")
results_[domain_name_map.get(domain_, domain_)] = False
return
for address in addresses:
has_flag = any(__is_ip_in_subnet(address, subnet) for subnet in ip_list_)
for ip_str in addresses:
ip_obj = ipaddress.ip_address(ip_str)
has_flag = any(ip_obj in sub_net for sub_net in ip_list_)
if not has_flag:
if dns_type_ == "AAAA":
ip_list_.append(address)
else:
ip_list_.append(address)
logger.info(f"Resolving【{domain_name_map.get(domain_, domain_)}{address} ({domain_})")
net_obj = ipaddress.ip_network(ip_obj, strict=False)
ip_list_.append(net_obj)
ip_cidr_item = IpCidrItem(ip_cidr=str(net_obj), domain=domain_,
timestamp=int(time.time()), nameserver=query_helper.nameserver)
cidr_details_.append(ip_cidr_item)
logger.info(f"Resolving【{domain_name_map.get(domain_, domain_)}{ip_str} ({domain_})")
except Exception as e:
logger.exception(f"处理 {domain_} 出错: {e}")
logger.warn(f"处理 {domain_} 出错: {e}")
results_[domain_name_map.get(domain_, domain_)] = False
async def resolve_all(domains_, ipv6_list_, ip_list_):
async def resolve_all(domains_: list[str], ipv6_list_: list[IPv6Network], ip_list_: list[IPv4Network],
details: list[IpCidrItem]):
tasks = [
resolve_and_check(domain_, results_v6, failed_msg, "AAAA", ipv6_list_)
resolve_and_check(domain_, results_v6, failed_msg, socket.AF_INET6, ipv6_list_, details)
for domain_ in domains_
]
tasks.extend([resolve_and_check(domain_, results, failed_msg, "A", ip_list_)
tasks.extend([resolve_and_check(domain_, results, failed_msg, socket.AF_INET, ip_list_, details)
for domain_ in domains_])
await asyncio.gather(*tasks)
@@ -604,31 +821,27 @@ class ToBypassTrackers(_PluginBase):
if not event_data or event_data.get("action") != "refresh_tracker_ips":
return
query_helper = DnsHelper(self._dns_input)
logger.info(f"开始通过 {query_helper.method_name} 解析DNS")
chnroute6_lists_url = "https://ispip.clang.cn/all_cn_ipv6.txt"
chnroute_lists_url = "https://ispip.clang.cn/all_cn.txt"
ipv6_list = []
ip_list = []
logger.info(f"开始通过 {query_helper.nameserver} 解析DNS")
ipv6_list: list[IPv6Network] = []
ip_list: list[IPv4Network] = []
domains = []
success_msg = []
failed_msg = []
results = {}
results: dict[str, bool] = {} # 解析结果
unsupported_msg = []
results_v6 = {}
results_v6: dict[str, bool] = {}
cidr_details: list[IpCidrItem] = []
exempted_cidr_details: list[IpCidrItem] = []
# 加载 CN IP 列表
if self._china_ipv6_route:
# Load Chnroute6 Lists
res = RequestUtils().get_res(url=chnroute6_lists_url)
if res is not None and res.status_code == 200:
chnroute6_lists = res.text[:-1].split('\n')
for ipr in chnroute6_lists:
ipv6_list.append(ipr)
ipv6_list = self._load_cn_ip_lists(family=IPv6Network)
if self._china_ip_route:
# Load Chnroute Lists
res = RequestUtils().get_res(url=chnroute_lists_url)
if res is not None and res.status_code == 200:
chnroute_lists = res.text[:-1].split('\n')
for ipr in chnroute_lists:
ip_list.append(ipr)
ip_list = self._load_cn_ip_lists(family=IPv4Network)
for ip in ipv6_list + ip_list:
cidr_details.append(IpCidrItem(ip_cidr=str(ip), domain="CN", timestamp=int(time.time())))
do_sites = {site.domain: site.name for site in SiteOper().list_order_by_pri() if
site.id in self._bypassed_sites}
domain_name_map = {}
@@ -645,70 +858,74 @@ class ToBypassTrackers(_PluginBase):
for custom_tracker in self._custom_trackers.split('\n'):
if custom_tracker:
try:
socket.inet_pton(socket.AF_INET, custom_tracker)
if self._bypass_ipv4:
ip_list.append(f"{custom_tracker}/32")
except socket.error:
try:
socket.inet_pton(socket.AF_INET6, custom_tracker)
address = ipaddress.ip_address(custom_tracker)
net = ipaddress.ip_network(address)
if isinstance(net, IPv4Network):
if self._bypass_ipv4:
ip_list.append(net)
elif isinstance(net, IPv6Network):
if self._bypass_ipv6:
ipv6_list.append(ipaddress.ip_network(f"{custom_tracker}/128", strict=False).compressed)
except socket.error:
ipv6_list.append(net)
except ValueError:
domains.append(custom_tracker)
v6_ips = []
v4_ips = []
asyncio.run(resolve_all(domains, v6_ips, v4_ips))
ipv6_list.extend([ipaddress.ip_network(f"{ad}/128", strict=False).compressed for ad in v6_ips])
ip_list.extend([f"{ad}/32" for ad in v4_ips])
v6_nets = []
v4_nets = []
asyncio.run(resolve_all(domains, v6_nets, v4_nets, cidr_details))
ipv6_list.extend(v6_nets)
ip_list.extend(v4_nets)
for result in results:
if results[result]:
success_msg.append(f"{result}】 Trackers已被添加")
exempted_ip = []
exempted_ipv6 = []
exempted_ip: list[IPv4Network] = []
exempted_ipv6: list[IPv6Network] = []
exempted_domains = []
for exempted_domain in self._exempted_domains.split('\n'):
if exempted_domain:
try:
socket.inet_pton(socket.AF_INET, exempted_domain)
if self._bypass_ipv4:
exempted_ip.append(f"{exempted_domain}")
except socket.error:
try:
socket.inet_pton(socket.AF_INET6, exempted_domain)
if self._bypass_ipv6:
exempted_ipv6.append(f"{exempted_domain}")
except socket.error:
exempted_domains.append(exempted_domain)
address = ipaddress.ip_address(exempted_domain)
net = ipaddress.ip_network(address)
asyncio.run(resolve_all(exempted_domains, exempted_ip, exempted_ipv6))
if isinstance(net, IPv4Network):
if self._bypass_ipv4:
exempted_ip.append(net)
elif isinstance(net, IPv6Network):
if self._bypass_ipv6:
exempted_ipv6.append(net)
exempted_cidr_details.append(IpCidrItem(ip_cidr=str(net), domain=exempted_domain,
timestamp=int(time.time())))
except ValueError:
exempted_domains.append(exempted_domain)
cidr_details_dict = {detail.ip_cidr: detail for detail in cidr_details}
asyncio.run(resolve_all(exempted_domains, exempted_ipv6, exempted_ip, exempted_cidr_details))
for ip in exempted_ip:
index = __search_ip(ip, ip_list)
if index == -1:
continue
ip_larger = ip_list[index]
ip_list.pop(index)
length = int(ip_larger.split('/')[1])
if length < 12:
remaining_ip = __exclude_ip_range(ip_larger, f"{ip}/{length + 8}")
ip_list.extend(remaining_ip)
while (index:= ToBypassTrackers._search_subnet(ip, ip_list)) != -1:
subnet = ip_list[index]
ip_list.pop(index)
source = cidr_details_dict[str(subnet)].domain if str(subnet) in cidr_details_dict else "CN"
logger.warn(f"Excluding subnet {subnet} ({source}) for exempted IP {ip}")
if subnet.prefixlen < 12:
new_subnet = IPv4Network((ip.network_address, subnet.prefixlen + 8), strict=False)
ip_list.extend(subnet.address_exclude(new_subnet))
for ip in exempted_ipv6:
index = __search_ip(ip, ipv6_list)
if index == -1:
continue
ip_larger = ipv6_list[index]
ipv6_list.pop(index)
length = int(ip_larger.split('/')[1])
if length < 32:
remaining_ip = __exclude_ip_range(ip_larger, f"{ip}/{min(32, length + 8)}")
ipv6_list.extend(remaining_ip)
self.ipv4_txt = "\n".join(ip_list)
self.ipv6_txt = "\n".join(ipv6_list)
self.save_data("ipv4_txt", self.ipv4_txt)
self.save_data("ipv6_txt", self.ipv6_txt)
while (index:=ToBypassTrackers._search_subnet(ip, ipv6_list)) != -1:
subnet = ipv6_list[index]
ipv6_list.pop(index)
source = cidr_details_dict[str(subnet)].domain if str(subnet) in cidr_details_dict else "CN"
logger.warn(f"Excluding subnet {subnet} ({source}) for exempted IP {ip}")
if subnet.prefixlen < 32:
new_subnet = IPv6Network((ip.network_address, min(32, subnet.prefixlen + 8)), strict=False)
ipv6_list.extend(subnet.address_exclude(new_subnet))
ipv4_txt = "\n".join(str(net) for net in ip_list)
ipv6_txt = "\n".join(str(net) for net in ipv6_list)
self.save_data("ipv4_txt", ipv4_txt)
self.save_data("ipv6_txt", ipv6_txt)
self.save_data("cidr_details", [detail.model_dump() for detail in cidr_details])
self.save_data("excluded_cidr_details", [detail.model_dump() for detail in exempted_cidr_details])
if self._notify:
res_message = success_msg + failed_msg
res_message = "\n".join(res_message)
self.post_message(title=f"【绕过Trackers】",
mtype=NotificationType.Plugin,
text=f"{res_message}"
)
self.post_message(
title=f"{self.plugin_name}",
mtype=NotificationType.Plugin,
text=f"{res_message}"
)

View File

@@ -1,86 +1,108 @@
import re
from typing import Optional, List, Callable
import ipaddress
import socket
from urllib.parse import urlparse
import dns.asyncresolver
import dns.resolver
from dns import asyncresolver, query
from dns.nameserver import Do53Nameserver, DoHNameserver, DoTNameserver, DoQNameserver
from dns.resolver import NoAnswer, NXDOMAIN
from app.log import logger
class DnsHelper:
def __init__(self, dns_server: str):
self.method_name = "Local"
self.doh_url = "https://dns.alidns.com/dns-query"
self.__resolver = dns.asyncresolver.Resolver()
self.__dns_query_method = self.__query_method(dns_server)
def __query_method(self, dns_input: str) -> Callable:
if not dns_input:
return self.query_dns_local
if dns_input.startswith('https://'):
self.doh_url = dns_input
self.method_name = dns_input
return self.query_dns_doh
udp_match = re.match(r"^(?:udp://)?(\[?.+?]?)(?::(\d+))?$", dns_input)
if udp_match:
try:
self.__resolver.nameservers = [udp_match.group(1).strip('[]')]
if udp_match.group(2):
self.__resolver.port = int(udp_match.group(2))
self.method_name = f"udp://{self.__resolver.nameservers[0]}:{self.__resolver.port}"
except Exception as e:
logger.warn(f'{e}, using default resolver')
return self.query_dns_local
return self.query_dns_udp
logger.warn(f'Unknown method {dns_input}, using default resolver')
return self.query_dns_local
def __init__(self, dns_server: str | None = None):
self._resolver = asyncresolver.Resolver()
self._use_tcp: bool = False
if dns_server:
self.nameserver = dns_server
async def query_dns(self, domain: str, dns_type: str = "A") -> Optional[List[str]]:
answers = await self.__dns_query_method(domain, dns_type)
return answers
@property
def nameserver(self) ->str:
nameserver = self._resolver.nameservers[0]
return str(nameserver)
async def query_dns_local(self, domain: str, dns_type: str = "A") -> Optional[List[str]]:
@nameserver.setter
def nameserver(self, value: str | None):
if value is None:
self._resolver = asyncresolver.Resolver()
return
self._parse_dns_server(value)
@staticmethod
def get_ip_from_hostname(hostname) -> str | None:
try:
answer = await self.__resolver.resolve(domain, dns_type)
return [record.address for record in answer if hasattr(record, "address")]
except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN):
return []
except Exception as e:
# logger.error(f"本地DNS查询错误: {e} {domain}")
# 获取IP地址
ip = socket.gethostbyname(hostname)
return ip
except socket.gaierror:
return None
async def query_dns_doh(self, domain: str, dns_type: str = 'A') -> Optional[List[str]]:
@staticmethod
def is_ip_address(hostname):
try:
# 尝试解析为IP地址
ipaddress.ip_address(hostname)
return True
except ValueError:
return False
def _parse_dns_server(self, dns_server: str):
if "://" not in dns_server:
dns_server = f"udp://{dns_server}"
parsed = urlparse(dns_server)
# check and resolve the hostname
hostname = parsed.hostname
if hostname is None:
return
if DnsHelper.is_ip_address(hostname):
address = hostname
hostname = None
else:
address = DnsHelper.get_ip_from_hostname(hostname)
if address is None:
return
nameserver = None
match parsed.scheme:
case "udp":
nameserver = Do53Nameserver(address, parsed.port or 53)
case "tcp":
nameserver = Do53Nameserver(address, parsed.port or 53)
self._use_tcp = True
case "https":
nameserver = DoHNameserver(url=dns_server)
case "tls":
nameserver = DoTNameserver(address=address, port=parsed.port or 853, hostname=hostname)
case "h3":
nameserver = DoHNameserver(url=dns_server.replace("h3://", "https://"),
http_version=query.HTTPVersion.H3)
case "quic":
nameserver = DoQNameserver(address=address, port=parsed.port or 853, server_hostname=hostname)
case _:
nameserver = None
if nameserver is None:
self._resolver = asyncresolver.Resolver()
return
self._resolver.nameservers = [nameserver]
async def resolve_name(self, domain: str, family: int = socket.AF_UNSPEC) -> list[str] | None:
"""
使用 DNS-over-HTTPS (DoH) 异步解析域名
异步解析域名
:param domain: 要解析的域名
:param dns_type: DNS 记录类型,例如 'A', 'AAAA'
:param family: The address family
- socket.AF_UNSPEC: both IPv4 and IPv6 addresses
- socket.AF_INET6: IPv6 addresses only
- socket.AF_INET: IPv4 addresses only
:return: IP 地址列表,或 None
"""
try:
query = dns.message.make_query(domain, dns_type)
response = await dns.asyncquery.https(query, self.doh_url)
return [
item.address for rrset in response.answer for item in rrset.items
if hasattr(item, "address")
]
except Exception as e:
return None
async def query_dns_udp(self, domain: str, dns_type: str = 'A') -> Optional[List[str]]:
"""
使用 UDP 异步方式解析域名
:param domain: 域名
:param dns_type: 记录类型,如 A、AAAA
:return: IP地址列表 或 None
"""
try:
answer = await self.__resolver.resolve(domain, dns_type)
return [record.address for record in answer]
except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN):
answer = await self._resolver.resolve_name(domain, family=family, tcp=self._use_tcp)
return [a for a in answer.addresses()]
except (NoAnswer, NXDOMAIN):
return []
except Exception:
except Exception as e:
logger.debug(f"DNS查询出错 ({domain}): {e} ")
return None

View File

@@ -1,2 +1,2 @@
dnspython~=2.7.0
aioquic~=1.2.0
dnspython~=2.8.0
aioquic~=1.2.0

View File

@@ -1 +1 @@
eyJoZGRvbGJ5LmNvbSI6IFsidC5oZGRvbGJ5LmNvbSJdLCAidGp1cHQub3JnIjogWyJ0cmFja2VyLXB1YmxpYy50anVwdC5vcmciXSwgIm5pY2VwdC5uZXQiOiBbInd3dy5uaWNlcHQubmV0Il0sICJyb3VzaS56aXAiOiBbImhpdHB0LmNvbSJdLCAicHRob21lLm5ldCI6IFsicHRob21lLm5ldCJdLCAiaGR0aW1lLm9yZyI6IFsiaGR0aW1lLm9yZyJdLCAiZWFzdGdhbWUub3JnIjogWyJwdC5lYXN0Z2FtZS5vcmciXSwgInB0dGltZS5vcmciOiBbInd3dy5wdHRpbWUub3JnIl0sICJtLXRlYW0uY2MiOiBbInRyYWNrZXIubS10ZWFtLmNjIiwgInRyYWNrZXIubS10ZWFtLmlvIl0sICI1MnB0LnNpdGUiOiBbIjUycHQuc2l0ZSJdLCAicWluZ3dhcHQuY29tIjogWyJ0cmFja2VyLnFpbmd3YS5wcm8iLCAidHJhY2tlci5xaW5nd2FwdC5jb20iXSwgImhka3lsLmluIjogWyJ0cmFja2VyLmhka3lsLmluIl0sICJyYWluZ2ZoLnRvcCI6IFsicmFpbmdmaC50b3AiXSwgImhkZmFucy5vcmciOiBbImhkZmFucy5vcmciXSwgInB0bGdzLm9yZyI6IFsicHRsLmdzIiwgInJlbGF5MDEucHRsLmdzIl0sICJtb25pa2FkZXNpZ24udWsiOiBbInRyYWNrZXIubW9uaWthZGVzaWduLnVrIiwgImRhaWtpcmFpLm1vbmlrYWRlc2lnbi51ayIsICJhbmltZS1uby1pbmRleC5jb20iXSwgInB0c2Jhby5jbHViIjogWyJwdHNiYW8uY2x1YiJdLCAidG90aGVnbG9yeS5pbSI6IFsidHJhY2tlci50b3RoZWdsb3J5LmltIl0sICJ1Mi5kbWh5Lm9yZyI6IFsiZGF5ZHJlYW0uZG1oeS5iZXN0Il0sICJieXIucHQiOiBbInRyYWNrZXIuYnlyLnB0Il0sICJodWRidC5odXN0LmVkdS5jbiI6IFsiaHVkYnQuaHVzdC5lZHUuY24iXSwgImlsb2xpY29uLmNvbSI6IFsidHJhY2tlci5pbG9saWNvbi5jYyJdLCAiaGl0cHQuY29tIjogWyJoaXRwdC5jb20iXSwgImJ0c2Nob29sLmNsdWIiOiBbInB0LmJ0c2Nob29sLmNsdWIiXSwgImhkYXJlYS5jbHViIjogWyJ0cmFja2VyLmhkYXJlYS5jbHViIl0sICJzcHJpbmdzdW5kYXkubmV0IjogWyJvbjYuc3ByaW5nc3VuZGF5Lm5ldCIsICJvbi5zcHJpbmdzdW5kYXkubmV0Il0sICJ6bXB0LmNjIjogWyJ6bXB0LmNjIl0sICJjYXJwdC5uZXQiOiBbInRyYWNrZXIuY2FycHQubmV0Il0sICJpY2MyMDIyLmNvbSI6IFsidHJhY2tlci5pY2MyMDIyLnh5eiJdLCAia2VlcGZyZHMuY29tIjogWyJ0cmFja2VyLmtlZXBmcmRzLmNvbSJdLCAicHR6b25lLnh5eiI6IFsicHR6b25lLnh5eiJdLCAiY3NwdC50b3AiOiBbInRyYWNrZXIuY3NwdC50b3AiLCAidHJhY2tlci5jc3B0LmNjIiwgInRyYWNrZXIuY3NwdC5kYXRlIl0sICJjcmFicHQudmlwIjogWyJjcmFicHQudmlwIl0sICJva3B0Lm5ldCI6IFsid3d3Lm9rcHQubmV0Il0sICJnYW1lZ2FtZXB0LmNvbSI6IFsid3d3LmdhbWVnYW1lcHQuY29tIl0sICJhdWRpZW5jZXMubWUiOiBbInQuYXVkaWVuY2VzLm1lIiwgInRyYWNrZXIuY2luZWZpbGVzLmluZm8iXSwgInhpbmd5dW5nZS50b3AiOiBbInRyYWNrZXIueGluZ3l1bmdlLnRvcCIsICJ0cmFja2VyLnhpbmd5dW5nZS5zYnMiXSwgImV0OC5vcmciOiBbImV0OC5vcmciLCAidC5ldDgub3JnIl0sICJkaXNjZmFuLm5ldCI6IFsiZGlzY2Zhbi54eXoiXX0=
eyI1MnB0LnNpdGUiOiBbIjUycHQuc2l0ZSJdLCAiYXVkaWVuY2VzLm1lIjogWyJ0LmF1ZGllbmNlcy5tZSIsICJ0cmFja2VyLmNpbmVmaWxlcy5pbmZvIl0sICJidHNjaG9vbC5jbHViIjogWyJwdC5idHNjaG9vbC5jbHViIl0sICJieXIucHQiOiBbInRyYWNrZXIuYnlyLnB0Il0sICJjYXJwdC5uZXQiOiBbInRyYWNrZXIuY2FycHQubmV0Il0sICJjcmFicHQudmlwIjogWyJjcmFicHQudmlwIl0sICJjc3B0LnRvcCI6IFsidHJhY2tlci5jc3B0LnRvcCIsICJ0cmFja2VyLmNzcHQuY2MiLCAidHJhY2tlci5jc3B0LmRhdGUiXSwgImRpc2NmYW4ubmV0IjogWyJkaXNjZmFuLnh5eiJdLCAiZWFzdGdhbWUub3JnIjogWyJwdC5lYXN0Z2FtZS5vcmciXSwgImV0OC5vcmciOiBbImV0OC5vcmciLCAidC5ldDgub3JnIl0sICJnYW1lZ2FtZXB0LmNvbSI6IFsid3d3LmdhbWVnYW1lcHQuY29tIl0sICJoZGFyZWEuY2x1YiI6IFsidHJhY2tlci5oZGFyZWEuY2x1YiJdLCAiaGRkb2xieS5jb20iOiBbInQuaGRkb2xieS5jb20iXSwgImhkZmFucy5vcmciOiBbImhkZmFucy5vcmciXSwgImhka3lsLmluIjogWyJ0cmFja2VyLmhka3lsLmluIiwgInd3dy5oZGt5bGluLnRvcCJdLCAiaGR0aW1lLm9yZyI6IFsiaGR0aW1lLm9yZyJdLCAiaGl0cHQuY29tIjogWyJoaXRwdC5jb20iXSwgImh1ZGJ0Lmh1c3QuZWR1LmNuIjogWyJodWRidC5odXN0LmVkdS5jbiJdLCAiaWNjMjAyMi5jb20iOiBbInRyYWNrZXIuaWNjMjAyMi54eXoiXSwgImlsb2xpY29uLmNvbSI6IFsidHJhY2tlci5pbG9saWNvbi5jYyJdLCAia2VlcGZyZHMuY29tIjogWyJ0cmFja2VyLmtlZXBmcmRzLmNvbSJdLCAibS10ZWFtLmNjIjogWyJ0cmFja2VyLm0tdGVhbS5jYyIsICJ0cmFja2VyLm0tdGVhbS5pbyJdLCAibW9uaWthZGVzaWduLnVrIjogWyJ0cmFja2VyLm1vbmlrYWRlc2lnbi51ayIsICJkYWlraXJhaS5tb25pa2FkZXNpZ24udWsiLCAiYW5pbWUtbm8taW5kZXguY29tIl0sICJuaWNlcHQubmV0IjogWyJ3d3cubmljZXB0Lm5ldCJdLCAib2twdC5uZXQiOiBbInd3dy5va3B0Lm5ldCJdLCAicHRob21lLm5ldCI6IFsicHRob21lLm5ldCJdLCAicHRsZ3Mub3JnIjogWyJwdGwuZ3MiLCAicmVsYXkwMS5wdGwuZ3MiXSwgInB0c2Jhby5jbHViIjogWyJwdHNiYW8uY2x1YiJdLCAicHR0aW1lLm9yZyI6IFsid3d3LnB0dGltZS5vcmciXSwgInB0em9uZS54eXoiOiBbInB0em9uZS54eXoiXSwgInFpbmd3YXB0LmNvbSI6IFsidHJhY2tlci5xaW5nd2EucHJvIiwgInRyYWNrZXIucWluZ3dhcHQuY29tIiwgInRyYWNrZXIucWluZ3dhcHQub3JnIl0sICJyYWluZ2ZoLnRvcCI6IFsicmFpbmdmaC50b3AiXSwgInJvdXNpLnppcCI6IFsiaGl0cHQuY29tIl0sICJzcHJpbmdzdW5kYXkubmV0IjogWyJvbjYuc3ByaW5nc3VuZGF5Lm5ldCIsICJvbi5zcHJpbmdzdW5kYXkubmV0Il0sICJ0anVwdC5vcmciOiBbInRyYWNrZXItcHVibGljLnRqdXB0Lm9yZyJdLCAidG90aGVnbG9yeS5pbSI6IFsidHJhY2tlci50b3RoZWdsb3J5LmltIl0sICJ1Mi5kbWh5Lm9yZyI6IFsiZGF5ZHJlYW0uZG1oeS5iZXN0Il0sICJ4aW5neXVuZ2UudG9wIjogWyJ0cmFja2VyLnhpbmd5dW5nZS50b3AiLCAidHJhY2tlci54aW5neXVuZ2Uuc2JzIl0sICJ6bXB0LmNjIjogWyJ6bXB0LmNjIl0sICJoaGFuY2x1Yi50b3AiOiBbInRyYWNrZXIuaGhhbmNsdWIudG9wIl0sICJoZGNpdHkuY2l0eSI6IFsic3luYy5sZW5pdGVyLm9yZyJdLCAib3VyYml0cy5jbHViIjogWyJvdXJiaXRzLmNsdWIiXX0=

View File

@@ -101,6 +101,7 @@ class AutoSubv2(_PluginBase):
_max_retries = None
_enable_merge = None
_enable_asr = None
_auto_detect_language = None
_huggingface_proxy = None
_faster_whisper_model_path = None
_faster_whisper_model = None
@@ -126,6 +127,7 @@ class AutoSubv2(_PluginBase):
self._faster_whisper_model_path = config.get('faster_whisper_model_path',
self.get_data_path() / "faster-whisper-models")
self._huggingface_proxy = config.get('proxy', True)
self._auto_detect_language = config.get('auto_detect_language', False)
self._translate_zh = config.get('translate_zh', False)
if self._translate_zh:
use_chatgpt = config.get('use_chatgpt', True)
@@ -407,16 +409,28 @@ class AutoSubv2(_PluginBase):
model = WhisperModel(
download_model(self._faster_whisper_model, local_files_only=False, cache_dir=cache_dir),
device="cpu", compute_type="int8", cpu_threads=psutil.cpu_count(logical=False))
segments, info = model.transcribe(audio_file,
language=lang if lang != 'auto' else None,
word_timestamps=True,
vad_filter=True,
temperature=0,
beam_size=5)
logger.info("Detected language '%s' with probability %f" % (info.language, info.language_probability))
try:
segments, info = model.transcribe(audio_file,
language=lang if lang != 'auto' else None,
word_timestamps=True,
vad_filter=True,
temperature=0,
beam_size=5)
logger.info("Detected language '%s' with probability %f" % (info.language, info.language_probability))
if lang == 'auto':
lang = info.language
if lang == 'auto':
lang = info.language
except ValueError as e:
if "max() iterable argument is empty" in str(e):
logger.info("音频文件中未检测到任何语言内容,生成空字幕文件以避免重复处理")
# 生成空的字幕文件,避免重复识别
self.__save_srt(f"{audio_file}.srt", [])
# 如果原本是auto检测设置一个默认语言
lang = 'und' if lang == 'auto' else lang
return True, lang
else:
raise e
subs = []
if lang in ['en', 'eng']:
@@ -481,9 +495,15 @@ class AutoSubv2(_PluginBase):
if not ret:
logger.info(f"字幕源偏好:{self._translate_preference} 获取音轨元数据失败")
return False, None, None
if not iso639.find(audio_lang) or not iso639.to_iso639_1(audio_lang):
# 如果开启了自动语言检测直接设置为auto跳过metadata的语言信息
if self._auto_detect_language:
logger.info("已开启自动语言检测将使用whisper模型自动识别语言")
audio_lang = 'auto'
elif not iso639.find(audio_lang) or not iso639.to_iso639_1(audio_lang):
logger.info(f"字幕源偏好:{self._translate_preference} 未从音轨元数据中获取到语言信息")
audio_lang = 'auto'
# 当字幕源偏好为origin_first时优先使用音轨语言
if self._translate_preference == "origin_first":
prefer_subtitle_langs = ['en', 'eng'] if audio_lang == 'auto' else [audio_lang,
@@ -570,7 +590,7 @@ class AutoSubv2(_PluginBase):
os.remove(f"{audio_file.name}.srt")
return ret, lang, Path(f"{subtitle_file}.{lang}.srt")
else:
logger.error(f"生成字幕失败")
logger.error("生成字幕失败")
return False, None, None
@staticmethod
@@ -810,8 +830,8 @@ class AutoSubv2(_PluginBase):
def __translate_to_zh(self, text: str, context: str = None) -> str:
if self._event.is_set():
raise UserInterruptException(f"用户中断当前任务")
return self._openai.translate_to_zh(text, context)
raise UserInterruptException("用户中断当前任务")
return self._openai.translate_to_zh(text, context, max_retries=self._max_retries)
def __process_batch(self, all_subs: list, batch: list) -> list:
"""批量处理逻辑"""
@@ -839,20 +859,17 @@ class AutoSubv2(_PluginBase):
def __process_single(self, all_subs: List[srt.Subtitle], item: srt.Subtitle) -> srt.Subtitle:
"""单条处理逻辑"""
for _ in range(self._max_retries):
idx = all_subs.index(item)
context = self.__get_context(all_subs, [idx], is_batch=False) if self._context_window > 0 else None
success, trans = self.__translate_to_zh(item.content, context)
idx = all_subs.index(item)
context = self.__get_context(all_subs, [idx], is_batch=False) if self._context_window > 0 else None
success, trans = self.__translate_to_zh(item.content, context)
if success:
item.content = f"{trans}\n{item.content}"
self._stats['line_fallback'] += 1
return item
time.sleep(1)
item.content = f"[翻译失败]\n{item.content}"
return item
if success:
item.content = f"{trans}\n{item.content}"
self._stats['line_fallback'] += 1
return item
else:
item.content = f"[翻译失败]\n{item.content}"
return item
def __translate_zh_subtitle(self, source_lang: str, source_subtitle: str, dest_subtitle: str):
self._stats = {'total': 0, 'batch_success': 0, 'batch_fail': 0, 'line_fallback': 0}
@@ -862,6 +879,13 @@ class AutoSubv2(_PluginBase):
logger.info(f"英文字幕合并:合并前字幕数: {len(subs)},合并后字幕数: {len(valid_subs)}")
else:
valid_subs = subs
if not valid_subs:
logger.warning("字幕文件为空或没有有效的字幕条目,跳过翻译")
# 创建一个空的字幕文件
self.__save_srt(dest_subtitle, [])
return
self._stats['total'] = len(valid_subs)
processed = []
current_batch = []
@@ -878,10 +902,13 @@ class AutoSubv2(_PluginBase):
processed += self.__process_items(valid_subs, current_batch)
self.__save_srt(dest_subtitle, processed)
success_rate = (self._stats['batch_success'] / self._stats['total'] * 100) if self._stats['total'] > 0 else 0.0
logger.info(f"""
翻译完成!
总处理条目: {self._stats['total']}
批次成功: {self._stats['batch_success']} ({(self._stats['batch_success'] / self._stats['total']) * 100:.1f}%)
批次成功: {self._stats['batch_success']} ({success_rate:.1f}%)
批次失败: {self._stats['batch_fail']}
行补偿翻译: {self._stats['line_fallback']}
""")
@@ -1179,6 +1206,20 @@ class AutoSubv2(_PluginBase):
}
]
},
{
'component': 'VCol',
'props': {'cols': 12, 'md': 4, 'v-show': 'enable_asr'},
'content': [
{
'component': 'VSwitch',
'props': {
'model': 'auto_detect_language',
'label': '自动检测语言',
'hint': '使用whisper模型自动检测语言而非依赖视频元数据'
}
}
]
},
{
'component': 'VCol',
'props': {'cols': 12, 'md': 4, 'v-show': 'enable_asr'},
@@ -1196,10 +1237,15 @@ class AutoSubv2(_PluginBase):
}
}
]
},
}
]
},
{
'component': 'VRow',
'content': [
{
'component': 'VCol',
'props': {'cols': 12, 'md': 4, 'v-show': 'enable_asr'},
'props': {'cols': 12, 'md': 12, 'v-show': 'enable_asr'},
'content': [
{
'component': 'VSwitch',
@@ -1498,6 +1544,7 @@ class AutoSubv2(_PluginBase):
"translate_preference": "english_first",
"translate_zh": False,
"enable_asr": True,
"auto_detect_language": False,
"faster_whisper_model": "base",
"proxy": True,
"use_chatgpt": True,

Some files were not shown because too many files have changed in this diff Show More