🚀 feat(model): Added support for Grounding tasks with YOLOE model for text/visual prompting and prompt-free detection & segmentation

Author: CVHub520

.gitignore

@@ -5,6 +5,8 @@
 __pycache__
 *.pyc
 *.egg-info
+*.pt
+*.onnx
 anylabeling/data
 /data
 /dist

README.md

@@ -19,18 +19,21 @@
 
 ![](https://user-images.githubusercontent.com/18329471/234640541-a6a65fbc-d7a5-4ec3-9b65-55305b01a7aa.png)
 
-
 https://github.com/user-attachments/assets/f517fa94-c49c-4f05-864e-96b34f592079
 
+https://github.com/user-attachments/assets/52cbdb5d-cc60-4be5-826f-903ea4330ca8
+
+<div align="center"><strong>Text/Visual Prompting and Prompt-free for Detection & Segmentation</strong></div>
+
 <br>
 
 <img src="https://github.com/user-attachments/assets/7f43bcec-96fd-48d1-bd36-9e5a440a66f6" width="100%" />
-<div align="center"><strong>Visual Prompting for Detection</strong></div>
+<div align="center"><strong>Detect Anything</strong></div>
 
 <br>
 
 <img src="https://github.com/user-attachments/assets/208dc9ed-b8c9-4127-9e5b-e76f53892f03" width="100%" />
-<div align="center"><strong>Visual Prompting for Segmentation</strong></div>
+<div align="center"><strong>Segment Anything</strong></div>
 
 <br>
 
@@ -40,6 +43,7 @@ https://github.com/user-attachments/assets/f517fa94-c49c-4f05-864e-96b34f592079
 
 ## 🥳 What's New
 
+- Added support for Grounding tasks with YOLOE model for text/visual prompting and prompt-free detection & segmentation
 - Bump version to [3.0.1](https://github.com/CVHub520/X-AnyLabeling/releases/tag/v3.0.1)
 - For more details, please refer to the [CHANGELOG](./CHANGELOG.md)
 
@@ -74,12 +78,12 @@ https://github.com/user-attachments/assets/f517fa94-c49c-4f05-864e-96b34f592079
 | 📏 Depth Estimation | Depth Anything |
 | 🧩 Segment Anything | SAM, SAM-HQ, SAM-Med2D, EdgeSAM, EfficientViT-SAM, MobileSAM, 
 | ✂️ Image Matting | RMBG |
-| 📍 Grounding | CountGD, GeCO, Grunding DINO, YOLO-World |
 | 💡 Proposal | UPN |
 | 🏷️ Tagging | RAM, RAM++ |
 | 📄 OCR | PP-OCR |
 | 🗣️ VLM | Florence2, 
 | 🛣️ Land Detection | CLRNet |
+| 📍 Grounding | CountGD, GeCO, Grunding DINO, YOLO-World, YOLOE |
 | 📚 Other | 👉 [model_zoo](./docs/en/model_zoo.md) 👈 |
 
 

README_zh-CN.md

@@ -19,27 +19,31 @@
 
 ![](https://user-images.githubusercontent.com/18329471/234640541-a6a65fbc-d7a5-4ec3-9b65-55305b01a7aa.png)
 
-
 https://github.com/user-attachments/assets/f517fa94-c49c-4f05-864e-96b34f592079
 
+https://github.com/user-attachments/assets/52cbdb5d-cc60-4be5-826f-903ea4330ca8
+
+<div align="center"><strong>基于文本/视觉提示或免提示的检测和分割统一模型</strong></div>
+
 <br>
 
 <img src="https://github.com/user-attachments/assets/7f43bcec-96fd-48d1-bd36-9e5a440a66f6" width="100%" />
-<div align="center"><strong>Visual Prompting for Detection</strong></div>
+<div align="center"><strong>检测一切</strong></div>
 
 <br>
 
 <img src="https://github.com/user-attachments/assets/208dc9ed-b8c9-4127-9e5b-e76f53892f03" width="100%" />
-<div align="center"><strong>Visual Prompting for Segmentation</strong></div>
+<div align="center"><strong>分割一切</strong></div>
 
 <br>
 
 <img src="https://github.com/user-attachments/assets/56c9a20b-c836-47aa-8b54-bad5bb99b735" width="100%" />
-<div align="center"><strong>Chatbot</strong></div>
+<div align="center"><strong>聊天机器人</strong></div>
 
 
 ## 🥳 新功能
 
+- 新增 `Grounding` 任务支持，集成 `YOLOE` 模型实现文本/视觉提示和无提示检测分割
 - X-AnyLabeling [v3.0.1](https://github.com/CVHub520/X-AnyLabeling/releases/tag/v3.0.1) 最新版本发布
 - 更多详情，请参考[更新日志](./CHANGELOG.md)
 
@@ -75,12 +79,12 @@ https://github.com/user-attachments/assets/f517fa94-c49c-4f05-864e-96b34f592079
 | 📏 **深度估计** | Depth Anything |
 | 🧩 **分割一切** | SAM, SAM-HQ, SAM-Med2D, EdgeSAM, EfficientViT-SAM, MobileSAM |
 | ✂️ **图像抠图** | RMBG |
-| 📍 **视觉定位** | CountGD, GeCO, Grunding DINO, YOLO-World |
 | 💡 **候选框提取** | UPN |
 | 🏷️ **图像标记** | RAM, RAM++ |
 | 📄 **光学字符识别** | PP-OCR |
 | 🗣️ **视觉语言模型** | Florence2 |
 | 🛣️ **车道线检测** | CLRNet |
+| 📍 **Grounding** | CountGD, GeCO, Grunding DINO, YOLO-World, YOLOE |
 | 📚 **其他** | 👉 [model_zoo](./docs/en/model_zoo.md) 👈 |
 
 

anylabeling/app_info.py

@@ -1,5 +1,5 @@
 __appname__ = "X-AnyLabeling"
 __appdescription__ = "Advanced Auto Labeling Solution with Added Features"
-__version__ = "3.0.0"
+__version__ = "3.0.1"
 __preferred_device__ = "CPU"  # GPU or CPU
 __url__ = "https://github.com/CVHub520/X-AnyLabeling"

anylabeling/configs/auto_labeling/yoloe_11s.yaml

@@ -0,0 +1,13 @@
+type: yoloe
+name: yoloe_11s-r20250524
+provider: THU
+display_name: YOLOE-11-S
+model_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/yoloe-11s-seg.pt
+model_pf_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/yoloe-11s-seg-pf.pt
+embedding_model_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/mobileclip_blt.pt
+input_height: 640
+input_width: 640
+iou_threshold: 0.70
+conf_threshold: 0.25
+max_det: 1000
+with_mask: false

anylabeling/configs/auto_labeling/yoloe_v8l.yaml

@@ -0,0 +1,13 @@
+type: yoloe
+name: yoloe_v8l-r20250524
+provider: THU
+display_name: YOLOE-v8-L
+model_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/yoloe-v8l-seg.pt
+model_pf_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/yoloe-v8l-seg-pf.pt
+embedding_model_path: https://github.com/CVHub520/X-AnyLabeling/releases/download/v3.0.0/mobileclip_blt.pt
+input_height: 640
+input_width: 640
+iou_threshold: 0.70
+conf_threshold: 0.25
+max_det: 1000
+with_mask: false

anylabeling/configs/models.yaml

@@ -180,6 +180,10 @@
   config_file: ":/yolo12s.yaml"
 - model_name: "yolo12x-r20250514"
   config_file: ":/yolo12x.yaml"
+- model_name: "yoloe_11s-r20250524"
+  config_file: ":/yoloe_11s.yaml"
+- model_name: "yoloe_v8l-r20250524"
+  config_file: ":/yoloe_v8l.yaml"
 - model_name: "yolov5_car_plate-r20230112"
   config_file: ":/yolov5_car_plate.yaml"
 - model_name: "yolov5l-r20230520"

anylabeling/services/auto_labeling/__init__.py

@@ -72,6 +72,7 @@
     "dfine",
     "yolo12",
     "u_rtdetr",
+    "yoloe",
 ]
 
 
@@ -96,6 +97,13 @@
     "edge_sam",
     "florence2",
     "geco",
+    "yoloe",
+]
+
+
+# --- skip_prediction_on_new_marks ---
+_SKIP_PREDICTION_ON_NEW_MARKS_MODELS = [
+    "yoloe",
 ]
 
 
@@ -163,6 +171,7 @@
     "dfine",
     "yolo12",
     "u_rtdetr",
+    "yoloe",
 ]
 
 
@@ -198,6 +207,7 @@
     "yolo11_pose_track",
     "yolox",
     "yolo12",
+    "yoloe",
 ]
 
 
@@ -243,6 +253,7 @@
     "dfine",
     "yolo12",
     "u_rtdetr",
+    "yoloe",
 ]
 
 

anylabeling/services/auto_labeling/model_manager.py

@@ -1797,6 +1797,26 @@ def _load_geco():
                     f"❌ Error in loading model: {model_config['type']} with error: {str(e)}"
                 )
                 return
+        elif model_config["type"] == "yoloe":
+            from .yoloe import YOLOE
+
+            try:
+                model_config["model"] = YOLOE(
+                    model_config, on_message=self.new_model_status.emit
+                )
+                self.auto_segmentation_model_unselected.emit()
+                logger.info(
+                    f"✅ Model loaded successfully: {model_config['type']}"
+                )
+            except Exception as e:  # noqa
+                template = "Error in loading model: {error_message}"
+                translated_template = self.tr(template)
+                error_text = translated_template.format(error_message=str(e))
+                self.new_model_status.emit(error_text)
+                logger.error(
+                    f"❌ Error in loading model: {model_config['type']} with error: {str(e)}"
+                )
+                return
         elif model_config["type"] == "u_rtdetr":
             from .u_rtdetr import U_RTDETR