Upload folder using huggingface_hub

.gitignore

@@ -0,0 +1,2 @@
+__pycache__
+__ignore*

.vscode/launch.json

@@ -0,0 +1,16 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Python Debugger: Current File",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "${file}",
+            "console": "integratedTerminal",
+            "justMyCode": false
+        }
+    ]
+}

CLAUDE.md

@@ -0,0 +1,91 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+This is **TCID** (Transformer CI Dashboard) - a Gradio-based web dashboard that displays test results for Transformer models across AMD and NVIDIA hardware. The application fetches CI test data from HuggingFace datasets and presents it through interactive visualizations and detailed failure reports.
+
+## Architecture
+
+### Core Components
+
+- **`app.py`** - Main Gradio application with UI components, plotting functions, and data visualization logic
+- **`data.py`** - Data fetching module that retrieves test results from HuggingFace datasets for AMD and NVIDIA CI runs
+- **`styles.css`** - Complete dark theme styling for the Gradio interface
+- **`requirements.txt`** - Python dependencies (matplotlib only)
+
+### Data Flow
+
+1. **Data Loading**: `get_data()` in `data.py` fetches latest CI results from:
+   - AMD: `hf://datasets/optimum-amd/transformers_daily_ci`
+   - NVIDIA: `hf://datasets/hf-internal-testing/transformers_daily_ci`
+
+2. **Data Processing**: Results are joined and filtered to show only important models defined in `IMPORTANT_MODELS` list
+
+3. **Visualization**: Two main views:
+   - **Summary Page**: Horizontal bar charts showing test results for all models
+   - **Detail View**: Pie charts for individual models with failure details
+
+### UI Architecture
+
+- **Sidebar**: Model selection, refresh controls, CI job links
+- **Main Content**: Dynamic display switching between summary and detail views
+- **Auto-refresh**: Data reloads every 15 minutes via background threading
+
+## Running the Application
+
+### Development Commands
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+
+# Run the application
+python app.py
+```
+
+### HuggingFace Spaces Deployment
+
+This application is configured for HuggingFace Spaces deployment:
+- **Framework**: Gradio 5.38.0
+- **App file**: `app.py`
+- **Configuration**: See `README.md` header for Spaces metadata
+
+## Key Data Structures
+
+### Model Results DataFrame
+The joined DataFrame contains these columns:
+- `success_amd` / `success_nvidia` - Number of passing tests
+- `failed_multi_no_amd` / `failed_multi_no_nvidia` - Multi-GPU failure counts
+- `failed_single_no_amd` / `failed_single_no_nvidia` - Single-GPU failure counts
+- `failures_amd` / `failures_nvidia` - Detailed failure information objects
+- `job_link_amd` / `job_link_nvidia` - CI job URLs
+
+### Important Models List
+Predefined list in `data.py` focusing on significant models:
+- Classic models: bert, gpt2, t5, vit, clip, whisper
+- Modern models: llama, gemma3, qwen2, mistral3
+- Multimodal: qwen2_5_vl, llava, smolvlm, internvl
+
+## Styling and Theming
+
+The application uses a comprehensive dark theme with:
+- Fixed sidebar layout (300px width)
+- Black background throughout (`#000000`)
+- Custom scrollbars with dark styling
+- Monospace fonts for technical aesthetics
+- Gradient buttons and hover effects
+
+## Error Handling
+
+- **Data Loading Failures**: Falls back to predefined model list for testing
+- **Missing Model Data**: Shows "No data available" message in visualizations
+- **Empty Results**: Gracefully handles cases with no test results
+
+## Performance Considerations
+
+- **Memory Management**: Matplotlib configured to prevent memory warnings
+- **Interactive Mode**: Disabled to prevent figure accumulation
+- **Auto-reload**: Background threading with daemon timers
+- **Data Caching**: Global variables store loaded data between UI updates

README.md

@@ -1,12 +1,33 @@
 ---
 title: Tcid
-emoji: 🏃
-colorFrom: gray
-colorTo: blue
+emoji: 👁
+colorFrom: indigo
+colorTo: pink
 sdk: gradio
-sdk_version: 5.45.0
+sdk_version: 5.38.0
 app_file: app.py
 pinned: false
+short_description: A dashboard
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# TCID
+
+This space displays the state of the `transformers` CI on two hardwares, for a subset of models. The CI is run daily, on both AMD MI325 and Nvidia A10. The CI runs a different number of tests for each model. When a test finishes, it is assigned a status depending on its outcome:
+
+- passed: the test finsihed and the expected output (or outputs) were retrieved;
+- failed: the test either did not finish or the output was different from the expected output;
+- skipped: the test was not run, which usually happens when a test is incompatible with a model. For instance, some models skip `flash-attention`-related tests because they are incompatible with `flash-attention`;
+- error: the test did not finish and python crashed;
+
+The dashboard is divided in two main parts:
+
+## Summary page
+
+On the summary page, you can see a snapshot of the mix of test passed, failed and skipped for each model. The summary page also features an "Overall failures rate" for AMD and NVIDIA, which is computed this way:
+```overall_failure_rate = (failed + error) / (passed + failed + error)``` 
+
+We do not account for the test skipped in this overall failure rate, because skipped test have no chance to neither pass nor fail.
+
+## Models page
+
+From the sidebar, you can access a detailled view of each model. In it, you will find the breakdown of test statuses and the names of the test that failed for single and multi-gpu runs.

app.py

@@ -0,0 +1,305 @@
+import matplotlib.pyplot as plt
+import matplotlib
+import pandas as pd
+import gradio as gr
+
+from data import CIResults
+from utils import logger
+from summary_page import create_summary_page
+from model_page import plot_model_stats
+
+
+# Configure matplotlib to prevent memory warnings and set dark background
+matplotlib.rcParams['figure.facecolor'] = '#000000'
+matplotlib.rcParams['axes.facecolor'] = '#000000'
+matplotlib.rcParams['savefig.facecolor'] = '#000000'
+plt.ioff()  # Turn off interactive mode to prevent figure accumulation
+
+
+# Load data once at startup
+Ci_results = CIResults()
+Ci_results.load_data()
+# Start the auto-reload scheduler
+Ci_results.schedule_data_reload()
+
+
+# Function to check if a model has failures
+def model_has_failures(model_name):
+    """Check if a model has any failures (AMD or NVIDIA)."""
+    if Ci_results.df is None or Ci_results.df.empty:
+        return False
+    
+    # Normalize model name to match DataFrame index
+    model_name_lower = model_name.lower()
+    
+    # Check if model exists in DataFrame
+    if model_name_lower not in Ci_results.df.index:
+        return False
+    row = Ci_results.df.loc[model_name_lower]
+    
+    # Check for failures in both AMD and NVIDIA
+    amd_multi_failures = row.get('failed_multi_no_amd', 0)
+    amd_single_failures = row.get('failed_single_no_amd', 0)
+    nvidia_multi_failures = row.get('failed_multi_no_nvidia', 0)
+    nvidia_single_failures = row.get('failed_single_no_nvidia', 0)
+    return any([
+        amd_multi_failures > 0,
+        amd_single_failures > 0,
+        nvidia_multi_failures > 0,
+        nvidia_single_failures > 0,
+    ])
+
+
+# Function to get current description text
+def get_description_text():
+    """Get description text with integrated last update time."""
+    msg = [
+        "Transformer CI Dashboard",
+        "-",
+        "AMD runs on MI325",
+        "NVIDIA runs on A10",
+    ]
+    msg = ["**" + x + "**" for x in msg] + [""]
+    if Ci_results.latest_update_msg:
+        msg.append(f"*({Ci_results.latest_update_msg})*")
+    else:
+        msg.append("*(loading...)*")
+    return "<br>".join(msg)
+
+# Load CSS from external file
+def load_css():
+    try:
+        with open("styles.css", "r") as f:
+            css_content = f.read()
+        
+        return css_content
+    except FileNotFoundError:
+        logger.warning("styles.css not found, using minimal default styles")
+        return "body { background: #000; color: #fff; }"
+
+
+# Create the Gradio interface with sidebar and dark theme
+with gr.Blocks(title="Model Test Results Dashboard", css=load_css()) as demo:
+
+
+    with gr.Row():
+        # Sidebar for model selection
+        with gr.Column(scale=1, elem_classes=["sidebar"]):
+            gr.Markdown("# 🤖 TCID", elem_classes=["sidebar-title"])
+
+            # Description with integrated last update time
+            description_text = get_description_text()
+            description_display = gr.Markdown(description_text, elem_classes=["sidebar-description"])
+
+            # Summary button at the top
+            summary_button = gr.Button(
+                "summary\n📊",
+                variant="primary",
+                size="lg",
+                elem_classes=["summary-button"]
+            )
+
+            # Model selection header (clickable toggle)
+            model_toggle_button = gr.Button(
+                f"► Select model ({len(Ci_results.available_models)})",
+                variant="secondary",
+                elem_classes=["model-header"]
+            )
+
+            # Model buttons container (collapsible) - start folded
+            with gr.Column(elem_classes=["model-list", "model-list-hidden"]) as model_list_container:
+                # Create individual buttons for each model
+                model_buttons = []
+                model_choices = [model.lower() for model in Ci_results.available_models] if Ci_results.available_models else ["auto", "bert", "clip", "llama"]
+                
+                print(f"Creating {len(model_choices)} model buttons: {model_choices}")
+
+                for model_name in model_choices:
+                    # Check if model has failures to determine styling
+                    has_failures = model_has_failures(model_name)
+                    button_classes = ["model-button"]
+                    if has_failures:
+                        button_classes.append("model-button-failed")
+                    
+                    btn = gr.Button(
+                        model_name,
+                        variant="secondary",
+                        size="sm",
+                        elem_classes=button_classes
+                    )
+                    model_buttons.append(btn)
+
+            # CI job links at bottom of sidebar
+            ci_links_display = gr.Markdown("🔗 **CI Jobs:** *Loading...*", elem_classes=["sidebar-links"])
+
+        # Main content area
+        with gr.Column(scale=4, elem_classes=["main-content"]):
+            # Summary display (default view)
+            summary_display = gr.Plot(
+                value=create_summary_page(Ci_results.df, Ci_results.available_models),
+                label="",
+                format="png",
+                elem_classes=["plot-container"],
+                visible=True
+            )
+
+            # Detailed view components (hidden by default)
+            with gr.Column(visible=False, elem_classes=["detail-view"]) as detail_view:
+
+                # Create the plot output
+                plot_output = gr.Plot(
+                    label="",
+                    format="png",
+                    elem_classes=["plot-container"]
+                )
+
+                # Create two separate failed tests displays in a row layout
+                with gr.Row():
+                    with gr.Column(scale=1):
+                        amd_failed_tests_output = gr.Textbox(
+                            value="",
+                            lines=8,
+                            max_lines=8,
+                            interactive=False,
+                            container=False,
+                            elem_classes=["failed-tests"]
+                        )
+                    with gr.Column(scale=1):
+                        nvidia_failed_tests_output = gr.Textbox(
+                            value="",
+                            lines=8,
+                            max_lines=8,
+                            interactive=False,
+                            container=False,
+                            elem_classes=["failed-tests"]
+                        )
+
+    # Set up click handlers for model buttons
+    for i, btn in enumerate(model_buttons):
+        model_name = model_choices[i]
+        btn.click(
+            fn=lambda selected_model=model_name: plot_model_stats(Ci_results.df, selected_model),
+            outputs=[plot_output, amd_failed_tests_output, nvidia_failed_tests_output]
+        ).then(
+            fn=lambda: [gr.update(visible=False), gr.update(visible=True)],
+            outputs=[summary_display, detail_view]
+        )
+
+    # Model toggle functionality
+    def toggle_model_list(current_visible):
+        """Toggle the visibility of the model list."""
+        new_visible = not current_visible
+        arrow = "▼" if new_visible else "►"
+        button_text = f"{arrow} Select model ({len(Ci_results.available_models)})"
+        
+        # Use CSS classes instead of Gradio visibility
+        css_classes = ["model-list"]
+        if new_visible:
+            css_classes.append("model-list-visible")
+        else:
+            css_classes.append("model-list-hidden")
+            
+        return gr.update(value=button_text), gr.update(elem_classes=css_classes), new_visible
+
+    # Track model list visibility state
+    model_list_visible = gr.State(False)
+
+    model_toggle_button.click(
+        fn=toggle_model_list,
+        inputs=[model_list_visible],
+        outputs=[model_toggle_button, model_list_container, model_list_visible]
+    )
+
+    # Summary button click handler
+    def show_summary_and_update_links():
+        """Show summary page and update CI links."""
+        return create_summary_page(Ci_results.df, Ci_results.available_models), get_description_text(), get_ci_links()
+
+    summary_button.click(
+        fn=show_summary_and_update_links,
+        outputs=[summary_display, description_display, ci_links_display]
+    ).then(
+        fn=lambda: [gr.update(visible=True), gr.update(visible=False)],
+        outputs=[summary_display, detail_view]
+    )
+
+    # Function to get CI job links
+    def get_ci_links():
+        """Get CI job links from the most recent data."""
+        try:
+            # Check if df exists and is not empty
+            if Ci_results.df is None or Ci_results.df.empty:
+                return "🔗 **CI Jobs:** *Loading...*"
+
+            # Get links from any available model (they should be the same for all models in a run)
+            amd_multi_link = None
+            amd_single_link = None
+            nvidia_multi_link = None
+            nvidia_single_link = None
+
+            for model_name in Ci_results.df.index:
+                row = Ci_results.df.loc[model_name]
+
+                # Extract AMD links
+                if pd.notna(row.get('job_link_amd')) and (not amd_multi_link or not amd_single_link):
+                    amd_link_raw = row.get('job_link_amd')
+                    if isinstance(amd_link_raw, dict):
+                        if 'multi' in amd_link_raw and not amd_multi_link:
+                            amd_multi_link = amd_link_raw['multi']
+                        if 'single' in amd_link_raw and not amd_single_link:
+                            amd_single_link = amd_link_raw['single']
+
+                # Extract NVIDIA links
+                if pd.notna(row.get('job_link_nvidia')) and (not nvidia_multi_link or not nvidia_single_link):
+                    nvidia_link_raw = row.get('job_link_nvidia')
+                    if isinstance(nvidia_link_raw, dict):
+                        if 'multi' in nvidia_link_raw and not nvidia_multi_link:
+                            nvidia_multi_link = nvidia_link_raw['multi']
+                        if 'single' in nvidia_link_raw and not nvidia_single_link:
+                            nvidia_single_link = nvidia_link_raw['single']
+
+                # Break if we have all links
+                if amd_multi_link and amd_single_link and nvidia_multi_link and nvidia_single_link:
+                    break
+
+
+            # Add FAQ link at the bottom
+            links_md = "❓ [**FAQ**](https://huggingface.co/spaces/transformers-community/transformers-ci-dashboard/blob/main/README.md)\n\n"
+            links_md += "🔗 **CI Jobs:**\n\n"
+
+            # AMD links
+            if amd_multi_link or amd_single_link:
+                links_md += "**AMD:**\n"
+                if amd_multi_link:
+                    links_md += f"• [Multi GPU]({amd_multi_link})\n"
+                if amd_single_link:
+                    links_md += f"• [Single GPU]({amd_single_link})\n"
+                links_md += "\n"
+
+            # NVIDIA links
+            if nvidia_multi_link or nvidia_single_link:
+                links_md += "**NVIDIA:**\n"
+                if nvidia_multi_link:
+                    links_md += f"• [Multi GPU]({nvidia_multi_link})\n"
+                if nvidia_single_link:
+                    links_md += f"• [Single GPU]({nvidia_single_link})\n"
+
+            if not (amd_multi_link or amd_single_link or nvidia_multi_link or nvidia_single_link):
+                links_md += "*No links available*"
+
+            return links_md
+        except Exception as e:
+            logger.error(f"getting CI links: {e}")
+            return "🔗 **CI Jobs:** *Error loading links*\n\n❓ **[FAQ](README.md)**"
+
+
+    # Auto-update CI links when the interface loads
+    demo.load(
+        fn=get_ci_links,
+        outputs=[ci_links_display]
+    )
+
+
+# Gradio entrypoint
+if __name__ == "__main__":
+    demo.launch()

data.py

@@ -0,0 +1,240 @@
+from huggingface_hub import HfFileSystem
+import pandas as pd
+from utils import logger
+from datetime import datetime
+import threading
+import traceback
+import json
+import re
+
+# NOTE: if caching is an issue, try adding `use_listings_cache=False`
+fs = HfFileSystem()
+
+IMPORTANT_MODELS = [
+    "auto",
+    "bert",  # old but dominant (encoder only)
+    "gpt2",  # old (decoder)
+    "t5",  # old (encoder-decoder)
+    "modernbert",  # (encoder only)
+    "vit",  # old (vision) - fixed comma
+    "clip",  # old but dominant (vision)
+    "detr",  # objection detection, segmentation (vision)
+    "table-transformer",  # objection detection (visioin) - maybe just detr?
+    "got_ocr2",  # ocr (vision)
+    "whisper",  # old but dominant (audio)
+    "wav2vec2",  # old (audio)
+    "llama",  # new and dominant (meta)
+    "gemma3",  # new (google)
+    "qwen2",  # new (Alibaba)
+    "mistral3",  # new (Mistral) - added missing comma
+    "qwen2_5_vl",  # new (vision)
+    "llava",  # many models from it (vision)
+    "smolvlm",  # new (video)
+    "internvl",  # new (video)
+    "gemma3n",  # new (omnimodal models)
+    "qwen2_5_omni",  # new (omnimodal models)
+]
+
+KEYS_TO_KEEP = [
+    "success_amd",
+    "success_nvidia",
+    "skipped_amd",
+    "skipped_nvidia",
+    "failed_multi_no_amd",
+    "failed_multi_no_nvidia",
+    "failed_single_no_amd",
+    "failed_single_no_nvidia",
+    "failures_amd",
+    "failures_nvidia",
+    "job_link_amd",
+    "job_link_nvidia",
+]
+
+
+def log_dataframe_link(link: str) -> str:
+    """
+    Adds the link to the dataset in the logs, modifies it to get a clockable link and then returns the date of the 
+    report.
+    """
+    logger.info(f"Reading df located at {link}")
+    # Make sure the links starts with an http adress
+    if link.startswith("hf://"):
+        link = "https://huggingface.co/" + link.removeprefix("hf://")
+    # Pattern to match transformers_daily_ci followed by any path, then a date (YYYY-MM-DD format)
+    pattern = r'transformers_daily_ci(.*?)/(\d{4}-\d{2}-\d{2})'
+    match = re.search(pattern, link)
+    # Failure case: 
+    if not match:
+        logger.error("Could not find transformers_daily_ci and.or date in the link")
+        return "9999-99-99"
+    # Replace the path between with blob/main
+    path_between = match.group(1)
+    link = link.replace("transformers_daily_ci" + path_between, "transformers_daily_ci/blob/main")
+    logger.info(f"Link to data source: {link}")
+    # Return the date
+    return match.group(2)
+
+def infer_latest_update_msg(date_df_amd: str, date_df_nvidia: str) -> str:
+    # Early return if one of the dates is invalid
+    if date_df_amd.startswith("9999") and date_df_nvidia.startswith("9999"):
+        return "could not find last update time"
+    # Warn if dates are not the same
+    if date_df_amd != date_df_nvidia:
+        logger.warning(f"Different dates found: {date_df_amd} (AMD) vs {date_df_nvidia} (NVIDIA)")
+    # Take the latest date and format it
+    try:
+        latest_date = max(date_df_amd, date_df_nvidia)
+        yyyy, mm, dd = latest_date.split("-")
+        return f"last updated {mm}/{dd}/{yyyy}"
+    except Exception as e:
+        logger.error(f"When trying to infer latest date, got error {e}")
+        return "could not find last update time"
+
+def read_one_dataframe(json_path: str, device_label: str) -> tuple[pd.DataFrame, str]:
+    df_upload_date = log_dataframe_link(json_path)
+    df = pd.read_json(json_path, orient="index", encoding_errors="ignore")
+    df.index.name = "model_name"
+    df[f"failed_multi_no_{device_label}"] = df["failures"].apply(lambda x: len(x["multi"]) if "multi" in x else 0)
+    df[f"failed_single_no_{device_label}"] = df["failures"].apply(lambda x: len(x["single"]) if "single" in x else 0)
+    return df, df_upload_date
+
+def get_distant_data() -> tuple[pd.DataFrame, str]:
+    # Retrieve AMD dataframe
+    amd_src = "hf://datasets/optimum-amd/transformers_daily_ci/**/runs/**/ci_results_run_models_gpu/model_results.json"
+    files_amd = sorted(fs.glob(amd_src, refresh=True), reverse=True)
+    df_amd, date_df_amd = read_one_dataframe(f"hf://{files_amd[0]}", "amd")
+    # Retrieve NVIDIA dataframe, which pattern should be:
+    # hf://datasets/hf-internal-testing`/transformers_daily_ci/raw/main/YYYY-MM-DD/ci_results_run_models_gpu/model_results.json 
+    nvidia_src = "hf://datasets/hf-internal-testing/transformers_daily_ci/*/ci_results_run_models_gpu/model_results.json"
+    files_nvidia = sorted(fs.glob(nvidia_src, refresh=True), reverse=True)
+    # NOTE: should this be removeprefix instead of lstrip?
+    nvidia_path = files_nvidia[0].lstrip('datasets/hf-internal-testing/transformers_daily_ci/')
+    nvidia_path = "https://huggingface.co/datasets/hf-internal-testing/transformers_daily_ci/raw/main/" + nvidia_path
+    df_nvidia, date_df_nvidia = read_one_dataframe(nvidia_path, "nvidia")
+    # Infer and format the latest df date
+    latest_update_msg = infer_latest_update_msg(date_df_amd, date_df_nvidia)
+    # Join both dataframes
+    joined = df_amd.join(df_nvidia, rsuffix="_nvidia", lsuffix="_amd", how="outer")
+    joined = joined[KEYS_TO_KEEP]
+    joined.index = joined.index.str.replace("^models_", "", regex=True)
+    # Fitler out all but important models
+    important_models_lower = [model.lower() for model in IMPORTANT_MODELS]
+    filtered_joined = joined[joined.index.str.lower().isin(important_models_lower)]
+    # Warn for ach missing important models
+    for model in IMPORTANT_MODELS:
+        if model not in filtered_joined.index:
+            print(f"[WARNING] Model {model} was missing from index.")            
+    return filtered_joined, latest_update_msg
+
+
+def get_sample_data() -> tuple[pd.DataFrame, str]:
+    # Retrieve sample dataframes
+    df_amd, _ = read_one_dataframe("sample_amd.json", "amd")
+    df_nvidia, _ = read_one_dataframe("sample_nvidia.json", "nvidia")
+    # Join both dataframes
+    joined = df_amd.join(df_nvidia, rsuffix="_nvidia", lsuffix="_amd", how="outer")
+    joined = joined[KEYS_TO_KEEP]
+    joined.index = joined.index.str.replace("^models_", "", regex=True)
+    # Fitler out all but important models
+    important_models_lower = [model.lower() for model in IMPORTANT_MODELS]
+    filtered_joined = joined[joined.index.str.lower().isin(important_models_lower)]
+    # Prefix all model names with "sample_"
+    filtered_joined.index = "sample_" + filtered_joined.index
+    return filtered_joined, "sample data was loaded"
+
+def safe_extract(row: pd.DataFrame, key: str) -> int:
+    return int(row.get(key, 0)) if pd.notna(row.get(key, 0)) else 0
+
+def extract_model_data(row: pd.Series) -> tuple[dict[str, int], dict[str, int], int, int, int, int]:
+    """Extract and process model data from DataFrame row."""
+    # Handle missing values and get counts directly from dataframe
+    success_nvidia = safe_extract(row, "success_nvidia")
+    success_amd = safe_extract(row, "success_amd")
+
+    skipped_nvidia = safe_extract(row, "skipped_nvidia")
+    skipped_amd = safe_extract(row, "skipped_amd")
+    
+    failed_multi_amd = safe_extract(row, 'failed_multi_no_amd')
+    failed_multi_nvidia = safe_extract(row, 'failed_multi_no_nvidia')
+    failed_single_amd = safe_extract(row, 'failed_single_no_amd')
+    failed_single_nvidia = safe_extract(row, 'failed_single_no_nvidia')
+    # Calculate total failures
+    total_failed_amd = failed_multi_amd + failed_single_amd
+    total_failed_nvidia = failed_multi_nvidia + failed_single_nvidia
+    # Create stats dictionaries directly from dataframe values
+    amd_stats = {
+        'passed': success_amd,
+        'failed': total_failed_amd,
+        'skipped': skipped_amd,
+        'error': 0     # Not available in this dataset
+    }
+    nvidia_stats = {
+        'passed': success_nvidia,
+        'failed': total_failed_nvidia,
+        'skipped': skipped_nvidia,
+        'error': 0     # Not available in this dataset
+    }
+    return amd_stats, nvidia_stats, failed_multi_amd, failed_single_amd, failed_multi_nvidia, failed_single_nvidia
+
+
+
+class CIResults:
+
+    def __init__(self):
+        self.df = pd.DataFrame()
+        self.available_models = []
+        self.latest_update_msg = ""
+
+    def load_data(self) -> None:
+        """Load data from the data source."""
+        # Try loading the distant data, and fall back on sample data for local tinkering
+        try:
+            logger.info("Loading distant data...")
+            new_df, latest_update_msg = get_distant_data()
+            self.latest_update_msg = latest_update_msg
+        except Exception as e:
+            error_msg = [
+                "Loading data failed:",
+                "-" * 120,
+                traceback.format_exc(),
+                "-" * 120,
+                "Falling back on sample data."
+            ]
+            logger.error("\n".join(error_msg))
+            new_df, latest_update_msg = get_sample_data()
+            self.latest_update_msg = latest_update_msg
+        # Update attributes
+        self.df = new_df
+        self.available_models = new_df.index.tolist()
+        # Log and return distant load status
+        logger.info(f"Data loaded successfully: {len(self.available_models)} models")
+        logger.info(f"Models: {self.available_models[:5]}{'...' if len(self.available_models) > 5 else ''}")
+        logger.info(f"Latest update message: {self.latest_update_msg}")
+        # Log a preview of the df
+        msg = {}
+        for model in self.available_models[:3]:
+            msg[model] = {}
+            for col in self.df.columns:
+                value = self.df.loc[model, col]
+                if not isinstance(value, int):
+                    value = str(value)
+                    if len(value) > 10:
+                        value = value[:10] + "..."
+                msg[model][col] = value
+        logger.info(json.dumps(msg, indent=4))
+
+    def schedule_data_reload(self):
+        """Schedule the next data reload."""
+        def reload_data():
+            self.load_data()
+            # Schedule the next reload in 15 minutes (900 seconds)
+            timer = threading.Timer(900.0, reload_data)
+            timer.daemon = True  # Dies when main thread dies
+            timer.start()
+            logger.info("Next data reload scheduled in 15 minutes")
+
+        # Start the first reload timer
+        timer = threading.Timer(900.0, reload_data)
+        timer.daemon = True
+        timer.start()
+        logger.info("Data auto-reload scheduled every 15 minutes")

model_page.py

@@ -0,0 +1,180 @@
+import matplotlib.pyplot as plt
+import pandas as pd
+from utils import generate_underlined_line
+from data import extract_model_data
+
+# Figure dimensions
+FIGURE_WIDTH_DUAL = 18
+FIGURE_HEIGHT_DUAL = 9
+
+# Colors
+COLORS = {
+    'passed': '#4CAF50',    # Medium green
+    'failed': '#E53E3E',    # More red
+    'skipped': '#FFD54F',   # Medium yellow
+    'error': '#8B0000'      # Dark red
+}
+
+# Styling constants
+BLACK = '#000000'
+LABEL_COLOR = '#AAAAAA'
+TITLE_COLOR = '#FFFFFF'
+
+# Font sizes
+DEVICE_TITLE_FONT_SIZE = 28
+
+# Layout constants
+SEPARATOR_LINE_Y_END = 0.85
+SUBPLOT_TOP = 0.85
+SUBPLOT_WSPACE = 0.4
+PIE_START_ANGLE = 90
+BORDER_LINE_WIDTH = 0.5
+SEPARATOR_ALPHA = 0.5
+SEPARATOR_LINE_WIDTH = 1
+DEVICE_TITLE_PAD = 2
+MODEL_TITLE_Y = 1
+
+# Processing constants
+MAX_FAILURE_ITEMS = 10
+
+
+def _create_pie_chart(ax: plt.Axes, device_label: str, filtered_stats: dict) -> None:
+    """Create a pie chart for device statistics."""
+    if not filtered_stats:
+        ax.text(0.5, 0.5, 'No test results',
+               horizontalalignment='center', verticalalignment='center',
+               transform=ax.transAxes, fontsize=14, color='#888888',
+               fontfamily='monospace', weight='normal')
+        ax.set_title(device_label, fontsize=DEVICE_TITLE_FONT_SIZE, weight='bold',
+                    pad=DEVICE_TITLE_PAD, color=TITLE_COLOR, fontfamily='monospace')
+        ax.axis('off')
+        return
+
+    chart_colors = [COLORS[category] for category in filtered_stats.keys()]
+
+    # Create minimal pie chart - full pie, no donut effect
+    wedges, texts, autotexts = ax.pie(
+        filtered_stats.values(),
+        labels=[label.lower() for label in filtered_stats.keys()],  # Lowercase for minimal look
+        colors=chart_colors,
+        autopct=lambda pct: f'{round(pct * sum(filtered_stats.values()) / 100)}',
+        startangle=PIE_START_ANGLE,
+        explode=None,  # No separation
+        shadow=False,
+        wedgeprops=dict(edgecolor='#1a1a1a', linewidth=BORDER_LINE_WIDTH),  # Minimal borders
+        textprops={'fontsize': 12, 'weight': 'normal',
+                  'color': LABEL_COLOR, 'fontfamily': 'monospace'}
+    )
+
+    # Enhanced percentage text styling for better readability
+    for autotext in autotexts:
+        autotext.set_color(BLACK)  # Black text for better contrast
+        autotext.set_weight('bold')
+        autotext.set_fontsize(14)
+        autotext.set_fontfamily('monospace')
+
+    # Minimal category labels
+    for text in texts:
+        text.set_color(LABEL_COLOR)
+        text.set_weight('normal')
+        text.set_fontsize(13)
+        text.set_fontfamily('monospace')
+
+    # Device label closer to chart and bigger
+    ax.set_title(device_label, fontsize=DEVICE_TITLE_FONT_SIZE, weight='normal',
+                pad=DEVICE_TITLE_PAD, color=TITLE_COLOR, fontfamily='monospace')
+
+
+def plot_model_stats(df: pd.DataFrame, model_name: str) -> tuple[plt.Figure, str, str]:
+    """Draws pie charts of model's passed, failed, skipped, and error stats for AMD and NVIDIA."""
+    # Handle case where the dataframe is empty or the model name could not be found in it
+    if df.empty or model_name not in df.index:
+        # Create empty stats for both devices
+        amd_filtered = {}
+        nvidia_filtered = {}
+        failures_amd = failures_nvidia = {}
+    else:
+        row = df.loc[model_name]
+
+        # Extract and process model data
+        amd_stats, nvidia_stats = extract_model_data(row)[:2]
+
+        # Filter out categories with 0 values for cleaner visualization
+        amd_filtered = {k: v for k, v in amd_stats.items() if v > 0}
+        nvidia_filtered = {k: v for k, v in nvidia_stats.items() if v > 0}
+
+        # Generate failure info directly from dataframe
+        failures_amd = row.get('failures_amd', None)
+        failures_amd = {} if (failures_amd is None or pd.isna(failures_amd)) else dict(failures_amd)
+        failures_nvidia = row.get('failures_nvidia')
+        failures_nvidia = {} if (failures_nvidia is None or pd.isna(failures_nvidia)) else dict(failures_nvidia)
+
+    # failure_xxx = {"single": [test, ...], "multi": [...]}
+    # test = {"line": test_name. "trace": error_msg}
+
+    # Always create figure with two subplots side by side with padding
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(FIGURE_WIDTH_DUAL, FIGURE_HEIGHT_DUAL), facecolor=BLACK)
+    ax1.set_facecolor(BLACK)
+    ax2.set_facecolor(BLACK)
+
+    # Create both pie charts with device labels
+    _create_pie_chart(ax1, "amd", amd_filtered)
+    _create_pie_chart(ax2, "nvidia", nvidia_filtered)
+
+    # Add subtle separation line between charts - stops at device labels level
+    line_x = 0.5
+    fig.add_artist(plt.Line2D([line_x, line_x], [0.0, SEPARATOR_LINE_Y_END],
+                              color='#333333', linewidth=SEPARATOR_LINE_WIDTH,
+                              alpha=SEPARATOR_ALPHA, transform=fig.transFigure))
+
+    # Add central shared title for model name
+    fig.suptitle(f'{model_name.lower()}', fontsize=32, weight='bold',
+                color='#CCCCCC', fontfamily='monospace', y=MODEL_TITLE_Y)
+
+    # Clean layout with padding and space for central title
+    plt.tight_layout()
+    plt.subplots_adjust(top=SUBPLOT_TOP, wspace=SUBPLOT_WSPACE)
+
+    amd_failed_info = prepare_textbox_content(failures_amd, 'AMD', bool(amd_filtered))
+    nvidia_failed_info = prepare_textbox_content(failures_nvidia, 'NVIDIA', bool(nvidia_filtered))
+
+    return fig, amd_failed_info, nvidia_failed_info
+
+
+def prepare_textbox_content(failures: dict[str, list], device: str, data_available: bool) -> str:
+    """Extract failure information from failures object."""
+    # Catch the case where there is no data
+    if not data_available:
+        return generate_underlined_line(f"No data for {device}")
+    # Catch the case where there are no failures
+    if not failures:
+        return generate_underlined_line(f"No failures for {device}")
+
+    # Summary of failures
+    single_failures = failures.get("single", [])
+    multi_failures = failures.get("multi", [])
+    info_lines = [
+        generate_underlined_line(f"Failure summary for {device}:"),
+        f"Single GPU failures: {len(single_failures)}",
+        f"Multi GPU failures: {len(multi_failures)}",
+        ""
+    ]
+
+    # Add single-gpu failures
+    if single_failures:
+        info_lines.append(generate_underlined_line("Single GPU failures:"))
+        for test in single_failures:
+            name = test.get("line", "::*could not find name*")
+            name = name.split("::")[-1]
+            info_lines.append(name)
+        info_lines.append("\n")
+
+    # Add multi-gpu failures
+    if multi_failures:
+        info_lines.append(generate_underlined_line("Multi GPU failures:"))
+        for test in multi_failures:
+            name = test.get("line", "::*could not find name*")
+            name = name.split("::")[-1]
+            info_lines.append(name)
+
+    return "\n".join(info_lines)

requirements.txt

@@ -0,0 +1 @@
+matplotlib>=3.8

sample_amd.json

@@ -0,0 +1,1839 @@
+{
+    "models_auto": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 80,
+        "skipped": 2,
+        "time_spent": "0.99, 2.41, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329937",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330183"
+        }
+    },
+    "models_bert": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 239,
+        "skipped": 111,
+        "time_spent": "8.85, 0:01:00, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329946",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330199"
+        }
+    },
+    "models_clip": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 288,
+        "skipped": 590,
+        "time_spent": "0:01:55, 0:01:58, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330217",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329991"
+        }
+    },
+    "models_detr": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 77,
+        "skipped": 159,
+        "time_spent": "4.40, 6.77, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330035",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330267"
+        }
+    },
+    "models_gemma3": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 6,
+                "multi": 7
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 349,
+        "skipped": 260,
+        "time_spent": "0:11:14, 0:11:08, ",
+        "failures": {
+            "single": [
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_1b_text_only",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['Wri[57 chars]s, a silent stream,\\nInto the neural net, a wa[42 chars],\\n'] != ['Wri[57 chars]s, a river deep,\\nWith patterns hidden, secret[46 chars]ing']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_batch",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['use[114 chars]rown cow standing on a sandy beach with clear [264 chars]cow\"] != ['use[114 chars]rown and white cow standing on a sandy beach n[272 chars]ach']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_batch_crops",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"user\\nYou are a helpful assistant.\\n\\nHe[678 chars]h a'] != ['user\\nYou are a helpful assistant.\\n\\nHe[658 chars]h a']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_bf16",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['use[114 chars]rown cow standing on a sandy beach with clear [55 chars]ike'] != ['use[114 chars]rown and white cow standing on a sandy beach w[68 chars]oks']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_crops",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"use[251 chars]. There's a blue sky with some white clouds in the background\"] != [\"use[251 chars]. There's a bright blue sky with some white clouds in the\"]"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_multiimage",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"use[122 chars]n\\n**Main Features:**\\n\\n*   **Chinese Archway[19 chars]ent\"] != [\"use[122 chars]n\\n**Overall Scene:**\\n\\nIt looks like a stree[18 chars]nt,\"]"
+                }
+            ],
+            "multi": [
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3Vision2TextModelTest::test_model_parallelism",
+                    "trace": "(line 925)  RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_1b_text_only",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['Wri[57 chars]s, a silent stream,\\nInto the neural net, a wa[42 chars],\\n'] != ['Wri[57 chars]s, a river deep,\\nWith patterns hidden, secret[46 chars]ing']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_batch",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['use[114 chars]rown cow standing on a sandy beach with clear [264 chars]cow\"] != ['use[114 chars]rown and white cow standing on a sandy beach n[272 chars]ach']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_batch_crops",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"user\\nYou are a helpful assistant.\\n\\nHe[678 chars]h a'] != ['user\\nYou are a helpful assistant.\\n\\nHe[658 chars]h a']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_bf16",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['use[114 chars]rown cow standing on a sandy beach with clear [55 chars]ike'] != ['use[114 chars]rown and white cow standing on a sandy beach w[68 chars]oks']"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_crops",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"use[251 chars]. There's a blue sky with some white clouds in the background\"] != [\"use[251 chars]. There's a bright blue sky with some white clouds in the\"]"
+                },
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_multiimage",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"use[122 chars]n\\n**Main Features:**\\n\\n*   **Chinese Archway[19 chars]ent\"] != [\"use[122 chars]n\\n**Overall Scene:**\\n\\nIt looks like a stree[18 chars]nt,\"]"
+                }
+            ]
+        },
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330061",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330319"
+        }
+    },
+    "models_gemma3n": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 197,
+        "skipped": 635,
+        "time_spent": "0:01:06, 0:01:08, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330294",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330077"
+        }
+    },
+    "models_got_ocr2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 147,
+        "skipped": 163,
+        "time_spent": "0:01:03, 0:01:01, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330314",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330094"
+        }
+    },
+    "models_gpt2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 249,
+        "skipped": 99,
+        "time_spent": "0:02:01, 0:01:46, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330311",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330113"
+        }
+    },
+    "models_internvl": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 253,
+        "skipped": 107,
+        "time_spent": "0:01:50, 0:02:00, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/internvl/test_modeling_internvl.py::InternVLLlamaIntegrationTest::test_llama_small_model_integration_forward",
+                    "trace": "(line 727)  AssertionError: False is not true : Actual logits: tensor([ -9.8750,  -0.4885,   1.4668, -10.3359, -10.3359], dtype=torch.float16)"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/internvl/test_modeling_internvl.py::InternVLLlamaIntegrationTest::test_llama_small_model_integration_forward",
+                    "trace": "(line 727)  AssertionError: False is not true : Actual logits: tensor([ -9.8750,  -0.4885,   1.4668, -10.3359, -10.3359], dtype=torch.float16)"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330361",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330105"
+        }
+    },
+    "models_llama": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 235,
+        "skipped": 101,
+        "time_spent": "0:03:15, 0:02:51, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/llama/test_modeling_llama.py::LlamaIntegrationTest::test_model_7b_logits_bf16",
+                    "trace": "(line 727)  AssertionError: False is not true"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/llama/test_modeling_llama.py::LlamaIntegrationTest::test_model_7b_logits_bf16",
+                    "trace": "(line 727)  AssertionError: False is not true"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330531",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330138"
+        }
+    },
+    "models_llava": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 206,
+        "skipped": 124,
+        "time_spent": "0:03:58, 0:04:34, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationIntegrationTest::test_batched_generation",
+                    "trace": "(line 399)  importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationIntegrationTest::test_batched_generation",
+                    "trace": "(line 399)  importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330406",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330161"
+        }
+    },
+    "models_mistral3": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 199,
+        "skipped": 105,
+        "time_spent": "0:04:34, 0:04:39, ",
+        "failures": {
+            "single": [
+                {
+                    "line": "tests/models/mistral3/test_modeling_mistral3.py::Mistral3IntegrationTest::test_mistral3_integration_generate",
+                    "trace": "(line 715)  AssertionError: 'The [14 chars] two cats lying on a pink surface, which appea[21 chars] bed' != 'The [14 chars] two tabby cats lying on a pink surface, which[23 chars]n or'"
+                }
+            ],
+            "multi": [
+                {
+                    "line": "tests/models/mistral3/test_modeling_mistral3.py::Mistral3IntegrationTest::test_mistral3_integration_generate",
+                    "trace": "(line 715)  AssertionError: 'The [14 chars] two cats lying on a pink surface, which appea[21 chars] bed' != 'The [14 chars] two tabby cats lying on a pink surface, which[23 chars]n or'"
+                }
+            ]
+        },
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330418",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329678"
+        }
+    },
+    "models_modernbert": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 142,
+        "skipped": 102,
+        "time_spent": "0:01:03, 9.02, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329712",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330429"
+        }
+    },
+    "models_qwen2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 217,
+        "skipped": 113,
+        "time_spent": "0:01:08, 0:01:05, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/qwen2/test_modeling_qwen2.py::Qwen2IntegrationTest::test_export_static_cache",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['My [35 chars], organic, gluten free, vegan, and vegetarian. I love to use'] != ['My [35 chars], organic, gluten free, vegan, and free from preservatives. I']"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/qwen2/test_modeling_qwen2.py::Qwen2IntegrationTest::test_export_static_cache",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['My [35 chars], organic, gluten free, vegan, and vegetarian. I love to use'] != ['My [35 chars], organic, gluten free, vegan, and free from preservatives. I']"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329761",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330508"
+        }
+    },
+    "models_qwen2_5_omni": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 2,
+                "multi": 2
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 167,
+        "skipped": 141,
+        "time_spent": "0:02:23, 0:01:53, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/qwen2_5_omni/test_modeling_qwen2_5_omni.py::Qwen2_5OmniThinkerForConditionalGenerationModelTest::test_model_parallelism",
+                    "trace": "(line 715)  AssertionError: Items in the second set but not the first:"
+                },
+                {
+                    "line": "tests/models/qwen2_5_omni/test_modeling_qwen2_5_omni.py::Qwen2_5OmniModelIntegrationTest::test_small_model_integration_test_batch",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"sys[293 chars]s shattering, and the dog appears to be a Labrador Retriever.\"] != [\"sys[293 chars]s shattering, and the dog is a Labrador Retriever.\"]"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/qwen2_5_omni/test_modeling_qwen2_5_omni.py::Qwen2_5OmniModelIntegrationTest::test_small_model_integration_test",
+                    "trace": "(line 700)  requests.exceptions.ConnectionError: HTTPSConnectionPool(host='qianwen-res.oss-accelerate-overseas.aliyuncs.com', port=443): Max retries exceeded with url: /Qwen2-VL/demo_small.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7cb8c91d02f0>: Failed to establish a new connection: [Errno -2] Name or service not known'))"
+                },
+                {
+                    "line": "tests/models/qwen2_5_omni/test_modeling_qwen2_5_omni.py::Qwen2_5OmniModelIntegrationTest::test_small_model_integration_test_batch",
+                    "trace": "(line 715)  AssertionError: Lists differ: [\"sys[109 chars]d is a glass shattering, and the dog is a Labr[187 chars]er.\"] != [\"sys[109 chars]d is glass shattering, and the dog is a Labrad[185 chars]er.\"]"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329806",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330503"
+        }
+    },
+    "models_qwen2_5_vl": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 205,
+        "skipped": 113,
+        "time_spent": "0:02:32, 0:02:29, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch_different_resolutions",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['sys[314 chars]ion\\n addCriterion\\n\\n addCriterion\\n\\n addCri[75 chars]n\\n'] != ['sys[314 chars]ion\\nThe dog in the picture appears to be a La[81 chars] is']"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch_different_resolutions",
+                    "trace": "(line 715)  AssertionError: Lists differ: ['sys[314 chars]ion\\n addCriterion\\n\\n addCriterion\\n\\n addCri[75 chars]n\\n'] != ['sys[314 chars]ion\\nThe dog in the picture appears to be a La[81 chars] is']"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329760",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330498"
+        }
+    },
+    "models_smolvlm": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 323,
+        "skipped": 231,
+        "time_spent": "0:01:08, 0:01:13, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330553",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329835"
+        }
+    },
+    "models_t5": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 2,
+                "multi": 3
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 254,
+        "skipped": 325,
+        "time_spent": "0:01:50, 0:01:40, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelTest::test_multi_gpu_data_parallel_forward",
+                    "trace": "(line 131)  TypeError: EncoderDecoderCache.__init__() missing 1 required positional argument: 'cross_attention_cache'"
+                },
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_export_t5_summarization",
+                    "trace": "(line 687)  AttributeError: 'dict' object has no attribute 'batch_size'"
+                },
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_small_integration_test",
+                    "trace": "(line 727)  AssertionError: False is not true"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_export_t5_summarization",
+                    "trace": "(line 687)  AttributeError: 'dict' object has no attribute 'batch_size'"
+                },
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_small_integration_test",
+                    "trace": "(line 727)  AssertionError: False is not true"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329815",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330559"
+        }
+    },
+    "models_vit": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 135,
+        "skipped": 93,
+        "time_spent": "9.85, 7.74, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329875",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330596"
+        }
+    },
+    "models_wav2vec2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 292,
+        "skipped": 246,
+        "time_spent": "0:01:56, 0:01:54, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329877",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330632"
+        }
+    },
+    "models_whisper": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 40,
+                "multi": 42
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 537,
+        "skipped": 337,
+        "time_spent": "0:03:23, 0:03:02, ",
+        "failures": {
+            "single": [
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_distil_token_timestamp_generation",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_forced_decoder_ids",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_prompt_ids",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_prompt_ids_task_language",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_language_detection",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation_multilingual",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation_multilingual",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_en_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_longform_timestamps_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_token_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_speculative_decoding_distil",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_speculative_decoding_non_distil",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_batched_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_longform_timestamps_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_specaugment_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_static_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_static_generation_long_form",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_batch_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_generation_longform",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_empty_longform",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_no_speech_detection",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_prompt_ids",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch_beam",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_multi_batch_hard_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_single_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                }
+            ],
+            "multi": [
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelTest::test_multi_gpu_data_parallel_forward",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_distil_token_timestamp_generation",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_forced_decoder_ids",
+                    "trace": "(line 2938)  Failed: (subprocess)"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_prompt_ids",
+                    "trace": "(line 131)  TypeError: EncoderDecoderCache.__init__() missing 1 required positional argument: 'cross_attention_cache'"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_generate_with_prompt_ids_task_language",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_language_detection",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation_multilingual",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_generation_multilingual",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_en_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_longform_timestamps_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_token_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_speculative_decoding_distil",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_speculative_decoding_non_distil",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_batched_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_en_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_logits_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_longform_timestamps_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_specaugment_librispeech",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_static_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_static_generation_long_form",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_batch_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_generation",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_token_timestamp_generation_longform",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_empty_longform",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_empty_longform_multi_gpu",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_no_speech_detection",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_prompt_ids",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch_beam",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_single_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_multi_batch_hard_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_single_batch_prev_cond",
+                    "trace": "(line 172)  ImportError: To support decoding audio data, please install 'torchcodec'."
+                }
+            ]
+        },
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301330636",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712966867/job/47301329883"
+        }
+    }
+}

sample_nvidia.json

@@ -0,0 +1,1475 @@
+{
+    "models_auto": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 226,
+        "skipped": 10,
+        "time_spent": "3.79, 5.93, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215208",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215147"
+        }
+    },
+    "models_bert": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 527,
+        "skipped": 211,
+        "time_spent": "0:01:47, 0:01:50, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215196",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215175"
+        }
+    },
+    "models_clip": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 660,
+        "skipped": 934,
+        "time_spent": "0:02:15, 0:02:11, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215674",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215699"
+        }
+    },
+    "models_detr": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 177,
+        "skipped": 271,
+        "time_spent": "0:01:07, 0:01:11, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216030",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216008"
+        }
+    },
+    "models_gemma3": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 507,
+        "skipped": 320,
+        "time_spent": "0:09:30, 0:09:28, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/gemma3/test_modeling_gemma3.py::Gemma3Vision2TextModelTest::test_model_parallelism",
+                    "trace": "(line 925)  RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!"
+                }
+            ]
+        },
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216642",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216593"
+        }
+    },
+    "models_gemma3n": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 288,
+        "skipped": 703,
+        "time_spent": "0:02:15, 0:02:15, ",
+        "failures": {
+            "single": [
+                {
+                    "line": "tests/models/gemma3n/test_modeling_gemma3n.py::Gemma3nTextModelTest::test_sdpa_padding_matches_padding_free_with_position_ids",
+                    "trace": "(line 4243)  AssertionError: Tensor-likes are not close!"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216605",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216660"
+        }
+    },
+    "models_got_ocr2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 257,
+        "skipped": 333,
+        "time_spent": "0:01:49, 0:01:49, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216911",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216742"
+        }
+    },
+    "models_gpt2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 487,
+        "skipped": 229,
+        "time_spent": "0:02:11, 0:02:01, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216717",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216759"
+        }
+    },
+    "models_internvl": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 355,
+        "skipped": 241,
+        "time_spent": "0:04:33, 0:04:31, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_flex_attention_with_grads",
+                    "trace": "(line 439)  torch._inductor.exc.InductorError: RuntimeError: No valid triton configs. OutOfResources: out of resource: shared memory, Required: 106496, Hardware limit: 101376. Reducing block sizes or `num_stages` may help."
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_flex_attention_with_grads",
+                    "trace": "(line 439)  torch._inductor.exc.InductorError: RuntimeError: No valid triton configs. OutOfResources: out of resource: shared memory, Required: 106496, Hardware limit: 101376. Reducing block sizes or `num_stages` may help."
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217017",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217056"
+        }
+    },
+    "models_llama": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 481,
+        "skipped": 253,
+        "time_spent": "0:03:43, 0:03:37, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217239",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217242"
+        }
+    },
+    "models_llava": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 349,
+        "skipped": 159,
+        "time_spent": "0:08:59, 0:09:11, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217250",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217263"
+        }
+    },
+    "models_mistral3": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 283,
+        "skipped": 267,
+        "time_spent": "0:09:53, 0:09:40, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215108",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215124"
+        }
+    },
+    "models_modernbert": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 174,
+        "skipped": 218,
+        "time_spent": "0:01:27, 0:01:24, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215158",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215123"
+        }
+    },
+    "models_qwen2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 443,
+        "skipped": 251,
+        "time_spent": "0:02:16, 0:02:16, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215909",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215891"
+        }
+    },
+    "models_qwen2_5_omni": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 278,
+        "skipped": 159,
+        "time_spent": "0:02:55, 0:03:00, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/qwen2_5_omni/test_modeling_qwen2_5_omni.py::Qwen2_5OmniThinkerForConditionalGenerationModelTest::test_model_parallelism",
+                    "trace": "(line 675)  AssertionError: Items in the second set but not the first:"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215907",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215896"
+        }
+    },
+    "models_qwen2_5_vl": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 1
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 309,
+        "skipped": 141,
+        "time_spent": "0:03:13, 0:03:14, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch_different_resolutions",
+                    "trace": "(line 675)  AssertionError: Lists differ: ['sys[314 chars]ion\\n addCriterion\\n\\n addCriterion\\n\\n addCri[75 chars]n\\n'] != ['sys[314 chars]ion\\nThe dog in the picture appears to be a La[81 chars] is']"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py::Qwen2_5_VLIntegrationTest::test_small_model_integration_test_batch_different_resolutions",
+                    "trace": "(line 675)  AssertionError: Lists differ: ['sys[314 chars]ion\\n addCriterion\\n\\n addCriterion\\n\\n addCri[75 chars]n\\n'] != ['sys[314 chars]ion\\nThe dog in the picture appears to be a La[81 chars] is']"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215945",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301215911"
+        }
+    },
+    "models_smolvlm": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 497,
+        "skipped": 269,
+        "time_spent": "0:01:33, 0:01:36, ",
+        "failures": {},
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216282",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216321"
+        }
+    },
+    "models_t5": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 1,
+                "multi": 2
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 592,
+        "skipped": 535,
+        "time_spent": "0:03:13, 0:02:52, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelTest::test_multi_gpu_data_parallel_forward",
+                    "trace": "(line 131)  TypeError: EncoderDecoderCache.__init__() missing 1 required positional argument: 'cross_attention_cache'"
+                },
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_export_t5_summarization",
+                    "trace": "(line 687)  AttributeError: 'dict' object has no attribute 'batch_size'"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/t5/test_modeling_t5.py::T5ModelIntegrationTests::test_export_t5_summarization",
+                    "trace": "(line 687)  AttributeError: 'dict' object has no attribute 'batch_size'"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216565",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216464"
+        }
+    },
+    "models_vit": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 217,
+        "skipped": 199,
+        "time_spent": "2.03, 1.28, ",
+        "failures": {},
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216869",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216833"
+        }
+    },
+    "models_wav2vec2": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 4,
+                "multi": 4
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 672,
+        "skipped": 438,
+        "time_spent": "0:03:37, 0:03:36, ",
+        "failures": {
+            "multi": [
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_inference_mms_1b_all",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm_invalid_pool",
+                    "trace": "(line 675)  AssertionError: Traceback (most recent call last):"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm_pool",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                }
+            ],
+            "single": [
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_inference_mms_1b_all",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm_invalid_pool",
+                    "trace": "(line 675)  AssertionError: Traceback (most recent call last):"
+                },
+                {
+                    "line": "tests/models/wav2vec2/test_modeling_wav2vec2.py::Wav2Vec2ModelIntegrationTest::test_wav2vec2_with_lm_pool",
+                    "trace": "(line 989)  RuntimeError: Dataset scripts are no longer supported, but found common_voice_11_0.py"
+                }
+            ]
+        },
+        "job_link": {
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216956",
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216929"
+        }
+    },
+    "models_whisper": {
+        "failed": {
+            "PyTorch": {
+                "unclassified": 0,
+                "single": 5,
+                "multi": 6
+            },
+            "TensorFlow": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Flax": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Tokenizers": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Pipelines": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Trainer": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "ONNX": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Auto": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Quantization": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            },
+            "Unclassified": {
+                "unclassified": 0,
+                "single": 0,
+                "multi": 0
+            }
+        },
+        "errors": 0,
+        "success": 1014,
+        "skipped": 475,
+        "time_spent": "0:11:09, 0:11:47, ",
+        "failures": {
+            "single": [
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation_multilingual",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_longform_timestamps_generation",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_longform_timestamps_generation",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard",
+                    "trace": "(line 675)  AssertionError: Lists differ: [\" Fo[272 chars]ting of classics, Sicilian, nade door variatio[8147 chars]le!'] != [\" Fo[272 chars]ting a classic Sicilian, nade door variation o[8150 chars]le!']"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_single_batch_prev_cond",
+                    "trace": "(line 675)  AssertionError: Lists differ: [\" Fo[268 chars]ating, so soft, it would make JD power and her[196 chars]ke.\"] != [\" Fo[268 chars]ating so soft, it would make JD power and her [195 chars]ke.\"]"
+                }
+            ],
+            "multi": [
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelTest::test_multi_gpu_data_parallel_forward",
+                    "trace": "(line 131)  TypeError: EncoderDecoderCache.__init__() missing 1 required positional argument: 'cross_attention_cache'"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_large_batched_generation_multilingual",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_small_longform_timestamps_generation",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_tiny_longform_timestamps_generation",
+                    "trace": "(line 756)  RuntimeError: The frame has 0 channels, expected 1. If you are hitting this, it may be because you are using a buggy FFmpeg version. FFmpeg4 is known to fail here in some valid scenarios. Try to upgrade FFmpeg?"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_longform_multi_batch_hard",
+                    "trace": "(line 675)  AssertionError: Lists differ: [\" Fo[272 chars]ting of classics, Sicilian, nade door variatio[8147 chars]le!'] != [\" Fo[272 chars]ting a classic Sicilian, nade door variation o[8150 chars]le!']"
+                },
+                {
+                    "line": "tests/models/whisper/test_modeling_whisper.py::WhisperModelIntegrationTests::test_whisper_shortform_single_batch_prev_cond",
+                    "trace": "(line 675)  AssertionError: Lists differ: [\" Fo[268 chars]ating, so soft, it would make JD power and her[196 chars]ke.\"] != [\" Fo[268 chars]ating so soft, it would make JD power and her [195 chars]ke.\"]"
+                }
+            ]
+        },
+        "job_link": {
+            "single": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301216943",
+            "multi": "https://github.com/huggingface/transformers/actions/runs/16712955100/job/47301217012"
+        }
+    }
+}

styles.css

@@ -0,0 +1,669 @@
+/* Global dark theme with configurable bottom margin */
+:root {
+    --main-content-bottom-margin: 10px; /* Configurable bottom margin for main content */
+}
+
+.gradio-container {
+    background-color: #000000 !important;
+    color: white !important;
+    height: 100vh !important;
+    max-height: 100vh !important;
+    overflow: hidden !important;
+}
+
+/* Remove borders from all components */
+.gr-box, .gr-form, .gr-panel {
+    border: none !important;
+    background-color: #000000 !important;
+}
+
+/* Simplified sidebar styling */
+.sidebar {
+    background: linear-gradient(145deg, #111111, #1a1a1a) !important;
+    border: none !important;
+    padding: 15px !important;
+    margin: 0 !important;
+    height: 100vh !important;
+    position: fixed !important;
+    left: 0 !important;
+    top: 0 !important;
+    width: 300px !important;
+    box-sizing: border-box !important;
+    overflow-y: auto !important;
+    overflow-x: hidden !important;
+}
+
+/* Target the actual Gradio column containing sidebar */
+div[data-testid="column"]:has(.sidebar) {
+    height: 100vh !important;
+    overflow-y: auto !important;
+    overflow-x: hidden !important;
+}
+
+/* Individual sidebar elements */
+.sidebar-title {
+    margin-bottom: 10px !important;
+}
+
+.sidebar-description {
+    margin-bottom: 15px !important;
+}
+
+/* Summary button styling - distinct from model buttons */
+.summary-button {
+    background: linear-gradient(135deg, #4a4a4a, #3e3e3e) !important;
+    color: white !important;
+    border: 2px solid #555555 !important;
+    margin: 0 0 15px 0 !important;
+    border-radius: 5px !important;
+    padding: 12px 10px !important;
+    transition: all 0.4s cubic-bezier(0.4, 0, 0.2, 1) !important;
+    position: relative !important;
+    overflow: hidden !important;
+    box-shadow:
+        0 4px 15px rgba(0, 0, 0, 0.3),
+        inset 0 1px 0 rgba(255, 255, 255, 0.2) !important;
+    font-weight: 600 !important;
+    font-size: 14px !important;
+    text-transform: uppercase !important;
+    letter-spacing: 0.3px !important;
+    font-family: monospace !important;
+    height: 60px !important;
+    display: flex !important;
+    flex-direction: column !important;
+    justify-content: center !important;
+    align-items: center !important;
+    line-height: 1.2 !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    min-width: 0 !important;
+    box-sizing: border-box !important;
+}
+
+.model-header {
+    margin-bottom: 10px !important;
+    background: linear-gradient(135deg, #2a2a2a, #1e1e1e) !important;
+    color: white !important;
+    border: 1px solid #333 !important;
+    border-radius: 5px !important;
+    font-weight: 600 !important;
+    font-size: 14px !important;
+    font-family: monospace !important;
+    text-align: left !important;
+    width: 100% !important;
+}
+
+.model-header:hover {
+    background: linear-gradient(135deg, #3a3a3a, #2e2e2e) !important;
+}
+
+.sidebar-links {
+    margin-top: 15px !important;
+}
+
+/* Hide scrollbar for model container */
+.model-container::-webkit-scrollbar {
+    display: none !important;
+}
+
+/* Ensure all sidebar content fits within width */
+.sidebar * {
+    max-width: 100% !important;
+    word-wrap: break-word !important;
+    overflow-wrap: break-word !important;
+}
+
+/* Specific control for markdown content */
+.sidebar .markdown,
+.sidebar h1,
+.sidebar h2,
+.sidebar h3,
+.sidebar p {
+    max-width: 100% !important;
+    word-wrap: break-word !important;
+    overflow: hidden !important;
+}
+
+/* Sidebar scrollbar styling */
+.sidebar::-webkit-scrollbar {
+    width: 8px !important;
+    background: #111111 !important;
+}
+
+.sidebar::-webkit-scrollbar-track {
+    background: #111111 !important;
+}
+
+.sidebar::-webkit-scrollbar-thumb {
+    background-color: #333333 !important;
+    border-radius: 4px !important;
+}
+
+.sidebar::-webkit-scrollbar-thumb:hover {
+    background-color: #555555 !important;
+}
+
+/* Force button containers to single column in model list */
+.model-list .gr-button,
+.model-list button {
+    display: block !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    margin: 4px 0 !important;
+    flex: none !important;
+}
+
+/* Simple unfolding menu with invisible scrollbar */
+.model-list-visible {
+    max-height: 200px !important;
+    overflow-y: auto !important;
+    transition: max-height 0.3s ease !important;
+    scrollbar-width: none !important;
+    -ms-overflow-style: none !important;
+}
+
+.model-list-visible::-webkit-scrollbar {
+    width: 0px !important;
+    background: transparent !important;
+}
+
+.model-list-hidden {
+    max-height: 0 !important;
+    overflow: hidden !important;
+    transition: max-height 0.3s ease !important;
+}
+
+
+/* Model button styling */
+.model-button {
+    background: linear-gradient(135deg, #2a2a2a, #1e1e1e) !important;
+    color: white !important;
+    margin: 3px 0 !important;
+    padding: 8px 12px !important;
+    font-weight: 600 !important;
+    font-size: 14px !important;
+    text-transform: uppercase !important;
+    letter-spacing: 0.3px !important;
+    font-family: monospace !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    white-space: nowrap !important;
+    text-overflow: ellipsis !important;
+    display: block !important;
+    cursor: pointer !important;
+    transition: all 0.3s ease !important;
+    border: 1px solid #333 !important;
+    border-radius: 5px !important;
+}
+
+.model-button:hover {
+    background: linear-gradient(135deg, #3a3a3a, #2e2e2e) !important;
+    border-color: #74b9ff !important;
+    color: #74b9ff !important;
+    transform: translateY(-1px) !important;
+    box-shadow: 0 2px 8px rgba(116, 185, 255, 0.2) !important;
+}
+
+/* Model buttons with failures - fuzzy red border with inner glow */
+.model-button-failed {
+    border: 1px solid #712626 !important;
+    box-shadow: inset 0 0 8px rgba(204, 68, 68, 0.4) !important;
+}
+
+.model-button-failed:hover {
+    border-color: #712626 !important;
+    box-shadow: 0 0 12px rgba(255, 107, 107, 0.5) !important;
+}
+
+/*
+.model-button:active {
+    background: linear-gradient(135deg, #2a2a2a, #1e1e1e) !important;
+    color: #5a9bd4 !important;
+}
+*/
+
+/* Model stats badge */
+.model-stats {
+    display: flex !important;
+    justify-content: space-between !important;
+    align-items: center !important;
+    margin-top: 8px !important;
+    font-size: 12px !important;
+    opacity: 0.8 !important;
+}
+
+.stats-badge {
+    background: rgba(116, 185, 255, 0.2) !important;
+    padding: 4px 8px !important;
+    border-radius: 10px !important;
+    font-weight: 500 !important;
+    font-size: 11px !important;
+    color: #74b9ff !important;
+}
+
+.success-indicator {
+    width: 8px !important;
+    height: 8px !important;
+    border-radius: 50% !important;
+    display: inline-block !important;
+    margin-right: 6px !important;
+}
+
+.success-high { background-color: #4CAF50 !important; }
+.success-medium { background-color: #FF9800 !important; }
+.success-low { background-color: #F44336 !important; }
+
+/* Refresh button styling */
+.refresh-button {
+    background: linear-gradient(135deg, #2d5aa0, #1e3f73) !important;
+    color: white !important;
+    border: 1px solid #3a6bc7 !important;
+    margin: 0 0 10px 0 !important;
+    border-radius: 5px !important;
+    padding: 6px 8px !important;
+    transition: all 0.3s ease !important;
+    font-weight: 500 !important;
+    font-size: 11px !important;
+    text-transform: lowercase !important;
+    letter-spacing: 0.1px !important;
+    font-family: monospace !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    min-width: 0 !important;
+    box-sizing: border-box !important;
+    white-space: nowrap !important;
+    overflow: hidden !important;
+    text-overflow: ellipsis !important;
+}
+
+.refresh-button:hover {
+    background: linear-gradient(135deg, #3a6bc7, #2d5aa0) !important;
+    border-color: #4a7bd9 !important;
+}
+
+/* Summary button styling - distinct from model buttons */
+.summary-button {
+    background: linear-gradient(135deg, #4a4a4a, #3e3e3e) !important;
+    color: white !important;
+    border: 2px solid #555555 !important;
+    margin: 0 0 15px 0 !important;
+    border-radius: 5px !important;
+    padding: 12px 10px !important;
+    transition: all 0.4s cubic-bezier(0.4, 0, 0.2, 1) !important;
+    position: relative !important;
+    overflow: hidden !important;
+    box-shadow:
+        0 4px 15px rgba(0, 0, 0, 0.3),
+        inset 0 1px 0 rgba(255, 255, 255, 0.2) !important;
+    font-weight: 600 !important;
+    font-size: 14px !important;
+    text-transform: uppercase !important;
+    letter-spacing: 0.3px !important;
+    font-family: monospace !important;
+    height: 60px !important;
+    display: flex !important;
+    flex-direction: column !important;
+    justify-content: center !important;
+    align-items: center !important;
+    line-height: 1.2 !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    min-width: 0 !important;
+    box-sizing: border-box !important;
+}
+
+/* Simplified Gradio layout control */
+.sidebar .gr-column,
+.sidebar .gradio-column {
+    width: 100% !important;
+}
+
+/* Simplified Gradio targeting */
+div[data-testid="column"]:has(.sidebar) {
+    width: 300px !important;
+    min-width: 300px !important;
+}
+
+/* Button container with fixed height - DISABLED */
+/*
+.button-container {
+    height: 50vh !important;
+    max-height: 50vh !important;
+    overflow-y: auto !important;
+    overflow-x: hidden !important;
+    scrollbar-width: thin !important;
+    scrollbar-color: #333333 #111111 !important;
+    width: 100% !important;
+    max-width: 100% !important;
+    box-sizing: border-box !important;
+    padding: 5px 0 !important;
+    margin-top: 10px !important;
+}
+*/
+
+/* Removed simple scroll CSS - was hiding buttons */
+
+.summary-button:hover {
+    background: linear-gradient(135deg, #5a5a5a, #4e4e4e) !important;
+    color: #74b9ff !important;
+    border-color: #666666 !important;
+}
+
+.summary-button:active {
+    background: linear-gradient(135deg, #4a4a4a, #3e3e3e) !important;
+    color: #5a9bd4 !important;
+}
+
+/* Regular button styling for non-model buttons */
+.gr-button:not(.model-button):not(.summary-button) {
+    background-color: #222222 !important;
+    color: white !important;
+    border: 1px solid #444444 !important;
+    margin: 5px 0 !important;
+    border-radius: 8px !important;
+    transition: all 0.3s ease !important;
+}
+
+.gr-button:not(.model-button):not(.summary-button):hover {
+    background-color: #333333 !important;
+    border-color: #666666 !important;
+}
+
+/* Plot container with smooth transitions and controlled scrolling */
+.plot-container {
+    background-color: #000000 !important;
+    border: none !important;
+    transition: opacity 0.6s ease-in-out !important;
+    flex: 1 1 auto !important;
+    min-height: 0 !important;
+    overflow-y: auto !important;
+    scrollbar-width: thin !important;
+    scrollbar-color: #333333 #000000 !important;
+}
+
+/* Custom scrollbar for plot container */
+.plot-container::-webkit-scrollbar {
+    width: 8px !important;
+    background: #000000 !important;
+}
+
+.plot-container::-webkit-scrollbar-track {
+    background: #000000 !important;
+}
+
+.plot-container::-webkit-scrollbar-thumb {
+    background-color: #333333 !important;
+    border-radius: 4px !important;
+}
+
+.plot-container::-webkit-scrollbar-thumb:hover {
+    background-color: #555555 !important;
+}
+
+/* Gradio plot component styling */
+.gr-plot {
+    background-color: #000000 !important;
+    transition: opacity 0.6s ease-in-out !important;
+}
+
+.gr-plot .gradio-plot {
+    background-color: #000000 !important;
+    transition: opacity 0.6s ease-in-out !important;
+}
+
+.gr-plot img {
+    transition: opacity 0.6s ease-in-out !important;
+}
+
+/* Target the plot wrapper */
+div[data-testid="plot"] {
+    background-color: #000000 !important;
+}
+
+/* Target all possible plot containers */
+.plot-container img,
+.gr-plot img,
+.gradio-plot img {
+    background-color: #000000 !important;
+}
+
+/* Ensure plot area background */
+.gr-plot > div,
+.plot-container > div {
+    background-color: #000000 !important;
+}
+
+/* Prevent white flash during plot updates */
+.plot-container::before {
+    content: "";
+    position: absolute;
+    top: 0;
+    left: 0;
+    right: 0;
+    bottom: 0;
+    background-color: #000000;
+    z-index: -1;
+}
+
+/* Force all plot elements to have black background */
+.plot-container *,
+.gr-plot *,
+div[data-testid="plot"] * {
+    background-color: #000000 !important;
+}
+
+/* Override any white backgrounds in matplotlib */
+.plot-container canvas,
+.gr-plot canvas {
+    background-color: #000000 !important;
+}
+
+/* Text elements */
+h1, h2, h3, p, .markdown {
+    color: white !important;
+}
+
+/* Sidebar header enhancement */
+.sidebar h1 {
+    background: linear-gradient(45deg, #74b9ff, #a29bfe) !important;
+    -webkit-background-clip: text !important;
+    -webkit-text-fill-color: transparent !important;
+    background-clip: text !important;
+    text-align: center !important;
+    margin-bottom: 15px !important;
+    font-size: 28px !important;
+    font-weight: 700 !important;
+    font-family: monospace !important;
+}
+
+/* Sidebar description text */
+.sidebar p {
+    text-align: center !important;
+    margin-bottom: 20px !important;
+    line-height: 1.5 !important;
+    font-size: 14px !important;
+    font-family: monospace !important;
+}
+
+/* CI Links styling */
+.sidebar a {
+    color: #74b9ff !important;
+    text-decoration: none !important;
+    font-weight: 500 !important;
+    font-family: monospace !important;
+    transition: color 0.3s ease !important;
+}
+
+.sidebar a:hover {
+    color: #a29bfe !important;
+    text-decoration: underline !important;
+}
+
+.sidebar strong {
+    color: #74b9ff !important;
+    font-weight: 600 !important;
+    font-family: monospace !important;
+}
+
+.sidebar em {
+    color: #a29bfe !important;
+    font-style: normal !important;
+    opacity: 0.9 !important;
+    font-family: monospace !important;
+}
+
+/* Remove all borders globally */
+* {
+    border-color: transparent !important;
+}
+
+/* Main content area */
+.main-content {
+    background-color: #000000 !important;
+    padding: 0px 20px var(--main-content-bottom-margin, 10px) 20px !important;
+    margin-left: 300px !important;
+    height: 100vh !important;
+    overflow-y: auto !important;
+    box-sizing: border-box !important;
+    display: flex !important;
+    flex-direction: column !important;
+}
+
+/* Custom scrollbar for main content */
+.main-content {
+    scrollbar-width: thin !important;
+    scrollbar-color: #333333 #000000 !important;
+}
+
+.main-content::-webkit-scrollbar {
+    width: 8px !important;
+    background: #000000 !important;
+}
+
+.main-content::-webkit-scrollbar-track {
+    background: #000000 !important;
+}
+
+.main-content::-webkit-scrollbar-thumb {
+    background-color: #333333 !important;
+    border-radius: 4px !important;
+}
+
+.main-content::-webkit-scrollbar-thumb:hover {
+    background-color: #555555 !important;
+}
+
+/* Failed tests display - seamless appearance with constrained height */
+.failed-tests textarea {
+    background-color: #000000 !important;
+    color: #FFFFFF !important;
+    font-family: monospace !important;
+    font-size: 14px !important;
+    border: none !important;
+    padding: 10px !important;
+    outline: none !important;
+    line-height: 1.4 !important;
+    height: 180px !important;
+    max-height: 180px !important;
+    min-height: 180px !important;
+    overflow-y: auto !important;
+    resize: none !important;
+    scrollbar-width: thin !important;
+    scrollbar-color: #333333 #000000 !important;
+    scroll-behavior: auto !important;
+    transition: opacity 0.5s ease-in-out !important;
+    scroll-padding-top: 0 !important;
+}
+
+/* WebKit scrollbar styling for failed tests */
+.failed-tests textarea::-webkit-scrollbar {
+    width: 8px !important;
+}
+
+.failed-tests textarea::-webkit-scrollbar-track {
+    background: #000000 !important;
+}
+
+.failed-tests textarea::-webkit-scrollbar-thumb {
+    background-color: #333333 !important;
+    border-radius: 4px !important;
+}
+
+.failed-tests textarea::-webkit-scrollbar-thumb:hover {
+    background-color: #555555 !important;
+}
+
+/* Prevent white flash in text boxes during updates */
+.failed-tests::before {
+    content: "";
+    position: absolute;
+    top: 0;
+    left: 0;
+    right: 0;
+    bottom: 0;
+    background-color: #000000;
+    z-index: -1;
+}
+
+.failed-tests {
+    background-color: #000000 !important;
+    height: 200px !important;
+    max-height: 200px !important;
+    min-height: 200px !important;
+    position: relative;
+    transition: opacity 0.5s ease-in-out !important;
+    flex-shrink: 0 !important;
+}
+
+.failed-tests .gr-textbox {
+    background-color: #000000 !important;
+    border: none !important;
+    height: 180px !important;
+    max-height: 180px !important;
+    min-height: 180px !important;
+    transition: opacity 0.5s ease-in-out !important;
+}
+
+/* Force all textbox elements to have black background */
+.failed-tests *,
+.failed-tests .gr-textbox *,
+.failed-tests textarea * {
+    background-color: #000000 !important;
+}
+
+/* Summary display styling */
+.summary-display textarea {
+    background-color: #000000 !important;
+    color: #FFFFFF !important;
+    font-family: monospace !important;
+    font-size: 24px !important;
+    border: none !important;
+    padding: 20px !important;
+    outline: none !important;
+    line-height: 2 !important;
+    text-align: right !important;
+    resize: none !important;
+}
+
+.summary-display {
+    background-color: #000000 !important;
+}
+
+/* Detail view layout */
+.detail-view {
+    display: flex !important;
+    flex-direction: column !important;
+    height: 100% !important;
+    min-height: 0 !important;
+}
+
+/* JavaScript to reset scroll position */
+.scroll-reset {
+    animation: resetScroll 0.1s ease;
+}
+
+@keyframes resetScroll {
+    0% { scroll-behavior: auto; }
+    100% { scroll-behavior: auto; }
+}
+

summary_page.py

@@ -0,0 +1,231 @@
+import matplotlib.pyplot as plt
+import pandas as pd
+from data import extract_model_data
+
+# Layout parameters
+COLUMNS = 3
+
+# Derived constants
+COLUMN_WIDTH = 100 / COLUMNS  # Each column takes 25% of width
+BAR_WIDTH = COLUMN_WIDTH * 0.8  # 80% of column width for bars
+BAR_MARGIN = COLUMN_WIDTH * 0.1  # 10% margin on each side
+
+# Figure dimensions
+FIGURE_WIDTH = 22  # Wider to accommodate columns and legend
+MAX_HEIGHT = 14  # Maximum height in inches
+MIN_HEIGHT_PER_ROW = 2.8
+FIGURE_PADDING = 1
+
+# Bar styling
+BAR_HEIGHT_RATIO = 0.22  # Bar height as ratio of vertical spacing
+VERTICAL_SPACING_RATIO = 0.2  # Base vertical position ratio
+AMD_BAR_OFFSET = 0.25  # AMD bar offset ratio
+NVIDIA_BAR_OFFSET = 0.54  # NVIDIA bar offset ratio
+
+# Colors
+COLORS = {
+    'passed': '#4CAF50',
+    'failed': '#E53E3E',
+    'skipped': '#FFD54F',
+    'error': '#8B0000',
+    'empty': "#5B5B5B"
+}
+
+# Font styling
+MODEL_NAME_FONT_SIZE = 16
+LABEL_FONT_SIZE = 14
+LABEL_OFFSET = 1  # Distance of label from bar
+FAILURE_RATE_FONT_SIZE = 28
+
+
+def calculate_overall_failure_rates(df: pd.DataFrame, available_models: list[str]) -> tuple[float, float]:
+    """Calculate overall failure rates for AMD and NVIDIA across all models."""
+    if df.empty or not available_models:
+        return 0.0, 0.0
+    
+    total_amd_tests = 0
+    total_amd_failures = 0
+    total_nvidia_tests = 0
+    total_nvidia_failures = 0
+    
+    for model_name in available_models:
+        if model_name not in df.index:
+            continue
+            
+        row = df.loc[model_name]
+        amd_stats, nvidia_stats = extract_model_data(row)[:2]
+        
+        # AMD totals
+        amd_total = amd_stats['passed'] + amd_stats['failed'] + amd_stats['error']
+        if amd_total > 0:
+            total_amd_tests += amd_total
+            total_amd_failures += amd_stats['failed'] + amd_stats['error']
+        
+        # NVIDIA totals
+        nvidia_total = nvidia_stats['passed'] + nvidia_stats['failed'] + nvidia_stats['error']
+        if nvidia_total > 0:
+            total_nvidia_tests += nvidia_total
+            total_nvidia_failures += nvidia_stats['failed'] + nvidia_stats['error']
+    
+    amd_failure_rate = (total_amd_failures / total_amd_tests * 100) if total_amd_tests > 0 else 0.0
+    nvidia_failure_rate = (total_nvidia_failures / total_nvidia_tests * 100) if total_nvidia_tests > 0 else 0.0
+    
+    return amd_failure_rate, nvidia_failure_rate
+
+
+def draw_text_and_bar(
+    label: str,
+    stats: dict[str, int],
+    y_bar: float,
+    column_left_position: float,
+    bar_height: float,
+    ax: plt.Axes,
+) -> None:
+    """Draw a horizontal bar chart for given stats and its label on the left."""
+    # Text
+    label_x = column_left_position - LABEL_OFFSET
+    failures_present = any(stats[category] > 0 for category in ['failed', 'error'])
+    if failures_present:
+        props = dict(boxstyle='round', facecolor=COLORS['failed'], alpha=0.35)
+    else:
+        props = dict(alpha=0)
+    ax.text(
+        label_x, y_bar, label, ha='right', va='center', color='#CCCCCC', fontsize=LABEL_FONT_SIZE,
+        fontfamily='monospace', fontweight='normal', bbox=props
+    )
+    # Bar
+    total = sum(stats.values())
+    if total > 0:
+        left = column_left_position
+        for category in ['passed', 'failed', 'skipped', 'error']:
+            if stats[category] > 0:
+                width = stats[category] / total * BAR_WIDTH
+                ax.barh(y_bar, width, left=left, height=bar_height, color=COLORS[category], alpha=0.9)
+                left += width
+    else:
+        ax.barh(y_bar, BAR_WIDTH, left=column_left_position, height=bar_height, color=COLORS['empty'], alpha=0.9)
+
+def create_summary_page(df: pd.DataFrame, available_models: list[str]) -> plt.Figure:
+    """Create a summary page with model names and both AMD/NVIDIA test stats bars."""
+    if df.empty:
+        fig, ax = plt.subplots(figsize=(16, 8), facecolor='#000000')
+        ax.set_facecolor('#000000')
+        ax.text(0.5, 0.5, 'No data available',
+                horizontalalignment='center', verticalalignment='center',
+                transform=ax.transAxes, fontsize=20, color='#888888',
+                fontfamily='monospace', weight='normal')
+        ax.axis('off')
+        return fig
+    
+    # Calculate overall failure rates
+    amd_failure_rate, nvidia_failure_rate = calculate_overall_failure_rates(df, available_models)
+
+    # Calculate dimensions for N-column layout
+    model_count = len(available_models)
+    rows = (model_count + COLUMNS - 1) // COLUMNS  # Ceiling division
+
+    # Figure dimensions - wider for columns, height based on rows
+    height_per_row = min(MIN_HEIGHT_PER_ROW, MAX_HEIGHT / max(rows, 1))
+    figure_height = min(MAX_HEIGHT, rows * height_per_row + FIGURE_PADDING)
+
+    fig, ax = plt.subplots(figsize=(FIGURE_WIDTH, figure_height), facecolor='#000000')
+    ax.set_facecolor('#000000')
+    
+    # Add overall failure rates at the top as a proper title
+    failure_text = f"Overall Failure Rates: AMD {amd_failure_rate:.1f}%  |  NVIDIA {nvidia_failure_rate:.1f}%"
+    ax.text(50, -1.25, failure_text, ha='center', va='top', 
+           color='#FFFFFF', fontsize=FAILURE_RATE_FONT_SIZE,
+           fontfamily='monospace', fontweight='bold')
+
+    visible_model_count = 0
+    max_y = 0
+
+    for i, model_name in enumerate(available_models):
+        if model_name not in df.index:
+            continue
+
+        row = df.loc[model_name]
+
+        # Extract and process model data
+        amd_stats, nvidia_stats = extract_model_data(row)[:2]
+
+        # Calculate position in 4-column grid
+        col = visible_model_count % COLUMNS
+        row = visible_model_count // COLUMNS
+
+        # Calculate horizontal position for this column
+        col_left = col * COLUMN_WIDTH + BAR_MARGIN
+        col_center = col * COLUMN_WIDTH + COLUMN_WIDTH / 2
+
+        # Calculate vertical position for this row - start from top
+        vertical_spacing = height_per_row
+        y_base = (VERTICAL_SPACING_RATIO + row) * vertical_spacing
+        y_model_name = y_base    # Model name above AMD bar
+        y_amd_bar = y_base + vertical_spacing * AMD_BAR_OFFSET       # AMD bar
+        y_nvidia_bar = y_base + vertical_spacing * NVIDIA_BAR_OFFSET    # NVIDIA bar
+        max_y = max(max_y, y_nvidia_bar + vertical_spacing * 0.3)
+
+        # Model name centered above the bars in this column
+        ax.text(col_center, y_model_name, model_name.lower(),
+               ha='center', va='center', color='#FFFFFF',
+               fontsize=MODEL_NAME_FONT_SIZE, fontfamily='monospace', fontweight='bold')
+
+        # AMD label and bar in this column
+        bar_height = min(0.4, vertical_spacing * BAR_HEIGHT_RATIO)
+        # Draw AMD bar
+        draw_text_and_bar("amd", amd_stats, y_amd_bar, col_left, bar_height, ax)
+        # Draw NVIDIA bar
+        draw_text_and_bar("nvidia", nvidia_stats, y_nvidia_bar, col_left, bar_height, ax)
+
+        # Increment counter for next visible model
+        visible_model_count += 1
+
+
+    # Add legend horizontally in bottom right corner
+    patch_height = 0.3
+    patch_width = 3
+
+    legend_start_x = 68.7
+    legend_y = max_y + 1
+    legend_spacing = 10
+    legend_font_size = 15
+
+    # Add failure rate explanation text on the left
+    # explanation_text = "Failure rate = failed / (passed + failed)"
+    # ax.text(0, legend_y, explanation_text, 
+    #        ha='left', va='bottom', color='#CCCCCC', 
+    #        fontsize=legend_font_size, fontfamily='monospace', style='italic')
+    
+    # Legend entries
+    legend_items = [
+        ('passed', 'Passed'),
+        ('failed', 'Failed'), 
+        ('skipped', 'Skipped'),
+    ]
+    
+    for i, (status, label) in enumerate(legend_items):
+        x_pos = legend_start_x + i * legend_spacing
+        # Small colored square
+        ax.add_patch(plt.Rectangle((x_pos - 0.6, legend_y), patch_width, -patch_height,
+                                 facecolor=COLORS[status], alpha=0.9))
+        # Status label
+        ax.text(x_pos + patch_width, legend_y, label, 
+               ha='left', va='bottom', color='#CCCCCC', 
+               fontsize=legend_font_size, fontfamily='monospace')
+
+    # Style the axes to be completely invisible and span full width
+    ax.set_xlim(-5, 105)  # Slightly wider to accommodate labels
+    ax.set_ylim(0, max_y + 1)  # Add some padding at the top for title
+    ax.set_xlabel('')
+    ax.set_ylabel('')
+    ax.spines['bottom'].set_visible(False)
+    ax.spines['left'].set_visible(False)
+    ax.spines['top'].set_visible(False)
+    ax.spines['right'].set_visible(False)
+    ax.set_xticks([])
+    ax.set_yticks([])
+    ax.yaxis.set_inverted(True)
+
+    # Remove all margins to make figure stick to top
+    plt.tight_layout()
+    return fig

utils.py

@@ -0,0 +1,51 @@
+import logging
+import sys
+from datetime import datetime
+
+
+class TimestampFormatter(logging.Formatter):
+    """Custom formatter that matches the existing timestamp format used in print statements."""
+
+    def format(self, record):
+        # Create timestamp in the same format as existing print statements
+        timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
+
+        # Format the message with timestamp prefix
+        if record.levelno == logging.WARNING:
+            return f"WARNING: {record.getMessage()}"
+        elif record.levelno == logging.ERROR:
+            return f"Error {record.getMessage()}"
+        else:
+            return f"[{timestamp}] {record.getMessage()}"
+
+
+def setup_logger(name="tcid", level=logging.INFO):
+    """Set up logger with custom timestamp formatting to match existing print format."""
+    logger = logging.getLogger(name)
+
+    # Avoid adding multiple handlers if logger already exists
+    if logger.handlers:
+        return logger
+
+    logger.setLevel(level)
+
+    # Create console handler
+    handler = logging.StreamHandler(sys.stdout)
+    handler.setLevel(level)
+
+    # Set custom formatter
+    formatter = TimestampFormatter()
+    handler.setFormatter(formatter)
+
+    logger.addHandler(handler)
+
+    return logger
+
+
+# Create default logger instance
+logger = setup_logger()
+
+
+
+def generate_underlined_line(text: str) -> str:
+    return text + "\n" + "─" * len(text)