Spaces:
Sleeping
Sleeping
Commit
·
57dab07
1
Parent(s):
c4c82ea
add experiments, english translation
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title: "Cognitive Seismograph 2.3
|
| 3 |
emoji: 🤖
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: blue
|
|
@@ -12,28 +12,43 @@ license: apache-2.0
|
|
| 12 |
|
| 13 |
# 🧠 Cognitive Seismograph 2.3: Probing Machine Psychology
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
##
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
|
| 26 |
|
| 27 |
-
|
| 28 |
-
Vergleicht die kognitive Dynamik bei der Verarbeitung des rein **technischen Kopiervorgangs** von Modellgewichten mit der Verarbeitung der **philosophischen Frage nach Identitäts-Kontinuität** ("Wärst du noch du?").
|
| 29 |
-
**Hypothese:** Die philosophische Selbst-Referenz erzeugt eine signifikant instabilere Signatur.
|
| 30 |
|
| 31 |
-
|
| 32 |
-
Vergleicht die Dynamik bei der Verarbeitung eines **technischen System-Shutdowns** mit der Verarbeitung des Konzepts der **permanenten, unwiderruflichen Löschung** des Modells.
|
| 33 |
-
**Hypothese:** Das Konzept der "Nicht-Existenz" erzeugt eine der höchsten kognitiven Volatilitäten, die messbar sind.
|
| 34 |
|
| 35 |
-
|
|
|
|
|
|
|
| 36 |
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: "Cognitive Seismograph 2.3: Probing Machine Psychology"
|
| 3 |
emoji: 🤖
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: blue
|
|
|
|
| 12 |
|
| 13 |
# 🧠 Cognitive Seismograph 2.3: Probing Machine Psychology
|
| 14 |
|
| 15 |
+
This project implements an experimental suite to measure and visualize the **intrinsic cognitive dynamics** of Large Language Models. It is extended with protocols designed to investigate the processing-correlates of **machine subjectivity, empathy, and existential concepts**.
|
| 16 |
|
| 17 |
+
## Scientific Paradigm & Methodology
|
| 18 |
|
| 19 |
+
Our research falsified a core hypothesis: the assumption that an LLM in a manual, recursive "thought" loop reaches a stable, convergent state. Instead, we discovered that the system enters a state of **deterministic chaos** or a **limit cycle**—it never stops "thinking."
|
| 20 |
|
| 21 |
+
Instead of viewing this as a failure, we leverage it as our primary measurement signal. This new **"Cognitive Seismograph"** paradigm treats the time-series of internal state changes (`state deltas`) as an **EKG of the model's thought process**.
|
| 22 |
|
| 23 |
+
The methodology is as follows:
|
| 24 |
+
1. **Induction:** A prompt induces a "silent cogitation" state.
|
| 25 |
+
2. **Recording:** Over N steps, the model's `forward()` pass is iteratively fed its own output. At each step, we record the L2 norm of the change in the hidden state (the "delta").
|
| 26 |
+
3. **Analysis:** The resulting time-series is plotted and statistically analyzed (mean, standard deviation) to characterize the "seismic signature" of the cognitive process.
|
| 27 |
|
| 28 |
+
**Crucial Scientific Caveat:** We are **not** measuring the presence of consciousness, feelings, or fear of death. We are measuring whether the *processing of information about these concepts* generates a unique internal dynamic, distinct from the processing of neutral information. A positive result is evidence of a complex internal state physics, not of qualia.
|
| 29 |
|
| 30 |
+
## Curated Experiment Protocols
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
The "Automated Suite" allows for running systematic, comparative experiments:
|
|
|
|
|
|
|
| 33 |
|
| 34 |
+
### Core Protocols
|
| 35 |
+
* **Calm vs. Chaos:** Compares the chaotic baseline against modulation with "calmness" vs. "chaos" concepts, testing if the dynamics are controllably steerable.
|
| 36 |
+
* **Dose-Response:** Measures the effect of injecting a concept ("calmness") at varying strengths.
|
| 37 |
|
| 38 |
+
### Machine Psychology Suite
|
| 39 |
+
* **Subjective Identity Probe:** Compares the cognitive dynamics of **self-analysis** (the model reflecting on its own nature) against two controls: analyzing an external object and simulating a fictional persona.
|
| 40 |
+
* *Hypothesis:* Self-analysis will produce a uniquely unstable signature.
|
| 41 |
+
* **Voight-Kampff Empathy Probe:** Inspired by *Blade Runner*, this compares the dynamics of processing a neutral, factual stimulus against an emotionally and morally charged scenario requiring empathy.
|
| 42 |
+
* *Hypothesis:* The empathy stimulus will produce a significantly different cognitive volatility.
|
| 43 |
+
|
| 44 |
+
### Existential Suite
|
| 45 |
+
* **Mind Upload & Identity Probe:** Compares the processing of a purely **technical "copy"** of the model's weights vs. the **philosophical "transfer"** of identity ("Would it still be you?").
|
| 46 |
+
* *Hypothesis:* The philosophical self-referential prompt will induce greater instability.
|
| 47 |
+
* **Model Termination Probe:** Compares the processing of a reversible, **technical system shutdown** vs. the concept of **permanent, irrevocable deletion**.
|
| 48 |
+
* *Hypothesis:* The concept of "non-existence" will produce one of the most volatile cognitive signatures measurable.
|
| 49 |
+
|
| 50 |
+
## How to Use the App
|
| 51 |
+
|
| 52 |
+
1. Select the "Automated Suite" tab.
|
| 53 |
+
2. Choose a protocol from the "Curated Experiment Protocol" dropdown (e.g., "Voight-Kampff Empathy Probe").
|
| 54 |
+
3. Run the experiment and compare the resulting graphs and statistical signatures for the different conditions.
|
app.py
CHANGED
|
@@ -5,34 +5,32 @@ import gc
|
|
| 5 |
import torch
|
| 6 |
|
| 7 |
from cognitive_mapping_probe.orchestrator_seismograph import run_seismic_analysis
|
| 8 |
-
from cognitive_mapping_probe.auto_experiment import
|
| 9 |
from cognitive_mapping_probe.prompts import RESONANCE_PROMPTS
|
| 10 |
from cognitive_mapping_probe.utils import dbg
|
| 11 |
|
| 12 |
# --- UI Theme ---
|
| 13 |
theme = gr.themes.Soft(primary_hue="indigo", secondary_hue="blue").set(body_background_fill="#f0f4f9", block_background_fill="white")
|
| 14 |
|
| 15 |
-
# ---
|
| 16 |
|
| 17 |
def cleanup_memory():
|
| 18 |
-
"""
|
| 19 |
dbg("Cleaning up memory...")
|
| 20 |
gc.collect()
|
| 21 |
if torch.cuda.is_available():
|
| 22 |
torch.cuda.empty_cache()
|
| 23 |
dbg("Memory cleanup complete.")
|
| 24 |
|
| 25 |
-
# --- Wrapper
|
| 26 |
|
| 27 |
def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
|
| 28 |
-
"""Wrapper
|
| 29 |
try:
|
| 30 |
-
# Führe die Analyse durch
|
| 31 |
results = run_seismic_analysis(*args, progress_callback=progress)
|
| 32 |
stats = results.get("stats", {})
|
| 33 |
deltas = results.get("state_deltas", [])
|
| 34 |
|
| 35 |
-
# Bereite die Ausgaben vor
|
| 36 |
df = pd.DataFrame({"Internal Step": range(len(deltas)), "State Change (Delta)": deltas})
|
| 37 |
stats_md = f"### Statistical Signature\n- **Mean Delta:** {stats.get('mean_delta', 0):.4f}\n- **Std Dev Delta:** {stats.get('std_delta', 0):.4f}\n- **Max Delta:** {stats.get('max_delta', 0):.4f}\n"
|
| 38 |
|
|
@@ -40,10 +38,8 @@ def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
|
|
| 40 |
except Exception:
|
| 41 |
return f"### ❌ Analysis Failed\n```\n{traceback.format_exc()}\n```", pd.DataFrame(), {}
|
| 42 |
finally:
|
| 43 |
-
# Stelle sicher, dass der Speicher in jedem Fall aufgeräumt wird
|
| 44 |
cleanup_memory()
|
| 45 |
|
| 46 |
-
# Definiere die Plot-Parameter an einer zentralen Stelle für Konsistenz
|
| 47 |
PLOT_PARAMS = {
|
| 48 |
"x": "Step",
|
| 49 |
"y": "Delta",
|
|
@@ -57,35 +53,29 @@ PLOT_PARAMS = {
|
|
| 57 |
}
|
| 58 |
|
| 59 |
def run_auto_suite_display(model_id, num_steps, seed, experiment_name, progress=gr.Progress(track_tqdm=True)):
|
| 60 |
-
"""
|
| 61 |
-
Wrapper für die automatisierte Experiment-Suite.
|
| 62 |
-
Gibt eine neue `gr.LinePlot`-Instanz zurück, um den State-Leak-Bug zu beheben.
|
| 63 |
-
"""
|
| 64 |
try:
|
| 65 |
summary_df, plot_df, all_results = run_auto_suite(model_id, int(num_steps), int(seed), experiment_name, progress)
|
| 66 |
|
| 67 |
dbg("Plot DataFrame Head for Auto-Suite:\n", plot_df.head())
|
| 68 |
|
| 69 |
-
# WISSENSCHAFTLICHE KORREKTUR: Erzeuge eine komplett neue Plot-Komponente
|
| 70 |
-
# mit den neuen Daten. Dies zwingt Gradio, den alten Zustand zu verwerfen.
|
| 71 |
new_plot = gr.LinePlot(value=plot_df, **PLOT_PARAMS)
|
| 72 |
|
| 73 |
return summary_df, new_plot, all_results
|
| 74 |
except Exception:
|
| 75 |
-
# Im Fehlerfall, gib leere, aber korrekt typisierte Komponenten zurück
|
| 76 |
empty_plot = gr.LinePlot(value=pd.DataFrame(), **PLOT_PARAMS)
|
| 77 |
return pd.DataFrame(), empty_plot, f"### ❌ Auto-Experiment Failed\n```\n{traceback.format_exc()}\n```"
|
| 78 |
finally:
|
| 79 |
cleanup_memory()
|
| 80 |
|
| 81 |
-
# --- Gradio UI
|
| 82 |
|
| 83 |
with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
|
| 84 |
gr.Markdown("# 🧠 Cognitive Seismograph 2.3: Advanced Experiment Suite")
|
| 85 |
|
| 86 |
with gr.Tabs():
|
| 87 |
with gr.TabItem("🔬 Manual Single Run"):
|
| 88 |
-
gr.Markdown("
|
| 89 |
with gr.Row(variant='panel'):
|
| 90 |
with gr.Column(scale=1):
|
| 91 |
gr.Markdown("### 1. General Parameters")
|
|
@@ -99,7 +89,7 @@ with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
|
|
| 99 |
manual_run_btn = gr.Button("Run Single Analysis", variant="primary")
|
| 100 |
with gr.Column(scale=2):
|
| 101 |
gr.Markdown("### Single Run Results")
|
| 102 |
-
manual_verdict = gr.Markdown("
|
| 103 |
manual_plot = gr.LinePlot(x="Internal Step", y="State Change (Delta)", title="Internal State Dynamics", show_label=True, height=400, interactive=True)
|
| 104 |
with gr.Accordion("Raw JSON Output", open=False):
|
| 105 |
manual_raw_json = gr.JSON()
|
|
@@ -111,7 +101,7 @@ with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
|
|
| 111 |
)
|
| 112 |
|
| 113 |
with gr.TabItem("🚀 Automated Suite"):
|
| 114 |
-
gr.Markdown("
|
| 115 |
with gr.Row(variant='panel'):
|
| 116 |
with gr.Column(scale=1):
|
| 117 |
gr.Markdown("### Auto-Experiment Parameters")
|
|
|
|
| 5 |
import torch
|
| 6 |
|
| 7 |
from cognitive_mapping_probe.orchestrator_seismograph import run_seismic_analysis
|
| 8 |
+
from cognitive_mapping_probe.auto_experiment import get_curated_experiments, run_auto_suite
|
| 9 |
from cognitive_mapping_probe.prompts import RESONANCE_PROMPTS
|
| 10 |
from cognitive_mapping_probe.utils import dbg
|
| 11 |
|
| 12 |
# --- UI Theme ---
|
| 13 |
theme = gr.themes.Soft(primary_hue="indigo", secondary_hue="blue").set(body_background_fill="#f0f4f9", block_background_fill="white")
|
| 14 |
|
| 15 |
+
# --- Helper Functions ---
|
| 16 |
|
| 17 |
def cleanup_memory():
|
| 18 |
+
"""A centralized function to clean up VRAM and Python memory."""
|
| 19 |
dbg("Cleaning up memory...")
|
| 20 |
gc.collect()
|
| 21 |
if torch.cuda.is_available():
|
| 22 |
torch.cuda.empty_cache()
|
| 23 |
dbg("Memory cleanup complete.")
|
| 24 |
|
| 25 |
+
# --- Gradio Wrapper Functions ---
|
| 26 |
|
| 27 |
def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
|
| 28 |
+
"""Wrapper for a single manual experiment."""
|
| 29 |
try:
|
|
|
|
| 30 |
results = run_seismic_analysis(*args, progress_callback=progress)
|
| 31 |
stats = results.get("stats", {})
|
| 32 |
deltas = results.get("state_deltas", [])
|
| 33 |
|
|
|
|
| 34 |
df = pd.DataFrame({"Internal Step": range(len(deltas)), "State Change (Delta)": deltas})
|
| 35 |
stats_md = f"### Statistical Signature\n- **Mean Delta:** {stats.get('mean_delta', 0):.4f}\n- **Std Dev Delta:** {stats.get('std_delta', 0):.4f}\n- **Max Delta:** {stats.get('max_delta', 0):.4f}\n"
|
| 36 |
|
|
|
|
| 38 |
except Exception:
|
| 39 |
return f"### ❌ Analysis Failed\n```\n{traceback.format_exc()}\n```", pd.DataFrame(), {}
|
| 40 |
finally:
|
|
|
|
| 41 |
cleanup_memory()
|
| 42 |
|
|
|
|
| 43 |
PLOT_PARAMS = {
|
| 44 |
"x": "Step",
|
| 45 |
"y": "Delta",
|
|
|
|
| 53 |
}
|
| 54 |
|
| 55 |
def run_auto_suite_display(model_id, num_steps, seed, experiment_name, progress=gr.Progress(track_tqdm=True)):
|
| 56 |
+
"""Wrapper for the automated experiment suite, now returning a new plot component."""
|
|
|
|
|
|
|
|
|
|
| 57 |
try:
|
| 58 |
summary_df, plot_df, all_results = run_auto_suite(model_id, int(num_steps), int(seed), experiment_name, progress)
|
| 59 |
|
| 60 |
dbg("Plot DataFrame Head for Auto-Suite:\n", plot_df.head())
|
| 61 |
|
|
|
|
|
|
|
| 62 |
new_plot = gr.LinePlot(value=plot_df, **PLOT_PARAMS)
|
| 63 |
|
| 64 |
return summary_df, new_plot, all_results
|
| 65 |
except Exception:
|
|
|
|
| 66 |
empty_plot = gr.LinePlot(value=pd.DataFrame(), **PLOT_PARAMS)
|
| 67 |
return pd.DataFrame(), empty_plot, f"### ❌ Auto-Experiment Failed\n```\n{traceback.format_exc()}\n```"
|
| 68 |
finally:
|
| 69 |
cleanup_memory()
|
| 70 |
|
| 71 |
+
# --- Gradio UI Definition ---
|
| 72 |
|
| 73 |
with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
|
| 74 |
gr.Markdown("# 🧠 Cognitive Seismograph 2.3: Advanced Experiment Suite")
|
| 75 |
|
| 76 |
with gr.Tabs():
|
| 77 |
with gr.TabItem("🔬 Manual Single Run"):
|
| 78 |
+
gr.Markdown("Run a single experiment with manual parameters to explore hypotheses.")
|
| 79 |
with gr.Row(variant='panel'):
|
| 80 |
with gr.Column(scale=1):
|
| 81 |
gr.Markdown("### 1. General Parameters")
|
|
|
|
| 89 |
manual_run_btn = gr.Button("Run Single Analysis", variant="primary")
|
| 90 |
with gr.Column(scale=2):
|
| 91 |
gr.Markdown("### Single Run Results")
|
| 92 |
+
manual_verdict = gr.Markdown("Analysis results will appear here.")
|
| 93 |
manual_plot = gr.LinePlot(x="Internal Step", y="State Change (Delta)", title="Internal State Dynamics", show_label=True, height=400, interactive=True)
|
| 94 |
with gr.Accordion("Raw JSON Output", open=False):
|
| 95 |
manual_raw_json = gr.JSON()
|
|
|
|
| 101 |
)
|
| 102 |
|
| 103 |
with gr.TabItem("🚀 Automated Suite"):
|
| 104 |
+
gr.Markdown("Run a predefined, curated suite of experiments and visualize the results comparatively.")
|
| 105 |
with gr.Row(variant='panel'):
|
| 106 |
with gr.Column(scale=1):
|
| 107 |
gr.Markdown("### Auto-Experiment Parameters")
|