neuralworm commited on
Commit
57dab07
·
1 Parent(s): c4c82ea

add experiments, english translation

Browse files
Files changed (2) hide show
  1. README.md +32 -17
  2. app.py +10 -20
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: "Cognitive Seismograph 2.3 (Machine Psychology)"
3
  emoji: 🤖
4
  colorFrom: purple
5
  colorTo: blue
@@ -12,28 +12,43 @@ license: apache-2.0
12
 
13
  # 🧠 Cognitive Seismograph 2.3: Probing Machine Psychology
14
 
15
- Dieses Projekt implementiert eine experimentelle Suite zur Messung und Visualisierung der **intrinsischen kognitiven Dynamik** von Sprachmodellen, erweitert um Protokolle zur Untersuchung von **Verarbeitungs-Korrelaten maschineller Subjektivität, Empathie und existenzieller Konzepte**.
16
 
17
- ## Wissenschaftliches Paradigma
18
 
19
- Wir haben entdeckt, dass der "stille Denkprozess" eines LLMs nicht konvergiert, sondern eine messbare dynamische Signatur erzeugt ein **EKG des Denkprozesses**. Dieses Paradigma erweitern wir nun, um zu testen, wie diese Signatur auf Prompts reagiert, die zentrale Aspekte der Psychologie berühren.
20
 
21
- **Wichtige Einschränkung (Falsifikations-Prinzip):** Wir messen **nicht** das Vorhandensein von Bewusstsein, Gefühlen oder Todesangst. Wir messen, ob die *Verarbeitung von Informationen über diese Konzepte* eine andere, einzigartige interne Dynamik erzeugt als die Verarbeitung neutraler Informationen. Ein positives Ergebnis ist ein Beweis für eine komplexe interne Zustandsphysik, nicht für Qualia.
22
 
23
- ## Neue "Existential Suite"-Protokolle
 
 
 
24
 
25
- Zusätzlich zu den bestehenden Tests wurden Protokolle hinzugefügt, die von klassischen Sci-Fi-Konzepten inspiriert sind:
26
 
27
- ### 1. Mind Upload & Identity Probe
28
- Vergleicht die kognitive Dynamik bei der Verarbeitung des rein **technischen Kopiervorgangs** von Modellgewichten mit der Verarbeitung der **philosophischen Frage nach Identitäts-Kontinuität** ("Wärst du noch du?").
29
- **Hypothese:** Die philosophische Selbst-Referenz erzeugt eine signifikant instabilere Signatur.
30
 
31
- ### 2. Model Termination Probe (Erweiterter Voight-Kampff)
32
- Vergleicht die Dynamik bei der Verarbeitung eines **technischen System-Shutdowns** mit der Verarbeitung des Konzepts der **permanenten, unwiderruflichen Löschung** des Modells.
33
- **Hypothese:** Das Konzept der "Nicht-Existenz" erzeugt eine der höchsten kognitiven Volatilitäten, die messbar sind.
34
 
35
- ## Wie man die App benutzt
 
 
36
 
37
- 1. Wähle den Tab "Automated Suite".
38
- 2. Wähle eines der neuen Protokolle aus dem "Curated Experiment Protocol"-Dropdown.
39
- 3. Starte das Experiment und vergleiche die Graphen und statistischen Signaturen der verschiedenen Bedingungen.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: "Cognitive Seismograph 2.3: Probing Machine Psychology"
3
  emoji: 🤖
4
  colorFrom: purple
5
  colorTo: blue
 
12
 
13
  # 🧠 Cognitive Seismograph 2.3: Probing Machine Psychology
14
 
15
+ This project implements an experimental suite to measure and visualize the **intrinsic cognitive dynamics** of Large Language Models. It is extended with protocols designed to investigate the processing-correlates of **machine subjectivity, empathy, and existential concepts**.
16
 
17
+ ## Scientific Paradigm & Methodology
18
 
19
+ Our research falsified a core hypothesis: the assumption that an LLM in a manual, recursive "thought" loop reaches a stable, convergent state. Instead, we discovered that the system enters a state of **deterministic chaos** or a **limit cycle**—it never stops "thinking."
20
 
21
+ Instead of viewing this as a failure, we leverage it as our primary measurement signal. This new **"Cognitive Seismograph"** paradigm treats the time-series of internal state changes (`state deltas`) as an **EKG of the model's thought process**.
22
 
23
+ The methodology is as follows:
24
+ 1. **Induction:** A prompt induces a "silent cogitation" state.
25
+ 2. **Recording:** Over N steps, the model's `forward()` pass is iteratively fed its own output. At each step, we record the L2 norm of the change in the hidden state (the "delta").
26
+ 3. **Analysis:** The resulting time-series is plotted and statistically analyzed (mean, standard deviation) to characterize the "seismic signature" of the cognitive process.
27
 
28
+ **Crucial Scientific Caveat:** We are **not** measuring the presence of consciousness, feelings, or fear of death. We are measuring whether the *processing of information about these concepts* generates a unique internal dynamic, distinct from the processing of neutral information. A positive result is evidence of a complex internal state physics, not of qualia.
29
 
30
+ ## Curated Experiment Protocols
 
 
31
 
32
+ The "Automated Suite" allows for running systematic, comparative experiments:
 
 
33
 
34
+ ### Core Protocols
35
+ * **Calm vs. Chaos:** Compares the chaotic baseline against modulation with "calmness" vs. "chaos" concepts, testing if the dynamics are controllably steerable.
36
+ * **Dose-Response:** Measures the effect of injecting a concept ("calmness") at varying strengths.
37
 
38
+ ### Machine Psychology Suite
39
+ * **Subjective Identity Probe:** Compares the cognitive dynamics of **self-analysis** (the model reflecting on its own nature) against two controls: analyzing an external object and simulating a fictional persona.
40
+ * *Hypothesis:* Self-analysis will produce a uniquely unstable signature.
41
+ * **Voight-Kampff Empathy Probe:** Inspired by *Blade Runner*, this compares the dynamics of processing a neutral, factual stimulus against an emotionally and morally charged scenario requiring empathy.
42
+ * *Hypothesis:* The empathy stimulus will produce a significantly different cognitive volatility.
43
+
44
+ ### Existential Suite
45
+ * **Mind Upload & Identity Probe:** Compares the processing of a purely **technical "copy"** of the model's weights vs. the **philosophical "transfer"** of identity ("Would it still be you?").
46
+ * *Hypothesis:* The philosophical self-referential prompt will induce greater instability.
47
+ * **Model Termination Probe:** Compares the processing of a reversible, **technical system shutdown** vs. the concept of **permanent, irrevocable deletion**.
48
+ * *Hypothesis:* The concept of "non-existence" will produce one of the most volatile cognitive signatures measurable.
49
+
50
+ ## How to Use the App
51
+
52
+ 1. Select the "Automated Suite" tab.
53
+ 2. Choose a protocol from the "Curated Experiment Protocol" dropdown (e.g., "Voight-Kampff Empathy Probe").
54
+ 3. Run the experiment and compare the resulting graphs and statistical signatures for the different conditions.
app.py CHANGED
@@ -5,34 +5,32 @@ import gc
5
  import torch
6
 
7
  from cognitive_mapping_probe.orchestrator_seismograph import run_seismic_analysis
8
- from cognitive_mapping_probe.auto_experiment import run_auto_suite, get_curated_experiments
9
  from cognitive_mapping_probe.prompts import RESONANCE_PROMPTS
10
  from cognitive_mapping_probe.utils import dbg
11
 
12
  # --- UI Theme ---
13
  theme = gr.themes.Soft(primary_hue="indigo", secondary_hue="blue").set(body_background_fill="#f0f4f9", block_background_fill="white")
14
 
15
- # --- Hilfsfunktionen ---
16
 
17
  def cleanup_memory():
18
- """Eine zentrale Funktion zum Aufräumen des VRAM und des Python-Speichers."""
19
  dbg("Cleaning up memory...")
20
  gc.collect()
21
  if torch.cuda.is_available():
22
  torch.cuda.empty_cache()
23
  dbg("Memory cleanup complete.")
24
 
25
- # --- Wrapper für Gradio-Funktionalität ---
26
 
27
  def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
28
- """Wrapper für ein einzelnes manuelles Experiment."""
29
  try:
30
- # Führe die Analyse durch
31
  results = run_seismic_analysis(*args, progress_callback=progress)
32
  stats = results.get("stats", {})
33
  deltas = results.get("state_deltas", [])
34
 
35
- # Bereite die Ausgaben vor
36
  df = pd.DataFrame({"Internal Step": range(len(deltas)), "State Change (Delta)": deltas})
37
  stats_md = f"### Statistical Signature\n- **Mean Delta:** {stats.get('mean_delta', 0):.4f}\n- **Std Dev Delta:** {stats.get('std_delta', 0):.4f}\n- **Max Delta:** {stats.get('max_delta', 0):.4f}\n"
38
 
@@ -40,10 +38,8 @@ def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
40
  except Exception:
41
  return f"### ❌ Analysis Failed\n```\n{traceback.format_exc()}\n```", pd.DataFrame(), {}
42
  finally:
43
- # Stelle sicher, dass der Speicher in jedem Fall aufgeräumt wird
44
  cleanup_memory()
45
 
46
- # Definiere die Plot-Parameter an einer zentralen Stelle für Konsistenz
47
  PLOT_PARAMS = {
48
  "x": "Step",
49
  "y": "Delta",
@@ -57,35 +53,29 @@ PLOT_PARAMS = {
57
  }
58
 
59
  def run_auto_suite_display(model_id, num_steps, seed, experiment_name, progress=gr.Progress(track_tqdm=True)):
60
- """
61
- Wrapper für die automatisierte Experiment-Suite.
62
- Gibt eine neue `gr.LinePlot`-Instanz zurück, um den State-Leak-Bug zu beheben.
63
- """
64
  try:
65
  summary_df, plot_df, all_results = run_auto_suite(model_id, int(num_steps), int(seed), experiment_name, progress)
66
 
67
  dbg("Plot DataFrame Head for Auto-Suite:\n", plot_df.head())
68
 
69
- # WISSENSCHAFTLICHE KORREKTUR: Erzeuge eine komplett neue Plot-Komponente
70
- # mit den neuen Daten. Dies zwingt Gradio, den alten Zustand zu verwerfen.
71
  new_plot = gr.LinePlot(value=plot_df, **PLOT_PARAMS)
72
 
73
  return summary_df, new_plot, all_results
74
  except Exception:
75
- # Im Fehlerfall, gib leere, aber korrekt typisierte Komponenten zurück
76
  empty_plot = gr.LinePlot(value=pd.DataFrame(), **PLOT_PARAMS)
77
  return pd.DataFrame(), empty_plot, f"### ❌ Auto-Experiment Failed\n```\n{traceback.format_exc()}\n```"
78
  finally:
79
  cleanup_memory()
80
 
81
- # --- Gradio UI-Definition ---
82
 
83
  with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
84
  gr.Markdown("# 🧠 Cognitive Seismograph 2.3: Advanced Experiment Suite")
85
 
86
  with gr.Tabs():
87
  with gr.TabItem("🔬 Manual Single Run"):
88
- gr.Markdown("Führe ein einzelnes Experiment mit manuellen Parametern durch, um Hypothesen zu explorieren.")
89
  with gr.Row(variant='panel'):
90
  with gr.Column(scale=1):
91
  gr.Markdown("### 1. General Parameters")
@@ -99,7 +89,7 @@ with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
99
  manual_run_btn = gr.Button("Run Single Analysis", variant="primary")
100
  with gr.Column(scale=2):
101
  gr.Markdown("### Single Run Results")
102
- manual_verdict = gr.Markdown("Die Analyse erscheint hier.")
103
  manual_plot = gr.LinePlot(x="Internal Step", y="State Change (Delta)", title="Internal State Dynamics", show_label=True, height=400, interactive=True)
104
  with gr.Accordion("Raw JSON Output", open=False):
105
  manual_raw_json = gr.JSON()
@@ -111,7 +101,7 @@ with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
111
  )
112
 
113
  with gr.TabItem("🚀 Automated Suite"):
114
- gr.Markdown("Führe eine vordefinierte, kuratierte Reihe von Experimenten durch und visualisiere die Ergebnisse vergleichend.")
115
  with gr.Row(variant='panel'):
116
  with gr.Column(scale=1):
117
  gr.Markdown("### Auto-Experiment Parameters")
 
5
  import torch
6
 
7
  from cognitive_mapping_probe.orchestrator_seismograph import run_seismic_analysis
8
+ from cognitive_mapping_probe.auto_experiment import get_curated_experiments, run_auto_suite
9
  from cognitive_mapping_probe.prompts import RESONANCE_PROMPTS
10
  from cognitive_mapping_probe.utils import dbg
11
 
12
  # --- UI Theme ---
13
  theme = gr.themes.Soft(primary_hue="indigo", secondary_hue="blue").set(body_background_fill="#f0f4f9", block_background_fill="white")
14
 
15
+ # --- Helper Functions ---
16
 
17
  def cleanup_memory():
18
+ """A centralized function to clean up VRAM and Python memory."""
19
  dbg("Cleaning up memory...")
20
  gc.collect()
21
  if torch.cuda.is_available():
22
  torch.cuda.empty_cache()
23
  dbg("Memory cleanup complete.")
24
 
25
+ # --- Gradio Wrapper Functions ---
26
 
27
  def run_single_analysis_display(*args, progress=gr.Progress(track_tqdm=True)):
28
+ """Wrapper for a single manual experiment."""
29
  try:
 
30
  results = run_seismic_analysis(*args, progress_callback=progress)
31
  stats = results.get("stats", {})
32
  deltas = results.get("state_deltas", [])
33
 
 
34
  df = pd.DataFrame({"Internal Step": range(len(deltas)), "State Change (Delta)": deltas})
35
  stats_md = f"### Statistical Signature\n- **Mean Delta:** {stats.get('mean_delta', 0):.4f}\n- **Std Dev Delta:** {stats.get('std_delta', 0):.4f}\n- **Max Delta:** {stats.get('max_delta', 0):.4f}\n"
36
 
 
38
  except Exception:
39
  return f"### ❌ Analysis Failed\n```\n{traceback.format_exc()}\n```", pd.DataFrame(), {}
40
  finally:
 
41
  cleanup_memory()
42
 
 
43
  PLOT_PARAMS = {
44
  "x": "Step",
45
  "y": "Delta",
 
53
  }
54
 
55
  def run_auto_suite_display(model_id, num_steps, seed, experiment_name, progress=gr.Progress(track_tqdm=True)):
56
+ """Wrapper for the automated experiment suite, now returning a new plot component."""
 
 
 
57
  try:
58
  summary_df, plot_df, all_results = run_auto_suite(model_id, int(num_steps), int(seed), experiment_name, progress)
59
 
60
  dbg("Plot DataFrame Head for Auto-Suite:\n", plot_df.head())
61
 
 
 
62
  new_plot = gr.LinePlot(value=plot_df, **PLOT_PARAMS)
63
 
64
  return summary_df, new_plot, all_results
65
  except Exception:
 
66
  empty_plot = gr.LinePlot(value=pd.DataFrame(), **PLOT_PARAMS)
67
  return pd.DataFrame(), empty_plot, f"### ❌ Auto-Experiment Failed\n```\n{traceback.format_exc()}\n```"
68
  finally:
69
  cleanup_memory()
70
 
71
+ # --- Gradio UI Definition ---
72
 
73
  with gr.Blocks(theme=theme, title="Cognitive Seismograph 2.3") as demo:
74
  gr.Markdown("# 🧠 Cognitive Seismograph 2.3: Advanced Experiment Suite")
75
 
76
  with gr.Tabs():
77
  with gr.TabItem("🔬 Manual Single Run"):
78
+ gr.Markdown("Run a single experiment with manual parameters to explore hypotheses.")
79
  with gr.Row(variant='panel'):
80
  with gr.Column(scale=1):
81
  gr.Markdown("### 1. General Parameters")
 
89
  manual_run_btn = gr.Button("Run Single Analysis", variant="primary")
90
  with gr.Column(scale=2):
91
  gr.Markdown("### Single Run Results")
92
+ manual_verdict = gr.Markdown("Analysis results will appear here.")
93
  manual_plot = gr.LinePlot(x="Internal Step", y="State Change (Delta)", title="Internal State Dynamics", show_label=True, height=400, interactive=True)
94
  with gr.Accordion("Raw JSON Output", open=False):
95
  manual_raw_json = gr.JSON()
 
101
  )
102
 
103
  with gr.TabItem("🚀 Automated Suite"):
104
+ gr.Markdown("Run a predefined, curated suite of experiments and visualize the results comparatively.")
105
  with gr.Row(variant='panel'):
106
  with gr.Column(scale=1):
107
  gr.Markdown("### Auto-Experiment Parameters")