Application Architecture¶

This guide covers the technical architecture, code organization, and development patterns of SPECTROview. It is intended for developers who want to understand, maintain, or extend the application.

MVVM Pattern¶

SPECTROview enforces a strict Model-View-ViewModel architecture. Every workspace follows the same three-layer separation of concerns:

Model — Pure data containers and domain logic. Models hold no references to Qt widgets and can be tested independently.
ViewModel — Business-logic orchestrator. The ViewModel reads and mutates Models, then notifies the View through Qt signals. It never imports or references View classes.
View — Qt widget layer. Views connect to ViewModel signals and call ViewModel methods in response to user actions.

graph LR
    A["User Action"] --> B["View (v_*)"]
    B -->|"method call"| C["ViewModel (vm_*)"]
    C -->|"method call"| D["Model (m_*)"]
    D -->|"return data"| C
    C -->|"emit Signal"| B
    B -->|"update UI"| A

File Naming Convention¶

Layer	Prefix	Example
View	`v_`	`v_workspace_spectra.py`
ViewModel	`vm_`	`vm_workspace_spectra.py`
Model	`m_`	`spectra_store.py`

Import Rules¶

Layer	Can Import	Cannot Import
View	ViewModel, Components	—
ViewModel	Model, `fit_engine`	View
Model	Standard libs only	View, ViewModel

Signal/Slot Communication¶

Views and ViewModels communicate exclusively via Qt signals and slots. A ViewModel must never call View methods directly, and a View must never modify Model state without going through its ViewModel.

# ── ViewModel defines signals ──
class VMWorkspaceSpectra(QObject):
    spectra_list_changed = Signal(list)       # ViewModel → View
    fit_progress_updated = Signal(int, int, int, float, int)

# ── View connects in __init__ ──
class VWorkspaceSpectra(QWidget):
    def __init__(self):
        self.vm.spectra_list_changed.connect(self._update_list)

Project Structure¶

spectroview/
├── __init__.py             # Constants, peak models, resource paths
├── main.py                 # Entry point, QMainWindow, cross-workspace wiring
├── application/            # API facade for the currently running desktop state
│   ├── service.py          # Dataset/spectra/fit/graph/map domain operations
│   ├── dispatch.py         # Queued Qt main-thread execution boundary
│   └── errors.py           # Stable integration error codes
│
├── model/                  # Data models (no Qt deps)
│   ├── spectra_store.py    # Tensor-centric SpectraStore & MapData structures
│   ├── workspace_io.py     # Unified serialization for Workspaces
│   ├── peak_model.py       # Helper functions for peak parameters
│   ├── m_graph.py          # Plot configuration model
│   ├── m_settings.py       # Persistent app settings (QSettings wrapper)
│   ├── m_io.py             # File loaders (TXT, CSV, WDF, SPC, TRPL, DAT)
│   ├── m_mva.py            # PCA + NMF engine
│   ├── m_fit_model_manager.py # Saved fit model file management
│   ├── m_file_converter.py # Batch file format converter
│   ├── m_quick_calculators.py     # Pure math logic for scientific calculators (Spot Size, Depth, Unit conversion)
│   ├── m_update_checker.py # Background GitHub release checker (QThread)
│   └── m_spc.py            # Galactic SPC binary reader
│
├── viewmodel/              # Business logic and data orchestration
│   ├── vm_workspace_spectra.py   # Spectra workspace logic (base class)
│   ├── vm_workspace_maps.py      # Maps workspace (extends Spectra VM)
│   ├── vm_workspace_graphs.py    # Graphs workspace logic
│   ├── vm_fit_model_builder.py   # Fit model file management orchestration
│   ├── vm_mva.py                 # MVA orchestration
│   ├── vm_settings.py            # Settings persistence
│   └── utils.py                  # Helpers, toast notifications
│
├── ai_agent/               # AI Data Chat module (multi-provider; runtime-optional)
│   ├── __init__.py              # Module docstring, keeps import guarded
│   ├── m_llm_client.py          # Multi-provider LLM clients (Ollama/OpenAI/Anthropic) + QThread workers
│   ├── m_conversation.py        # Conversation model (+ m_conversation_store.py: history)
│   ├── m_prompt_manager.py      # System-prompt / rules / knowledge assembly
│   ├── mcp/server.py            # Shared FastMCP chat + full desktop tool profiles
│   ├── mcp/runtime.py           # Opt-in loopback Streamable HTTP lifecycle
│   ├── config/, prompts/, rules/, knowledge/, examples/, utils/  # Prompt assets + helpers (see ai_agent.md)
│   ├── vm_chat.py               # Chat ViewModel (prompt-tier selection, agentic loop)
│   └── v_chat_panel.py          # Floating chat dialog (+ v_history_dialog.py)
│
├── view/                   # Qt widgets and UI layout
│   ├── v_workspace_spectra.py    # Spectra workspace View
│   ├── v_workspace_maps.py       # Maps workspace View (extends Spectra)
│   ├── v_workspace_graphs.py     # Graphs workspace View
│   ├── v_quick_calculators.py    # Scientific calculators GUI
│   └── components/               # Shared / reusable widgets
│       ├── v_spectra_viewer.py        # Matplotlib spectra canvas
│       ├── v_spectra_legend_editor.py  # Batch spectrum label/color editor
│       ├── v_fit_model_builder.py     # Baseline + Peak + Fit controls
│       ├── v_peak_table.py            # Interactive peak parameter table
│       ├── v_map_viewer.py            # Heatmap / wafer canvas
│       ├── v_map_viewer_dialog.py     # Detachable map viewer window
│       ├── v_map_list.py              # Loaded maps list panel
│       ├── v_graph.py                 # Single graph widget (matplotlib)
│       ├── v_mva.py                   # PCA/NMF controls and plots
│       ├── v_fit_results.py           # Fit results DataFrame table
│       ├── v_data_filter.py           # Dynamic query filter panel
│       ├── v_dataframe_table.py       # Generic DataFrame viewer
│       ├── v_spectra_list.py          # Spectrum list with checkboxes
│       ├── v_moretab.py               # Metadata / tab panel
│       ├── v_settings.py              # Fit/view settings dialog
│       ├── v_menubar.py               # Main menu bar
│       ├── v_about.py                 # About dialog
│       ├── v_update_banner.py         # Update notification banner widget
│       ├── v_user_manual.py           # Built-in user manual viewer
│       ├── customized_widgets.py      # Palette combobox, custom toolbar
│       └── customize_graph/           # Graph customization dialog (package)
│           ├── customize_graph_dialog.py     # CustomizeGraphDialog (singleton, tab host)
│           ├── customize_legend.py           # Legend/Color tab
│           ├── customize_axis.py             # Axis tab (scale/limits/breaks; incl. wafer/2Dmap Z color-scale)
│           ├── customize_annotations.py      # Annotations tab
│           ├── customize_more_options.py     # More Options tab (plot options, theme, fonts, sorting, trendline/histogram)
│           └── customize_annotation_dialogs.py  # EditLineDialog/EditTextDialog/ColorDelegate
│
├── fit_engine/             # High-performance batch fitting
│   ├── vbf_engine.py            # Orchestrator (VBFengine)
│   ├── evaluator.py             # Parameter mapping (VBFevaluator)
│   ├── optimizer.py             # Batched Levenberg-Marquardt
│   ├── models.py                # Batched peak functions + Jacobians
│   ├── scalar_models.py         # Fallback scalar functions + PEAK_MODEL_REGISTRY
│   ├── vbf_thread.py            # QThread wrapper
│   ├── baseline.py              # Baseline algorithms (arPLS, airPLS, etc)
│   └── noise.py                 # Noise estimation functions
│
└── resources/              # Icons, stylesheets, user manual assets
    ├── icons/
    ├── styles/
    └── user_manual/

Application Entry Point¶

spectroview/main.py creates the QMainWindow and instantiates all workspaces as tabs in a QTabWidget:

graph LR
    Main["main.py"] --> Tabs["QTabWidget"]
    Tabs --> S["Spectra"]
    Tabs --> M["Maps"]
    Tabs --> G["Graphs"]
    Main --> W["setup_connections()"]
    W -->|"inject ref"| M
    W -->|"signal"| S
    W -->|"signal"| Tabs

The setup_connections() method in main.py wires cross-workspace dependencies:

Maps → Graphs: VMWorkspaceMaps.set_graphs_workspace(v_graphs) injects a reference so Maps can send profiles and DataFrames directly to the Graphs workspace.
Maps → Spectra: main.py connects send_spectra_to_workspace to the Spectra ViewModel's receive_spectra(), which ingests deep copies of the selected map spectra into the Spectra tab's SpectraStore.
Fit Results → Graphs: Both Spectra and Maps emit fit_results_updated with a pd.DataFrame that can be forwarded to the Graphs workspace for statistical plotting.

Keeping the startup path lean¶

Importing main.py transitively imports every module reachable from it, and that import graph dominates startup time. Two families of packages are deliberately kept off that graph and imported at their point of use instead:

Package	Import cost	Where it is now imported
`anthropic`, `openai`, `ollama`, `mcp`	~4.5 s combined	`Main.open_ai_chat()`, `_load_*()` in `ai_agent/m_llm_client.py`, `MCPHub._open_session()`
`scipy.stats`, `scipy.interpolate`, `scipy.linalg`	~1.3 s combined	`PlotRenderer._plot_histogram()` / `WaferPlot.plot()`, `SpectraStore.batch_preprocess()`, `VMMVA._build_from_all_maps()`, `build_heatmap_grid()`, `optimizer._batched_solve()`

scipy carries an extra constraint: a purely lazy import just moves the stall to the first histogram or wafer render, which made opening a saved .graphs workspace visibly slower. Main.showEvent() therefore starts a daemon thread (_prewarm_heavy_imports) that imports the three scipy submodules right after the first paint — the window is up at ~2 s and scipy is warm ~1 s later, so every function-local import scipy... is a sys.modules hit by the time a user can trigger one. The LLM SDKs are deliberately not prewarmed: most sessions never open the chat, and the panel already shows a loading state.

Verify with python -X importtime -c "import spectroview.main" before adding a new module-level import of a heavy third-party package — if it isn't needed to paint the first window, defer it. tests/performance/test_startup_imports.py fails if one of these reappears on the startup path.

Data Lifecycle¶

Spectra: Loading → Processing → Fitting → Results¶

sequenceDiagram
    participant View
    participant VM as ViewModel
    participant Engine as FitEngine

    View->>VM: load_spectra(paths)
    VM-->>View: spectra_list_changed

    View->>VM: subtract_baseline(), add_peak()

    View->>VM: fit(apply_all)
    VM->>Engine: fit_spectra()
    Engine-->>VM: results
    VM-->>View: fit_results_updated

Maps: Loading → Extraction → Heatmap → Profile¶

sequenceDiagram
    participant View
    participant VM as ViewModel
    participant Viewer as MapViewer

    View->>VM: load_map_files(paths)
    VM-->>View: maps_list_changed

    View->>VM: select_map(name)
    VM-->>View: map_data_updated
    View->>Viewer: plot_heatmap()

    View->>VM: extract_profile()
    VM-->>View: switch_to_graphs_tab

Workspace Inheritance¶

VMWorkspaceMaps extends VMWorkspaceSpectra. This means every fitting, baseline, peak, and serialization feature available in the Spectra workspace is automatically available in the Maps workspace, with additional map-specific overrides:

classDiagram
    class VMWorkspaceSpectra {
        +store
        +fit()
        +save_work()
    }

    class VMWorkspaceMaps {
        +maps
        +load_map_files()
        +select_map()
    }

    class VMWorkspaceGraphs {
        +dataframes
        +graphs
        +create_graph()
    }

    VMWorkspaceSpectra <|-- VMWorkspaceMaps

Reusable Component System¶

Workspaces are composed from shared components that follow the same signal-based pattern:

Component	File	Purpose
`VSpectraViewer`	`v_spectra_viewer.py`	Matplotlib canvas for spectrum display, zoom/pan, peak/baseline interaction
`SpectraLegendEditorDialog`	`v_spectra_legend_editor.py`	Transactional palette, legend-limit, label, and color editor (100-row batches)
`VFitModelBuilder`	`v_fit_model_builder.py`	X-correction, spectral range, baseline, peaks, fit controls
`VPeakTable`	`v_peak_table.py`	Editable table of peak parameters (center, FWHM, amplitude, bounds)
`VMapViewer`	`v_map_viewer.py`	Heatmap/wafer canvas with Z/X range sliders, mask, profile extraction
`VMapViewerDialog`	`v_map_viewer_dialog.py`	Detachable always-on-top window wrapping `VMapViewer`
`VGraph`	`v_graph.py`	Matplotlib graph widget supporting 10+ plot styles
`VDataFilter`	`v_data_filter.py`	Dynamic `pandas` `.query()` filter builder
`VFitResults`	`v_fit_results.py`	Color-coded fit results table
`VMVA`	`v_mva.py`	PCA/NMF controls and embedded plotting
`CustomizeGraphDialog`	`customize_graph/customize_graph_dialog.py`	Singleton dialog for graph annotation, legends, and axis customization

Persistence & Serialization¶

SPECTROview delegates all loading and saving operations to the unified WorkspaceIO class (workspace_io.py), which isolates IO logic from the ViewModels. Each workspace uses its own save/load format via WorkspaceIO:

Workspace	File Extension	Key Strategy
`Spectra`	`.spectra`	ZIP archive (`format_version: 2`): `metadata.json` (`store_meta` per map) + per-map NPZ arrays. Written by the unified `WorkspaceIO.save_workspace()`.
`Maps`	`.maps`	Same unified ZIP format as `.spectra` (`WorkspaceIO.save_workspace()`), whose metadata additionally carries `maps_metadata`, `map_type`, and the fit-results DataFrame.
`Graphs`	`.graphs`	ZIP archive (`format_version: 4`) via `WorkspaceIO.save_workspace()`: `metadata.json` (plots + DataFrame sources) + compressed DataFrames.

Spectrum Serialization Flow¶

# Save (format_version 2): SpectraStore → ZIP archive (metadata.json + NPZ arrays)
metadata = {
    "format_version": 2,
    "store_meta": {                 # keyed by map name (== fname for single spectra)
        "sample_001": {
            "fnames": ["sample_001"],
            "is_active": [True],
            "baseline_config": {...},
            "peak_params": [...],
            "range_min": None, "range_max": None,
            # ...
        }
    }
}
# arrays: per-map NPZ blocks produced by SpectraStore.to_npz_dict(map_name)
# (raw x0 float64 axis + Y0 float32 intensities, plus any processed/fit arrays)

Threading Model¶

Long-running operations run on QThread subclasses to prevent UI freezing. All threads emit progress signals that the ViewModel relays to the View's progress bar:

Thread Class	Location	Purpose
`VBFthread`	`fit_engine/vbf_thread.py`	Batched fitting (primary engine)
`UpdateCheckerWorker`	`model/m_update_checker.py`	Background GitHub release check at startup
`LLMWorker` / `APIWorker` / `AnthropicWorker`	`ai_agent/m_llm_client.py`	Streaming chat requests for AI Data Chat — one worker per backend (Ollama / OpenAI-compatible / Anthropic)

Thread lifecycle:

ViewModel instantiates the thread and connects finished / progress signals.
Thread .start() — runs run() on a separate OS thread.
On completion, the finished signal triggers _on_fit_finished() in the ViewModel.
ViewModel emits result signals → View updates UI.

Cross-Workspace Communication¶

SPECTROview avoids a global event bus. Instead, main.py uses dependency injection and direct signal connections:

# main.py → setup_connections()
# 1. Inject reference: Maps VM can call Graphs workspace methods
self.v_maps_workspace.vm.set_graphs_workspace(self.v_graphs_workspace)

# 2. Maps → Spectra: ingest spectra sent from the Maps workspace
self.v_maps_workspace.vm.send_spectra_to_workspace.connect(
    self.v_spectra_workspace.vm.receive_spectra
)

# 3. Signal: Maps requests tab switch after sending profile
self.v_maps_workspace.vm.switch_to_graphs_tab.connect(
    lambda: self.tabWidget.setCurrentWidget(self.v_graphs_workspace)
)

Global Constants (`init.py`)¶

spectroview/__init__.py defines application-wide constants:

Constant	Purpose
`PEAK_MODELS`	Registered peak shapes (`Gaussian`, `Lorentzian`, `PseudoVoigt`, `Fano`, ...)
`FIT_PARAMS`	Default fitting parameters (`max_ite`, `xtol`, `ftol`, bounds)
`PLOT_STYLES`	Available graph types (`point`, `scatter`, `box`, `bar`, `line`, `wafer`, ...)
`X_AXIS_UNIT`, `Y_AXIS_UNIT`	Axis label registries
`AXIS_LABELS`	Autocomplete suggestions for graph labels
`ICON_DIR`	Resolved path to `resources/icons/`
`PLOT_POLICY_LIGHT`, `PLOT_POLICY_DARK`	Matplotlib stylesheet paths

Update Checker¶

Overview¶

SPECTROview ships a lightweight, opt-out update system that queries the GitHub Releases API in the background and displays a dismissable banner when a newer version is found. The banner can download the wheel attached to that exact release, verify its GitHub SHA-256 digest when available, close the application, install the wheel with pip, and relaunch it.

No extra dependency is required — only Python's built-in urllib.

Key Files¶

File	Role
`model/m_update_checker.py`	`QThread` workers for release checks and wheel downloads; creates the detached pip-install/restart helper
`view/components/v_update_banner.py`	Slim 36 px banner widget inserted at position 0 of the central layout
`model/m_settings.py`	Stores `enabled`, `skipped_version`, and `last_check_date` in `QSettings`
`main.py`	Starts the thread via `QTimer.singleShot(2000, ...)` from `showEvent`

Flow¶

sequenceDiagram
    participant Main
    participant Timer as QTimer (2 s)
    participant Worker as UpdateCheckerWorker
    participant GitHub as api.github.com
    participant Banner as VUpdateBanner
    participant Helper as Update helper
    actor User

    Main->>Timer: showEvent → singleShot(2000)
    Timer->>Worker: _start_update_check() → worker.start()
    Worker->>GitHub: GET /repos/CEA-MetroCarac/SPECTROview/releases/latest
    GitHub-->>Worker: JSON {tag_name, html_url, body, assets[]}
    Worker->>Worker: compare versions
    alt newer version found
        Worker-->>Main: update_available(tag, notes, url, wheel URL, SHA-256)
        Main->>Banner: insertWidget(0, VUpdateBanner(...))
        User->>Banner: Update
        Banner->>GitHub: Download release wheel in QThread
        GitHub-->>Banner: Wheel bytes
        Banner->>Helper: Start detached updater and close app
        Helper->>Helper: pip install --upgrade wheel
        Helper->>Main: Relaunch installed SPECTROview
    end
    Worker-->>Main: check_finished → set_last_check_date(today)

Design Decisions¶

Decision	Rationale
`QThread` instead of `QNetworkAccessManager`	Pure Python `urllib` avoids Qt networking module complexity; thread is simpler to test
2-second startup delay	Ensures the UI is fully painted before the network request starts
Once-per-day throttle	Avoids redundant requests; the date is persisted via `QSettings`
Silent check failure	`URLError`, `OSError`, `json.JSONDecodeError` are caught — offline machines see no error
Version comparison via tuples	`_parse_version('v26.29.0') → (26, 29, 0)` handles `v`-prefixed tags and non-numeric parts gracefully
Release asset source	GitHub provides the version, release notes, wheel URL, and integrity digest in one public API response; PyPI remains the normal package-install channel
Detached helper	The helper waits for Qt to exit, installs with the launching interpreter, then clears `PYTHONPATH`/`PYTHONHOME` and restarts from a temporary working directory so a checkout cannot shadow the freshly installed package
Skip vs Dismiss	Skip persists the exact tag — the banner re-appears for the next release. Dismiss hides only for the session

Adding / Modifying the Checker¶

Release automation lives in .github/workflows/release.yml. Pushing a vMAJOR.MINOR.PATCH tag whose value matches spectroview.VERSION builds the wheel and source distribution, attaches them to a GitHub release, and publishes them to PyPI through the pypi environment's Trusted Publisher.

Before the first automated PyPI publication, register a Trusted Publisher for CEA-MetroCarac/SPECTROview in the PyPI project's Publishing settings with workflow filename release.yml and environment pypi. This lets GitHub Actions obtain a short-lived publishing token without storing a long-lived PyPI API token in the repository.

To add a "disable updates" toggle to the Settings dialog, bind MSettings.set_check_for_updates() to a QCheckBox in v_settings.py — the _start_update_check() method in main.py already reads this flag before starting the thread.

Deep-Dive Documentation¶

Topic	Page	Summary
Data Architecture: SpectraStore	spectra_store.md	`MapData`, `MapInfo`, `SpectrumProxy`, data hierarchy, preprocessing pipeline, persistence
Spectra Workspace	spectra.md	`VMWorkspaceSpectra`, spectrum lifecycle, baseline/peak pipeline, fit model management
Maps Workspace	maps.md	`VMWorkspaceMaps`, hyperspectral data loading, heatmap rendering, coordinate handling
Graphs Workspace	graphs.md	`VMWorkspaceGraphs`, DataFrame management, plot creation, `VGraph` rendering
Vectorized Batch Fit Engine (`VBF Engine`)	vbf_engine.md	Batched LM optimizer, analytical Jacobians, adding new peak models
Multivariate Analysis	mva.md	PCA/NMF implementation, data pipeline, export to `Graphs`
AI Data Chat	ai_agent.md	Multi-provider LLM chat (Ollama/OpenAI/Anthropic/…), MCP tool calling, system prompt, Graphs workspace integration

Running & Testing¶

# Run from source
python -m spectroview.main

# Install in editable mode
pip install -e .

# Run tests
pytest

# Build documentation
mkdocs serve

Dependencies¶

Package	Constraint	Purpose
`PySide6`	—	Qt 6 bindings (not PyQt)
`matplotlib`	`< 3.10.9`	Plotting backend for spectra and maps
`numpy`	`< 2.0.0`	Numerical array operations
`scipy`	—	Interpolation, KDTree, SVD
`pandas`	—	DataFrame management
`renishawWiRE`	—	Renishaw `.wdf` file reader
`superqt`	—	Enhanced Qt widgets (range sliders)
`ollama` / `openai` / `anthropic` / `mcp`	`ollama ≥ 0.4`, `mcp ≥ 1.28`	LLM backends + Model Context Protocol for AI Data Chat. Core dependencies (no `[ai]` extras group) — a plain `pip install` includes them; the feature is only optional to use at runtime.
`truststore`	`≥ 0.9` (Python ≥ 3.10)	Routes TLS verification through the OS trust store (corporate/internal CAs) for cloud AI providers; import is guarded