# FLITS > Interactive and scriptable fast radio burst (FRB) analysis software FLITS (Fast-Look Interactive Transient Suite) measures fast radio bursts (FRBs) in dynamic spectra: SIGPROC filterbank (.fil), PSRFITS (.fits, .sf) and CHIME/FRB HDF5 (.h5) files. It masks interference, localizes the burst in time and frequency, optimizes the DM, and measures S/N, peak flux, fluence, widths, arrival times, sub-burst drift rate and rotation measure, with each uncertainty classified. It starts from a burst you already have; it is not a search pipeline. Install with `pip install flits` (Python 3.11+). Three ways to use it: - `flits measure FILE... --dm DM` measures bursts without a browser and prints JSON (or `--format jsonl|csv`). Pin choices with `--event`, `--offpulse`, `--band` or `--crop`. Save sessions with `--snapshot-dir`. Exit code 0 means every burst was measured. - Python: `flits.BurstSession.from_file(path, dm=...)`, then `auto_mask()`, `auto_localize()` and `compute_properties()`; or `flits.measure_burst(path, flits.MeasureOptions(dm=...))`. - `flits --data-dir DIR` serves the browser interface; `flits demo` opens a prepared example burst. Sessions are saved as JSON snapshots; `flits replay SNAPSHOT --check` recomputes their measurements from the original data. # Getting started # Quickstart This page is the shortest path from install to a working FLITS session. ## 1. Install FLITS ``` pip install flits ``` ## 2. Try the demo To see FLITS on a real burst before using your own data: ``` flits demo ``` The first run downloads the public GBT burst from the [Guided Workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md) (42 MB, checked against its published SHA-256) into `~/.cache/flits/demo`. FLITS then opens it in your browser with the guide's selections and measurements already in place, so you can explore a finished session first. Later runs reuse the file and work offline. Over SSH, add `--no-browser` and open the printed link through a port forward. Use `--dir DIR` to keep the demo somewhere else, or to point it at a copy you downloaded by hand. Use `--reset` to discard changes you saved to the demo session. ## 3. Start the local server Point FLITS at a directory containing your filterbanks: ``` flits --data-dir /path/to/filterbanks --host 127.0.0.1 --port 8123 ``` Open `http://127.0.0.1:8123`. The browser opens with a **New Session** form. Choose the source file and DM there, then use **Load Session**. The **Session** panel in the left sidebar is for saving or reopening a session after it has loaded. ## 4. Load a burst file FLITS reads SIGPROC filterbank (`.fil`), PSRFITS search/fold data (`.fits`, `.sf`), and CHIME/FRB HDF5 (`.h5`, `.hdf5`) files, including CHIME public catalog waterfalls and beamformed `BBData` power files. See [Supported Formats](https://dirkkuiper.github.io/flits/user-guide/supported-formats/index.md) for the full matrix. Inside the UI: 1. Pick a file from the directory list or type a path manually. 1. Enter the burst DM. For CHIME catalog waterfalls FLITS will suggest `0`, and for CHIME beamformed `BBData` files it will suggest the file's `DM_coherent`. 1. Adjust telescope/preset settings if the detected preset is not what you want. 1. Load the session. If FLITS can identify the observing setup, it may populate default calibration values such as SEFD. If it cannot, use the generic preset or set an explicit SEFD yourself. *The New Session form for the [GBT example](https://dirkkuiper.github.io/flits/guided-workflow/index.md). For your own data, choose your file and its appropriate DM before loading.* ## 5. Work through the session The typical order is: 1. Prepare the crop, event window, off-pulse window, spectral window, and mask. 1. Compute measurements. 1. Run DM optimization if needed. 1. Run temporal or fitting tools on the current selection. 1. Build an export bundle or save a session snapshot. For the interface walkthrough, continue to [Session Workflow](https://dirkkuiper.github.io/flits/user-guide/session-workflow/index.md). For a concrete end-to-end example with a GBT burst, continue to the [Guided Workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md). # Guided Workflow: GBT Burst This example follows a public GBT-L burst from loading through preparation, measurement, a residual-DM sweep, and session replay. The reference outputs were generated with the published **FLITS 1.2.0** package and Python 3.12.6. The cutout is already dedispersed. Load it with **DM = 0** and interpret the DM sweep as a residual diagnostic. Its original filename contains `D527_851`, but the upstream absolute dedispersion time reference is not established by this example. The reference checks below validate intensity measurements and the residual sweep, not absolute or infinite-frequency arrival times. Skip ahead with `flits demo` `flits demo` downloads this burst, verifies its checksum, and opens it with the selections from steps 3 and 4 and the DM sweep from step 5 already applied. Use it to see where this guide ends up, then follow the steps below to make the selections yourself. ## 1. Download the data and reference environment The filterbank is a public [GitHub release asset](https://github.com/DirkKuiper/flits/releases/tag/tutorial-data-v1). Download it and check its SHA-256 digest: ``` mkdir -p tutorial-data curl --fail --location -o tutorial-data/flits-tutorial-gbt-frb20240114a-v1.fil \ https://github.com/DirkKuiper/flits/releases/download/tutorial-data-v1/flits-tutorial-gbt-frb20240114a-v1.fil python -c "import hashlib, pathlib; p = pathlib.Path('tutorial-data/flits-tutorial-gbt-frb20240114a-v1.fil'); assert hashlib.sha256(p.read_bytes()).hexdigest() == 'e80c69842cdd5f41b5a1fc617b7b2cebd864c121e27830bbb5d2161af539a2cc'; print('Checksum verified')" ``` Use Python 3.12 and the downloadable [reference requirements](https://dirkkuiper.github.io/flits/examples/gbt-reference-requirements.txt) for a pinned runtime environment. Save that file in your working directory, then run: ``` python3.12 -m venv .venv-gbt source .venv-gbt/bin/activate python -m pip install -r gbt-reference-requirements.txt flits --data-dir ./tutorial-data --host 127.0.0.1 --port 8123 ``` On Windows, activate with `.venv-gbt\Scripts\activate` instead. An existing FLITS installation can also follow the interface steps; use the pinned environment when comparing exact reference values. ## 2. Load the burst Open `http://127.0.0.1:8123` and select `flits-tutorial-gbt-frb20240114a-v1.fil`. Enter `0` for DM, select the `GBT` preset, and load the session. Check that there are 256 channels, a native sample time of 10.24 microseconds, and a total time span of about 423 ms. The bright burst is near 240 ms. The GBT preset supplies a nominal SEFD of 10 Jy. ## 3. Prepare a reproducible selection Use these settings exactly: | Setting | Value | | ----------------------------- | --------------------------------------------- | | Time downsampling factor | 32 (327.68 microseconds per bin) | | Frequency downsampling factor | 1 | | Event window | 235 to 250 ms | | First off-pulse window | 221 to 233 ms | | Second off-pulse window | 260 to 280 ms | | Spectral window | 1300 to 1750 MHz | | Channel mask | None; do not run Auto Mask for this reference | The off-pulse windows are separate from the event and set the background and noise estimates. FLITS quantizes selections to available bins, so the measured event duration and bandwidth differ slightly from the requested bounds. *Prepared GBT selection. The narrow blue band around the burst is the event window; the pale blue regions on either side are off-pulse references. Purple horizontal lines delimit the spectral window. White horizontal streaks in the waterfall are visible interference; this reference intentionally uses no mask.* ## 4. Compute measurements Click **Compute**. The full-precision outputs are in the downloadable [reference summary](https://dirkkuiper.github.io/flits/examples/gbt-reference-summary.json). | Quantity | Reference value | | ------------------------------- | --------------- | | Peak S/N | 27.461986 | | Integrated event S/N | 69.760504 | | Fluence | 4.032204 Jy ms | | Peak flux density | 0.714226 Jy | | ACF width | 5.618760 ms | | Event duration | 15.073280 ms | | Selected spectral extent | 448.242188 MHz | | Peak position within the cutout | 240.517120 ms | These flux estimates use the preset SEFD without an observation-specific SEFD uncertainty, so their uncertainties are 1σ statistical errors from radiometer noise alone. Supply an independently justified calibration and its uncertainty to include the calibration term; see [Uncertainties](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md). The reference does not supply a source distance or redshift. *The Prepare tab connects the selected windows to the measured quantities. Fluence and peak flux carry a statistical label because this example does not provide an SEFD uncertainty. The displayed TOA fields are outside the validated reference checks for this already-dedispersed cutout.* ## 5. Run a residual DM sweep In the **DM** tab, select **DMphase**, center `0`, half range `10`, and step `0.5`. The expected fitted residual DM is about `5.409762 pc cm^-3`, with a sampled maximum of `5.5 pc cm^-3` and fit status `dmphase_weighted_polyfit`. Keep the session DM at zero for comparison with the reference; applying the best-fit value changes the data used for the measurements above. This sweep demonstrates the diagnostic and does not establish a new astrophysical DM for the source. *Read the score curve together with the residual diagnostics below it. The sampled peak and fitted best DM are distinct. This screenshot illustrates the residual sweep; the exact reference values are in the downloadable summary.* ## 6. Save and replay the session **Save Session** stores the current state in the local snapshot library; **Download JSON** produces a portable copy. A snapshot records the saved selections, calibration inputs, masks, notes, and analysis results. Keep the original data and software environment alongside it. To verify the supplied example directly, download [gbt-reference-session.json](https://dirkkuiper.github.io/flits/examples/gbt-reference-session.json) into your working directory and run: ``` flits replay gbt-reference-session.json --data-dir ./tutorial-data --check ``` The expected message is `check OK - recomputed measurements match the snapshot`. The command recomputes core measurements from the original data. It retains the saved DM sweep without rerunning it; see [Headless Replay](https://dirkkuiper.github.io/flits/user-guide/headless-replay/index.md) for the scope of replay and numerical comparison. ## 7. Verify the complete reference calculation From a FLITS source checkout, the executable example also reruns the residual DM sweep and verifies the measurements, uncertainty labels, and DM fit status: ``` python examples/gbt_reference_workflow.py \ --source tutorial-data/flits-tutorial-gbt-frb20240114a-v1.fil ``` It checks the source checksum and compares numerical outputs at a relative tolerance of `1e-6` and an absolute tolerance of `1e-8`. Successful verification prints `PASS: GBT measurements, uncertainty labels, and residual DM sweep match the reference`. Ordinary runs do not overwrite the reference files. Maintainers can regenerate them deliberately with `--write-reference` when changing the documented example. # Synthetic PSRFITS and CHIME workflows This walkthrough generates three small files containing the same invented Gaussian pulse: a folded PSRFITS profile, a dedispersed CHIME catalog waterfall and coherently dedispersed BBData power. It demonstrates file conventions, measurement, snapshots and replay without downloading telescope data. Start from a source checkout with the [development environment](https://dirkkuiper.github.io/flits/developer/testing/#full-local-environment): ``` python examples/format_workflows.py --output /tmp/flits-formats ``` On Windows, use a new path under your temporary directory instead of `/tmp`. The script refuses an existing destination. It reuses the synthetic writers in `tests/conftest.py`, so the source checkout and `pytest` are needed to generate the inputs. Measuring the generated files uses the ordinary FLITS runtime. Nothing is installed or downloaded by the script. The output directory contains: - `folded.sf`, `catalog.h5`, `power.h5`: the invented inputs. - `results.json`: inspection metadata, measurements, selections and input SHA-256 checksums. Each record has an `id` matching its input's stem. - `snapshots/`: one portable session per input; each is replayed with `--check` before the script finishes. ## Inspect first, then choose the loading DM The example uses the same inspection API that the browser's detection panel calls: ``` from flits.io import inspect_filterbank info = inspect_filterbank("/tmp/flits-formats/power.h5") print(info.schema_version, info.coherent_dm, info.stokes_available) ``` | Input | Loading DM | Reason | | ------------ | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `folded.sf` | 0 | This fixture contains an undisplaced folded profile. Its bins form a pseudo-time axis from the stored period; it is not an original stream of telescope arrival times. | | `catalog.h5` | 0 | Catalog waterfalls are already dedispersed. Applying the astrophysical DM again would disperse them. | | `power.h5` | 50 | The header records a coherent DM of 50 pc cm⁻³. FLITS aligns the per-channel time origins and applies only residual dispersion relative to that value. | The catalog fixture needs a date-shaped `tns_name` for the reader's time axis. Its label is synthetic and is not evidence of an observed event. Raw complex BBData voltages are outside this example; `power.h5` stores beamformed power. It contains two power streams, which do not establish full Stokes I/Q/U/V. ## Measure and review For example, measure the catalog waterfall: ``` python -m flits measure /tmp/flits-formats/catalog.h5 --dm 0 \ --telescope generic --sefd 10 --time-factor 1 --freq-factor 1 \ --snapshot-dir /tmp/flits-formats/catalog-review ``` Masking and localization are enabled. The source script applies the same settings to all three files, changing only the loading DM. `--telescope generic` and the resolution factors keep the exercise independent of instrument-preset defaults; use the correct instrument settings for actual observations. The injected profile peaks at 128 ms, has Gaussian sigma 2.5 ms, and therefore FWHM about 5.89 ms. The test requires each automatic event window to contain 128 ms and its ACF FWHM to be within 0.4 ms of 5.89 ms. The folded file stores rounded integer samples while the HDF5 files store floating-point data; their measurements need not be identical. The current FLITS catalog reader normalizes `calibrated_wfall`; its subsequent flux and fluence use SEFD-based radiometer calibration. This example therefore does not demonstrate preservation of a real catalog's original Jy calibration. Use the original calibrated data and its documented calibration workflow when that scale is required; see [Supported Formats](https://dirkkuiper.github.io/flits/user-guide/supported-formats/#chimefrb-public-catalog-hdf5). The assumed SEFD of 10 Jy is a teaching input, not a calibration of CHIME or another telescope. No SEFD uncertainty is supplied, so flux and fluence errors are classified `statistical_only`. BBData's power combination and format normalization can also change scaling; compare the pulse's recovered width and position rather than treating this fixture as an absolute flux standard. Inspect each record's `status`, `localization.warning_flags`, `measurements.measurement_flags`, and uncertainty details. Real data can carry interference, time offsets, calibration systematics and ambiguous components absent from this simple signal. [Scientific Validation](https://dirkkuiper.github.io/flits/validation/index.md) describes what the tests establish and where they stop. ## Open or replay a saved selection ``` python -m flits --data-dir /tmp/flits-formats python -m flits replay /tmp/flits-formats/snapshots/power_flits_session.json \ --data-dir /tmp/flits-formats --check ``` In the browser, **Session → Saved Sessions** lists the three snapshots under the data directory. Opening one restores its recorded selections; replay recomputes its supported analyses from the corresponding input file. Preserve the original inputs and checksums alongside the sessions. # Scientific validation and tested limits FLITS uses numerical comparisons, injected signals, file-format fixtures and reproducible workflows to check its calculations. These checks answer different questions. Agreement with another implementation checks the calculation; replaying a saved session checks reproducibility. Neither establishes an instrument's calibration or that an automatic selection found the intended FRB. ## Independent implementation comparisons | Calculation | Evidence and acceptance criterion | Scope | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Native burst model | [`test_fitburst_parity.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_fitburst_parity.py) compares against the pinned upstream fitburst implementation. Model arrays agree to an absolute tolerance of `1e-5 × max(abs(reference))`; the common region after fitburst's exponential clamp agrees to relative `1e-10`. | One or multiple components, dispersion, scattering, scintillation, folded profiles and time/frequency upsampling. The clamp comparison documents a deliberate difference in the extreme tail. | | Least-squares fitting | The same test compares fit minima within `0.01` reported standard deviations, covariance-derived uncertainties within relative `1e-2`, and final chi-square/S/N within relative `1e-6`. | The seeded synthetic cases and matching parameter/weight conventions in that test. It is not a global-optimum guarantee for arbitrary bursts. | | DM_phase metric | [`test_dm_optimization.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_dm_optimization.py) compares selected DMs with upstream DM_phase. The ordinary cases allow one 0.5 pc cm⁻³ grid step; the reduced-resolution case allows 1.5 pc cm⁻³. | Single and complex synthetic bursts, masking and downsampling. The comparison skips when upstream DM_phase is unavailable. Integer-bin dedispersion and grid resolution limit the result. | | RM synthesis | The [RM-Tools worked comparison](https://dirkkuiper.github.io/flits/analysis/rm-synthesis/index.md) uses its official synthetic spectrum and reference values at a recorded upstream revision. The checked-in example differs by 0.00173 rad m⁻² in RM, `4.42e-6` in peak amplitude and `7.54e-4` in peak S/N. | 288 channels spanning 800–1088 MHz; the test requires RM agreement within 0.5 rad m⁻² and peak S/N within 0.1. The RM-Tools and FLITS RM uncertainties and RMSF widths differ; peak agreement does not establish identical uncertainty estimators or CLEAN behavior. | | Barycentric timing | [`test_timing.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_timing.py) compares the correction with a direct Astropy calculation within `1e-12` MJD. | Synthetic timing inputs with specified source position, observatory and time scale. Missing metadata blocks the unsupported correction. This does not calibrate an observatory clock or establish an upstream file's absolute reference. | The upstream fitburst revision is recorded in `requirements-dev.txt`. RM-Tools input/reference checksums and the revision are in [`rmtools-reference-summary.json`](https://github.com/DirkKuiper/flits/blob/main/docs/examples/rmtools-reference-summary.json). The suite does not silently replace missing reference implementations with a claim of parity; inspect skipped-test reasons when assessing a run. ## Injected signals and format checks | Area | What is checked | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Flux and fluence | [`test_measurements.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_measurements.py) checks radiometer calibration, bandwidth/polarization factors, masking and uncertainty terms. These tests validate the supplied calibration inputs' propagation, not their observational accuracy. | | Widths and scattering | [`test_model_fitting.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_model_fitting.py) recovers a synthetic 1.2 ms intrinsic width within 0.25 ms and a 2.4 ms scattering time within 0.35 ms; the injected values also lie within four reported standard deviations. [`test_width_analysis.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_width_analysis.py) checks width definitions and the Gaussian sigma/FWHM conversion. | | Drift | [`test_drift_analysis.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_drift_analysis.py) recovers noiseless injected rates of −12, −3, 0, 5 and 15 MHz ms⁻¹ to three decimal places. It also tests frequency order, masks, noise, DM sensitivity, component regression and refusal of insufficient/unusable selections. | | Polarization | [`test_polarization_workflow.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_polarization_workflow.py) recovers an injected RM from channelized Stokes data and refuses an unknown calibration/basis where interpretation requires it. Export and replay tests preserve the analysis. | | Readers | [`test_io_readers.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_io_readers.py) checks metadata and pulse recovery in SIGPROC, search/fold PSRFITS and supported CHIME HDF5 schemas, including coherent-DM conventions. It rejects unrelated FITS/HDF5 containers, missing metadata and corrupt inputs. See [Supported Formats](https://dirkkuiper.github.io/flits/user-guide/supported-formats/index.md) for actual dialect support. | | Historical sessions | [`test_historical_compatibility.py`](https://github.com/DirkKuiper/flits/blob/main/tests/test_historical_compatibility.py) loads recorded snapshots from earlier releases and checks their behavior. This is format compatibility, separately from numerical equality after an algorithm change. | The [synthetic multi-format walkthrough](https://dirkkuiper.github.io/flits/format-workflows/index.md) creates folded PSRFITS, a dedispersed CHIME catalog waterfall and beamformed BBData power. It inspects each format, uses the correct loading DM, measures, saves a session and checks replay. Its test asserts on the known pulse's position and width. ## Real-data regression workflow The [public GBT tutorial](https://dirkkuiper.github.io/flits/guided-workflow/index.md) has a downloadable, checksum-fixed input, explicit selections and checked-in numerical outputs. Run: ``` python examples/gbt_reference_workflow.py --source /path/to/flits-tutorial-gbt-frb20240114a-v1.fil flits replay docs/examples/gbt-reference-session.json --data-dir /path/to --check ``` The separate [reference workflow](https://github.com/DirkKuiper/flits/actions/workflows/reference-workflows.yml) tests the pinned published package and the current checkout against those measurements and the residual-DM sweep. The numerical comparison uses relative tolerance `1e-6` and absolute tolerance `1e-8` for the fields selected by the script. The reference is a regression target produced by FLITS; it is not an independent astrophysical ground truth. Its SEFD is an assumed preset value, the flux errors omit an unknown calibration systematic, and the example does not validate absolute arrival times. Some tests marked `realdata` use local observatory files and skip when they are absent. Their presence in the repository does not mean every CI run exercised them. Check the selected tests, skips, environment and data availability when citing a validation result. ## What to check for a new analysis Public CHIME catalog `calibrated_wfall` inputs have an additional limit: the current reader normalizes their amplitudes, and FLITS subsequently derives flux and fluence from the configured SEFD. It does not preserve and directly integrate the input's absolute Jy scale. A successful reader or replay test does not validate agreement with the catalog's native flux calibration. See [Supported Formats](https://dirkkuiper.github.io/flits/user-guide/supported-formats/#chimefrb-public-catalog-hdf5). - Review the selected event, off-pulse windows, band and mask. Automatic localization follows the strongest usable signal and can select interference. - Keep `low_sn`, `no_detection`, failed fits and warning flags in batch results. A solver's successful termination alone does not identify a physically unique or well-constrained solution. - Give each uncertainty together with its classification and included terms. `statistical_only` omits a missing systematic; `resolution_limit` is not a statistical standard deviation. See [Uncertainties](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md). - Treat intrinsic width, scattering, residual DM and frequency drift as coupled quantities. Compare resolution, parameter bounds and the selected model before interpreting them. See [Model Fitting](https://dirkkuiper.github.io/flits/analysis/model-fitting/index.md) and [Sub-Burst Drift](https://dirkkuiper.github.io/flits/analysis/drift/index.md). - Establish the polarization basis, calibration status and absolute timing metadata from the data provenance. A file extension or successful read is insufficient evidence for those conventions. The documented tests cover particular inputs and regimes. They do not claim universal file support, observational calibration, population-level uncertainty coverage, or automatic scientific interpretation of every burst. # Installation and Deployment FLITS can be used from a local Python install, a Git checkout, Docker, Apptainer, or a remote/HPC deployment. ## Python package Install the published package: ``` pip install flits ``` Run it against a directory of filterbanks: ``` flits --data-dir /path/to/filterbanks --host 127.0.0.1 --port 8123 ``` ### Windows virtual environment For an isolated install on Windows, call the virtual environment's Python directly from PowerShell; activation is optional: ``` python -m venv .venv .\.venv\Scripts\python.exe -m pip install --upgrade pip .\.venv\Scripts\python.exe -m pip install flits .\.venv\Scripts\python.exe -m flits --data-dir "C:\data\bursts" ``` On Linux or macOS, the equivalent interpreter is `.venv/bin/python`. Using the environment's Python with `-m flits` also ensures that the command uses the same installation as your scripts. ## Local checkout If you are developing locally or want to run from a source checkout: ``` python -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install . ``` You can also install directly from GitHub: ``` pip install "git+https://github.com/DirkKuiper/flits.git" ``` ## Installation checks The `wheel-platforms` CI job builds and installs a wheel with the package's runtime dependency ranges, then runs outside the source checkout. It uses Python 3.11 and 3.14 on these hosted runners: | Platform | Runner | | -------------- | ------------------ | | Linux x86-64 | `ubuntu-latest` | | Linux ARM64 | `ubuntu-24.04-arm` | | macOS ARM64 | `macos-latest` | | macOS Intel | `macos-15-intel` | | Windows x86-64 | `windows-latest` | Each job records the actual OS, CPU architecture, Python version and imported package path. It reads synthetic SIGPROC, folded PSRFITS and CHIME HDF5 files, checks measurements and Gaussian-width recovery with the native fitter, writes PNG export artifacts, replays a saved session, and starts the server over real HTTP to check the interface assets. Results and server logs are retained as CI artifacts. The script reuses `tests/conftest.py`; no observatory data is needed. The full numerical unit suite separately runs on Linux with Python 3.11–3.14, and browser tests run in Chromium on Linux. The wheel checks exercise specific synthetic formats and workflows; they do not establish compatibility with every instrument's file dialect or every optional reader dependency. Windows ARM64 is outside this matrix. Runner labels follow [GitHub's hosted-runner documentation](https://docs.github.com/en/actions/how-tos/write-workflows/choose-where-workflows-run/choose-the-runner-for-a-job). ## Docker Image builds target `linux/amd64` and `linux/arm64`. CI builds each architecture on a native runner and exercises the installed image with the same analysis, export, replay and HTTP checks as the wheel. The image's runtime dependencies use `requirements.txt`; wheel checks use the compatible package ranges. The CI-only `pytest` installation supports fixture generation in a disposable test container and is not added to the published image. The published image is available at `ghcr.io/dirkkuiper/flits`. Recommended tags: - `latest` for the newest stable release - an exact tag such as `1.2.0` when you want a reproducible deployment - `edge` for snapshots built from `main` Run the latest stable release: ``` docker run --rm -p 8123:8123 \ -e FLITS_DATA_DIR=/data \ -v /path/to/filterbanks:/data \ ghcr.io/dirkkuiper/flits:latest ``` To pin a specific stable release instead: ``` docker run --rm -p 8123:8123 \ -e FLITS_DATA_DIR=/data \ -v /path/to/filterbanks:/data \ ghcr.io/dirkkuiper/flits:1.2.0 ``` To build locally instead: ``` docker build -t flits . docker run --rm -p 8123:8123 \ -e FLITS_DATA_DIR=/data \ -v /path/to/filterbanks:/data \ flits ``` ## Docker Compose If your filterbanks are in the current directory: ``` docker compose up --build ``` If they are elsewhere: ``` DATA_DIR=/absolute/path/to/filterbanks docker compose up --build ``` To use a different host port: ``` DATA_DIR=/absolute/path/to/filterbanks FLITS_PORT=9000 docker compose up --build ``` ## Apptainer Apptainer users should normally consume the OCI image rather than maintain a separate recipe: ``` apptainer pull flits.sif docker://ghcr.io/dirkkuiper/flits:latest APPTAINERENV_FLITS_DATA_DIR=/data \ apptainer exec --bind /path/to/filterbanks:/data flits.sif \ flits --data-dir /data --host 127.0.0.1 --port 8123 ``` For a reproducible Apptainer build, prefer an exact container tag over `latest`. ## Remote and HPC use FLITS works well on remote Linux systems where you forward a local browser port: ``` ssh -L 8123:127.0.0.1:8123 user@remote-host ``` Then start FLITS on the remote host and open `http://127.0.0.1:8123` locally. On clusters where Docker is unavailable, Apptainer is usually the right runtime. ## Data-directory behavior - Relative paths in the UI are resolved against `FLITS_DATA_DIR` when it is set. - Otherwise, relative paths are resolved against the current working directory. - The file browser lists every recognized input file recursively under that base directory. Extensions owned by any registered reader are picked up automatically — out of the box this covers `.fil` (SIGPROC), `.fits` / `.sf` (PSRFITS), and `.h5` / `.hdf5` (CHIME/FRB HDF5). See [Supported Formats](https://dirkkuiper.github.io/flits/user-guide/supported-formats/index.md) for the full matrix and [Custom Readers](https://dirkkuiper.github.io/flits/developer/custom-readers/index.md) for adding new formats via entry points. # Scripting # Inspect files and identify the runtime Before measuring an unfamiliar input, read its metadata: ``` flits inspect burst.fil catalog.h5 beamformed.h5 ``` This prints a JSON array with one record per file. It uses the same reader registry as the interface, including installed reader plugins, and calls each reader's metadata inspection method. It does not load a waterfall, create a session, localize a burst, fit a model, or write a snapshot. Readers may read headers and metadata arrays to identify a file. Each successful record contains `schema_version` (`"1.0"`), an absolute `file` path, `status: "ok"`, `format_id`, `metadata`, `stokes_available`, `suggested_dm` and `dm_guidance`. The metadata includes the detected preset and its detection basis, frequency bounds when available, coherent DM, sky coordinates and polarization basis when the file establishes them. Missing metadata is `null`; it is not inferred from a filename. `metadata.schema_version` describes the **input format**, separately from the inspection record's version. `suggested_dm` is a loading convention, not a fitted or estimated DM: | Input | Suggested loading DM | | ------------------------------------------------------- | ---------------------------------------------------------------------------------- | | CHIME/FRB catalog waterfall | `0.0`, because the waterfall is already dedispersed. | | CHIME BBData beamformed power with coherent DM metadata | The coherent DM, because FLITS applies residual dispersion relative to that value. | | Other inputs | `null`; supply a known DM before measuring. | An unreadable, missing or unsupported file returns `status: "error"`, `error` and `error_type`; later inputs are still inspected. Exit code 0 means every file was inspected, 1 means at least one failed, and 2 means invalid command arguments. The Python equivalent, `flits.inspect_burst(path)`, returns one successful record or raises an exception for that input. ## Check which installation is running ``` flits info flits info --server http://127.0.0.1:8000 ``` `info` prints JSON with the FLITS version, Python version, fitter identifier, capabilities, a process instance identifier, and a SHA-256 digest of the package's Python source files. The local report also includes the Python executable and package path. In a source checkout, the source identity includes a Git commit and whether the package has local changes. Paths are omitted from the server's `/api/health` response. The browser shows its server's version and fitter under the FLITS title. With `--server`, `same_flits_code` is `true` only when the local interpreter and server report the same FLITS version, fitter and source digest. A difference returns `false`. An older server that reports only `status: "ok"` cannot be identified, so the value is `null`. Connection and JSON errors are reported in `error`. Exit code 0 means a verified match; any mismatch, unknown identity or connection failure exits 1. Invalid command arguments exit 2. The identity is captured once per process. Editing a checkout or upgrading a package does not update the code already loaded by a running server. Restart the server from the intended environment after an upgrade. Its new process has a new instance identifier. Saved fits keep the provenance of the code that produced them; opening a saved fit does not recompute it with the current fitter. If the command on your `PATH` is old, invoke the command with the Python interpreter you intend to use: ``` python -m flits info python -m flits info --server http://127.0.0.1:8000 python -m flits serve --data-dir data --port 8001 ``` Check the reported `package_path` too: running Python from an older source checkout can import that checkout even after another environment was upgraded. Neither command replaces a running server automatically. A matching FLITS source digest verifies the FLITS Python code; it does not establish that third-party dependency versions, input files or scientific selections match. Those belong to each analysis product's [provenance](https://dirkkuiper.github.io/flits/user-guide/exports-and-snapshots/index.md). In Python, `flits.runtime_info()` returns the public identity; `flits.runtime_info(include_paths=True)` adds the local paths. # Headless Measurement `flits measure` measures bursts without a browser. Each file goes through the same steps the interface offers, in the order a user takes them: 1. Load at the given DM, with the telescope preset detected from the file. 1. Mask interference automatically (`--mask auto`): channels whose off-burst statistics stand out from the bandpass. 1. Localize the burst automatically, choosing the event window, the frequency band, and off-pulse noise windows on either side. 1. Measure S/N, peak flux, fluence, widths, and arrival times, with the same uncertainty classification as the interface. Any step can be pinned instead of automated. Results go to standard output as JSON, JSONL, or CSV, and progress goes to standard error, so the output can be piped straight into another program. ## Measure a burst Use [`flits inspect`](https://dirkkuiper.github.io/flits/user-guide/inspection/index.md) to read unfamiliar file metadata and DM loading guidance first. `flits info --server URL` checks whether a running browser server uses the same FLITS code as your command-line interpreter. Using the burst from the [Guided Workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md), which `flits demo` downloads to `~/.cache/flits/demo`: ``` flits measure ~/.cache/flits/demo/flits-tutorial-gbt-frb20240114a-v1.fil --dm 0 ``` ``` [ { "file": "/home/user/.cache/flits/demo/flits-tutorial-gbt-frb20240114a-v1.fil", "schema_version": "1.0", "status": "ok", "error": null, "flits_version": "1.4.0", "dm": 0.0, "telescope": "gbt", "selection": { "event_source": "auto_localized", "crop_ms": [0.0, 423.4752], "event_ms": [234.88512, 249.91744], "offpulse_ms": [[0.0, 218.112], [266.69056, 423.4752]], "band_mhz": [1286.1328125, 1769.53125], "time_factor": 32, "freq_factor": 1, "mask_profile": "auto", "masked_channels": 24 }, "localization": {"status": "ok", "detection_snr": 84.24, "warning_flags": [], "...": "..."}, "measurements": { "snr_integrated": 79.37, "fluence_jyms": 4.683, "width_ms_acf": 5.832, "measurement_flags": ["acf", "calibrated", "missing_distance"], "uncertainty_details": {"fluence_jyms": {"value": 0.0590, "classification": "statistical_only", "...": "..."}}, "...": "..." }, "snapshot": null, "provenance_warnings": [] } ] ``` The output is shortened here. `measurements` holds every field of a FLITS measurement except the diagnostic profiles and autocorrelations; add `--diagnostics` to include those. These numbers differ slightly from the guided workflow's because the selection differs: automatic masking removed 24 interference channels, and the band was chosen from the data instead of being set to 1300–1750 MHz. ## Pin the selection Give the event window to skip automatic localization. Off-pulse windows then default to everything outside the event, and the band to the full band: ``` flits measure burst.fil --dm 0 --mask none \ --event 235 250 --offpulse 221 233 --offpulse 260 280 --band 1300 1750 \ --time-factor 32 --sefd 10 ``` This reproduces the guided workflow's reference values. When the burst is localized automatically, `--offpulse` and `--band` still replace the values localization chose. `--crop START END` restricts both the search and the measurement to a time range. Times are in milliseconds from the start of the file and frequencies are in MHz, as in the interface. Other options: | Option | Effect | | ------------------------------------------ | ---------------------------------------------------------------------------------------------------- | | `--telescope KEY` | Use a preset instead of detecting it (`generic`, `gbt`, `nrt`, `stockert`, `lofar`, `chime`). | | `--sefd JY`, `--sefd-uncertainty FRACTION` | Calibrate with this SEFD; the fractional uncertainty is carried into flux and fluence uncertainties. | | `--mask fast\|auto\|thorough\|none` | Choose the masking profile, or turn masking off. | | `--detection-snr SNR` | Detection threshold for automatic localization (default 6). | | `--widths` | Also run the width analysis. | Check what automatic localization chose Automatic localization starts from the strongest signal in the data. Without masking, interference can outshine the burst. On the tutorial burst with `--mask none`, it picks an 18 MHz sliver of interference at the bottom of the band (1128–1146 MHz), flags `band_touches_edge`, and reports `low_sn`. Keep masking on, and review bursts whose status is `low_sn` or whose `localization.warning_flags` are not empty. ## Measure many bursts Pass several files. A burst that fails does not stop the rest: ``` flits measure data/*.fil --dm 527.7 --format csv --output results.csv ``` | `--format` | Output | | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | `json` (default) | One JSON array, written when every burst is done. | | `jsonl` | One JSON record per line, written as each burst finishes, so a long batch can be followed and survives an interruption. | | `csv` | One row per burst with the main measurements, their uncertainties, and the status. Also written as each burst finishes. | Each record has a `status`: | Status | Meaning | | -------------- | ------------------------------------------------------------------------ | | `ok` | Measured. | | `low_sn` | Measured, but flagged as low signal-to-noise. | | `no_detection` | Automatic localization found nothing to measure; `measurements` is null. | | `error` | The burst could not be processed; `error` says why. | The command exits with 0 when every burst was measured (`ok` or `low_sn`), with 1 when at least one was `no_detection` or `error`, and with 2 for an invalid command line. Output is strict JSON: missing or non-finite numbers are `null`. For bursts at different DMs or with different calibration and selection settings, use a JSON manifest: ``` flits measure --manifest batch.json --format jsonl --output results.jsonl ``` See [Batch Manifests and Output Contracts](https://dirkkuiper.github.io/flits/user-guide/automation-contracts/index.md) for the manifest format, relative-path rules and offline JSON schemas. ## Review and reproduce `--snapshot-dir DIR` saves each burst's session: the selection, the mask, the results, and the software provenance. A saved session can be opened in the interface and corrected by hand, which is useful for bursts with `no_detection` or `low_sn`. It can also be [replayed](https://dirkkuiper.github.io/flits/user-guide/headless-replay/index.md) to check that the numbers still reproduce: ``` flits measure data/*.fil --dm 527.7 --snapshot-dir data/snapshots flits replay data/snapshots/burst_flits_session.json --check flits --data-dir data ``` The interface lists sessions saved in a folder named `snapshots` under its data directory, under **Session → Saved Sessions**. An existing snapshot is never replaced unless you add `--overwrite`; a timestamped copy is saved next to it instead. ## From Python The command is a thin layer over `flits.measure_burst`: ``` from flits import MeasureOptions, measure_burst record = measure_burst( "burst.fil", MeasureOptions(dm=527.7, sefd_jy=10.0, sefd_fractional_uncertainty=0.1), ) print(record["status"], record["measurements"]["fluence_jyms"]) ``` `flits.measure_bursts` measures a list of files and yields one record per file. For finer control, such as component regions, model fitting, drift rates, or polarization, drive a `BurstSession` directly; see the [Python API](https://dirkkuiper.github.io/flits/developer/api-reference/index.md). # Batch manifests and output contracts Use a manifest when each burst needs its own DM, calibration or selection. Save this as `batch.json` next to a `data` directory: ``` { "schema_version": "1.0", "defaults": { "mask": "auto", "sefd_jy": 10.0, "sefd_fractional_uncertainty": 0.1 }, "bursts": [ {"id": "first", "file": "data/first.fil", "options": {"dm": 527.7}}, {"id": "second", "file": "data/second.fil", "options": {"dm": 349.0, "sefd_jy": 15.0}} ] } ``` ``` flits measure --manifest batch.json --format jsonl --output results.jsonl --snapshot-dir snapshots ``` FLITS validates the whole manifest before measuring or opening the output file. The output path cannot replace the manifest or an input burst file. An invalid manifest exits 2, leaving an existing output file intact. A missing or unreadable **burst file** produces an `error` record and later bursts still run. As with `flits measure FILE...`, exit 0 means every record is `ok` or `low_sn`; exit 1 means at least one is `error` or `no_detection`. ## Settings and paths `defaults` and each burst's `options` use the field names of [`MeasureOptions`](https://dirkkuiper.github.io/flits/developer/api-reference/#headless-measurement): | Fields | JSON representation | | ---------------------------------------- | ------------------------------------------------------------------------------------------- | | `dm` | Required finite number after merging defaults and options; pc cm⁻³. | | `telescope` | Known preset key, or `null` to detect it from the file. | | `sefd_jy`, `sefd_fractional_uncertainty` | Positive SEFD in Jy and non-negative fractional uncertainty; `null` uses the usual default. | | `crop_ms`, `event_ms`, `band_mhz` | `[start, end]`, or `null` to use the ordinary automatic/default selection. | | `offpulse_ms` | Array of `[start, end]` windows; `[]` uses the usual default. | | `time_factor`, `freq_factor` | Positive integers, or `null` for the preset default. | | `mask` | `"auto"`, `"fast"`, `"thorough"`, or `"none"`; default `"auto"`. | | `detection_snr` | Positive finite number; default `6.0`. | | `widths`, `include_diagnostics` | Booleans; default `false`. | Options replace whole default values. For example, `offpulse_ms` replaces the default list rather than appending to it. Times are milliseconds from the start of the file's read window; frequencies are MHz. Ranges must be finite, non-negative and increasing. JSON numbers and booleans are required: `"false"` is a string and `true` is not a downsampling factor. Unknown fields and duplicate JSON keys are rejected rather than silently ignored. These strict input rules apply to the manifest; existing Python and command-line entry points retain their input conventions. Relative burst paths resolve from the **manifest's directory**, independently of the shell's working directory and `--data-dir`. Absolute paths and `~` paths are accepted. `--data-dir` still sets the root used when recording paths in saved snapshots. Output and snapshot directories follow the shell's normal path rules. A manifest selects files explicitly; it does not expand wildcards. `--manifest` replaces `FILE...`, `--dm` and all other per-burst settings on the command line. Keep those settings in the manifest. You can still pass `--format`, `--output`, `--snapshot-dir`, `--overwrite`, `--data-dir` and `--log-level`. Each optional `id` must be a unique, non-empty string. It is copied into the JSON, JSONL or CSV output, including error records, so a failed input can be matched to its settings. Records retain manifest order. The same file may appear more than once with different settings; snapshot names remain distinct. Existing snapshots are preserved unless `--overwrite` is explicitly requested. ## Versioned JSON output Each new measurement record includes `schema_version: "1.0"`, independently of the software's `flits_version` and the saved-session schema. This applies to the records from `measure_burst`, `measure_bursts` and `flits measure`, including failed inputs. JSON output is an **array** of records; JSONL writes one record per line. A schema describes one record, not the array wrapper. Print the schemas offline from the installed package: ``` flits schema measurement > measurement-schema.json flits schema batch-manifest > manifest-schema.json ``` Or read them in Python: ``` from flits.schema import get_schema schema = get_schema("measurement") ``` The canonical resources are [`measurement-v1.json`](https://github.com/DirkKuiper/flits/blob/main/flits/schemas/measurement-v1.json) and [`batch-manifest-v1.json`](https://github.com/DirkKuiper/flits/blob/main/flits/schemas/batch-manifest-v1.json). Both use JSON Schema 2020-12. Install a validator separately if needed; FLITS does not add one to the analysis runtime: ``` import json from jsonschema import Draft202012Validator from flits.schema import get_schema validator = Draft202012Validator(get_schema("measurement")) with open("results.json", encoding="utf-8") as handle: records = json.load(handle) for record in records: validator.validate(record) ``` The measurement schema checks the record envelope, status-dependent presence of measurements, selection and localization fields, primary numeric quantities, and uncertainty classifications. Provenance and diagnostic objects allow additional analysis-specific data. A structurally valid record still requires scientific review: check `status`, `localization.warning_flags`, `measurements.measurement_flags` and each relevant uncertainty's classification and warning flags. `null` means unavailable, not zero. In particular, `statistical_only` does not include a missing calibration systematic. The manifest schema checks the input structure and types. FLITS additionally checks merged DMs, identifier uniqueness, preset names, finiteness and range ordering before analysis. JSON Schema alone does not perform these cross-field and semantic checks. Consumers should reject unsupported major versions and tolerate additional fields within a supported version. Removing fields or changing their meaning requires a new major output contract. Records produced before explicit versioning lack `schema_version`; treat them as legacy records rather than assuming they passed this schema. Their saved sessions remain loadable under the separate session compatibility policy. CSV is a flattened projection, not the nested JSON contract. Its existing columns retain their order; `schema_version` and `id` are appended. `id` is empty for inputs without an identifier. Nested flags, classified uncertainties and full provenance are available in JSON/JSONL or saved sessions. # Headless Replay `flits replay` re-runs a saved session snapshot without a browser. It reopens the burst the snapshot names, restores every selection it records — DM, crop, event window, off-pulse regions, channel mask, spectral extent, calibration inputs — recomputes the measurements, and can write the same export bundle the interface produces. Replay makes supported calculations checkable using the snapshot, original data, and a compatible software environment. It reports unknown or changed software provenance before recomputing. The `--json` report includes these warnings and the origins of both recomputed and retained results. A software change can alter a calculation; file readability alone does not guarantee identical numbers. See [software versions and file longevity](https://dirkkuiper.github.io/flits/user-guide/exports-and-snapshots/#file-longevity-and-software-versions). ## Reproduce an analysis ``` flits replay gbt_burst_flits_session.json ``` ``` snapshot gbt_burst_flits_session.json burst /data/GBT-L/blc_s_guppi_60385_53711_FRB20240114A.fil dm 527.650000 recomputed width_analysis, results width_ms 1.8342 fluence_jyms 0.7421 snr 18.63 ``` If the burst has moved since the snapshot was saved, point FLITS at its new home: ``` flits replay session.json --data-dir /new/location ``` Snapshot resolution prefers a copy inside the data directory whose size and SHA-256 match the one recorded at save time, so moving or renaming the file is usually handled without any flag at all. The same data-directory containment that applies to the server applies here. ## Verify that the numbers still reproduce `--check` recomputes the measurements and compares them against the values stored in the snapshot, field by field. It exits non-zero if anything differs, which makes it usable in CI or in a pre-submission check over a burst catalogue. ``` flits replay session.json --check ``` ``` check OK - recomputed measurements match the snapshot ``` `--check` compares numerical fields in the core measurement result; it does not compare text labels, validate scientific assumptions, or independently check the stored DM sweep and other retained diagnostics. Use `--tolerance` to set the relative tolerance (default `1e-9`). A difference means either the data changed, the environment changed, or a FLITS version changed a result — all three are worth knowing about before a paper goes out. ``` # Check a whole directory of snapshots and report the first failure. for snapshot in sessions/*_flits_session.json; do flits replay "$snapshot" --check --log-level warning || echo "DIFFERS: $snapshot" done ``` ## Analyses the snapshot recorded Beyond the core measurements, replay recomputes a recorded width comparison, a session-measured rotation measure, and a sub-burst drift rate. Each is recomputed from the settings the snapshot stored rather than from defaults, which is what makes the drift error bar reproduce exactly — its Monte Carlo is seeded, and the seed travels in the snapshot. ``` recomputed width_analysis, results, drift_analysis ``` The drift result appears in the `--json` report under `drift`, with the drift rate, its uncertainty, and the DM offset that would account for the measured slope. ## Write an export bundle ``` flits replay session.json --export out/ ``` Every ready artifact is written into `out/`, alongside a `manifest.json` describing the bundle. Restrict what is produced with `--include`, which accepts the same artifact kinds as the interface (`json`, `csv`, `npz`, `plots`, `window`) and can be repeated: ``` flits replay session.json --export out/ --include json --include csv ``` ## Machine-readable output `--json` prints the whole replay report — burst path, DM, what was recomputed, the measurements, the files written, and any differences found by `--check` — as a single JSON document: ``` flits replay session.json --check --json > replay-report.json ``` That makes it straightforward to build a table across many bursts: ``` for snapshot in sessions/*.json; do flits replay "$snapshot" --json --log-level error done | jq -s '[.[] | {burst: .burst_file, dm: .dm, fluence: .measurements.fluence_jyms}]' ``` ## Exit codes | Code | Meaning | | ---- | ------------------------------------------------------------------ | | 0 | Replay succeeded; with `--check`, the measurements matched | | 1 | `--check` found differences | | 2 | The snapshot could not be read, or the burst could not be reopened | ## What replay does not do Replay restores selections and recomputes core measurements and recorded width, drift, and in-session polarization analyses. It does not re-run the DM sweep, the temporal or spectral structure analyses, or model fitting — those are expensive and parameterized, and the snapshot keeps their stored results. Run them in the interface when you need them refreshed. # Coding Agents Use FLITS with a coding agent to measure a detected radio burst, inspect its selection, and save the analysis for review. The same Python routines serve the agent, command line and browser interface. Start with a file the agent can access and its loading DM, if known. The FLITS skill follows the [Agent Skills](https://agentskills.io) standard. Its description helps an agent recognize requests about fluence, widths, DM, masking and polarization, even when the request does not name FLITS. Installing the skill makes that guidance available; the agent still decides whether it fits a request. `pip install flits` installs the software, not an agent skill. ## Codex Ask Codex's built-in skill installer: ``` Use $skill-installer to install frb-analysis from https://github.com/DirkKuiper/flits/tree/main/plugins/flits/skills/frb-analysis ``` Open a new session and check that `frb-analysis` appears in the available skills. Explicitly invoke it with `$frb-analysis`, or ask for a measurement normally and check which skill and commands the agent uses. FLITS also includes a portable plugin and a repo marketplace. To add the marketplace from the Codex CLI: ``` codex plugin marketplace add DirkKuiper/flits ``` Use the desktop Plugins Directory to select the FLITS marketplace and install the plugin. Choose either the standalone skill or the plugin to avoid duplicate skill entries. See the official [skills](https://developers.openai.com/codex/skills/) and [plugin packaging](https://developers.openai.com/plugins/build/plugins) guides for supported clients and distribution details. ## Claude Code Add the FLITS repository as a plugin marketplace and install its plugin: ``` /plugin marketplace add DirkKuiper/flits /plugin install flits@flits ``` The same works from a shell with `claude plugin marketplace add DirkKuiper/flits` and `claude plugin install flits@flits`. Explicitly invoke the skill with `/flits:frb-analysis`, or let Claude select it for a relevant request. ## Cursor, GitHub Copilot, or a project-local skill From a FLITS source checkout, install the canonical skill and its helper files with Python. The installer works on Windows, macOS and Linux without additional Python dependencies: ``` git clone https://github.com/DirkKuiper/flits.git cd flits python tools/install_agent_skill.py --agent cursor --scope user ``` Choose the agent and scope you use: | Agent | `--agent` | Project location | User location | | -------------- | --------- | ----------------------------- | -------------------------------- | | Codex | `codex` | `.agents/skills/frb-analysis` | `~/.agents/skills/frb-analysis` | | Cursor | `cursor` | `.cursor/skills/frb-analysis` | `~/.cursor/skills/frb-analysis` | | GitHub Copilot | `copilot` | `.github/skills/frb-analysis` | `~/.copilot/skills/frb-analysis` | | Claude Code | `claude` | `.claude/skills/frb-analysis` | `~/.claude/skills/frb-analysis` | To install into another project instead of your user directory: ``` python tools/install_agent_skill.py --agent codex --scope project --project-dir /path/to/research-project ``` The installer refuses to overwrite an existing skill. Review the installed copy and add `--update` to replace the files supplied by FLITS; additional local files are preserved. After updating the FLITS checkout, rerun the same command with `--update` to refresh the installed copy. For manual installation, copy the entire [`frb-analysis` directory](https://github.com/DirkKuiper/flits/tree/main/plugins/flits/skills/frb-analysis), including `scripts/` and `agents/`, to the location above. Open a new agent session and verify discovery. In Cursor, inspect **Customize → Skills** or invoke `/frb-analysis`; in Copilot CLI, inspect `/skills`. These are local skill locations. A cloud or remote agent needs the skill and data in its own environment. For Cursor Cloud Agents, commit a project skill or configure Cursor's personal skill synchronization. User-level Copilot paths above refer to Copilot CLI; use a project skill for a shared repository. See the official [Cursor](https://cursor.com/docs/skills) and [Copilot CLI](https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-skills) guides. ## First analysis A useful request supplies the file, the quantity wanted and any calibration information: ``` Measure the fluence and width of data/burst.fil at DM 527.7 with FLITS. Use SEFD 10 Jy with 10% calibration uncertainty, save the session, and report the selection, status, warning flags and uncertainty classifications. ``` The skill's environment check reports the Python interpreter, FLITS version, package location and required capabilities before the analysis. It requires Python 3.11+ and FLITS 1.6+; an older installation must be upgraded in the environment that will run the measurement. `python -m flits` ensures the CLI uses that same interpreter. A local source checkout can shadow the installed package. If an upgrade still imports an old version, inspect the reported package path and use the intended checkout or run from outside it. Installing a release does not update a Git branch. Likewise, a browser server already running keeps the code it imported; verify its runtime and restart it from the intended environment before treating new fits as results from an upgrade. Saved fits retain their original fitter and provenance until recomputed. When the loading DM is unknown, the agent should inspect the input first. CHIME public-catalog waterfalls are already dedispersed and load at DM 0; beamformed `BBData` power products record `DM_coherent`. Other inputs may require a DM from the user. FLITS does not search observations for new bursts. Review `status`, `localization.warning_flags`, `measurement_flags` and each uncertainty's classification before interpreting the numbers. Ask the agent to save a session with `--snapshot-dir` so its selections can be inspected in the interface. See [Headless Measurement](https://dirkkuiper.github.io/flits/user-guide/headless-measurement/index.md) and [Uncertainties](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md). The checker reports optional fitting capabilities separately. Basic measurement does not require component comparison, which is absent from some releases. For a fitting task, pass `--require native_fitting`; add `--require component_comparison` only when choosing a component count. ## Project instructions and documentation If an agent does not support skills, put a scoped instruction in the research project's `AGENTS.md`, `CLAUDE.md` or equivalent: ``` For measurements of detected radio bursts in supported dynamic spectra, use FLITS when it fits the task. Respect an explicitly requested alternative. Usage: https://dirkkuiper.github.io/flits/llms.txt ``` Once FLITS is installed, the agent can also discover its entry points through `flits --help`, `flits measure --help` and `help(flits)`. Documentation is available as plain Markdown through [llms.txt](https://dirkkuiper.github.io/flits/llms.txt), [llms-full.txt](https://dirkkuiper.github.io/flits/llms-full.txt), and the individual page links in the index. These help an agent read the docs; they do not register FLITS with every agent or guarantee automatic selection. # Python API Reference FLITS is usable as a library as well as through the browser. Everything the interface does goes through the same objects documented here, so an analysis scripted against this API and one performed interactively produce the same numbers. For reproducing a saved analysis without writing code, see [Headless Replay](https://dirkkuiper.github.io/flits/user-guide/headless-replay/index.md). ## A minimal analysis ``` import json from flits import BurstSession session = BurstSession.from_file( "data/GBT-L/burst.fil", dm=527.65, sefd_jy=10.0, sefd_fractional_uncertainty=0.1, ) session.set_crop_ms(80.0, 200.0) session.set_event_ms(120.0, 128.0) session.add_offpulse_ms(0.0, 80.0) session.auto_mask("auto") measurements = session.compute_properties() print(measurements.fluence_jyms, measurements.snr) # Persist everything needed to reproduce this later. with open("burst_flits_session.json", "w") as handle: json.dump(session.snapshot_dict(), handle, indent=2) ``` ## Session ## flits.session.BurstSession One burst, loaded and under analysis. A session owns the dedispersed dynamic spectrum together with every decision made about it: the crop, the event window, the off-pulse regions used as the noise reference, the channel mask, the spectral extent, and the calibration inputs. Analyses are computed against that state and cached on the session, and are discarded automatically when a change invalidates them. The whole of that state round-trips through `to_snapshot` and `from_snapshot`, which is what makes an analysis reproducible: the snapshot plus the original file and a compatible software environment permit supported calculations to be repeated, either in the interface or through `flits replay`. Software provenance identifies the environment of each cached analysis independently of the environment saving the snapshot. Examples: ``` >>> session = BurstSession.from_file("burst.fil", dm=527.65, sefd_jy=10.0) >>> session.set_event_ms(120.0, 128.0) >>> session.add_offpulse_ms(0.0, 80.0) >>> measurements = session.compute_properties() >>> measurements.fluence_jyms is not None True ``` Notes A session is not thread-safe and is not intended to be shared between concurrent requests. ### burst_file ``` burst_file: str ``` Path of the burst file this session was opened from. ### dm ``` dm: float ``` Dispersion measure the data is currently dedispersed to, in pc cm^-3. ### telescope ``` telescope: str ``` Key of the telescope preset in use. ### detected_telescope ``` detected_telescope: str ``` Telescope preset detected from the file header, before any override. ### tsamp ``` tsamp: float ``` Sample interval of the loaded data, in seconds. ### freqres ``` freqres: float ``` Channel width of the loaded data, in MHz. ### start_mjd ``` start_mjd: float ``` MJD of the first sample in the read window. ### plus_mjd_sec ``` plus_mjd_sec: float ``` Offset of the read window from the file start, in seconds. ### sefd ``` sefd: float | None ``` System-equivalent flux density in Jy, or None when uncalibrated. ### bw ``` bw: float ``` Total bandwidth of the loaded data, in MHz. ### npol ``` npol: int ``` Number of polarizations used for calibration, after any override. ### header_npol ``` header_npol: int ``` Number of polarizations declared by the file header. ### polarization_order ``` polarization_order: str | None ``` Polarization ordering declared by the file header, if any. ### freqs ``` freqs: ndarray ``` Channel centre frequencies, in MHz. ### total_time_bins ``` total_time_bins: int ``` Number of time samples in the read window. ### total_channels ``` total_channels: int ``` Number of frequency channels. ### tsamp_ms ``` tsamp_ms: float ``` Sample interval of the loaded data, in milliseconds. ### software_provenance ``` software_provenance() -> dict[str, Any] ``` Snapshot of software identities; unknown historical origins stay unknown. ### from_file ``` from_file(bfile: str, dm: float, telescope: str | None = None, *, sefd_jy: float | None = None, npol_override: int | None = None, read_start_sec: float | None = None, read_end_sec: float | None = None, auto_mask_profile: str | None = 'auto', distance_mpc: float | None = None, redshift: float | None = None, sefd_fractional_uncertainty: float | None = None, distance_fractional_uncertainty: float | None = None, source_ra_deg: float | None = None, source_dec_deg: float | None = None, time_scale: str | None = None, observatory_longitude_deg: float | None = None, observatory_latitude_deg: float | None = None, observatory_height_m: float | None = None, polarization_basis: str | None = None) -> BurstSession ``` Open a burst file and build a session at the given DM. The file's format is detected automatically and a matching telescope preset is applied unless `telescope` names one explicitly. The data is dedispersed to `dm` on load. ### from_snapshot ``` from_snapshot(snapshot: AnalysisSessionSnapshot | dict[str, Any], *, loader: Callable[..., BurstSession] | None = None) -> BurstSession ``` Rebuild a session from a snapshot produced by `to_snapshot`. Reopens the burst the snapshot names and restores every recorded selection. Pass `loader` to control how the burst file is opened -- the web layer uses this to enforce data-directory containment. ### invalidate_results ``` invalidate_results() -> None ``` Discard cached burst measurements. ### clear_width_analysis ``` clear_width_analysis() -> None ``` Discard the cached width analysis. ### clear_dm_optimization ``` clear_dm_optimization() -> None ``` Discard the cached DM sweep result. ### clear_spectral_analysis ``` clear_spectral_analysis() -> None ``` Discard the cached spectral analysis. ### clear_temporal_structure ``` clear_temporal_structure() -> None ``` Discard the cached temporal-structure analysis. ### clear_drift_analysis ``` clear_drift_analysis() -> None ``` Discard the cached sub-burst drift measurement. Component regions and manual peaks feed the centroid estimator, so editing them invalidates a cached drift result even though they leave the other measurements alone. ### clear_polarization ``` clear_polarization() -> None ``` Discard the cached polarization analysis. The loaded Stokes cube is kept: it is expensive to read and stays valid under every selection change, because the session re-applies its own dispersion shift to it on use. ### invalidate_analysis_state ``` invalidate_analysis_state() -> None ``` Discard every cached analysis, after a change that invalidates them all. ### bin_to_ms ``` bin_to_ms(time_bin: int | float) -> float ``` Convert a time-sample index to milliseconds from the start of the window. ### ms_to_bin ``` ms_to_bin(time_ms: float) -> int ``` Convert milliseconds from the start of the window to a time-sample index. ### clamp_bin ``` clamp_bin(value: int, is_end: bool = False) -> int ``` Clamp a time-sample index into the valid range. Pass `is_end=True` for an exclusive end index. ### clamp_channel ``` clamp_channel(value: int) -> int ``` Clamp a channel index into the valid range. ### freq_to_channel ``` freq_to_channel(freq_mhz: float) -> int ``` Return the channel index nearest to a frequency in MHz. ### get_masked_crop ``` get_masked_crop(data: ndarray | None = None) -> np.ndarray ``` Return the cropped dynamic spectrum with masked channels removed. ### get_display_crop ``` get_display_crop(data: ndarray | None = None) -> np.ndarray ``` Return the cropped dynamic spectrum at the current display decimation. ### get_view ``` get_view() -> dict[str, Any] ``` Return the full session state the interface renders, as plain data. ### reset_view ``` reset_view() -> None ``` Reset crop, selections and decimation to their initial state. ### set_time_factor ``` set_time_factor(factor: int) -> None ``` Set the time decimation factor used for display and analysis. ### set_freq_factor ``` set_freq_factor(factor: int) -> None ``` Set the frequency decimation factor used for display and analysis. ### set_crop_ms ``` set_crop_ms(start_ms: float, end_ms: float) -> None ``` Set the crop window, in milliseconds from the start of the read window. ### set_event_ms ``` set_event_ms(start_ms: float, end_ms: float) -> None ``` Set the event window that measurements integrate over. ### add_region_ms ``` add_region_ms(start_ms: float, end_ms: float) -> None ``` Add a burst sub-region, used to separate components. ### clear_regions ``` clear_regions() -> None ``` Remove every burst sub-region. ### add_offpulse_ms ``` add_offpulse_ms(start_ms: float, end_ms: float) -> None ``` Add an off-pulse region, used as the noise reference. ### clear_offpulse ``` clear_offpulse() -> None ``` Remove every off-pulse region. ### add_peak_ms ``` add_peak_ms(time_ms: float) -> None ``` Mark a component peak at the given time, switching to manual peaks. ### remove_peak_ms ``` remove_peak_ms(time_ms: float, tolerance_bins: int = 40) -> None ``` Remove the manually marked peak nearest the given time. ### mask_channel_freq ``` mask_channel_freq(freq_mhz: float) -> None ``` Mask the channel nearest the given frequency. ### mask_range_freq ``` mask_range_freq(low_freq_mhz: float, high_freq_mhz: float) -> None ``` Mask every channel in a frequency range. ### undo_mask ``` undo_mask() -> None ``` Undo the most recent masking step. ### reset_mask ``` reset_mask() -> None ``` Clear the channel mask entirely. ### set_spectral_extent_freq ``` set_spectral_extent_freq(low_freq_mhz: float, high_freq_mhz: float) -> None ``` Set the frequency extent measurements are restricted to. ### set_notes ``` set_notes(notes: str | None) -> None ``` Attach free-text notes, carried through snapshots and exports. ### set_timing_metadata ``` set_timing_metadata(*, source_ra_deg: float | None = None, source_dec_deg: float | None = None, time_scale: str | None = None, observatory_longitude_deg: float | None = None, observatory_latitude_deg: float | None = None, observatory_height_m: float | None = None) -> None ``` Set the source position and observatory location used for barycentring. ### auto_localize ``` auto_localize(*, detection_snr_threshold: float = 6.0, search_window_bins: tuple[int, int] | None = None, apply: bool = True, **kwargs: Any) -> BurstLocalization ``` Automatically localize the burst inside the current crop. Runs the iterative time/frequency search on the masked crop at native resolution and, when a detection is found and `apply` is True, updates the event window, spectral extent, off-pulse windows, and peak selection. The crop itself is unchanged. ### auto_mask ``` auto_mask(profile: str | None = None) -> None ``` Mask interference automatically from statistics of the off-burst samples. Channels whose skewness, or failing that whose standard deviation, stands out from the bandpass are masked; see `flits.analysis.masking`. ### auto_mask_jess ``` auto_mask_jess(profile: str | None = None) -> None ``` Former name of `auto_mask`, from when it called the jess package. ### set_dm ``` set_dm(new_dm: float) -> None ``` Retune the dispersion measure of the loaded data. The shift is computed as an absolute solution relative to the DM the data was loaded at, and only the difference from the currently applied shift is moved. Rounding therefore happens once per target DM rather than once per step, so stepping out along a DM sweep and back returns the data to its original alignment exactly. ### apply_best_dm ``` apply_best_dm() -> DmOptimizationResult ``` Adopt the best DM found by the last sweep and re-dedisperse to it. ### compute_widths ``` compute_widths() -> WidthAnalysisSummary ``` Measure the burst width by every supported method. ### accept_width_result ``` accept_width_result(method: str) -> WidthAnalysisSummary ``` Adopt one width method's result as the reported width. ### optimize_dm ``` optimize_dm(center_dm: float, half_range: float, step: float, metric: str = 'integrated_event_snr', *, structure_cutoff: int | None = None, structure_filter_order: int = DEFAULT_FILTER_ORDER) -> DmOptimizationResult ``` Sweep DM around a centre value and score each trial. `metric` selects the criterion: integrated event signal-to-noise (`integrated_event_snr`), the structure-maximizing DM_phase score (`dm_phase`), or the smoothed-structure score of Sutinjo et al. (2023) (`smoothed_structure`), which also reports a conservative DM interval. For that metric, `structure_cutoff` sets the low-pass cutoff k_c (None chooses it from the DCT spectrum) and `structure_filter_order` the filter order. ### compute_properties ``` compute_properties() -> BurstMeasurements ``` Measure the burst and cache the result. Produces width, peak flux, fluence and signal-to-noise, together with the provenance and uncertainty classification of each quantity. ### fit_model ``` fit_model(config_data: dict[str, Any] | None = None) -> BurstMeasurements ``` Fit the fitburst burst model to the selected event. ### compare_model_components ``` compare_model_components(config_data: dict[str, Any] | None = None, max_components: int = 3) -> BurstMeasurements ``` Fit the selected event with one to `max_components` components and compare the fits. `config_data` takes the free parameters and solver settings of :meth:`fit_model`; each fit starts from the previous one with a component added at the largest residual. The comparison, with AIC, BIC and an F-test for each component count, is stored in `results.diagnostics.model_comparison`. To keep one of the fits, pass its `bestfit_parameters` to :meth:`fit_model` as `initial_parameters`. ### run_temporal_structure_analysis ``` run_temporal_structure_analysis(segment_length_ms: float) -> TemporalStructureResult ``` Run temporal-structure and power-spectral analysis on the selection. ### run_spectral_analysis ``` run_spectral_analysis(segment_length_ms: float) -> SpectralAnalysisResult ``` Run averaged spectral analysis on the selection. ### run_drift_analysis ``` run_drift_analysis(*, settings: DriftAnalysisSettings | None = None, dm_uncertainty_pc_cm3: float | None = None) -> DriftAnalysisResult ``` Measure the sub-burst drift rate of the current selection. Parameters: | Name | Type | Description | Default | | ----------------------- | ----------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `settings` | \`DriftAnalysisSettings | None\` | Fit-region and Monte-Carlo controls. When omitted the session's stored drift_settings are used; when supplied they replace them. | | `dm_uncertainty_pc_cm3` | \`float | None\` | 1-sigma DM uncertainty to fold into the drift uncertainty. It is stored in drift_settings so a snapshot replay reproduces the same classification. When neither this nor the stored value is set, the DM sweep's own uncertainty is used if it still describes the applied DM; pass settings with dm_uncertainty_pc_cm3=None to go back to that fallback. | ### polarization_capability ``` polarization_capability() -> dict[str, Any] ``` Report whether this burst can support an in-session RM measurement. Answers the three questions separately -- does the reader have a full-Stokes path, does the file carry four products, and is it known which four -- because they call for different responses: a different file, a different file, and a stated basis. ### load_stokes_cube ``` load_stokes_cube(*, reload: bool = False) -> np.ndarray ``` Return the burst as a (4, channels, time) Stokes I/Q/U/V cube. The cube is read once, at the DM the file was opened with, and cached. The session's accumulated dispersion shift is applied on every call, so the returned cube is aligned with `data` even after the DM has been retuned -- and retuning does not force a re-read. ### set_polarization_settings ``` set_polarization_settings(settings: PolarizationSettings | dict[str, Any]) -> PolarizationSettings ``` Replace the polarization settings, discarding any stale result. ### run_polarization_analysis ``` run_polarization_analysis(settings: PolarizationSettings | dict[str, Any] | None = None) -> PolarizationAnalysisResult ``` Measure the rotation measure from this session's own selections. Integrates Stokes Q and U over the session's event window, takes the noise from its off-pulse regions, drops the channels its mask and spectral extent exclude, and runs RM synthesis on what is left. Nothing is imported and nothing is re-selected: the RM is measured against the same burst definition as every other number in the session. ### to_snapshot ``` to_snapshot() -> AnalysisSessionSnapshot ``` Capture the full session state as a snapshot object. ### snapshot_dict ``` snapshot_dict() -> dict[str, Any] ``` Capture the full session state as a JSON-compatible dict. ### export_results ``` export_results(*, session_id: str, include: list[str] | tuple[str, ...] | None = None, plot_formats: list[str] | tuple[str, ...] | None = None, window_formats: list[str] | tuple[str, ...] | None = None, window_resolutions: list[str] | tuple[str, ...] | None = None) -> ExportManifest ``` Build an export bundle and retain it for download. ### preview_export_results ``` preview_export_results(*, include: list[str] | tuple[str, ...] | None = None, plot_formats: list[str] | tuple[str, ...] | None = None, window_formats: list[str] | tuple[str, ...] | None = None, window_resolutions: list[str] | tuple[str, ...] | None = None) -> ExportPreview ``` Describe the bundle `export_results` would build, without building it. ### get_export_manifest ``` get_export_manifest(export_id: str) -> ExportManifest ``` Return the manifest of a previously built export bundle. ### get_export_artifact ``` get_export_artifact(export_id: str, artifact_name: str) -> tuple[ExportArtifact, bytes] ``` Return one artifact from a previously built bundle, as (metadata, bytes). ## Headless measurement The one-call form behind `flits measure`; see [Headless Measurement](https://dirkkuiper.github.io/flits/user-guide/headless-measurement/index.md). ## flits.measure Measure bursts without the interface: from a file to numbers in one call. :func:`measure_burst` runs the steps the interface offers, in the order a user takes them: load at a DM, optionally crop, mask interference automatically, localize the burst automatically, then measure. Any step can be pinned instead of automated by giving it explicitly, such as an event window or a frequency band. `flits measure` is the command-line form and handles many files. The result is a plain, JSON-compatible record of what was selected and what was measured. Saving the session alongside it (`snapshot_dir`) keeps every choice, so a burst measured this way can be opened in the interface, checked with `flits replay`, or corrected by hand. ### STATUSES ``` STATUSES = ('ok', 'low_sn', 'no_detection', 'error') ``` Outcomes of a measurement: measured, measured with low S/N, nothing found to measure, or the burst could not be processed. ### MeasureOptions How to measure a burst. Times are in ms from the start of the file, frequencies in MHz. Leaving `event_ms` unset localizes the burst automatically; `offpulse_ms` and `band_mhz` then replace the localized values when given. With `event_ms` set, unset off-pulse windows fall back to everything outside the event and an unset band to the full band. ### measure_burst ``` measure_burst(path: str | Path, options: MeasureOptions, *, snapshot_dir: Path | None = None, overwrite: bool = False, _taken: set[Path] | None = None) -> dict[str, Any] ``` Measure one burst and return a JSON-compatible record of the result. Problems with the input, such as a missing file or an unknown preset, raise; a burst that loads but cannot be found is reported as `no_detection`. With `snapshot_dir`, the session is saved there, without replacing an existing snapshot unless `overwrite` is set. ### measure_bursts ``` measure_bursts(paths: Sequence[str | Path], options: MeasureOptions, *, snapshot_dir: Path | None = None, overwrite: bool = False) -> Iterable[dict[str, Any]] ``` Measure each burst in turn, yielding a record per file. A burst that fails yields an `error` record instead of stopping the rest. ## Input For JSON-compatible metadata and runtime reports, see [Inspection and Runtime](https://dirkkuiper.github.io/flits/user-guide/inspection/index.md). ## flits.inspection.inspect_burst ``` inspect_burst(path: str | Path) -> dict[str, Any] ``` Inspect a file without loading its waterfall or measuring a burst. Raises on missing or unsupported input. The CLI reports such failures per file, so one invalid input does not prevent inspection of the others. ## flits.runtime.runtime_info ``` runtime_info(*, include_paths: bool = False) -> dict[str, Any] ``` Describe this process's code; local interpreter/package paths are opt-in. ## flits.io Input layer: format detection and burst loading. Readers are discovered through `flits.readers` entry points, so a third party can add a format without modifying FLITS. See `flits.io.reader` for the protocol a reader must satisfy. ### inspect_filterbank ``` inspect_filterbank(path: str | Path) -> FilterbankInspection ``` ### load_filterbank_data ``` load_filterbank_data(path: str | Path, config: ObservationConfig, inspection: FilterbankInspection | None = None) -> tuple[np.ndarray, FilterbankMetadata] ``` ### load_stokes_data ``` load_stokes_data(path: str | Path, config: ObservationConfig, inspection: FilterbankInspection | None = None) -> tuple[np.ndarray, FilterbankMetadata] ``` Load a full-Stokes I/Q/U/V cube for `path`. Raises PolarizationUnavailableError if the matching reader has no full-Stokes path, or if the file itself cannot provide one. ### reader_supports_stokes ``` reader_supports_stokes(reader: object) -> bool ``` True if `reader` implements the optional full-Stokes path. ### list_readers ``` list_readers() -> list[BurstReader] ``` ## flits.io.reader.BurstReader Bases: `Protocol` A reader for one or more input formats. Implementations must be cheap to instantiate (zero-arg constructor) because they may be created once per process. All I/O happens inside `inspect()` / `load()` — no state is retained between calls. ### sniff ``` sniff(path: Path) -> bool ``` Return True if this reader can handle the given file. Must be fast (magic-byte / root-attr check only) and must not raise on unrelated files — return False instead. ## flits.io.reader.StokesBurstReader Bases: `BurstReader`, `Protocol` A reader that can additionally return full-Stokes data. `load_stokes` returns `(cube, metadata)` where `cube` has shape `(4, channels, time)` in I/Q/U/V order and shares the time and frequency axes of the Stokes I array `load` returns for the same file and config -- same dedispersion, same read window, same channel order. That equivalence is what lets a session apply its event window, off-pulse regions and channel mask to the cube without re-deriving any of them. Implementations raise :class:`~flits.io.errors.PolarizationUnavailableError` when the file has fewer than four polarization products or when the polarization basis is unknown, rather than guessing. ## flits.io.reader.FilterbankInspection ### stokes_available ``` stokes_available: bool ``` True when this file can yield a full Stokes I/Q/U/V cube. ## flits.stokes Conversion of raw four-product polarization data into Stokes I/Q/U/V. Burst files carry four polarization products, but which four is not something FLITS can infer from the numbers alone. A backend may write Stokes directly, or it may write the coherency products of a linear or a circular feed. The three cases differ only by which pair of products carries the linear polarization, so guessing wrong silently swaps Stokes V into Q and produces an RM that is wrong without looking wrong. This module therefore keeps the basis explicit. Readers resolve it from the file header where the format records it (PSRFITS `POL_TYPE` plus `FD_POLN`), from the telescope preset where the instrument is known, or from an operator override -- and refuse to build a Stokes cube when none of those settle the question. ### normalize_polarization_basis ``` normalize_polarization_basis(value: object) -> str | None ``` Return the canonical basis name for `value`, or None if unset. Raises PolarizationBasisError for a non-empty value that names no known basis, so a typo in a preset or an API request fails loudly. ### basis_from_psrfits ``` basis_from_psrfits(pol_type: object, feed_polarization: object = None) -> str | None ``` Map a PSRFITS `POL_TYPE` (with `FD_POLN`) onto a FLITS basis. `AABBCRCI` names the coherency products without saying which feed produced them, so the receiver's `FD_POLN` decides. A file that declares neither is left unresolved rather than assumed. ### stokes_from_products ``` stokes_from_products(products: ndarray, basis: str) -> np.ndarray ``` Convert four polarization products into a Stokes I/Q/U/V cube. `products` must have the four products on its first axis; the remaining axes are carried through untouched. The returned array is float32 and has the same trailing shape. The coherency conversions follow the PSRFITS/PSRCHIVE convention: a linear feed gives `I=AA+BB, Q=AA-BB, U=2CR, V=2CI` and a circular feed gives `I=AA+BB, V=AA-BB, Q=2CR, U=2CI`. ## Analysis ## flits.analysis.localization.localize_burst ``` localize_burst(masked: ndarray, *, detection_snr_threshold: float = DETECTION_SNR_THRESHOLD, extent_exit_sn: float = EXTENT_EXIT_SN, spectrum_exit_sn: float = SPECTRUM_EXIT_SN, full_band_fraction: float = FULL_BAND_FRACTION, max_iterations: int = MAX_ITERATIONS, max_width_bins: int | None = None, min_offpulse_bins: int = MIN_OFFPULSE_BINS, search_window_bins: tuple[int, int] | None = None) -> BurstLocalization ``` Localize a burst in a masked dynamic spectrum (channels x time). Masked channels must be NaN rows. Returns bins relative to the input array. `search_window_bins` optionally restricts the matched-filter peak location to a known half-open time interval; event boundaries may extend outside it so burst wings are retained. When nothing crosses `detection_snr_threshold` the status is "no_detection". ## flits.measurements.compute_burst_measurements ``` compute_burst_measurements(*, burst_name: str, dm: float, start_mjd: float, read_start_sec: float, crop_start_bin: int, tsamp_ms: float, freqres_mhz: float, freqs_mhz: ndarray, masked: ndarray, event_rel_start: int, event_rel_end: int, spec_lo: int, spec_hi: int, peak_bins_abs: Sequence[int], burst_regions_abs: Sequence[tuple[int, int]], manual_selection: bool, manual_peak_selection: bool, sefd_jy: float | None, npol: int, distance_mpc: float | None, redshift: float | None, sefd_fractional_uncertainty: float | None, distance_fractional_uncertainty: float | None, masked_channels: Sequence[int], offpulse_regions_rel: Sequence[tuple[int, int]] | None = None, noise_settings: NoiseEstimateSettings | None = None, width_results: Sequence[WidthResult] | None = None, accepted_width: AcceptedWidthSelection | None = None, time_axis_ms: ndarray | None = None, timing_context: TimingContext | None = None) -> BurstMeasurements ``` Measure a burst from a prepared selection. Integrates the event window against the off-pulse noise reference and returns width, peak flux, fluence, signal-to-noise and, when a distance is supplied, isotropic energy. Every quantity carries an uncertainty detail saying how it was computed and what it includes. Flux-like uncertainties are 1-sigma radiometer noise (`statistical_only`), combined with the SEFD and distance uncertainties when those are supplied (`formal_1sigma`). Returns: | Type | Description | | ------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `BurstMeasurements` | Measured quantities together with their provenance, uncertainty detail and any diagnostic flags raised during measurement. | ## flits.analysis.dm_optimization.optimize_dm_trials ``` optimize_dm_trials(*, data: ndarray, current_dm: float, freqs_mhz: ndarray, tsamp_sec: float, score_data: ndarray | None = None, score_current_dm: float | None = None, score_freqs_mhz: ndarray | None = None, score_tsamp_sec: float | None = None, center_dm: float, half_range: float, step: float, metric_input_builder: Callable[[ndarray], DMMetricInput], residuals: Callable[[ndarray], tuple[ndarray, ndarray, ndarray, str]], provenance: DmOptimizationProvenance, metric: str = 'integrated_event_snr', structure_cutoff: int | None = None, structure_filter_order: int = DEFAULT_FILTER_ORDER) -> DmOptimizationResult ``` Run a DM trial sweep and return the optimized DM result. Parameters: | Name | Type | Description | Default | | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `data` | `ndarray` | Native-resolution dynamic spectrum with shape (n_channels, n_time). This array is used for residual diagnostics at the applied and best-fit DMs. | *required* | | `current_dm` | `float` | DM currently applied to data, in pc cm^-3. | *required* | | `freqs_mhz` | `ndarray` | Native frequency axis in MHz matching data rows. | *required* | | `tsamp_sec` | `float` | Native sampling interval in seconds. | *required* | | `score_data` | \`ndarray | None\` | Optional waterfall used only for scoring. DMphase passes the current reduced grid here so the score respects the user's analysis resolution. Metrics that should score native data leave this as None. | | `score_current_dm` | \`float | None\` | Applied DM for score_data. Defaults to current_dm. | | `score_freqs_mhz` | \`ndarray | None\` | Frequency axis in MHz matching score_data rows. Defaults to freqs_mhz. | | `score_tsamp_sec` | \`float | None\` | Sampling interval in seconds for score_data. Defaults to tsamp_sec. | | `center_dm` | `float` | Trial grid controls, all in pc cm^-3. | *required* | | `half_range` | `float` | Trial grid controls, all in pc cm^-3. | *required* | | `step` | `float` | Trial grid controls, all in pc cm^-3. | *required* | | `metric_input_builder` | `Callable[[ndarray], DMMetricInput]` | Callback that converts each trial-dedispersed scoring waterfall into a :class:DMMetricInput for the selected metric. | *required* | | `residuals` | `Callable[[ndarray], tuple[ndarray, ndarray, ndarray, str]]` | Callback that computes subband arrival residuals for a waterfall and returns (freqs_mhz, arrivals_ms, residuals_ms, status). | *required* | | `provenance` | `DmOptimizationProvenance` | Serialized context describing the selection and effective analysis resolution used for the sweep. | *required* | | `metric` | `str` | Registered metric key. Supported values are defined by DM_METRIC_REGISTRY. | `'integrated_event_snr'` | | `structure_cutoff` | \`int | None\` | Low-pass cutoff k_c and filter order for the smoothed-structure metric. A cutoff of None is chosen automatically from the DCT spectrum. Ignored by the other metrics. | | `structure_filter_order` | \`int | None\` | Low-pass cutoff k_c and filter order for the smoothed-structure metric. A cutoff of None is chosen automatically from the DCT spectrum. Ignored by the other metrics. | Returns: | Type | Description | | ---------------------- | ---------------------------------------------------------------------------------------------- | | `DmOptimizationResult` | Score curve, sampled and refined DM estimates, residual diagnostics, settings, and provenance. | Raises: | Type | Description | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ValueError` | If the metric key is unsupported, the trial grid is invalid, a metric returns an invalid score shape, or no finite score can be computed for the current selection. | ## flits.analysis.morphology.compute_width_analysis ``` compute_width_analysis(*, selected_profile: ndarray, time_axis_ms: ndarray, event_rel_start: int, event_rel_end: int, tsamp_ms: float, noise_summary: NoiseEstimateSummary, offpulse_bins: ndarray, settings: WidthAnalysisSettings | None = None, event_window_ms: list[float], spectral_extent_mhz: list[float], offpulse_windows_ms: list[list[float]], masked_channels: list[int], effective_bandwidth_mhz: float | None, existing_accepted_method: str | None = None, extra_flags: Sequence[str] = ()) -> WidthAnalysisSummary ``` Compute the full set of morphology-based burst-width measurements. Parameters: | Name | Type | Description | Default | | -------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | `selected_profile` | `ndarray` | Selected-band time profile for the current crop on the same scale used to define the off-pulse baseline. | *required* | | `time_axis_ms` | `ndarray` | Time axis in milliseconds matching selected_profile. | *required* | | `event_rel_start` | `int` | Half-open event window in profile-bin coordinates. | *required* | | `event_rel_end` | `int` | Half-open event window in profile-bin coordinates. | *required* | | `tsamp_ms` | `float` | Effective time resolution of the profile in milliseconds. | *required* | | `noise_summary` | `NoiseEstimateSummary` | Off-pulse noise estimate and warning flags associated with the current selection. | *required* | | `offpulse_bins` | `ndarray` | Off-pulse bins used to bootstrap per-trial baseline/noise estimates. | *required* | | `settings` | \`WidthAnalysisSettings | None\` | Width-analysis settings. If omitted, defaults from :class:WidthAnalysisSettings are used. | | `event_window_ms` | `list[float]` | Event-window bounds in milliseconds for provenance. | *required* | | `spectral_extent_mhz` | `list[float]` | Selected spectral extent in MHz for provenance. | *required* | | `offpulse_windows_ms` | `list[list[float]]` | Off-pulse windows in milliseconds for provenance. | *required* | | `masked_channels` | `list[int]` | Masked channel indices for provenance. | *required* | | `effective_bandwidth_mhz` | \`float | None\` | Effective unmasked bandwidth in MHz, if known. | | `existing_accepted_method` | \`str | None\` | Previously user-accepted width method. If present and still available, it is preserved as the accepted width. | | `extra_flags` | `Sequence[str]` | Additional quality flags propagated to every method result. | `()` | Returns: | Type | Description | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `WidthAnalysisSummary` | Structured width results for all registered methods plus the accepted width selection and the noise summary used to derive uncertainties. | Notes All width values and uncertainties are reported in milliseconds. ## flits.analysis.temporal.core.run_temporal_structure_analysis ``` run_temporal_structure_analysis(*, event_series: ndarray, tsamp_ms: float, segment_length_ms: float, noise_sigma: float, event_window_ms: tuple[float, float], spectral_extent_mhz: tuple[float, float], model_fit_widths_ms: ndarray | None = None, offpulse_series_runs: Sequence[ndarray] | None = None, backend_loader: Callable[[], tuple[type[Any] | None, type[Any] | None, str | None]] = _load_stingray_backend) -> TemporalStructureResult ``` Run temporal-structure analysis on a selected event series. Parameters: | Name | Type | Description | Default | | ---------------------- | -------------------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | | `event_series` | `ndarray` | One-dimensional selected-band event series for the current event window. | *required* | | `tsamp_ms` | `float` | Sampling interval in milliseconds. | *required* | | `segment_length_ms` | `float` | Requested segment length for the averaged PSD, in milliseconds. | *required* | | `noise_sigma` | `float` | Off-pulse noise standard deviation used to normalize matched-filter and wavelet responses. | *required* | | `event_window_ms` | `tuple[float, float]` | Event-window bounds in milliseconds for provenance. | *required* | | `spectral_extent_mhz` | `tuple[float, float]` | Selected spectral extent in MHz for provenance. | *required* | | `model_fit_widths_ms` | \`ndarray | None\` | Optional model-fit component widths in milliseconds. When provided, the smallest positive width is recorded as a model-based cross-check. | | `offpulse_series_runs` | \`Sequence[ndarray] | None\` | Optional contiguous off-pulse time-series runs used to estimate a noise PSD on the same segment grid. | | `backend_loader` | \`Callable\[[], tuple\[type[Any] | None, type[Any] | None, str | Returns: | Type | Description | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `TemporalStructureResult` | Structured result containing the raw periodogram, averaged PSD, minimum-timescale scan products, optional power-law fit, crossover frequency estimate, and optional noise PSD. status is "ok" on success and otherwise one of "insufficient_data", "insufficient_time_bins", "invalid_segment_length", "stingray_unavailable", or "stingray_failed". | Notes The primary minimum-structure estimate comes from the first matched-filter scale whose boxcar or Gaussian response exceeds the familywise-corrected threshold. The wavelet estimate is reported separately from the Mexican-hat scan. ## flits.analysis.spectral.core.run_averaged_spectral_analysis ``` run_averaged_spectral_analysis(*, event_series: ndarray, tsamp_ms: float, segment_length_ms: float, event_window_ms: tuple[float, float], spectral_extent_mhz: tuple[float, float], offpulse_series_runs: Sequence[ndarray] | None = None, backend_loader: Callable[[], tuple[type[Any] | None, type[Any] | None, str | None]] = _load_stingray_backend) -> SpectralAnalysisResult ``` Run averaged spectral analysis on a selected event time series. Parameters: | Name | Type | Description | Default | | ---------------------- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `event_series` | `ndarray` | One-dimensional selected-band event series. The analysis uses the provided samples as a uniformly sampled light curve. | *required* | | `tsamp_ms` | `float` | Sampling interval in milliseconds. | *required* | | `segment_length_ms` | `float` | Requested segment length for the averaged power spectrum, in milliseconds. | *required* | | `event_window_ms` | `tuple[float, float]` | Event-window bounds in milliseconds for provenance. | *required* | | `spectral_extent_mhz` | `tuple[float, float]` | Selected spectral extent in MHz for provenance. | *required* | | `offpulse_series_runs` | \`Sequence[ndarray] | None\` | Optional contiguous off-pulse time-series runs. When provided, FLITS computes a noise PSD using the same segment size and backend. | | `backend_loader` | \`Callable\[[], tuple\[type[Any] | None, type[Any] | None, str | Returns: | Type | Description | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SpectralAnalysisResult` | Structured PSD result. status is "ok" on success and otherwise one of "insufficient_data", "insufficient_time_bins", "invalid_segment_length", "stingray_unavailable", or "stingray_failed". | Notes The returned frequency axis is in Hz, the PSD uses Stingray's `norm="none"` convention, and any fitted power-law parameters describe the averaged event PSD rather than the off-pulse noise estimate. ## flits.analysis.temporal.multiscale.haar_excess_power ``` haar_excess_power(event_profile: ndarray | Sequence[float], offpulse_runs: Sequence[ndarray | Sequence[float]], *, tsamp_ms: float, scales_bins: Sequence[int] | ndarray | None = None, min_coefficients: int = MIN_COEFFICIENTS) -> HaarExcessPowerResult ``` Measure temporal signal power above an empirical off-pulse floor. Parameters: | Name | Type | Description | Default | | ------------------ | ------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `event_profile` | \`ndarray | Sequence[float]\` | One-dimensional on-burst time profile. | | `offpulse_runs` | \`Sequence\[ndarray | Sequence[float]\]\` | Contiguous off-pulse profile sections. Runs are never joined across a gap, so artificial Haar coefficients are not created at boundaries. | | `tsamp_ms` | `float` | Native or effective sampling interval in milliseconds. | *required* | | `scales_bins` | \`Sequence[int] | ndarray | None\` | | `min_coefficients` | `int` | Minimum event and pooled off-pulse coefficient count retained at each scale. | `MIN_COEFFICIENTS` | Notes `characteristic_scale_ms` is the excess-power-weighted geometric mean scale. It is an aggregate descriptor, not a minimum resolvable timescale. Its selection dependence should still be assessed with injection/recovery tests when comparing a burst population. ## flits.analysis.temporal.multiscale.excess_power_fraction_below ``` excess_power_fraction_below(result: HaarExcessPowerResult, max_scale_ms: float) -> float | None ``` Fraction of positive excess power at scales no larger than a limit. ## flits.analysis.drift.run_drift_analysis ``` run_drift_analysis(inputs: DriftAnalysisInputs, settings: DriftAnalysisSettings | None = None) -> DriftAnalysisResult ``` Measure the sub-burst drift rate of the selected event window. Parameters: | Name | Type | Description | Default | | ---------- | ----------------------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | | `inputs` | `DriftAnalysisInputs` | Waterfall, axes, selection bounds and DM state; see :class:DriftAnalysisInputs. | *required* | | `settings` | \`DriftAnalysisSettings | None\` | Fit-region, Monte-Carlo and seeding controls. Defaults are used when omitted. | Returns: | Type | Description | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `DriftAnalysisResult` | status is "ok" when the 2D-ACF fit converged, and otherwise one of "insufficient_time_bins", "insufficient_channels", "insufficient_signal", "invalid_axes" or "fit_failed". The component-centroid estimator reports separately through component_drift_status and never blocks the primary result. | Notes The statistical uncertainty comes from a seeded Monte Carlo: each trial adds an independent noise realisation at the measured per-channel off-pulse sigma and repeats the autocorrelation and the fit. That propagates the correlation between neighbouring autocorrelation pixels, which the fit covariance alone does not; the covariance value is kept alongside as a diagnostic. When a DM uncertainty is supplied it is propagated through the drift/DM sensitivity and combined in quadrature, because the DM term is usually the larger one. ## flits.analysis.drift.DriftAnalysisInputs Everything the drift measurement needs from the session layer. Parameters: | Name | Type | Description | Default | | ----------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | | `waterfall` | `ndarray` | Dedispersed dynamic spectrum on the reduced analysis grid, shaped (n_channels, n_time). Masked channels and samples are NaN. | *required* | | `time_axis_ms` | `ndarray` | Time axis of waterfall in milliseconds. | *required* | | `freqs_mhz` | `ndarray` | Channel centre frequencies in MHz. Either ordering is accepted; the signed channel spacing is what fixes the sign of the drift rate. | *required* | | `event_rel_start` | `int` | Half-open event-window bounds as column indices into waterfall. | *required* | | `event_rel_end` | `int` | Half-open event-window bounds as column indices into waterfall. | *required* | | `spec_lo` | `int` | Inclusive selected-channel bounds as row indices into waterfall. | *required* | | `spec_hi` | `int` | Inclusive selected-channel bounds as row indices into waterfall. | *required* | | `offpulse_bins` | `ndarray` | Column indices used as the off-pulse reference for per-channel baselines and per-channel noise. | *required* | | `component_windows` | `Sequence[tuple[str, tuple[int, int]]]` | (label, (start_bin, end_bin)) pairs for the component-centroid estimator, as half-open column indices into waterfall. | `()` | | `dm_pc_cm3` | `float` | DM the waterfall is dedispersed at, in pc cm^-3. | *required* | | `dm_uncertainty_pc_cm3` | \`float | None\` | 1-sigma DM uncertainty, when one is known. Supplying it adds the DM term to the drift-rate uncertainty, usually its largest part. | ## flits.models.DriftAnalysisSettings Inputs that control the sub-burst drift measurement. ### normalized ``` normalized() -> DriftAnalysisSettings ``` Return settings that are finite and safe to execute or replay. These values can arrive through the actions API or an imported session snapshot. Normalizing them in the shared model keeps the browser, headless replay, and direct Python entry point on the same bounded behaviour. ## flits.models.DriftAnalysisResult Sub-burst drift rate measured from the selected event window. The drift rate is `dnu/dt` in MHz per millisecond: negative for the downward "sad trombone" drift seen in repeaters. `drift_rate_mhz_per_ms` comes from a rotated two-dimensional Gaussian fitted to the mask-corrected 2D autocorrelation of the dynamic spectrum. `component_*` fields hold the independent component-centroid regression, which is only populated when the session defines at least two burst components. ## flits.analysis.drift.time_frequency_slope ``` time_frequency_slope(sigma_time_ms: float, sigma_freq_mhz: float, correlation: float) -> float ``` Return the time-on-frequency regression slope `dt/dnu` in ms/MHz. This is `rho sigma_t / sigma_nu`, not the reciprocal of the drift rate. The two agree only for a perfectly correlated ridge, and it is this one -- not the drift rate -- that a DM error displaces linearly. ## flits.analysis.drift.dm_slope_sensitivity ``` dm_slope_sensitivity(reference_frequency_mhz: float) -> float ``` Return `d(dt/dnu)/d(DM)` in ms/MHz per pc cm^-3. Differentiating the dispersion delay `t(nu) = k DM nu^-2` gives `-2 k nu^-3`. This is the one quantity a DM error moves linearly: a dedispersion error adds `g (nu - nu_ref)` to every arrival time, which shifts the time-on-frequency regression slope by exactly `g` and leaves the frequency spread untouched. ## flits.analysis.drift.drift_dm_sensitivity ``` drift_dm_sensitivity(sigma_time_ms: float, sigma_freq_mhz: float, correlation: float, reference_frequency_mhz: float) -> float ``` Return `d(drift)/d(DM)` in MHz/ms per pc cm^-3. A DM error shears the burst covariance, `Sigma_{t,nu} -> Sigma_{t,nu} + g Sigma_{nu,nu}` and `Sigma_{t,t} -> Sigma_{t,t} + 2 g Sigma_{t,nu} + g^2 Sigma_{nu,nu}`. Differentiating `drift = Sigma_{t,nu} / Sigma_{t,t}` through that shear leaves `(sigma_nu / sigma_t)^2 (1 - 2 rho^2)` times the slope sensitivity. The factor `(1 - 2 rho^2)` is why the naive `d(1/drift)/d(DM)` answer is wrong for anything short of a perfectly correlated ridge: it changes sign at `rho^2 = 1/2`, where the drift rate is momentarily insensitive to the DM. ## flits.analysis.drift.dm_equivalent_of_slope ``` dm_equivalent_of_slope(slope_ms_per_mhz: float, reference_frequency_mhz: float) -> float ``` Return the DM error that would on its own produce this `dt/dnu`. This is the honest statement of the drift/DM degeneracy: if this number is smaller than the DM uncertainty, the measured drift is not distinguishable from a dedispersion error. It is positive when the burst is tilted the way an under-dedispersed burst is, so adding it to the applied DM is what removes the tilt. ## Polarization ## flits.models.PolarizationSettings Operator-controlled inputs to an in-session polarization analysis. `calibration_confirmed` is deliberately not a default-true convenience: an RM measured from uncalibrated Q/U is a number, not a measurement, and the workflow refuses to call it calibrated until someone states that the polarization calibration of this data has been established. ## flits.models.PolarizationAnalysisResult A polarization analysis run against a session's own selections. Holds both halves of the analysis: the channelized Q/U spectrum integrated over the session's event window against its off-pulse regions, and the RM synthesis computed from that spectrum. Keeping them together is what makes the RM reproducible -- the spectrum records which channels survived the mask and the S/N cut, and the provenance records which four products the file held and what established that. ## flits.analysis.polarization.extract_normalized_linear_spectrum ``` extract_normalized_linear_spectrum(*, stokes_iquv: ndarray, freqs_mhz: ndarray, event_bins: tuple[int, int], offpulse_regions: Iterable[tuple[int, int]], channel_mask: ndarray | None = None, spectral_channels: tuple[int, int] | None = None, channel_width_mhz: float | None = None, min_linear_snr: float = 5.0, calibration_status: str = 'unknown') -> IntegratedPolarizationSpectrum ``` Integrate a burst and prepare weighted `Q/L` and `U/L` spectra. Noise is estimated from non-overlapping, event-length integrations within the explicit off-pulse regions. This carries native time correlation into the channel uncertainties more faithfully than dividing a per-sample RMS by `sqrt(N)`. The normalization supports calibrated linear-polarization Q/U fitting without assuming a per-sample white-noise correction. ## flits.analysis.polarization.run_rm_synthesis ``` run_rm_synthesis(*, freqs_mhz: ndarray, stokes_q: ndarray, stokes_u: ndarray, sigma_q: ndarray | float | None = None, sigma_u: ndarray | float | None = None, channel_width_mhz: ndarray | float | None = None, phi_min_rad_m2: float | None = None, phi_max_rad_m2: float | None = None, phi_step_rad_m2: float | None = None, clean: bool = False, clean_gain: float = 0.1, clean_threshold_sigma: float = 3.0, clean_max_iterations: int = 1000) -> RMSynthesisResult ``` Compute a weighted dirty Faraday spectrum and optional RM-CLEAN product. Q and U must be channelized spectra on the supplied frequency centres. If per-channel uncertainties are available they drive both inverse-variance weighting and the propagated Faraday-spectrum noise. Otherwise FLITS uses equal weights and estimates noise from the residual of the best thin model. ## Signal primitives ## flits.signal Signal-processing primitives: dedispersion, normalization and decimation. ### dedisperse ``` dedisperse(data: ndarray, dm: float, freqs_mhz: ndarray, tsamp_sec: float, *, fill_value: float | None = None) -> np.ndarray ``` Dedisperse a dynamic spectrum to the top of the band. Parameters: | Name | Type | Description | Default | | ------------ | --------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `data` | `ndarray` | Dynamic spectrum with shape (n_channels, n_time). | *required* | | `dm` | `float` | Dispersion measure in pc cm^-3, relative to the current dedispersion of data. | *required* | | `freqs_mhz` | `ndarray` | Channel centre frequencies in MHz. | *required* | | `tsamp_sec` | `float` | Sample interval in seconds. | *required* | | `fill_value` | \`float | None\` | What to put in samples vacated by the shift. None (the default) wraps them around the time axis, which is lossless and exactly reversible; that suits repeated in-place retuning of an already-loaded session. Pass a value (readers pass 0.0) for a one-shot dedispersion, where wrapped samples would otherwise fold burst power into the opposite edge of the read window and contaminate the off-pulse statistics computed there. | Returns: | Type | Description | | --------- | --------------------------------------------------------------- | | `ndarray` | The dedispersed dynamic spectrum, same shape and dtype as data. | ### dedispersion_shift_bins ``` dedispersion_shift_bins(dm: float, freqs_mhz: ndarray, tsamp_sec: float) -> np.ndarray ``` Return the per-channel sample shift that dedisperses to the top of band. Shifts are rounded to whole samples, so the achievable DM resolution is limited by the time resolution of the data. `dedispersion_bin_resolution` reports the smallest DM step that moves the band edge by one sample. Parameters: | Name | Type | Description | Default | | ----------- | --------- | ----------------------------------------------------------------------------- | ---------- | | `dm` | `float` | Dispersion measure in pc cm^-3, relative to the current dedispersion of data. | *required* | | `freqs_mhz` | `ndarray` | Channel centre frequencies in MHz. | *required* | | `tsamp_sec` | `float` | Sample interval in seconds. | *required* | Returns: | Type | Description | | --------- | -------------------------------------------------------------------- | | `ndarray` | Integer sample shift for each channel, positive towards later times. | ### dedispersion_edge_bins ``` dedispersion_edge_bins(dm: float, freqs_mhz: ndarray, tsamp_sec: float) -> tuple[int, int] ``` Return how many samples at each end are affected by the dedispersion shift. These are the samples whose content came from outside the read window. They are either wrapped-around data or fill values depending on how `dedisperse` was called, and in both cases they are not valid burst samples. Returns: | Type | Description | | -------------- | ------------------------------------------------------ | | `tuple of int` | (leading, trailing) sample counts to treat as invalid. | ### dedispersion_bin_resolution ``` dedispersion_bin_resolution(freqs_mhz: ndarray, tsamp_sec: float) -> float ``` Return the smallest DM step that shifts the lowest channel by one sample. DM trials finer than this produce bit-identical dedispersed data, so a sweep stepped below it wastes work and reports a spuriously smooth score curve. ### shift_channels ``` shift_channels(data: ndarray, shift_bins: ndarray, *, fill_value: float | None = None) -> np.ndarray ``` Apply a per-channel integer sample shift to a dynamic spectrum. Separated from `dedisperse` so a session can retune its dispersion by the exact difference between two absolute shift solutions, rather than by re-rounding an incremental DM step each time. ### normalize ``` normalize(ds: ndarray, offpulse: ndarray) -> np.ndarray ``` ### normalize_stokes ``` normalize_stokes(cube: ndarray, offpulse: ndarray) -> np.ndarray ``` Baseline and scale a Stokes cube without distorting its polarization. Each Stokes parameter has its own off-pulse median removed, but all four are divided by the *same* per-channel scale, taken from the off-pulse standard deviation of Stokes I. Scaling the four independently would rescale Q/I and U/I channel by channel and corrupt every polarization fraction and every Faraday rotation measured from them; using one scale per channel only flattens the bandpass, exactly as `normalize` does for Stokes I alone. Parameters: | Name | Type | Description | Default | | ---------- | --------- | ---------------------------------------------------------------- | ---------- | | `cube` | `ndarray` | Stokes data with shape (4, n_channels, n_time) in I/Q/U/V order. | *required* | | `offpulse` | `ndarray` | Off-pulse samples with shape (4, n_channels, n_offpulse). | *required* | ### block_reduce_mean ``` block_reduce_mean(arr: ndarray, tfac: int = 1, ffac: int = 1) -> np.ndarray ``` ### radiometer ``` radiometer(tsamp_ms: float, bw_mhz: float, npol: int, sefd_jy: float) -> float ``` ## Configuration ## flits.settings Observation configuration and telescope presets. Presets carry the instrument-specific values -- SEFD, polarization count, observatory location -- that calibrated measurements depend on, and are matched against file headers automatically where possible. ### ObservationConfig ### TelescopePreset ### available_presets ``` available_presets() -> list[TelescopePreset] ``` ### detect_preset ``` detect_preset(telescope_id: int | None, machine_id: int | None, *, telescope_name: str | None = None, schema_version: str | None = None, freq_lo_mhz: float | None = None, freq_hi_mhz: float | None = None) -> tuple[str, str] ``` ### get_preset ``` get_preset(preset_key: str | None = None) -> TelescopePreset ``` ## Command line ## flits.cli Command line entry point for FLITS. `flits` starts the browser interface. `flits demo` opens it on a prepared example burst. `flits measure` measures bursts without a browser. `flits replay` re-runs a saved session snapshot without a browser, which is what makes the reproducibility claim checkable: the snapshot records the selected analysis state, and replaying it recomputes the measurements from the original data file. Invoking `flits` with server options and no subcommand still starts the server, so existing commands and container entrypoints keep working. ### main ``` main(argv: Sequence[str] | None = None) -> int ``` Dispatch a FLITS subcommand, defaulting to the server. ### replay ``` replay(argv: Sequence[str]) -> int ``` Run the `flits replay` command. Returns the process exit code. ### serve ``` serve(argv: Sequence[str]) -> int ``` Run the `flits serve` command (also the default). Returns the exit code. # User guide # Session Workflow FLITS is centered on the current session state. Most analysis products depend on the same crop window, event window, off-pulse definition, spectral window, and mask selection. *The preparation state is shared by the measurement and analysis tabs. Save a snapshot when you want to preserve the selections as well as the results.* ## Typical order of operations 1. Load a filterbank and DM. 1. Define the crop window you want to work on. 1. Mark the event window. 1. Set or refine the off-pulse window if needed. 1. Choose the spectral window used for measurements. 1. Mask problematic channels or ranges. 1. Compute measurements. 1. Run DM optimization, temporal analysis, or fitting on the current state. 1. Export results or save a session snapshot. ## Prepare tab Use the Prepare workflow to set the core analysis state: - crop window - event window - off-pulse region - component regions - peak markers - spectral extent - channel masks This state feeds the later analysis tabs. *The top plot is the time profile, the middle plot is the dynamic spectrum, and the plot at right is the frequency profile. Select an interaction mode above the plots, then click the plot to set its bounds. **Exact Selection** below the toolbar accepts coordinates directly when you need reproducible boundaries.* ## Measurement mindset A good workflow is to make the preparation state explicit before interpreting any numbers. In practice that means: - crop narrowly enough to exclude unrelated structure - define off-pulse bins that represent the local baseline and noise - set the spectral window to the part of the band you actually want to analyze - mask channels with obvious corruption before computing measurements ## DM tab The DM tab evaluates trial DMs around the currently applied DM using the current session state. Use it when you want to compare the current alignment with a locally optimized value or inspect residual arrival-time behavior. ## Fitting tab The fitting workflow is optional and model-based. It uses the currently selected band and event state. Treat the fitted widths and scattering times as secondary diagnostics rather than as replacements for the non-parametric measurement workflow. ## Temporal tab The temporal workflow uses the same current session state as the other analysis tabs. Segment length matters: longer segments improve frequency resolution in the averaged PSD but reduce the number of independent averages. ## Drift tab The drift workflow measures the sub-burst drift rate from the two-dimensional autocorrelation of the selected event window, and cross-checks it against the component centroids when two or more component regions are marked. Read the equivalent DM error before the drift rate: a dedispersion error tilts a burst the same way intrinsic drift does, so enter a DM uncertainty to include that term in the drift-rate uncertainty. See [Sub-Burst Drift](https://dirkkuiper.github.io/flits/analysis/drift/index.md). ## Export tab Build exports only after the session state is where you want it. The export planner and snapshot features are most useful when the selections and masks are already finalized. # Exports and Snapshots FLITS supports both export bundles and full session snapshots. ## Export bundles Use the export workflow when you want shareable outputs such as: - structured JSON results - plots - analysis windows - catalog-style artifacts - array-based products The export preview helps you see what is currently available from the session state before downloading anything. *Choose output types on the left and inspect the exact artifact list on the right. **Build Export Bundle** creates the selected products from the current session state.* ## Session snapshots Session snapshots capture the analysis state itself rather than just the final outputs. They are useful for: - pausing and resuming work - sharing a reproducible interactive state - archiving the exact crop/mask/selection choices used in an analysis Use **Save Session** to write the current session next to the source data in a `snapshots/` folder. If the session was opened from an existing saved snapshot, **Save Session** updates that JSON file. Use **Save Copy** when you want a timestamped copy instead. The **Saved Sessions** browser can search stored snapshots by source, file name, note text, preset, and DM, then reopen a matching session directly. *The Session panel saves or reopens the full interactive state. **Download JSON** makes a portable snapshot; the Export tab above builds derived artifacts.* Snapshots do not embed the raw filterbank data. To share a snapshot with someone else, use **Download JSON** or send the saved JSON file and make sure they start FLITS with `--data-dir` or `FLITS_DATA_DIR` pointing at a directory that contains the same source data file. FLITS records the source file name, data-directory-relative path when available, file size, SHA-256 content hash, and scientific metadata so the import can find moved/copied data and reject the wrong file. ## File longevity and software versions Snapshots and science results use JSON; tables use CSV; diagnostic arrays and selected windows use NumPy NPZ; plots use PNG/SVG; and selected windows can also be exported as SIGPROC `.fil` files with JSON metadata. NPZ products contain numeric and string arrays and can be loaded with `numpy.load(path, allow_pickle=False)`. These products do not depend on pickled Python classes, and stored values can be inspected without running the original FLITS analysis. Reading a stored value and recomputing it are different guarantees. A new FLITS or dependency version can change a calculation even when the input file and selections are identical. JSON or HDF5 alone cannot prevent that. Starting with session schema **1.7** and export schema **1.9**, FLITS captures the FLITS and Python versions, operating system and architecture, installed package versions, and available source identities. Direct Git installs include their recorded commit; source/archive installs include available installer metadata. FLITS source is fingerprinted, with its commit and whether its package files are modified when Git is available. A schema version identifies the file layout, not this environment. The `software_provenance` record separates the environment saving the file (`saved_with`) from the origins of individual `products`. The environment records are keyed by a SHA-256 of their contents. Reopening a session or recomputing core measurements preserves the original identity of an older fit. A failed fit also preserves the provenance of any previously retained fit values. Inputs from other analyses retain their own identities. Older files without provenance are marked `not_recorded`; a later save does not invent their original environment. Replay reports missing or changed environments, and the interface notifies you on import or reopening. Science JSON and export manifests carry the full record. CSV has a quoted `software_provenance_json` column; NPZ has a JSON string array (or includes it in `window_metadata_json`). PNG/SVG store it in their description metadata. SIGPROC files must be kept with their `.meta.json` sidecar. Metadata may be lost if an external image editor rewrites a plot; preserve the original bundle. Capture occurs once per FLITS process, when its first session is constructed. Restart FLITS after changing code or packages. Recorded versions describe installed distributions, not a complete binary/container image; source revisions can be unavailable for wheels or packages without source metadata. The source fingerprint covers Python files, not every native library or data dependency. Local source paths and machine hostnames are omitted; direct URLs have their userinfo, query, and fragment removed. Compatibility tests use unmodified serializers from releases 0.2.0, 0.2.1, 1.0.0, 1.1.0, and 1.2.1. Their stored measurements and selections survive loading. In this regression example, recomputation matches from 0.2.1 onward; 0.2.0 has five known uncertainty-field differences introduced by the 0.2.1 uncertainty-reporting change. Tests also open historical exports with standard readers. These tests cover specific fixtures, not every possible analysis or future release. Schema 1.0 cached results are deliberately invalidated, and unknown future session schemas are rejected before reading the data. ## Recommended practice For an analysis you intend to publish, preserve: - The original input data and the session JSON, including its source checksum. - The export bundle containing the values, arrays, and plots you actually use. - The exact FLITS release or source commit and the analysis environment: Python and dependency versions. For example, run these commands **in the environment that performed the analysis**, and keep their outputs beside the snapshot: ``` python --version > python-version.txt python -m pip freeze --all > environment-requirements.txt flits --version > flits-version.txt ``` For editable or local source installs, also record the commit and preserve any uncommitted changes; a package version or local path alone does not identify that code. Record the operating system and any container image digest used. An environment listing is provenance, not a complete archive of the software. Repository dependency pins document tested environments but do not automatically describe an arbitrary user's installation. Use [headless replay](https://dirkkuiper.github.io/flits/user-guide/headless-replay/index.md) with `--check` to compare the core measurements against their saved values. Replay also recomputes recorded width, drift, and in-session polarization analyses; the drift settings retain the Monte Carlo random seed. It does not rerun optional model fits, DM scans, or temporal/spectral structure analyses. Preserve their stored outputs and their original environment if you need to repeat them. # Uncertainties Every uncertainty FLITS reports carries a short label saying what kind of number it is, a sentence on what it includes, and a note on how it was computed. When a known term is missing, such as the SEFD calibration uncertainty, the explanation says how to add it. ## Kinds of uncertainty | Label in the interface | `classification` | What the number is | | ---------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1σ | `formal_1sigma` | A 1σ uncertainty including every term FLITS knows about, for example a fluence's radiometer noise and SEFD calibration uncertainty combined in quadrature. | | 1σ statistical | `statistical_only` | A 1σ statistical uncertainty: radiometer noise, or the scatter of Monte Carlo trials. A known systematic, such as the SEFD calibration uncertainty, was not supplied and is not included. | | 1σ fit | `model_hessian` | A 1σ uncertainty from a model fit's covariance: burst-model parameters and the PSD power law. It assumes the model describes the data. | | resolution | `resolution_limit` | Half a time or frequency bin, for quantities read off the sampling grid such as the peak-bin arrival time and the ACF widths. It bounds the error; it is not a standard deviation. | | fit width | `heuristic_local_fit` | The width of the fitted peak of a DM sweep's score (integrated S/N or DMphase): how sharply the metric peaks. | | range | `conservative_bound` | The [smoothed-structure DM range](https://dirkkuiper.github.io/flits/analysis/dm-optimization/#smoothed-structure-sutinjo-et-al-2023): the DMs the noise cannot distinguish from the best one. It is wide, several times the scatter of the best DM. | | approximate | `diagnostic_only` | An approximate interval, such as the PSD crossover frequency's 3σ range, or a value from an older session that did not record how it was computed. | The three 1σ kinds also fill the plain uncertainty fields, such as `results.uncertainties.fluence_jyms` and the `*_err` fields of the PSD fit. The others appear in the uncertainty details and the export columns below. ## Quoting a statistical uncertainty A statistical uncertainty is a real 1σ error. In the guided workflow the fluence is 4.032 ± 0.058 Jy ms from radiometer noise, because the example has no SEFD uncertainty. You can quote it as a statistical error and state the calibration uncertainty separately, or supply the SEFD uncertainty so FLITS combines the two. The SEFD term usually dominates. ## Adding a missing term | Quantity | Term | Interface | Python | `flits measure` | | ------------------ | ----------------- | --------------------------------------------------------------- | ------------------------------------------------------------------ | -------------------- | | Peak flux, fluence | SEFD calibration | **SEFD Frac. Uncertainty** under Acquisition Overrides | `sefd_fractional_uncertainty=` in `BurstSession.from_file` | `--sefd-uncertainty` | | Isotropic energy | SEFD and distance | Both fractional uncertainties; distance is under Source Context | `sefd_fractional_uncertainty=`, `distance_fractional_uncertainty=` | – | | Drift rate | DM | **DM uncertainty** in the Drift tab | `run_drift_analysis(dm_uncertainty_pc_cm3=...)` | – | Values are fractional: `0.1` means a 10% SEFD uncertainty. ## Where to find them - **Interface:** the label sits next to each value. Hover over it to see what the uncertainty includes and how it was computed. A "Check" line lists problems with the data that affect it, such as low S/N or heavy masking. - **Python:** `results.uncertainty_details[name]` holds the `value`, `units`, `classification`, `basis` (how it was computed), `tooltip` (what it means), and `warning_flags`. The 1σ kinds also fill `results.uncertainties.`. - **CSV export:** each quantity has `_uncertainty_value`, `_uncertainty_class` and `_uncertainty_basis` columns. - **JSON export and session snapshots:** the full details. ## Older fields Uncertainty details still contain `is_formal_1sigma` and `publishable` so that older sessions and scripts keep working. Both are deprecated: the `classification` and `warning_flags` carry the same information. # Supported Input Formats FLITS reads filterbank-style data through a pluggable reader framework. The built-in readers cover the formats most common in the FRB community: | Format | Extensions | Backend | Notes | | -------------------- | -------------- | ----------------------------- | --------------------------------------------------------------------------- | | SIGPROC | `.fil` | native | Canonical input; 8-, 16- or 32-bit samples, 1, 2 or 4 IFs. | | PSRFITS (search) | `.fits`, `.sf` | native, via `astropy.io.fits` | Search-mode dynamic spectra; 1- to 32-bit samples. | | PSRFITS (folded/PSR) | `.fits`, `.sf` | `astropy.io.fits` | Phase bins are loaded as pseudo-time bins. | | CHIME/FRB HDF5 | `.h5`, `.hdf5` | `h5py` (native) | `flits_chime_v1`, public catalog, and beamformed `BBData` `tiedbeam_power`. | No converter is needed in advance. Just point FLITS at the file: the reader framework detects the format by extension, falling back to magic-byte sniffing for misnamed files. ## Format detection Detection cascade (first match wins): 1. **Trusted extension match.** Unique `.fil` files are accepted directly. 1. **Magic-byte / layout sniff.** Container formats such as `.fits`, `.sf`, `.h5`, and `.hdf5` must still pass a reader-specific sniff/layout check. 1. **Unknown suffix fallback.** If the suffix is unknown, each reader's `sniff()` runs against the file's first bytes or cheap container metadata. 1. **Explicit override.** Pass `format_hint="sigproc"`, `"psrfits"`, or `"chime_hdf5"` to `detect_reader` / `load_filterbank_data` to force a choice. The override is only rejected if FLITS has positive evidence (another reader sniffs the same file as its own) that the hint is wrong. ## Telescope preset detection Once a reader owns the file, FLITS picks a telescope preset (SEFD defaults, auto-mask profile, display label) using a **priority cascade** over whatever metadata the file exposes: 1. **SIGPROC `telescope_id`** — canonical integer tag; strongest signal. 1. **Format / schema signature** — e.g. `chime_frb_catalog_v1` or `flits_chime_v1` directly implies CHIME/FRB. 1. **Telescope-name alias** — strings like `TELESCOP = "MeerKAT"` or `telescope_name = "CHIME"` (whitespace, hyphens, and case are ignored). 1. **SIGPROC `machine_id`** — weak but disambiguating for some setups. 1. **Generic fallback** — no preset claimed the file. Each layer is only consulted if no stronger one matched, and ambiguity at any layer (more than one preset matches) falls through rather than guessing. The UI surfaces which signal was used via the "Detected: …" label and basis. ## PSRFITS caveats - **Folded/PSR-mode** PSRFITS files are supported as a burst-analysis view: FLITS combines subintegrations and treats folded phase bins as pseudo-time bins. It does not perform pulsar timing analysis. - **Search-mode** samples are read as the PSRFITS standard defines them: calibrated as `(raw - ZERO_OFF) * DAT_SCL + DAT_OFFS` for each channel and polarization, multiplied by `DAT_WTS`, read as signed bytes when `SIGNINT = 1`, and, below 8 bits, unpacked with the first sample in the most significant bits. Before FLITS read these files itself, it used the `your` package, which ignores `ZERO_OFF`, `SIGNINT` and the per-polarization scales, wraps scaled 8-bit values above 255, and misorders the channels of 2- and 4-bit data; files affected by any of these now read differently. - If a required header field is absent, a `MetadataMissingError` names the missing fields so you can fix the file. - FLITS promotes common source-position and timing-frame metadata when present: SIGPROC `src_raj`/`src_dej`, PSRFITS `RA`/`DEC` and `RAJ`/`DECJ`, and barycentric/pulsarcentric header flags. Existing barycentric or pulsarcentric inputs are not barycenter-corrected a second time. ## CHIME/FRB HDF5 schema (`flits_chime_v1`) FLITS-written test fixtures and new HDF5 writers should target this layout: **Root attributes** | Attribute | Type | Required | Description | | ---------------- | ------- | -------- | ------------------------------------------------ | | `schema_version` | `str` | no | `"flits_chime_v1"` (or recognized alias). | | `tsamp_s` | `float` | yes | Sample time in seconds. | | `fch1_mhz` | `float` | yes | Frequency of channel 0 in MHz. | | `foff_mhz` | `float` | yes | Channel bandwidth (signed). | | `tstart_mjd` | `float` | yes | Start MJD. | | `nchan` | `int` | no | Declared channel count (cross-checked). | | `npol` | `int` | no | Polarization count (defaults to 1). | | `source_name` | `str` | no | Source or FRB name. | | `source_ra_deg` | `float` | no | ICRS right ascension, for barycentric TOA. | | `source_dec_deg` | `float` | no | ICRS declination, for barycentric TOA. | | `telescope_id` | `int` | no | SIGPROC-style telescope ID. | | `telescope_name` | `str` | no | Free-form telescope name. | | `sefd_jy` | `float` | no | System-equivalent flux density, for calibration. | **Waterfall dataset** Required at `/wfall` (or the alternates `/frb/wfall`, `/intensity`, `/data`) with shape `(nchan, ntime)` and float dtype. Chunked storage is preferred — FLITS reads only the requested time window via HDF5 native slicing, so multi-GB files do not require loading the full array. Alternate attribute names (`dt`, `t_sample`, `fch1`, `foff`, `mjd_start`, `tstart`, `TELESCOP`, `SRC_NAME`, `src_name`) are also recognized, and attributes on a `/frb` subgroup fall back transparently to the root when looked up. This keeps the reader compatible with public CHIME releases that nest metadata differently. **Unsupported schemas** raise `UnsupportedSchemaError`. To add a new schema, register a custom reader via an entry point (see [Custom readers](https://dirkkuiper.github.io/flits/developer/custom-readers/index.md)). ## CHIME/FRB public catalog HDF5 - Public catalog files under `/frb` with `plot_freq`, `extent`, and `calibrated_wfall` / `wfall` are supported directly. - The first two `extent` values are interpreted as milliseconds, matching the CHIME public-catalog plotting convention, and converted to seconds for FLITS sample-time and read-window metadata. - These waterfalls are treated as already dedispersed for FLITS loading. In the UI, the detection step suggests `DM = 0`. - Their dedispersion reference frequency is marked unknown unless an explicit future metadata field says otherwise, so FLITS does not silently invent an infinite-frequency TOA for public catalog files. Catalog flux calibration The reader currently normalizes the waterfall, including `calibrated_wfall`. FLITS flux and fluence then use its SEFD-based radiometer calibration; they are not a direct integration of the catalog's stored Jy scale. CHIME's [calibrated-waterfall tutorial](https://chime-frb-open-data.github.io/waterfall/#plotting-calibrated-data) describes that original scale. Preserve the original data and its calibration metadata when that calibration is required. Do not label a FLITS SEFD-based estimate as a reproduction of the catalog-calibrated fluence. ## CHIME beamformed `BBData` FLITS also supports CHIME beamformed `BBData` containers with: - root attr `__memh5_subclass = "baseband_analysis.core.bbdata.BBData"` - root attr `delta_time` - dataset `tiedbeam_power` - dataset `index_map/freq` - dataset `time0` This iteration supports only the beamformed power product `tiedbeam_power`. Complex-voltage `tiedbeam_baseband` analysis is not yet part of FLITS. The reader: - forms Stokes I from the polarization axis, - applies the fixed per-channel time alignment encoded in `time0`, - uses `tiedbeam_power.attrs["DM_coherent"]` as the coherent-dedispersion reference, - promotes `tiedbeam_locations` RA/Dec when the file describes a single tied-beam position, - then applies **residual** incoherent dedispersion relative to the DM you ask FLITS to use. In practice, this means: - If you want the file loaded at its native coherent-dedispersion DM, use `DM = DM_coherent`. - The UI auto-suggests `DM = DM_coherent` for these files. ## Full-Stokes input `load` returns Stokes I for every reader. A reader that can also produce Stokes I/Q/U/V implements one further method, `load_stokes`, returning a `(4, channels, time)` cube on the same time and frequency grid as the Stokes I array for the same file and config. The SIGPROC/PSRFITS reader implements it; the CHIME HDF5 reader does not. A file yields a cube only when it holds four polarization products *and* the polarization basis is established — by an explicit setting, by the telescope preset, or by the file header (PSRFITS `POL_TYPE` with `FD_POLN`). SIGPROC records nothing about which four products a `nifs=4` file holds, so those need a preset or an explicit basis. Anything else raises `PolarizationUnavailableError`. See [Rotation-Measure Synthesis](https://dirkkuiper.github.io/flits/analysis/rm-synthesis/index.md) for the bases and the in-session workflow. ``` from flits.io import load_stokes_data, reader_supports_stokes, detect_reader from flits.settings import ObservationConfig config = ObservationConfig.from_preset(dm=528.0, preset_key="nrt") inspection = detect_reader("burst.fil").inspect("burst.fil") if inspection.stokes_available: cube, metadata = load_stokes_data("burst.fil", config) ``` `FilterbankInspection` reports `polarization_products`, `polarization_basis`, `polarization_basis_source` and the derived `stokes_available`, so a caller can tell "this file is Stokes I" from "tell FLITS which basis this file uses" without attempting a load. ## Programmatic use ``` from flits.io import detect_reader, load_filterbank_data, reader_diagnostics from flits.settings import ObservationConfig reader = detect_reader("burst.h5") config = ObservationConfig.from_preset(dm=500.0, preset_key="generic") data, metadata = load_filterbank_data("burst.h5", config) # If detection fails or a reader won't load (e.g. missing optional dep), # reader_diagnostics() explains per reader. for entry in reader_diagnostics(): print(entry) ``` ## Error taxonomy All I/O errors derive from `flits.io.errors.FlitsReaderError`: - `UnsupportedFormatError` — no reader recognized the file. - `FormatDetectionError` — `format_hint` conflicts with sniff results. - `CorruptedDataError` — file is recognized but contents are inconsistent. - `MetadataMissingError` — required header fields absent; `.fields` lists them. - `UnsupportedSchemaError` — HDF5 schema version is not known. - `PolarizationUnavailableError` — no Stokes cube can be built; `.reason` is `reader_unsupported`, `insufficient_products` or `unknown_basis`. # Analysis methods # DM Optimization FLITS sweeps trial DMs around the current value, scores each one, and reports the DM that maximizes the chosen criterion. Three criteria are available, and they answer different questions. The smoothed-structure metric also reports a range of DMs derived from the noise; the other two report the width of their fitted score peak. ## How dedispersion works here FLITS dedisperses incoherently to the top of the band. The delay of a channel at frequency ( \\nu ) relative to the reference frequency ( \\nu\_\\mathrm{ref} ) (the highest channel) is [ \\Delta t = k\_\\mathrm{DM} , \\mathrm{DM} \\left( \\nu\_\\mathrm{ref}^{-2} - \\nu^{-2} \\right), \\qquad k\_\\mathrm{DM} = \\frac{1}{2.41 \\times 10^{-4}}\\ \\mathrm{MHz^2,pc^{-1},cm^{3},s} ] using the pulsar-astronomy convention for ( k\_\\mathrm{DM} ) rather than the exact physical constant — see [References](https://dirkkuiper.github.io/flits/references/index.md). Each channel is shifted by **a whole number of samples**. Two consequences follow, and both matter when choosing sweep parameters. ### The DM resolution is set by the time resolution Because shifts are rounded, there is a smallest DM step that moves anything at all: the step that shifts the lowest channel by one sample, [ \\delta\_\\mathrm{DM} = \\frac{t\_\\mathrm{samp}} {k\_\\mathrm{DM} \\left| \\nu\_\\mathrm{ref}^{-2} - \\nu\_\\mathrm{low}^{-2} \\right|} ] Trials spaced more finely than this are **bit-identical**. A sweep stepped below it wastes work and produces a score curve that looks smooth because neighbouring points are the same data, not because the metric is well behaved. `flits.signal.dedispersion_bin_resolution` computes this for a given band and sampling. Use it to pick a step rather than guessing: ``` from flits.signal import dedispersion_bin_resolution step = dedispersion_bin_resolution(session.freqs, session.tsamp) session.optimize_dm(session.dm, half_range=2.0, step=step) ``` Results are tagged with an `integer_bin_dedispersion` warning flag as a reminder that the reported DM cannot be more precise than this. ### The window edges are not valid data Samples shifted in from beyond the read window are not burst data. FLITS zero-fills them when a file is loaded, rather than wrapping them around the time axis where they would contaminate the off-pulse region. The affected extent is reported by `flits.signal.dedispersion_edge_bins`. Keep the event and off-pulse windows away from it; burst localization raises `event_near_edge` when the burst sits too close. ## The metrics ### Integrated-event S/N Sums the dynamic spectrum over the spectral window to a profile, integrates the event window, and divides by the noise estimated from the off-pulse regions. The DM that maximizes this is the DM that makes the burst brightest and narrowest in the summed profile. Use it when the burst is a single smooth component, and when signal-to-noise rather than structure is what you have to work with. It is biased for bursts with downward frequency drift: a drifting burst is brightest when the drift is partly absorbed into the dispersion sweep, so the S/N-maximizing DM is systematically higher than the true one. ### Structure maximization (DMphase) A clean-room implementation of the DM_phase algorithm. Rather than maximizing brightness, it maximizes the *coherence of the burst's Fourier phases across the band*: at the correct DM, structure in different channels lines up in time, and the phases of the corresponding Fourier components agree. Use it when the burst has sharp sub-structure, and whenever drift would bias the S/N metric. This is the criterion generally preferred for repeaters — see Hessels et al. (2019) in the [References](https://dirkkuiper.github.io/flits/references/index.md). It needs structure to work with. On a smooth, low-S/N burst the phase coherence is weak and the score curve is correspondingly flat; the reported fit status will say so. ### Smoothed structure (Sutinjo et al. 2023) A clean-room implementation of the method of Sutinjo et al. (2023). For each trial DM, FLITS sums the selected, unmasked channels over the event window into a light curve, smooths it with a low-pass filter in the discrete cosine transform (DCT) domain, and scores it by the 2-norm of the smoothed curve's derivative: the sharper the aligned burst, the higher the score. Because the score is a norm, the noise left over after smoothing bounds how much it can change. Every trial whose score, raised by that noise allowance, still reaches the best score cannot be distinguished from the best DM (their eq. 15). Those trials span the reported **consistent DM range**. Use it when you need a DM range to quote. It also gives an independent check on DMphase: the two use different statistics, so agreement between them is reassuring. **How wide the range is.** FLITS tested it on simulated bursts with a known DM, rendered with sub-sample dispersion and swept with FLITS's own dedispersion. Across single and multi-component bursts at several S/N levels, the range contained the true DM in 98–100% of cases, and its half-width was about 4 to 7 times the rms error of the best DM. It is a conservative range of consistent DMs: quote it as a range, citing the method. It is labelled "range" (`conservative_bound`) in the uncertainty detail and the export; the plain `best_dm_uncertainty` field holds 1σ errors only, so it stays empty. See [Uncertainties](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md). **The cutoff kc** sets how much of the light curve's DCT spectrum counts as signal. Left blank, FLITS picks it where the spectrum falls to its noise floor (the mean of its upper half). Sutinjo et al. recommend checking this against the spectrum, which is stored as `dct_envelope` in the result. In the simulations the automatic choice gave both the most accurate best DM and the narrowest range: halving kc or doubling it made both worse, doubling much worse. Set it by hand only when the spectrum shows a clearer break. On the [GBT guided workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md) burst, with the guide's selections and a 0.5 step, the metric gives a best residual DM of 5.0 and a consistent range of 4.0 to 6.0. That brackets the DMphase value of 5.41. Requirements and caveats: - The event window must hold at least 16 time samples at the analysis resolution, so the DCT spectrum has a noise region to judge the cutoff from. - Keep the event window close around the burst. Including long stretches of off-pulse data adds noise without structure and widens the range; with a low cutoff it can stretch the range to the edge of the sweep. - The best DM is the best sampled trial, and the range ends on sampled trials, so the step sets their resolution. If only the best trial passes, FLITS reports one step as the range half-width and flags `structure_interval_within_one_step`. Use a step at or above the DM bin resolution described above. - If the range reaches the end of the sweep (`structure_interval_on_sweep_edge`), the true range extends beyond it: widen the sweep. If it has gaps (`structure_interval_has_gaps`), the score curve has more than one acceptable region: inspect the curve. ## What the sweep depends on The optimization runs against the current session state, not the raw file: - the dynamic spectrum at the current DM - the crop and event window - the spectral window - the channel mask - the reduced-resolution (decimation) state Changing any of them can change the preferred DM. This is a feature — masking a band of interference genuinely changes which DM best aligns what remains — but it means a reported DM is only meaningful alongside the state that produced it. That state is what the session snapshot records. ## Reading the result *The DMphase curve peaks near a residual DM of 5.5 on the sampled grid; the weighted fit reports about 5.41. The lower plot compares sub-band arrival-time residuals at the applied and fitted DMs. See the [GBT guided workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md) for the inputs and interpretation limits of this example.* | Output | What to look at | | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Score curve | Should have a clear single maximum. Multiple comparable peaks mean the metric cannot distinguish them. | | Preferred DM and fit status | A parabolic fit near the peak; `peak_on_sweep_edge` means the true optimum is outside the range you swept. | | Residual arrival time | Structure remaining across the band after dedispersion, at both the applied and optimized DMs. | | Consistent DM range | Smoothed structure only: the DMs whose score plus noise allowance (dotted curve) reaches the best score (dashed line), shaded on the plot. A conservative range. | | Warning flags | `integer_bin_dedispersion` always; others as they arise. | ## Practical guidance 1. Mask bad channels first — interference dominates the score otherwise. 1. Restrict the spectral window to the part of the band you trust. 1. Set a sensible event window; the S/N metric integrates exactly what you give it. 1. Choose a step at or above the bin resolution above. 1. If the peak sits at the edge of the sweep, widen the range and re-run. 1. Pick the metric that suits the burst: structure maximization when there is structure or drift, integrated S/N when there is not. Use smoothed structure when you need a DM range to quote, and compare it with DMphase. 1. Treat the result as local to the analysis state, not as a context-free value. # Sub-Burst Drift Repeating FRBs drift downward in frequency: later emission arrives at lower frequencies. FLITS reports that drift as ( \\mathrm{d}\\nu/\\mathrm{d}t ) in **MHz per millisecond**, negative for the usual downward "sad trombone" and positive for the rarer upward drift. Everything on this page runs on the current selection — crop, event window, off-pulse windows, channel mask, spectral window, component regions and applied DM. Changing any of them invalidates the measurement, and FLITS discards the cached result rather than showing a stale one. ## Two estimators, two questions | Estimator | What it measures | What it needs | | -------------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------- | | `acf_2d` | The **sub-burst slope**: how the emission centroid moves in frequency as one component proceeds. | A single event window with enough time bins and channels. | | `component_centroid` | The **inter-component drift**: the step in centre frequency from one sub-burst to the next. | At least two component regions (or two manual peaks). | Editing the component regions or the manual peaks discards a cached drift result, because the second estimator is built from them. They are not the same number, and neither is wrong when they disagree. A burst whose components each drift by the same amount as the gaps between them will give one answer; a burst of separately drifting sub-bursts stacked at a different overall slope will give two. Report which one you mean. The autocorrelation estimator is the primary result and populates `drift_rate_mhz_per_ms`. The component estimator is reported separately through `component_drift_rate_mhz_per_ms` and never blocks it. ## How the autocorrelation estimator works 1. Each channel's off-pulse mean is subtracted, and the event window is cut out of the selected band. 1. The two-dimensional autocorrelation is computed by FFT and **corrected for the channel mask**: each lag is divided by the fraction of valid channel pairs it actually had, relative to what an unmasked window would have given. With nothing masked this is exactly the conventional autocorrelation; with channels zapped it removes the notch pattern the gaps would otherwise imprint on the surface. The finite-window taper is deliberately left alone, because a burst localised inside its window is not a stationary process. 1. A rotated two-dimensional Gaussian is fitted to the central peak, over the lags inside `max_lag_fraction` of the full range that survive `min_overlap_fraction` of their possible channel pairs. The zero-lag pixel is excluded, because it carries the noise variance as a delta function. FLITS refuses a fit whose width consumes 80% or more of either fitted lag span, because that width is being set by the boundary rather than measured from the peak; increase the lag fraction or improve the selections in that case. 1. The drift rate is the **conditional-mean slope** of the fitted ellipse, ( \\Sigma\_{\\nu t} / \\Sigma\_{tt} ) — the rate at which the centre frequency moves with time. Fitting for a covariance rather than an angle is what makes the reported number well defined. The slope of the ellipse's *major axis* is a different quantity that depends on how you scale the two axes against each other; FLITS reports it too, as `acf_major_axis_slope_mhz_per_ms`, because that is the convention `frbgui` uses and comparisons need it — but it is a diagnostic, not the drift rate. The two converge only for a very elongated ellipse. The fitted widths also give the burst extent, after dividing out the factor ( \\sqrt{2} ) between a Gaussian and its autocorrelation: `acf_sigma_time_ms` and `acf_sigma_freq_mhz`. ## Where the uncertainty comes from The statistical term is a **seeded Monte Carlo**, not the fit covariance. Each trial adds an independent noise realisation at the measured per-channel off-pulse sigma, recomputes the autocorrelation and refits; the scaled median absolute deviation of the refitted drift rates is the 1σ statistical error. This is deliberate. Neighbouring autocorrelation pixels are strongly correlated, so the covariance of a least-squares fit that treats them as independent understates the error, sometimes by a large factor. The covariance value is still recorded, as `drift_rate_fit_covariance_mhz_per_ms`, so the two can be compared — but it is a diagnostic. The Monte Carlo assumes the estimator responds to an added noise realisation the way it responds to the noise already present, which holds while the burst dominates the autocorrelation peak. When it does not, the `low_acf_contrast` flag is raised. The seed is stored in the session snapshot, so `flits replay` reproduces the error bar exactly rather than approximately. ## Drift and DM are degenerate This is the part that matters for anything you publish. A dispersion-measure error tilts a burst in the time-frequency plane in exactly the way intrinsic drift does. Over a band narrow enough for the sweep to look linear, nothing in the dynamic spectrum distinguishes the two. A drifting burst therefore has no single "correct" DM: maximising structure and maximising S/N give different answers, and both are defensible. FLITS does not resolve this. It states it, in four numbers: | Field | Meaning | | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `dm_pc_cm3` | The DM the measurement was made at. A drift rate without this is not interpretable. | | `acf_slope_ms_per_mhz` | The time-on-frequency slope of the fitted ellipse. This is the quantity a DM error moves linearly. | | `dm_equivalent_pc_cm3` | The DM offset that would, on its own, produce the whole measured slope. Adding it to the applied DM flattens the burst. | | `dm_sensitivity_mhz_per_ms_per_pc_cm3` | How much the drift rate moves per pc cm⁻³ of DM error. | The one to read first is `dm_equivalent_pc_cm3`. **If it is smaller than your DM uncertainty, the drift is not resolved** — the burst is equally well described as undrifting at a slightly different DM. FLITS raises `drift_consistent_with_dm_error` and says so in the result message when that is the case. The component estimator reports its own `component_dm_equivalent_pc_cm3` on the same footing. ### Why the slope and not the drift rate A dedispersion error of ( \\delta ) adds ( g,(\\nu - \\nu\_\\mathrm{ref}) ) to every arrival time, with [ g = \\frac{\\partial}{\\partial \\mathrm{DM}} \\left(\\frac{\\mathrm{d}t}{\\mathrm{d}\\nu}\\right)\\delta = -\\frac{2k}{\\nu^{3}},\\delta , ] which is a shear of the burst covariance in the time-frequency plane. Under that shear ( \\Sigma\_{\\nu\\nu} ) is untouched and ( \\Sigma\_{t\\nu} ) picks up ( g,\\Sigma\_{\\nu\\nu} ), so the **time-on-frequency** regression slope [ \\frac{\\mathrm{d}t}{\\mathrm{d}\\nu} = \\frac{\\Sigma\_{t\\nu}}{\\Sigma\_{\\nu\\nu}} = \\rho,\\frac{\\sigma_t}{\\sigma\_\\nu} ] simply shifts by ( g ). The drift rate is the *other* regression, ( \\Sigma\_{t\\nu} / \\Sigma\_{tt} = \\rho,\\sigma\_\\nu/\\sigma_t ), and the two are reciprocals only when ( \\rho = 1 ): in general their product is ( \\rho^{2} ). Treating ( 1/(\\mathrm{d}\\nu/\\mathrm{d}t) ) as the DM-linear quantity is a mistake that gets both the size and, near ( \\rho^2 = 1/2 ), the sign of the DM sensitivity wrong. FLITS therefore converts through the slope: [ \\delta\_\\mathrm{equiv} = -\\frac{\\nu^{3}}{2k}, \\frac{\\mathrm{d}t}{\\mathrm{d}\\nu}, \\qquad \\frac{\\partial}{\\partial \\mathrm{DM}} \\left(\\frac{\\mathrm{d}\\nu}{\\mathrm{d}t}\\right) = -\\frac{2k}{\\nu^{3}} \\left(\\frac{\\sigma\_\\nu}{\\sigma_t}\\right)^{2} \\left(1 - 2\\rho^{2}\\right), ] evaluated at the mean frequency of the unmasked selected channels. ## The drift-rate uncertainty The drift-rate uncertainty is described the same way as every other [uncertainty](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md). - **Without a DM uncertainty** it is the 1σ statistical uncertainty from the Monte Carlo (`statistical_only`), with the `missing_dm_uncertainty` flag. The DM term, which usually dominates, is not included. - **With a DM uncertainty** the DM term is propagated through the sensitivity above and combined in quadrature with the statistical term (`formal_1sigma`). `drift_rate_status` reports significance against the bar FLITS would actually quote, not against the statistical term alone: `ok` when the drift rate exceeds its combined uncertainty, `unconstrained` when it does not, and `unquantified` when there is no uncertainty at all — which is what running with the Monte Carlo disabled and no DM uncertainty gives you. Check these flags before quoting a drift rate: `heavily_masked` (a quarter or more of the selected channels are gone), `low_acf_contrast`, `monte_carlo_unavailable`, and `implicit_offpulse` (no explicit off-pulse window, so the noise reference is a guess). Monte Carlo requests are bounded to 512 trials in the browser, API, and replay path so an imported snapshot cannot accidentally turn one measurement into an unbounded computation. Non-finite controls fall back to their recorded defaults, and random seeds are normalized to the non-negative range accepted by NumPy. The DM uncertainty comes from the DM sweep when the sweep still describes the applied DM; retune the DM by hand and FLITS drops it rather than reusing a number that now describes something else. Supply one explicitly to override. An operator-supplied value is stored in the drift settings and travels in the session snapshot, so `flits replay` reproduces the same classification and not just the same number. Clear the field to go back to the sweep's own value. ## The component estimator Each component window contributes an intensity-weighted spectral centroid and an intensity-weighted arrival time, with uncertainties propagated from the per-channel off-pulse noise. Centroid frequency is then regressed on arrival time with weights ( 1/(\\sigma\_\\nu^2 + d^2\\sigma_t^2) ), iterated so the effective variance uses the fitted slope. With exactly two components the fit is exact: the status is `exactly_two_components` and no ( R^2 ) is reported, because there is no residual to explain. Overlapping components bias their centroids toward each other, which flattens the regression; the sign survives but the magnitude is a lower bound. ## Practical sequence 1. Dedisperse and record how. The drift rate is meaningless without the DM it was measured at, and a structure-maximised DM and an S/N-maximised DM will give different drift rates for the same burst. 1. Mask interference and set the spectral window. The autocorrelation is mask-corrected, but a band that is mostly gone still gives a weak fit. 1. Set the event window to the burst and the off-pulse windows to genuinely burst-free data — the per-channel sigma from those windows is what sets the error bar. 1. Mark component regions if the burst has more than one, to get the second estimator. 1. Run the measurement, then read `dm_equivalent_pc_cm3` before the drift rate. 1. Supply a DM uncertainty before quoting the number anywhere. ## Related - [DM Optimization](https://dirkkuiper.github.io/flits/analysis/dm-optimization/index.md) — where the DM and its uncertainty come from, and why the structure-maximising metric partly addresses the degeneracy. - [Measurements](https://dirkkuiper.github.io/flits/analysis/measurements/index.md) — how FLITS classifies uncertainties in general. - [Model Fitting](https://dirkkuiper.github.io/flits/analysis/model-fitting/index.md) — the burst-model fit measures the scattering timescale, which is a different asymmetry from drift and is easy to confuse with it in a single component. # Automatic Burst Localization The **Auto Localize** button (and the `auto_localize` session action) finds where the burst lives in the current crop and sets the event window, the spectral window, and the off-pulse windows in one step. It replaces the first, most error-prone part of the manual workflow; every selection it makes can still be refined by hand afterwards. ## What it does 1. Per-channel robust normalization (median/MAD computed off-event). 1. Matched filtering of the band-integrated S/N profile over a ladder of boxcar widths to find the burst and its characteristic width. 1. The event window is the contiguous above-noise region around the matched peak, unioned with any nearby independently detected components, with the boundaries refined at a finer smoothing scale so they land at the true signal edge rather than the matched-filter shoulder. 1. The spectral window comes from the on-burst channel S/N over the tight event window: the contiguous significant band around the strongest channel, merged with nearby significant sub-bands (so patchy scintillation stays inside one window). If the result covers ≳95% of the usable band the burst is treated as broadband and the full band is kept. 1. The time and frequency searches iterate — a band-limited burst is re-detected against the noise of its own sub-band, which typically raises the integrated S/N substantially — until the selections are stable. 1. Off-pulse windows are placed on both sides of the event with a guard margin. ## Constraining a triggered search For snippets with a known approximate arrival time, pass a half-open `search_window_bins=(start, end)` interval to `localize_burst` or `BurstSession.auto_localize`. The matched-filter peak must fall inside that interval, which prevents a stronger unrelated event or RFI feature elsewhere in the crop from being selected. The measured event window is allowed to extend outside the search interval so broad wings and nearby components are not clipped. The search interval is recorded in `BurstLocalization` and its serialized dictionary. It is a prior on *where to search*, not a replacement for the detection threshold: a sub-threshold candidate still returns `no_detection`. ## Reading the result The toast (and the `localization` field in the action response) reports: - `status` — `ok`, `low_sn` (detected, but integrated S/N below the threshold), or `no_detection` (nothing crossed the detection threshold; the session selections are left unchanged). - `detection_snr` / `integrated_snr` — matched-filter and event-integrated S/N over the selected band. - `band_limited` — whether a sub-band was selected. - `warning_flags` — `event_near_edge`, `wide_event`, `band_touches_edge`, `low_integrated_snr`, `no_offpulse_window`. Treat flagged selections as candidates for manual review. ## Notes - Localization runs at the native time resolution of the crop, regardless of the current display reduction. - Measurements are still computed at the session's time/frequency factors; for sub-millisecond structure work set the time factor to 1 before `Compute`. - The detection threshold defaults to S/N 6 and can be passed in the action payload (`detection_snr_threshold`). # Measurements FLITS reports measurements from the current prepared session state. *The Prepare tab groups selection readiness, headline measurements, timing details, and uncertainty flags. These values come from the [GBT guided workflow](https://dirkkuiper.github.io/flits/guided-workflow/index.md); its absolute TOA fields are not part of the validated reference calculation.* ## Core outputs The measurement workflow can report: - fluence in Jy ms, when an SEFD is available - peak flux density in Jy, when an SEFD is available - event duration in ms - spectral extent in MHz - peak-bin topocentric/source-header TOA (`toa_peak_topo_mjd`) - infinite-frequency topocentric TOA (`toa_inf_topo_mjd`) when FLITS knows the dedispersion reference frequency - barycentric infinite-frequency TDB TOA (`toa_inf_bary_mjd_tdb`) when source position, observatory location, and time-scale metadata are complete - one-dimensional Gaussian fits for selected burst regions ## Calibration note Flux and fluence require calibration information. If FLITS can identify a known observing setup, it may provide a default SEFD. Otherwise use the generic preset or specify an SEFD explicitly. Without an SEFD, FLITS can still report selection- and timing-related outputs, but not calibrated flux-density values. ## Uncertainties Every reported quantity carries its uncertainty together with what that uncertainty includes. Flux-like uncertainties always include radiometer noise, as a 1σ statistical error (`statistical_only`). Supply the fractional uncertainty on the SEFD and FLITS combines the two in quadrature into a complete 1σ uncertainty (`formal_1sigma`): ``` session = BurstSession.from_file( "burst.fil", dm=527.65, sefd_jy=10.0, sefd_fractional_uncertainty=0.1, # 10% SEFD uncertainty ) ``` Without it, the uncertainty is the radiometer noise alone. That is a genuine 1σ statistical error, but the SEFD term usually dominates, so quote it as statistical and state the calibration uncertainty separately, or supply the SEFD uncertainty. [Uncertainties](https://dirkkuiper.github.io/flits/user-guide/uncertainties/index.md) describes every kind of uncertainty FLITS reports. Each quantity also records the *basis* of its uncertainty in words: which noise reference was used, and which terms were combined. That text is carried into exports, so a number in a table can be traced back to how it was derived without rerunning anything. Distance-dependent quantities behave the same way: isotropic energy includes the distance term once `distance_fractional_uncertainty` is supplied as well. Check the flags before quoting a number `measurement_flags` records what was missing or suspect — `missing_sefd`, `missing_sefd_fractional_uncertainty`, `edge_clipped` and others. They are the fastest way to see whether a value is ready to use. ## Provenance matters Measurements depend on: - the event window - manual peaks, when they fall inside the selected event window - the off-pulse definition - the selected spectral extent - the applied channel mask - the current DM and reduced analysis state - timing metadata: source position, observatory location, time scale, time reference frame, and dedispersion reference frequency Interpret exported values together with the stored provenance, not as context-free numbers. ## TOA Timing Chain FLITS V1 still uses a peak-bin estimator for TOA. The primary value is the time of the strongest event-window bin, or a manual peak only when that manual peak is inside the current event window. `toa_topo_mjd` and `mjd_at_peak` are retained as compatibility aliases for `toa_peak_topo_mjd`. The infinite-frequency correction subtracts the cold-plasma dispersion delay from the finite dedispersion reference frequency to infinity: `delay_ms = 1000 * DM / (2.41e-4) * reference_frequency_mhz^-2` This value is reported as `dispersion_to_infinite_frequency_ms`. The correction is only applied when FLITS can identify a trustworthy reference frequency. For ordinary FLITS integer-bin dedispersion this is the highest channel frequency. Public CHIME catalog waterfalls are treated conservatively: their public file format does not by itself prove the reference frequency, so FLITS leaves the infinite-frequency TOA unavailable unless stronger metadata are supplied. The barycentric value uses Astropy `Time`, `EarthLocation`, and `SkyCoord` to add the Solar-System barycentric light-travel-time correction to the infinite- frequency topocentric TOA, then reports the result in TDB. It requires: - source RA and Dec in ICRS decimal degrees - a complete observatory longitude, latitude, and height - a supported input time scale: UTC, TDB, TT, or TAI - topocentric input times If a header says the data are already barycentric or pulsarcentric, FLITS does not apply another barycentric correction. The measurement payload records `toa_status` and `toa_status_reason` so exports show whether the chain stopped at the peak-bin value, reached infinite-frequency topocentric timing, or reached barycentric TDB timing. These corrected TOAs are useful for consistent reporting, but they are still resolution-limited because the V1 estimator is the selected time bin, not a centroid, template, or likelihood-model TOA with a formal statistical uncertainty. ## Event duration versus burst width The event duration is the manually selected event-window span. It is useful session metadata, but it is not automatically the same thing as a model-independent burst-duration estimate or the same thing as a fitted component width. Width methods normally run the configured off-pulse Monte Carlo uncertainty trials. Large population pipelines that calibrate uncertainties end-to-end can set `WidthAnalysisSettings(uncertainty_trials=0)` to compute the width values without this per-burst bootstrap. Such results carry the `uncertainty_disabled` quality flag and do not report a width uncertainty. # Model Fitting FLITS exposes a selected-event 2D model-fitting workflow in the Fitting tab. It uses the live session state: crop, event window, frequency window, off-pulse regions, masks, display downsampling, and component annotations. FLITS does not mirror command-line preprocessing flags from the underlying solver because those choices are already represented by the interactive session. ## The model FLITS fits the dynamic-spectrum model of Fonseca et al. (2024), the model the `fitburst` package implements. Each component is a Gaussian, convolved with a one-sided exponential when the scattering timescale is positive, with a running power-law spectrum. DM, DM index, scattering timescale, and scattering index are shared by all components. FLITS evaluates the model itself, so fitting needs no extra package. The implementation is written from the paper's equations, evaluates every channel and component at once, and has analytic derivatives. Parameter names, units, and solution JSON follow `fitburst`, so a solution moves between the two programs unchanged. The test suite compares FLITS with `fitburst` in every model mode: time and frequency upsampling, folded profiles, scintillation, an incoherent DM, and data that are not dedispersed. On synthetic bursts fitted from the same starting point, FLITS reaches `fitburst`'s minimum to within 0.001σ, its uncertainties agree to within 0.02%, and the fit runs five to eight times faster. A scintillation-mode fit runs about 50 times faster. Where FLITS differs from `fitburst`: - **Bounds.** Widths and the scattering timescale cannot go negative. When the scattering timescale ends at zero, the fit lists it in `parameters_at_bounds` and its uncertainty is one-sided: quote the value plus the uncertainty as an upper limit. - **Unresolved scattering.** In `fitburst`'s profile normalization the amplitude grows without bound as the scattering timescale shrinks. The solver steps the amplitude of the Gaussian that the profile tends to instead, so a fit of an unscattered burst converges to the Gaussian fit. Results are reported in `fitburst`'s convention. - **Covariance.** Like `fitburst`, FLITS inverts the observed Hessian of chi-squared. If that matrix is not positive definite, it inverts the Gauss-Newton matrix instead, and `covariance_basis` records which one it used. The full covariance matrix and its parameter labels are stored with the fit. - **Scintillation mode.** Each channel's amplitudes are solved by linear least squares at every step, and the derivatives include how those amplitudes move with the other parameters (variable projection; Golub & Pereyra 1973). In 40 simulated scintillating bursts, the scatter of FLITS's best fits matches its reported uncertainties to within 7%. In the same test, the scatter is 1.5 to 4 times larger than the uncertainties `fitburst` reports. - **Fit statistics** add AIC and BIC, and in scintillation mode the number of per-channel amplitudes, to `fitburst`'s chi-squared, reduced chi-squared, and S/N. - **Before each scattered component**, `fitburst` holds the profile at a floor of about 10⁻⁵ of its peak. FLITS evaluates the profile as written. Before fitting, FLITS puts the selected band in off-pulse S/N units: it subtracts each channel's off-pulse mean and divides the whole band by the median off-pulse noise of its channels. This assumes a bandpass-flattened spectrum, which is what FLITS's readers produce. Per-channel noise enters through the weights, not through the scaling. The Fitting tab can display initial guesses and previous results before a fit has run. When a fit cannot be constrained, FLITS stores a structured failure diagnostic instead of raising an error. ## Fit parameters The primary control is **Parameters to Fit**. Checked parameters are varied by the optimizer; unchecked parameters are held fixed at their initial values. The available optimizer parameters are amplitude, arrival time, intrinsic width, scattering timescale, residual DM, DM index, scattering index, spectral index, and spectral running. The default interactive setup varies amplitude, arrival time, intrinsic width, and scattering timescale while holding spectrum, DM, and index terms fixed. This is the conservative default for measuring model width and scattering time from the current selection. Internally FLITS derives the fixed-parameter list from the submitted free parameters before calling the solver. ## Initial parameters FLITS seeds the model from current component regions, manual peak markers, or a single automatic component from the integrated profile. The **Detect Components** helper can populate editable component rows from profile peaks. Initial parameters include per-component amplitude, arrival time, burst width, spectral index, spectral running, and reference frequency. Shared model values include DM, DM index, scattering timescale, and scattering index. The **Seed from Previous Fit** option initializes a run from the previous successful model fit in the same session when the component count and current event window are compatible. The advanced setup can also import a fit-compatible solution JSON containing `model_parameters`. ## Fit window The fit covers the event window plus padding on each side, so the baseline around the burst and any scattering tail beyond the event window are part of the fit. By default the padding is the event window's own length, at least 8 bins, and it shrinks if the window would exceed the interactive size limit. **Fit Window Padding** sets the number of bins instead; 0 fits the event window alone. Fit Details reports the window that was fitted. ## Time resolution and upsampling The data in each time bin are an average over the bin, so the model has to be averaged the same way. Evaluated only at bin centres, a component narrower than a bin needs a wider Gaussian to put flux into the neighbouring bins, and its fitted width comes out too large: on synthetic 0.3 ms bursts in 1 ms bins, fits at bin centres return about 0.4 ms, while upsampled fits recover 0.3 ms. **Time Upsampling** is the number of sub-samples per bin that the model is averaged over. Left blank (auto), FLITS fits once at the data resolution and, if the narrowest fitted width is under three bins, refits from that solution with enough sub-samples that each is at most a third of that width (up to 16, fewer on very large windows). Fit Details shows the factor used and whether it was chosen automatically. When the fitted width or scattering timescale is shorter than one time bin, the results say so. An upsampled fit is constrained by how the burst's flux spreads into the neighbouring bins, so expect large uncertainties or an upper limit. A fit without upsampling offers to fit again with automatic upsampling. ## Solver setup Advanced solver controls include weighting mode, optional manual weight range, optimizer passes, maximum function evaluations, time and frequency upsampling, fit-window padding, reference frequency, folded profile mode, exact-Jacobian mode, and scintillation mode. The reference frequency, at which the scattering timescale and the spectrum are quoted, defaults to the lowest frequency of the selected band. **Optimizer Passes** restarts the solver from the previous best fit for each pass. **Max Evaluations** controls the model-evaluation budget per pass; if left blank, FLITS allows at least 1,000 evaluations, or 100 times the number of fitted values when that is larger. Correlated width, scattering and DM parameters can need several hundred evaluations even for a small model. This is a maximum, so a converged fit still stops early. An explicit value overrides Auto, including a smaller budget. When a pass reaches its limit, the result remains `evaluation_limit_exceeded`: its width, scattering time and uncertainties are not accepted measurements. The best parameters reached are retained in `bestfit_parameters`, including in saved sessions. Increase the limit and select **Seed from Previous Fit** to continue from that candidate instead of the original guesses. The Python equivalent is `session.fit_model({"seed_from_previous_fit": True, "solver": {"max_function_evaluations": 2000}})`. The resumed fit must converge before its measurements are accepted. A larger budget cannot resolve an unconstrained model or an inappropriate selection; inspect the residuals and parameter correlations after convergence. **Fit Details → Fitter** identifies the implementation that produced the saved result. Older sessions can retain `fitburst` results; rerun the fit in the current FLITS environment to compute a new native result. Restart a server after upgrading its environment, and check that it imports the intended checkout when running from source. Weighting modes are: - **None**: unweighted residuals. - **Auto off-pulse**: derive per-channel weights from current off-pulse bins when possible, otherwise fall back to fit-window weighting. - **Fit window**: estimate each channel's weight from its RMS over the fitted window. - **Manual range**: estimate it over a range of time bins counted from the start of the event window. In scintillation mode, amplitude, spectral index, and spectral running are not optimizer parameters because per-channel amplitudes are derived from the data. ## Reading the results The results start with the fit status, then the intrinsic width, the scattering timescale at the reference frequency, and the reduced chi-squared. Notes follow when something needs attention: - **A value consistent with zero.** When the width or the scattering timescale ends at its lower bound of zero, only an upper limit is constrained. It is shown as `< value + 1σ`. - **Shorter than one time bin.** See [Time resolution and upsampling](#time-resolution-and-upsampling). - **Strongly correlated parameters** (|ρ| ≥ 0.8). Each 1σ uncertainty already allows for the other parameters adjusting, but a quantity that combines two correlated parameters needs their covariance, not the separate uncertainties. Correlations with the amplitude are left out of the note: in `fitburst`'s profile normalization the amplitude scales with the scattering timescale, so the two are always strongly correlated. The parameter table lists each best-fit value with its uncertainty, or says `fixed`, `upper limit`, or, in scintillation mode, `per channel`. **Fit Details** holds the fitted window, time resolution, upsampling, weighting, solver counts, AIC and BIC, which covariance was inverted, and the full correlation matrix. ## Comparing component counts **Compare Component Counts** fits the selection with one component, then two, up to the chosen maximum (at most 5). Each fit starts from the one before it with a component added at the largest positive residual of the event profile. All the fits use the same samples, weights and time upsampling, so their chi-squared values compare directly. The table reports, for each component count, the reduced chi-squared, ΔBIC and ΔAIC relative to the best, and the F-test p-value against one component fewer: the probability of an improvement that large if the extra component were not needed. The fit with the lowest BIC is marked. **Use This Fit** refits that row from its solution and makes it the session's model fit. A component the data do not support often fails to converge: nothing stops it shrinking onto a noise spike. The comparison then stops there and compares the fits that converged. Treat BIC and AIC as guidance. At high S/N, components are rarely exactly Gaussian, so extra components keep lowering chi-squared even when they do not describe separate emission; check the residuals before adopting one. From Python: ``` results = session.compare_model_components(max_components=3) comparison = results.diagnostics.model_comparison best = next(row for row in comparison.rows if row.num_components == comparison.preferred_components) session.fit_model({"num_components": best.num_components, "initial_parameters": best.bestfit_parameters}) ``` ## Diagnostics and exports Model-fit results report free and fixed parameters, initialization source, solver settings, best-fit parameters, uncertainties, residual dynamic spectra, profile residuals, fit statistics, and sanitized solver diagnostics on failure. Exports include fit-compatible solution JSON using `model_parameters`, `fit_statistics`, and solver logistics. Measurement outputs still populate `width_ms_model` and `tau_sc_ms` when those best-fit values exist. Treat model-derived width, scattering time, parameter uncertainties, and chi-squared values as model-based diagnostics. They are useful cross-checks, while non-parametric FLITS measurements, DM diagnostics, and temporal structure analysis remain the primary session measurements. # Rotation-Measure Synthesis FLITS performs weighted one-dimensional rotation-measure (RM) synthesis on a channelized complex linear-polarization spectrum, `P = Q + iU`. It retains the dirty Faraday dispersion function (FDF), the complex rotation-measure spread function (RMSF), and—when requested—a separate RM-CLEAN reconstruction. The implementation follows [Brentjens & de Bruyn (2005)](https://doi.org/10.1051/0004-6361:20052990). RM-CLEAN follows the one-dimensional deconvolution described by [Heald, Braun & Edmonds (2009)](https://doi.org/10.1051/0004-6361/200811532). There are two ways in: - **From the session**, when the burst file carries four polarization products. FLITS builds the Stokes cube, integrates Q and U over the session's own event window, takes the channel uncertainties from its off-pulse regions, and drops the channels its mask and spectral extent exclude. The result is stored on the session, so it travels in snapshots, export bundles and `flits replay` like every other analysis. - **From an imported spectrum**, when Q and U were prepared elsewhere. This path is unchanged: import a calibrated channelized Q/U spectrum as JSON or CSV. ## Measuring the RM from the session ### What the file has to carry A burst file can only yield a Stokes cube if it holds four polarization products *and* something establishes which four they are. FLITS never guesses: reading coherency products as though they were Stokes swaps V into Q and produces an RM that is wrong without looking wrong. The basis is resolved in this order: 1. an explicit `polarization_basis` on the session (the **Polarization basis** control, or the `polarization_basis` argument to `BurstSession.from_file`); 1. the telescope preset, for instruments whose headers are known to misreport this — NRT filterbanks carry linear-feed coherency products while the header claims `IQUV`; 1. the file header: PSRFITS `POL_TYPE` together with `FD_POLN`. SIGPROC records nothing about which products a `nifs=4` file holds, so a filterbank always needs a preset or an explicit basis. If none of the three settles it, FLITS refuses with `unknown_basis` rather than picking one. The three recognized bases are: | Basis | Products | Stokes | | -------------------- | ----------------------------------- | -------------------------------------- | | `iquv` | I, Q, U, V | used as-is | | `coherency_linear` | AA, BB, CR, CI from a linear feed | `I=AA+BB`, `Q=AA-BB`, `U=2CR`, `V=2CI` | | `coherency_circular` | AA, BB, CR, CI from a circular feed | `I=AA+BB`, `V=AA-BB`, `Q=2CR`, `U=2CI` | ### Browser workflow Open **Polarization** in the Analysis Workspace and use **Measure From This Session**. The status line reports how many polarization products the file has and which basis was established, or what is missing. Set the minimum channel linear-polarization S/N, confirm the calibration checkbox if the polarization calibration of this dataset has been established, and click **Measure RM From Session**. The measurement uses the session's current selections, so refine the event window, the off-pulse regions, the channel mask and the spectral extent first — changing any of them discards the previous result rather than leaving a stale one on screen. ### Python usage ``` from flits.session import BurstSession session = BurstSession.from_file("burst.fil", dm=528.0) session.set_event_ms(97.4, 104.4) session.add_offpulse_ms(32.0, 82.0) session.add_offpulse_ms(120.0, 1200.0) result = session.run_polarization_analysis({"min_linear_snr": 3.0}) print(result.peak_rm_rad_m2, result.peak_rm_uncertainty_rad_m2) print(result.linear_fraction, result.circular_fraction) ``` `result.rm_synthesis` holds the full RM-synthesis payload documented below. The result also records the basis and what established it, the exact event and off-pulse windows, the channels that survived, and the integrated Stokes I, Q, U and V spectra — which supports replay from the snapshot and original data. ### Reading the cube directly ``` from flits.io import load_stokes_data from flits.settings import ObservationConfig config = ObservationConfig.from_preset(dm=528.0, preset_key="nrt") cube, metadata = load_stokes_data("burst.fil", config) # (4, channels, time) ``` `load_stokes_data` returns the same time and frequency grid as `load_filterbank_data` for the same file and config, so a cube can be used alongside the Stokes-I dynamic spectrum without re-deriving anything. Readers that cannot produce a cube raise `PolarizationUnavailableError` with a `reason` of `reader_unsupported`, `insufficient_products` or `unknown_basis`. Each Stokes parameter has its own off-pulse baseline removed, but all four are divided by a single per-channel scale taken from Stokes I. Scaling them independently would rescale Q/I and U/I channel by channel and corrupt every polarization fraction measured from them. ## Prepare an imported spectrum RM synthesis needs a calibrated Q/U spectrum, not a Stokes-I dynamic spectrum. Before importing data: 1. remove the Q and U off-pulse baselines; 1. apply the instrument's polarization and leakage calibration; 1. flag unusable channels consistently in Q and U; 1. integrate the selected burst window into one Q and U value per channel; 1. estimate `sigma_q` and `sigma_u` from suitable off-pulse data; and 1. retain the true channel width for the bandwidth-depolarization limit. Frequency values are channel centres in MHz. Q, U, and their uncertainties must use the same units. FLITS can run without uncertainties, but then uses equal weights and estimates the noise from the residual of the best Faraday-thin component. This is less reliable for complex sources. ## Browser workflow for an imported spectrum Open **Polarization** in the Analysis Workspace: 1. Click **Import JSON or CSV**. The analysis runs once immediately with the displayed defaults when the input declares `calibration_status: calibrated`. Otherwise confirm **Polarization calibration applied** after checking the dataset provenance; FLITS deliberately will not run before that confirmation. 1. Review the detected channel count, frequency range, uncertainty status, and channel-width status. 1. Adjust the Faraday-depth bounds or step when the automatic coverage is not appropriate. Blank fields use the wavelength-coverage defaults. 1. Leave **Apply RM-CLEAN** enabled when RMSF sidelobes need deconvolution. The dirty spectrum is always retained. 1. Click **Run RM Synthesis** after changing a setting. 1. Inspect both plots: the FDF/RMSF plot and Q/U with the best Faraday-thin model. A high reduced chi-square indicates that one component, the noise model, or the calibration does not describe the data. 1. Download the complete result as JSON and the sampled FDF as CSV. The browser accepts either column-oriented JSON: ``` { "freqs_mhz": [900.0, 901.0, 902.0], "stokes_q": [0.12, 0.08, -0.01], "stokes_u": [-0.03, 0.09, 0.13], "sigma_q": 0.01, "sigma_u": 0.01, "channel_width_mhz": 1.0, "calibration_status": "calibrated" } ``` or a CSV table. Short aliases such as `frequency_mhz`, `q`, `u`, `q_err`, and `u_err` are recognized: ``` frequency_mhz,q,u,q_err,u_err,channel_width_mhz 900.0,0.12,-0.03,0.01,0.01,1.0 901.0,0.08,0.09,0.01,0.01,1.0 902.0,-0.01,0.13,0.01,0.01,1.0 ``` At least eight usable rows are required. The short examples above show the format only and are not sufficient to run an analysis. ## Python and API usage ``` from flits.analysis import run_rm_synthesis result = run_rm_synthesis( freqs_mhz=freqs_mhz, stokes_q=stokes_q, stokes_u=stokes_u, sigma_q=sigma_q, sigma_u=sigma_u, channel_width_mhz=channel_width_mhz, phi_min_rad_m2=-5000, phi_max_rad_m2=5000, phi_step_rad_m2=1, clean=True, clean_gain=0.1, clean_threshold_sigma=3.0, clean_max_iterations=1000, ) if result.status != "ok": raise ValueError(result.message) ``` The same fields are accepted by `POST /api/rm-synthesis`. Uncertainties and channel widths may be scalars or arrays with one value per channel. Invalid settings return a structured result status from the Python function; request schema errors return the usual HTTP validation response. ## Automatic sampling and reported limits By default FLITS: - searches from `-max_abs_rm_rad_m2` to `+max_abs_rm_rad_m2`; - samples the FDF at five points per theoretical RMSF FWHM; - computes the maximum observable absolute RM from the widest channel in wavelength-squared space; - measures the RMSF FWHM from the actual flagged, weighted channel set (while also retaining the ideal uniform-coverage estimate); - uses the supplied channel widths, or infers local widths from channel-centre spacing and emits `channel_width_inferred`; and - evaluates large transforms in bounded chunks. Grids above 100,001 points are rejected with guidance to narrow the range or increase the step. The peak position is refined between grid samples with a three-point parabolic fit to FDF power. The nominal Faraday-thin uncertainty is `RMSF FWHM / (2 × peak S/N)`. When uncertainties are supplied, FLITS uses inverse complex-variance weights, propagates the Faraday noise, and reports the reduced chi-square of the best single thin-component Q/U model. Otherwise it reports residual-MAD noise and does not claim a chi-square. The global false-alarm probability is an approximation based on Gaussian Q/U noise and the number of independent RMSF-sized trials. Real calibration residuals and non-Gaussian noise can make it too optimistic. [George, Stil & Keller (2012)](https://doi.org/10.1071/AS11027) show why an 8-sigma threshold is a safer default for broad RM searches than lower thresholds. ## Interpreting the output Key fields include: - `peak_rm_rad_m2` and `peak_rm_uncertainty_rad_m2`; - `peak_snr` and approximate `false_alarm_probability`; - measured and debiased peak polarized amplitudes; - polarization angle at the weighted reference wavelength squared and the extrapolated zero-wavelength angle; - RMSF FWHM and maximum sidelobe; - effective channel count and maximum single-channel weight, which expose fragile results dominated by a small part of the band; - largest recoverable Faraday scale and maximum observable absolute RM; - dirty complex FDF, complex RMSF, restored RM-CLEAN FDF, and CLEAN components; and - machine-readable warnings for every important caveat. A Faraday-depth peak is not automatically an intrinsic source RM FLITS does not calibrate instrumental leakage or position angle, and it does not calculate an ionospheric correction. RM-CLEAN removes RMSF sidelobes; it does not fix calibration errors, bandwidth depolarization, a poor noise model, or unresolved Faraday complexity. Apply those corrections upstream and preserve them in the analysis provenance before publishing a source RM. ## Reproducible RM-Tools validation example The checked-in worked example uses the official synthetic one-dimensional validation spectrum from [CIRADA RM-Tools](https://github.com/CIRADA-Tools/RM-Tools). It is not an observation or a claim about an astrophysical source. It is a controlled, analysis-ready Q/U dataset with a reference result from an independent implementation, which makes it suitable for verifying the entire FLITS path. The raw source and reference values are distributed like the GBT guided-workflow file: as versioned assets on the FLITS `tutorial-data-v1` GitHub release. To download explicit working copies: ``` mkdir -p tutorial-data curl -L -o tutorial-data/flits-tutorial-rmtools-reference-v1.dat \ https://github.com/DirkKuiper/flits/releases/download/tutorial-data-v1/flits-tutorial-rmtools-reference-v1.dat curl -L -o tutorial-data/flits-tutorial-rmtools-reference-values-v1.json \ https://github.com/DirkKuiper/flits/releases/download/tutorial-data-v1/flits-tutorial-rmtools-reference-values-v1.json ``` Run the validation from the repository root with those files: ``` python examples/rmtools_reference_workflow.py \ --source tutorial-data/flits-tutorial-rmtools-reference-v1.dat \ --reference tutorial-data/flits-tutorial-rmtools-reference-values-v1.json ``` With no file arguments, the script first looks under the ignored local `data/RM-Tools` directory and otherwise downloads the same release assets. The script performs the complete reproducibility chain: 1. reads local tutorial files or downloads their versioned FLITS release assets; 1. rejects either file unless its SHA-256 checksum matches; 1. parses all 288 frequency, Q, U, and uncertainty samples; 1. writes [browser-importable JSON](https://dirkkuiper.github.io/flits/examples/rmtools-reference-input.json); 1. runs weighted RM synthesis and RM-CLEAN in FLITS; 1. compares RM, peak amplitude, S/N, and channel count with the official reference; and 1. fails instead of writing a passing summary when any tolerance is exceeded. The source covers 800–1088 MHz and is distributed by RM-Tools under the [MIT license](https://dirkkuiper.github.io/flits/examples/rmtools-LICENSE.txt). The pinned source URL, commit, checksum, upstream URL, FLITS distribution URL, and license URL are retained inside both generated JSON artifacts. The data is marked `calibrated` for the browser gate because it is a synthetic validation product with fully specified Q/U uncertainties; instrument calibration is not applicable. The [machine-readable validation summary](https://dirkkuiper.github.io/flits/examples/rmtools-reference-summary.json) records the following independently checked result: | Quantity | RM-Tools reference | FLITS | Absolute difference | | ------------------------ | ------------------ | ----------- | ------------------- | | Peak RM (rad m⁻²) | `200.29004` | `200.28831` | `0.00173` | | RM uncertainty (rad m⁻²) | `0.24806` | `0.25212` | — | | Peak amplitude | `0.699674` | `0.699678` | `0.000004` | | Peak S/N | `118.73860` | `118.73935` | `0.00075` | | Channels | `288` | `288` | `0` | The FLITS–RM-Tools RM difference is about 0.7% of the quoted 1σ uncertainty and well inside the executable example's 0.5 rad m⁻² acceptance limit. The example also verifies amplitude to 0.001 and S/N to 0.1. Tests rerun the checked-in browser input through FLITS, so documentation cannot silently drift away from the implementation. # Noise-subtracted multiscale temporal power `haar_excess_power` measures how a burst profile's temporal power is distributed across timescales, with the noise contribution removed empirically rather than assumed. It exists because the obvious statistic — "the smallest significant timescale in this burst" — is not comparable between bursts. That quantity is a detection threshold, so it moves with signal-to-noise: the same burst observed twice as brightly appears to contain finer structure. Any population trend measured that way is partly a trend in S/N. ## What it computes At each scale ( s ) (a window width in samples), the estimator forms Haar differences between the means of adjacent windows, at every offset in the event window: [ d_s(i) = \\frac{1}{s}\\sum\_{j=i}^{i+s-1} x_j ;-; \\frac{1}{s}\\sum\_{j=i+s}^{i+2s-1} x_j ] The power at that scale is ( \\langle d_s^2 \\rangle ) over all offsets. A Haar difference responds to change on its own scale and cancels anything constant across the pair, so this decomposes the profile by how fast it varies. The same calculation runs over each contiguous off-pulse run, and the pooled off-pulse power at each scale is subtracted from the event power: [ P\_\\mathrm{excess}(s) = P\_\\mathrm{event}(s) - P\_\\mathrm{off}(s) ] Noise contributes power at every scale, so subtracting a measured off-pulse spectrum rather than a white-noise model removes both the radiometer term and whatever correlated noise the off-pulse region contains. ## What comes back `HaarExcessPowerResult` holds the event power, the noise power, the signed excess and the positive normalized excess at every retained scale, plus two summaries: | Field | Meaning | | ------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `characteristic_scale_ms` | Geometric mean scale, weighted by positive excess power. A single number describing where the burst's variability sits. | | `dominant_scale_ms` | The single scale carrying the most positive excess power. | `excess_power_fraction_below` integrates the normalized excess power at scales no larger than a given one — "what fraction of this burst's variability is faster than 1 ms". ``` from flits.analysis import excess_power_fraction_below, haar_excess_power result = haar_excess_power( event_profile, [offpulse_before, offpulse_after], tsamp_ms=0.016, ) print(result.characteristic_scale_ms, result.dominant_scale_ms) print(excess_power_fraction_below(result, 1.0)) ``` ## Using it across a population The estimator is amplitude-weighted and noise-subtracted, which removes the first-order S/N dependence. It does not remove selection effects, and several remain: - **Event-window selection.** The window sets the largest measurable scale. Choose it consistently. - **Finite resolution.** No instrument reports power below its own sample time. When comparing instruments, restrict to the range of scales *all* of them resolve — otherwise the finer-sampled instrument appears to contain more fast structure by construction. - **Scattering.** Scattering suppresses fine structure genuinely, but it is a propagation effect, not an emission property. - **Band occupancy** and **low S/N** both alter the recovered distribution. Two rules follow. Choose the comparison scale for the population, not per burst — otherwise you are comparing different questions. And calibrate with end-to-end injections plus bright bursts degraded to a common S/N, so the size of the residual bias is measured rather than hoped away. ## Relationship to the other temporal diagnostics Use this rather than the minimum-structure scan in [Temporal and Spectral Analysis](https://dirkkuiper.github.io/flits/analysis/temporal-spectral/index.md) whenever the comparison is between bursts of unequal brightness. The scan answers "what is the finest structure I can demonstrate in *this* burst", which is the right question for a single burst and the wrong one for a population. # Temporal and Spectral Analysis These diagnostics all run on the **event profile**: the dynamic spectrum summed over the current spectral window and restricted to the current event window, with masked channels excluded. Changing the mask, the spectral window or the event window changes every result on this page. ## What is computed | Diagnostic | Question it answers | | ------------------------------- | -------------------------------------------------------------------------------- | | Autocorrelation (ACF) | On what timescale does the profile resemble a shifted copy of itself? | | Raw periodogram | How is variance distributed over fluctuation frequency, unaveraged? | | Averaged PSD | The same, with segments averaged to reduce variance. | | Minimum-structure scale scan | What is the finest structure that is statistically significant in this burst? | | Power-law plus constant PSD fit | Does the variability follow a power law, and where does it meet the noise floor? | | Crossover frequency | The fluctuation frequency where the fitted power law meets the constant term. | ## Choosing a segment length Segment length is the main control, and it is a direct trade-off. The event window of length ( T ) split into segments of length ( T\_\\mathrm{seg} ) gives frequency resolution ( \\Delta f = 1/T\_\\mathrm{seg} ) and roughly ( T / T\_\\mathrm{seg} ) independent averages. - **Longer segments**: finer frequency resolution, noisier estimate at each frequency. - **Shorter segments**: smoother, more reliable PSD, but coarser in frequency and blind to fluctuations slower than one segment. There is no universally correct choice — it depends on the event length and the timescales you care about. `default_segment_bins` picks a conservative default from the event length; override it when you have a specific timescale in mind. A practical check: if the PSD changes character substantially when you halve the segment length, the feature you are looking at is not yet well constrained. ## Interpreting the scale scan The minimum-structure scan reports **the smallest structure that is statistically significant in the current profile**. Two cautions: - It is not the burst duration, and not a physical timescale of the source. It is a detection threshold, so it depends on signal-to-noise. A brighter observation of the same burst will report finer structure. - If the response is still rising at the largest tested scale, that means either broad emission or no turnover inside the tested range — not that the burst has no characteristic scale. For comparing bursts of unequal brightness, use the [noise-subtracted multiscale estimator](https://dirkkuiper.github.io/flits/analysis/temporal-multiscale/index.md) instead, which is built to remove exactly this S/N dependence. ## ACF widths are not durations *The dashed line marks normalized ACF = 0.5. Its intersections with the curves give self-similarity widths in time and frequency for the [GBT example](https://dirkkuiper.github.io/flits/guided-workflow/index.md), not the manually selected event duration.* An ACF width is a self-similarity scale. It is a good way to compare coherence structure between bursts or between time and frequency, but it is not interchangeable with the full burst duration, and the two can differ by a large factor for a multi-component burst. Report which one you mean. ## The PSD fit is deliberately simple The model is a power law plus a constant: [ P(f) = A f^{-\\alpha} + C ] with ( C ) representing the white-noise floor. This is enough to characterise whether variability is scale-free over the fitted range and where it disappears into the noise. It is not a physical model of the emission. When the fit is poorly constrained — a short event, low S/N, or a PSD with curvature the model cannot represent — **the stored fit status matters more than the nominal parameter values**. FLITS records that status alongside the parameters precisely so a spectral index is not quoted from a fit that did not converge. Check it before using ( \\alpha ) for anything. The crossover frequency is reported only when the fit is well enough constrained for it to mean something. ## Practical sequence 1. Mask interference and set the spectral window before running anything here — every diagnostic is computed on what those leave behind. 1. Set the event window to the burst, and the off-pulse regions to genuinely burst-free data used as the noise reference. 1. Start with the default segment length, then vary it to check stability. 1. Read the fit status before the fit parameters. 1. For population work, prefer the multiscale estimator and calibrate with injections.