Narrative Feature Extraction
Computational annotation of movies and stories for cognitive science and naturalistic neuroimaging.
This site is the entry point for browsing every part of the project: the Phase 1 scoping review, the annotation format specification and pipeline, the how-to guides for growing the corpus, the draft review paper with its figures, and the interactive segment browser.
What the project does
A human supplies a movie or an audio/text story; the pipeline returns a hierarchical, semantically organized, second-by-second set of annotations — visual, audio, language, social, situational, and affective — produced by a curated set of best-in-class models. Features that do not apply to a stimulus (e.g. visual features for an audio-only story) are returned as explicit nulls, so every annotation shares one constant shape and the corpus stacks into rectangular matrices for analysis.
At a glance
- 83 stimuli annotated (~7.8 h): 53 audiovisual, 29 audio-only, 1 text-only.
- 95 channels per stimulus across six feature classes, on a common 1 Hz grid.
- Local-first models on Apple-Silicon GPU/CPU; HDF5 + JSON output with a MATLAB reader.
How this book is organized
| Part | Contents |
|---|---|
| Overview | Project overview and the full contents & user guide. |
| Phase 1 — Scoping review | Survey of computational annotation tools per feature class, the semantic hierarchy, redundancy analysis, and best-in-class recommendations. |
| Phase 2 — Pipeline & format | The annotation format spec, the frozen feature set, and the extractors that run. |
| Phase 3 — Corpus | How the corpus was assembled and how to add your own movies/stories. |
| Phase 4 — Analysis & dissemination | Corpus analysis and tools, the review paper, and the interactive browser. |
Start here
- Review paper (draft) — the models/algorithms behind each annotation and the empirical structure of the annotation space across the corpus.
- Contents & user guide — the full map of tools, datasets, derivatives, and how to load / view / inspect them (with MATLAB recipes).
- Interactive segment browser — rank stimulus segments by any combination of features.
The source repository holds the pipeline (
src/nfe/), the MATLAB reader (matlab/), and the annotated corpus derivatives. This documentation book is generated from the canonical Markdown indocs/bytools/build_book.py.