Language features

Low-level lexical & word-level features

Per-word lexical/psycholinguistic annotation is a mature, mostly CPU-friendly pipeline problem rather than a deep-learning frontier. The backbone is a fast tokenizer/POS/NER/dependency engine (spaCy) that segments the transcript and tags each word, followed by table lookups against well-established norm databases (SUBTLEX/wordfreq for frequency; Brysbaert for concreteness; Kuperman for AoA; Warriner/NRC-VAD for valence/arousal/dominance; NRC-EmoLex/LIWC for affect categories). The one genuinely model-dependent feature is word-level surprisal/entropy, computed from an incremental LLM (GPT-2 through larger autoregressive models) via the minicons or surprisal packages. For time-alignment, all of these are per-token features that inherit timestamps from forced alignment of the transcript to the audio (e.g. WhisperX/Montreal Forced Aligner) — that alignment step is upstream and out of scope here but is the mechanism that turns per-word features into a per-second movie timeline. The main design decisions for a small lab are: (1) which norm databases to license/merge, (2) how to handle out-of-vocabulary words and lemmatization for lookups, and (3) which LLM to use for surprisal given the size/fit tradeoff.

Recommended (best-in-class): spaCy (en_core_web_trf/lg) as the tokenizer/POS/NER/dependency backbone; wordfreq + SUBTLEX-US for frequency; Brysbaert concreteness + Kuperman AoA + Warriner/NRC-VAD norm joins for psycholinguistic per-word features; NRC EmoLex for discrete emotion channels; minicons (or surprisal) with GPT-2-medium/Pythia for word surprisal & entropy

Likely redundant:

• NLTK — superseded by spaCy for tokenization/POS/NER (keep only for WordNet synset access)
• surprisal package — direct overlap with minicons; pick one surprisal library
• ANEW (1,034 words) — superseded by Warriner VAD (13,915) and NRC-VAD (20,000)
• MRC Psycholinguistic Database — legacy imageability/concreteness, superseded by Brysbaert and Glasgow norms
• LIWC-22 affect categories — most per-word affect content is reproducible with free NRC EmoLex + Warriner VAD + spaCy POS; LIWC is justified mainly for its validated pronoun/person-reference and function-word category scheme
• Glasgow Norms — overlaps Brysbaert/Kuperman/Warriner on the same dimensions at smaller coverage; useful only as a single-source merge or for its imageability/familiarity/semantic-size/gender scales
• VADER — sentence/social-media-tuned sentiment, redundant with Warriner+NRC for per-word valence

Open questions:

• Time-alignment is upstream: per-word features are only as good as the forced alignment (WhisperX / Montreal Forced Aligner) that assigns each token a start/end timestamp — confirm which aligner the pipeline uses before binning to per-second.
• Lemmatization vs surface-form lookup: norm databases key on lemmas/base forms while surprisal and frequency can be surface-form; need a consistent OOV and morphology strategy (e.g. fall back lemma->surface->subword).
• Whether to merge Warriner and NRC-VAD (and with what priority) for valence/arousal/dominance, vs picking one to avoid double-counting correlated features.
• Choice of surprisal LM: smaller GPT-2/Pythia models often fit human reading-time/neural data better than the largest LLMs — decide whether the goal is best psycholinguistic fit (small model) or best raw language modeling (large model).
• Coverage/OOV handling for proper nouns and disfluencies in spoken-narrative transcripts, which are common in movies/audio stories but absent from norm databases.
• LLM-generated norm estimates (2024-2026 work) could fill OOV gaps for concreteness/valence/AoA, but require validation/fine-tuning against the human tables before use.

Syntactic & grammatical structure

Sentence/utterance-level syntactic annotation for movie/audio narratives rests on a mature stack: a few NLP pipelines (spaCy, Stanza, Trankit) emit dependency parses, POS/morphology (tense/aspect/mood as UD feats), and lemmas per token; dedicated constituency parsers (benepar, supar, Stanza/CoreNLP conparse) yield phrase-structure trees from which complexity metrics (tree depth, Yngve/Frazier scores, L2SCA clause/T-unit ratios) are derived; coreference (Maverick, fastcoref, Stanza-coref) links mentions across the timeline; and lightweight libraries (textstat, readability) plus DA classifiers handle readability and dialogue acts. All core tools are open-source and CPU-runnable; transformer variants want a modest GPU. The main workflow is: parse transcript utterances time-aligned to the movie/audio timeline (via forced alignment elsewhere), then compute per-utterance/per-second syntactic features. Most complexity, readability, and Yngve/Frazier metrics are derived post-hoc from parser output rather than being separate models.

Recommended (best-in-class): spaCy 3.8 (en_core_web_trf) for dependency/POS/morphology; Stanza 1.10 for UD-scheme parses + built-in constituency + coref in one tool; benepar for constituency trees feeding complexity metrics; Maverick for coreference resolution; L2SCA (+ Yngve/Frazier from trees) as the derived syntactic-complexity layer

Likely redundant:

• Trankit — superseded by spaCy/Stanza for English; keep only for multilingual or superior sentence-segmentation needs
• supar — overlaps benepar/Stanza; use only when you need joint dep+constituency or CRF marginals
• fastcoref/F-coref — superseded by Maverick on accuracy+efficiency; keep for high-throughput batch coref
• textstat — overlaps py-readability-metrics; surface-only, redundant with L2SCA’s deeper syntactic complexity
• Stanford CoreNLP constituency/coref — largely superseded by Stanza’s neural pipeline and Maverick

High-level semantics, discourse & narrative

This subclass covers meaning at the sentence/passage/scene level: dense semantic embeddings, topic structure, semantic coherence/similarity over time, discourse relations, and narrative structure (arcs, turning points, summaries, LLM-derived tags). For a movie/audio-story timeline the practical pipeline is: align a transcript (with word/utterance timestamps from ASR), window it into rolling segments (e.g. per-sentence, per-utterance, or sliding N-second windows), then emit per-segment features. The dominant, most flexible feature source in 2024-2026 is dense passage embeddings from instruction-tuned encoders (Qwen3-Embedding, E5/multilingual-e5, GTE, BGE, plus API options like OpenAI/Voyage/Gemini), from which you derive coherence (cosine similarity between adjacent windows), drift, and topic clusters. BERTopic gives interpretable topics-over-time. Instruction-tuned LLMs (Llama-3, Qwen2.5, GPT-4o-class) used as zero/few-shot feature extractors now cover the harder, more abstract signals — turning points, discourse relations, event/scene segmentation, narrative tags, and structured summaries — that no specialized supervised model does robustly. Note: in this neuroimaging context, the LAST-token/last-layer hidden states of autoregressive LLMs (GPT-2 through Llama-3) are themselves a primary per-word/per-TR feature and the current standard for naturalistic language-comprehension encoding models.

Recommended (best-in-class): Qwen3-Embedding-0.6B (lab default open encoder; scale to 4B/8B if GPU allows) as the backbone for all semantic features; Sliding-window cosine coherence/drift pipeline on top of those embeddings to produce per-second time-resolved scalars; BERTopic with topics-over-time for interpretable topic timelines (reuses the same embeddings); Autoregressive LLM hidden-state + surprisal extractor (GPT-2 / Llama-3.1-8B) for word-level / per-TR comprehension features (the neuroimaging standard); Instruction-tuned LLM (Llama-3.1-8B-Instruct / Qwen2.5-7B-Instruct local, or GPT-4o/Claude API) as a zero/few-shot extractor for narrative stage, turning points, discourse relations, and structured tags

Likely redundant:

• multilingual-e5, GTE, BGE — pick ONE primary open encoder; the others are interchangeable alternatives to Qwen3-Embedding, not separate features
• OpenAI/Voyage/Gemini embedding APIs — superseded for most needs by open encoders (Qwen3/E5); use only if a leaderboard-topping API embedding is required and off-site transcript transfer is acceptable
• Supervised RST parser (DMRST) and supervised turning-point models — largely subsumed by the instruction-tuned LLM extractor, which is more flexible on spoken transcripts; keep the supervised ones only when standardized, reproducible discourse labels are specifically required
• Standalone raw-embedding clustering — superseded by BERTopic, which wraps clustering with labeling and dynamic topics-over-time