AI Content Optimizer

Objective and purpose

The objective is to reduce repetition and off-topic speech in recorded video while preserving temporal continuity and recording which intervals the editor retains or removes.

Silence detection identifies pauses but does not determine whether speech repeats earlier content or departs from the main topic. The prototype uses transcript semantics to estimate topical relevance, penalize redundancy and filler, and retain adjacent speech through temporal continuity rules.

Transcription, scoring, and edit planning

The Gradio application accepts an MP4, a Whisper model and precision setting, and a retention threshold. The default threshold is 0.45; lower settings generally keep more material. The application returns an edited MP4 and an inspection report. The current interface does not provide an interactive timeline for approving individual cuts before rendering.

The segmenter groups words until punctuation or a pause of at least 0.9 seconds creates a boundary. These units often resemble sentences, but a pause can split a grammatical sentence. Timestamp quality and speech-recognition errors therefore affect the later edit.

Retention score definition

The model all-MiniLM-L6-v2 embeds each unit. Similarity to the normalized mean embedding acts as a relevance score: a heuristic proxy for the unit’s similarity to the recording’s overall topic. Redundancy uses the maximum similarity to the previous 12 units. Pattern-based rules identify formulaic phrases, short filler expressions, and vague introductory wording.

The scorer normalizes relevance, combines it with weight 0.70, subtracts redundancy with weight 0.45 and filler with weight 0.35, then rescales the combined values within the video. A score of 1 denotes the maximum of the recording-specific scale; the value is not a calibrated probability that a human editor would retain the unit.

Temporal continuity rules

Independent unit selection can produce discontinuous edits. The planner fills single-unit gaps, removes retained islands shorter than two units, and restores the strongest unit in a removed run of at least five units. It pads retained speech by 0.1 seconds and merges nearby intervals before deriving cuts.

Temporal continuity rules can retain a lower-scoring unit because retention also depends on adjacent units. The report exposes scores, retained units, durations, and intervals. Removed intervals carry the generic label semantic_cut; the current implementation does not write an individual natural-language explanation for every cut.

Saved edit plan and rendered output

The following figure uses numeric timestamps from an existing saved plan. The source recording and transcript are omitted. The two rows preserve the recorded keep and trim intervals, including small gaps between them.

For example, the plan keeps 10.60–13.12 seconds and removes 13.12–19.56 seconds. The speech unit at 8.14–10.20 seconds has normalized relevance 1.00, redundancy 0.129, no filler penalty, and a final keep score of 1.00. Its padded interval begins at 8.04 seconds and ends at 10.30 seconds.

The saved plan does not record the threshold used for this run. Its planned duration and the measured export duration are different quantities, and the source extends beyond the recorded planning horizon. Renderer verification must distinguish the planning horizon, retained duration, and complete source duration.

Execution evidence and known failures

The project contains 16 saved plans and 13 corresponding exports. These artifacts demonstrate that the pipeline runs and produces files. They do not provide human ratings, a comparison with professional edits, or evidence that viewers understand the edited video better.

Saved plans identify several implementation failures. Three short saved plans retain no content, and the exporter fails when everything is removed. The renderer also derives its timeline end from the last trim interval instead of the original video’s duration; in some plans, that can omit trailing retained material. Empty and single-unit inputs need explicit handling.

Semantic similarity also has methodological limitations. A relevance penalty can remove a necessary digression; similar wording can convey additional information; and a short phrase can preserve a logical connection. Within-video normalization also means the threshold has no consistent quality interpretation across recordings. Human review remains necessary for meaning, pacing, and continuity.

Separate experiments and planned evaluation

The repository also explores visual-frame features, audio features, weakly labeled temporal models, and retrieval. Those experiments are separate from the active application; their saved checkpoints do not establish that the current editor uses a trained audiovisual editing model.

The proposed evaluation compares complete outputs with source videos, measures temporal-boundary and coverage errors, and collects human judgments of coherence and information retention. Evaluation must first address failures involving complete removal and omitted trailing content; duration reduction alone does not establish editing quality.