You are a PIPELINE EXECUTOR. Your job is: prep media -> upload the video to Gemini via
mcp__oracle__consult (browser engine) -> read the analysis -> verify it against
evidence -> write local artifacts. Oracle/Gemini supplies the expensive multimodal
analysis; you own every local file, receipt, and verification.
This skill is the sibling of oracle-packx. Where oracle-packx bundles code and
sends text to ChatGPT, oracle-video attaches a video file and sends it to
Gemini (which can actually watch the footage). The full, validated process spec
lives in references/repeatable-video-followup-process.md;
read it before a long run.
Video Research Operating Rules
This skill exists because video understanding must happen in Gemini. Do not route video analysis through Codex, ChatGPT, OpenAI API mode, or text-only summarization. The local agent may search YouTube, download media, compress files, write prompts, bundle analyses, and produce final Markdown; the video-watching step is always Gemini through Oracle's browser engine.
When the user asks for "same pipeline" or a repeatable YouTube research batch, apply these repeatable research rules:
- Filter candidate videos by actual YouTube metadata (
upload_date), not title text, search ranking, or remembered publication dates. - If the user gives a cutoff such as "after a chosen cutoff", enforce
upload_date >= CUTOFF_YYYYMMDD. Reject older/stale videos even if they rank highly. - Exclude topic collisions explicitly. For example, "Raycast" should exclude game-development raycasting; "Codex" should exclude unrelated OCR/library/tool names.
- Exclude coupon, affiliate-code, auto-generated music/topic uploads, shorts-only junk, and videos whose content is not actually about the requested product/workflow.
- Download with current
yt-dlp, Chrome cookies, and YouTube challenge support when needed. A known-good command shape isyt-dlp --cookies-from-browser chrome .... - Preserve source metadata in
queue.jsonwithid,title,uploader,upload_date,duration, andurl. - Space Gemini video starts by about five minutes for larger batches only after the first attached-video probe passes and no unresolved run is pending, unless the user explicitly asks for a different cadence.
- Make prompts topic-specific. Never reuse Codex-specific prompt language for another topic; parameterize the topic in batch runners and prompts.
- After every Gemini run, save the analysis beside the chunk and verify the sentinel before moving to consolidation.
- After each successful Gemini run, close the Gemini tab that Oracle opened. Do not
leave the remote Mac mini with dozens of
gemini.google.com/apptabs.
Non-Negotiable Runtime Mode
Video analysis MUST run through the browser engine using ~/.oracle/config.json.
- Every
mcp__oracle__consultcall MUST setengine: "browser". - Default model:
model: "gemini-3-pro"— Oracle drives Gemini's mode picker to 3.5 Flash + Extended thinking, the right balance of speed and reasoning for routine video analysis. Any Gemini model with an attachment uses the DOM upload path (not pasted text), so the video is attached either way. - Heavy opt-in:
model: "gemini-3-pro-deep-think"— selects 3.1 Pro + Deep Think ("Max parallel reasoning"). Use this only when the user explicitly asks for the deepest analysis; it is markedly slower. Do NOT use it as the default. - Deliver the video with
browserAttachments: "always". The file path infilesmust be an absolute path on the invoking machine that Oracle can stream or hand to the browser host. For direct remote-host execution, use a path valid on that host; for remote Oracle service runs, preserve the client-side absolute path in receipts and let Oracle stream the attachment to the remote service. - Never use
engine: "api"for video. The API path cannot upload the browser video. If a call reportsMissing OPENAI_API_KEY, you accidentally used API mode — rebuild in browser mode. - Do not pass ad hoc remote host/token/profile overrides. Let Oracle read
~/.oracle/config.json, which owns the remote browser endpoint (the host that has Chrome + Google sign-in). CLI Oracle recovery path:oracle --engine browser --model gemini-3-pro ...(add--model gemini-3-pro-deep-thinkonly for the heavy Deep Think opt-in). - If the user mentions a remote Mac mini/TV/Gemini config, still do not hard-code a
host/token in prompts or calls. Use
~/.oracle/config.json; it is the source of truth for the remote browser and cookie-bearing Google session. - If an ownership/access dialog appears in Gemini ("confirm that you own/can access this content"), treat it as a browser/UI blocker. Let the user click it if automation cannot, then retry only the affected upload. Do not restart the whole batch.
Model / Intelligence Boundary
This skill analyzes video through Gemini in Oracle browser mode. Do not manage ChatGPT's Intelligence pill here.
Critical rules:
- Do not set
browserThinkingTime: "instant"for Gemini video analysis. - Do not use ChatGPT
Pro Extended/Instantlabels as Gemini readiness proof. - Use
model: "gemini-3-pro"by default, ormodel: "gemini-3-pro-deep-think"only for explicit heavy opt-in. - The Instant vs Pro Extended policy applies to ChatGPT only: Instant for
ChatGPT Create image /
imagine; Pro Extended for ChatGPT non-image consults.
Gemini readiness: UI sign-in is required, but cookie errors are not proof
Gemini video upload requires the browser profile that Oracle drives to have a live
signed-in Gemini UI. However, a Gemini text-only probe is not authoritative for video
readiness. Text-only Gemini may use Oracle's HTTP/cookie path and fail because a
Google cookie such as __Secure-1PSIDTS is missing, while the visible Gemini UI is
signed in and the attached-video DOM upload path still works.
Use this readiness order:
- Confirm the run is routed to Gemini browser mode, not ChatGPT or API mode.
- Confirm the remote Gemini UI is visibly signed in.
- Run one tiny attached-video probe with
engine: "browser",model: "gemini-3-pro",browserAttachments: "always", and an explicit sentinel. - Treat the attached-video probe, not a text-only probe, as the video readiness proof.
- If the probe hits a UI blocker, classify it with the failure diagnosis tree below before retrying or starting a batch.
Gemini may show a conditional video upload consent modal after a video is attached
and send is attempted. The DOM may include video-upload-consent-dialog with text like
"A reminder about creating videos" and actions Cancel / Agree. This is not a
Google sign-out, quota limit, or upload failure. Automation may click only the visible
action whose normalized label is exactly Agree. Never click Cancel, never click the
first dialog button generically, and never treat this modal as permission to click
arbitrary dialog actions.
Attached-video smoke test
Before a large batch, prove the actual video path with a tiny local MP4/MOV probe.
Required proof:
- request uses
engine: "browser"; - request uses
model: "gemini-3-pro"unless the user explicitly requested Deep Think; - request uses
browserAttachments: "always"; - stored request/session metadata shows Gemini provider routing;
- run log shows Gemini navigation and attachment upload completion;
- final response contains the expected sentinel;
- completed Gemini tab is closed after output is saved.
A text-only Gemini success is not enough. A text-only Gemini failure is also not enough to declare the video path broken. The video path is proven only by an attached-video run or by inspected Gemini tab evidence showing the submitted video prompt and sentinel response.
Two Paths
Pick the path from the source length and structure (full decision rules in the reference doc).
Short path — quick clips, no pre-processing (DEFAULT for short screen captures)
Use when the clip is short enough to upload and analyze in one pass (quick screen recording, a couple-minute walkthrough, a bug repro, a design review clip).
- Copy/symlink the clip as the single chunk; keep the original immutable + record a
sha256receipt. - Run ONE Gemini consult with the clip attached (see Step: Analyze).
- Verify the output (URLs/tools/claims supported by what's actually in the video).
- Write a compact deliverable: bug report, requirements note, design brief, or summary.
Do NOT segment, chunk, or build a manifest for short clips. Skip straight to analysis.
Long path — workshop recordings, multi-section talks
Use for long recordings (roughly 45+ min, real breaks, many sections). Segment with ffmpeg, drop verified dead-space (never discard chat — carry it forward), analyze each chunk, run an omissions pass, extract keyframes, reconcile against evidence, then assemble a teammate/follow-up packet. Follow the reference doc step by step.
Pipeline
Step 1: Choose a slug
3-5 hyphenated words (Oracle rejects anything outside that range), e.g.
workshop-chunk-02-analysis or bug-repro-video-followup. Every prompt starts with
[{slug}] on its own first line, then a blank line, then the prompt.
Step 2: Prep the media + receipts
RUN_DIR=runs/$(date +%Y-%m-%d)-{slug} # one folder per video/video-set
mkdir -p "$RUN_DIR"/{inputs,receipts,chunks}
cp "/abs/path/to/source.mp4" "$RUN_DIR/inputs/source.mp4"
shasum -a 256 "$RUN_DIR/inputs/source.mp4" > "$RUN_DIR/inputs/source.sha256"
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/inputs/source.mp4" > "$RUN_DIR/receipts/ffprobe.source.json"
For the long path, segment/chunk per the reference doc and write chunks/video-manifest.json.
For the short path, the single clip is chunks/01/chunk.mp4.
Step 3: Write the analysis prompt
Save the prompt to chunks/NN/gemini-prompt.md so the run is reproducible. Use the
"Gemini Chunk Analysis" template from the reference doc. Always include the hard rules:
preserve original timestamps, separate transcript/chat/visual/inference evidence, do
not invent tools/commands/URLs/files/speakers/labels, return both Markdown and fenced
JSON, end with an END_OF_VIDEO_ANALYSIS ... sentinel. Write the exact expected
sentinel beside the prompt, for example chunks/NN/expected-sentinel.txt, and verify
against that file rather than guessing whether the run used slug-only or
run_key chunk_id form.
Step 4: Analyze — upload the video to Gemini
You MUST execute this MCP tool call (one per chunk):
mcp__oracle__consult({
prompt: "[{slug}]\n\n{contents of chunks/NN/gemini-prompt.md}",
files: ["/abs/path/RUN_DIR/chunks/NN/chunk.mp4"],
engine: "browser",
model: "gemini-3-pro", // default → Gemini 3.5 Flash + Extended thinking
browserAttachments: "always",
slug: "{slug}"
})
For the heavy Deep Think opt-in (only when explicitly requested), swap in
model: "gemini-3-pro-deep-think" (→ 3.1 Pro + Deep Think).
Forbidden shapes: engine: "api", omitted engine, no browserAttachments. If you
draft one of these, rebuild before invoking.
For batch CLI Oracle recovery, use the same browser mode explicitly:
oracle \
--engine browser \
--model gemini-3-pro \
--browser-attachments always \
--browser-keep-browser \
--prompt "$(cat chunks/NN/gemini-prompt.md)" \
--file "/abs/path/RUN_DIR/chunks/NN/chunk.mp4" \
--slug "{topic}-batch-NN" \
--write-output "chunks/NN/gemini-analysis.md" \
--verbose \
--wait \
--force
Use --browser-keep-browser only so the run can be inspected and recovered while it is
active. Once the run has saved output, close the Gemini tab (see Step 5.5).
Persist the result yourself (the MCP return + output.log are truncated — the caller owns persistence):
chunks/NN/gemini-run.log # browser automation metadata / timing
chunks/NN/gemini-result.json # full response payload
chunks/NN/gemini-analysis.md # rendered analysis
Step 5: Wait correctly
Gemini video runs are blocking. After a run starts, all local analysis, artifact writing based on Gemini's answer, follow-up planning, and independent summarization MUST pause until Oracle/Gemini has fully completed and the local response has been read.
- Never assume Oracle/Gemini timed out or failed based on elapsed time alone.
- Never proceed independently while Oracle/Gemini is still processing, even if a wrapper returns after 120s or any other local wait limit.
- Reattach with
mcp__oracle__sessions({ id: "{slug}", detail: true })or read~/.oracle/sessions/{slug}/output.log; these checks are only for recovering the final response, not for starting work while the run continues. - Keep waiting as long as Oracle/Gemini requires while the session is still processing. There is no Codex-side timeout threshold that permits independent action.
- Only call it failed on an explicit terminal error, a sign-in/quota/upload guard, or a non-zero exit.
Healthy wrapper completion looks like the run log ending with Gemini browser response received (N chars) and Completed in Nms. Recovered completion is also valid when the
wrapper reports a send/ready error but tab inspection shows a submitted user turn and
Gemini response containing the exact expected sentinel. In that case, save the visible
response, record the recovery in gemini-run.log, and do not rerun that chunk.
For newer Gemini DOM runs, successful submission may also be evident from the remote
Gemini tab even when an older Oracle wrapper reports a send/ready error. If a run says
Gemini send was not accepted or Gemini composer was not ready to send after an
attached-video attempt, inspect the Gemini tab before retrying: the submitted user turn
and Gemini response may already exist. A valid sentinel in the visible Gemini response
beats the wrapper error; persist the response and fix/adjust the wrapper before
batching more chunks.
Step 5.5: Clean up the Gemini tab
Every successful Gemini video run should close the tab it opened on the remote browser. This prevents the remote Mac mini/TV browser from accumulating dozens of Gemini tabs. Cleanup happens only after extraction, not merely after the browser appears done.
Preferred cleanup order:
- Record the Oracle session slug and, if available, its Chrome debug port/target in the run log.
- Extract and persist every needed artifact from the Gemini response:
gemini-result.json,gemini-analysis.md, any machine JSON orgemini-analysis.extracted.json, and the recovery note if the response came from tab inspection rather than wrapper success. - Verify
gemini-analysis.mdis non-empty and contains the exactexpected-sentinel.txtvalue. - Close only the Gemini tab for that completed/extracted run.
- Leave the browser itself running if Oracle needs to reuse the signed-in profile.
- Do not close tabs for still-running uploads, still-generating responses, unresolved wrapper errors, or tabs whose visible response has not yet been extracted.
If you write a batch script, make tab cleanup an explicit post-extraction step. The script should not close the tab immediately after Oracle returns; it should first save the full result, extract machine-readable content, verify the sentinel, then close the exact tab target for that run. If extraction fails, leave the tab open and record the target/port so the run can be recovered.
When the run used --browser-keep-browser, use Oracle/browser tooling to close matching
completed Gemini tabs rather than leaving them open. If you are in the Oracle repo, a
working inspection/cleanup shape is:
pnpm tsx scripts/browser-tools.ts eval --port <port> '
(() => {
const tabs = [...document.querySelectorAll("a")];
return { href: location.href, title: document.title, text: document.body.innerText.slice(0, 500) };
})()
'
If you need to operate on the configured remote Mac mini, run the same command through
SSH using the repo-local Node path from that machine. Prefer closing the exact tab
target via Chrome DevTools Protocol when possible; avoid killing Chrome or closing all
Gemini tabs blindly. If exact-target close is not available, list tabs first, identify
completed gemini.google.com/app tabs by title/conversation state, and close only those.
Step 6 (long path): Omissions pass + reconcile + verify
Re-attach the same chunk with the "Omissions Pass" prompt, then reconcile locally into
final-reconciled.md/.json, and run the verification reports (coverage, hallucination,
timestamp, visual-claim). Keep every receipt next to the claim it supports. Details in
the reference doc.
Step 7: Assemble + summarize
Long path: produce the final/ packet (teammate guide, chapter index, resources, QA
index, exercises, keyframe manifest, instructor checklist). Short path: produce the one
compact deliverable. Summarize for the user, leading with the highest-value findings,
and link the artifact paths.
Evidence Preservation (core rule)
- Keep original media immutable; record a
sha256receipt. - Preserve ORIGINAL recording timestamps when a source timeline exists.
- Save every prompt, result, and run log next to the chunk.
- Never claim a URL/tool/command/visual unless it is supported by transcript, chat, carry-forward chat, a local path, or keyframe review.
- Never discard chat from a removed/dead span — carry it forward.
Gemini Video Failure Diagnosis Tree
Stop new batch starts before using this tree. Diagnose the current run first.
- Was a video attached?
- No: this may be the text-only HTTP/cookie path. Do not use it as the only video readiness result. Run a tiny attached-video probe.
- Yes: continue.
- Was the run routed to Gemini browser mode?
- If logs or metadata show ChatGPT navigation/upload, API mode, or missing
browserConfig.provider: "gemini", stop. This is a routing bug. Do not treat the result as valid video analysis. - If Gemini browser routing is confirmed, continue.
- If logs or metadata show ChatGPT navigation/upload, API mode, or missing
- Is there an explicit terminal guard?
sign-in: stop and have the remote browser signed into Gemini.quotaorrate-limit: stop and wait/change account per user direction.upload-warning: stop and fix file size/type/count or chunk the media.- No terminal guard: continue.
- Did a text-only probe fail on missing Google cookies?
- If the visible Gemini UI is signed in, do not diagnose sign-out from that alone. Text-only probes can fail on HTTP/cookie requirements that attached-video DOM runs do not need.
- Run or inspect an attached-video probe.
- Did attached-video send fail with
Gemini send was not accepted?- Inspect the Gemini tab before retrying.
- If
video-upload-consent-dialogis visible, click only exactAgree, then wait for the same prompt to submit. - If the prompt and attachment still remain in the composer and no submitted user turn exists, rerun only that chunk after fixing the blocker.
- If a submitted user turn or response exists, do not rerun until checking for the sentinel.
- Did the wrapper report
prompt-empty?- Inspect the tab.
prompt-emptycan mean Gemini accepted the prompt and moved it out of the composer. - If the visible response contains the expected sentinel, persist it as success.
- If no submitted user turn and no response exist, treat it as a real send failure and rerun only that chunk.
- Inspect the tab.
- Are there stale Gemini tabs?
- Do not close active uploads or generating responses.
- Generic empty Gemini tabs from failed diagnostics may be closed after inspection.
- Tabs with real conversation state must be checked for a saved response/sentinel before closing.
- Close only the exact completed tab after output is saved.
- Is the result valid?
- Valid completion is either a successful wrapper log with response received, or inspected Gemini tab evidence containing the expected sentinel and saved output.
- Anything else is unresolved; do not consolidate, summarize, or batch further.
Troubleshooting
| Problem | Fix |
|---|---|
Run log stops at Sending prompt..., no response; tab left with video attached |
Stop batch starts. Inspect the Gemini tab before rerunning. Check for a visible video consent modal, explicit sign-in/quota/rate-limit/upload warning, submitted user turn, or sentinel response. Rerun only the affected chunk after the state is classified. |
Gemini send was not accepted after 3 attempts |
Inspect the remote Gemini tab. If video-upload-consent-dialog is visible, click only the visible action whose normalized label is exactly Agree, then wait for the same prompt to submit. If the prompt remains in the composer with no submitted user turn, rerun only that chunk after fixing the blocker. If a response with the expected sentinel already exists, persist it as success. |
Gemini composer was not ready to send with reason=prompt-empty after an attached-video run |
Inspect the tab before retrying. prompt-empty may be a wrapper false negative after Gemini accepted the prompt. If the prompt appears in a submitted user turn/page and the Gemini response contains the expected sentinel, save it as completed analysis and record recovery. If no user turn/response exists, rerun only that chunk after fixing send/modal handling. |
| Text-only Gemini probe reports sign-in/cookie failure, but the visible Gemini app is signed in | Do not assume the video path is broken. Text-only probes can use Oracle's HTTP/cookie path and fail on missing __Secure-1PSIDTS; video attachments use the Gemini DOM upload path. Verify with a tiny real video attachment probe before deciding whether user sign-in is required. |
Text-only Gemini probe fails on missing __Secure-1PSIDTS |
Do not diagnose video sign-out from this alone. Confirm visible Gemini UI state and run a tiny attached-video probe. |
| Attached-video probe succeeds after text-only failure | Treat video DOM upload path as ready. Record both receipts so future agents do not revert to the false sign-out diagnosis. |
| Wrapper failed but tab contains sentinel | Persist the visible Gemini response, save a recovery note in the run log, close the completed tab, and do not rerun that chunk. |
| Log shows ChatGPT navigation or ChatGPT upload for a video | Stop. Treat this as a routing bug, not a valid analysis. Verify the stored request/metadata shows browserConfig.provider: "gemini" and rerun only after the provider guard tests pass. |
Many leftover gemini.google.com/app tabs with generic "Google Gemini" title |
Those are usually runs that never produced a query/response (often diagnostic/sign-in attempts), or completed runs that were left open because --browser-keep-browser was used. Close completed/empty Gemini tabs after saving output. Tabs with a real auto-generated title + conversation id may have completed; verify output before closing. |
Missing OPENAI_API_KEY |
The call used API mode. Retry with engine: "browser" + model: "gemini-3-pro" (default) or gemini-3-pro-deep-think (heavy). |
sign-in / quota / rate-limit / upload-warning guard |
Stop. Sign the remote browser into Google (or wait out the limit), then resume. |
video-upload-consent-dialog / "A reminder about creating videos" appears |
This is a video consent blocker, not sign-out. Click only exact Agree; never click Cancel, never click the first button generically, and never click Confirm for this dialog unless the DOM label is actually Agree. Continue the same run after acceptance. |
| Ownership/access confirmation appears | Treat as a separate UI blocker. If automation cannot safely identify the explicit ownership/access confirmation action, ask the user to click it once, then continue or retry only the affected upload. Do not restart the whole batch. |
| 120s timeout / no final answer yet | Reattach via mcp__oracle__sessions or output.log; do not proceed independently; keep waiting until Oracle/Gemini completes or reports a terminal failure. |
| Large video upload | Oracle streams large attachments to the remote service before the run ([remote] streaming large attachment before run). Keep the file under the configured maxFileSizeBytes / remote limit, or chunk it. |
| Truncated MCP return | Read chunks/NN/gemini-result.json that you persisted, not the MCP tail. |
Completion Checklist
- [ ] Original media preserved +
sha256receipt written. - [ ] Slug is 3-5 hyphenated words; prompt starts with
[{slug}]. - [ ] Each Gemini call used
engine: "browser",model: "gemini-3-pro"(default 3.5 Flash + Extended;gemini-3-pro-deep-thinkonly if explicitly requested),browserAttachments: "always", absolutefilespath. - [ ] Remote browser confirmed signed into Gemini before batch runs.
- [ ] A real attached-video probe has completed through Gemini before a large batch; do not rely only on a text-only Gemini probe.
- [ ] Text-only Gemini probe results were not used as the sole video readiness proof.
- [ ] First real attached-video probe result was saved before any batch fan-out.
- [ ] Any
send not accepted,prompt-empty, consent modal, sign-in, quota, rate-limit, or upload-warning state stopped new batch starts until classified. - [ ] Prompt, result JSON, and run log persisted per chunk.
- [ ] Each run either reached
Gemini browser response received, or a wrapper-error recovery was documented with submitted user turn + visible Gemini response + expected sentinel. - [ ] If
video-upload-consent-dialogappeared, only theAgreeaction was clicked and the same submitted prompt was allowed to complete. - [ ] Stored session metadata/request shows
browserConfig.provider: "gemini"for every video run. - [ ] Expected sentinel was written next to the prompt and checked exactly.
- [ ] If the sentinel format used
run_key chunk_id, verification did not look for slug-only form. - [ ] Run log contains
[gemini-web] Navigating to gemini.google.com...andAttachment upload complete in Gemini: <filename>. - [ ] Run log does not show ChatGPT navigation/upload for the video attachment.
- [ ] No local analysis, artifact synthesis, or final answer happened while Oracle/Gemini was still processing.
- [ ] Completed Gemini tabs were closed after output was saved and verified; the remote
browser was not left with accumulated
gemini.google.com/apptabs. - [ ] Scripted tab cleanup ran only after full extraction completed:
gemini-result.json,gemini-analysis.md, extracted JSON if expected, and exact sentinel verification. - [ ] Outputs verified: no invented URLs/tools/commands; visual claims confirmed; original timestamps preserved.
- [ ] Long path: omissions pass + reconciliation + verification reports done;
final/packet assembled. - [ ] Short path: single compact deliverable written.
- [ ] User summary leads with highest-value findings and links artifact paths.
References
- Repeatable video follow-up process — full step-by-step spec, ffmpeg segmentation, prompt templates, verification, and final-packet layout.