# Plan: collect about 700 OPENLANE detail pages, then review with Gemini 3.8 Flash

**Prepared 4 October 2026. This is an execution plan, not a record that 700 cars have already been collected or assessed.** “700 hundred” is interpreted as **about 700 cars**. The current snapshot has 1,476 visible filtered Buy Now cards and 100 signed-in detail captures, so reaching 700 in that same snapshot means about **600 additional unique auction IDs**. Recheck live availability before a new run: two of the first 100 had already sold by the time their detail pages were reviewed.

Use this with [OPENLANE_END_TO_END_GUIDE.md](OPENLANE_END_TO_END_GUIDE.md), which explains the existing OPENLANE/Scrappa price screen and preserved files. The target here is to collect **reliable source facts first** and perform model interpretation later. A search-result asking gap is never a dealer margin.

## 0. Confirm the access route and select the 700 IDs

OPENLANE Europe’s [buyer terms](https://www.openlane.eu/en-gb/cms/terms-and-conditions) make the account holder responsible for platform use and allow suspension for suspected misuse. I found no published, account-specific “safe requests per minute” number in those terms. **No interval can guarantee the account will not be limited.** The most reliable route for recurring 700-car exports is to ask OPENLANE whether it offers an authorized feed/export or written access guidance. If using the signed-in site, stay within the account’s normal authorized access; do not use proxy rotation, extra accounts, CAPTCHA bypass, or repeated retries after a block. This is an operational risk control, not a claim about legal permission.

Freeze a dated work queue before opening detail pages. Record the exact selection rule in `selection_policy`, because “top 700” can mean different things:

- **Recommended for dealer leads:** 600 cars prioritized by positive after-VAT euro gap, stronger close-ad evidence, relevant configuration and current availability, plus 100 stratified controls across price/evidence bands. Controls reveal false negatives and help calibrate the model. The already captured 100 can be included if they still fit the policy; reuse their saved detail records unless stale or incomplete.
- **If the intention is literally top 700 by gap percentage:** sort the dated screening CSV by `asking_gap_after_vat_pct`, then take 700 unique IDs. Expect many cheap, heavily damaged cars at the top. Retain that exact rule for reproducibility.

The queue should contain `run_id`, auction ID, original card rank, queue rank, source snapshot date, selection reason, prior capture hash/status, and priority. Do not overwrite the 3 October card snapshot or the first 100 detail records. Save a new run directory such as `runs/2026-10-04-detail-700/` with its own manifest and events.

## 1. Learn from the 100-car pilot, then set a collection budget

The first 100 signed-in pages yielded a median of about **2,681 characters of full page text** and **32 gallery URLs per car**. Ten had no captured condition text; 15 had under 100 characters; delivery text was captured on 28. The gallery references totaled 3,353 images. Pages took roughly 6–9 seconds each in the earlier interactive capture, before deliberately slower pacing. Those are measured observations from this snapshot, not a throughput promise.

**Starting policy for a new automated browser run:** one authenticated tab and **one detail page in flight**. Begin with 20 cars at **no faster than one page start every 25–30 seconds**, including page load and section expansion. Inspect the site responses, extraction completeness and account state. If clean, continue at the same pace through a 50-car pilot, then consider **20–30 seconds per car** as a working range. Do not increase concurrency merely to hit a deadline. For 700 pages, page starts alone at that range take roughly **4–6 hours**; login, slow pages, breaks and review add time. Split the queue into 50–100-car sessions with a saved checkpoint between sessions. This pace is a cautious starting assumption, **not a published OPENLANE allowance**.

Count **actual site requests**, not just “cars completed.” One car page may load images and background data. Avoid downloading whole galleries or PDFs in the 700-car baseline. Capture the displayed text, DOM photo URLs and authenticated report URL; request extra assets only when the condition source is missing or a car becomes a finalist. Do not run parallel browser tabs, a hidden HTTP scraper and an image downloader at the same time against the same account.

| Signal | Collector action |
| --- | --- |
| Normal page, correct auction ID and populated specs | Capture once, validate, save immediately, then wait for the next scheduled start. |
| Slow page or transient 5xx | Save an event and try **one** later retry after a longer pause. Repeated 5xx means stop the session and investigate. |
| HTTP 429 or explicit rate-limit notice | Honor `Retry-After` if supplied. Otherwise stop this session, reduce the planned pace, and resume only after a substantial cooldown or OPENLANE guidance. |
| HTTP 403, CAPTCHA, “unusual activity,” access suspension | **Stop**. Do not switch account/IP or bypass the challenge. Check account status and ask OPENLANE for the appropriate access method. |
| Login expired | Pause the queue; resume only after the normal user login. Never store the password or session cookie in the dataset. |
| Sold overlay | Mark sold with observation time. If **Show it to me, anyway** works, capture the historical report; do not count the car as currently purchasable. |

The pacing controller must be separate from extraction, with a configurable minimum interval, a daily/session cap, a stop flag, and a log of page start/end and response outcome. A stopped run must resume by auction ID without revisiting completed records unnecessarily.

## 2. Collect one complete, auditable text record per car

For each queued auction ID, use the existing authenticated browser session and open the normal detail URL `https://www.openlane.eu/en/car/info?auctionId=<id>`. Wait until the visible reference/specifications match the requested auction ID; a generic OPENLANE title is a loading state, not a completed capture. If sold, handle the overlay as above. Expand **Condition** and **Delivery options**; read the page’s specifications, equipment, documents and service/keys/inspection sections. Record the **Download** report URL. Do not press **Buy Now**, submit a bid, or mistake the page’s **Save** control for downloading a car report.

### Two layers per car

**Raw layer, append-only:** capture timestamp, run ID, auction ID, URL, page status, sold overlay status, complete visible page text, exact expanded condition text, exact expanded delivery text, report URL, gallery image URLs, and an optional page HTML snapshot if available through the normal browser. Preserve original language, currency, tax labels and exact wording. Store a SHA-256 hash of the text and the source card hash. If the same ID is recaptured, append a new version with its own time/hash; never mutate the old observation.

**Normalized layer, derived from raw:** make/model/trim; year, km, power, fuel, gearbox; body and van length/roof/windows/seats/cargo layout; pickup country and location; VAT regime and displayed Buy Now price; keys, documents, inspection and maintenance; prior accident and drive/flatbed status; exterior/interior/mechanical/tyre damage items; equipment; delivery options with destination, method, dates, amount and VAT basis. Every field needs `source_section`, `source_quote_or_offset`, `observed_at`, and one of `observed`, `inferred`, `estimated`, `unknown`.

Suggested files in the new run directory:

```text
queue.jsonl                 one row per selected auction ID and selection reason
capture_events.jsonl        append-only starts, finishes, waits, errors and stops
openlane_detail_raw.jsonl   versioned signed-in text + report/photo references
openlane_detail_facts.jsonl deterministic normalized fields and validation flags
capture_status.sqlite       resumable status index and hashes (derived from JSONL)
```

The event log should distinguish `queued`, `captured`, `partial`, `sold_captured`, `condition_missing`, `login_required`, `rate_limited`, `blocked`, `not_found`, and `failed`. A completed record is not just “HTTP 200”: it needs the right auction ID, substantive specifications, a condition status (`present`, `explicitly empty`, `external report`, or `missing`), and a saved report URL/status. Preserve a null when the source did not show a delivery price; do not turn “not quoted” into zero cost.

Collector state machine (implementation outline, not a command to run blindly):

```text
load dated queue and append-only captures; index latest status by auction_id
open one normal signed-in browser tab
for each queued ID not already validly captured in this run:
    wait until the next permitted page-start time
    append START event with timestamp and ID
    navigate to the normal car-info URL
    wait for matching auction reference and populated specifications
    if sold overlay: note it; open historical details if allowed
    expand Condition and Delivery options; read equipment/specs/documents
    capture exact section text, report URL and gallery URLs
    validate ID and completeness; classify condition as present/empty/external/missing
    append raw JSONL record; flush to disk before leaving this car
    append SUCCESS, PARTIAL, SOLD or ERROR event with duration and page state
    update resumable status index from the saved record
    if login/rate-limit/block stop condition: end the session
```

Do **not** define “complete” by a crude character cutoff alone: a short explicit damage note can be valid, while an empty accordion can mean an external report was not loaded. The reviewer must see that distinction. Retry only the missing section/report later, rather than reloading all successful cars.

### What to fetch conditionally

- **Baseline for all ~700:** visible specification, equipment, documents, maintenance and condition text; report URL; photo URLs. This is the highest information per site load and per later Gemini token.
- **Only when condition text is absent/very short or an external condition report is referenced:** use the site's normal report-download path if it works, with the same pace controller and separate `pdf_status` (`not_requested`, `saved_verified`, `failed`, `not_offered`). A URL alone is **not** a saved PDF. The first 100 have URLs but not local PDFs.
- **Only finalists or discrepancies:** locally save a small, relevant photo set or report PDF, then inspect visually. The user's first-pass preference is descriptions over spending model tokens on every gallery. Existing local photo copies can still power the dashboard carousel.
- **Delivery quote:** if the section requires extra loading or the destination is not the intended one, mark `quote_needed`. Quoted prices are destination- and time-specific and usually shown ex VAT. The current-bid total must not be reused as the Buy Now total.

Before committing each record, check auction ID/card agreement, numeric year/km/price plausibility, duplicate damage lines, condition status, account-address redaction in any export, and that the report link points to that auction ID. Keep contradictions (for example card mileage vs detail mileage) as two sourced facts plus a flag.

## 3. Prepare Gemini input after collection, without reloading OPENLANE

Google lists the stable model ID as **`gemini-3.8-flash`**, with text/PDF input, structured outputs and Batch API support ([model card](https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash)). Do **not** have Gemini browse OPENLANE for each car. Feed it only the saved, cleaned evidence package so the 700 website visits are not repeated.

Make one compact `review_input` per auction ID, versioned by `source_hash` and `prompt_version`:

1. Deterministically extract the card price/VAT, market comparison counts and titles, exact OPENLANE specs and raw condition/equipment/document/delivery sections. Keep important verbatim lines; remove navigation, similar-car suggestions and repeated UI boilerplate.
2. Redact the account's street address, login details, cookies and any unrelated personal information **before** sending data to Google. Keep pickup city/country and the quoted destination city needed for transport analysis. Review Google account/data settings before using authenticated material; [Google's pricing page](https://ai.google.dev/gemini-api/docs/pricing) distinguishes free and paid data-use terms.
3. Include explicit `missing_sections` and contradictions. A missing condition section is not “good condition.” Send the original price evidence tier, ad count, variant caveats and URLs/IDs, but do not ask Gemini to hallucinate fresh live market ads.
4. Keep photos and PDFs out of the first text-only pass. Add a second multimodal request only for missing damage evidence or promising finalists. Track which photo/report pages the second pass actually saw.
5. Ask for **structured JSON** against a pinned schema. Google documents structured outputs with schema validation ([structured-output guide](https://ai.google.dev/gemini-api/docs/structured-output)). Validate again locally; a syntactically valid JSON object can still contain unsupported claims.

### Required output fields

Use a stable schema with `auction_id`, `source_hash`, `prompt_version`, `model_id`, `reviewed_at`, `source_completeness`, `variant`, `observed_condition_items[]` (location, issue, severity, exact evidence quote/section), `documents_and_history`, `equipment_highlights`, `critical_flags[]`, `delivery` (observed quote, currency, tax basis, destination, or null), `repair_range_eur` and `prep_range_eur` (nullable, assumptions), `liquidity_reason`, `buyer_appeal_reason`, eight nullable **1–5** scores, `evidence_level`, `weighted_score` (nullable), `decision_bucket` (`reject`, `needs_quote`, `review`, `candidate`), and `next_checks[]`. Keep observations separate from estimates. Never output a precise repair amount for hidden crash damage or a profit/margin if essential acquisition and sale costs are missing.

The score definitions and weights are already fixed in [DEALER_ASSESSMENT_RUBRIC.md](DEALER_ASSESSMENT_RUBRIC.md). Keep that file/prompt version with each batch. Use **low thinking** for straightforward text extraction; route uncertain variants, contradictory records and large apparent gaps to a **medium-thinking second pass** or a human. Google lists low, medium and high thinking for this model; `minimal` is unsupported ([model card](https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash)).

## 4. Calibrate on the existing 100 before processing all 700

Do a 20–30-car Gemini pilot first: include severe-damage cases, apparently clean cars, vans with different cabin layouts, sold cars and cars with missing condition text. Compare Gemini's **facts** with the saved OPENLANE text and its scores with the existing 100-car dealer reviews; do not treat either model's opinion as ground truth. Manually check all major safety/title/drivability flags and a random sample of ordinary cars. Measure: source-quote accuracy, missed serious damage, false “clean” claims, variant errors, cost estimates without basis, schema failures and N/A handling.

Fix the prompt/schema once, bump `prompt_version`, and rerun the pilot until the failure types are acceptable. Only then submit the remaining records. After the full run, manually audit **at least 30–50 stratified cars** (top apparent opportunities, borderline scores, missing reports, vans, and random controls). Every potential purchase still needs a live report and current cost check.

## 5. Use Gemini Batch for the non-urgent text review

Google's [Batch API](https://ai.google.dev/gemini-api/docs/batch-api) accepts a JSONL input file of requests, each with a user-defined `key` returned with its result; it is intended for large, non-urgent jobs, with a target turnaround of 24 hours and **50% of standard API cost**. The [Gemini 3.8 Flash pricing page](https://ai.google.dev/gemini-api/docs/pricing) gives the current per-token prices; recheck before submitting. On 4 October 2026, paid Batch introductory rates are **$0.375 per million input tokens** and **$1.875 per million output tokens (including thinking)** through 31 December 2026. This is an API estimate only, not a guarantee of final spend.

Use one keyed request per car, e.g. `run_id:auction_id:source_hash:prompt_version`, and save the exact uploaded JSONL, batch job ID/status, raw response JSONL, per-car usage and parse errors. Split into modest batches (for example 50–100 cars) so a bad prompt or schema does not affect all 700 and so a subset can be retried. Do not submit the same key twice unless deliberately creating a new version. Estimate cost from a **count-tokens pilot** and observed output/thinking usage, then set a budget; Google provides a [token-counting API](https://ai.google.dev/api/tokens) and project-specific [rate limits](https://ai.google.dev/gemini-api/docs/rate-limits). The batch model is economical for this non-urgent review, but model cost and OPENLANE access risk are separate concerns.

Pseudocode for the **file workflow**, deliberately separating preparation from any paid submission:

```text
for each captured, validated auction ID:
    payload = compact_evidence(raw_capture, pricing_snapshot)
    key = run_id + ':' + auction_id + ':' + source_hash + ':' + prompt_version
    append {key, request: {contents: [payload], structured_output_config: schema_version}}
validate one input per key; count tokens on a representative pilot
submit a 20–30-car pilot; save job ID and every raw response
validate schema + evidence quotes; compare with human-reviewed cases
submit remaining 50–100-car batches; join outputs by key, never by line order
write review JSONL and derived dashboard rows only after validation
```

Use the exact current SDK/request syntax from Google's Batch documentation when implementing `structured_output_config`; the pseudocode is **not** an API request body. Gemini outputs should never overwrite the raw OPENLANE capture, prior GPT assessment or price snapshot. Store separate `review_source=gemini-3.8-flash`, source hash, schema/prompt version and human corrections.

## 6. Refresh the dealer dashboard without losing provenance

For each car, show the captured detail date, sold/active-at-capture state, report/PDF status, condition completeness, and whether the review is **Gemini text-only**, **Gemini plus selected photos/PDF**, or **human verified**. Keep original advertised prices and every comparable ID. Provide filters for critical damage, missing documents/condition, variant uncertainty, evidence level and all eight scores. Sort by plausible opportunity only after filtering unresolved critical issues; the original after-VAT asking gap remains visible but is not profit.

Only a finalist with current listing availability, exact comparable ads, verified Buy Now fees/VAT, transport quote, repair/prep quote and achievable sale price should enter a dealer margin calculation. If any input is unknown, show `quote needed` rather than a false profit number.

## Completion criteria for a 700-car run

- The manifest has 700 unique IDs and a recorded selection policy. The raw detail archive has one validated record or an explicit terminal status for **every** ID; no silent skips.
- No concurrent account access or uncontrolled retries occurred. All rate-limit, login, block and sold events are preserved. Collection can resume from a checkpoint without revisiting successful pages.
- Every record carries source URL, capture time, text hash, condition status, equipment/spec text, report URL/status, and photo URL list; source contradictions remain visible.
- Gemini input is redacted, compact and pinned to exact source hashes. Every output joins by keyed ID and passes schema and evidence checks; missing evidence produces null/N/A.
- The 20–30-car calibration and 30–50-car stratified final audit are logged. Critical defects are not turned into precise repair/profit figures without quotes.
- The dashboard preserves the original card/price snapshot, the authenticated detail version and the model/human review as **separate layers**.

**Current baseline to preserve:** `openlane_buy_now_all_visible_raw.json`, `openlane_buy_now_all_visible.sqlite`, `scrappa_all_visible_responses.jsonl`, `openlane_top100_pct_manifest.json`, `openlane_top100_full_details.jsonl`, `dealer_assessment_*.jsonl`, and the existing dashboard. The first 100 have authenticated text and report links, **not locally saved report PDFs**. The proposed 700-car collector and Gemini batch runner have **not** been built or executed by this document.