UK Used-Car Marketplace Development: The CarVendors Case Study

The problem: 150 cars, no API, and nobody willing to type them in twice

A UK dealer's live stock usually exists in exactly one place — their own website. No API, no export, no feed. Getting 150-plus cars onto a marketplace by hand means somebody re-keying spec, price, mileage and thirty to sixty photographs per vehicle, then doing it again every time a price moves.

At that volume nobody does it reliably, and the failure is worse than it looks. A stale price does not just lose a sale; it destroys trust at the exact moment a buyer picks up the phone.

A marketplace has to satisfy two people at once. The buyer needs to find the right car in under a minute. The sales manager needs the stock to be correct without anyone touching it. Those pull in opposite directions, and most of the engineering below exists to serve both.

This page covers three things: how stock arrives on its own, why a broken source cannot empty the site, and how buyers actually find what they filter for. The result is live at carvendors.co.uk.

How do you import dealer stock when the supplier has no API?

The pipeline reads a partner dealer's own public listing and detail pages and turns them into marketplace records. Four stages:

Read. Requests go out with a deliberate delay between them and a bounded response size, so the source server never sees anything resembling a hammering.

Recover the registration. The real UK registration comes from the detail page, never from the URL slug — slugs lie, and a purpose-built validator rejects them. That validator accepts five historic UK plate format families; widening it from a narrower pattern recovered two vehicles that had previously been silently discarded.

Normalise. Everything is mapped onto the marketplace's own schema on the way in. This is where most of the value sits, and it gets its own section below.

Publish. The importer writes the published status and date directly. The marketplace's search query reads that status, with an active-dealer flag as the only other gate. No admin queue, no CSV step, no re-keying — the row the importer writes is the row the buyer searches.

Ingestion runs on stock the client is contractually entitled to list, with the source operator's knowledge, under an agreement between them. Only public pages are read, rate-limited, never authenticated. Each source's terms are assessed before a feed is built.

A second route handles dealers whose systems do publish an XML feed, reusing the same schema, the same per-dealer isolation and the same change detection. Feed URLs live in a database column rather than in code, and a scheduled dispatcher picks up every active dealer independently. Onboarding a new dealer is a configuration change, not a release.

All of it runs on the dealer's existing shared cPanel hosting. No VPS, no container platform, no new monthly bill.

What happens when a dealer stock feed breaks or the source site goes down?

On 21 January 2026 the source listing page failed to fetch. The run aborted after 48 seconds, and not one live advert was touched. The next day's run completed normally.

That is the failure most integrations get wrong. The source site changes its HTML overnight, or goes down, or rate-limits mid-import. A naive importer sees zero cars, concludes the dealer sold everything, and deletes 150 live adverts. The marketplace shows an empty forecourt and nobody notices until a customer calls.

The protection is deliberately boring:

• The pipeline imports before it touches live inventory — never the other way round.
• Active stock is counted before the import and again after.
• Two independent checks are then combined with a non-empty-result test, and that single expression is the only thing authorised to switch anything off. Processed vehicles must reach 85% of vehicles found; post-import active stock must be at least 80% of the pre-import count.
• Both are evaluated per dealer, so one dealer's bad day cannot touch another's listings.
• If the database connection cannot be re-established immediately before cleanup, the run marks itself unhealthy and skips cleanup rather than half-deleting.
• Even when the checks pass, cleanup is soft and staged: missing vehicles are deactivated in batches and stay in the database, and only records inactive for more than seven days are ever hard-deleted. A car that vanishes for a day because of a site glitch comes back with its images and history intact.

What the checks do not cover is wrong data. They stop listings disappearing. They do nothing about listings being incorrect. A source returning a plausible partial set inside both thresholds still prunes, and if the source's HTML shifts so price parses out of the wrong field, both checks read healthy and the wrong prices publish. There is no price-anomaly detection today. The seven-day grace limits the damage in the first case; the second is in What isn't built yet, below.

On a healthy run — 17 February 2026 — the checks read 99.3% success and 100.6% inventory ratio, cleanup was approved, and three genuinely sold cars were deactivated. The correct outcome, arrived at defensively.

How often can a dealer stock feed refresh without hammering the source?

Refreshing 150 vehicles properly means 150 detail-page fetches, 150 attribute writes and thousands of image references. Hourly, and you hammer the dealer's server. Weekly, and your prices are a week stale.

The resolution is a content fingerprint. Every vehicle reduces to a twelve-field signature — title, price, mileage, description, model, year, fuel, transmission, colour, engine size, body style and image count — with text normalised first, so cosmetic churn on the dealer's site does not register but a real price drop or an added photo does.

The ordering is what matters: the fingerprint is checked first, in a single indexed lookup, before any attribute write, any enrichment call, any image work. A match short-circuits the entire per-vehicle pipeline.

From the 17 February run: 153 vehicles found, 148 proven unchanged and skipped, 4 records rewritten, and 101 image rows written against roughly 4,350 image references crawled.

That last number looks like a contradiction, so: images are stored as URL references, never downloaded as binaries. That is why 4,350 references cost 101 rows. It is also the honest reason a dealer's photograph can 404 in a buyer's browser — the marketplace does not control the hosting of that imagery. The trade-off buys storage cost near zero and a refresh cycle fast enough to run several times a day.

Because an unchanged car costs almost nothing to check, stock can refresh multiple times daily rather than once — so a price change reaches buyers the same run, while the source server still sees only polite, rate-limited traffic.

Why do vehicle search filters return the wrong cars? Normalising messy dealer data

A buyer ticks Blue. The dealer's Pearlescent Midnight Blue Metallic car does not appear. As far as that buyer is concerned, the car is not on the site.

Source data is filthy, and that is nobody's fault. Colours arrive as "Pearlescent Midnight Blue Metallic" and "Red/Silver". Transmissions as "DSG", "auto", and the genuine observed typo "automatc". Fuel as "desiel". Every one of those breaks a search facet, and a broken facet means stock nobody can find.

• Colour — 136 real-world source strings map onto 22 canonical marketplace colours, after stripping finish words (metallic, pearl, matt, gloss) and reducing two-tone values to their first colour. Extraction uses four ranked strategies, each validated against a 52-word colour whitelist.
• Transmission and fuel — explicit typo dictionaries built from real observed misspellings. Anything unrecognised is logged for the next mapping pass rather than silently dropped, so the system tells you which variant to add next.
• Manufacturer — titles resolve against 34 name variants covering 32 distinct brands, cached in memory for the run.
• Photographs — de-duplicated on the content hash in the filename, always keeping the larger variant. The source publishes every image twice; on 17 February the de-duplication halved the set exactly across all 151 vehicles that reported an image count.
• Validation is two-tier, and the split is commercial rather than technical. A missing registration or an implausible price is a hard reject. Price-on-application, a missing colour or unusual mileage are warnings that still publish. Genuine POA stock reaches the site; a £12,000,000 hatchback never does.

Advanced vehicle search filters: how faceted search narrows hundreds of cars to three

UK buyers filter on many axes at once — make, model, body style, fuel, transmission, price band, mileage band, year range, colour, doors, seats, postcode, dealer. Hand-coding the combinations produces a nest of conditionals nobody dares extend.

Instead, every filter is a condition appended to one flat array with a bound parameter, joined once at the end. Twenty-five parameters compose into a single SQL statement with no nested branching: 13 exact-match dimensions generated from one parameter list, 5 multi-select dimensions that each expand into an OR group inside the AND chain — so "Hatchback OR SUV" AND "Petrol OR Diesel" resolves in one pass — and 6 ranges mapped onto indexed columns. Ten indexes across five tables cover exactly the columns buyers filter on; the migration is idempotent, additive, and ships its own rollback statements.

Every user-supplied value is bound as a parameter rather than concatenated — this is the most-attacked input on a marketplace. Pagination inputs are clamped, so a hand-edited URL cannot produce a negative offset or a crawler-triggered error that Google records.

What a buyer actually notices:

• Filtered URLs are server-rendered, so every filter combination is a shareable, indexable landing page — 47 of the site's 124 sitemap URLs are pre-filtered searches.
• Filters persist across pagination, and dealer context survives "Clear All", so a dealer's own traffic is never handed to a competitor's cars.
• Filter changes debounce with in-flight request cancellation, so ticking four body styles quickly fires one query rather than four.
• A 24-card results page costs five batched database queries regardless of card count (four for signed-out visitors), replacing a pattern where every card queried independently for its own photographs and saved state — well over a hundred statements behind the page carrying all the commercial traffic.

Deployment, monitoring and who owns the code

Delivery. Three environments — local, staging, production — with CI deploying on push and a curated exclusion manifest so version history, CI configuration and dependency trees never reach the live server. The rule is written into the repository: nothing reaches production until the same change has been checked on staging. Making that practical was its own piece of work — the codebase was audited for environment-coupled paths (roughly 71 hardcoded URL and path defects across about 35 files), and the database-stored production URL was replaced with runtime detection from the request, so identical code runs on a laptop, on staging and in production with no per-environment edits.

Unattended operation. Fetching retries with backoff behind explicit timeouts. A circuit breaker aborts a run rather than committing a partial dataset. When a single detail page dies, listing-page data is retained and flagged so the car stays on sale with price, mileage and a photograph rather than vanishing. Database resilience across a multi-minute crawl is handled at four levels: retried initial connect, a proactive reconnect after the crawl, a connection heartbeat, and per-run verification. Statistics failures are explicitly non-blocking, so observability can never stop stock going live. Every run emails a result summary on success and on failure, and top-level failures are also written to an error-log table. Housekeeping runs itself — logs purged after seven days, snapshots rotated, tables optimised weekly.

Ownership. The client owns the code, the domain, the data and the feed configuration, with a documented handover — README, runbook, environment matrix and credential inventory — and could move to another developer. That is the one thing a platform vendor cannot offer without abandoning its own business model.

What isn't built yet

Every claim above this section is checkable against a live site. These are the things that are not there yet.

1. A stock ceiling on current hosting. The import is linear — 153 vehicles in about four and a half minutes — against a thirty-minute execution limit. That breaks somewhere around a thousand vehicles per run on shared hosting. Past that it needs chunked runs across cron windows, per-dealer scheduling, or a VPS. Designed, not built.
2. No anomaly detection on price or field corruption. The safety checks stop adverts vanishing; nothing yet stops a wrong price publishing if the source's HTML shifts. Price-delta detection — holding any car whose price moves more than a set percentage in one run — is the obvious next thing to build.
3. No automated tests on the ingestion pipeline, and none around the search query builder. Given how many pages depend on that single search API, it is the most obvious insurance to add.
4. No caching layer. Every search runs live queries.
5. No search-query logging. Nothing records zero-result rate, query volume, or which filters buyers actually use.
6. Result ordering ships as newest-first with an optional image-first mode on landing pages. A full multi-factor ranking algorithm is designed in detail and is not implemented — no scores table, no rotation code, no view log anywhere in the codebase. Read it as a costed next phase, not a shipped feature.

A case study that only lists wins is a brochure.

A note on these numbers

The ingestion figures — 153 found, 148 skipped, 4 rewritten, 101 image rows, the run time, and the two safety-check readings — are read from dated production run logs. The January failure comes from the pipeline's own job log.

Query counts and filter dimensions are counted by reading the shipped code. The system has no runtime query log, so those are structural claims about how the code is written, not instrumented measurements. They are stated that way deliberately.

There are no conversion or enquiry-volume figures on this page. Per-vehicle engagement events are recorded, at real volume — but they have not been filtered for crawler and development traffic, and they are the client's data to release rather than mine. Anything presented as a business outcome before that work is done would be a projection wearing a measurement's clothes.

This page is dated, and it gets updated as real operating data accumulates.