Education & Workforce
Job Market Intelligence API
Labour-market data is abundant and almost entirely unstructured. The value is not in collecting postings; it is in making a million of them say something consistent.
- Job boards
- HR software vendors
- Education providers
- Staffing firms
- Workforce agencies
What your customers would see
example interfaceMockupIllustrative mockup of the interface we would build. Figures shown are examples, not results from a live deployment.
The problem this solves
Every recruitment and HR product wants to answer the same handful of questions. What does this role pay in this city right now. Which skills are rising and which are fading. Is hiring in this sector accelerating or stalling. The raw material for all of it sits in job postings, which are public, numerous and formatted differently by every employer on earth.
Postings are messier than they look. The same job carries forty different titles. Salaries appear as ranges, hourly rates, annualised equivalents, or not at all. Agencies repost the same vacancy across six boards, so naive counts overstate demand by a wide margin. Aggregator sites republish each other. Without deduplication, title normalisation and salary standardisation you are measuring posting behaviour rather than hiring demand.
There is also a collection constraint that shapes the whole build. What may be gathered and what may be redistributed are different questions, and they are governed by each source's terms rather than by whether a crawler can technically reach the page. A design that assumes everything public is reusable will not survive contact with a client's legal review.
Who buys this
Job boards, HR software vendors, Education providers, Staffing firms, Workforce agencies
What the API returns
What comes back
- Normalised job titles mapped to an occupational taxonomy
- Salary distributions by role, seniority and location with the sample size attached
- Skill demand counts and trend direction, extracted from posting text
- Posting volume series with duplicates and reposts collapsed
- Employer-level hiring activity where the source terms permit attribution
- Emerging and declining skill lists with the evidence window stated
- Time-to-fill proxies derived from posting lifetime
- Coverage and confidence flags per role and region, so thin cells are visible
Endpoints
| Method | Path | Returns |
|---|---|---|
| GET | /v1/roles/{role}/market | Salary, demand and trend summary for one role and region |
| GET | /v1/skills/demand | Skill demand ranking with trend direction |
| GET | /v1/trends/postings | Posting volume series by sector, region and seniority |
| POST | /v1/normalise | Map raw titles and skill strings onto the taxonomy |
| GET | /v1/regions/{region}/summary | Market summary for one labour market |
| GET | /v1/coverage | Where coverage is thin and why |
Request and response
illustrativeRequest
{
"role": "data-engineer",
"region": "GB-LND",
"seniority": ["mid", "senior"],
"window": "last_90_days",
"include": ["salary", "skills", "trend"]
}Response
{
"role": { "normalised": "data-engineer", "taxonomy": "esco:2522.4" },
"region": "GB-LND",
"window": { "from": "2026-07-23", "to": "2026-10-21" },
"postings": { "raw": 4182, "after_dedup": 2611, "dedup_ratio": 0.376 },
"salary": {
"currency": "GBP",
"basis": "annual_fte",
"disclosed_share": 0.58,
"p25": 62000, "p50": 74500, "p75": 89000,
"sample": 1514,
"confidence": "high"
},
"skills_rising": [
{ "skill": "dbt", "share": 0.34, "change_pts": 7.1 },
{ "skill": "iceberg", "share": 0.11, "change_pts": 4.8 }
],
"skills_falling": [
{ "skill": "hadoop", "share": 0.04, "change_pts": -3.2 }
],
"trend": { "direction": "up", "change_pct": 0.09, "vs": "previous_90_days" },
"coverage_note": "Senior cell thin outside central London; see /v1/coverage."
}Where the data comes from
| Permitted job posting sources | Employer career sites and boards whose terms allow collection and redistribution of derived measures — we check each one rather than assuming public means reusable |
| Official labour statistics | National statistical agency series on employment, vacancies and earnings, which anchor the posting-derived figures to something authoritative |
| Occupational taxonomies | ESCO, O*NET or national equivalents, giving titles and skills a stable spine |
| Your own posting or placement data | Where you are a board or staffing firm, your own records are usually the cleanest and best-licensed source you have |
| Geographic reference data | Travel-to-work areas and administrative boundaries, so a market is defined by commuting reality rather than postcode |
| Currency and cost-of-living reference data | For comparing pay across regions without pretending the comparison is simple |
The intelligence layer
The difference between a demo and a service is mostly in this section. These are the parts that keep the answer trustworthy once real traffic is hitting it.
- Title normalisation using a trained classifier over the taxonomy rather than string matching, because the same role has dozens of names and the same name covers several roles
- Deduplication on employer, location, text similarity and posting window together — single-field matching misses agency reposts entirely
- Salary standardisation to annual full-time equivalent with explicit handling of hourly, daily and range-only disclosures
- Disclosure-bias correction, since employers who publish salary are not a random sample of employers
- Skill extraction from posting text with taxonomy mapping and negation handling, so 'no Python required' is not counted as Python demand
- Minimum-cell thresholds below which a figure is withheld rather than published as noise
- Benchmarking of posting-derived volume against official vacancy statistics, with divergence reported rather than smoothed away
- Monitoring for source composition shifts, which is the usual cause of a sudden apparent change in a market
What it changes for the business
Credible answers in your product
Users get salary and demand figures with sample sizes attached rather than a number they have no reason to believe.
Course and curriculum decisions
Education providers can see which skills are actually appearing in postings before committing to a new programme.
Content with a long tail
Role and region market pages are durable, updatable material that keeps earning traffic.
Sales conversations
Staffing and HR vendors can open with evidence about a client's own market instead of a generic pitch.
How it fits into your product
How it embeds
- REST JSON for your own market pages and dashboards
- Bulk extracts on a schedule where you need the data inside your own warehouse
- Normalisation endpoint you can call against your existing records to align them with the taxonomy
- Webhooks on material trend changes in roles or regions you have subscribed to
- Embeddable salary and demand components if you would rather not build the charts
- Per-tenant scoping so your own customers can query only their permitted segments
Included in every build
- Authentication, scoped API keys and per-customer rate limiting
- Input validation with predictable, documented error responses
- Versioned endpoints so your integration never breaks without notice
- Request and response logging for audit and dispute resolution
- Latency, error-rate and usage monitoring with alerting
- OpenAPI specification, sandbox environment and client examples
- Backups, failover design and a documented recovery procedure
- An agreed support arrangement once the service is live
How we would build it
about 20 weeksDiscovery and feasibility
Weeks 1–3
This one depends heavily on data access, so we start by proving the sources exist, can be licensed for your use, and actually carry the signal the model needs. If they do not, you find out here.
Architecture and baseline model
Weeks 4–6
A written architecture and a baseline model measured against a holdout period, so you know what accuracy you are buying before the full build starts.
Production build
Weeks 7–16
Data pipelines, model training and serving, historical archive, validation harness, API, security, monitoring and documentation.
Validation and integration
Weeks 17–20
Backtesting against held-out history, accuracy reporting your team can audit, integration support and an agreed retraining cadence.
Commercial models
There is no fixed price list, because the cost is driven by data complexity, integration count, model requirements, request volume and the support level you need. Scope is agreed in writing before development starts.
Fixed-scope prototype
A defined slice of the service, built and priced as one piece of work. The usual first step when the business case needs proving internally.
Fixed-price build
An agreed scope against a written architecture, delivered for an agreed price. Best when the requirements are already clear.
Build then retainer
We build the service, then operate and improve it under a monthly agreement covering monitoring, updates and model maintenance.
White-label product
The service runs under your brand, your domain and your pricing, with MLPivot invisible to your customers.
Questions buyers ask
Can you cover any job board?
No, and this is the main constraint on the build. Collection and redistribution are governed by each source's terms, not by whether the page is reachable. Some boards permit derived analytics, some forbid it outright, and some allow collection but not resale. We map that during discovery and design around the sources that do not permit your intended use.
How reliable are the salary figures?
They depend on disclosure. Roughly half of postings in most markets state pay, and the ones that do skew towards certain employer types, so we apply a disclosure correction and report the sample size on every cell. Where the sample is too thin we withhold the figure instead of publishing a median of eleven postings.
Is posting volume the same as hiring?
It is a proxy and should be presented as one. Reposts, speculative advertising and agency duplication all inflate raw counts, which is why deduplication matters so much. We benchmark against official vacancy statistics and report where the two diverge.
How current is the data?
Daily collection for high-volume sources, weekly for the rest, with every response carrying an as-of date. Official statistics lag by weeks or months by nature, so the API keeps posting-derived and official figures as separate fields rather than blending them into one number.
Can we resell this to our own customers?
Usually yes for derived aggregates, rarely for the underlying postings. The distinction follows from the source licences. We would document exactly which outputs you may pass on, in writing, before the build rather than after.
Related blueprints
Salary Benchmarking API
Role, level and region specific pay ranges with stated provenance, as a compensation layer inside HR, payroll or recruitment software.
Workforce Planning Forecast API
Forecasts of future role demand, internal capacity and external skills supply, as a planning layer inside HR, ERP or public-sector workforce software.
Skills Assessment API
Standardised skill tests, scoring and gap analysis delivered as an API your HR or learning platform embeds under its own brand.
Want this built for your platform?
Tell us what your product does today and what you want another system to be able to ask it. We will come back with an approach, a scope and a price.