Building a Job Market Intelligence Dashboard with Free ATS Data

Investors, researchers, and product teams make better decisions with real-time labor market data. Which sectors are expanding headcount? What skills are companies hiring for right now? Which cities are tech hiring hubs?

TL;DR: Build a free real-time job market dashboard using Greenhouse, Lever, and Ashby public APIs — no paid subscription needed. Track hiring velocity, skills demand (Python, Kubernetes, LLM, etc.), department breakdown, and remote adoption across 200+ tech companies. A SQLite backend with weekly collection and a Flask API serves the analytics — data that talent intelligence platforms charge $500+/month for.

This guide builds a functional job market intelligence dashboard using only free, public ATS APIs — no paid data subscription required.

What You Can Track

Using Greenhouse, Lever, and Ashby public APIs across 200+ tech companies:

• Hiring velocity: How many roles a company is actively filling (weekly delta)
• Department breakdown: Engineering vs Sales vs Marketing hire ratios
• Skills demand: Which technologies appear most in job descriptions
• Remote adoption: What percentage of roles are remote or hybrid
• Geographic distribution: Where companies are hiring (for offices)
• Seniority distribution: Junior vs senior mix

Data Collection Backend

from apify_client import ApifyClient
from datetime import datetime
import sqlite3
import json

client = ApifyClient('YOUR_API_TOKEN')

# Define your company universe
TECH_COMPANIES = [
    # Developer infrastructure
    'vercel', 'netlify', 'cloudflare', 'supabase', 'planetscale', 'neon',
    'railway', 'render', 'fly', 'turso',
    # Fintech
    'stripe', 'brex', 'ramp', 'mercury', 'plaid',
    # Productivity
    'notion', 'linear', 'loom', 'coda', 'clickup',
    # AI
    'anthropic', 'cohere', 'mistral', 'together',
    # ... add more
]

def create_schema(conn):
    conn.execute("""
        CREATE TABLE IF NOT EXISTS job_snapshots (
            id TEXT PRIMARY KEY,
            ats TEXT,
            company_slug TEXT,
            title TEXT,
            department TEXT,
            location TEXT,
            is_remote BOOLEAN,
            employment_type TEXT,
            url TEXT,
            description_snippet TEXT,
            published_at TEXT,
            collected_at TEXT,
            week TEXT
        )
    """)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS collection_runs (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            run_date TEXT,
            companies_checked INTEGER,
            jobs_collected INTEGER,
            new_jobs INTEGER
        )
    """)
    conn.commit()

def weekly_collection(conn):
    week = datetime.utcnow().strftime('%Y-W%W')
    
    run = client.actor('themineworks/ats-jobs').call(run_input={
        'companies': TECH_COMPANIES,
        'maxJobsPerCompany': 200,
        'includeDescription': True,
    })
    
    jobs_collected = 0
    new_jobs = 0
    
    for job in client.dataset(run['defaultDatasetId']).iterate_items():
        job_id = f"{job['ats']}_{job['id']}"
        
        # Check if already exists
        existing = conn.execute('SELECT id FROM job_snapshots WHERE id = ?', (job_id,)).fetchone()
        if not existing:
            new_jobs += 1
        
        # Upsert
        desc = job.get('description_plain', '') or ''
        conn.execute("""
            INSERT OR REPLACE INTO job_snapshots VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (
            job_id, job['ats'], job['company_slug'], job['title'],
            job.get('department'), job.get('location'), job.get('is_remote'),
            job.get('employment_type'), job['url'],
            desc[:500],  # First 500 chars as snippet
            job.get('published_at'),
            datetime.utcnow().isoformat(),
            week,
        ))
        jobs_collected += 1
    
    conn.commit()
    conn.execute("""
        INSERT INTO collection_runs (run_date, companies_checked, jobs_collected, new_jobs)
        VALUES (?, ?, ?, ?)
    """, (datetime.utcnow().isoformat(), len(TECH_COMPANIES), jobs_collected, new_jobs))
    conn.commit()
    
    print(f"Week {week}: {jobs_collected} total, {new_jobs} new")
    return new_jobs

Analytics Queries

def get_hiring_velocity(conn, top_n: int = 20) -> list[dict]:
    """Companies ranked by current open role count."""
    rows = conn.execute("""
        SELECT company_slug, COUNT(*) as open_roles,
               SUM(CASE WHEN is_remote THEN 1 ELSE 0 END) as remote_roles
        FROM job_snapshots
        WHERE collected_at > datetime('now', '-7 days')
        GROUP BY company_slug
        ORDER BY open_roles DESC
        LIMIT ?
    """, (top_n,)).fetchall()
    
    return [{'company': r[0], 'open_roles': r[1], 'remote_roles': r[2]} for r in rows]

def get_skills_demand(conn) -> list[dict]:
    """Skills frequency across all current job descriptions."""
    skills = ['Python', 'TypeScript', 'Rust', 'Go', 'Kubernetes', 'AWS', 'LLM', 'React', 'PostgreSQL']
    
    results = []
    total_jobs = conn.execute("SELECT COUNT(*) FROM job_snapshots WHERE collected_at > datetime('now', '-7 days')").fetchone()[0]
    
    for skill in skills:
        count = conn.execute("""
            SELECT COUNT(*) FROM job_snapshots
            WHERE collected_at > datetime('now', '-7 days')
            AND (LOWER(title) LIKE ? OR LOWER(description_snippet) LIKE ?)
        """, (f'%{skill.lower()}%', f'%{skill.lower()}%')).fetchone()[0]
        
        results.append({
            'skill': skill,
            'mentions': count,
            'pct_of_jobs': round(count / max(total_jobs, 1) * 100, 1),
        })
    
    return sorted(results, key=lambda x: x['mentions'], reverse=True)

def get_department_breakdown(conn) -> dict:
    """Distribution of open roles by department."""
    rows = conn.execute("""
        SELECT 
            CASE 
                WHEN LOWER(department) LIKE '%engineer%' OR LOWER(department) LIKE '%eng%' THEN 'Engineering'
                WHEN LOWER(department) LIKE '%sales%' OR LOWER(department) LIKE '%revenue%' THEN 'Sales'
                WHEN LOWER(department) LIKE '%marketing%' OR LOWER(department) LIKE '%growth%' THEN 'Marketing'
                WHEN LOWER(department) LIKE '%product%' OR LOWER(department) LIKE '%design%' THEN 'Product/Design'
                WHEN LOWER(department) LIKE '%data%' OR LOWER(department) LIKE '%analytics%' THEN 'Data'
                ELSE 'Other'
            END as dept_category,
            COUNT(*) as count
        FROM job_snapshots
        WHERE collected_at > datetime('now', '-7 days')
        GROUP BY dept_category
        ORDER BY count DESC
    """).fetchall()
    
    return {r[0]: r[1] for r in rows}

Simple Web Dashboard (Flask)

from flask import Flask, jsonify, render_template_string

app = Flask(__name__)

DASHBOARD_HTML = """
<!DOCTYPE html><html>
<head><title>Job Market Intelligence</title>
<style>
  body { font-family: monospace; background: #000; color: #f5f5f5; padding: 2rem; }
  h1 { color: #3b82f6; letter-spacing: -0.02em; }
  table { width: 100%; border-collapse: collapse; margin: 1rem 0; }
  th, td { text-align: left; padding: 0.5rem; border-bottom: 1px solid #1a1a1a; }
  th { color: #3b82f6; font-size: 0.75rem; text-transform: uppercase; letter-spacing: 0.1em; }
</style>
</head>
<body>
  <h1>Job Market Intelligence Dashboard</h1>
  <div id="content">Loading...</div>
  <script>
    fetch('/api/summary').then(r => r.json()).then(data => {
      document.getElementById('content').innerHTML = `
        <h2>Top Hiring Companies</h2>
        <table><tr><th>Company</th><th>Open Roles</th><th>Remote</th></tr>
        ${data.velocity.map(c => `<tr><td>${c.company}</td><td>${c.open_roles}</td><td>${c.remote_roles}</td></tr>`).join('')}
        </table>
        <h2>Skills in Demand</h2>
        <table><tr><th>Skill</th><th>Job Mentions</th><th>% of Jobs</th></tr>
        ${data.skills.map(s => `<tr><td>${s.skill}</td><td>${s.mentions}</td><td>${s.pct_of_jobs}%</td></tr>`).join('')}
        </table>`;
    });
  </script>
</body></html>"""

@app.route('/')
def index():
    return render_template_string(DASHBOARD_HTML)

@app.route('/api/summary')
def summary():
    conn = sqlite3.connect('jobs.db')
    return jsonify({
        'velocity': get_hiring_velocity(conn),
        'skills': get_skills_demand(conn),
        'departments': get_department_breakdown(conn),
    })

if __name__ == '__main__':
    conn = sqlite3.connect('jobs.db')
    create_schema(conn)
    app.run(debug=True)

With a week of data collection, this dashboard gives you a real-time view of tech hiring trends that would otherwise require a paid talent intelligence platform costing $500+/month.

Frequently Asked Questions

What job market signals can you track using free public ATS APIs?

Public ATS APIs expose: role titles and departments (what companies are building), required skills extracted from job descriptions (which technologies are becoming table stakes), location and remote/hybrid distribution (where work is happening), posting velocity (how fast companies are growing), and role seniority mix (whether a company is hiring senior-heavy or building a junior-heavy team). None of this requires any authentication — it is all public information that most people simply don’t collect systematically.

How do you measure hiring velocity for a company using public job posting data?

Count new job postings per week and compute a 4-week rolling average. Velocity is the week-over-week change in that rolling average. A company going from 5 new postings/week to 15 over a month has 3x velocity — a strong hiring signal. Track velocity per department: an engineering velocity spike while sales is flat means a product buildout; the inverse means a go-to-market push. Store weekly snapshots so you can compute velocity over any time window.

How do you track skills demand trends across tech companies using ATS data?

Collect job descriptions from Greenhouse, Lever, and Ashby weekly. Use Claude Haiku to extract skill mentions in batches of 10 descriptions per API call. Store skill counts by week in a skills_demand table with (skill, week, count) rows. Query with a 12-week rolling window to detect rising skills (week-over-week growth > 10%), declining skills (consistent drops), and emerging skills (appeared in fewer than 5% of postings 6 months ago but now in 15%+). This is how you spot technologies crossing the adoption chasm.

What database is best for storing and querying job market intelligence data?

SQLite is sufficient for a single-developer setup tracking 200 companies — it handles millions of rows without configuration and stores the entire dataset in one portable file. Postgres becomes worthwhile when you need concurrent writes from multiple collection workers, or when the dataset exceeds ~5 million rows. Use a star schema: a central jobs fact table with foreign keys to companies, skills, and locations dimension tables. This makes aggregations and trend queries efficient without complex denormalization.

How much does it cost to build and run a job market intelligence dashboard?

Initial setup using public ATS APIs costs nothing in data access fees. The Apify ATS Jobs actor costs approximately $2-5/month under PPE billing to collect 10,000 postings weekly. If you add Claude Haiku for skills extraction, add $3-8/month for 10,000 descriptions. Total ongoing cost: under $15/month for intelligence that Revelio Labs, Lightcast, or LinkedIn Talent Insights charge $500-5,000/month to provide.