A web app for viewing and analyzing mediacounts statistics for media files on Wikimedia Commons.
Queries can be run on individual files or category trees. Queries are run server-side and results are stored so users can save, revisit, and share them. Users must log in via Wikimedia to run queries, for the sake of managing load and being able to save results.
This application uses the mediacounts dataset, which stores counts of how often media files from upload.wikimedia.org have been transferred to users. It also pulls and displays usage of files across Wikimedia wikis.
- Node.js
- Wikimedia OAuth credentials
- Go to Wikimedia OAuth Consumer Registration
- Select OAuth 1.0a
- Fill in your application details:
- Application name: Your application name
- OAuth "callback" URL: Your app's callback URL, e.g.
http://localhost:3000/auth/callback - Applicable grants: Select "User identity verification only"
- Submit (approval should be instant given no permissions)
npm install
cp .env.sample .envEdit .env with your configuration.
Development mode (with auto-reload):
npm run devProduction mode:
npm startThe server will run on http://localhost:3000
Endpoints are authenticated (except where noted) to manage load and avoid abusive scrapers.
GET /api/media/stats
Parameters:
filename(required) - Commons filenamestart(optional) - Start date in YYYYMMDD format (default: 32 days ago)end(optional) - End date in YYYYMMDD format (default: 2 days ago, since mediacounts data may lag)granularity(optional) -dailyormonthly(default: daily)referer(optional) - Referer filter (default: all-referers)agent(optional) - Agent type filter (default: all-agents)
POST /api/category/stats
JSON body fields:
category(required) - Category namestart(optional) - Start date in YYYYMMDD formatend(optional) - End date in YYYYMMDD formatgranularity(optional) -dailyormonthly(default: daily)depth(optional) - Subcategory depth 0-10 (default: 0)referer(optional) - Referer filteragent(optional) - Agent type filter
Always runs asynchronously and returns a job reference immediately:
{
"jobId": "abc123...",
"status": "pending",
"statusUrl": "/api/jobs/abc123..."
}GET /api/jobs/:jobId
Authentication: Not required (permalink is shareable)
Returns job status, progress, and results when complete.
GET /api/jobs
Parameters:
limit(optional) - Number of jobs to return (default: 50, max: 100)
The response includes an X-Total-Count header with the user's total job
count, so clients can tell when the list is truncated.
GET /api/media/search
Parameters:
query(required) - Search termlimit(optional) - Number of results (default: 10, max: 50)
GET /api/category/files
Parameters:
category(required) - Category namedepth(optional) - Subcategory depth 0-10 (default: 0)
GET /api/stats
Authentication: Not required (shown on the public /stats page)
Returns lifetime totals: queries completed, files analyzed, media requests counted, distinct categories queried, categories scanned (subcategories included), processing time, single-file lookups, and users served. These counters persist after old jobs are deleted.
GET /api/session
Returns authentication status and user information.
GET /login- Initiate OAuth login flowGET /auth/callback- OAuth callback handlerPOST /logout- Logout and clear session
API endpoints are rate-limited per authenticated user to 500 requests per minute.
Outbound Wikimedia API traffic is throttled by a global scheduler (one
request per 100ms process-wide). Jobs run concurrently (up to
MAX_CONCURRENT_JOBS) and take turns round-robin, so short jobs finish
quickly while long ones run, and total upstream load never grows with the
number of jobs. A 429 from Wikimedia pauses all outbound traffic for the
requested interval.