Toolkit for scraping, storing, and analyzing data from the Indonesia Stock Exchange (IDX / Bursa Efek Indonesia). Covers market data, company fundamentals, corporate actions, and broker flows — with time-series storage, Parquet export, and graph analysis pipelines.
git clone https://github.com/yourusername/idx-bei.git
cd idx-bei
uv syncFor a self-contained, zero-boilerplate exploration of the data pipeline, foreign flow, Bandarmology, and vectorized backtesting, open the interactive walkthrough script:
Launch the complete stack (Unified Web Dashboard, REST API, WebSocket server, and Neo4j graph database) with a single command:
docker compose up -d
# Dashboard & REST API: http://localhost:8000
# Neo4j Graph Browser: http://localhost:7474# 1. Scrape all snapshot datasets (company profiles, financial ratios, corporate actions, brokers)
uv run idx all
# 2. Inspect dataset inventory, 2026 calendar gaps, and tiered backfill recommendations
uv run idx status
# 3. Concurrent async backfill across company boards & shareholders (952 tickers)
uv run idx company --all-details --concurrency 8
# 4. Daily ingestion (today's OHLCV, broker summary & index flow)
uv run idx daily
# 5. Concurrent historical backfill (for backtesting)
uv run idx backfill --start 20260101 --end 20260807 --concurrency 8
# 6. Export all time-series and snapshots to Parquet (supports --incremental)
uv run idx parquet
# 7. Compact daily time-series partitions into monthly Snappy Parquet files
uv run idx compact# Screen top long-term DCA compounders with forensic anti-trap protection
idx compounder --top 15
# Analyze specific ticker for forensic quality, Justified PBV, and DCA suitability
idx compounder BMRI
idx compounder BBCA
# Inspect identified accounting value traps (one-off earnings distortions)
idx compounder --show-traps# Analyze specific ticker for dividend yield, DPR, and Ex-Date trap risk
idx dividend BBCA
idx dividend PTBA
# Screen top dividend opportunities across the entire market
idx dividend --screen --min-yield 8.0# Daily foreign flow accumulation & stealth accumulation briefing
idx signals
# Scan for stealth accumulation (quiet foreign buying during consolidation)
idx bandarmology --stealth# Backtest strategy holding returns, Sharpe ratios, and max drawdowns
idx backtest --strategy foreign_flow --holding 20 --top 10
idx backtest --strategy composite_alpha --holding 20 --stop-loss 7.0 --take-profit 15.0# Resolve multi-hop UBO hierarchy and corporate holding tree
uv run idx graph --ubo BBCA
# Rank top corporate board powerbrokers by network centrality
uv run idx graph --centrality
# Detect circular cross-holding loops between listed companies
uv run idx graph --cross-holdings# Track month-over-month position accumulation/distribution across all 1% & 5% holders
uv run idx drift --latest
# Ingest and standardize raw KSEI monthly/weekly shareholder reports (file or URL)
uv run idx drift --ingest data/ksei/raw_ksei_202607.csv
# Inspect specific tycoon position changes (e.g. Lo Kheng Hong, Prajogo Pangestu)
uv run idx drift --tycoon "LO KHENG HONG"# Generate daily 8-screen decision briefing (writes Markdown + JSON into data/briefings/)
uv run idx signals
# Inspect Top-N broker concentration ratios & institutional footprint
uv run idx bandarmology
# Detect stealth institutional accumulation vs retail traps (Smart Money Delta)
uv run idx bandarmology --stealth
# Optional: broadcast briefing summary to Discord/Slack/Telegram webhook
uv run idx signals --webhook-url "https://discord.com/api/webhooks/..."Runs eight quantitative decision-support screens over Parquet exports:
- Sector Rotation & Market Regime Radar — Sector Relative Strength (RS Alpha) vs COMPOSITE (IHSG), leading/lagging rotation, and macro regime (Bullish Expansion, Bearish Contraction, Rangebound)
- Composite Alpha Rankings — multi-factor ranking (Value + Smart Money Flow + Technical Momentum + Clean Audit)
- Foreign Flow Radar — net foreign buying/selling as % of free float (accumulate/distribute)
-
Bandarmology & Broker Dominance — Top-1 (
$CR_1$ ), Top-3 ($CR_3$ ), Top-5 ($CR_5$ ) broker concentration and institutional vs retail footprint -
Stealth Accumulation vs Retail Trap — detects flat-price stealth institutional loading (
$\Delta > 3.0$ ) vs retail exit pumps ($\Delta < 0.5$ ) -
Audit Risk Shield — stocks with non-clean audit opinions (
WDP/TMP/TL) - Dilution Watch — recent private placements, warrants, and capital reductions
- Sharia Value Screen — sharia-flagged, PER < 12, ROE ≥ 12%, DER ≤ 2
- Pasar Nego Crossing Radar — stealth off-market block crossings (value > Rp25B, nego share > 75%)
Resolve whether to BUY, HOLD, or SELL BEFORE CUM DATE when a stock announces a dividend:
# Deep-dive dividend decision analysis for a specific stock (auto USD/IDR conversion)
uv run idx dividend BBCA
uv run idx dividend PTBA
# Screen high-yield dividend opportunities and rank by Dividend Trap Risk
uv run idx dividend --screen --min-yield 4.0 --limit 20Read the full tactical strategy guide in docs/DIVIDEND_DECISION_GUIDE.md.
# Optional: compile modern React 19 SPA frontend (Vite + TypeScript)
cd frontend && bun install && bun run build && cd ..
# Launch Unified Web Dashboard & API (TradingView charts, live WS stream, REST docs)
uv run idx dashboard
# Or equivalently
uv run idx serve --port 8000
# Start Model Context Protocol (MCP) Server for Claude, Cursor, Antigravity
uv run idx mcpThe unified web dashboard automatically serves the compiled modern React 19 SPA from frontend/dist (with automatic fallback to dashboard/). It features:
- Stock Screener: Multi-factor quantitative filtering, valuation tiers, tycoon stakes, and blue-chip filters.
- Charts & Flow: TradingView Lightweight Charts v5 with live ticks, EMA-20/50, Bollinger Bands, and Net Foreign Flow sub-panel.
- Bandarmology & Institutional Radar: Smart Money Delta tracking, stealth institutional accumulation scanner, and retail trap alerts.
- Dividend Decision & Trap Radar: Dividend yield rankings, 0–100 Trap Risk scoring, and 3-way arbitrage comparison (Naive Hold vs Pre-Cum Exit vs Post-Ex Rebuy).
- Quantitative Strategy Simulator: Interactive vectorized backtester with customizable holding periods, stop loss / take profit rules, Sharpe/Sortino ratios, and interactive equity curve plotting.
- Data Ingestion & Backfill Horizon Status: Live timeseries health matrix, calendar gap detection separating national holidays from missing trading sessions, and 4-tier quantitative backfill horizon guidance with 1-click execution.
- Super-Insiders & Conglomerates: Tycoon portfolio tracking and corporate ownership cluster graphs.
The MCP server exposes 14 standard JSON-RPC tools for AI assistants:
-
idx_get_market_regime: inspect macro regime (Bullish Expansion, Bearish Contraction, Rangebound) and sector rotation Relative Strength (Alpha) against IHSG. -
idx_analyze_dividend: evaluate dividend announcements (Yield, DPR, Trap Risk 0–100, Buy/Hold/Sell verdict). -
idx_get_signals: daily 8-screen briefing summary. -
idx_query_stock: OHLCV, Net Foreign Flow & VWAP. -
idx_get_company_profile: board directors, commissioners, major shareholders & subsidiaries. -
idx_query_broker_flow: Top-N broker market share,$CR_1/CR_3/CR_5$ concentration ratios. -
idx_screen_stealth_accumulation: Wyckoff accumulation phases (Phases A–D), Smart Money Delta$\Delta$ , and retail trap detection. -
idx_run_backtest: vectorized quantitative strategy simulator across 6 strategies with customizable holding periods, stop loss/take profit, and volatility parity position sizing. -
idx_get_technical_signals: RSI-14, EMA 20/50/200, Bollinger Bands, ATR-14 & trend regime. -
idx_compare_peers: sector-relative valuation benchmarking. -
idx_search_announcements: public disclosures & PDF filing links. -
idx_screen_sharia_value: profitable, low-debt Sharia stocks. -
idx_get_super_insiders: billionaire/tycoon ownership holdings. -
idx_execute_sql: safe read-only DuckDB SQL queries over Parquet datasets.
idx-bei/
├── pyproject.toml # Root UV workspace configuration
├── uv.lock # Consolidated workspace lockfile
├── frontend/ # Modern React 19 + TypeScript + Vite SPA
│ ├── src/ # Components, hooks, services, types
│ ├── package.json # Bun / Node package definition
│ └── vite.config.ts # Vite bundler config
├── python/ # Python package & scripts
│ ├── src/idx/ # Core package (src-layout)
│ │ ├── core/ # HTTP client (sync & async), DuckDB query layer, KSEI ownership engine, currency rates
│ │ ├── scrapers/ # Domain scrapers (company, trading, corporate, financial, news, async backfillers)
│ │ ├── pipelines/ # Incremental Parquet export, daily ingestion, compaction
│ │ ├── mcp/ # Model Context Protocol (MCP) server (13 tools)
│ │ ├── dividend.py # Dividend decision engine & trap risk analyzer
│ │ ├── ingestion.py # Dataset inventory, calendar gap detection & backfill recommendations
│ │ ├── backtest.py # Vectorized strategy simulator & dividend arbitrage backtester
│ │ ├── graph.py # Neo4j UBO tree resolution, board centrality & ingestion
│ │ ├── api.py # FastAPI REST microservice & WebSocket broadcast server
│ │ ├── signals.py # 8 decision-support screens & stealth accumulation model
│ │ └── cli.py # CLI implementation
│ ├── tests/ # Pytest suite (181 passing unit tests, >=85% coverage)
│ └── pyproject.toml # Package config (uv/setuptools)
├── notebooks/ # Interactive research walkthrough scripts (quant & neo4j)
├── docs/ # Empirical API specs, architecture, and decision guides
├── data/ # Generated datasets (gitignored)
│ ├── timeseries/ # Historical OHLCV, broker, index partitions
│ ├── parquet/ # Columnar exports (daily and monthly compacted)
│ └── briefings/ # Daily signal briefings (md + json)
├── frontend/ # Modern React 19 + TypeScript SPA (TradingView charts, Vis.js graph, simulator)
├── docker-compose/ # Neo4j & PostgreSQL service configs
└── dashboard/ # Reference vanilla dashboard (index.html, css/, js/)
See CONTRIBUTING.md for developer onboarding and guidelines.
# Run automated pytest suite with coverage enforcement (175+ unit tests, >=85% coverage)
uv run pytest python/tests --cov=idx --cov-fail-under=85
# Run Mypy static type checker
uv run mypy python/src/idx
# Run Ruff linter and formatter
uv run ruff check python/src python/tests
uv run ruff format python/src python/tests# 1. Start Neo4j container
docker compose up -d
# 2. Ingest full 952-company network (12,000+ insiders, 5,400+ subsidiaries)
uv run idx graph --ingest
# 3. Open Neo4j Browser UI
# URL: http://localhost:7474 (user: neo4j, password: password)The repository includes a 100% native Go implementation (CGO_ENABLED=0) providing sub-millisecond REST & WebSocket APIs, pure Go Parquet analytics, vectorized technical indicators, and uTLS Cloudflare bypass ingestion:
# Build & install native Go binaries (idx, idx-server, idx-sync)
make install
# Cross-compile static ARM64 binaries for Android Termux
make build-arm64
# Start high-performance REST & WebSocket server on port 8000
idx serve --port 8000
# Screen top institutional DCA compounders
idx compounder --top 15
idx compounder BBRI
# Inspect Parquet OHLCV and compute RSI-14, EMA-20/50/200, Bollinger Bands, ATR-14
idx stock BBCA 10
# Ingest daily market-close data via standalone uTLS
idx sync100% Native Termux Standalone Architecture (Zero PRoot / Zero container overhead): Built with pure-Go uTLS Chrome/Safari fingerprint impersonation to bypass Cloudflare WAF natively and parquet analytics without heavy C-extensions:
# Build natively inside Termux or copy precompiled bin/idx-sync-android-arm64
make build-arm64
# Daily scheduled ingestion (OHLCV, Broker summary, Index summary, USD/IDR rate)
./bin/idx-sync -data-dir ./data
# Or schedule with Termux cronie (16:45 WIB):
# 45 16 * * 1-5 /data/data/com.termux/files/home/idx-bei/scripts/termux_schedule.sh >> /data/data/com.termux/files/home/idx-sync.log 2>&1| Layer | Tools |
|---|---|
| Systems Engine & Server | Go 1.24+ (Pure Go, 0 CGO, parquet-go, tls-client, websocket) |
| Data Science & MCP | Python 3.13+ (Astral uv workspace) |
| Web API & Realtime | Native Go 1.22+ http.ServeMux & WebSocket / FastAPI fallback |
| Frontend SPA | React 19, TypeScript, Vite, Tailwind/CSS tokens, Lucide |
| Charting & Visualization | TradingView Lightweight Charts v5, Vis.js Network |
| HTTP Client | Pure Go uTLS (tls-client) & Python curl_cffi |
| Query & Storage | Parquet (parquet-go & pyarrow), DuckDB, JSON |
| Databases | Neo4j (graph), PostgreSQL (relational) |
| Quantitative Engine | Pure Go technical indicator calculations, Python numpy/pandas |
| AI Assistant Protocol | Model Context Protocol (MCP stdio) |
| Infrastructure | Docker Compose |
| Endpoint | Data | Status |
|---|---|---|
GetStockSummary |
Daily OHLCV, foreign flow, bid/offer | 🟢 |
GetBrokerSummary |
Broker transaction volume & value | 🟢 |
GetIndexSummary |
IHSG, LQ45, sectoral indices | 🟢 |
GetCompanyProfiles |
Listed company directory | 🟢 |
GetCompanyProfilesDetail |
Board, shareholders, subsidiaries | 🟢 |
GetApiDataPaginated |
P/E, P/B, ROA, ROE, EPS, NPM | 🟢 |
GetIssuedHistory |
Corporate actions (15 categories) | 🟢 |
GetBrokerSearch |
Exchange member directory | 🟢 |
GetNewsSearch |
Market news & headlines | 🟢 |
GetAllAnnouncement |
Company disclosures & PDF filings | 🟢 |
See API_VERIFICATION_SPEC.md for full endpoint documentation.
MIT — see LICENSE.
Disclaimer: For educational and research purposes only. Comply with IDX terms of service.