Ryan Malloy
5248b5c093
Added: Why section, env vars table, merge strategy explanation, NORAD encoding table, regime classification, library usage examples, SQL output format, and development instructions.
204 lines
6.3 KiB
Markdown
204 lines
6.3 KiB
Markdown
# pg-orrery-catalog
|
|
|
|
TLE catalog builder for [pg_orrery](https://pg-orrery.warehack.ing) — download, merge, and load satellite catalogs from Space-Track, CelesTrak, and SatNOGS into PostgreSQL.
|
|
|
|
## Why
|
|
|
|
pg_orrery needs TLEs in a table. The satellite catalog has 30,000+ tracked objects spread across multiple data sources, each with different freshness. Getting them all merged and loaded into PostgreSQL is tedious: download from each source, handle different auth methods, deduplicate by epoch, format as SQL, pipe to psql.
|
|
|
|
`pg-orrery-catalog` handles the whole pipeline.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
# Run directly (no install needed)
|
|
uvx pg-orrery-catalog --help
|
|
|
|
# Or install
|
|
uv pip install pg-orrery-catalog
|
|
|
|
# For direct database loading (adds psycopg)
|
|
uv pip install "pg-orrery-catalog[pg]"
|
|
```
|
|
|
|
Requires Python 3.11+.
|
|
|
|
## Usage
|
|
|
|
### Download TLE data
|
|
|
|
```bash
|
|
pg-orrery-catalog download # All configured sources
|
|
pg-orrery-catalog download --source celestrak # CelesTrak only (no auth)
|
|
pg-orrery-catalog download --source spacetrack # Space-Track (needs credentials)
|
|
pg-orrery-catalog download --force # Re-download even if cached
|
|
```
|
|
|
|
Files are cached in `~/.cache/pg-orrery-catalog/` and reused for 24 hours.
|
|
|
|
### Build a merged catalog
|
|
|
|
```bash
|
|
pg-orrery-catalog build | psql -d mydb # SQL to stdout
|
|
pg-orrery-catalog build --table satellites -o catalog.sql # Save SQL file
|
|
pg-orrery-catalog build --format 3le -o merged.tle # 3LE text output
|
|
pg-orrery-catalog build --format json -o catalog.json # JSON with metadata
|
|
```
|
|
|
|
When the same NORAD ID appears in multiple sources, the entry with the newest epoch wins. This means CelesTrak SupGP data (operator-provided, fresher epochs) automatically overrides stale Space-Track entries.
|
|
|
|
### Build from specific files
|
|
|
|
```bash
|
|
pg-orrery-catalog build /path/to/spacetrack.tle /path/to/celestrak.tle
|
|
```
|
|
|
|
### Load directly into PostgreSQL
|
|
|
|
```bash
|
|
pg-orrery-catalog load \
|
|
--database-url postgresql:///mydb \
|
|
--table satellites \
|
|
--create-index
|
|
```
|
|
|
|
### Check status
|
|
|
|
```bash
|
|
pg-orrery-catalog info --cache
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Three layers (highest precedence first): CLI flags > environment variables > config file.
|
|
|
|
### Environment variables
|
|
|
|
| Variable | Purpose |
|
|
|----------|---------|
|
|
| `SPACETRACK_USER` | Space-Track login email |
|
|
| `SPACETRACK_PASSWORD` | Space-Track password |
|
|
| `SOCKS_PROXY` | SOCKS5 proxy for CelesTrak/SatNOGS (e.g. `localhost:1080`) |
|
|
| `DATABASE_URL` | PostgreSQL connection URL for `load` command |
|
|
| `PG_ORRERY_TABLE` | Default table name |
|
|
|
|
### Config file
|
|
|
|
`~/.config/pg-orrery-catalog/config.toml`:
|
|
|
|
```toml
|
|
[spacetrack]
|
|
user = "you@example.com"
|
|
password = "secret"
|
|
|
|
[celestrak]
|
|
proxy = "localhost:1080"
|
|
supgp_groups = ["starlink", "oneweb", "planet", "orbcomm"]
|
|
|
|
[output]
|
|
table = "satellites"
|
|
|
|
[database]
|
|
url = "postgresql://localhost/mydb"
|
|
```
|
|
|
|
## Sources
|
|
|
|
| Source | Auth | Coverage | Freshness |
|
|
|--------|------|----------|-----------|
|
|
| [Space-Track](https://www.space-track.org) | Login required | Full catalog (~30k+ on-orbit) | Hours to days |
|
|
| [CelesTrak](https://celestrak.org) | None | Active sats + operator SupGP | Minutes to hours |
|
|
| [SatNOGS](https://db.satnogs.org) | None | Community-tracked objects | Varies |
|
|
|
|
CelesTrak's supplemental GP groups are configured via `supgp_groups`. These contain operator-submitted ephemerides (SpaceX for Starlink, OneWeb, Planet, Orbcomm) that are typically hours fresher than Space-Track's catalog.
|
|
|
|
## Merge strategy
|
|
|
|
Sources are processed in order: Space-Track (base catalog), CelesTrak active, SatNOGS, then SupGP groups (freshest last). For each NORAD ID, the entry with the newest TLE epoch wins.
|
|
|
|
Example merge output:
|
|
|
|
```
|
|
spacetrack_everything: 33053 objects (33053 new, 0 updated)
|
|
celestrak_active: 14376 objects (2 new, 0 updated)
|
|
satnogs_full: 1488 objects (121 new, 5 updated)
|
|
supgp_starlink: 9703 objects (77 new, 7398 updated)
|
|
Total: 33253 unique objects
|
|
Regimes: LEO: 31542, GEO: 1203, MEO: 385, HEO: 123
|
|
```
|
|
|
|
The 7,398 Starlink updates are fresher epochs from SpaceX overriding stale Space-Track entries.
|
|
|
|
## NORAD ID decoding
|
|
|
|
TLEs use a 5-character NORAD catalog field. With 100,000+ tracked objects, numeric encoding ran out of space. `pg-orrery-catalog` handles all four encoding cases, matching the C implementation in pg_orrery's vendored SGP4 library:
|
|
|
|
| Case | Format | Range | Example |
|
|
|------|--------|-------|---------|
|
|
| Traditional | `ddddd` | 0 -- 99,999 | `25544` = 25,544 (ISS) |
|
|
| Alpha-5 | `Ldddd` | 100,000 -- 339,999 | `T0002` = 270,002 |
|
|
| Super-5 (3) | `xxxxX` | 340,000 -- 906,309,663 | `0000A` = 340,000 |
|
|
| Super-5 (4) | `xxxXd` | 906,309,664+ | `000A0` = 906,309,664 |
|
|
|
|
Alpha-5 skips letters I and O (ambiguous with 1 and 0). Super-5 uses a base-64 alphabet: 0-9, A-Z, a-z, `+`, `-`.
|
|
|
|
## SQL output format
|
|
|
|
The generated SQL matches pg_orrery conventions:
|
|
|
|
```sql
|
|
DROP TABLE IF EXISTS satellites;
|
|
CREATE TABLE IF NOT EXISTS satellites (
|
|
id serial,
|
|
name text,
|
|
tle tle
|
|
);
|
|
|
|
INSERT INTO satellites (name, tle) VALUES ('ISS (ZARYA)', E'1 25544U 98067A ...\n2 25544 51.6400 ...');
|
|
```
|
|
|
|
## Orbital regime classification
|
|
|
|
Objects are classified by mean motion (revolutions per day):
|
|
|
|
| Regime | Mean Motion | Description |
|
|
|--------|-------------|-------------|
|
|
| LEO | > 11.25 rev/day | Low Earth Orbit (period < 128 min) |
|
|
| MEO | > 1.8 rev/day | Medium Earth Orbit (GPS, navigation) |
|
|
| GEO | > 0.9 rev/day | Near-synchronous (comms, weather) |
|
|
| HEO | <= 0.9 rev/day | Highly elliptical (Molniya, tundra, GTO) |
|
|
|
|
## Library usage
|
|
|
|
```python
|
|
from pg_orrery_catalog.tle import decode_norad, parse_3le_file
|
|
from pg_orrery_catalog.catalog import merge_sources
|
|
from pg_orrery_catalog.regime import regime_summary
|
|
from pg_orrery_catalog.output.sql import generate_sql
|
|
|
|
# Parse and merge multiple files
|
|
merged, stats = merge_sources(["spacetrack.tle", "celestrak.tle"])
|
|
print(f"{stats.total_unique} unique objects")
|
|
|
|
# Regime breakdown
|
|
print(regime_summary(merged))
|
|
# {'LEO': 31542, 'MEO': 385, 'GEO': 1203, 'HEO': 123}
|
|
|
|
# Generate pg_orrery-compatible SQL
|
|
sql = generate_sql(merged, table="my_catalog")
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
git clone git@git.supported.systems:warehack.ing/pg-orrery-catalog.git
|
|
cd pg-orrery-catalog
|
|
uv venv && uv pip install -e ".[dev]"
|
|
pytest tests/ -v # 58 tests
|
|
ruff check src/ tests/ # Lint
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|