# Accuracy benchmark: Astro Agents vs NASA/JPL Horizons

**Engine:** astro-agents 1.3.0, positions from NASA/JPL DE440 via Skyfield 1.55. The run was replayed on 1.3.0 from the cached Horizons answers, and all 12,400 values are identical to the first run on 1.0.3.
**Reference:** the NASA/JPL Horizons API (ephemeris DE441), queried on 2026-10-26.
**Sample:** 1,240 instants (TT), 1800-01-03 to 2199-12-29, × 10 bodies = **12,400 comparisons per measurement**.

## Results in one paragraph

**Positions:** the engine's positions agree with JPL Horizons to **0.024″ at worst** (the Moon, at the 1800 and 2200 ends of the range). The Sun and the planets agree to **0.000003″** (a few metres along the line of sight). This is compared in the ICRF, with no precession or nutation model involved.

**Longitudes as served:** the ecliptic longitudes the API returns are **within 0.062″ of Horizons from 1962 to 2026**, the period where Horizons corrects its reference frame with measured Earth-orientation data. Outside that period they are within **0.53″**. That gap is one rotation of the reference frame shared by all ten bodies (IAU 2006/2000A in the engine vs IAU76/80 in Horizons); it is not a position error.

**One exception:** Neptune at 1.19″ on 1842-02-05, when it was 0.08° from the Sun's centre, behind the solar disk.

## Results

All values in **arcseconds**.

| Body | ICRF position: mean | ICRF position: max | Longitude as served, 1962–2026: mean | Longitude as served, 1962–2026: max | Longitude as served, 1800–2200: mean | Longitude as served, 1800–2200: p99 | Longitude as served, 1800–2200: max |
|---|---|---|---|---|---|---|---|
| Sun | 0.000001 | 0.000002 | 0.0473 | 0.0488 | 0.196 | 0.506 | 0.521 |
| **Moon** | **0.004969** | **0.024372** | **0.0466** | **0.0513** | **0.195** | **0.495** | **0.510** |
| Mercury | 0.000001 | 0.000003 | 0.0473 | 0.0505 | 0.196 | 0.506 | 0.524 |
| Venus | 0.000001 | 0.000002 | 0.0473 | 0.0503 | 0.196 | 0.506 | 0.522 |
| Mars | 0.000000 | 0.000001 | 0.0477 | 0.0506 | 0.196 | 0.506 | 0.523 |
| Jupiter | 0.000000 | 0.000000 | 0.0477 | 0.0497 | 0.196 | 0.507 | 0.521 |
| Saturn | 0.000000 | 0.000000 | 0.0481 | 0.0504 | 0.196 | 0.508 | 0.522 |
| Uranus | 0.000000 | 0.000000 | 0.0476 | 0.0489 | 0.196 | 0.506 | 0.522 |
| Neptune | 0.000000 | 0.000000 | 0.0479 | 0.0491 | 0.197 | 0.508 | 1.186 ¹ |
| Pluto | 0.000000 | 0.000000 | 0.0503 | 0.0615 | 0.198 | 0.514 | 0.528 |
| **All bodies** | **0.000497** | **0.024372** | **0.0478** | **0.0615** | **0.196** | **0.507** | **1.186** ¹ |

The median is 0.000000″ for the ICRF positions and 0.0476″ for the longitudes as served over 1962–2026.

Sample size: N = 1,240 instants per body (12,400 overall) for the ICRF and 1800–2200 columns. The 1962–2026 columns are a subset of 304 instants per body (3,040 overall).

¹ On 1842-02-05 (JD 2393872.3 TT), Neptune was 0.08° from the Sun's centre, behind the solar disk. There the gravitational deflection of light is over 1.75″, and the two implementations treat a line of sight passing through the Sun differently. No other instant of any body exceeds 0.528″.

### The Moon

The Moon is the body most sensitive to errors of centre (geocentric vs topocentric) and of time.

| Moon | Mean | Median | p99 | Max |
|---|---|---|---|---|
| ICRF position, 1800–2200 | 0.0050″ | 0.0027″ | 0.0204″ | 0.0244″ |
| ICRF position, 2020–2030 (276 instants) | 0.0012″ | 0.0012″ | 0.0016″ | 0.0017″ |
| Longitude as served, 1962–2026 | 0.0466″ | 0.0466″ | 0.0510″ | 0.0513″ |

The Moon's residual grows towards 1800 and 2200, which is where DE440 and DE441 (the version Horizons uses) differ most for the lunar orbit. In the present decade the two agree to about 0.002″.

Like-for-like checks: the Moon's longitude residual matches the Sun's and the planets' at the same instants, so the geocentre and the time scale are configured identically.

### The longitude residual is the reference frame, not the positions

At each instant, the signed longitude residual is almost the same for all ten bodies. Spread across bodies (max minus min): median 0.011″, p99 0.032″ (max 0.79″, the Neptune case above). A position error would differ from body to body; this residual is a single rotation of the ecliptic-of-date frame:

- **The engine** uses IAU 2006 precession and IAU 2000A nutation, the current IAU standard.
- **Horizons' quantity 31 (`ObsEcLon`)** is defined in the IAU76/80 ecliptic of date. Inside its Earth-orientation (EOP) coverage (1962 onward), Horizons corrects it with the IERS celestial pole offsets, and the two frames then agree to about 0.05″. Outside that coverage the offsets are held fixed. The difference between the IAU 1976 and IAU 2006 precession then grows at roughly 0.25″ per century, to about 0.5″ at 1800 and 2200.

## Method

### What the engine computes

From `src/astro/ephemeris.py`:

| | Engine |
|---|---|
| Centre | Geocentre (Earth, NAIF 399) |
| Position | Apparent: light-time, annual aberration, gravitational deflection (Skyfield `observe().apparent()`) |
| Frame | True ecliptic and equinox of date, IAU 2006 precession / IAU 2000A nutation (Skyfield `ecliptic_frame`) |
| Longitude | Tropical = ecliptic longitude of date, degrees |
| Targets | Sun 10, Moon 301, planet **barycentres** 1, 2, 4, 5, 6, 7, 8, 9 |
| Time | Input in UT, converted to TT with Skyfield's built-in ΔT tables |
| Ephemeris | DE440 excerpt 1800–2200 (`src/astro/data/de440_1800_2200.bsp`) |

### Horizons configuration

Horizons is configured identically wherever Horizons allows it:

| | Measurement 1: ICRF position | Measurement 2: longitude as served |
|---|---|---|
| Engine side | `Earth.at(t).observe(body)`: astrometric vector, light-time corrected, ICRS | `ephemeris.body_state(t, body)["longitude"]`, the value every endpoint returns |
| Horizons | `EPHEM_TYPE=VECTORS`, `VEC_CORR=LT`, `REF_SYSTEM=ICRF`, `REF_PLANE=FRAME` | `EPHEM_TYPE=OBSERVER`, `QUANTITIES=31` (ObsEcLon: apparent, light-time, deflection, aberration, ecliptic of date) |
| Common to both | `CENTER=500@399` (geocentre), the same targets (Sun 10, Moon 301, barycentres 1–9), `TIME_TYPE=TT`, `TLIST` of the same Julian dates | |
| Metric | Angle between the two directions | Absolute difference of longitudes |

- **Same TT instants on both sides.** The positions are compared at identical TT instants, so a different ΔT model can neither hide nor create a position error.
- **Barycentres, not planet centres.** Horizons is asked for exactly the bodies the engine reads. For bodies with moons the offset to the planet centre is under 0.1″; it applies equally to both sides here.
- **Latitudes are not compared.** The benchmark covers longitudes only.

### Sampling

1,000 instants uniform over 1800-01-03 to 2199-12-29, plus 240 uniform over 2020–2030 (a dense recent subset), drawn with a fixed seed (440). Julian dates are rounded to 1e-6 day, and both sides use the rounded values. Some of the 1,000 uniform instants also fall in 2020–2030, so the dense subset holds 276 instants.

### ΔT (UT → TT)

This is reported separately and does not enter the figures above.

From 1962, Horizons' `TDB-UT` is TDB−**UTC**, not TDB−UT1. Beyond about 78 days ahead it holds its last prediction constant. Against the engine's TT−UT1:

| Period | Difference | Effect on the Moon for an input given in UT |
|---|---|---|
| 1962–2031 | ≤ 0.71 s (this is UT1−UTC, < 0.9 s by definition) | ≤ 0.4″ |
| 1800–1962 | ≤ 0.26 s | ≤ 0.14″ |
| 2031–2200 | Up to 151 s: Horizons keeps today's value, the engine extrapolates Earth's slowing rotation | Up to 83″ in 2200 |

ΔT in 2200 is unknown to everyone (its uncertainty is minutes). Far-future charts given in UT carry that uncertainty whatever the engine.

## Caveats

- **DE440 vs DE441.** Horizons computes from DE441, the engine from DE440. For the planets they are the same within this range (differences under a metre); the Moon's few hundredths of an arcsecond near 1800 and 2200 come from that difference, not from the computation.
- **Other engines.** Vedaksha (arthiqlabs) publishes 0.096″ mean against Horizons. On positions (ICRF), this benchmark gives a mean of 0.0005″ and a max of 0.024″. Their metric and frame may differ from ours, so the numbers are not strictly like for like. Against Horizons' own ecliptic-of-date longitudes, the engine averages 0.048″ over 1962–2026 and 0.196″ over 1800–2200; the latter is the frame difference explained above.
- **What is not covered.** Lunar nodes and Lilith are not compared: Horizons has no equivalent quantity. House cusps, ayanamsas and divisional charts are cross-checked against XALEN in `tests/test_engine_oracle.py`, not here.

## Reproduce

```bash
pip install -e .
python benchmarks/horizons_benchmark.py            # replays from the cached Horizons answers in benchmarks/data/horizons/
python benchmarks/horizons_benchmark.py --refresh  # queries Horizons again (about 500 requests)
```

**Outputs:**
- `benchmarks/results/summary.json`: every statistic above, also served by `GET https://astro-agent.dev/v1/benchmark`.
- `benchmarks/results/samples.csv`: one row per instant and body, with the engine longitude, Horizons longitude, difference and ICRF angle. Published at https://astro-agent.dev/benchmark-samples.csv.

**Checking one row without the source:** query Horizons for the row's TT instant with the configuration above. To compare with the API, `POST /v1/western/positions` takes a UT instant (`"time_basis": "utc"`), so convert TT to UT first; the ΔT caveat above then applies.