The music of the primes · part 2 of 3

Beyond the diagonal: the secret of the last per cent

Part one left a small, roughly one-per-cent deviation in the lowest blocks. Noise, or the trace of something deeper? The answer hides in pairs of primes — and, surprisingly, in the very first zeros of the zeta function itself.

David Hessabout 8 min readfollows part 1
Re(s) = ½ · critical lineIm(s) →14.1321.0225.0130.4232.9437.5940.9243.3348.0149.77

When part one showed that the zeros of the Riemann zeta function remember the primes through Berry's formula, one awkward question remained: what about that small, roughly one-per-cent deviation in the lowest blocks?

The follow-up study takes the same millions of zeros and tests them against the more advanced formula of Eugene Bogomolny and Jonathan Keating. The result? The mysterious “noise” turned out to be real physics — and the primes revealed further, even more remarkable layers of order.

200,000zeros in the tested zeros6 blocks
≤ 0.93 σdeviation of the new formula from the data
4–10%depth of the “echoes” of the first zeros
3·10⁻¹³relative accuracy of the explicit formula

01The end of the one-per-cent mystery

Berry's formula from part one rests on the so-called diagonal approximation. The number variance is really a double sum over pairs of prime orbits — and Berry kept only the pairs in which an orbit meets itself.

Picture this · an orchestra

The diagonal approximation is like estimating the loudness of an orchestra as the sum of each instrument playing solo. Most of the time that comes out nearly right. But the instruments hear each other — their waves add up and cancel. Those are the off-diagonal terms: the interplay of two different primes with similar periods.

Bogomolny and Keating included this interplay. How often two primes sit close together is described by the famous Hardy–Littlewood conjecture on prime pairs — and the formula follows from it without a single free parameter.

The last per cent: noise or physics?
Below the gold line a deviation is indistinguishable from random noise.
0 σ1 σ2 σ3 σ4 σlargest deviation from the data in standard errors (block zeros6, 200,000 zeros)1 σ — statistical noise levelBerrydiagonal terms only2.4–3.8 σBogomolny–Keating+ off-diagonal terms≤ 0.93 σ
No free parameter: the off-diagonal correction rests on the Hardy–Littlewood constants for prime pairs; nothing is fitted.

So that earlier one per cent was not noise but a genuine signature of the missing off-diagonal corrections. Once we add them, data and theory merge.

02Two arithmetic engines

The formula carries two arithmetic components: repetitions of prime powers Φd and the Hardy–Littlewood factor for prime pairs Φo. What happens when we switch them off one at a time?

Two arithmetic engines
What happens to the prediction when we “switch off” one of the factors?
0 %1 %2 %3 %4 %5 %change in the predicted number variance Σ²(L)without Φdprime-power repetition off4.4–4.6 %without ΦoHardy–Littlewood off≈ 0.5 %
Φd carries the bulk of the smooth variance correction. Φo is almost invisible in the variance but tunes the fine structure of the form factor and the fast oscillations.
Switched offWhat happensWhere the factor matters most
Both factors (only the “zeta part” of the formula remains)a failure as dramatic as pure GUE — deviations of tens of σ—
Φd · prime-power repetitionnumber variance changes by 4.4–4.6%smooth correction of the variance
Φo · Hardy–Littlewoodchange of only about 0.5%fine structure of the form factor, fast oscillations

At very short distances the two factors are tightly coupled and cannot be separated: they balance each other in an exactly computable way.

03Resurgence: high zeros remember the lowest ones

The most magical result of the study confirms a prediction by Berry and Keating about so-called resurgence — literally “rebirth”. Look at the statistics of zeros high up, around height 106 or even 1021, and you find three things:

  • the pair correlation of the zeros shows tiny dips of 4 to 10%,
  • these dips sit exactly at the heights of the first ten zeros, from γ₁ = 14.13 to γ₁₀ = 49.77,
  • and the number variance Σ²(L) shows peaks exactly where the window length matches multiples of the first zeros.
An echo of the first zeros
Exactly where dips 4–10% deep appear in the statistics of zeros at heights 10⁶ to 10²¹.
γ₁14.13γ₂21.02γ₃25.01γ₄30.42γ₅32.94γ₆37.59γ₇40.92γ₈43.33γ₉48.01γ₁₀49.77position of the dip in the pair correlation of high zeros = height of a low zero γₖ
A schematic of positions, not depths: all dips are drawn alike. γ₁ to γ₁₀ are the heights of the first ten zeta zeros.

Zeros a million to 10²⁰ times higher carry a living memory of the very beginning — of the first zeros of the zeta function.

The Bogomolny–Keating formula describes this echo with remarkable accuracy. The random primes of the Cramér model produce nothing of the kind: the echo is purely arithmetic.

04A grand audit of classic predictions

The study also works as an audit of other pillars of the mathematical physics of the zeta function:

period 1Fast oscillations. On a fine grid the number variance shows rapid ripples with period 1 — exactly as predicted.
r ≤ 100The “decorated factorial”. Riemann–Siegel series coefficients up to order r = 100 match Berry's asymptotic formula.
n ≤ 999Riemann's formula for π(x). With the first 1,000 zeros, simple rounding gives the correct prime count for every n ≤ 999.
3·10⁻¹³The music of the primes. The explicit formula with a Gaussian test function links zeros and prime powers to this relative accuracy.
The zeros “sing out” the primes
Riemann's explicit formula: a smooth curve plus waves from the zeta zeros. Hover over the chart.
actual count π(x)no zeros — smooth term onlywith the first 30 zeros
048121601020304050xnumber of primes ≤ x
Our illustrative computation (Riemann's J(x) with Möbius inversion, first 30 zeros). Thirty waves already draw the steps at the primes 2, 3, 5, 7, 11, … The study goes much further: with 1,000 zeros, rounding gives the correct π(n) for every n ≤ 999.

Replace the real zeros with “smooth” fictitious stand-ins of the same density and the formula for π(x) fails completely. The primes simply require the exact positions of the real zeros — just as in part one the zeros required the exact positions of the primes. It is a two-way street.

05The bottom line

✓ What the study showed

The behaviour of the zeros at long range and in the details is not random. It is a symphony written in the arithmetic of the primes — from off-diagonal corrections to the echoes of the first zeros — and parameter-free formulas capture it to within one standard error.

✗ What does not follow

None of this proves or disproves the Riemann Hypothesis. The formulas assume the zeros lie on the critical line and test how precisely the hidden order resonates.

Glossary
Diagonal approximation
In the double sum over pairs of orbits, keep only the pairs in which both orbits are identical.
Off-diagonal terms
Contributions from pairs of different prime orbits with nearby periods. Their weight is set by correlations between primes.
Hardy–Littlewood conjecture
A prediction of how often pairs of primes with a given gap occur (for example twin primes p and p + 2).
Spectral form factor
The Fourier transform of the pair correlation of the zeros — it shows which “periods” sound in the spectrum.
Resurgence
The phenomenon whereby information about low zeros reappears in the asymptotics of high zeros.
Riemann–Siegel series
A fast formula for computing ζ on the critical line; its coefficients at high orders grow like a “decorated factorial”.
Paper, data and code

Study II · Beyond the diagonal approximation (Bogomolny–Keating)

The whole series on Zenodo: Study I · Study II · Study III · licence CC BY 4.0 (text, data) and MIT (code)

The charts are interactive — hover over the curves and points.

2 komentáře

Napsat komentář

Vaše e-mailová adresa nebude zveřejněna. Vyžadované informace jsou označeny *