Skip to content

fix: truncate float literals to exact float64 precision#501

Merged
gka merged 2 commits into
mainfrom
fix/no-loss-of-precision
Mar 2, 2026
Merged

fix: truncate float literals to exact float64 precision#501
gka merged 2 commits into
mainfrom
fix/no-loss-of-precision

Conversation

@ljodea

@ljodea ljodea commented Feb 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Truncates 45 float literal constants in src/lib/helpers/math.ts to their exact IEEE 754 float64 representations
  • These are statistical constants (Wichura 1988 probit function) where extra digits exceeded float64 precision (~15-17 significant digits)
  • Uses String(Number(x)) to find the shortest exact representation — all values are bit-for-bit identical at runtime
  • Fixes all 45 no-loss-of-precision oxlint warnings

Test plan

  • pnpm exec oxlint --type-aware — 0 no-loss-of-precision warnings
  • pnpm test — 685/685 tests pass
  • pnpm check — 0 svelte-check errors

🤖 Generated with Claude Code

The 45 statistical constants (Wichura 1988 probit function) had more
significant digits than IEEE 754 float64 can represent, triggering
oxlint no-loss-of-precision warnings. Replace each literal with its
shortest exact float64 representation. No behavioral change — JS was
already silently discarding the extra digits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@netlify

netlify Bot commented Feb 27, 2026

Copy link
Copy Markdown

Deploy Preview for svelteplot ready!

Name Link
🔨 Latest commit 1cd9c91
🔍 Latest deploy log https://app.netlify.com/projects/svelteplot/deploys/69a5dd2d35689c00085a8aba
😎 Deploy Preview https://deploy-preview-501--svelteplot.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@gka gka merged commit 7de9f93 into main Mar 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants