Dune Analytics Announces Mid‑Point Progress on Its Migration to DuneSQL
April 20 2024 marked the start of a three‑month phase‑out of SparkSQL and PostgreSQL in favour of the newly‑launched DuneSQL engine. With 30 days elapsed, the company released a detailed update on the rollout, highlighting new functionality, performance gains and the forthcoming deprecation of legacy tools.
Why DuneSQL Matters
DuneSQL is positioned as the next‑generation query layer for on‑chain data, promising faster execution, higher precision and a richer feature set than the platform’s previous engines. The migration is part of Dune’s broader strategy to give analysts, developers and creators a “fastest, most powerful and convenient crypto‑data querying experience on the planet.”
Key capabilities introduced with DuneSQL include:
- Query‑as‑view support – users can treat any query as a reusable view.
- Wei‑level numeric types – native
UINT256andINT256fields enable exact handling of token amounts. - Adjustable performance tiers – queries can be assigned a performance profile that matches their computational intensity.
- Materialized views and scheduled runs – allow the creation of pre‑computed datasets and automated refresh cycles.
- Custom data uploads – analysts can ingest private datasets and query them alongside on‑chain tables.
- Migration tooling – the “Harmonizer” assists in translating legacy SparkSQL scripts to DuneSQL syntax.
30‑Day Milestone: What Has Changed
1. Enhanced Harmonizer Tool
The migration assistant, dubbed Harmonizer, received a substantial upgrade. It now handles more sophisticated conversions, such as turning Spark’s explode(array_column) pattern into Trino’s cross join unnest(array_column). The tool can also port over Ethereum V1 queries written for SparkSQL. A full list of merged pull‑requests is available on the project’s GitHub repository.
2. Performance Improvements
Since the launch, DuneSQL has rolled out performance tiers and materialized versions of high‑traffic tables like nft.trades and dex.trades. These changes have largely resolved earlier bottlenecks. To further address optimizer timeouts on complex queries, the team plans to release a full materialized‑view feature in late June. By breaking large queries into a chain of smaller, pre‑computed views, users can sidestep stage‑related timeouts and benefit from more predictable runtimes.
3. Spellbook Migration
The internal “Spellbook” query builder, previously powered by SparkSQL, has completed its testing phase on DuneSQL. Over the next few weeks Dune will switch the default Spellbook to the new engine and move the SparkSQL version into maintenance‑only status (no new pull requests, removal from the UI dropdown). Creators will experience a seamless transition, while contributors will receive a more detailed rollout schedule soon. Once the DuneSQL Spellbook is fully validated, the Dune team will handle the migration of all existing spells.
4. Documentation and Communication
A high‑level migration timeline, along with an FAQ, is now live in Dune’s documentation portal. The guide details the sunset schedule for SparkSQL and PostgreSQL, outlines the steps required to move custom queries, and provides contact information for support.
Analyst Perspective
- Reduced friction for on‑chain analysis – The introduction of native 256‑bit integer types removes the need for work‑arounds when dealing with token balances, making code cleaner and less error‑prone.
- Scalability for heavier workloads – Adjustable performance levels and materialized views give power users a way to scale queries without over‑loading the shared cluster, a common pain point on the legacy platform.
- Lower barrier to entry for data contributors – The ability to upload private datasets and query them alongside public on‑chain tables widens Dune’s appeal to firms that need to blend proprietary data with blockchain metrics.
- Potential short‑term disruption – While the migration tools are maturing, users may still encounter translation quirks, especially for highly complex SparkSQL scripts. The ongoing feedback loop encouraged by Dune is crucial to smoothing these issues.
Key Takeaways
| Takeaway | Implication |
|---|---|
| DuneSQL is now feature‑complete for most day‑to‑day analytics | Users can immediately leverage views, high‑precision types and scheduled queries. |
| Harmonizer now supports advanced SparkSQL constructs | Migration of legacy queries should become faster and require fewer manual edits. |
| Materialized views arriving late June | Allows decomposition of large, timeout‑prone queries into manageable pipelines. |
| Spellbook fully transitioning to DuneSQL | No functional change for creators; behind‑the‑scenes performance improvements expected. |
| Documentation and support channels updated | Clear guidance is available, reducing uncertainty during the remaining 60‑day sunset period. |
Outlook
Dune’s commitment to a rapid yet transparent migration suggests the platform will soon become the de‑facto standard for on‑chain data analysis. The blend of performance‑tuned execution, precise numeric handling and native support for user‑uploaded datasets positions DuneSQL to handle the growing data demands of DeFi, NFT, and broader crypto analytics. Stakeholders are encouraged to test the updated Harmonizer, experiment with materialized views when they launch, and keep an eye on the Spellbook rollout schedule to ensure uninterrupted workflow.
For further details, consult Dune’s migration documentation or reach out to the support email provided in the original announcement.
Source: https://dune.com/blog/dune-sql-migration-update
