feat: add complete Carquet writer through QueryExportWriter (#1048)
Add a private
CarquetExportWriterthat consumes theQueryResponseproduced by the existingQueryExportWriterand writes through the existing Carquet output adapter. This completes the scalar and nested writer implementation while productionCOPY TOcontinues to use the Arrow writer.The implementation supports scalar values, NULLs, LIST/fixed ARRAY and STRUCT, plus the existing string representations of INTERVAL and graph values. Both scalar and nested timestamps use NeuG’s millisecond unit. It retains compression, dictionary encoding and explicit row group options, validates response shape before opening output, and aborts partial output on write or finalization failure. The private C Data bridge owns its temporary buffers and needs no Arrow C++ SDK.
This PR uses only dependencies already merged into main. It has no dependency on the Carquet reader PR, changes no public IO/query interfaces or production registrations, and adds no Carquet submodule or adjacent-patch changes. The existing Arrow reader is used only as an independent test oracle. Tests generate files in memory; no new binary fixtures are checked in. Feature documentation is under
doc/source/extensions/carquet_writer.md.Validation
- Debug and Release: all 6 CI-selected Parquet CTest targets pass, including 16 writer tests, 10 adapter tests, 46 existing Parquet tests and 3 Carquet C probes.
- Debug and Release Python backend/export regression: 88 passed in each build; 6 cases skipped for the missing comprehensive_graph dataset, and 6 remote credential-dependent cases deselected.
- ASan/UBSan: all 16 writer tests pass with 48 Carquet C sources and 4 private C++ implementation sources instrumented. The remaining NeuG core/dependencies are not instrumented.
- The 4 private C++ sources compile after removing Arrow include directories. Clean pinned Carquet source plus the unchanged main patch builds with pkg-config unavailable.
- clang-format 10.0.1, git diff whitespace checks, documentation links and navigation pass.
Linux/system-Arrow CI and the full Sphinx documentation build were not available locally.
Local delivery
- Branch:
codex/carquet-r07-complete-writer- Commit:
48a4e1dce0e68d331e978bbc13395b4892ce39e0- Base:
ef76f38ff4af60d4681bdf9cc1a76e0a4476940d- Size: 10 files, +2543/-1 (implementation 1314 lines; tests 1120 lines; build/docs +109/-1).
- Validation logs:
/Users/luo/workspace/neug/build/review-r07-expanded-dmfpp0um
版权所有:中国计算机学会技术支持:开源发展技术委员会
京ICP备13000930号-9
京公网安备 11010802047560号
The one data index for your agentic applications
NeuG (pronounced “new-gee”) is a high-performance, graph-native transactional database that runs embedded in your application or behind a service. It provides durable storage, explicit transactions, Cypher-native querying, and in-place graph analytics.
Built on this data foundation, NeuG is the one data index for your agentic applications—indexing structure, semantics, and exact keywords over the same managed data. For more information, see the NeuG documentation.
News
Previous news
- **2026-06** — NeuG v0.1.3: [GDS extensions](https://neug.io/docs/extensions/load_gds/), [`COPY TEMP`](https://neug.io/docs/data_io/import_data/), [Node.js client](https://neug.io/docs/reference/nodejs_api/) - **2026-05** — NeuG v0.1.2: [`LOAD FROM`](https://neug.io/docs/data_io/load_data/), [Parquet](https://neug.io/docs/extensions/load_parquet/) & [HTTPFS](https://neug.io/docs/extensions/load_httpfs/) extensions - **2026-03** — NeuG v0.1 released - **2025-06** — GraphScope Flex, the engine foundation behind NeuG, set an [LDBC SNB Interactive Benchmark record](https://graphscope.io/blog/tech/2025/06/12/graphscope-flex-achieved-record-breaking-on-ldbc-snb-interactive-workload-declarative) with 80,000+ QPSInstallation
The packages support Linux and macOS on x86_64 and ARM64. Windows users can run NeuG through WSL2; native Windows support is on the roadmap. For more detailed instructions (including C++ from source), see the installation guide.
Python · requires Python 3.8+
Node.js · requires Node.js 20+ (since v0.1.3)
Quick Example
The same data can be queried by graph structure, vector similarity, or exact keywords. With the extensions installed and
ServiceandRunbookdata already loaded:Create an HNSW index · Create a full-text index
One Data, Indexed Three Ways
NeuG provides complementary ways to retrieve and analyze the same entities and properties:
Structure is native to NeuG’s graph storage. Vector and full-text indexes are maintained with the same underlying graph properties: graph changes and index changes commit atomically, and committed indexes recover with the graph through checkpoints and the write-ahead log.
Embedded or Service
Run NeuG in-process for local agent workflows and low-overhead analytics. When concurrent applications need network access, expose the same runtime as a service with
db.serve().See the reproducible dual-mode benchmark for complete results and methodology.
Development & Contributing
For building NeuG from source, see the Development Guide. We welcome contributions — please read the Contributing Guide before submitting issues or pull requests.
AI-Assisted Workflow
We apply an AI-assisted Spec-Driven workflow inspired by GitHub Spec-Kit:
/create-issuecommand in your IDE, or submit an issue manually/create-prcommand in your IDE, or submit a PR manuallyFor more details, see the AI-Assisted Development Guide.
Acknowledgements
NeuG builds upon the excellent work of the open-source community. We would like to acknowledge:
License
NeuG is distributed under the Apache License 2.0.