Skip to content

Commit

Permalink
ARROW-844: [Format] Update README documents in format/
Browse files Browse the repository at this point in the history
Added a section reflecting specification maturity and stability.

Author: Wes McKinney <[email protected]>

Closes apache#556 from wesm/ARROW-844 and squashes the following commits:

03dbb71 [Wes McKinney] Update README documents in format/
  • Loading branch information
wesm committed Apr 18, 2017
1 parent 7f20f6e commit 38efabe
Showing 1 changed file with 15 additions and 5 deletions.
20 changes: 15 additions & 5 deletions format/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,14 @@

## Arrow specification documents

> **Work-in-progress specification documents**. These are discussion documents
> created by the Arrow developers during late 2015 and in no way represents a
> finalized specification.
Currently, the Arrow specification consists of these pieces:

- Metadata specification (see Metadata.md)
- Physical memory layout specification (see Layout.md)
- Metadata serialized representation (see Message.fbs)
- Logical Types, Schemas, and Record Batch Metadata (see Schema.fbs)
- Encapsulated Messages (see Message.fbs)
- Mechanics of messaging between Arrow systems (IPC, RPC, etc.) (see IPC.md)
- Tensor (Multi-dimensional array) Metadata (see Tensor.fbs)

The metadata currently uses Google's [flatbuffers library][1] for serializing a
couple related pieces of information:
Expand All @@ -35,4 +33,16 @@ couple related pieces of information:
schema, and enable a system to send and receive Arrow row batches in a form
that can be precisely disassembled or reconstructed.

## Arrow Format Maturity and Stability

We have made significant progress hardening the Arrow in-memory format and
Flatbuffer metadata since the project started in February 2016. We have
integration tests which verify binary compatibility between the Java and C++
implementations, for example.

Major versions may still include breaking changes to the memory format or
metadata, so it is recommended to use the same released version of all
libraries in your applications for maximum compatibility. Data stored in the
Arrow IPC formats should not be used for long term storage.

[1]: http://github.com/google/flatbuffers

0 comments on commit 38efabe

Please sign in to comment.