Add VersionFieldSerializer, update README.md

README.md

@@ -275,7 +275,7 @@ In this example, FieldSerializer will be used for SomeClass. FieldSerializer is
 
 By default, most classes will end up using FieldSerializer. It essentially does what hand written serialization would, but does it automatically. FieldSerializer does direct assignment to the object's fields. If the fields are public, protected, or default access (package private), bytecode generation is used for maximum speed (see [ReflectASM](https://github.com/EsotericSoftware/reflectasm)). For private fields, setAccessible and cached reflection is used, which is still quite fast.
 
-Other general purpose serializes are provided, such as BeanSerializer, TaggedFieldSerializer, and CompatibleFieldSerializer. Additional serializers are available in a separate project on github, [kryo-serializers](https://github.com/magro/kryo-serializers).
+Other general purpose serializes are provided, such as BeanSerializer, TaggedFieldSerializer, CompatibleFieldSerializer, and VersionFieldSerializer. Additional serializers are available in a separate project on github, [kryo-serializers](https://github.com/magro/kryo-serializers).
 
 ## KryoSerializable
 
@@ -525,6 +525,8 @@ For some needs, especially long term storage of serialized bytes, it can be impo
 
 TaggedFieldSerializer only serializes fields that have a @Tag annotation. This is less flexible than FieldSerializer, which can handle most classes without needing annotations, but allows TaggedFieldSerializer to support both renaming fields and adding new fields without invalidating previously serialized bytes. Fields with the @Tag annotation should never be removed, instead they can be marked with the @Deprecated annotation. Deprecated fields can optionally be renamed so they don't clutter the class (eg, `ignored`, `ignored2`).
 
+VersionFieldSerializer supports fields to have a @Since annotation to indicate version. This will handle most backward-compatibility critical cases with little overhead, but removing, renaming or changing the type of any field may invalidate previous serialized bytes. For a particular field, value in @Since should never change once created.
+
 Alternatively, CompatibleFieldSerializer can be used, which writes a simple schema before the object data the first time the class is encountered in the serialized bytes. Like FieldSerializer, it can serialize most classes without needing annotations. Fields can be added or removed without invalidating previously serialized bytes, but changing the type of a field is not supported.
 
 FieldSerializer is most efficient, but does not allow the serialized classes to be changed. TaggedFieldSerializer is very efficient, usually the serialized size is only increased by 1 additional byte per field and there is no speed penalty. TaggedFieldSerializer allows classes to evolve and is the ideal solution when backward compatibility is required. CompatibleFieldSerializer is much less efficient, both in speed and size. The schema it writes contains field names as strings and it has to allocate buffers during serialization and deserialization. CompatibleFieldSerializer should only be used if TaggedFieldSerializer cannot.

src/com/esotericsoftware/kryo/serializers/VersioningFieldSerializer.java → src/com/esotericsoftware/kryo/serializers/VersionFieldSerializer.java

@@ -31,44 +31,46 @@
 import static com.esotericsoftware.minlog.Log.DEBUG;
 import static com.esotericsoftware.minlog.Log.debug;
 
-/** Serializes objects using direct field assignment, with versioning backward compatibility. Fields can be
- * added without invalidating previously serialized bytes. Note that removing, renaming or changing the type of a field is not supported.
+/** Serializes objects using direct field assignment, with versioning backward compatibility.
+ * Fields can be added without invalidating previously serialized bytes.
+ * Note that removing, renaming or changing the type of a field is not always supported.
  * In addition, forward compatibility is not considered.
+ * (Use with caution, this could cause cached objects seems invalidated from the view of older version of your program)
  * <p>
- * There is additional overhead compared to {@link FieldSerializer}. 
- * A varible length version code is appended before object. 
+ * There is a little additional overhead compared to {@link FieldSerializer}.
+ * A variable length version code is appended before every object.
  * When deserializing, input version will be examined to decide which field to skip.
+ * If there is any unrecognized field in input (which not included in current type), the deserialization may fail.
  * <p>
+ *
  * @author Tianyi HE <[email protected]> */
-public class VersioningFieldSerializer<T> extends FieldSerializer<T> {
+public class VersionFieldSerializer<T> extends FieldSerializer<T> {
 
-    // use magic number to assume data format, this is not precise but will work
-    // unicode char: 釒
-    private final static char FORMAT_MAGIC = 37330;
-
-    // minimal version of each field
+    // version of each field
     private int[] fieldVersion;
 
+    // version of current type
     private int typeVersion = 0;
 
     /**
-     * modification on persist entities must use since to annotate new fields
-     * in order to maintain backward compatibility
+     * incremental modification on persist objects must add {@link Since} new fields
      */
     @Retention(RetentionPolicy.RUNTIME)
     @Target(ElementType.FIELD)
     public @interface Since {
 
         /**
-         * Version of field, must less than 32768
+         * Version of annotated field, default is 0, and must be incremental to
+         * maintain compatibility.
          *
          * @return
          */
-        short value() default 0;
+        int value() default 0;
     }
 
-    public VersioningFieldSerializer(Kryo kryo, Class type) {
+    public VersionFieldSerializer(Kryo kryo, Class type) {
         super(kryo, type);
+        // make sure this is done before any read / write operations
         initializeCachedFields();
     }
 
@@ -106,9 +108,7 @@ public void removeField(CachedField field) {
     @Override
     public void write(Kryo kryo, Output output, T object) {
         CachedField[] fields = getFields();
-        // write magic number
-        output.writeChar(FORMAT_MAGIC);
-        // write version
+        // write type version
         output.writeVarInt(typeVersion, true);
         // write fields
         for (int i = 0, n = fields.length; i < n; i++) {
@@ -120,21 +120,12 @@ public void write(Kryo kryo, Output output, T object) {
     public T read(Kryo kryo, Input input, Class<T> type) {
         T object = create(kryo, input, type);
         kryo.reference(object);
-        // read magic
-        char magic = input.readChar();
-        int version = 0;
-        // magic indicates whether input supports versioning
-        if (magic != FORMAT_MAGIC) {
-            // roll back previous operation, first 2 bytes may overlap the first field
-            input.setPosition(input.position() - 2);
-            if (DEBUG)
-                debug("No version information, using 0");
-        } else {
-            // format is verified, read version
-            version = input.readVarInt(true);
-        }
+
+        // read input version
+        int version = input.readVarInt(true);
         CachedField[] fields = getFields();
         for (int i = 0, n = fields.length; i < n; i++) {
+            // field would not present in input, skip it
             if (fieldVersion[i] > version) {
                 if (DEBUG)
                     debug("Skip field " + fields[i].getField().getName());