release_1.0.0-changelog_known_issues.md

title
Release 1.0.0 – Changelog and Known Issues

• toc {:toc}

Changelog

Sub-task

• [FLINK-7] - [GitHub] Enable Range Partitioner
• [FLINK-146] - Sorted output not working
• [FLINK-1982] - Remove dependencies on Record for Flink runtime and core
• [FLINK-2676] - Add abstraction for keyed window state
• [FLINK-2680] - Create a dedicated aligned-event time window operator
• [FLINK-2730] - Add CPU/Network utilization graphs to new web dashboard
• [FLINK-2732] - Add access to the TaskManagers' log file and out file in the web dashboard.
• [FLINK-2850] - Limit the types of jobs which can run in detached mode
• [FLINK-2853] - Apply JMH on MutableHashTablePerformanceBenchmark class.
• [FLINK-2869] - Apply JMH on IOManagerPerformanceBenchmark class.
• [FLINK-2889] - Apply JMH on LongSerializationSpeedBenchmark class
• [FLINK-2890] - Apply JMH on StringSerializationSpeedBenchmark class.
• [FLINK-2900] - Remove Record-API dependencies from Hadoop Compat module
• [FLINK-2901] - Several flink-test ITCases depend on Record API features
• [FLINK-2906] - Remove Record-API
• [FLINK-2919] - Apply JMH on FieldAccessMinibenchmark class.
• [FLINK-2920] - Apply JMH on KryoVersusAvroMinibenchmark class.
• [FLINK-2933] - Flink scala libraries exposed with maven should carry scala version
• [FLINK-2972] - Remove Twitter Chill dependency from flink-java module
• [FLINK-3057] - [py] Provide a way to pass information back to the plan process
• [FLINK-3070] - Create an asynchronous state handle interface
• [FLINK-3071] - Add asynchronous materialization thread
• [FLINK-3140] - NULL value data layout in Row Serializer/Comparator
• [FLINK-3195] - Restructure examples projects and package streaming examples
• [FLINK-3201] - Enhance Partitioned State Interface with State Types
• [FLINK-3208] - Rename Gelly vertex-centric model to scatter-gather
• [FLINK-3224] - The Streaming API does not call setInputType if a format implements InputTypeConfigurable
• [FLINK-3275] - [py] Add support for Dataset.setParallelism()
• [FLINK-3285] - Skip Maven deployment of flink-java8
• [FLINK-3312] - Add convenience accessor methods for extended state interface
• [FLINK-3327] - Attach the ExecutionConfig to the JobGraph and make it accessible to the AbstractInvocable.
• [FLINK-3363] - JobManager does not shut down dedicated executor properly
• [FLINK-3365] - BlobLibraryCacheManager does not shutdown Timer thread
• [FLINK-3371] - Move TriggerCotext and TriggerResult to their own classes
• [FLINK-3384] - Create atomic closable queue for communication between Kafka Threads
• [FLINK-3388] - Expose task accumulators via JMX
• [FLINK-3401] - AscendingTimestampExtractor should not fail on order violation
• [FLINK-3437] - Fix UI router state for job plan
• [FLINK-3468] - Change Timestamp Extractors to not support negative timestamps
• [FLINK-3470] - EventTime WindowAssigners should error on negative timestamps

Bug

• [FLINK-1278] - Remove the Record special code paths
• [FLINK-1644] - WebClient dies when no ExecutionEnvironment in main method
• [FLINK-1989] - Sorting of POJO data set from TableEnv yields NotSerializableException
• [FLINK-2115] - TableAPI throws ExpressionException for "Dangling GroupBy operation"
• [FLINK-2263] - ExecutionGraph uses Thread.sleep to delay execution retries
• [FLINK-2315] - Hadoop Writables cannot exploit implementing NormalizableKey
• [FLINK-2348] - IterateExampleITCase failing
• [FLINK-2351] - Deprecate config builders in InputFormats and Output formats
• [FLINK-2369] - On Windows, in testFailingSortingDataSinkTask the temp file is not removed
• [FLINK-2392] - Instable test in flink-yarn-tests
• [FLINK-2443] - [CompactingHashTable] GSA Connected Components fails with NPE
• [FLINK-2491] - Operators are not participating in state checkpointing in some cases
• [FLINK-2504] - ExternalSortLargeRecordsITCase.testSortWithLongAndShortRecordsMixed failed spuriously
• [FLINK-2586] - Unstable Storm Compatibility Tests
• [FLINK-2608] - Arrays.asList(..) does not work with CollectionInputFormat
• [FLINK-2622] - Scala DataStream API does not have writeAsText method which supports WriteMode
• [FLINK-2624] - RabbitMQ source / sink should participate in checkpointing
• [FLINK-2662] - CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators."
• [FLINK-2671] - Instable Test StreamCheckpointNotifierITCase
• [FLINK-2674] - Rework windowing logic
• [FLINK-2695] - KafkaITCase.testConcurrentProducerConsumerTopology failed on Travis
• [FLINK-2719] - ProcessFailureStreamingRecoveryITCase>AbstractProcessFailureRecoveryTest.testTaskManagerProcessFailure failed on Travis
• [FLINK-2739] - Release script depends on the order of parent module information
• [FLINK-2747] - TypeExtractor does not correctly analyze Scala Immutables (AnyVal)
• [FLINK-2752] - Documentation is not easily differentiable from the Flink homepage
• [FLINK-2757] - DataSinkTaskTest fails on Windows
• [FLINK-2758] - TaskManagerRegistrationTest fails on Windows
• [FLINK-2759] - TaskManagerProcessReapingTest fails on Windows
• [FLINK-2769] - Web dashboard port not configurable on client side
• [FLINK-2771] - IterateTest.testSimpleIteration fails on Travis
• [FLINK-2797] - CLI: Missing option to submit jobs in detached mode
• [FLINK-2799] - Yarn tests cannot be executed with DEBUG log level
• [FLINK-2800] - kryo serialization problem
• [FLINK-2823] - YARN client should report a proper exception if Hadoop Env variables are not set
• [FLINK-2826] - transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking
• [FLINK-2827] - Potential resource leak in TwitterSource#loadAuthenticationProperties()
• [FLINK-2832] - Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement
• [FLINK-2838] - Inconsistent use of URL and Path classes for resources
• [FLINK-2845] - TimestampITCase.testWatermarkPropagation
• [FLINK-2879] - Links in documentation are broken
• [FLINK-2913] - Close of ObjectOutputStream should be enclosed in finally block in FsStateBackend
• [FLINK-2914] - Missing break in ZooKeeperSubmittedJobGraphStore#SubmittedJobGraphsPathCacheListener#childEvent()
• [FLINK-2930] - ExecutionConfig execution retry delay not respected
• [FLINK-2934] - Remove placehoder pages in Web dashboard
• [FLINK-2937] - Typo in Quickstart->Scala API->Alternative Build Tools: SBT
• [FLINK-2938] - Streaming docs not in sync with latest state changes
• [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic)
• [FLINK-2950] - Markdown presentation problem in SVM documentation
• [FLINK-2954] - Not able to pass custom environment variables in cluster to processes that spawning TaskManager
• [FLINK-2958] - StreamingJobGraphGenerator sets hard coded number execution retry
• [FLINK-2963] - Dependence on SerializationUtils#deserialize() should be avoided
• [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs
• [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster
• [FLINK-2979] - RollingSink does not work with Hadoop 2.7.1
• [FLINK-2987] - Flink 0.10 fails to start on YARN 2.6.0
• [FLINK-2989] - Job Cancel button doesn't work on Yarn
• [FLINK-2990] - Scala 2.11 build fails to start on YARN
• [FLINK-2992] - New Windowing code is using SerializationUtils with wrong classloader
• [FLINK-3000] - Add ShutdownHook to YARN CLI to prevent lingering sessions
• [FLINK-3005] - Commons-collections object deserialization remote command execution vulnerability
• [FLINK-3009] - Cannot build docs with Jekyll 3.0.0
• [FLINK-3011] - Cannot cancel failing/restarting streaming job from the command line
• [FLINK-3013] - Incorrect package declaration in GellyScalaAPICompletenessTest.scala
• [FLINK-3019] - CLI does not list running/restarting jobs
• [FLINK-3020] - Local streaming execution: set number of task manager slots to the maximum parallelism
• [FLINK-3022] - Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide
• [FLINK-3024] - TimestampExtractor Does not Work When returning Long.MIN_VALUE
• [FLINK-3025] - Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug
• [FLINK-3032] - Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict
• [FLINK-3043] - Kafka Connector description in Streaming API guide is wrong/outdated
• [FLINK-3047] - Local batch execution: set number of task manager slots to the maximum parallelism
• [FLINK-3048] - DataSinkTaskTest.testCancelDataSinkTask
• [FLINK-3052] - Optimizer does not push properties out of bulk iterations
• [FLINK-3054] - Remove R (return) type variable from SerializationSchema
• [FLINK-3059] - Javadoc fix for DataSet.writeAsText()
• [FLINK-3061] - Kafka Consumer is not failing if broker is not available
• [FLINK-3062] - Kafka Producer is not failing if broker is not available/no partitions available
• [FLINK-3067] - Kafka source fails during checkpoint notifications with NPE
• [FLINK-3080] - Cannot union a data stream with a product of itself
• [FLINK-3081] - Kafka Periodic Offset Committer does not properly terminate on canceling
• [FLINK-3082] - Confusing error about ManualTimestampSourceFunction
• [FLINK-3087] - Table API do not support multi count in aggregation.
• [FLINK-3098] - Cast from Date to Long throw compile error.
• [FLINK-3100] - Signal handler prints error on normal shutdown of cluster
• [FLINK-3101] - Flink Kafka consumer crashes with NPE when it sees deleted record
• [FLINK-3103] - Remove synchronization in FsStateBackend#FsCheckpointStateOutputStream#close()
• [FLINK-3108] - JoinOperator's with() calls the wrong TypeExtractor method
• [FLINK-3117] - Storm Tick Tuples are not supported
• [FLINK-3118] - Check if MessageFunction implements ResultTypeQueryable
• [FLINK-3121] - Watermark forwarding does not work for sources not producing any data
• [FLINK-3125] - Web dashboard does not start when log files are not found
• [FLINK-3134] - Make YarnJobManager's allocate call asynchronous
• [FLINK-3136] - Scala Closure Cleaner uses wrong ASM import
• [FLINK-3138] - Method References are not supported as lambda expressions
• [FLINK-3143] - Update Clojure Cleaner's ASM references to ASM5
• [FLINK-3144] - [storm] LocalCluster prints nothing without a configured logger
• [FLINK-3145] - Storm examples can't be run without flink-java as dependency
• [FLINK-3151] - YARN kills Flink TM containers due to memory overuse (outside heap/offheap)
• [FLINK-3156] - FlinkKafkaConsumer fails with NPE on notifyCheckpointComplete
• [FLINK-3157] - Web frontend json files contain author attribution
• [FLINK-3166] - The first program in ObjectReuseITCase has the wrong expected result, and it succeeds
• [FLINK-3171] - Consolidate zoo of wrapper classes for input/output-stream to data-input/output-view
• [FLINK-3173] - Bump org.apache.httpcomponents.httpclient version to 4.2.6
• [FLINK-3175] - KafkaITCase.testOffsetAutocommitTest
• [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned
• [FLINK-3180] - MemoryLogger does not log direct memory
• [FLINK-3185] - Silent failure during job graph recovery
• [FLINK-3188] - Deletes in Kafka source should be passed on to KeyedDeserializationSchema
• [FLINK-3189] - Error while parsing job arguments passed by CLI
• [FLINK-3196] - InputStream should be closed in EnvironmentInformation#getRevisionInformation()
• [FLINK-3197] - InputStream not closed in BinaryInputFormat#createStatistics
• [FLINK-3206] - Heap size for non-pre-allocated off-heap memory
• [FLINK-3218] - Merging Hadoop configurations overrides user parameters
• [FLINK-3220] - Flink does not start on Hortonworks Sandbox 2.3.2 due to missing class
• [FLINK-3236] - Flink user code classloader should have Flink classloader as parent classloader
• [FLINK-3240] - Remove or document DataStream(.global|.forward)
• [FLINK-3242] - User-specified StateBackend is not Respected if Checkpointing is Disabled
• [FLINK-3243] - Fix Interplay of TimeCharacteristic and Time Windows
• [FLINK-3247] - Kafka Connector unusable with quickstarts - shading issue
• [FLINK-3248] - RMQSource does not provide a constructor for credentials or other options
• [FLINK-3250] - Savepoint coordinator requires too strict parallelism match
• [FLINK-3251] - Checkpoint stats show ghost numbers
• [FLINK-3252] - Checkpoint stats only displayed after click on graph
• [FLINK-3254] - CombineFunction interface not respected
• [FLINK-3255] - Chaining behavior should not depend on parallelism
• [FLINK-3260] - ExecutionGraph gets stuck in state FAILING
• [FLINK-3261] - Tasks should eagerly report back when they cannot start a checkpoint
• [FLINK-3266] - LocalFlinkMiniCluster leaks resources when multiple jobs are submitted
• [FLINK-3267] - Disable reference tracking in Kryo fallback serializer
• [FLINK-3268] - Unstable test JobManagerSubmittedJobGraphsRecoveryITCase
• [FLINK-3271] - Using webhdfs in a flink topology throws classnotfound exception
• [FLINK-3274] - Prefix Kafka connector accumulators with unique id
• [FLINK-3280] - Wrong usage of Boolean.getBoolean()
• [FLINK-3281] - IndexOutOfBoundsException when range-partitioning empty DataSet
• [FLINK-3286] - Remove JDEB Debian Package code from flink-dist
• [FLINK-3287] - Flink Kafka Consumer fails due to Curator version conflict
• [FLINK-3289] - Double reference to flink-contrib
• [FLINK-3292] - Bug in flink-jdbc. Not all JDBC drivers supported
• [FLINK-3293] - Custom Application Name on YARN is ignored in deploy jobmanager mode
• [FLINK-3296] - DataStream.write*() methods are not flushing properly
• [FLINK-3300] - Concurrency Bug in Yarn JobManager
• [FLINK-3304] - AvroOutputFormat.setSchema() doesn't work in yarn-cluster mode
• [FLINK-3314] - Early cancel calls can cause Tasks to not cancel properly
• [FLINK-3322] - MemoryManager creates too much GC pressure with iterative jobs
• [FLINK-3328] - Incorrectly shaded dependencies in flink-runtime
• [FLINK-3333] - Documentation about object reuse should be improved
• [FLINK-3337] - mvn test fails on flink-runtime because curator classes not found
• [FLINK-3338] - Kafka deserialization issue - ClassNotFoundException
• [FLINK-3339] - Checkpointing NPE when using filterWithState
• [FLINK-3340] - Fix object juggling in drivers
• [FLINK-3341] - Kafka connector's 'auto.offset.reset' inconsistent with Kafka
• [FLINK-3342] - Operator checkpoint statistics state size overflow
• [FLINK-3350] - Increase timeouts on Travis Builds
• [FLINK-3351] - RocksDB Backend cannot determine correct local db path
• [FLINK-3352] - RocksDB Backend cannot determine correct hdfs path
• [FLINK-3353] - CSV-related tests may fail depending on locale
• [FLINK-3357] - Drop JobId.toShortString()
• [FLINK-3359] - Make RocksDB file copies asynchronous
• [FLINK-3361] - Wrong error messages for execution retry delay and akka ask pause config values
• [FLINK-3368] - Kafka 0.8 consumer fails to recover from broker shutdowns
• [FLINK-3369] - RemoteTransportException should be instance of CancelTaskException
• [FLINK-3373] - Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems
• [FLINK-3385] - Fix outer join skipping unprobed partitions
• [FLINK-3392] - Unprotected access to elements in ClosableBlockingQueue#size()
• [FLINK-3393] - ExternalProcessRunner wait to finish copying error stream
• [FLINK-3394] - Clear up the contract of MutableObjectIterator.next(reuse)
• [FLINK-3396] - Job submission Savepoint restore logic flawed
• [FLINK-3400] - RocksDB Backend does not work when not in Flink lib folder
• [FLINK-3412] - Remove implicit conversions JavaStream / ScalaStream
• [FLINK-3415] - TimestampExctractor accepts negative watermarks
• [FLINK-3416] - [py] .bat files fail when path contains spaces
• [FLINK-3420] - Remove "ReadTextFileWithValue" from StreamExecutionEnvironment
• [FLINK-3423] - ExternalProcessRunnerTest fails on Windows
• [FLINK-3424] - FileStateBackendtest.testStateOutputStream fails on windows
• [FLINK-3425] - DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows
• [FLINK-3426] - JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows
• [FLINK-3432] - ZookeeperOffsetHandlerTest fails on windows
• [FLINK-3434] - Return value from flinkYarnClient#deploy() should be checked against null
• [FLINK-3438] - ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings
• [FLINK-3439] - Remove final Long.MAX_VALUE Watermark in StreamSource
• [FLINK-3440] - Kafka should also checkpoint partitions where no initial offset was retrieved
• [FLINK-3448] - WebRuntimeMonitor: Move initialization of handlers to start() method
• [FLINK-3449] - createInput swallows exception if TypeExtractor fails
• [FLINK-3450] - RocksDB Backed Window Fails with KryoSerializer
• [FLINK-3453] - Fix TaskManager logs exception when sampling backpressure while task completes
• [FLINK-3458] - Shading broken in flink-shaded-hadoop
• [FLINK-3478] - Flink serves arbitary files through the web interface
• [FLINK-3483] - Job graph visualization not working properly in OS X Chrome
• [FLINK-3485] - The SerializedListAccumulator value doest seem to be right
• [FLINK-3499] - Possible ghost references in ZooKeeper completed checkpoint store
• [FLINK-3511] - Flink library examples not runnable without adding dependencies
• [FLINK-3512] - Savepoint backend should not revert to "jobmanager"
• [FLINK-3513] - Fix interplay of automatic Operator UID and Changing name of WindowOperator
• [FLINK-3518] - Stale docs for quickstart setup
• [FLINK-3525] - Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close()
• [FLINK-3526] - ProcessingTime Window Assigner and Trigger are broken
• [FLINK-3534] - Cancelling a running job can lead to restart instead of stopping
• [FLINK-3540] - Hadoop 2.6.3 build contains /com/google/common (guava) classes in flink-dist.jar
• [FLINK-3554] - Bounded sources should emit a Max Watermark when they are done
• [FLINK-3556] - Unneeded check in HA blob store configuration
• [FLINK-3562] - Update docs in the course of EventTimeSourceFunction removal
• [FLINK-3565] - FlinkKafkaConsumer does not work with Scala 2.11
• [FLINK-3569] - Test cases fail due to Maven Shade plugin

Improvement

• [FLINK-8] - [GitHub] Implement automatic sample histogram building for Range Partitioning
• [FLINK-553] - Add getGroupKey() method to group-at-time operators
• [FLINK-573] - Clean-up MapOperators in optimizer
• [FLINK-734] - Integrate web job client into JobManager web interface
• [FLINK-987] - Extend TypeSerializers and -Comparators to work directly on Memory Segments
• [FLINK-1045] - Remove Combinable Annotation
• [FLINK-1228] - Add REST Interface to JobManager
• [FLINK-1240] - We cannot use sortGroup on a global reduce
• [FLINK-1513] - Remove GlobalConfiguration Singleton
• [FLINK-1666] - Clean-up Field Expression Code
• [FLINK-1702] - Authenticate via Kerberos from the client only
• [FLINK-1778] - Improve normalized keys in composite key case
• [FLINK-1903] - Joins where one side uses a field more than once don't work
• [FLINK-1947] - Make Avro and Tachyon test logging less verbose
• [FLINK-2017] - Add predefined required parameters to ParameterTool
• [FLINK-2021] - Rework examples to use ParameterTool
• [FLINK-2075] - Shade akka and protobuf dependencies away
• [FLINK-2185] - Rework semantics for .setSeed function of SVM
• [FLINK-2213] - Configure number of vcores
• [FLINK-2239] - print() on DataSet: stream results and print incrementally
• [FLINK-2342] - Add new fit operation and more tests for StandardScaler
• [FLINK-2344] - Deprecate/Remove the old Pact Pair type
• [FLINK-2380] - Allow to configure default FS for file inputs
• [FLINK-2399] - Fail when actor versions don't match
• [FLINK-2455] - Misleading I/O manager error log messages
• [FLINK-2488] - Expose attemptNumber in RuntimeContext
• [FLINK-2518] - Avoid predetermination of ports for network services
• [FLINK-2523] - Make task canceling interrupt interval configurable
• [FLINK-2524] - Add "getTaskNameWithSubtasks()" to RuntimeContext
• [FLINK-2559] - Fix Javadoc (e.g. Code Examples)
• [FLINK-2646] - Rich functions should provide a method "closeAfterFailure()"
• [FLINK-2667] - Rework configuration parameters
• [FLINK-2716] - Checksum method for DataSet and Graph
• [FLINK-2788] - Add type hint with TypeExtactor call on Hint Type
• [FLINK-2795] - Print JobExecutionResult for interactively invoked jobs
• [FLINK-2882] - Improve performance of string conversions
• [FLINK-2893] - Rename recovery configuration keys
• [FLINK-2895] - Duplicate immutable object creation
• [FLINK-2897] - Use distinct initial indices for OutputEmitter round-robin
• [FLINK-2898] - Invert Travis CI build order
• [FLINK-2902] - Web interface sort tasks newest first
• [FLINK-2904] - Web interface truncated task counts
• [FLINK-2932] - Flink quickstart docs should ask users to download from https, not http
• [FLINK-2936] - ClassCastException when using EventTimeSourceFunction in non-EventTime program
• [FLINK-2940] - Deploy multiple Scala versions for Maven artifacts
• [FLINK-2947] - Coloured Scala Shell
• [FLINK-2949] - Add method 'writeSequencefile' to DataSet
• [FLINK-2961] - Add support for basic type Date in Table API
• [FLINK-2962] - Cluster startup script refers to unused variable
• [FLINK-2966] - Improve the way job duration is reported on web frontend.
• [FLINK-2974] - Add periodic offset commit to Kafka Consumer if checkpointing is disabled
• [FLINK-2976] - Save and load checkpoints manually
• [FLINK-2981] - Update README for building docs
• [FLINK-2984] - Support lenient parsing of SVMLight input files
• [FLINK-2991] - Extend Window Operators to Allow Efficient Fold Operation
• [FLINK-2993] - Set default DelayBetweenExecutionRetries to 0
• [FLINK-2994] - Client sysout logging does not report exceptions
• [FLINK-3003] - Add container allocation timeout to YARN CLI
• [FLINK-3017] - Broken 'Slots' link on Streaming Programming Guide
• [FLINK-3023] - Show Flink version + commit id for -SNAPSHOT versions in web frontend
• [FLINK-3028] - Cannot cancel restarting job via web frontend
• [FLINK-3030] - Enhance Dashboard to show Execution Attempts
• [FLINK-3039] - Trigger KeyValueState cannot be Scala Int
• [FLINK-3040] - Add docs describing how to configure State Backends
• [FLINK-3042] - Define a way to let types create their own TypeInformation
• [FLINK-3044] - In YARN mode, configure FsStateBackend by default.
• [FLINK-3045] - Properly expose the key of a kafka message
• [FLINK-3046] - Integrate the Either Java type with the TypeExtractor
• [FLINK-3049] - Move "Either" type to package "org.apache.flink.types"
• [FLINK-3050] - Add custom Exception type to suppress job restarts
• [FLINK-3051] - Define a maximum number of concurrent inflight checkpoints
• [FLINK-3055] - ExecutionVertex has duplicate method getParallelSubtaskIndex and getSubTaskIndex
• [FLINK-3056] - Show bytes sent/received as MBs/GB and so on in web interface
• [FLINK-3063] - [py] Remove combiner
• [FLINK-3069] - Make state materialization asynchronous
• [FLINK-3073] - Activate streaming mode by default
• [FLINK-3074] - Make ApplicationMaster/JobManager akka port configurable
• [FLINK-3076] - Display channel exchange mode in the plan visualizer
• [FLINK-3077] - Add "version" command to CliFrontend for showing the version of the installation
• [FLINK-3083] - Add docs how to configure streaming fault tolerance
• [FLINK-3084] - File State Backend should not write very small state into files
• [FLINK-3085] - Move State Backend Initialization from "registerInputOutput()" to "invoke()"
• [FLINK-3114] - Read cluster's default parallelism upon remote job submission
• [FLINK-3115] - Update Elasticsearch connector to 2.X
• [FLINK-3116] - Remove RecordOperator
• [FLINK-3120] - Set number of event loop thread and number of buffer pool arenas to same number
• [FLINK-3122] - Generalize value type in LabelPropagation
• [FLINK-3123] - Allow setting custom start-offsets for the Kafka consumer
• [FLINK-3124] - Introduce a TaskInfo object to better represent task name, index, attempt number etc.
• [FLINK-3127] - Measure backpressure in Flink jobs
• [FLINK-3131] - Expose checkpoint metrics
• [FLINK-3132] - Restructure streaming guide
• [FLINK-3133] - Introduce collect()/coun()/print() methods in DataStream API
• [FLINK-3135] - Add chainable driver for UNARY_NO_OP strategy
• [FLINK-3147] - HadoopOutputFormatBase should expose fields as protected
• [FLINK-3154] - Update Kryo version from 2.24.0 to 3.0.3
• [FLINK-3160] - Aggregate operator statistics by TaskManager
• [FLINK-3161] - Externalize cluster start-up and tear-down when available
• [FLINK-3169] - Drop {{Key}} type from Record Data Model
• [FLINK-3172] - Specify jobmanager port in HA mode
• [FLINK-3174] - Add merging WindowAssigner
• [FLINK-3176] - Window Apply Website Example
• [FLINK-3178] - Make Closing Behavior of Window Operators Configurable
• [FLINK-3181] - The vertex-centric SSSP example and library method send unnecessary messages during the first superstep
• [FLINK-3194] - Remove web client
• [FLINK-3198] - Rename Grouping.getDataSet() method and add JavaDocs
• [FLINK-3200] - Use Partitioned State Abstraction in WindowOperator
• [FLINK-3209] - Remove Unused ProcessingTime, EventTime and AbstractTime
• [FLINK-3213] - Union of two streams with different parallelism should adjust parallelism automatically
• [FLINK-3219] - Implement DataSet.count using a single operator
• [FLINK-3232] - Add option to eagerly deploy channels
• [FLINK-3233] - PartitionOperator does not support expression keys on atomic types
• [FLINK-3234] - SortPartition does not support KeySelectorFunctions
• [FLINK-3235] - Drop Flink-on-Tez code
• [FLINK-3244] - Add log messages to savepoint coordinator restore
• [FLINK-3246] - Consolidate maven project names with *-parent suffix
• [FLINK-3249] - Wrong "unknown partition state/input gate" error messages
• [FLINK-3258] - Merge AbstractInvokable's registerInputOutput and invoke
• [FLINK-3259] - Redirect programming guides to new layout
• [FLINK-3262] - Remove fuzzy versioning from Bower dependencies
• [FLINK-3265] - RabbitMQ Source not threadsafe: ConcurrentModificationException
• [FLINK-3270] - Add example for reading and writing to Kafka
• [FLINK-3273] - Remove Scala dependency from flink-streaming-java
• [FLINK-3276] - Move runtime parts of flink-streaming-java to flink-runtime
• [FLINK-3278] - Add Partitioned State Backend Based on RocksDB
• [FLINK-3290] - [py] Generalize OperationInfo transfer
• [FLINK-3299] - Remove ApplicationID from Environment
• [FLINK-3303] - Move all non-batch specific classes in flink-java to flink-core
• [FLINK-3305] - Remove JavaKaffee serializer util
• [FLINK-3306] - Auto type registration at Kryo is buggy
• [FLINK-3308] - [py] Remove debug mode
• [FLINK-3309] - [py] Resolve maven warnings
• [FLINK-3310] - Add back pressure statistics to web frontend
• [FLINK-3334] - Change default log4j.properties Conversion pattern to include year-month-day in the timestamp
• [FLINK-3336] - Add Semi-Rebalance Data Shipping for DataStream
• [FLINK-3347] - TaskManager ActorSystems need to restart themselves in case they notice quarantine
• [FLINK-3348] - taskmanager.memory.off-heap missing bc documentation
• [FLINK-3355] - Allow passing RocksDB Option to RocksDBStateBackend
• [FLINK-3358] - Create constructors for FsStateBackend in RocksDBBackens
• [FLINK-3364] - Don't initialize SavepointStore in JobManager constructor
• [FLINK-3375] - Allow Watermark Generation in the Kafka Source
• [FLINK-3376] - Add an illustration of Event Time and Watermarks to the docs
• [FLINK-3378] - Consolidate TestingCluster and FokableFlinkMiniCluster
• [FLINK-3379] - Refactor TimestampExtractor
• [FLINK-3386] - Kafka consumers are not properly respecting the "auto.offset.reset" behavior
• [FLINK-3389] - Add Pre-defined Options settings for RocksDB State backend
• [FLINK-3395] - Polishing the web UI
• [FLINK-3402] - Refactor Common Parts of Stream/Batch Documentation
• [FLINK-3413] - Remove implicit Seq to DataStream conversion
• [FLINK-3419] - Drop partitionByHash from DataStream
• [FLINK-3421] - Remove all unused ClassTag context bounds in the Streaming Scala API
• [FLINK-3422] - Scramble HashPartitioner hashes
• [FLINK-3430] - Remove "no POJO" warning in TypeAnalyzer
• [FLINK-3435] - Change interplay of Ingestion Time and Event Time
• [FLINK-3436] - Remove ComplexIntegrationITCase
• [FLINK-3455] - Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1
• [FLINK-3459] - Make build SBT compatible
• [FLINK-3460] - Make flink-streaming-connectors' flink dependencies provided
• [FLINK-3461] - Remove duplicate condition check in ZooKeeperLeaderElectionService
• [FLINK-3469] - Improve documentation for grouping keys
• [FLINK-3509] - Update Hadoop versions in release script and on travis to the latest minor version
• [FLINK-3520] - Periodic watermark operator should emit current watermark in close()
• [FLINK-3521] - Make Iterable part of method signature for WindowFunction
• [FLINK-3522] - Storm examples "PrintSampleStream" throws an error if called without arguments
• [FLINK-3527] - Scala DataStream has no transform method
• [FLINK-3528] - Add Incremental Fold for Non-Keyed Window Operator
• [FLINK-3535] - Decrease logging verbosity of StackTraceSampleCoordinator
• [FLINK-3536] - Make clearer distinction between event time and processing time
• [FLINK-3538] - DataStream join API does not enforce consistent usage
• [FLINK-3548] - Remove unnecessary generic parameter from SingleOutputStreamOperator
• [FLINK-3559] - Don't print pid file check if no active PID

New Feature

• [FLINK-1723] - Add cross validation for model evaluation
• [FLINK-2157] - Create evaluation framework for ML library
• [FLINK-2390] - Replace iteration timeout with algorithm for detecting termination
• [FLINK-2411] - Add basic graph summarization algorithm
• [FLINK-2435] - Add support for custom CSV field parsers
• [FLINK-2720] - Add Storm-CountMetric in flink-stormcompatibility
• [FLINK-2728] - Add missing features to new web dashboard
• [FLINK-2871] - Add OuterJoin strategy with HashTable on outer side
• [FLINK-2905] - Add intersect method to Graph class
• [FLINK-2951] - Add Union operator to Table API.
• [FLINK-2955] - Add operations introduction in Table API page.
• [FLINK-2978] - Integrate web submission interface into the new dashboard
• [FLINK-2996] - Add config entry to define BlobServer port
• [FLINK-3001] - Add Support for Java 8 Optional type
• [FLINK-3002] - Add an EitherType to the Java API
• [FLINK-3058] - Add Kafka consumer for new 0.9.0.0 Kafka API
• [FLINK-3093] - Introduce annotations for interface stability
• [FLINK-3102] - Allow reading from multiple topics with one FlinkKafkaConsumer
• [FLINK-3170] - Expose task manager metrics via JMX
• [FLINK-3192] - Add explain support to print ast and sql physical execution plan.

Task

• [FLINK-1681] - Remove the old Record API
• [FLINK-2973] - Add flink-benchmark with compliant licenses again
• [FLINK-3112] - Remove unused RecordModelPostPass class
• [FLINK-3113] - Remove unused global order methods from GenericDataSinkBase
• [FLINK-3155] - Update Flink docker version to latest stable Flink version
• [FLINK-3186] - Deprecate DataSink.sortLocalOutput() methods
• [FLINK-3366] - Rename @Experimental annotation to @PublicEvolving
• [FLINK-3367] - Annotate all user-facing API classes with @Public or @PublicEvolving
• [FLINK-3484] - Add setSlotSharingGroup documentation
• [FLINK-3490] - Bump Chill version to 0.7.4

Test

• [FLINK-2480] - Improving tests coverage for org.apache.flink.streaming.api
• [FLINK-2573] - Add Kerberos test case
• [FLINK-2803] - Add test case for Flink's memory allocation
• [FLINK-3517] - Number of job and task managers not checked in scripts

Wish

• [FLINK-2429] - Remove the "enableCheckpointing()" without interval variant
• [FLINK-2995] - Set default number of retries to larger than 0
• [FLINK-3377] - Remove final flag from ResultPartitionWriter class

Known Issues

• The mvn clean verify command does not succeed for the source release, because of issues with the Maven shade plugin. This has been resolved in the release-1.0 branch.
• FLINK-3578 - Scala DataStream API does not support Rich Window Functions.