Apache Flume 1.7.0 发布，日志服务器
欢迎加入运维网交流群：263444886Apache Flume 1.7.0 发布了，Flume 是一个分布式、可靠和高可用的服务，用于收集、聚合以及移动大量日志数据，使用一个简单灵活的架构，就流数据模型。这是一个可靠、容错的服务。
本次更新如下：
新特性
[*] - Implement Taildir Source
改进
[*] - Make SpoolDir work with Sub-Directories
[*] - Build flume by jdk 7 in default
[*] - Add an optional parameter to specify the expected input text encoding for the netcat sourcef the netcat source
[*] - Configurable poll delay for spooling directory source
[*] - HTTP Source to support generic Stream Handler
[*] - Allow pollableSource backoff times to be configurable
[*] - Kafka Source reading multiple topics
[*] - A Kafka Channel defined as parseAsFlumeEvent=false cannot be correctly used by a Flume source
[*] - Kafka Source - Message Offset and Partition add to headers
[*] - Performance improvement on TailDir source
[*] - Add static Schema URL to AvroEventSerializer configuration
[*] - Support New Kafka APIs
[*] - Kafka Source/Sink should optionally read/write Flume records
[*] - Kafka Channel partition topic by key
[*] - Kafka Sink should be able to select which header as the key
[*] - Allow RollingFileSink to specify a file prefix and a file extension.
[*] - Bump Rat version
[*] - AsyncHBaseSink - Failure callbacks should log the exception that caused them
[*] - Add includePattern option in SpoolDirectorySource configuration
[*] - TaildirSource is underperforming with huge parent directories
[*]
- Integrate checkstyle for non-test>
[*]
- Integrate checkstyle for test>
[*] - make raw data appearing in log messages explicit
[*] - Add file path to the header in TaildirSource
[*] - Fix issues with flume-checkstyle module
[*] - Add localhost escape sequence to HDFS sink
[*] - Kafka channel and sink should enable statically assigned partition per event via header
[*] - Flume-Kafka Source with new Consumer
[*] - Flume-Kafka-Sink with new Producer
[*] - Flume-Kafka-Channel with new APIs
Bug 修复
[*] - Hdfs Sink File Rollover
[*] - Exception while syncing from Flume to HDFS
[*] - Flume build occasionally fails with OutOfMemoryError on Windows.
[*] - ResettableFileInputStream can’t support ucs-4 character
[*] - SpoolingDirectory is unable to handle empty files
[*] - Building flume from trunk failing with dependency error
[*] - NullPointerException in Kafka Sink test
[*] - Thrift Source tests fail on Oracle JDK 8
[*] - Some TestFileChannelRestart tests are extremely slow
[*] - Remove unneeded repository declarations in pom.xml
[*] - flume-ng –conf parameter is not used when starting a flume agent
[*] - ResettableFileInputStream returns negate values from read() method
[*] - Spooldir source does not log channel exceptions
[*] - High CPU on KafkaSink
[*] - Documented transaction handling semantics incorrect
[*] - Add documentation for EventValidator
[*] - NPE in KafkaSourceCounter
[*] - Optional channel errors slows down the Source to Main channel event rate
[*] - HDFS Sink does not use configured timezone for rounding
[*] - Make maximum tolerated failures before shutting down and recreating client in AsyncHbaseSink configurable
[*] - Kafka Channel timeout property is overridden by default value
[*] - Async HBase sink FD leak on client shutdown
[*] - How to include this Flume Patch in Flume 1.5.2 ?
[*] - Kerberos configuration error when using short names in multiple HDFS Sinks
[*] - Upgrade Derby version to 10.11.1.1
[*] - Error when specifying empty replace string in Search and Replace Interceptor
[*] - Hive Sink skipping first transaction in each Batch of Hive Transactions
[*] - Move Hive sink out of preview mode
[*] - flume_env script should handle jvm parameters like -javaagent -agentpath -agentlib
[*] - TailDirSource throws FileNotFound Exception if ~/.flume directory is not created already
[*]
- SyslogTcpSource uses Deprecated>
[*] - Malformed Syslog messages can lead to OutOfMemoryException
[*] - Hive sink - abort remaining transactions on shutdown
[*] - flume-ng.ps1 Error running script to start an agent on Windows
[*] - Hive Sink tests need to create table with transactional property set
[*] - Upgrade commons-collections to 3.2.2
[*] - ChannelCounter of SpillableMemoryChannel doesn’t register actually.
[*] - Windows Launch Script fails in plugins dir code
[*] - Optional Channels can cause OOMs
[*] - Fixes to DateTime computations
[*] - Revert FLUME-2712 and FLUME-2886
[*] - AsyncHBase sink NPE when Channel.getTransaction() fails
[*] - Document Kerberos setup for Kafka channel
[*] - NetcatSource - SocketChannel not closed when session is broken
[*]
- Flume>
[*] - The kafka channel using new APIs will be stuck when the sink is avro sink
[*] - Kafka Channel Should Not Commit Offsets When Stopping
[*] - HDFSSequenceFile Should Sync Writer
[*] - Bump AsyncHBase version
[*] - KafkaSource tests arbitrarily fail
[*] - Upgrade recursive SpoolDir to use Java7 features
[*] - Docs: Fixed parameters on Replicating Channel Selector documentation example
[*] - Flume fails to build on Windows
[*]
->
[*] - SyslogAgent possible NPE on stop()
[*] - Handle offset migration in the new Kafka Channel
[*] - Some tests are broken in TestReliableSpoolingFileEventReader and TestSpoolingFileLineReader
[*] - Handle offset migration in the new Kafka Source
文档
[*] - FLUME-2548 brings SSLv2Hello back for Avro Sink, but UG says it is one of the protocols to exclude
[*] - Document Fault Tolerant Config parameters in FlumeUserGuide
[*] - Documentation for Pollable Source config parameters introduced in FLUME-2729
[*] - Update Website Team page with new Committer’s
[*] - Typo in Twitter source warning
[*] - Document new cachePatternMatching option for TaildirSource
[*] - FlumeUserGuide - error in Kafka Source properties table
[*] - Document secure Kafka Sink/Source/Channel setup
[*] - Minor mistake in NetCat Source example in documentation
[*] - Add missing configuration parameter to SequenceSource docs
任务
[*] - Bump java target version to 1.7
测试
[*] - testSourceCounter in TestSyslogUdpSource is flaky
点此查看发行主页和完整更新日志
下载地址
页:
[1]