Spring Batch for Rust, offers a robust and flexible framework for the development of batch processing applications, addressing the challenges of handling large-scale data processing tasks efficiently and reliably. It provides developers a comprehensive toolkit for building enterprise-grade batch applications.
Feature | Description |
---|---|
mongodb | Enable reader and writer for Mongodb database |
rdbc-postgres | Enable rdbc reader and writer for Postgres database |
rdbc-mysql | Enable rdbc reader and writer for Mysql and MariaDb databases |
rdbc-sqlite | Enable rdbc reader and writer for Sqlite database |
json | Enable json reader and writer |
csv | Enable csv reader and writer |
fake | Enable fake reader. Useful for generate fake dataset |
logger | Enable logger writer. Useful for debugging |
- XML reader and writer
- Filter items
- Kafka reader and writer
- Pulsar reader and writer
- Retry/Skip policies
- Save execution data in database
Make sure you activated the suitable features crate on Cargo.toml:
[dependencies]
spring-batch-rs = { version = "<version>", features = ["<full|json|csv|fake|logger>"] }
Then, on your main.rs:
fn main() -> Result<(), BatchError> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1995,Peugeot,205,City car
2021,Mazda,CX-30,SUV Compact
1967,Ford,Mustang fastback 1967,American car";
let reader = CsvItemReaderBuilder::new()
.delimiter(b',')
.has_headers(true)
.from_reader(csv.as_bytes());
let processor = UpperCaseProcessor::default();
let writer = JsonItemWriterBuilder::new().from_path(temp_dir().join("cars.json"));
let step: Step<Car, Car> = StepBuilder::new()
.reader(&reader) // set csv reader
.writer(&writer) // set json writer
.processor(&processor) // set upper case processor
.chunk(2) // set commit interval
.skip_limit(2) // set fault tolerance
.build();
let job = JobBuilder::new().start(Box::new(&step)).build();
let result = job.run();
assert!(result.is_ok());
assert!(step.get_status() == StepStatus::Success);
Ok(())
}
- Generate CSV file from JSON file with processor
- Generate JSON file from CSV string with fault tolerance
- Generate JSON file from fake persons
- Generate CSV file without headers from fake persons
- Insert records into Mysql database
- Log records from Postgres database
- Read records from MongoDb database
- Write records to MongoDb database
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions