Skip to content

A toolkit for building enterprise-grade batch applications

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT
Notifications You must be signed in to change notification settings

sboussekeyt/spring-batch-rs

Folders and files

NameName
Last commit message
Last commit date
Nov 18, 2024
Apr 1, 2024
Nov 12, 2024
Aug 11, 2024
Nov 12, 2024
Aug 2, 2024
Jan 10, 2024
Nov 18, 2024
Nov 12, 2024
Dec 22, 2023
Dec 22, 2023
Apr 1, 2024
Jan 9, 2024

Spring-Batch for Rust

๐Ÿž A toolkit for building enterprise-grade batch applications

crate docs build status Discord chat CodeCov license

Spring-Batch for Rust

Spring Batch for Rust, offers a robust and flexible framework for the development of batch processing applications, addressing the challenges of handling large-scale data processing tasks efficiently and reliably. It provides developers a comprehensive toolkit for building enterprise-grade batch applications.

Features

Feature Description
mongodb Enable reader and writer for Mongodb database
rdbc-postgres Enable rdbc reader and writer for Postgres database
rdbc-mysql Enable rdbc reader and writer for Mysql and MariaDb databases
rdbc-sqlite Enable rdbc reader and writer for Sqlite database
json Enable json reader and writer
csv Enable csv reader and writer
fake Enable fake reader. Useful for generate fake dataset
logger Enable logger writer. Useful for debugging

Roadmap

  • XML reader and writer
  • Filter items
  • Kafka reader and writer
  • Pulsar reader and writer
  • Retry/Skip policies
  • Save execution data in database

Getting Started

Make sure you activated the suitable features crate on Cargo.toml:

[dependencies]
spring-batch-rs = { version = "<version>", features = ["<full|json|csv|fake|logger>"] }

Then, on your main.rs:

fn main() -> Result<(), BatchError> {
    let csv = "year,make,model,description
   1948,Porsche,356,Luxury sports car
   1995,Peugeot,205,City car
   2021,Mazda,CX-30,SUV Compact
   1967,Ford,Mustang fastback 1967,American car";

    let reader = CsvItemReaderBuilder::new()
        .delimiter(b',')
        .has_headers(true)
        .from_reader(csv.as_bytes());

    let processor = UpperCaseProcessor::default();

    let writer = JsonItemWriterBuilder::new().from_path(temp_dir().join("cars.json"));

    let step: Step<Car, Car> = StepBuilder::new()
        .reader(&reader) // set csv reader
        .writer(&writer) // set json writer
        .processor(&processor) // set upper case processor
        .chunk(2) // set commit interval
        .skip_limit(2) // set fault tolerance
        .build();

    let job = JobBuilder::new().start(Box::new(&step)).build();
    let result = job.run();

    assert!(result.is_ok());
    assert!(step.get_status() == StepStatus::Success);

    Ok(())
}

Examples

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions