Data loading

Bringing in data into a system is always a complex task. Throughout your enterprise data topology, you have streams of data that you need to move, collect, and process on the way to their destinations. BDL provides a low-latency ingest infrastructure tool that lets you create continuous data ingest pipelines using a drag and drop UI within an integrated development environment (IDE), powered by the StreamsSets Data Collector.

See Documentation

Universal data loading
Data Collector

The Data Collector is a lightweight, powerful design and execution engine that streams data in real time. You can use it to route and process data in your data streams.

The Data Collector has the following benefits:

  • Design and execute data pipelines that are resilient to data drift, without hand coding
  • Early warning and actionable detection of outliers and anomalies on a stage-by-stage basis throughout the pipeline
  • Rich in-stream data cleansing and preparation capabilities to ensure timely delivery of consumption-ready data
  • Large number of built-in connectors for source and destination systems