Data warehousing vendor Greenplum is claiming to enhance data loading speeds to the tune of as much as four terabytes an hour.

The technology is part of the company's bid to challenge players such as Teradata, Oracle and Netezza. Customers are running into cost and performance constraints with competing solutions, and are looking for scalable software solutions to meet their needs, opined Paul Salazar, vice president of marketing.

  According to Greenplum, this is different from traditional bulk loading technologies used by most mainstream database and MPP appliance vendors that push data from a single source, often over a single or small number of parallel channels. The aforementioned situation can result in bottlenecks and higher load times.

“With our approach we hit fully linear parallelism because we take all the source systems and we essentially do what we call scatter the data," explained Ben Wether, director of product management at Greenplum. "We break it up into chunks that are sprayed across hundreds or thousands of parallel streams into the database and received…by all the nodes of the database in parallel. The essence of it is we eliminate all the bottlenecks.”

Performance scales with the number of Greenplum Database nodes, and the technology supports both large batch and continuous near-real-time loading patterns, company officials said. Data can be transformed and processed in-flight, leveraging all nodes of the database in parallel.

Final gathering and storage of data to disk takes place on all nodes simultaneously, with data automatically partitioned across nodes and optionally compressed, Greenplum officials explained.