Big data app development brought to you by Hortonworks, Cascading

Big data app development brought to you by Hortonworks, Cascading

Apr 23, 2014
Pam Baker

Like in other technologies, there’s a huge need for applications and big data is no exception. Subsequently there is a need to assist developers in getting such to market and in play quickly. To that end, Hortonworks has added Concurrent’s Cascading SDK to its Hadoop distribution. Such helps developers operationalize their data. In addition, Hortonworks will certify, support and deliver Cascading–the most widely used App development framework for data applications on Hadoop.

“As more enterprises realize they are in the business of data, the need for simple, powerful tools for big data application development is a must-have to survive in today’s competitive climate,” said Gary Nakamura, CEO, Concurrent, in a statement to the press. “Our deepened relationship with Hortonworks furthers our commitment to Hadoop and drives new innovation around the development of enterprise data applications.”

Upcoming releases of Cascading will also support Apache Tez, a general data-processing fabric and MapReduce replacement that provides a powerful framework for executing a complex topology of tasks. According to the press release:

“Tez executes on top of Apache Hadoop YARN, a sub-project of Hadoop which separates resource management and processing components. YARN fundamentally enables a broader array of interaction patterns for data stored in HDFS beyond MapReduce and makes Hadoop 2.0 a more general data processing platform.

In addition, thousands of companies that already use Cascading, Lingual, Scalding or Cascalog or any other dynamic programming language APIs and frameworks built on top of Cascading, have the flexibility to seamlessly migrate to newer versions of HDP that support Apache Tez, with zero investment required to take advantage of this improved processing environment.”

It will be interesting to watch how quickly developers take to this and how many new apps show up as a result. My bet is it will be plenty.