Apache Livy - A REST Interface for Apache Spark
Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN. Livy provides the following features: Interactive Scala, Python, and R shells Batch submissions in Scala, Java, Python Multiple users can share the same server (impersonation support) Can be used for submitting jobs from anywhere with REST Does not require any code change to your programs Support Spark1/ Spark2, Scala 2.10/2.11 within one build. Architecture Core Functionalities Livy offers three modes to run Spark jobs: Using programmatic API Running interactive statements through REST API Submitting batch applications with REST API In the following sections, I will provide the details of these 3 modes. Details: https://hortonworks.com/blog/livy-a-rest-interface-for-apache-spark/ Guide: https://dzone.com/articles/quick-star