Use Sparkling Water with Amazon EMR H2O Sparkling Water 3.42.0.3-1-3.1 documentation

Publish date: 2024-05-04

This section describes how to use Sparkling Water with Amazon EMR via the Web Services UI. An example describing how to do this using the AWS CLI is available in the Analytics at Scale: H2O, Apache Spark and R on AWS EMR blog post (courtesy of Red Oak Strategic).

To use Sparkling Water with an EMR cluster, you can use a premade H2O template.

  • Log in to the AWS Marketplace.

  • In the AMI & SaaS search bar, search for H2O, and select H2O Artificial Intelligence to open H2O in the marketplace. Review the information on this page. Note that the Delivery Methods section provides two options:

    Click Continue after reviewing to open the Launch on the EC2 page.

  • H2O Artificial Intelligence
  • The Launch on the EC2 page provides information about launch options. On the Manual Launch tab:

    Click Launch with CloudFormation Console to begin creating your stack.

  • Launch On EC2 page
  • On the Select Template page, enter https://s3-us-west-2.amazonaws.com/h2o-cloud/aws-template/TemplateEMR.json in the Specify an Amazon S3 template URL field.

  • Note that the Specify Details page now includes an EMR Options section. Enter a name for the stack, update any options as desired, and then click Next to continue.

  • Specify details - EMR
  • Enter any optional tags and/or permissions on the Options page. Click Next to continue.

  • Options page
  • Review the Stack configuration. Click Create to create the Stack or click Previous to return to another page and edit any information.

  • After your EMR cluster is created, you can ssh into your head node. In the head node, you will find an H2O folder with Sparkling Water inside. To start any H2O jobs with Sparkling Water, follow the instructions located on the download page (https://www.h2o.ai/download/).

    Note: It is important to add the following Spark configuration to any of your EMR jobs:

    --conf spark.dynamicAllocation.enabled=false --conf spark.scheduler.minRegisteredResourcesRatio=1 

    and for EMR 5.x (5.32.0 and newer) also:

    --conf spark.yarn.heterogeneousExecutors.enabled=false 

    ncG1vNJzZmirY2Ourq3ZqKWar6NjsLC5jqFpqGWimrmmrdKeZqyokae4rbXNoGSwmaSav3C%2Fz5qppGVjY35wf41taWdoXmh6cnmSZ2honJ%2BYfKWxz6WmsqWVo8Fwrdaslp6lomO1tbnL