Presto on Amazon EMR

Presto is a distributed query engine for running interactive datasets on a wide variety of sources. I found it blazingly fast on most of the datasets I use - typically in parquet format, after initial processing.

If you are using Amazon athena, you are already using presto as the engine behind the scenes. However, it is possible you want to install presto on EMR for a few reasons such as

Control the version of presto you use and upgrade/pin the version when you choose.
you might want to use it for running larger queries than the current limits imposed by Athena (30 minute query limit as of this writing).
run it on hadoop files along with files on S3.

When creating the cluster, select Presto option and the option to choose the Hive metastore (if applicable)

Presto on EMR

Once the cluster is created, you can ssh into the master node and access the presto CLI. Schema here maps to the “database” in athena. Catalog maps to the one-per-account catalog in Athena/Glue.

[hadoop@ip-xxx-xx-xx-xxx ~]$ presto-cli --catalog hive --schema sales_parquet
WARNING: History file is not readable/writable: /home/hadoop/.presto_history. History will not be available during this session.
presto:sales_parquet> show tables;
           Table
----------------------------
 fact_order_item
 fact_order
 fact_page_view
 usage_snapshot
(4 rows)

Query 20181228_210708_00013_rapbe, FINISHED, 5 nodes
Splits: 70 total, 70 done (100.00%)
0:01 [4 rows, 177B] [7 rows/s, 316B/s]

presto:sales_parquet> select count(*) from usage_snapshot;
  _col0
----------
 84919353
(1 row)

Query 20181228_210718_00014_rapbe, FINISHED, 4 nodes
Splits: 47 total, 47 done (100.00%)
0:04 [84.9M rows, 0B] [22.1M rows/s, 0B/s]