YARN

Yet Another Resource Negotiator

YARN Architecture

Launching an application

yarn jar application.jar DriverClass input output
spark-submit --master yarn application.py input output

List running jobs

yarn application -list

See application logs

yarn logs -applicationId 

Kill an application

yarn application -kill 

Capacity Scheduler Queues

WebUI

YARN UI