Skip to content
README.md 401 B
Newer Older
Marcelo Pasin's avatar
Marcelo Pasin committed
# hadoop-docker


Marcelo Pasin's avatar
Marcelo Pasin committed
docker compose up -d
Marcelo Pasin's avatar
Marcelo Pasin committed

Marcelo Pasin's avatar
Marcelo Pasin committed
docker exec -it --user hadoop namenode bash
Marcelo Pasin's avatar
Marcelo Pasin committed

Marcelo Pasin's avatar
Marcelo Pasin committed
hdfs dfs -mkdir /user
hdfs dfs -mkdir /user/hadoop
hdfs dfs -mkdir input
hdfs dfs -put *.txt input
Marcelo Pasin's avatar
Marcelo Pasin committed

Marcelo Pasin's avatar
Marcelo Pasin committed
STRJAR=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-streaming-3.4.1.jar
Marcelo Pasin's avatar
Marcelo Pasin committed

Marcelo Pasin's avatar
Marcelo Pasin committed
hadoop jar $STRJAR -mapper mapper.py -reducer reducer.py -input '/user/hadoop/input/*.txt' -output '/user/hadoop/output'
Marcelo Pasin's avatar
Marcelo Pasin committed

Marcelo Pasin's avatar
Marcelo Pasin committed
hdfs dfs -cat output/*