MapReduce at ETHZ - ETH Zurich | Flashcards & Summaries

Select your language

Suggested languages for you:
Log In Start studying!

Lernmaterialien für MapReduce an der ETHZ - ETH Zurich

Greife auf kostenlose Karteikarten, Zusammenfassungen, Übungsaufgaben und Altklausuren für deinen MapReduce Kurs an der ETHZ - ETH Zurich zu.

TESTE DEIN WISSEN

What does a reduce function do on a high level?

Lösung anzeigen
TESTE DEIN WISSEN

It takes as input a key with an associated list of values and returns a list of values or key-value pairs. Often the type of the final key and value are the same as of the intermediate data.

Lösung ausblenden
TESTE DEIN WISSEN

Are the input key-value pairs of mappers sorted by key?

Lösung anzeigen
TESTE DEIN WISSEN

Yes

Lösung ausblenden
TESTE DEIN WISSEN

Do MapReduce splits always correspond to HDFS blocks?

Lösung anzeigen
TESTE DEIN WISSEN

No, since splits respects logical record boundaries, they might contain data from multiple HDFS blocks.

Lösung ausblenden
TESTE DEIN WISSEN

Is one single Reducer applied to all values associated with the same key?

Lösung anzeigen
TESTE DEIN WISSEN

Yes, this is the principle behind partitioning: one Reducer is responsible for all values associated with a particular key.

Lösung ausblenden
TESTE DEIN WISSEN

In Hadoop MapReduce do the key-value pairs a Reducer outputs have to be of the same type as its input pairs?

Lösung anzeigen
TESTE DEIN WISSEN

Yes

Lösung ausblenden
TESTE DEIN WISSEN

What is a combiner in MapReduce?

Lösung anzeigen
TESTE DEIN WISSEN

A combiner processes the output data from the Mapper before passing it to the Reducer and is optional. Its there so less shuffling has to be done which can reduce network congestion.

Lösung ausblenden
TESTE DEIN WISSEN

What does a map function do on a high level?

Lösung anzeigen
TESTE DEIN WISSEN

It takes as input a key-value pair and emits a list of key-value pairs.

Lösung ausblenden
TESTE DEIN WISSEN

Does each mapper have to generate the same number of key/value pairs as its input had?

Lösung anzeigen
TESTE DEIN WISSEN

Yes

Lösung ausblenden
TESTE DEIN WISSEN

Is the JobTracker or the TaskTracker responsible for scheduling mappers and reducers and making sure all nodes are correctly running?

Lösung anzeigen
TESTE DEIN WISSEN

JobTracker

Lösung ausblenden
TESTE DEIN WISSEN

What's the core principle of Hadoop?

Lösung anzeigen
TESTE DEIN WISSEN

Bring the query to the data. (same as short-circuiting, datanode and tasktracker/regionserver on the same machine)

Lösung ausblenden
TESTE DEIN WISSEN

Mapper and Reducer are functions that are passed to the map and reduce function respectively. What is the default behaviour of the MapReduce job if the Mapper and Reducer are not changed?

Lösung anzeigen
TESTE DEIN WISSEN

Mapper and Reducers are identity functions by default and there is always a shuffling phase that sorts by input key so the whole job performs sorting by input key.

Lösung ausblenden
TESTE DEIN WISSEN

What is the role of the Context object that is passed to the map and reduce function respectively?

Lösung anzeigen
TESTE DEIN WISSEN

It allows the Mapper/Reducer to interact with the rest of the Hadoop system. The context can be used to set parameters to mappers and reducers running on different nodes.

Lösung ausblenden
  • 96592 Karteikarten
  • 1718 Studierende
  • 87 Lernmaterialien

Beispielhafte Karteikarten für deinen MapReduce Kurs an der ETHZ - ETH Zurich - von Kommilitonen auf StudySmarter erstellt!

Q:

What does a reduce function do on a high level?

A:

It takes as input a key with an associated list of values and returns a list of values or key-value pairs. Often the type of the final key and value are the same as of the intermediate data.

Q:

Are the input key-value pairs of mappers sorted by key?

A:

Yes

Q:

Do MapReduce splits always correspond to HDFS blocks?

A:

No, since splits respects logical record boundaries, they might contain data from multiple HDFS blocks.

Q:

Is one single Reducer applied to all values associated with the same key?

A:

Yes, this is the principle behind partitioning: one Reducer is responsible for all values associated with a particular key.

Q:

In Hadoop MapReduce do the key-value pairs a Reducer outputs have to be of the same type as its input pairs?

A:

Yes

Mehr Karteikarten anzeigen
Q:

What is a combiner in MapReduce?

A:

A combiner processes the output data from the Mapper before passing it to the Reducer and is optional. Its there so less shuffling has to be done which can reduce network congestion.

Q:

What does a map function do on a high level?

A:

It takes as input a key-value pair and emits a list of key-value pairs.

Q:

Does each mapper have to generate the same number of key/value pairs as its input had?

A:

Yes

Q:

Is the JobTracker or the TaskTracker responsible for scheduling mappers and reducers and making sure all nodes are correctly running?

A:

JobTracker

Q:

What's the core principle of Hadoop?

A:

Bring the query to the data. (same as short-circuiting, datanode and tasktracker/regionserver on the same machine)

Q:

Mapper and Reducer are functions that are passed to the map and reduce function respectively. What is the default behaviour of the MapReduce job if the Mapper and Reducer are not changed?

A:

Mapper and Reducers are identity functions by default and there is always a shuffling phase that sorts by input key so the whole job performs sorting by input key.

Q:

What is the role of the Context object that is passed to the map and reduce function respectively?

A:

It allows the Mapper/Reducer to interact with the rest of the Hadoop system. The context can be used to set parameters to mappers and reducers running on different nodes.

MapReduce

Erstelle und finde Lernmaterialien auf StudySmarter.

Greife kostenlos auf tausende geteilte Karteikarten, Zusammenfassungen, Altklausuren und mehr zu.

Jetzt loslegen

Das sind die beliebtesten StudySmarter Kurse für deinen Studiengang MapReduce an der ETHZ - ETH Zurich

Für deinen Studiengang MapReduce an der ETHZ - ETH Zurich gibt es bereits viele Kurse, die von deinen Kommilitonen auf StudySmarter erstellt wurden. Karteikarten, Zusammenfassungen, Altklausuren, Übungsaufgaben und mehr warten auf dich!

Das sind die beliebtesten MapReduce Kurse im gesamten StudySmarter Universum

Mapeh

University of San Agustin

Zum Kurs
MAP

Universidad de La Coruña

Zum Kurs
MAP A2

Bergische Universität Wuppertal

Zum Kurs

Die all-in-one Lernapp für Studierende

Greife auf Millionen geteilter Lernmaterialien der StudySmarter Community zu
Kostenlos anmelden MapReduce
Erstelle Karteikarten und Zusammenfassungen mit den StudySmarter Tools
Kostenlos loslegen MapReduce