Introduction
The Java Stream mapMulti method was added in Java 16. It allows us to replace each element in a Java Stream with one or more elements. I’ll cover the Java Stream mapMulti method in detail here.
A quick recap of Java Stream’s flatMap
The flatMap is an intermediate operation. When applied on a stream, it returns a new stream after applying the passed mapping function to each element of the stream. The mapping function takes an element and returns a stream (for each element). The flatMap operation thus replaces each element with the contents of the stream returned by the mapping function. Its method signature is:
<R> Stream<R> flatMap(Function<? super T,? extends Stream<? extends R>> mapper)
The mapping function takes elements of type T and returns a stream having elements of type R (Stream<R> ignoring the wildcards).
Stream flatMap example
Let us say we have a Java record for a Department and each department has a list of courses. We have two departments with courses for each.
record Department(String name, List<String> courses) {
}
List<Department> departments = List.of(
new Department("Maths",
List.of("Advanced Maths", "Discrete Maths")),
new Department("Computer Science",
List.of("Algorithms", "Data Structures", "Computer Architecture"))
);We can use the flatMap to collect the courses as a list, as shown below.
List<String> allCourses = departments.stream()
.flatMap(department -> department.courses().stream())
.collect(Collectors.toList());
System.out.println(allCourses); //[Advanced Maths, Discrete Maths, Algorithms, Data Structures, Computer Architecture]The function passed to flatMap replaced each department (String) with a stream having a list of courses (Stream<String>) for that department. The flatMap method took care of flatMapping the elements in all the mapped streams (courses) as one Stream<String>. Finally, the Collectors.toList() collected the elements in the resultant steam into a list.
Java Stream mapMulti method
I’ll start with the mapMulti method signature,
default <R> Stream<R> mapMulti(BiConsumer<? super T,? super Consumer<R>> mapper)
The mapMulti is an intermediate operation, and it takes a BiConsumer mapper function and returns a stream (this BiConsumer is called as the mapping function – not to be confused with a Function). The new stream has each element of the original stream replaced with one or more elements. Here, the passed mapper function does the replacement by calling the consumer argument (of the BiConsumer) with the replacement elements.
More specifically, when the stream pipeline invokes the mapper function for an element, it also gets a Consumer instance (the two arguments of the mapper BiConsumer are the element and the consumer instance). The mapper function computes the replacement elements for the input element (maybe zero, one or more). It then passes those mapped elements or replacement elements to the Consumer instance.
Our first MapMulti example
We have a list of strings and we create a stream pipeline out of it. We then use mapMulti to map each element of the stream into multiple elements (in the example, it is two).
List<String> input = List.of("a", "b", "c");
List<String> result = input.stream()
.<String>mapMulti((element, consumer) -> {
consumer.accept(element + "-1");
consumer.accept(element + "-2");
})
.collect(Collectors.toList());
System.out.println(result);We pass a BiConsumer of String and a Consumer<String> to the mapMulti (as a lambda expression) and we replace each element with two new elements. Those elements are passed to the Consumer argument of the BiFunction. We need not worry about how that Consumer argument is passed as it is taken care by the stream pipeline and mapMulti implementation. The important point to understand here is that those replacement elements then become part of the resultant stream, replacing the original element. For example, the element “a” is replaced by “a-1” and “a-2”. At last, we collect the elements of the stream into a list and print it.
The result of running the above is,
[a-1, a-2, b-1, b-2, c-1, c-2]
Since each of the three elements is replaced with two elements each, we have six strings.
Java Stream mapMulti vs flatMap
We use both flatMap and mapMulti to do a one-to-many transformation of the stream elements. We can achieve the same result as the earlier code (which used mapMulti) using a flatMap as shown below.
input.stream()
.flatMap(element -> Stream.of(element + "-1", element + "-2"))
.collect(Collectors.toList());Then when should we use a flatMap and when to use a mapMulti and what is the difference between a flatMap and a mapMulti?
The difference lies in how they both perform the one-to-many transformation.
- When using a flatMap, we create and return a new stream for each (flat) mapping. This could be costly (overhead) in some cases, especially when replacing with a small number of elements. To skip or remove an element, we have to return an empty stream. In cases like this, using mapMulti could lead to performance improvement (We will see the performance numbers towards the end of this post).
- As you would have noticed, using flatMap is declarative, whereas when using mapMulti it results to be an imperative code. Sometimes, using such imperative code could be easier and lead to better readability (arguably, in the above example, it is the case). In such cases, using a mapMulti would be preferable.
More mapMulti examples
We’ve already seen an example of mapMulti replacing an element with more than one. It is also possible to replace an element with nothing.
Using the similar example, here, we map an element into two elements only if the length of the element it even. Hence, strings whose length are odd will be dropped from the stream.
List<String> input = List.of("apple", "pear", "peach", "banana");
List<String> result = input.stream()
.<String>mapMulti((element, consumer) -> {
if (element.length() % 2 == 0) {
consumer.accept(element + "-1");
consumer.accept(element + "-2");
}
})
.collect(Collectors.toList());
System.out.println(result);It prints,
[pear-1, pear-2, banana-1, banana-2]
Let us look at another interesting example shown in the Javadoc of mapMulti. Shown below, we have a nested list of Iterables.
List<Object> nestedList = List.of(1, List.of(2, List.of(3, 4)), 5);
We would like to flatten it and get the integer elements alone. We can recursively expand them using a helper function and using mapMulti send the integers into the consumer whenever we have an integer. This will return a stream having integers alone.
private void expandIterable(Object e, Consumer<Object> c) {
if (e instanceof Iterable<?> elements) {
for (Object ie : elements) {
expandIterable(ie, c);
}
} else if (e != null) {
c.accept(e);
}
}
Stream<Object> expandedStream = nestedList.stream()
.mapMulti(this::expandIterable);
.forEach(System.out::println);
The method reference used in mapMulti can be written as a lambda expression as .mapMulti((object, consumer) -> expandIterable(object, consumer));
The first argument could be an integer or a nested Iterable (a List here) and the second argument is the consumer passed by the stream implementation. The expandIterable function checks the type of the object and if it is an Iterable, it calls itself recursively for each element contained within the Iterable. Otherwise (if it is an Integer), it passes it to the consumer. Hence, as a result, we flatten each of the nested lists and return a stream of objects with integers alone.
Running this, we get,
1 2 3 4 5
Do not use an infinite stream inside mapMulti mapper
From the looks of it, it might appear that we can always replace a piece of code using a flatMap with a mapMulti. But that is not true when the flatMap returns an infinite stream (and the stream pipeline has a short-circuiting operation like a limit()).
Shown below, we have a list of three strings. We create a stream and use flatMap, which uses an infinite stream to generate the suffix for the elements (surely sounds silly, but bear with it as an example). Next, we limit the number of elements in the stream to five using limit() and print the stream elements.
List<String> list = List.of("a", "b", "c");
list.stream()
.flatMap(element -> Stream.iterate(0, i -> i + 1)
.map(suffix -> element + "-" + suffix))
.limit(5)
.forEach(System.out::println);When we run this, it prints,
a-0 a-1 a-2 a-3 a-4
NOTE: Beware Java 8 Streams are not completely Lazy and hence running above code using JDK 8 will result in an infinite loop.
Let us replace flatMap with mapMulti.
list.stream()
.mapMulti((element, consumer) -> {
Stream.iterate(0, i -> i + 1)
.map(suffix -> element + "-" + suffix)
.forEach(consumer);
})
.limit(5)
.forEach(System.out::println);
System.out.println("Done"); //Never gets hereWe get the same output as before, but there would be an infinite loop. The last print statement will never be executed.
a-0 a-1 a-2 a-3 a-4
When a flatMap returns an infinite stream, if the stream pipeline has short-circuiting operations, the stream pipeline will terminate. For mapMulti, it is not the case. This is because, by definition and contract of mapMulti, the BiConsumer mapper function should convert an element into one or more elements and by that requirement, the mapping cannot use an infinite stream, which implies that it will never terminate.
Type inference issue with mapMulti
If you had noticed in the first mapMulti example, I had used a type witness for the Consumer. Here’s another simple example.
List<String> list = List.of("a", "b", "c", "d");
List<String> result = list.stream()
.mapMulti((element, consumer) -> consumer.accept(element))
.collect(Collectors.toList());
System.out.println(result);Running this code, we get the below error.
java: incompatible types: inference variable T has incompatible bounds
equality constraints: java.lang.String
lower bounds: java.lang.ObjectThe mapMulti mapper converts an element of type T to one or more elements of type R and hence we need a Consumer<R>. But the compiler cannot determine what type of Consumer to pass and hence it passes a Consumer<Object>. To workaround the problem, we provide a hint (type witness) for the Consumer’s type parameter.
List<String> result = list.stream()
.<String>mapMulti((element, consumer) -> consumer.accept(element))
.collect(Collectors.toList());
System.out.println(result); //[a, b, c, d]Performance of mapMulti
Time to look into how mapMulti performs when comparing with flatMap. As usual, I’ll be using Java JMH for benchmarks.
We have two functions doing the same thing (functionally), but the first one uses flatMap, and the second uses the mapMulti.
It starts with an infinite stream with a limit of 100K elements (integers). For each of the integers, we suffix (append) characters from ‘a’ to ‘z’.
First, we use flatMap to flatten an integer to 26 strings (for example, for integer 0, it will be 0-a, 0-b, 0-c,… 0-z). Then the second benchmark test does the same using the mapMulti.
package com.javadevcentral.mapMulti;
//imports not shown
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
public class FlatMap_MapMulti_Benchmark {
@Benchmark
@Fork(value = 2)
@Measurement(iterations = 10, time = 1)
@Warmup(iterations = 5, time = 1)
public List<String> flatMap() {
return Stream.iterate(1, i -> i + 1)
.limit(100_000)
.flatMap(num -> IntStream.rangeClosed('a', 'z')
.mapToObj(i -> num + "-" + (char) i))
.collect(Collectors.toList());
}
@Benchmark
@Fork(value = 2)
@Measurement(iterations = 10, time = 1)
@Warmup(iterations = 5, time = 1)
public List<String> mapMulti() {
return Stream.iterate(1, i -> i + 1)
.limit(100_000)
.<String>mapMulti((num, consumer) -> {
for (char c = 'a'; c <= 'z'; c++) {
consumer.accept(num + "-" + c);
}
})
.collect(Collectors.toList());
}
}Here are the benchmark results,
Benchmark Mode Samples Score Score error Units c.j.mapMulti.FlatMap_MapMulti_Benchmark.flatMap avgt 20 318.776 94.230 ms/op c.j.mapMulti.FlatMap_MapMulti_Benchmark.mapMulti avgt 20 240.144 88.563 ms/op
The code using mapMulti takes 240 milliseconds per operation (ms/op) whereas the flatMap code takes 318 ms/op.
Let us re-check the performance number by using @OperationsPerInvocation annotation as each method operates on 100K numbers. It will help us test the performance of flatMap vs mapMulti per element.
To do that, we just replace,
@OutputTimeUnit(TimeUnit.MILLISECONDS)
with
@OutputTimeUnit(TimeUnit.NANOSECONDS) @OperationsPerInvocation(100_000)
As before, the results show that mapMulti is faster than flatMap.
Benchmark Mode Samples Score Score error Units c.j.mapMulti.FlatMap_MapMulti_Benchmark.flatMap avgt 20 2707.993 434.600 ns/op c.j.mapMulti.FlatMap_MapMulti_Benchmark.mapMulti avgt 20 1934.614 162.427 ns/op
Filtering out the empty optional from a list
There is an interesting use-case of mapMulti with Optionals. We have a List<Optional<Integer>>. And we want to extract only the non-optional elements as a List<Integer>. We could use flatMap or a mapMulti for this. Let us compare the performance of each.
package com.javadevcentral.mapMulti;
//imports not shown
@BenchmarkMode(Mode.AverageTime)
public class FlatMap_MapMulti_OptionalBenchmark {
private static List<Optional<Integer>>optionals;
static {
optionals = IntStream.range(0, 100_000)
.mapToObj(Optional::of)
.collect(Collectors.toCollection(ArrayList::new));
optionals.addAll(IntStream.range(0, 50_000)
.mapToObj(i -> Optional.<Integer>empty())
.toList());
Collections.shuffle(optionals);
}
@Benchmark
@Fork(value = 2)
@Measurement(iterations = 10, time = 1)
@Warmup(iterations = 5, time = 1)
public List<Integer> filter() {
return optionals.stream()
.filter(Optional::isPresent)
.map(Optional::get)
.collect(Collectors.toList());
}
@Benchmark
@Fork(value = 2)
@Measurement(iterations = 10, time = 1)
@Warmup(iterations = 5, time = 1)
public List<Integer> flatMap() {
return optionals.stream()
.flatMap(Optional::stream)
.collect(Collectors.toList());
}
@Benchmark
@Fork(value = 2)
@Measurement(iterations = 10, time = 1)
@Warmup(iterations = 5, time = 1)
public List<Integer> mapMulti() {
return optionals.stream()
//.<Integer>mapMulti((o, consumer) -> o.ifPresent(consumer))
.<Integer>mapMulti(Optional::ifPresent)
.collect(Collectors.toList());
}
}We build the list of optionals with 100K integers (0, 1, 2, 3.. 99,999) and add 50K empty optionals and shuffle the list.
Here we have three methods, each doing the same thing (to get the non-empty elements).
- The first method uses a simple filter and map on the stream to extract the integer elements from the list of optionals.
- The second one uses flatMap by calling the stream() method on the Optional. The Optional#stream was added in Java 9. (Refer to Optional – New methods in Java 9 through 11 post to learn the new methods added to the Optional class)
- Finally comes the interesting part. When using mapMulti, the method reference used in the above code might be confusing. Looking at the equivalent lambda expression (commented code) will make it clear. In the mapping function, we get the optional element and a Consumer<Integer> and we call ifPresent on the optional and pass the consumer. Internally, if the optional is not empty, the underlying integer value will be passed to the consumer. Hence, I have written this lambda expression using Type::method version of method reference. (I have a post on Java method references explaining all the types of method references in Java).
Here are the benchmark results:
Benchmark Mode Samples Score Score error Units c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.filter avgt 20 1.797 0.108 ms/op c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.flatMap avgt 20 2.903 0.315 ms/op c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.mapMulti avgt 20 1.855 0.420 ms/op
The performance of mapMulti and filter are very similar, whereas using flatMap is costlier here.
Since we have 150K elements, to use OperationsPerInvocation, we replace the @OutputTimeUnit(TimeUnit.MILLISECONDS) with the following.
@OutputTimeUnit(TimeUnit.NANOSECONDS) @OperationsPerInvocation(150_000)
The results are:
Benchmark Mode Samples Score Score error Units c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.filter avgt 20 12.344 1.942 ns/op c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.flatMap avgt 20 19.902 1.582 ns/op c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.mapMulti avgt 20 10.375 0.828 ns/op
Summary of performance benchmark
With this, can we say that using mapMulti is always better than flatMap?
It depends..
For the examples scenarios or use-cases I’ve taken, it turned out that mapMulti was performing better than a flatMap. You have to do similar benchmarking for your use-case and workload to determine which would be better.
Replacing with primitive types
The mapMulti accepted a BiConsumer of types T and Consumer<R>. They have also added mapMulti methods for mapping elements to primitive types (int, long, and double) viz., mapMultiToInt, mapMultiToLong and mapMultiToDouble.
mapMultiToInt
The mapMultiToInt accepts a BiConsumer of types T (as before) and IntConsumer. An IntConsumer is a functional interface which accepts a primitive int value. Hence when we map a stream element to a primitive int, it returns an IntStream and there is no boxing involved.
IntStream mapMultiToInt(BiConsumer<? super T,? super IntConsumer> mapper)
Let us look at an example. We have a list of fruit names and we use mapMultiToInt to map each string to its length. This returns an IntStream and we all sum() on it to get the sum of all fruit name’s length.
List<String> strings = List.of("apple", "pear", "grapes", "banana");
int sumOfLengths = strings.stream()
.mapMultiToInt((fruit, intConsumer) -> intConsumer.accept(fruit.length()))
.sum();
System.out.println(sumOfLengths); //21As another example, we have a Student and Exam model, as shown below with some data. Each student has taken up a list of exams and the exam object has the marksScored data.
record Exam(String name, int marksScored) {
}
record Student(String name, List<Exam> exams) {
}
List<Student> studentDetails = List.of(
new Student("S1",
List.of(new Exam("E1", 98),
new Exam("E2", 78))),
new Student("S2",
List.of(new Exam("E1", 61),
new Exam("E2", 72)))
);Let’s say we want to find the average mark scored among all students and all exams. We stream the students list, and for each student, we go through each of the exams taken up. For each exam, we get the marksScored and pass it to the mapMultiToInt’s IntConsumer. Finally, we call the average() on the IntStream to get the average.
double average = studentDetails.stream()
.mapMultiToInt((student, intConsumer) -> student.exams()
.forEach(exam -> intConsumer.accept(exam.marksScored())))
.average()
.orElse(0);
System.out.println(average); //77.25mapMultiToLong and mapMultiToDouble
The mapMultiToLong and mapMultiToDouble work in a similar way to mapMultiToInt, but for long and double, respectively. Their method signatures are shown below.
LongStream mapMultiToLong(BiConsumer<? super T,? super LongConsumer> mapper) DoubleStream mapMultiToDouble(BiConsumer<? super T,? super DoubleConsumer> mapper)
The second argument of the BiConsumer of mapMultiToLong is a LongConsumer (a primitive version of the Consumer interface for a long). Similarly, the mapMultiToDouble’s BiConsumer takes a DoubleConsumer. The mapMultiToLong and mapMultiToDoublee return a LongStream and DoubleStream, respectively.
As a simple example, let us map a Long or a Double to three values, as shown below.
List<Long> longValues = List.of(10L, 20L);
long sumOfLongs = longValues.stream()
.mapMultiToLong((longVal, longConsumer) -> {
long primitiveLongVal = longVal; // unbox once
longConsumer.accept(primitiveLongVal - 1);
longConsumer.accept(primitiveLongVal);
longConsumer.accept(loprimitiveLongValngVal + 1);
})
.sum();
System.out.println(sumOfLongs);//90
List<Double> doubleValues = List.of(10.4, 20.1);
double sumOfDoubles = doubleValues.stream()
.mapMultiToDouble((doubleVal, doubleConsumer) -> {
double primitiveDoubleVal = doubleVal; //unbox once
doubleConsumer.accept(primitiveDoubleVal - 1);
doubleConsumer.accept(primitiveDoubleVal);
doubleConsumer.accept(primitiveDoubleVal + 1);
})
.sum();
System.out.println(sumOfDoubles);//91.5The mapMulti in primitive specialization of streams
In the final section of the Java Streams mapMulti post, let us look at the mapMulti methods added to the primitive specialization of streams (IntStream, LongStream, and DoubleStream).
In the IntStream, we have the mapMulti method, as shown below.
default IntStream mapMulti(IntStream.IntMapMultiConsumer mapper)
where the IntMapMultiConsumer is a functional interface. It takes a primitive int value and an IntConsumer.
@FunctionalInterface
interface IntMapMultiConsumer {
void accept(int value, IntConsumer ic);
}
@FunctionalInterface
public interface IntConsumer {
void accept(int value);
}To summarize, the mapMulti method on IntStream takes a primitive int and an IntConsumer and returns an IntStream back.
The mapMulti methods on LongStream and DoubleStream have similar methods.
LongStream mapMulti(LongStream.LongMapMultiConsumer mapper)
DoubleStream mapMulti(DoubleStream.DoubleMapMultiConsumer mapper)
interface LongMapMultiConsumer {
void accept(long value, LongConsumer lc);
}
interface DoubleMapMultiConsumer {
void accept(double value, DoubleConsumer dc);
}Examples are shown below.
IntStream intStream = IntStream.of(10, 20);
int sumOfInts = intStream
.mapMulti((intVal, intConsumer) -> {
intConsumer.accept(intVal - 1);
intConsumer.accept(intVal);
intConsumer.accept(intVal + 1);
})
.sum();
System.out.println(sumOfInts); //90
LongStream longStream = LongStream.of(10, 20);
long sumOfLongs = longStream
.mapMulti((longVal, longConsumer) -> {
longConsumer.accept(longVal - 1);
longConsumer.accept(longVal);
longConsumer.accept(longVal + 1);
})
.sum();
System.out.println(sumOfLongs); //90
DoubleStream doubleStream = DoubleStream.of(10.4, 20.1);
double sumOfDoubles = doubleStream
.mapMulti((doubleVal, doubleConsumer) -> {
doubleConsumer.accept(doubleVal - 1);
doubleConsumer.accept(doubleVal);
doubleConsumer.accept(doubleVal + 1);
})
.sum();
System.out.println(sumOfDoubles); //91.5Conclusion
In this detailed post on Java Stream mapMulti, we saw how a mapMulti method differs from a flatMap. We saw examples of mapMulti methods and saw the performance of it measured using JMH.
References
When and how to perform one to 0..n mapping Stream mapMulti over flatMap
