Introduction

The Java Stream mapMulti method was added in Java 16. It allows us to replace each element in a Java Stream with one or more elements. I’ll cover the Java Stream mapMulti method in detail here.

A quick recap of Java Stream’s flatMap

The flatMap is an intermediate operation. When applied on a stream, it returns a new stream after applying the passed mapping function to each element of the stream. The mapping function takes an element and returns a stream (for each element). The flatMap operation thus replaces each element with the contents of the stream returned by the mapping function. Its method signature is:

<R> Stream<R> flatMap(Function<? super T,? extends Stream<? extends R>> mapper)

The mapping function takes elements of type T and returns a stream having elements of type R (Stream<R> ignoring the wildcards).

Stream flatMap example

Let us say we have a Java record for a Department and each department has a list of courses. We have two departments with courses for each.

record Department(String name, List<String> courses) {
}

List<Department> departments = List.of(
        new Department("Maths",
                List.of("Advanced Maths", "Discrete Maths")),
        new Department("Computer Science",
                List.of("Algorithms", "Data Structures", "Computer Architecture"))
);

We can use the flatMap to collect the courses as a list, as shown below.

List<String> allCourses = departments.stream()
        .flatMap(department -> department.courses().stream())
        .collect(Collectors.toList());
System.out.println(allCourses); //[Advanced Maths, Discrete Maths, Algorithms, Data Structures, Computer Architecture]

The function passed to flatMap replaced each department (String) with a stream having a list of courses (Stream<String>) for that department. The flatMap method took care of flatMapping the elements in all the mapped streams (courses) as one Stream<String>. Finally, the Collectors.toList() collected the elements in the resultant steam into a list.

Java Stream mapMulti method

I’ll start with the mapMulti method signature,

default <R> Stream<R> mapMulti(BiConsumer<? super T,? super Consumer<R>> mapper)

The mapMulti is an intermediate operation, and it takes a BiConsumer mapper function and returns a stream (this BiConsumer is called as the mapping function - not to be confused with a Function). The new stream has each element of the original stream replaced with one or more elements. Here, the passed mapper function does the replacement by calling the consumer argument (of the BiConsumer) with the replacement elements.

More specifically, when the stream pipeline invokes the mapper function for an element, it also gets a Consumer instance (the two arguments of the mapper BiConsumer are the element and the consumer instance). The mapper function computes the replacement elements for the input element (maybe zero, one or more). It then passes those mapped elements or replacement elements to the Consumer instance.

Our first MapMulti example

We have a list of strings and we create a stream pipeline out of it. We then use mapMulti to map each element of the stream into multiple elements (in the example, it is two).

List<String> input = List.of("a", "b", "c");
List<String> result = input.stream()
        .<String>mapMulti((element, consumer) -> {
            consumer.accept(element + "-1");
            consumer.accept(element + "-2");
        })
        .collect(Collectors.toList());
System.out.println(result);

We pass a BiConsumer of String and a Consumer<String> to the mapMulti (as a lambda expression) and we replace each element with two new elements. Those elements are passed to the Consumer argument of the BiFunction. We need not worry about how that Consumer argument is passed as it is taken care by the stream pipeline and mapMulti implementation. The important point to understand here is that those replacement elements then become part of the resultant stream, replacing the original element. For example, the element “a” is replaced by “a-1” and “a-2”. At last, we collect the elements of the stream into a list and print it.

The result of running the above is,

[a-1, a-2, b-1, b-2, c-1, c-2]

Since each of the three elements is replaced with two elements each, we have six strings.

Java Stream mapMulti vs flatMap

We use both flatMap and mapMulti to do a one-to-many transformation of the stream elements. We can achieve the same result as the earlier code (which used mapMulti) using a flatMap as shown below.

input.stream()
        .flatMap(element -> Stream.of(element + "-1", element + "-2"))
        .collect(Collectors.toList());

Then when should we use a flatMap and when to use a mapMulti and what is the difference between a flatMap and a mapMulti?

The difference lies in how they both perform the one-to-many transformation.

  1. When using a flatMap, we create and return a new stream for each (flat) mapping. This could be costly (overhead) in some cases, especially when replacing with a small number of elements. To skip or remove an element, we have to return an empty stream. In cases like this, using mapMulti could lead to performance improvement (We will see the performance numbers towards the end of this post).
  2. As you would have noticed, using flatMap is declarative, whereas when using mapMulti it results to be an imperative code. Sometimes, using such imperative code could be easier and lead to better readability (arguably, in the above example, it is the case). In such cases, using a mapMulti would be preferable.

More mapMulti examples

We’ve already seen an example of mapMulti replacing an element with more than one. It is also possible to replace an element with nothing.

Using the similar example, here, we map an element into two elements only if the length of the element it even. Hence, strings whose length are odd will be dropped from the stream.

List<String> input = List.of("apple", "pear", "peach", "banana");
List<String> result = input.stream()
        .<String>mapMulti((element, consumer) -> {
            if (element.length() % 2 == 0) {
                consumer.accept(element + "-1");
                consumer.accept(element + "-2");
            }
        })
        .collect(Collectors.toList());
System.out.println(result);

It prints,

[pear-1, pear-2, banana-1, banana-2]

Let us look at another interesting example shown in the Javadoc of mapMulti. Shown below, we have a nested list of Iterables.

List<Object> nestedList = List.of(1, List.of(2, List.of(3, 4)), 5);

We would like to flatten it and get the integer elements alone. We can recursively expand them using a helper function and using mapMulti send the integers into the consumer whenever we have an integer. This will return a stream having integers alone.

private void expandIterable(Object e, Consumer<Object> c) {
    if (e instanceof Iterable<?> elements) {
        for (Object ie : elements) {
expandIterable(ie, c);
        }
    } else if (e != null) {
        c.accept(e);
    }
}

Stream<Object> expandedStream = nestedList.stream()
        .mapMulti(this::expandIterable);
        .forEach(System.out::println);

The method reference used in mapMulti can be written as a lambda expression as .mapMulti((object, consumer) -> expandIterable(object, consumer));

The first argument could be an integer or a nested Iterable (a List here) and the second argument is the consumer passed by the stream implementation. The expandIterable function checks the type of the object and if it is an Iterable, it calls itself recursively for each element contained within the Iterable. Otherwise (if it is an Integer), it passes it to the consumer. Hence, as a result, we flatten each of the nested lists and return a stream of objects with integers alone.

Running this, we get,

1
2
3
4
5

Do not use an infinite stream inside mapMulti mapper

From the looks of it, it might appear that we can always replace a piece of code using a flatMap with a mapMulti. But that is not true when the flatMap returns an infinite stream (and the stream pipeline has a short-circuiting operation like a limit()).

Shown below, we have a list of three strings. We create a stream and use flatMap, which uses an infinite stream to generate the suffix for the elements (surely sounds silly, but bear with it as an example). Next, we limit the number of elements in the stream to five using limit() and print the stream elements.

List<String> list = List.of("a", "b", "c");

list.stream()
        .flatMap(element -> Stream.iterate(0, i -> i + 1)
                .map(suffix -> element + "-" + suffix))
        .limit(5)
        .forEach(System.out::println);

When we run this, it prints,

a-0
a-1
a-2
a-3
a-4

NOTE: Beware Java 8 Streams are not completely Lazy and hence running above code using JDK 8 will result in an infinite loop.

Let us replace flatMap with mapMulti.

list.stream()
        .mapMulti((element, consumer) -> {
            Stream.iterate(0, i -> i + 1)
                    .map(suffix -> element + "-" + suffix)
                    .forEach(consumer);
        })
        .limit(5)
        .forEach(System.out::println);
System.out.println("Done"); //Never gets here

We get the same output as before, but there would be an infinite loop. The last print statement will never be executed.

a-0
a-1
a-2
a-3
a-4

When a flatMap returns an infinite stream, if the stream pipeline has short-circuiting operations, the stream pipeline will terminate. For mapMulti, it is not the case. This is because, by definition and contract of mapMulti, the BiConsumer mapper function should convert an element into one or more elements and by that requirement, the mapping cannot use an infinite stream, which implies that it will never terminate.

Type inference issue with mapMulti

If you had noticed in the first mapMulti example, I had used a type witness for the Consumer. Here’s another simple example.

List<String> list = List.of("a", "b", "c", "d");
List<String> result = list.stream()
        .mapMulti((element, consumer) -> consumer.accept(element))
        .collect(Collectors.toList());
System.out.println(result);

Running this code, we get the below error.

java: incompatible types: inference variable T has incompatible bounds
    equality constraints: java.lang.String
    lower bounds: java.lang.Object

The mapMulti mapper converts an element of type T to one or more elements of type R and hence we need a Consumer<R>. But the compiler cannot determine what type of Consumer to pass and hence it passes a Consumer<Object>. To workaround the problem, we provide a hint (type witness) for the Consumer’s type parameter.

List<String> result = list.stream()
        .<String>mapMulti((element, consumer) -> consumer.accept(element))
        .collect(Collectors.toList());
System.out.println(result); //[a, b, c, d]

Performance of mapMulti

Time to look into how mapMulti performs when comparing with flatMap. As usual, I’ll be using Java JMH for benchmarks.

We have two functions doing the same thing (functionally), but the first one uses flatMap, and the second uses the mapMulti.

It starts with an infinite stream with a limit of 100K elements (integers). For each of the integers, we suffix (append) characters from ‘a’ to ‘z’.

First, we use flatMap to flatten an integer to 26 strings (for example, for integer 0, it will be 0-a, 0-b, 0-c,… 0-z). Then the second benchmark test does the same using the mapMulti.

package com.javadevcentral.mapMulti;

//imports not shown

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
public class FlatMap_MapMulti_Benchmark {

    @Benchmark
    @Fork(value = 2)
    @Measurement(iterations = 10, time = 1)
    @Warmup(iterations = 5, time = 1)
    public List<String> flatMap() {
        return Stream.iterate(1, i -> i + 1)
                .limit(100_000)
                .flatMap(num -> IntStream.rangeClosed('a', 'z')
                        .mapToObj(i -> num + "-" + (char) i))
                .collect(Collectors.toList());

    }

    @Benchmark
    @Fork(value = 2)
    @Measurement(iterations = 10, time = 1)
    @Warmup(iterations = 5, time = 1)
    public List<String> mapMulti() {
        return Stream.iterate(1, i -> i + 1)
                .limit(100_000)
                .<String>mapMulti((num, consumer) -> {
                    for (char c = 'a'; c <= 'z'; c++) {
                        consumer.accept(num + "-" + c);
                    }
                })
                .collect(Collectors.toList());
    }
}

Here are the benchmark results,

Benchmark                                           Mode  Samples    Score  Score error  Units
c.j.mapMulti.FlatMap_MapMulti_Benchmark.flatMap     avgt       20  318.776       94.230  ms/op
c.j.mapMulti.FlatMap_MapMulti_Benchmark.mapMulti    avgt       20  240.144       88.563  ms/op

The code using mapMulti takes 240 milliseconds per operation (ms/op) whereas the flatMap code takes 318 ms/op.

Let us re-check the performance number by using @OperationsPerInvocation annotation as each method operates on 100K numbers. It will help us test the performance of flatMap vs mapMulti per element.

To do that, we just replace

@OutputTimeUnit(TimeUnit.MILLISECONDS)

with

@OutputTimeUnit(TimeUnit.NANOSECONDS)
@OperationsPerInvocation(100_000)

As before, the results show that mapMulti is faster than flatMap.

Benchmark                                           Mode  Samples     Score  Score error  Units
c.j.mapMulti.FlatMap_MapMulti_Benchmark.flatMap     avgt       20  2707.993      434.600  ns/op
c.j.mapMulti.FlatMap_MapMulti_Benchmark.mapMulti    avgt       20  1934.614      162.427  ns/op

Filtering out the empty optional from a list

There is an interesting use-case of mapMulti with Optionals. We have a List<Optional<Integer>>. And we want to extract only the non-optional elements as a List<Integer>. We could use flatMap or a mapMulti for this. Let us compare the performance of each.

package com.javadevcentral.mapMulti;

//imports not shown 

@BenchmarkMode(Mode.AverageTime)
public class FlatMap_MapMulti_OptionalBenchmark {
    private static List<Optional<Integer>>optionals;

    static {
        optionals = IntStream.range(0, 100_000)
                .mapToObj(Optional::of)
                .collect(Collectors.toCollection(ArrayList::new));
        optionals.addAll(IntStream.range(0, 50_000)
                .mapToObj(i -> Optional.<Integer>empty())
                .toList());
        Collections.shuffle(optionals);
    }

    @Benchmark
    @Fork(value = 2)
    @Measurement(iterations = 10, time = 1)
    @Warmup(iterations = 5, time = 1)
    public List<Integer> filter() {
        return optionals.stream()
                .filter(Optional::isPresent)
                .map(Optional::get)
                .collect(Collectors.toList());
    }

    @Benchmark
    @Fork(value = 2)
    @Measurement(iterations = 10, time = 1)
    @Warmup(iterations = 5, time = 1)
    public List<Integer> flatMap() {
        return optionals.stream()
                .flatMap(Optional::stream)
                .collect(Collectors.toList());
    }

    @Benchmark
    @Fork(value = 2)
    @Measurement(iterations = 10, time = 1)
    @Warmup(iterations = 5, time = 1)
    public List<Integer> mapMulti() {
        return optionals.stream()
                //.<Integer>mapMulti((o, consumer) -> o.ifPresent(consumer))
                .<Integer>mapMulti(Optional::ifPresent)
                .collect(Collectors.toList());
    }
}

We build the list of optionals with 100K integers (0, 1, 2, 3.. 99,999) and add 50K empty optionals and shuffle the list.

Here we have three methods, each doing the same thing (to get the non-empty elements).

  1. The first method uses a simple filter and map on the stream to extract the integer elements from the list of optionals.
  2. The second one uses flatMap by calling the stream() method on the Optional. The Optional#stream was added in Java 9. (Refer to Optional – New methods in Java 9 through 11 post to learn the new methods added to the Optional class)
  3. Finally comes the interesting part. When using mapMulti, the method reference used in the above code might be confusing. Looking at the equivalent lambda expression (commented code) will make it clear. In the mapping function, we get the optional element and a Consumer<Integer> and we call ifPresent on the optional and pass the consumer. Internally, if the optional is not empty, the underlying integer value will be passed to the consumer. Hence, I have written this lambda expression using Type::method version of method reference. (I have a post on Java method references explaining all the types of method references in Java).

Here are the benchmark results:

Benchmark                                                   Mode  Samples  Score  Score error  Units
c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.filter      avgt       20  1.797        0.108  ms/op
c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.flatMap     avgt       20  2.903        0.315  ms/op
c.j.mapMulti.FlatMap_MapMulti_OptionalBenchmark.mapMulti    avgt       20  1.855        0.420  ms/op

The performance of mapMulti and filter are very similar, whereas using flatMap is costlier here.

Since we have 150K elements, to use OperationsPerInvocation, we replace the @OutputTimeUnit(TimeUnit.MILLISECONDS) with the following.

@OutputTimeUnit(TimeUnit.NANOSECONDS)
@OperationsPerInvocation(150_000)

The results are:

Benchmark                                                   Mode  Samples   Score  Score error  Units
c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.filter      avgt       20  12.344        1.942  ns/op
c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.flatMap     avgt       20  19.902        1.582  ns/op
c.t.mapMulti.FlatMap_MapMulti_OptionalBenchmark.mapMulti    avgt       20  10.375        0.828  ns/op

Summary of performance benchmark

With this, can we say that using mapMulti is always better than flatMap?

It depends..

For the examples scenarios or use-cases I’ve taken, it turned out that mapMulti was performing better than a flatMap. You have to do similar benchmarking for your use-case and workload to determine which would be better.

Replacing with primitive types

The mapMulti accepted a BiConsumer of types T and Consumer<R>. They have also added mapMulti methods for mapping elements to primitive types (int, long, and double) viz., mapMultiToInt, mapMultiToLong and mapMultiToDouble.

mapMultiToInt

The mapMultiToInt accepts a BiConsumer of types T (as before) and IntConsumer. An IntConsumer is a functional interface which accepts a primitive int value. Hence when we map a stream element to a primitive int, it returns an IntStream and there is no boxing involved.

IntStream mapMultiToInt(BiConsumer<? super T,? super IntConsumer> mapper)

Let us look at an example. We have a list of fruit names and we use mapMultiToInt to map each string to its length. This returns an IntStream and we all sum() on it to get the sum of all fruit name’s length.

List<String> strings = List.of("apple", "pear", "grapes", "banana");
int sumOfLengths = strings.stream()
        .mapMultiToInt((fruit, intConsumer) -> intConsumer.accept(fruit.length()))
        .sum();
System.out.println(sumOfLengths); //21

As another example, we have a Student and Exam model, as shown below with some data. Each student has taken up a list of exams and the exam object has the marksScored data.

record Exam(String name, int marksScored) {
}

record Student(String name, List<Exam> exams) {

}
List<Student> studentDetails = List.of(
        new Student("S1",
                List.of(new Exam("E1", 98),
                        new Exam("E2", 78))),
        new Student("S2",
                List.of(new Exam("E1", 61),
                        new Exam("E2", 72)))
);

Let’s say we want to find the average mark scored among all students and all exams. We stream the students list, and for each student, we go through each of the exams taken up. For each exam, we get the marksScored and pass it to the mapMultiToInt’s IntConsumer. Finally, we call the average() on the IntStream to get the average.

double average = studentDetails.stream()
        .mapMultiToInt((student, intConsumer) -> student.exams()
                .forEach(exam -> intConsumer.accept(exam.marksScored())))
        .average()
        .orElse(0);
System.out.println(average); //77.25

mapMultiToLong and mapMultiToDouble

The mapMultiToLong and mapMultiToDouble work in a similar way to mapMultiToInt, but for long and double, respectively. Their method signatures are shown below.

LongStream mapMultiToLong(BiConsumer<? super T,? super LongConsumer> mapper)

DoubleStream mapMultiToDouble(BiConsumer<? super T,? super DoubleConsumer> mapper)

The second argument of the BiConsumer of mapMultiToLong is a LongConsumer (a primitive version of the Consumer interface for a long). Similarly, the mapMultiToDouble’s BiConsumer takes a DoubleConsumer. The mapMultiToLong and mapMultiToDouble return a LongStream and DoubleStream, respectively.

As a simple example, let us map a Long or a Double to three values, as shown below.

List<Long> longValues = List.of(10L, 20L);
long sumOfLongs = longValues.stream()
        .mapMultiToLong((longVal, longConsumer) -> {
            long primitiveLongVal = longVal; // unbox once
            longConsumer.accept(primitiveLongVal - 1);
            longConsumer.accept(primitiveLongVal);
            longConsumer.accept(loprimitiveLongValngVal + 1);
        })
        .sum();
System.out.println(sumOfLongs);//90

List<Double> doubleValues = List.of(10.4, 20.1);
double sumOfDoubles = doubleValues.stream()
        .mapMultiToDouble((doubleVal, doubleConsumer) -> {
            double primitiveDoubleVal = doubleVal; //unbox once
            doubleConsumer.accept(primitiveDoubleVal - 1);
            doubleConsumer.accept(primitiveDoubleVal);
            doubleConsumer.accept(primitiveDoubleVal + 1);
        })
        .sum();
System.out.println(sumOfDoubles);//91.5

The mapMulti in primitive specialization of streams

In the final section of the Java Streams mapMulti post, let us look at the mapMulti methods added to the primitive specialization of streams (IntStream, LongStream, and DoubleStream).

In the IntStream, we have the mapMulti method, as shown below.

default IntStream mapMulti(IntStream.IntMapMultiConsumer mapper)

where the IntMapMultiConsumer is a functional interface. It takes a primitive int value and an IntConsumer.

@FunctionalInterface
interface IntMapMultiConsumer {
    void accept(int value, IntConsumer ic);
}

@FunctionalInterface
public interface IntConsumer {
    void accept(int value);
}

To summarize, the mapMulti method on IntStream takes a primitive int and an IntConsumer and returns an IntStream back.

The mapMulti methods on LongStream and DoubleStream have similar methods.

LongStream mapMulti(LongStream.LongMapMultiConsumer mapper)

DoubleStream mapMulti(DoubleStream.DoubleMapMultiConsumer mapper)

interface LongMapMultiConsumer {
    void accept(long value, LongConsumer lc);
}

interface DoubleMapMultiConsumer {
    void accept(double value, DoubleConsumer dc);
}

Examples are shown below.

IntStream intStream = IntStream.of(10, 20);
int sumOfInts = intStream
        .mapMulti((intVal, intConsumer) -> {
            intConsumer.accept(intVal - 1);
            intConsumer.accept(intVal);
            intConsumer.accept(intVal + 1);
        })
        .sum();
System.out.println(sumOfInts); //90

LongStream longStream = LongStream.of(10, 20);
long sumOfLongs = longStream
        .mapMulti((longVal, longConsumer) -> {
            longConsumer.accept(longVal - 1);
            longConsumer.accept(longVal);
            longConsumer.accept(longVal + 1);
        })
        .sum();
System.out.println(sumOfLongs); //90

DoubleStream doubleStream = DoubleStream.of(10.4, 20.1);
double sumOfDoubles = doubleStream
        .mapMulti((doubleVal, doubleConsumer) -> {
            doubleConsumer.accept(doubleVal - 1);
            doubleConsumer.accept(doubleVal);
            doubleConsumer.accept(doubleVal + 1);
        })
        .sum();
System.out.println(sumOfDoubles); //91.5

Conclusion

In this detailed post on Java Stream mapMulti, we saw how a mapMulti method differs from a flatMap. We saw examples of mapMulti methods and saw the performance of it measured using JMH.

References