I’m trying stream the data from an HTTP (GET) response to another HTTP (POST) request. With old HttpURLConnection I would take the responses OutputStream, read parts into a buffer and write them to the requests InputStream.
I’ve already managed to do the same with HttpClient in Java 11 by creating my own Publisher that is used in the POST to write the request body. The GET request has a
ofByteArrayConsumer that sends the chunks to the custom Publisher which itself then sends the chunks to the subscribing HTTP POST request.
But I think this is not the correct approach as it looks like there is something in the API that looks like this could be done directly without implementing publishers and subscribers myself.
HttpResponse.BodyHandlers.ofPublisher() which returns a
Publisher<List<ByteBuffer> which I can use for the HTTP GET request. Unfortunately for my POST request, there is
HttpRequest.BodyPublishers.fromPublisher which expects a
Publisher<? extends ByteBuffer> so it seems that the fromPublisher only works for a publisher that holds a complete ByteBuffer and not one that sends several ByteBuffers for parts of the data.
Do I miss something here to be able to connect the BodyPublisher from one request to the other?
You’re not missing anything. This is simply a use case that is not supported out of the box for now. Though the mapping from
List<ByteBuffer> is trivial, the inverse mapping is less so. One easy (if not optimal) way to adapt from one to the other could be to collect all the buffers in the list into a single buffer – possibly combining
HttpResponse.BodyHandlers.buffering() if you want to control the amount of bytes in each published
List<ByteBuffer> that you receive from upstream.