Spring Batch : MultiResourcePartitioner How to set order resources by name files Code Answer

Hello Developer, Hope you guys are doing great. Today at Tutorial Guruji Official website, we are sharing the answer of Spring Batch : MultiResourcePartitioner How to set order resources by name files without wasting too much if your time.

The question is published on by Tutorial Guruji team.

I have Job Spring Batch which read and write multiple files for that i’m use MultiResourcePartitioner for reading multiple files.

When i set the list of files to resources (partitionner.setResources(listFiles)) the pMultiResourcePartitioner randomly processes files does not respect the order of files in listFiles

this my code :

    @Bean
    @JobScope
    public MultiResourcePartitioner paritioner() throws IOException {
        MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
        Resource[] resources = new Resource[listFiles.size()];
        ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();

        HashMap<Integer, String> collect = listFiles
            .stream().sorted(Comparator.reverseOrder()).collect(HashMap<Integer, String>::new, (map, streamValue) -> map.put(map.size(), streamValue), (mapFirst, mapSecond) -> {
            });
        collect.forEach((index, value) -> resources[index] = resolver.getResource("file:" + value));

        partitioner.partition(1);
        partitioner.setResources(resources);
        return partitioner;
    }

How to avoid random process file in MultiResourcePartitioner

Answer

When you decide to use a partitioned step, you should be assuming that partitions will be processed in parallel by different workers (which could be local threads or remote worker JVMs). Going parallel is not compatible with ordering. The order in which partitions are processed in undefined.

In your case, the MultiResourcePartitioner will create a partition per file, and each file will be processed by a distinct worker. The order in which you pass resources to the partitioner does not (and should not) matter.

We are here to answer your question about Spring Batch : MultiResourcePartitioner How to set order resources by name files - If you find the proper solution, please don't forgot to share this with your team members.

Related Posts

Tutorial Guruji