How to execute multiple SQL queries defined in a SQL file

I have an SQL file containing many SQL queries like so :

CREATE TABLE ..... ;
ALTER TABLE .... ;

I want to execute these queries in a single spark.sql call How to achieve this in JAVA ?

Thanks

Answer

Spark doesn’t support executing multiple queries with a single call to spark.sql (see this SO answer). If you don’t mind calling spark.sql multiple times, you can read the .sql file into a string, split it into separate queries, and then pass it to spark.sql, like so:

import java.nio.file.*;

public static void main(String[] args) {
    String[] commands;

    try {
        Path sqlPath = FileSystems.getDefault().getPath("query.sql");
        String sql = new String(Files.readAllBytes(sqlPath));
        commands = sql.split(";", 0);
    } catch (Exception e) {
        System.out.println("failed to read SQL file");
        return;
    }

    for (int i = 0; i < commands.length; i++) {
        spark.sql(commands[i]);
    }
}

Leave a Reply

Your email address will not be published. Required fields are marked *