Spark 2.4.0 Avro Java – cannot resolve method from_avro Code Answer

Hello Developer, Hope you guys are doing great. Today at Tutorial Guruji Official website, we are sharing the answer of Spark 2.4.0 Avro Java – cannot resolve method from_avro without wasting too much if your time.

The question is published on by Tutorial Guruji team.

I’m trying to run a spark stream from a kafka queue containing Avro messages.

As per I should be able to use from_avro to convert column value to Dataset<Row>.

However, I’m unable to compile the project as it complains from_avro cannot be found. I can see the method declared in package.class of the dependency.

How can I use the from_avro method from org.apache.spark.sql.avro in my Java code locally?

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;

import static org.apache.spark.sql.functions.*;
import org.apache.spark.sql.avro.*;

public class AvroStreamTest {
    public static void main(String[] args) throws IOException, InterruptedException {

     // Creating local sparkSession here...

        Dataset<Row> df = sparkSession
                .option("kafka.bootstrap.servers", "host:port")
                .option("subscribe", "avro_queue")

        // Cannot resolve method 'from_avro'..."value"), jsonFormatSchema)).writeStream().format("console")



  <!-- more dependencies below -->


It seems like Java is unable to import names from sql.avro.package.class


It’s because of the generated class names, importing it as import org.apache.spark.sql.avro.package$; and then using package$.MODULE$.from_avro(...) should work

We are here to answer your question about Spark 2.4.0 Avro Java – cannot resolve method from_avro - If you find the proper solution, please don't forgot to share this with your team members.

Related Posts

Tutorial Guruji