AWS Java Lambda not connecting to Internet – no VPC

I created a Lambda function in AWS, it does not have a VPC configured, which seems to be the common reason for missing internet access, yet it seemingly cannot connect to the internet. I’m sure I’m making some obvious mistake but I can’t really think of anything.

I’ve tried to recreate the connections my Lambda uses in a main function locally and it works just fine there.

Output of main method on my machine

2021-09-12 14:54:05  INFO  FetchCommodityPrice - google.com reached? true
2021-09-12 14:54:05  INFO  FetchCommodityPrice - fred.stlouisfed.org reached? true
2021-09-12 14:54:05  DEBUG Logger - [FredApi#observations] ---> GET https://api.stlouisfed.org/fred/series/observations?series_id=PSOILUSDM&api_key=[REMOVED]&observation_start=2021-06-01&file_type=json HTTP/1.1
2021-09-12 14:54:05  DEBUG Logger - [FredApi#observations] ---> END HTTP (0-byte body)
2021-09-12 14:54:06  DEBUG Logger - [FredApi#observations] <--- HTTP/1.1 200 OK (1035ms)
2021-09-12 14:54:06  DEBUG Logger - [FredApi#observations] connection: keep-alive
2021-09-12 14:54:06  DEBUG Logger - [FredApi#observations] content-length: 483
[...response body...]
2021-09-12 14:54:06  DEBUG Logger - [FredApi#observations] <--- END HTTP (483-byte body)
2021-09-12 14:54:06  DEBUG FetchCommodityPrice - latest: 1451.68977904762

Output of my Lambda execution

START RequestId: [request-id] Version: $LATEST
2021-09-12 12:58:06 [request-id] INFO  FetchCommodityPrice - google.com reached? false
2021-09-12 12:58:09 [request-id] INFO  FetchCommodityPrice - fred.stlouisfed.org reached? false
2021-09-12 12:58:09 [request-id] DEBUG FetchCommodityPrice - Fetching data for commodity PSOILUSDM
2021-09-12 12:58:09 [request-id] DEBUG Logger - [FredApi#observations] ---> GET https://api.stlouisfed.org/fred/series/observations?series_id=PSOILUSDM&api_key=[REMOVED]&observation_start=2021-06-12&file_type=json HTTP/1.1
2021-09-12 12:58:09 [request-id] DEBUG Logger - [FredApi#observations] ---> END HTTP (0-byte body)
END RequestId: [request-id]
REPORT RequestId: [request-id]  Duration: 10010.46 ms   Billed Duration: 10000 ms   Memory Size: 150 MB Max Memory Used: 134 MB Init Duration: 1904.89 ms   
2021-09-12T12:58:14.479Z [request-id] Task timed out after 10.01 seconds

Code of my Lambda Handler

package com.kwjoshua.soyprice;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.kwjoshua.soyprice.fred.FredApi;
import com.kwjoshua.soyprice.fred.Observation;
import com.kwjoshua.soyprice.fred.SeriesObservations;
import feign.Feign;
import feign.jackson.JacksonDecoder;
import feign.slf4j.Slf4jLogger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.net.InetAddress;
import java.time.LocalDate;
import java.util.Optional;

@SuppressWarnings("unused")
public class FetchCommodityPrice implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> {
    private static final Logger logger = LoggerFactory.getLogger(FetchCommodityPrice.class);

    private final ObjectMapper objectMapper;
    private final String fredApiKey;
    private final FredApi fredApi;

    public FetchCommodityPrice() {
        this.objectMapper = JsonMapper.builder().findAndAddModules().build();
        this.fredApiKey = System.getenv("FRED_API_KEY");
        this.fredApi = createFredApiClient(System.getenv("FRED_API_URL"));
    }

    @Override
    public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent requestEvent, Context context) {
        Optional<Commodity> commodity = Commodity.byPath(requestEvent.getPath());
        if (commodity.isEmpty()) {
            return new APIGatewayProxyResponseEvent().withStatusCode(404);
        }
        testConnection();
        try {
            return okResponse(getCommodityPrice(commodity.get()));
        } catch (Exception e) {
            return new APIGatewayProxyResponseEvent().withStatusCode(500);
        }
    }

    private static void testConnection() {
        logReachable("google.com");
        logReachable("fred.stlouisfed.org");
    }

    private static void logReachable(String host) {
        try {
            logger.info(host + " reached? " + InetAddress.getByName(host).isReachable(2000));
        } catch (IOException e) {
            logger.error("Error trying to reach host '" + host + "'", e);
        }
    }

    private CommodityPrice getCommodityPrice(Commodity commodity) {
        logger.debug("Fetching data for commodity " + commodity.getId());
        SeriesObservations observations = fredApi.observations(
                commodity.getId(), fredApiKey, LocalDate.now().minusMonths(3).toString());
        Observation lastObservation = observations.getObservations().get(observations.getObservations().size() - 1);
        logger.debug("last observed price: " + lastObservation.getValue());
        return new CommodityPrice(lastObservation.getValue(), lastObservation.getDate());
    }

    private APIGatewayProxyResponseEvent okResponse(CommodityPrice commodityPriceJson) throws JsonProcessingException {
        return new APIGatewayProxyResponseEvent()
                .withStatusCode(200)
                .withBody(objectMapper.writeValueAsString(commodityPriceJson))
                .withIsBase64Encoded(false);
    }

    private static FredApi createFredApiClient(String fredApiUrl) {
        return Feign.builder()
                    .logger(new Slf4jLogger())
                    .logLevel(feign.Logger.Level.FULL)
                    .decoder(new JacksonDecoder())
                    .target(FredApi.class, fredApiUrl);
    }

    public static void main(String[] args) {
        testConnection();
        FredApi api = createFredApiClient("https://api.stlouisfed.org/fred");
        SeriesObservations observations = api.observations(Commodity.OIL.getId(), args[0], "2021-06-01");
        logger.debug("latest: " + observations.getObservations().get(observations.getObservations().size() - 1).getValue());
    }
}

VPC configuration of Lambda

Screenshot illustrating no VPC is configured

Answer

It was a timeout issue. The site does answer, but much slower than it does when I connect from my local network.

Increasing the maximum lambda function timeout solved the issue.