Sentry not reporting all errors from AWS Lambda

[rynkowsg]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node

SDK Version

9.11.0

Framework Version

No response

Link to Sentry event

No response

Reproduction Example/SDK Setup

No response

Steps to Reproduce

Node: 18.20.6

Config

const Sentry = require("@sentry/node");

//...

Sentry.init({
  dsn: sentryDsn,
  environment: stage,
  release: APP_VERSION,
  // When enabled, stack traces are automatically attached to all messages logged. This defaults to false.
  attachStacktrace: false,
  // the total amount of breadcrumbs that should be captured. This defaults to 100.
  maxBreadcrumbs: 100,
  debug: true,
  defaultIntegrations: false,
  includeLocalVariables: true,
  integrations: [
    // Capture console logs as breadcrumbs. (default)
    Sentry.consoleIntegration(),
    // Deduplicate certain events to avoid receiving duplicate errors. (default)
    // Sentry.dedupeIntegration(),
    // Add local variables to exception frames. (default)
    Sentry.localVariablesIntegration({ captureAllExceptions: true, }),
    // Allows the SDK to provide original functions and method names, even when
    // those functions or methods are wrapped by our error or breadcrumb handlers. (default)
    Sentry.functionToStringIntegration(),
    // Registers handlers to capture global uncaught exceptions. (default)
    Sentry.onUncaughtExceptionIntegration(),
    // Registers handlers to capture global unhandled promise rejections. (default)
    Sentry.onUnhandledRejectionIntegration(),
    // Extracts all non-native attributes from the error object and attaches them to the event as extra data.
    Sentry.extraErrorDataIntegration(),
    // Allows you to configure linked errors. (default)
    Sentry.linkedErrorsIntegration({limit: 10, key: "cause"}),
    // Capture spans & breadcrumbs for http requests. (default)
    Sentry.httpIntegration(),
    // Capture spans & breadcrumbs for node fetch requests. (default)
    Sentry.nativeNodeFetchIntegration(),
    // enable Express.js middleware tracing
    Sentry.expressIntegration(),
  ],
  // Data to be set to the initial scope
  initialScope: {
    tags: {
      "cu.gitRev": GIT_REV,
      "cu.gitTag": GIT_TAG,
      "cu.appVersion": APP_VERSION,
      "cu.releaseDate": RELEASE_DATE,
    },
  },
  // Performance Monitoring
  tracesSampleRate: 1.0, //  Capture 100% of the transactions
  // Set sampling rate for profiling - this is relative to tracesSampleRate
  profilesSampleRate: 1.0,
});

// ..


const app = express();

// use middlewares
// use routes

Sentry.setupExpressErrorHandler(app);

app.use((err, req, res, next) => {
  logger.info("Error handler start ", err);
  logger.info(`Sentry: ${res.sentry}`);
  if (res.headersSent) {
    return next(err);
  }

  const isStatusError = err instanceof StatusError;
  const status = typeof err.status === "number" ? err.status : 500;
  const message = isStatusError ? err.message : "Server error. Please retry.";

  if (status >= 500) {
    console.error(err);
  }

  if (err instanceof ValidationError) {
    res.status(err.statusCode).json(err);
    return res.end();
  }

  res.status(status).json({
    ...(isStatusError ? err : {}),
    message: message,
  });
  return res.end();
});

const handler = serverless(app, { binary: ["application/pdf"] });
// ...

sample testing endpoints in passed to Express.js router:

r.get("/api/status", (req, res) => res.status(200).json({ status: "api-v2 up" }));
r.get("/api/throw", async () => {
  const someVariable = "contains some value";
  console.log("test log from Console");
  logger.info(`variable value: ${someVariable}`);
  throw new Error(`Sentry test! ${moment().utc().format()}`);
});
r.post("/api/throw2", async (req, res) => {
  const { msg } = req.body;
  throw new Error(`${msg} ${moment().utc().format()}`);
});

Expected Result

All errors getting reported to Sentry.

Actual Result

Not all errors are reported to Sentry.

I set debug: true in options and I noticed these:

2025-04-07T10:31:37.648000+00:00 2025/04/07/[$LATEST]17f2b086d6674e16a9ca0fc0dc3509e3 2025-04-07T10:31:37.648Z  1212eb8a-f7c5-4239-841d-2ea330a15875    DEBUG   Sentry Logger [debug]: @opentelemetry_sentry-patched/instrumentation-http outgoingRequest on request error() Error: socket hang up                                                                                                                                     
    at connResetException (node:internal/errors:720:14)                                                                                                                                                            
    at TLSSocket.socketOnEnd (node:_http_client:525:23)                                                                                                                                                            
    at TLSSocket.emit (node:events:529:35)                                                                                                                                                                         
    at TLSSocket.emit (node:domain:489:12)                                                                                                                                                                         
    at endReadableNT (node:internal/streams/readable:1400:12)                                                                                                                                                      
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {                                                                                                                               
  code: 'ECONNRESET'                                                                                                                                                                                               
}                                                                                                                                                                                                                  
2025-04-07T10:31:37.687000+00:00 2025/04/07/[$LATEST]17f2b086d6674e16a9ca0fc0dc3509e3 2025-04-07T10:31:37.687Z  1212eb8a-f7c5-4239-841d-2ea330a15875    INFO    Sentry Logger [log]: Recording outcome: "network_error:error"                                                                                                                                                                                                          
2025-04-07T10:31:37.705000+00:00 2025/04/07/[$LATEST]17f2b086d6674e16a9ca0fc0dc3509e3 2025-04-07T10:31:37.705Z  1212eb8a-f7c5-4239-841d-2ea330a15875    ERROR   Sentry Logger [error]: Error while sending envelope: Error: socket hang up                                                                                                                                                                                             
    at connResetException (node:internal/errors:720:14)                                                                                                                                                            
    at TLSSocket.socketOnEnd (node:_http_client:525:23)                                                                                                                                                            
    at TLSSocket.emit (node:events:529:35)                                                                                                                                                                         
    at TLSSocket.emit (node:domain:489:12)                                                                                                                                                                         
    at endReadableNT (node:internal/streams/readable:1400:12)                                                                                                                                                      
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {                                                                                                                               
  code: 'ECONNRESET'                                                                                                                                                                                               
}                 

fulllog.txt

For scenario where I call /api/throw, then /api/throw2 and /api/status.

The first error got though, the second error is not reported.

While I've been testing it today on a deployed lambda, Sentry seems to drop more events then pass through. And it is not they are not accepted - they are not reaching to the Sentry server. The project it belongs are still running low numbers - in stats I see, on total of 16 error, 16 got accepted, but there should be more then 16 errors reported.

These logs come from SDK 9.11.0, but I tested with v8 and I got similar results.
The code runs on AWS Lambda.

[mydea]

Hey, thanks for writing in.

Could the problem be that the Sentry events are not being flushed before the serverless handler is exited? You need to make sure to call Sentry.flush() before the function is ended - @sentry/node generally assumes to be long-lived and will flush in the background by default.

[rynkowsg]

Ok, I can try to ensure Sentry.flush is called before the end of function.

Still, isn't odd, this situation? There is one request causing error - that one is reported. Then there is another causing different error - this one is not reported. And at the end there is a call to /status endpoint to ensure the lambda still is running. One event got though, the other not. And lambda after that still stay warm, so should be able to flush in the background. Especially I see in the logs an attempt:

INFO    Sentry Logger [log]: Flushing client reports based on interval.
INFO    Sentry Logger [log]: Flushing outcomes...
INFO    Sentry Logger [log]: Sending outcomes: [ { reason: 'network_error', category: 'error', quantity: 1 } ]

[lforst]

I know you selected SaaS as an option in the issue template but where are you pointing the DSN to. Network error seems rather funky. Like some infra issues, or routing, or something that is out of Sentry's realm of control actually.

[rynkowsg]

This is a SaaS setup. The DSN format is: https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@xxxxxxxxxxxxxxxxx.ingest.us.sentry.io/0000000000000000.

The service is running from the eu-west-2 region.
I looked now at VPC that lambda uses and it is pretty much public only VPC. Should be alright on our side.
I wouldn't post it if we had issues reaching other services, but we don't.

Yesterday before I produced the log I posted here, for over an hour I've been producing errors to capture and I looked at the Sentry debug logs in deployed lambda and I saw quite a few of these errors.

[lforst]

Ok it's really frustrating not to see the exact error in the logs. I am gonna add a debug statement there.

[lforst]

I added a log message. Would you mind bumping to the latest version and looking in the logs again? Thanks!

[rynkowsg]

I checked it last evening. The log didn't gave us much more above of what was there already.

2025-04-09T21:38:45.421000+00:00 2025/04/09/[$LATEST]a2d2eab647324434a1178b3e40c12e22 2025-04-09T21:38:45.421Z  d173998f-3e7c-4c6c-bc6f-88d383f46f26    DEBUG   Sentry Logger [debug]: @opentelemetry_sentry-patched/instrumentation-http outgoingRequest on request error() Error: socket hang up
    at connResetException (node:internal/errors:720:14)
    at TLSSocket.socketOnEnd (node:_http_client:525:23)
    at TLSSocket.emit (node:events:529:35)
    at TLSSocket.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ECONNRESET'
}
2025-04-09T21:38:45.468000+00:00 2025/04/09/[$LATEST]a2d2eab647324434a1178b3e40c12e22 2025-04-09T21:38:45.468Z  d173998f-3e7c-4c6c-bc6f-88d383f46f26    INFO    Sentry Logger [log]: Recording outcome: "network_error:error"
2025-04-09T21:38:45.468000+00:00 2025/04/09/[$LATEST]a2d2eab647324434a1178b3e40c12e22 2025-04-09T21:38:45.468Z  d173998f-3e7c-4c6c-bc6f-88d383f46f26    ERROR   Sentry Logger [error]: Encountered error running transport request: Error: socket hang up
    at connResetException (node:internal/errors:720:14)
    at TLSSocket.socketOnEnd (node:_http_client:525:23)
    at TLSSocket.emit (node:events:529:35)
    at TLSSocket.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ECONNRESET'
}
2025-04-09T21:38:45.484000+00:00 2025/04/09/[$LATEST]a2d2eab647324434a1178b3e40c12e22 2025-04-09T21:38:45.484Z  d173998f-3e7c-4c6c-bc6f-88d383f46f26    ERROR   Sentry Logger [error]: Error while sending envelope: Error: socket hang up
    at connResetException (node:internal/errors:720:14)
    at TLSSocket.socketOnEnd (node:_http_client:525:23)
    at TLSSocket.emit (node:events:529:35)
    at TLSSocket.emit (node:domain:489:12)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ECONNRESET'
}

fulllog-250409.txt

[rynkowsg]

At the end I think it was a failure on my side. Like @mydea posted a few days before:

make sure to call Sentry.flush() before the function is ended - @sentry/node generally assumes to be long-lived and will flush in the background by default.

When integrating Sentry I followed the Express guide, even though we run our app as AWS Lambda. I hadn't added Sentry.flush() until yesterday. Once I placed it just before returning from the lambda, the problem was resolved. Despite the lambda container staying warm, probably AWS was cutting off any open connections when the function terminated, leading to the socket errors.

I believe we can close this issue. Thanks for the assistance, and sorry for any confusion caused! 🙏