"Operational issue - Multiple services (N. Virginia)
Oct 20 2:22 AM PDT We have applied initial mitigations and we are observing early
signs of recovery for some impacted AWS Services. During this time, requests
may continue to fail as we work toward full resolution. We recommend customers
retry failed requests. While requests begin succeeding, there may be additional
latency and some services will have a backlog of work to work through, which
may take additional time to fully process. We will continue to provide updates
as we have more information to share, or by 3:15 AM.
Oct 20 2:01 AM PDT We have identified a potential root cause for error rates
for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the
issue appears to be related to DNS resolution of the DynamoDB API endpoint in
US-EAST-1. We are working on multiple parallel paths to accelerate recovery.
This issue also affects other AWS Services in the US-EAST-1 Region. Global services
or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB
Global tables may also be experiencing issues. During this time, customers may
be unable to create or update Support Cases. We recommend customers continue
to retry any failed requests. We will continue to provide updates as we have
more information to share, or by 2:45 AM.
Oct 20 1:26 AM PDT We can confirm significant error rates for requests made to
the DynamoDB endpoint in the US-EAST-1 Region. This issue also affects other
AWS Services in the US-EAST-1 Region as well. During this time, customers may
be unable to create or update Support Cases. Engineers were immediately engaged
and are actively working on both mitigating the issue, and fully understanding
the root cause. We will continue to provide updates as we have more information
to share, or by 2:00 AM.
Oct 20 12:51 AM PDT We can confirm increased error rates and latencies for multiple
AWS Services in the US-EAST-1 Region. This issue may also be affecting Case Creation
through the AWS Support Center or the Support API. We are actively engaged and
working to both mitigate the issue and understand root cause. We will provide
an update in 45 minutes, or sooner if we have additional information to share.
Oct 20 12:11 AM PDT We are investigating increased error rates and latencies
for multiple AWS services in the US-EAST-1 Region. We will provide another update
in the next 30-45 minutes.
"
"Operational issue - Multiple services (N. Virginia)
Oct 20 2:22 AM PDT We have applied initial mitigations and we are observing early
signs of recovery for some impacted AWS Services. During this time, requests
may continue to fail as we work toward full resolution. We recommend customers
retry failed requests. While requests begin succeeding, there may be additional
latency and some services will have a backlog of work to work through, which
may take additional time to fully process. We will continue to provide updates
as we have more information to share, or by 3:15 AM.
Oct 20 2:01 AM PDT We have identified a potential root cause for error rates
for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the
issue appears to be related to DNS resolution of the DynamoDB API endpoint in
US-EAST-1. We are working on multiple parallel paths to accelerate recovery.
This issue also affects other AWS Services in the US-EAST-1 Region. Global services
or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB
Global tables may also be experiencing issues. During this time, customers may
be unable to create or update Support Cases. We recommend customers continue
to retry any failed requests. We will continue to provide updates as we have
more information to share, or by 2:45 AM.
Oct 20 1:26 AM PDT We can confirm significant error rates for requests made to
the DynamoDB endpoint in the US-EAST-1 Region. This issue also affects other
AWS Services in the US-EAST-1 Region as well. During this time, customers may
be unable to create or update Support Cases. Engineers were immediately engaged
and are actively working on both mitigating the issue, and fully understanding
the root cause. We will continue to provide updates as we have more information
to share, or by 2:00 AM.
Oct 20 12:51 AM PDT We can confirm increased error rates and latencies for multiple
AWS Services in the US-EAST-1 Region. This issue may also be affecting Case Creation
through the AWS Support Center or the Support API. We are actively engaged and
working to both mitigate the issue and understand root cause. We will provide
an update in 45 minutes, or sooner if we have additional information to share.
Oct 20 12:11 AM PDT We are investigating increased error rates and latencies
for multiple AWS services in the US-EAST-1 Region. We will provide another update
in the next 30-45 minutes.
"
Wow! Not THAT is how you acknowledge and respond to an issue. Clear communication
and expectations.
"Operational issue - Multiple services (N. Virginia)
Oct 20 2:22 AM PDT We have applied initial mitigations and we are observing early
signs of recovery for some impacted AWS Services. During this time, requests
may continue to fail as we work toward full resolution. We recommend customers
retry failed requests. While requests begin succeeding, there may be additional
latency and some services will have a backlog of work to work through, which
may take additional time to fully process. We will continue to provide updates
as we have more information to share, or by 3:15 AM.
Oct 20 2:01 AM PDT We have identified a potential root cause for error rates
for the DynamoDB APIs in the US-EAST-1 Region. Based on our investigation, the
issue appears to be related to DNS resolution of the DynamoDB API endpoint in
US-EAST-1. We are working on multiple parallel paths to accelerate recovery.
This issue also affects other AWS Services in the US-EAST-1 Region. Global services
or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB
Global tables may also be experiencing issues. During this time, customers may
be unable to create or update Support Cases. We recommend customers continue
to retry any failed requests. We will continue to provide updates as we have
more information to share, or by 2:45 AM.
Oct 20 1:26 AM PDT We can confirm significant error rates for requests made to
the DynamoDB endpoint in the US-EAST-1 Region. This issue also affects other
AWS Services in the US-EAST-1 Region as well. During this time, customers may
be unable to create or update Support Cases. Engineers were immediately engaged
and are actively working on both mitigating the issue, and fully understanding
the root cause. We will continue to provide updates as we have more information
to share, or by 2:00 AM.
Oct 20 12:51 AM PDT We can confirm increased error rates and latencies for multiple
AWS Services in the US-EAST-1 Region. This issue may also be affecting Case Creation
through the AWS Support Center or the Support API. We are actively engaged and
working to both mitigate the issue and understand root cause. We will provide
an update in 45 minutes, or sooner if we have additional information to share.
Oct 20 12:11 AM PDT We are investigating increased error rates and latencies
for multiple AWS services in the US-EAST-1 Region. We will provide another update
in the next 30-45 minutes.
"