Final Update: Tuesday, August 29th 2017 15:10 UTC
We’ve confirmed that all systems are back to normal as of August 29th 2017 14:40 UTC. Our logs indicate that the incident started on August 29th, 2017 13:10 UTC. During the course of this incident, VSTS customers may have observed a delay in notifications.
The load on the affected SMTP server has been reduced allowing for notifications to no longer be delayed. As a longer term solution we are assessing the balance of the traffic across all of the SMTP servers.
We apologize for any inconvenience this may have caused.
Sincerely,
Randy
Update: Tuesday, August 29th 2017 14:43 UTC
Our DevOps team have identified that the notification delays are happening due to high load of e-mails received by our SMTP servers. We are actively working with our partner teams to implement a mitigation strategy. Currently we are working on diverting traffic off of the affected SMTP servers to servers with lower load. This change is actively being worked on.
- Next Update: Before Tuesday, August 29th 2017 18:43 UTC
Sincerely,
Randy
Initial Update: Tuesday, August 29th 2017 13:47 UTC
We are investigating Notification Delays affecting users of the service. Users in may experience delayed notification for events like Work Item Changes and Pull Request updates.
- Next Update: Before Tuesday, August 29th 2017 15:00 UTC
Sincerely,
Randy