Final Update: Monday, May 28th 2018 11:23 UTC
We’ve confirmed that all systems are back to normal as of May 28th 2018 10:03 UTC. Our logs show the incident started on May 28th 2018 06:25 UTC and that during the 3 hours and 38 minutes that it took to resolve the issue. Active customers experienced performance degradation across multiple geographic regions during this time. Sorry for any inconvenience this may have caused.
- Root Cause: We are continuing to investigate the root cause
- Chance of Re-occurrence: High
- Lessons Learned: We are working to understand the root cause of the incident. A public postmortem will be created in 3 days for this incident
- Incident Timeline: 3 hours & 38 minutes – May 28th 2018 06:25 UTC UTC through May 28th 2018 10:03 UTC
Sincerely,
Brett
Update: Monday, May 28th 2018 10:22 UTC
Our DevOps team continues to investigate issues across multiple geographic regions. Root cause is not fully understood at this time, though it is suspected to be with identity services. The team is actively mitigating the issue by adding capacity. The problem began at May 28th 2018 06:25 UTC. We currently have no estimated time for resolution.
- Next Update: Before Monday, May 28th 2018 11:30 UTC
Sincerely,
Brett
Update: Monday, 28 May 2018 08:55 UTC
Our DevOps team continues to investigate performance issues with VSTS. The initial symptoms indicate that we are running low on capacity, hence we are in the process of adding 3 additional Application Tiers to oursservice. The problem began at May 28th 2018 06:25 UTC.
- Next Update: Before Monday, May 28th 2018 10:15 UTC
Sincerely,
RajeshKumar
We're investigating Performance Degradation in West Europe, North Central US & South Central US. Customers hosted in this regions may experience slowness accessing their VSTS accounts.
- Next Update: Before Monday, May 28th 2018 08:15 UTC
Sincerely,
RajeshKumar