We detected the issue due to the Cloudflare outage at 7:49am MDT (UTC -6). The VictorOps Technical Support Team was notified at 7:53am MDT. After verifying the issue and acknowledging Cloudflare’s status we created our StatusPage incident at 8am MDT.
Cloudflare offered resolution at 8:11am MDT and VictorOps became fully operational. We remained in a monitoring state until 9:12am MDT, upon that time we resolved the incident.
As always, we welcome and encourage you to reach out to the VictorOps Support team with any questions: victorops-support@splunk.com
Posted Jul 02, 2019 - 14:26 MDT
Resolved
We've been actively monitoring the implemented fix and the overall situation at the moment. Based on positive technical observations, we're moving this incident from Monitoring to Resolved.
Thank you for your patience during this time.
Again, we sincerely apologize for any inconvenience this issue may have caused. As a follow-up to this issue, our teams will be conducting a Post Incident Review and will follow-up with our findings on this StatusPage. If you have any immediate questions in reference to this incident, please don't hesitate to contact our Support Team with an email to victorops-support@splunk.com or submit the form on our Contact Support page (https://victorops.com/contact-support/).
Posted Jul 02, 2019 - 09:12 MDT
Monitoring
Portal access has been restored.
Posted Jul 02, 2019 - 08:11 MDT
Update
We are continuing to work on a fix for this issue.
Posted Jul 02, 2019 - 08:02 MDT
Identified
The VictorOps portal is currently unavailable via web and mobile. We are ingesting alerts, processing incidents, and delivering notifications to users as normal. You can ack or resolve incidents by responding to phone call and SMS notifications.
Cloudflare is facing network issues (which is in turn having downstream effects on those who utilize its services, including VictorOps) and is working towards a resolution. Please see https://www.cloudflarestatus.com/
We are actively working to resolve this issue as quickly as possible.
Use the Subscribe to Updates option or follow @VOSupport on Twitter for updates. If you have any immediate questions or concerns, our support team is standing by to respond via email victorops-support@splunk.com or submit the form on our Contact Support page (https://victorops.com/contact-support/).