Andrea Saez
posted this on Jan 17 04:13 pm
On January 17, 2012 we experienced a few delivery issues on our server. Here's what you need to know.
What went wrong
A network switch and an optical fiber between our datacenters failed due to bad weather. This affected a switch; some data was corrupted and had an impact on all information that was connected to that specific switch.
What we're doing to improve this
- Add a permanent VPN between datacenters for faster failover scenario
- Route the fiber optical directly on firewall instead of routing it in our core router
- CDN: Spread the infrastructure to a different geographical location
We certainly apologize to all our customers for any inconvenience we may have caused during this time.
- The CakeMail Team
We are experiencing a few issues with delivery - some campaigns may be delayed. We are looking into the problem.
Update 5:16PM EST: The problem is due to a faulty switch. Currently replacing it.
Update 7:02PM EST: Still on the issue.
Update 7:32PM EST: Starting to process campaigns again. We're not yet on full capacity though and we have some backlog.
Update 8:44PM EST: It appears that the dedicated fiber between our two datacenters have been cut. Things are now routing through a VPN but it is slower. ETA for this is 1:00AM EST. Full sending capacity is expected only then.
Update 10:30PM EST: Although the fiber still needs to be repaired, we currently resolved all delivery delays. Let us know if you still experience issues.
Update 1:00AM EST: All fixed. Thank you for your patience!