Cellular Connection Issues
June 13th. 22:21 UTC
Update
Customer Success are monitoring 2G/3G connection/authentication delays due to an increase in reconnection load on those radio access technologies (RATs). Our affected global gateway partner is continuing work to increase signaling capacity for 2G/3G connection/authentication.
June 13th. 17:34 UTC
Update
All core nodes and redundant links have been restored to normal service levels, including both London and Paris data centers, and devices are continuing to reconnect (with higher than normal system load). A unique system failure in vendor hardware triggered a state issue that propagated across redundant data centers and affected redundant links. Root cause analysis is still under way and a post-mortem will follow with additional information.
Customer devices may still see intermittent cellular network connection/authentication issues as device modules attempt to reconnect following the initial outage, but this is now improving. Due to excessive signaling load during restoration efforts, which occurred as waiting devices attempted to reconnect simultaneously, increased GRX/IPX link capacity remains elevated to mitigate cellular network connection/authentication delays.
June 13th. 16:29 UTC
Update
Hardware links have been successfully restored by an upstream cellular gateway provider. Customer devices may see intermittent cellular network connection/authentication issues as device modules attempt to reconnect following the initial outage. Due to excessive signaling load during restoration efforts, which occurred as waiting devices attempted to reconnect simultaneously, increased GRX/IPX link capacity remains elevated to mitigate cellular network connection/authentication delays.
June 13th. 14:29 UTC
Update
Affected devices have continued to reconnect following the restoration of network services at our global gateway partner. Due to excessive signaling load during restoration efforts, which occurred as waiting devices attempted to reconnect simultaneously, our global gateway partner also increased GRX/IPX link capacity to mitigate cellular network connection/authentication delays. Some devices reconnecting during the period of excessive signaling load timed out while attempting a connection/authentication to the cellular network, and some of those devices have reconnected.
Update
June 12th. 23:58 UTC
Network services are beginning to come back online. Our service provider’s engineering and success teams are continuing to monitor as service returns toward standard levels
June 12th. 22:56 UTC
Update
Network partner engineers are beginning to reroute traffic following systemwide IP failures causing the disruption of service. We will continue to update as we receive more information on progress toward full service restoration.
June 12th. 20:19 UTC
Update
Solid GPS is continuing to monitor as our SIM card partner works to restore service. We are awaiting estimated time to resolution and will update when available.
June 12th. 19:08 UTC
Update
Issue confirmed to be wider in scope
June 12th. 18:47 UTC
Update
An issue has been identified with a cellular gateway provider affecting all network connectivity routed out of their global point of presence. The service provider has escalated the issue and is actively monitoring for service recovery.
June 12th. 18:27 UTC
Update
We are investigating possible issues connecting to the cellular network for some customers.