A search outage affected all Mintlify-powered sites after our upstream search provider executed a migration on our search indices. The migration rendered existing indices unavailable, causing search functionality to return errors or no results across all customer sites. The incident was resolved by failing over to backup indices.
Scope: All Mintlify sites with search functionality
User experience: Search queries returned errors or empty results for the duration of the outage
Duration of impact: ~42 minutes
Our upstream search provider initiated a migration of our search indices without adequate coordination. This migration caused the primary indices to become unavailable, which in turn broke search queries across all Mintlify sites.
Within approximately 10 minutes of detecting the incident, the team initiated a failover to pre-existing backup indices. Once the migration to backup indices completed, full search functionality was restored.
Resolved
A search outage affected all Mintlify-powered sites after our upstream search provider executed a migration on our search indices. The migration rendered existing indices unavailable, causing search functionality to return errors or no results across all customer sites. The incident was resolved by failing over to backup indices.
Scope: All Mintlify sites with search functionality
User experience: Search queries returned errors or empty results for the duration of the outage
Duration of impact: ~42 minutes
Our upstream search provider initiated a migration of our search indices without adequate coordination. This migration caused the primary indices to become unavailable, which in turn broke search queries across all Mintlify sites.
Within approximately 10 minutes of detecting the incident, the team initiated a failover to pre-existing backup indices. Once the migration to backup indices completed, full search functionality was restored.
Monitoring
We have implemented a system wide fix and search recovery should be in progress for all sites. We are monitoring to make sure this is the case.
Identified
We have identified the issue with an upstream provider and are working to rollout a fix to the system.
Investigating
We are seeing errors with our search database and working to investigate and understand the cause to restore functionality.