650 Concurrent Requests, 250 Million+ Records - Big Data Search in Milliseconds
A leading global brand protection company specializing in international domain registration services wanted to build a single function search that runs on 250 million+ records. They needed to search domain names from the list and see if any matching domain names have cropped up. Knowing this will help them identify potential risks or frauds for customers that have registered with them.
We built a big data search engine that searches through the entire World’s domain names (250+ million records) and returns results within milliseconds.
- Nightly Data fetch to pull entire world’s domain names
- Storing strategy for this data so that it’s “searchable”
- API endpoint that searches domain names at zuper speed (650 concurrent requests returning 500 domain names each within 1 second)
- Hosted on Azure using Redis for awesome performance
Azure Pipeline, Azure DevOps, .NET Core, Cosmos DB, Azure Search Service, Azure Redis Cache, Azure SQL, Azure DevOps
- Quick and timely identification of domain name frauds
- Identifying potential risks and frauds due to domain name duplication or fraudulent domain names