Introduction
Staying informed in the crypto market is essential, and having quick access to relevant information is crucial for making trading decisions. At Infiniteblocks Savvy, we host a vast collection of over 4,000 cryptocurrency news Snippets, with new ones added daily. These Snippets provide valuable information to help our users navigate the complexities of crypto trading.
To make this information easily accessible, we developed a powerful search application using Vertex AI Agent Builder, enabling users to search efficiently through our extensive database of Snippets.
About the Application
Our search application utilizes Vertex AI Search, a core component of Vertex AI Agent Builder. This tool provides advanced search capabilities using natural language processing, result filtering, highlighting, and ranking adjustments. It connects to a Datastore, which acts as a repository for our Snippets data.
Application Architecture
The application consists of two primary Cloud Functions:
- Search Engine Serving: Handles user queries and retrieves relevant search results.
- Refreshing Datastore Documents: Updates the Datastore daily with the latest Snippets.
Breaking Down Implementation
1. Firestore Database
Firestore serves as the database for storing Snippets. Cloud Scheduler triggers a workflow that retrieves and processes the latest Snippets into Firestore.
2. Datastores
Datastores are structured repositories used by Vertex AI Search applications. Data from Firestore is staged in a Cloud Storage Bucket and imported into the Datastore for efficient querying.
3. Vertex AI Search App
The Vertex AI Search App leverages Google's AI capabilities to process raw data into structured formats, generate semantic embeddings, and provide accurate search results.
Challenges
Building the Vertex AI Search application was a rewarding experience, but it came with challenges, including:
- In-preview features: Some functionalities were unstable, requiring troubleshooting.
- Documentation issues: Frequent platform updates led to inconsistent documentation.
- Lack of resources: Limited examples and community support made troubleshooting challenging.
Conclusion
Using Vertex AI Search, we created a search engine that enhances user experience and makes information retrieval efficient. This project was not only a valuable learning experience but also a significant step forward in leveraging AI to solve real-world problems.