This handler was implemented using the replicate library that is provided by Replicate.The required arguments to establish a connection are,
model_name: Model name which you want to access in MindsDB. e.g ‘air-forever/kandinsky-2’
version: version hash/id which you want to use in MindsDB.
api_key: API key from Replicate Platform you can found here.
Before you can use Replicate, it’s essential to authenticate by setting your API token in an environment variable named REPLICATE_API_TOKEN.
This token acts as a key to enable access to Replicate’s features.
Using pip:
If you’re working in a standard Python environment (using pip for package management), set your token as an environment variable by running the following command
in your terminal:On Linux, Mac:
Copy
Ask AI
export REPLICATE_API_TOKEN='YOUR_TOKEN'
On Windows:
Copy
Ask AI
set REPLICATE_API_TOKEN=YOUR_TOKEN
Using Docker:
For Docker users, the process slightly differs. You need to pass the environment variable directly to the Docker container when running it. Use this command:
Copy
Ask AI
docker run -e REPLICATE_API_TOKEN='YOUR_TOKEN' -p 47334:47334 -p 47335:47335 mindsdb/mindsdb
Again, replace ‘YOUR_TOKEN’ with your actual Replicate API token.
To use this handler and connect to a Replicate cluster in MindsDB, you need an account on Replicate. Make sure to create an account by following this link.To establish the connection and create a model in MindsDB, use the following syntax:
Copy
Ask AI
CREATE MODEL blipPREDICT textUSING engine = 'replicate', model_name= 'salesforce/blip', version ='2e1dddc8621f72155f24cf2e0adbde548458d3cab9f00c0139eea840d0ac4746', api_key = 'r8_BpO.........................';
You can use the DESCRIBE PREDICTOR query to see the available parameters that you can specify to customize your predictions:
Now, you can use the established connection to query your ML Model as follows:
Copy
Ask AI
SELECT *FROM mindsdb.blipWHERE image="https://images.unsplash.com/photo-1686847266385-a32745169de4"AND question="Is there lion in image?"USINGtask="visual_question_answering";
+------------+--------------------------------------------------------------+-------------------------+| text | image | question |+------------+--------------------------------------------------------------+-------------------------+| Answer: no | https://images.unsplash.com/photo-1686847266385-a32745169de4 | Is there lion in image? |+------------+--------------------------------------------------------------+-------------------------+
+---------------------------------------------------+--------------------------------------------------------------+| text | image |+---------------------------------------------------+--------------------------------------------------------------+| Caption: a bird is sitting on the back of a horse | https://images.unsplash.com/photo-1686847266385-a32745169de4 |+---------------------------------------------------+--------------------------------------------------------------+
Image Text Matching
Copy
Ask AI
SELECT *FROM mindsdb.blipWHERE image="https://images.unsplash.com/photo-1686847266385-a32745169de4"AND caption="Bird having horse Riding"USINGtask="image_text_matching";
OUTPUT
Copy
Ask AI
+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+| text | image | caption |+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+| The image and text is matched with a probability of 0.7730.The image feature and text feature has a cosine similarity of 0.3615. | https://images.unsplash.com/photo-1686847266385-a32745169de4 | Bird having horse Riding |+-----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------+
This is just an one model used in this example there are more with vast variation and usescases.
Also there is no limit to imagination, how can you use this.Note: Replicate provides only a few free predictions, so choose your predictions wisely. Don’t let the machines have all the fun, save some for yourself! 😉