Replicate text2video

This handler was implemented using the replicate library that is provided by Replicate.

The required arguments to establish a connection are,

• model_name: Model name which you want to access in MindsDB. e.g ‘air-forever/kandinsky-2’
• version: version hash/id which you want to use in MindsDB.
• api_key: API key from Replicate Platform you can found here.

Before you can use Replicate, it’s essential to authenticate by setting your API token in an environment variable named REPLICATE_API_TOKEN. This token acts as a key to enable access to Replicate’s features.

1. Using pip:

If you’re working in a standard Python environment (using pip for package management), set your token as an environment variable by running the following command in your terminal:

On Linux, Mac:

export REPLICATE_API_TOKEN='YOUR_TOKEN'

On Windows:

set REPLICATE_API_TOKEN=YOUR_TOKEN

2. Using Docker:

For Docker users, the process slightly differs. You need to pass the environment variable directly to the Docker container when running it. Use this command:

docker run -e REPLICATE_API_TOKEN='YOUR_TOKEN' -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

Again, replace ‘YOUR_TOKEN’ with your actual Replicate API token.

Usage

To use this handler and connect to a Replicate cluster in MindsDB, you need an account on Replicate. Make sure to create an account by following this link.

To establish the connection and create a model in MindsDB, use the following syntax:

CREATE MODEL video_ai

PREDICT output

USING

    engine = 'replicate',

    model_name= 'deforum/deforum_stable_diffusion',

    version ='e22e77495f2fb83c34d5fae2ad8ab63c0a87b6b573b6208e1535b23b89ea66d6',

    api_key = 'r8_HEH............';

You can use the DESCRIBE PREDICTOR query to see the available parameters that you can specify to customize your predictions:

DESCRIBE PREDICTOR mindsdb.video_ai.features;

OUTPUT

+-------------------+---------+-----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| inputs            | type    | default                                                               | description                                                                                                                                                          |

+-------------------+---------+-----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| fps               | integer | 15                                                                    | Choose fps for the video.                                                                                                                                            |

| seed              | integer | -                                                                     | Random seed. Leave blank to randomize the seed                                                                                                                       |

| zoom              | string  | 0: (1.04)                                                             | zoom parameter for the motion                                                                                                                                        |

| angle             | string  | 0:(0)                                                                 | angle parameter for the motion                                                                                                                                       |

| sampler           | -       | plms                                                                  | -                                                                                                                                                                    |

| max_frames        | integer | 100                                                                   | Number of frames for animation                                                                                                                                       |

| translation_x     | string  | 0: (0)                                                                | translation_x parameter for the motion                                                                                                                               |

| translation_y     | string  | 0: (0)                                                                | translation_y parameter for the motion                                                                                                                               |

| color_coherence   | -       | Match Frame 0 LAB                                                     | -                                                                                                                                                                    |

| animation_prompts | string  | 0: a beautiful portrait of a woman by Artgerm, trending on Artstation | Prompt for animation. Provide 'frame number : prompt at this frame', separate different prompts with '|'. Make sure the frame number does not exceed the max_frames. |

+-------------------+---------+-----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Now, you can use the established connection to query your ML Model as follows:

SELECT *

FROM video_ai

WHERE animation_prompts='a human and animals are friends by Asher Brown Durand, trending on Artstation'

USING

 max_frames=119;

OUTPUT

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+

| output                                                                                                                                                                                                                                                                                                                    | prompt                                 |

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+

| https://replicate.delivery/pbxt/gSgRjNlxIgJWBB8KeebKeRmBjZx0wqX7JC41U0pvIfPCYVzEB/out.mp4 | a human and animals are friends by Asher Brown Durand, trending on Artstation |

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+

Checkout generated video here as above link will not work, reason is given below.

• IMPORTANT NOTE: PREDICTED URL will only work for 24 hours after prediction.

Note: Replicate provides only a few free predictions, so choose your predictions wisely. Don’t let the machines have all the fun, save some for yourself! 😉