Working with object storage

Review MinIO

As we will see later, we will need object storage to store the data used for training the models and for saving them afterward. As we saw at the beginning, MinIO is already configured on our node and should be accesed through:

https://minio-ui-minio.apps.cluster-79242.dynamic.redhatworkshops.io

You will need to use the following credentials to log into the MinIO dashboard:

  • User: minio

  • Password: minio123

In the Object Browser landing page you will spot two different buckets: the one used for model training/saving and the one used for storing the pipelines artifacts.

MinIO Buckets

Crate DataConnection

At this point, we can use the s3 bucket to store data. In order to make it accessible from our Jupyter environment, we need to create a DataConnection in our node, indicating the set of s3 configuration values.

Red Hat OpenShift AI creates DataConnection resources as OpenShift secrets. Those secrets store all the configuration values that facilitate connecting datascience projects to object storage buckets. Those buckets can contain either the dataset used for the training or the AI models to be served. Let’s configure the DataConnection.

  1. Open again the OpenShift AI web console and navigate to the Connections tab. There, select Create connection.

  2. In the new pop-up window, complete the following fields:

    • Connection type: Select S3 compatible object storage - v1

    • Connection Name: Name for the secret. I’m using s3-storage.

    • Access key: Username defined when deploying MinIO. Normally, minio.

    • Secret key: Add the MinIO password. In my case, I used minio123.

    • Endpoint: You can get the API endpoint URL from the MinIo service in OCP. Mine is http://minio-service.minio.svc.cluster.local:9000.

    • Region: It doesn’t really matter here. The default value is us-east-1.

    • Bucket: Indicate the MinIO bucket name created before. As we saw, I created the s3-storage folder to store my trained model.

  3. When completed, select Create.

A new Secret called aws-connection-s3-storage will be created in your ai-edge-project namespace containing all the specified values stored in base64 format.

DataConnection