Torch Object Storage#

To handle who can access Datasets, the necessary IAM resources/policies must be created. This includes

A TOS Bucket
An IAM Policy
Linking Users to Bucket with a IAM Policy

Setting up the bucket#

For instructions on creating TOS Buckets for Volce Engine, go to TOS Bucket and create a bucket.

Create IAM Policy#

Go to the IAM Policy Management page and create a custom policy.

Select the JSON Editor tab and enter the information below. Remember to replace bucket_name with the created bucket name from above:

{
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "tos:PutObject",
              "tos:GetObject",
              "tos:ListBucket",
              "tos:DeleteObject",
              "tos:GetBucketLocation"
          ],
          "Resource": [
              "trn:tos:::<bucket_name>",
              "trn:tos:::<bucket_name>/*"
          ]
      }
  ]
}

Link User and Bucket with IAM Policy Permissions#

Once the IAM Policy is created, you can add any created user or user groups to this policy.

To do so, go to the IAM Policy Management page and select the created IAM Policy from the previous step.

Select the authorization tab and select the desired users and groups.

Construct URI#

URIs are constructed as follows with examples for bucket name my_bucket and endpoint tos-s3-region.my_endpoint.com:

tos://<endpoint><bucket>

# Example
tos://tos-s3-region.my_endpoint.com/my_bucket

Follow Configure Data Storage to add the bucket to OSMO.

Note

The endpoint chosen should be the s3 endpoint which starts with toc-s3.