filecollections

« Previous section Next section »

UCloud Developer Guide / Orchestration of Resources / Storage / Drives (FileCollection)

Drives (FileCollection)

A FileCollection is an entrypoint to a user's files

Rationale

📝 NOTE: This API follows the standard Resources API. We recommend that you have already read and understood the concepts described here.


This entrypoint allows the user to access all the files they have access to within a single project. It is important to note that a file collection is not the same as a directory! Common real-world examples of a file collection is listed below:

NameTypical pathComment

Home directory

/home/$username/

The home folder is typically the main collection for a user

Work directory

/work/$projectId/

The project 'home' folder

Scratch directory

/scratch/$projectId/

Temporary storage for a project

The provider of storage manages a 'database' of these file collections and who they belong to. The file collections also play an important role in accounting and resource management. A file collection can have a quota attached to it and billing information is also stored in this object. Each file collection can be attached to a different product type, and as a result, can have different billing information attached to it. This is, for example, useful if a storage provider has both fast and slow tiers of storage, which is typically billed very differently.

All file collections additionally have a title. This title can be used for a user-friendly version of the folder. This title does not have to be unique, and can with great benefit choose to not reference who it belongs to. For example, if every user has exactly one home directory, then it would make sense to give this collection "Home" as its title.


📝 Provider Note: This is the API exposed to end-users. See the table below for other relevant APIs.

End-UserProvider (Ingoing)Control (Outgoing)

Table of Contents

1. Examples

2. Remote Procedure Calls

3. Data Models

Example: An example collection

Frequency of useCommon

Actors

  • An authenticated user (user)

Communication Flow: Kotlin

/* In this example we will see a simple collection. This collection models the 'home' directory of a user. */


/* 📝 NOTE: Collections are identified by a unique (UCloud provided) ID */

FileCollections.retrieve.call(
    ResourceRetrieveRequest(
        flags = FileCollectionIncludeFlags(
            filterCreatedAfter = null, 
            filterCreatedBefore = null, 
            filterCreatedBy = null, 
            filterIds = null, 
            filterMemberFiles = null, 
            filterProductCategory = null, 
            filterProductId = null, 
            filterProvider = null, 
            filterProviderIds = null, 
            hideProductCategory = null, 
            hideProductId = null, 
            hideProvider = null, 
            includeOthers = false, 
            includeProduct = false, 
            includeSupport = false, 
            includeUpdates = false, 
        ), 
        id = "54123", 
    ),
    user
).orThrow()

/*
FileCollection(
    createdAt = 1635151675465, 
    id = "54123", 
    owner = ResourceOwner(
        createdBy = "user", 
        project = null, 
    ), 
    permissions = ResourcePermissions(
        myself = listOf(Permission.ADMIN), 
        others = emptyList(), 
    ), 
    providerGeneratedId = null, 
    specification = FileCollection.Spec(
        product = ProductReference(
            category = "example-ssd", 
            id = "example-ssd", 
            provider = "example", 
        ), 
        title = "Home", 
    ), 
    status = FileCollection.Status(
        resolvedProduct = null, 
        resolvedSupport = null, 
    ), 
    updates = emptyList(), 
)
*/
Communication Flow: Curl
# ------------------------------------------------------------------------------------------------------
# $host is the UCloud instance to contact. Example: 'http://localhost:8080' or 'https://cloud.sdu.dk'
# $accessToken is a valid access-token issued by UCloud
# ------------------------------------------------------------------------------------------------------

# In this example we will see a simple collection. This collection models the 'home' directory of a user.

# 📝 NOTE: Collections are identified by a unique (UCloud provided) ID

# Authenticated as user
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/retrieve?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false&id=54123" 

# {
#     "id": "54123",
#     "specification": {
#         "title": "Home",
#         "product": {
#             "id": "example-ssd",
#             "category": "example-ssd",
#             "provider": "example"
#         }
#     },
#     "createdAt": 1635151675465,
#     "status": {
#         "resolvedSupport": null,
#         "resolvedProduct": null
#     },
#     "updates": [
#     ],
#     "owner": {
#         "createdBy": "user",
#         "project": null
#     },
#     "permissions": {
#         "myself": [
#             "ADMIN"
#         ],
#         "others": [
#         ]
#     },
#     "providerGeneratedId": null
# }
Communication Flow: Visual

Example: Renaming a collection

Frequency of useCommon

Actors

  • An authenticated user (user)

Communication Flow: Kotlin

/* In this example, we will see how a user can rename one of their collections. */


/* 📝 NOTE: Renaming must be supported by the provider */

FileCollections.retrieveProducts.call(
    Unit,
    user
).orThrow()

/*
SupportByProvider(
    productsByProvider = mapOf("ucloud" to listOf(ResolvedSupport(
        product = Product.Storage(
            allowAllocationRequestsFrom = AllocationRequestsGroup.ALL, 
            category = ProductCategoryId(
                id = "example-ssd", 
                name = "example-ssd", 
                provider = "example", 
            ), 
            chargeType = ChargeType.DIFFERENTIAL_QUOTA, 
            description = "Fast storage", 
            freeToUse = false, 
            hiddenInGrantApplications = false, 
            name = "example-ssd", 
            pricePerUnit = 1, 
            priority = 0, 
            productType = ProductType.STORAGE, 
            unitOfPrice = ProductPriceUnit.PER_UNIT, 
            version = 1, 
            balance = null, 
            id = "example-ssd", 
            maxUsableBalance = null, 
        ), 
        support = FSSupport(
            collection = FSCollectionSupport(
                aclModifiable = null, 
                usersCanCreate = true, 
                usersCanDelete = true, 
                usersCanRename = true, 
            ), 
            files = FSFileSupport(
                aclModifiable = false, 
                isReadOnly = false, 
                searchSupported = true, 
                sharesSupported = false, 
                streamingSearchSupported = false, 
                trashSupported = false, 
            ), 
            maintenance = null, 
            product = ProductReference(
                category = "example-ssd", 
                id = "example-ssd", 
                provider = "example", 
            ), 
            stats = FSProductStatsSupport(
                accessedAt = null, 
                createdAt = null, 
                modifiedAt = null, 
                sizeInBytes = null, 
                sizeIncludingChildrenInBytes = null, 
                unixGroup = null, 
                unixOwner = null, 
                unixPermissions = null, 
            ), 
        ), 
    ))), 
)
*/

/* As we can see, the provider does support the rename operation. We now look at our collections. */

FileCollections.browse.call(
    ResourceBrowseRequest(
        consistency = null, 
        flags = FileCollectionIncludeFlags(
            filterCreatedAfter = null, 
            filterCreatedBefore = null, 
            filterCreatedBy = null, 
            filterIds = null, 
            filterMemberFiles = null, 
            filterProductCategory = null, 
            filterProductId = null, 
            filterProvider = null, 
            filterProviderIds = null, 
            hideProductCategory = null, 
            hideProductId = null, 
            hideProvider = null, 
            includeOthers = false, 
            includeProduct = false, 
            includeSupport = false, 
            includeUpdates = false, 
        ), 
        itemsPerPage = null, 
        itemsToSkip = null, 
        next = null, 
        sortBy = null, 
        sortDirection = null, 
    ),
    user
).orThrow()

/*
PageV2(
    items = listOf(FileCollection(
        createdAt = 1635151675465, 
        id = "54123", 
        owner = ResourceOwner(
            createdBy = "user", 
            project = null, 
        ), 
        permissions = ResourcePermissions(
            myself = listOf(Permission.ADMIN), 
            others = emptyList(), 
        ), 
        providerGeneratedId = null, 
        specification = FileCollection.Spec(
            product = ProductReference(
                category = "example-ssd", 
                id = "example-ssd", 
                provider = "example", 
            ), 
            title = "Home", 
        ), 
        status = FileCollection.Status(
            resolvedProduct = null, 
            resolvedSupport = null, 
        ), 
        updates = emptyList(), 
    )), 
    itemsPerPage = 50, 
    next = null, 
)
*/

/* Using the unique ID, we can rename the collection */

FileCollections.rename.call(
    bulkRequestOf(FileCollectionsRenameRequestItem(
        id = "54123", 
        newTitle = "My Awesome Drive", 
    )),
    user
).orThrow()

/*
Unit
*/

/* The new title is observed when we browse the collections one more time */

FileCollections.browse.call(
    ResourceBrowseRequest(
        consistency = null, 
        flags = FileCollectionIncludeFlags(
            filterCreatedAfter = null, 
            filterCreatedBefore = null, 
            filterCreatedBy = null, 
            filterIds = null, 
            filterMemberFiles = null, 
            filterProductCategory = null, 
            filterProductId = null, 
            filterProvider = null, 
            filterProviderIds = null, 
            hideProductCategory = null, 
            hideProductId = null, 
            hideProvider = null, 
            includeOthers = false, 
            includeProduct = false, 
            includeSupport = false, 
            includeUpdates = false, 
        ), 
        itemsPerPage = null, 
        itemsToSkip = null, 
        next = null, 
        sortBy = null, 
        sortDirection = null, 
    ),
    user
).orThrow()

/*
PageV2(
    items = listOf(FileCollection(
        createdAt = 1635151675465, 
        id = "54123", 
        owner = ResourceOwner(
            createdBy = "user", 
            project = null, 
        ), 
        permissions = ResourcePermissions(
            myself = listOf(Permission.ADMIN), 
            others = emptyList(), 
        ), 
        providerGeneratedId = null, 
        specification = FileCollection.Spec(
            product = ProductReference(
                category = "example-ssd", 
                id = "example-ssd", 
                provider = "example", 
            ), 
            title = "My Awesome Drive", 
        ), 
        status = FileCollection.Status(
            resolvedProduct = null, 
            resolvedSupport = null, 
        ), 
        updates = emptyList(), 
    )), 
    itemsPerPage = 50, 
    next = null, 
)
*/
Communication Flow: Curl
# ------------------------------------------------------------------------------------------------------
# $host is the UCloud instance to contact. Example: 'http://localhost:8080' or 'https://cloud.sdu.dk'
# $accessToken is a valid access-token issued by UCloud
# ------------------------------------------------------------------------------------------------------

# In this example, we will see how a user can rename one of their collections.

# 📝 NOTE: Renaming must be supported by the provider

# Authenticated as user
curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/retrieveProducts" 

# {
#     "productsByProvider": {
#         "ucloud": [
#             {
#                 "product": {
#                     "balance": null,
#                     "maxUsableBalance": null,
#                     "name": "example-ssd",
#                     "pricePerUnit": 1,
#                     "category": {
#                         "name": "example-ssd",
#                         "provider": "example"
#                     },
#                     "description": "Fast storage",
#                     "priority": 0,
#                     "version": 1,
#                     "freeToUse": false,
#                     "allowAllocationRequestsFrom": "ALL",
#                     "unitOfPrice": "PER_UNIT",
#                     "chargeType": "DIFFERENTIAL_QUOTA",
#                     "hiddenInGrantApplications": false,
#                     "productType": "STORAGE"
#                 },
#                 "support": {
#                     "product": {
#                         "id": "example-ssd",
#                         "category": "example-ssd",
#                         "provider": "example"
#                     },
#                     "stats": {
#                         "sizeInBytes": null,
#                         "sizeIncludingChildrenInBytes": null,
#                         "modifiedAt": null,
#                         "createdAt": null,
#                         "accessedAt": null,
#                         "unixPermissions": null,
#                         "unixOwner": null,
#                         "unixGroup": null
#                     },
#                     "collection": {
#                         "aclModifiable": null,
#                         "usersCanCreate": true,
#                         "usersCanDelete": true,
#                         "usersCanRename": true
#                     },
#                     "files": {
#                         "aclModifiable": false,
#                         "trashSupported": false,
#                         "isReadOnly": false,
#                         "searchSupported": true,
#                         "streamingSearchSupported": false,
#                         "sharesSupported": false
#                     },
#                     "maintenance": null
#                 }
#             }
#         ]
#     }
# }

# As we can see, the provider does support the rename operation. We now look at our collections.

curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/browse?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false" 

# {
#     "itemsPerPage": 50,
#     "items": [
#         {
#             "id": "54123",
#             "specification": {
#                 "title": "Home",
#                 "product": {
#                     "id": "example-ssd",
#                     "category": "example-ssd",
#                     "provider": "example"
#                 }
#             },
#             "createdAt": 1635151675465,
#             "status": {
#                 "resolvedSupport": null,
#                 "resolvedProduct": null
#             },
#             "updates": [
#             ],
#             "owner": {
#                 "createdBy": "user",
#                 "project": null
#             },
#             "permissions": {
#                 "myself": [
#                     "ADMIN"
#                 ],
#                 "others": [
#                 ]
#             },
#             "providerGeneratedId": null
#         }
#     ],
#     "next": null
# }

# Using the unique ID, we can rename the collection

curl -XPOST -H "Authorization: Bearer $accessToken" -H "Content-Type: content-type: application/json; charset=utf-8" "$host/api/files/collections/rename" -d '{
    "items": [
        {
            "id": "54123",
            "newTitle": "My Awesome Drive"
        }
    ]
}'


# {
# }

# The new title is observed when we browse the collections one more time

curl -XGET -H "Authorization: Bearer $accessToken" "$host/api/files/collections/browse?includeOthers=false&includeUpdates=false&includeSupport=false&includeProduct=false" 

# {
#     "itemsPerPage": 50,
#     "items": [
#         {
#             "id": "54123",
#             "specification": {
#                 "title": "My Awesome Drive",
#                 "product": {
#                     "id": "example-ssd",
#                     "category": "example-ssd",
#                     "provider": "example"
#                 }
#             },
#             "createdAt": 1635151675465,
#             "status": {
#                 "resolvedSupport": null,
#                 "resolvedProduct": null
#             },
#             "updates": [
#             ],
#             "owner": {
#                 "createdBy": "user",
#                 "project": null
#             },
#             "permissions": {
#                 "myself": [
#                     "ADMIN"
#                 ],
#                 "others": [
#                 ]
#             },
#             "providerGeneratedId": null
#         }
#     ],
#     "next": null
# }
Communication Flow: Visual

Remote Procedure Calls

browse

Browses the catalog of available resources

retrieve

Retrieve a single resource

retrieveProducts

Retrieve product support for all accessible providers

This endpoint will determine all providers that which the authenticated user has access to, in the current workspace. A user has access to a product, and thus a provider, if the product is either free or if the user has been granted credits to use the product.

See also:

Searches the catalog of available resources

create

Creates one or more resources

delete

Deletes one or more resources

init

Request (potential) initialization of resources

RequestResponseError

This request is sent by the client, if the client believes that initialization of resources might be needed. NOTE: This request might be sent even if initialization has already taken place. UCloud/Core does not check if initialization has already taken place, it simply validates the request.

rename

Request renaming of FileCollection

updateAcl

Updates the ACL attached to a resource

Data Models

FSCollectionSupport

Declares which FileCollection operations are supported for a product

data class FSCollectionSupport(
    val aclModifiable: Boolean?,
    val usersCanCreate: Boolean?,
    val usersCanDelete: Boolean?,
    val usersCanRename: Boolean?,
)
Properties


FSFileSupport

Declares which file-level operations a product supports

data class FSFileSupport(
    val aclModifiable: Boolean?,
    val trashSupported: Boolean?,
    val isReadOnly: Boolean?,
    val searchSupported: Boolean?,
    val streamingSearchSupported: Boolean?,
    val sharesSupported: Boolean?,
)
Properties


FSProductStatsSupport

Declares which stats a given product supports

data class FSProductStatsSupport(
    val sizeInBytes: Boolean?,
    val sizeIncludingChildrenInBytes: Boolean?,
    val modifiedAt: Boolean?,
    val createdAt: Boolean?,
    val accessedAt: Boolean?,
    val unixPermissions: Boolean?,
    val unixOwner: Boolean?,
    val unixGroup: Boolean?,
)
Properties


FSSupport

data class FSSupport(
    val product: ProductReference,
    val stats: FSProductStatsSupport?,
    val collection: FSCollectionSupport?,
    val files: FSFileSupport?,
    val maintenance: Maintenance?,
)
Properties


FileCollection

A Resource is the core data model used to synchronize tasks between UCloud and Provider.

data class FileCollection(
    val id: String,
    val specification: FileCollection.Spec,
    val createdAt: Long,
    val status: FileCollection.Status,
    val updates: List<FileCollection.Update>,
    val owner: ResourceOwner,
    val permissions: ResourcePermissions?,
    val providerGeneratedId: String?,
)

For more information go here.

Properties


FileCollection.Spec

data class Spec(
    val title: String,
    val product: ProductReference,
)
Properties


FileCollection.Status

Describes the current state of the Resource

data class Status(
    val resolvedSupport: ResolvedSupport<Product.Storage, FSSupport>?,
    val resolvedProduct: Product.Storage?,
)

The contents of this field depends almost entirely on the specific Resource that this field is managing. Typically, this will contain information such as:

  • A state value. For example, a compute Job might be RUNNING

  • Key metrics about the resource.

  • Related resources. For example, certain Resources are bound to another Resource in a mutually exclusive way, this should be listed in the status section.

Properties


FileCollection.Update

Describes an update to the Resource

data class Update(
    val timestamp: Long,
    val status: String?,
)

Updates can optionally be fetched for a Resource. The updates describe how the Resource changes state over time. The current state of a Resource can typically be read from its status field. Thus, it is typically not needed to use the full update history if you only wish to know the current state of a Resource.

An update will typically contain information similar to the status field, for example:

  • A state value. For example, a compute Job might be RUNNING.

  • Change in key metrics.

  • Bindings to related Resources.

Properties


FileCollectionIncludeFlags

data class FileCollectionIncludeFlags(
    val filterMemberFiles: MemberFilesFilter?,
    val includeOthers: Boolean?,
    val includeUpdates: Boolean?,
    val includeSupport: Boolean?,
    val includeProduct: Boolean?,
    val filterCreatedBy: String?,
    val filterCreatedAfter: Long?,
    val filterCreatedBefore: Long?,
    val filterProvider: String?,
    val filterProductId: String?,
    val filterProductCategory: String?,
    val filterProviderIds: String?,
    val filterIds: String?,
    val hideProductId: String?,
    val hideProductCategory: String?,
    val hideProvider: String?,
)
Properties


MemberFilesFilter

Filter for member files.

enum class MemberFilesFilter {
    SHOW_ONLY_MINE,
    SHOW_ONLY_MEMBER_FILES,
    DONT_FILTER_COLLECTIONS,
}

A member files collection must use the following format to be recognized: "Member Files: $username"

Properties


FileCollectionsRenameRequestItem

data class FileCollectionsRenameRequestItem(
    val id: String,
    val newTitle: String,
)
Properties


Last updated