Raw Storage

Raw Storage Key Features

• Upload Files: Easily upload files to the Raw Storage for persistent storage and retrieval.

• Download Files: Retrieve files from the Raw Storage for local use or processing.

• Delete Files: Manage file storage by deleting unwanted or obsolete files.

• Search Capabilities: Leverage search functionalities to find specific files based on criteria such as filename, metadata, or tags.

Examples

Functions

list

List all files in a dataset.

Arguments

• resource_dto (DatasetDto): Dataset resource to list files from.
• metadata_filter (Optional[dict]): Metadata filter.

Yields

• File metadata for each file in the dataset.

Example

from odp.client import OdpClient
from odp.dto import Metadata
from odp.dto.catalog import DatasetDto, DatasetSpec
from odp.dto.common.contact_info import ContactInfo    
client = OdpClient()

# DatasetDto
my_dataset = {...}
my_filter = {...}

for file_metadata in client.raw.list(my_dataset, my_filter):
    print(file_metadata)

create_file

Create and upload a file to a dataset.

Arguments

• resource_dto (DatasetDto): Dataset resource to associate the file with.
• file_metadata_dto (FileMetadataDto): Metadata for the file, including name and MIME type.
• contents (bytes): Contents of the file.

Returns

• The metadata of the created file.

Example

from odp.client import OdpClient
from odp.dto import Metadata
from odp.dto.catalog import DatasetDto, DatasetSpec
from odp.dto.common.contact_info import ContactInfo
from odp.client.dto.file_dto import FileMetadataDto

client = OdpClient()

# DatasetDto
my_dataset = {...}

# FileMetadataDto
file_metadata_dto = {...}

# File contents
contents = b"Hello, World!"

try:
    file_metadata = client.raw.create_file(resource_dto=my_dataset, file_metadata_dto=file_metadata_dto, contents=contents)
    print("File created successfully:", file_metadata)
except OdpValidationError as e:
    print("Validation Error:", e)
except OdpResourceNotFoundError as e:
    print("Resource Not Found Error:", e)

download_file

Download a file from a dataset.

Arguments

• resource_dto (DatasetDto): Dataset resource containing the file.
• file_metadata_dto (FileMetadataDto): Metadata of the file to download.
• destination_path (str): Local path to save the downloaded file.

Example

from odp.client import OdpClient
from odp.dto import Metadata
from odp.dto.catalog import DatasetDto, DatasetSpec
from odp.dto.common.contact_info import ContactInfo
from odp.client.dto.file_dto import FileMetadataDto

client = OdpClient()

# DatasetDto
my_dataset = {...}

# FileMetadataDto
file_metadata_dto = {...}

# Destination_path
destination_path = "path/to/save/file.txt"

try:
    client.raw.download_file(my_dataset, file_metadata_dto, destination_path)
    print("File downloaded successfully.")
except OdpValidationError as e:
    print("Validation Error:", e)
except OdpResourceNotFoundError as e:
    print("Resource Not Found Error:", e)

delete_file

Delete a file from a dataset.

Arguments

• resource_dto (DatasetDto): Dataset resource containing the file.
• file_metadata_dto (FileMetadataDto): Metadata of the file to delete.

Raises

• OdpResourceNotFoundError: If the file or dataset is not found.

Example

from odp.client import OdpClient
from odp.dto import Metadata
from odp.dto.catalog import DatasetDto, DatasetSpec
from odp.dto.common.contact_info import ContactInfo
from odp.client.dto.file_dto import FileMetadataDto

client = OdpClient()

# DatasetDto
my_dataset = {...}

# FileMetadataDto
file_metadata_dto = {...}

try:
    client.raw.delete_file(my_dataset, file_metadata_dto)
    print("File deleted successfully.")
except OdpResourceNotFoundError as e:
    print("Resource Not Found Error:", e)