⚡ Sangsakala Proxy 🏠 Home
Skip to content

Ray - Add Support for Azure Blob Storage / ADLS #5844

Closed

Description

@jbauer12

Is your feature request related to a problem? Please describe.
There is a lack of integration with Azure Blob Storage, though GCS and S3 are well supported.

Describe the solution you'd like
Azure Blob Storage is supported in the same way as GCS and S3 for the Ray Offline Store.

Describe alternatives you've considered
It's difficult to find an offline store with Azure Blob Storage integration. Perhaps the Spark Offline Store could be used as well somehow.

Additional context

Thank you!

Activity

  1. jbauer12 commented on Jan 26, 2026

    @jbauer12
    ContributorAuthor

    The first tests worked by installing adlfs pip package and setting

    pip install adlfs
    export AZURE_STORAGE_ANON=False

    (so that it uses DefaultAzureCredential instead of anonymous) and defining the file sources as follows (using DFS endpoints):

    customer_daily_profile = FileSource(
        name="customer_daily_profile",
        path="abfs://xxx@xxxx.dfs.core.windows.net/xxx/customer_daily_profile.parquet",
        timestamp_field="event_timestamp",
        created_timestamp_column="created",
    )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions