Skip to content

add overriding truncated_namespace condition for truncating paths for digital ocean

Foggy requested to merge foggy1/pleroma:features/s3-uploader-digital-ocean into develop

Digital Ocean Spaces (their file store) supports the S3 api, but it has an asymmetrical opinion about bucket names. Consider the following config:

config :pleroma, Pleroma.Uploaders.S3,
  bucket: "space_name/folder_name",
  public_endpoint: "htttps://space_name.nyc3.digitaloceanspaces.com"

(Note: s3 creds via ex_aws are configured as normal, albeit with DO's region naming scheme.)

In this case, if I have a Digital Ocean Space called space_name, it lives at the above public endpoint. Spaces are sort of like buckets, but since each one on its own costs money you can't go horizontal with the buckets quite like you can on S3. For this reason, you're much more likely to always use a folder, as above.

With the above config, Pleroma.Uploaders.S3.put_item/1 will successfully save my images to folder_name in the proper space. However, Pleroma.Uploaders.S3.get_item/1 uses the following code to try and get the url:

{:ok,
 {:url,
  Path.join([
    Keyword.fetch!(config,:public_endpoint),
    bucket_with_namespace,
    strict_encode(URI.decode(file))
])}}

If no :namespace opt is passed, the url is thus htttps://space_name.nyc3.digitaloceanspaces.com/space_name/folder_name/#{filename}. If :bucket_namespace opt is passed, it would be bucket_namespace:space_name/....

Yet for Digital Ocean, the proper URI for accessing filename in folder_name is htttps://space_name.nyc3.digitaloceanspaces.com/folder_name/#{filename}. The prefix of space_name on the public endpoint is all Digital Ocean accepts for accessing this particular "space", despite the fact that it accepts the space_name/folder_name schema on put.

I have added an option called :truncated_namespace for this purpose which, when specified in the above config, truncates the path between :public_endpoint and the :filename to be solely that which is specified as a value.

config :pleroma, Pleroma.Uploaders.S3,
  bucket: "space_name/folder_name",
  public_endpoint: "htttps://space_name.nyc3.digitaloceanspaces.com",
  truncated_namespace: "folder_name"

Which will yield htttps://space_name.nyc3.digitaloceanspaces.com/#{truncated_namespace}/#{filename}.

The above config works with Digital Ocean Spaces given the MR'd code change. I was wishy-washy about the name of the opt, let me know if there is something more appropriate. I did not go with an opt name specific to Digital Ocean since this might be useful for other API's.

Edited by feld

Merge request reports