Posted inNumérique

S3 migration: Segment error and hidden videos from channel

d71a876ab463922f105d212d0610fe9980892118 S3 migration: Segment error and hidden videos from channel

Hi there, i have already tried to ask on Matrix, but maybe this would be a better place:

I have initiated a migration from local (network storage) to S3 (Backblaze). Uploads work, playback without proxy works now too (i had to learn how important the custom bucket rules are, oops, some failed jobs there), but with the proxy, i am getting a segment error similar to this already closed issue.

The nginx proxy is adopted from the Mastodon docs with minor adjustments:

 location @s3 { limit_except GET OPTIONS HEAD { # GET didnt seem enough deny all; } resolver;
# proxy_set_header Host YOUR_S3_HOSTNAME; # backblaze generally doesnt seem to like this one proxy_set_header Connection ''; proxy_set_header Authorization ''; proxy_hide_header Set-Cookie; proxy_hide_header 'Access-Control-Allow-Origin'; proxy_hide_header 'Access-Control-Allow-Methods';
# proxy_hide_header 'Access-Control-Allow-Headers'; # this seemed to get rid of errors like "range" proxy_hide_header x-amz-id-2; proxy_hide_header x-amz-request-id; proxy_hide_header x-amz-meta-server-side-encryption; proxy_hide_header x-amz-server-side-encryption; proxy_hide_header x-amz-bucket-region; proxy_hide_header x-amzn-requestid; proxy_ignore_headers Set-Cookie; proxy_pass $s3_backend$uri; proxy_intercept_errors off; proxy_cache tubecache; proxy_cache_valid 48h; proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504; proxy_cache_lock on; expires 1y; add_header Cache-Control public; add_header 'Access-Control-Allow-Origin' '*'; add_header X-Cache-Status $upstream_cache_status; 

Now i wonder if the resulting commit of that bugreport would fix my issue as well, even tho i indeed see 206 statuscodes. And how i could cherry-pick that commit without bigger setup changes (i suppose i can replace that symlink with a git version though, now that i write about it).

Edit: just removing that proxy from the peertube config makes it two errors:

CORS Missing Allow Origin XHROPTIONShttps://tube-tchncs-de.s3.eu-central-003.backblazeb2.com/videos/...
CORS Missing Allow Origin

Second issue

A user just reported that almost all of their videos are not visible anymore. If i click to see all videos as moderator, their videos are still there and neither private nor unlisted. The hidden videos appear to be migrated ones, to object storage. This is very confusing and i am not sure how to address this problem at the moment.


Laisser un commentaire