Resolved Attachment Store move from S3 to External Data

Discussion in 'Other' started by Tai, Apr 8, 2016.

  1. Tai

    Tai New Member

    #1 , Apr 8, 2016
    Last edited: Apr 8, 2016
    Hi,

    I'm trying to move all my attachments from S3 into the local external data directory. I've set it to "Store file in External Data directory" in the options and then used the "Move Attachment Data" tool in the Rebuild Cache area. Unfortunately, the attachments are still being served from S3.

    Running XF 1.5.7 and [bd] Attachment Store 1.2.1

    What can I do to get attachments to be served from /data instead of my S3 bucket?
     
  2. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    What did it say when you ran the move tool?
     
  3. Tai

    Tai New Member

    It just completed the move and said something along the lines of "Cache rebuilt successfully."
     
  4. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    How many files do you have on S3 (estimated is okie). And how long did the tool run? After the run, do you have anything in /data/attachment-files?
     
  5. Tai

    Tai New Member

    Around 16,000 files I would think. Yes, there are files in /data/attachment-files
     
  6. Tai

    Tai New Member

    The tool ran for about 40 to 60 minutes, maybe longer.
     
  7. Tai

    Tai New Member

    @xfrocks Is there anything that can be done?
     
  8. Tai

    Tai New Member

    Still no response, my resource files are leading to dead S3 links. How can I get off this addon?
     
  9. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    Hi there,

    Sorry for the late response. As you said earlier, you have 16k files in S3? How many are there in /data/attachment-files after the move? (They should have the same number of files fyi). Also, have you deleted your s3 bucket? How can they lead to dead links?
     
    1. Tai

      Tai New Member

      Tai @xfrocks There's 9091 files, I'm not sure how many are on S3. I haven't deleted the S3 bucket. However, the dead links do work if I change the Amazon URL to the relevant forum URL.

      For example, this is an attachment to one thread:

      This is the link on the thread (Amazon link), the file doesn't exist: https://smd-vault.s3.amazonaws.com/2014/08/3635_2b8c889e2580b3669fcb522b38de9b3f.zip

      If I change it to the data directory for my website it works: https://starmadedock.net/data/2014/08/3635_2b8c889e2580b3669fcb522b38de9b3f.zip

      It also looks like it didn't move those files to the attachment-files directory.
       
  10. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    Can you get the row dump for xf_attachment_data where data_id = 3635?
     
    1. Tai

      Tai New Member

      Tai @xfrocks Yea, I already removed that attachment. I can try to find another attachment where this the case? Unless attachments aren't deleted when you remove them from the thread?
       
    2. xfrocks

      xfrocks XenForo rocks!
      Staff Member

      xfrocks @Tai Any data is good as long as it's returning wrong url (s3) while the file is in external data.
       
    3. Tai

      Tai New Member

      Tai @xfrocks I PM'd you the dump, I took a look and 3635 wasn't deleted. So, I assume XF doesn't remove the entry from the DB when it's removed from a thread.
       
  11. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    XenForo remove the row from db after a while. But only if you do hard-delete thread though.

    Anyway, I have checked the dump. According to it, the file is currently in s3, so that's why the s3 url is being generated... That means the add-on performed the file move but failed to update its database (for some unknown reason). I will need access to your server and db for further debug, can you provide the credentials?
     
    1. Tai

      Tai New Member

      Tai @xfrocks Any updates @xfrocks ? I've sent all the details to you in a PM.
       
  12. xfrocks

    xfrocks XenForo rocks!
    Staff Member

    @Tai I have marked your issue as resolved. Let me know if you still need assistance regarding this.
     
Loading...