Duplicate pics on ancient posts

[Mark J 243]

Certainly no big deal, but is there a reason that most of the large threads going back to 2014 seem to have duplicates of every pic. I understand quoted pics. It makes each page take double time when trying to go through 50-100 pages of some of the largest threads. 

Again in the scheme of today’s world, this is so minor, but I am just curious. 

[F.W 5,734]

Because many pics are kind of recycled in different threads.

[Mark J 243]

I was asking about individual posts like https://peefans.com/topic/61-b2939s-picture-collection/

 

 

[Sophie 24,410]

This is an unfortunate side effect of a server move way back when. All images in posts were duplicated. 

If anyone would like to provide me with a list of links I could go through them all and manually delete the duplicates.

[Mark J 243]

29 minutes ago, Sophie said:

This is an unfortunate side effect of a server move way back when. All images in posts were duplicated. 

If anyone would like to provide me with a list of links I could go through them all and manually delete the duplicates.

Thanks @Sophie for the clarification and explanation. 

It would be insane to go through them manually. Like I said, I was just curious

[Scot_Lover 1,877]

Yeah, I was thinking of the duplicates too, Just didn't think I could find a way to locare them all.

@Admin, are you able to do a comparison search at all?

[Admin 14,788]

As Sophie said it's down to when the forum software got changed. Overall I think the current software is better, but this was an unfortunate side effect for a lot of the old picture threads.

When you upload an image to this software, you have the option to 'insert into post' in a specific position, or if you don't do that, it'll automatically just add the picture to the end of your post. When the migration happened it seems that the previous attachments were all automatically added to the end of the posts, despite the fact that many pictures were already 'inserted into the post'. This created a fair few duplicates. Not sure how much sense that makes, but basically there is a reason it happened, but unfortunately solving it would be a manual process. If anyone reading this would like to tidy up their own thread though, just removing the duplicates in a single thread shouldn't actually take too long. 🙂 

If not, there's some great pictures so I guess we'll just have to enjoy them twice. 😛 I would guess that given the amount of new pictures posted regularly, a large amount of people probably don't go back to the 2014/2015 posts anyway.  

[Scot_Lover 1,877]

I was just mindful of the storage requirements, I do client work on an Azure service, and I have to account for every byte, lol.

Everything has to be accounted and paid for.

[Admin 14,788]

1 hour ago, Scot_Lover said:

I was just mindful of the storage requirements, I do client work on an Azure service, and I have to account for every byte, lol.

Everything has to be accounted and paid for.

Haha, very true.

Luckily the picture isn't uploaded twice - only one file is used, it's simply inserted into the post twice through the glitch (so no extra space is used). You're right though that from a bandwidth perspective it's not ideal, but thar's not much of an issue/cost compared to storage. Thanks though. 🙂 

[wetmanjf 2,787]

I have been deleting the duplicate pictures as I have time...but I am sure there are a LOT of them

 

Wetman

[likesToLick 10,216]

On 10/18/2018 at 7:41 AM, Admin said:

Luckily the picture isn't uploaded twice - only one file is used, it's simply inserted into the post twice through the glitch (so no extra space is used).

Given that these old posts have all been corrupted in the same way by the server move,  might it be possible to write some query code that would make a single pass across the database and correct each occurrence of the error as it finds them?

I'm not familiar with the software used for the forum platform.  What kind of database does it use?  Is the interface open?  Is there an API?

Of course,  you'd want to be very sure of your backup and recovery procedures, before you risked running a process like that.

Edit:  I just took a look at the Invision site,  and it appears that database access of the kind I was suggesting is locked away behind a cloudwall,  so perhaps my idea will be no use to you.

Edited October 26, 2018 by likesToLick
update