Probably many of you are hoarding terabytes of data. How do you keep it organized, I'm particularly intrested in pictures. For now I see 4 options available to keep them organized:
>filenames + multiplatform, does not require additional software - limited by amount of max character per filename / path
>directory structure + easier than (1) to traverse within file explorer, multiplatform, does not require additional software - needs you to create a fuckton of symlinks, because one pic might belong to multiple directories
>tags + pretty much the same as 1, but without limits and its' easier to search for particular files + multiple types of tags, ie. separate characters and artist tags - significantly slower than 1 - requires a decent tagging software (and I don't know any decent one) - requires you to store your pictures as TIFFs, which takes a ton of storage place and has no real advantage above ie. PNG for the sole purpose of storing pics containing undressed anime characters
>database or 3rd party solution (which probably depends on databases) +fast +all advantages of tags -requires additional software -which might not be multiplatform -cannot search and files under file explorer
Filenames. Then I can search for what I want. Hopefully I can transition to a filesystem which supports longer file names and trees within the next 5 years. (Looking at you ReFS)
Hunter Myers
...
Luke Gomez
Mind sharing your naming pattern? something like: wokada_(takouji)-vocaloid-hatsune_miku-luka_megurine-kagamine_rin-body_writing-no_bra-self_shot-vertical-underboob-twintails-high_res looks like a mess, is hard to write and tends to be problematic with no-wildcard searches
Adam Turner
>- needs you to create a fuckton of symlinks, because one pic might belong to multiple directories I usually just pick one even if it can go in many
Bentley Bell
That's what I used to do in past, but apparently I couldn't find anything when I wanted to.
Adam Jones
Filenames and directory structure. Exclusively, no tags or anything.
Pictures I took get put in folders named by date. (example: "2017-01-21 Description of thing I photographed") I've actually read an article or two on photography websites saying don't do this, their argument being that you'll want to find pictures but not remember when you took them. I've never run into that problem, myself, maybe I just think chronologically.
For images I save off Sup Forums I just put them into a folder by category. If there's an obvious way to name the file (say the image has text on it) then I do so.
I heartily recommend a search program called "Everything" to winfags. Someone wrote a Linux version called Angrysearch but I haven't tried it.
if that happens (usually to me it happens with porn) then I just pick one and leave an empty text file in the other directory saying "X video is saved in Y folder"
Jace Murphy
Thanks for your reply, user. Atm the only type of files I cannot manage by putting them in some subfodlers are pics from anime and a few games (which I don't play, but Overwatch asses are nice too look at, arent they?) I'm currently thinking about structure: /{series name or game name}/{character name}/pics as for the "main" sorting method, but there are a few characters where I have over a thounsand pics, also my "unknown" folder is growing at the alarming rate and I have no idea what to do about it. I'll probably and up deleting a massive part of my collection (everything that isn't at least 8/10), but it's still going to left me with over a dozen thousands pics. I thought about hydrus network, but I don't like the idea of uploading everything to get it tagged, especially with my shitty connection.
>I heartily recommend a search program called "Everything" to winfags. Someone wrote a Linux version called Angrysearch but I haven't tried it. Yup, Everything is at least great-tier program, but since I want my solution to be as multiplatfom as possible (and independent of both filesystems and external programs), it's not gonna help in this case.
Anthony Ross
>Someone wrote a Linux version called Angrysearch but I haven't tried it. there's also fsearch
Lucas Cook
well no matter how you decide to do it you're gonna have to go through all those thousands and thousands of pictures and add the organizational information. Be that a filename, a place in the directory structure, tags, whatever. You just have to sit down and start plowing through it.
the moral of the story is to not let yourself get in the habit of just dumping things into a big "to be sorted" area, but to organize them as you acquire them. same way it's easier to keep your room neat than it is to clean a messy room.
Leo Anderson
didnt read
Carter Parker
That's actually not the case, I happen to made a several tries on sorting them. I converted everything to tiff format, stripped it from metadata, renamed to md5 hash of file and started tagging with Photoshop Elements Organizer. What I noticed is: PEO is well made tagging software... ...but besides of that it's a garbage bloatware which created a miles long log file in my firewall, and I don't feel like keeping this trash on my PC just because I can tag pics with it. Also, it allowed me to tag files only on my main workstation with windows os. Now I have about 80% of my collection in tiff, some tagged, some not and the rest is mixed filenames and fomats. I kinda regret doing what I did, but there's no comeback because I don't keep backup history that long (it started several years ago) Now I'm looking for some better way to get things done, that's why I made this thread. Spending a fuckton of time to actually sort/tag them is not a issue, I have plenty of spare time and I do love to go trough some of these pics from time to time
Landon Bailey
Does anyone here hoard data on blurays or other disc based media instead of a hard drives? I've been considering going that way since it seems to cost significantly less per GB with blurays until you get into storing enough data to use a RAID 6 with decent capacity hard drives. The initial cost for the equipment is also much cheaper (a bluray burner vs building a NAS) and there wouldn't be the reoccurring cost of replacing failing hard drives (though I don't know how often that happens or costs in the long run).
I'm planning on keeping some information on the data in the file name and keeping copies of the directory trees stored in text files so they can be examined and searched with grep without needing to insert the disc.
Christian Young
Depends. If you want to use it as a long term backup - sure, why not. If you want to access the data stored quite often, I'd rather got myself a NAS. B Disks are easy to damage, pain in ass to use (you need to swap them if you want to get data from more than one at the time) and generally slower than NAS with even 5.4k drives inside. Also, you can configure your NAS to keep your data redundant with raid 1 or some other software solution, while storing unique data on BD would require creating copies manually.
Isaac Richardson
>Depends. If you want to use it as a long term backup - sure, why not. Most of the data I store currently just sits around and is rarely accessed, which is why I started to consider blurays or other more archival focused options.
>Also, you can configure your NAS to keep your data redundant with raid 1 or some other software solution I'd consider a RAID to be necessary since I wouldn't want terabytes worth of data I spent time collecting to become unrecoverable in the event of a single hard drive failing. From what I've read quality blurays are much more stable than HDDs, so long as you're storing them correctly, making redundant storage not as necessary.
Brody Reed
>you're storing them correctly That's one thing (and might be harder to achieve than you think) >making redundant storage not as necessary And that's WRONG Redundant storage is always a necessity, with the exception of storing data you won't mind to lose.
Kevin James
I have folders separated by show, and a bash script which uses exiftool to tag all the images in those folders with the name of the show. Now I want to come up with a way to do a reverse image search or something and scrape tags from boorus.
Michael Barnes
Mind sharing code? I had a similar idea. Btw, what tool do you use to convert your images to tiff?
Hunter Jones
>That's one thing (and might be harder to achieve than you think) Keep them in a box in my closet out of the sun. I already keep my room's humidity and temperature pretty stable year round.
>Redundant storage is always a necessity If one HDD fails you lose terabytes of data that can take weeks to retrieve again. If one BD-R fails (which is significantly less like than an HDD failing if you're using quality BD-Rs) you only lose 25 GB of data that you can replace in a couple hours, and with BD-Rs data that can't be easily replaced can have a second copy made for significantly less cost than duplicating an entire hard drive. Redundant storage is not as necessary.
Connor Powell
What about having a sqlite database? >use "find . -exec md5sum {} \;" to generate list of hashes and directories >put it to csv file >import to sqlite table >add tags table >use md5 as foreign key
I think the biggest flaw of this idea is constantly having to update the database manually. And I don't really know how to deal with deleted files. It's not like I'm gonna keep list of removed files and then drop them from table. It's way too much work.
Easton Bailey
>can take weeks to retrieve again naaah, it wouldn't Anyways, I get your point. If it's really much cheaper than NAS then go for it, seems like a decent solution for your case
This is pretty close to what you want. You could also find the last version of Picasa and use that.
Jackson Brooks
is right It will work only if you have less than a few hundreds items in each directory.
I mentioned hydrus earlier, I might end up using it, but that's not the perfect solution either.
Lucas Sanchez
>It will work only if you have less than a few hundreds items in each directory. It really isn't. It gets a bit time consuming to find specific images if your folder has a thousand images, but you can use tags if you really needed to.
Matthew White
Sure I'll put it on pastebin, you'll have to change the base directory to match yours. Also I don't convert to tiff jpg and png both support having exif info. pastebin.com/di1DmvVG
Blake Cruz
thanks btw, afaik PNG does not support exif tags
Blake Robinson
exiftool seems able to write them, it does complain about using nonstandard tagging but it still tags them.
Cooper Clark
intresting, I'll check it out
Cooper Edwards
Just looked it up, png doesn't use exif but it has similar standard that lets you attach strings. w3.org/TR/PNG/#11textinfo
Colton Phillips
79 characters is max, still - good to know
Lucas Collins
>How do you keep it organized I don't.
I do have a few tags folder in which I symlink images and doujins but I'm very lazy so it's not that useful.
Caleb White
I use the world's most advanced OS
Ryder Taylor
>Devices: Windows kek, enjoy your world's most advanced OS ™
John Cox
a small 120gb SSD for games that don't work in wine
is your os advanced enough to do this? i didn't think so
James Davis
snail is impure and a whore
Gavin Reed
Name 1 (one) os that can't. Even wangblows lets you search files by tags, type, orientation, etc. For example, if one writes: orientation:gay in explorer's search box, you are going to appear as a result
Memes aside, how is no options provided for user and no software making mac os "the most advanced" one?
Brayden Diaz
your really dumb don't reply to my posts anymore
Lucas Sullivan
>uses mac os >shills mac os >calls other dumb
man, I love this board
Kevin Sanchez
i told u not to reply to me anymore
Blake Young
OK
Nolan Gomez
Movies are thrown into a folder with practically no regard to naming Music sorted by Artist / Album / CD TV Shows / Anime sorted by Series / Season Pictures are being organized currently with no regard to file name but sorted by used IE reaction / expression.
Jose Sanders
>Movies are thrown into a folder with practically no regard to naming same >Music sorted by Artist / Album / CD same >TV Shows / Anime sorted by Series / Season same >Pictures are being organized currently with no regard to file name but sorted by used IE reaction / expression. Aaaand here I am with my problem.
I guess I'll get rid of most pics saving only ones that are really god-tier, and then go with
../series/ character/...(pics) ...(pics)
plus some tags, auto ones for series/character (exiftool script) and some manually-added ones I guess I'll go with either Picasa or Adobe Bridge for tagging, because first one is lighweight (sort of, at least) and second one is installed on my pc anyways, because my job requires me to use Photoshop from time to time. All files will be converted to tiff, space shouldn't be an issue if I'm going to delete most of them.
That's my conclusion for now, I'm off to sleep. If anyone's got questions or some valuable input - I'll check it tomorrow and reply (unless thread goes to the archive)
Grayson Howard
your so dumb lol
Ethan Wright
I would say just dump them in a folder. It worked for me for at least 2 or 3 since I started "collecting" random images online. I only feel like organizing them because I have around 15 gigabytes of them.
Jacob Cook
Pictures folder: 132.497 items
Yeah, renaming is not an option. At all. I just move them to folders.