|
| 1 | +# memetagfs |
| 2 | + |
| 3 | +This is a hierarchic tag filesystem implemented in Go. The name implies it's designed to store memes but you can use it |
| 4 | +for any files. The name was chosen so that it's unique enough to find in Google. |
| 5 | + |
| 6 | +# Motivation |
| 7 | + |
| 8 | +This project mostly replicates my older one called [jtagsfs](https://github.com/rkfg/jtagsfs). I removed some unused features |
| 9 | +and added some that I needed, also the codebase is much more clean now. This version uses less RAM and is more portable, doesn't |
| 10 | +have runtime dependencies so I actually run it on my SOHO router from a USB stick. I optimize it to perform good enough on a |
| 11 | +580 MHz/128 Mb MIPS system so it should be blazing fast on a typical desktop PC. |
| 12 | + |
| 13 | +# Platforms |
| 14 | +## Supported OS |
| 15 | + |
| 16 | +Linux only for now. There's no platform-specific code in the filesystem itself but it uses [the FUSE library](https://github.com/bazil/fuse) |
| 17 | +that currently only supports Linux and BSD and I don't use BSD. Tell me if it works there. Windows support is usually tricky for |
| 18 | +custom filesystems and requires 3rd party driver installation and whatnot. |
| 19 | + |
| 20 | +## Supported architectures |
| 21 | + |
| 22 | +- x86_64 |
| 23 | +- MIPS |
| 24 | +- i686 (not tested but should be okay) |
| 25 | +- ARM (not tested but should be okay) |
| 26 | + |
| 27 | +# Installation |
| 28 | + |
| 29 | +Get a prebuilt binary from the [releases tab](https://github.com/rkfg/memetagfs/releases) or install a Go toolchain and do `go get github.com/rkfg/memetagfs` |
| 30 | +The ready to launch binary should appear in `~/go/bin/memetagfs`. You can copy it wherever you like. If you crosscompile for other |
| 31 | +architectures make sure you have a crosscompiler which is required to build the SQLite library (automatically). On Debian you can |
| 32 | +install `gcc-arm-linux-gnueabi` package for ARM and `gcc-multilib-mipsel-linux-gnu` for MIPS. You'll need a regular GCC environment |
| 33 | +anyway to build for x86. |
| 34 | + |
| 35 | +# Usage |
| 36 | + |
| 37 | +## Big fat warning |
| 38 | + |
| 39 | +Due to the way filesystems work there's a thing you should know beforehand. **NEVER** delete directories under the `browse` directory. |
| 40 | +Any file managers and commands I ever heard do that recursively which means they will visit all possible combinations of tags removing |
| 41 | +all files they'll find effectively wiping most or all of your files! It's not possible to delete the directories themselves in `browse` |
| 42 | +but files can and will be deleted. If you need to delete a tag do it only in the `tags` directory which is made exactly for managing |
| 43 | +tags. If you need to change the tags a file belongs to, move it to another combination of tags. Deleting a file physically deletes it |
| 44 | +from the storage, forever! Not just from this particular combination of tags. |
| 45 | + |
| 46 | +## Creating tags |
| 47 | + |
| 48 | +First, you need to create an empty directory for the database and storage. |
| 49 | +Create another empty directory where your filesystem should be mounted to. Then launch |
| 50 | +`memetagfs -s /path/to/storage -d /path/to/database.db /path/to/mountpoint` |
| 51 | +to mount it. You'll have 2 directories inside, `browse` and `tags`. Create your tags inside the `tags` directory (as directories). |
| 52 | +If you want some tags only be visible inside other tags, create them accordingly. For example, you may have tags |
| 53 | +like `HD`, `BD` or `DVD` that are only applicable for the `video` tag and there's no reason to have them |
| 54 | +visible in `pictures` tag. You then create the `video` tag, cd to it and create those `HD`, `BD` or `DVD` |
| 55 | +tags inside. |
| 56 | + |
| 57 | +It's also possible to _include_ a tag subtree to another tag when two distinct tags share the same children. Suppose you have |
| 58 | +`pics` and `vids` tags for pictures and videos (memes, for example). Most of the child tags are the same for both categories. |
| 59 | +One solution could be just putting all those tags at the same root level so they're visible in both and if it works for you, great! |
| 60 | +But what if you also have completely different root tags like `music` or `books`? You certainly don't want to see `cats` or `pepe` |
| 61 | +among the music-related tags. This was not possible to solve reliably in `jtagsfs` but it is in `memetagfs`! It supports a special |
| 62 | +**tag group** tag type that can be included in other regular tags. Prepend the tag name with `!` and it becomes a tag group. |
| 63 | +Tag groups are not visible by themselves. Create some tags you want to share between other tags under a tag group, for example, `!memes`. Let's say you've created `cats`, `dogs` and `frogs` inside `!memes`. Now you should include this group to both `pics` and `vids`. For that simply rename the directories adding the tag group name (without `!`) between pipes like this: `pics |memes|` and |
| 64 | +`vids |memes|`. Now when you open these tags (see [Browsing](#Browsing)) you'll see not only the direct children you might have added |
| 65 | +to them (specific to `pics` and `vids`) but also everything you've added to `!memes`. You can include more than one tag group |
| 66 | +separating them with commas. For example, you might want to create a `!quality` group with `high`, `medium` and `low` tags representing |
| 67 | +the perceived quality of content. But that can also be used for `music`! So then you rename the root tags as this: |
| 68 | +`vids |memes, quality|` (space after comma is optional) and `music |quality|`. |
| 69 | + |
| 70 | +## Browsing |
| 71 | + |
| 72 | +The `browse` directory is where you put your files to and where you query them. |
| 73 | +For example, you want to tag a movie with tags `video`, `HD`, `thriller` and `sci-fi`. You cd to |
| 74 | +`/path/to/mountpoint/browse/video/HD/thriller/sci-fi/@` (the order isn't important) and copy your file there. |
| 75 | +The `@` directory marks the end of the tags set. |
| 76 | + |
| 77 | +By default tags are concatenated with boolean AND so when you search for a file you specify a set of tags that it should |
| 78 | +have. You may also exclude some of the tags with `_`. For example, you want to find all pictures of cats but without dogs (because some |
| 79 | +pictures might have both and you tagged them with both tags). For that you go to `/path/to/mountpoint/browse/pics/cats/_/dogs/@` and see |
| 80 | +the result of the query. Most of the time the default behavior will be just what you need. |
| 81 | + |
| 82 | +To change tags just move the file to another tag path. If you remove the file, it's removed from the storage forever, |
| 83 | +not just from this tag or set of tags! If you want to remove one or several tags, move the file to the path that doesn't |
| 84 | +contain these tags. If the same file is visible there you can drop it to the `@@` directory instead so there's no conflict. |
| 85 | + |
| 86 | +There's also a special `@@` directory which shows internal file IDs and all tags the file has. It's useful to check |
| 87 | +if a file is tagged correctly. You may move files from and to this directory just as fine as to/from `@`. Don't try |
| 88 | +to change tags by renaming files here, the only part that matters is the actual filename. Tags are shown just for |
| 89 | +information and changing them would lead to renaming a file to itself. |
| 90 | + |
| 91 | +You may move, rename and delete tags in the `tags` directory. If a tag has files on it you won't be able to delete such tag. |
| 92 | + |
| 93 | +## Duplicates |
| 94 | + |
| 95 | +Due to the nature of semantic filesystems sometimes you can get more than one file with the same name in the query results. Consider the |
| 96 | +following example: file 1.jpg that belongs to tags `pics`, `cats` and another file 1.jpg that belongs to `pics` and `dogs`. It's perfectly valid but what would happen if you visit just `pics` tag? Both files would appear and without special measures there's no way |
| 97 | +for other programs to differentiate between these two. Such situation is handled by prefixing the filenames with the IDs of the database |
| 98 | +records. So in this case the files would look like `1231|1.jpg` and `389|1.jpg` (if their internal IDs are 1231 and 389). If you rename |
| 99 | +either of the files, the deduplication mechanic will turn off and you'll see the original filenames again. Moving such files around |
| 100 | +is fine, the deduplicating prefix is transparently removed. As a consequence, you can't use the `|` symbol in the filenames. |
| 101 | + |
| 102 | +# Checking for errors |
| 103 | + |
| 104 | +Software has bugs. It's inevitable. But losing data because of that is unacceptable (even though it happens sometimes). Memetagfs |
| 105 | +can check the database and storage for contradictions and salvage the files that for some reason lost all database records and became |
| 106 | +invisible. Use `--fsck` parameter to scan the internals and report the number of errors found. Use an additional `-f` flag to fix those |
| 107 | +errors. All unreferenced files will be put to a root tag `lost+found` (it will be created if it doesn't exist) and you can sort them |
| 108 | +later. Any database records that refer to non-existing files will be deleted. |
0 commit comments