diff --git a/README.md b/README.md index 1d48d4a..c665bca 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,14 @@ # bit-inspector "Bit Inspector" (Bir) is a tool to manage files. - * It is inspired by Git, but its goals are slightly different. -Features - * Tracking hash sum of files - * Backing up directory to another location like: local, FTP(S), SFTP, HTTP(S), S3, Bir server - * Optionally you can set a password to protect your files (if a not trusted external location like FTP or S3 is used) - * Detecting bit rots (to keep your files forever) - found bit rots can be repaired. - * Files are handled in a "Bir repository" - * You can clone a remote repository - * You can save your local repository to a remote repository. - * Your files are versioned. You can travel in history and return to an older version of the whole repo or only a changed/deleted directory or file. - -What is not supported: - * Conflict resolution is not supported. (If you wish something not yet suuported to be added, please, let's start a discussion here or at forum.nanoboot.org) - * "Bir" is not intended to be used by many read/write users. - * Several people changing one Bir repository must be avoided. - * "Bir" is not intended to be used by many read users and only one read/write user. - * One Bir repository can be used by more users, but only one user can change it. - * Branches are not supported - -## Requirements +## Requirements to run "Bit Inspector" Requirements to run "Bit Inspector": * Java 19 ## How to build "Bit Inspector" on your own + Requirements to build "Bit Inspector: * Java 19 * Maven @@ -43,7 +25,97 @@ Add to your .bashrc: alias bir='java -jar {path to bit-inspector jar with dependencies file}/bit-inspector-0.0.0-SNAPSHOT-jar-with-dependencies.jar' -## Syntax +## Features + +### Feature 1 : Bit Rot Detection (to keep your files forever) + + * Hash sum of files is tracked. + * If last modification date of file in database and on the disk are the same, but the calculated checksum is different, then a bit rot was detected. + * New files are added to the database, deleted files are removed from the database. + +Found bit rots can be probably repaired, but only in case, you use also "Feature 2 : Backup of files" + +Inspired by: + + * https://github.com/ambv/bitrot + * https://github.com/laktak/chkbit-py + +#### File .birignore + +You can create file .birignore containing the names of the files/directories you wish to ignore +* each line should contain exactly one name +* lines starting with # are skipped +* you may use Unix shell-style wildcards + +#### File .bir.sqlite3 + +An SQLite database + +#### File .bir.sqlite3.sha512 + +Last calculated hash sum of file ".bir.sqlite3" + +### Feature 2 : Backup of files + +Inspired by Git, but goals are slightly different. + * https://github.com/joshnh/Git-Commands + * https://git-scm.com/docs + +#### File .birbackup +remotes=remote1,remote2,remote3 + +#### Features + + * Backing up directory to another location like: local, FTP(S), SFTP, HTTP(S), S3, Bir server and others + * Optionally you can set a password to protect your files (if a not trusted external location like FTP or S3 is used) + + * Files are handled in a "Bir repository" + +#### What is not supported + + * Conflict resolution is not supported. (If you wish something not yet supported to be added, please, let's start a discussion here or at forum.nanoboot.org) + * "Bir" is not intended to be used by many read/write users. + * Several people changing one Bir repository must be avoided. + * "Bir" is not intended to be used by many read users and only one read/write user. + * One Bir repository can be used by more users, but only one user can change it. + * Branches are not supported + +## Bir repository + +* Your files are versioned. You can travel in history and return to an older version of the whole repo or only a changed/deleted directory or file. + +You can: + * clone a remote Bir repository + * or use an existing local Bir repository + * or create a new empty Bir repository + * or create a new Bir repository using an existing directory + * or save your local repository to a remote repository. + +### Structure + +#### Directory .bir/objects + +.bir/objects/{??}/{?????...} + +#### Directory .bir/pack +.bir/pack/pack-{sha-512}.pack + +#### File .bir/birindex.{number} + +#### File .bir/birlog + +Contains index number of last bir index. + +#### file .bir/description + +#### File .bir/config + +``` +pack-file.files-until-size.mb=100 +pack-file.max-size.mb=1000 +``` + +## Commands bir {command} [{arg1} {arg2} {argn}] @@ -51,28 +123,26 @@ Example: ``` bir clone path=/home/johndoe/mydir url={local path or s3 bucket or FTP server or website url} ``` - -### Arguments +Arguments path={path to directory} * default:. (current working directory) -## Bir repository -To use Bir, a Bir repository is needed. +### bitrot -You can: - * clone a remote Bir repository - * or use an existing local Bir repository - * or create a new empty Bir repository - * or create a new Bir repository using an existing directory +Checks for bitrots in current directory -### Command : clone : Cloning a remote repo +### clone : Cloning a remote repo + +``` +bir clone {url} [[--bare +``` #### Local ``` -bir clone {path to local directory = Bir repository} +bir clone {path to another local Bir repository - path to directory} ``` #### S3 @@ -99,31 +169,15 @@ bir clone http[s]://[{user}:{password}]@{host url}:{port}/{directory} bir clone bir:://[{user}:{password}]@{host url}:{port}/[path to repository/]{repository name} ``` -### Using an existing Bir repository - -### Creating a new empty Bir repository - -Go to wanted directory or use argument dir={path to wanted directory} and run: +### init +Init commands creates new directory .bir with its structure ``` -bir init [dir to wanted directory] +bir init ``` -### Creating a Bir repository using an existing directory - - -Go to wanted directory or use argument dir={path to wanted directory} and run: - -``` -bir init [dir to wanted directory] -``` - - -### Commands - -Inspired by: - * https://github.com/joshnh/Git-Commands - * https://git-scm.com/docs + * Creating a new empty Bir repository + * Creating a Bir repository using an existing directory ### help @@ -135,8 +189,6 @@ Inspired by: ### reset -### mv - ### tag ### revert @@ -147,25 +199,12 @@ Inspired by: ### gc -### clone - ### fsck ### check ### bundle -``` -bir clone {url} [[--bare -``` - -### init - -Init commands creates new directory .bir with its structure -``` -bir init [dir={path to directory}] -``` - ### remote add {remote name} ``` @@ -184,63 +223,14 @@ bir commit [-m "{message}"] ### bir pull -### bir status - -### bir add - -### bir rm - -### bir stash - ### bir log ### bir diff +### bir prune - -Table FILE – add new columns – linux_rights, owner, group - - -### - -## Structure of .bir directory - -.bir -.bir/objects -.bir/remotes/{remote name} -.bir/bir.sqlite3 ... an SQLite database -.bir/bir.sqlite3.sha512 ...last calculated hash sum of file ".bir.sqlite3". -.bir/config -.bir/description - - -## File .birignore - -You can create file .birignore containing the names of the files/directories you wish to ignore -* each line should contain exactly one name -* lines starting with # are skipped -* you may use Unix shell-style wildcards - -## Detection of bit rots - -If the file last modification date in database and on the disk are the same, but the calculated checksum is different, then a bit rot was is probably detected. - - -New files are added to the database, deleted files are removed from the database. - - -This program cannot restore files with bitrot. - * You have to backup up your files (do snapshots). - - -It is inspired by: - - * https://github.com/ambv/bitrot - * https://github.com/laktak/chkbit-py +bir prune origin --since 2021-10-04 ## Todo -New tables: FILE_HISTORY - - - +Table FILE – add new columns – linux_rights, owner, group diff --git a/src/main/java/org/nanoboot/bitinspector/commands/CheckCommand.java b/src/main/java/org/nanoboot/bitinspector/commands/CheckCommand.java index ca78700..58cb931 100644 --- a/src/main/java/org/nanoboot/bitinspector/commands/CheckCommand.java +++ b/src/main/java/org/nanoboot/bitinspector/commands/CheckCommand.java @@ -361,7 +361,7 @@ public class CheckCommand implements Command { if (processedCount % 100 == 0) { double progress = ((double) processedCount) / countOfFilesToCalculateHashSum * 100; LOG.info("Update - Progress: " + processedCount + "/" + countOfFilesToCalculateHashSum + " " + String.format("%,.2f", progress) + "%"); - LOG.info("Remains: " + Duration.of(rtc.remainingSecondsUntilEnd(), TimeUnit.SECOND).toHumanString()); + LOG.info("Remains: " + Duration.of(rtc.getRemainingSecondsUntilEnd(), TimeUnit.SECOND).toHumanString()); } File file = new File("./" + absolutePathOfFileInDb);