Skip to content

Have --catch-up=SESSION add the new hash files to SESSION#5966

Open
magnumripper wants to merge 1 commit into
openwall:bleeding-jumbofrom
magnumripper:catch-up-add
Open

Have --catch-up=SESSION add the new hash files to SESSION#5966
magnumripper wants to merge 1 commit into
openwall:bleeding-jumbofrom
magnumripper:catch-up-add

Conversation

@magnumripper

@magnumripper magnumripper commented Mar 24, 2026

Copy link
Copy Markdown
Member

After the --catch-up=SESSION run finished, add its input files to SESSION's recovery file so that SESSION can simply be resumed for continuing with its old files as well as the new ones.

This can be disabled with a new --no-catch-up-add option.

@magnumripper
magnumripper requested a review from solardiz March 24, 2026 12:13
@magnumripper
magnumripper force-pushed the catch-up-add branch 3 times, most recently from 583cdd4 to f379506 Compare March 24, 2026 16:52
@magnumripper

Copy link
Copy Markdown
Member Author

This is tested in all relevant ways I could think of, such as with --fork and with any combination of single/multiple files.

@magnumripper
magnumripper force-pushed the catch-up-add branch 2 times, most recently from 6b3cdd9 to b3f441c Compare March 24, 2026 23:57
@magnumripper
magnumripper force-pushed the catch-up-add branch 3 times, most recently from afdfa88 to 8c322b4 Compare April 9, 2026 19:21
After the --catch-up=SESSION run caught up, its input files are now
added to SESSION's recovery file so that it can simply be resumed for
continuing with its old files as well as the new ones.

This can be disabled with a new --no-catch-up-add option.
Comment thread src/options.c
{"restore", FLG_RESTORE_SET, FLG_RESTORE_CHK, 0, ~FLG_RESTORE_SET & ~GETOPT_FLAGS, OPT_FMT_STR_ALLOC, &options.session},
{"session", FLG_SESSION, FLG_SESSION, FLG_CRACKING_SUP, OPT_REQ_PARAM, OPT_FMT_STR_ALLOC, &options.session},
{"catch-up", FLG_ONCE, 0, 0, OPT_REQ_PARAM, OPT_FMT_STR_ALLOC, &options.catchup},
{"catch-up-add", FLG_ONCE, 0, 0, USUAL_REQ_CLR | OPT_TRISTATE, NULL, &options.catchup_add},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this OPT_TRISTATE? (Just a question, not implying there's anything wrong.)

@solardiz solardiz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I thought of just merging this quickly, but then I saw it's complicated. We can still just merge or we can discuss first... which I guess neither of us has much time for now - I certainly don't. What do you suggest?

Comment thread doc/OPTIONS
file so you can just resume that session. This will end up just as if both
sets of hashes were in the original session to start with, with no wasted
resources or time (assuming salted hashes). If you do not wish the original
session file to be touched, also add the --no-catch-up-add option.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that we don't have the --no-catch-up-add option documented in its own section in OPTIONS, we just mention it here. This may be fine. Maybe we already do similar for other obscure options?

Comment thread src/recovery.c
return ret;
}

void rec_add_files(char *session)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rec_add_files is quite some code and quite some duplication of logic (filename suffix, locking) with our existing rec file writing code. As an alternative, maybe we could be adding the extra files to rec_argv and rec_argc, and then our existing code would just happen to write them into the rec file at next sync point (which we could trigger explicitly if needed).

Comment thread src/recovery.c
fprintf(catchup_file, "%s\n", current->data);

/* Append original tail */
fprintf(catchup_file, "%s", catchup_data + offset);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use fwrite. We read this data with fread.

Comment thread src/recovery.c

/* Append original tail */
fprintf(catchup_file, "%s", catchup_data + offset);
fclose(catchup_file);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In our usual code, we don't just fclose. We check with ferror first (to detect any errors of fprintf), then fflush (with its return value checked), ftruncate (ditto), and fsync (ditto). OK, ftruncate may be unneeded here if we're sure we can only increase the size, but the rest are probably good to have?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding "sure we can only increase the size", I'm confused as to whether we're rewriting the same file we read from, or some other file. If it's a different file, then its previous content may have been unrelated and larger, and then we'd need to truncate.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, indeed same file - this code literally seeks to offset 0 again. Then maybe we're fine without truncation.

Comment thread src/recovery.c
if (fclose(rec_file))
pexit("fclose");
rec_file = NULL;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused - why do we delete and close this session's file? I don't seem to fully understand the logic this implements. Also, what would these two filenames be in typical usage?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants