Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt sqlite with proper caching #2219

Open
1 of 4 tasks
ppfeister opened this issue Jul 4, 2024 · 3 comments
Open
1 of 4 tasks

Adopt sqlite with proper caching #2219

ppfeister opened this issue Jul 4, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@ppfeister
Copy link
Member

Description

  • Switch from text files for each username to a single sqlite database
  • Allow for text file output as an option to better support legacy?
  • Properly cache and expire results in database (only cache positive hits)

Code of Conduct

  • I agree to follow this project's Code of Conduct
@ppfeister ppfeister added the enhancement New feature or request label Jul 4, 2024
@sdushantha
Copy link
Member

I like your suggestion @ppfeister, it prevents having several .txt files scattered across the file system.

With the results being stored in a sqlite DB, would we add a few extra flags Sherlock so that i can list and remove results from the DB?

@ppfeister
Copy link
Member Author

Proper implementation in my mind would require results to be cached for a standard amount of time (TBD), allowing the result to be overwritten when expired (reduce rate limit hits), with a flag to skip caching new entries, and a flag to ignore presently cached entries.

Say, --no-cache and --force-check or similar.

I didn't see a need for any further options to manipulate the db, but if any should be added for better UX, they should be behind a sub-command of some type rather than even more flags than we already have. Say, sherlock cache --clear and sherlock cache --clear-site or similar. Only issue this specific example raises is ambiguity between cache the command and cache the username, but something similar should be workable.

@ppfeister
Copy link
Member Author

sherlock-cache likely to be added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
2 participants