Add configurable User-Agent for VulnerableCode scrapers (Fixes #410) #2077

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

warrior-hub wants to merge 2 commits into aboutcode-org:main from warrior-hub:add-user-agent

warrior-hub commented Dec 20, 2025

Problem:

Web scrapers did not declare a User-Agent.
Some websites block requests without a proper User-Agent.

Solution:

Added a default User-Agent: VulnerableCode
Configurable via environment variable: VULNERABLECODE_USER_AGENT
Created helper function http.get() to be used for all future requests

This addresses issue #410

warrior-hub and others added 2 commits

December 20, 2025 20:20


          Add configurable User-Agent for web requests

b1d23c8


          Merge branch 'main' into add-user-agent

f61ec44

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet