Generalize `silence_polling` to `silence_queries`

[namolnad]

          Yes, I think `silence_queries` would work 👍🏻

Originally posted by @rosa in #198 (comment)

As mentioned in #198, silence_polling only silences the polling queries (which makes sense), and continues to log for all other querying methods. I think many, especially in a development environment, would be interested in a silence_queries setting to fully silence heart_beat, semaphore, etc queries and keep their logs clean. We likely should maintain silence_polling, at least for now, for backwards compatibility.

[namolnad]

@rosa do you know of a global way we can wrap all the querying methods in solid_queue in a similar fashion to how the polling is wrapped currently? Or is it more likely that we'll just need to pick off the methods one-by-one and make sure new functionality gets wrapped similarly? Happy to take a stab at this one once that approach gets nailed down. Thanks!

[iJackUA]

Up to this issue, as it really quite distracting in dev mode.

With dozens of jobs / recurring jobs and settings

RAILS_LOG_TO_STDOUT=true
RAILS_LOG_LEVEL=debug

It's literally an infinite flood in the terminal with SolidQueue related SQL logs.
While developer is mostly interested in few messages types "Job Enqueued" / "Performing Job" / "Performed Job" / "Error trace if there is an error".

I am also a bit confused trying setting different "logger" for SolidQueue, trying to redirect it's logs and using other log_level. But it is not redirecting all logs, only from here https://github.com/rails/solid_queue/blob/main/lib/solid_queue/log_subscriber.rb

Also there is ActiveJob with it's own logger in the equation. And while trying to set custom logger to SolidQueue.logger /
ActiveJob::Base.logger I still could not synchronize them to show me only "significant logs" like INFO level.

One workaround for this is

bin/jobs

require_relative "../config/environment"
require "solid_queue/cli"

# Keep using single Rails.logger for everything, but for this instance of Ruby env set log level INFO in dev 
Rails.logger.level = Logger::INFO if Rails.env.local?

SolidQueue::Cli.start(ARGV)

It keeps logger setting in one place (basic Rails logger config), log level for web app itself in dev can be still kept debug, but SolidQueue now shows me only more or less important "solid_queue/log_subscriber" events and ActiveJob's "enqueue/performed" logs.

[TheNotary]

I second the motion to make development logs relevant. These tick:

SolidQueue-1.1.4 Prune dead processes (4.2ms) size: 0
SolidQueue-1.1.4 Unblock jobs (2.3ms) limit: 500, size: 0

I followed your solution, iJackUA, but put it into config/environments/development.rb which feels slightly more conventional.

(config/environments/development.rb)

config.active_job.verbose_enqueue_logs = true
config.active_job.queue_adapter = :solid_queue
config.solid_queue.silence_polling = true
config.solid_queue.logger = ActiveSupport::Logger.new(STDOUT, level: Logger::INFO)

It's a shame there's no TRACE level of logging, because I feel as though these kinds of recurring messages don't make sense to print to the dev log unless they're troubleshooting something specific and opt into it. Would it be possible to add a config.solid_queue.verbose_logs parameter that allows us to limit the verbosity on all reoccurring messages? Also one of my biggest quality impressions is when ctrl+c results in a stack of logs being puked to the console:

^CSolidQueue-1.1.4 Shutdown Dispatcher (10.2ms) pid: 6664, hostname: "lappy-4.local", process_id: 546, name: "dispatcher-73b47bda61e49f6a0439", polling_interval: 1, batch_size: 500, concurrency_maintenance_interval: 600
SolidQueue-1.1.4 Release claimed jobs (3.4ms) size: 0
SolidQueue-1.1.4 Shutdown Worker (25.5ms) pid: 6665, hostname: "lappy-4.local", process_id: 547, name: "worker-070ad4b9331441fa8320", polling_interval: 0.1, queues: "*", thread_pool_size: 3
SolidQueue-1.1.4 Supervisor terminated gracefully (119.0ms) process_id: 545, supervisor_pid: 6661, supervised_processes: [6664, 6665]
SolidQueue-1.1.4 Shutdown Supervisor (35.8ms) pid: 6661, hostname: "lappy-4.local", process_id: 545, name: "supervisor-6d90f9601efc219169bf"

Those logs are printed at the INFO level, so this trick doesn't even resolve them when in development. I think they make sense to log in production, but in development when you've rigged solid queue up just for the sake of ActionCable updates flowing to the views correctly, they would never be found helpful, right?

@rosa, do you know the 'logging vision' for solid_queue, especially as it pertains to Rails users leveraging it in development? Months ago, it seemed like this was the best way to observe app behavior as jobs interact with the database and ActionCable broadcast hooks.

[diogovernier]

Hi @rosa, @namolnad! I put together a PR (#732) that implements the silence_queries approach discussed here and in #198.

It replaces silence_polling with a unified silence_queries config that covers polling, heartbeats, concurrency maintenance, process pruning, and scheduler dynamic task reloading. silence_polling is kept as a backward compatible alias.

Would love your feedback on the direction. Happy to adjust if you had something different in mind!

[rosa]

Closing the loop here: this ended up solved in Rails rather than in the gem. rails/rails#56899 adds a sql_notifications: false option to the database config, which turns off all SQL notifications for that connection. Setting it on the queue database silences polling, heartbeats, semaphores, pruning, everything, and it's simpler than maintaining a list of things to silence on our side. It'll be in Rails 8.2; silence_polling stays for earlier versions. There's now a note about it next to silence_polling in the README (#795). Thanks all!