ruby on rails - Sidekiq - Intermittent nonstart -
i'm dealing frustrating bug - sidekiq doesn't start processing jobs. it's annoyingly intermittent.
when happens, sidekiq.log
shows no change (and running). queue size remains @ 0. i'm guessing it's getting lost in redis.
in case matters, i've got 3 environments of same app running on same machine (different environments), put in (but problem preceded them). 1 redis running on default port.
i know redis-cli monitor
spews out stuff fast, , there's much, it's hard me figure out going on.
1491479154.190069 [0 127.0.0.1:38763] "brpop" "queue:default" "2" 1491479154.289507 [0 127.0.0.1:38753] "brpop" "queue:default" "2" 1491479154.289544 [0 127.0.0.1:38752] "brpop" "queue:default" "2" /// snip... many lines of these 1491479154.364985 [0 127.0.0.1:38772] "multi" 1491479154.365090 [0 127.0.0.1:38772] "incrby" "stat:processed" "0" 1491479154.365107 [0 127.0.0.1:38772] "incrby" "stat:processed:2017-04-06" "0" 1491479154.365128 [0 127.0.0.1:38772] "incrby" "stat:failed" "0" 1491479154.365133 [0 127.0.0.1:38772] "incrby" "stat:failed:2017-04-06" "0" 1491479154.365139 [0 127.0.0.1:38772] "del" "server.name:32533:6befea0f66c0:workers" 1491479154.365149 [0 127.0.0.1:38772] "expire" "server.name:32533:6befea0f66c0:workers" "60" 1491479154.365160 [0 127.0.0.1:38772] "exec" 1491479154.365896 [0 127.0.0.1:38772] "multi" 1491479154.365962 [0 127.0.0.1:38772] "sadd" "processes" "server.name:32533:6befea0f66c0" 1491479154.365977 [0 127.0.0.1:38772] "exists" "server.name:32533:6befea0f66c0" 1491479154.365987 [0 127.0.0.1:38772] "hmset" "server.name:32533:6befea0f66c0" "info" "{\"hostname\":\"server.name\",\"started_at\":1491373877.8700902,\"pid\":2217,\"tag\":\"demo\",\"concurrency\":25,\"queues\":[\"default\"],\"labels\":[],\"identity\":\"server.name:32533:6befea0f66c0\"}" "busy" "0" "beat" "1491479154.3655496" "quiet" "false" 1491479154.366035 [0 127.0.0.1:38772] "expire" "server.name:32533:6befea0f66c0" "60" 1491479154.366046 [0 127.0.0.1:38772] "rpop" "server.name:32533:6befea0f66c0-signals" 1491479154.366058 [0 127.0.0.1:38772] "exec" 1491479154.389484 [0 127.0.0.1:38751] "brpop" "queue:default" "2"
how 1 debug redis , sidekiq solve issue? wanted pipe grep
can't in redis-cli
interactive shell...
ideas?
there 2 instances different environments running. removed staging, production worked fine. guess wasn't set correctly.
Comments
Post a Comment