2011-05-24 01:16:58 utc |
phaeron |
hello, I have a weird issue with a process. a participant being called twice successfully doesn't get called the the third time. |
2011-05-24 01:18:56 utc |
phaeron |
http://pastie.org/1964063 |
2011-05-24 01:19:21 utc |
phaeron |
it goes fine until .. echo "IN SUC" |
2011-05-24 01:19:29 utc |
phaeron |
but never calls the next line |
2011-05-24 01:30:15 utc |
jmettraux |
hello |
2011-05-24 01:31:20 utc |
jmettraux |
phaeron: which participant it it ? |
2011-05-24 01:32:08 utc |
phaeron |
jmettraux: sorry almost slept |
2011-05-24 01:32:13 utc |
phaeron |
4:30 am here :) |
2011-05-24 01:32:24 utc |
phaeron |
img_status_update |
2011-05-24 01:32:42 utc |
jmettraux |
does the process run into some error ? |
2011-05-24 01:32:51 utc |
phaeron |
called twice successfully inside do_build_image |
2011-05-24 01:33:01 utc |
phaeron |
no it just sits waiting there |
2011-05-24 01:33:04 utc |
phaeron |
no errors |
2011-05-24 01:33:28 utc |
jmettraux |
where exactly in do_build_image ? |
2011-05-24 01:34:21 utc |
phaeron |
it returns successfully from do_build_image , I see the BUILD IMAGE RETURNED , and the IN SUC |
2011-05-24 01:35:05 utc |
jmettraux |
but that's not in do_build_image |
2011-05-24 01:35:12 utc |
jmettraux |
I don't understand |
2011-05-24 01:35:44 utc |
phaeron |
do_build_image is a suprocess |
2011-05-24 01:35:48 utc |
phaeron |
*subprocess |
2011-05-24 01:35:53 utc |
phaeron |
as you can see of course |
2011-05-24 01:36:03 utc |
jmettraux |
after IN SUC, there is no call to that subprocess |
2011-05-24 01:36:25 utc |
phaeron |
yes there is a call to img_status_update |
2011-05-24 01:36:28 utc |
jmettraux |
are you 100% that engine.process(wfid).errors is empty ? |
2011-05-24 01:36:36 utc |
phaeron |
but it never gets it |
2011-05-24 01:37:36 utc |
phaeron |
the errors directory is empty |
2011-05-24 01:37:55 utc |
phaeron |
and ruote-kit doesn't show anything either |
2011-05-24 01:38:02 utc |
phaeron |
any way else I can be sure |
2011-05-24 01:38:21 utc |
jmettraux |
ok |
2011-05-24 01:39:01 utc |
jmettraux |
may I see the code of img_status_update ? |
2011-05-24 01:39:26 utc |
phaeron |
sure it's in python , as it talks to ruote over amqp |
2011-05-24 01:39:37 utc |
jmettraux |
stop |
2011-05-24 01:39:41 utc |
phaeron |
:) |
2011-05-24 01:40:03 utc |
jmettraux |
are you sure the message got over from ruote over amqp ? |
2011-05-24 01:40:33 utc |
phaeron |
so it got there twice, the third time I am not sure , |
2011-05-24 01:41:00 utc |
phaeron |
rabbitmq doesn't seem to have any messages pending for that exchange / queue |
2011-05-24 01:41:05 utc |
jmettraux |
it'd be interesting to know where exactly it got lost |
2011-05-24 01:41:14 utc |
jmettraux |
also, when you say twice |
2011-05-24 01:41:23 utc |
phaeron |
yes I am tearing my hair out |
2011-05-24 01:41:31 utc |
jmettraux |
is it "each time I run this process, the third time it gets lost" |
2011-05-24 01:42:00 utc |
phaeron |
yes |
2011-05-24 01:42:13 utc |
phaeron |
which is very strange of course |
2011-05-24 01:42:38 utc |
jmettraux |
you could try to run the engine with RuoteKit.engine.noisy = true |
2011-05-24 01:42:49 utc |
jmettraux |
which will make it bleed its activity to the stdout |
2011-05-24 01:42:59 utc |
phaeron |
one second , I just noticed something interesting |
2011-05-24 01:43:13 utc |
phaeron |
rabbitmqctl list_consumers -p boss |
2011-05-24 01:43:28 utc |
jmettraux |
then if I were you, I'd add some debug output to the AmqpParticipant to double check a) did it receive the message b) did it put it succcessfully into the amqp rabbithole |
2011-05-24 01:43:42 utc |
phaeron |
showed three img_status_update consumers |
2011-05-24 01:43:51 utc |
phaeron |
restarting rabbitmq it shows one |
2011-05-24 01:43:52 utc |
phaeron |
.. |
2011-05-24 01:44:18 utc |
jmettraux |
could the process crash "consumers" ? |
2011-05-24 01:44:29 utc |
jmettraux |
so that the third time, there is nothing to consume ? |
2011-05-24 01:44:46 utc |
phaeron |
I am not sure how it happened. but now it works |
2011-05-24 01:45:09 utc |
phaeron |
it seems I got the ampq connections into a weird state |
2011-05-24 01:45:18 utc |
jmettraux |
maybe you guys have implemented error logs for your python consumers ? |
2011-05-24 01:45:19 utc |
jmettraux |
ok |
2011-05-24 01:45:21 utc |
jmettraux |
great |
2011-05-24 01:45:27 utc |
jmettraux |
solved |
2011-05-24 01:45:28 utc |
phaeron |
sorry for the noise |
2011-05-24 01:45:38 utc |
jmettraux |
no worries, glad to help a fellow developer |
2011-05-24 01:45:38 utc |
phaeron |
yes there are error logs and dumps and everything |
2011-05-24 01:45:52 utc |
phaeron |
and it always worked twice , but the third time disappeared |
2011-05-24 01:46:14 utc |
phaeron |
anyway |
2011-05-24 01:46:17 utc |
jmettraux |
you should stop saying "disappeared" |
2011-05-24 01:46:42 utc |
phaeron |
ok :) , well rabbitmq sent it to a "phantom" consumer |
2011-05-24 01:46:47 utc |
jmettraux |
ah ok |
2011-05-24 01:47:02 utc |
jmettraux |
thanks |
2011-05-24 01:49:15 utc |
phaeron |
probably a crash at some point and the consumer was not unregistered |
2011-05-24 01:49:33 utc |
phaeron |
because with clean shutdown the consumer is no longer there in rabbitmq |
2011-05-24 01:50:36 utc |
phaeron |
jmettraux: another question , I think lbt had reported already the '${__result__} == false' issue ? |
2011-05-24 01:50:47 utc |
phaeron |
false is a json False |
2011-05-24 01:53:50 utc |
phaeron |
I can only find https://github.com/jmettraux/ruote/issues/27 |
2011-05-24 01:56:29 utc |
phaeron |
ok you're probably busy |
2011-05-24 02:02:30 utc |
jmettraux |
phaeron: try "$__result__ == false" |
2011-05-24 02:04:36 utc |
jmettraux |
if the workitem field holds the value false, then '${__result__} == false' will be substituted to 'false == false' which should yield true |
2011-05-24 02:04:58 utc |
phaeron |
I will, thanks, probably needs more debugging output to check it |
2011-05-24 02:05:04 utc |
phaeron |
but now it 5 am .. |
2011-05-24 02:05:10 utc |
phaeron |
thanks , and sorry for the noise |
2011-05-24 02:05:16 utc |
jmettraux |
no worries |
2011-05-24 11:06:13 utc |
jmettraux |
jeffmess: hello and welcome to #ruote |
2011-05-24 11:25:29 utc |
jeffmess |
jmettraux: thanks! :) |
2011-05-24 11:40:42 utc |
jmettraux |
tosch_le: critical mass am Freitag ! My bike is ready |
2011-05-24 12:32:14 utc |
tosch_le |
jmettraux: great! have fun! |
2011-05-24 22:30:53 utc |
jmettraux |
kitplummer: hello and welcome to #ruote |
2011-05-24 22:33:46 utc |
kitplummer |
thanks jmettraux - looks like your effort is going to come in handy for me. ;) |
2011-05-24 22:33:54 utc |
kitplummer |
much appreciated. |
2011-05-24 22:33:59 utc |
jmettraux |
ah great |
2011-05-24 22:34:00 utc |
jmettraux |
enjoy |
2011-05-24 22:34:17 utc |
kitplummer |
i have a lot to learn. |
2011-05-24 22:36:17 utc |
jmettraux |
IRC and mailing list should help |
2011-05-24 22:36:55 utc |
kitplummer |
i'm sure it will. need to get my head wrapped around ruote-backend to - so i can use resque workers. |
2011-05-24 22:37:42 utc |
jmettraux |
ruote-backend : are you working for sandbox ? |
2011-05-24 22:38:01 utc |
kitplummer |
sandbox? |
2011-05-24 22:38:16 utc |
jmettraux |
he's the author of ruote-backend |
2011-05-24 22:38:36 utc |
kitplummer |
nopeā¦but, hoping to use it. |
2011-05-24 22:40:10 utc |
kitplummer |
this is what i'm looking at: https://github.com/dolores/ruote-backend |
2011-05-24 22:40:37 utc |
jmettraux |
understood |
2011-05-24 22:45:25 utc |
jmettraux |
the author sometimes log in here as "sandbox", I guess that if you have issues with that storage, you can use the "issues" facilities on github to communicate |
2011-05-24 22:47:43 utc |
kitplummer |
sounds good, will do. |
2011-05-24 22:48:34 utc |
jmettraux |
let me lookup a conversation I had with him, I'm not sure he wants to maintain that project for a wider audience (wider than him and his company) |
2011-05-24 22:49:43 utc |
kitplummer |
hmmn, that might explain why the gem wasn't in rubygems.org |
2011-05-24 22:50:52 utc |
jmettraux |
can't find the conversation anymore |
2011-05-24 22:51:30 utc |
kitplummer |
no worries, thanks for the tip tho. |
2011-05-24 23:05:21 utc |
kitplummer |
is there another resque/redis backend? |
2011-05-24 23:10:16 utc |
jmettraux |
there is ruote-redis, but it's not resque driven |
2011-05-24 23:10:30 utc |
jmettraux |
http://ruote.rubyforge.org/source.html |
2011-05-24 23:11:01 utc |
kitplummer |
k. i could probably be convinced that we don't need resque. but, the worker stuff is so simple there. |
2011-05-24 23:11:26 utc |
jmettraux |
what's your goal ? Place work in a resque queue ? |
2011-05-24 23:11:34 utc |
jmettraux |
and have it picked by a consumer ? |
2011-05-24 23:11:59 utc |
kitplummer |
yup. distributed workers, listening to discrete queues. |
2011-05-24 23:12:18 utc |
jmettraux |
you can write a custom participant |
2011-05-24 23:12:30 utc |
kitplummer |
each phase of the workflow would create a job for a different external system interface. |
2011-05-24 23:12:57 utc |
jmettraux |
ruote-amqp does that with amqp |
2011-05-24 23:13:11 utc |
kitplummer |
hmmn. |
2011-05-24 23:13:16 utc |
jmettraux |
"place task on the queue, wait for reply, when reply is here, reply to engine" |
2011-05-24 23:13:22 utc |
kitplummer |
yep. |
2011-05-24 23:13:41 utc |
jmettraux |
storage != participant |
2011-05-24 23:13:48 utc |
kitplummer |
right. |
2011-05-24 23:14:37 utc |
kitplummer |
i'm assuming not all storage systems support multiple participants per queue. |
2011-05-24 23:16:42 utc |
jmettraux |
storage systems don't care about participants |
2011-05-24 23:17:08 utc |
jmettraux |
they care about the persistence of ruote, not about the distribution of work to participants |
2011-05-24 23:17:32 utc |
kitplummer |
got it. |
2011-05-24 23:18:56 utc |
kitplummer |
does the Ruote::Engine support remote registration? just looking through examples now. |
2011-05-24 23:20:32 utc |
jmettraux |
yes |
2011-05-24 23:22:15 utc |
jmettraux |
well, it depends on the storage, if it can be reached remotely, then yes |
2011-05-24 23:31:39 utc |
kitplummer |
cool, that's the mapping i was thinking about between storage and participantā¦and what is attractive for me - wrt resque. |
2011-05-24 23:32:00 utc |
kitplummer |
thanks for the info jmettraux |
2011-05-24 23:32:13 utc |
jmettraux |
ah, you're welcome |