ruote tmp/log_2011-09-22.html

2011-09-22 00:17:20 utc pbh_ (though i'm not sure if something like Worker#add_hook('next_participant', &callback) would be more natural, or something like :on_next_participant in the process definition itself)
2011-09-22 00:19:36 utc jmettraux those hooks would be for production purposes ?
2011-09-22 00:20:59 utc pbh_ well, i guess i'm just imagining two use cases, the case on the mailing list where creation of participants went out of control, and the testing use case where you want to see if a new participant was reached
2011-09-22 00:21:29 utc pbh_ but i'm not sure whether it makes sense to add hooks or not, it might just be more complexity (and possible error cases) than not having them at all
2011-09-22 00:21:43 utc jmettraux create of participants went out of control ?
2011-09-22 00:22:17 utc jmettraux in the process definition itself, the "listen" expression can be of use: http://ruote.rubyforge.org/exp/listen.html
2011-09-22 00:22:32 utc pbh_ http://groups.google.com/group/openwferu-users/browse_thread/thread/1e2f3e16b9400b74 was what i was referring to
2011-09-22 00:23:18 utc pbh_ though it sounds like they're checking the "history" (in storage?) to check if things look odd, which sounds pretty similar to having a hook
2011-09-22 00:23:43 utc jmettraux participants themselves are hooks
2011-09-22 00:24:42 utc jmettraux I think you want/need a engine.wait_for(:next_participant) or something like that
2011-09-22 00:24:47 utc pbh_ ah, so it does look like listen would be very similar to adding a hook
2011-09-22 00:26:11 utc pbh_ yeah, i think you are right
2011-09-22 00:26:35 utc pbh_ if anything just wait_for(:next_participant)
2011-09-22 00:26:56 utc jmettraux I should generalize wait_for so that your can specifiy a message name, like engine.wait_for('dispatched') which would be equivalent to wait_for(:next_participant)
2011-09-22 00:27:49 utc pbh_ i think that would be totally general too, because any transition you would test would be a transition to a participant
2011-09-22 00:28:02 utc pbh_ since on_error, on_timeout, if statements, etc. all go to a participant
2011-09-22 00:28:53 utc jmettraux on_error, on_timeout, on_cancel may point to subprocesses too
2011-09-22 00:29:44 utc pbh_ true, though that'll still be a subprocess that is implemented via some sequence/concurrence etc of participants, no?
2011-09-22 00:30:20 utc pbh_ i guess another workaround for testing purposes would be that you could continuously wait_for(1) until you are in a different participant
2011-09-22 00:34:11 utc jmettraux https://gist.github.com/1233739 if the error expression is called, there will be no participant involved
2011-09-22 00:35:33 utc jmettraux in tests, I always know in advance which participant I'll reach, so I do engine.wait_for(:toto)
2011-09-22 00:36:16 utc jmettraux if I don't care about the participant name, I do engine.wait_for(x) where x is the number of msgs required to reach the next participant
2011-09-22 00:36:33 utc jmettraux if I introduce engine.wait_for('dispatched') I can simplify the counting process
2011-09-22 00:36:42 utc pbh_ yeah, that seems like the easiest thing
2011-09-22 00:37:51 utc pbh_ oh, while i'm at it, another minor point of confusion
2011-09-22 00:38:40 utc pbh_ i think the documentation listed a few things that were new as of 2.2.1 (though it could have been as of 2.1)
2011-09-22 00:39:02 utc pbh_ is the current version of ruote for ruote-kit 2.2.0, 2.2.1, or something else, and is 2.2.1 released yet?
2011-09-22 00:41:23 utc jmettraux https://rubygems.org/gems/ruote
2011-09-22 00:41:40 utc jmettraux I should probably add that info on the front page
2011-09-22 00:41:49 utc jmettraux (lots of todo points from you today)
2011-09-22 00:42:09 utc pbh_ ha, i hope i'm not just a source of work!
2011-09-22 00:44:01 utc jmettraux feedback is always welcome, so thanks !
2011-09-22 00:44:02 utc pbh_ also, i mostly prefer the alternative names you introduced in 2.2.1, which is part of the reason for my curiosity
2011-09-22 00:44:52 utc jmettraux there are quite a few people out there using master ruote, thanks to bundler it's very easy (it's also making me lazy about releasing)
2011-09-22 00:48:35 utc pbh_ ah, interesting
2011-09-22 00:49:38 utc pbh_ i was considering that, though i was a little confused as to (a) whether it would break any of the related tools (ruote-kit, ruote-redis) and (b) whether master always passes the associated tests (i haven't looked deeply at ci to check whether things are automatically moved between branches based on tests or anything)
2011-09-22 00:50:58 utc pbh_ (that said, i think everyone is getting a lazy about releasing now that github is out, i'm stuck using factory_girl's git version because they haven't released in forever)
2011-09-22 00:51:04 utc jmettraux I'm trying to move all the subprojects together, it's harder for subprojects owned by other people (I'm trying to let them follow)
2011-09-22 00:56:53 utc pbh_ anyway, have to run, thanks for your help as always
2011-09-22 00:58:03 utc jmettraux you're welcome, take care !
2011-09-22 07:41:58 utc lbt good morning jmettraux
2011-09-22 07:42:08 utc jmettraux lbt: good morning
2011-09-22 07:42:45 utc lbt if you have a minute I'd like to enhance the amqp error handling
2011-09-22 07:43:03 utc jmettraux I'll try to help the best I can
2011-09-22 07:43:25 utc lbt so when a remote participant has an exception in the consume
2011-09-22 07:43:46 utc lbt I would like to behave as if a local one has an exception and take the process into error
2011-09-22 07:44:00 utc lbt that's reasonably easy to do remotely
2011-09-22 07:44:30 utc lbt trap the unhandled exception and do a reply with "something" (__error__?) flagged
2011-09-22 07:44:48 utc lbt then the issue is how to raise on the Worker side
2011-09-22 07:45:03 utc lbt I'm considering on_reply
2011-09-22 07:45:33 utc jmettraux why don't you simply raise in the receiver ?
2011-09-22 07:45:57 utc lbt the receiver is not in a worker ... it's in a storage
2011-09-22 07:46:14 utc jmettraux ?
2011-09-22 07:46:20 utc lbt sec
2011-09-22 07:48:03 utc lbt https://github.com/kennethkalmer/ruote-amqp/blob/master/lib/ruote-amqp/receiver.rb#L29
2011-09-22 07:48:09 utc lbt RuoteAMQP::Receiver.new(engine_or_storage)
2011-09-22 07:48:36 utc lbt actually it's not in a storage at the moment
2011-09-22 07:48:41 utc jmettraux ok
2011-09-22 07:49:06 utc lbt but I wasn't sure if raising in the receive was the right place from the worker PoB
2011-09-22 07:49:10 utc lbt PoV
2011-09-22 07:49:23 utc lbt the on_reply seems more logical
2011-09-22 07:49:25 utc lbt however
2011-09-22 07:49:41 utc lbt I did wonder about the receiver having a new msg type of error
2011-09-22 07:49:48 utc jmettraux no, you are right, raising in the receiver won't wrok
2011-09-22 07:49:51 utc jmettraux work
2011-09-22 07:49:55 utc jmettraux let me cook something
2011-09-22 07:50:36 utc lbt well, what are you thinking
2011-09-22 07:50:49 utc lbt I'm trying to maintain grow familiarity with the code
2011-09-22 07:51:07 utc lbt so a suggestion as to the correct area ... and why
2011-09-22 07:51:11 utc lbt would be nice
2011-09-22 07:51:22 utc jmettraux looking for an example
2011-09-22 07:52:19 utc jmettraux https://github.com/jmettraux/ruote-beanstalk/blob/master/lib/ruote/beanstalk/receiver.rb#L120-145
2011-09-22 07:52:23 utc lbt https://github.com/jmettraux/ruote/blob/master/lib/ruote/exp/fe_error.rb
2011-09-22 07:52:29 utc jmettraux the beanstalk equivalent
2011-09-22 07:52:41 utc lbt great .. looking
2011-09-22 07:53:43 utc jmettraux our ruote-amqp only deals with receive(item) and launch(item), this beanstalk implementation knows how to emit errors when receiving
2011-09-22 07:54:02 utc lbt *nod* ... that's perfect
2011-09-22 07:54:03 utc jmettraux if you feel like porting that to ruote-amqp
2011-09-22 07:54:07 utc lbt yes please
2011-09-22 07:54:54 utc lbt now... would using __error__ be the right thing ... it seems not to be used much and it's in the fields space which seems a bit userspace-ey
2011-09-22 07:55:23 utc lbt so I didn't want to ressurect it if you'd rather it died :)
2011-09-22 07:56:02 utc jmettraux not sure, the way you see in ruote-beanstalk is the right one (90%)
2011-09-22 07:56:29 utc lbt type == 'error'
2011-09-22 07:56:34 utc lbt OK
2011-09-22 07:56:41 utc jmettraux BTW, are those people Meego people ? https://github.com/kennethkalmer/ruote-amqp/issues/5
2011-09-22 07:56:53 utc lbt yes .. on the Intel side
2011-09-22 07:57:09 utc lbt he's right ... I think I mentioned that many moons ago
2011-09-22 07:57:22 utc jmettraux What did Kenneth reply ?
2011-09-22 07:58:03 utc lbt I honestly don't recall - I suspect it was left as "it works"
2011-09-22 07:58:09 utc jmettraux ah ok
2011-09-22 07:58:34 utc lbt now we have amqp protocol changes coming up then we should review the implementation
2011-09-22 07:58:46 utc jmettraux ok
2011-09-22 07:59:06 utc lbt there are also some other things like only accepting the incoming message after it's gone to disk
2011-09-22 07:59:14 utc lbt so read, write, ack
2011-09-22 08:00:47 utc lbt nb.... http://autodoc.meego.com/boss/processes/Project/CE/Trunk/Testing/
2011-09-22 08:01:02 utc lbt live update every minute of process running on backend
2011-09-22 08:01:22 utc jmettraux very cool
2011-09-22 08:02:02 utc jmettraux extremely nice
2011-09-22 08:02:08 utc lbt my next thing ... 'soon' ... is to try to put a table into a pdef of 'participant/version constraint'
2011-09-22 08:02:50 utc lbt so I can run participant 1.0 and 2.0 and have them 'peek' at the wi to accept/refuse
2011-09-22 08:03:17 utc lbt we may need to do that differently in the amqp - read/peek/nack
2011-09-22 08:03:23 utc jmettraux participants have an #accept?(wi) optional method
2011-09-22 08:03:27 utc lbt in ruby it would be the accept?
2011-09-22 08:03:29 utc lbt yeah
2011-09-22 08:03:57 utc lbt so I'd like them to use the same (best practice) data struct in the WI
2011-09-22 08:04:52 utc lbt maybe they can have an accept_version? which is called by accept? if present and by amqp peek in remotes
2011-09-22 08:05:20 utc lbt probably that's all userside until it works :)
2011-09-22 08:06:01 utc lbt anyhow... back to the error thing. My main concern was a workable solution by the time the next release happens
2011-09-22 08:06:17 utc lbt looks like there's no problem there if it's already in beanstalk :)
2011-09-22 08:06:38 utc lbt I may need to tweak amqp of course - but that's not too hard
2011-09-22 08:07:33 utc jmettraux it seems like you need a convention, "hey this message contains an error, let's propagate it to ruote"
2011-09-22 08:08:27 utc lbt yes - at the remote end it will catch an exception around the consume and normally sets a wi field
2011-09-22 08:09:12 utc jmettraux ok
2011-09-22 08:09:41 utc lbt https://meego.gitorious.org/meego-infrastructure-tools/ruote-amqp-pyclient/blobs/master/RuoteAMQP/participant.py#line75
2011-09-22 08:10:43 utc jmettraux python code looks nice
2011-09-22 08:10:53 utc jmettraux very cool
2011-09-22 08:11:26 utc lbt https://meego.gitorious.org/meego-infrastructure-tools/boss-standard-workflow/trees/master/participants
2011-09-22 08:11:53 utc jmettraux <3
2011-09-22 08:12:30 utc lbt yeah ... we're still at the "where do we split things up" stage
2011-09-22 08:12:42 utc lbt logic in the participant or the process
2011-09-22 08:12:49 utc lbt it's a hard question
2011-09-22 08:13:07 utc jmettraux +1
2011-09-22 08:13:57 utc lbt typically I'm looking to see if 2 users/groups would choose the participant in their 'local' workflow
2011-09-22 08:14:14 utc lbt eg check_mentions_bug
2011-09-22 08:14:31 utc lbt will refuse a package where the changelog has no bug ref
2011-09-22 08:14:48 utc lbt some people like that .. some don't ... local convention
2011-09-22 08:16:06 utc jmettraux you're "touching" heterogenous populations with your tool
2011-09-22 08:16:09 utc jmettraux must be epic
2011-09-22 08:16:37 utc lbt well... the objective is to make it the "standard" build/qa approach for meego
2011-09-22 08:16:48 utc lbt (or whatever comes after :) )
2011-09-22 08:17:41 utc lbt the nice thing is the coupling is so loose and generally handled by BOSS/ruote
2011-09-22 08:20:18 utc lbt hmm ... can I get cc'ed on all amqp issues?
2011-09-22 08:20:52 utc lbt sorry ... ruote
2011-09-22 08:22:24 utc jmettraux on all the ruote-ampq issues ?
2011-09-22 08:22:37 utc jmettraux I guess you only have to watch kenneth's ruote-amqp project
2011-09-22 08:22:45 utc jmettraux https://github.com/kennethkalmer/ruote-amqp/
2011-09-22 08:22:56 utc jmettraux let me try something
2011-09-22 08:23:44 utc lbt ah, I guess that makes sense
2011-09-22 08:24:22 utc jmettraux added a mention to you in the issue
2011-09-22 08:24:30 utc jmettraux should generate a notification for you
2011-09-22 08:25:13 utc lbt watching mail
2011-09-22 08:25:56 utc jmettraux or your https://github.com/lbt "inbox"
2011-09-22 08:26:57 utc lbt got mail - I suspect 'watch' did it
2011-09-22 08:27:34 utc lbt right ... standup ... then wife->dentist .... I may catch you later :)
2011-09-22 08:27:48 utc lbt have a good one and thanks ... as usual
2011-09-22 08:28:11 utc jmettraux ok, take care !
2011-09-22 08:28:14 utc jmettraux thanks to you
2011-09-22 22:33:46 utc pbh_ hmmm
2011-09-22 22:34:16 utc pbh_ is participant registration with the engine only used when launching a process? (vs when executing the process instance with a worker)
2011-09-22 22:35:49 utc pbh_ ACTION is trying to figure out why RuoteKit only registers participants in config/initializers/ruote-kit.rb when it is not in the worker rake task
2011-09-22 22:36:58 utc jmettraux hello, that's debatable; we're trying to put the configuration under conf/ and the worker task is simple
2011-09-22 22:37:10 utc jmettraux granted, one could read some conf/ from the rake task
2011-09-22 22:37:54 utc jmettraux feel free to do it like you feel is right
2011-09-22 22:38:05 utc pbh_ ah, i guess what i'm trying to figure out
2011-09-22 22:38:06 utc jmettraux well, always
2011-09-22 22:38:27 utc pbh_ is when i run rake exec ruote:run_worker
2011-09-22 22:38:44 utc pbh_ is it figuring out which participants to dispatch based on data that's already in the engine?
2011-09-22 22:38:56 utc jmettraux yes
2011-09-22 22:39:36 utc pbh_ so if i want to add a new participant implementation to ruotekit
2011-09-22 22:40:00 utc pbh_ i presumably need to require it somewhere so that it can get instantiated
2011-09-22 22:40:37 utc jmettraux currently, you'd need to add it to the initializer and then [re]start so that the [new] set of participants gets registered
2011-09-22 22:41:12 utc pbh_ that seems reasonable
2011-09-22 22:41:37 utc jmettraux BTW, I added https://github.com/jmettraux/ruote/commit/056731f935102937d5af94bb1873cfa75a4d29ff example: https://github.com/jmettraux/ruote/blob/master/test/unit/ut_3_wait_logger.rb#L41-58
2011-09-22 22:42:01 utc pbh_ oh, excellent
2011-09-22 22:42:54 utc pbh_ that should make testing easier
2011-09-22 22:45:38 utc pbh_ so to check that you went to the right next participant
2011-09-22 22:46:03 utc pbh_ you'd do Engine#wait_for('dispatch') and then assert that the work item's participant was the expected one?
2011-09-22 22:46:14 utc pbh_ err, .participant_name
2011-09-22 22:51:21 utc jmettraux yes
2011-09-22 22:51:25 utc jmettraux there is a slight catch
2011-09-22 22:52:06 utc jmettraux 'dispatch' is the message that triggers the dispatching, the next message is 'dispatched', but for most participant it occurs right after the consume/on_workitem method is called
2011-09-22 22:52:15 utc jmettraux so it might be a bit "early" in some instances
2011-09-22 22:54:53 utc pbh_ ah, that makes sense
2011-09-22 22:56:01 utc jmettraux sometimes in tests, especially with Ruote::StorageParticipant, I do "@engine.wait_for(:toto_participant); @engine.wait(1)" so that it waits until the "dispatched"
2011-09-22 22:56:10 utc jmettraux now we can do @engine.wait_for('dispatched')
2011-09-22 23:01:52 utc pbh_ excellent
2011-09-22 23:02:04 utc pbh_ btw, just wrote my first participant
2011-09-22 23:02:33 utc jmettraux great, what does it do ?
2011-09-22 23:02:41 utc pbh_ ha, not much
2011-09-22 23:03:28 utc pbh_ it just adds a field to the workitem with the key ActiveSupport::SecureRandom.hex(5) and a value of ActiveSupport::SecureRandom.hex(10)
2011-09-22 23:04:06 utc pbh_ but its working on rails 3.0.x + a Ruote::Redis::Storage + ruote-kit + a rake worker locally
2011-09-22 23:04:40 utc jmettraux way cool
2011-09-22 23:04:55 utc pbh_ btw, presumably on_error gets triggered if a participant throws any exception?
2011-09-22 23:05:05 utc pbh_ or exits with some non-zero error code?
2011-09-22 23:05:17 utc jmettraux if it throws yes
2011-09-22 23:07:31 utc pbh_ oh, also
2011-09-22 23:07:45 utc pbh_ is on_reply a convention, or is it actually called by the worker
2011-09-22 23:08:07 utc pbh_ e.g., if i had any method on my participant that called reply_to_engine with the workitem and its associated metadata, would it work just fine?
2011-09-22 23:08:56 utc jmettraux it should
2011-09-22 23:09:32 utc jmettraux on_reply is mostly used by "proxy participants", ie participants that are gateway to 'further' systems, like a amqp queue or ...
2011-09-22 23:09:59 utc jmettraux it's called when the associated receiver 'receives' the workitem and the participant wants to have a last say
2011-09-22 23:10:32 utc jmettraux if the communication is completely encapsulated in the #on_workitem/#consume, then #on_reply is not necessary
2011-09-22 23:12:00 utc pbh_ ah, yeah, i still find the receiver/listener bits a little confusing, but i think i'll probably be completely avoiding them (because the interaction will be through rails rather than AMQP or anything)
2011-09-22 23:12:27 utc jmettraux understood
2011-09-22 23:13:48 utc jmettraux with AMQP and co, participants are a bit like average people, they got the the postbox or the post office to send their letters, but a receiver (the postman) delivers the letters to their home
2011-09-22 23:14:08 utc pbh_ yeah, it seems like AMQP is like a transactional SMTP
2011-09-22 23:14:20 utc jmettraux nicely said
2011-09-22 23:14:50 utc pbh_ so it seems like there are two ways to handle a delayed web/human type participant with rails
2011-09-22 23:15:10 utc pbh_ one option is to use the StorageParticipant and then do some kind of lookup from a rails controller
2011-09-22 23:16:02 utc pbh_ the other option is to code a custom participant that gets the workitem, dumps it to activerecord somewhere, and then later after some interaction with some rails controller, the workitem is taken from the db and reply_to_engine is called on the custom participant
2011-09-22 23:17:23 utc pbh_ will both of those work?
2011-09-22 23:17:37 utc jmettraux yes
2011-09-22 23:17:52 utc pbh_ (and in the storageparticipant case, it seems like there are a few lookup options, is there some sensible way to do things?)
2011-09-22 23:18:19 utc pbh_ (e.g., ruote-on-rails seems to lookup items for a given user by assuming that a storageparticipant with a given name is the same as the logged in user name)
2011-09-22 23:18:30 utc jmettraux I tend to use the StorageParticipant for humans, though it won't play nice with will_paginate and kaminari
2011-09-22 23:18:49 utc jmettraux so dumping to one's own model sometimes makes sense
2011-09-22 23:19:23 utc jmettraux most of the time you want "role == participant name", like in "ceo", "accountant", ...
2011-09-22 23:19:32 utc jmettraux where one role is mapped to multiple users
2011-09-22 23:19:44 utc jmettraux so you have to provide your own role/user mapping
2011-09-22 23:20:09 utc pbh_ though presumably there might be multiple actions for the same user, no?
2011-09-22 23:20:20 utc jmettraux yes
2011-09-22 23:20:28 utc pbh_ like, the ceo might have an approval controller and a budget controller
2011-09-22 23:20:54 utc pbh_ do you usually end up just reading all of the workitems for the role and then filtering them?
2011-09-22 23:21:00 utc jmettraux or you could have a unique TaskController and render a different form depending on the task
2011-09-22 23:21:09 utc jmettraux yes
2011-09-22 23:24:44 utc pbh_ ah, yeah i guess that's what ruote-on-rails does
2011-09-22 23:25:01 utc pbh_ a collect/flatten on RuoteKit.engine.storage_participant.by_participant(pname)
2011-09-22 23:26:36 utc pbh_ (though i guess just a .query should work too)
2011-09-22 23:27:52 utc pbh_ (which from the source looks like a thin wrapper on top of select/collect)
2011-09-22 23:30:16 utc jmettraux some storage provide better implementations (ie not "load all in memory and weed out the necessary" approach)
2011-09-22 23:31:24 utc jmettraux s/storage/storages/
2011-09-22 23:33:42 utc pbh_ ah, interesting
2011-09-22 23:34:05 utc jmettraux at least I/we try to do it this way