ruote tmp/log_2012-11-01.html

2012-11-01 00:59:11 utc mburnett i got my little experiment with killing workers to work, i was also killing associated participant (sleepers) and hadn't setup timeouts
2012-11-01 01:08:06 utc jmettraux oh, cool
2012-11-01 01:09:36 utc mburnett yeah, this is a great library..also, I really like that you wrote it as a library rather than a direct service
2012-11-01 01:24:48 utc jmettraux trying to not get too much in the way...
2012-11-01 09:27:47 utc jmettraux mburnett: about submitting jobs to a grid engine and waiting for them to complete
2012-11-01 09:28:48 utc jmettraux maybe looking at ruote-amqp and how it features participants/listeners might help (
2012-11-01 09:29:08 utc jmettraux s/listener/receiver/
2012-11-01 09:36:34 utc jmettraux basically the participant is signing the outgoing "workitem" with the string version of its "fei" (flow expression id), and the external system is supposed, when it replies, to include that identifier
2012-11-01 09:36:58 utc jmettraux so that the receiver, in the engine, knows what expression / workflow instance to resume
2012-11-01 20:56:49 utc kindjal Trying hard to find a working example of ruote + amqp + rabbit...
2012-11-01 20:56:55 utc kindjal finding lots of examples that don't work
2012-11-01 20:57:09 utc kindjal or say "ruote-amqp" but mean "RuoteAMQP" which seem different
2012-11-01 20:57:36 utc kindjal Any hints?
2012-11-01 20:58:14 utc phaeron we have a working solution with ruote-amqp and rabbitmq and python participant
2012-11-01 20:58:15 utc phaeron s
2012-11-01 20:58:26 utc kindjal is it on github?
2012-11-01 20:58:51 utc phaeron lbt: ping where is the last place you put boss-bundle
2012-11-01 20:58:58 utc phaeron kindjal: will rpm packages do ?
2012-11-01 20:59:03 utc kindjal sure
2012-11-01 20:59:05 utc lbt umm
2012-11-01 20:59:39 utc phaeron
2012-11-01 20:59:47 utc phaeron kindjal: boss and boss-bundle there
2012-11-01 20:59:51 utc lbt
2012-11-01 20:59:57 utc phaeron it's underdocumented unforunately
2012-11-01 21:00:06 utc kindjal phaeron: I'll take a look. Thanks!
2012-11-01 21:00:22 utc phaeron welcome
2012-11-01 21:00:30 utc phaeron I gtg now but I will login later
2012-11-01 21:00:52 utc mburnett jmettraux: thanks for the suggestion about amqp, i'm looking into it now
2012-11-01 21:01:07 utc jmettraux kindjal: for ruote-amqp examples that work, I know for sure that the ruote-amqp specs do pass, so those specs (and the README) should be OK
2012-11-01 21:01:14 utc lbt
2012-11-01 21:01:15 utc jmettraux mburnett: you're welcome
2012-11-01 21:02:03 utc lbt mmm - didnt' read the surrounding discussion either
2012-11-01 21:02:16 utc kindjal jmettraux: I should just read the ./spec, shouldn't I…. I'll do that too.
2012-11-01 21:02:39 utc jmettraux kindjal: the ruote-amqp readme and then the spec
2012-11-01 21:03:28 utc kindjal jmettraux: Yes, I've been over the read me… but being new to ruote, and not very experienced with Ruby at all, it's difficult to put together how it all works to build something new.
2012-11-01 21:03:32 utc jmettraux and if you want to make a particular example "work", we can help, but you'd need to tell us which example
2012-11-01 21:03:43 utc kindjal I'll get there.
2012-11-01 21:04:18 utc jmettraux I was afraid of that, people learning wide instead of deep and then labelling it all as "not working" :-(
2012-11-01 21:05:00 utc jmettraux also if you spot issues, we can fix them
2012-11-01 21:06:05 utc kindjal I'm sorry, I really am doing that aren't I.
2012-11-01 21:06:06 utc kindjal My fault
2012-11-01 21:06:10 utc lbt as a new-to-ruby guy I had problems getting up to speed with it all; I recall thinking some of the examples were a triumph of ruby-minimalism over clarity of understanding :)
2012-11-01 21:06:28 utc jmettraux lol
2012-11-01 21:06:40 utc lbt idoms should be banned in tutorials :)
2012-11-01 21:07:02 utc jmettraux the next step is to ban them from all code
2012-11-01 21:07:03 utc lbt or mentioned as elegant usage styles
2012-11-01 21:07:20 utc lbt I love them in real code
2012-11-01 21:07:23 utc kindjal phaeron left… but I see his "boss" app uses RuoteAMQP, which is not the same as Ruote::Amqp is it?
2012-11-01 21:07:39 utc lbt kindjal: phaeron and I work together
2012-11-01 21:07:45 utc jmettraux +1 for commented out "this code, in layman Ruby, would look like"
2012-11-01 21:08:03 utc kindjal lbt: what's the story with RuoteAMQP?
2012-11-01 21:08:07 utc lbt kindjal: python
2012-11-01 21:08:21 utc kindjal but boss is #!/usr/bin/ruby?
2012-11-01 21:08:28 utc lbt *g*
2012-11-01 21:08:39 utc lbt that's standard protocols for you
2012-11-01 21:09:01 utc lbt so we have boss/ruby talk over amqp to participants/python
2012-11-01 21:09:18 utc lbt we choose to do that btw
2012-11-01 21:09:35 utc kindjal umm
2012-11-01 21:09:36 utc kindjal
2012-11-01 21:09:36 utc kindjal ?
2012-11-01 21:09:54 utc kindjal which is not
2012-11-01 21:09:55 utc jmettraux argh, out of date rdoc, I should simply nuke it
2012-11-01 21:10:25 utc kindjal I am so confused
2012-11-01 21:10:30 utc lbt oops - sorry, my mistake
2012-11-01 21:10:44 utc kindjal eg: $engine, :launchitems => true )
2012-11-01 21:10:51 utc kindjal that is not "how it is done" with ruote-amqp
2012-11-01 21:10:55 utc jmettraux sorry, this is all outdated
2012-11-01 21:11:05 utc jmettraux is the "source"
2012-11-01 21:11:19 utc kindjal I so want a working example that makes use of that.
2012-11-01 21:11:30 utc jmettraux listed via
2012-11-01 21:12:15 utc kindjal yes, that's how I got to ruote-amqp, but then you go googling for "ruote ruote-amqp examples" and you get a bunch of github projects that say "ruote-amqp" but actually use RuoteAMQP
2012-11-01 21:12:37 utc kindjal I'll just try the spec examples and read me from ruote-amqp
2012-11-01 21:13:03 utc kindjal btw, I'm super excited about ruote, and yes, I immediately jumped into a complex example I want to build...
2012-11-01 21:13:10 utc kindjal just want to say "this looks so cool!"
2012-11-01 21:13:10 utc kindjal ;)
2012-11-01 21:13:42 utc jmettraux was looking at but the example by Hartog it points out is "not responding" :-(
2012-11-01 21:14:45 utc jmettraux ACTION adds to his todo: "write pong example for ruote-amqp"
2012-11-01 21:17:41 utc jmettraux lbt: if you come across those "too elegant to be readable" examples, please fire an issue report ;-)
2012-11-01 21:17:50 utc lbt will do :)
2012-11-01 21:17:53 utc lbt and kindjal: apologies for confusing you too
2012-11-01 21:18:06 utc kindjal lbt: no worries
2012-11-01 21:18:38 utc lbt I will vouch for ruote over amqp as being a great solution
2012-11-01 21:19:02 utc kindjal that's what my gut tells me… I just need to make it work! ;)
2012-11-01 21:48:18 utc kindjal if a participant launches a grid job, say via SGE, and the job takes days, how does ruote know the job's health... if it dies, we want to know, if it's complete, we want to progress.
2012-11-01 21:52:14 utc lbt so the participant takes the jobdesc from amqp; launches the 'grid job' and stores the workitem. it then exits
2012-11-01 21:52:40 utc lbt when your job is done the workitem is loaded and sent back to ruote
2012-11-01 21:52:58 utc lbt if you reboot the client and the job restarts, that's fine
2012-11-01 21:53:31 utc lbt that's one way you could do it
2012-11-01 21:53:46 utc lbt the "job is done" is up to you
2012-11-01 21:54:03 utc kindjal so the grid job would hold the responsibility to say "I'm done"...
2012-11-01 21:54:17 utc kindjal but if it died, how would ruote know, so that the workflow could restart it?
2012-11-01 21:54:23 utc kindjal it'd have to have a heartbeat monitor.
2012-11-01 21:54:24 utc lbt you're asking the wrong way round
2012-11-01 21:54:42 utc lbt you don't design your systems around ruote as such
2012-11-01 21:54:59 utc lbt you design your systems and use ruote to communicate
2012-11-01 21:55:16 utc lbt if you would like a heartbeat for other reasons, then have one
2012-11-01 21:55:29 utc kindjal I guess I'm trying to learn what ruote "does by itself"...
2012-11-01 21:56:12 utc lbt it asks systems to do things and enables you to flexibly do things based on the results
2012-11-01 21:57:24 utc lbt eg we use it to close bugs when QA tests pass if the changelog mentions "FIXES" a bug and it's built in a certain project
2012-11-01 21:57:57 utc kindjal I think I understand what you're saying...
2012-11-01 21:58:05 utc kindjal I think.
2012-11-01 21:58:07 utc kindjal :)
2012-11-01 21:58:34 utc kindjal If I have remote processes as 'workers', I'll need to know when they die, or error, or complete...
2012-11-01 21:58:48 utc kindjal and I think I'll have to build these bits… and then communicate via ruote to "do the right thing"
2012-11-01 21:58:58 utc kindjal I think
2012-11-01 21:59:03 utc kindjal I'll go work on it
2012-11-01 21:59:05 utc kindjal thanks.
2012-11-01 21:59:26 utc lbt ping me if you want to bounce ideas about
2012-11-01 21:59:35 utc kindjal Thanks, I probably wil..
2012-11-01 21:59:43 utc lbt I tend to use ruote with multiple servers running different tasks
2012-11-01 22:00:01 utc lbt rather than small sw components/functions
2012-11-01 22:01:35 utc lbt so 'build an rpm' on one server; create an installable image on another; run tests on devices using that image on another; update DB according to results; notifications for success/fail
2012-11-01 22:02:33 utc lbt if the image builder goes down it's designed to come up and continue with it's own queue (image can take many minutest to build)
2012-11-01 22:03:06 utc lbt the db updated is more RPC - transaction is pretty instant
2012-11-01 22:03:58 utc kindjal I follow you.
2012-11-01 22:04:06 utc lbt we're not quite at the : pull from amqp; do work; ack message+respond atomically- AMQP flow yet (I don't think)
2012-11-01 22:04:51 utc lbt also we did some stress work at Nokia a year or so ago
2012-11-01 22:05:09 utc lbt and ran millions of processes or something
2012-11-01 22:05:24 utc lbt found some minor mem leaks but in general all went well
2012-11-01 22:05:53 utc lbt we need to fixup the "admin interface" which can be a pita
2012-11-01 22:06:22 utc lbt join #meego-boss btw
2012-11-01 22:06:38 utc lbt we have an irc logger which is sometimes interesting
2012-11-01 22:08:58 utc lbt this is one of our more complex processes:
2012-11-01 22:09:50 utc lbt we don't like jmettraux to see what twisted things we do with his baby though ...
2012-11-01 22:15:04 utc phaeron I showed him that one before and he said it was ok :D
2012-11-01 22:17:24 utc kindjal Thanks for the tips. I'm going to go dig into it more. ;)
2012-11-01 22:49:38 utc jmettraux ok #meego-boss is super noisy
2012-11-01 22:50:10 utc jmettraux just hoping ruote specific conversation stay here
2012-11-01 22:50:22 utc jmettraux lbt thanks so much for your help here :-)
2012-11-01 22:52:40 utc phaeron yes it reports on every event that happens in the build system
2012-11-01 22:54:09 utc lbt jmettraux: np - the #meego-boss is interesting every now and again:
2012-11-01 23:49:58 utc jmettraux cool stuff