ruote tmp/log_2013-01-16.html

2013-01-16 00:27:21 utc jmettraux side note, just wrote that: sorry for the noise
2013-01-16 01:11:23 utc randomcamel how extensible would you guys say Ruote is? we have our own systems for distributing and scheduling jobs out to workers, and I'm looking at whether it's possible to integrate Ruote with those things. I guess what would be needed would be callbacks or hooks to extend or replace Ruote events.
2013-01-16 01:12:02 utc jmettraux why not simply ruote participants and receivers that interface with your system?
2013-01-16 01:13:03 utc jmettraux à la or
2013-01-16 01:18:05 utc randomcamel when I looked into that a while ago, I couldn't figure out where I would implement more sophisticated scheduling (e.g. arbitrary priorities or limited resources).
2013-01-16 01:19:24 utc randomcamel (I didn't look at either of those two things, but that was one of the fundamental problems with using any of the FIFO work queues.)
2013-01-16 01:20:57 utc jmettraux seems like you already have a level of indirection
2013-01-16 01:21:36 utc jmettraux I guess a ruote participant would just have to know which of your work queue and how to flag a task with a priority
2013-01-16 01:22:12 utc jmettraux probably the priority label is set by default in the participant and can be overriden via the workitem or the process definition
2013-01-16 01:22:28 utc jmettraux sorry, just letting my imagination loose, I have no idea of your context
2013-01-16 01:30:34 utc randomcamel no worries, I like ideas. =) it's video encoding and packaging workflows.
2013-01-16 01:30:52 utc jmettraux sweet :-)
2013-01-16 01:32:03 utc randomcamel we've been working on other aspects and putting off the nuts and bolts of how we describe workflows (which basically have to be dynamically generated from an existing kind of specification) and realizing I should have thought about it harder.
2013-01-16 01:33:25 utc randomcamel remembering that I liked Ruote's DSL but I was really really wary (and still am) of deep integration with and modification of existing job distribution systems, since IME they all have their own very strong ideas of how the world works and it's just as much work to conform to the tool's worldview as it is to write a basic version yourself.
2013-01-16 01:34:22 utc jmettraux understood
2013-01-16 01:34:29 utc randomcamel but coming up now to what we actually need to express in our workflows, I'm wondering if I should have involved Ruote from the beginning, and if it's possible to do it now.
2013-01-16 01:37:23 utc jmettraux if ruote can "compose" with your system, ie if your system present a well defined {sur|inter}face, I'm sure it can
2013-01-16 01:37:46 utc jmettraux at least that would be invaluable in order to develop your own orchestration component
2013-01-16 01:37:52 utc jmettraux ruote or not
2013-01-16 09:57:53 utc hartog hi all...
2013-01-16 10:00:16 utc mister_solo good morning
2013-01-16 10:06:52 utc hartog Does anyone here has any suggestions?
2013-01-16 10:07:28 utc jmettraux hartog: hello
2013-01-16 10:08:04 utc jmettraux please look at
2013-01-16 10:09:54 utc jmettraux you should mention platforms and versions when you report issues like that
2013-01-16 10:09:57 utc hartog 1.8.0, 1.8.1, 1.9.0, etc. refer to gem version?
2013-01-16 10:10:17 utc jmettraux yes
2013-01-16 10:10:34 utc jmettraux from what I remember from your setting, you were already using ruote-mon
2013-01-16 10:10:37 utc jmettraux so it worked
2013-01-16 10:11:07 utc jmettraux at some point you upgraded to mongo 1.8.1 and it stopped working
2013-01-16 10:11:37 utc jmettraux do not be afraid to look at your version control diffs, they have lots of info for you
2013-01-16 10:11:42 utc hartog unwillingly, unknowingly and undocumented - story of ruby
2013-01-16 10:11:47 utc jmettraux proceed with small steps
2013-01-16 10:12:12 utc hartog problem is, I did not upgrade
2013-01-16 10:12:25 utc hartog but my mongo gem is not pinned down
2013-01-16 10:12:28 utc hartog so I will make it so
2013-01-16 10:12:39 utc jmettraux your Gemfile.lock is pinning it
2013-01-16 10:12:54 utc jmettraux be a bit more methodical and your life will be easier
2013-01-16 10:14:03 utc hartog I am pretty methodical
2013-01-16 10:14:19 utc hartog and as I suspected; the version of mongo (1.8.1) did not change since jan. 9
2013-01-16 10:14:29 utc hartog on my development machine it works, on production it does not
2013-01-16 10:14:36 utc jmettraux what OS?
2013-01-16 10:14:40 utc hartog both carry the same (boasted) mongod version
2013-01-16 10:14:43 utc jmettraux what OSes
2013-01-16 10:14:50 utc hartog dev=ubuntu, prod=debian
2013-01-16 10:15:12 utc jmettraux are you sure it's mongo 1.8.1 on both?
2013-01-16 10:15:30 utc hartog both have the same gemfile lock
2013-01-16 10:15:50 utc hartog down to the md5sum
2013-01-16 10:18:18 utc jmettraux ok it's solved, it would have been interesting to know what difference(s) lead to the issue
2013-01-16 10:18:32 utc hartog i will take a source build mongo to see what happens - thx for your help and pointers
2013-01-16 10:19:06 utc jmettraux you're welcome
2013-01-16 10:24:31 utc hartog and voila; it works
2013-01-16 10:24:34 utc hartog when I die, I will be placed in that circle of hell where you have to solve dependency problems all day
2013-01-16 10:31:10 utc jmettraux :-)
2013-01-16 10:31:38 utc jmettraux better better than being reincarnated as a goat or as a windows sysadmin
2013-01-16 10:33:09 utc hartog I would love to be reincarnated as a goat - just eat, sleep, head-butt-your-peers and climb-stuff all day. Seems perfect to me... :D
2013-01-16 10:33:50 utc jmettraux :D, yeah, well said
2013-01-16 13:00:47 utc hartog jmettraux: as an extension to 'testing participants' on the ruote website:
2013-01-16 13:06:51 utc jmettraux ok, thanks, I'm still left with writing the documentation
2013-01-16 13:07:03 utc jmettraux as I said, people prefer to write code ;-)
2013-01-16 13:09:22 utc hartog i know...
2013-01-16 13:09:33 utc hartog I will add more comments as I further distill this
2013-01-16 13:09:55 utc jmettraux adding that to my todo list
2013-01-16 13:10:00 utc hartog btw; there is nothing in there that is not documented (except for the thingy with the fei)
2013-01-16 13:10:04 utc hartog you just have to look hard
2013-01-16 13:11:06 utc jmettraux yeah, that's the trick, people complain about the documentation enough that I'm motivated to write it
2013-01-16 13:12:34 utc jmettraux for now I linked to your gist:
2013-01-16 13:12:57 utc jmettraux I'll try to find so time during the week-end to refine your work into some documentation
2013-01-16 13:24:00 utc hartog i'll see if i can write a few words so you can dress it up further - I should have some time some where
2013-01-16 13:29:42 utc jmettraux have to go, thanks and thanks for bearing my grumpiness ;-)
2013-01-16 13:58:28 utc hartog john; I know you scour the IRC logs from time to time - I added a little readme to, hope it helps ;)
2013-01-16 14:26:24 utc MCamou hi all
2013-01-16 14:26:35 utc MCamou I am looking at Hartog's participant testing gist
2013-01-16 14:26:47 utc MCamou which comes at just the right time for us as we are just getting into testing :)
2013-01-16 14:26:51 utc MCamou quick question
2013-01-16 14:27:12 utc MCamou if a participant sets a variable, how can I check its value in a test?
2013-01-16 14:27:51 utc MCamou we're using fexp.set_variable inside our participant
2013-01-16 14:29:19 utc hartog ACTION is clueless :D
2013-01-16 14:30:17 utc hartog john suggested once to steer clear from interacting with variables (setting, etc) and try to keep it within the workitem. I try to live by that pragma...
2013-01-16 14:30:35 utc hartog but you could use mocking for testing the set_variable has been called
2013-01-16 14:31:06 utc MCamou hmmmm… I didn't get the memo re: variables :)
2013-01-16 14:31:32 utc MCamou but I'll try mocking
2013-01-16 14:31:34 utc hartog with mocha something like: ```subject.fexp.expects(:set_variable)```
2013-01-16 14:31:50 utc hartog variables are engine_variables
2013-01-16 14:32:02 utc hartog afai[k/c]
2013-01-16 14:32:26 utc hartog workitem.fields are workflow variables, so to speak
2013-01-16 14:32:50 utc MCamou right but we see variables as something a bit less permanent
2013-01-16 14:34:09 utc hartog you have my blessings on the issue - I try to understand ruote, but it is though-stuff
2013-01-16 14:34:25 utc hartog so I cannot for the world give you any good advise on how (not) to use variables
2013-01-16 14:35:38 utc MCamou hartog: thanks, using mocks should help a lot :)
2013-01-16 14:36:45 utc MCamou BTW, small typo. In your gist you're calling the module RuoteHelpers and calling config.include RuoteHelper (without the 's')
2013-01-16 14:37:03 utc hartog okthx!
2013-01-16 14:37:36 utc hartog fixed :)
2013-01-16 14:39:03 utc MCamou oh another one :) in the test you require 'spec_helper' and the file is named ruote_helper.rb
2013-01-16 14:39:45 utc MCamou and of course we're missing a require 'ruote' or something
2013-01-16 14:39:57 utc MCamou or Ruote::LocalParticipant won't be found
2013-01-16 14:41:22 utc hartog but obviously; it is a hook for you to base your work on
2013-01-16 14:41:31 utc MCamou yes of course :)
2013-01-16 14:41:36 utc MCamou thanks for your great work!
2013-01-16 14:41:50 utc hartog normally when you use rspec, there would/should be a spec/ and spec/spec_helper.rb file
2013-01-16 14:42:01 utc hartog np
2013-01-16 15:13:20 utc hartog ACTION is off
2013-01-16 15:13:24 utc hartog wave!
2013-01-16 15:15:04 utc MCamou hartog: one more thing… in your RuoteHelpers#new_workitem, merge should be merge!
2013-01-16 21:21:10 utc randomcamel so I'm a little confused about what Ruote::Worker actually does. in my mind a "worker" is the thing that actually does the task work, but AFAICT in Ruote that's actually the participant.
2013-01-16 21:22:12 utc randomcamel it looks like maybe Worker is some kind of artifact from previous versions?
2013-01-16 22:28:47 utc jmettraux randomcamel: hello
2013-01-16 22:28:59 utc jmettraux no, worker are not artifacts from previous versions
2013-01-16 22:29:15 utc jmettraux the execution of a workflow instance is the sum of little actions
2013-01-16 22:29:24 utc jmettraux those little actions are performed by workers
2013-01-16 22:29:48 utc jmettraux among those actions are participant actions
2013-01-16 22:30:16 utc jmettraux it helps if you say to yourself "ruote worker"
2013-01-16 22:33:10 utc jmettraux I chose the name "worker" because it picks work on a queue, performs it and potentially queues more work
2013-01-16 22:38:33 utc randomcamel huh, they are sparsely mentioned in the docs. are they picking work off a queue that's done behind the scenes in the storage?
2013-01-16 22:39:55 utc jmettraux yes
2013-01-16 22:44:22 utc randomcamel so if you were to set up a group of machines to execute process instances, would there be a handful of machines running engine+worker, and then the majority being dumber nodes that are just participants?
2013-01-16 22:45:02 utc jmettraux out of the box, the workers run the participants
2013-01-16 22:45:15 utc jmettraux so it's a handful of worker machines
2013-01-16 22:45:42 utc jmettraux unless your participants are just proxy to another job system
2013-01-16 22:46:08 utc jmettraux in that case you'd have the ruote worker machines and the other job system machines
2013-01-16 22:47:23 utc randomcamel is the engine a central-ish component, or are all the worker+participant machines also running the engine?
2013-01-16 22:48:05 utc jmettraux the Engine is deprecated in favour of the Dashboard in ruote 2.3.x
2013-01-16 22:48:45 utc jmettraux it's only a dashboard, it just wraps the storage or the couple worker+storage with methods for querying workflow executions, interrupting them or launching new ones
2013-01-16 22:49:38 utc jmettraux all the recent examples should avoid Engine and use Dashboard. If you find an outdated example, please link me to it and I'll fix it if possible
2013-01-16 22:50:11 utc randomcamel =)
2013-01-16 22:50:30 utc randomcamel is there an easy way to submit doc patches?
2013-01-16 22:51:04 utc jmettraux yes,
2013-01-16 22:51:12 utc randomcamel ah hah! sweet.
2013-01-16 22:51:37 utc randomcamel writing and correcting docs is a good way to learn.
2013-01-16 22:53:03 utc jmettraux thanks! fixed
2013-01-16 22:57:02 utc randomcamel so when a participant has finished its action, is it the worker that decides on the next step, based on the process definition?
2013-01-16 22:57:35 utc jmettraux seen from above, yes
2013-01-16 22:57:37 utc randomcamel (this is all really helpful, by the way. it's a different model and terminology than I've seen before, so I'm trying to map out each things responsibilities.)
2013-01-16 22:58:01 utc jmettraux :-)
2013-01-16 23:03:53 utc randomcamel I am very interested in not re-implementing this stuff myself.