ruote log_2010-07-09

2010-07-09 06:39:42 utc dhf good morning/evenng all, I do not seem to get this test process to start. Noting gets printed and nothing gets logged. I am using Logging to log within the application.
2010-07-09 06:52:39 utc jmettraux dhf: hello, how is the engine initialized ?
2010-07-09 06:53:22 utc dhf @@storage =
2010-07-09 06:53:23 utc dhf @@worker =
2010-07-09 06:53:23 utc dhf @@engine =
2010-07-09 06:54:05 utc dhf @@storage_path = '/tmp/phynet-ruote'
2010-07-09 06:54:29 utc jmettraux does it create files under /tmp/phynet-ruote ?
2010-07-09 07:04:33 utc jmettraux dhf: if you add a "sleep 5" before the end of the spec, it will let the engine do its work before the ruby process exits
2010-07-09 07:05:40 utc dhf I added a wait_for the return value from the tp.launch and it still fails. I will try the sleep
2010-07-09 07:06:19 utc dhf what does the ARGV << "-d" flag in your rspec example
2010-07-09 07:06:36 utc jmettraux ?
2010-07-09 07:06:52 utc jmettraux sorry, I have no rspec examples
2010-07-09 07:07:22 utc jmettraux there is an error in your participant
2010-07-09 07:07:40 utc jmettraux the class Ruote::Workitem doesn't respond to the [] method
2010-07-09 07:07:58 utc jmettraux engine.process(wfid).errors will be non-empty
2010-07-09 07:08:41 utc jmettraux this works :
2010-07-09 07:09:43 utc jmettraux engine.context.logger.noisy = true
2010-07-09 07:10:04 utc jmettraux is useful when learning/debugging, it spits out the whole engine activity to STDOUT
2010-07-09 07:10:39 utc dhf was the puts "Storing #{workitem['name']" nt right. I noticed you used inspect. I was trying to test my knowledge of using one field/value
2010-07-09 07:11:05 utc dhf how do you do that?
2010-07-09 07:11:34 utc jmettraux ?
2010-07-09 07:11:46 utc dhf If you want you can have these test and samples when I am done learning. They may help others.
2010-07-09 07:11:54 utc jmettraux the class Ruote::Workitem doesn't respond to the [] method
2010-07-09 07:11:59 utc jmettraux ah thank you very much
2010-07-09 07:12:05 utc dhf setting noisy to true spits out everything?
2010-07-09 07:12:21 utc jmettraux verbose mode, yes
2010-07-09 07:13:06 utc dhf Ok. If the ['name'] waas wrong, how do I get one value is that the ${f:name} ?
2010-07-09 07:14:15 utc jmettraux workitem.fields['name']
2010-07-09 07:15:14 utc jmettraux for example :
2010-07-09 07:15:25 utc dhf missed the fields. are there other methods to access the same information. In my test I will be useing all. That way I have a reference of the carious ways to do something
2010-07-09 07:15:28 utc jmettraux (though those BlockParticipants are not recommended)
2010-07-09 07:15:45 utc jmettraux let me check
2010-07-09 07:17:50 utc jmettraux you could also use those two :
2010-07-09 07:18:49 utc jmettraux you can also re-open Workitem to add some methods :
2010-07-09 07:19:51 utc dhf ok so lookup will work. Then where is the $ syntax used? I understand opening but that is last resort
2010-07-09 07:20:35 utc jmettraux opening is OK since you're the "last mile user"
2010-07-09 07:21:39 utc dhf true
2010-07-09 07:24:39 utc dhf are the 'name' and the 'task' that are used fields or variables? Also could ${f:name} and ${f:task} be used?
2010-07-09 07:24:57 utc jmettraux well
2010-07-09 07:25:16 utc jmettraux OK, looking back at your code to determine the context
2010-07-09 07:26:10 utc jmettraux ${name} means "substitute me with the value of the workitem field "name" turned into a String
2010-07-09 07:26:33 utc jmettraux it's equivalent to ${f:name} and ${field:name}
2010-07-09 07:26:54 utc jmettraux
2010-07-09 07:29:16 utc dhf I used engine.register_participant :saveimage, istore, :task => 'save' and the inspect did not show task at all. Is that niot the right syntax?
2010-07-09 07:29:33 utc dhf I want to pass exactly what I expect form the participant
2010-07-09 07:31:05 utc jmettraux
2010-07-09 07:31:25 utc jmettraux since ruote 2.1.x, the preference is to register "stateless" participants
2010-07-09 07:31:44 utc dhf I have read that many times today
2010-07-09 07:31:52 utc jmettraux engine.register_participant(name_or_regex, klass, options)
2010-07-09 07:32:02 utc jmettraux name_or_regex is a string or a regular expression
2010-07-09 07:32:09 utc jmettraux klass is a instance of Class
2010-07-09 07:32:28 utc jmettraux options is a hash of options that are passed when is called
2010-07-09 07:32:33 utc jmettraux
2010-07-09 07:33:02 utc dhf this is a statelesss i thought. So the task is in opts?
2010-07-09 07:33:11 utc jmettraux yes
2010-07-09 07:33:21 utc jmettraux but your initial implementation is stateful
2010-07-09 07:33:31 utc jmettraux since you instantiate the participant before registering it
2010-07-09 07:33:31 utc dhf a suym is ok instead of a string?
2010-07-09 07:33:38 utc jmettraux yes
2010-07-09 07:34:34 utc dhf do not understand the instantiate making it statefull? how would you do it with out the instantiating?
2010-07-09 07:35:10 utc jmettraux if you pass the class instead of an instance (of that class), ruote will instantiate the participant only when needed
2010-07-09 07:35:34 utc jmettraux this is necessary if you want ruote to run in multiple workers (multiple ruby processes)
2010-07-09 07:36:19 utc jmettraux line 54 is the right thing
2010-07-09 07:37:50 utc jmettraux then you can do something like line 16 - 17
2010-07-09 07:40:03 utc jmettraux argh
2010-07-09 07:40:11 utc jmettraux 16:36 jmettraux: line 54 is the right thing
2010-07-09 07:40:14 utc jmettraux means nothing
2010-07-09 07:40:35 utc lbt morning all :)
2010-07-09 07:40:54 utc jmettraux line 56 is better
2010-07-09 07:41:00 utc jmettraux lbt: morning !
2010-07-09 07:42:51 utc lbt been busy recently? it's been quiet around here...
2010-07-09 07:43:28 utc jmettraux :) I don't generate questions by myself
2010-07-09 07:43:49 utc dhf John has been GREAT
2010-07-09 07:44:03 utc jmettraux ACTION blushes
2010-07-09 07:44:05 utc lbt ah... I usually talk to myself if no-one asks me anything.... :)
2010-07-09 07:44:28 utc lbt I've been making .debs for ruote too FYI
2010-07-09 07:44:36 utc jmettraux cool
2010-07-09 07:45:00 utc dhf Oh great another guru I can tap. I have to wait all day for John. But it gies me lots of time to experiment.
2010-07-09 07:45:09 utc lbt dhf: I notice you're having issues getting ruote going ...
2010-07-09 07:45:28 utc lbt well, I'm new to it too but sure... if I can help...
2010-07-09 07:46:39 utc dhf have been jumping ahead to learn. I can do the simple blocked things. trying to learn the using my own participants. Writing simple rspec test and wil be giving John the samples when I am done to use in doc.
2010-07-09 07:47:35 utc lbt OK
2010-07-09 07:47:49 utc lbt I'm focusing mainly on the amqp side for remote participants
2010-07-09 07:48:37 utc dhf Good I will be getting to that. I use beanstalk for the rest of the system but it is still experimental
2010-07-09 07:49:19 utc dhf in what I am doing. I have to convince management to do all this. I am a one person company ;)
2010-07-09 07:50:11 utc jmettraux thanks for trying ruote ;)
2010-07-09 07:50:17 utc lbt well, IMHO the benefits of ruote come with loose coupling of process steps
2010-07-09 07:51:03 utc lbt and that's why I personally am looking for an rpc connection from the control logic to the participant logic
2010-07-09 07:51:17 utc lbt but that isn't required...
2010-07-09 07:51:29 utc jmettraux rpc connection ?
2010-07-09 07:51:42 utc lbt remote procedure call
2010-07-09 07:51:50 utc dhf I love it. I worked with others several years ago and was SVP R&D for a company that wrote one in python. It was not accepted by the management. Public companies with management :(
2010-07-09 07:52:08 utc dhf The decided to use oracle because of the name.
2010-07-09 07:52:29 utc lbt *nod* ....
2010-07-09 07:53:03 utc dhf Is tehre a way to get a list of all registered participants?
2010-07-09 07:53:10 utc lbt heh...
2010-07-09 07:53:16 utc lbt my job for today:
2010-07-09 07:53:22 utc lbt write a "ps" for ruote
2010-07-09 07:54:01 utc lbt s/write/write or find/ .... john's bound to have one
2010-07-09 07:54:27 utc jmettraux engine.processes
2010-07-09 07:54:45 utc lbt jmettraux: what are your thoughts on having ~1 participant per process/script ?
2010-07-09 07:55:05 utc jmettraux lbt: no context, no thoughts
2010-07-09 07:55:06 utc lbt (for non-trivial ones obviously)
2010-07-09 07:55:56 utc lbt well - the examples are mainly single-file scripts with an engine/worker/participant (ISTR)
2010-07-09 07:55:57 utc jmettraux dhf: unfortunately, there is only engine.context.plist.names
2010-07-09 07:56:20 utc jmettraux or engine.context.plist.send(:get_list)
2010-07-09 07:56:50 utc jmettraux lbt: 1+ participants is great
2010-07-09 07:56:59 utc jmettraux and examples are just examples
2010-07-09 07:57:22 utc jmettraux lbt : I now RPC, but what do you mean by :
2010-07-09 07:57:23 utc jmettraux 16:51 lbt: and that's why I personally am looking for an rpc connection from the control logic to the participant logic
2010-07-09 07:57:26 utc jmettraux ?
2010-07-09 07:57:35 utc jmettraux which control logic ?
2010-07-09 07:57:37 utc lbt the amqp connection solution
2010-07-09 07:57:51 utc lbt that's 'rpc'
2010-07-09 07:57:54 utc lbt ish
2010-07-09 07:57:59 utc jmettraux ah got it
2010-07-09 07:58:53 utc lbt dhf: also... just since you mentioned python. I wrote a python client for ruote
2010-07-09 08:03:02 utc dhf ibt: as soon as you said that, I thought of an use to tie in my old product into what I am trying to do. I owned a healthcare software company for 18 years. the product was written in Python
2010-07-09 08:03:18 utc lbt :)
2010-07-09 08:03:24 utc jmettraux lbt : in the system I'm developing right now, I have like 5-6 participants
2010-07-09 08:04:42 utc lbt how do you structure the code ?
2010-07-09 08:05:15 utc jmettraux lbt: well, it's a rails app
2010-07-09 08:05:23 utc lbt OK
2010-07-09 08:05:39 utc jmettraux I put the ruote initialization in config/initializers/ruote.rb
2010-07-09 08:05:57 utc jmettraux and the participants under lib/{appname}/participants/*.rb
2010-07-09 08:06:08 utc jmettraux (the participant implementation)
2010-07-09 08:06:49 utc lbt I guess I'm significantly influenced because my mental model is multiple systems with minimal human interaction (so no single web app)
2010-07-09 08:07:41 utc jmettraux as long as you have a ruby file requiring the others, you're good to go
2010-07-09 08:08:42 utc jmettraux ACTION keeps forgetting lbt is kind of new to ruby
2010-07-09 08:09:00 utc lbt "kind of" !!! :)
2010-07-09 08:10:00 utc lbt btw... are you interested in the deb/rpm packaging ?
2010-07-09 08:10:20 utc jmettraux lbt: well
2010-07-09 08:10:26 utc jmettraux rubygems works well
2010-07-09 08:11:16 utc jmettraux and I don't have the time to maintain debs
2010-07-09 08:11:24 utc lbt *nod*
2010-07-09 08:12:07 utc jmettraux thanks anyway !
2010-07-09 08:12:08 utc dhf would it make sense to have a routine go through the participants directory and register them. The registration is one of my issues. I want to be able to drop a new participant in a directory and tell the system to register it.
2010-07-09 08:12:28 utc jmettraux dhf: it sure could be done
2010-07-09 08:12:35 utc lbt dhf: interesting; I want that too
2010-07-09 08:12:40 utc dhf I am doing that with another part of the system and it works well
2010-07-09 08:12:54 utc lbt I wrote a 'register' to work with amqp
2010-07-09 08:12:56 utc jmettraux you'd have to provide a class method that tells the engine under which name/regex to register the participant
2010-07-09 08:13:04 utc lbt so a remote participant can register with the core engine
2010-07-09 08:13:12 utc lbt dynamically
2010-07-09 08:13:44 utc dhf I would name the file image_participant.rb and register it using image as the name
2010-07-09 08:13:53 utc jmettraux excellent idea
2010-07-09 08:13:55 utc lbt jmettraux: and currently there's a silent override.... I wondered if that could change
2010-07-09 08:14:23 utc dhf the issue would be otopns, which I still hav not gotten working yet. I do not see them where I think I should in the participant.
2010-07-09 08:14:31 utc jmettraux lbt: please read that email thread : it's related
2010-07-09 08:14:58 utc lbt ACTION reads
2010-07-09 08:15:24 utc jmettraux dhf: in this gist I use the options passed at registration
2010-07-09 08:15:33 utc lbt heh... I should read my mail b4 coming here
2010-07-09 08:16:16 utc jmettraux lbt : I'm not convinced by dynamic registration of participants, so far dynamic participants have been good
2010-07-09 08:16:45 utc dhf That is why I asked about getting a list of participants registered. I want to signal the system to check for updates or on a schedule.
2010-07-09 08:17:05 utc lbt jmettraux: I'll sort out the use case.
2010-07-09 08:17:44 utc jmettraux lbt: please understand that dynamic registration is too heavy handed
2010-07-09 08:17:53 utc dhf using the current list and checking against the current list. might keep my own that are registerd this way along with the timestamp to chek if changed since registered.
2010-07-09 08:17:58 utc jmettraux multiplexing withing participants is way easier and robust
2010-07-09 08:18:21 utc jmettraux within participants
2010-07-09 08:18:32 utc dhf jmettraus: a good example of that would be great
2010-07-09 08:18:48 utc jmettraux dhf: you could also override the ParticipantList component
2010-07-09 08:19:10 utc lbt jmettraux: I'm talking about adding a completely new bit of logic on a remote system.
2010-07-09 08:19:33 utc jmettraux lbt: I think we should stop discussing about that, we're completely off
2010-07-09 08:19:56 utc dhf I will be investigating options when I get to that point. I was asking for good example of multiplexing within a participant.
2010-07-09 08:20:03 utc jmettraux aaaaaaaaaaaaaaaaaaaaah
2010-07-09 08:20:16 utc lbt ACTION is quiet now ;)
2010-07-09 08:20:57 utc dhf If I ask too many questions, tell me to be quiet also. no problem.
2010-07-09 08:25:16 utc jmettraux :)
2010-07-09 08:25:25 utc jmettraux dhf:
2010-07-09 08:25:41 utc jmettraux multiplexing is from line 6 to 10
2010-07-09 08:26:00 utc jmettraux the participant catches a wide range workitem
2010-07-09 08:26:16 utc jmettraux and then, based on the participant name, decides on what to do with them
2010-07-09 08:26:18 utc dhf got it and like it.
2010-07-09 08:26:42 utc dhf can you say not handled? if get one not valid?
2010-07-09 08:26:59 utc jmettraux ?
2010-07-09 08:27:16 utc jmettraux aaaaaaaaaah
2010-07-09 08:27:20 utc jmettraux let me check the doc
2010-07-09 08:28:20 utc jmettraux dhf: it's a kind of new feature
2010-07-09 08:28:32 utc jmettraux the participant can reject a workitem
2010-07-09 08:28:52 utc dhf what happens when that is done?
2010-07-09 08:28:53 utc lbt neat
2010-07-09 08:29:16 utc jmettraux a new despatchment
2010-07-09 08:29:25 utc jmettraux let me lookup another piece of doc
2010-07-09 08:29:35 utc jmettraux (this is more useful when there are multiple workers
2010-07-09 08:30:06 utc dhf it would go to another worker, but what happens if totally rejected
2010-07-09 08:30:12 utc dhf all reject it
2010-07-09 08:30:40 utc jmettraux sorry, this reject thing is useless in that case
2010-07-09 08:30:58 utc jmettraux you should simply raise an exception
2010-07-09 08:31:04 utc jmettraux and the process would go into an error
2010-07-09 08:31:12 utc dhf ok that is fine
2010-07-09 08:31:58 utc jmettraux then you can fix the issue and then replay_at_error
2010-07-09 08:33:08 utc dhf on the: engine.register_participant :saveimage, Participants::ImageParticipant, :task => 'save' where would I find the :task? I thought you said it is passed in the opts on initialize, but that is blank when I puts it
2010-07-09 08:33:36 utc jmettraux let me check
2010-07-09 08:34:33 utc jmettraux dhf : got it : you have to change the initialize header of your participant
2010-07-09 08:34:41 utc jmettraux def initialize (opts)
2010-07-09 08:34:44 utc jmettraux instead of
2010-07-09 08:34:50 utc jmettraux def initialize (opts={})
2010-07-09 08:38:16 utc dhf how do you pass multiple options
2010-07-09 08:38:22 utc lbt nb... I totally get dynamic participants.
2010-07-09 08:39:14 utc jmettraux dhf: engine.register_participant :x, k, 'option1' => 'value1', 'option2' => 'value2'
2010-07-09 08:39:18 utc jmettraux lbt: great
2010-07-09 08:40:02 utc lbt 100% not what I'm on about.... but I'll "show you the code" at some point too :)
2010-07-09 08:40:06 utc lbt anyhow...
2010-07-09 08:40:23 utc lbt dhf: one thing that got me about options...
2010-07-09 08:40:43 utc lbt they're "kept" until the process initialises a participant with those values
2010-07-09 08:42:04 utc lbt so you can do engine.register_participant :robot1, robot, 'IP' => '' and engine.register_participant :robot2, robot, 'IP' => ''
2010-07-09 08:42:22 utc lbt (if my ruby is correct)
2010-07-09 08:42:35 utc lbt and you have 2 participants of class robot.
2010-07-09 08:42:54 utc lbt each is likely to be instantiated many times
2010-07-09 08:43:01 utc lbt by different processes
2010-07-09 08:43:04 utc jmettraux (provided robot contains a class)
2010-07-09 08:43:38 utc jmettraux like a printer is a participant in different processes
2010-07-09 08:43:40 utc lbt but each time a robot1 is made, it'll be passed options containg that IP which will probably go into self._ip
2010-07-09 08:43:58 utc lbt err, @ip
2010-07-09 08:44:49 utc lbt printer is a good example jmettraux
2010-07-09 08:50:39 utc jmettraux lbt: I have to tell that your saying that participants (right next to the engine) are more like proxies has been haunting me
2010-07-09 08:51:02 utc lbt for amqp
2010-07-09 08:52:14 utc lbt (I hope in a good way :) )
2010-07-09 08:52:24 utc jmettraux in a good way :)
2010-07-09 08:53:56 utc jmettraux have to escape for a while
2010-07-09 09:06:29 utc dhf or you could pass them when the participant is call in the process.
2010-07-09 09:26:30 utc dhf jmettraux: where can you get the errors? I tried using "@engine.process(wfid).errors" and I get a undefined method 'errors' for nilClass error. I put it before and after the wait_for.
2010-07-09 09:27:12 utc jmettraux if engine.process(wfid) returns nil, it means the process hasn't yet been started or is already over
2010-07-09 09:27:48 utc dhf so how can I tell if it ended with errors?
2010-07-09 09:28:12 utc jmettraux if you place the engine.process(wfid).errors right after the launch it will break
2010-07-09 09:28:27 utc jmettraux place it only after the wait_for(wfid)
2010-07-09 09:29:59 utc dhf get the nillClass error there also
2010-07-09 09:31:04 utc jmettraux dhf: could you please prepare a pastie of that portion of code ?
2010-07-09 09:31:39 utc jmettraux wait
2010-07-09 09:31:41 utc jmettraux no need
2010-07-09 09:32:03 utc dhf
2010-07-09 09:32:06 utc jmettraux if your process executes successfully, it will be gone after the wait_for() so it's good
2010-07-09 09:33:02 utc dhf so how can you check if it processed successfully?
2010-07-09 09:33:30 utc jmettraux sorry, out of the box, no history logging
2010-07-09 09:34:35 utc dhf I mean real time. How do I know a process was good. What can be checked?
2010-07-09 09:34:35 utc jmettraux you could have a look at my test bench or at
2010-07-09 09:35:03 utc jmettraux you could place a "final" participant that keeps track of the wfids of the process that terminated
2010-07-09 09:35:45 utc jmettraux dhf: what is the context ? spec or production ?
2010-07-09 09:36:30 utc jmettraux dhf: if wait_for(wfid) returns it means two things a) process wound up in error or b) process terminated
2010-07-09 09:38:25 utc dhf both actually. but mainly in production. When a process is done. I need to know it was clean. Example was the [] issue earlier. If there was a bad participant r other. need to know. I assume I could then check if process(wfid) was nil that was no error
2010-07-09 09:39:09 utc jmettraux engine.processes returns all the currently alive processes
2010-07-09 09:39:42 utc jmettraux process_in_errors = { |ps| ps.errors.length > 0 }
2010-07-09 09:41:58 utc dhf I forced an error and got the errors after the wait_for. So if I do if.engine.process(wfid).nil? means there were no errors.
2010-07-09 09:42:13 utc jmettraux yes
2010-07-09 09:42:39 utc jmettraux (or it means you're asking too soon and the worker hasn't yet kicked in)
2010-07-09 09:43:02 utc jmettraux in production you'll rarely watch a process that closely
2010-07-09 09:43:17 utc jmettraux you'll fire them and check from time to time if they go wrong
2010-07-09 09:43:28 utc dhf No but I want to log that a process had an error.
2010-07-09 09:45:53 utc jmettraux you could implement a logger/history : (an example)
2010-07-09 09:47:42 utc jmettraux you could also add an on_error attribute at the root of your process definition
2010-07-09 09:47:58 utc jmettraux
2010-07-09 09:48:22 utc jmettraux too many possibilities :(
2010-07-09 09:49:52 utc dhf using your process_in_errors = { |ps| ps.errors.length > 0 } when does the errors clear? I found many form when I was experimenting that still show up?
2010-07-09 09:51:20 utc dhf does an errored process stay around? how can it be fixed and restarted? I know I have lots of questions. But my goal is the get this sample test done this weekend.
2010-07-09 09:53:27 utc jmettraux yes, it stays around
2010-07-09 09:53:34 utc jmettraux the fix depends on the error
2010-07-09 09:53:46 utc jmettraux use the replay_at_error to restart the process
2010-07-09 09:54:16 utc jmettraux
2010-07-09 12:49:34 utc kennethkalmer jmettraux: thanks ! :D
2010-07-09 12:49:46 utc jmettraux you're welcome :)
2010-07-09 12:50:06 utc kennethkalmer how are things going over here ?
2010-07-09 12:51:08 utc jmettraux doing fine
2010-07-09 12:51:22 utc jmettraux and you and the cup ?
2010-07-09 12:52:06 utc kennethkalmer i'm doing great, just got back from snowboarding in Lesotho (
2010-07-09 12:52:15 utc jmettraux cool
2010-07-09 12:52:19 utc kennethkalmer getting ready for the last two games of the swc
2010-07-09 12:52:56 utc jmettraux
2010-07-09 12:54:04 utc kennethkalmer thanks, looks awesome