ruote log_2010-07-12

2010-07-12 07:45:16 utc dhf jmettraux: Saw the directory enhancement. Have a couple of suggestions that I was thinking about tha I did not mention. You there?
2010-07-12 07:45:34 utc jmettraux dhf: I'm here
2010-07-12 07:45:41 utc jmettraux maybe it's better to reply to that email
2010-07-12 07:45:48 utc dhf got a minute to talk about?
2010-07-12 07:45:53 utc jmettraux it keeps the conversation intact
2010-07-12 07:45:58 utc jmettraux yes, I have time
2010-07-12 07:49:10 utc dhf on the participant registration. after thinking about it I wold suggest that you register it in a wild card format. That way they can be used multiple ways. The other. Some way to not re-register if it has not changes. I do not know the impact of reregistering a participant, but if not changed no need to register on the fly. I see it so that I can give the system a signal some how to reevaluate the directory for new or changed participants. Could
2010-07-12 07:49:10 utc dhf look at the last time and compare for anything new.
2010-07-12 07:50:47 utc jmettraux OK
2010-07-12 07:50:58 utc jmettraux I'm writing that down
2010-07-12 07:52:09 utc dhf does it make sense?
2010-07-12 07:52:44 utc dhf could have the ability to force a re-eval and just an update.
2010-07-12 07:52:52 utc jmettraux OK
2010-07-12 07:53:05 utc dhf of course ton start up the time would be nil
2010-07-12 07:53:15 utc dhf so it would force all.
2010-07-12 07:53:35 utc jmettraux IIRC, it's already a bit like that, so I just have to ensure with some testing
2010-07-12 08:03:35 utc jmettraux maybe I went too fast
2010-07-12 08:03:43 utc jmettraux I shouldn't have added this register_from_dir
2010-07-12 08:04:25 utc jmettraux in case of multiple workers on multiple machines, if a participant dir gets changed in one machine and not on the other....
2010-07-12 08:07:41 utc dhf it would cause additional new ones to be registered. actually good
2010-07-12 08:08:16 utc dhf as long as the workers are the same it does not matter where they are.
2010-07-12 08:08:19 utc jmettraux but with different results depending on the worker
2010-07-12 08:08:26 utc dhf i am sorry participants
2010-07-12 08:08:32 utc jmettraux as long the code of the participants is the same...
2010-07-12 08:09:06 utc dhf different workers using different participants with the same name?
2010-07-12 08:09:34 utc jmettraux yes
2010-07-12 08:09:42 utc jmettraux out of sync participants
2010-07-12 08:10:12 utc jmettraux perhpas you'll never use more than 1 worker
2010-07-12 08:10:21 utc jmettraux but I have to think with multiple workers
2010-07-12 08:10:30 utc dhf How do the get registered today? The worker registers the participant? I thought you registered only through the engine. I have not used more than one yet.
2010-07-12 08:10:51 utc tosch_le just my 2 cents: i don't have need for register_from_dir and i believe if there is anybody who has the need, he may perfectly well come up with his own specialized solution
2010-07-12 08:11:12 utc jmettraux tosch_le: that's what I'm thinking right now
2010-07-12 08:11:47 utc jmettraux when a participant is registered, it's registered via the engine, the registration is done in the central data (the storage)
2010-07-12 08:11:55 utc jmettraux so that all the workers know about the participant
2010-07-12 08:12:12 utc jmettraux if you introduce directory reload it's fine
2010-07-12 08:12:33 utc jmettraux but if you have multiple machines, you'd better be sure all of them have the same code
2010-07-12 08:12:43 utc dhf so I do not see where the issues is you first mentioned.
2010-07-12 08:12:48 utc jmettraux else some old version of a participant might be handed a workitem
2010-07-12 08:12:58 utc dhf true
2010-07-12 08:13:19 utc dhf but is that not part of management of the system.
2010-07-12 08:13:39 utc dhf it is a convience item.
2010-07-12 08:15:32 utc dhf This way a new participant can be added or modified. If one changes and is on multiple machines it would need to be pushed to those machines.
2010-07-12 08:16:15 utc tosch_le imho, participant registration is the job of the ruote integrator, not ruote itself
2010-07-12 08:16:37 utc jmettraux +1
2010-07-12 08:17:12 utc tosch_le register_from_dir is a nice example for what is possible and may be mentioned on the website, but there are definitely some caveeats
2010-07-12 08:17:16 utc dhf My experience with workflows/bpm is that as workflows get added new participants are added. this way there is no code change.
2010-07-12 08:17:57 utc tosch_le there is a code change: the participant implementation is somewhat new
2010-07-12 08:18:00 utc jmettraux by workflow, you mean process definition ?
2010-07-12 08:18:04 utc dhf If you write a fixed workflow and do not add to it much, then code changes are not a problem. I have seen new workflows added all the time.
2010-07-12 08:18:09 utc dhf yes
2010-07-12 08:18:22 utc dhf on process def is a workflow
2010-07-12 08:18:32 utc jmettraux OK
2010-07-12 08:18:50 utc dhf but the participant is not the main code. it is external
2010-07-12 08:18:59 utc tosch_le but ruote is already flexible enough to handle that
2010-07-12 08:19:39 utc dhf As I see it now with out the option. you have to change the code that sets up the ruote engine to register participants when ever one is added
2010-07-12 08:20:04 utc jmettraux or simply do engine.register_participant :x, ParticipantClass
2010-07-12 08:20:12 utc dhf If tehre is some way to do the same thing now I am intereested inknowing what it is.
2010-07-12 08:20:33 utc tosch_le register_from_dir is one way to cope with that problem, i see some others, too, for instance using general purpose participants
2010-07-12 08:20:56 utc tosch_le it depends what you're needs are
2010-07-12 08:21:35 utc tosch_le for me, (un-)registering participants during runtime is no problem.
2010-07-12 08:21:56 utc dhf I agree registration is easy. but would you not have to then restart the engine since the engine is a running process. in the application somewhere.
2010-07-12 08:22:27 utc tosch_le why should i need to restart the engine?
2010-07-12 08:22:30 utc dhf if you add a new participant, how do you tell the engine about it on the fly without restarting it
2010-07-12 08:22:42 utc jmettraux in fact
2010-07-12 08:22:43 utc tosch_le engine.register does the trick
2010-07-12 08:22:54 utc jmettraux yes
2010-07-12 08:23:27 utc tosch_le there's no need to call that on launch time, you can call it at any moment
2010-07-12 08:24:18 utc dhf sorry not following. engine is an object in an application that is running. How do you tell it that a new participant exists weeks after the application was started. does not have to be weeks could be a short time
2010-07-12 08:24:20 utc tosch_le even if the participant implementation changes, there is no need to restart the whole engine: just re-register the participant and you're done
2010-07-12 08:24:24 utc jmettraux in fact, each time a worker has to deliver a workitem, it looks up the participant info in the participant list (which basically looks up the info in the storage)
2010-07-12 08:24:42 utc jmettraux since there is one central storage (oh the bottleneck), everything is OK
2010-07-12 08:25:22 utc dhf so in theory you could create a process to register new participants?
2010-07-12 08:25:24 utc jmettraux you can call engine.register_participant at anytime
2010-07-12 08:25:40 utc tosch_le yes
2010-07-12 08:25:44 utc jmettraux in theory you could do it from a Rakefile, a piece of ruby code
2010-07-12 08:26:15 utc tosch_le why not using a process? just come up with a ParticipantRegistrationParticipant and you're done
2010-07-12 08:26:22 utc tosch_le ;-)
2010-07-12 08:26:29 utc jmettraux ouch
2010-07-12 08:26:41 utc dhf but rake starts a new engine.
2010-07-12 08:27:00 utc dhf I like the procss if we can not keep the by directory
2010-07-12 08:27:03 utc tosch_le yeah, but the engine is just a dashboard, in that case to the storage
2010-07-12 08:27:04 utc jmettraux dhf: not necessarily
2010-07-12 08:27:14 utc dhf am I missing something about starting new engines?
2010-07-12 08:27:17 utc jmettraux yes
2010-07-12 08:27:24 utc tosch_le the storage is shared by all engines, so the running one will get notified
2010-07-12 08:27:37 utc jmettraux exactly
2010-07-12 08:27:55 utc dhf so I could start asecond engine on the fly to register a new participant?
2010-07-12 08:28:00 utc jmettraux yes
2010-07-12 08:28:10 utc tosch_le one caveeat, though: that is right for partipants which are not already instanciated
2010-07-12 08:28:24 utc dhf or I actually like creating a participant to do the work
2010-07-12 08:29:05 utc jmettraux ouch
2010-07-12 08:29:17 utc dhf why?
2010-07-12 08:29:59 utc jmettraux sounds painful
2010-07-12 08:30:03 utc jmettraux why not
2010-07-12 08:30:32 utc tosch_le to repeat myself, and i suppose that are John#: "imho, participant registration is the job of the ruote integrator, not ruote itself"
2010-07-12 08:30:44 utc tosch_le s/John/John's worries/
2010-07-12 08:30:53 utc jmettraux there is an interesting thing, here
2010-07-12 08:31:07 utc jmettraux my task is to make integrators' life easier
2010-07-12 08:31:37 utc dhf why painful? I realize it might be better to use something like rake to start an engine register the new participant then quit.
2010-07-12 08:31:51 utc jmettraux ok
2010-07-12 08:32:27 utc dhf I jsut like the directory approach. forces the participants in one area. easy to maintain and update.
2010-07-12 08:32:34 utc jmettraux tosch_le: there's a potential pain point : when a catchall participant is registered and you innocently what to add a participant and it gets stuck *after* the catchall participant
2010-07-12 08:33:52 utc tosch_le yes, that's a problem. it's the reason i'm not perfectly happy with '.+' as participant name
2010-07-12 08:33:55 utc jmettraux so editing the participant list "en bloc" is interesing
2010-07-12 08:34:05 utc jmettraux interesting
2010-07-12 08:34:23 utc dhf Help me, if a catchall is registered, and you later register via a new engine it will be registered after the catchall and will not be ever used?
2010-07-12 08:34:38 utc jmettraux exactly
2010-07-12 08:34:53 utc tosch_le but that would be a problem, too, if the participants from a dir are registered using wildcards. what about a.rb vs aa.rb?
2010-07-12 08:35:14 utc jmettraux of course you can pass a "position" parameter, but the novice integrator will make the error (98%)
2010-07-12 08:36:00 utc jmettraux toch_le: look at the last test : http://github.com/jmettraux/ruote/blob/ruote2.1/test/functional/ft_43_participant_register_from_dir.rb
2010-07-12 08:36:20 utc jmettraux mixing alphabetical order and class.participant_regex
2010-07-12 08:36:47 utc dhf could d aht I am doing with another part of my system not ruote but I do keep a file of the ordered definitions of documents. The order is critical. That file is used to register the definitions in the right order.
2010-07-12 08:39:11 utc dhf not as easy as just dropping it in a directory, but when order is important it works. I did not realize order of participants was important in Ruote
2010-07-12 08:39:44 utc tosch_le it's important when using wildcards
2010-07-12 08:39:50 utc jmettraux it's important only if you go with regexes instead of absolute participant names
2010-07-12 08:39:55 utc jmettraux :) double answer
2010-07-12 08:42:30 utc dhf sorry lost me again. Why is it important with regex? If part of the file name is use as part of the name pike patientparticipant.rb register it as patient.+ how does the order make a difference
2010-07-12 08:42:57 utc tosch_le of course, there could be a solution for the multiple workers and participants loaded by dir problem: just save a checksum of the participant file during participant registration in the storage
2010-07-12 08:43:23 utc tosch_le and the time the file was last modified, obviously
2010-07-12 08:43:30 utc tosch_le but hey, that is overkill
2010-07-12 08:43:42 utc jmettraux dhf: /^u.+/ vs /^user-.+/
2010-07-12 08:44:03 utc jmettraux if u is placed before user
2010-07-12 08:44:32 utc jmettraux all the workitem for participants whose name begin with u will go to u
2010-07-12 08:44:46 utc jmettraux and the /^user-.+/ will never see a workitem
2010-07-12 08:45:07 utc tosch_le …and user-tosch is used in a process definition, then the participant for u.+ will eat all my cookies :-)
2010-07-12 08:45:18 utc jmettraux cookie-monster
2010-07-12 08:45:36 utc jmettraux and Germany will never reach the finale ;)
2010-07-12 08:46:05 utc dhf so you would have to have files in the directory as u_participant.rb and user_participant.rb? put a line in the file that indicates the registration name
2010-07-12 08:46:29 utc dhf is this a real case? or a protection of an edge?
2010-07-12 08:46:50 utc jmettraux I like regex participants
2010-07-12 08:47:00 utc dhf I agree I do to
2010-07-12 08:47:27 utc jmettraux dhf: http://github.com/jmettraux/ruote/blob/ruote2.1/test/functional/ft_43_participant_register_from_dir.rb has three different tests covering order and regexes, it gives a good POV on the participant list job
2010-07-12 08:48:37 utc dhf I saw that. look for an order file in the directory. If it exists use it for the order. otherwise alpah order
2010-07-12 08:49:02 utc jmettraux and participant classes have potential "participant_regex" class method
2010-07-12 08:49:59 utc dhf that works. but if order is important, something has to provide that order
2010-07-12 08:51:21 utc dhf That does not solve the order that tosch_le mentioned. Plus there is nothing saying that the directory approach must be used
2010-07-12 08:51:31 utc jmettraux ?
2010-07-12 08:51:43 utc jmettraux the order problem is sovled
2010-07-12 08:51:45 utc jmettraux solved
2010-07-12 08:51:47 utc tosch_le dhf: it does solve the problem
2010-07-12 08:52:00 utc tosch_le the user has to make sure he provides the right order
2010-07-12 08:52:41 utc jmettraux if there is a xxxx_participant.rb and xxxx is a number, the number is dropped (but the directory listing will have considered it when ordering the files (alpha order))
2010-07-12 08:52:44 utc dhf I like the "order file" it would contain user_participant.rb followed by u_participant.rb then they would be ordered in the right order. if the files was not there register in alpha order.
2010-07-12 08:52:58 utc dhf that works also
2010-07-12 08:53:05 utc dhf to force the right order.
2010-07-12 08:53:42 utc tosch_le yeah, but that's to much pain to maintain the order
2010-07-12 08:53:54 utc tosch_le naming the files is easier imho
2010-07-12 08:54:03 utc jmettraux actually, it all boils down to that : http://gist.github.com/472271
2010-07-12 08:54:11 utc jmettraux the participant list is only a list in the storage
2010-07-12 08:54:18 utc dhf I saw your comment about that in the file. It does solve the order problem but as you said the novice will make a mistake but simple doc solves that
2010-07-12 08:54:36 utc jmettraux you can read it and then update it
2010-07-12 08:55:02 utc jmettraux and since all workers share the same storage...
2010-07-12 08:55:34 utc jmettraux it's a list of [ regex, [ classname, options* ] ]
2010-07-12 08:57:22 utc dhf I still like what you already did. It seems to solve the issues. If the implementor does not want to use it, they do not have to. or they can do a mixture I wold assume.
2010-07-12 08:58:57 utc tosch_le i'm unsure if there should be code parts within ruote which need signposts around it warning using the code might break when dealing with multiple workers
2010-07-12 09:01:22 utc dhf Multiple workers on the same machine a problem? I still do not see the problem.
2010-07-12 09:01:40 utc jmettraux what I'm going to do : remove the register_from_dir, but provide one or two methods for easy edition of the participants as a list
2010-07-12 09:02:04 utc tosch_le dhf: you'll really have to make sure the participant files are identical
2010-07-12 09:02:22 utc tosch_le and multiple workers may be on multiple machines
2010-07-12 09:02:45 utc dhf on same machine? why would they not be? on multiple machines I can see the issue
2010-07-12 09:03:14 utc dhf John. if you do remove it please send my that module.
2010-07-12 09:03:50 utc jmettraux dhf: OK
2010-07-12 09:04:04 utc dhf but multimachine workers is more advanced. I would think that the majority would use the directory
2010-07-12 09:04:10 utc tosch_le dhf: even on the same machine someone may use another directory
2010-07-12 09:04:37 utc dhf to me those are documentation issues
2010-07-12 09:04:53 utc dhf if you do not want to use it don't
2010-07-12 09:05:06 utc dhf it is just an option
2010-07-12 09:05:16 utc jmettraux dhf: http://gist.github.com/472278 line 158
2010-07-12 09:05:50 utc jmettraux maybe I'll keep it around, I wanted to use it from RK
2010-07-12 09:07:39 utc dhf Please do.
2010-07-12 09:08:24 utc dhf I might be wrong but I would guess it is 80+% cold easily use it without a problem
2010-07-12 09:09:12 utc tosch_le you may increase that to more than 95% ;-)
2010-07-12 09:09:49 utc jmettraux maybe all participant lists should be directory based
2010-07-12 09:10:06 utc dhf it is just such a logical approach.
2010-07-12 09:10:25 utc tosch_le no, it's to unflexible.
2010-07-12 09:10:42 utc tosch_le what about participant generation on the fly?
2010-07-12 09:10:56 utc jmettraux yes
2010-07-12 09:10:57 utc dhf I would not make it a requirement. just an option.
2010-07-12 09:11:37 utc dhf actually makes Ruote more flexible by having it.
2010-07-12 09:11:54 utc tosch_le have to leave for a while, meeting now.
2010-07-12 09:11:59 utc jmettraux ciao
2010-07-12 09:12:02 utc tosch_le ciao!
2010-07-12 09:12:10 utc jmettraux I have heard this argument before ;)
2010-07-12 09:12:36 utc dhf I know. too much flexibility is just as bad as too little
2010-07-12 09:13:29 utc jmettraux thanks for the great conversation anyway
2010-07-12 09:14:01 utc dhf I saw your comments about RK and I was thinking this would make that easier also.
2010-07-12 09:14:35 utc dhf one approach on the catchall is to unregister it then load and re register
2010-07-12 09:14:53 utc dhf there are probably better ways though
2010-07-12 09:17:20 utc jmettraux yes, you can give a position when registering, or use the update all approach I showed in this channel before
2010-07-12 09:17:29 utc jmettraux I have to package and document that
2010-07-12 09:18:35 utc dhf I must have missed it. you can say that catchall is last, always?
2010-07-12 09:19:10 utc jmettraux when you register a participant, you can tell where in the list it goes
2010-07-12 09:19:17 utc jmettraux you could say go top for example
2010-07-12 09:20:32 utc jmettraux have to move, ttyl !
2010-07-12 10:43:10 utc kennethkalmer hey guys !
2010-07-12 10:43:20 utc kennethkalmer jmettraux: thanks for the boston.com photo essay link
2010-07-12 12:00:47 utc jmettraux kennethkalmer: you're welcome