ruote tmp/log_2011-09-29.html

2011-09-29 05:12:19 utc u1106 did anybody see my question ~12 hrs ago?
2011-09-29 05:15:53 utc jmettraux u1106: hello, yes
2011-09-29 05:16:08 utc jmettraux if there is no one replying here, consider using the mailing list: http://groups.google.com/group/openwferu-users
2011-09-29 05:16:19 utc u1106 good morning
2011-09-29 05:16:45 utc u1106 (with your instantiation of morning :)
2011-09-29 05:17:09 utc u1106 yep, haven't subscribed yet, but I should
2011-09-29 05:17:53 utc jmettraux the beanstalk code only has the "engine" side
2011-09-29 05:18:36 utc u1106 ok, so I guessed that correctly
2011-09-29 05:19:03 utc jmettraux implementing a "participant" on the other side is quite easy: https://github.com/jmettraux/ruote-beanstalk/blob/master/test/functional/ft_0_participant.rb#L31-53
2011-09-29 05:19:17 utc u1106 So the remote participant side exists nowhere in a public repo?
2011-09-29 05:19:29 utc u1106 Ah, I looked a t that one yesterday
2011-09-29 05:20:16 utc jmettraux do you work with David, in Finland ?
2011-09-29 05:20:30 utc u1106 exactly, I am the one
2011-09-29 05:21:05 utc u1106 I was looking for the place where an error is created
2011-09-29 05:22:02 utc u1106 I understood that ruote-beanstalk uses its own wire protocol
2011-09-29 05:22:29 utc jmettraux yes
2011-09-29 05:22:52 utc u1106 i.e. it sends arrays over the wire, not just the hash, which is in ruote storage
2011-09-29 05:25:43 utc jmettraux an implementation is free to send what it wants/needs
2011-09-29 05:25:43 utc u1106 as long as remote participant and engine agree on the format :)
2011-09-29 05:25:43 utc jmettraux exactly
2011-09-29 05:25:43 utc u1106 I basically copied the ruote-beanstalk code to our ruote-amqp:
2011-09-29 05:25:43 utc jmettraux well, it's rather as long as the proxy and the remote agree on the format
2011-09-29 05:25:44 utc jmettraux the engine sees nothing of that business
2011-09-29 05:25:59 utc u1106 that's waht I meant, poor wording on my side
2011-09-29 05:27:08 utc u1106 if item.has_key?('fields') and item['fields'].has_key?('__error__')
2011-09-29 05:27:08 utc u1106 # a workitem that resulted in an error
2011-09-29 05:27:08 utc u1106 @context.error_handler.action_handle('dispatch', item['fei'],
2011-09-29 05:27:08 utc u1106 ReceiveError.new(item['fei'],item['fields']['__error__']))
2011-09-29 05:27:08 utc u1106 else
2011-09-29 05:27:08 utc u1106 receive(item) # workitem resumes in its process instance
2011-09-29 05:27:50 utc u1106 so the __error__ thing is from our wire protocol, that works fine
2011-09-29 05:28:30 utc u1106 the error object is also created, but when ruote-kit tries to display it, it gets an execption
2011-09-29 05:29:08 utc jmettraux do you have details ?
2011-09-29 05:29:22 utc u1106 of the exception?
2011-09-29 05:30:09 utc jmettraux yes
2011-09-29 05:30:48 utc u1106 not handy right now, I have hacked the code since then, but I have it in git so I can reproduce it
2011-09-29 05:31:16 utc jmettraux ok
2011-09-29 05:33:06 utc u1106 If I guessed it right ruote-kit tried to access a date attribute which wasn't there and then Ruby complained that nil had no dup method
2011-09-29 05:34:02 utc jmettraux ok
2011-09-29 05:34:36 utc jmettraux if I see the details I can determine where the problem is and then we can fix it
2011-09-29 05:35:10 utc jmettraux I should probably make ruote-kit more forgiving
2011-09-29 05:35:27 utc u1106 anywhy I compared the error object created by the code above to an error object created by and explicit error exp in the workflow, and they were very different
2011-09-29 05:35:49 utc u1106 the force error had an embedded workitem
2011-09-29 05:36:23 utc u1106 the one created by the code above did not contain the embedded workitem
2011-09-29 05:37:01 utc u1106 so I guess ruote-kit might have wanted to access the embedded workitem (guessing does not mean reading the source ;)
2011-09-29 05:38:13 utc u1106 generally: should an error object contain the workitem from the time when the error happened?
2011-09-29 05:39:11 utc jmettraux I just looked at https://github.com/jmettraux/ruote/blob/master/lib/ruote/svc/error_handler.rb#L54-67 it grabs the workitem by itself
2011-09-29 05:39:49 utc jmettraux provided the "fei" is the right one
2011-09-29 05:42:14 utc u1106 hmm, don't you overwrite the fei parmater in this code?
2011-09-29 05:42:43 utc u1106 sorry nonsense, I looked wrong
2011-09-29 05:44:02 utc jmettraux if the fei is wrong, fexp will be nil, and NilClass has no h method, so it's another problem surfacing in ruote-kit
2011-09-29 05:44:33 utc u1106 ok, have to get that code from git and try again
2011-09-29 05:52:35 utc u1106 BTW: I have seen 2 fei attributes in workitems. one in the "top level" and another one in fields.paramaters. Is that by design? Are they always the same ones (I haven't checked)
2011-09-29 05:54:35 utc jmettraux I can't remember writing any code that puts feis in fields.parameters, sorry
2011-09-29 05:57:54 utc u1106 ok, I'll keep an eye on it. If I see it again I can check were it comes from. But at least that means using the fei from the top level when calling error_handler.action_dispatch should be the right thing to do
2011-09-29 06:00:32 utc jmettraux +1
2011-09-29 06:11:40 utc u1106 BTW you have worked with JSON a lot, is there an easy way to pretty print (auto indent) a JSON message so the structure gets readable for humans (would be easy to write one, but I was so sure that it must exist that I didn't start to do it...)
2011-09-29 06:13:24 utc jmettraux Rufus::Json.pretty_encode(your_array_or_hash)
2011-09-29 06:13:30 utc jmettraux or else
2011-09-29 06:13:34 utc u1106 thanks
2011-09-29 06:13:43 utc jmettraux require 'pp'; pp your_array_or_hash
2011-09-29 06:13:59 utc jmettraux not JSON, but close
2011-09-29 06:14:03 utc jmettraux if you like colours:
2011-09-29 06:14:24 utc jmettraux gem install awesome_print
2011-09-29 06:14:26 utc jmettraux then
2011-09-29 06:14:36 utc jmettraux require 'ap'; ap whatever
2011-09-29 06:15:01 utc jmettraux pp is available with all the rubies
2011-09-29 06:18:08 utc u1106 http://www.pastie.org/2609593
2011-09-29 06:21:34 utc u1106 I was wrong when I said the work item is not embedded, it actually is. For some reason the forced error is ~1800 bytes, while "my own" one is ~880 bytes for the same work item, need to look again what the difference really is
2011-09-29 06:24:57 utc jmettraux error#at returns nil probably
2011-09-29 06:29:31 utc jmettraux preparing a fix
2011-09-29 06:47:27 utc jmettraux u1106: this should help: https://github.com/jmettraux/ruote/commit/349b049decddbb068053f1beb92014d43d20c173
2011-09-29 06:50:25 utc u1106 thanks, I'll try
2011-09-29 06:58:35 utc u1106 hmm, just cloned the whole tree, but your commit did not come. (Not so familiar with github), of course this is a small patch and I could just copy paste it from the web interface
2011-09-29 07:01:07 utc jmettraux which command did you use to clone ?
2011-09-29 07:01:43 utc u1106 git clone https://github.com/jmettraux/ruote.git
2011-09-29 07:01:47 utc u1106 nope
2011-09-29 07:01:54 utc u1106 the git one
2011-09-29 07:02:06 utc u1106 git://github.com/jmettraux/ruote.git
2011-09-29 07:02:30 utc u1106 oops sorry, my bad. Wrong virtual machine
2011-09-29 07:03:20 utc u1106 I have several virtual machines here. I cloned the tree to the vm where I have ruote running, but tried to find the commit in another vm
2011-09-29 07:04:02 utc u1106 of course it's tehre
2011-09-29 07:21:55 utc u1106 jmettraux: thanks a lot, the ruote-kit error is gone
2011-09-29 07:23:42 utc jmettraux ah, great, you're welcome !
2011-09-29 07:33:44 utc u1106 unrelated question: last week I when learning about ruote somewhere I read that field.__result__ has a special meaning. (Whether the task was successful or not) It thought I read it on http://ruote.rubyforge.org/ But when trying to find that paragraph again I was unable to do so (and google, too). Is that documented somewhere or could I possibly have read it as a comment in one of our own source files? I'm quite sure I did not dream it...
2011-09-29 07:36:02 utc jmettraux it's used in two rare instances
2011-09-29 07:36:23 utc jmettraux a) it gathers the block participant last line result
2011-09-29 07:37:07 utc jmettraux b) expressions like '_if' and 'given' use it when deciding which branch to follow
2011-09-29 07:37:20 utc jmettraux I just grepped in the website source but it's not mentioned
2011-09-29 07:37:28 utc jmettraux sorry about that
2011-09-29 07:42:51 utc u1106 Somewhere it must be mentioned somewhere, otherwise I wouldn't have learned about it its existence. b.) already answers my next question, which would have been whether the result has any effect to the workflow :)
2011-09-29 07:44:37 utc u1106 there is also an __error__ field (at least we have it in our code, not 100% sure whether it exists in ruote implementation) If it is a general thing, does that one have effects to the workflow?
2011-09-29 07:46:06 utc jmettraux not really
2011-09-29 07:46:52 utc jmettraux there is no expression that uses it
2011-09-29 07:47:12 utc jmettraux except for the "listen" one
2011-09-29 07:47:29 utc u1106 so it's 100% our own implemetation?
2011-09-29 07:47:57 utc u1106 except? so it's not our own implementation
2011-09-29 07:48:30 utc jmettraux you shouldn't bother about __result__ and __error__, they are reserved to ruote
2011-09-29 07:48:57 utc jmettraux of course, you're free to set them, but that might interfere
2011-09-29 07:49:59 utc u1106 ok. I bothered about them because we have participants written in Python and they communicate via AMQP
2011-09-29 07:50:43 utc u1106 so I guess we must follow ruote internals well on the Python side, otherwise we do interfere in unexpected ways
2011-09-29 07:50:59 utc jmettraux not really
2011-09-29 07:51:26 utc jmettraux those special fields are surrounded by __ __ don't touch them
2011-09-29 07:51:29 utc jmettraux that's all
2011-09-29 07:54:55 utc u1106 Ok they have accessors result and error so touching the __ stuff is not nice. But whatever way it's done it will end up __result__ or __error__ being set
2011-09-29 07:55:39 utc u1106 if you say don't touch them my error implementation I pasted here this morning is wrong
2011-09-29 07:55:57 utc jmettraux I may be mistaken
2011-09-29 07:56:08 utc u1106 because it works by setting field.__error__
2011-09-29 07:56:18 utc jmettraux ok, so be ti
2011-09-29 07:56:21 utc jmettraux so be it
2011-09-29 07:57:08 utc jmettraux It was simply not my intention to force people to know too much about the "internals" in order to use ruote
2011-09-29 07:57:39 utc jmettraux also, I'm sorry, but I don't see your code, so I'm following you remotely
2011-09-29 07:57:46 utc u1106 I definitely had somewhat vague feelings when doing it (lbt suggested it, maybe after discussing with you) , that's why I asked where it's documented
2011-09-29 07:58:21 utc jmettraux ok, let me look at the code of ruote-amqp
2011-09-29 07:59:15 utc u1106 yes, I fully agree "normal" ruote users don't need to know that. But we are not "normal" because our Python side needs to know a bit more about internals
2011-09-29 07:59:42 utc jmettraux seems you only need to know about the internals of ruote-amqp
2011-09-29 08:00:43 utc jmettraux I see nothing about __result__ or __error__ in ruote-amqp
2011-09-29 08:00:45 utc u1106 not the participant logic if we do it right, but our wrappers around AMQP (analgogously to ruote-beanstalk) need to know about "hidden" internals I guess
2011-09-29 08:01:29 utc jmettraux maybe you need to clean any __result__ or __error__ fields before giving a workitem back to ruote proper
2011-09-29 08:01:34 utc u1106 __result__ just has accessors
2011-09-29 08:02:16 utc u1106 __error__ might not have been there before, but I added it now to signal an error
2011-09-29 08:02:46 utc jmettraux so what's happening is that you guys are using fields to pass more informations ?
2011-09-29 08:03:09 utc u1106 on the receiver side it looks like this:
2011-09-29 08:03:12 utc u1106 if item.has_key?('fields') and item['fields'].has_key?('__error__')
2011-09-29 08:03:12 utc u1106 (08:26:12) u1106: # a workitem that resulted in an error
2011-09-29 08:03:12 utc u1106 (08:26:12) u1106: @context.error_handler.action_handle('dispatch', item['fei'],
2011-09-29 08:03:35 utc u1106 (sorry my irc timestamps from earlier)
2011-09-29 08:03:56 utc jmettraux why not doing something like "if item['__error__']" ie placing the __error__ in the workitem top level and not in fields ?
2011-09-29 08:04:49 utc u1106 yes that's what we do. And as I said I had a somewhat vague feeling when doing it, that's why I was interested to understand how it works
2011-09-29 08:05:12 utc jmettraux ah ok
2011-09-29 08:05:41 utc u1106 sorry, I cannot answer to your last question. That was done by lbt before I started on the project
2011-09-29 08:06:16 utc jmettraux ok, understood
2011-09-29 08:07:00 utc u1106 I was somewhat surprised to learn that ruote-beanstalk uses a special wire protocol. They singnal the error outside of the workitem as a message type, if I read the code correctly
2011-09-29 08:07:30 utc u1106 we signal the error inside the workitem, even inside the fields
2011-09-29 08:08:03 utc u1106 these are just observations, I'm to new to the system that I would have opinions
2011-09-29 08:08:21 utc u1106 besides the opinion that we should not do dangerous stuff ;)
2011-09-29 08:08:26 utc jmettraux lol
2011-09-29 08:08:30 utc jmettraux no worries
2011-09-29 08:08:51 utc jmettraux opinions of people new to systems are very valuable
2011-09-29 08:09:09 utc jmettraux so don't worry too much
2011-09-29 08:12:07 utc u1106 anyway thanks a lot for all your time here
2011-09-29 08:12:31 utc jmettraux thanks for catching the issue, much appreciated
2011-09-29 08:16:17 utc u1106 I didn't catch it, I just hit it and ruby catched that nil has no dup method. If I still work a couple of weeks with the code I might be able to catch such trivial issues myself
2011-09-29 12:30:27 utc ruote_logger u1106-office: (memo from jmettraux at Thu Sep 29 06:29:46 +0900 2011) if there is no one replying here, consider using the mailing list: http://groups.google.com/group/openwferu-users