ruote log_2010-09-28

2010-09-28 05:17:30 utc jmettraux dgoodlad: welcome on #ruote
2010-09-28 05:17:40 utc dgoodlad Hi jmettraux
2010-09-28 05:18:02 utc dgoodlad Are you actually around or is that an auto-welcome bot? :)
2010-09-28 05:18:09 utc jmettraux I'm around, hello
2010-09-28 05:18:49 utc dgoodlad I'm running into a bit of an issue, and have found a fix but would like to confirm that it's not breaking other things
2010-09-28 05:19:00 utc jmettraux OK
2010-09-28 05:19:40 utc dgoodlad For some reason when a workitem flows through a concurrence section of the process, it ends up breaking Engine::processes
2010-09-28 05:20:03 utc jmettraux do you have a way for me to reproduce the issue ?
2010-09-28 05:20:08 utc dgoodlad yes, one second
2010-09-28 05:21:59 utc dgoodlad so, this is the patch I had to make to make it work again: http://gist.github.com/600451
2010-09-28 05:22:11 utc dgoodlad and I'll grab the code that was breaking it, one more minute :)
2010-09-28 05:23:39 utc dgoodlad https://gist.github.com/4793154a6f44e7170989 << that is our list of participants and the process def
2010-09-28 05:24:02 utc dgoodlad everything works fine when you fire up the process initially, the workitem sits at the validator (storage participant)
2010-09-28 05:24:47 utc dgoodlad then if you proceed past the validator participant, Engine::processes fails at the line I patched, complaining that Hash has no method 'owner'
2010-09-28 05:25:07 utc jmettraux great report
2010-09-28 05:25:23 utc jmettraux let me package that with a test
2010-09-28 05:25:25 utc jmettraux many thanks !
2010-09-28 05:25:49 utc dgoodlad you're welcome :)
2010-09-28 05:26:07 utc dgoodlad I am very unfamiliar with the internals of ruote, so I wasn't sure if what I changed was actually right
2010-09-28 05:26:29 utc jmettraux spot on, it's just ruby code after all
2010-09-28 05:27:05 utc dgoodlad it appears that those 'schedules' passed to ProcessStatus#initialize are always hashes with 'owner' keys, not objects with 'owner' methods
2010-09-28 05:27:10 utc dgoodlad Is that correct?
2010-09-28 05:27:25 utc jmettraux yes
2010-09-28 05:28:24 utc dgoodlad great, I'll keep this patch applied here then! :)
2010-09-28 05:28:36 utc jmettraux fixing for 2.1.11 here
2010-09-28 05:28:46 utc dgoodlad sorry I didn't supply a test, but we were more just fiddling with it to 'make it work' than anything else
2010-09-28 05:28:54 utc jmettraux no worries
2010-09-28 05:28:58 utc jmettraux it helps a lot
2010-09-28 05:31:49 utc jmettraux seems already fixed on 2.1.11 (edge)
2010-09-28 05:32:16 utc jmettraux trying with 2.1.10
2010-09-28 05:32:53 utc dgoodlad oh really? weird... I was using rev b1d1046b60b4 and it was broken
2010-09-28 05:35:23 utc jmettraux let me double check something
2010-09-28 05:35:46 utc dgoodlad Forgot to push? ;)
2010-09-28 05:36:14 utc jmettraux no no
2010-09-28 05:36:30 utc jmettraux ProcessStatus is only instantianted in lib/ruote/engine.rb, no other entry points
2010-09-28 05:36:40 utc dgoodlad hmm
2010-09-28 05:36:54 utc jmettraux are you using bundler ?
2010-09-28 05:36:57 utc dgoodlad yes
2010-09-28 05:37:06 utc jmettraux no residual gem ?
2010-09-28 05:37:30 utc dgoodlad ~/Work/Trike/JasperDocs(new_workflow*) $ gem list ruote
2010-09-28 05:37:31 utc dgoodlad *** LOCAL GEMS ***
2010-09-28 05:37:37 utc dgoodlad but...
2010-09-28 05:38:04 utc jmettraux wait, I think it's my fault, but I can't find a way to reproduce the issue
2010-09-28 05:38:24 utc dgoodlad I have two ruote-* versions in $GEM_HOME/bundler/gems/
2010-09-28 05:38:29 utc dgoodlad I'll rm the older one
2010-09-28 05:38:50 utc jmettraux it shouldn't be that
2010-09-28 05:39:02 utc dgoodlad yeah come to think of it, the backtraces pointed at the correct directory
2010-09-28 05:39:13 utc jmettraux it's just me who can't reproduce the thing
2010-09-28 05:39:16 utc dgoodlad hehe ok
2010-09-28 05:39:28 utc dgoodlad we're using ruote-kit to set things up in a rails app
2010-09-28 05:39:35 utc dgoodlad and fs storage for the engine and participants
2010-09-28 05:40:31 utc dgoodlad https://gist.github.com/63db5dbb4d443e1b9eed
2010-09-28 05:40:35 utc dgoodlad that's the backtrace I get
2010-09-28 05:47:44 utc jmettraux got it
2010-09-28 05:51:37 utc dgoodlad oh?]
2010-09-28 05:53:37 utc jmettraux dgoodlad: many thanks http://github.com/jmettraux/ruote/commit/fa3ebdcbe164903ad94262a228aea53642990f6a
2010-09-28 05:56:24 utc dgoodlad jmettraux: awesome, thanks for including so quickly!
2010-09-28 06:04:33 utc tosch_le good morning!
2010-09-28 06:04:41 utc tosch_le did i hear patches?
2010-09-28 06:05:51 utc jmettraux tosch_le: good morning, yeah dgoodlad just contributed a patch
2010-09-28 06:06:16 utc tosch_le ah, yes, as seen on github :-)
2010-09-28 06:06:24 utc tosch_le thanks, dgoodlad!
2010-09-28 06:06:54 utc dgoodlad tosch_le: np :) that's why open source is nice, I can fix bugs in libraries when I hit them
2010-09-28 12:20:52 utc jmettraux lbt: hello
2010-09-28 12:21:32 utc lbt jmettraux: hi
2010-09-28 12:21:45 utc jmettraux how are you doing ?
2010-09-28 12:21:55 utc lbt I wish I had more time :)
2010-09-28 12:22:18 utc lbt we are pushing ahead with BOSS and ruote
2010-09-28 12:22:24 utc jmettraux have you seen such a message "NOT_ALLOWED - parameters for queue 'test1' in vhost 'ruote-test' not equivalent in AMQP::Protocol::Queue::Declare" ?
2010-09-28 12:22:40 utc lbt no
2010-09-28 12:22:42 utc jmettraux (I'm trying ruote-amqp with rabbitmq 2.1.0
2010-09-28 12:22:45 utc jmettraux OK
2010-09-28 12:22:50 utc jmettraux how's the push ?
2010-09-28 12:23:01 utc lbt actually... second...
2010-09-28 12:23:32 utc lbt amqplib.client_0_8.exceptions.AMQPChannelException: (403, u"ACCESS_REFUSED - access to queue 'image_queue' in vhost 'imgvhost' refused for user 'img'", (50, 10), 'Channel.queue_declare')
2010-09-28 12:23:45 utc lbt so, close but no cigar
2010-09-28 12:23:53 utc jmettraux thanks anyway
2010-09-28 12:23:56 utc lbt that was needing rabbitmqctl set_permissions -p imgvhost img '.*' '.*' '.*'
2010-09-28 12:24:17 utc lbt where img is the user
2010-09-28 12:25:04 utc lbt well, the test team are getting used to it... running lots of runs
2010-09-28 12:27:22 utc lbt http://wiki.meego.com/Release_Infrastructure/BOSS/Performance/Results
2010-09-28 12:27:22 utc lbt up to 20k flows
2010-09-28 12:27:22 utc jmettraux ACTION gets reminded he has to fix the ruote wiki
2010-09-28 12:27:22 utc lbt some interesting problems... they find some flows get lost
2010-09-28 12:27:22 utc lbt and a couple of crashes
2010-09-28 12:27:25 utc jmettraux I wish they were documented
2010-09-28 12:27:27 utc lbt 19995 of 20000 workflows arrived
2010-09-28 12:27:34 utc lbt yeah. I'm nagging them :)
2010-09-28 12:28:01 utc jmettraux very nice output, many thanks
2010-09-28 12:28:17 utc lbt yeah... pretty.... but I'm not sure what it says :)
2010-09-28 12:28:54 utc lbt the point for me is that is demonstrates scaling enough to satisfy the business
2010-09-28 12:29:01 utc jmettraux yes, inspiring anyway
2010-09-28 12:29:22 utc lbt we're looking at ruote-kit for the webui side
2010-09-28 12:29:49 utc lbt and something called Alice/Wonderland for rabbitmq monitoring (but that's erlang/qjuery)
2010-09-28 12:30:15 utc lbt had a couple of issues with r-k but no time to sanitise them for you
2010-09-28 12:30:47 utc lbt mem usage went >1Gb when the fs-store had a huge backlog of wflows
2010-09-28 12:31:12 utc jmettraux so you have issues with rk and issues with the fs storage
2010-09-28 12:31:25 utc lbt NoMethodError at /_ruote/processes
2010-09-28 12:31:26 utc lbt undefined method `position' for #
2010-09-28 12:31:42 utc jmettraux ok
2010-09-28 12:32:08 utc lbt yes... possibly lost wflows in testing
2010-09-28 12:32:32 utc lbt and this error which I'm blaming on the fact that we're halfway between releases atm
2010-09-28 12:33:01 utc jmettraux ok
2010-09-28 12:33:12 utc lbt our focus at the moment is the actual process - ruote/amqp is generally being fine :) .... so thanks
2010-09-28 12:33:46 utc lbt oh, and I've deployed ruote on meego.com too ... FYI
2010-09-28 12:33:53 utc jmettraux cool
2010-09-28 12:34:10 utc lbt yeah... the qa managers are really very interested
2010-09-28 12:34:32 utc lbt automated updates to bugzilla when changelogs with bug# passes into accepted...
2010-09-28 12:35:31 utc lbt I'm trying to get us a ruby developer too... fingers crossed
2010-09-28 12:36:09 utc jmettraux is it so hard to find them ?
2010-09-28 12:36:28 utc lbt no, to justify and get extra team resource :)
2010-09-28 12:42:14 utc jmettraux damn, rabbitmq is a timesink
2010-09-28 12:54:12 utc jmettraux yeah, the ruby code of your team is exotic
2010-09-28 13:07:00 utc jmettraux lbt: thanks anyway for the output, but it's a bit of a pain in the ass for me to make sense of the bench
2010-09-28 13:07:28 utc jmettraux if you guys want fixes, give me steps in plain english, it will be easier
2010-09-28 13:07:43 utc lbt *nod* ... I wouldn't touch it for now :)
2010-09-28 13:08:01 utc lbt yes... I've asked them to narrow down and reproduce
2010-09-28 13:08:39 utc jmettraux great
2010-09-28 13:08:56 utc lbt I am hoping to do a review of the amqp stuff soon
2010-09-28 13:09:12 utc lbt I would like to keep it independent of daemon-kit
2010-09-28 13:09:24 utc lbt that's a lot of overhead for people that don't need it
2010-09-28 13:09:52 utc lbt thoughts?
2010-09-28 13:09:57 utc jmettraux ruote-amqp ? or your own amqp stuff ?
2010-09-28 13:10:02 utc lbt ruote-amqp
2010-09-28 13:10:26 utc jmettraux it's overhead for me because I have to maintain it and I don't use it
2010-09-28 13:13:34 utc jmettraux sorry I'm mad because I just got into ruote-amqp and the specs are broken (though nothing changed)
2010-09-28 13:14:01 utc lbt I'd be happy to try and help maintain r-a
2010-09-28 13:14:40 utc jmettraux I'm afraid of the meego flavour it'd take
2010-09-28 13:15:06 utc jmettraux maybe I should just not care
2010-09-28 13:15:20 utc lbt heh, I'd rather you cared
2010-09-28 13:16:41 utc lbt we considered it to be a proxy interface at one point
2010-09-28 13:17:24 utc jmettraux in which sense ?
2010-09-28 13:18:07 utc lbt ruote does something and amqp is there to pass that along to a participant that thinks it is local
2010-09-28 13:18:26 utc jmettraux indeed
2010-09-28 13:18:35 utc lbt so it's a translucent distribution interface
2010-09-28 13:18:48 utc lbt transparent would be great ... but... :)
2010-09-28 13:19:27 utc jmettraux for me ruote-amqp should limit itself to what its name implies : the boundary between ruote and amqp
2010-09-28 13:19:37 utc jmettraux things go out, things come back
2010-09-28 13:19:58 utc lbt I see the signals sent by ruote as roughly "here's a wi" and "cancel"
2010-09-28 13:20:10 utc jmettraux +1
2010-09-28 13:20:12 utc lbt and it listens for an incoming wi
2010-09-28 13:20:37 utc lbt my mental model for a participant is 2-threads
2010-09-28 13:20:50 utc lbt worker and a "listen for cancel" thread
2010-09-28 13:21:10 utc jmettraux +1
2010-09-28 13:21:22 utc lbt listen for cancel is generically "control" and may include interrogation one day (50% done)
2010-09-28 13:21:31 utc jmettraux ah yes, that's true, ruote-amqp also proposes a participant "framework"
2010-09-28 13:22:14 utc lbt you mean inside the consume() ?
2010-09-28 13:22:34 utc jmettraux wait, let me check
2010-09-28 13:22:36 utc lbt (nb everything ^^^ is ruby agnostic)
2010-09-28 13:23:01 utc jmettraux I'm wrong, please forget my "participant framework" remark
2010-09-28 13:23:28 utc lbt OK
2010-09-28 13:23:34 utc lbt good :)
2010-09-28 13:23:50 utc lbt I honestly think our only contention was the remote-registration
2010-09-28 13:24:17 utc jmettraux ah yes
2010-09-28 13:24:21 utc lbt which ~ to a local Participant having access to the Storage/Engine ?
2010-09-28 13:25:04 utc jmettraux well
2010-09-28 13:25:10 utc lbt I need a solution for it but if you really think it doesn't happen like that then we can re-design it
2010-09-28 13:25:12 utc jmettraux if you want
2010-09-28 13:26:34 utc jmettraux seems so far now
2010-09-28 13:27:18 utc lbt I think ruote-kit may be a good answer actually
2010-09-28 13:31:25 utc lbt actually jmettraux; I think what I was after is ruote as a process-engine; not a process library
2010-09-28 13:31:46 utc jmettraux ?
2010-09-28 13:32:01 utc lbt where I can have multiple remote collections of processes (under VCS) for different teams
2010-09-28 13:32:25 utc lbt they can all register their process(es) and participants and launch them
2010-09-28 13:32:46 utc lbt and ruote dynamically gains (and loses) processes and participants
2010-09-28 13:33:20 utc jmettraux so you think that ruote is a process library or a process engine currently ?
2010-09-28 13:33:32 utc lbt the code is an engine
2010-09-28 13:33:55 utc lbt most processes are instantiated in applications which call library functions
2010-09-28 13:34:32 utc lbt if I want to change the process I stop the code(and worker), change and restart
2010-09-28 13:34:36 utc jmettraux yeah, I have a mindset where participants are re-used across processes, might not fit all situations
2010-09-28 13:34:36 utc lbt (I think)
2010-09-28 13:34:57 utc jmettraux so you want a meta-engine ?
2010-09-28 13:35:04 utc lbt I use it as such today :)
2010-09-28 13:35:33 utc lbt which explains why remote registration (even via a workerless ruote-kit) is good for me
2010-09-28 13:36:09 utc jmettraux great
2010-09-28 13:36:47 utc jmettraux remote registration is easy to implement via a coordination between two participants
2010-09-28 13:37:09 utc jmettraux the remote participant passes the detail to the local participant which does the registration locally
2010-09-28 13:37:35 utc lbt heh... I think that's how my very first implementation started...
2010-09-28 13:37:44 utc lbt meta-process indeed
2010-09-28 13:37:52 utc jmettraux virtual participants are fun too
2010-09-28 13:38:18 utc lbt virtual ?
2010-09-28 13:38:50 utc jmettraux that's what I was trying to explain you all along a few months back
2010-09-28 13:39:16 utc lbt ACTION hangs his head
2010-09-28 13:39:36 utc jmettraux real inbox, all the emails
2010-09-28 13:39:49 utc jmettraux virtual inbox, all the emails that contain the word "fish"
2010-09-28 13:40:14 utc jmettraux sorry, another way to skin a cat
2010-09-28 13:40:38 utc lbt *nod*... I recall
2010-09-28 13:41:07 utc jmettraux you had an objection with that, but I can't remember
2010-09-28 13:41:34 utc lbt yes... too visible iirc
2010-09-28 13:41:45 utc lbt I like the readability of processes
2010-09-28 13:43:16 utc lbt and I think multiplexing came up too....
2010-09-28 13:43:49 utc jmettraux kennethkalmer: hello
2010-09-28 13:43:54 utc lbt his ears.... they burn....
2010-09-28 13:44:03 utc jmettraux lol
2010-09-28 13:44:10 utc kennethkalmer hi jmettraux, lbt, et al :)
2010-09-28 13:44:12 utc kennethkalmer coffee ?
2010-09-28 13:44:22 utc lbt always good
2010-09-28 13:44:27 utc jmettraux you have five minutes for a rabbitmq question ?
2010-09-28 13:44:34 utc jmettraux espresso please :-)
2010-09-28 13:44:45 utc kennethkalmer definitely, let me prep those drinks and then we figure it out
2010-09-28 13:46:47 utc jmettraux I have ruby 1.8.7p249, amqp 0.6.7, rabbitmq 2.1.0 (same issue with 1.7.x)
2010-09-28 13:47:21 utc jmettraux amqp is calling
2010-09-28 13:47:22 utc kennethkalmer ACTION emits espresso's
2010-09-28 13:47:24 utc jmettraux @mq.send(Protocol::Queue::Declare.new({ :queue => name,
2010-09-28 13:47:25 utc jmettraux :nowait => true }.merge(opts)))
2010-09-28 13:47:29 utc jmettraux multiple times
2010-09-28 13:48:35 utc jmettraux ACTION grabs the espresso, thanks !
2010-09-28 13:48:49 utc jmettraux for example : http://gist.github.com/601022
2010-09-28 13:49:19 utc lbt ah :durable=>true....
2010-09-28 13:49:30 utc tosch_le ACTION grabs an espresso, too, thanks!
2010-09-28 13:49:32 utc kennethkalmer yeah
2010-09-28 13:49:37 utc jmettraux there's only one spec, but the call is done multiple times, with :durable => true, but not the first time
2010-09-28 13:49:41 utc jmettraux tosch_le : hello
2010-09-28 13:49:44 utc kennethkalmer think the queue has to be declared durable the first time
2010-09-28 13:49:51 utc jmettraux and it complains
2010-09-28 13:50:00 utc jmettraux let me add the caller output
2010-09-28 13:51:03 utc kennethkalmer k
2010-09-28 13:51:18 utc jmettraux http://gist.github.com/601031
2010-09-28 13:53:36 utc kennethkalmer peculiar, having a look around the code again
2010-09-28 13:53:42 utc kennethkalmer is this on anyones fork ?
2010-09-28 13:54:08 utc jmettraux it's my fork
2010-09-28 13:55:11 utc jmettraux used to work
2010-09-28 13:55:50 utc jmettraux last time I pushed
2010-09-28 13:57:31 utc lbt http://github.com/jmettraux/ruote-amqp/commit/e55999ec49ad4920ac468b66fba1a906639b14cc#L0R155 ?
2010-09-28 13:58:07 utc lbt no... maybe not
2010-09-28 13:58:08 utc jmettraux it worked at that point
2010-09-28 13:58:30 utc jmettraux I was looking at the same commit
2010-09-28 13:59:36 utc lbt I think the issue is you should be using MQ.queue( 'test7', :durable => true ) ... to keep with Ruote's "thou shalt not lose data"
2010-09-28 14:00:01 utc lbt but as to why it used to work....
2010-09-28 14:00:17 utc kennethkalmer ah, I have a guess that when that queue was created the first time (by r-a or another client) it wasn't created with durable
2010-09-28 14:00:23 utc lbt ah good point
2010-09-28 14:00:28 utc jmettraux it's the point
2010-09-28 14:00:29 utc kennethkalmer restarting rabbit should solve it
2010-09-28 14:00:37 utc kennethkalmer i'm just guessing
2010-09-28 14:00:46 utc jmettraux ok
2010-09-28 14:00:51 utc kennethkalmer let me run the code on this side quickly against rabbit 2.0.0
2010-09-28 14:01:08 utc lbt you may find alice useful at some point
2010-09-28 14:01:24 utc lbt http://github.com/auser/alice
2010-09-28 14:01:30 utc jmettraux restarted, same issue
2010-09-28 14:01:42 utc jmettraux lbt : thanks
2010-09-28 14:04:09 utc jmettraux OK fixed
2010-09-28 14:04:36 utc kennethkalmer wow
2010-09-28 14:04:39 utc kennethkalmer how ?
2010-09-28 14:04:52 utc jmettraux as you said
2010-09-28 14:05:14 utc jmettraux http://github.com/jmettraux/ruote-amqp/blob/ruote2.1/spec/workitem_listener_spec.rb#L35
2010-09-28 14:05:19 utc jmettraux adding :durable => true
2010-09-28 14:05:47 utc jmettraux many thanks guys
2010-09-28 14:06:06 utc kennethkalmer pleasure, please send me a pull request when you're 100% done
2010-09-28 14:06:09 utc lbt np
2010-09-28 14:06:34 utc lbt BTW... may want to be careful about some of the promises amqp makes about persistence
2010-09-28 14:06:38 utc jmettraux OK, I'm handling coffeaddict_nl's stuff as well
2010-09-28 14:06:44 utc jmettraux aah
2010-09-28 14:06:51 utc lbt I'm not sure amqplib respects them 100%
2010-09-28 14:07:46 utc lbt somewhere down the line I need to run some torture tests... I'm sure rabbit is fine... but just saying
2010-09-28 14:07:59 utc jmettraux OK
2010-09-28 14:08:16 utc kennethkalmer jmettraux & tosch_le: http://gist.github.com/601056
2010-09-28 14:08:24 utc kennethkalmer can I push that up to ruote-kit ?
2010-09-28 14:09:18 utc jmettraux what does it change ?
2010-09-28 14:09:44 utc tosch_le i'd say: feel free to do so
2010-09-28 14:09:53 utc kennethkalmer it registers the respond_to plugin only with RuoteKit::Application, not globally with Sinatra
2010-09-28 14:10:15 utc kennethkalmer i'm currently embedding rk into another sinatra app and it is causing havoc for me
2010-09-28 14:11:02 utc kennethkalmer keeping respond_to locally to rk solves all my issues and doesn't affect rk negatively
2010-09-28 14:11:05 utc jmettraux could it be related ? http://github.com/jmettraux/ruote-kit/blob/master/lib/ruote-kit/helpers/render_helpers.rb#L45-50
2010-09-28 14:12:08 utc jmettraux please push
2010-09-28 14:12:30 utc kennethkalmer thanks
2010-09-28 14:13:22 utc jmettraux tosch_le: how should we resync our repos ?
2010-09-28 14:13:49 utc lbt wow... there was some latency on that coffee... only just got here!
2010-09-28 14:13:56 utc lbt ACTION goes to sip
2010-09-28 14:13:56 utc jmettraux lol
2010-09-28 14:14:00 utc kennethkalmer :)
2010-09-28 14:14:01 utc lbt l8r :)
2010-09-28 14:15:05 utc tosch_le jmettraux: should be simple: just pull & push
2010-09-28 14:15:30 utc jmettraux OK
2010-09-28 14:16:02 utc kennethkalmer git ftw
2010-09-28 14:16:16 utc tosch_le :-)
2010-09-28 14:17:08 utc tosch_le kennethkalmer push *now* or my repo will have to wait till tomorrow ;-)
2010-09-28 14:17:21 utc kennethkalmer busy busy :)
2010-09-28 14:17:31 utc kennethkalmer waiting for bundle install so I can run tests
2010-09-28 14:18:26 utc jmettraux my ruote-amqp is back in the green, pushed
2010-09-28 14:18:29 utc tosch_le ok, have to leave now.
2010-09-28 14:18:37 utc kennethkalmer tosch_le: cheers !
2010-09-28 14:18:54 utc jmettraux tosch_le : cheers !
2010-09-28 14:19:02 utc tosch_le kennethkalmer: don't forget to pull before you commit ;-) (my usual mistake)
2010-09-28 14:19:11 utc tosch_le bye and thanks!
2010-09-28 14:19:21 utc kennethkalmer ciao !
2010-09-28 14:29:21 utc jmettraux kennethkalmer: good to have you back in the game
2010-09-28 14:29:31 utc kennethkalmer good to be back
2010-09-28 14:29:48 utc kennethkalmer starting to strike the balance between work and foss again, and it is soooooooooo overdue
2010-09-28 14:36:41 utc kennethkalmer having epic timeouts with github :/
2010-09-28 14:45:31 utc kennethkalmer signing out for today
2010-09-28 14:45:39 utc kennethkalmer will see you guys tomorrow !
2010-09-28 14:45:53 utc kennethkalmer have a spectacular evening :)