[27/06/2013 10:39]  hi people!
[27/06/2013 10:40]  anyone alive? :)
[27/06/2013 10:41]  I'd like to know if ruote is useful for my open
source project
[27/06/2013 10:41]  with some kind of special requirements. I'm willing
to develop any missing piece too
[27/06/2013 10:41]  (if worthwhile)
[27/06/2013 10:48]  in short, my main question is: can I create a
decentralized ruote system? so that I don't need a fixed set of workers and a
central orchestrator?
[27/06/2013 10:49]  I'd also need a secure way to communicate between
nodes, too =)
[27/06/2013 10:49]  (ssl or similar would probably do the trick)
[27/06/2013 10:49]  hartog: kubicek, lbt, uxp, weeb1e? :-P
[27/06/2013 10:51]  hey edulix
[27/06/2013 10:52]  hi lbt how are ya doing
[27/06/2013 10:52]  good :) and yes, it's a quiet channel depending on TZ
[27/06/2013 10:53]  decentralised or distributed?
[27/06/2013 10:53]  or resilient?
[27/06/2013 10:53]  lbt: decentralized
[27/06/2013 10:53]  we use ruote with AMQP to do worker/engine
[27/06/2013 10:53]  lbt: like email, for example
[27/06/2013 10:54]  so each ruote process runs on a single engine
[27/06/2013 10:54]  lbt: I'm not yet very familiar with your terminology
[27/06/2013 10:54]  lbt: what's an engine?
[27/06/2013 10:55]  the engine steps through the process definition and
hands tasks to participants which run on workers
[27/06/2013 10:56]  lbt: well let me just introduce you the general idea
of what I have to do, maybe this way is easier?
[27/06/2013 10:56]  it manages sending/receiving the worksheets and merges
responses
[27/06/2013 10:56]  yup
[27/06/2013 10:57]  okey, so I'm working in agora  ciudadana (online at
agoravoting.com), a voting system. next version, we're going to add support for
secure encrypted election
[27/06/2013 10:57]  the tally process involves multiple "authorities"
that need to process cooperatively an election
[27/06/2013 10:58]  each authority is as much independent as possible,
because secret of the vote comes from the fact that at least one authority is
honest and has not been hacked
[27/06/2013 10:59] --> MCamou has joined this channel (~Adium@wan.taric.es).
[27/06/2013 10:59]  so, I need a workflow for processing elections.
agoravoting.com main server would ask one of the authorities to orchestrate the
tally, and send the others the tasks that need to be done
[27/06/2013 11:00]  'main server' ?
[27/06/2013 11:00]  sounds like a star network?
[27/06/2013 11:01]  one task for example executes in parallel and
concurrently a process at each authority, this process does the tally (it runs
an external software that does the crypto stuff, comunicating with each authority)
[27/06/2013 11:01]  lbt: well main server might not be the best way to
name it. it's the web server, but there could be other web server accesing the
same authorities
[27/06/2013 11:02]  the authorities work like.. like public repos for
linux distros, they are independent/distributed
[27/06/2013 11:02]  but they cooperate as you can see =)
[27/06/2013 11:03]  I actually started working in my own distributed
workflow system, but if I find something that I can adapt to my usecase, I can
do that too https://github.com/agoraciudadana/frestq
[27/06/2013 11:04]  lbt: so what do you think? :-P
[27/06/2013 11:04] --> mister_solo has joined this channel
(~anonymous@wan.taric.es).
[27/06/2013 11:04]  ruote would be fine in such a topology
[27/06/2013 11:05]  I'm still visualising
[27/06/2013 11:06]  yeah, I understand it's not trivial
[27/06/2013 11:06]  Ruote essentially works in a star setup
[27/06/2013 11:07]  engine talks to participants which do tasks to finish a
process
[27/06/2013 11:07]  'talk to' can be over a network and amqp is probably best
[27/06/2013 11:08]  you can have multiple engines but they would all share
a common DB
[27/06/2013 11:09]  that DB can be redis/mongo etc
[27/06/2013 11:09]  lbt: uhm, is that necesary, that they share all a
common db? I was looking for something more distributed. Like email, I can send
an email to multiple addresses, but not all email servers in the world share the
same email database
[27/06/2013 11:09]  wait :)
[27/06/2013 11:09]  ok hehe
[27/06/2013 11:09]  however a ruote instance like this isn't 'heavy'
[27/06/2013 11:10]  and there is no problem running a task next to the
engine to start processes
[27/06/2013 11:10]  so running a federation of ruote systems is pretty easy
[27/06/2013 11:10]  but not a common use-case
[27/06/2013 11:10]  in fact I'm looking at a federation system myself
[27/06/2013 11:11]  nice
[27/06/2013 11:11]  (we develop a distro and want to federate QA)
[27/06/2013 11:11]  so I'm interested in key-based inter-authentication
[27/06/2013 11:11]  signing of messages etc
[27/06/2013 11:12]  so the initial description talks about how to make
ruote scale/resilient
[27/06/2013 11:12]  but as you say - that's not your main interest
[27/06/2013 11:12]  lbt: in the design of frestq, I send single tasks to
nodes. those task are "simple/single" to the sender of the task. but then the
receiver node can do sees the received task as a sequential task, to which new
subtasks can be added, in composition, and transparently to the sender
[27/06/2013 11:12]  but it shows where the network/APIs lie
[27/06/2013 11:13]  yup
[27/06/2013 11:13]  so you start a "build a car" job which turns out to
have a lot of steps
[27/06/2013 11:13]  exactly
[27/06/2013 11:13] --> chiradeep has joined this channel (~chiradeep@59.92.204.47).
[27/06/2013 11:13]  in the view of agoravoting.com, there would be only
a "tally" job
[27/06/2013 11:14]  also ruote over AMQP is pretty good at handling
multiple languages (python/ruby etc)
[27/06/2013 11:14]  the messages serialise to json
[27/06/2013 11:14]  so "take a task description and reply sometime"
[27/06/2013 11:15]  lbt: that's nice. btw, one of the steps I would have
is an user in the authirity needs to accept the tally task
[27/06/2013 11:15]  it's not going to be completely automated
[27/06/2013 11:15]  and I'm glad you already have this in ruote
[27/06/2013 11:15]  yep - another think we actually do
[27/06/2013 11:15]  yeah =)
[27/06/2013 11:15]  eg we do community based QA - so users vote but then
one of a group has to approve
[27/06/2013 11:16]  so that's why I don't want to reinvent the wheel, I
want to use the wheel and if needed, improve it, why not hehe
[27/06/2013 11:16]  yes, I think ruote would be worth exploring
[27/06/2013 11:16]  the various layers support security in different ways too
[27/06/2013 11:16]  yeah. I have spent quite some time already in
frestq, but I'm quite open minded
[27/06/2013 11:18]  another point is that I'm using something that uses
standard protocols, and something people already use, it's going to be more
tested, have more tools available and I'll probably need to do less and be able
contribute for others, so I'm going to explore this option
[27/06/2013 11:18]  I just whish I had seen it before =)
[27/06/2013 11:18]  wish*
[27/06/2013 11:19]  ruote is really quite powerful
[27/06/2013 11:19]  haha - I think maybe I'll email this chat to jmettraux