Javelin

• On trust: who do you trust?
  • Brokers? Yes...need to. This could be a big flaw.
  • Anything else? Probabilistically, no. (You'd better not have to, because you can't.)
  • Fault tolerance, byzantine computation, etc. This is really hard. Fault tolerant CILK? If individual operators are functions, reapply function elsewhere if timeout/crash.
  • Timeout is a hard problem: how do you know how long your program should run?
• Bounds on resource consumption...
  • Economic model is good way to decide bounds, but you still have the problem of enforcing the model.
• Brokerage architecture is the contribution of this paper, IMHO.
• They got quite a bit out of the design - 3 tier architecture.
• Can we change the architecture and deal with some of these issues?
  • add 4th tier? - client-side policeman
  • add hierarchy - abstract away trust, increase bandwidth of brokers.
    • As go up hierarchy, abstract away lower details - i.e. if operate at top level, can know about load distribution, etc. If at low-level, can't have a high-level broker guarantee entire computation.
• Why coarse-grained, independent jobs?
  • minimize job spawning:job execution overhead
  • communication meltdown for large number of jobs.
• What class of applications can Javelin not handle?
  • interactive applications
  • data-centric as opposed to compute-centric computing - distributed databases: move the computing to the data. Can't do that here in Javelin.
• Economic data modeling, buying replicated data, etc.
  • Sounds an awful lot like Mariposa's macroeconomic model for affinity scheduling of workers.
• Kuszmaul - optimizing work stealing to match bandwidths of branches in a tree of worker nodes (similar: programming on Clumps). Is this applicable?
• Encrypted computing.
  • Dave Wagner: if have a particular problem to solve in encrypted form, there is a restricted subset of problems that can easily be handled by encrypted computing. For most problems, practically impossible. Consider, for instance, composition of functions.
  • Maybe (maybe) stuff like blinded database lookup, but even that's fishy.
  • conclusion... RED HERRING
• Joe Hellerstein:
  • We have many idle toasters, but we're not trying to make toast all the time...
  • I.e., do we really need this infrastructure?
  • David Culler: don't buy the hype. Really about harnessing a (large) set of computing to solve one particular task. Is there a better strategy for putting together a system that harnesses a set of (arbitrary) machines?
• What about rsh + perl + glustat? I.e., what are minimum requirements for a distributed processing environment?
  • distributed file space
  • unpriviledged account, sandboxed execution
  • interpreted execution to deal with heterogeneity
  • communication (between spawners, spawnees, ...)
• What are the design principles?
  • Compare to Glunix, Condor, LSF, ...