Paper “Partisan: Enabling Cloud-Scale Erlang Applications”

https://arxiv.org/abs/1802.02652

• Models:
  • Full mesh model
  • client-server model
  • peer-to-peer model
  • publish-subscribe model
• Usually, the runtime support for distributed applications is not needed. But several counter-examples: Akka cluster, Microsoft Orleans, Distributed Erlang
• single topology is insufficient
• Partisan: distributed programming model + distributed layer = alternative to Distributed Erlang.
• 2 important improvements over Distributed Erlang:
  • addition of multiple runtime-selectable cluster topologies
  • ability to gain additional parallelism by distributing messages over multiple communication channels
• Default topology: full mesh
• But can distribute traffic over multiple connections to avoid congestion problem observed in Distributed Erlang
• Consider 2 application scenarios:
  • distributed database with large objects on smaller clusters (10s of nodes)
  • lightweight replicated k-v store (100-1000s of nodes)
• communication channels -> 30x improvements on point-to-point massaging, 13.5x on distributed database application.
• specify topology at runtime -> 256 nodes in cluster -> 1024 nodes
• Contribution of this paper:
  • design of the Partisan programming model, that supports the runtime specification of multiple cluster topologies
  • design of the channel-based full mesh backend
  • open source implementation of Partisan
  • detailed evaluation of Partisan
• 3.1 Partisan Programming Model:
  • 2 sets of operations: membership operations (join + remove Erlang nodes from the cluster) messaging operations (async deliver messages between nodes)
  • topology-agnostic and async
  • Messaging is async and best-effort: in full mesh - directly to the node, in peer-to-peer mode a message may have to be forwarded through several nodes.
• 3.2 API
  • Join: join a node to the cluster
  • Self Leave / Leave
  • Members
  • Forward Message
  • Cast Message
• 3.3 Topologies:
  • Static: explicit configuration at the deployment time
  • Full Mesh: single TCP connection for Erlang nodes + Channels
  • Client-Server
  • Peer-to-Peer: all clients connect to one other client in the system
  • Publish-subscribe: Partisan will connect to preconfigured AMQP message broker for node-to-node messaging and dissemination of cluster membership information.
• 3.5 Impl
  • Erlang 19.3
  • 6.7KLOC
  • Open Source GitHub
• 4. Evaluation
• 6 Conclusion:
  • 13.5x-30x performance improvement for real Erlang applications
  • Performance gains for Riak Core -> for all similar apps.