The modern "data center" is a complex arena, with physical and virtual servers, multiple OS environments, and complex networking that frequently spans multiple locations. The need to simplify has never been greater. Mesos, an innovative open-source cluster management platform, transforms the whole data center into a single pool of compute, memory, and storage resources that can be allocated, automated, and scaled as if working with a single super-computer. Mesos is an ideal environment for deploying containerized applications at scale, and it's generating a huge buzz in the big data world as a saner environment for running Spark and Hadoop. Mesos in Action introduces the Apache Mesos cluster manager and the concept of application-centric infrastructure. It guides readers from their first steps in deploying a highly-available Mesos cluster through deploying applications in production and writing native Mesos frameworks. It will show how to scale to thousands of nodes, while providing resource isolation between processes using Linux and Docker containers. It contains practical techniques for deploying applications using popular key frameworks, including Marathon, Chronos, and Aurora. Along the way, the book dives into Mesos internals, including fault tolerance, slave attributes, and resource scheduling and Mesos administration, including logging, monitoring, framework authorization, and slave recovery. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.
Over 50 recipes on the core features of Apache Mesos and running big data frameworks in Mesos About This Book Learn to install and configure Mesos to suit the needs of your organization Follow step-by-step instructions to deploy application frameworks on top of Mesos, saving you many hours of research and trial and error Use this practical guide packed with powerful recipes to implement Mesos and easily integrate it with other application frameworks Who This Book Is For This book is for system administrators, engineers, and big data programmers. Basic experience with big data technologies such as Hadoop or Spark would be useful but is not essential. A working knowledge of Apache Mesos is expected. What You Will Learn Set up Mesos on different operating systems Use the Marathon and Chronos frameworks to manage multiple applications Work with Mesos and Docker Integrate Mesos with Spark and other big data frameworks Use networking features in Mesos for effective communication between containers Configure Mesos for high availability using Zookeeper Secure your Mesos clusters with SASL and Authorization ACLs Solve everyday problems and discover the best practices In Detail Apache Mesos is open source cluster sharing and management software. Deploying and managing scalable applications in large-scale clustered environments can be difficult, but Apache Mesos makes it easier with efficient resource isolation and sharing across application frameworks. The goal of this book is to guide you through the practical implementation of the Mesos core along with a number of Mesos supported frameworks. You will begin by installing Mesos and then learn how to configure clusters and maintain them. You will also see how to deploy a cluster in a production environment with high availability using Zookeeper. Next, you will get to grips with using Mesos, Marathon, and Docker to build and deploy a PaaS. You will see how to schedule jobs with Chronos. We'll demonstrate how to integrate Mesos with big data frameworks such as Spark, Hadoop, and Storm. Practical solutions backed with clear examples will also show you how to deploy elastic big data jobs. You will find out how to deploy a scalable continuous integration and delivery system on Mesos with Jenkins. Finally, you will configure and deploy a highly scalable distributed search engine with ElasticSearch. Throughout the course of this book, you will get to know tips and tricks along with best practices to follow when working with Mesos. Style and approach This step-by-step guide is packed with powerful recipes on using Apache Mesos and shows its integration with containers and big data frameworks.
The ultimate guide to managing, building, and deploying large-scale clusters with Apache Mesos About This Book Master the architecture of Mesos and intelligently distribute your task across clusters of machines Explore a wide range of tools and platforms that Mesos works with This real-world comprehensive and robust tutorial will help you become an expert Who This Book Is For The book aims to serve DevOps engineers and system administrators who are familiar with the basics of managing a Linux system and its tools What You Will Learn Understand the Mesos architecture Manually spin up a Mesos cluster on a distributed infrastructure Deploy a multi-node Mesos cluster using your favorite DevOps See the nuts and bolts of scheduling, service discovery, failure handling, security, monitoring, and debugging in an enterprise-grade, production cluster deployment Use Mesos to deploy big data frameworks, containerized applications, or even custom build your own applications effortlessly In Detail Apache Mesos is open source cluster management software that provides efficient resource isolations and resource sharing distributed applications or frameworks. This book will take you on a journey to enhance your knowledge from amateur to master level, showing you how to improve the efficiency, management, and development of Mesos clusters. The architecture is quite complex and this book will explore the difficulties and complexities of working with Mesos. We begin by introducing Mesos, explaining its architecture and functionality. Next, we provide a comprehensive overview of Mesos features and advanced topics such as high availability, fault tolerance, scaling, and efficiency. Furthermore, you will learn to set up multi-node Mesos clusters on private and public clouds. We will also introduce several Mesos-based scheduling and management frameworks or applications to enable the easy deployment, discovery, load balancing, and failure handling of long-running services. Next, you will find out how a Mesos cluster can be easily set up and monitored using the standard deployment and configuration management tools. This advanced guide will show you how to deploy important big data processing frameworks such as Hadoop, Spark, and Storm on Mesos and big data storage frameworks such as Cassandra, Elasticsearch, and Kafka. Style and approach This advanced guide provides a detailed step-by-step account of deploying a Mesos cluster. It will demystify the concepts behind Mesos.
Scale applications with high availability and optimized resource management across data centers Key Features Create clusters and perform scheduling, logging, and resource administration with Mesos Explore practical examples of managing complex clusters at scale with real-world data Write native Mesos frameworks with Python Book Description Apache Mesos is an open source cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks. This book will help you build a strong foundation of Mesos' capabilities along with practical examples to support the concepts explained throughout the book. Learn Apache Mesos dives straight into how Mesos works. You will be introduced to the distributed system and its challenges and then learn how you can use Mesos and its framework to solve data problems. You will also gain a full understanding of Mesos' internal mechanisms and get equipped to use Mesos and develop applications. Furthermore, this book lets you explore all the steps required to create highly available clusters and build your own Mesos frameworks. You will also cover application deployment and monitoring. By the end of this book, you will have learned how to use Mesos to make full use of machines and how to simplify data center maintenance. What you will learn Deploy and monitor a Mesos cluster Set up servers on AWS to deploy Mesos components Explore Mesos resource scheduling and the allocation module Deploy Docker-based services and applications using Mesos Marathon Configure and use SSL to protect crucial endpoints of your Mesos cluster Debug and troubleshoot services and workloads on a Mesos cluster Who this book is for This book is for DevOps and data engineers and administrators who work with large data clusters. You’ll also find this book useful if you have experience working with virtualization, databases, and platforms such as Hadoop and Spark. Some experience in database administration and design will help you get the most out of this book.
How can Apache Mesos make a difference in your organization? With this practical guide, you’ll learn how this cluster manager directs your datacenter’s resources, and provides real time APIs for interacting with (and developing for) the entire cluster. You’ll learn how to use Mesos as a deployment system, like Ansible or Chef, and as an execution platform for building and hosting higher-level applications, like Hadoop. Author David Greenberg shows you how Mesos manages your entire datacenter as a single logical entity, eliminating the need to assign fixed sets of machines to applications. You’ll quickly discover why Mesos is the ultimate DevOps tool. Understand Mesos architecture, and learn how it manages CPU, memory, and other resources across a cluster Build an application on top of Mesos with Marathon, a platform for hosting services on Mesos Create new, production-ready frameworks for Mesos Write a custom executor to provide richer interaction between the Mesos scheduler and workers Dive into advanced topics, including the reconciliation process, Docker integration, dynamic reservations, and persistent volumes Learn about today’s Mesos initiatives that will likely become tomorrow’s features