Architecture and implementation of warden in Foundry Cloud

Label Foundry CloudWardenFrameworkvirtualizing techniqueDEA
3404 people read comments(4) Collection Report

In the cloud foundry, when the application developer's application by the cloud foundry component DEA to run, the application resource isolation and control is particularly important, and the existence of the warden good solution to this problem.

Cloud foundry warden projects for the primary purpose is to provide a simple interface to manage the isolation environment. These isolated environment can be called "container", they can be used in the CPU, memory usage, disk usage and equipment access to do corresponding restrictions.

This paper will discuss the implementation of warden analysis from four aspects:

  1. Function introduction and framework implementation of warden
  2. The external interface of warden framework and its realization
  3. Internal module and implementation of warden framework
  4. Operation example of warden

Function introduction and framework implementation of warden

Warden function introduction

Because cloud foundry V1 DEA component when you run the application, design their own existence some defects, isolation and restriction of resources in the running process that is running with a DEA application is not very good, so in the cloud foundry V2 introduced the warden of the module.

Warden receives management regarding the application of DEA component to send the request, in this part of the management request, with lightweight virtualization technology, virtualization host operating system, execute request specific content inside the container. The warden of the specific effects of the use of application between each other is not perceived and resources to complete the isolation and their respective resource use there is an upper limit. If cloud foundry application resource isolation and restriction mechanism does not exist, runs on a DEA with multiple applications, when load increases, tried to compete for resources, when the resources have been depleted, greatly reducing availability and security of the application.

In isolation and restriction of resources and the warden mainly provides three dimensions of user custom isolation and restriction: memory, disk, and network bandwidth; also the warden also provides the following dimensions of resource isolation and limit, but only to provide default values, and does not provide user-defined settings: CPU, CPUACCT and devices. The

At the same time, the warden as a virtual container, also provides many API commands, which allow the user to perform the warden container management. The main command is as follows: the copy in, copy out, create, destroy, echo, error, info and limit bandwidth, limit disk, memory limit, limit the CPU and link, list, the message, net, net out, Ping, run, spawn, stop, and stream and. These features can see simple commands:James Bayer warden and docker document for more.

Warden framework implementation

In the specific implementation to warden framework, need to declare a concept related to warden:
  • Warden: in cloud foundry in the realization of application resource isolation and control framework, including, warden client, warden server, warden protocol and the warden container;
  • The server warden server:warden framework in the end, is mainly responsible for receiving the client request and request processing execution;
  • Implementation of client side of the warden client:warden framework, cloud foundry is DEA ng component calls, to achieve the warden server to send a specific request;
  • The definition of warden_client and warden_server communication protocol warden request message in the protocol:warden framework;
  • The container warden container:warden framework, management and operation of the application, isolation and restricted resources in container units.

The warden framework is a typical C/S architecture, as shown below:

The external interface of warden framework and its realization

Though the warden module is the cloud foundry is an integral part of, but without the aid of cloud foundry, warden can still used to manage the warden container, and the container internal run applications such as.

If warden running inside the cloud foundry, DEA ng component is embedded the warden client and to the warden client and warden server communication is established, distribution management application request; if the warden exist alone, you can communicate through the tools the command-line warden repl (Read-Eval-Print loop) Rui and the warden server, users through the command line initiated container management request. This chapter will take the above two on external interface warden framework and implementation.

Communication between warden and dea_ng

The use of warden in Cloud Foundry, and dea_ng is almost entirely used together. In the deployment of DEA ng, regardless of cloud foundry clusters installed a plurality of components of the DEA ng, where each of the components of the DEA ng nodes will install a warden, thus the warden and DEA ng for one-to-one relationship.

The following is a schematic diagram of the interaction of warden and dea_ng:

From the above diagram shows, accept the request from dea_ng, the distribution of the container request is divided into the following steps:
  1. Dea_ng through app access request message middleware NATS management;
  2. Dea_ng according to the type of request, and through the Warden: Protocol protocol to create a corresponding container request;
  3. Dea_ng and warden_server have been established by waren_client transmitting the container connection request.

Warden and REPL command line interaction

Warden can also separate installation on a machine, when the need to manage the warden, by way of the repl command line, start a process, create the warden client, and is responsible for receiving the user in the command line input warden container management command, followed by the warden client to the warden for the server to send request.

From the above, the communication mode of REPL and dea_ng and warden are almost the same, the only difference between the use of. The following is a schematic diagram of warden and repl command line interaction:

Internal module and implementation of warden framework

Already mentioned the warden framework is the C / S structure, abstract, it's running include: 1. The warden client to the warden for the server to send the container request; 2. By the warden server receives the request and deal with; server 3.warden_server warden container for execution request.

The following is a brief sketch of warden framework:

Implementation of warden_server framework

The main function of warden_server for receiving warden_client requests, and distributed processing. Warden server implementation, through eventmachine start a server, and listens to the local a UNIX domain socket file, the final will be forwarding all the received requests to the handle of the ClientConnection. ClientConnection first from monitoring the sock file read data, and data read from the read request, then identify the requested type. Finally, the type of the request, the implementation of the corresponding operation. If the request for a specific warden Container, container from registered with the registry to find out corresponding warden container, then to the warden container implementation shell scripts.

When warden_server disptach requests, mainly to complete the request distribution. Request can be mainly divided into two categories, a class is for the specific operation of the internal container, such as according to create the warden container request execution new container operation, the specified container execution run a task, to the specified container to perform the destruction operation:; another container for information access, such as the warden container Ping, access to all of the container hanles information. The warden executive echo command etc.. Following this type of request warden_server.

Implementation of container warden

Warden_server is mainly responsible for the management of warden container management, including the creation, destruction and other settings. When warden container created and set up by warden container, is responsible for their own operation. Applications running inside the warden container, due to the warden container provides resource isolation and control the environment, in order to achieve application resource isolation and restriction.

Warden resource isolation and restriction is the core of the whole cloud foundry, the warden container resource isolation and restricted mainly by CGroup mechanism in Linux kernel, quota and TC (traffic controller).

This part will be from the warden container file structure, life cycle, the network is configured to start, and elaborates the warden container 'implementation.

Container warden file structure

The warden container can be thought of as a simple version of the operating system. So there also exists their own internal file directory, just create completed, the internal file structure are mainly the following six folder: / bin and / etc, /jobs, / lib, /run, / TMP and / MNT.
The bin file is executable file, such as wshd, WSH, iomux-spawn, iomux-link. This part can be executable file uses, or below will elaborate.

/etc is stored and host related files and directories. These files include system configuration files, including host name system, network parameter setting etc..

/jobs from the warden container stored in the external transmission job information.

/libs storage file system important.

/run to store the tasks in the container.

The / TMP is mainly used for storing temporary files, the file is mainly a rootfs file and rootfs folder with a streamlined system.

/mnt is mainly used for mount file directory, and storage container creates added device files.

Life cycle of container warden

The warden container of life cycle including the main, creating, using and delete these three processes, which will involve the specific operation of many of the warden container in the using process.

The first is the creation of container warden. After the warden server warden container to create a received request by executing the script to create the container. Can be summarized into the following several parts: system initialization file, configuration container attribute, the configuration container network and operation warden container is one of the most important Guardian process wshd role of this part of the script. After created after the warden container, about a warden per container, will generate a container handle and container for record.

Followed by the use of container warden. Warden container consists of two cases, one is the user request through the warden client, and then after the warden server management, such as users of the warden container configuration resource constraints, users to warden container internal transfer files, and so on; another situation is when the warden container running inside a web application, the warden container external users through the application service access internal application. In the first case, is through warden container The internal washd process fork shell process, and the user commands transmitted through the WSH to wshd, and wshd commands to be executed by shell.

Also included in the warden container life cycle to remove its own. Delete process, first of all let wshd request to the container all internal process sends a pkill -TERM, and wait for the process to the end, if there is no end directly kill the process, then wshd exit, and clear the container file directory.

Network configuration of container warden

Warden container network configuration:

First of all, warden for each container to create a virtual network, and through the warden host for port mapping. As the host for the specified port on the request forwarded by DNAT to the specified warden container. When the warden container foreign send request, through its virtual network card sent external request, sent to the host also virtual virtual network card and the virtual network card as the gateway of container, request processing, finally through the host is sent to the network.


The above is a brief introduction to the warden architecture.

About the author:

Sun Hongliang,DAOCLOUDSoftware engineer. Knowledge and technology of two years to calculate the main research in the field of cloud PaaS. That lightweight container virtualization technology, will bring deep impact to the PaaS field, and even decide to go to the future of PaaS technology.

Reproduced please indicate the source.

The more out of my understanding, certainly in some places defects and mistakes. I hope this article can be of some help to contact the warden architecture and implementation of the people, if you are interested in this field, and have better ideas and suggestions, please contact me.

My email address:

Guess you're looking for
View comments
* the above user comments represent the personal views do not represent the views or position of the CSDN website
    personal data
    • Visit81740 times
    • Integral:One thousand three hundred and twenty-eight
    • Grade
    • Rank:18621st name
    • Original47
    • Reproduced:0
    • Translation:1
    • Comments:49
    Blog column
    Contact information
    Latest comments