An interesting blog about "Why distributed systems are hard to program" can be found here.
Middle-ware is the bridge that connects application program and the communication infrastructure for basic message passing and support for reliable channels across different hardware and software platforms etc.
"Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination."
It is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Although zookeeper doesn't completely prevent partial failures, it provides a set of tools to build distributed applications which can handle failures safely.
Zookeeper Deployment steps..System Requirements
Supported platfroms
- GNU/Linux - development and production platform for both server and client.
- Sun Solaris - development and production platform for both server and client.
- FreeBSD - development and production platform for clients only.
(Java NIO selector support in the FreeBSD JVM is broken.)
- Win32 - development platform only for both server and client.
- MacOSX - development platform only for both server and client.
- Sun Java, release 1.6 (JDK 6) or greater
- Apache Ant
- Apache Subversion(svn) or Git
Setting up the environment(Linux is used as the development platform for rest of the guide)
- Install the Sun JDK 6
- Use a native package managing system like APT or any other popular front-end for apt to install tools listed below. (eg.Synaptic package manager)
- otherwise you can download these packages from the web and install them manually.
- Use these commands in the Terminal to install , $ sudo add-apt-repository ppa:ferramroberto/java sudo apt-get update sudo apt-get install sun-java6-jdk sun-java6-jre
- set the JAVA_HOME system variable export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.26 export PATH=$PATH:$JAVA_HOME/bin
- If your system has installed more than one version of Java such as open jdk and sun-jdk, configure the default java version of the system as sun-java6-jdk by entering the following command in a terminal window and selecting the correct choice.$ sudo update-alternatives --config java
- Verify the installation using $ javac -version $ java -version
- Terminal out put should be like, java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
- Install Apache ant
- $ sudo apt-get install ant
- Verify the installation using
- $ ant -version
- Install subversion
- Verify the installation using
- $ svn help
- If you intend to use git as the version control system
- Set the Java heap size.
- Since zookeeper uses a in-memory data model, swapping can seriously degrade its performance. So it is very important to set the java heap size according to the memory utilization of the system.
- To set the heap size first open the configuration fire, .bashrc located in home directory,
$ sudo vim /home/user/.bashrc- Adding this line to the end of the file will set the heap size to 1GB(min),2GB(max) $ export JAVA_OPTS="-Xms1g -Xmx2g or $ export JAVA_OPTS="-Xms1024m -Xmx2048m
- Using subversion
- enter the command below in the terminal to get a working copy of zookeeper source code
- Using Git
- This will give the source code of zookeeper in read-only mode. That means you can do the changes to the source code in your local copy only. If you intend to make changes in the code and contribute developing the zookeeper you will have to create a git account and setup a personal repository in the git-hub first. Commit(pull) request to the original repository can be made through this account. Get more details here.
Building the source Zookeeper build process is developed using the build scripting tool called ant. Apache Ant is one of the heavily used build tool in Java development projects and very popular among open source community. Ant can automate tasks such as compiling the source code, building deployment packages and automatic dependency checking etc. Ant uses a script called "build.xml", which descibes how to carry out the compiling and building process. Detailed tutorial about "How to use ant" can be found here.
Go to the top level directory of the zookeeper trunk.
Use the command below to list out all the targets defined in the build.xml.$ ant -p
Terminal out put should be something like this.Buildfile: /home/user/checkouts/zookeeper/build.xml Main targets: bin-package Build binary distribution binary Make tarball without source and documentation call-test-cppunit to execute cppunit tests checkstyle Run optional third-party tool targets clean Clean. Delete the build files, and their directories clean-eclipse Clean eclipse files clover Instrument the Unit tests using Clover. compile-native Make C binding deb Make deb package docs Generate forrest-based documentation. eclipse Create eclipse project files javadoc Generate javadoc javadoc-dev Generate javadoc for zookeeper developers package Build distribution package-native Make C binding tarball releaseaudit Release Audit activities rpm Make rpm package tar Make release tarball test to run core and contrib tests test-contrib to run contrib tests test-cppunit to run cppunit test cases Default target: jar
These targets can be invoked using$ ant <target>
As a example, to build the binary distribution simply use,$ ant bin-package
Tips and Tricks...
- checking out the source using subversion
- you need to set the proxy server settings for the subversion client in,
- $HOME/.subversion/servers or
- /etc/subversion/servers
- Checking out the source using git
- Building source using ant, online
- If you have set the proxy server settings in the system, you need to enforce them by using,