QueryDB

DP is a well-described solution to a common software problem. Its benefits: Already defined to solve a problem. Increase code reusability and robustness. Faster devlopment and new developers in team can understand it easily DP defined in to 3 categories: Creational  - Used to construct objects such that they can be decoupled from their implementing system. Structural  - Used to form large object structures between many disparate objects Behavioral  - Used to manage algorithms, relationships, and responsibilities between objects. Creational : Singleton - Singleton pattern restricts the instantiation of a class and ensures that only one instance of the class exists in the jvm. We have different approaches for Singleton but all of these follow below bullets: Private constructor Private static variable of same class i.e. only instance of class. Public static method of class that returns the instance. A few points to think about before implementation: Y

To get your windows version open command prompt and execute following command"- C:\Users\ds> wmic os get osarchitecture OSArchitecture 64-bit C:\Users\ds>

There are 5 types of data types: Strings: Redis string is a sequence of bytes. Strings in Redis are binary safe, having fixed length, So, one can store anything up to 512 megabytes in one string. 127.0.0.1:6379> SET name "Dinesh Sachdev" OK 127.0.0.1:6379> get name "Dinesh Sachdev" Hashes: It is a collection of key value pairs. Every hash can store up to 232 - 1 field-value pairs (more than 4 billion). 127.0.0.1:6379> HMSET user:1 username dineshSachdev password dinesh123 address 123 OK 127.0.0.1:6379> HGETALL user:1 1) "username" 2) "dineshSachdev" 3) "password" 4) "dinesh123" 5) "address" 6) "123" 127.0.0.1:6379> HGET user:1 username "dineshSachdev" 127.0.0.1:6379> HGET user:1 password "dinesh123" Lists: Redis Lists are simply lists of strings, sorted by insertion order. You can add elements to a Redis List on the head or on the tail. 127.

Redis is an open source, BSD licensed, advanced key-value store. Redis holds its database entirely in memory, using the disk only for persistence. Redis can replicate data to any number of slaves. Installing Redis Download the latest stable release tarball or wget http://download.redis.io/releases/redis-stable.tar.gz Untar it:            tar xzf redis-stable.tar.gz Change Directory or make entry to '.bashrc':           cd redis-stable Proceed to with the make command:           make Run the recommended make test:           make test Start Redis hduser@slave:~$ redis-server 8413:C 02 Feb 13:29:24.587 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf 8413:M 02 Feb 13:29:24.591 # You requested maxclients of 10000 requiring at least 10032 max file descriptors. 8413:M 02 Feb 13:29:24.592 # Redis can't set maximum open files to 10032 because of OS err

Yarn as cluster manager While using Yarn we do not need to start Spark workers and master. For Spark standalone cluster manager refer http://querydb.blogspot.in/2016/01/installing-and-setting-up-spark-152.html YARN is a cluster manager introduced in Hadoop 2.0 that allows diverse data processing frameworks to run on a shared resource pool, and is typically installed on the same nodes as the Hadoop filesystem (HDFS). Running Spark on YARN in these environments is useful because it lets Spark access HDFS data quickly, on the same nodes where the data is stored. Using YARN in Spark is straightforward: you set an environment variable that points to your Hadoop configuration directory, then submit jobs to a special master URL with spark-submit. ·          Set Hadoop configuration directory as the environment variable HADOOP_CONF_DIR ·          Then, submit your application as follows: spark-submit --master yarn yourapp ·          Launch spark-shell: spark-sh