background preloader

Theory

Facebook Twitter

What is machine learning? Machine-learning algorithms are responsible for the vast majority of the artificial intelligence advancements and applications you hear about.

What is machine learning?

(For more background, check out our first flowchart on "What is AI? " here.) What is HDFS, Map Reduce, YARN, HBase, Hive, Pig, Mongodb in Apache Hadoop Big Data. Apache Hadoop is an open source framework written in Java language.

What is HDFS, Map Reduce, YARN, HBase, Hive, Pig, Mongodb in Apache Hadoop Big Data

Open source means it is freely available and we can even change its source code as per the requirements. It is a platform for large data storage as well as processing. It efficiently processes large volumes of data on a cluster of commodity hardware (low-performance systems which are compatible with Microsoft Windows or Linux). Most of Hadoop code is written by Yahoo, IBM, Facebook, Cloudera. What Is Data Mining? This chapter provides a high-level orientation to data mining technology.

What Is Data Mining?

What Is Data Mining? Data mining is the practice of automatically searching large stores of data to discover patterns and trends that go beyond simple analysis. Data mining uses sophisticated mathematical algorithms to segment the data and evaluate the probability of future events. A Comparison Of NoSQL Database Management Systems And Models. A previous version of this article was written by O.S.

A Comparison Of NoSQL Database Management Systems And Models

Tezer. Introduction When most people think of a database, they often envision the traditional relational database model that involves tables made up of rows and columns. While relational database management systems still handle the lion’s share of data on the internet, alternative data models have become more common in recent years as developers have sought workarounds to the relational model’s limitations. NoSQL Database Types: Understanding the Differences. Mapreduce Example in Apache Hadoop. MapReduce Tutorial: Introduction In this MapReduce Tutorial blog, I am going to introduce you to MapReduce, which is one of the core building blocks of processing in Hadoop framework.

Mapreduce Example in Apache Hadoop

Before moving ahead, I would suggest you to get familiar with HDFS concepts which I have covered in my previous HDFS tutorial blog. Phases of MapReduce - How Hadoop MapReduce Works - TechVidvan. MapReduce is one of the core components of Hadoop that processes large datasets in parallel by dividing the task into a set of independent tasks.

Phases of MapReduce - How Hadoop MapReduce Works - TechVidvan

In this MapReduce Tutorial, you will study the working of Hadoop MapReduce in detail. It covers all the phases of MapReduce job execution like Input Files, InputFormat, InputSplits, RecordReader, Mapper, Combiner, Partitioner, Shuffling, and Sorting, Reducer, RecordWriter, and OutputFormat in detail. Let us first see what Hadoop MapReduce is. Introduction to MapReduce. What Is a Hadoop Cluster? What Is a Hadoop Cluster?

What Is a Hadoop Cluster?

Apache Hadoop is an open source, Java-based, software framework and parallel data processing engine. It enables big data analytics processing tasks to be broken down into smaller tasks that can be performed in parallel by using an algorithm (like the MapReduce algorithm), and distributing them across a Hadoop cluster. A Hadoop cluster is a collection of computers, known as nodes, that are networked together to perform these kinds of parallel computations on big data sets.

Understanding Hadoop Yarn. YARN — Yet Another Resource Negotiator… YARN — Yet Another Resource Negotiator, is a part of Hadoop 2 version, is one of the two major components of Apache Hadoop (with HDFS).

Understanding Hadoop Yarn. YARN — Yet Another Resource Negotiator…

It plans the use of cluster resources as well as the treatments applied to the data. Above is the basic architecture of Yarn, where Resource Manager is the core component of the entire architecture, which is responsible for the management of resources including RAMs, CPUs, and other resources throughout the cluster. Application Master is responsible for application scheduling throughout the life cycle, Node Manager is responsible for the supply and isolation of resources on this node. The Resource Manager: it controls the resource management of the cluster, also makes allocation decisions. The resource manager has two main components: Scheduler and Applications Manager. Data acquisition and understanding of Team Data Science Process.

What Is a Key-Value Database? NoSQL wide-column stores demystified. Many people believe NoSQL to be ancient technology.

NoSQL wide-column stores demystified

In the world of databases, however, NoSQL is considered a baby — even though it’s been around since the early ’70s. How’s that possible? Well, NoSQL wasn’t really popular until the late 2000s, when both Google and Amazon put a lot of research and resources into it. What Is The Difference Between Descriptive, Predictive and Prescriptive Analytics. Data – particularly Big Data – isn’t that useful on its own.

What Is The Difference Between Descriptive, Predictive and Prescriptive Analytics

It needs to be analysed before it can be acted on, and we refer to the lessons that we learn from the analytics as insights. Insights inform the actions we take with the aim of creating business growth and driving efficiency. As a basic example, consider a machine. Don’t worry about what the machine does – for the purposes of this discussion – all we need to know is that sometimes it breaks down. It’s fairly easy to collect one dataset showing the times that the machine fails, and another dataset showing what the machine is doing. Predictive Analytics vs Descriptive Analytics. What is machine learning? SAN vs. NAS vs. DAS: Competing or Complementary? How to Work with NoSQL Database in Python using PyMongo.

Before beginning the NoSQL Database in Python, let’s find out about NoSQL.NoSQL expands to “Not Only SQL”. It lends us a way to store and retrieve data that we can model in forms other than relational (tables). NoSQL databases largely find use in applications involving big data and real-time uses. The reason we call them “Not Only SQL” is because they may support query languages that are SQL-like. We can use NoSQL to store data in forms like key-value, document, columnar, and graph.