StudentShare
Contact Us
Sign In / Sign Up for FREE
Search
Go to advanced search...
Free

Data Warehouses with Big Data - Term Paper Example

Cite this document
Summary
The author of the paper concludes that while traditional data warehousing still puts emphasis on the use of forms and queries, advanced data warehouses are based on data tables that are extensively interlinked with each other in a multidimensional environment. …
Download full paper File format: .doc, available for editing
GRAB THE BEST PAPER92.7% of users find it useful
Data Warehouses with Big Data
Read Text Preview

Extract of sample "Data Warehouses with Big Data"

? Data Warehouses with Big Data I.D. of the Data Warehouses with Big Data In the last two decades, database management technologies have continuously evolved to yield more powerful systems for large scale storage and retrieval of data. Basic database administration is traditionally based on Relational Database Management System (RDBMS) and database administration suites like Microsoft SQL and Access, Oracle, MySQL, etc. are used (Ricardo 2011). However, technocrats and industries were not satisfied with RDBMS, and consequently data warehousing (DW) technologies were introduced during the 2000s. Basically, data warehousing is a relatively more intelligent and refined database administration system which can handle large amount of data. At the commercial level, data warehousing is further evolving in the form of Big Data which is aimed at storing and retrieving enormous amount of asynchronous and disparate data across distributed computing systems (Kusnetzky 2010). According to Ricardo (2011), traditional form of database administration is based on the fundamental technique of identifying and manipulating the different characteristic entities in a given dataset. These entities can be termed as database elements. Most important database elements are forms, fields, tables, and queries. In an RDBMS framework, these elements of a database are related with each other using simple administrator defined relationships. RDBMS can thus help an administrator to organize data in an intelligent and retrievable way. However, this technology is not always helpful to arrange data into multiple layers so as to facilitate more efficient stacking, less errors, and context aware distribution. Although the basic concepts of RDBMS are still in extensive use in different high level database applications (Kimball and Ross 2011), large scale information storage services are now progressing toward multidimensional management of data. Evolution of DW schema took place with regard to the needs of the industries and research institutes. It can be stated that evolution of data warehousing was initially aimed to mitigate the limitations of preexisting database management systems. According to Baru et al (2013), the database management industry has considerably matured and developed its own dynamics and techniques over the last twenty years. But in the past few years, data warehousing technologies have become commercially important. With the advent of Big Data, the database management industry has now developed “increased volume, velocity, and variety” of data storage, retrieval, and even processing systems (Baru et al 2013, p. 60). Experts like Devlin (2011) have gone to the extent of stating that Big Data is a better and independent form of database administration technology vis-a-vis data warehousing. But the author appears to be more critical toward the traditional data warehousing technologies. From a holistic viewpoint, synchronization of data warehousing with cloud computing facilities is precondition to Big Data (Baru et al 2013; Kusnetzky 2010). Consequently, “taxonomy of data” (see Devlin 2011, section 2) in the realm of Big Data can be regarded as a viable cornerstone in the evolution of contemporary DW schema. For more details, refer to Figure – 1. Figure – 1: Taxonomy of data as viewed at the point of transition from data warehousing to Big Data techniques. The figure shows six main varieties of data named multiplex, textual, compound, derived, atomic, and measurement. (Devlin 2011, section 2) RDBMS is a database management system that is based on defining, linking, and organizing different database elements like tables, forms, queries, etc. However, a standard DW schema gives maximum importance to the data tables. As such, data tables are organized with the help of “dimensional modeling” (Kimball and Ross 2011, p. 16). This is a method of database administration which is based on simplicity and architectural coherence of distributed database systems with complex warehousing capabilities. Dimensional modeling culminates at multidimensional data structures, which can be stratified and stacked inside limited space too (Jensen, Pedersen, and Thomsen 2010). Consequently, DW schema is based on the basic technique of stacking and layering huge amounts of data and then interconnecting them in a most suitable and intelligent way. There are three major configurations in this data management system: Star schema, snowflake schema, and fact constellation schema. The construction of star schema is done on a central data table termed as the fact table. Several dimensional tables are then connected to this fact table. These dimension tables can be used to define and stratify huge amount of information around a single fact table. In snowflake schema, the dimension tables are further normalized to provide even more layers of data storage and processing. The most complicated configuration is the fact constellation schema. This kind of schema has bigger and more complex star schema arrangement of a large number of smaller star schema data sets. Consequently, a fact constellation schema must possess large number of internal links and connectors. (Kimball and Ross 2011; Jensen, Pedersen, and Thomsen 2010, pp. 10-15) New storage paradigms are emerging with the lapse of time. Although experts like Devlin (2011) tend to focus more on the differences between data warehousing and Big Data (see Figure – 2), the two technologies can hardly be regarded to have evolved differently. Figure – 2: In his article, Devlin (2011, section 1) refers to the figure above as a traditional data warehouse design. However, the middle portion of this diagram shows a layered configuration of enterprise data warehouse. This is a direct consequence of dimensional modeling, which is an imperative in Big Data implementation as well. In strict sense, more complex data warehousing synchronized with cloud computing can give rise to Big Data storage mechanisms. For complex sets of data, mere enhanced memory may not be enough. Sorting, storage, retrieval, searching, and navigation are some of the key processes that are embedded in modern database management. Consequently, a key paradigm of Big Data implementation is based on increased processor speeds. For very high processing speeds, shared hardware access through cloud computing is utilized in Big Data facilities. (Baru et al 2013; Kusnetzky 2010) Another key paradigm in modern data warehousing is dimensional modeling. This is a key concept that is seminal in implementing a Big Data facility and develops into multiple layers of data storage system. Dimensional modeling gives rise two multidimensional interpretation of data. In RDBMS, data is organized in two dimensional tables having rows and columns. Location of a given piece of data set would be traced on the basis of cell number and column number. But in advanced data warehousing, data can be organized in three dimensional cubes. Each cell of a cubical arrangement must be defined with three parameters instead of two. Hence, chances of errors are lessened and more data is stored. Application of the concept of dimensional modeling taken from traditional data warehousing is a fundamental requisite of Big Data. (Kimball and Ross 2011; Jensen, Pedersen, and Thomsen 2010) Furthermore, the main drive behind Big Data is due to the desire of accomplishing extremely complicated tasks for advanced research purposes. For example, research and development in the field of bioinformatics calls for high speed processing of enormous genomic data. Big Data helps the geneticists to achieve this aim (for more details, see Figure – 3). So Big Data also involves the paradigm of developing continuously expanding storage system to accomplish the previously impossible storage and retrieval tasks. Big Data or advanced data warehousing systems of today are thus being benchmarked on the basis of continuous performance and versatility (Baru et al 2013). Figure – 3: The figure above shows an example of Big Data use in bioinformatics. As per the industry standards, even a general Big Data storage and retrieval system can process one peta-byte of data in just 750 days! Since bioinformatics data generated in the realm of genomic research and development are extremely large, only a Big Data implementation system can meet the processing requirements that were regarded to be unattainable even only a few years ago. (Trelles, Prins, Snir, and Jansen 2011, p. 224) Majority of computer science experts like Kimball and Ross (2011) agree upon the fact that Big Data is the advanced version of data warehousing. While traditional data warehousing still puts emphasis on the use of forms and queries, advanced data warehouses are based on data tables that are extensively interlinked with each other in a multidimensional environment. When this kind of complex arrangement is combined with high speed cloud computing, commercial level Big Data utilities are created. According to Baru et al (2013), the main qualities of Big Data include high speed processing and enormous memory units. Big Data signifies industry level usage of pervasive computing with the help of multilevel information processing, storage, and retrieval. References Baru, C., Bhandarkar, M., Nambiar, R., Poess, M., & Rabl, T. 2013, Benchmarking Big Data Systems and the BigData Top100 List. Big Data, 1, pp. 60-64. Devlin, B. 2011, Will data warehousing survive the advent of big data? - Strata. Strata - Making Data Work. Available: http://strata.oreilly.com/2011/01/data-warehouse-big-data.html. Last accessed on September 22, 2013 Jensen, C.S., Pedersen, T.B. and Thomsen, C. 2010, Multidimensional Databases and Data Warehousing. San Rafael: Morgan and Claypool. Kimball, R., & Ross, M. 2011, The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling. Hoboken: Wiley. Kutnetzky, D. 2010, What is "Big Data?" | ZDNet. ZDNet | Technology News, Analysis, Comments and Product Reviews for IT Professionals. Available: http://www.zdnet.com/blog/virtualization/what-is-big-data/1708. Last accessed on September 22, 2013 Ricardo, C. 2011, Database Illuminated. Sudbury: Jones and Bartlett. Trelles, O., Prins, P., Snir, M., & Jansen, R. C. 2011, Big data, but are we ready? Nature reviews Genetics, 12, p. 224 Read More
Cite this document
  • APA
  • MLA
  • CHICAGO
(“An Essay of Data warehouses with Big Data Example | Topics and Well Written Essays - 1250 words”, n.d.)
An Essay of Data warehouses with Big Data Example | Topics and Well Written Essays - 1250 words. Retrieved from https://studentshare.org/information-technology/1485924-an-essay-of-data-warehouses-with-big-data
(An Essay of Data Warehouses With Big Data Example | Topics and Well Written Essays - 1250 Words)
An Essay of Data Warehouses With Big Data Example | Topics and Well Written Essays - 1250 Words. https://studentshare.org/information-technology/1485924-an-essay-of-data-warehouses-with-big-data.
“An Essay of Data Warehouses With Big Data Example | Topics and Well Written Essays - 1250 Words”, n.d. https://studentshare.org/information-technology/1485924-an-essay-of-data-warehouses-with-big-data.
  • Cited: 0 times

CHECK THESE SAMPLES OF Data Warehouses with Big Data

Data Warehousing in the Telecommunications Industry

This report "Data Warehousing in the Telecommunications Industry" presents the big data as one of the hot topics of the year 2012 that is a compilation of multiple large and complex data sets so that useful information may be extracted for decision making of the organizations (Dignan, 2011).... hellip; There are several demerits of the big data which the organizations should consider before adopting the big data for decision making.... As big data is using more commonly the new technologies which are relatively less mature, therefore these technologies carry more risk that might directly impact the organization's decision through the big data....
7 Pages (1750 words) Report

Data Warehouse and State-of-the-art Application

data warehouses can have both currents as well as historical data.... The author of this essay "data Warehouse" casts light on the phenomenon of the data warehouse.... According to the text, in computing, the term data warehouse (DW) refers to a centrally managed and integrated database containing both raw and formatted data.... hellip; data warehousing is a process of centralized data management and retrieval....
5 Pages (1250 words) Essay

Warehousing and Inventory Management

WMS usually make use of data capture facilities such as wireless LANs, RFID, mobile computers, and barcode readers.... data collection is followed by either batch synchronization or wireless transmission to a central database.... They data-track inventory during production and interpret between existing ERP and WMS systems Warehouse management should be incorporated with the rest of the business to help in streamlining operations, reducing costs and speeding up order fulfilment....
7 Pages (1750 words) Essay

Data Model and Data Warehouse Design Architecture & XML

Since, data warehouses are capable of handling server tasks connected to querying, therefore, the proposed implementation of data warehouse would support the company's transactions systems to complete transactions within a desirable time frame by running the reports and queries on a server.... The Director of the company is concerned that a data warehouse has to be developed which can help in analyzing the company's performance (in terms… The company has many clinics located in the main cities of the United Kingdom, each running its own operational database system to store records of all its For the purpose of this scenario, the information sources are represented by these operational database systems which are actually relational databases, and are required to be integrated in the data warehouse....
12 Pages (3000 words) Essay

Data Model and Data Warehouse Design Architecture & XML

The Director of the company is interested to deploy a data warehouse system which can provide analytical views in form of reports, containing data, graphs, and charts, to help the top management in making effective business decisions to increase the productivity, in other words, sales of the company.... This centralized database is a relational database which will represent the information source for the data warehouse in this...
12 Pages (3000 words) Essay

Data warehouse, data mart and business intelligence

On the other hand, data marts are more aligned with specific company motives, system requirements and planning and analysis are performed in an effective manner and as a result implementation, design, installation and testing are less expensive as compared to data warehouses (Demarest, 1993; Firestone, 1997).... data warehouses and Data Marts: A Dynamic View.... Business intelligence offers a wide variety of tools and techniques for collecting, storing, processing, and distributing huge volumes of data and information to improve business decision making capabilities....
2 Pages (500 words) Essay

W3 Asig Selection Criteria and Data Warehouses

This specialized software allows application systems to focus on the user interface, data validation and screen Selection Criteria and data warehouses al Affiliation Selection Criteria and data warehouses You are the administrator of adata warehouse for a hotel chain with leading presence in 10 western states in the United States.... Write a proposal describing the criteria you will use to select the RDBMS for your data ware house.... Make your assumptions clear. A relational… management system (RDBMS) was defined as a “special system software that is used to manage the organization, storage, access, security and integrity of data....
2 Pages (500 words) Essay

Recent Developments in Data Warehouses and Its Application in E-Commerce

As the paper "Recent Developments in data warehouses and Its Application in E-Commerce" outlines, the data warehouse provides a platform for effective data extraction/mining.... The review paper seeks to provide an insight into the recent developments of data warehouses and their technical application in the field of e-commerce.... In essence, the data warehouse comprises exhaustive and abridged information, integrated data, metadata, and historical data....
17 Pages (4250 words) Article
sponsored ads
We use cookies to create the best experience for you. Keep on browsing if you are OK with that, or find out how to manage cookies.
Contact Us