Welcome!

FinTech Journal Authors: Yeshim Deniz, Pat Romanski, Liz McMillan, Elizabeth White, Zakia Bouachraoui

Related Topics: @DXWorldExpo, Linux Containers, Agile Computing, @CloudExpo, FinTech Journal

@DXWorldExpo: Blog Post

NoSQL: Filling the Gaps By @MapR | @CloudExpo [#Cloud #BigData]

Today's web based apps demand databases that are flexible, high performing, and easily scalable.

NoSQL: Filling the Gaps in Your Traditional Relational Database

With the many different characteristics of NoSQL databases available today, it's not always clear how to best categorize the different NoSQL offerings. Typically, though, NoSQL databases are labeled according to the associated data model, most commonly: key-value, wide-column, document, and graph. But more important than the differences between them are the reasons why they are growing in popularity as a whole. In general, NoSQL databases are meant to fill some of the capability gaps found in traditional relational database management systems (RDBMS).

Today's web-based applications are highly demanding, so the databases that hold their vast stores of information are not only expected to be very flexible in nature (supporting various data formats), but also to manage extreme performance and scaling. Enterprise architects have many great technology choices today, and it makes more sense than ever to spend the time to assess application requirements thoroughly before suggesting an appropriate database.

Initially promoted as a departure from, and improvement upon, the traditional RDBMS model, NoSQL is considered an independent and specialized database technology to support complex business needs. But NoSQL advantages don't necessarily represent an opportunity for outright replacement of RDBMSs. An enterprise architect's recommendation of RDBMS versus NoSQL comes down to the nature of the operational requirements, in other words, particular use cases. In some cases, an RDBMS is more appropriate, and sometimes quicker, for specific data management tasks compared to NoSQL, and the use of NoSQL could be problematic for the application as well as for the business. A look at some of the fundamental features of database technologies will lay the groundwork for a general understanding of how NoSQL and RDBMSs differ, and how the former can fill in the gaps of the latter.

RDBMS: transactional, consistent

RDBMSs have been around for decades and have become the foundation for many business applications today. They can guarantee good performance with a volume of thousands of transactions per second which, decades ago, was enough. They also support multi-update transactions, known by computer scientists as "ACID transactions." The ACID transaction principles--atomicity, consistency, isolation, and durability--ensure data integrity even when multiple data elements are updated together. But modern internet applications - especially those operating in real time, such as fraud detection, risk analysis, advertising, and gaming - involve millions of transactions per second, and typically don't need multi-update transactional guarantees. RDBMSs struggle to manage the sheer volume of these newer online transaction processing (OLTP) applications. RDBMS vendors are investing time, money, and effort into overcoming these issues, but for now the gap remains.

Another strength of RDBMSs is the implementation of SQL ("Structured Query Language") as the de-facto standard for data processing tasks like data query, data definition, data manipulation, etc. SQL, and thus RDBMSs, require a predefined tabular structure entailing rows and columns to ensure that data is stored in a consistent and expected format. The widespread knowledge of SQL along with the consistency of the data model make RDBMSs well-suited for business intelligence analysts. Any application that requires multi-element updates, such as those dealing with financial data, and that takes advantage of the power and ubiquity of SQL, should use an RDBMS.

NoSQL: scalable, flexible

Most NoSQL databases give up on multi-row ACID transaction capabilities to achieve cost-effective scalability and performance. For a certain class of use cases, this tradeoff is perfectly acceptable, especially when the data set has records that are independent of one another, so there is no requirement to make updates that require additional immediate updates. And with regard to the data model, NoSQL databases, in contrast with RDBMSs, use different formats to store data. This flexibility lets application developers store heterogeneous record formats in the same database, which is often needed for large-scale data sets loaded from numerous distinct sources. The most popular NoSQL data formats are key-value, wide-column, and document (like JSON). Among them, the key-value databases are the simplest, while wide-column databases provide the closest data model to RDBMS tables. NoSQL databases bypass some of the hard constraints used in RDBMS architectures to achieve data storage flexibility, scalability, and performance.

Use cases: NoSQL suitability over RDBMS

There are many applications for which the traditional ACID-driven RDBMS model is not the easiest or best option. Here's another look at what NoSQL can do:

  • A key-value database is ideal if your application requires only the storage of data for the purposes of a quick lookup. In this situation, an RDBMS is unnecessarily complex, and a key-value database will be more than sufficient to meet your application's needs. The value itself can be as simple or as complex as you require.
  • A wide-column database is ideal if your data is structured like an RDBMS table but the columns might be different across rows. This means that different types of records can be stored in the same table, and existing rows can be easily updated to accommodate new columns.
  • A document database is ideal if your data entails hierarchical objects. RDBMSs could accomplish this with the help of object relational mapping (ORM) tools, but the work is often complex, and it's not worth complicating matters when NoSQL solutions are available.
  • A graph database is ideal if your application deals with connected networks of entities or large trees. Graph databases are good at quickly identifying linkages between related entities.

Major influencing factors: scaling and performance

RDBMSs were not originally designed for horizontal scaling - in other words, adding more hardware servers to the mix -- so they get overwhelmed when the load and data increase beyond expectations. RDBMSs are not great at distributing data across servers, in the process known as "sharding." Either they don't have the ability to automatically shard, thus requiring application-level coding to distribute data, or the scale-out architecture is expensive.

Here, NoSQL solutions have an advantage. A NoSQL database does not have to break up records into distinct pieces across servers. They are always logically stored in a single place, thus simplifying the distribution process. And since there is no referential integrity required between these logical entities, NoSQL is easily able to perform automatic data sharding. Though NoSQL solutions have certain limitations when compared to the relational model, they were designed with the intention of providing high scalability. A NoSQL solution can scale horizontally over a distributed environment and support high availability.

Overall performance depends largely on the selection of appropriate technology (NoSQL or RDBMS) for a specific use case. Thus, if NoSQL is selected for the wrong use case, it can hurt the performance of the application. Apart from factors like network, caching, and disk I/O, performance is dictated by the data and its allocation across the distributed storage - and NoSQL is capable of handling large volumes of data in a clustered environment, boosting performance as a result.

It's important to remember that NoSQL is not a substitute for traditional RDBMSs, but an alternate solution to address a different set of use cases for which RDBMSs were not designed. Many NoSQL technologies exist on the market today, and if your NoSQL requirements include enterprise-grade reliability, high performance, and fast responsiveness, a production-proven NoSQL database worth investigating is MapR-DB by MapR Technologies.

More Stories By Dale Kim

Dale is Director of Industry Solutions at MapR. His technical and managerial experience includes work with relational databases, as well as non-relational data in the areas of search, content management, and NoSQL. Dale holds an MBA from Santa Clara University, and a BA in Computer Science from the UC Berkeley.

IoT & Smart Cities Stories
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
DXWorldEXPO LLC, the producer of the world's most influential technology conferences and trade shows has announced the 22nd International CloudEXPO | DXWorldEXPO "Early Bird Registration" is now open. Register for Full Conference "Gold Pass" ▸ Here (Expo Hall ▸ Here)
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.