Better on the cloud: IoT, Big Data, and AI

Leading cloud providers offer business solutions that tap the power of IoT, Big Data, AI, and other emerging technologies.

Image by:

Pixabay. Modified by Opensource.com. CC BY-SA 4.0

Just a few years ago, companies used innovation and digital transformation mostly to differentiate themselves and to stay competitive. The dramatic growth in digital technologies and cloud computing over the last couple of years has since changed this mindset.

Today, organizations must be innovative and leverage the latest technologies simply to stay in business. Enterprises that implement online retail, banking, and other services aren’t considering these channels as just another route to increase their revenue. They realize that online services are fast becoming their primary revenue channel. According to data analyzed over three months for Forrester's report, The Digital Business Imperative, 84% of US banking customers used online banking for their transactions, and 43% used a mobile phone for these activities.

Organizations are working quickly to review and analyze their processes and seize opportunities for digital transformation. It is important to understand that to undergo digital transformation, companies may need to completely re-engineer their current processes to make use of technologies like the Internet of Things (IoT), Big Data analytics, artificial intelligence, and others instead of doing patchwork on existing processes to adapt to digital technologies. Furthermore, it is also important for senior IT executives to consider digital initiatives in tandem with their cloud strategy instead of treating them in isolation.

There is no doubt that the infrastructure required to support the compute power, storage, scale, and speed of these technologies is best provided by the cloud. The question you need to ask is: “Are we going to develop and host the infrastructure we need to support digital, or are we going to leverage the constantly improving robust infrastructure and services that cloud providers are offering?”

Organizations that plan for massive growth and transformation are usually the ones that invest considerably in technology to support innovative ideas. Advancements in the digital space help fuel innovation, but without a solid cloud strategy accompanied by agile development processes, ideas are likely to remain simply as ideas on paper and will take forever to get materialized into products or solutions that provide value. In today’s context, digital and cloud are almost inseparable due to the robust infrastructure, services, and tools available on the cloud to support digital initiatives. All leading cloud providers offer competing solutions and services to help organizations move forward with their digital initiatives at a rapid pace.

Ignoring the digital revolution will be disastrous to organizations as well as individuals.

In my article, Transforming with the Cloud: If Not Now, When?, I touched upon some of the key areas that organizations should consider to transform their businesses with the cloud. I have structured this article in a similar fashion, focusing on the role of cloud technology to support key digital technologies such as artificial intelligence, Big Data, analytics, and the Internet of Things. The goal is to provide a high-level overview of the digital landscape and to discuss how leading cloud providers are helping enterprises with their digital initiatives.

Artificial intelligence

The digital revolution, which centers around mass production of computers and communication devices, has changed the way businesses have operated over the last several decades, resulting in constant improvements in every possible area, ranging from generating new ideas for products and services to innovative product designs to improving the customer experience. At present, the world is going through another, possibly even stronger revolution: the use of artificial intelligence to perform complex cognitive tasks to solve business problems in ways that were previously either highly complicated or extremely resource-intensive.

Most organizations deal with business propositions that yield small to medium value and require a high volume of human work, such as reviewing large numbers of documents like RFPs to understand requirements and estimate costs. AI is proving to be the best alternative to handle cases like these, which are currently handled by human beings, but the volume of work involved challenges businesses on the feasibility of continuing those without AI.

AI systems try to mimic the human brain, which uses patterns to generate perceptions, and logic to drive the structured approach of analyzing a situation from a rational perspective. AI systems process large volumes of data that come from various sources such as sensors, online applications, textual data from social media, and the like. AI processes this data using perception to analyze patterns and incorporates machine learning to utilize structured evaluation methods and rational decision-making, not only to extract pieces of meaningful information but also to assemble this information to make valuable decisions.

The cloud plays a significant role in enhancing the power of applications that incorporate AI. Almost all major players in the cloud business have developed AI services that use powerful cognitive engines to process structured data such as relational databases and unstructured data from NoSQL databases, sensors, etc., that get uploaded to the cloud. The pattern-matching algorithms and logic components built into these cognitive engines are highly sophisticated and powerful. Data and compute power are the two most critical requirements to make these engines effective. The engines predict more accurately with larger datasets. AI applications such as image recognition, video analytics, natural language processing, and speech recognition leverage machine learning using highly sophisticated neural networks that perform detection and prediction from large volumes of data. Parallel processing with the use of GPUs (graphics processing units) help these data processing and computations run faster.

Building and implementing such robust GPU-based parallel processing engines on-premise is expensive and resource-intensive. The cloud addresses this problem by providing APIs to access machine-learning services such as video analytics, speech recognition, process automation, vision detection, natural language processing, etc. Behind these APIs are complex infrastructures that combine the power of clusters of GPU-based compute engines, neural networks, and data lakes.

Big Data and analytics

With the growth of internet, cloud, and social media, we've also seen the exponential growth of data across the world. According to statistics on Big Data generation in the last five years, the average volume of data created in the world every day is about 2.3 trillion gigabytes. When the nature of the data was more structured and organized, companies relied on data warehouses and BI applications to help make important data-driven business decisions. Traditional data warehouses were built based on relational databases that could be queried using SQL; the data could be extracted, transformed, and loaded from one or more data sources via ETL jobs that ran up to several times a day.

This approach has proved to be ineffective when it comes to handling and modifying continuous streams of real-time data flowing from multiple dissimilar sources such as social media, IoT, the public web, and relational databases. Big Data analytics, which helps examine large structured and unstructured data sets, has become a major enabler for enterprises, helping them make critical business decisions by providing insight and knowledge through data mining, predictive analytics, and forecasting. The evolution of Big Data processing has led to the data lake, a centralized repository that stores structured and unstructured data as is and permits the use of various tools and approaches to address business questions.

The cloud's ability to scale vertically and horizontally makes it the ideal platform for Big Data hosting and analytics. With vertical scaling, it is possible to increase the capacity of a server by adding resources as needed by applications. Horizontal scaling allows businesses to expand hardware resources as processing requirements increase. Hadoop, which led the Big Data revolution, is designed as a distributed system so that it can scale. Parallel processing is an important part of its design, enabling the system to process multiple independent small tasks, such as serving data stores and file systems, processing streaming data, and handling queries in tandem.

Cloud-based systems offer high bandwidth, enormous amounts of memory, and scalable processing power to help Big Data applications with improved real-time processing and analysis of streaming data. The cloud is a clear choice for applications running large workloads and storing enormous volumes of data. Cloud providers offer highly scalable database services coupled with tools and services to support information management, business intelligence, and analytics.

Internet of Things

The Internet of Things (IoT) refers to the universe of connected devices such as security sensors, surveillance cameras, smartphones, wearable devices like smartwatches, and even household appliances like washing machines, refrigerators, and others that have the ability to communicate and transfer data over the network without direct human interaction. This disruptive technology has not only empowered consumers by providing greater control of household systems and appliances, but it has also enabled organizations with data that provides greater insight into critical areas of business interest, opening the doors for innovative products, solutions, and new business opportunities.

IoT will impact all industries, from manufacturing to logistics to healthcare—we are not far from a time when nearly everything in the world is connected. Business adoption of IoT is already growing exponentially as the number of connected devices continues to increase. According to a recent Gartner study, businesses could already be using as many as 3.1 billion IoT devices today, and by 2020 this number will likely increase to about 7.6 billion.

IoT plays an essential role in optimizing production, managing supply chains, tracking assets, making financial decisions, and improving the customer experience. A Forbes report on IoT’s impact on business points out a few areas where the use of IoT is becoming more dominant.

Digital transformation and IoT adoption in the fitness industry has been remarkable. Wearable devices with built-in sensors can constantly collect data on physical activities such as distance traveled and calories burned and can monitor sleep patterns to provide detailed analyses and insight for continuous healthcare. IoT applications are designed to use data from connected devices, and the sophisticated tools available in the cloud let you visualize, explore, and build complex analytics. For complex IoT applications that use several devices, it is important to understand the state of the devices and to frequently communicate with the application components that leverage those devices. It is also essential to ensure secure identity and access between devices and applications.

As with any revolution in technology, IoT also faces challenges. With more devices connected to the internet, the volume of data generated is immense. This puts significant pressure on the internet and creates a need for an infrastructure that can transmit and store this data more efficiently. With the number of connected devices constantly increasing, there is a push to create "edge" devices that are intelligent enough to perform some processing and send the results to servers instead of sending massive amounts of data to central servers for processing.

For example, surveillance cameras typically send videos to a central recording device, which records only when it detects motion. Imagine the network impact if hundreds of high-definition surveillance cameras sent video feeds constantly to this central server. It's also worth noting that as the physical distance between the devices and the server increases, network transmission latency increases. With edge computing, video cameras are smart enough to sense motion, and they send videos to the cloud-based central recording system only when they detect motion. This drastically reduces the volume of data that is transmitted over the network and increases efficiency. Cloud and IoT complement each other: Connected devices generate huge amounts of data and the cloud provides the infrastructure to store, process, and analyze the data.

Taking everything into account…

Technology is certainly a major driver in any business today, and companies that struggle to integrate technology effectively will eventually find it difficult to succeed or even sustain their place in the marketplace. Connected devices, social media, and massive volumes of several forms of structured and unstructured data feeds have paved the way to further leverage technology and transform businesses. Digital transformation, which includes artificial intelligence, Big Data analytics, Internet of Things, and other emerging technologies, is fast becoming a key requirement for organizations to be innovative and remain competitive. Digital has opened doors for better analytics and decision-making, leading enterprises to explore, analyze, and obtain new insights and ideas to grow their business.

Offering a mature collection of services, tools, and security, the cloud is an ideal platform for any business developing a digital transformation strategy. All leading cloud providers offer comprehensive and competitive solutions, tools, and services to address even the most complex digital transformation initiatives for their customers. Furthermore, the horizontal and vertical scaling of infrastructure offered by the cloud makes it highly suitable for the robust compute requirements digital technologies demand.

Artificial intelligence encompasses machine learning, speech recognition, speech synthesis, image recognition, image comparison, video analytics, and many other applications. It is widely implemented by enterprises to improve customer experience, implement chatbots, develop training, and more. Cloud providers offer APIs that application developers can utilize to build intelligent AI-enabled applications.

Big Data analytics takes data analytics to a different dimension, providing the means to gather better insights from structured data such as feeds from conventional relational databases and unstructured data such as data streams from social media. It enables businesses to better understand customer perceptions, for example, by analyzing comments and conversations on social media. Implementing Big Data and its associated tools on-premise requires a massive investment of time and money. A cloud platform provides all the tools and the elastic compute power needed to help businesses focus on the benefits of analytics rather than worrying about implementation, maintenance, and support.

Internet of Things (IoT) connects devices to the internet and provides individuals and organizations greater control of their homes, lives, and businesses. A rain sensor-based watering system, for example, monitors humidity, automatically waters fields, promotes better growth of crops, and cuts costs. Edge computing provides better processing ability to devices, eliminating the need to transfer large volumes of data to central servers for processing. There are plenty of solutions and tools offered by leading cloud providers for edge computing and IoT in general.

Ignoring the digital revolution will be disastrous to organizations as well as individuals. Humans and businesses generate trillions of gigabytes of data every day. Whether we realize it or not, digital is transforming our lives. It is essential for organizations to develop and implement a digital strategy that works with cloud initiatives and tools.