Creating a Data Map: How to Design Access to a Variety of Data Sources

Before you sit down and dive into this amazing blog… take a second and look at your surroundings. Just in your field of vision, how many connected devices do you see? I’m willing to bet that it’s at least 3-4. And do you know what all of those devices have in common? Data. They’re all generating data.

But, here’s the biggest point to take away as you navigate the world of information: Data is Not Created Equally – Know Your Source

There’s lots of different data points and creation mechanisms. To that extent, let’s start with Embedded Data. IDC’s report tells us that by 2025, embedded data will constitute nearly 20% of all data created — three quarters the size of productivity data and closing fast. To clarify, embedded data, could come from a number of origin points including, wearable devices, IoT, cars, building automation, machine tools, RFID readers, chip cards, and so much more.

According to the report, the embedding of computing power in a large number of endpoint devices has become a key contributor to data growth in our present era. Today, the number of embedded system devices feeding into datacenters is less than one per person globally, and over the next 10 years, that number will increase to more than four per person. While data from embedded systems tends to be very efficient compared with data from entertainment and other consumer usage, the number of files generated will be very large, measuring in the quintillions per year. All these embedded devices creating data fuel the growth and value of Big Data applications and metadata.

For example, let’s say you’re a content provider. You’d want to analyze the meta data of this source to better understand usage, latency, satisfaction, and even where to introduce new services.

This also means, as a data center operator, you have to be very careful around storage and data processing. If this is your line of business, be sure to plan accordingly.

Next, we have productivity data. As mentioned earlier, embedded data will constitute nearly 20% of all data created — three quarters the size of productivity data. Productivity data comes from a set of traditional computing platforms such as PCs, servers, phones, and tablets. Remember, this information will also continue to rise as we become much more mobile. To that extent, by 2025, connected users will number 75% of the world’s population, including previously unconnected groups like young children, the elderly, and people in emerging markets. The growth of real-time data that’s generated from all of these data points will cause a shift in the type of digital storage needed in the future. Furthermore, it’ll heighten the focus on low-latency responsiveness from enterprise edge storage solutions and offerings.

IDC estimates that the percentage of data in the datasphere that is processed, stored, or delivered by public cloud datacenters will nearly double to 26% from 2016 to 2025. Such clouds will process, store, or deliver not just IT services but also entertainment, grid telemetry, and telecommunications.

(article continues after break)

Keep Moving Forward with Aptude

Aptude is your own personal IT professional services firm. We provide our clients with first class resources in a continuous, cost-containment fashion.

Our support services will free up your senior IT staff from the overwhelming burden of day-to-day maintenance issues. Now they’ll have time to launch those new projects and applications you’ve been waiting for. Simply put, we can free up your resources and contain your costs. Let’s have a quick chat to discuss our exclusive services.

CONTACT US

Using the Right Tools to Navigate the Sea of Data

OK, so we know that data is being created all the time. We also know that understanding sources of data and leveraging that information is really important and valuable. Now, what tools can you use to actually get real-world value out of data that’s being created in our world? Well, there are lots. And, not enough space in this blog to cover them all. But, let’s talk about one that can, literally, help you map out your data sets – MapR.

MapR provides access to a variety of data sources from a single computer cluster, including big data workloads such as Apache Hadoop and Apache Spark, a distributed file system, a multi-model database management system, and event stream processing, combining analytics in real-time with operational applications. The cool part is the flexibility of this solution. So much so that the technology runs on both commodity hardware and public cloud computing services. This means that you have a lot of options around the ingestion of data and where your data engine actually resides.

Let’s briefly look at a real-world application. Let’s assume that you’re trying to ingest data from several data sets. And, let’s assume that the purpose here is to create an AI engine that helps you make decisions based on the data that’s ingested. This is where a solution like MapR can really help. The MapR platform natively supports both big data applications and data science applications like cognitive engines. They also support a variety of open APIs on top of their patented enterprise offering to deliver the most value and capability for the enterprise. Furthermore, for your AI-driven project, MapR supports the AI software development lifecycle, exploration, training, deployment and putting the models into a production environment. This includes full support for real-time event streaming and even hot-swappable models.

Basically, MapR supports AI/ML and analytics workloads. As mentioned earlier, this can all be done on a single platform and on one cluster. Depending on your own deployment, MapR supports hybrid and multi-cloud environments with open APIs, which allows you to avoid cloud lock-in and continue to run legacy applications as is. Finally, MapR works closely with both Docker and Kubernetes which really helps with the containerization of stateful applications.

This All Sounds Great! What Next?

Let’s pump the breaks here a little. You now have an idea as to some amazing tools that can be used around some really advanced projects. But, even with a ship that has an amazing engine, you’ll still need a guide to help you understand and navigate the sea of data. This is where I’ll give you a big piece of advice: don’t set sail alone. Working with a good partner can help you understand your business goals, how data plays a key role, and how to actually leverage data to your advantage. In fact, in having conversations around data utilization, there’s a solid chance you’ll learn about use-cases and success stories you didn’t even know about.

Tools like MapR offer powerful integration points into data utilization, cognitive systems, big data, and much more. Designing the right architecture will take a combination of the right tools, a good architecture, and someone helping guide the way.

What Are You Working On?

Looking for intelligent technological solutions? Seeking consultation on your upcoming projects? Need a quote for services? Contact Aptude’s executive team directly. It’s amazing just how much one little email can rapidly accelerate your productivity.

Guy DeRosa

Senior Vice President & Director of Operations
EMAIL GUY
p: 630.692.6700
ext. 406

Srinath Parepally

Vice President of International Development
EMAIL SRINATH
p: 630.692.6700
ext. 402

Uday Mehta

Vice President of Development
EMAIL UDAY
p: 630.692.6700
ext. 403

Ravi Das

Sr. Business Development Manager
EMAIL RAVI
p: 630.692.6700
ext. 409

Gain Time, Increase Currency, Contact Us

It’s amazing how one quick email can change your life. Give us a shout! We’ll get back to you right away with the right person for what you’re looking to accomplish.

You don't have to go into a lot of detail now, we'll dial it in later.
=
What our clients are saying…

schneider-national

Aptude provides onsite and offshore Oracle DBA support, which includes troubleshooting, back-up, recovery, migration, upgrades, and daily maintenance of Oracle database servers. Aptude has been working with our team for the past four years and we continue to use them and are satisfied with their work

Warren E., Schneider National

build

Aptude provided Build.com a Java, MySQL, Webservices and other UI based solution in the business domain of analyzing and reporting on user activities for our ecommerce website. Utilizing Omniture’s APIs to download, parse, and regenerate and upload back so that we could be more effective in our marketing. I was satisfied with their project work and delivery and would consider utilizing them for future projects.” Build.com

Dan D., build.com

ugahead

Aptude provided us with Oracle DBA migration support, including an upgrade from Oracle 11.1 to Oracle 11.2, and the project was completed on time and to specifications. The project manager and project consultants were responsive and proactive, resulting in a successful conclusion to the work. I would definitely contract with them again, and have recommended them to other technical offices at the University of Georgia.

David C., University of Georgia

wolters-kluwer

Thank you for the hard work your team has put forth to staff the contract positions at Wolters Kluwer. Aptude has consistently scored high in our supplier carding and even more important you are a vendor we can always trust. I am especially impressed with your ability to tackle our positions that other vendors have not been able to fill.

Andrea G., Wolters Kluwer
By |2019-01-07T17:54:05+00:00January 7th, 2019|

About the Author:

Bill Kleyman
Bill Kleyman Industry Analyst | Board Advisory Member | Writer/Blogger/Speaker | Executive | Millennial | Techie Bill is an enthusiastic technologist with experience in a variety of industries. This includes data center, cloud, virtualization, security, AI, mobility, edge solutions, and much more. His architecture work includes large virtualization and cloud deployments as well as focusing on overcoming emerging business challenges. Bill enjoys writing, blogging, and educating colleagues around everything that is technology. During the day, Bill is the Director of Technology Solutions at EPAM where he works with AI, machine learning, blockchain solutions, DevOps, cloud, and advanced technologies to help engineer the digital future. Bill's whitepapers, articles, video blogs and podcasts have been published and referenced on WindowsITPro, Data Center Knowledge, InformationWeek, NetworkComputing, TechTarget, DarkReading, Forbes, CBS Interactive, Slashdot, and many others. As an active member of the technology industry, Bill was ranked #16 globally in the Onalytica study which reviewed the top 100 most influential individuals in the cloud landscape; and #4 in a different Onalytica study reviewing the industry's top Data Security Experts. Email bill@kleyman.org

It's What We Do