In the next five to ten years the way people and businesses make decisions, may change dramatically. Data and the knowledge people can get from it will be the centre of this process.
Unconceivable amounts of data will be produced by individuals, corporations or devices. Vendors and software will provide tools to extract knowledge from that huge amount of data and people will easily access that knowledge to make decisions almost in real time. New opportunities and challenges are arising with the arriving of IoT1 and big data2.
The benefits of big data in an organization include “enhanced data sharing through transparency, improved performance though analysis, augmented market segmentation, increased decision support through advanced analytics and greater ability to innovate products, services and business models” 3.
The first challenge to many companies today is still to understand well the data they are dealing with. “Listening” to your own data requires analysis. Rather than managing the data, it is necessary to interpret and use it to generate decisions. This requires improved analytics to help companies identify many new factors that determine, for example, the prices of each of their customer segments and products.
The second challenge is to get more data from external sources. Companies will soon realise the advantage of gathering all kinds of external types of data that can impact their businesses: structured data, semi structured and unstructured data from new sources like multimedia content, social media and sensors/devices data. Still a lot of effort/investment in hardware, software and data processing techniques will be necessary to store and extract knowledge from that huge amount of plain and simple data.
The third challenge is “opening the door” to others. After extracting knowledge from the data and giving it to the company value chain and decision making players, companies will find out that other parties will be interested and willing to pay access to their data. So, what data does your company have that might be of public interest? What data and knowledgeable info can it provide to other parties?
Storing data at an affordable cost is going to change the future data retention policies of companies. Big data will allow a huge or unlimited extension of the data’s lifetime. One or three months of data is not enough when you want to look deeper into it.
It’s important to understand that having access to huge amounts of data isn’t necessarily better when it comes to getting knowledge from data. Most businesses today don’t use a fraction of the data they have access to, so the priority should be the first challenge, asking the right questions and making a good use of the existing data.
As TIBCO specialist building integration services I can already envision that Big Data and IoT will create more opportunities in my working field, mainly for two reasons: 1) everything needs to be connected and 2) data needs to be delivered at an increasingly fast speed.
TIBCO is already showing what it can offer in this new Big Data paradigm: The TIBCO Fast Data Platform brings together integration, analytics and event processing to solve customer problems like be able to clean, merge, enrich and shape data; perform visual and predictive analytics; and apply data prep and real-time math to streaming data.
Also, the next big TIBCO announcements point towards the future of big data, through projects like Project Flogo, an ultra-lightweight integration software solution for IoT and Graph Database, a transanalytical database for big data.
Surely more challenges to this new big data paradigm will come up after companies find out what to do with their data. Some are already emerging, linked to creating recommendation systems, cost/time optimizations, new product developments and better decision making. In the near future others are sure to follow, artificial intelligence and machine learning will lead us to a brave new world of fast paced and intelligent decisions, that will revolutionise businesses and the world as we know.
1 The definition of IoT given by the International Telecommunication Union (ITU) is “A global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies”. According to Gartner, the IoT population will reach 26 billion units in 2020 and that IoT products and service suppliers will generate incremental revenue exceeding $300 billion, mostly in services. There are various kinds of IoT applications, intelligent transportation systems, e-health or smart home are some of them.
2 ““Big Data” refers to a combination of an approach to informing decision making with analytical insight derived from data, and a set of enabling technologies that enable that insight to be economically derived from at times very large, diverse sources of data.” by John Akred – Founder and CTO, Silicon Valley Data Science
3 in Journal of Technology Research (AABRI), Bernice Purcell, Holy Family University
ANTONIO QUEIROZ
TIBCO Consultant at Polarising
