The Hindu Editorial Analysis- 29th June 2023 | Current Affairs & Hindu Analysis: Daily, Weekly & Monthly - UPSC PDF Download

Mahalanobis in the era of Big Data and AI

Why in News?

Professor P.C. Mahalanobis, who introduced statistics to India, is a scientist whose absence is felt dearly even today. Mahalanobis’s lifelong courtship with statistics, his unwavering and fearless leadership to advance a statistics and survey culture in India, the founding of the Indian Statistical Institute — “a mighty monument of his handicraft” — and his nurturing of a generation of outstanding academicians have all left behind an enduring legacy.

Prasanta Chandra Mahalanobis (1893-1972)

• He is considered the father of modern statistics in India, founded the Indian Statistical Institute (ISI) and shaped the Planning Commission of India.
• He also devised a statistical method called ‘Fractile Graphical Analysis’, used to compare socio-economic conditions of varied groups.

Contributions

• Mahalanobis Distance: In 1930 he proposed the Mahalanobis Distance for the first time, which is a measure of comparison between two data sets.
• Indian Statistical Institute: In 1932 he founded ISI in Kolkata.
• Journal: In 1933 he started ‘Sankhya: The Indian Journal of Statistics’.
• He established National Sample Survey and set up the Central Statistical Organisation to coordinate statistical activities in 1950
• In 1955 he became the member of Planning Commission
• He was instrumental in formulating India’s second five-year-plan (1956-1961), which laid the blueprint for industrialisation and development in India.

Applications of Big Data

• Healthcare and IoT: Using big data tools, data regarding patient experience is collected and used by doctors to give better treatment. IoT devices can sense a symptom of probable coming disease in the human body and prevent it from giving advanced treatment.
• Education Sector: Online educational course conducting organizations utilize big data to search for candidates, interested in that course.
• Government Sector: Big data helps in making faster and more informed decisions regarding various political programs and also helps in identifying areas that are in immediate need of attention. Big Data is hugely used for deceit recognition in the domain of cyber security.
• Product Development: Big data is used by companies like Netflix to anticipate user demand. By categorizing important characteristics of previous and present products or services and analyzing the relationship between those characteristics and the commercial success of the offerings, they create prediction models for new products and services.
• Operational Efficiency: Big data analyze and assess production, customer feedback and returns, and other factors that reduce outages and anticipate future demands. Big data can also be used to improve decision-making in accordance with the needs of the marketplace.
• Customer Experience: In order to enhance the engagement process and increase the value provided, big data enables you to collect information from social media, site traffic, call logs, and other sources.
• Drive Innovation: Big data may support innovation by analyzing the relationships between people, institutions, things, and processes, and then coming up with fresh applications for those insights.
• Fraud and Compliance: Security landscapes and compliance requirements are constantly evolving. Big data makes it easier to spot patterns in data that lead to fraud and can consolidate a lot of data to speed up regulatory reporting.
• Machine Learning: Big data enable us to teach machines instead of programming them. That is made possible by the availability of massive data to train machine learning models.

Challenges

• Storage: Despite the development of new storage technology, data volumes double in size approximately every two years. Organizations still have trouble keeping up with their data and figuring out how to store it efficiently.
• Privacy: Data privacy and Net neutrality are the twin issues of digitization that are brought on by big data analytics.
• Security: For organizations, security is a major concern. Information that isn’t encrypted is vulnerable to theft or damage from cyber criminals.
• Technological Challenges: Big data have many inherent limitations, including a poor infrastructure for gathering and managing such vast amounts of data, concerns with storage and computing, scalability, and streaming.
• Rapid Enhancement: Big data technology is changing rapidly. Keeping up with big data technology is an ongoing challenge.

Government Initiatives and interventions

Following are some of the government level initiatives and interventions in the field of big data:

National Data & Analytics Platform

• NITI Aayog launched the National Data & Analytics Platform (NDAP).
• The platform intends to democratize access to public government data by making data available on a user-friendly platform that is accessible, interoperable, and interactive.
• It hosts foundational datasets from various government agencies, presents them coherently, and provides tools for analytics and visualization.

Big Data Management Policy

• India has formulated a Big Data Management Policy and is in the process of establishing a Data Analytics Centre.
• The “Big Data Management Policy” created by CAG is a beginning point when it comes to auditing sizable amounts of data produced by the public sector in the states and union territories.
• The Mission is being implemented with all the Technology Innovation Hubs (TIHs) undertaking activities under the following four major categories i.e.,

1. Technology Development
2. Human Resource Development
3. Entrepreneurship Development
4. International Collaboration.

Way forward

Establishing Data Management Protocols: The Data Management protocols ensure that data satisfies the following characteristics:

• Authenticity – Data is created through the process it claims
• Integrity – Data is complete, accurate, and trustworthy.
• Relevance – Data is appropriate and relevant for the identified purpose.
• Usability – Data is readily accessible in a convenient manner.
• Security – Data is secure and accessible only to authorized parties.

Adequate data protection laws incorporating some of the best practices followed in the World.

Economically, the integration of big data and blockchain technology has the potential to revolutionize the way we approach and utilize big data in the present.

Conclusion

Big data has many applications, therefore it makes sense why there is so much hype surrounding it. It is important because of the way an organization uses it, not because of how much of it they have managed to collect. Its analysis can be done rapidly and efficiently with some big data solutions. These Big Data solutions are utilized to benefit from the massive amounts of data in almost all industry verticals.