ICT30005 Professional Issues In IT

Introduction

This paper seeks to provide an overview of data profiling inside an organisation. Data profiling in regard to an organization is the process of examining database data and obtaining data information. Data profiling is used to understand data content, sources, structure, and relationships. It aids in identifying data anomalies and quality (Abedjan et al., 2018). These statistics are used in data profiling. A briefing research paper by Abedjan et al. (2018) summarises research findings for a specific audience. Organisations employ purpose-built data profiling tools and related procedures for improving data accuracy in a company database. This study provides a comprehensive examination of data profiling in businesses.

Literature Review

In this paper, this section would provide a review of literature related to data profiling along with the relevant issues.

Current Concerns Regarding Data Profiling

Data Quality Problem

According to Kwon, Lee and Shin (2014), the topic of data quality in regard to data profiling has been the emphasis of this peer-reviewed study in this review. The article under consideration identifies data quality as a primary source of worry for businesses and their operations. According to the literature of Talib (2013), the problem of data quality is caused by a growth in the diffusion of data as well as the exchange of data across organisations. It is the demand for data uniformity that is causing anxiety among enterprises. Following the results of the paper, it is concluded that, for businesses to fulfil their objectives, they are now concerned with determining the dependability of data sources, the medium of data storage, and standardised data collection and storage (Pambreni, Khatibi, Azam and Tham, 2019). The extraction of data and transformation of data for the organisation are two additional considerations of post-data profiling activities.

Poor Quality Data and its Consequences

In the opinion of Abedjan et al. (2018), low-quality data has a significant impact on the organisation. In this article, it is suggested that the functioning and administration of companies will alter as a result of the debate. The company suffers a revenue loss as a consequence of the poor quality of the data collected. In a single year, the data quality issue results in a loss of around $9.7 million per year (Anodot Ltd., 2022). According to the data quality statistics, businesses in US alone lose $3.1 trillion every year only because of the poor quality of data (Anodot Ltd., 2022). According to the findings of this review research, poor quality data is the most significant factor contributing to the decline of a firm or an organisation. According to Hazan et al. (2014), poor quality data has a significant impact on an organization’s revenue creation as well as employee performance.

Data Cleansing

Data profiling assists in data cleaning by identifying susceptible data (Ilyas and Chu, 2019). Missing values, skewed distributions and outliers are discussed in this article. After data purification, data profiling is repeated. This would ensure data purification. Adopting data cleaning will assist the company to ensure data quality. This article addresses data cleaning as a crucial part of data profiling, which helps the company flourish. The reviewed paper states that this procedure is repeated to assure data purification.

Analysis of Data Profiling

Process of Data Extraction in Data Profiling

Data extraction is a crucial part of data profiling, according to Abedjan et al (2018). A quick examination of length, range, data type, variance and frequency is required. This article reviews the requirement for obtaining data through collecting data and transforming it into a format that can be easily evaluated. A framework is created by separating important text fragments and extracting essential information from them. The data extraction approach enables extensive analysis and data acquisition.

Data Quality Management

Data quality management ensures the organization’s data assets. There are three main areas to address data quality management. Quality management is based on the totality of data assets. The next stage for management of the data quality is to check data accuracy. Being accurate, data must fulfil both external and internal criteria. This study of Dong and Srivastava (2015) examines internal and external needs as well as business processes along with decision making. Data quality management requires honesty. The company uses data integration to transport data from one source to another.

Benefits of Data Profiling

According to the reviewed article of Abedjan et al. (2018), the various benefits of data profiling are the improvement of data quality and reducing the time for the implementation cycle. It helps in improving the understanding of the data for an organization. This reviewed article states that a corporate database is probable to be precise and it is expected to maintain data quality. It can be achieved effectively with the help of data profiling. The statistical data are at first collected from different platforms and then it is used to gain the perspective of the people and their experiences. It helps the organization to assess the needs for achieving the aim of the organization. There are also specific concerns related to data profiling, and it includes the tampering of information.

Data Profiling and Categorization

According to Skinner, Edwards and Smith (2020), deconstructing and gathering data is an issue. This is done to prevent duplicate values, mistakes, white spaces, unnecessary characters, and symbols. According to the article of Reeve (2013), data profiling is used to eliminate undesired letters, spaces and symbols from text, but no mistakes. To find and eliminate errors from data, several data processing methods such as text mining, data mining, and extraction must be used. The findings of data profiling are applied to data mining methods, allowing information to be retrieved and found from an organization’s database. Traditional data mining approaches may also be classified (Han, Pei and Kamber, 2011). It is done to get a meaningful data structure. This article describes categorization as a learning process where articles are classified based on their content. This article explains the classifier’s abilities to perform group assignments. It presents each data category as a binary issue. These approaches analyse words or phrases in the text and the phase frequency.

Recommendation

It is suggested that businesses utilise data profiling to improve data quality. Quality of data is critical to an organization’s revenue growth. It is suggested since it minimises data cycle time. The organisation should establish a standard for data profiling that includes results release and reporting to key stakeholders (Skinner, Edwards and Smith, 2020). It is advised to ensure data source dependability and uniformity to achieve an organization’s business objectives.

Conclusion

This paper concluded that data profiling is an essential practice for enterprises. The company may utilise data profiling to maintain data quality and improve data accuracy. The main problems of data profiling are data purification and data quality. It indicates that data cleaning is done to ensure data quality. A literature evaluation for data profiling using data extraction and collecting. The document efficiently provides the company with data profiling advantages. It concludes that poor data quality is the cause of many organisations’ revenue loss and gives statistical evidence to support this. The study covers categorisation as an essential part of data profiling and does it well.

References

Abedjan et al. (2018). Data Profiling. Retrieved view https://www.morganclaypool.com/doi/abs/10.2200/S00878ED1V01Y201810DTM052

Abedjan et al. (2018). Data Profiling. Morgan & Claypool Publishers.

Anodot Ltd. (2022). The Price You Pay for Poor Data Quality. Retrieved view https://www.anodot.com/blog/price-pay-poor-data-quality/#:~:text=According%20to%20Gartner%20research%2C%20%E2%80%9Cthe,due%20to%20poor%20data%20quality.

Dong, X. and Srivastava, D. (2015). Big Data Integration. Morgan & Claypool Publishers.

Han, J., Pei, J. and Kamber, M. (2011). Data Mining: Concepts and Techniques. Elsevier.

Hazan et al. (2014). Data quality for data science, predictive analytics, and big data in supply chain management: An introduction to the problem and suggestions for research and applications. International Journal of Production Economics, 154, pp. 72-80.

Ilyas, I. and Chu, X. (2019). Data Cleaning. Morgan & Claypool.

Kwon, O., Lee, N. and Shin, B. (2014). Data quality management, data usage experience and acquisition intention of big data analytics. International Journal of Information Management, 34(3), pp. 387-394.

Pambreni, Y., Khatibi, A., Azam, S. and Tham, J. (2019). The influence of total quality management toward organization performance. Management Science Letters , 9(9), pp. 1397-1406.

Reeve, A. (2013). Managing Data in Motion: Data Integration Best Practice Techniques and Technologies. Newnes.

Skinner, J., Edwards, A. and Smith, A. (2020). Qualitative Research in Sport Management. Routledge.

Talib, F. (2013). An overview of total quality management: understanding the fundamentals in service organization. International Journal of Advanced Quality Management, 1(1), pp. 1-20.