資料管理是以經濟實惠的方式,安全、有效率地收集、保存和使用資料的實務。資料管理的目標是要協助人員、組織和連網物品在政策和法規的限制內最佳化資料的使用,以便制定決策並採取讓組織享有最大效益的動作。隨著組織越來越仰賴無形資產來創造價值,強大的資料管理策略將前所未有地變得更加重要。
The objective of data management is to help people, organizations, and online items to optimize the use of data within the limits of policy and law, in order to formulate decisions and take action to maximize the benefits for the organization. As organizations increasingly rely on non-physical assets to generate value, the powerful data management strategy will become even more important than ever.
管理組織中的數位資料涉及廣泛的工作、政策、程序和實務範圍。資料管理的工作範圍很廣,包含以下因素:
There are a wide range of tasks, policies, procedures and practices involved in managing digital data in the organization.
- 建立、存取和更新橫跨不同資料層內的資料
- 儲存橫跨多個雲端與內部部署之間的資料
- 提供高可用性和災難還原
- 使用逐漸成長的各種應用程式、分析和演算法中的資料
- 確保資料隱私權和安全性
- 根據保留排程和合規性需求封存和銷毀資料
正式資料管理策略滿足使用者和管理員的活動、資料管理科技的功能、法規需求的要求,以及組織需求,以從其資料取得價值。
The formal data management strategy responds to the activities of users and managers, the functions of data management technology, legal requirements and organizational needs in order to derive value from their data.
在當今數位經濟時代,資料是一種資本,也是數位產品和服務的經濟生產要素。就像汽車製造商如果缺乏必要的金融資本,就無法製造新車款一樣,如果缺乏資料來餵養車載演算法,就無法使汽車發揮自主功能。資料此全新的角色對於競爭策略和運算未來具有深遠的含意。
In today’s digital economy, data is a capital and a factor in the economic production of digital goods and services. Just as car manufacturers lack the necessary financial capital, they cannot create new cars, and when data is not available to feed auto-carriage algorithms, they cannot make cars self-functional.
鑒於資料所具有的中央和關鍵任務角色,無論規模或類型,每間組織都需要強大的管理實務和健全的管理系統。
Given the central and critical role of the data, every organization needs strong governance practices and sound management systems, regardless of their size or type.
現代企業需要一個資料管理解決方案,提供有效方式來管理不同但統一的資料層資料。資料管理系統建立在資料管理平台上,可以包括 資料庫、資料湖和資料倉儲、巨量資料管理系統、資料分析等。
Current businesses need a data management solution to provide effective ways to manage different but uniform data layers , 所有這些元件會作為一個「資料公用程式」一起工作,提供組織對其應用程式所需的資料管理功能,以及使用這些應用程式所產生的資料分析和演算法。儘管目前工具有助於資料庫管理員 (DBA) 自動化許多傳統的管理工作,但由於大多數資料庫部署的規模和複雜度,仍然經常需要手動介入。每次需要手動介入時,發生錯誤的機會隨之增加。減少手動資料管理的需求是新型資料管理技術「自主資料庫」的的主要目標。 All these components will work as a "data utility" to provide the organization's data management functionality for its applications, as well as the data analysis and algorithms generated by the applications. While the current tools help the database manager (DBA) to automate many of the traditional management, the pattern and complexity of most databases deployments often require manual intervention. Each time a manual intervention is required, the opportunity for error increases. The need to reduce manual data management is the main objective of the new data management technology, 持續交付軟體的最關鍵步驟是連續整合 (CI)。CI 是一種開發實務,開發人員會將其程式碼變更 (通常是小型、增量的變更) 提交至集中的來源儲存區域,而這會啟動一組自動化的組建和測試。此儲存區域可讓開發人員在將錯誤傳送給實際環境執行之前,先提早自動擷取錯誤。連續整合管道通常涉及一系列步驟,從程式碼提交到執行基本的自動檢查/靜態分析、擷取相依性,以及最終建立軟體並在建立組建使用者自建物件之前執行一些基本單位測試。Github、Gitlab 等原始程式碼管理系統提供 Webhook 整合,讓 Jenkins 等 CI 工具可以在每次存入程式碼之後,開始執行自動化組建和測試。 The key step for a continuous delivery software is continuous integration (CI). The CCI is a development implementation in which developers submit their code changes (usually small, incremental changes) to a centralized source storage area, which triggers a self-inflicted build and test. This storage area allows developers to take early automatic errors before sending errors to the actual environment. The continuous integration conduit usually involves a series of steps, from the code to the basic automatic check/silent analysis, calibration, and, ultimately, the building of software and some basic unit tests before building their own objects. The original code management systems such as Github, Gitlab, etc., provide Webhook integration, allowing Jenkins and other CI tools to start building and testing on their own. 資料管理平台是收集並分析組織內大量資料的基礎系統。商業資料平台一般包括資料庫廠商或協力廠商所開發的管理軟體工具。這些資料管理解決方案可以協助 IT 團隊和 DBA 執行典型任務,例如: These data management solutions can assist the IT team and DBA in their typical tasks, such as: 日漸普及的雲端資料庫平台,能使企業透過高成本效益的方式快速擴展或縮小規模。部分可供服務使用,甚至能讓組織節省更多成本。 The increasingly popular allows businesses to expand quickly or narrow down in a cost-effective manner. Some of the services are available and even save organizations more costs. 自主資料庫是以雲端為基礎,使用人工智慧 (AI) 和機器學習自動化許多由 DBA 執行的資料管理工作,包括管理資料庫備份、安全性和效能調整。 The autonomous database is cloud-based, using artificial intelligence (AI) and machine learning to automate many DBA-run data management, including management database backup, security and efficiency adjustments. 自主資料庫又稱為自主驅動資料庫,為資料管理提供了巨大的優點,包括: The autonomous database, also known as as the autonomous database , provides significant advantages for data management, including: 逐漸普遍的雲端資料平台可讓企業以經濟實惠的方式快速擴大或縮小。部分可供服務使用,甚至能讓組織節省更多成本。 The increasingly widespread cloud-based data platform allows businesses to expand or shrink rapidly in a cost-effective manner. Some of the services are available, and even more cost-effective for the organization.
在許多方面,巨量資料就是字面上的意思,代表大量的資料。但巨量資料的形式也比傳統資料更廣泛,而且資料的收集速度也很快。思考一下每天或每分鐘來自 Facebook 等社交媒體的所有資料。資料的數量、多樣性和速度使得其對於企業如此寶貴,但也使得管理變得非常複雜。
In many ways,
隨著越來越多資料從不同來源 (如攝影機、社交媒體、錄音和物聯網 (IoT) 裝置) 收集,大數據管理系統已浮現。這些系統專事於三個一般領域。
As more and more data are collected from different sources (such as cameras, social media, audio recording and network (IOT) devices), large data management systems have surfaced. These systems are dedicated to three general domains.
- 巨量資料整合從批量資料到流媒體資料帶來了不同類型的資料,並將其加以轉換以供使用。
- 大數據管理以有效率地、安全地和可靠地方式處理資料並儲存於資料湖或資料倉儲,通常是使用物件儲存體進行。
- 巨量資料分析透過分析發現新的洞見,包括圖形分析,並使用機器學習和 AI 視覺化來建立模型。
公司正在運用大數據改善並加速產品開發、預測性維護、客戶體驗、安全性、營運效率等等。隨著大資料擴大,商機也會擴大。
The company is using large data to improve and accelerate product development, predictive maintenance, client experience, safety, operating efficiency, and so on. As big data expands, business opportunities expand.
現今資料管理的大多數挑戰都是源自於商業步調加快和資料加速擴散的緣故。組織可用的資料多樣性、速度和數量不斷擴大,組織尋求更有效的管理工具跟上腳步。組織面臨的部分主要挑戰包括以下項目:
Most of the challenges in data management today stem from the faster pace of business and the faster expansion of data. The multiplicity, speed and quantity of data available to the organization are constantly expanding, and the organization seeks more effective management tools to keep pace. Some of the major challenges facing the organization include the following:
缺乏資料洞見 |
目前收集和儲存的資料來自於許多多元的來源 (例如感應器、智慧裝置、社交媒體和攝影機),而且資料來源和多樣性也不斷增加。但是,如果組織不知道自己擁有什麼資料,資料的位置,以及這些資料的使用方式,則這些資料都沒有用處。資料管理解決方案需要調整及效能,才能及時提供有意義的洞見。 The data currently collected and stored comes from many diverse sources (e.g. sensors, smart devices, social media and cameras) and the sources and multiplicity of data are constantly increasing. But these data are useless if the organization does not know what it has, where it is located and how it is used. Data management solutions need to be adapted and effective in order to provide meaningful insights in time. |
難以維護資料管理效能層次 cannot maintain the data management effectiveness layer |
組織隨時都在獲取、儲存和使用更多資料。為了在這不斷擴展的等級中保持峰值回應時間,組織需要持續監控資料庫正在回答的問題類型,並隨著查詢變化而更改索引,但不能影響效能表現。 In order to maintain a peak response time in this continuously expanding level, the organization needs to maintain continuous monitoring of the type of questions that the database is answering and change the index as queries change, without affecting performance. |
因應不斷變更的資料需求所面臨的挑戰 challenges faced in responding to changing data needs |
合規性法規複雜、橫跨多司法管轄區,而且不斷改變。組織需要能夠輕鬆地檢閱其資料,並識別屬於全新或經修改之需求的任何內容。尤其是必須偵測、追蹤和監控個人識別資訊 (PII) 是否符合日益嚴格的全球隱私權法規。 Organizations need to be able to easily access their data and identify anything that is new or modified. In particular, they must detect, track and monitor whether personal knowledge (PII) is in line with the increasingly stringent rules of global privacy law. |
輕鬆處理及轉換資料 Easy Process and Convert Data |
收集和辨識資料本身並不能提供任何價值,組織需要處理資料。如果需要花費大量的時間和精力將資料轉換為他們需要的分析內容,那麼這類分析就不會進行。因此便會失去該資料的潛在價值。 The collection and identification of data does not in itself provide any value, and the organization needs to process the data. If it takes a lot of time and effort to convert the data into the analysis they need, then this kind of analysis will not take place. |
實際儲存資料的需要 practical data storage needs |
在資料管理的新世界,組織會將資料儲存於多個系統中,包括將資料以任何格式儲存於單一存放庫的資料倉儲和非結構化資料湖。組織的資料科學家需要一種方法來快速、輕鬆地將資料從其原始格式轉換為他們需要的形狀、格式或模型,以便進行廣泛的分析。 In the new world of data management, the organization will store the data in multiple systems, including storage and non-structured data lakes where the data is stored in any format in a single repository. The organization’s data scientists need a way to quickly and easily convert the data from its original format to the shape, format or model they need for broad analysis. |
需求持續最佳化 IT 靈活性和成本 Needs to continuously optimize IT spirituality and costs |
藉助雲端資料管理系統可用性,組織現在可以選擇要將資料保留在內部部署環境、雲端,還能與兩者混合中混合使用。IT 組織需要評估內部部署和雲端環境之間的相同性程度,以維持 IT 靈活性並降低成本。 By contributing to the availability of the cloud data management system, the organization can now choose to keep the data in its internal deployment environment, cloud cover, and mix it with the two. The IT organization needs to evaluate the same level of internal deployment as the cloud cover to maintain IT viability and reduce costs. |
歐盟於 2018 年 5 月頒布和實施的一般資料保護規範 (GDPR) 包括個人資料管理和處理的七大原則。這些原則包括合法性、公平性、透明性、目的限制、準確性、儲存線置、完整性和機密性等等。
The EU General Data Protection Regulations (GDPR) issued and implemented in May 2018 include seven principles of personal data management and processing. These principles include legality, fairness, transparency, purpose limits, accuracy, saving lines, integrity and confidentiality.
GDPR 及其他跟隨其腳步的法律 (如加拿大消費者隱私權法,CCPA) 正在改變資料管理的面貌。這些需求提供標準化的資料保護法,讓個人可以掌控其個人資料及其使用方式。實際上,當組織未能在資料擷取時獲得知情同意、對於資料使用或地區的掌控能力不佳,或未能符合資料抹除或可移植性需求時,此法律將消費者轉變成利害關係人並具有實際的法律追索權。
GDPRs and other laws that follow them (e.g. Canada Consumer Privacy Act, CCPA) are changing the face of data management. These requirements provide standardized data protection laws that allow individuals to control their personal data and how they use them. In practice, when organizations fail to obtain informed consent when data are extracted, have poor control over data use or areas, or fail to meet data elimination or portability needs, the law transforms consumers into stakeholders with effective legal recourse.
處理資料管理的挑戰需要一套全方位、深思熟慮的最佳實務。儘管特定最佳實務會隨著相關資料類型和產業而有所不同,但以下最佳實務可以處理組織現今面臨的主要資料管理挑戰:
The challenge of handling data management requires a comprehensive and well-thought-out set of best practices. While certain best practices will vary according to the type of data and industry involved, the following best practices can address the major data management challenges facing the organization today:
建立探索層來辨識您的資料 creates a search layer to identify your data |
在您組織資料層上的探索層允許分析師和資料科學家搜尋和瀏覽資料集,使您的資料可用。 The search layer in your organization's data layer allows analysts and data scientists to search for and browse data sets to make your data available. |
開發資料科學環境,有效地重新利用您的資料 Develops the scientific environment for effectively reusing your data |
資料科學環境會盡可能將資料轉置工作自動化,簡化資料模型的建立和評估。消除手動轉置資料需求的一組工具可以加速新模型的假設和測試。 As much as possible, the data science environment will automate data conversion and simplify data modelling and evaluation. A set of tools to eliminate manual data conversion needs can accelerate the simulation and testing of the new model. |
使用自主技術在您不斷擴展的資料層中保持性能水準 uses |
自主資料功能使用 AI 和機器學習持續監控資料庫查詢,並隨著查詢變更最佳化索引。如此可讓資料庫維持快速反應時間,並讓 DBA 和資料科學家有時間從事費時的手動工作。 Autonomous data functions use AI and machine learning to continuously monitor databases and improve the best indexes as they do. This allows the database to maintain a quick reaction time and allows DBA and data scientists time to work manually at a time-consuming. |
利用探索來掌握合規要求 uses exploration to control the requirements of the rules |
新工具運用資料探索檢閱資料並識別需要偵測、追蹤和監控的連接鏈結,以達成多個司法管轄區的合規性。隨著全球的合規性需求增加,此功能對於風險和安全性人員也日趨重要。 The new tool explores data access and recognizes links that need to be detected, tracked and monitored in order to achieve regulatory compliance in multiple jurisdictions. As global demand for compliance increases, this function is increasingly important for risk and safety. |
請確定您使用的是整合式資料庫 Make sure you use an integrated database |
融合式資料庫是一個資料庫,原生支援所有現代資料類型,以及內建於一個產品中的最新開發模型。最好的融合式資料庫能夠執行各種工作負載,包括圖表、IoT、區塊鏈及機器學習。 The integration database is a database that supports all modern data types and the latest development models built up in one product. The best integration database can perform all kinds of work loads, including |
確保資料庫平台具有所需的效能、規模及可用性,以支援您的業務 ensures that the database platform is as effective, structured and accessible as it needs to support your business |
整合資料的目標是能夠分析資料,做出更妥善、更即時的決策。這是一個可擴充的高效能資料庫平台,企業運用進階分析和機器學習,快速分析多個來源的資料,讓企業做出更好的業務決策。 The goal of integrating data is to be able to analyse data and make better and more immediate decisions. This is an expanded and efficient database platform where businesses use advanced analysis and machine learning to quickly analyse multiple sources of data and enable them to make better business decisions. |
使用通用的查詢層來管理多樣的資料儲存形式 manages multiple data storage forms using a common query layer |
新技術正讓資料管理存放庫可以共同運作,消弭之間的差異。橫跨許多種資料儲存的常見查詢層可讓資料科學家、分析師和應用程式無需了解資料的儲存位置,也無須手動將資料轉置成可用的格式,即可存取資料。 The new technology is enabling the data management repository to work together to eliminate the differences between the two. A common query layer across many types of data storage allows data scientists, analysts and applications to access data without knowing where to store the data and without having to manually convert it into an available format. |
資料科學橫跨多個學科,利用各種科學方法、流程、演算法及系統從資料中萃取出價值。資料科學家將一系列技術 (包括統計、電腦科學和商業知識) 結合起來,分析從網路、智慧型手機、客戶、感測器和其他來源收集的資訊。
The data scientists combine a range of techniques (including statistics, computer science and business knowledge) to analyse information collected from the Internet, smart phones, customers, sensors and other sources.
有了資料成為企業資本的新角色,組織正在尋找已經知道的數位新創公司和顛覆者:「資料」是一個寶貴的資產,可協助您在競爭對手之前找出趨勢、做出決策及採取行動。資料在價值鏈的新定位正帶領組織主動尋求從此新資本獲取價值的更好方法。
With data becoming a new player in business, the organization is looking for known digital start-ups and respondents: "Data" is a valuable asset that can help you identify, make decisions and take action before competing rivals. The new location of the value chain is leading the organization to seek better ways to gain value from this new asset.
深入了解最佳資料管理可為您做的工作,包括雲端的自主式策略 (PDF) 和可擴充的高效能資料庫雲功能的優點。
- Oracle Autonomous Database
全球第一個自主驅動資料庫 - Oracle Database
全球頂尖的融合式多模型資料庫管理系統 - Oracle Exadata
無與倫比的 Oracle Database 效能、規模與可用性 - Oracle Autonomous Data Warehouse
無複雜性的資料倉儲
注册有任何问题请添加 微信:MVIP619 拉你进入群
打开微信扫一扫
添加客服
进入交流群
发表评论