e-Catalog Data Sources - PKC - Obsidian Publish

> [[e-Catalog Technical Requirements|Previous Section: Technical Requirements]] The sample data for the e-Catalog project was obtained from [[LKPP]]. Because it is an operational application designed for the public display of data content, the pre-existing e-Catalog application can be considered a solitary data source. On the contrary, the Proof of Concept (PoC) program should demonstrate a mechanism for ensuring accountability for all data changes. The scheme for product detail data is thus derived from the data set provided from the start. # Data Published in Public Space In this project, the Ethereum blockchain will be used to identify data change trails. (We chose the [[Goerli]] test network specifically for demonstration purposes.) As a result, Etherscan will automatically record and make public all Ethereum transaction activities. This Proof of Concept program will make all changes visible to the public by publishing our data change activities to this blockchain-based public space. To demonstrate the concept of data traceability, all price changes displayed in the e-Catalog app must be linked to a specific change event on the Etherscan website. We chose to write two Solidity contracts to register these data change activities on Ethereum: [[SubmitChangeEvents]] and [[AccountableChange]]. As a result, changes to product details will be published by the e-Catalog application and should be visible as event entries on the blockchain log. However, changes that are not published on the Ethereum block are considered illegal. This ensures that a product's prices or other detailed data values cannot be changed without accountability. If the data content is large, it should be referenced as an IPFS hash value for the primary data source, and only the aforementioned authentication and authorization mechanism should change the reference. ## Authentication and Authority The most important aspects of any data system are user authentication and data change authority assignment. To ensure consistency in authenticity and authority, this Proof of Concept program will use Ethereum as the programable security model to provide a single source of authentication and authority assignment. In other words, the Smart Contracts embedded in this PoC define the rules that govern authentication and authority in the context of this e-Catalog app. This PoC may use a combination of [[multi-signature]] and Ethereum-based wallet authentication and authorization to ensure the irrefutability of user actions. The blockchain, on the other hand, keeps a public record of how data ownership and even software update authority are transferred. All changes to data content that is not stored on the main chain should be marked with time stamps and change authorization certificates issued by the Smart Contract system. These data markers will assist the public and the administration in tracking who and when data updates were performed. ## LKPP provisioned data set In this collaborative project, LKPP provided our team with a sample data set. A total of one thousand laptop data points were provided to Aqua Software Technologies. The data schema is shown below: | key | sample data value | data type | |:-------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------| | nama_produk | HP 240 G7 | string | | no_produk | X4522100001-PEP-001442670Y | string | | harga_utama | 6849315 | integer | | harga_pemerintah | 6849315 | integer | | jumlah_stok | 0 | number | | produk_gambar | "https://e-katalog.lkpp.go.id/katalog/produk/download/gambar/14032447.png" | array of strings (URL) separated by "|" | | berlaku_sampai | "\"2023-12-31T00:00:00\"" | date | | nama_manufaktur | "hp inc" | string | | no_produk_penyedia | 224U5PA | string | | uom | "Unit" | string | | jenis_produk | "import" | string | | kode_kbki | 4522100001 | number | | tkdn_produk | null | null object | | nilai_bmp | null | null object | | sum_tkdn_bmp | 0 | number | | nama_pemilik_sertifikat_tkdn | "PT. Armindo Intercorp " | string | | Kind_of_tkdn_product | null | null object | | spesifikasi |  "Koneksi : Realtek 802.11ac (2x2) Wi-Fi\u00c2\u00ae and Bluetooth\u00c2\u00ae 5 Combo | VGA : UMA | RAM : 4GB DDR4 | Warna : Dark ash silver | Storage : 256GB SSD M.2 | Deskripsi : CPU i3-1005G1 , Graphics UMA , RAM 4GB DDR4 , Storage 256GB SSD M.2 , ODD - , OS Win 10 Home , Display 14\" HD , Warranty ( NON ADP) 1/1/0 | Type/Model : HP 240 G7 | Ukuran Layar : 14\" HD | Processor : i3-1005G1" | string | | tanggal_tkdn | "\"2022-08-31T00:00:00\"" | date | | tanggal_expire_tkdn | "\"2025-08-31T00:00:00\"" | date | | link_p3dn | "http://tkdn.kemenperin.go.id//search.php?where=sertifikat&what=" | string (URL) | | nama_komoditas_kbki | "Laptop termasuk notebook dan subnotebook" | string | | created_date | "\"2020-10-11T00:00:00\"" | | | modified_date | "null" | date | | komoditas_id | 340 | string | | nama_komoditas | "Peralatan Elektronik Perkantoran dan Peralatan Pendukungnya" | string | | kelas_harga | "nasional" | string | | produk_kategori_id | "5590" | string | | nama_kategori | "Laptop/Notebook" | string | | manufaktur_id | "33228" | string | | penyedia_id | "420323" | string | | nama_penyedia | "PT Hewlett-Packard Indonesia" | string | By visual inspection, the detailed product data to be displayed in the e-Catalog app is a set of laptop computers. There are a total of 34 columns, and 1000 computer models are represented. The attribute name for each column is denoted in Indonesian language. The actual Excel data file is linked here. [[Laptops_Original.xlsx]] # Product Data converted to JSON format One data entry is shown in the following JSON format: ```JSON { "data": { "nama_produk": "HP 240 G7", "no_produk": "X4522100001-PEP-001442670Y", "harga_utama": 6849315, "harga_pemerintah": 6849315, "jumlah_stok": 0, "produk_gambar": "https://e-katalog.lkpp.go.id/katalog/produk/download/gambar/14032447", "berlaku_sampai": "\"2023-12-31T00:00:00\"", "nama_manufaktur": "hp inc", "no_produk_penyedia": "224U5PA", "uom": "Unit", "jenis_produk": "import", "kode_kbki": 4522100001, "tkdn_produk": null, "nilai_bmp": null, "sum_tkdn_bmp": 0, "nama_pemilik_sertifikat_tkdn": "PT. Armindo Intercorp ", "Kind_of_tkdn_product": null, "spesifikasi": "Koneksi : Realtek 802.11ac (2x2) Wi-Fi\u00c2\u00ae and Bluetooth\u00c2\u00ae 5 Combo | VGA : UMA | RAM : 4GB DDR4 | Warna : Dark ash silver | Storage : 256GB SSD M.2 | Deskripsi : CPU i3-1005G1 , Graphics UMA , RAM 4GB DDR4 , Storage 256GB SSD M.2 , ODD - , OS Win 10 Home , Display 14\" HD , Warranty ( NON ADP) 1/1/0 | Type/Model : HP 240 G7 | Ukuran Layar : 14\" HD | Processor : i3-1005G1", "tanggal_tkdn": "\"2022-08-31T00:00:00\"", "tanggal_expire_tkdn": "\"2025-08-31T00:00:00\"", "link_p3dn": "http://tkdn.kemenperin.go.id//search.php?where=sertifikat&what=", "nama_komoditas_kbki": "Laptop termasuk notebook dan subnotebook", "created_date": "\"2020-10-11T00:00:00\"", "modified_date": "null", "komoditas_id": "340", "nama_komoditas": "Peralatan Elektronik Perkantoran dan Peralatan Pendukungnya", "kelas_harga": "nasional", "produk_kategori_id": "5590", "nama_kategori": "Laptop/Notebook", "manufaktur_id": "33228", "penyedia_id": "420323", "nama_penyedia": "PT Hewlett-Packard Indonesia" } } ``` This data entry is automatically extracted from Excel and converted into JSON objects. Finding a way to automatically convert data content to be displayed in e-Catalog is the function of this system, the detailed mechanism is explained in [[e-Catalog User Interface Design]]. For the PoC, `no_produk`, which means: "product number", will be used as the primary key for this data table. ### Adopt a Data-Driven Approach for Information presentation A visual examination of a sample data point reveals the presence of the following data types: string, URL string, null string, integer number, floating point number, date, government code, and many others. These various data types can be automatically converted into machine representations for display in the e-Catalog user interface. Configuring how to present this data should be done in a data-driven manner, so that when a more effective data presentation mechanism becomes available, changes in source code will be avoided, and only relevant configuration files for display options will need to be updated. # User Data In this Proof of Concept program, the Users data model was created solely for demonstration purposes. The goal of creating a new data model is to take advantage of Ethereum's user authentication authority. Because this is an exploratory program, we must find a way to combine two types of authentication systems: one from the existing authentication services of LKPP or other Indonesian government agencies, and one from Ethereum as a public authentication service. As a result, we decided to use [[OpenZeppelin]]'s RBAC implementation to create a [[Role-based Access Control]] (RBAC) user data model. This data model is depicted in the [[e-Catalog Data Architecture]] section. It also refers to the design principle of [[Account abstraction]], which was first proposed by [[Vitalik Buterin]]. Keep user data separate from product data to maximize technology reuse and avoid implementing a system that has not been tested or critiqued by the general public. To maintain a data lineage trail between the Ethereum account system and the locally built user data set, all changes in the local user data set should be marked with an Ethereum-issued transaction ID and an Ethereum-based timestamp. # Software Data In Ethereum smart contract system, the contractual code themselves are considered to be a form of immutable data, and it is also being recorded on the Ethereum blockchain. They can also be seem directly on [[Etherscan]]. Moreover, the source code that works with the written smart contracts written in JavaScript and other supporting programming languages are stored on Github. The detail software data layout is explained in [[e-Catalog Version Control System]]. # Conclusion Finally, by incorporating Ethereum blockchain technology into the e-Catalog application, we can ensure the transparency, traceability, and accountability of all data. The public Ethereum-blockchain serves as the single source of truth, and data evidence provided by the blockchain can be incorporated into off-chain data content in the form of [[timestamp|timestamps]]", "[[authentication certificate|authentication certificates]]", and contract/transaction addresses. All of this data content can be traced back to the public blockchain on the [[Etherscan]] website. > [[e-Catalog Data Collection and Transformation|Next Section: Data Collection and Transformation]]