Open the announcement
NEW · Development continues on the new WeldExp Modules calculators: CIC, SFC, RSW and T85.
Zenit Programs

WeldExp database development

Full control over the data

Fast, controlled data upload using Codex, Excel templates, GitHub validations and a test database.

· Zenit Programs

HomeFull control over the data

Manual database loading is a time-consuming process that requires careful attention. For this reason, when loading the WeldExp database, data extraction, organisation, testing and verification are carried out as an interconnected workflow supported by artificial intelligence.

Current data-loading workflow

  1. Reading the data and verifying the source with the help of Codex.
  2. Creating a temporary data-collection Excel workbook from a template so that the data can be organised clearly.
  3. Preparing the SQL files and Markdown notes, then uploading them to the GitHub repository using MCP.
  4. Running the database-loading test and the required validations.

The test result determines the next step:

  • if a validation or data-structure error occurs, the process stops with an error and the correction is returned to the Codex-assisted preparation stage;
  • if the test succeeds, production data loading can begin after human review.

Creating the test database

When content is uploaded to GitHub, the test database can be created in two ways:

  1. With an offline host, from a seed package that is manually refreshed on GitHub and contains a database backup made up of CSV and SQL files. In this case, the tests are run by the Linux runners provided by GitHub.
  2. With an online host, from an automatically refreshed seed package using the API configured for the production database before every new data-loading task.

Automation and human control

Human review cannot be omitted; a clear Excel document is still required for this purpose. The aim of the workflow is fast and controlled data loading through the combined use of the trigger rules configured in the database, the schema structure and the validation procedures stored on GitHub.

The process applies the PDCA cycle known from ISO 9001 to machine-assisted data processing and control: plan, do, check and, where necessary, act.

Codex also performs separate forced verification steps to ensure that no data is omitted and that typographical errors caused by inaccurate OCR extraction or processing limitations are identified in time.

Screenshots of the workflow

The first two images show the work instruction recorded on GitHub for the development of the data-loading process.

First part of the WeldExp data-upload work instruction recorded on GitHub
1. Work instruction on GitHub for development of the upload process.
Second part of the WeldExp data-upload work instruction recorded on GitHub
2. Continuation on GitHub of the work instruction for development of the upload process.

The third image shows the validation and database test that run fully automatically during the GitHub upload.

Result of validation and database test run automatically on GitHub upload
3. Validation and database test run automatically on GitHub upload.

The fourth image shows test errors that could cause problems during loading into the production database.

Errors reported by the database test that prevent production upload
4. Errors reported by the database test that prevent production upload.