Python +OpenCV

---

شرح پروژه

Request details

Goal

Reproduce an existing web form using OpenCV, Python and Javascript

Problem

We have a large number of forms which we need to migrate to a modern UI and UX consistent standard. The forms we intend to reproduce are built with different frameworks, situated inside different applications, have different visual styles and were built at different points in time. For many of these forms, we do not have access to source code (except the generated “View Source"). So in terms of appearance and code, there is substantial heterogeneity between the forms that need to be reproduced.

Solution approach

Step 1- Train OpenCV to recognize form fields, field-groups, and field interactions

Train OpenCV to recognize

Form fields and elements (Text box, Text area, Slider, Drop-down, Multi-select, Radio buttons, Check-boxes, Calendar Controls, etc.)

Field Groups - Where form fields are grouped for logical purposes and for presentation in particular order, form fields inside containers / tab-strip controls

Interactions - Repeater / Remover (Click + to add a row, Click - to remove a row of form elements, etc.), Hide / Show controls (open / close)

Form values (possibly can be extracted from code or using OpenCV)

Mandatory flags (Either text stating “All fields are mandatory”, or Asterisk next to a field label)

The trained model must be able to recognize the type of field regardless of how visually different it looks, or is styled, compared to another field of a similar type. Two text boxes might look very different but the model must be determine that both of them are text-boxes and belong to the same field-type.

Also, the trained model must be able to recognize when fields are “grouped”.

Step 2 - Draw selection area (DOM elements or area where form elements are present in a form)

The user should be able to use their mouse to select the area in the web application where the form fields are present.

Step 3 - Interact with the form programmatically to ensure form elements are visible for OpenCV to recognize

When the user selects a form on screen, execute two functions using JS / headless browser:

Set focus to a form field and exit focus without inputting anything. This will trigger alert messages if any to popup for required / mandatory fields. (This function may not be needed if Mandatory field determination is done using OpenCV.)

Make a random selection / input a random value in each field. This will trigger the validation alert message. This message should be captured using Python code / OpenCV.

Step 4 - Extract field attributes from code

Extract relevant information from the code (such as the URL, HTTP Post URI, Whether field is required or not, JS event (onBlur, onChange, onInput, etc.) attached to a field, server-side function attached to the field (Post / GET), field name, field ID, values in array, etc.

Step 5 - Load all data into meta.json and ui.json

The information extracted using OpenCV and the source code must be populated in two React-supported json files. One json is for the meta information (meta.json) and the other is for the UI schema (ui.json) in the form. (Sample formats of the json files will be shared.)

Using a combination of these two json files (meta.json and ui.json), the form must be rendered in the browser.

The final form will look different, but have the same sequence of form fields, grouping of form fields, JS events / functions attached to fields, mandatory conditions, alert messages, POST / GET URIs, array values.

Not needed

All CSS in the original form will be discarded and the generated form will have standard CSS and JS. Javascript functions if any will not be needed, except where the form has interaction (Hide / Show fields, Add / Delete Row, etc.), which can be captured by OpenCV.

What we will share

We will share

a sample list of forms which can be used for training OpenCV to recognize the form fields

sample meta.json and ui.json

مهارت ها و تخصص های مورد نیاز

---

---