Generalizing production testing operations for IoT devices
Korhonen, Kimmo (2021-12-16)
Korhonen, Kimmo
K. Korhonen
16.12.2021
© 2021 Kimmo Korhonen. Ellei toisin mainita, uudelleenkäyttö on sallittu Creative Commons Attribution 4.0 International (CC-BY 4.0) -lisenssillä (https://creativecommons.org/licenses/by/4.0/). Uudelleenkäyttö on sallittua edellyttäen, että lähde mainitaan asianmukaisesti ja mahdolliset muutokset merkitään. Sellaisten osien käyttö tai jäljentäminen, jotka eivät ole tekijän tai tekijöiden omaisuutta, saattaa edellyttää lupaa suoraan asianomaisilta oikeudenhaltijoilta.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:oulu-202112179403
https://urn.fi/URN:NBN:fi:oulu-202112179403
Tiivistelmä
A rapidly increasing number of new IoT products entering the market puts strain on the testing effort required to manufacture them. Every device needs to be tested at the manufacturing site before it can be shipped to the customer. This testing process during manufacturing is called Production Testing. If test automation systems running these tests are developed with a single system engineering approach, the number of test automation systems becomes unmaintainable. In addition to this, the development cost of such test automation system has to be covered by each product. Reusability of test automation and test assets is low when resources cannot be efficiently shared between products.
Existing solutions for generalizing testing effort from a single system approach to support multiple products were reviewed from the literature. Software Product Line Engineering is recognized as a possible solution, but its adoption requires organizational, economical, and technical changes. This thesis studied technical solutions for how test automation system could be developed to support the testing of multiple products.
Test automation system was designed based on existing literature, and two example products were used to mimic realistic IoT products. Work followed to define test requirements for two example products, implement tests, and execute them for the first example product. After tests passed for the first example product, they were executed for the second product. Test failures and evident problems were marked as variation points, and they were analysed. Test attributes that needed to be varied were recognized, and four different sources for that information were identified. Test logic was identified as one of the sources for attributes, and there was no need to variate it. Matching configuration was created for other sources: component, test, and hardware configuration. Tests were successfully executed for the second example product after introducing the variation via configuration files.
Prototype implementation succeeded in its goal to create production test automation system capable of testing two different example products using common test assets. Needed variation was introduced successfully through configuration files. This thesis shows that general test assets can be created for production testing, despite the fact that production testing is tightly coupled to the target hardware.
Future work continues by testing additional hardware platforms to reveal more variation points. This helps to develop production testing test automation to support a wider range of hardware platforms and components. Storing the hardware-specific configuration data to the device looks promising topic for further study.
Existing solutions for generalizing testing effort from a single system approach to support multiple products were reviewed from the literature. Software Product Line Engineering is recognized as a possible solution, but its adoption requires organizational, economical, and technical changes. This thesis studied technical solutions for how test automation system could be developed to support the testing of multiple products.
Test automation system was designed based on existing literature, and two example products were used to mimic realistic IoT products. Work followed to define test requirements for two example products, implement tests, and execute them for the first example product. After tests passed for the first example product, they were executed for the second product. Test failures and evident problems were marked as variation points, and they were analysed. Test attributes that needed to be varied were recognized, and four different sources for that information were identified. Test logic was identified as one of the sources for attributes, and there was no need to variate it. Matching configuration was created for other sources: component, test, and hardware configuration. Tests were successfully executed for the second example product after introducing the variation via configuration files.
Prototype implementation succeeded in its goal to create production test automation system capable of testing two different example products using common test assets. Needed variation was introduced successfully through configuration files. This thesis shows that general test assets can be created for production testing, despite the fact that production testing is tightly coupled to the target hardware.
Future work continues by testing additional hardware platforms to reveal more variation points. This helps to develop production testing test automation to support a wider range of hardware platforms and components. Storing the hardware-specific configuration data to the device looks promising topic for further study.
Kokoelmat
- Avoin saatavuus [31657]