Build Infrastructure Proposals
Proposals for build system improvements, please comment on individual idea on mailing list, or add suggestions here.
DONE Get Jenkins builders off of Ubuntu 12.04. 12.04LTS is now out of support and has us pegged to some very old versions of needed packages (like scons, and the coverage report generator gcovr). It also gives us some false success - there is at least one unit test (stack/tests/stacktests) that happens to pass on this old version, but all more recent Linux systems time out the test (deadman timer exception)
Use a consistent scons version (see also previous item). This is mostly for Linux: other platforms don't have a “distro version” of scons but rather install it via pip during provisioning. Note the recent release of 3.0 higlighted that the problem is not only distro-version too old, it can also be upstream (pip) version is too new. For linux, install it directly, perhaps in a virtualenv, and don't use distro version? IOT-2755
DONE Install the Gerrit trivial-rebase functionality. This would let patchsets which don't change code (e.g. rebase, comment change) collect and retain votes and comments made against the previous version.
Update out of date build tools appropriately. In particular, the Android SDK and NDK are quite out of date, but an uplift has a number of moving parts - in the code tree it needs to be tested carefully before the builder can be adjusted.
Unit-test data collection:
Insufficient collection of problems: right now it looks like if a test dies with a core dump, that's not recorded in the xml files, and the Jenkins publisher doesn't see them, so you get cute things like the overview page saying no failures, while the build log has very noisy statements about a failure. IOT-2510
The log collection logic (“publisher”) is at the end of this page.
unit test results from other builders than the one named unit-tests are not collated overall. Windows and Tizen also run some unit tests. Is this a problem? These other platforms may have their own logic for deciding unit-test failure.
auto_build.py unit_tests performs back to back non-secure and secure tests. the generated logfile names are the same, so the publisher collects only the secure set if both finish running - the non-secure ones are overwritten. That may not normally be a problem, but there could be some differences.
Support selective unit testing (a Jira ticket is about to be filed on this). Affects developers, not Jenkins use.
Make whether or not to run tests under valgrid selectable (currently the logic is “if linux, run under valgrind”). For example, some developers were building iotivity on Raspberry Pi running Raspbian, which passes the “is it Linux” test, but the platform is way too small to support the valgrind runs.
Jenins xunit Publisher
194 - publisher:
195 name: xunit-report
197 - xunit:
198 thresholdmode: '1'
199 test-time-margin: 3000
201 - gtest:
202 pattern: 'out/**/test_out/*.xml'
203 skip-if-no-test-files: true
204 fail-if-not-new: false
205 deleteoutput: true
206 stoponerror: false