Skip to content

Update testing to allow same fw on several boards #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 12, 2016
Merged

Update testing to allow same fw on several boards #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions docs/PORT_BOARD.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,23 @@ const char *board_id = "0000";
Now running `progen generate -t uvision` will create project files including the new board that can be developed and debugged. For more information about the yaml format [see the project_generator documentation.](https://github.com/project-generator/project_generator/wiki/Getting_started)

### Adding Board to Automated Tests
Update `test/into.py`. There are various dictionaries that you will need to add to
* Add the name of the interface (as specified in the projects.yaml) to FIRMWARE_NAME_TO_BOARD_ID and associate it with the board ID
Update `test/into.py` so the new board has at least one configuration in SUPPORTED_CONFIGURATIONS
```python
SUPPORTED_CONFIGURATIONS = [
# Board ID Firmware Bootloader Target
...
( 0x240, 'k20dx_frdmk64f_if', 'k20dx_bl', 'FRDM-K64F' ),
...
]
```
Configration Fields
* Board ID - The ID assigned to the board type.
* Firmware - The name of the firmware as shown in projects.yaml.
* Bootloader - The name of the bootloader firmware as shown in projects.yaml. Only required on HICs with a DAPLink bootloader.
* Target - Name of the target for this board.
* If this is an mbed official board then the target should match the name in the mbed platform page URL. For example the K64F is located at https://developer.mbed.org/platforms/FRDM-K64F/ so it would have the target name `FRDM-K64F`. Using this naming convention allows the automated tests to use the RESTful Compile API. the automated tests will build the target UART application on-the-fly in the cloud using the RESTful Compile API, download it to the PC, then download the resulting image to the target.
* If it is not, you will need to build the UART application yourself and supply it to `test\run_tests.py` via --targetdir. In this case, the target is the application image filename sans extension.

* Add to TARGET_NAME_TO_BOARD_ID. What you use for the key will depend on whether your board is mbed enabled _and_ registered with mbed.org.
* If it is, the automated tests will build the target UART application on-the-fly in the cloud using the RESTful Compile API, download it to the PC, then download the resulting image to the target. (The source code for the application is in an mbed.org mercurial repository.) In this case, the dictionary key is the board name as registered in mbed.org.
* If it is not, you will need to build the UART application yourself and supply it to `test\run_tests.py` via --targetdir. In this case, the dictionary key is the application image filename sans extension.
* You may need to update one or more other dictionaries. See comments in the code for guidance.
You may need to update one or more other dictionaries. See comments in the code for guidance.

See [Automated Tests](AUTOMATED_TESTS.md) for more information related to automated testing.
24 changes: 6 additions & 18 deletions test/daplink_firmware.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,14 +144,11 @@ def __init__(self, name, bundle, directory):
'be added to HIC_STRING_TO_ID in info.py' % string_hic
self._hic_id = info.HIC_STRING_TO_ID[string_hic]

# Set board ID
self._board_id = None
if name in info.FIRMWARE_NAME_TO_BOARD_ID:
self._board_id = info.FIRMWARE_NAME_TO_BOARD_ID[name]
else:
assert self._type is not self.TYPE.INTERFACE, 'Unknown board ' \
'"%s" must be added to FIRMWARE_NAME_TO_BOARD_ID in ' \
'info.py' % name
# Check firmware name and type
assert self._type in self.TYPE, "Invalid type %s" % self._type
if self._type is self.TYPE.INTERFACE:
assert name in info.FIRMWARE_SET, 'Unknown board "%s" must be ' \
'added to SUPPORTED_CONFIGURATIONS in info.py' % name

# Set file paths
self._bin_path = self._directory + os.sep + '%s_crc.bin' % name
Expand All @@ -170,12 +167,7 @@ def __init__(self, name, bundle, directory):
self._valid = True

def __str__(self):
board_id = self.board_id
if board_id is None:
board_id = 0
return "Name=%s Board ID=0x%04x HIC ID=0x%08x" % (self.name,
board_id,
self.hic_id)
return "Name=%s" % (self.name)

@property
def valid(self):
Expand All @@ -190,10 +182,6 @@ def name(self):
def hic_id(self):
return self._hic_id

@property
def board_id(self):
return self._board_id

@property
def type(self):
return self._type
Expand Down
5 changes: 0 additions & 5 deletions test/firmware.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,6 @@ def hic_id(self):
"""HIC ID for the type of board this firmware can run on"""
raise NotImplementedError()

@property
def board_id(self):
"""Board ID for the type of board this firmware can run on"""
raise NotImplementedError()

@property
def type(self):
"""Build type - either interface or bootloader"""
Expand Down
131 changes: 46 additions & 85 deletions test/info.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,90 +26,49 @@
'sam3u2c': 0x97969903,
}

# Add new firmware with a hard coded board ID here
FIRMWARE_NAME_TO_BOARD_ID = {
'k20dx_frdmk22f_if': 0x0231,
'k20dx_frdmk64f_if': 0x0240,
'kl26z_microbit_if': 0x9900,
'kl26z_nrf51822_if': 0x9900,
'lpc11u35_lpc812xpresso_if': 0x1050,
'lpc11u35_ssci1114_if': 0x1114,
'lpc11u35_efm32gg_stk_if': 0x2015,
'sam3u2c_nrf51dk_if': 0x1100,
'k20dx_frdmk20dx_if': 0x0230,
'k20dx_frdmkw24f_if': 0x0280,
'k20dx_frdmkl02z_if': 0x0250,
'k20dx_frdmkl05z_if': 0x0210,
'k20dx_frdmkl25z_if': 0x0200,
'k20dx_frdmkl26z_if': 0x0260,
'k20dx_frdmkl46z_if': 0x0220,
'sam3u2c_nrf51mkit_if': 0x1070,
'sam3u2c_nrf51dongle_if': 0x1120,
'lpc11u35_archble_if': 0x9009,
'lpc11u35_archpro_if': 0x9004,
'lpc11u35_archmax_if': 0x9011,
'lpc11u35_hrm1017_if': 0x1017,
'lpc11u35_sscity_if': 0x1019,
'lpc11u35_ssci824_if': 0x1018,
'lpc11u35_lpc824xpresso_if': 0x0824,
'lpc11u35_lpc4088dm_if': 0x1062,
'lpc11u35_lpc4088qsb_if': 0x1060,
'k20dx_rbl_if': 0x1090,
'k20dx_rblnano_if': 0x1095,
'lpc11u35_tiny_if': 0x9012,
'lpc11u35_c027_if': 0x1234,
'lpc11u35_arm_watch_stm32f411_if': 0x5050,
'lpc11u35_arm_watch_efm32_if': 0x5051,
'lpc11u35_arm_watch_nrf51_if': 0x5052,
'lpc11u35_archlink_if': 0x9013,
}

# Maps the target test application to the ID of the board it is built to run
# on.
#
# If the application is built on the fly using the RESTful Compile API, the
# map key must correspond to one of the boards on
# https://developer.mbed.org/platforms/{board name here}
#
# If the application is pre-built and supplied via --targetdir, the map
# key is the target application filename sans extension. Note that the key
# is also used as the board name in the test output. Thus, name the target
# application after the board to avoid confusing results.
#
TARGET_NAME_TO_BOARD_ID = {
'FRDM-K22F': 0x0231,
'NXP-LPC800-MAX': 0x1050,
'FRDM-K64F': 0x0240,
'Microbit': 0x9900,
'Nordic-nRF51-DK': 0x1100,
'FRDM-K20D50M': 0x0230,
#k20dx_frdmk24f_if
#k20dx_frdmkl02z_if
'FRDM-KL05Z': 0x0210,
'KL25Z': 0x0200,
#k20dx_frdmkl26z_if
'FRDM-KL46Z': 0x0220,
'Nordic-nRF51822': 0x1070,
'Nordic-nRF51-Dongle': 0x1120,
'Seeed-Arch-BLE': 0x9009,
'Seeeduino-Arch-Pro': 0x9004,
'Seeed-Arch-Max': 0x9011,
'mbed-HRM1017': 0x1017,
'Switch-Science-mbed-TY51822r3': 0x1019,
'Switch-Science-mbed-LPC824': 0x1018,
'LPC1114FN28': 0x1114,
'LPCXpresso824-MAX': 0x0824,
'EA-LPC4088-Display-Module': 0x1062,
'EA-LPC4088': 0x1060,
'RedBearLab-nRF51822': 0x1090,
'RedBearLab-BLE-Nano': 0x1095,
'Seeed-Tiny-BLE': 0x9012,
'u-blox-C027': 0x1234,
#lpc11u35_arm_watch_stm32f411_if
#lpc11u35_arm_watch_efm32_if
#lpc11u35_arm_watch_nrf51_if
'Seeed-Arch-Link': 0x9013,
}
SUPPORTED_CONFIGURATIONS = [
# Board ID Firmware Bootloader Target
( 0x200, 'k20dx_frdmkl25z_if', 'k20dx_bl', 'KL25Z' ),
( 0x210, 'k20dx_frdmkl05z_if', 'k20dx_bl', 'FRDM-KL05Z' ),
( 0x220, 'k20dx_frdmkl46z_if', 'k20dx_bl', 'FRDM-KL46Z' ),
( 0x230, 'k20dx_frdmk20dx_if', 'k20dx_bl', 'FRDM-K20D50M' ),
( 0x231, 'k20dx_frdmk22f_if', 'k20dx_bl', 'FRDM-K22F' ),
( 0x240, 'k20dx_frdmk64f_if', 'k20dx_bl', 'FRDM-K64F' ),
( 0x250, 'k20dx_frdmkl02z_if', 'k20dx_bl', None ),
( 0x260, 'k20dx_frdmkl26z_if', 'k20dx_bl', None ),
( 0x280, 'k20dx_frdmkw24f_if', 'k20dx_bl', None ),
( 0x824, 'lpc11u35_lpc824xpresso_if', None, 'LPCXpresso824-MAX' ),
( 0x1017, 'lpc11u35_hrm1017_if', None, 'mbed-HRM1017' ),
( 0x1018, 'lpc11u35_ssci824_if', None, 'Switch-Science-mbed-LPC824' ),
( 0x1019, 'lpc11u35_sscity_if', None, 'Switch-Science-mbed-TY51822r3' ),
( 0x1050, 'lpc11u35_lpc812xpresso_if', None, 'NXP-LPC800-MAX' ),
( 0x1060, 'lpc11u35_lpc4088qsb_if', None, 'EA-LPC4088' ),
( 0x1062, 'lpc11u35_lpc4088dm_if', None, 'EA-LPC4088-Display-Module' ),
( 0x1070, 'sam3u2c_nrf51mkit_if', 'sam3u2c_bl', 'Nordic-nRF51822' ),
( 0x1090, 'k20dx_rbl_if', 'k20dx_bl', 'RedBearLab-nRF51822' ),
( 0x1095, 'k20dx_rblnano_if', 'k20dx_bl', 'RedBearLab-BLE-Nano' ),
( 0x1100, 'sam3u2c_nrf51dk_if', 'sam3u2c_bl', 'Nordic-nRF51-DK' ),
( 0x1114, 'lpc11u35_ssci1114_if', None, 'LPC1114FN28' ),
( 0x1120, 'sam3u2c_nrf51dongle_if', 'sam3u2c_bl', 'Nordic-nRF51-Dongle' ),
( 0x1234, 'lpc11u35_c027_if', None, 'u-blox-C027' ),
( 0x2015, 'lpc11u35_efm32gg_stk_if', None, None ),
( 0x5050, 'lpc11u35_arm_watch_stm32f411_if', None, None ),
( 0x5051, 'lpc11u35_arm_watch_efm32_if', None, None ),
( 0x5052, 'lpc11u35_arm_watch_nrf51_if', None, None ),
( 0x9004, 'lpc11u35_archpro_if', None, 'Seeeduino-Arch-Pro' ),
( 0x9009, 'lpc11u35_archble_if', None, 'Seeed-Arch-BLE' ),
( 0x9011, 'lpc11u35_archmax_if', None, 'Seeed-Arch-Max' ),
( 0x9012, 'lpc11u35_tiny_if', None, 'Seeed-Tiny-BLE' ),
( 0x9013, 'lpc11u35_archlink_if', None, 'Seeed-Arch-Link' ),
( 0x9900, 'kl26z_microbit_if', 'kl26z_bl', 'Microbit' ),
]

BOARD_ID_TO_BUILD_TARGET = {config[0]: config[3] for config in
SUPPORTED_CONFIGURATIONS}
FIRMWARE_SET = set((config[1] for config in SUPPORTED_CONFIGURATIONS))
TARGET_SET = set((target[3] for target in SUPPORTED_CONFIGURATIONS if
target[3] is not None))

BOARD_ID_LOCKED_WHEN_ERASED = set([
0x0231, # K22F
Expand All @@ -120,6 +79,10 @@
0x0210, # KL05Z
])

TARGET_WITH_COMPILE_API_LIST = [config[3] for config in
SUPPORTED_CONFIGURATIONS if
config[3] is not None]

#Hack until these targets have an image with a valid vector table
TARGET_WITH_BAD_VECTOR_TABLE_LIST = [
'Microbit',
Expand All @@ -134,5 +97,3 @@
'Seeed-Tiny-BLE',
'Seeed-Arch-Link',
]

BOARD_ID_TO_BUILD_TARGET = {v: k for k, v in TARGET_NAME_TO_BOARD_ID.items()}
101 changes: 43 additions & 58 deletions test/run_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
from firmware import Firmware
from target import load_target_bundle, build_target_bundle
from test_daplink import daplink_test
import info

DEFAULT_TEST_DIR = './test_results'

Expand Down Expand Up @@ -376,13 +377,6 @@ def build_test_configurations(self, parent_test):
continue
board_list.append(board)

# Create a table mapping each board id to a target
board_id_to_target = {}
for target in self._target_list:
assert target.board_id not in board_id_to_target, 'Multiple ' \
'targets found for board id "%s"' % target.board_id
board_id_to_target[target.board_id] = target

# Create a list for bootloader firmware and interface firmware
bootloader_firmware_list = []
filtered_interface_firmware_list = []
Expand All @@ -397,77 +391,68 @@ def build_test_configurations(self, parent_test):
else:
assert False, 'Unsupported firmware type "%s"' % firmware.type

# Create a table mapping name to object with that name
TARGET_NAME_TO_TARGET = {target.name: target for target in
self._target_list}
FIRMWARE_NAME_TO_FIRMWARE = {firmware.name: firmware for firmware in
filtered_interface_firmware_list}
BL_NAME_TO_BL = {firmware.name: firmware for firmware in
bootloader_firmware_list}

# Explicitly specified boards must be present
fw_name_set = set(fw.name for fw in filtered_interface_firmware_list)
if self._firmware_filter is not None:
assert self._firmware_filter == fw_name_set

# Create a table mapping each hic to a bootloader
hic_id_to_bootloader = {}
for firmware in bootloader_firmware_list:
hic_id = firmware.hic_id
assert hic_id not in hic_id_to_bootloader, 'Duplicate ' \
'bootloaders for HIC "%s" not allowed'
hic_id_to_bootloader[hic_id] = firmware

# Create a test configuration for each interface and supported board
# combination
# Create test configurations for each supported configuration
test_conf_list = []
self._untested_firmware = []
for firmware in filtered_interface_firmware_list:
board_id = firmware.board_id
hic_id = firmware.hic_id
bl_firmware = None
untested_firmware = set(filtered_interface_firmware_list)
for board_id, fw_name, bl_fw_name, target_name in info.SUPPORTED_CONFIGURATIONS:
target = None
if_firmware = None
bl_firmware = None
if target_name in TARGET_NAME_TO_TARGET:
target = TARGET_NAME_TO_TARGET[target_name]
if fw_name in FIRMWARE_NAME_TO_FIRMWARE:
if_firmware = FIRMWARE_NAME_TO_FIRMWARE[fw_name]
if bl_fw_name in BL_NAME_TO_BL:
bl_firmware = BL_NAME_TO_BL[bl_fw_name]

target_required = self._test_ep
bl_required = self._load_bl or self._test_daplink
if if_firmware is None:
# Skip configuration
continue
if target_required and target is None:
# Skip configuration
test_info.info('No target to test firmware %s' % fw_name)
continue
if bl_required and bl_firmware is None:
# Skip configuration
test_info.info('No bootloader to test firmware %s' % fw_name)
continue
# Check if there is a board to test this firmware
# and if not skip it
if board_id not in board_id_to_board_list:
self._untested_firmware.append(firmware)
test_info.info('No board to test firmware %s' % firmware.name)
continue

# Get target
target = None
target_required = self._test_ep
if board_id in board_id_to_target:
target = board_id_to_target[board_id]
elif target_required:
self._untested_firmware.append(firmware)
test_info.info('No target to test firmware %s' %
firmware.name)
continue

# Check for a bootloader
bl_required = self._load_bl or self._test_daplink
if hic_id in hic_id_to_bootloader:
bl_firmware = hic_id_to_bootloader[hic_id]
elif bl_required:
self._untested_firmware.append(firmware)
test_info.info('No bootloader to test firmware %s' %
firmware.name)
test_info.info('No board to test firmware %s' % fw_name)
continue

# Create a test configuration for each board
board_list = board_id_to_board_list[board_id]
for board in board_list:
if firmware.hic_id != board.hic_id:
test_info.warning('FW HIC ID %s != Board HIC ID %s' %
(firmware.hic_id, board.hic_id))
if bl_firmware is not None:
if firmware.hic_id != bl_firmware.hic_id:
test_info.warning('FW HIC ID %s != BL HIC ID %s' %
(firmware.hic_id,
bl_firmware.hic_id))
if target is not None:
assert firmware.board_id == target.board_id

test_conf = TestConfiguration(firmware.name + ' ' + board.name)
test_conf.if_firmware = firmware
test_conf = TestConfiguration(if_firmware.name + ' ' +
board.name)
test_conf.if_firmware = if_firmware
test_conf.bl_firmware = bl_firmware
test_conf.board = board
test_conf.target = target
test_conf_list.append(test_conf)
# remove this from the untested list
if if_firmware in untested_firmware:
untested_firmware.remove(if_firmware)
assert bl_firmware not in untested_firmware

self._untested_firmware = list(untested_firmware)
self._test_configuration_list = test_conf_list


Expand Down
Loading