User Tools

Site Tools


iotivity_protocol_conformance_test_guide

CTT Feature Selection

Select Network Interface

  • Choose OptionNetwork Interface menu. A window will appear (like below image).
  • Select Network Interface & IP. Press OK button.

Select DUT

  • Choose FileSelect DUT menu. A window will appear (like below image).
  • Device list will appear in left side. Select the target device. Press next button.

  • Select DUT's Interface and Testing Profile. Press next button.

  • Choose PICS configuration file for target device.

CTT Test Run

Go to Test Run menu . From that Test Run option can be selected to run test case.

Tutorial

Building IUTSimulator

Clone the IoTivity Code
Checkout Test branch
Run the command scons MODULE=iut WITH_RD=1 (support with RD)
The IUTSimulator Application should be available in /bin/linux folder

Run Server Tests

Pre-Condition

  • Start IUT Simulator in the desired server mode
    1. ./IUTSimulator 1 4 21 (justworks server) or ./IUTSimulator 1 4 22 (randompin server) - IPV4
    2. ./IUTSimulator 1 6 21 (justworks server) or ./IUTSimulator 1 6 22 (randompin server) - IPV6
    3. Select menu 104
    4. Select menu 6 (if collection is required to test)
  • From CTT, Choose FileSelect DUT menu.
  • Device list will appear in left side. Select the target device. Press Next button.
  • Select DUT's Interface.
  • Under OCF Server Type, select All
  • Under Testing Profile, select OCF 1.3 Server
  • Press Next button.
  • Browse & select the PICS file PICS_Server.json
  • Press Next button.

Note: Unless until the PICS file not supports the testcase, Choose all the test cases to be run for the profile chosen. For Example: For Introspection test cases to be run (1.3.2-1 and 1.3.2-2 ), you may need to generate the Single resource JSON profile or need to modify the IUTSimulator/CTT code to support both using single PICS file. Above conditions were written based on 1.3.1 IoTivity and CTT 2.0.0 Version. Please read CTT release notes time to time for any update/change from above precondition/steps. PICS file has to be generated using CMS tool and follow the guide from Comarch and get access to generate the PICS of your own for Server, Client and WES.

General condition for all sections:

  • When CTT prompts “Please manually perform a factory reset on the IUT and confirm when complete”, Just press enter and continue.
  • During WES test Cases, Factory reset means, you need to reset the IUT and make sure its boot with its SoftAP.
  • If CTT prompts “Please initiate device to revert to 'ready for OTM' state”, Reset the IUT and Continue. (Give the same command used earlier bring-up IUT should be followed to reset)

CT1.1.1 to CT1.2.3

  • Select desired TCs from Test(s) List
  • Click on Test RunRun All Test Cases
  • When CTT prompts to power cycle On:
    • At IUTSimulator, press 0 & Enter to close it
    • Start IUTSimulator again, using +20 for Security argument
      • e.g. if IUT Simulator was running as justworks server(./IUTSimulator 1 6 21), then now run it using ./IUTSimulator 1 6 41
      • Start necessary resources
    • At CTT side, press Ok

CT1.2.6

  • From CTT, Select CT1.2.6 from Test(s) List
  • Click on Test RunRun All Test Cases
  • After a while, CTT will ask to update representation of a resource.
  • At the IUTSimulator side, select menu 34
  • From CTT side, check the resource URI to update and select the matching resource no. from IUTSimulator (e.g. for /AirFlowResURI, select 3)
  • Please follow the next steps depending on which resource was selected on the previous step
    • If selected resource URI contains /BinarySwithResURI, then type value & press enter, then select menu option 4 & then type 1 (or 0, alternatively)
    • If selected resource URI contains /TemperatureResURI, then type temperature & press enter, then select menu option 3 & then type 22.22 (or 33.33, alternatively)
    • If selected resource URI contains /AirFlowResURI, then type speed & press enter, then select menu option 1 & then type 20 (or 30, alternatively)
    • If selected resource URI contains /Vendor/AirConditioner/Swinger, then type x.com.vendor.swing.on & press enter, then select menu option 4 & then type 1 (or 0, alternatively)
    • If selected resource URI contains /Vendor/AirConditioner/TimerClock, then type x.com.vendor.timer.hour & press enter, then select menu option 1 & then type 10 (or 5, alternatively)
  • Repeat the above 2 steps for all the resource CTT prompts to update

CT1.7.2.2

  • From IUT Simulator side, start it as randompin server
  • From CTT, Select CT1.7.2.2 from Test(s) List
  • Click on Test RunRun All Test Cases
  • When CTT prompts to input random pin, get the value from IUTSimulator window and type it on CTT popup

CT1.7.2.1 to CT1.7.10.5 (Except CT1.7.2.2)

  • Select desired TCs from Test(s) List
  • Click on Test RunRun All Test Cases
  • When CTT prompts to power cycle On, press Ok

Run Server Tests (MFG)

Pre-Condition

  • Replace the root server certificate with the CTT root certificate:
    1. open oic_svr_db_server_mfg.json
    2. replace mfgtrustca to CTTtrustca (save mfgtrustca for entering to CTT)
    3. run: ./json2cbor oic_svr_db_server_mfg.json oic_svr_db_server_mfg.dat
  • Start IUT Simulator in the desired server mode
    1. ./IUTSimulator 1 4 23 (mfg server)- IPV4
    2. ./IUTSimulator 1 6 23 (mfg server)- IPV6
  • From CTT, Choose File → Select DUT menu.
  • Device list will appear in left side. Select the target device. Press Next button.
  • Select DUT's Interface.
  • Under OCF Server Type, select All
  • Under Testing Profile, select OCF 1.3 Server
  • Press Next button.
  • Browse & select the PICS file PICS_server_OCF10.json
  • Press Next button.

CT1.1.1 to CT1.2.3, CT1.2.6, CT1.7.2.2 tests run similarly to Server Tests

CT1.7.2.3-2

  • From CTT, Select CT1.7.2.3-2 from Test(s) List
  • Click on Test Run → Run All Test Cases
  • Enter the server root certificate in the dialog box in PEM Format (if the certificate have DER format please convert it into the PEM format)
  • (WARNING! When entering a certificate in CTT, make sure that it does not have the special character: “\r”)

Run Client Tests

Pre-Condition

  • From CTT, Choose FileSelect DUT menu.
  • Start IUT Simulator in the desired client mode
    1. ./IUTSimulator 1 4 11 (justworks client) or ./IUTSimulator 1 4 12 (randompin client)
    2. Select menu 12
  • On CTT, in left side, Client will appear under OCF Client Devices. Select the target client device. Press Next button.
  • Select DUT's Interface.
  • Under OCF Server Type, select All
  • Under Testing Profile, select OCF 1.3 Client [default]
  • Press Next button.
  • Browse & select the PICS file PICS_Client.json
  • Press Next button.

Note: Choose all the test cases for the client profile and run it in one shot. Introspection testcases not having dependencies to generate a separate single resource PICS or need a modification in the code until 1.3.1 IoTivity code and 2.0.0 CTT version. Please read CTT release notes for any changes/Update from the above mentioned step. Above steps are based on the 1.3.1 IoTivity and 2.0.0 CTT version.

CT2.1.1

  • From CTT, Select TC CT2.1.1 from Test(s) List
  • Click on Test RunRun All Test Cases
  • After a while, CTT will ask to send multicast discovery.
  • At the IUTSimulator side, select menu 12
  • On CTT side, a pop up will ask whether all resources are discovered or not, click on Yes

CT2.2.2

  • From CTT, Select TC CT2.2.2 from Test(s) List
  • Click on Test RunRun All Test Cases
  • After a while, CTT will ask to send multicast discovery and also send Unicast GET to a particular resource
  • At the IUTSimulator side, select menu 12
  • At the IUTSimulator side, select menu 17, and from the Discovered resource list, select the resource no. that was required by CTT
  • On CTT side, whenever a pop up shows up for confirmation, click on Yes
  • Repeat previous 2 step for all the resource asked by CTT

CT2.2.3

  • From CTT, Select TC CT2.2.3 from Test(s) List
  • Click on Test RunRun All Test Cases
  • After a while, CTT will ask to send multicast discovery and also send Unicast POST to a particular resource
  • At the IUTSimulator side, select menu 12
  • At the IUTSimulator side, select menu 22, and from the Discovered resource list, select the resource no. that was required by CTT
  • Please follow the next steps depending on which resource was selected on the previous step
    • If selected resource URI contains /BinarySwithResURI, then type value & press enter, then select menu option 4 & then type 1 (or 0, alternatively)
    • If selected resource URI contains /TemperatureResURI, then type temperature & press enter, then select menu option 3 & then type 22.22 (or 33.33, alternatively)
  • On CTT side, whenever a pop up shows up for confirmation, click on Yes
  • Repeat previous 2 step for all the resource asked by CTT

CT2.2.6

  • From CTT, Select TC CT2.2.6 from Test(s) List
  • Click on Test RunRun All Test Cases
  • After a while, CTT will ask to send multicast discovery and also send Unicast Observe to any resource
  • At the IUTSimulator side, select menu 12
  • At the IUTSimulator side, select menu 25, and from the Discovered resource list, select the resource no. for /TemperatureResURI
  • CTT will send notification to client, and ask if DUT has received it or not.
  • If the notification is received on IUTSimulator, click Yes on CTT
  • CTT will ask to cancel observing the resource
  • At the IUTSimulator side, select menu 26, and from the Discovered resource list, select the resource no. for /TemperatureResURI
  • On CTT side, if any pop up shows up for confirmation, click on Yes

Run WES(WiFi Easy Setup) Tests

The Device running IUTSimulator should have SoftAP running so that the WES onboarding can happen. Refer Ubuntu or respective OS support website to configure the SoftAP using hostapd or using any other tool. SoftAP setup configuration is out of scope of this topic.

Pre-Condition

  • Ensure CTT and IUT are in same network initially for the CTT to discover IUT
  • From CTT, Choose FileSelect DUT menu.
  • Start IUT Simulator in the desired server mode
    1. ./IUTSimulator 1 4 21 (justworks server) or ./IUTSimulator 1 4 22 (randompin server) - IPv4
    2. ./IUTSimulator 1 6 21 (justworks server) or ./IUTSimulator 1 6 22 (randompin server) - IPv6
    3. Select menu 109
  • On CTT, in left side, Server will appear under OCF Client Devices. Select the target client device. Press Next button.
  • Select DUT's Interface.
  • Under OCF Server Type, select All
  • Under Testing Profile, select OCF 1.3 Server
  • Press Next button.
  • Browse & select the PICS file PICS_WES.json
  • Press Next button.

Note: PICS has to be generated through CMS utility. Please refer the CTT package for understanding of the PICS fields and its arguments. You may require permission from OCF authority to Generate the PICS file. Else You can modify the existing Sample PICS given in CTT package. Recommendation is to use the CMS tool to generate PICS

CT1.1.1 to CT1.3.1

Procedure is same as for Server and Client test execution explained in above sections. Only difference is, initially CTT asks to connect to initiate factory of IUT. At this point reset the IUT and boot IUT with its SoftAP & provide the login details in CTT when it prompts. If you are already connected in SoftAP, press enter when CTT prompts for Factory Reset.

CT1.3.3-1

When CTT prompts to do an Factory Reset in IUT, Rest the IUT to connect in SoftAP mode and press Enter. Enter IUT SoftAP credentials, when CTT prompts for it. And follow the instruction there after.

CT1.3.3-2

When CTT prompts to do an Factory Reset in IUT, Rest the IUT to connect in SoftAP mode and press Enter. Enter IUT SoftAP credentials, when CTT prompts for it. And follow the instruction there after. As part of the test case, CTT waits for the SoftAP to drop-off. Ensure that CTT is able to join another network to avoid test case failure due to no network availability after SoftAP drop-off. Once SoftAP is not visible, CTT prompts to do an Factory reset and start on-boarding the IUT & continue there after.

iotivity_protocol_conformance_test_guide.txt · Last modified: 2018/02/08 11:18 by Sankar Selvam R