How to Communicate with AWS IoT

Registering your Microvisor-enabled device with AWS IoT and preparing AWS to interact with it involves four key steps:

1. Create an access policy for your device.
2. Create an AWS IoT thing to represent your device.
3. Create a client certificate and signing keys.
4. Bind the policy to the certificate, and the certificate to the thing.

In this guide's workflow, steps 2-4 are embodied in a single step, below.

1. Go to the AWS Console and then select Internet of Things > IoT Core from the Services icon.
2. Navigate to Manage > Security > Policy.
3. Click the Create policy button.
4. Give the policy a name.
5. Under Policy document, click on the JSON tab.
6. In the boilerplate policy, set the values of the Action and Resource keys to "*".
7. Click Create.

1. Navigate to Manage > All devices > Things.
2. Click the Create things button.
3. Make sure Create single thing is selected and click Next.
4. For the thing's name enter your device's SID. Now click Next.
5. Make sure Auto-generate a new certificate (recommended) is selected and click Next.
6. Select the policy you created in Step 1, above, and click Create thing.
7. AWS will allow you to download all the certificates and keys. Do so now — you will use these later.

On the AWS IoT Console, navigate to Settings. Copy the Device data endpoint. Use this value as your MQTT broker hostname, which you will upload in a moment.

AWS IoT's MQTT port is 8883.

The next phase involves transferring the client certificate, its private key, and the CA certificate to the device. This is a three-step process:

1. Convert each secret to the supported format (DER).
2. Upload each secret to secure storage in the Microvisor cloud.
3. The application requests the secrets from the Microvisor cloud.

Each of the authentication files you downloaded from AWS IoT must be converted to the DER format. You use the openssl command to achieve this. For the certificates, use this form:

1
openssl x509 -inform pem -in <CA_CERT_FILE_NAME> -outform der -out AmazonRootCA1.der
2
openssl x509 -inform pem -in <CLIENT_CERT_FILE_NAME> -outform der -out ${MV_DEVICE_SID}-cert.der

To convert the private key, use:

openssl pkcs8 -topk8 -in <PRIVATE_KEY_FILE_NAME> -inform pem -out ${MV_DEVICE_SID}-private_key.der -outform der -nocrypt

${MV_DEVICE_SID} is a shell environment variable that holds your device's SID. You will have set this if you have followed the Get Started with the Microvisor Nucleo Development Board guide. If not run this:

export MV_DEVICE_SID=<YOUR_DEVICE_SID>

Secrets are uploaded on a file by file basis using the Microvisor REST API. At this time, you need to use a command line tool like curl to perform this task. The Twilio CLI will gain this functionality shortly.

1
curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
2
  -d "Key=cert" \
3
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-cert.der)" \
4
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}
5

6
curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
7
  -d "Key=root-CA" \
8
  -d "Value=$(hexdump -v -e '1/1 "%02x"' AmazonRootCA1.der)" \
9
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}
10

11
curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Secrets \
12
  -d "Key=private_key" \
13
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-private_key.der)" \
14
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

Additionally, we recommend treating the MQTT broker and port as secrets. This saves you from the need to bake them into your app:

1
curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
2
  -d "Key=broker-host" \
3
  -d "Value=<YOUR_AWS_DEVICE_DATA_ENDPOINT>" \
4
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}
5

6
curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
7
  -d "Key=broker-port" \
8
  -d "Value=<YOUR_AWS_BROKER_PORT>" \
9
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

Each of the uploaded secrets is now available to be accessed by your application using Microvisor system calls. To do so, your code must first establish a network connection and then open a data channel of type MV_CHANNELTYPE_CONFIGFETCH. You can now retrieve the required secrets:

1
uint8_t deviceid[35] = {0};
2
mvGetDeviceId(deviceid, 34);
3

4
struct MvConfigKeyToFetch root_ca = {
5
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
6
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
7
  .key = {
8
    .data = (uint8_t*)"root-CA",
9
    .length = 7
10
  }
11
};
12

13
MvConfigKeyToFetch cert = {
14
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
15
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
16
  .key = {
17
    .data = (uint8_t*)"cert",
18
    .length = 4
19
  }
20
};
21

22
MvConfigKeyToFetch private_key = {
23
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
24
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
25
  .key = {
26
    .data = (uint8_t*)"private_key",
27
    .length = 11
28
  }
29
};
30

31
MvConfigKeyToFetch broker_host = {
32
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
33
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
34
  .key = {
35
    .data = (uint8_t*)"broker-host",
36
    .length = 11
37
  }
38
};
39

40
MvConfigKeyToFetch broker_port = {
41
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
42
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
43
  .key = {
44
    .data = (uint8_t*)"broker-port",
45
    .length = 11
46
  }
47
};
48

49
MvConfigKeyToFetch keys[5] = {root_ca, cert, private_key, broker_host, broker_port};
50

51
MvConfigKeyFetchParams params = {
52
  .num_items = 5;
53
  .keys_to_fetch = keys
54
};
55

56
enum MvStatus status = mvSendConfigFetchRequest(fetch_channel_handle, &params);
57
assert(status == MV_STATUS_OK);

The requested secrets will be retrieved asynchronously and their availability to the application signaled by a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE to the fetch channel's notification center. Call mvReadConfigFetchResponseData() to retrieve the bulk data, and then call mvReadConfigResponseItem() for each specific item:

1
MvConfigResponseData response;
2
enum MvStatus status = mvReadConfigFetchResponseData(fetch_channel_handle, &response);
3
assert(status == MV_STATUS_OK);
4

5
if (response.result != MV_CONFIGFETCHRESULT_OK) {
6
  server_error("Could not fetch config data (status: %d)", response.result);
7
  return;
8
}
9

10
if (response.num_items != 5) {
11
  server_error("Could not get all items (retrieved %d items)", response.num_items);
12
  return;
13
}
14

15
MvConfigKeyFetchResult result;
16
uint8_t  read_buffer[3072] __attribute__ ((aligned(512))) = {0};
17
uint32_t read_buffer_used = 0;
18

19
struct SizedStringBuffer buf = {
20
  .data = read_buffer,
21
  .size = sizeof(read_buffer)
22
  .length = &read_buffer_used
23
};
24

25
MvConfigResponseReadItemParams params = {
26
  .item_index = 0,
27
  .result = &result,
28
  .buf = &buf;
29
};
30

31
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
32
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
33
// NOTE The function 'consume_binary()' converts the API-submitted string to binary data
34
//      It is not included here: please see the MQTT demo repo for details
35
if (!consume_binary(root_ca, &root_ca_len, read_buffer, &read_buffer_used) {
36
  server_error("Could not unpack root CA");
37
  return;
38
}
39

40
params.item_index = 1;
41
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
42
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
43
if (!consume_binary(client_cert, &client_cert_len, read_buffer, &read_buffer_used) {
44
  server_error("Could not unpack client cert");
45
  return;
46
}
47

48
params.item_index = 2;
49
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
50
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
51
if (!consume_binary(private_key, &private_key_len, read_buffer, &read_buffer_used) {
52
  server_error("Could not unpack private key");
53
  return;
54
}
55

56
params.item_index = 3;
57
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
58
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
59
memcpy(mqtt_broker_host, read_buffer, read_buffer_used);
60
broker_host_len = mqtt_broker_host_len;
61

62
params.item_index = 4;
63
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
64
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
65
read_buffer[read_buffer_used] = '\0';
66
mqtt_broker_port = strtol((const char *)read_buffer, NULL, 10);

With the secrets loaded, they can now be referenced when you establish a second network data channel, this time of type MV_CHANNELTYPE_MQTT. You can close down your fetch channel now.

enum MvStatus status = mvCloseChannel(&fetch_channel_handle);

Rather than repeat the basics of Microvisor MQTT usage — for which see How to Issue MQTT Requests Under Microvisor — we'll focus on the key settings you need for AWS IoT usage.

When you connect to the AWS IoT broker, make sure your connection parameters set MV_MQTTPROTOCOLVERSION_V5 as the value of the MvMqttConnectRequest property protocol_version. AWS IoT uses MQTT 5.

The MvMqttConnectRequest properties tls_credentials and authentication will be set as follows:

1
struct MvMqttAuthentication authentication = {
2
  .method = MV_MQTTAUTHENTICATIONMETHOD_NONE,
3
  .username_password = {
4
    .username = {NULL, 0},
5
    .password = {NULL, 0}
6
  }
7
};
8

9
struct MvSizedString device_certs[] = {
10
  {
11
    .data = (uint8_t *)cert,
12
    .length = cert_len
13
  },
14
};
15

16
struct MvTlsCertificateChain device_certificate = {
17
  .num_certs = 1,
18
  .certs = device_certs
19
};
20

21
struct MvSizedString key = {
22
  .data = (uint8_t *)private_key,
23
  .length = private_key_len
24
};
25

26
struct MvOwnTlsCertificateChain device_credentials = {
27
  .chain = device_certificate,
28
  .key = key
29
};
30

31
struct MvSizedString ca_certs[] = {
32
  {
33
    .data = (uint8_t *)root_ca,
34
    .length = root_ca_len
35
  },
36
};
37

38
struct MvTlsCertificateChain server_ca_certificate = {
39
  .num_certs = 1,
40
  .certs = ca_certs
41
};
42

43
struct MvTlsCredentials tls_credentials = {
44
  .cacert = server_ca_certificate,
45
  .clientcert = device_credentials,
46
};
47

48
struct MvMqttConnectRequest request = {
49
  .protocol_version = MV_MQTTPROTOCOLVERSION_V5,
50
  .host = {
51
    .data = broker_host,
52
    .length = broker_host_len
53
  },
54
  .port = broker_port,
55
  .clientid = {
56
    .data = client,
57
    .length = client_len
58
  },
59
  .authentication = authentication,
60
  .tls_credentials = &tls_credentials,
61
  .keepalive = 60,
62
  .clean_start = 0,
63
  .will = NULL,
64
};

Issue the request with:

enum MvStatus status = mvMqttRequestConnect(mqtt_channel_handle, &request);

If the returned value of status is MV_STATUS_OKAY, your MQTT channel's notification center will shortly receive a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE. Call mvMqttGetNextReadableDataType() and check the returned data type. This value will be MV_MQTTREADABLEDATATYPE_CONNECTRESPONSE if the response is the result of your connection attempt. In this case, call mvMqttReadConnectResponse() and Microvisor will populate the MvMqttConnectResponse you pass into the system call:

1
struct MvMqttConnectResponse {
2
  enum MvMqttRequestState request_state,
3
  uint32_t reason_code
4
}

If the value of request_state is MV_MQTTCONNECTSTATUS_REQUESTCOMPLETED then your application has successfully connected to the AWS IoT MQTT broker, and you can subscribe to channels and then begin publishing messages to them.

If you receive another value, check the status it indicates. You should check you have downloaded, converted, and uploaded the correct CA and client certificates, and the client private key. Check they are being stored correctly and are accessible by your MQTT connection code. Make sure you are passing the certificates to AWS IoT correctly.

Once the device is connected to the AWS IoT broker, one of the actions it can take is to start publishing messages, subscribing to topics, and receiving messages from those topics. This is standard MQTT functionality so, again, we won't cover the process here. For AWS IoT, you must ensure that the topic to which you will be publishing is correctly enabled in your policy. At the start of this guide, we set an 'allow all' policy for convenience, but we really should implement one that's more secure and targets your device.

1. Go to the AWS Console and then select Internet of Things > IoT Core from the Services icon.

2. Navigate to Manage > Security > Policy .

3. Click on your existing policy's name.

4. Click on Edit active version .

5. Click on JSON .

6. Copy the following Statement object and paste it on over the existing one:

   Copy code block

   1
     "Statement": [
   2
       {
   3
         Effect": "Allow",
   4
         "Action": "iot:Connect",
   5
         "Resource": "arn:aws:iot:<region>:<account_id>:client/${iot:Connection.Thing.ThingName}"
   6
       },
   7
       {
   8
         "Effect": "Allow",
   9
         "Action": "iot:Publish",
   10
         "Resource": "arn:aws:iot:<region>:<account_id>:topic/sensor/device/${iot:Connection.Thing.ThingName}"
   11
       },
   12
       {
   13
         "Effect": "Allow",
   14
         "Action": "iot:Subscribe",
   15
         "Resource": [
   16
         "arn:aws:iot:<region>:<account_id>:topicfilter/command/device/${iot:Connection.Thing.ThingName}",
   17
         "arn:aws:iot:<region>:<account_id>:topicfilter/command/device/all"
   18
         ]
   19
       },
   20
       {
   21
         "Effect": "Allow",
   22
         "Action": "iot:Receive",
   23
         "Resource": [
   24
         "arn:aws:iot:<region>:<account_id>:topic/command/device/${iot:Connection.Thing.ThingName}",
   25
         "arn:aws:iot:<region>:<account_id>:topic/command/device/all"
   26
         ]
   27
       }
   28
     ]

7. Replace all instances of <region> with your own AWS region, and all instances of <account_id> with your own AWS account ID (numbers only; omit any hyphens).

8. Check the JSON is valid: the AWS console will warn you of structural errors. Look for mis-paired array and object markers.

9. Make sure Set the edited version as the active version for this policy is checked.

10. Click the Save as new version button.

This policy is configured for the MQTT demo code. It includes the topic sensor/device/<DEVICE_SID> for message posting, and the topic command/device/<DEVICE_SID> for receiving messages. It also assumes your AWS IoT Thing's name matches your device's SID.