How to Communicate with AWS IoT

Microvisor is ready to work with Amazon Web Services’ AWS IoT offering, which uses MQTT as its messaging protocol. This guide will show you how to configure AWS IoT to receive data from a Microvisor-enabled device, and what you need to add to your own Microvisor-backed application to authenticate with AWS IoT.

It is not a guide to using Microvisor’s MQTT functionality. Please see How to Issue MQTT Requests Under Microvisor if that’s your need. Additionally, AWS IoT has some differences with a standard MQTT implementation. You can review these differences in the AWS IoT documentation.

Set up AWS IoT

Registering your Microvisor-enabled device with AWS IoT and preparing AWS to interact with it involves four key steps:

1. Create an access policy for your device.
2. Create an AWS IoT thing to represent your device.
3. Create a client certificate and signing keys.
4. Bind the policy to the certificate, and the certificate to the thing.

In this guide’s workflow, steps 2-4 are embodied in a single step, below.

1. Create an access policy

1. Go to the AWS Console and then select Internet of Things > IoT Core from the Services icon.
2. Navigate to Manage > Security > Policy.
3. Click the Create policy button.
4. Give the policy a name.
5. Under Policy document, click on the JSON tab.
6. In the boilerplate policy, set the values of the Action and Resource keys to "*".
7. Click Create.

2. Create a thing on AWS IoT

1. Navigate to Manage > All devices > Things.
2. Click the Create things button.
3. Make sure Create single thing is selected and click Next.
4. For the thing’s name enter your device’s SID. Now click Next.
5. Make sure Auto-generate a new certificate (recommended) is selected and click Next.
6. Select the policy you created in Step 1, above, and click Create thing.
7. AWS will allow you to download all the certificates and keys. Do so now — you will use these later.

Acquire AWS data

On the AWS IoT Console, navigate to Settings. Copy the Device data endpoint. Use this value as your MQTT broker hostname, which you will upload in a moment.

AWS IoT’s MQTT port is 8883.

Provision the application

The next phase involves transferring the client certificate, its private key, and the CA certificate to the device. This is a three-step process:

1. Convert each secret to the supported format (DER).
2. Upload each secret to secure storage in the Twilio cloud.
3. The application requests the secrets from the Twilio cloud.

1. Convert secrets

Each of the authentication files you downloaded from AWS IoT must be converted to the DER format. You use the openssl command to achieve this. For the certificates, use this form:

openssl x509 -inform pem -in <CA_CERT_FILE_NAME> -outform der -out AmazonRootCA1.der
openssl x509 -inform pem -in <CLIENT_CERT_FILE_NAME> -outform der -out ${MV_DEVICE_SID}-cert.der

To convert the private key, use:

openssl pkcs8 -topk8 -in <PRIVATE_KEY_FILE_NAME> -inform pem -out ${MV_DEVICE_SID}-private_key.der -outform der -nocrypt

${MV_DEVICE_SID} is a shell environment variable that holds your device’s SID. You will have set this if you have followed the Get Started with the Microvisor Nucleo Development Board guide. If not run this:

export MV_DEVICE_SID=<YOUR_DEVICE_SID>

2. Upload secrets

Secrets are uploaded on a file by file basis using the Microvisor REST API. At this time, you need to use a command line tool like curl to perform this task. The Twilio CLI will gain this functionality shortly.

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=cert" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-cert.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=root-CA" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' AmazonRootCA1.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Secrets \
  -d "Key=private_key" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-private_key.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

Additionally, we recommend treating the MQTT broker and port as secrets. This saves you from the need to bake them into your app:

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=broker-host" \
  -d "Value=<YOUR_AWS_DEVICE_DATA_ENDPOINT>" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=broker-port" \
  -d "Value=<YOUR_AWS_BROKER_PORT>" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

3. Request secrets

Each of the uploaded secrets is now available to be accessed by your application using Microvisor system calls. To do so, your code must first establish a network connection and then open a data channel of type MV_CHANNELTYPE_CONFIGFETCH. You can now retrieve the required secrets:

uint8_t deviceid[35] = {0};
mvGetDeviceId(deviceid, 34);

struct MvConfigKeyToFetch root_ca = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"root-CA",
    .length = 7
  }
};

MvConfigKeyToFetch cert = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"cert",
    .length = 4
  }
};

MvConfigKeyToFetch private_key = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"private_key",
    .length = 11
  }
};

MvConfigKeyToFetch broker_host = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"broker-host",
    .length = 11
  }
};

MvConfigKeyToFetch broker_port = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"broker-port",
    .length = 11
  }
};

MvConfigKeyToFetch keys[5] = {root_ca, cert, private_key, broker_host, broker_port};

MvConfigKeyFetchParams params = {
  .num_items = 5;
  .keys_to_fetch = keys
};

enum MvStatus status = mvSendConfigFetchRequest(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK);

The requested secrets will be retrieved asynchronously and their availability to the application signaled by a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE to the fetch channel’s notification center. Call mvReadConfigFetchResponseData() to retrieve the bulk data, and then call mvReadConfigResponseItem() for each specific item:

MvConfigResponseData response;
enum MvStatus status = mvReadConfigFetchResponseData(fetch_channel_handle, &response);
assert(status == MV_STATUS_OK);

if (response.result != MV_CONFIGFETCHRESULT_OK) {
  server_error("Could not fetch config data (status: %d)", response.result);
  return;
}

if (response.num_items != 5) {
  server_error("Could not get all items (retrieved %d items)", response.num_items);
  return;
}

MvConfigKeyFetchResult result;
uint8_t  read_buffer[3072] __attribute__ ((aligned(512))) = {0};
uint32_t read_buffer_used = 0;

struct SizedStringBuffer buf = {
  .data = read_buffer,
  .size = sizeof(read_buffer)
  .length = &read_buffer_used
};

MvConfigResponseReadItemParams params = {
  .item_index = 0,
  .result = &result,
  .buf = &buf;
};

status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
// NOTE The function 'consume_binary()' converts the API-submitted string to binary data
//      It is not included here: please see the MQTT demo repo for details
if (!consume_binary(root_ca, &root_ca_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack root CA");
  return;
}

params.item_index = 1;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
if (!consume_binary(client_cert, &client_cert_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack client cert");
  return;
}

params.item_index = 2;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
if (!consume_binary(private_key, &private_key_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack private key");
  return;
}

params.item_index = 3;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
memcpy(mqtt_broker_host, read_buffer, read_buffer_used);
broker_host_len = mqtt_broker_host_len;

params.item_index = 4;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
read_buffer[read_buffer_used] = '\0';
mqtt_broker_port = strtol((const char *)read_buffer, NULL, 10);

With the secrets loaded, they can now be referenced when you establish a second network data channel, this time of type MV_CHANNELTYPE_MQTT. You can close down your fetch channel now.

enum MvStatus status = mvCloseChannel(&fetch_channel_handle);

Connect to AWS IoT

Rather than repeat the basics of Microvisor MQTT usage — for which see How to Issue MQTT Requests Under Microvisor — we’ll focus on the key settings you need for AWS IoT usage.

When you connect to the AWS IoT broker, make sure your connection parameters set MV_MQTTPROTOCOLVERSION_V5 as the value of the MvMqttConnectRequest property protocol_version. AWS IoT uses MQTT 5.

The MvMqttConnectRequest properties tls_credentials and authentication will be set as follows:

struct MvMqttAuthentication authentication = {
  .method = MV_MQTTAUTHENTICATIONMETHOD_NONE,
  .username_password = {
    .username = {NULL, 0},
    .password = {NULL, 0}
  }
};

struct MvSizedString device_certs[] = {
  {
    .data = (uint8_t *)cert,
    .length = cert_len
  },
};

struct MvTlsCertificateChain device_certificate = {
  .num_certs = 1,
  .certs = device_certs
};

struct MvSizedString key = {
  .data = (uint8_t *)private_key,
  .length = private_key_len
};

struct MvOwnTlsCertificateChain device_credentials = {
  .chain = device_certificate,
  .key = key
};

struct MvSizedString ca_certs[] = {
  {
    .data = (uint8_t *)root_ca,
    .length = root_ca_len
  },
};

struct MvTlsCertificateChain server_ca_certificate = {
  .num_certs = 1,
  .certs = ca_certs
};

struct MvTlsCredentials tls_credentials = {
  .cacert = server_ca_certificate,
  .clientcert = device_credentials,
};

struct MvMqttConnectRequest request = {
  .protocol_version = MV_MQTTPROTOCOLVERSION_V5,
  .host = {
    .data = broker_host,
    .length = broker_host_len
  },
  .port = broker_port,
  .clientid = {
    .data = client,
    .length = client_len
  },
  .authentication = authentication,
  .tls_credentials = &tls_credentials,
  .keepalive = 60,
  .clean_start = 0,
  .will = NULL,
};

Issue the request with:

enum MvStatus status = mvMqttRequestConnect(mqtt_channel_handle, &request);

If the returned value of status is MV_STATUS_OKAY, your MQTT channel’s notification center will shortly receive a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE. Call mvMqttGetNextReadableDataType() and check the returned data type. This value will be MV_MQTTREADABLEDATATYPE_CONNECTRESPONSE if the response is the result of your connection attempt. In this case, call mvMqttReadConnectResponse() and Microvisor will populate the MvMqttConnectResponse you pass into the system call:

struct MvMqttConnectResponse {
  enum MvMqttRequestState request_state,
  uint32_t reason_code
}

If the value of request_state is MV_MQTTCONNECTSTATUS_REQUESTCOMPLETED then your application has successfully connected to the AWS IoT MQTT broker, and you can subscribe to channels and then begin publishing messages to them.

If you receive another value, check the status it indicates. You should check you have downloaded, converted, and uploaded the correct CA and client certificates, and the client private key. Check they are being stored correctly and are accessible by your MQTT connection code. Make sure you are passing the certificates to AWS IoT correctly.

Tighten your AWS access policy

Once the device is connected to the AWS IoT broker, one of the actions it can take is to start publishing messages, subscribing to topics, and receiving messages from those topics. This is standard MQTT functionality so, again, we won’t cover the process here. For AWS IoT, you must ensure that the topic to which you will be publishing is correctly enabled in your policy. At the start of this guide, we set an ‘allow all’ policy for convenience, but we really should implement one that’s more secure and targets your device.

This policy is configured for the MQTT demo code. It includes the topic sensor/device/<DEVICE_SID> for message posting, and the topic command/device/<DEVICE_SID> for receiving messages. It also assumes your AWS IoT Thing’s name matches your device’s SID.