Menu

Expand
Rate this page:

How to Communicate with AWS IoT

Microvisor is ready to work with Amazon Web Services’ AWS IoT offering, which uses MQTT as its messaging protocol. This guide will show you how to configure AWS IoT to receive data from a Microvisor-enabled device, and what you need to add to your own Microvisor-backed application to authenticate with AWS IoT.

It is not a guide to using Microvisor’s MQTT functionality. Please see How to Issue MQTT Requests Under Microvisor if that’s your need. Additionally, AWS IoT has some differences with a standard MQTT implementation. You can review these differences in the AWS IoT documentation.

This guide assumes you have already set up an account with AWS. If you have not done so, you can create an AWS account here.

Please be aware that outside of free tier usage, AWS IoT is a billable service. AWS charges for device connections, messaging, and other services. Please familiarize yourself with AWS IoT pricing before you continue.

Set up AWS IoT

Registering your Microvisor-enabled device with AWS IoT and preparing AWS to interact with it involves four key steps:

  1. Create an access policy for your device.
  2. Create an AWS IoT thing to represent your device.
  3. Create a client certificate and signing keys.
  4. Bind the policy to the certificate, and the certificate to the thing.

In this guide’s workflow, steps 2-4 are embodied in a single step, below.

1. Create an access policy

  1. Go to the AWS Console and then select Internet of Things > IoT Core from the Services icon.
  2. Navigate to Manage > Security > Policy.
  3. Click the Create policy button.
  4. Give the policy a name.
  5. Under Policy document, click on the JSON tab.
  6. In the boilerplate policy, set the values of the Action and Resource keys to "*".
  7. Click Create.

The access policy set in the above step provides access to all AWS IoT resources and allows all actions. This is for simplicity, but in production you should limit devices to the actions and resources they need and no more. You may wish to do so in development too. We’ll provide more tightly controlled policy later.

2. Create a thing on AWS IoT

  1. Navigate to Manage > All devices > Things.
  2. Click the Create things button.
  3. Make sure Create single thing is selected and click Next.
  4. For the thing’s name enter your device’s SID. Now click Next.
  5. Make sure Auto-generate a new certificate (recommended) is selected and click Next.
  6. Select the policy you created in Step 1, above, and click Create thing.
  7. AWS will allow you to download all the certificates and keys. Do so now — you will use these later.

Acquire AWS data

On the AWS IoT Console, navigate to Settings. Copy the Device data endpoint. Use this value as your MQTT broker hostname, which you will upload in a moment.

AWS IoT’s MQTT port is 8883.

Provision the application

The next phase involves transferring the client certificate, its private key, and the CA certificate to the device. This is a three-step process:

  1. Convert each secret to the supported format (DER).
  2. Upload each secret to secure storage in the Twilio cloud.
  3. The application requests the secrets from the Twilio cloud.

1. Convert secrets

Each of the authentication files you downloaded from AWS IoT must be converted to the DER format. You use the openssl command to achieve this. For the certificates, use this form:

openssl x509 -inform pem -in <CA_CERT_FILE_NAME> -outform der -out AmazonRootCA1.der
openssl x509 -inform pem -in <CLIENT_CERT_FILE_NAME> -outform der -out ${MV_DEVICE_SID}-cert.der

To convert the private key, use:

openssl pkcs8 -topk8 -in <PRIVATE_KEY_FILE_NAME> -inform pem -out ${MV_DEVICE_SID}-private_key.der -outform der -nocrypt

${MV_DEVICE_SID} is a shell environment variable that holds your device’s SID. You will have set this if you have followed the Get Started with the Microvisor Nucleo Development Board guide. If not run this:

export MV_DEVICE_SID=<YOUR_DEVICE_SID>

2. Upload secrets

Secrets are uploaded on a file by file basis using the Microvisor REST API. At this time, you need to use a command line tool like curl to perform this task. The Twilio CLI will gain this functionality shortly.

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=cert" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-cert.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=root-CA" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' AmazonRootCA1.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Secrets \
  -d "Key=private_key" \
  -d "Value=$(hexdump -v -e '1/1 "%02x"' ${MV_DEVICE_SID}-private_key.der)" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

Additionally, we recommend treating the MQTT broker and port as secrets. This saves you from the need to bake them into your app:

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=broker-host" \
  -d "Value=<YOUR_AWS_DEVICE_DATA_ENDPOINT>" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

curl -X POST -s https://microvisor.twilio.com/v1/Devices/${MV_DEVICE_SID}/Configs \
  -d "Key=broker-port" \
  -d "Value=<YOUR_AWS_BROKER_PORT>" \
  -u ${TWILIO_ACCOUNT_SID}:${TWILIO_AUTH_TOKEN}

3. Request secrets

Each of the uploaded secrets is now available to be accessed by your application using Microvisor system calls. To do so, your code must first establish a network connection and then open a data channel of type MV_CHANNELTYPE_CONFIGFETCH. You can now retrieve the required secrets:

uint8_t deviceid[35] = {0};
mvGetDeviceId(deviceid, 34);

struct MvConfigKeyToFetch root_ca = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"root-CA",
    .length = 7
  }
};

MvConfigKeyToFetch cert = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"cert",
    .length = 4
  }
};

MvConfigKeyToFetch private_key = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"private_key",
    .length = 11
  }
};

MvConfigKeyToFetch broker_host = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"broker-host",
    .length = 11
  }
};

MvConfigKeyToFetch broker_port = {
  .scope = MV_CONFIGKEYFETCHSCOPE_DEVICE;
  .store = MV_CONFIGKEYFETCHSTORE_CONFIG;
  .key = {
    .data = (uint8_t*)"broker-port",
    .length = 11
  }
};

MvConfigKeyToFetch keys[5] = {root_ca, cert, private_key, broker_host, broker_port};

MvConfigKeyFetchParams params = {
  .num_items = 5;
  .keys_to_fetch = keys
};

enum MvStatus status = mvSendConfigFetchRequest(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK);

You can find the fully working code from which these snippets were taken in our MQTT demo repo.

The requested secrets will be retrieved asynchronously and their availability to the application signaled by a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE to the fetch channel’s notification center. Call mvReadConfigFetchResponseData() to retrieve the bulk data, and then call mvReadConfigResponseItem() for each specific item:

MvConfigResponseData response;
enum MvStatus status = mvReadConfigFetchResponseData(fetch_channel_handle, &response);
assert(status == MV_STATUS_OK);

if (response.result != MV_CONFIGFETCHRESULT_OK) {
  server_error("Could not fetch config data (status: %d)", response.result);
  return;
}

if (response.num_items != 5) {
  server_error("Could not get all items (retrieved %d items)", response.num_items);
  return;
}

MvConfigKeyFetchResult result;
uint8_t  read_buffer[3072] __attribute__ ((aligned(512))) = {0};
uint32_t read_buffer_used = 0;

struct SizedStringBuffer buf = {
  .data = read_buffer,
  .size = sizeof(read_buffer)
  .length = &read_buffer_used
};

MvConfigResponseReadItemParams params = {
  .item_index = 0,
  .result = &result,
  .buf = &buf;
};

status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
// NOTE The function 'consume_binary()' converts the API-submitted string to binary data
//      It is not included here: please see the MQTT demo repo for details
if (!consume_binary(root_ca, &root_ca_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack root CA");
  return;
}

params.item_index = 1;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
if (!consume_binary(client_cert, &client_cert_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack client cert");
  return;
}

params.item_index = 2;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
if (!consume_binary(private_key, &private_key_len, read_buffer, &read_buffer_used) {
  server_error("Could not unpack private key");
  return;
}

params.item_index = 3;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
memcpy(mqtt_broker_host, read_buffer, read_buffer_used);
broker_host_len = mqtt_broker_host_len;

params.item_index = 4;
status = mvReadConfigResponseItem(fetch_channel_handle, &params);
assert(status == MV_STATUS_OK && result == MV_CONFIGFETCHRESULT_OK);
read_buffer[read_buffer_used] = '\0';
mqtt_broker_port = strtol((const char *)read_buffer, NULL, 10);

With the secrets loaded, they can now be referenced when you establish a second network data channel, this time of type MV_CHANNELTYPE_MQTT. You can close down your fetch channel now.

enum MvStatus status = mvCloseChannel(&fetch_channel_handle);

Connect to AWS IoT

Rather than repeat the basics of Microvisor MQTT usage — for which see How to Issue MQTT Requests Under Microvisor — we’ll focus on the key settings you need for AWS IoT usage.

When you connect to the AWS IoT broker, make sure your connection parameters set MV_MQTTPROTOCOLVERSION_V5 as the value of the MvMqttConnectRequest property protocol_version. AWS IoT uses MQTT 5.

The MvMqttConnectRequest properties tls_credentials and authentication will be set as follows:

struct MvMqttAuthentication authentication = {
  .method = MV_MQTTAUTHENTICATIONMETHOD_NONE,
  .username_password = {
    .username = {NULL, 0},
    .password = {NULL, 0}
  }
};

struct MvSizedString device_certs[] = {
  {
    .data = (uint8_t *)cert,
    .length = cert_len
  },
};

struct MvTlsCertificateChain device_certificate = {
  .num_certs = 1,
  .certs = device_certs
};

struct MvSizedString key = {
  .data = (uint8_t *)private_key,
  .length = private_key_len
};

struct MvOwnTlsCertificateChain device_credentials = {
  .chain = device_certificate,
  .key = key
};

struct MvSizedString ca_certs[] = {
  {
    .data = (uint8_t *)root_ca,
    .length = root_ca_len
  },
};

struct MvTlsCertificateChain server_ca_certificate = {
  .num_certs = 1,
  .certs = ca_certs
};

struct MvTlsCredentials tls_credentials = {
  .cacert = server_ca_certificate,
  .clientcert = device_credentials,
};

struct MvMqttConnectRequest request = {
  .protocol_version = MV_MQTTPROTOCOLVERSION_V5,
  .host = {
    .data = broker_host,
    .length = broker_host_len
  },
  .port = broker_port,
  .clientid = {
    .data = client,
    .length = client_len
  },
  .authentication = authentication,
  .tls_credentials = &tls_credentials,
  .keepalive = 60,
  .clean_start = 0,
  .will = NULL,
};

Issue the request with:

enum MvStatus status = mvMqttRequestConnect(mqtt_channel_handle, &request);

If the returned value of status is MV_STATUS_OKAY, your MQTT channel’s notification center will shortly receive a notification of type MV_EVENTTYPE_CHANNELDATAREADABLE. Call mvMqttGetNextReadableDataType() and check the returned data type. This value will be MV_MQTTREADABLEDATATYPE_CONNECTRESPONSE if the response is the result of your connection attempt. In this case, call mvMqttReadConnectResponse() and Microvisor will populate the MvMqttConnectResponse you pass into the system call:

struct MvMqttConnectResponse {
  enum MvMqttRequestState request_state,
  uint32_t reason_code
}

If the value of request_state is MV_MQTTCONNECTSTATUS_REQUESTCOMPLETED then your application has successfully connected to the AWS IoT MQTT broker, and you can subscribe to channels and then begin publishing messages to them.

If you receive another value, check the status it indicates. You should check you have downloaded, converted, and uploaded the correct CA and client certificates, and the client private key. Check they are being stored correctly and are accessible by your MQTT connection code. Make sure you are passing the certificates to AWS IoT correctly.

Tighten your AWS access policy

Once the device is connected to the AWS IoT broker, one of the actions it can take is to start publishing messages, subscribing to topics, and receiving messages from those topics. This is standard MQTT functionality so, again, we won’t cover the process here. For AWS IoT, you must ensure that the topic to which you will be publishing is correctly enabled in your policy. At the start of this guide, we set an ‘allow all’ policy for convenience, but we really should implement one that’s more secure and targets your device.

  1. Go to the AWS Console and then select Internet of Things > IoT Core from the Services icon.
  2. Navigate to Manage > Security > Policy.
  3. Click on your existing policy’s name.
  4. Click on Edit active version.
  5. Click on JSON.
  6. Copy the following Statement object and paste it on over the existing one:
      "Statement": [
        {
          Effect": "Allow",
          "Action": "iot:Connect",
          "Resource": "arn:aws:iot:<region>:<account_id>:client/${iot:Connection.Thing.ThingName}"
        },
        {
          "Effect": "Allow",
          "Action": "iot:Publish",
          "Resource": "arn:aws:iot:<region>:<account_id>:topic/sensor/device/${iot:Connection.Thing.ThingName}"
        },
        {
          "Effect": "Allow",
          "Action": "iot:Subscribe",
          "Resource": [
          "arn:aws:iot:<region>:<account_id>:topicfilter/command/device/${iot:Connection.Thing.ThingName}",
          "arn:aws:iot:<region>:<account_id>:topicfilter/command/device/all"
          ]
        },
        {
          "Effect": "Allow",
          "Action": "iot:Receive",
          "Resource": [
          "arn:aws:iot:<region>:<account_id>:topic/command/device/${iot:Connection.Thing.ThingName}",
          "arn:aws:iot:<region>:<account_id>:topic/command/device/all"
          ]
        }
      ]
  7. Replace all instances of <region> with your own AWS region, and all instances of <account_id> with your own AWS account ID (numbers only; omit any hyphens).
  8. Check the JSON is valid: the AWS console will warn you of structural errors. Look for mis-paired array and object markers.
  9. Make sure Set the edited version as the active version for this policy is checked.
  10. Click the Save as new version button.

This policy is configured for the MQTT demo code. It includes the topic sensor/device/<DEVICE_SID> for message posting, and the topic command/device/<DEVICE_SID> for receiving messages. It also assumes your AWS IoT Thing’s name matches your device’s SID.

Rate this page:

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd by visiting Twilio's Stack Overflow Collective or browsing the Twilio tag on Stack Overflow.

Thank you for your feedback!

Please select the reason(s) for your feedback. The additional information you provide helps us improve our documentation:

Sending your feedback...
🎉 Thank you for your feedback!
Something went wrong. Please try again.

Thanks for your feedback!

thanks-feedback-gif