Honeycomb Terraform AWS Integrations

This repo contains a set of modules in the modules folder for resources in AWS using Terraform to send observability data to Honeycomb.

📣 Adopting version 1.0.0

As of v1.0.0 of this module, version 5 of the AWS Provider is required. If you still need support for version 4 of the AWS Provider, continue to use v0.5.0.

How does this work?

Supported Integrations

Available supported integrations include:

• CloudWatch Logs
• CloudWatch Metrics
• RDS Logs
• Logs from a S3 Bucket

To use an individual Terraform integration, refer to the integration's README. Otherwise, refer to the configuration instructions below to configure all supported integrations.

Usage

First, add the minimal Terraform configuration, which includes the required fields for all supported Terraform integrations:

module "honeycomb-aws-integrations" {
  source = "honeycombio/integrations/aws"

  # aws cloudwatch logs integration
  cloudwatch_log_groups = [module.log_group.cloudwatch_log_group_name] // CloudWatch Log Group names to stream to Honeycomb.

  # aws rds logs integration
  enable_rds_logs  = true
  rds_db_name      = var.db_name
  rds_db_engine    = "mysql"
  rds_db_log_types = ["slowquery"] // valid types include general, slowquery, error, and audit (audit will be unstructured)

  # aws metrics integration
  # enable_cloudwatch_metrics = true

  # s3 logfile - alb access logs
  s3_bucket_arn  = var.s3_bucket_arn
  s3_parser_type = "alb" // valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval

  #honeycomb
  honeycomb_api_key = var.honeycomb_api_key             // Honeycomb API key.
  honeycomb_dataset = "terraform-aws-integrations-test" // Your Honeycomb dataset name that will receive the logs.

  # Users generally don't need to set this unless they're using Secure Tenancy
  honeycomb_api_host = var.honeycomb_api_host
}

Then, set the Honeycomb API key for Terraform to use, by setting the HONEYCOMB_API_KEY environment variable.

export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY

Then, set up AWS credentials for the intended AWS account where the resources will be created and managed. Please refer to Terraform documentation for more details and options.

export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION

Now you can run terraform plan/apply in sequence.

For more configuration options, refer to USAGE.md.

IAM role names and permissions boundaries

By default, every IAM role these modules create is named by Terraform using the integration's name as a prefix, and has no permissions boundary. If your organization requires specific role names or a permissions boundary, each integration accepts *_role_name and *_role_permissions_boundary inputs. Refer to USAGE.md for the full list.

⚠️ An IAM role's name cannot be changed in place. Setting a *_role_name on a deployment that was created without one destroys and recreates the role, which in turn replaces the resources that reference it: the Kinesis Firehose delivery stream, the CloudWatch Logs subscription filter or metric stream, and the Lambda functions. Run terraform plan first, and expect a brief gap in telemetry delivery while the replacements happen.

Adding or changing a *_role_permissions_boundary is an in-place update to the role and does not replace anything.

Examples

Examples that use this module can be found in examples/.

Development

Tests

Test cases that run against local code are in tests/. To set up:

1. Set the API key used by Terraform setting the TF_VAR_HONEYCOMB_API_KEY environment variable.

export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY

2. Set up AWS credentials. Please see Terraform documentation for more details and options.

export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION

3. Run terraform init to initialize the working directory.

4. terraform plan and terraform apply will now work as expected, as will terraform destroy.

5. Test cases also run as part of the pipeline. See test-terraform-module.yml

Docs

Configuration option documentation is autogenerated by running make generate-docs, and put in each module's USAGE.md.

Please regenerate and commit before merging any changes to this repository.

Linters

We use tflint and terraform fmt, and enforce this linting with a GitHub Action.

You can run make terraform-format to automatically fix formatting issues.

Contributions

Features, bug fixes and other changes to this module are gladly accepted. Please open issues or a pull request with your change.

All contributions will be released under the Apache License 2.0.