9.7 KiB

Raw Blame History

Testing

driftctl uses both unit tests and acceptance tests. Acceptance tests are not required, but at least a good unit test coverage is required for a PR to be merged. This documentation section's goal is about how we manage our test suite in driftctl.

driftctl uses gotestsum to wrap go test, you can install required tools to run test with make install-tools.

To run unit test simply run.

$ make install-tools
$ make test

Before the test suite starts, we run golangci-lint. If there are any linter issues, you have to fix them first.

For the driftctl team, code coverage is very important as it helps show which part of your code is not covered. We kindly ask you to check your coverage to ensure every important part of your code is tested. We do not expect 100% coverage for each line of code, but at least every critical part of your code should be covered. For example, we don't care about covering NewStruct() constructors if there is no big logic inside. Remember, a covered code does not mean that all conditions are tested and asserted, so be careful to test the right things. A bug can still happen in a covered part of your code.

We use the golden file pattern to assert on results. Golden files could be updated with -update flag. For example, I've made modifications to s3 bucket policy, I could update golden files with the following command:

$ go test ./pkg/remote/aws/ --update s3_bucket_policy_no_policy

⚠️ Beware that updating golden files may call external services.

In the example above, as we are using mocked AWS responses in json golden files, you should have to configure proper resources on AWS side before running an update. For convenience, we try to put, as much as possible, terraform files used to generate golden files in test folders.

A quick way to get started is to copy/paste an existing test and adapt it to your needs.

Unit testing

Unit testing should not use any external dependency, so we mock all calls to the cloud provider's SDK (see below for more details on mocking).

Mocks

In driftctl unit test suite, each call to the cloud provider's SDK should be mocked. We use mocks generated by mockery in our tests. See below each step to create a mock for a new AWS service (e.g. EC2).

Create a mock interface in test/aws/ec2.go

package aws

import (
	"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
)

type FakeEC2 interface {
	ec2iface.EC2API
}

Use mockery to generate a full mocked struct mockery --name FakeS3 --dir ./test/aws
Mock a response in your test (list IAM users for example)

client := mocks.FakeIAM{}
client.On("ListUsersPages",
    &iam.ListUsersInput{},
    mock.MatchedBy(func(callback func(res *iam.ListUsersOutput, lastPage bool) bool) bool {
        callback(&iam.ListUsersOutput{Users: []*iam.User{
            {
                UserName: aws.String("test-driftctl"),
            },
            {
                UserName: aws.String("test-driftctl2"),
            },
        }}, true)
        return true
    })
).Once().Return(nil)

⚠️ If you have several mocks on the same method, the "mock" library will evaluate code in your MatchedBy multiple times even if the first parameter does not match. It means your callback will always be called, this is an unwanted behaviour most of the time! A workaround is to manage flags but this is an ugly solution, here is an example using a boolean flag:

mockCalled := false
client := mocks.FakeIAM{}
client.On("ListUsersPages",
    &iam.ListUsersInput{},
    mock.MatchedBy(func(callback func(res *iam.ListUsersOutput, lastPage bool) bool) bool {
        if mockCalled {
            return false
        }
        callback(&iam.ListUsersOutput{Users: []*iam.User{
            {
                UserName: aws.String("test-driftctl"),
            },
            {
                UserName: aws.String("test-driftctl2"),
            },
        }}, true)
        mockCalled = true
        return true
    })
).Once().Return(nil)

🙏 We are still looking for a better way to handle this, contributions are welcome.

References:

Acceptance testing

driftctl provides a kind of acceptance test framework (test/acceptance) to help you run those tests. The goal here is to apply some terraform code, and then run a series of Check. A Check consists of running driftctl and checking for results using json output. driftctl uses assertion struct to help you check output results. See below for more details.

Each acceptance test should be prefixed by TestAcc_ and should be run using the environment var DRIFTCTL_ACC=true.

$ DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go

Credentials

Acceptance tests need credentials to perform real world action on cloud providers:

Read/write access are required to perform terraform action
Read only access is required to execute driftctl

Recommended way to run acceptance tests is to use two distinct credentials:

One for terraform related actions
One for driftctl scan

In our acceptance tests, we may need read/write permissions during specific contexts (e.g. terraform init, apply, destroy) or lifecycle (PreExec and PostExec). If needed, you can override environment variables in those contexts by adding ACC_ prefix on environment variables.

AWS

You can use ACC_AWS_PROFILE to override AWS named profile used for terraform operations.

$ ACC_AWS_PROFILE=read-write-profile AWS_PROFILE=read-only-profile DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go

In the example below, the driftctl AWS profile must have read/write permissions and will be used for both terraform operations and driftctl run.

This is not the recommended way to run tests as it may hide permissions issues.

$ AWS_PROFILE=driftctl DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go

Workflow

OnStart You may run some code before everything
terraform apply
For each declared check loop
- PreExec
- driftctl scan
- check results
- PostExec
OnEnd
terraform destroy

⚠️ driftctl tests handle terraform resources removal, but it is up to you to remove potential unmanaged resources added in PreExec step !

Example

The following test runs terraform to create an EC2 instance. Then, we add a new tag (ENV: production) to the instance. Finally, we check the drift.

func TestAcc_AwsInstance_WithBlockDevices(t *testing.T) {
	var mutatedInstanceId string
	acceptance.Run(t, acceptance.AccTestCase{
		// This path should contain terraform files
		Path: "./testdata/acc/aws_instance",
		// Pass args to driftctl execution
		// DO NOT PASS --output flag as it is handled automatically by test runner
		// You may use a .driftignore file in your test directory or use filters to limit driftctl scope
		// Try to be minimalist as possible as test will be easier to maintain over time
		Args: []string{"scan"}, // TODO add filter to limit scan scope to aws_instances
		Checks: []acceptance.AccCheck{
			{ // First check does not have any PreExec or PostExec
				Check: func(result *acceptance.ScanResult, stdout string, err error) {
					if err != nil {
						t.Fatal(err)
					}
					// Assert that no drift are detected
					result.AssertDriftCountTotal(0)
					// We could assert on analysis object directly
					// Below we check for infra strictly in sync, beware that this check should fail
					// if you run your acceptance test on a messy cloud provider state (existing dangling resources for example)
					// without using filter or driftignore
					//
					// Note that the result struct is composed of analysis result AND assertion library
					// You could use result.Equal() directly for example
                    result.True(result.Analysis.IsSync())
				},
			},
			{
				// In this PreExec, we retrieve the created instance ID and add a new tag
				// using AWS SDK
				// We store the instance ID in a var to assert on it after driftctl run
				PreExec: func() {
					client := ec2.New(awsutils.Session())
					response, err := client.DescribeInstances(&ec2.DescribeInstancesInput{
						Filters: []*ec2.Filter{
							{
								Name: aws.String("instance-state-name"),
								Values: []*string{
									aws.String("running"),
								},
							},
							{
								Name: aws.String("tag:Name"),
								Values: []*string{
									aws.String("test_instance_1"),
								},
							},
						},
					})
					if err != nil {
						t.Fatal(err)
					}
					if len(response.Reservations[0].Instances) != 1 {
						t.Fatal("Error, unexpected number of instances found, manual check required")
					}
					mutatedInstanceId = *response.Reservations[0].Instances[0].InstanceId
					_, _ = client.CreateTags(&ec2.CreateTagsInput{
						Resources: []*string{&mutatedInstanceId},
						Tags: []*ec2.Tag{
							{
								Key:   aws.String("Env"),
								Value: aws.String("Production"),
							},
						},
					})
				},
				// Check that driftctl detected a drift on manually modified instances
				Check: func(result *acceptance.ScanResult, stdout string, err error) {
					if err != nil {
						t.Fatal(err)
					}
					result.AssertResourceHasDrift(
						mutatedInstanceId,
						awsresources.AwsInstanceResourceType,
                        analyser.Change{
                            Change: diff.Change{
                                Type: diff.CREATE,
                                Path: []string{"Tags", "Env"},
                                From: nil,
                                To:   "Production",
                            },
                        },
					)
				},
			},
		},
	})
}

9.7 KiB Raw Blame History