9.7 KiB
Testing
driftctl uses both unit test and acceptance test. Acceptance test are not required, but at least a good unit test coverage is required for a PR to be merged. This documentation section's goal is about how we manage our test suite in driftctl.
driftctl uses gotestsum to wrap go test, you can install required tools to run test with make install-tools
To run unit test simply run
make install-tools
make test
Before the test suite starts, we run golangci-lint.
If there are any linter issues, you have to fix them first.
For the driftctl team, code coverage is very important as it helps show which part of your code is not covered.
We kindly ask you to check your coverage to ensure every important part of your code is tested.
We do not expect 100% coverage for every line of code, but at least every critical part of your code should be covered.
For example, we don't care about covering NewStruct() constructors if there is no big logic inside.
Remember, a covered code does not mean that every condition is tested and asserted, so be careful to test the right things.
A bug can still happen in a covered part of your code.
We use the golden file pattern to assert on results. Golden files could be updated with -update flag.
For example, I've made some modifications to s3 bucket policy, I could update golden files with the following command:
go test ./pkg/remote/aws/ --update s3_bucket_policy_no_policy
⚠️ Beware that updating golden files may call external services. In the example above, as we are using mocked AWS responses in json golden files, you should have to configure proper resources on AWS side before running an update. For convenience, we try to put, as much as possible, terraform files used to generate golden files in test folders.
A quick way to get started is to copy/paste an existing test and adapt it to your needs.
Unit testing
Unit testing should not use any external dependency, so we mock every call to the cloud provider's SDK (see below for more details on mocking)
Mocks
In driftctl unit test suite, every call to the cloud provider's SDK should be mocked. We use mocks generated by mockery in our tests. See below each step to create a mock for a new AWS service (e.g. EC2).
- Create a mock interface in
test/aws/ec2.go
package aws
import (
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
)
type FakeEC2 interface {
ec2iface.EC2API
}
- Use mockery to generate a full mocked struct
mockery --name FakeS3 --dir ./test/aws - Mock a response in your test (list IAM users for example)
client := mocks.FakeIAM{}
client.On("ListUsersPages",
&iam.ListUsersInput{},
mock.MatchedBy(func(callback func(res *iam.ListUsersOutput, lastPage bool) bool) bool {
callback(&iam.ListUsersOutput{Users: []*iam.User{
{
UserName: aws.String("test-driftctl"),
},
{
UserName: aws.String("test-driftctl2"),
},
}}, true)
return true
})
).Once().Return(nil)
⚠️ If you have several mocks on the same method, the "mock" library will evaluate code in your MatchedBy multiple times even if the first parameter does not match.
It means your callback will always be called, this is an unwanted behaviour most of the time!
A workaround is to manage flags but this is an ugly solution, here is an example using a boolean flag:
mockCalled := false
client := mocks.FakeIAM{}
client.On("ListUsersPages",
&iam.ListUsersInput{},
mock.MatchedBy(func(callback func(res *iam.ListUsersOutput, lastPage bool) bool) bool {
if mockCalled {
return false
}
callback(&iam.ListUsersOutput{Users: []*iam.User{
{
UserName: aws.String("test-driftctl"),
},
{
UserName: aws.String("test-driftctl2"),
},
}}, true)
mockCalled = true
return true
})
).Once().Return(nil)
🙏 We are still looking for a better way to handle this, contributions are welcome.
References:
Acceptance testing
driftctl provides a kind of acceptance test framework (test/acceptance) to help you run those tests.
The goal here is to apply some terraform code, and then run a series of Check.
A Check consists of running driftctl and checking for results using json output.
driftctl uses assertion struct to help you check output results. See below for more details.
Each acceptance test should be prefixed by TestAcc_ and should be run using env var DRIFTCTL_ACC=true.
DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go
Credentials
Acceptance tests need credentials to perform real world action on cloud providers:
- Read/write access are required to perform terraform action
- Read only access is required for driftctl execution
Recommended way to run acc tests is to use two distinct credentials: one for terraform related actions, and one for driftctl scan.
In our acceptance tests, we may need read/write permissions during specific contexts
(e.g. terraform init, apply, destroy)or lifecycle (PreExec and PostExec).
If needed, you can override environment variables in those contexts by adding ACC_ prefix on env variables.
AWS
You can use ACC_AWS_PROFILE to override AWS named profile used for terraform operations.
ACC_AWS_PROFILE=read-write-profile AWS_PROFILE=read-only-profile DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go
In the example below, the driftctl AWS profile must have read/write permissions and will be used
for both terraform operations and driftctl run.
This is not the recommended way to run tests as it may hide permissions issues.
AWS_PROFILE=driftctl DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go
Workflow
OnStartYou may run some code before everything- terraform apply
- For each declared check loop
PreExec- driftctl scan
- check results
PostExec
OnEnd- terraform destroy
⚠️ driftctl tests handle terraform resources removal, but it is up to you to remove potential unmanaged resources added in PreExec step !
Example
The following test runs terraform to create an EC2 instance. Then, we add a new tag (ENV: production) to the instance. Finally, we check the drift.
func TestAcc_AwsInstance_WithBlockDevices(t *testing.T) {
var mutatedInstanceId string
acceptance.Run(t, acceptance.AccTestCase{
// This path should contain terraform files
Path: "./testdata/acc/aws_instance",
// Pass args to driftctl execution
// DO NOT PASS --output flag as it is handled automatically by test runner
// You may use a .driftignore file in your test directory or use filters to limit driftctl scope
// Try to be minimalist as possible as test will be easier to maintain over time
Args: []string{"scan"}, // TODO add filter to limit scan scope to aws_instances
Checks: []acceptance.AccCheck{
{ // First check does not have any PreExec or PostExec
Check: func(result *acceptance.ScanResult, stdout string, err error) {
if err != nil {
t.Fatal(err)
}
// Assert that no drift are detected
result.AssertDriftCountTotal(0)
// We could assert on analysis object directly
// Below we check for infra strictly in sync, beware that this check should fail
// if you run your acceptance test on a messy cloud provider state (existing dangling resources for example)
// without using filter or driftignore
//
// Note that the result struct is composed of analysis result AND assertion library
// You could use result.Equal() directly for example
result.True(result.Analysis.IsSync())
},
},
{
// In this PreExec, we retrieve the created instance ID and add a new tag
// using AWS SDK
// We store the instance ID in a var to assert on it after driftctl run
PreExec: func() {
client := ec2.New(awsutils.Session())
response, err := client.DescribeInstances(&ec2.DescribeInstancesInput{
Filters: []*ec2.Filter{
{
Name: aws.String("instance-state-name"),
Values: []*string{
aws.String("running"),
},
},
{
Name: aws.String("tag:Name"),
Values: []*string{
aws.String("test_instance_1"),
},
},
},
})
if err != nil {
t.Fatal(err)
}
if len(response.Reservations[0].Instances) != 1 {
t.Fatal("Error, unexpected number of instances found, manual check required")
}
mutatedInstanceId = *response.Reservations[0].Instances[0].InstanceId
_, _ = client.CreateTags(&ec2.CreateTagsInput{
Resources: []*string{&mutatedInstanceId},
Tags: []*ec2.Tag{
{
Key: aws.String("Env"),
Value: aws.String("Production"),
},
},
})
},
// Check that driftctl detected a drift on manually modified instances
Check: func(result *acceptance.ScanResult, stdout string, err error) {
if err != nil {
t.Fatal(err)
}
result.AssertResourceHasDrift(
mutatedInstanceId,
awsresources.AwsInstanceResourceType,
analyser.Change{
Change: diff.Change{
Type: diff.CREATE,
Path: []string{"Tags", "Env"},
From: nil,
To: "Production",
},
},
)
},
},
},
})
}