Right, so you’ve decided to automate your deployment process. Good for you. Manually dragging and dropping files onto a server is a fantastic way to spend an afternoon you’ll never get back, and we’re not doing that anymore. Welcome to AWS CodePipeline, the service that strings together your other services into something resembling a proper CI/CD conveyor belt. Think of it as the grumpy, pedantic foreman on your digital factory floor. It doesn’t do the work itself, but it stands there with a clipboard, yelling at CodeBuild to compile your code and telling CodeDeploy where to shove the resulting artifact.

Its entire job is orchestration. You give it a blueprint—a series of stages and actions—and it will execute them in order, passing the output from one step to the next, and generally making sure everything happens in the right place at the right time. When it works, it’s glorious. When it fails, you’ll want to strangle its digital ghost. Let’s make sure it’s the former.

The Anatomy of a Pipeline

A pipeline is just a fancy JSON document (they call it a structure) that defines a linear sequence of events. It’s broken down into stages, and each stage contains one or more actions. A stage is a logical division, like “Source,” “Build,” or “Deploy.” An action is the actual task being performed within that stage, like “pull code from this GitHub repo” or “run this buildspec.yml file.”

The magic glue that holds it all together is the artifact. An artifact is simply a package of files, a ZIP archive, that gets passed from one action to the next. The Source action produces an artifact (your source code). The Build action consumes that source artifact and (if you’ve set it up right) produces a new, built artifact (like a JAR file or a Docker image). The Deploy action then consumes that built artifact. CodePipeline’s job is to manage the storage and hand-off of these artifacts between stages using Amazon S3.

Defining the Beast: A CloudFormation Example

While you can click through the console, you’re a professional, so we’re defining this as code. Here’s a CloudFormation snippet that defines a simple pipeline pulling from GitHub, building with CodeBuild, and—for the sake of this example—dropping the output into an S3 bucket for deployment. Note the ArtifactStore section; this is the central S3 bucket where all the intermediate files are kept.

Resources:
  MyAppPipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineServiceRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref ArtifactStoreBucket
      Stages:
        - Name: Source
          Actions:
            - Name: GitHub-Source
              ActionTypeId:
                Category: Source
                Owner: ThirdParty
                Version: '1'
                Provider: GitHub
              Configuration:
                Owner: my-github-username
                Repo: my-awesome-repo
                Branch: main
                OAuthToken: '{{resolve:secretsmanager:my/github/token::SecretString:token}}' # Seriously, use Secrets Manager
              OutputArtifacts:
                - Name: SourceOutput
              RunOrder: 1

        - Name: Build
          Actions:
            - Name: CodeBuild-Action
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: '1'
                Provider: CodeBuild
              Configuration:
                ProjectName: !Ref MyAppBuildProject
              InputArtifacts:
                - Name: SourceOutput
              OutputArtifacts:
                - Name: BuildOutput
              RunOrder: 1

        - Name: Deploy-To-S3
          Actions:
            - Name: S3-Deploy
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: '1'
                Provider: S3
              Configuration:
                BucketName: !Ref MyWebsiteBucket
                Extract: true # Crucial: unzips the build artifact before uploading
              InputArtifacts:
                - Name: BuildOutput
              RunOrder: 1

The Crucial Gotchas and Best Practices

  1. The IAM Nightmare: CodePipeline needs a terrifyingly broad set of permissions to assume roles, trigger builds, and push artifacts. The console will offer to create a default role for you, which is a great way to learn what the iam:PassRole permission does at 2 a.m. Define your roles explicitly in CloudFormation or Terraform. The principle of least privilege is your only friend here.

  2. Artifact Names are Your Anchor: Notice the InputArtifacts and OutputArtifacts sections. The Name you define here is how you reference this specific package of files in subsequent stages. Misspell this, and your pipeline will fail with an error message that’s unhelpful at best. Be consistent.

  3. The Silent Failure of Bad Buildspecs: The most common point of failure is the buildspec.yml file in your CodeBuild stage. If CodeBuild fails, CodePipeline just shrugs and turns the stage red. The real logs—the detailed, “why did my npm install fail” logs—are in CodeBuild. Get in the habit of clicking through to the CodeBuild project execution details to find the real error. Never assume the Pipeline UI is telling you the whole story.

  4. Manual Approvals Are a Trap: You can add a manual approval action. It sends an email. This is a fantastic way to bring all automation to a screeching halt until someone checks their spam folder. If you need a gating mechanism, use a automated test suite in a stage or integrate with a proper chatOps tool like Slack.

  5. Version Everything: The Version: '1' in the ActionTypeId isn’t a suggestion. AWS updates these action types, and if you leave it blank or use an outdated version, your pipeline might just stop working one Tuesday after an AWS update. Pin your versions. Always.

The real power—and complexity—comes when you start adding parallel actions (like running unit tests and integration tests at the same time) or branching pipelines based on conditions. But start here. Get this linear flow working rock-solid first. Because a foreman is only as good as the blueprint you give him. And this one hates typos.