15.5 Multi-Attach: Sharing an io2 Volume Across Instances
Right, so you need a single block of storage that multiple EC2 instances can read and write to simultaneously. Your first thought is probably a network file system, and you’d be right 99% of the time. But what if your application is so deeply, pathologically tied to block-level semantics that NFS or Lustre just won’t cut it? What if you need sub-millisecond latency and can’t tolerate a filesystem protocol getting in the way? Enter Multi-Attach for io2 volumes. It’s the high-performance, high-stakes way to share a single disk across multiple instances in the same Availability Zone.
Let’s be brutally honest upfront: this is a specialist tool, not a everyday Swiss Army knife. The designers at AWS didn’t just bolt this on; they built it for a specific class of problems, like lifting and shifting a legacy clustered database (think Oracle RAC or some SQL Server configurations) that expects direct access to a shared block device. It’s brilliant for that. It’s a catastrophic foot-gun for almost everything else.
How It Actually Works (Without the Marketing Fluff)
You’re not magically sharing a physical disk. Under the hood, AWS is doing something clever but conceptually simple. When you enable Multi-Attach on an io2 volume, you’re creating a single provisioned block of storage, but the control plane creates multiple “block device endpoints”—one for each instance you attach it to.
All writes from any attached instance are synchronously replicated to every other attached instance’s view of the volume. Yes, you read that right: synchronous replication. This is the core of the entire mechanism and the source of both its power and its peril. A write isn’t considered committed until it’s landed on the volume and acknowledged by the block storage endpoints of every other attached instance. This guarantees data consistency but introduces a latency penalty. The more instances you attach, the worse this inter-node communication latency gets.
The Code: Enabling and Attaching
You can’t just take any old EBS volume and share it. It must be a provisioned IOPS io2 volume (or io2 Block Express), and you must enable Multi-Attach at creation time. You can’t enable it later. Here’s how you do it with the AWS CLI:
# Create the Multi-Attach enabled volume
aws ec2 create-volume \
--volume-type io2 \
--iops 1000 \
--size 100 \
--availability-zone us-east-1a \
--multi-attach-enabled
# Note the VolumeId from the output: vol-1234567890abcdef0
# Now attach it to your first instance (i-001)
aws ec2 attach-volume \
--volume-id vol-1234567890abcdef0 \
--instance-id i-001 \
--device /dev/sdf
# And attach the SAME volume to your second instance (i-002) in the same AZ
aws ec2 attach-volume \
--volume-id vol-1234567890abcdef0 \
--instance-id i-002 \
--device /dev/sdf
Notice that both instances see the device as /dev/sdf. That’s fine from AWS’s perspective, but what happens on the OS inside those instances is your problem to manage, which brings us to the most critical part.
The Monumentally Important Caveat: The Cluster-Aware Filesystem
This is the part the documentation glosses over, so lean in. You must use a cluster-aware filesystem like Oracle Cluster File System (OCFS2) or Veritas File System. Why? Because a regular filesystem like ext4 or XFS has absolutely no idea that another operating system is also writing to the same block device simultaneously.
If you just mount a standard Linux filesystem on both instances, you are inviting absolute, instantaneous, and spectacular data corruption. The filesystem’s metadata (inodes, journal, free space maps) will be clobbered by each instance, each thinking it has exclusive access. The result will be a charred, smoking ruin of data. The cluster-aware FS uses distributed locking mechanisms to coordinate access to the metadata, preventing this digital murder-suicide pact.
Performance and Pricing Realities
Remember that synchronous replication? It has a cost. Your write latency will be higher than on a standard io2 volume because it has to coordinate with every other attached instance. Your provisioned IOPS are also shared across all attached instances. If you provision 1000 IOPS, that’s the total throughput for the entire volume, not per instance. All attached instances are competing for that same I/O budget.
And on pricing? You pay for the volume itself, and you pay for the number of attachments. That’s right, Multi-Attach has a per-attachment-hour fee on top of the provisioned storage and IOPS. It’s an expensive feature, as it should be. This isn’t for fooling around.
When To Use This (And When To Run Away)
Use it for:
- Lifting and shifting a tightly-coupled legacy application that was built for a SAN.
- Specific high-performance computing (HPC) workloads that are designed for shared block access.
- Situations where you absolutely need block-level semantics and consistency across instances.
Run, don’t walk, away from it for:
- A simple shared file share (use EFS or FSx).
- A shared home directory (seriously, use EFS).
- Anything that doesn’t have built-in, robust coordination for concurrent block access.
It’s a powerful tool, but it demands respect. Use it wisely, and test your failure scenarios thoroughly in a non-production environment. You’ve been warned.