Making Optimal Deployment Choices for an Encoding System on AWS
by Karthikeyan N (Principal Engineer, Media Server Technologies)
Choose the right Amazon EC2 instance type and instance storage type to minimize OPEX and uniquely position your encoding workflow.
Global media companies and studios including Netflix, Amazon, 20th Century Fox and Warner Bros. are increasingly harnessing the cloud to deliver optimal quality video and audio streams in a scalable and easy manner. Amazon Web Services (AWS) has emerged as a popular choice for video encoding in public cloud systems. Even the largest streaming service – Netflix – has been deploying its encoding systems on AWS for the last five years to effectively scale its services.
THINKode, Ittiam’s ML based chunk adaptive encoder software, is also designed to run on AWS. In the process of deploying THINKode on AWS, we identified two key factors for optimizing the performance and deployment costs – the appropriate Amazon EC2 instance type and the Amazon instance storage type. These factors will critically influence the OPEX for any video encoding deployment on AWS.
AWS offers different instance families and types that vary in cost and capabilities. The process of selecting the optimal instance family for our use case can therefore be complex. To simplify our decision-making, we have been performing extensive tests and comparative studies to determine the most suitable option for encoding.
Take a look at our key findings.
Amazon EC2 instance family: Which one is for you?
We chose Compute Optimized (CO) instance family as it drives higher performance and is the best suitable for encoding tasks that are compute intensive. CO instances are clocked at a higher CPU frequency (2.9GHz) compared to other instances (~2.3GHz), and have higher ECUs per dollar.
Amazon EC2 instance families
- General Purpose (GP)
- Compute Optimized (CO)
- Memory Optimized (MO)
- Accelerated Computing Instances – GPU (ACI)
- Storage Optimized (SO)
The two groups of CO instances – C3 and C4 – comprise different types of instances based on their exact hardware configuration.
C3
- High frequency Intel Xeon E5-2680 v2 (Ivy Bridge) processors
- SSD storage and network storage (EBS)
C4
- High frequency Intel Xeon E5-2666 v3 (Haswell) processors, optimized specifically for EC2
- No SSD, only network storage (EBS)
Going for the right Amazon EC2 instance group
Having concluded that CO is the most suitable instance family for encoding, our next step was to select the appropriate instance group for deployment – C3 or C4. We performed profiling experiments on comparable instances from each group and compared the performance with respect to the following parameters, which are critical for encoding.
Cloud storage access speed: A source file was downloaded and the encoded output was egressed to the cloud storage. Our goal was to find out if the total access speed is adequate to maximize the encoding performance.
Disk throughput: The source file, the intermediate files generated as part of the encoding process, and the encoded output were stored in the instance storage. We then checked if the disk throughput was sufficient.
Encoding performance: The encoding time was used to measure the performance of the encoding system.
Take a look at the various options available within the C3 and C4 instance families and their characteristics.
| Model | vCPU | ECU | Memory (GiB) | Instance storage (GB) | Price |
|---|---|---|---|---|---|
| c3.large | 2 | 7 | 3.75 | 2 x 16 SSD | $0.105 per hour |
| c3.xlarge | 4 | 14 | 7.5 | 2 x 40 SSD | $0.21 per hour |
| c3.2xlarge | 8 | 28 | 15 | 2 x 80 SSD | $0.42 per hour |
| c3.4xlarge | 16 | 55 | 30 | 2 x 160 SSD | $0.84 per hour |
| c3.8xlarge | 32 | 108 | 60 | 2 x 320 SSD | $1.68 per hour |
| Model | vCPU | ECU | Memory (GiB) | Instance storage (GB) | Price |
|---|---|---|---|---|---|
| c4.large | 2 | 8 | 3.75 | EBS Only | $0.1 per hour |
| c4.xlarge | 4 | 16 | 7.5 | EBS Only | $0.199 per hour |
| c4.2xlarge | 8 | 31 | 15 | EBS Only | $0.398 per hour |
| c4.4xlarge | 16 | 62 | 30 | EBS Only | $0.796 per hour |
| c4.8xlarge | 36 | 132 | 60 | EBS Only | $1.591 per hour |
Based on the characteristics, we shortlisted c3.2xlarge and c4.2xlarge, both comparable in their processing capability, to represent their respective groups for profiling.
Cloud storage access speed
We used Amazon Simple Storage Service (Amazon S3) as the cloud storage type for content ingest/egress.
| Experiment results | Instance type | Download throughput | Upload throughput |
|---|---|---|---|
| Both the instance groups deliver almost the same performance for upload and download, to and from Amazon S3. | c4.2xlarge | ~119 MBps | ~135 MBps |
| c3.2xlarge | ~103 MBps | ~130 MBps |
Disk throughput
Amazon EC2 provides the following instance storage options.
Amazon EC2 Instance Store: Access storage from disks physically attached to the host computer.
Amazon Elastic Block Store (Amazon EBS): Durable, block-level storage volumes that can be attached to a running instance.
Additionally, Amazon EBS provides the following volume types, which differ in performance characteristics and price.
| Volume type | General purpose SSD (gp2)* | Provisioned IOPS SSD (io1) | Throughput optimized HDD (st1) | Cold HDD (sc1) |
|---|---|---|---|---|
| Volume size | 1 GiB – 16 TiB | 4 GiB – 16 TiB | 500 GiB – 16 TiB | 500 GiB – 16 TiB |
| Max. IOPS/volume | 10,000 | 20,000 | 500 | 250 |
| Max. Throughput/volume | 160 MiB/s | 320 MiB/s | 500 MiB/s | 250 MiB/s |
| Price | $0.10 per GB-month |
|
$0.045 per GB-month | $0.025 per GB-month |
Since the EBS volume type general purpose SSD (gp2) provides adequate performance at a lower price and hence the best balance between storage size, price and performance, we have used it further in our experiments.
| Experiment results | Source | Read throughput | Write throughput |
|---|---|---|---|
| SSD provides more than 3.3x read throughput than EBS. However, SSD provides inconsistent write throughput, while EBS provides consistent throughput.
Since inconsistent write throughput can often become a performance bottleneck, EBS emerges as the optimal instance storage type for us. |
EBS | ~120 MBps | ~119 MBps |
| SSD | ~393 MBps | 50 to 120 MBps (fluctuating) |
Encoding performance
| Experiment results | Instance type | Encoding time |
|---|---|---|
The encoding performance on c4.2xlarge is 1.15x that of c3.2xlarge at similar cost. The C4 instance group thus emerges as the right choice for encoding. |
c4.2xlarge | 13min 12sec |
| c3.2xlarge | 15min 11sec |
Our profiling experiments for all the parameters show that C4 instance group combined with EBS instance storage type will enable the best balance of performance and cost.
| Parameters | Experiment results | Optimal choice |
|---|---|---|
| File storage access speed | Upload/download speed to/from S3 is almost the same for both instance groups | C3 or C4 |
| Disk throughput | SSD write throughput is inconsistent, while EBS provides consistent throughput | EBS storage. C3 or C4 (since both offer the EBS option) |
| Encoding performance | Performance on c4.2xlarge is 1.15x that on c3.2xlarge at similar cost | C4 |
Choosing the exact C4 instance type
Our final task was to choose the exact instance type by measuring the encoding performance on different C4 instance types. We anticipated that the multi-core scalability of all the software elements in our workflow would affect the performance on high core count instances. In our experiments, we found a near-linear increase in performance from C4.large to C4.2xlarge. With the price also increasing linearly, C4.2xlarge seemed a good base to compare against even higher performance variants.
| Experiment results | Instance type | Encoding time |
|---|---|---|
c4.4xlarge enables 1.46x more encoding performance than c4.2xlarge. However, its cost is 2x than c4.2xlarge. c4.2xlarge instance type thus offers the best combination of price and performance. |
c4.2xlarge | 13min 12sec |
| c4.4xlarge | 9min 3sec |
Conclusion
Based on our comparative tests, we conclude the following are the best deployment options on AWS for encoding:
Amazon EC2 instance family: Compute Optimized (CO), Group: C4, Type: c4.2xlarge
Amazon instance storage: EBS, Volume Type: General Purpose SSD (gp2)
For more insights, reach out to us @mkt@www.ittiam.com