Medusa vs Magento: Performance comparison

Grzegorz Tomaka
March 29, 2024
Let’s build together!

Let’s talk about how we can build your commerce project — tailored to your business, powered by Rigby

Table of contents

Medusa is 6.5x faster than Magento! Find out in the article how we checked it.

Magento is an old player in the eCommerce field, known for its comprehensive but complex system. Recently, it's shifted towards a headless approach, trying to keep up with the modern digital market. But does this shift put it on par with newer, API-first platforms like Medusa, designed from the ground up to be flexible and efficient?

This comparison is about seeing if Magento, with its new headless approach, can match the performance of platforms built to be headless from day one.

We’ve also prepared a whitepaper that compares additional factors between Magento and Medusa. If you're considering a transition from Magento, this guide covers everything from platform comparisons to migration strategies.

Check it out - Migration from Magento to Medusa: A Complete Guide for CTO

Migration from Magento to Medusa Whitepaper

What is Medusa?

medusa-strona

Medusa is an open-source, headless commerce platform built with modern JavaScript. It's designed to suit any eCommerce need, including complex setups like multi-vendor marketplaces. Its API-first approach and decoupled architecture enable rapid development and scalability.

What is Magento (Adobe Commerce)?

magento-strona

Magento, now Adobe Commerce, is a leading open-source e-commerce platform known for its extensive feature set. It supports a headless architecture, allowing businesses to create custom front-end experiences while leveraging Magento's powerful e-commerce capabilities on the backend.

What We Compare

In our comparison of Medusa and Magento, we're diving deep into API performance, focusing on how efficiently each platform handles data requests. API performance is crucial because it directly impacts the speed and responsiveness of an eCommerce site, affecting everything from user experience to SEO rankings.

During API performance testing, we will measure average and 95th percentile load time based on http_req_duration.

http_req_duration

To compare Medusa and Magento's API performance, we focus on the metric http_req_duration This metric represents the total time it takes for an HTTP request to complete, from the initial request to the final byte of the response. It encompasses all aspects of the request cycle, making it a comprehensive measure of the efficiency of your e-commerce platform's backend.

The Importance of Average Loading Time and the 95th Percentile

  • Average Loading Time: This metric gives us a general idea of how long it typically takes for an API call to load across all tests. It's helpful in establishing a baseline performance but can sometimes hide issues like occasional delays or outliers.
  • 95th Percentile: Instead of just looking at averages, we focus on the 95th percentile to better understand performance under load. This metric helps us see how the platform performs under stress, excluding the most extreme outliers. It's crucial for e-commerce because it reflects the experience of nearly all users.

Why is Loading Speed Important?

Loading speed is crucial because even minor delays can drastically lower conversion rates and sales. Fast-loading sites meet user expectations, improve SEO rankings, and are more likely to keep users engaged. In short, a quicker website directly translates to a better user experience and increased revenue.

To learn more, check out our other article - "How to Increase Sales in the Online Store?"

Test Cases: Why Focus on Product Fetching?

We've focused on fetching product data as a primary test case for our comparison. This decision is based on the premise that viewing product information is often the first step in a customer's journey to purchase. Efficiently loading product details keeps potential buyers engaged and reduces bounce rates.

How We Conduct the Test

Here's how we're conducting our performance test:

Tool k6 for Load Testing

For our testing, we're using k6, a powerful open-source load-testing tool. k6 is built to simulate real-world traffic on web applications, providing insights into how systems perform under various conditions. k6 allows us to simulate traffic and examine API performance.

To ensure our test results focus purely on server response times and are not influenced by network latency variations, we've set up our k6 testing environment directly on the server.

Testing Environment: Uniform Server Specifications

We're running both Medusa and Magento on identical server setups to eliminate hardware as a variable. Each platform is hosted on a DigitalOcean Droplet, with 4 GiB of RAM and 2 virtual CPUs. This specification ensures that both platforms have equal resources, making our performance comparison as fair as possible.

No Cache Systems

To assess each platform's real performance, we're conducting our tests without any cache systems in place. This approach allows us to measure the true backend performance of Medusa and Magento without the influence of caching mechanisms that could skew the results.

Product Database

Both platforms are populated with 1,000 identical products. This ensures that our tests compare like for like, with both systems handling an equal amount of data during the test scenarios.

Deployment

MedusaWe used Medusa version 1.20.2 with Node 18.20.0 on our server using Docker.

MagentoWe used Magento Open Source 2.4.6-p4, running on PHP 8.1 and Apache

Types of Tests

To evaluate Medusa and Magento's performance, we've designed a test series that simulates different user load scenarios. Each scenario is applied to both platforms and across two distinct types of requests:

  • Single product retrieval
  • Fetching 15 Products with an Offset of 500.

This approach allows us to assess how each platform handles varying degrees of demand and specific data retrieval tasks.

Test Scenarios

  1. 50 Iterations with 1 User: This test simulates a user making 50 consecutive requests. It's designed to assess how the platforms perform under a light, consistent load, providing insight into their efficiency in handling individual user actions.
  2. 10 Users Simultaneously (for 30s): By simulating 10 users accessing the platform simultaneously, this test evaluates the platform's ability to handle multiple requests concurrently. It's a basic concurrency test, reflecting a realistic scenario for small to medium-sized eCommerce sites during low to moderate-traffic periods.
  3. 25 Users Simultaneously (for 30s): This scenario significantly increases the concurrency, testing the platforms' performance under a medium load. It simulates a more intense shopping period, such as a sale event, during which many users browse and interact with the site simultaneously.
  4. 50 Users Simultaneously (for 30s): The most demanding test, this simulates a high-traffic event, challenging the platforms' scalability and resource management under heavy load. It's indicative of peak shopping times, such as significant sales or holiday shopping rushes.

Requests Tested

  • Single Product Retrieval: Measures the platforms' response times when a user requests information for a single product.
  • Fetching 15 Products with an Offset of 500: Evaluates the platforms' performance in paginated data retrieval, a common scenario for users browsing through large collections of products.

Test: Single Product Retrieval

Magento endpoint

/rest/V1/products/[product_sku]

Medusa endpoint

/store/products/[prod_id]

50 Iterations with 1 User

a. Magento

k6-1
Metric Value
avg 184.14 ms
min 146.24 ms
med 185.6 ms
max 293.22 ms
p(90) 202.21 ms
p(95) 214.25 ms

b. Medusa

k6-2
Metric Value
avg 29.88 ms
min 17.89 ms
med 26.44 ms
max 80.61 ms
p(90) 39.37 ms
p(95) 49.52 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 184.14 ms 29.88 ms 6.16
min 146.24 ms 17.89 ms 8.17
med 185.6 ms 26.44 ms 7.02
max 293.22 ms 80.61 ms 3.64
p(90) 202.21 ms 39.37 ms 5.14
p(95) 214.25 ms 49.52 ms 4.33
HTTP Request Duration (ms) - Single Product [Test - 50 Iterations with 1 User].svg

10 users simultaneously for 30 s

a. Magento

k6-3
Metric Value
avg 1080 ms
min 192.96 ms
med 1060 ms
max 4980 ms
p(90) 1200 ms
p(95) 1290 ms

b. Medusa

k6-4
Metric Value
avg 154.21 ms
min 56.6 ms
med 149.56 ms
max 410.13 ms
p(90) 194.78 ms
p(95) 214.06 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 1080 ms 154.21 ms 7.00
min 192.96 ms 56.6 ms 3.41
med 1060 ms 149.56 ms 7.09
max 4980 ms 410.13 ms 12.14
p(90) 1200 ms 194.78 ms 6.16
p(95) 1290 ms 214.06 ms 6.03
HTTP Request Duration (ms) - Single Product [Test - 10 users simultaneously for 30 s] (1).svg

25 users simultaneously for 30 s

a. Magento

k6-5
Metric Value
avg 2880 ms
min 1290 ms
med 2580 ms
max 7060 ms
p(90) 3300 ms
p(95) 6150 ms

b. Medusa

k6-6
Metric Value
avg 416.93 ms
min 243.46 ms
med 400.08 ms
max 955.38 ms
p(90) 506.95 ms
p(95) 558.85 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 2880 ms 416.93 ms 6.91
min 1290 ms 243.46 ms 5.30
med 2580 ms 400.08 ms 6.45
max 7060 ms 955.38 ms 7.39
p(90) 3300 ms 506.95 ms 6.51
p(95) 6150 ms 558.85 ms 11.01
HTTP Request Duration (ms) - Single Product [Test - 25 users simultaneously for 30 s].svg

50 users simultaneously for 30 s

a. Magento

k6-7
Metric Value
avg 5730 ms
min 2750 ms
med 5430 ms
max 9950 ms
p(90) 8660 ms
p(95) 9100 ms

b. Medusa

k6-8
Metric Value
avg 936.39 ms
min 494.78 ms
med 884.96 ms
max 2570 ms
p(90) 1150 ms
p(95) 1240 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 5730 ms 936.39 ms 6.12
min 2750 ms 494.78 ms 5.56
med 5430 ms 884.96 ms 6.13
max 9950 ms 2570 ms 3.87
p(90) 8660 ms 1150 ms 7.53
p(95) 9100 ms 1240 ms 7.34
HTTP Request Duration (ms) - Single Product [Test - 50 users simultaneously for 30 s].svg

Test: Fetching 15 Products with an Offset of 500

Magento endpoint

/rest/V1/products?searchCriteria[pageSize]=15&searchCriteria[currentPage]=34

Medusa endpoint

/store/products?limit=15&offset=500

50 Iterations with 1 User

a. Magento

k6-9
Metric Value
avg 441.35 ms
min 357.93 ms
med 442.7 ms
max 545.1 ms
p(90) 483.18 ms
p(95) 490.76 ms

b. Medusa

k6-10
Metric Value
avg 62.15 ms
min 38.97 ms
med 56.64 ms
max 203.11 ms
p(90) 72.19 ms
p(95) 82.08 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 441.35 ms 62.15 ms 7.10
min 357.93 ms 38.97 ms 9.18
med 442.7 ms 56.64 ms 7.81
max 545.1 ms 203.11 ms 2.68
p(90) 483.18 ms 72.19 ms 6.69
p(95) 490.76 ms 82.08 ms 5.98
HTTP Request Duration (ms) - 15 Products [Test - 50 Iterations with 1 User].svg

10 users simultaneously for 30 s

a. Magento

k6-11
Metric Value
avg 2160 ms
min 473.95 ms
med 2060 ms
max 6620 ms
p(90) 2360 ms
p(95) 2610 ms

b. Medusa

k6-12
Metric Value
avg 460.23 ms
min 166.26 ms
med 439.68 ms
max 1480 ms
p(90) 597.98 ms
p(95) 656.33 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 2160 ms 460.23 ms 4.69
min 473.95 ms 166.26 ms 2.85
med 2060 ms 439.68 ms 4.69
max 6620 ms 1480 ms 4.47
p(90) 2360 ms 597.98 ms 3.95
p(95) 2610 ms 656.33 ms 3.98
HTTP Request Duration (ms) - 15 Products [Test - 10 users simultaneously for 30 s].svg

25 users simultaneously for 30 s

a. Magento

k6-13
Metric Value
avg 5490 ms
min 3260 ms
med 5090 ms
max 9660 ms
p(90) 8540 ms
p(95) 9070 ms

b. Medusa

k6-14
Metric Value
avg 1160 ms
min 759.49 ms
med 1110 ms
max 2670 ms
p(90) 1350 ms
p(95) 1430 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 5490 ms 1160 ms 4.73
min 3260 ms 759.49 ms 4.29
med 5090 ms 1110 ms 4.59
max 9660 ms 2670 ms 3.62
p(90) 8540 ms 1350 ms 6.33
p(95) 9070 ms 1430 ms 6.34
HTTP Request Duration (ms) - 15 Products [Test - 25 users simultaneously for 30 s].svg

50 users simultaneously for 30 s

a. Magento

k6-15
Metric Value
avg 14200 ms
min 6810 ms
med 12690 ms
max 21290 ms
p(90) 19940 ms
p(95) 20330 ms

b. Medusa

k6-16
Metric Value
avg 1960 ms
min 489.28 ms
med 1890 ms
max 3700 ms
p(90) 2260 ms
p(95) 2900 ms

c. Magento vs Medusa

http_req_duration Magento Medusa How many times faster is Medusa?
avg 14200 ms 1960 ms 7.24
min 6810 ms 489.28 ms 13.92
med 12690 ms 1890 ms 6.72
max 21290 ms 3700 ms 5.76
p(90) 19940 ms 2260 ms 8.82
p(95) 20330 ms 2900 ms 7.01
HTTP Request Duration (ms) - 15 Products [Test - 50 users simultaneously for 30 s].svg

Summary

Our performance comparison between Magento and Medusa APIs shows Medusa's superior speed. Even though responses from Medusa's API are heavier regarding data size, Medusa's response times are undeniably much faster across the board.

In tests simulating multiple users accessing the system simultaneously, we observed a significant difference in response times and the volume of data processed. The Medusa API allowed us to execute more requests in the same timeframe, fetching more data.

95th percentile (p(95)) times from all tests reveals the significant differences in performance:

p(95) Magento Medusa How many times faster is Medusa?
Single Product - 50 iterations with 1 user 214.25 ms 49.52 ms 4.33
Single Product - 10 users simultaneously for 30s 1290 ms 214.06 ms 6.03
Single Product - 25 users simultaneously for 30s 6150 ms 558.85 ms 11.01
Single Product - 50 users simultaneously for 30s 9100 ms 1240 ms 7.34
15 Products - 50 iterations with 1 user 490.76 ms 82.08 ms 5.98
15 Products - 10 users simultaneously for 30s 2610 ms 656.33 ms 3.98
15 Products - 25 users simultaneously for 30s 9070 ms 1430 ms 6.34
15 Products - 50 users simultaneously for 30s 20330 ms 2900 ms 7.01
ALL TEST AVERAGE 6156.88 ms 891.36 ms 6.50
  • For single product fetches across 50 iterations with a single user, Medusa was 4.33 times faster than Magento.
  • When the load increased to 10, 25, and 50 users simultaneously for 30 seconds, Medusa consistently outperformed Magento, being 6.03, 11.01, and 7.34 times faster, respectively.
  • The performance advantage extended to fetching 15 products simultaneously, with Medusa being 5.98 to 7.01 times faster across different user loads.

Medusa outperforms Magento, being 6.5 times faster on the same server setup. This means Medusa can efficiently handle many more requests and users with the same server resources.

In today's market, every millisecond counts towards making a sale. Choosing a fast eCommerce backend like Medusa could be the key to staying ahead.

Got a project in mind? Let’s talk

Jacob Zbąski
Co-founder & CEO

“We build engines for growth, tailored to how your business actually works. Let’s talk about how we can help bring your vision to life.”

Jacob Zbąski
Co-founder & CEO