MongoDB Aggregation Guide (with example)

This article will cover the aggregate queries in MongoDB and how to take advantage of indexes for speeding up the queries.

In MongoDB typically we use find() command for most of the queries. But to deal with large number of documents by applying filters, sorting, groups, reshaping and modifying the documents we need a systematic approach and there we use aggregation framework.

This article covers:

1. What is Aggregation in MongoDB?
2. Stages of MongoDB aggregation pipeline.
3. Aggregation pipeline syntax.
4. Stage limits of an aggregation pipeline.
5. Document Size Limit.
6. Aggregation pipeline example.

1) What is Aggregation in MongoDB?

Aggregation is a way of processing a large number of documents in a collection by means of passing them through different stages. These stages are also known as Pipeline.

The stages in a pipeline can be filter , sort , group , reshape and modifying the documents.

The most common usecase of using Aggregation is to calculate aggregare values for groups of documents. This is very much similar to the aggregation available in SQL with the GROUP BY clause and COUNT, SUM and AVG functions. It can also perform relational-like joins, reshape documents, create new and update them.

There are other methods too for obtaining aggregate data in MongoDB, but the aggregation framework is the most recommended approach.

There are single purpose methods like estimatedDocumentCount() , count() , and distinct() which can be appended to a find() query. These are quick to use but has limited scope.

2) Stages of MongoDB aggregation pipeline:

• $match stage — filters documents → like a WHERE in SQL.
• $group stage — aggregate data → like SQL’s GROUP BY.
• $sort stage — arranges documents → like SQL’s ORDER BY.

They are used in sequence to prcess and transform data efficiently in MongoDB queries.

The input of the pipeline can be a single collection, where other collection can be merged later down the pipeline. Then pipeline performs transformations on the data untill our goal is achieved.

This is the way to break down a complex query into easier stages, in each of the stage we complete a different operation on the data. So, by the end of the query pipeline we acheived a desired outcome.

This approach allows us to check how our query is functioning at every stage. We can examine both input and output at every stage. The output of each stage will be the input of next stage.

There are no limit on the number of stages used in the query, or how we combine them. There are best practices to be followed for achieving optimum query performance.

Bitveen

3) Aggregation pipeline syntax:

db.collectionName.aggregate(pipeline, options)

• Where collectionName — is the name of a collection.
• pipeline — is an array that contains the aggregation stages.
• options — is an optional object that allows us to control how the aggregation behaves.

Here is the structure of an aggregation pipeline syntax:

pipeline = [
  { $match : { ... } },
  { $group : { ... } },
  { $sort  : { ... } }
]

4) Stage limits of an aggregation pipeline:

Aggregation uses memory. Each stage can use upto 100 MB of RAM. We can get an error from the database if we exceed this limit.

The solution is to use page to disk, we have to wait little longer is the only disadvantage of using page to disk because it is slower to work on the disk rather than in memory.

Example of using page to disk is by setting allowDiskUse to true like this:

db.collectionName.aggregation(pipeline, { allowDiskUse: true })

Note: for shared services allowDiskUse option is not always available. For example: Atlas M0, M2, and M5 clusters disable this option.

5) Document Size Limit:

📌 MongoDB has a maximum size limit of 16MB per document. Even if we use aggregation query, this rule still applies.

So, What’s the issue?
If our aggregation creates a big document, like we are trying to combine a lot of data into one single result, we might hit that 16MB limit — and our query will fail.

✅ Safe Way: Let MongoDB returns many small documents.

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } }
])

// 📌 The output gives us one document per customer, like:

[
  { "_id": "cust1", "total": 1200 },
  { "_id": "cust2", "total": 800 }
]

/* Each document is small, we're safe. 
   MongoDB returns them as a cursor (like a list we can scroll through).
   This is the default behaviour.
*/

🧐 In this query, some places using $ and others not, why is it so?
Answer: Use $ before field names (like “$amount” , $customerId) when reffering to fields in our documents.
Don’t use $ when writing custom field names or values in our output documents.

| When writing           | Use `$`? | Example                     |
|------------------------|----------|-----------------------------|
| MongoDB operator       | ✅ Yes   | `$match`, `$group`, `$sum`  |
| Referencing a field    | ✅ Yes   | `"$amount"`, `"$status"`    |
| Naming your output     | ❌ No    | `total`, `count`, `status`  |
| Matching a value       | ❌ No    | `{ status: "shipped" }`     |

❌ Risky Way: Trying to combine everything into one single document.

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: null, allOrders: { $push: "$$ROOT" } } }
])

/* 
  📌 This groups all shipped orders into a single document and 
     Puts all orders inside an array called allOrders.
     Output like:
*/

{
  "_id": null,
  "allOrders": [
    {
      "_id": ObjectId("..."),
      "customerId": "cust1",
      "amount": 500,
      "status": "shipped"
    },
    {
      "_id": ObjectId("..."),
      "customerId": "cust2",
      "amount": 300,
      "status": "shipped"
    },
    // ... and thousands more like this
  ]
}

🛑 If this allOrders array grows too big (over 16MB), MongoDB will throw an error: “ BSONObj size must be <= 16793600 ”

❌ Also Risky: When using $out or $merge. These stages write the output into another collection instead of returning it.

// Using $out:
db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: null, allOrders: { $push: "$$ROOT" } } },
  { $out: "summary" }
])

// Using $merge:
db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: null, allOrders: { $push: "$$ROOT" } } },
  { $merge: { into: "summary", whenMatched: "merge", whenNotMatched: "insert" } }
])

🧐 How $out and $merge works?
MongoDB tries to store that big document (with thousands of orders) inside a new collection called summary.

Output of the summary collection (if it does not goes beyond limit 16MB)

{
  "_id": null,
  "allOrders": [
    { "_id": "...", "customerId": "cust1", "amount": 500, "status": "shipped" },
    { "_id": "...", "customerId": "cust2", "amount": 300, "status": "shipped" },
    ...
  ]
}

🛑 Again — if the size of this document goes beyond 16MB, the write to summary collection will fail.

Bitveen

6) Aggregation pipeline example.

Two collections: universities and courses are being used in the example, with the below documents (where data is unreal).

// First collection: universities,
// We can insert the data into the collection using below command:
db.universities.insertMany([
  {
    country : 'Spain',
    city : 'Salamanca',
    name : 'USAL',
    location : {
      type : 'Point',
      coordinates : [ -5.6711512,17, 40.9605592 ]
    },
    students : [
      { year : 2013, number : 24774 },
      { year : 2014, number : 23166 },
      { year : 2015, number : 21913 },
      { year : 2016, number : 21715 }
    ]
  },
  {
    country : 'Spain',
    city : 'Salamanca',
    name : 'UPSA',
    location : {
      type : 'Point',
      coordinates : [ -5.4491191,17, 40.9621732 ]
    },
    students : [
      { year : 2013, number : 4788 },
      { year : 2014, number : 4821 },
      { year : 2015, number : 6550 },
      { year : 2016, number : 6125 }
    ]
  }
])

// Second collection: courses,
// We can insert the data into the collection using below command:

db.courses.insertMany([
  {
    university : 'USAL',
    name : 'Computer Science',
    level : 'Excellent'
  }
  {
    university : 'USAL',
    name : 'Electronics',
    level : 'Intermediate'
  }
  {
    university : 'USAL',
    name : 'Communication',
    level : 'Excellent'
  }
])

⭐ MongoDB `$match`

The $match stage of aggregation allows us to filter the data, and get documents which matches the given condition criteria.

For example, if we want to work with data in which country is Spain and city is Salamanca the below query will be applied:

db.universities.aggregate([
  {
    $match : { country: "Spain", city: "Salamanca" }
  }
])

The output is an array of documents (objects) containing all the fields as below:

[
  {
    country : 'Spain',
    city : 'Salamanca',
    name : 'USAL',
    location : {
      type : 'Point',
      coordinates : [ -5.6711512,17, 40.9605592 ]
    },
    students : [
      { year : 2013, number : 24774 },
      { year : 2014, number : 23166 },
      { year : 2015, number : 21913 },
      { year : 2016, number : 21715 }
    ]
  },
  {
    country : 'Spain',
    city : 'Salamanca',
    name : 'UPSA',
    location : {
      type : 'Point',
      coordinates : [ -5.4491191,17, 40.9621732 ]
    },
    students : [
      { year : 2013, number : 4788 },
      { year : 2014, number : 4821 },
      { year : 2015, number : 6550 },
      { year : 2016, number : 6125 }
    ]
  }
]

⭐ MongoDB `$project`

It is very rare that we need to retrieve all fields of documents. So it is good practice to return only those fields which are required to avoid processing unnecessary data.

The $project stage is used to fetch only necessary fields.

We will take an example where we need the fields: country , city and name.

• We must explicitly write _id : 0 when the field is not required to fetch. If we don’t explicitly define the _id will come as part of result.
• Apart from _id fild, we can specify fieldName : 1 for the fields we want to fetch.

// Aggregate query example of using $project:
db.universities.aggregate([
  { $project : { 
      _id : 0,
      country: 1,
      city : 1,
      name : 1
   }
]).pretty()

📌 Notes : .pretty() is only used in the MongoDB shell or command line interfaces like mongosh. It does not affect the results or performance — only how they’re displayed.

This will return an array of object(s) in response, as follows:

[
  { "country" : "Spain", "city" : "Salamanca", "name" : "USAL" }
  { "country" : "Spain", "city" : "Salamanca", "name" : "UPSA" }
]

⭐ MongoDB `$group`

Using the $group stage, we can perform all aggregation queries, such as finding counts, totals, averages, or maximums.

In the example below, we want to count the number of documents per university in our universities collection.

The query will be:

// aggregate query to get total counts of university using `$group`
db.universities.aggregate([
  { $group : { 
      _id : '$name',
      totalDocs : { $sum : 1 }
    } 
  }
])

This will return an array of object(s) with _id containing University name and totalDocs is sum of documents count with that University.

[
  { _id: 'USAL',  totalDocs: 1 }
  { _id: 'UPSA',  totalDocs: 1 }
]

⭐ `$group` aggregation operators:

The $group stage of an aggregation pipeline supports certain expressions (operators) that allow users to perform arithmetic, array, boolean and other operations.

Following operators can be used in $group stage:

| Operator | Meaning                                                                 |
|----------|-------------------------------------------------------------------------|
| `$count` | Counts the number of documents in each group.                          |
| `$max`   | Returns the highest value of a field in each group.                    |
| `$min`   | Returns the lowest value of a field in each group.                     |
| `$avg`   | Calculates the average of a field in each group.                       |
| `$sum`   | Adds up values of a field in each group.                               |
| `$push`  | Adds values to an array field in the result.                           |

⭐ MongoDB `$out`

$out stage allows us to carry results of aggregation over into a new collection, or into an existing one. Using $out we can save the results of aggregation:

• Into a new collection (which will be created),
  { $out: “newOrders” } (creats a new collection newOrders with the results.)
• Replacing an existing collection,
  { $out: "ordersArchive" } (if "ordersArchive" exists it deletes existing collection, then writes new data into it.)
• Or add to an existing collection (without deleting what’s already there).
  db.orders.aggregate([… , { $merge: “orders” }])

We can think of $out like "Save As" in MongoDB — either we are creating something new or overwriting/replacing something that already exists.

The $out stage must be the ‘last’ stage in the pipeline.

Following query is using aggregation with more than one stage ($group and $out stage).

db.universities.aggregate([
  {
    $group : { 
      _id : '$name',
      totalDocs : { $sum : 1 }
    }
  },
  {
    $out : 'aggResults'
  }
])

Now, the content of the new aggResults collection is:

db.aggResults.find();

// the output of the query:
[
  { _id : 'UPSA',  'totalDocs' : 1 },
  { _id : 'USAL',  'totalDocs' : 1 }
]

⭐ MongoDB `$unwind`

The $unwind stage is used in MongoDB to break apart array fields, creating a separate document for each element in the array.

This is useful when you want to process or group individual array items (which stages like $group can’t do directly on arrays).

Example of Input document:

{
  country: "Spain",
  city: "Salamanca",
  name: "USAL",
  location: {
    type: "Point",
    coordinates: [ -5.6722512, 17, 40.9607792 ]
  },
  students: [
    { year: 2014, number: 24774 },
    { year: 2015, number: 23166 },
    { year: 2016, number: 21913 },
    { year: 2017, number: 21715 }
  ]
}

Aggregation pipeline with `$unwind`

db.universities.aggregate([
  { $match: { name: "USAL" } },
  { $unwind: "$students" }
]).pretty()

This will:

• Matches the document where name is USAL.
• Unwinds the students array.
• Returns 4 documents, one for each student/year.

Sample Output (2 of 4):

{ name: "USAL", country: "Spain", students: { year: 2014, number: 24774 } }
{ name: "USAL", country: "Spain", students: { year: 2015, number: 23166 } }

Each resulting document keeps the same fields (like country, city, name), but students now holds only one array item at a time.

⭐ MongoDB `$sort`

The $sort stage is used to order documents in the aggregation pipeline based on the value of one or more fields.

It is typically used after filtering ($match), projecting ($project), or deconstructing arrays ($unwind) to control the result order.

| Purpose             | Description                                                           |
| ------------------- | --------------------------------------------------------------------- |
| Sort by field value | Orders documents by one or more fields.                               |
| Ascending sort      | Use `1` to sort from lowest to highest.                               |
| Descending sort     | Use `-1` to sort from highest to lowest.                              |
| Combine with stages | Often used with `$match`, `$project`, and `$unwind` to format output. |

Example — Sort a university’s students by number (descending):

db.universities.aggregate([
  { $match: { name: "USAL" } },
  { $unwind: "$students" },
  { $project: { _id: 0, "students.year": 1, "students.number": 1 } },
  { $sort: { "students.number": -1 } }
])

Output:

{ "students": { "year": 2014, "number": 24774 } }
{ "students": { "year": 2015, "number": 23166 } }
{ "students": { "year": 2016, "number": 21913 } }
{ "students": { "year": 2017, "number": 21715 } }

⭐ MongoDB `$limit`

The $limit stage restricts the number of documents that pass through the pipeline. It's commonly used after $sort to return only the top N results.

| Purpose              | Description                                                 |
| -------------------- | ----------------------------------------------------------- |
| Restrict result size | Limits the number of output documents.                      |
| Used with `$sort`    | Often follows `$sort` to get the top results.               |
| Used with `$match`   | Can also follow `$match` directly when filtering is enough. |

Example — Return only the top 2 student years (by student count) for “USAL”:

db.universities.aggregate([
  { $match: { name: "USAL" } },
  { $unwind: "$students" },
  { $project: { _id: 0, "students.year": 1, "students.number": 1 } },
  { $sort: { "students.number": -1 } },
  { $limit: 2 }
])

Note: $limit must be placed after $sort to ensure you're limiting the sorted results, not the original document order.

Output:

{ "students": { "year": 2014, "number": 24774 } }
{ "students": { "year": 2015, "number": 23166 } }

⭐ MongoDB `$addFields`

The $addFields stage is used to add new fields or modify existing fields in documents as they pass through the aggregation pipeline.

| Purpose                | Description                                                        |
| ---------------------- | ------------------------------------------------------------------ |
| Add new fields         | Inserts new fields into the document.                              |
| Modify existing fields | Can also overwrite the value of an existing field.                 |
| Works mid-pipeline     | Can be placed anywhere in the pipeline to shape data.              |
| Similar to `$set`      | `$addFields` and `$set` are functionally the same in MongoDB 4.2+. |

Example — Add a field foundation_year to a university document:

db.universities.aggregate([
  { $match: { name: "USAL" } },
  { $addFields: { foundation_year: 1218 } }
])

Output:

{
  name: "USAL",
  country: "Spain",
  city: "Salamanca",
  foundation_year: 1218,
  students: [
    { year: 2014, number: 24774 },
    { year: 2015, number: 23166 },
    ...
  ]
}

💡 Tip: $addFields is great for enriching data without altering the original collection.

⭐ MongoDB `$count`

The $count stage is used to return the total number of documents that pass through the pipeline up to that point.

| Purpose               | Description                                                                    |
| --------------------- | ------------------------------------------------------------------------------ |
| Count documents       | Returns the number of documents seen so far in the pipeline.                   |
| Custom field name     | You can give the result field any name (e.g., `total`, `count`, etc.).         |
| Often used at the end | Typically placed as the last stage in the pipeline.                            |
| Useful for analytics  | Helps measure how many items meet certain filter or transformation conditions. |

Example — Count how many student records exist across all universities:

db.universities.aggregate([
  { $unwind: "$students" },
  { $count: "total_documents" }
])

Output:

{ "total_documents": 8 }

💡 Tip: $count is a shortcut for using $group with a $sum: 1 — but much simpler to write!

⭐ MongoDB `$lookup`

The $lookup stage is used to join documents from another collection, similar to a SQL LEFT JOIN.

It allows you to enrich documents with related data stored in a different collection.

| Purpose             | Description                                                                  |
| ------------------- | ---------------------------------------------------------------------------- |
| Join collections    | Fetches matching documents from another collection and embeds them.          |
| Works like SQL JOIN | Matches a local field with a foreign field in another collection.            |
| Result goes in `as` | Matching documents are placed in a new array field (e.g., `courses`).        |
| Indexing matters    | Index the join fields for better performance (`localField`, `foreignField`). |

Example:

Join universities with courses where university.name matches courses.university:

db.universities.aggregate([
  { $match: { name: "USAL" } },
  { $project: { _id: 0, name: 1 } },
  {
    $lookup: {
      from: "courses",
      localField: "name",
      foreignField: "university",
      as: "courses"
    }
  }
])

Output:

{
  "name": "USAL",
  "courses": [
    { "university": "USAL", "name": "Computer Science", "level": "Excellent" },
    { "university": "USAL", "name": "Electronics", "level": "Intermediate" },
    { "university": "USAL", "name": "Communication", "level": "Excellent" }
  ]
}

💡 Tip: Always index the localField and foreignField used in $lookup for optimal performance.

⭐ MongoDB `$sortByCount`

The $sortByCount stage is a shortcut for $group + $count + $sort, used to count the occurrences of each unique value in a field and sort them in descending order.

| Purpose                  | Description                                                              |
| ------------------------ | ------------------------------------------------------------------------ |
| Count unique values      | Groups documents by a field and counts how many times each value occurs. |
| Sorted automatically     | Results are sorted in descending order of count.                         |
| Shortcut for `$group`    | Equivalent to `$group` + `$count` + `$sort` in one step.                 |
| Great for frequency data | Quickly shows the most or least common values in a field.                |

Example — Count how many courses exist at each level:

db.courses.aggregate([
  { $sortByCount: "$level" }
])

Output:

{ "_id": "Excellent", "count": 2 }
{ "_id": "Intermediate", "count": 1 }

💡 Tip: Use $sortByCount when you want a quick frequency breakdown of any field in your collection.

⭐ MongoDB `$facet`

The $facet stage allows you to run multiple aggregation pipelines in parallel on the same input. This is especially useful for reporting, dashboards, or summarizing data in multiple ways — all within a single aggregation query.

| Purpose                     | Description                                                             |
| --------------------------- | ----------------------------------------------------------------------- |
| Run multiple pipelines      | Executes multiple sub-pipelines in parallel on the same data.           |
| Useful for reporting        | Great for generating grouped, sorted, or summarized views side-by-side. |
| Reduces overhead            | Avoids repeating `$match`, `$lookup`, etc., multiple times.             |
| Returns results as sub-docs | Each sub-pipeline result is returned under its own field.               |

Example:

Generate two reports for "USAL" university:

1. Count of courses by level
2. Year with the fewest students

db.universities.aggregate([
  { $match: { name: "USAL" } },
  {
    $lookup: {
      from: "courses",
      localField: "name",
      foreignField: "university",
      as: "courses"
    }
  },
  {
    $facet: {
      countingLevels: [
        { $unwind: "$courses" },
        { $sortByCount: "$courses.level" }
      ],
      yearWithLessStudents: [
        { $unwind: "$students" },
        { $project: { _id: 0, students: 1 } },
        { $sort: { "students.number": 1 } },
        { $limit: 1 }
      ]
    }
  }
])

Output:

{
  "countingLevels": [
    { "_id": "Excellent", "count": 2 },
    { "_id": "Intermediate", "count": 1 }
  ],
  "yearWithLessStudents": [
    {
      "students": {
        "year": 2017,
        "number": 21715
      }
    }
  ]
}

💡 Tip: $facet is like running multiple reports in one go — with shared logic and better performance.

Exercise

⭐ $group + $sort aggregation stages
To calculate totals and order them, we often combine $group and $sort. This is useful for summary reports — like finding the total number of students per university and listing them by size.

| Stage     | Purpose                                                                  |
| --------- | ------------------------------------------------------------------------ |
| `$unwind` | Breaks the `students` array into individual documents.                   |
| `$group`  | Groups by university name and sums up the total students per university. |
| `$sort`   | Orders the results by total student count (highest to lowest).           |

Query: Total students per university (sorted)

db.universities.aggregate([
  { $unwind: "$students" },
  {
    $group: {
      _id: "$name",
      totalalumni: { $sum: "$students.number" }
    }
  },
  { $sort: { totalalumni: -1 } }
])

Output:

{ "_id": "USAL", "totalalumni": 91568 }
{ "_id": "UPSA", "totalalumni": 22284 }

💡 Tip: Always place $sort after $group to sort on aggregated values like totals, averages, or counts.

Bitveen

Performance: ⚡ Aggregation Pipeline Performance Tips

MongoDB’s aggregation pipeline is designed to optimize queries automatically, but writing efficient pipelines manually is still very important for performance.

| Tip                         | Description                                                                                |
| --------------------------- | ------------------------------------------------------------------------------------------ |
| Place `$match` early        | Always use `$match` before `$sort` to reduce the number of documents being sorted.         |
| Use indexes in first stage  | Indexes are only used if `$match` or `$sort` is at the **very beginning** of the pipeline. |
| Avoid full collection scans | Without an index, MongoDB scans all documents (which is slower).                           |
| Use `explain()`             | To analyze whether your query is using indexes or doing a full scan.                       |

Example: Check index usage with explain()

const pipeline = [
  { $match: { name: "USAL" } },
  { $sort: { "students.number": -1 } }
];

db.universities.aggregate(pipeline, { explain: true })

This will return a plan that shows whether MongoDB used an index or performed a collection scan.

Summary:

• Use indexed fields in your first $match or $sort.
• Keep heavy operations like $group, $unwind, and $sort after filtering.
• Use .explain() to inspect and optimize slow pipelines.

If you find my content helpful or want to support my work, consider buying me a coffee ☕ Your contributions will help me keep creating valuable resources.

👉 Buy Ankit Maheshwari a Coffee

Follow me on Medium for the upcoming articles.

Connect with me on LinkedIn.

Bitveen