REST API

Aggregations

RESTHeart Ulabase

Configuration

The URL and credentials the examples on this page use

localhost:8080 with admin:secret Saved in your browser

Setup Guide

Where to run the examples on this page

To run the examples on this page, you need a RESTHeart instance.

Option 1: Use Ulabase (Recommended)

The fastest way to get started is with Ulabase. Create a free service in minutes:

  1. Sign up at ulabase.com

  2. Create a free API service

  3. Set up your root user following the Root User Setup guide

  4. Use the configuration panel above to set your service URL and credentials

Tip
All code examples on this page will automatically use your configured Ulabase credentials.

Option 2: Run RESTHeart Locally

If you prefer local development, follow the Setup Guide to install RESTHeart on your machine.

Note
Local instances run at http://localhost:8080 with default credentials admin:secret

An aggregation processes the documents of a collection through a series of stages, the pipeline, and returns the result: a count, a sum, a group, a transformed document.

With RESTHeart you declare a MongoDB aggregation on a collection, and a client runs it with a GET.

Running Aggregations

To execute an aggregation, send a GET request to:

GET /collection/_aggrs/aggregation-name?avars={"var1":"value1"}

RESTHeart will process the request and return the computed results in the response body.

When you need an aggregation

Since v9.16 a client cannot send its own query unless its permission allows it (see allowFilter). The permission alone still serves the simple reads:

  • the caller’s documents, or the public ones: GET /collection, with the permission’s readFilter, paged with page and pagesize;

  • one document by id: GET /collection/<id>, with the same readFilter;

  • the fields not to show: the permission’s projectResponse;

  • the count: GET /collection/_size, which counts the documents the readFilter allows.

An aggregation is needed when the client picks a criterion, such as a search, a category, a range or an order other than the default, or when it wants a computed result. A permission can also offer a few declared alternatives to the client, an order or an optional condition, without an aggregation: see the alternatives a permission declares.

Tip
By default, aggregation results are returned directly in the response without being written to the database. For persistent results, use Materialized Views.

Defining Aggregations

Aggregations must be defined in the collection’s metadata before they can be used.

Creating an Aggregation

To define an aggregation, use a PATCH request to update the collection metadata:

cURL

curl -i -X PATCH "[RESTHEART-URL]/mycollection?wm=upsert" \
  -u "[BASIC-AUTH]" \
  -H "Content-Type: application/json" \
  -d '{
  "aggrs": [
    {
      "uri": "sales-by-region",
      "stages": [
        { "$match": { "status": "completed" } },
        { "$group": {
            "_id": "$region",
            "total": { "$sum": "$amount" }
          }
        },
        { "$sort": { "total": -1 } }
      ]
    }
  ]
}'

HTTPie

echo '{
  "aggrs": [
    {
      "uri": "sales-by-region",
      "stages": [
        { "$match": { "status": "completed" } },
        { "$group": {
            "_id": "$region",
            "total": { "$sum": "$amount" }
          }
        },
        { "$sort": { "total": -1 } }
      ]
    }
  ]
}' | http PATCH "[RESTHEART-URL]/mycollection?wm=upsert" \
  Authorization:"Basic [BASIC-AUTH]" \
  Content-Type:application/json

JavaScript

fetch('[RESTHEART-URL]/mycollection?wm=upsert', {
  method: 'PATCH',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    "aggrs": [
      {
        "uri": "sales-by-region",
        "stages": [
          { "$match": { "status": "completed" } },
          { "$group": {
              "_id": "$region",
              "total": { "$sum": "$amount" }
            }
          },
          { "$sort": { "total": -1 } }
        ]
      }
    ]
  })
})
.then(response => {
  if (response.ok) {
    console.log('Write request executed successfully');
  } else {
    console.error('Write request failed:', response.status);
  }
})
.catch(error => console.error('Error:', error));

This defines an aggregation named sales-by-region that: 1. Filters for completed sales 2. Groups documents by region 3. Calculates the total amount for each region 4. Sorts the results by total in descending order

Aggregation Properties

Property Description Required

uri

The name used in the URL path: /collection/_aggrs/uri

Yes

stages

Array of MongoDB aggregation pipeline stages

Yes

allowDiskUse

Allows operations to use more than 100MB of memory

No (default: false)

Parameterizing Aggregations

Make your aggregations dynamic by using variables that can be passed at runtime.

The $var Operator

Use the $var operator in your aggregation stages to reference variables:

{
  "aggrs": [
    {
      "uri": "sales-by-product",
      "stages": [
        { "$match": { "product": { "$var": "productName" } } },
        { "$group": { "_id": "$month", "sales": { "$sum": "$amount" } } }
      ]
    }
  ]
}

Passing Variables

Since 9.9.0, the simplest way to provide variable values is a plain query parameter named after the $var it binds:

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/sales-by-product?productName=Widget%20Pro" \
  -u "[BASIC-AUTH]"

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/sales-by-product" \
  productName=="Widget Pro" \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

const params = new URLSearchParams({ productName: "Widget Pro" });

fetch(`[RESTHEART-URL]/mycollection/_aggrs/sales-by-product?${params}`, {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

This returns the sales of the "Widget Pro" product only.

Every query parameter binds the $var with its name, except those RESTHeart reserves: filter, sort, page, pagesize, keys, hint, avars and similar. A request can bind as many variables as the pipeline needs, and each value is parsed on its own.

Each value is interpreted as JSON when it parses as one, and otherwise taken literally as a string:

Query parameter Bound value

?status=A

the string "A". It is not valid JSON, so it is taken as it is: no quotes needed

?limit=10

the number 10

?tags=[1,2,3]

the array [1, 2, 3]

?opts={"limit":10,"skip":20}

the object {"limit": 10, "skip": 20}

?date={"$date":1700000000000}

the BSON date it represents. The MongoDB Extended JSON types work too: $date, $oid, $numberLong and the others

Note
This shorthand works the same way for change stream variables, since both are bound through the same mechanism.
Note
A variable is a value. Since v9.17 a query parameter that the pipeline reads, and whose value carries a MongoDB operator such as ?status={"$ne":null}, gets 400. See Operator Injection Protection.

Legacy: the avars Query Parameter

Before 9.9.0 all the variables went in one JSON object, in the avars query parameter. This form is still supported. Two cases need it: a variable with the name of a reserved query parameter, such as page or sort, and a RESTHeart older than 9.9.0.

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/sales-by-product" \
  -u "[BASIC-AUTH]" \
  -G --data-urlencode 'avars={"productName":"Widget Pro"}'

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/sales-by-product" \
  avars=='{"productName":"Widget Pro"}' \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

const params = new URLSearchParams({
  avars: JSON.stringify({"productName":"Widget Pro"})
});

fetch(`[RESTHEART-URL]/mycollection/_aggrs/sales-by-product?${params}`, {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

You can combine both forms in the same request. If a variable name is given both as a flat query parameter and inside avars, the avars value wins.

Default Values

You can specify default values for variables:

{ "$var": [ "sortField", { "date": -1 } ] }

This uses the provided sortField value if available, otherwise defaults to sorting by date in descending order.

Example with default value:

{
  "aggrs": [
    {
      "uri": "recent-orders",
      "stages": [
        { "$sort": { "$var": [ "sortBy", { "date": -1 } ] } },
        { "$limit": 10 }
      ]
    }
  ]
}

Dot Notation for Nested Variables

You can access nested properties in variables using dot notation. Since a flat query parameter’s value is parsed as JSON when it looks like one (see above), it can carry a whole object:

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/my-pipeline" \
  -u "[BASIC-AUTH]" \
  -G --data-urlencode 'config={"limit":10,"skip":20}'

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/my-pipeline" \
  config=='{"limit":10,"skip":20}' \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

const params = new URLSearchParams({
  config: JSON.stringify({"limit":10,"skip":20})
});

fetch(`[RESTHEART-URL]/mycollection/_aggrs/my-pipeline?${params}`, {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

The legacy form, avars={"config":{"limit":10,"skip":20}}, works identically.

In the aggregation definition:

{ "$limit": { "$var": "config.limit" } }

This resolves to { "$limit": 10 }.

Predefined Variables

RESTHeart provides several predefined variables that you can use in your aggregations:

Variable Description

@user

The authenticated account. Its properties depend on how the caller authenticated: @user._id against MongoDB users, @user.userid against the file realm, @user.sub with a token. A name the account does not carry fails the request with variable not bound unless the $var declares a default, and a value supplied by the caller for a @ name is ignored. See Identifying the caller

@mongoPermissions

The user’s MongoDB permissions (e.g., @mongoPermissions.readFilter). A pipeline applies the readFilter only if it uses it: see Aggregations and permissions

@page

Current page number from query parameter

@pagesize

Page size from query parameter

@skip

Calculated as (page-1)*pagesize

@limit

Same as @pagesize

Pagination in Aggregations

Unlike regular document queries, pagination in aggregations must be handled explicitly using the $skip and $limit stages.

Use the predefined variables to implement pagination:

{
  "aggrs": [
    {
      "uri": "paginated-results",
      "stages": [
        { "$match": { "active": true } },
        { "$sort": { "lastName": 1 } },
        { "$skip": { "$var": "@skip" } },
        { "$limit": { "$var": "@limit" } }
      ]
    }
  ]
}

Request with pagination:

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/paginated-results?page=3&pagesize=25" \
  -u "[BASIC-AUTH]"

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/paginated-results" \
  page==3 pagesize==25 \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

const params = new URLSearchParams({
  page: 3,
  pagesize: 25
});

fetch(`[RESTHEART-URL]/mycollection/_aggrs/paginated-results?${params}`, {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

This skips the first 50 documents and returns the next 25.

Conditional Stages with $ifvar

Since RESTHeart 7.3, you can include stages conditionally based on whether specific variables are provided.

Basic Conditional Stage

Include a stage only if a variable is provided:

{
  "uri": "conditional-pipeline",
  "stages": [
    { "$match": { "type": "product" } },
    { "$ifvar": [ "category", { "$match": { "category": { "$var": "category" } } } ] }
  ]
}

The second stage only applies if the category variable is provided.

Multiple Required Variables

Require multiple variables for a stage:

{ "$ifvar": [ ["minPrice", "maxPrice"],
  { "$match": {
      "price": {
        "$gte": { "$var": "minPrice" },
        "$lte": { "$var": "maxPrice" }
      }
    }
  }
]}

Else Clause

Provide an alternative stage when variables are missing:

{ "$ifvar": [ "sortBy",
  { "$sort": { "$var": "sortBy" } },
  { "$sort": { "createdAt": -1 } }
]}

This sorts by the provided field if sortBy is given, otherwise sorts by creation date.

Materialized Views

Create persistent collections based on aggregation results using the $merge stage:

{
  "aggrs": [
    {
      "uri": "sales-summary",
      "stages": [
        { "$group": { "_id": "$category", "totalSales": { "$sum": "$amount" } } },
        { "$merge": { "into": "categorySalesSummary" } }
      ]
    }
  ]
}

When this aggregation is executed, results are written to the categorySalesSummary collection:

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/sales-summary" \
  -u "[BASIC-AUTH]"

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/sales-summary" \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

fetch('[RESTHEART-URL]/mycollection/_aggrs/sales-summary', {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

The response will be empty, but a new collection is created or updated:

cURL

curl -i -X GET "[RESTHEART-URL]/categorySalesSummary" \
  -u "[BASIC-AUTH]"

HTTPie

http GET "[RESTHEART-URL]/categorySalesSummary" \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

fetch('[RESTHEART-URL]/categorySalesSummary', {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));
HTTP/1.1 200 OK
[
  { "_id": "electronics", "totalSales": 253489.99 },
  { "_id": "furniture", "totalSales": 187245.50 }
]

Incremental Updates

The $merge stage is more efficient than the older $out stage because it can update existing documents rather than replacing the entire collection each time.

Aggregations and permissions

The readFilter, writeFilter and projectResponse of a permission bound the requests a client shapes: a read, a query, a write. They do not apply to aggregations. The stages of an aggregation are written by whoever writes the collection’s metadata, and the pipeline itself decides which documents it reads and which fields it returns.

To apply the caller’s readFilter, the pipeline matches on it:

{ "$match": { "$var": ["@mongoPermissions.readFilter", {}] } }

The default {} matters. Without a readFilter in the permission the variable is not bound, and a $var with no default fails the request.

A pipeline can also segregate the documents with the caller’s identity, for instance { "$match": { "owner": { "$var": "@user._id" } } }. To hide fields, it uses a $project stage.

When a client moves from ?filter to an aggregation, the pipeline must do the segregation that the readFilter used to do on the GET.

Security Considerations

Operator Injection Protection

The pipeline is written by whoever defines the aggregation. What comes from the request is a value. With aggregation-check-operators enabled, the default, RESTHeart keeps it so in two ways.

mongo:
    aggregation-check-operators: true  # Default setting

A value with an operator is refused. An operator is a key starting with $, as in {"$ne": null}. The request gets 400:

GET /orders/_aggrs/by-status?status={"$ne":null}

HTTP/1.1 400 Bad Request
illegal query parameter status: aggregation variables cannot include operators

The check covers:

  • every value in avars;

  • since v9.17, each plain query parameter that the pipeline reads with $var or $ifvar. A parameter the pipeline does not read is not bound and is not checked;

  • since v9.17, the predefined variables whose value can come from the client, such as @request.body.<path> and @user.<property>. @mongoPermissions and @filter are not checked: their values are filters.

MongoDB Extended JSON types, such as {"$date": 1700000000000} and {"$oid": "…​"}, are values and are accepted.

A string starting with $ stays a string. Since v9.17, a variable whose value is a string like "$name" or "$$ROOT" is bound as {"$literal": "$name"} wherever MongoDB would read it as a field path or as a variable: in every expression, such as $project, $addFields, $group, and $expr inside a $match. In a $match stage outside $expr the string is already a query value, and is left as it is. A default written in the definition is never wrapped.

For example, with this stage:

{ "$project": { "label": { "$var": "label" } } }

?label=$password returns the string $password in label. Before v9.17 it returned the value of the password field.

Two consequences:

  • a pipeline that let the client choose a field through a variable, such as {"$group": {"_id": {"$var": "groupBy"}}} with ?groupBy=$category, now groups by the literal string. Declare the alternatives in the pipeline with $ifvar, one optional stage per field the client may choose;

  • in the query stages other than $match ($geoNear.query, $graphLookup.restrictSearchWithMatch, $search) a value with a string starting with $ is refused by MongoDB.

With aggregation-check-operators: false a variable can carry operators and field paths into the pipeline. This is strongly discouraged.

Aggregation Pipeline Security

Since v9 RESTHeart checks each pipeline against two blacklists, of stages and of operators, before running it:

Stage Blacklisting

Certain pipeline stages can be restricted to prevent unauthorized data access or modification:

  • $out - Writing to collections

  • $merge - Merging data into collections

  • $lookup - Cross-collection joins

  • $graphLookup - Graph traversal operations

  • $unionWith - Combining collections

Operator Blacklisting

JavaScript-executing operators are blocked by default for security:

  • $where - JavaScript filter expressions

  • $function - Custom JavaScript functions

  • $accumulator - JavaScript-based aggregation

Configuration Example

mongo:
  aggregationSecurity:
    blacklistedStages:
      - $out
      - $merge
      - $lookup
      - $graphLookup
      - $unionWith
    blacklistedOperators:
      - $where
      - $function
      - $accumulator
    allowCrossDatabaseOperations: false
    allowJavaScriptExecution: false

When security violations occur, RESTHeart returns HTTP 403 Forbidden with specific violation types:

  • BLACKLISTED_STAGE - Attempted to use a prohibited pipeline stage

  • BLACKLISTED_OPERATOR - Attempted to use a prohibited operator

  • CROSS_DATABASE_ACCESS - Attempted to access collections in a different database

  • JAVASCRIPT_EXECUTION - Attempted to execute JavaScript code

Important
These checks are enabled by default.

Transaction Support

Execute aggregations within a transaction by including the sid and txn parameters:

cURL

curl -i -X GET "[RESTHEART-URL]/mycollection/_aggrs/my-pipeline?sid=session-id&txn=transaction-id" \
  -u "[BASIC-AUTH]"

HTTPie

http GET "[RESTHEART-URL]/mycollection/_aggrs/my-pipeline" \
  sid==session-id txn==transaction-id \
  Authorization:"Basic [BASIC-AUTH]"

JavaScript

const params = new URLSearchParams({
  sid: 'session-id',
  txn: 'transaction-id'
});

fetch(`[RESTHEART-URL]/mycollection/_aggrs/my-pipeline?${params}`, {
  method: 'GET',
  headers: {
    'Authorization': 'Basic [BASIC-AUTH]'
  }
})
.then(response => response.json())
.then(data => {
  console.log('Retrieved data:', data);
})
.catch(error => console.error('Error:', error));

See the Transactions documentation for details.

Example Use Cases

Monthly Sales Report

{
  "uri": "monthly-sales",
  "stages": [
    { "$match": {
        "date": {
          "$gte": { "$var": [ "startDate", { "$date": "2023-01-01T00:00:00Z" } ] },
          "$lte": { "$var": [ "endDate", { "$date": "2023-12-31T23:59:59Z" } ] }
        }
      }
    },
    { "$group": {
        "_id": { "$dateToString": { "format": "%Y-%m", "date": "$date" } },
        "count": { "$sum": 1 },
        "totalAmount": { "$sum": "$amount" }
      }
    },
    { "$sort": { "_id": 1 } }
  ]
}

User Activity Analytics

{
  "uri": "user-activity",
  "stages": [
    { "$match": { "userId": { "$var": "userId" } } },
    { "$group": {
        "_id": "$activityType",
        "count": { "$sum": 1 },
        "lastActivity": { "$max": "$timestamp" }
      }
    },
    { "$sort": { "count": -1 } }
  ]
}