Upgrade from 9.15 to 9.16

RESTHeart 9.16 has two breaking changes for deployments. Check each one against yours:

If you write plugins and one of them builds the key of a change stream worker, see also For plugin developers: the key of a change stream worker.

What changed

A read sends its own query only if its permission allows it

A filter written by the client can reveal the fields that projectResponse hides. A condition on a hidden field tells which documents match it, and a sort on it tells the order of its values. For this reason 9.16 refuses these queries unless the permission allows them.

Before 9.16, a permission that allowed a GET also let the client add filter, sort and sort_by.

From 9.16, a GET with one of them gets 403, unless its permission allows it. This applies to _size too. The client gets the message filter, sort and sort_by are not allowed. The log of RESTHeart says more, at INFO:

403 for GET /mydb/orders: filter, sort and sort_by are not allowed by the permission that matches the request: query through an aggregation, or set mongo.allowFilter on the permission

For each query a client sends, choose one way:

  • An aggregation. Declare it on the collection and call it with /coll/_aggrs/<name>. The pipeline must apply the caller’s readFilter itself: see Aggregations and permissions.

  • A choice declared by the permission. For an order, or an optional condition the client picks by value, the permission is enough: see the alternatives a permission declares.

  • allowFilter on the permission, for a role that must write its own queries:

"mongo": { "allowFilter": true }

To keep the behaviour of 9.15 on every permission, change the default of the mongo service:

mongo:
  default-allow-filter: true

Three things do not change. The root role can always query. The requests a plugin allows through the ACLRegistry are not checked. Writes are not affected: a bulk PATCH or DELETE still needs a filter, and allowBulkPatch and allowBulkDelete decide about it.

In the MCP catalog, the query and size actions do not show filter to a caller whose permissions do not allow it.

See allowFilter.

Change streams do not apply the readFilter and projectResponse

Before 9.16, a change stream added the caller’s readFilter and projectResponse to its stages. A stream with no stages of its own sent each user only the events of the documents the readFilter allowed.

From 9.16, a change stream does not do it any more. The stream itself must send each user only their own events. It does it in one of two ways:

  • notify_when on a @user variable;

  • a $match stage that uses @user or @mongoPermissions.

A stream under a permission with a readFilter that does neither is refused with 403. Without this check, after the upgrade it would send every event to everyone, with no error. The client gets Cannot open change stream: not allowed. The log of RESTHeart says why, at WARN:

Change stream /mydb/orders/_streams/mine refused: this change stream is allowed by a permission with a readFilter, which change streams do not apply since 9.16: segregate the events with notify_when on @user._id, or in the stages, or declare "segregation": "none" for a stream meant for every event

Before (9.15), with a readFilter of {"owner": "@user._id"} on the permission:

{ "uri": "mine", "stages": [] }

After (9.16):

{
  "uri": "mine",
  "stages": [],
  "notify_when": { "fullDocument::owner": { "$var": "@user._id" } }
}

The server reads @user._id from the caller’s account when the stream opens, never from the request. Use the variable the account carries: @user._id with MongoDB users, @user.sub with a JWT, @user.userid with the file realm.

Two more cases:

  • a stream that hid fields with the projectResponse now hides them with a $project stage;

  • a stream meant for every event declares "segregation": "none", and is not refused.

See Change streams and permissions.

For plugin developers: the key of a change stream worker

A change stream worker, the object that holds the MongoDB cursor of a stream, is now identified by the URL of the stream, its stages resolved for the request, and the JSON mode. Before 9.16 it was identified by the URL, the avars of the request and the JSON mode.

The constructor of ChangeStreamWorkerKey changed accordingly. A plugin compiled for 9.15 that calls it fails at runtime with NoSuchMethodError:

  • before (9.15): new ChangeStreamWorkerKey(String url, BsonDocument avars, JsonMode jsonMode);

  • after (9.16): new ChangeStreamWorkerKey(String url, BsonArray stages, JsonMode jsonMode).

To find the worker a request opens or shares, for instance to count the cursors of a database, do not build the key yourself. Use the new method, which builds it as the change stream handler does:

var key = ChangeStreamWorkerKey.of(mongoRequest);
var sharesCursor = ChangeStreamWorkers.getInstance().get(key).isPresent();

of throws when the stream does not exist or a variable of its stages is not bound. In both cases the handler refuses the request and opens no cursor.

What is new

These need no change on your side:

  • mongo.sort sets the order of the reads a permission allows: see sort.

  • A permission can offer the client a few choices, picked by name or by value, with $ifvar and with $var defaults: see the alternatives a permission declares.

  • All the clients of a change stream share one MongoDB cursor, whatever their notify_when value.

  • notify_when accepts @user variables: see Events per user.