Binary Files

Introduction

RESTHeart is a very good fit for headless CMS use cases, as it allows to effectively manage and aggregate content and its metadata, such as images, audios and videos, delivering them via a REST API.

RESTHeart offers complete binary files management. It’s possible to create, read and delete even huge files. RESTHeart makes use of MongoDB’s GridFS, a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB.

You can read more about GridFS here.

Example with http

Let’s use httpie to upload a PNG image named “dataflow.png

Note: RESTHeart’s HTTP clients must adhere to the multipart/form-data specification. However the specification allows a form to upload multiple files at the same time, but RESTHeart prohibits that: while it’s possible to POST / PUT several different non-binary parts for each file, it’s mandatory to upload one single binary file per each POST / PUT request. This limitation is necessary to univocally relate all the optional parts to the unique file.

POST a binary file

In this example we assume RESTHeart is running on localhost, port 8080.

Create the default restheart database, if none exists yet:

Request
PUT / HTTP/1.1
Response
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 1o6j8dt1f5y6jlu05t0blw2q4g280cgdv8253ilqhyoskoi5de
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-04T09:24:37.171231Z
Connection: keep-alive
Content-Length: 0
Content-Type: application/json
Date: Thu, 04 Jul 2019 09:09:37 GMT
ETag: 5d1dc2510951267987cf8ab1
X-Powered-By: restheart.org

Create the collection for hosting files. It must end with .files to mark this as a special collection for files:

Request
PUT /mybucket.files HTTP/1.1
Response
HTTP/1.1 201 Created
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 1o6j8dt1f5y6jlu05t0blw2q4g280cgdv8253ilqhyoskoi5de
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-04T09:26:41.654633Z
Connection: keep-alive
Content-Length: 0
Content-Type: application/json
Date: Thu, 04 Jul 2019 09:11:41 GMT
ETag: 5d1dc2cd0951267987cf8ab2
X-Powered-By: restheart.org

Then POST the file:

Request
POST /mybucket.files HTTP/1.1

@dataflow.png
Response
HTTP/1.1 201 Created
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 5r9jcdp1yzfa1uwx0nety86bgwz5fq5wmd17qjjajcbxw5ptzy
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-05T07:08:09.510520Z
Connection: keep-alive
Content-Length: 0
Content-Type: application/json
Date: Fri, 05 Jul 2019 06:53:10 GMT
Location: http://localhost:8080/mybucket.files/5d1ef3d50951267987cf8ab4
X-Powered-By: restheart.org

The Location HTTP header returns the file’s location:

Location: http://localhost:8080/mybucket.files/5d1ef3d50951267987cf8ab4

Note that the location contains the object ID automatically generated by MongoDB (see the string 5d1ef3d50951267987cf8ab4 at the end of the above Location URL). This is a unique identifier and it’s convenient in many situation, but it’s not always desirable. In many case it would be better to explicitly name the resource with something more readable and meaningful. To set the resource name it is necessary to upload the file by using the PUT verb instead of POST. We’ll show this later.

If you GET the Location, RESTHeart actually returns the file’s metadata:

Request
GET http://localhost:8080/mybucket.files/5d1ef3d50951267987cf8ab4 HTTP/1.1
Response
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 5r9jcdp1yzfa1uwx0nety86bgwz5fq5wmd17qjjajcbxw5ptzy
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-05T07:15:17.848962Z
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 244
Content-Type: application/json
Date: Fri, 05 Jul 2019 07:00:17 GMT
ETag: 5d1ef3d50951267987cf8ab3
X-Powered-By: restheart.org

{
    "_id": {
        "$oid": "5d1ef3d50951267987cf8ab4"
    }, 
    "_links": {
        "rh:data": {
            "href": "/mybucket.files/5d1ef3d50951267987cf8ab4/binary"
        }
    }, 
    "chunkSize": 261120, 
    "filename": "file", 
    "length": {
        "$numberLong": "232201"
    }, 
    "md5": "543dacb294d6fad4eb019b33b7192a4d", 
    "metadata": {
        "_etag": {
            "$oid": "5d1ef3d50951267987cf8ab3"
        }, 
        "contentType": "image/png"
    }, 
    "uploadDate": {
        "$date": 1562309590133
    }
}

The actual binary content is available by appending the binary postfix, like this:

http://localhost:8080/mybucket.files/5d1ef3d50951267987cf8ab4/binary

Note that the browser might show a popup window asking for credentials. As usual, use admin:secret.

Once a file has been created it becomes immutable: only its metadata can be updated. Practically, to update an existing file it’s necessary to delete and re-create it.

Uploading files with PUT

In the previous examples, the mybucket.files owner by default assigned to new files a resource name coming from MongoDB. If we want to set a meaningful URL then we need to send a PUT, like this:

Request
PUT /mybucket.files/dataflow.png HTTP/1.1

@dataflow.png
Response
HTTP/1.1 201 Created
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 5r9jcdp1yzfa1uwx0nety86bgwz5fq5wmd17qjjajcbxw5ptzy
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-05T07:24:04.146509Z
Connection: keep-alive
Content-Length: 0
Content-Type: application/json
Date: Fri, 05 Jul 2019 07:09:04 GMT
X-Powered-By: restheart.org

If we GET the resulting resource, here is the full HTTP response:

Request
GET /mybucket.files/dataflow.png HTTP/1.1
Response
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 5r9jcdp1yzfa1uwx0nety86bgwz5fq5wmd17qjjajcbxw5ptzy
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-05T07:39:57.206643Z
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 246
Content-Type: application/json
Date: Fri, 05 Jul 2019 07:24:57 GMT
ETag: 5d1ef4f50951267987cf8ab6
X-Powered-By: restheart.org

{
    "_id": "dataflow.png", 
    "_links": {
        "rh:data": {
            "href": "/mybucket.files/dataflow.png/binary"
        }
    }, 
    "chunkSize": 261120, 
    "filename": "file", 
    "length": {
        "$numberLong": "232201"
    }, 
    "md5": "543dacb294d6fad4eb019b33b7192a4d", 
    "metadata": {
        "_etag": {
            "$oid": "5d1ef4f50951267987cf8ab6"
        }, 
        "contentType": "image/png"
    }, 
    "uploadDate": {
        "$date": 1562309877118
    }
}

Note that in this case the resource has a much nicer URL:

http://localhost:8080/mybucket.files/dataflow.png

Which is easier to read and link than the automatically generated name.

The “properties” part

To add optional form data parts to the request it is necessary to embed the data into the properties field name. The content of this field is automatically parsed as JSON data, so it must be valid JSON.

In the following example, we set the “author” and the “filename”:

Request
PUT /mybucket.files/dataflow.png HTTP/1.1

@dataflow.png properties='{"author":"SoftInstigate", "filename":"dataflow"}'
Response
HTTP/1.1 201 Created
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 3e6cbhpj5auv459nvu51y292xo13sco265hmw6ul7ecgsm5tix
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-09T11:41:55.806010Z
Connection: keep-alive
Content-Length: 0
Content-Type: application/json
Date: Tue, 09 Jul 2019 11:26:55 GMT
X-Powered-By: restheart.org

The JSON will be merged into in the metadata section:

Request
GET /mybucket.files/dataflow.png HTTP/1.1
Response
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ETag, X-Powered-By
Auth-Token: 3e6cbhpj5auv459nvu51y292xo13sco265hmw6ul7ecgsm5tix
Auth-Token-Location: /tokens/admin
Auth-Token-Valid-Until: 2019-07-09T11:43:16.133898Z
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 266
Content-Type: application/json
Date: Tue, 09 Jul 2019 11:28:16 GMT
ETag: 5d247a4c0fcdc80169e486e3
X-Powered-By: restheart.org

{
    "_id": "dataflow.png", 
    "_links": {
        "rh:data": {
            "href": "/mybucket.files/dataflow.png/binary"
        }
    }, 
    "chunkSize": 261120, 
    "filename": "dataflow", 
    "length": {
        "$numberLong": "232201"
    }, 
    "md5": "543dacb294d6fad4eb019b33b7192a4d", 
    "metadata": {
        "_etag": {
            "$oid": "5d247a4c0fcdc80169e486e3"
        }, 
        "author": "SoftInstigate", 
        "contentType": "image/png", 
        "filename": "dataflow"
    }, 
    "uploadDate": {
        "$date": 1562671692338
    }
}

GET a binary file

As shown above, when we GET the representation for the newly created file the response contains only its metadata.

Appending `/binary` at the end of the above URL makes returns the actual binary content stored into GridFS. Open

http://localhost:8080/mybucket.files/dataflow.png/binary

If we paste this URL to a browser’s address bar then the image is displayed. This allows RESTHeart to serve as a very basic but powerful digital asset management system.