Chapter 15. Using the System APIs

All DreamFactory versions include a web-based administration console used to manage all aspects of the platform. While this console offers a user-friendly solution for performing tasks such as managing APIs, administrators, and business logic, many companies desire to instead automate management through scripting. There are two notable reasons for doing so:

  • Multi-environment administration: APIs should always be rigorously tested in a variety of test and QA environments prior to being deployed to production. While DreamFactory does offer a service export/import mechanism, it's often much more convenient to write custom scripts capable of automating multi-environment service creation.
  • Integration with third party services: The complexity associated with creating new SaaS products such as API monetization can be dramatically reduced thanks to the ability to integrate DreamFactory into the solution. Following payment, the SaaS could interact with DreamFactory to generate a new role-based access control, API key, and define a volume limit. The new API key could then be presented to the subscriber.

In this chapter we'll walk you through several examples explaining exactly how these two use cases can be achieved.

Create a Database API

To create a database API, you'll send a POST request to the /api/v2/system/service endpoint. The request payload will contain all of the API configuration attributes. For instance this payload reflects what would be required to create a MySQL API:

{
  "resource":[
  {
    "id":null,
    "name":"mysql",
    "label":"MySQL API",
    "description":"MySQL API",
    "is_active":true,
    "type":"mysql",
    "config":{
      "max_records":1000,
      "host":"HOSTNAME",
      "port":3306,
      "database":"DATABASE",
      "username":"USERNAME",
      "password":"PASSWORD"
    },
    "service_doc_by_service_id":null
  }
  ]
}

After submitting a successful request, a 201 Created status code is returned along with the newly created service's ID:

{
  "resource": [
    {
      "id": 194
    }
  ]
}

Retrieving Service Types and Configuration Schemas

Each service type naturally requires a different configuration schema. For instance most database service types require that a host name, username, and password are provided, whereas the AWS S3 service type requires an AWS access key ID, secret access key, and AWS region. You can obtain a list of supported service types and associated configuration schemas by issuing a GET request to /api/v2/system/service_type. This will return a rather lengthy response containing the names and configuration schemas, a tiny portion of which is recreated here:

{
  "resource": [
    {
      "name": "adldap",
      "label": "Active Directory",
      "description": "A service for supporting Active Directory integration",
      "group": "LDAP",
      "singleton": false,
      "dependencies_required": null,
      "subscription_required": "SILVER",
      "service_definition_editable": false,
      "config_schema": [
        {
          "alias": null,
          "name": "host",
          "label": "Host",
          "description": "The host name for your AD/LDAP server.",
          "native": [],
          "type": "string",
          "length": 255,
        ...
        }
      ]
    }
  ]
}

If you just want to retrieve a list of service type names, issue the same GET request but with the fields=name parameter attached:

/api/v2/system/service_type?fields=name

This will return a list of service type names:

{
  "resource": [
    {
      "name": "adldap"
    },
    {
      "name": "amqp"
    },
    {
      "name": "apns"
    },
    ...
    {
      "name": "user"
    },
    {
      "name": "v8js"
    },
    {
      "name": "webdav_file"
    }
  ]
}

Retrieve API details

To retrieve configuration details about a specific API, issue a GET request to /api/v2/system/service. You can pass along either an API ID or the API name (namespace). For instance to retrieve a service configuration by ID, you'll pass the ID like this:

/api/v2/system/service/8

It is likely more natural to reference an API by it's namespace. You can pass the name in using the filter parameter:

/api/v2/system/service?filter=name=mysql

In both cases, the response will look like this:

{
  "resource": [
    {
      "id": 8,
      "name": "mysql",
      "label": "MySQL API",
      "description": "MySQL API",
      "is_active": true,
      "type": "mysql",
      "mutable": true,
      "deletable": true,
      "created_date": "2019-02-27 02:14:17",
      "last_modified_date": "2019-08-20 20:40:15",
      "created_by_id": "1",
      "last_modified_by_id": "3",
      "config": {
        "service_id": 8,
        "options": null,
        "attributes": null,
        "statements": null,
        "host": "database.dreamfactory.com",
        "port": 3306,
        "database": "employees",
        "username": "demo",
        "password": "**********",
        "schema": null,
        "charset": null,
        "collation": null,
        "timezone": null,
        "modes": null,
        "strict": null,
        "unix_socket": null,
        "max_records": 1000,
        "allow_upsert": false,
        "cache_enabled": false,
        "cache_ttl": 0
      }
    }
  ]
}

Creating a Scripted API Deployment Solution

Now that you understand how to create and query DreamFactory-managed APIs, your mind is probably racing regarding all of the ways at least some of your administrative tasks can be automated. Indeed, there are many different ways to accomplish this, because all modern programming languages support the ability to execute HTTP requests. In fact, you might consider creating a simple shell script that executes curl commands. Begin by creating a JSON file that contains the service creation request payload:

{
    "resource":[
        {
            "id":null,
            "name":"mysqltest09032019",
            "label":"mysql test",
            "description":"mysql test",
            "is_active":true,
            "type":"mysql",
            "config":{
                "max_records":1000,
                "host":"HOSTNAME",
                "port":3306,
                "database":"DATABASE",
                "username":"USERNAME",
                "password":"PASSWORD"
            },
                "service_doc_by_service_id":null
        }
    ]
}

Name this file mysql-production.json, and don't forget to update the authentication placeholders. Next, create a shell script that contains the following code:

#!/bin/bash

curl -d @mysql-production.json \
    -H "Content-Type: application/json" \
    -H "X-DreamFactory-Api-Key: YOUR_API_KEY" \
    -X POST https://YOUR_DOMAIN_HERE/api/v2/system/service

Save this script as create-service.sh and then update the permissions so it's executable before running it:

$ chmod u+x create-service.sh
$ ./create-service.sh
{"resource":[{"id":196}]}

Of course, this is an incredibly simple example which can be quickly built upon to produce a more robust solution. For instance you might create several JSON files, one for development, testing, and production, and then modify the script to allow for environment arguments:

$ ./create-service.sh production

Clear the DreamFactory Service Cache

For performance purposes DreamFactory caches all service definitions so the configuration doesn't have to be repeatedly read from the system database. Therefore when editing a service you'll need to subsequently clear the service cache in order for your changes to take effect. To clear the cache for a specific service, issue a DELETE request to the following URI, appending the service ID to it:

/api/v2/system/admin/session/8

To clear the cache for all defined services, issue a DELETE request to the following URI:

/api/v2/system/admin/session

Managing Roles

After creating an API, you'll typically want to generate a role-based access control and API key. API-based role management is a tad complicated because it involves bit masks. The bit masks are defined as follows:

Verb Mask
GET 1
POST 2
PUT 4
PATCH 8
DELETE 16

To create a role, you'll send a POST request to /api/v2/system/role, accompanied by a payload that looks like this:

{
  "resource":[
  {
    "name": "MySQL Role",
    "description": "MySQL Role",
    "is_active": true,
    "role_service_access_by_role_id": [
      {
        "service_id": SERVICE_ID,
        "component": "_table/employees/*",
        "verb_mask": 1,
        "requestor_mask": 3,
        "filters": [],
        "filter_op": "AND"
      },
      {
        "service_id": SERVICE_ID,
        "component": "_table/supplies/*",
        "verb_mask": 1,
        "requestor_mask": 3,
        "filters": [],
        "filter_op": "AND"
      }
    ],
    "user_to_app_to_role_by_role_id": []
  }
  ]
}

This payload assigns the role two permissions:

  • It can send GET requests to the _table/employees/* endpoint associated with the API identified by SERVICE_ID.
  • It can send GET and POST requests to the _table/supplies/* endpoint associated with the API identified by SERVICE_ID.

In both cases, the requestor_mask was set to 3, because you'll add permission masks together to achieve the desired permission level. For instance GET (1) + POST 2 = 3. If you wanted to allow all verbs, you'll add all of the masks together 1 + 2 + 4 + 8 + 16 = 31.

You can learn more about role management in our wiki.

Managing API Keys

It's possible to create and manage API keys via the system API. To retrieve a list of all API keys (known as Apps in DreamFactory lingo), send a GET request to the URI /api/v2/system/app. You'll receive a list of apps that look like this:

{
  "id": 5,
  "name": "weatherappapi",
  "api_key": "API_KEY",
  "description": "weatherappapi",
  "is_active": true,
  "type": 0,
  "path": null,
  "url": null,
  "storage_service_id": null,
  "storage_container": null,
  "requires_fullscreen": false,
  "allow_fullscreen_toggle": true,
  "toggle_location": "top",
  "role_id": 2,
  "created_date": "2019-02-28 17:55:29",
  "last_modified_date": "2019-02-28 17:55:29",
  "created_by_id": "3",
  "last_modified_by_id": null,
  "launch_url": ""
},

To retrieve just the application name and associated API key, identify the desired fields using the fields parameter:

/api/v2/system/app?fields=name,api_key

Here's an example response. Note the launch_url attribute is always returned:

{
  "resource": [
    {
      "name": "weatherappapi",
      "api_key": "API_KEY",
      "launch_url": ""
    },
    {
      "name": "HR Application",
      "api_key": "API_KEY",
      "launch_url": ""
    },
    {
      "name": "MySQL",
      "api_key": "API_KEY",
      "launch_url": ""
    }
  ]
}

You can also return each application's defined role using the related parameter. Issue a GET request to the following URI:

/api/v2/system/app?related=role_by_role_id

This will return a list of apps, and additionally any roles associated with the app. Note how in the following example a nested JSON object called role_by_role_id includes the role definition:

{
  "id": 5,
  "name": "Weather Application API",
  "api_key": "API_KEY",
  "description": "Weather Application API",
  "is_active": true,
  "type": 0,
  "path": null,
  "url": null,
  "storage_service_id": null,
  "storage_container": null,
  "requires_fullscreen": false,
  "allow_fullscreen_toggle": true,
  "toggle_location": "top",
  "role_id": 2,
  "created_date": "2019-02-28 17:55:29",
  "last_modified_date": "2019-02-28 17:55:29",
  "created_by_id": "3",
  "last_modified_by_id": null,
  "launch_url": "",
  "role_by_role_id": {
    "id": 2,
    "name": "Weather App API Role",
    "description": "Role for Weather App API",
    "is_active": true,
    "created_date": "2019-02-28 17:54:56",
    "last_modified_date": "2019-02-28 17:54:56",
    "created_by_id": "3",
    "last_modified_by_id": null
  }
}