When we design a user interface (UI) for our application, we want it to be user-friendly, for the user to have a good user experience (UX). How about application programming interfaces (API)? Shouldn’t an API be designed to be user-friendly as well? And in this case, who are the main users/consumers of APIs? Developers, of course!
This post will cover a few points on designing the JSON for API responses. API requests are not covered, cos of the Robustness Principle – “Be conservative in what you send, be liberal in what you accept”. A common security mantra goes, “Never Trust the User”, and so, while we can lay out clear guidelines on API requests, we can expect all kinds of input. What we can control, is our API’s output.
When we code an API in a dynamically typed language such as PHP or Javascript, we sometimes forget that some of the users consuming our API may be using a strongly typed language like Java or Swift (the languages for Android and iOS apps accordingly). This post is a result of having been on both sides of the fence 😛
Some definitions before we start…”dev” is a short form for “developer”, JSON is a data-interchange format consisting of name/value pairs, each pair being a property (some would call them key/value pairs, but we’ll just use the definition from json.org). An API consists of one or more endpoints, e.g. https://example.com/products and https://example.com/merchants (some would call them Products API and Merchants API, but no, these are just 2 endpoints in an API). And now, here are the points, each starting with a title in bold followed by its rationale:
1) camelCase for names
The JSON response is usually parsed and cast into an object of a certain class, e.g. Entity jsonObj = new Entity(jsonString); in Java. In a piece of code where all variables are camelCased, which would look more consistent – jsonObj.last_name or jsonObj.lastName?
If the JSON response has a lot of name/value pairs or contains a large array, eliminating underscores in names would save some bandwidth, e.g. {"myList":[{"myName":"a"},{"myName":"b"}]} vs {"my_list":[{"my_name":"a"},{"my_name":"b"}]}.
This, as well as the following points, is not dogmatic. It’s alright if you wish to use underscores for names, e.g. mapping straight to database fields or query string params, just be consistent – do not have a mix of underscored names and camelCased names.
2) Use null only for objects, empty values for other data types
In Swift, the null value nil can only be assigned to Optionals, not to normal data types. For example, var x: Int? = nil is allowed but not var x: Int = nil. To use the value of an Optional, it must be unwrapped first, e.g. x! + 2. Life would be easier for Swift devs if null is not used for primitive data types such as strings and integers.
There are limitations when writing a Swift framework SDK that can be imported in an Objective-C app, one of them being Swift Optionals. MyPersonClass?, String? and even Int can be exposed to Objective-C but not Int? – don’t ask me why 😛 It would be non-trivial for a Swift dev to model a JSON response of { age: null; }.
In Java, int is a primitive type and cannot be null, while Integer is a class and an object of its type can be null. Having a lot of objects with the possibility of them being null would probably invoke nightmares of NullPointerException among Java devs lol.
What then would constitute empty values for primitive data types? Taking a leaf out of PHP’s book – [] for arrays, "" for strings, 0 for integers and false for booleans. For cases where 0 cannot be used for an integer, -1 can be used, e.g. duration: -1 to indicate infinite duration, recordsPerPage: -1 to indicate no limit on the records per page (maximum no. of records per page), age: -1 to indicate age unknown, etc.
Different checks have to be used for null and empty strings in Java and Swift, unlike PHP and Javascript.
// PHP
$s = '';
echo (! $s) ? 'empty' : 'non-empty';
$s = null;
echo (! $s) ? 'empty' : 'non-empty'; // same check as for ''
// Javascript
var s = '';
console.log(!s ? 'empty' : 'non-empty');
s = null; // works also if s = undefined
console.log(!s ? 'empty' : 'non-empty'); // same check as for ''
// Java
String s = "";
System.out.println(s.equals("") ? "empty" : "non-empty"); // crash if s is null
s = null; // s.equals() will throw NullPointerException
System.out.println(s == null ? "empty" : "non-empty");
// Swift
var s: String? = "" // cannot use String if value can be null
print(s == "" ? "empty" : "non-empty")
s = nil
print(s == nil ? "empty" : "non-empty")
3) Consistent data types
If a property is meant to be an integer, e.g. id, keep it as an integer throughout all objects and across all endpoints. Do not use integers sometimes and strings at other times, e.g. {"id":1,"name":"John Doe","spouse":{"id":"2","name":"Jane Doe"}}.
If a property is meant to be an array, e.g. {"children":[{"id":1,"age":3},{"id":2,"age":5}]}, keep it as an array throughout all objects and across all endpoints. Do not change it into an object suddenly, e.g. {"children":{"id":1,"age":3}} just because there’s only 1 element. And if there are no elements, use an empty array for the value, i.e. [], and not null (which is for objects only).
Be careful of edge cases. Came across a situation where https://example.com/products yielded { per_page: 10 } but https://example.com/products?per_page=10 would yield { per_page: "10" }, the former being an integer and the latter being a string.
4) Use strings for id
Using integers for id has its limitations, one of which being its range, which is about 2^64 on current 64-bit platforms. This is probably why the Facebook Graph API uses strings for id – the number of posts probably has already exceeded the maximum integer value.
Also, numbers as large as 64 bits may pose problems with some programming languages (such as JavaScript where parseInt(Number.MAX_SAFE_INTEGER + 1) fails), which is why the Twitter API returns ids in both integer and string formats.
If the format is changed in the future, e.g. use passport number (please don’t) or UUID instead of integers, API consumers (especially Java/Swift devs) need not rush to update their code just because the data type has changed.
5) Do not omit properties
Compare {"name":"Alice","children":3} and {"name":"Bob"} – the children property is omitted as Bob has no children. The dev consuming this response would have to ensure no crashes happen when this happens, probably by including extra if-else statements.
Codable was introduced in Swift 4. The JSON responses above can be decoded into a Person object with let person = try JSONDecoder().decode(Person.self, from: jsonString.data(using: .utf8)). But special handling would be needed for the children property as it may not always be there. In the code snippet below, the entire init method could have been omitted if children is always present. In this example, this property cannot be declared using Int? as Int can be exposed to Objective-C but not Int? (mentioned earlier in point 2).
@objc public class Person: NSObject, Codable {
@objc public var id: Int = 0
@objc public var name: String? = nil
@objc public var address: Address? = nil
@objc public var children: Int = 0
required public init(from decoder: Decoder) throws {
let values = try decoder.container(keyedBy: CodingKeys.self)
id = try values.decode(Int.self, forKey: .id)
name = try values.decode(String?.self, forKey: .name)
address = try values.decodeIfPresent(Address.self, forKey: .address)
do {
children = try values.decode(Int.self, forKey: .children)
} catch _ {
children = 0
}
}
}
6) Same top-level properties across all endpoints
When consuming an API, it is common to write classes to model the responses and entities inside the responses. This would make it trivial to cast the response into an object using a decoding library with the class, as compared to manually parsing the response and initialising the properties by hand. Imagine responses from 2 endpoints: {"status":"ok","name":"John","address":{"street":"Beta Ave","zip":123}} from the Person endpoint and {"status":"ok","products":[{"id":91,"name":"Water"}],"recordsPerPage":1,"page":1,"pages":10,"records":91} from the Products endpoint.
Besides having Address and Product classes to model the entities, there would need to be separate classes to model the different responses, e.g. PersonResponse and ProductsResponse. In Java and Swift, generics would probably be used to share common code in handling the various responses, e.g. func handleResponse<T: Codable>(completionHandler: @escaping (Response<T>) -> Void) in Swift, where T would be PersonResponse or ProductsResponse. Now, imagine the responses from 100 endpoints with 100 different top-level properties 😛
A suggestion would be to only have 3 properties at the top level of all responses: data, error, meta. All would be objects, which makes it trivial to decode into classes and to use null when no value exists. status is deliberately omitted as devs should learn to use HTTP status codes to determine if a response is successful or not, and not depend on a string property with no standardised list of values and propensity for misspellings/miscapitalisation.
3 sample responses are shown below – an error response, a Person response and a Products response. Metadata such as application version (which should be present regardless of success or error) and pagination info should not be mixed with the actual data. Having error as an object allows the adding of other properties in the future. Note that for the Products response, data remains an object and does not change suddenly into an array. In general, if there is an error, data should be null.
// Error response
// Application version format:
// <project version>-<repo branch>-<commit hash>-<ISO 8601 timestamp in UTC timezone>
// Project version is from project config file such as composer.json
{
"data": null,
"error": {
"message": "Person not found"
},
"meta": {
"version": "v0.1.0-develop-52e5752-20220112T1315Z"
}
}
// Response from Person endpoint
{
"data": {
"name": "John Doe",
"address": {
"street": "Beta Ave",
"zip": 123
}
},
"error": null,
"meta": {
"version": "v0.1.0-develop-52e5752-20220112T1315Z"
}
}
// Response from Products endpoint
// Use "items" instead of resource name (e.g. "products") to be consistent
// Using snake_case here as "l" & "I" can be mixed up if spelt as "totalItems"
{
"data": {
"items": [
{
"id": 91,
"name": "Mineral Water"
}
]
},
"error": null,
"meta": {
"version": "v0.1.0-develop-52e5752-20220112T1315Z",
"pagination": {
"items_per_page": 10,
"page": 10,
"total_items": 91,
"total_pages": 10
}
}
}
7) Consistent naming for similar properties across all endpoints
A picture paints a thousand words – this negative example should speak volumes.
// Response from Employee endpoint
{
"id": 1,
"lastName": "Woe",
"age": 30,
"spouse": {
"personId": 15,
"familyName": "Doe",
"years": 29
}
}
// Response from Manager endpoint
{
"employeeId": 5,
"surname": "Foe",
"howOld": 40
}
8) Use UTC time zone and ISO 8601 for timestamps
Always explicitly set the timezone as UTC when creating dates. If not, the dates would default to the time zone of the server the code is running on, which may not always be set as UTC – a dev in Singapore may set up an AWS EC2 instance which defaults to the Singapore timezone (UTC+08:00), another dev in US may set up another instance in the Eastern Time Zone, etc.
Using ISO 8601 for timestamps in request headers or parameters, e.g. 2018-09-07T01:30:00.123456+08:00, can be useful in determining the time zone of the client, especially when troubleshooting requests from mobile devices. I came across a case where the app on a client’s mobile phone would always run into authentication issues with the API server and it was traced to the request timestamp header being always 1 hour behind that of the server – the client was from Indonesia while the server was in Singapore, go figure 😛
Some would suggest using Unix time, an integer independent of time zones. The main problem is that it is not human-readable. When you are fighting fire and debugging JSON from API requests/responses, the last thing you want to do is to use a convertor to convert every Unix timestamp you see into UTC and then to your timezone. Which is easier and faster to read – 1536255123456 or 2018-09-07T01:30:00.123456+08:00?
9) Use 0, 1, -1 for boolean properties
Supposing we are implementing feature flags and that the settings are embedded in the JSON response, e.g. { "featureFlags": { "someFeature": 1 } }. For the sample JSON, the existing/new feature will be turned on if the value of the someFeature flag is 1, turned off if the value is 0, unspecified if the value is -1 or if the someFeature flag is not set at all. How the application behaves when the flag is unspecified depends on the consensus within the team – generally, if the flag is unspecified, the application should keep to the current behaviour so as to avoid nasty surprises.
Using 0 and 1 makes it trivial for the consumer of the API response to check, e.g. if (featureFlags.someFeature) {}. They can be mapped directly to the TINYINT(1) data type in a MySQL database as well, without requiring the API to do additional casting.
Why not use boolean true and false, e.g. { "someFeature": true }? Besides taking up more bytes and having no value to indicate unspecified (null should not be used as it is a different data type and meant for objects only), it begets the question of how the value would be stored in the database. The MySQL database engine does not have a built-in Boolean type and uses TINYINT(1) instead, which means the API will need to do some casting from integer 0/1 to boolean false/true. And please do not suggest storing the strings “true” and “false” in a VARCHAR column in the database 😛
Using string values such as “on”, “off”, “yes”, “no”, “enabled”, “disabled”, “true”, “false” and so on begets the same opinion covered in point 6 as to why the status property is omitted. Application behaviour should not depend on a string property with no standardised list of values and propensity for misspellings/miscapitalisation, e.g., “on”, “ON”, “On”, “oN”, “om”, “no”. In the programming world, 0 always means false and 1 always means true.
Conclusion
The gist of all the above points to the KISS principle – “Keep it Simple, Stupid”. And one of the ways of doing that is consistency. You may not agree with all the points above, and that is alright – just make sure your API is consistent throughout in its design, saving your users guessing games and making their coding lives easier. Make your API developer-friendly today – ad huc!
[UPDATE 12 OCT 2018]
Gave a talk based on this blog post at the Singapore PHP Community Combined Meetup 2018 on 26 Sep, held in conjunction with PHPConf.Asia 2018. The talk elaborates with more examples. Here’s the video recording and the slides 🙂
[UPDATE 12 OCT 2020]
Added point 9 on using 0, 1, -1 for boolean properties.