... thoughts on embedding HTML layout/presentation in JSON.

Something that has been bothering me a lot lately in hypermedia API formats is that JSON (and transitively JSON-LD) is purely data-oriented and it is awkward to intermingle layout.

JSON's processing model has two key aspects:

declaration order doesn't matter

{"foo": 1, "bar": 2} == {"bar": 2, "foo": 1}

you cannot use the same identifier

JSON.parse('{"foo": 1, "foo": 2}') == {"foo": 2}

And that's an issue because it means you can't easily declare layout/presentation like you would in HTML/XML. For example:

<!-- Declaration order matters. For example, this: -->
<form>
  <div>hello world</div>
  <input name='foo'>
  <input name='bar'>
</form>
<!-- ... is different than this: -->
<form>
  <input name='bar'>
  <input name='foo'>
  <div>hello world</div>
</form>

As well as you can use the same identifier multiple times and it doesn't override:

<!-- Declaration order matters. For example, this: -->
<form>
  <div>foo</div>
  <div>bar</div>
</form>
<!-- ... is different than this: -->
<form>
  <div>foo</div>
</form>

The JSON-friendly way to make this work is to make your objects declare a tree. This is what it would look like:

{
  @type: Form,
  children: [{
      @type: Div,
      children: ["hello world"]
    }, {
      @type: Input,
      name: "foo",
    },{
      @type: Input,
      name: "bar",
    }
  ]
}

And this is awkward at best.

HTML, on the other hand, is great at representing layout/presentation, but it is bad at representing machine-readable data. For example, this is what intermingling UI and data looks like with microdata:

<!-- Here is an example of embedding data with presentation in HTML -->
<div itemscope itemtype='http://schema.org/WebSite'>
  <!-- It is awkward to have to create these 'fake' UI elements 
    just to convey data -->
  <a itemscope itemprop='url'>http://example.com</a>
  <meta itemprop='name' content='My WebSite'>
  <!-- It is super hard to represent nested structures because meta tags
    aren't recursive -->
  <meta itemprop='description' content='foo bar>
  <form itemprop='potentialAction' itemscope 
    itemtype='http://schema.org/SearchAction'>
    <input itemprop='query' name='q'>
  </form>
</div>

What follows is an exploration of what combining both worlds would look like: embedding HTML layout in JSON, optimizing for producers as opposed to consumers (i.e. starting from the assumption that you want to make this easy for "writers" being cool paying the cost for "readers").

HTML (-ish) layout controls in JSON (-ish).

These are @goto's personal notes from a variety of internal discussions at Google.

This is super draft-y, read at your own risk.

The basic idea was simple: bring all the goodness of the well stablished and well thought out hypermedia controls in HTML to JSON objects. Everybody is familiar with those and there is no need to re-invent the wheel.

Here [0, 1, 2, 3] is some background reading if you are interested in the motivations.

I'll go over the most basic hypermedia controls available in HTML and a mechanism to map them to JSON. At the end, I'll go over a couple of extensions to what HTML provides to address a few needs from APIs.

Processing model

This isn't really JSON (and transitively JSON-LD): declaration order matters. That's important because it is designed to declare layout/presentation and data, as opposed to just data + hypermedia controls. That's super unfortunate, but I think necessary.
an object with a name that starts with @ starts new nodes, it is the equivalent of <> tags in XML/HTML. e.g. @a == <a>
a primitive with a name that starts with @ is an attribute of the parent node, it is the equivalent of attributes in XML/HTML. e.g. @a { @href: "foo" } == <a href="foo" >
custom properties (everything that isn't started with a @*) are added to the parent node's context.
There is some sort of JSON-LD-like processing that converts all these things into a bound graph.
// Comments are allowed.

The media type would be something like application/vnd.json-ish.

Affordances

We use a reserved token (@) to make the distinction between text and hypertext. Here are the main hypertext affordances:

Tag	Control
@a	Outbound links
@form	Templated links
@link	Inbound links
@meta	Metadata

@a

The simplest (and possibly most powerful) hypermedia control is the ability to express outbound links.

{
  kind: "issue",
  id: "1234",
  name: "this is an issue",
  description: "things are not working!",

  @a: { @href: "/issues", @rel: "home", @text: "All issues", @itemprop: "home" }
}

@form

The second important hypermedia control for APIs is the ability to perform operations/actions. On the web, this is typically done via <form>. This is roughtly what it maps to in JSON:

{
  kind: "collection",
  name: "All issues",
  description: "The collection of all issues",

  // Here is one GET form that has a text and a checkbox.
  @form: { @action: "/search", @method: "GET" ,
    @input: { @type: "text", @name: "query"},
 
    // Pick many of many.
    @input: { @type: "checkbox", @name: "openOnly"},

    // Pick one of many.
    @input: { @type: "radio", @name: "priority", @value: "P1"},
    @input: { @type: "radio", @name: "priority", @value: "P2"},

    // Multiple ways ot submit a form
    @input: { @type: "submit", @value: "cancel", @text: "Bah, nevermind!"},
    @input: { @type: "submit", @value: "done", @text: "Search!"},
  },

  // Forms can perform POSTs too as well as allow selects.
  @form: { @action: "/create", @method: "POST",
    @select: {
      @name: "issueType",
      @option: { @value: "1", @text: "New issue"},
      @option: { @value: "2", @text: "New feature"},
      @option: { @value: "3", @text: "New bug"},
    },

    // A submit button
    @input: { @type: "submit"},
  }
}

Note that, while <checkbox>/<select> don't have a UI equivalency in APIs, they still have hard semantics on how they are supposed to be used: <checkbox> boolean select-many-of-many and <select>s are select-one-of-many. <radio>s don't exist, because they are equivalent to selects.

@link

Links gives developers the ability to include external resources in the current resource (much link <img> tags work).

{
  // Here is an example of alternate links.
  @link: { @rel: "alternate", @href: "/issues/1234", @type: "text/xml"},

  kind: "issue",
  name: "A specific issue",

  author: {
    // Here is an example of a basic inbound link (clients should fetch and
    // insert the contents of the target page into this document tree).
    @link: { @rel: "import", @href: "/users/1234"},
  }
}

Microdata, RDFa, schema.org and opengraph

Suppose you had the following JSON payload:

{
  kind: "Film",
  title: "The Rock"
}

That obviously can't be understood by a generic client because it lacks common semantics. You could use microdata to give clients semantic hints:

{
  @itemtype: "http://schema.org/Movie",
  kind: "Film",
  title: "The Rock",

  // The schema.org Movie equivalent of title.
  name: "The Rock",
}

Similarly, you could use RDFa to annotate your objects:

{
  kind: "Film",
  name: "The Rock",
  @meta: { @property: "og:type", @content: "video.movie"},
  @meta: { @property: "og:title", @text: "The Rock"}
}

Now your clients can use either your proprietary vocabulary or the standardized types in external vocabularies. Here is one example of how this fits with http://schema.org/Action.

{
  @itemtype: "http://schema.org/WebSite",

  name: "hello world",
  url: "http://cnn.com",

  // Here is one GET form that allows you to search given a query.
  @form: {
    @method: "GET",
    @itemtype: "http://schema.org/SearchAction",
    @itemprop: "potentialAction",
    @input: { @required: true, @name: "q", @itemprop: "query", @text: "Puppies"}
  },
}

Extensions

There are a few things that HTML doesn't quite yet support well, so we extend it with a few key affordances.

Nested <input>s

{
  "hello": "world",

  // Forms can perform POSTs too as well as allow selects.
  @form: { @action: "/selects", @method: "POST", @enctype: "application/json",
    @input: { @name: "person", @type: "group",
      @input: { @name: "name", @type: "text", @value: "hello"}
    }
  }
}

Would lead to the following POST request:

POST /widgets/abc123 HTTP/1.1
Host: api.example.com
Content-Length: ...
Content-Type: application/json

{
  "person": {
    "name": "foo",
  }
}

PATCH, PUT and DELETE

We allow PATCH, PUT and DELETE to be a value of the "method" property of <form>.

{
  hello: "world",
 
  // Clients may send a DELETE HTTP request to the server.
  @form: {
    @action: "/resources/123",
    @method: "DELETE"
    @input: { @type: "submit"},
  },
}

This is what PUTs would look like.

{
  hello: "world",

  // Here is one GET form that has a text and a checkbox.
  @form: {
    @action: "/resources/123",
    @method: "PUT",

    // This object goes into the HTTP body of PUT.
    @input: { @type: "hidden", @name: "newvalue", @value: "foo"},

    // You may even require the client to provide an input before
    // you can PUT.
    @input: { @type: "text", @name: "newkey2", @value: "bar"},

    // A submit button
    @input: { @type: "submit"},
  },
}

This is what PATCHs would look like.

{
  count: "3",

  @form: {
    @action: "/resources/123",
    @method: "PATCH",
    @enctype: "application/json-patch",
    // This is a JSON-PATCH object.
    @input: {
      @type: "hidden", @name: "foo",
      @text: {
        @replace: "/count",
        @value: "5"
      }
    },
  },
}

This is obviously more of an exploration than an actual proposal, but I think serves to isolate a problem: JSON doesn't work well for layout/presentation. I don't know yet what's the best way to solve this problem, but I think it is an important factor while picking your hypermedia API format.

Hello World

Tuesday, September 1, 2015

JSON-ish