Engineering

How to create a news feed across multiple content types with Contentful's Vault

As we’ve said previously we’re big fans of Contentful. Recently, for a news app, we used the Contentful Vault for Android to sync content from Contentful’s Content Delivery API to the Android device’s local database.

The app’s main news feed presents the user with the latest entries across several content types to be displayed in the main news feed.

Vault handled most of what we needed straight out of the box. We loved not having to worry about syncing content from the server – Vault took care of that. We did need to write a little code to query more than one content type ordered by the entry date. In this article, we’ll explain our approach.

Since Contentful’s query engine assumes a single content type both in Vault and in their Content Delivery API, we could not use the Contentful query engine to generate the news feed. Once we realized that Vault’s query API is only a thin layer over a SQLite database, we were encouraged sufficiently to explore the feasibility of writing our own SQLite queries to create the feeds.

With this idea in mind, we peeked under the hood to see how the Vault database is structured and how the tables and columns are named so that we could build our own queries. We exported the database created by Vault with a convenient shell script called Humpty Dumpty. We then used SQLiteBrowser to get to know Vault’s scheme and to test our queries.

To be able to support both 1:1 and 1:N in the same way Vault uses a links table.

CREATE TABLE `links$en-US` (
    `parent` STRING NOT NULL, 
    `child` STRING NOT NULL, 
    `field` STRING NOT NULL, 
    `is_asset` INT NOT NULL, 
    `position` INT NOT NULL, 
    UNIQUE (`parent`, `child`, `field`, `is_asset`, `position`)
)

The necessary work to join the multiple content types tables basically involves using a UNION to multiple queries using the links table.

    private static String selectAllUnion(String q1, String q2, String q3) {
        return "SELECT * FROM (" + q1 + " UNION " + q2 + " UNION " + q3 + ")";
    }

In our high level Java code, we have

    public static String buildFeedQuery() {
        final String articleTableName = new Article$$ModelHelper().getTableName() + CONTENT_LOCALE;
        final String linkTableName = new Link$$ModelHelper().getTableName() + CONTENT_LOCALE;
        final String categoryTableName = new Category$$ModelHelper().getTableName() + CONTENT_LOCALE;
        final String galleryTableName = new Gallery$$ModelHelper().getTableName() + CONTENT_LOCALE;
        final String categoriesIds = buildCategoriesIdsSet();
        String feedQuery =
                selectAllUnion(
                        articlesQuery(categoryTableName, articleTableName, categoriesIds),
                        linksQuery(categoryTableName, linkTableName, categoriesIds),
                        galleryQuery(galleryTableName, categoryTableName, categoriesIds)
                ) + orderByDate();
        return feedQuery;
    }

These table names require a dependency on build.gradle: apt 'com.contentful.vault:compiler:0.9.12'. We gathered it was safe to use these special classes because Vault itself uses them in their Getting Started examples.

And that’s it! Now we are able to query more than one content type at the same time and as a bonus we could leverage CursorAdapter paging capabilities.

If you try this approach or something similar in your app, please let us know in the comments.