Last month we unrolled another project: A social media center for one of Germany's largest insurance companies. The R+V Newsroom streams activities by R+V and R+V24 from eight different sources, giving a total of 14 different accounts polled, adding about 9 new posts each day. Until today, about 200,000 imports have been performed.
It's all about APIs
Polling APIs is different from usual single-box coding where errors are usually your own bad. Since we need to handle uncertainties implied by remote systems, our code needs to be particularly robust. Since we're talking to eight different systems, we need to talk eight different API languages. And since we're regularly checking data streams, we need a reliable way to find the point we left off for each of them.
Each network has its own associated importer that knows a) how to fetch new posts and b) how to write them to the database. With API client gems like twitter, koala, yt and xing_api, this is no big deal. Some sources had to be consumed more manually, like Wordpress RSS feeds, and to display press releases we're even scraping their web page.
The key for robustness is asynchronous import scheduled by whenever and performed by Sidekiq. When an import fails (due to an API unavailability like during the the DNS DDoS last month), a clever configuration makes sure it is retried a number of times before we're notified of its failure.
But it's not just flexible input, the R+V Newsroom also provides its data in various formats. Besides the website, it offers widgets that can be embedded into other pages and display a limited, filtered list of posts. Additionally, the post stream is available as RSS and JSON, leaving the choice on how to consume the data to the user.
Great UX with reasonable effort
In the frontend, the R+V Newsroom makes use of a number of great libraries to leverage user experience.
We further sped up the page by delivering images in the right sizes for each device. Since monitor resolution is ever increasing, images need to be served in higher resolutions, too. At the same time, the traffic from mobile devices increases—devices, that mostly have little bandwidth and usually sport far smaller screens than desktop computers. By using the awesome lazysizes, we could accomplish a flexible and maintainable delivery of "just right" images to whatever device visits the page. Read more in this in-depth blog post.
Another notable improvement was made to file uploading in the backend. You all know these plain file input fields that make any webpage feel like it's 20 years old. It does not have to be this way! jQuery-File-Upload is a mature library that lets you build awesome, snappy file uploads that are customizable and play well with server-side components like CarrierWave. See an in-depth example in the makandropedia.
Hosting by makandra
Having this new platform with a great UX, R+V decided to add the other half of UX: reliable servers. The Newsroom is hosted on makandra's powerful Rails infrastucture. Like web applications from Audi, ABUS and ProSieben, the R+V Newsroom runs on redundant machines that are properly scaled and caringly maintained.
With all this in place, the R+V Newsroom is a solid platform with a snappy UI in both frontend and backend, looking forward to its next 100,000 imports.
Four party cooperation
The R+V Newsroom is the result of a multi-company endeavor. Conception, project management and customer communication was done by d.tales, a content marketing agency from Munich. Christian Ringleb crafted these clean, bluey designs, while we were growing the Newsroom line by line. Thanks for the sweet cooperation!